A NEW CLASS OF ACTIVATION FUNCTIONS BASED ON THE CORRECTING AMENDMENTS OF GOMPERTZ–MAKEHAM TYPE

TitleA NEW CLASS OF ACTIVATION FUNCTIONS BASED ON THE CORRECTING AMENDMENTS OF GOMPERTZ–MAKEHAM TYPE
Publication TypeJournal Article
Year of Publication2019
AuthorsKYURKCHIEV NIKOLAY, ILIEV ANTON, RAHNEV ASEN
JournalDynamic Systems and Applications
Volume28
Issue2
Start Page243
Pagination16
Date Published02.2019
ISSN1056-2176
AMS Subject Classification41A46
Abstract

We will explore the interesting methodological task for constructing new activation functions using ”correcting amendments” of ”Gompertz-Makeham-type” (GMAF). We also define the new family of recurrence generated activation functions based on ”Gompertz–Makeham correction” - (RGGMAF).

We prove upper and lower estimates for the Hausdorff approximation of the sign function by means of this new class of parametric activation functions - (RGGMAF). Numerical examples, illustrating our results are given.

PDFhttps://acadsol.eu/dsa/articles/28/2/2.pdf
DOI10.12732/dsa.v28i2.2
Refereed DesignationRefereed
Full Text

[1] N. Guliyev, V. Ismailov, A single hidden layer feedforward network with only one neuron in the hidden layer san approximate any univariate function, Neural Computation, 28 (2016), 1289-1304.
[2] D. Costarelli, R. Spigler, Approximation results for neural network operators activated by sigmoidal functions, Neural Networks, 44 (2013), 101-106.
[3] D. Costarelli, G. Vinti, Pointwise and uniform approximation by multivariate neural network operators of the max-product type, Neural Networks, 81 (2016), 81-90.
[4] D. Costarelli, R. Spigler, Solving numerically nonlinear systems of balance laws by multivariate sigmoidal functions approximation, Computational and Applied Mathematics, 37, No. 1 (2018), 99-133.
[5] D. Costarelli, G. Vinti, Convergence for a family of neural network operators in Orlicz spaces, Mathematische Nachrichten, 290, No. 2-3 (2017), 226-235.
[6] J. Dombi, Z. Gera, The Approximation of Piecewise Linear Membership Functions and Lukasiewicz Operators, Fuzzy Sets and Systems, 154, No. 2 (2005), 275-286.
[7] I. A. Basheer, M. Hajmeer, Artificial Neural Networks: Fundamentals, Computing, Design, and Application, Journal of Microbiological Methods, 43 (2000), 3-31.
[8] Z. Chen, F. Cao, The Approximation Operators with Sigmoidal Functions, Computers & Mathematics with Applications, 58 (2009), 758-765.
[9] Z. Chen, F. Cao, The Construction and Approximation of a Class of Neural Networks Operators with Ramp Functions, Journal of Computational Analysis and Applications, 14 (2012), 101-112.
[10] Z. Chen, F. Cao, J. Hu, Approximation by Network Operators with Logistic Activation Functions, Applied Mathematics and Computation, 256 (2015), 565- 571.
[11] D. Costarelli, R. Spigler, Constructive Approximation by Superposition of Sigmoidal Functions, Anal. Theory Appl., 29 (2013), 169-196.
[12] N. Kyurkchiev, A new class of activation function based on the correcting amendments, Int. J. for Sci., Res. and Developments, 6, No. 2 (2018), 565-568.
[13] N. Kyurkchiev, A. Iliev, S. Markov, Some techniques for recurrence generating of activation functions, LAP LAMBERT Academic Publishing (2017), ISBN: 978-3-330-33143-3.
[14] A. Iliev, N. Kyurkchiev, S. Markov, A Note on the New Activation Function of Gompertz Type, Biomath Communications, 4, No. 2 (2017).
[15] W. Duch, N. Jankowski, Survey of neural transfer functions, Neural Computing Surveys, 2 (1999), 163-212.
[16] F. Hausdorff, Set Theory, 2nd ed., Chelsea Publ., New York (1962).
[17] B. Sendov, Hausdorff Approximations, Kluwer, Boston (1990).
[18] N. I. Akhiezer, Theory of approximation, Nauka, Moscow (1965), 2nd ed., (in Russian), English translation (of the 1st edition): N. I. Achiezer, Theory of approximation, Translated by Charles J. Hyman: Frederick Ungar Publishing, New York (1956).
[19] E. Voronovskaya, Odd polynomials of least deviation, Dokl. AN SSSR, 159, No. 4 (1964), 715-718. (in Russian)
[20] S. Markov, Bl. Sendov, On the numerical evaluation of a class of polynomials of best approximation, Ann. Univ. Sofia, Fac. Math., 61 (1966/67), 17-27. (in Bulgarian).
[21] A. Andreev, N. Kyurkchiev, Approximation of some impulse functions - implementation in programming environment MATHEMATICA, Proceedings of the 43 Spring Conference of the Union of Bulgarian Mathematicians, Borovetz, April 2-6, 2014, 111-117.
[22] N. Kyurkchiev, A. Andreev, Hausdorff approximation of functions different from zero at one point - implementation in programming environment MATHEMATICA, Serdica J. of Computing, 7, No. 2 (2013), 135-142.
[23] N. Kyurkchiev, A. Andreev, Synthesis of slot aerial grids with Hausdorff-type directive patterns - implementation in programming envoronment Mathematica, C. R. Acad. Bulg. Sci., 66, No. 11 (2013), 1521-1528.
[24] N. Kyurkchiev, A. Andreev, Approximation and Antenna and Filters synthesis: Some Moduli in Programming Environment MATHEMATICA, LAP LAMBERT Academic Publishing, Saarbrucken (2014), ISBN: 978-3-659-53322-8.
[25] N. Kyurkchiev, A. Iliev, On the Hausdorff distance between the Heaviside function and some transmuted activation functions, Mathematical Modelling in Engineering Applications, 2, No. 1 (2016), 1-5.
[26] N. Kyurkchiev, S. Markov, Hausdorff approximation of the sign function by a class of parametric activation functions, Biomath Communications, 3, No. 2 (2016).
[27] A. Iliev, N. Kyurkchiev, S. Markov, A Family of Recurrence Generated Parametric Activation Functions with Applications to Neural Networks, International Journal on Research Innovations in Engineering Science and Technology, 2, No. 1 (2017), 60-68.
[28] N. Kyurkchiev, A. Iliev, S. Markov, Families of Recurrence Generated Three and Four Parametric Activation Functions, International Journal for Scientific Research & Development, 5, No. 1 (2017), 746-750.
[29] V. Kyurkchiev, A. Iliev, N. Kyurkchiev, On Some Families of Recurrence Generated Activation Functions, International Journal of Scientific Engineering and Applied Science, 3, No. 3 (2017), 243-248.
[30] N. Kyurkchiev, A. Iliev, A Note On The New Fibonacci Hyperbolic Tangent Activation Function, International Journal of Innovative Science Engineering and Technology, 4, No. 5 (2017), 364-368.
[31] A. Golev, A. Iliev, N. Kyurkchiev, A Note on the Soboleva’ Modified Hyperbolic Tangent Activation Function, Int. J. of Sci., Eng. and Techn., 4, No. 6 (2017), 177-182.
[32] A. Malinova, A. Golev, A. Iliev, N. Kyurkchiev, A Family of Recurrence Generating Activation Functions Based on Gudermann Function, International Journal of Engineering Researches and Management Studies, 4, No. 8 (2017), 38-48.
[33] N. Kyurkchiev, A Note on the Volmer’s Activation (VA) Function, C. R. Acad. Bulg. Sci., 70, No. 6 (2017), 769-776.
[34] V. Kyurkchiev, N. Kyurkchiev, A Family of Recurrence Generated Functions Based on ”Half-Hyperbolic Tangent Activation Function”, Biomedical Statistics and Informatics, 2, No. 3 (2017), 87-94.
[35] N. Kyurkchiev, A. Iliev, A new class of 2-parametric deterministic activation functions, Int. J. of Advanced Research in Computer and Communication Engineering, 7, No. 3 (2018), 481-483.
[36] N. Kyurkchiev, A new class activation functions with application in the theory of impulse technics, Journal of Mathematical Sciences and Modelling, 1, No. 1 (2018), 15-20.
[37] C. Zhang, S. Zhou, B. Chain, Hybrid Epidemics-A Case Study on Computer Worm Conficker, PLoS ONE, 10, No. 5 (2015), e0127478.
[38] P. Porras, H. Saidi, V. Yegneswaran, An Analysis of Conficker’S Logic and Rendezvous Points, SRI international technical report, March 19, (2009).
[39] N. Kyurkchiev, A. Iliev, A. Rahnev, T. Terzieva, A new analysis of Code Red and Witty worms behavior, Communications in Applied Analysis, 23 (2), 2019, 267-285.
[40] N. Kyurkchiev, A. Iliev, A. Rahnev, T. Terzieva, Some New Approaches for Modelling Large-Scale Worm Spreading on the Internet. II, Neural, Parallel, and Scientific Computations, 27 (2019). (to appear).