Quick Hidden Layer Size Tuning in ELM for Classification Problems

  • Audi Albtoush Faculty of Computer Science and Information Technology, Jerash University, Jordan
  • Manuel Fernandez-Delgado Santiago De Compostela University, Spain
  • Haitham Maarouf Santiago De Compostela University, Spain
  • Asmaa Jameel Al Nawaiseh Software Engineering Department, Mut'ah university, Jordan
Keywords: Extreme Learning Machine, Number of Hidden Nodes, Moving Average, Exponential Moving Average, Divide-and-conquer

Abstract

The extreme learning machine is a fast neural network with outstanding performance. However, the selection of an appropriate number of hidden nodes is time-consuming, because training must be run for several values, and this is undesirable for a real-time response. We propose to use moving average, exponential moving average, and divide-and-conquer strategies to reduce the number of training’s required to select this size. Compared with the original, constrained, mixed, sum, and random sum extreme learning machines, the proposed methods achieve a percentage of time reduction up to 98\% with equal or better generalization ability.

References

Al-Btoush, A., Abbadi, M., Hassanat, A., Tarawneh, A., Hasanat, A., and Prasath, S. New features for eye-tracking systems: Preliminary results. In Intl Conf Inf Comm Syst (2019), IEEE, pp. 179–184.

Al-Btoush, A., Fern´andez-Delgado, M., Cernadas, E., and Barro, S. Extreme learning machine with confidence interval based bias initialization. In 2021 Second International Conference on Intelligent Data Science Technologies and Applications (IDSTA) (2021), IEEE, pp. 23–30.

Al Nawaiseh, A. J., Albtoush, A., Al Nawaiseh, S. J., et al. Evaluate database management system quality by analytic hierarchy process (ahp) and simple additive weighting (saw) methodolog. MENDEL 28, 2 (2022), 67–75.

Albtoush, A., Fernandez-Delgado, M., Cernadas, E., and Barro, S. Quick extreme learning machine for large-scale classification. Neural Computing and Applications 34, 8 (2022), 5923–5938.

Cao, L., Yue, Y., Zhang, Y., and Cai, Y. Improved crow search algorithm optimized extreme learning machine based on classification algorithm and application. IEEE Access 9 (2021), 20051–20066.

Dani, Y., Gunawan, A. Y., Khodra, M. L., and Indratno, S. W. Detecting outliers using modified recursive pca algorithm for dynamic streaming data. MENDEL 29, 2 (2023), 237–244.

Deng, W., Bai, Z., Huang, G., and Zheng, Q. A fast SVD-hidden-nodes based extreme learning machine for large-scale data analytics. Neural Netw 77 (2016), 14–28.

Elsheikh, A., Saba, A., Elaziz, M., Lu, S., et al. Deep learning-based forecasting model for COVID-19 outbreak in Saudi Arabia. Process Saf Environ 149 (2021), 223–233.

Feng, G., Huang, G.-B., Lin, Q., and Gay, R. Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE T Neur Netw 20, 8 (2009), 1352–1357.

Hsieh, C.-J., Si, S., and Dhillon, I. A divideand- conquer solver for kernel support vector machines. In Intl Conf Mach Learn (2014), pp. 566–574.

Huang, G.-B., and Chen, L. Convex incremental extreme learning machine. Neurocomputing 70, 16-18 (2007), 3056–3062.

Huang, G.-B., and Chen, L. Enhanced random search based incremental extreme learning machine. Neurocomputing 71, 16-18 (2008), 3460– 3468.

Huang, G.-B., Zhu, Q.-Y., Mao, K., Siew, C.-K., Saratchandran, P., and Sundararajan, N. Can threshold networks be trained directly? IEEE T Circuits Syst II: Express Briefs 53, 3 (2006), 187–191.

Huang, G.-B., Zhu, Q.-Y., and Siew, C.-K. Extreme learning machine: a new learning scheme of feedforward neural networks. In IEEE Intl J Conf Neur Netw (2004), vol. 2, pp. 985–990.

Huang, G.-B., Zhu, Q.-Y., and Siew, C.-K. Extreme learning machine: theory and applications. Neurocomputing 70, 1-3 (2006), 489–501.

Huang, Y., and Lai, D. Hidden node optimization for extreme learning machine. AASRI Procedia 3 (2012), 375–380. Conf on Modelling, Identification and Control.

Lai, J., Wang, X., Li, R., Song, Y., and Lei, L. BD-ELM: a regularized extreme learning machine using biased dropconnect and biased dropout. Math Probl Eng 2020 (2020).

Lan, Y., Soh, Y. C., and Huang, G.-B. A constructive enhancement for online sequential extreme learning machine. In Intl J Conf Neural Netw (2009), Ieee, pp. 1708–1713.

Mahmoudi, M. R., and Baroumand, S. Modeling the stochastic mechanism of sensor using a hybrid method based on seasonal autoregressive integrated moving average time series and generalized estimating equations. ISA Transactions (2021), 300–305.

Marques, N. C., and Gomes, C. Implementing an intelligent moving average with a neural network. In Proc European Conf Artif Intel. IOS Press, 2010, pp. 1129–1130.

Miche, Y., Sorjamaa, A., Bas, P., Simula, O., Jutten, C., and Lendasse, A. OP-ELM: optimally pruned extreme learning machine. IEEE T Neur Netw 21, 1 (2009), 158–162.

Rong, H.-J., Ong, Y.-S., Tan, A.-H., and Zhu, Z. A fast pruned-extreme learning machine for classification problem. Neurocomputing 72, 1-3 (2008), 359–366.

Schuler, J. P. S., Romani, S., Abdel- Nasser, M., Rashwan, H., and Puig, D. Grouped pointwise convolutions reduce parameters in convolutional neural networks. MENDEL 28, 1 (2022), 23–31.

Sheela, K. G., and Deepa, S. N. Review on methods to fix number of hidden neurons in neural networks. Math Probl Eng 2013 (2013), 1–11.

Simila, T., and Tikka, J. Multiresponse sparse regression with application to multidimensional scaling. In Intl Conf Artif Neural Netw (2005), Springer, pp. 97–102.

Song, S., Wang, M., and Lin, Y. An improved algorithm for incremental extreme learning machine. Syst Sci & Control Eng 8, 1 (2020), 308–317.

Suresh, S., Saraswathi, S., and Sundararajan, N. Performance enhancement of extreme learning machine for multi-category sparse data classification problems. Engin Appl Artif Intel 23, 7 (2010), 1149–1157.

Uribe, I. M. Predictive model of the enso phenomenon based on regression trees. MENDEL 29, 1 (2023), 7–14.

Yu, D., and Deng, L. Efficient and effective algorithms for training single-hidden-layer neural networks. Pattern Recogn Lett 33, 5 (2012), 554–558.

Zhu, Q.-Y., Qin, A. K., Suganthan, P. N., and Huang, G.-B. Evolutionary extreme learning machine. Patt Recogn 38, 10 (2005), 1759–1763.

Zhu, W., Miao, J., and Qing, L. Constrained extreme learning machines: A study on classification cases. arXiv:1501.06115, 2015.

Published
2024-07-15
How to Cite
[1]
Albtoush, A., Fernandez-Delgado, M., Maarouf, H. and Al Nawaiseh, A. 2024. Quick Hidden Layer Size Tuning in ELM for Classification Problems. MENDEL. 30, 1 (Jul. 2024), 1-14. DOI:https://doi.org/10.13164/mendel.2024.1.001.
Section
Research articles