Activation functions performance in multilayer perceptron for time series forecasting

Nur Haizum, Abd Rahman and Yin, Chin Hui and Hani Syahida, Zulkafli (2024) Activation functions performance in multilayer perceptron for time series forecasting. In: AIP Conference Proceedings. The 6th ISM International Statistical Conference (ISM-VI) 2023 , 19–20 September 2023 , Shah Alam, Malaysia. pp. 1-10., 3123 (1). ISBN 978-0-7354-5030-1 (Published)

Preview	Pdf 2024 Activation Functions Performance in Multilayer Perceptron for Time Series Forecasting.pdf Download (44kB) | Preview
	Pdf Activation functions performance in multilayer perceptron for time series forecasting.pdf Restricted to Repository staff only Download (1MB) | Request a copy

Abstract

Activation functions are important hyperparameters in neural networks, applied to calculate the weighted sum of inputs and biases and determine whether a neuron can be activated. Choosing the most suitable activation function can assist neural networks in training faster without sacrificing accuracy. This study aims to evaluate the performance of three activation functions, Sigmoid, Hyperbolic Tangent (Tanh), and Rectified Linear Unit (ReLU) in the hidden layer of Multilayer Perceptron (MLP) for time series forecasting. To evaluate the activation functions, three simulated non-linear time series were generated using the Threshold Autoregressive (TAR) model, and two real datasets, the Canadian Lynx series and Wolf’s Sunspot data, were employed. The Mean Square Error (MSE) and Mean Absolute Error (MAE) were computed to measure the performance accuracy. The analysis of the real data revealed that the Tanh function exhibited the lowest MSE and MAE, with values of 1.345 and 0.945, respectively. The Sigmoid function yielded MSE and MAE values of 1.520 and 1.005, while the ReLU function resulted in values of 1.562 and 1.018. These findings align with the simulation results, confirming that the Tanh function is the most effective for time series forecasting. Therefore, it is recommended to replace the commonly used Sigmoid function with Tanh for an accurate forecast.

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	Artificial neural networks, Solar activity, Energy forecasting, Time series analysis
Subjects:	Q Science > QA Mathematics
Faculty/Division:	Center for Mathematical Science
Depositing User:	Miss Amelia Binti Hasan
Date Deposited:	02 Sep 2024 05:57
Last Modified:	05 Sep 2024 04:21
URI:	http://umpir.ump.edu.my/id/eprint/42460
Download Statistic:	View Download Statistics

Actions (login required)

View Item