A Simple Heuristic Approach to Improve Performance of Extreme Learning Machine
AbstractNeural networks (NNs) is used to solve many engineering and science problem. Generally, feedforward architecture is preferred and gradient-based learning algorithms are extensively operated to tune all parameters of NN iteratively. This training method is a conventional one, but training process takes a long time due to the slowness of gradient-based learning algorithms. This slowness has been an important drawback in their applications. To overcome this disadvantage, extreme learning machine (ELM) concept introduced to science community in near past. Essentially, ELM is a data-driven learning algorithm for single-hidden layer feedforward neural networks (SLFNs). This algorithm provides extremely fast learning speed. In this study, performance of SLFNs learned by ELM algorithm is investigated on the problem of highly nonlinear dynamic system identification. As a result of studies on selected benchmark problems in the literature, it has been seen that ELM may not provide a good generalization success due to randomly chosen the number of hidden nodes and weight parameters for inputs in SLFN. For both the training and the test data set, very poor results have been obtained and observed surprisingly during the above-mentioned studies. Here, a simple heuristic approach has been proposed in this study in order to eliminate this bad situation and the findings obtained with this approach are discussed. Based on the obtained experimental results, it has been shown that the proposed approach determines the optimal the number of hidden nodes and a reasonable random selection of input weights required for a good generalization performance.