Forecasting of Sea Level Time Series using RNN and LSTM Case Study in Sunda Strait

Sea-level forecasting is essential for coastal development planning and minimizing their significant consequences in coastal operations, such as naval engineering and navigation. Conventional sea level predictions, such as tidal harmonic analysis, do not consider the influence of non-tidal elements and require long-term historical sea level data. In this paper, two deep learning approaches are applied to forecast sea level. The first deep learning is Recurrent Neural Network (RNN), and the second is Long Short Term Memory (LSTM). Sea level data was obtained from IDSL (Inexpensive Device for Sea Level Measurement) at Sebesi, Sunda Strait, Indonesia. We trained the model for forecasting 3, 5, 7, 10, and 14 days using three months of hourly data in 2020 from 1st May to 1st August. We compared forecasting results with RNN and LSTM with the results of the conventional method, namely tidal harmonic analysis. The LSTM’s results showed better performance than the RNN and the tidal harmonic analysis, with a correlation coefficient of R2 0.97 and an RMSE value of 0.036 for the 14 days prediction. Moreover, RNN and LSTM can accommodate non-tidal harmonic data such as sea level anomalies.


Introduction
Information on the sea level is necessary for the operation of ports, in particular, for the scheduling of naval transportation and the port services operation [1]. Ships with deep draft depend on the sea level forecasting for their navigation in a shallow coastal area. Moreover, historical data on the sea level is vital for the planning of coastal and offshore buildings, and the sea level prediction is necessary during the construction phase of the structures [2]. Another importance of predicting sea level is to gather information on climate change how likely the rise of sea level is in the future so that we can minimize the consequent, especially in a coastal area such as flooding [3]. The sea level consists of two general elements, tidal and non-tidal elements. The dominant forces shaping the sea level are the tidal elements because tidal is the most stable pattern in oceanographic [4]. The non-tidal elements come from wind or other occurrences that are affecting the sea level [5]. Tidal harmonic analysis (THA) is a conventional way to forecast sea level. THA assumes sea LONTAR KOMPUTER VOL. 12 level as a superposition of harmonic components or tidal representatives. Typically, using Least Square Estimation (LSE), the tidal components in a given area can be reliably calculated using the measured sea level. However, the forecasted sea level is the tidal parts only since THA cannot predict the non-tidal components. Another major downside, THA needs long historical sea level data to extract all tidal elements.
Various attempts proposed to improve the accuracy of the THA method for predicting the sea level, such as using multivariate least-square harmonic estimation [6] and combining the LSE method with the Inaction Method (IM) [7]. Moreover, there is also another approach for predicting sea-level namely ARIMA (Autoregressive Integrated Moving Average) [8], SARIMA (Seasonal Autoregressive Integrated Moving Averaged) [9], and Holtz-Winters Exponential Smoothing [10]. The prediction performance from ARIMA, SARIMA, and Holtz-Winters for seven prediction days measured by RMSE are 0.226, 0.155, and 0.134, respectively. These results offer a challenge to finding a better predictive model. ARIMA, SARIMA, and Holt-Winter approaches are methods based on time series analysis. Machine learning approaches such as ANN (Artificial Neural Network) to forecast sea-level is discussed in [11] where ANN is applied to predict sea level anomalies. In 2018, Imani et al. [12] applied relevance vector machine models and extreme learning machines to predict sea-level. Rizkina et al. [13] in 2019 compared the nonlinear autoregressive (NAR) neural network and tidal harmonic analysis to obtain a short-term prediction of sea level in Semarang. Using the NAR method, they obtain prediction with an R-value of 0.9566 and an RMSE value of 0.0736.
This research aims to investigate a reliable sea-level prediction approach that needs only reasonably short-term historical data. Sea level prediction is carried out by applying the Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM), two popular deep learning methods. RNN is a type of neural network designed to handle sequential data. Not only suitable for solving classification problems in Natural Language Problem (NLP), the RNN is also suitable for predicting time series data. However, RNN often encounters problems such as vanishing the gradient, where the loss function decreases exponentially. It interferes with the learning process. LSTM is an improvement RNN to avoid this problem. LSTM architecture consists of a memory cell, input gate, forget gate, and output gate. As a study area, we chose the Sebesi station in Sunda Strait, Indonesia. Sea level data comes from measurements by IDSL or Inexpensive Device for Sea Level measurement. The RNN and LSTM in predicting the sea level is then compared to the traditional approach using THA. Figure 1 shows our research methodology. The study utilizes the sea level data obtained from the IDSL at Sebesi station in Sunda Strait, Indonesia. The raw data is pre-processed to remove noises as well as to fill in missing data. The data set is then separated into training also testing data and then tested for three forecasting methods, the traditional THA and the deep learning methods using the RNN and LSTM. We performed several forecasting scenarios based on the length of days to forecast for 3, 5, 7, 10, and 14 days. The prediction results from these three methods are then compared qualitatively and quantitatively. The performance is measured using the correlation coefficients (R 2 ) together with the Root Mean Square Error (RMSE). These steps in the methodology of this paper are explained in more detail in Figure 1.

Sea Level Data
The data are collected from a sea-level measurement device called IDSL (Inexpensive Device for Sea Level measurement). The IDSL is a joint effort between the European Commission's Joint Research Centre, The Marine Research Centre of MMAF, and The Indonesia Society of Tsunami Experts (IATSI). The station supports the tsunami early warning system in the Sunda Strait due to its proximity to the Krakatau Volcano that generated the tsunami in 2018 [14]. The Sebesi station was selected for IDSL due to its important location as a tourism destination and community livelihoods. Monitoring of sea levels due to global climate change is also vital for sealevel rise risk mitigation. The location of Sebesi Station is between the lines 5°56' 9.7692" South Latitude and 105°30' 43.5816" East Longitude as shown in Figure 2.

Preprocessing Data
The sea level data from the IDSL is recorded initially every 20 seconds. It is unnecessary to use detailed temporal spacing data for sea-level forecasting since they include unnecessary harmonic components, especially high frequencies. Instead, we use hourly data for sea level forecasting. Moreover, the measured sea level data contained missing values, which will lead to a problem for the forecasting process by using a deep learning approach. To that end, the original 20s temporal data is transformed into hourly data by calculating the average.
In this paper we use 3.5 months data, i.e. 1 th May -15 th August 2020. The original 20 s temporal data is shown in the top of Figure 3, whereas the hourly sea level data is depicted in the middle of Figure 3. The missing values in the original data are filled with interpolated data shown in the lower plot of Figure 3.
The preprocessed data is separated into training and testing data as depicted by Figure 4. We use three months of sea level data, 1 th May -31 th July 2020 for training to forecast 3, 5, 7, 10, and 14 days ahead.

Tidal Harmonic Analysis
As stated in [15], the sea level is mainly generated by tidal waves, which are strongly influenced by the moon and the sun. The tide is affected by the relative astronomical location of the earth with the moon and the sun. Changes in the astronomical location lead to the rising and dropping of the sea level. These astronomical forces that shape the tide could be considered as a summation of harmonic components, which is the core principle of the Tidal Harmonic Analysis (THA) method [6,7]. This fact makes the tide the most predictable oceanographic physical phenomenon, especially compared to wind and waves [4]. As a result, the tide can be expressed as (1) Here, h 0 is the mean of sea level or MSL, f i is the node factor, H i is the i th amplitude component of the tidal component-i, ω i , and g i are the the i th tidal constituent and its phase, respectively. High and low tides can be calculated by searching the critical value of the formula (1), by calculating the first derivative of the formula, equated to zero as in equation (2). Nevertheless, the equation ( cannot capture the variation in the peak tide since the peak of superposition of harmonic waves in the equation (1) are constant and appear periodically.
There is no analytic solution for the equation (2). The parameters H i , f i , ω i , and g i are estimated using the Least Square Estimation (LSE) method with measured sea level as the reference in many tidal harmonic analysis methods [16]. As a consequence of the THA approach, the THA method cannot predict non-tidal elements at sea level. Moreover, the THA method requires a relatively long historical sea level data set to correctly describe the low frequencies component of the sea level.

Recurrent Neural Network
RNN is a family of artificial neural network (ANN) architecture that processes interrelated repetitive input typically in sequential data, including time-series data. The RNN differs from the traditional Neural Network. Each processing produced in the RNN is influenced by the current input and an internal state resulting from the previous input processing. When the RNN makes a decision, the time step t − 1 can influence the decision to be taken at time step t [17]. The RNN is a method commonly used for forecasting problems. In [18], the RNN is used for forecasting wind speed, where they also compare the method with the ARIMA model. In [19], the RNN is used for forecasting for electricity consumption in medium-to-long-term, where they conclude that the RNN produces better prediction than the three-layer multilayer perceptron model.
The architecture of the RNN can be illustrated as in Figure 5. Here, for each time step t, the state S t is calculated from the input X t and the prior state S t-1 by following the formula (3) Here, W ix and W sx are the weight matrix at input and hidden layer respectively, and b x denotes the bias. The function tanh(x) is an activation function that is defined by the following formula where the range of function tanh(x) is from −1 to 1.
The output value Y t is given by the following formula where Y t denotes the output, W ox is the weighted matrix at output layer, S t is the states, and b y is the bias.

Long Short-Term Memory
LSTM was initiated by Hochreiter and Schmidhuber [20]. It is an improved version of RNN where LSTM is equipped with a memory cell that could reserve information for a long time. Vanishing and exploding gradient problems are problems in the RNN that fail to capture long-term dependencies, thereby reducing the accuracy of a prediction on RNN [21]. Meanwhile, the LSTM could store or discard data since each neuron has several gates that regulate the memory of each neuron. Figure 6 shows the layout of the gates of LSTM. In LSTM there are three gates, namely f t , i t , and o t as shown in Figure 6. Gate f t is the forget gate, i t is the input gate, and o t is the output gate [2].
The first step in assembling the LSTM is to differentiate the necessary and unnecessary data in the first gate (forget-gate). A sigmoid function (6) defines this process. If the value of the sigmoid function is 0 then the data will be discarded. If the value is 1 then the data will be updated or passed through. X t is the current input value and S t−1 is the hidden state of the prior value. W and b are coefficients where their values are determined from the training process. The next process is in the input gate. The input gate controls how many states from the current input will pass through. The output is determined by the sigmoid function (7) in the first layer, and the hyperbolic tangent function (8) in the second layer, where the value of the cell state C t is defined in equation (9).
And the final gate unit component is the output gate, which decides the internal state forward and the cell state or memory cell to forward old information with additional new information to the next cell state.
By following the calculation at the output gate as defined in equation (10), the output value, namely the cell state value C t will be forwarded for the next memory cell calculation, and the current hidden state S t , will be generated using equation (8).

Evaluation
In this evaluation section, the optimal model search process is carried out, especially for using deep learning models, namely RNN and LSTM. In searching for the optimal model for this deep learning model, some stages are carried out. The first is searching for the optimal model by calculating the scoring metric using the Correlation Coefficient and Mean Square Error. If in the first stage of the search for the model the best model is obtained, then in the second stage by finding the optimal number of lookbacks from the deep learning model by taking into account the computational time in the training process itself, by using these two stages the model with the best results is obtained.

Performance Analysis
The performance of both RNN and LSTM is evaluated by calculating the Root Mean Square Error (RMSE) and R 2 to see which one is better. The RMSE is a formula for measuring the average error and is written as follows.
In equation (12), N is the total value of input data,ŷ i is the output of the prediction value and y i is the target value [22].
The coefficient of determination R 2 is a useful quantitative usually used to measure how well the output of a model in predicting the target. Values of R 2 are between 0 and 1 [23]. The R 2 is defined as follows in equation (13)(14)(15), A positive MBE means that the predictions are overestimating, and a negative MBE means an underestimation.

Result and Discussion
As explained in the previous section, we use the sea level data from 1 th May -15 th August 2020 at Sebesi Station. Using three months of sea level data for training data for the Tidal Harmonic Analysis, RNN, and LSTM model, predict the next 3 to 14 days ahead. Based on Table 1, the LSTM model shows better performance in predicting sea-level compared to the Tidal Harmonic Analysis and RNN model.   Table 1, the RMSE value of RNN is smaller than the RMSE value of THA for all prediction days. It shows that prediction with RNN provides a closer estimate than THA. The result is consistent with the value of the coefficient deterministic R 2 . The coefficient deterministic of RNN is closer to one than THA. It means that the predicted results of RNN have a better match with the original data than THA. However, prediction with LSTM gives better results than RNN for all prediction days, both measured by RMSE and by R 2 . As an improved version of RNN, the performance of LSTM is better than RNN due to its ability to handle information for a long time that overcomes vanishing and exploding gradient problems in RNNs. By analyzing the length of the prediction day, it can be seen that the longer the prediction day, the better the prediction result. On longer prediction days, prediction errors will be more distributed. Qualitative comparison between the prediction for 14 days with Tidal Harmonic Analysis, RNN, and LSTM is shown in Figure 7. Based on Figure 7 we can see that the prediction results using THA cannot accurately estimate the sea level around the top and bottom. The predictive graph of THA is periodic and very well accommodates the tidal component of the sea level but fails to accommodate the non-tidal element that affects the sea level. top and bottom of the sea level. These predictions are also better than the previous prediction methods using ARIMA, SARIMA, and Holt-Winters for testing with seven prediction days which give RMSE 0.226, 0.155, and 0.134, respectively [10]. The RMSE values of RNN and LSTM for seven prediction days are0.054 and 0.039, respectively.

Conclusion
We predict sea level using deep learning approaches, the Recurrent Neural Network together with Long Short-Term Memory. As training data, we use three months time series of sea level data to train the model to predict 3, 5, 7, 10, and 14 days. We conducted a prediction with Tidal Harmonic Analysis, RNN, and LSTM method. All methods we used can predict with reasonably good performance and follow the measured value of the buoy. However, the deep learning approach shows better prediction than the Tidal Harmonic Analysis. Moreover, LSTM shows performance improvement in predicting long-term sea-level data (14 days) with a R 2 value of 0.97 and RMSE value of 0.036. These results are better than RNN with a R 2 value of 0.96 and the RMSE value of 0.047.