# AnalyticBridge

A Data Science Central Community

Subscribe to DSC Newsletter

# Non-Linear ARIMA using neural nets?

Anyone has done something like this? What kind on NN did you use and was it significantly better than ordinary ARIMA?

Views: 2453

### Replies to This Discussion

Hi Mehran,

I'm Burak for Turkey and I'll try to make an impression on you, about ANN and Time Series, If I can of course :)

When I was working on my graduate thesis at collage, I used to Backpropagation Network with Delta Bar Delta weight updating algorithm (offline and supervized learning). I forecast; Turkish Lira / USD Exchange Rates (Period : Daily, Range: 2002-2005 for Training, 2005-2006 for forecasting) I programmed this network on Visual Basic.Net and I'll tray to give an abstract of my results in here. I used many of types of ANNs and I decided to best way of the forecasting of time series are Feedforward Backpropagation Networks. And yes there are absolutely significant differences between ANN and other techniques espicially "When the relations of series are both not linear and unseenable easily"

In Accordance With : Mean Error Criteria

In Accordance With : Mean Absolute Error Criteria

In Accordance With : Mean Squared Error Criteria

In Accordance With : Mean Percentage Error Criteria

In Accordance With : Mean Absolute Percentage Error Criteria

I have given the most useful criterias that uses to performance analyzing of forecast. I hope you have the information about this criterias and you'll able to make comparsion of ANN and other models. By the way ;

ANN : Artificial Neural Network Model 4-8-1 with backpropagation algorithm and its diagram ;

Training Set : 03.01.2002 – 01.11.2005 n=999
Prediction Set : 02.11.2005 – 09.05.2006
Model : USD = f(USD(t-1), XU100(t-1), Gold(t-1), FED_Euro(t-1))

USD : USD/TL Exchange Rate
XU_100 : Istanbul Stock Exchange 100 Index
Gold : Istanbul Gold Market TL Price of Gold
Fed_Euro : Federal Reserve Bank of USA EURO/USD exchange rate

Activation Function : Hyperbolic Tangent
Normalization Range of Data : (0,1) (continuous)
Learning Rate : 0.01
Lambda for Adaptive Learning : 0.0001
Beta for Adaptive Learning : 0.02
Momentum Term : 0.90

REG MODEL : Multiple Regression Model with same regressors (but used to optimal lag specification)
GARCH MODEL : Generalized Autoregressive Conditional Heteroscedasticity Model with same regressors (but used to optimal lag specification)
VAR MODEL : Vector Autoregressive Model with same regressors (but used to optimal lag specification)

PS : All the time series are growth series of rates, so those were found stationary at constant level as per Augmented Dickey Fuller and DF(82) tests. so Yt and Xi,t ~I(0) i=1...k

Well, I hope to help you with my research results (actually it's 200 pages but I'm afraid it's Turkish and I don't think that you know Turkish)

Final of Final I'm going to show you both Exchange rate and ANN Forecast for outside of training
I think the last image which I uploeded here doesn't show, so I'll try to put it here.

I
Red Lines : USD/TL Growth of Daily Exchange Rates
Blu Lines : Backpropagated ANN Forecasts over the training period

Have a Nice Day :)

Burak.
Hi Burak,

Thanks for your response. Your NN forecast is impressive if it is based on a previously unseen data set, not the training set. From your diagram I can see that your topology is rather 4-8-4-1, i.e. you have two hidden layers, which in many cases makes your model overly complex.

Anyways, what I meant by this question wasn't about using an NN to do the job as in its conventional use. The idea is to further develop an ARIMA model with a regression component (which can accept multiple variables) such that the regression component be non-linear (instead of the usual linear). But remember that you still need to preserve all the good stuff from ARIMA.
You're welcome. Yes it was unseen data by network. And before I reached this arcitecture, I'd tried many of specification of ; number of neurons, hidden layers, network types etc.. So it was best choice for that time series. You know, architecture (type of network, number of neurons, hidden layers, total layer, supervised or unsupervised train, use momoent or not, value of training rate, etc...) is totally subjective and it's change sets to sets.

I think about your question and I imagine that : Each time series includes linear and non-linear elements. If you decompose data to 2 part ;

1) Linear Part
2) Non-Linear Part.

You can forecast ;

Linear Part : Using ARIMA
Non-Linear Part : Using ANN (as far as I'm concerned it has to be Backpropagation Network)

So, you have got an Hybride Sytem that consist of ANN + ARIMA. I think in this way, we have got the power of ARIMA on forecasting Linear Data and also power of NN on forecasting Non-Linear Data. Finally your Forecast consist of : sum of ANN and ARIMA forecasts.

Yt = NLYt + LYt ----> Decomposed Linear and Non-Linear Parts

Frcst(NLYt) ---> Taken by ANN
Frcst(LYt) ---> Taken by ARIMA

Frcst(Yt) = Frcst(NLYt) + Frcst(LYt)

There are some techniques for decomposition of data (Moving Average, Multiplicative Dec., Additive Dec., X11 and X12) You must use one suitable of these techniques.

I haven't tried this way, but I guess this hybride system which I defined above, more powerful both ANN and ARIMA for forecasting non-linear time series. If I try it one day, I'll inform you about results.

Burak.
Hi, Burak:
I have been using ARIMA for sales forecast for my company and I came across this hybrid method that decompose series into linear and non-linear parts. That result that you posed seems very promising and I would like to learn more about it. Could you point me to some resources that will give me some introduction about this? Most of all, I am using SAS/ETS for my forecast and I need to know how to implement this method. Do I have to learn how to program in Visual Basic.net? Thanks.
Perhaps you should check the work of Johan Suykens, University of Leuven, Belgium. He wa my promotor during my PhD and I remember that he did a lot of research in this area.

Succes.

Kind regards,

Bart

## On Data Science Central

© 2020   TechTarget, Inc.   Powered by