Non-Linear ARIMA using neural nets? - AnalyticBridge2020-10-25T11:42:18Zhttps://www.analyticbridge.datasciencecentral.com/forum/topics/2004291:Topic:1113?commentId=2004291%3AComment%3A7305&feed=yes&xn_auth=noPerhaps you should check the…tag:www.analyticbridge.datasciencecentral.com,2009-06-09:2004291:Comment:462922009-06-09T13:39:30.974ZBart Hamershttps://www.analyticbridge.datasciencecentral.com/profile/BartHamers
Perhaps you should check the work of Johan Suykens, University of Leuven, Belgium. He wa my promotor during my PhD and I remember that he did a lot of research in this area.<br />
<br />
Succes.<br />
<br />
Kind regards,<br />
<br />
Bart
Perhaps you should check the work of Johan Suykens, University of Leuven, Belgium. He wa my promotor during my PhD and I remember that he did a lot of research in this area.<br />
<br />
Succes.<br />
<br />
Kind regards,<br />
<br />
Bart Hi, Burak:
I have been using…tag:www.analyticbridge.datasciencecentral.com,2009-06-08:2004291:Comment:462422009-06-08T15:37:22.932ZYi-Chun Tsaihttps://www.analyticbridge.datasciencecentral.com/profile/YiChunTsai
Hi, Burak:<br />
I have been using ARIMA for sales forecast for my company and I came across this hybrid method that decompose series into linear and non-linear parts. That result that you posed seems very promising and I would like to learn more about it. Could you point me to some resources that will give me some introduction about this? Most of all, I am using SAS/ETS for my forecast and I need to know how to implement this method. Do I have to learn how to program in Visual Basic.net? Thanks.
Hi, Burak:<br />
I have been using ARIMA for sales forecast for my company and I came across this hybrid method that decompose series into linear and non-linear parts. That result that you posed seems very promising and I would like to learn more about it. Could you point me to some resources that will give me some introduction about this? Most of all, I am using SAS/ETS for my forecast and I need to know how to implement this method. Do I have to learn how to program in Visual Basic.net? Thanks. You're welcome. Yes it was un…tag:www.analyticbridge.datasciencecentral.com,2008-03-18:2004291:Comment:75532008-03-18T21:21:24.985ZBurak Çelikhttps://www.analyticbridge.datasciencecentral.com/profile/Burak
You're welcome. Yes it was unseen data by network. And before I reached this arcitecture, I'd tried many of specification of ; number of neurons, hidden layers, network types etc.. So it was best choice for that time series. You know, architecture (type of network, number of neurons, hidden layers, total layer, supervised or unsupervised train, use momoent or not, value of training rate, etc...) is totally subjective and it's change sets to sets.<br />
<br />
I think about your question and I imagine that…
You're welcome. Yes it was unseen data by network. And before I reached this arcitecture, I'd tried many of specification of ; number of neurons, hidden layers, network types etc.. So it was best choice for that time series. You know, architecture (type of network, number of neurons, hidden layers, total layer, supervised or unsupervised train, use momoent or not, value of training rate, etc...) is totally subjective and it's change sets to sets.<br />
<br />
I think about your question and I imagine that : Each time series includes linear and non-linear elements. If you decompose data to 2 part ;<br />
<br />
1) Linear Part<br />
2) Non-Linear Part.<br />
<br />
You can forecast ;<br />
<br />
Linear Part : Using ARIMA<br />
Non-Linear Part : Using ANN (as far as I'm concerned it has to be Backpropagation Network)<br />
<br />
So, you have got an Hybride Sytem that consist of ANN + ARIMA. I think in this way, we have got the power of ARIMA on forecasting Linear Data and also power of NN on forecasting Non-Linear Data. Finally your Forecast consist of : sum of ANN and ARIMA forecasts.<br />
<br />
Yt = NLYt + LYt ----> Decomposed Linear and Non-Linear Parts<br />
<br />
Frcst(NLYt) ---> Taken by ANN<br />
Frcst(LYt) ---> Taken by ARIMA<br />
<br />
Frcst(Yt) = Frcst(NLYt) + Frcst(LYt)<br />
<br />
There are some techniques for decomposition of data (Moving Average, Multiplicative Dec., Additive Dec., X11 and X12) You must use one suitable of these techniques.<br />
<br />
I haven't tried this way, but I guess this hybride system which I defined above, more powerful both ANN and ARIMA for forecasting non-linear time series. If I try it one day, I'll inform you about results.<br />
<br />
Burak. Hi Burak,
Thanks for your re…tag:www.analyticbridge.datasciencecentral.com,2008-03-18:2004291:Comment:73052008-03-18T02:08:08.465ZMehran Gomarihttps://www.analyticbridge.datasciencecentral.com/profile/MehranGomari
Hi Burak,<br />
<br />
Thanks for your response. Your NN forecast is impressive if it is based on a previously unseen data set, not the training set. From your diagram I can see that your topology is rather 4-8-4-1, i.e. you have two hidden layers, which in many cases makes your model overly complex.<br />
<br />
Anyways, what I meant by this question wasn't about using an NN to do the job as in its conventional use. The idea is to further develop an ARIMA model with a regression component (which can accept multiple…
Hi Burak,<br />
<br />
Thanks for your response. Your NN forecast is impressive if it is based on a previously unseen data set, not the training set. From your diagram I can see that your topology is rather 4-8-4-1, i.e. you have two hidden layers, which in many cases makes your model overly complex.<br />
<br />
Anyways, what I meant by this question wasn't about using an NN to do the job as in its conventional use. The idea is to further develop an ARIMA model with a regression component (which can accept multiple variables) such that the regression component be non-linear (instead of the usual linear). But remember that you still need to preserve all the good stuff from ARIMA. I think the last image which…tag:www.analyticbridge.datasciencecentral.com,2008-03-14:2004291:Comment:62962008-03-14T06:19:58.553ZBurak Çelikhttps://www.analyticbridge.datasciencecentral.com/profile/Burak
I think the last image which I uploeded here doesn't show, so I'll try to put it here.<br />
<br />
<p style="text-align: left;"><img src="http://storage.ning.com/topology/rest/1.0/file/get/2167222214?profile=original" alt="" width="795" height="541"/></p>
I<br />
Red Lines : USD/TL Growth of Daily Exchange Rates<br />
Blu Lines : Backpropagated ANN Forecasts over the training period<br />
<br />
Have a Nice Day :)<br />
<br />
Burak.
I think the last image which I uploeded here doesn't show, so I'll try to put it here.<br />
<br />
<p style="text-align: left;"><img src="http://storage.ning.com/topology/rest/1.0/file/get/2167222214?profile=original" alt="" width="795" height="541"/></p>
I<br />
Red Lines : USD/TL Growth of Daily Exchange Rates<br />
Blu Lines : Backpropagated ANN Forecasts over the training period<br />
<br />
Have a Nice Day :)<br />
<br />
Burak. Hi Mehran,
I'm Burak for Tur…tag:www.analyticbridge.datasciencecentral.com,2008-03-14:2004291:Comment:62912008-03-14T06:05:44.015ZBurak Çelikhttps://www.analyticbridge.datasciencecentral.com/profile/Burak
Hi Mehran,<br />
<br />
I'm Burak for Turkey and I'll try to make an impression on you, about ANN and Time Series, If I can of course :)<br />
<br />
When I was working on my graduate thesis at collage, I used to Backpropagation Network with Delta Bar Delta weight updating algorithm (offline and supervized learning). I forecast; Turkish Lira / USD Exchange Rates (Period : Daily, Range: 2002-2005 for Training, 2005-2006 for forecasting) I programmed this network on Visual Basic.Net and I'll tray to give an abstract of…
Hi Mehran,<br />
<br />
I'm Burak for Turkey and I'll try to make an impression on you, about ANN and Time Series, If I can of course :)<br />
<br />
When I was working on my graduate thesis at collage, I used to Backpropagation Network with Delta Bar Delta weight updating algorithm (offline and supervized learning). I forecast; Turkish Lira / USD Exchange Rates (Period : Daily, Range: 2002-2005 for Training, 2005-2006 for forecasting) I programmed this network on Visual Basic.Net and I'll tray to give an abstract of my results in here. I used many of types of ANNs and I decided to best way of the forecasting of time series are Feedforward Backpropagation Networks. And yes there are absolutely significant differences between ANN and other techniques espicially "When the relations of series are both not linear and unseenable easily"<br />
<br />
In Accordance With : Mean Error Criteria<br />
<br />
<p style="text-align: left;"><img src="http://storage.ning.com/topology/rest/1.0/file/get/2167224770?profile=original" alt="" width="456" height="175"/></p>
<br />
In Accordance With : Mean Absolute Error Criteria<br />
<br />
<p style="text-align: left;"><img src="http://storage.ning.com/topology/rest/1.0/file/get/2167224975?profile=original" alt="" width="456" height="175"/></p>
<br />
In Accordance With : Mean Squared Error Criteria<br />
<br />
<p style="text-align: left;"><img src="http://storage.ning.com/topology/rest/1.0/file/get/2167225045?profile=original" alt="" width="456" height="175"/></p>
<br />
In Accordance With : Mean Percentage Error Criteria<br />
<br />
<p style="text-align: left;"><img src="http://storage.ning.com/topology/rest/1.0/file/get/2167225299?profile=original" alt="" width="456" height="175"/></p>
<br />
In Accordance With : Mean Absolute Percentage Error Criteria<br />
<br />
<p style="text-align: left;"><img src="http://storage.ning.com/topology/rest/1.0/file/get/2167225686?profile=original" alt="" width="456" height="184"/></p>
<br />
I have given the most useful criterias that uses to performance analyzing of forecast. I hope you have the information about this criterias and you'll able to make comparsion of ANN and other models. By the way ;<br />
<br />
ANN : Artificial Neural Network Model 4-8-1 with backpropagation algorithm and its diagram ;<br />
<br />
<p style="text-align: left;"><img src="http://storage.ning.com/topology/rest/1.0/file/get/2167225926?profile=original" alt="" width="531" height="527"/></p>
<br />
Training Set : 03.01.2002 – 01.11.2005 n=999<br />
Prediction Set : 02.11.2005 – 09.05.2006<br />
Model : USD = f(USD(t-1), XU100(t-1), Gold(t-1), FED_Euro(t-1))<br />
<br />
USD : USD/TL Exchange Rate<br />
XU_100 : Istanbul Stock Exchange 100 Index<br />
Gold : Istanbul Gold Market TL Price of Gold<br />
Fed_Euro : Federal Reserve Bank of USA EURO/USD exchange rate<br />
<br />
Activation Function : Hyperbolic Tangent<br />
Normalization Range of Data : (0,1) (continuous)<br />
Learning Rate : 0.01<br />
Lambda for Adaptive Learning : 0.0001<br />
Beta for Adaptive Learning : 0.02<br />
Momentum Term : 0.90<br />
<br />
REG MODEL : Multiple Regression Model with same regressors (but used to optimal lag specification)<br />
GARCH MODEL : Generalized Autoregressive Conditional Heteroscedasticity Model with same regressors (but used to optimal lag specification)<br />
VAR MODEL : Vector Autoregressive Model with same regressors (but used to optimal lag specification)<br />
<br />
PS : All the time series are growth series of rates, so those were found stationary at constant level as per Augmented Dickey Fuller and DF(82) tests. so Yt and Xi,t ~I(0) i=1...k<br />
<br />
Well, I hope to help you with my research results (actually it's 200 pages but I'm afraid it's Turkish and I don't think that you know Turkish)<br />
<br />
Final of Final I'm going to show you both Exchange rate and ANN Forecast for outside of training