Question

Intro: I’m using MATLAB’s Neural Network Toolbox in an attempt to forecast time series one step into the future. Currently I’m just trying to forecast a simple sinusoidal function, but hopefully I will be able to move on to something a bit more complex after I obtain satisfactory results.

Problem: Everything seems to work fine, however the predicted forecast tends to be lagged by one period. Neural network forecasting isn’t much use if it just outputs the series delayed by one unit of time, right?

Code:

t = -50:0.2:100;
noise = rand(1,length(t));
y = sin(t)+1/2*sin(t+pi/3);
split = floor(0.9*length(t));
forperiod = length(t)-split;
numinputs = 5;
forecasted = [];
msg = '';
for j = 1:forperiod
    fprintf(repmat('\b',1,numel(msg)));
    msg = sprintf('forecasting iteration %g/%g...\n',j,forperiod);
    fprintf('%s',msg);

    estdata = y(1:split+j-1);
    estdatalen = size(estdata,2);

    signal = estdata;
    last = signal(end);

    [signal,low,high] = preprocess(signal'); % pre-process
    signal = signal';

    inputs = signal(rowshiftmat(length(signal),numinputs));
    targets = signal(numinputs+1:end);

    %% NARNET METHOD
    feedbackDelays = 1:4;
    hiddenLayerSize = 10;
    net = narnet(feedbackDelays,[hiddenLayerSize hiddenLayerSize]);
    net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};
    signalcells = mat2cell(signal,[1],ones(1,length(signal)));
    [inputs,inputStates,layerStates,targets] = preparets(net,{},{},signalcells);
    net.trainParam.showWindow = false;
    net.trainparam.showCommandLine = false;
    net.trainFcn = 'trainlm';  % Levenberg-Marquardt
    net.performFcn = 'mse';  % Mean squared error
    [net,tr] = train(net,inputs,targets,inputStates,layerStates);
    next = net(inputs(end),inputStates,layerStates);


    next = postprocess(next{1}, low, high); % post-process
    next = (next+1)*last;

    forecasted = [forecasted next];
end

figure(1);
plot(1:forperiod, forecasted, 'b', 1:forperiod, y(end-forperiod+1:end), 'r');
grid on;

Note:
The function ‘preprocess’ simply converts the data into logged % differences and ‘postprocess’ converts the logged % differences back for plotting. (Check EDIT for preprocess and postprocess code)

Results:

BLUE: Forecasted Values

RED: Actual Values

Can anyone tell me what I’m doing wrong here? Or perhaps recommend another method to achieve the desired results (lagless prediction of sinusoidal function, and eventually more chaotic timeseries)? Your help is very much appreciated.

EDIT:
It’s been a few days now and I hope everyone has enjoyed their weekend. Since no solutions have emerged I’ve decided to post the code for the helper functions ‘postprocess.m’, ‘preprocess.m’, and their helper function ‘normalize.m’. Maybe this will help get the ball rollin.

postprocess.m:

function data = postprocess(x, low, high)

% denormalize
logdata = (x+1)/2*(high-low)+low;

% inverse log data
sign = logdata./abs(logdata);
data = sign.*(exp(abs(logdata))-1);

end

preprocess.m:

function [y, low, high] = preprocess(x)

% differencing
diffs = diff(x);
% calc % changes
chngs = diffs./x(1:end-1,:);
% log data
sign = chngs./abs(chngs);
logdata = sign.*log(abs(chngs)+1);
% normalize logrets
high = max(max(logdata));
low = min(min(logdata));
y=[];
for i = 1:size(logdata,2)
    y = [y normalize(logdata(:,i), -1, 1)];
end

end

normalize.m:

function Y = normalize(X,low,high)
%NORMALIZE Linear normalization of X between low and high values.

if length(X) <= 1
    error('Length of X input vector must be greater than 1.');
end

mi = min(X);
ma = max(X);
Y = (X-mi)/(ma-mi)*(high-low)+low;

end

Answer 1

I didn’t check you code, but made a similar test to predict sin() with NN. The result seems reasonable, without a lag. I think, your bug is somewhere in synchronization of predicted values with actual values.
Here is the code:

%% init & params
t = (-50 : 0.2 : 100)';
y = sin(t) + 0.5 * sin(t + pi / 3);
sigma = 0.2;
n_lags = 12;
hidden_layer_size = 15;
%% create net
net = fitnet(hidden_layer_size);
%% train
noise = sigma * randn(size(t));
y_train = y + noise;
out = circshift(y_train, -1);
out(end) = nan;
in = lagged_input(y_train, n_lags);
net = train(net, in', out');
%% test
noise = sigma * randn(size(t)); % new noise
y_test = y + noise;
in_test = lagged_input(y_test, n_lags);
out_test = net(in_test')';
y_test_predicted = circshift(out_test, 1); % sync with actual value
y_test_predicted(1) = nan;
%% plot
figure, 
plot(t, [y, y_test, y_test_predicted], 'linewidth', 2); 
grid minor; legend('orig', 'noised', 'predicted')

and the lagged_input() function:

function in = lagged_input(in, n_lags)
    for k = 2 : n_lags
        in = cat(2, in, circshift(in(:, end), 1));
        in(1, k) = nan;
    end
end