Question

I’m looking for a way to ignore specific entries in matrices for subsequent linear regression in MATLAB

I have two matricies: y =

    9.3335    7.8105    5.8969    3.5928
    23.1580   19.6043   15.3085    8.2010
    40.1067   35.2643   28.9378   16.6753
    56.4697   51.8224   44.5587   29.3674
    70.7238   66.5842   58.8909   42.7623
    83.0253   78.4561   71.1924   53.8532

and x =

    300   300   300   300
    400   400   400   400
    500   500   500   500
    600   600   600   600
    700   700   700   700
    800   800   800   800

I need to do linear regression on the points where y is between 20 and 80, so I need a way to fully automate the process. I tried making the outlying y values [and their corresponding x values] NaNs, but during linear regression, matlab included the NaNs in the calculations so I got NaN outputs. Can anyone suggest a good way to ignore those entries or to ignore NaNs completely calculations? (NOTE: the columns in y will often have different combinations of values, so I can’t eliminate the whole row).

Answer 1

Since you perform your regression on each column separately, you can simply form an index into rows with the valid y-values:

nCols = size(x,2);

results = zeros(2,nCols);
validY = y>20 & y<80; %# a logical array the size of y with valid entries
nValid = sum(validY,1);

for c = 1:nCols
    % results is [slope;intercept] in each column
    results(:,c) = [x(validY(:,c),c),ones(nValid(c),1)]\y(validY(:,c),c);

end