Question

Background

I have 4 data sets: one is weather data with time and pressure and another is a pressure sensor data set with the same; time and pressure. Essentially, both are a time series. The longer time series is the weather data which has about 64008 data points for both variables. The shorter time series for the pressure sensors is 51759. You could say that the shorter time series is a subset of the longer time series with some missing data points. Regardless, I want to get pressure for the weather but only for the times that my sensor has.

Motivation

So basically, I am trying to implement a while loop so that for every equivalent time pf my pressure sensor, and whether data, I will take the pressure from the weather data. I don’t need to record time from the weather data because I can just use the time sequence from my pressure sensor.

Example

To a get an idea of what I am talking about, I did a sample script and it runs just fine.

x(:,1) = (1:50)';
x(:,2) = (51:100)';
y(:,1) = [1:12 20:25 40:45]';

i = 1;
j = 1;
while i < numel(y)+1
     if y(i) == x(j,1)
        a(i,1) = x(j,2);
        i = i + 1;
        j = j + 1;
     else 
        j = j + 1;    
    end
end

a
% check
size(y)
size(a)

As you can see, I made a vector of x with a long series in 2 columns. And then I made a subset of values of vector y which includes data points that are contained in the x vectors. I run my script, the size of a matches y which means that the size comes out to be the same. I also saw that the matrix itself had the same values. So it works. Unless this is a simplified version where I’m missing something. Either way, my real script is below.

% Pressure Data
west_time;
west_pressure;
% Weather Data
weather_data(:,1) = weather_time;
weather_data(:,2) = weather_pressure;
% Initialize vector
weather_pressure_sensor = zeros(numel(west_time));

% Obtaining the pressure from the weather data at a 
% particular point in time when it corresponds 
% with the time from my pressure sensor
i = 1;
j = 1;
while i < numel(west_time),
   if west_time(i) == weather_data(j,1)
       weather_pressure_sensor(i,:) = weather_data(j,2);
       i = i + 1;
       j = j + 1;
   else 
       i = i;
       j = j + 1;
   end  
end

% Weather Pressure
weather_pressure_final = weather_pressure_sensor(:,2);

However, when i go to my data set, I run into an error code:

Attempted to access weather_data(64009,1); index out of
bounds because size(weather_data)=[64008,2].

Error in data_timeset2 (line 69)
    if west_time(i) == weather_data(j,1)

I was wondering if I could get some assistance with my code. Am I missing something or did I not define something? This is the way I’ve always done while loops so I don’t know why it decides to fail me now. But in any case, I’m sure it’s something really trivial and stupid but I can’t figure out for the life of me. Or maybe someone has another way…? Either way, much appreciated in advance!

Answer 1

If the time points in your data set are unique, there is a much better way to do this.

t1 = [...]; #% time series 1
t2 = [...]; #% time series 2; is a subset of t1
p1 = [...]; #% pressure series 1; same length as t1
p2 = [...]; #% pressure series 2; same length as t2

[t1, index] = sort(t1); #% make monotonic if it isn't already
p1 = p1(index); #% apply same sorting to pressures
[t2, index] = sort(t2); #% same for t2, p2
p2 = p2(index);

[Lia, Locb] = ismember(t2, t1); #% Lia contains indices of t2 that are in t1
                                #% Locb contains indices of t1 that are in t2
final_p = p1(Locb); #% get the values of p1 where t2 existed in t1