Question

I have started monitoring my ISP’s downtimes with a looping PHP script which checks the connection automatically every 5 seconds and stores the result in MySQL database. The scripts checks if it’s able to reach a couple of remote websites and logs the result. The time and status of the check are always stored in the database.

The structure of the table is following:

id (auto increment)
time (time stamp)
status (varchar)

Now to my issue.

I have the data, but I don’t know how to use it to achieve the result I would like to get. Basically I would like to find all the periods of time when the connection was down and for how long the connection was down.

For instance if we have 10 rows with following data

0 | 2012-07-24 22:23:00 | up
1 | 2012-07-24 22:23:05 | up
2 | 2012-07-24 22:23:10 | down
3 | 2012-07-24 22:23:16 | down
4 | 2012-07-24 22:23:21 | up
5 | 2012-07-24 22:23:26 | down
6 | 2012-07-24 22:23:32 | down
7 | 2012-07-24 22:23:37 | up
8 | 2012-07-24 22:23:42 | up
9 | 2012-07-24 22:23:47 | up

the query should return the periods (from 22:23:10 to 22:23:21, and from 22:23:26 to 22:23:37). So the query should find always the time between the first time the connection goes down, and the first time the connection is up again.

One method I thought could work was finding all the rows where the connection goes down or up, but how could I find these rows? And is there some better solution than this?

I really don’t know what the query should look like, so the help would be highly appreciated.

Thank you, regards Lassi

Answer 1

Here’s one approach.

Start by getting the status rows in order by timestamp (inline view aliased as s). Then use MySQL user variables to keep the values from previous rows, as you process through each row.

What we’re really looking for is an ‘up’ status that immediately follows a sequence of ‘down’ status. And when we find that row with the ‘up’ status, what we really need is the earliest timestamp from the preceding series of ‘down’ status.

So, something like this will work:

SELECT d.start_down
     , d.ended_down
  FROM (SELECT @i := @i + 1 AS i
             , @start := IF(s.status = 'down' AND (@status = 'up' OR @i = 1), s.time, @start) AS start_down
             , @ended := IF(s.status = 'up' AND @status = 'down', s.time, NULL) AS ended_down
             , @status := s.status
         FROM (SELECT t.time
                    , t.status
                 FROM mydata t
                WHERE t.status IN ('up','down')
                ORDER BY t.time ASC, t.status ASC
              ) s
         JOIN (SELECT @i := 0, @status := 'up', @ended := NULL, @start := NULL) i
      ) d
WHERE d.start_down IS NOT NULL
  AND d.ended_down IS NOT NULL

This works for the particular data set you show.

What this doesn’t handle (what it doesn’t return) is a ‘down’ period that is not yet ended, that is, a sequence of ‘down’ status with no following ‘up’ status.

To avoid a filesort operation to return the rows in order, you’ll want a covering index on (time,status). This query will generate a temporary (MyISAM) table to materialize the inline view aliased as d.

NOTE: To understand what this query is doing, peel off that outermost query, and run just the query for the inline view aliased as d (you can add s.time to the select list.)

This query is getting every row with an ‘up’ or ‘down’ status. The “trick” is that it is assigning both a “start” and “end” time (marking a down period) on only the rows that end a ‘down’ period. (That is, the first row with an ‘up’ status following rows with a ‘down’ status.) This is where the real work is done, the outermost query just filters out all the “extra” rows in this resultset (that we don’t need.)

SELECT @i := @i + 1 AS i
     , @start := IF(s.status = 'down' AND (@status = 'up' OR @i = 1), s.time, @start) AS start_down
     , @ended := IF(s.status = 'up' AND @status = 'down', s.time, NULL) AS ended_down
     , @status := s.status
     , s.time
  FROM (SELECT t.time
             , t.status
          FROM mydata t
         WHERE t.status IN ('up','down')
         ORDER BY t.time ASC, t.status ASC
       ) s
  JOIN (SELECT @i := 0, @status := 'up', @ended := NULL, @start := NULL) i

The purpose of inline view aliased as s is to get the rows ordered by timestamp value, so we can process them in sequence. The inline view aliased as i is just there so we can initialize some user variables at the start of the query.

If we were running on Oracle or SQL Server, we could make use of “analytic functions” or “ranking functions” (as they are named, respectively.) MySQL doesn’t provide anything like that, so we have to “roll our own”.