Question

I have a MySQL database that I am keeping temperature readings from several different sensors. I initially thought of using three different tables to store my data:

mysql> select * from sensor_info;
+----+------------------+------+--------+
| id | address          | name | active |
+----+------------------+------+--------+
|  1 | 28D684CD02000057 | NULL |      1 |
|  2 | 28099B49030000D8 | NULL |      1 |
|  3 | 28339ACD0200004B | NULL |      1 |
+----+------------------+------+--------+

mysql> select * from data_period limit 4;
+----+---------------------+
| id | ts                  |
+----+---------------------+
|  1 | 2012-06-30 09:35:02 |
|  2 | 2012-06-30 09:36:22 |
|  3 | 2012-06-30 09:37:46 |
|  4 | 2012-06-30 09:40:36 |
+----+---------------------+

mysql> select * from data_points limit 4;
+----+-------------+-----------+-------+
| id | data_period | sensor_id | data  |
+----+-------------+-----------+-------+
|  1 |           1 |         1 | 77.90 |
|  2 |           1 |         2 | 77.34 |
|  3 |           1 |         3 | 77.56 |
|  4 |           2 |         1 | 78.01 |
+----+-------------+-----------+-------+

What I’m trying to do is to take my stored data and put it into a CSV file so I can display it using dygraphs Javascript library. I need to get my data into a format like this:

date,temp1,temp2,temp3
2012-06-30 09:35:02,77.90,77.34,77.56
2012-06-30 09:36:22,78.01,77.36,77.59
....

Every way I start to do this (using PHP), I seem to make this overly complicated and have to put queries inside loops inside loops. Am I making this harder on myself than I need to?

Will most of the work be done using the queries or using PHP? Down the road, I will also want to add code that will place NULL in the CSV if a temperature reading is missing from a particular timestamp.

I’m not looking for a very specific answer, I just want to know what direction I should go. I don’t even know how to start to format my data from the database or if I should try looking at a different format to store my info in the database.

Answer 1

I’d run a single select query to join the lot together. You can use an outer join where there might not be data.

SELECT data_period.ts AS date, dp1.data AS temp1, dp2.data AS temp2, dp3.data AS temp3
FROM data_period
LEFT OUTER JOIN data_points AS dp1 ON dp1.data_period=data_period.id AND dp1.sensor_id=1
LEFT OUTER JOIN data_points AS dp2 ON dp2.data_period=data_period.id AND dp2.sensor_id=2
LEFT OUTER JOIN data_points AS dp3 ON dp3.data_period=data_period.id AND dp3.sensor_id=3

(See: SQL Fiddle Example )

That should give you a single set of results that you can just loop through.

If you really want MySQL to do most of the work, you can change the first line to (I think)
SELECT data_period.ts +’,’+ IFNULL(dp1.data,’NULL’) +’,’+ IFNULL(dp2.data,’NULL’) +’,’+ IFNULL(dp3.data,’NULL’)

To Synthesize the comment and the answer and to get this out to a file:

SELECT data_period.ts AS date, dp1.data AS temp1, dp2.data AS temp2, dp3.data AS temp3
INTO OUTFILE '/home/user/output.csv'
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '\\'
LINES TERMINATED BY '\n'
FROM data_period LEFT OUTER JOIN data_points AS dp1 ON dp1.data_period=data_period.id      
AND dp1.sensor_id=1 LEFT OUTER JOIN data_points AS dp2 ON  
dp2.data_period=data_period.id   AND dp2.sensor_id=2 LEFT OUTER JOIN data_points AS dp3 
ON dp3.data_period=data_period.id AND dp3.sensor_id=3;