Question

I have no experience with relational databases and before I write C++ code to implement solution to my problem, I would like to check if using a database would provide an easy solution. Here is my problem:

I have a set of physical samples and simple measurement, which produces a real number result on each sample. Measurement is performed multiple times on all samples available (new samples are added periodically) and results are stored in the database as table with SAMPLE_ID and RESULT columns. Each measurement is stored as a new table containing its results (table name identifies specific measurement). Or, if it makes more sense, each measurement adds a column to global table with current results (column name identifies specific measurement). I will create tables through C++ API and receive reports (query results) the same way. I need at least two reports (simple ASCII text is fine):

1. List of all samples with their best (highest) result.
2. For a small subset of measurements, list of samples where the result of most recent measurement is worse (lower) than any preceding one (in selected subset).

What is the database query to produce each report?

Answer 1

Yes, a database will work well for that.

You will need a column to store either a date or a timestamp so you can distinguish between sample results. Without a column like that, “most recent measurement” is meaningless. (The order of rows in a table is essentially meaningless.)

You probably don’t need anyone to develop a front end; try just entering data manually or loading a CSV file through the dbms’s bulk loader. (Every modern dbms has one; their names vary.)

And you probably don’t need a reporting specialist to build reports. The query output is often all you need in research.

Some queries are simple, and others are perhaps not simple but at least straightforward. Code below was tested in PostgreSQL, but should work in any dbms that supports common table expressions and row constructors.

create table measurements (
  sample_id integer not null,
  measurement_time timestamp not null,
  measurement real not null check(measurement >= 0 and measurement <= 30),
  primary key (sample_id, measurement_time)
);

insert into measurements values 
(1, '2012-02-02 08:03', 13.89),
(2, '2012-02-02 00:00', 13.86),
(1, '2012-02-02 00:25', 25.07),
(1, '2012-02-02 03:32', 25.38),
(1, '2012-02-02 05:47', 16.64),
(2, '2012-02-02 08:03', 16.16),
(2, '2012-02-02 07:25', 25.85),
(3, '2012-02-02 08:03', 14.78),
(3, '2012-02-02 09:29', 17.08),
(3, '2012-02-02 10:31', 13.41),
(4, '2012-02-02 12:38', 20.98),
(5, '2012-02-02 08:03', 25.00),
(5, '2012-02-02 14:02', 16.27),
(5, '2012-02-02 03:32', 12.10),
(5, '2012-02-02 17:47', 21.34),
(6, '2012-02-02 18:32', 17.16),
(6, '2012-02-02 18:33', 21.59),
(7, '2012-02-02 20:07', 21.47),
(8, '2012-02-02 21:58', 11.50),
(8, '2012-02-02 22:53', 21.01);

-- All samples with their highest measurement.
select sample_id, max(measurement)
from measurements
group by sample_id
order by sample_id;

-- Most recent measurement lower than any preceeding measurement.
-- Another way of saying this is that the max() measurement isn't the 
-- latest measurement.
with max_measurements as (
  select m.*
  from measurements m
  inner join (select sample_id, max(measurement) measurement
              from measurements
              group by sample_id) max_m 
      on max_m.sample_id = m.sample_id 
     and max_m.measurement = m.measurement
),
latest_measurement as (
  select m.*
  from measurements m
  inner join (select sample_id, max(measurement_time) measurement_time
              from measurements
              group by sample_id) max_m 
      on max_m.sample_id = m.sample_id 
     and max_m.measurement_time = m.measurement_time
)
select m.* 
from max_measurements m
where row(m.sample_id, m.measurement_time) not in (select sample_id, measurement_time 
                                                   from latest_measurement);