Question

I have a SQL database with rows containing prices of EUR/USD for a given datetime:

i want to split these data into groups of 1 minute and get some information, I’ve written a python program that call the sqlite3 database. For now, i have this program:

def SQLQuerySpecial(currency,start,end):
    sql = 'SELECT f.datetime, f.buy, c.minp, c.maxp, c.avgp FROM {c} f INNER JOIN (SELECT MIN(datetime) mindate, MAX(datetime) maxdate, MIN(sell) minp, MAX(sell) maxp, AVG(buy) avgp FROM {c} GROUP BY strftime("%Y-%m-%d %H:%M",datetime)) c ON f.datetime = c.mindate or f.datetime = c.maxdate'.format(c = currency)

When i do:

    SQLQuerySpecial("EUR", "2002-01-02 01:27:00", "2002-01-02 01:34:00")

I get the result:

(‘2002-01-02 01:26:33’, 0.8913, 0.8913, 0.8913, 0.8913)

(‘2002-01-02 01:27:12’, 0.8946, 0.8946, 0.8947, 0.89462)

(‘2002-01-02 01:27:49’, 0.8946, 0.8946, 0.8947, 0.89462)

(‘2002-01-02 01:28:00’, 0.8947, 0.8946, 0.8947, 0.8946454545454546)

(‘2002-01-02 01:28:00’, 0.8946, 0.8946, 0.8947, 0.8946454545454546)

(‘2002-01-02 01:28:43’, 0.8946, 0.8946, 0.8947, 0.8946454545454546)

(‘2002-01-02 01:28:43’, 0.8946, 0.8946, 0.8947, 0.8946454545454546)

(‘2002-01-02 01:29:49’, 0.8946, 0.8946, 0.8947, 0.89465)

(‘2002-01-02 01:29:49’, 0.8947, 0.8946, 0.8947, 0.89465)

(‘2002-01-02 01:29:52’, 0.8946, 0.8946, 0.8947, 0.89465)

(‘2002-01-02 01:29:52’, 0.8947, 0.8946, 0.8947, 0.89465)

(‘2002-01-02 01:30:41’, 0.8945, 0.8945, 0.8946, 0.8945299999999999)

… and so one for 21 lines of result.

What i want with my function between 01:27 and 01:34 parameters, is to obtain only 8 lines:

2002-01-02 01:27:xx , buy price @ 01:27 , max buy price between 01:27 and 01:28 , min buy price between 01:27 and 01:28, buy price @ 1:28

2002-01-02 01:28:xx , buy price @ 01:28 , max buy price between 01:28 and 01:29 , min buy price between 01:28 and 01:29, buy price @ 1:29

2002-01-02 01:29:xx …

…

to 2002-01-02 01:34:xx …

I honestly tried to find the right way to do it, but its tricky to me, i’m not very good in SQL. Thanks for reading and help.

special thanks to unutbu

Answer 1

Try this:

sql = '''
    SELECT DISTINCT
        f.dm,
        f1.price AS first,
        f.maxp AS max,
        f.minp AS min, 
        f2.price AS last
    FROM (
        SELECT datetime, strftime("%Y-%m-%d %H:%M",datetime) dm,
               MIN(datetime) mindate, MAX(datetime) maxdate,
               MIN(price) minp, MAX(price) maxp
        FROM {c} 
        WHERE datetime >= ? AND datetime < ?
        GROUP BY dm
        ) f
    INNER JOIN {c} f1
    ON f1.datetime = f.mindate
    INNER JOIN {c} f2
    ON f2.datetime = f.maxdate
    '''.format(c = currency)

• dm stands for “date-minute”. We group by date-minutes.
• There are some rows with the same datetime. SELECT DISTINCT
  eliminates duplicate rows.
• The first and last price from each group appear on different rows. To
  join them on the same row requires a JOIN. This is the purpose of the
  two INNER JOINS above.