Question

I have a table with the following column structure:

sqlite> create table stocks(stockticker varchar(4), tradedate date, open double, high double, low double, close double, volume integer, adjclose double);

What I want to do is save daily stock data for a group of stocks into this one table but populate it with whatever historical data that currently exists. So I saved the historicals for XLI from yahoo/finance into a file called XLI.csv

In mySQL I use to be able to do the following:

sqlite> LOAD DATA INLINE 'XLI.csv'
   ...> INTO TABLE stocks
   ...> FIELDS TERMINATED BY ','
   ...> LINES TERMINATED BY '\n'
   ...> IGNORE 1 LINES
   ...> (@tradedate, open, high, low, close, volume, adjclose)
   ...> SET stockticker = 'XLI',
   ...> tradedate = STR_TO_DATE(@tradedate, '%m/%d/%Y');
Error: near "LOAD": syntax error
sqlite>

Now I understand SQLITE has many differences but I am a newbie and all the examples I have found typically have the following command structure:

sqlite> .import <filename> <table>

This does not help me because I need to also set the ‘stockticker’ field, ignore the first line (headers) and ensure the date is imported correctly. I haven’t found any examples in the documentation or on the internet that shows something like what I want to do.

I would kindly appreciate your help.

Answer 1

import the raw data into a staging table, then select the contents into the tick table (add a where condition to ignore the values from the first line)

to convert the date string, you can create a valid sqlite date string (e.g.: yyyy-mm-dd) by re-arranging the original date string and insert that into the destination date column.

insert into tick (stockticker, tradedate, open, high, low, close, volume, adjclose)
  select stockticker, (substr(date, 7) || '-' || substr(date, 1, 2) || '-' || substr(date, 4, 2)),
  open, high, low, close, volume, adjclose from stage