Question

This are the first three lines of my text file:

  Dist    Mv  CL Typ  LTef  logg Age Mass  B-V    U-B    V-I    V-K    V   [Fe/H] l           b         Av   Mbol
  0.033 14.40  5 7.90 3.481  5.10  1 0.15  1.723  1.512  3.153  5.850 17.008  0.13   0.50000   0.50000  0.014 12.616
  0.033  7.40  5 6.50 3.637  4.62  7 0.71  1.178  0.984  1.302  2.835 10.047 -0.56   0.50000   0.50000  0.014  6.125
  0.052 11.70  5 7.40 3.529  4.94  2 0.31  1.541  1.167  2.394  4.565 15.393 -0.10   0.50000   0.50000  0.028 10.075

Assuming I have the right columns, how do I import this?

Bonus: is it possible/are there tools to create the schema from these kinds of files automatically?

Answer 1

At lowest level you could just use COPY command (or \copy in psql if you don’t have access to superuser account, which COPY requires to load data from file). Unfortunately you have to create table structure at first (there is no built-in guess-by-header feature), but it looks straightforward to write one.

Choose whaterer suitable datatype you need e.g. real for single precision floating-point (IEEE 754), double precision or numeric if you need arbitrary precision numbers:

CREATE TABLE measurement
(
    "Dist" double precision,
    "Mv" double precision,
    "CL" double precision,
    "Typ" double precision,
    "LTef" double precision,
    "logg" double precision,
    "Age" double precision,
    "Mass" double precision,
    "B-V" double precision,
    "U-B" double precision,
    "V-I" double precision,
    "V-K" double precision,
    "V" double precision,
    "[Fe/H]" double precision,
    "l" double precision,
    "b" double precision,
    "Av" double precision,
    "Mbol" double precision
);

Another thing is that your file contains multiple spaces between values, so it’s better to transform it into single-tab-delimited entries (there are plenty of tools to do this):

$ sed 's/  */\t/g' import.csv
Dist    Mv  CL  Typ LTef    logg    Age Mass    B-V U-B V-I V-K V   [Fe/H]  l   b   Av  Mbol
0.033   14.40   5   7.90    3.481   5.10    1   0.15    1.723   1.512   3.153   5.850   17.008  0.13    0.50000 0.50000 0.014   12.616
0.033   7.40    5   6.50    3.637   4.62    7   0.71    1.178   0.984   1.302   2.835   10.047  -0.56   0.50000 0.50000 0.014   6.125
0.052   11.70   5   7.40    3.529   4.94    2   0.31    1.541   1.167   2.394   4.565   15.393  -0.10   0.50000 0.50000 0.028   10.075

Finally you can import your file straight into Postgres database, for example:

=> \copy measurement FROM '/path/import.csv' (FORMAT csv, DELIMITER E'\t', HEADER 'true')
=> TABLE measurement;
 Dist  |  Mv  | CL | Typ | LTef  | logg | Age | Mass |  B-V  |  U-B  |  V-I  |  V-K  |   V    | [Fe/H] |  l  |  b  |  Av   |  Mbol  
-------+------+----+-----+-------+------+-----+------+-------+-------+-------+-------+--------+--------+-----+-----+-------+--------
 0.033 | 14.4 |  5 | 7.9 | 3.481 |  5.1 |   1 | 0.15 | 1.723 | 1.512 | 3.153 |  5.85 | 17.008 |   0.13 | 0.5 | 0.5 | 0.014 | 12.616
 0.033 |  7.4 |  5 | 6.5 | 3.637 | 4.62 |   7 | 0.71 | 1.178 | 0.984 | 1.302 | 2.835 | 10.047 |  -0.56 | 0.5 | 0.5 | 0.014 |  6.125
 0.052 | 11.7 |  5 | 7.4 | 3.529 | 4.94 |   2 | 0.31 | 1.541 | 1.167 | 2.394 | 4.565 | 15.393 |   -0.1 | 0.5 | 0.5 | 0.028 | 10.075
(3 rows)