Question

I want to load a plain file into Greenplum database using external tables.
Can I specify input format for timestamps/date/time fields? (If you know the answer for PostgreSQL, please reply as well)

For example, with Oracle I can use DATE_FORMAT DATE MASK ‘YYYYMMDD’ to tell how to parse the date. For Netezza I can specify DATESTYLE ‘YMD’. For Greenplum I cannot find the answer. I can describe fields as char, and then parse them during the load, but this is an ugly workaround.

Here is my tentative code:

CREATE EXTERNAL TABLE MY_TBL (X date, Y time, Z timestamp ) 
LOCATION (
 'gpfdist://host:8001/file1.txt',
 'gpfdist://host:8002/file2.txt'
) FORMAT 'TEXT' (DELIMITER '|' NULL '')

Answer 1

It appears that you can:

SET DATESTYLE = 'YMD';

before SELECTing from the table. This will affect the interpretation of all dates, though, not just those from the file. If you consistently use unambiguous ISO dates elsewhere that will be fine, but it may be a problem if (for example) you need to also accept ‘D/M/Y’ date literals in the same query.

This is specific to GreenPlum’s CREATE EXTERNAL TABLE and does not apply to SQL-standard SQL/MED foreign data wrappers, as shown below.

---

What surprises me is that PostgreSQL proper (which does not have this CREATE EXTERNAL TABLE feature) always accepts ISO-style YYYY-MM-DD and YYYYMMDD dates, irrespective of DATESTYLE. Observe:

regress=> SELECT '20121229'::date, '2012-12-29'::date, current_setting('DateStyle');
    date    |    date    | current_setting 
------------+------------+-----------------
 2012-12-29 | 2012-12-29 | ISO, MDY
(1 row)

regress=> SET DateStyle = 'DMY';
SET
regress=> SELECT '20121229'::date, '2012-12-29'::date, current_setting('DateStyle');
    date    |    date    | current_setting 
------------+------------+-----------------
 2012-12-29 | 2012-12-29 | ISO, DMY
(1 row)

… so if GreenPlum behaved the same way, you should not need to do anything to get these YYYYMMDD dates to be read correctly from the input file.

Here’s how it works with a PostgreSQL file_fdw SQL/MED foreign data wrapper:

CREATE EXTENSION file_fdw;

COPY (SELECT '20121229', '2012-12-29') TO '/tmp/dates.csv' CSV;

SET DateStyle = 'DMY';

CREATE SERVER csvtest FOREIGN DATA WRAPPER file_fdw;

CREATE FOREIGN TABLE csvtest (
    date1 date,
    date2 date
) SERVER csvtest OPTIONS ( filename '/tmp/dates.csv', format 'csv' );

SELECT * FROM csvtest ;
   date1    |   date2    
------------+------------
 2012-12-29 | 2012-12-29
(1 row)

The CSV file contents are:

20121229,2012-12-29

so you can see that Pg will always accept ISO dates for CSV, irrespective of datestyle.

If GreenPlum doesn’t, please file a bug. The idea of DateStyle changing the way a foreign table is read after creation is crazy.