Home/ Questions/Q 3660948
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-19T01:15:19+00:00

I have several billion rows of data in CSV files. Each row can have

  • 0

I have several billion rows of data in CSV files. Each row can have anything from 10 to 20 columns. I want to use COPY FROM to load the data into a table containing 20 columns. If a specific CSV row only contains 10 columns of data, then I expect COPY FROM to set the rest of the columns (for which the values are missing) to NULL. I specify DEFAULT NULL on every column in the CREATE TABLE statement.

MY QUESTION:
Can this be done using COPY FROM?

EDIT: Greenplum (a database based upon PostgreSQL) has a switch named FILL MISSING FIELDS, which does what I describe (see their documentation here). What workarounds would you recommend for PostgreSQL?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Write a pre-processing script to just add some extra commas on the lines that don’t have enough columns, or to transform the CSV into TSV (tab-separated) and put “\N” in the extra columns.

    • 0