Question

I’m developing a shell script that loops through a series of Postgres database table names and dumps the table data. For example:

# dump data

psql -h $SRC_IP_ADDRESS -p 5432 -U postgres -c "BEGIN;" AWARE

do
 :
 pg_dump -U postgres -h $IP_ADDRESS -p 5432 -t $i -a --inserts MYDB >> \
 out.sql
done

psql -h $IP_ADDRESS -p 5432 -U postgres -c "COMMIT;" MYDB

I’m worried about concurrent access to the database, however. Since there is no database lock for Postgres, I tried to wrap a BEGIN and COMMIT around the loop (using psql, as shown above). This resulted in an error message from the psql command, saying that:

WARNING:  there is no transaction in progress

Is there any way to achieve this? If not, what are the alternatives?

Thanks!

Answer 1

Your script has two main problems. The first problem is practical: a transaction is part of a specific session, so your first psql command, which just starts a transaction and then exits, has no real effect: the transaction ends when the command completes, and later commands do not share it. The second problem is conceptual: changes made in transaction X aren’t seen by transaction Y until transaction X is committed, but as soon as transaction X is committed, they’re immediately seen by transaction Y, even if transaction Y is still in-progress. This means that, even if your script did successfully wrap the entire dump in a single transaction, this wouldn’t make any difference, because your dump could still see inconsistent results from one query to the next. (That is: it’s meaningless to wrap a series of SELECTs in a transaction. A transaction is only meaningful if it contains one or more DML statements, UPDATEs or INSERTs or DELETEs.)

However, since you don’t really need your shell-script to loop over your list of tables; rather, you can just give pg_dump all the table-names at once, by passing multiple -t flags:

pg_dump -U postgres -h $IP_ADDRESS -p 5432 \
    -t table1 -t table2 -t table3 -a --inserts MYDB >> out.sql

and according to the documentation, pg_dump “makes consistent backups even if the database is being used concurrently”, so you wouldn’t need to worry about setting up a transaction even if that did help.

(By the way, the -t flag also supports a glob notation; for example, -t table* would match all tables whose names begin with table.)