i’m batching updates with jdbc
ps = con.prepareStatement("");
ps.addBatch();
ps.executeBatch();
but in the background it seems, that the prostgres driver sends the query bit by bit to the database.
org.postgresql.core.v3.QueryExecutorImpl:398
for (int i = 0; i < queries.length; ++i)
{
V3Query query = (V3Query)queries[i];
V3ParameterList parameters = (V3ParameterList)parameterLists[i];
if (parameters == null)
parameters = SimpleQuery.NO_PARAMETERS;
sendQuery(query, parameters, maxRows, fetchSize, flags, trackingHandler);
if (trackingHandler.hasErrors())
break;
}
is there a possibility to let him send 1000 a time to speed it up?
AFAIK is no server-side batching in the fe/be protocol, so PgJDBC can’t use it.. Update: Well, I was wrong. PgJDBC (accurate as of 9.3) does send batches of queries to the server if it doesn’t need to fetch generated keys. It just queues a bunch of queries up in the send buffer without syncing up with the server after each individual query.See:
Even when generated keys are requested the extended query protocol is used to ensure that the query text doesn’t need to be sent every time, just the parameters.
Frankly, JDBC batching isn’t a great solution in any case. It’s easy to use for the app developer, but pretty sub-optimal for performance as the server still has to execute every statement individually – though not parse and plan them individually so long as you use prepared statements.
If autocommit is on, performance will be absolutely pathetic because each statement triggers a commit. Even with autocommit off running lots of little statements won’t be particularly fast even if you could eliminate the round-trip delays.
A better solution for lots of simple
UPDATEs can be to:COPYnew data into aTEMPORARYorUNLOGGEDtable; andUPDATE ... FROMtoUPDATEwith aJOINagainst the copied tableFor COPY, see the PgJDBC docs and the
COPYdocumentation in the server docs.You’ll often find it’s possible to tweak things so your app doesn’t have to send all those individual
UPDATEs at all.