Home/ Questions/Q 7831513
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-02T11:46:00+00:00

I’m using PostgreSQL 8.2.9, Solr 3.1, Tomcat 5.5 I have following problem: When I

  • 0

I’m using PostgreSQL 8.2.9, Solr 3.1, Tomcat 5.5

I have following problem:
When I execute delta-import – /dataimport?command=delta-import – any update queries to database are not responding for about 30 seconds.

I can easily repeat this behaviour (using psql or hibernate):

PSQL:

  1. Execute delta-import
  2. Immediately in psql – run SQL query: ‘UPDATE table SET … WHERE id = 1;’ several times
  3. The second/third/… time – I must wait ~30 seconds for query to return

Hibernate:

  1. In logs – hibernate waits ~30 seconds on method ‘doExecuteBatch(…)’ after setting query parameters

No other queries are executed when I’m testing this. On the other hand when I’m executing other commands (like full-import, etc.)- everything works perfectly fine.

In Solr’s dataconfig.xml:

  • I have readOnly attribute set to true on PostgreSQL dataSource.
  • deltaImportQuery, deltaQuery, … on entity tags don’t lock database (simple SELECT’s)

Web app (using hibernate) logs:

2012-01-08 18:54:52,403 DEBUG my.package.Class1... Executing method: save
2012-01-08 18:55:26,707 DEBUG my.package.Class1... Executing method: list

Solr logs:

INFO: [] webapp=/search path=/dataimport params={debug=query&command=delta-import&commit=true} status=0 QTime=1
2012-01-08 18:54:50 org.apache.solr.handler.dataimport.DataImporter doDeltaImport
INFO: Starting Delta Import
...
2012-01-08 18:54:50 org.apache.solr.handler.dataimport.JdbcDataSource$1 call
INFO: Time taken for getConnection(): 4
...
FINE: Executing SQL: select ... where last_edit_date > '2012-01-08 18:51:43'
2012-01-08 18:54:50 org.apache.solr.core.Config getNode
...
FINEST: Time taken for sql :4
...
INFO: Import completed successfully
2012-01-08 18:54:50 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start commit(optimize=true,waitFlush=false,waitSearcher=true,expungeDeletes=false)
2012-01-08 18:54:53 org.apache.solr.core.Config getNode
...
2012-01-08 18:54:53 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
...
2012-01-08 18:54:53 org.apache.solr.handler.dataimport.DocBuilder execute
INFO: Time taken = 0:0:2.985

There’re no ‘SELECT … FOR UPDATE / LOCK / etc.’ queries in above logs.

I have set logging for PostgreSQL – there’re no locks. Even sessions are set to:

Jan 11 14:33:07 test postgres[26201]: [3-1] <26201> LOG:  execute <unnamed>: SET SESSION CHARACTERISTICS AS TRANSACTION READ ONLY
Jan 11 14:33:07 test postgres[26201]: [4-1] <26201> LOG:  execute <unnamed>: SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL READ UNCOMMITTED

Why is this happening? This looks like some kind of database lock but then why when import is completed (2 secs) queries are still waiting (for 30 secs)?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Ok. It was hard to find solution to this problem.

    The reason was the underlying platform – disk write saturation. There was too many small “disk-write”‘s that consumed all disk-write power.

    Now we have new agreement with our service layer provider.

    Test query:

    while true ; do echo "UPDATE table_name SET type='P' WHERE type='P'" | psql -U user -d database_name ; sleep 1 ; done

    plus making changes via out other application, plus update index simultaneously.

    This was before platform change:
    Before platform change

    And here is how it works now:
    enter image description here

    • 0