Question

I have a quite serious performance issue with the following statement that i can’t fix myself.

Given Situation

• I have a postgres 8.4 Database with Postgis 1.4 installed
• I have a geospatial table with ~9 Million entries. This table has a (postgis) geometry column and a tsvector column
• I have a GIST Index on the geometry and a VNAME Index on the vname column
• Table is ANALYZE‘d

I want to excecute a to_tsquerytext search within a subset of these geometries that should give me all affecteded ids back.

The area to search in will limit the 9 Million datasets to approximately 100.000 and the resultset of the ts_query inside this area will most likely give an output of 0..1000 Entries.

Problem

The query analyzer decides that he wants to do an Bitmap Index Scan on the vname first, and then aggreates and puts a filter on the geometry (and other conditions I have in this statement)

Query Analyzer output:

Aggregate  (cost=12.35..12.62 rows=1 width=510) (actual time=5.616..5.616 rows=1 loops=1)
->  Bitmap Heap Scan on mxgeom g  (cost=8.33..12.35 rows=1 width=510) (actual time=5.567..5.567 rows=0 loops=1)
     Recheck Cond: (vname @@ '''hemer'' & ''hauptstrasse'':*'::tsquery)
     Filter: (active AND (geom && '0107000020E6100000010000000103000000010000000B0000002AFFFF5FD15B1E404AE254774BA8494096FBFF3F4CC11E40F37563BAA9A74940490200206BEC1E40466F209648A949404DF6FF1F53311F400C9623C206B2494024EBFF1F4F711F404C87835954BD4940C00000B0E7CA1E4071551679E0BD4940AD02004038991E40D35CC68418BE49408EF9FF5F297C1E404F8CFFCB5BBB4940A600006015541E40FAE6468054B8494015040060A33E1E4032E568902DAE49402AFFFF5FD15B1E404AE254774BA84940'::geometry) AND (mandator_id = ANY ('{257,1}'::bigint[])))
     ->  Bitmap Index Scan on gis_vname_idx  (cost=0.00..8.33 rows=1 width=0) (actual time=5.566..5.566 rows=0 loops=1)
           Index Cond: (vname @@ '''hemer'' & ''hauptstrasse'':*'::tsquery)

which causes a LOT of I/O – AFAIK It would be smarter to limit the geometry first, and do the vname search after.

Attempted Solutions

To achieve the desired behaviour i tried to

1. I Put the geom @@ AREA into a subselect -> Did not change the execution plan
2. I created a temporary view with the desired area subset -> Did not change the execution plan
3. I created a temporary table of the desired area -> Takes 4~6 seconds to create, so that made it even worse.

Btw, sorry for not posting the actual query: I think my boss would really be mad at me if I did, also I’m looking more for theoretical pointers for someone to fix my actual query. Please ask if you need further clarification

---

EDIT

Richard had a very good point: You can achieve the desired behaviour of the Query Planner with the width statement. The bad thing is that this temporary Table (or CTE) messes up the vname index, thus making the query return nothing in some cases.

I was able to fix this with creating a new vname on-the-fly with to_tsvector(), but this is (too) costly – around 300 – 500ms per query.

My Solution

I ditched the vname search and went with a simple LIKE('%query_string%') (10-20 ms/query), but this is only fast in my given environment. YMMV.

Answer 1

There have been some improvements in statistics handling for tsvector (and I think PostGIS too, but I don’t use it). If you’ve got the time, it might be worth trying again with a 9.1 release and see what that does for you.

However, for this single query you might want to look at the WITH construct.

http://www.postgresql.org/docs/8.4/static/queries-with.html

If you put the geometry part as the WITH clause, it will be evaluated first (guaranteed) and then that result-set will filtered by the following SELECT. It might end up slower though, you won’t know until you try.

It might be an adjustment to work_mem would help too – you can do this per-session (“SET work_mem = …”) but be careful with setting it too high – concurrent queries can quickly burn through all your RAM.

http://www.postgresql.org/docs/8.4/static/runtime-config-resource.html#RUNTIME-CONFIG-RESOURCE-MEMORY