Question

I have a long string to be queried in parallel. What I wanted to do is to parse the string into smaller units, and send each unit to the remote database and get the query results for each of them.

My parallelization works like this: Each parsed string needs to query the remote database, so I created a threadpool, and added several number of runnables that query the database for each parsed string. But I have only one single database instance in that case. So the code roughly looks like this:

for (String s : bigQuery)
   // this function connects to that database instance and queries the string "s"
   queryMyDatabase(s, databaseInstance); 

Unfortunately my threadpool implementation does not increase my query speed, and I was wondering whether it is because I used only one single client/instance. Do I need to implement a connection pool in that case? I am using CouchDB as a remote database, do you have any recommendations for a connection pool?

Answer 1

Assuming your threading implementation is correct the issue could be is that all your queries are sharing the same connection to DB. You can try dedicate every query it’s own, separate DB connection (from the Pool if you like).

Another possibility is that DB just can’t handle your requests fast enough so parallelism on your side doesn’t give you much.