Question

If there’s a queue of work todo in a table that is going to be periodically polled by a number of different worker clients…what’s the best way to prevent each worker from getting the same item to work on?

Say a table like: ItemId, LastAttemptDateTime, AttemptCount, and various item details.

Given an index on LastAttemptDateTime and sorted in ascending order and various clients are querying the table to grab an item to be worked on.

I use a stored procedure in MS SQL to do this…something like:

CREATE PROCEDURE GetNextQueueItem AS

SET NOCOUNT ON 

DECLARE @ItemId INT

UPDATE myqueue SET @ItemId=ItemId, AttemptCount=AttemptCount+1, LastAttemptDateTime=GetDate() 
WHERE ItemId=(SELECT TOP 1 ItemId 
FROM myqueue 
ORDER BY LastAttemptDateTime ASC)

SELECT ItemId, AttemptCount, and various item detail fields 
FROM myqueue 
WHERE ItemId = @ItemId

I’m fairly new to PostgreSQL and was wondering if there’s alternate approaches available. (The TOP 1 will change to LIMIT 1.)

Answer 1

PostgreSQL equivalent could look like this:

CREATE OR REPLACE FUNCTION get_next_queue_item()
  RETURNS SETOF myqueue AS
$BODY$
BEGIN
    RETURN QUERY
    UPDATE myqueue
    SET    attempt_count = attempt_count + 1
          ,last_attempt_ts = now()
    WHERE  item_id = (
        SELECT item_id
        FROM   myqueue 
        ORDER  BY last_attempt_ts
        LIMIT  1
        )
    RETURNING myqueue.*;

END;
$BODY$
  LANGUAGE plpgsql VOLATILE;

Major points

• You only need 1 statement to do it all. UPDATE can return the updated row in the same command with the RETURNING clause.
  _{State of the row is post-update. There is ways to get the pre-update state if needed.}

• No need for any variables.

• I changed all identifiers to lower case, which is the cleanest style in PostgreSQL.

• I renamed your column LastAttemptDateTime to last_attempt_ts
  ts .. for “timestamp”, because that’s the name of the timestamp / datetime type in Postgres.

• As you mentioned yourself, LIMIT 1 instead of TOP 1.

• I use RETURNS SETOF myqueue as return type.
  ^{myqueue is the associated row-type of the table myqueue – for every table or view a row-type of the same name is automatically created in PostgreSQL.}
  This declaration allows for multiple rows to be returned, but LIMIT 1 guarantees that it will only ever be one.

• This return type allows for RETURN QUERY to return the resulting row directly without any intermediate step. Fast, clean.

Actually, you don’t need a plpgsql function at all. You can do it with a simple SQL statement:

UPDATE myqueue
SET    attempt_count = attempt_count + 1
      ,last_attempt_ts = now()
WHERE  item_id = (
    SELECT item_id
    FROM   myqueue 
    ORDER  BY last_attempt_ts
    LIMIT  1
    )
RETURNING myqueue.*;