Question

The ORDER BY clause is decribed in the PostgreSQLdocumentation as:

ORDER BY expression [ ASC | DESC | USING operator ] [ NULLS { FIRST | LAST } ] [, ...]

Can someone give me some examples how to use the USING operator? Is it possible to get an alternating order of the resultset?

Answer 1

A very simple example would be:

> SELECT * FROM tab ORDER BY col USING <

But this is boring, because this is nothing you can’t get with the traditional ORDER BY col ASC.

Also the standard catalog doesn’t mention anything exciting about strange comparison functions/operators. You can get a list of them:

    > SELECT amoplefttype::regtype, amoprighttype::regtype, amopopr::regoper 
      FROM pg_am JOIN pg_amop ON pg_am.oid = pg_amop.amopmethod 
      WHERE amname = 'btree' AND amopstrategy IN (1,5);

You will notice, that there are mostly < and > functions for primitive types like integer, date etc and some more for arrays and vectors and so on. None of these operators will help you to get a custom ordering.

In most cases where custom ordering is required you can get away using something like ... ORDER BY somefunc(tablecolumn) ... where somefunc maps the values appropriately. Because that works with every database this is also the most common way. For simple things you can even write an expression instead of a custom function.

Switching gears up

ORDER BY ... USING makes sense in several cases:

• The ordering is so uncommon, that the somefunc trick doesn’t work.
• You work with a non-primitive type (like point, circle or imaginary numbers) and you don’t want to repeat yourself in your queries with strange calculations.
• The dataset you want to sort is so large, that support by an index is desired or even required.

I will focus on the complex datatypes: often there is more than one way to sort them in a reasonable way. A good example is point: You can “order” them by the distance to (0,0), or by x first, then by y or just by y or anything else you want.

Of course, PostgreSQL has predefined operators for point:

    > CREATE TABLE p ( p point );
    > SELECT p <-> point(0,0) FROM p;

But none of them is declared usable for ORDER BY by default (see above):

    > SELECT * FROM p ORDER BY p;
    ERROR:  could not identify an ordering operator for type point
    TIP:  Use an explicit ordering operator or modify the query.

Simple operators for point are the “below” and “above” operators <^ and >^. They compare simply the y part of the point. But:

    >  SELECT * FROM p ORDER BY p USING >^;
    ERROR: operator > is not a valid ordering operator
    TIP: Ordering operators must be "<" or ">" members of __btree__ operator families.

ORDER BY USING requires an operator with defined semantics: Obviously it must be a binary operator, it must accept the same type as arguments and it must return boolean. I think it must also be transitive (if a < b and b < c then a < c). There may be more requirements. But all these requirements are also necessary for proper btree-index ordering. This explains the strange error messages containing the reference to btree.

ORDER BY USING also requires not just one operator to be defined but an operator class and an operator family. While one could implement sorting with only one operator, PostgreSQL tries to sort efficiently and minimize comparisons. Therefore, several operators are used even when you specify only one – the others must adhere to certain mathematical constraints – I’ve already mentioned transitivity, but there are more.

Switching Gears up

Let’s define something suitable: An operator for points which compares only the y part.

The first step is to create a custom operator family which can be used by the btree index access method. see

    > CREATE OPERATOR FAMILY xyzfam USING btree;   -- superuser access required!
    CREATE OPERATOR FAMILY

Next we must provide a comparator function which returns -1, 0, +1 when comparing two points. This function WILL be called internally!

    > CREATE FUNCTION xyz_v_cmp(p1 point, p2 point) RETURNS int 
      AS $$BEGIN RETURN btfloat8cmp(p1[1],p2[1]); END $$ LANGUAGE plpgsql;
    CREATE FUNCTION

Next we define the operator class for the family. See the manual for an explanation of the numbers.

    > CREATE OPERATOR CLASS xyz_ops FOR TYPE point USING btree FAMILY xyzfam AS 
        OPERATOR 1 <^ ,
        OPERATOR 3 ?- ,
        OPERATOR 5 >^ ,
        FUNCTION 1 xyz_v_cmp(point, point) ;
    CREATE OPERATOR CLASS

This step combines several operators and functions and also defines their relationship and meaning. For example OPERATOR 1 means: This is the operator for less-than tests.

Now the operators <^ and >^ can be used in ORDER BY USING:

> INSERT INTO p SELECT point(floor(random()*100), floor(random()*100)) FROM generate_series(1, 5);
INSERT 0 5
> SELECT * FROM p ORDER BY p USING >^;
    p    
---------
 (17,8)
 (74,57)
 (59,65)
 (0,87)
 (58,91)

Voila – sorted by y.

To sum it up: ORDER BY ... USING is an interesting look under the hood of PostgreSQL. But nothing you will require anytime soon unless you work in very specific areas of database technology.

Another example can be found in the Postgres docs. with source code for the example here and here. This example also shows how to create the operators.