Question

I am designing a database with a single table for a special scenario I need to implement a solution for. The table will have several hundred million rows after a short time, but each row will be fairly compact. Even when there are a lot of rows, I need insert, update and select speeds to be nice and fast, so I need to choose the best indexes for the job.

My table looks like this:

create table dbo.Domain
(
    Name varchar(255) not null,
    MetricType smallint not null, -- very small range of values, maybe 10-20 at most
    Priority smallint not null, -- extremely small range of values, generally 1-4
    DateToProcess datetime not null,
    DateProcessed datetime null,

    primary key(Name, MetricType)
);

A select query will look like this:

select Name from Domain
where MetricType = @metricType
    and DateProcessed is null
    and DateToProcess < GETUTCDATE()
order by Priority desc, DateToProcess asc

The first type of update will look like this:

merge into Domain as target
using @myTablePrm as source
on source.Name = target.Name
    and source.MetricType = target.MetricType
when matched then
    update set
        DateToProcess = source.DateToProcess,
        Priority = source.Priority,
        DateProcessed = case -- set to null if DateToProcess is in the future
            when DateToProcess < DateProcessed then DateProcessed
            else null end
when not matched then
    insert (Name, MetricType, Priority, DateToProcess)
    values (source.Name, source.MetricType, source.Priority, source.DateToProcess);

The second type of update will look like this:

update Domain
set DateProcessed = source.DateProcessed
from @myTablePrm source
where Name = source.Name and MetricType = @metricType

Are these the best indexes for optimal insert, update and select speed?

-- for the order by clause in the select query
create index IX_Domain_PriorityQueue
    on Domain(Priority desc, DateToProcess asc)
    where DateProcessed is null;

-- for the where clause in the select query
create index IX_Domain_MetricType
    on Domain(MetricType asc);

Answer 1

Observations:

• Your updates should use the PK
• Why not use tinyint (range 0-255) to make the rows even narrower?
• Do you need datetime? Can you use smalledatetime?

Ideas:

• Your SELECT query doesn’t have an index to cover it. You need one on (DateToProcess, MetricType, Priority DESC) INCLUDE (Name) WHERE DateProcessed IS NULL
  `: you’ll have to experiment with key column order to get the best one

• You could extent that index to have a filtered indexes per MetricType too (keeping DateProcessed IS NULL filter). I’d do this after the other one when I do have millions of rows to test with