Question

SCENARIO

I need to select records from test_userData based on a 1-to-1 match from test_userCheck on the columns customer or account_info. The code below will create a mock-up of the tables and will populate with random data for the purpose of my question. Based on this code, it’s looking for any records where test_userData.customer = 'Guerrero, Unity' or test_userData.account_info = 'XXXXXXXXXXXXXXXX0821', and should return three rows (confirmation_id = 6836985, 5502798, and 3046441)

PROBLEM

As it stands, the query returns what I need… however, my real userData table has almost 2 million records, and the userCheck table has about 10,000. The query takes about 7 seconds as it is and I think that’s way too long. I’m also worried because the userData table will start to grow quickly (by tens of thousands of unique records a day), and I envision my current method becoming unmanageable.

QUESTION

Any ideas on how I can optimize this to scale with millions of records? The data resides on a shared SQL 2008 server with limited permissions.

--setup temporary testing tables
IF EXISTS
(
  SELECT * FROM dbo.sysobjects
  WHERE id = object_id(N'[dbo].[test_userData]')
         AND OBJECTPROPERTY(id, N'IsUserTable') = 1
)

DROP TABLE [dbo].[test_userData]

GO

IF EXISTS
(
  SELECT * FROM dbo.sysobjects
  WHERE id = object_id(N'[dbo].[test_userCheck]')
         AND OBJECTPROPERTY(id, N'IsUserTable') = 1
)

DROP TABLE [dbo].[test_userCheck]

GO

CREATE TABLE [dbo].[test_userData](
    [id] [int] IDENTITY(1,1) NOT NULL,
    [merchant_id] [int] NOT NULL,
    [sales_date] [datetime] NOT NULL,
    [confirmation_id] [int] NOT NULL,
    [customer] [nvarchar](max) NOT NULL,
    [total] [smallmoney] NOT NULL,
    [account_info] [nvarchar](max) NOT NULL,
    [email_address] [nvarchar](max) NOT NULL
 CONSTRAINT [PK_test_userData] PRIMARY KEY CLUSTERED 
(
    [id] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

GO

CREATE TABLE [dbo].[test_userCheck](
    [confirmation_id] [int] NOT NULL,
    [customer] [nvarchar](max) NOT NULL,
    [total] [smallmoney] NOT NULL,
    [account_info] [nvarchar](max) NOT NULL
 CONSTRAINT [PK_test_userCheck] PRIMARY KEY CLUSTERED 
(
    [confirmation_id] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

GO

--insert some random user transactions
INSERT INTO [dbo].[test_userData] (merchant_id,sales_date,confirmation_id,customer,total,account_info,email_address) VALUES 
('99','03/25/2010','3361424','Soto, Ahmed','936','XXXXXXXXXXXXXXXX8744','[e-mail]'),
('17','09/12/2010','6710165','Holcomb, Eden','1022','XXXXXXXXXXXXXXXX6367','[e-mail]'),
('32','05/04/2010','4489509','Foster, Nasim','1463','XXXXXXXXXXXXXXXX7115','[e-mail]'),
('95','01/02/2011','5384061','Browning, Owen','523','XXXXXXXXXXXXXXXX0576','[e-mail]'),
('91','08/21/2010','6075234','Dawson, McKenzie','141','XXXXXXXXXXXXXXXX3580','[e-mail]'),
('63','01/29/2010','1055619','Mathews, Keefe','1110','XXXXXXXXXXXXXXXX2682','[e-mail]'),
('27','10/20/2010','1819662','Clarke, Briar','1474','XXXXXXXXXXXXXXXX7481','[e-mail]'),
('82','03/05/2010','3184936','Holman, Dana','560','XXXXXXXXXXXXXXXX7080','[e-mail]'),
('24','06/11/2010','1007427','Kirk, Desiree','206','XXXXXXXXXXXXXXXX3681','[e-mail]'),
('49','06/17/2010','6137066','Foley, Sopoline','1831','XXXXXXXXXXXXXXXX1718','[e-mail]'),
('22','05/08/2010','3545367','Howell, Uriel','638','XXXXXXXXXXXXXXXX1945','[e-mail]'),
('5','10/25/2010','6836985','Little, Caryn','743','XXXXXXXXXXXXXXXX0821','[e-mail]'),
('91','06/16/2010','6852582','Buckner, Chiquita','99','XXXXXXXXXXXXXXXX1533','[e-mail]'),
('63','06/12/2010','7930230','Nolan, Wyoming','1192','XXXXXXXXXXXXXXXX1291','[e-mail]'),
('32','02/01/2010','8407102','Cummings, Deacon','1315','XXXXXXXXXXXXXXXX4375','[e-mail]'),
('75','06/29/2010','5502798','Guerrero, Unity','858','XXXXXXXXXXXXXXXX8000','[e-mail]'),
('50','09/13/2010','8312525','Russo, Yvette','1680','XXXXXXXXXXXXXXXX2046','[e-mail]'),
('11','04/13/2010','6204132','Small, Calista','426','XXXXXXXXXXXXXXXX0269','[e-mail]'),
('16','01/01/2011','7522507','Mosley, Thor','1459','XXXXXXXXXXXXXXXX8451','[e-mail]'),
('5','01/27/2010','1472120','Case, Kiona','1419','XXXXXXXXXXXXXXXX7097','[e-mail]'),
('70','02/17/2010','1095935','Snyder, Tanner','1655','XXXXXXXXXXXXXXXX8556','[e-mail]'),
('63','11/10/2010','3046441','Guerrero, Unity','629','XXXXXXXXXXXXXXXX0807','[e-mail]'),
('22','08/19/2010','5435100','Turner, Patrick','1133','XXXXXXXXXXXXXXXX6734','[e-mail]'),
('96','10/05/2010','6381992','May, Dominic','1858','XXXXXXXXXXXXXXXX7227','[e-mail]'),
('96','02/26/2010','8630748','Chandler, Olympia','1016','XXXXXXXXXXXXXXXX4001','[e-mail]');

--insert a random fraud transaction to check against (based on customer and account_info only)
INSERT INTO [dbo].[test_userCheck] (confirmation_id, customer, total, account_info) VALUES
('2055015', 'Guerrero, Unity', '20.02', 'XXXXXXXXXXXXXXXX0821');

--get result, which is correct
SELECT     a.confirmation_id, a.customer, a.total, a.account_info, a.email_address
FROM         dbo.test_userData AS a RIGHT OUTER JOIN
                      dbo.test_userCheck AS b ON a.customer = b.customer OR a.account_info = b.account_info;

DROP TABLE [dbo].[test_userData];
DROP TABLE [dbo].[test_userCheck];

Answer 1

Create the appropriate index or indices. Just based on your question, I’d suggest two indices, one on test_userData.customer, and a second index on test_userData.account_info