Question

I am tasked with the following:
The customer wants to read a number (possibly thousands) of values from a csv file. I have to use these values in an SQL select statement like this:

SELECT * FROM myTable WHERE myTable.Value IN (cvsValue1, csvValue 2, ..., csvValueN)

The question is: Will this work for an arbitrary number of csv values, and will it perform well for large number of values?

I will need to save the SQL as a string internally in my C# application for later use. (if that makes a difference for alternative solutions)

Answer 1

You really don’t want to do that. It would be better to dump those values into an indexed table and use IN as a subquery (which typically implements a SEMI JOIN (more info) vs the array of strings (which is typically implement as a series of OR operations.

from BOL:

Including an extremely large number of values (many thousands) in an
IN clause can consume resources and return errors 8623 or 8632. To
work around this problem, store the items in the IN list in a table.

Error 8623:

The query processor ran out of internal resources and could not
produce a query plan. This is a rare event and only expected for
extremely complex queries or queries that reference a very large
number of tables or partitions. Please simplify the query. If you
believe you have received this message in error, contact Customer
Support Services for more information.

Error 8632:

Internal error: An expression services limit has been reached. Please
look for potentially complex expressions in your query, and try to
simplify them.