Question

I just imported a huge text file into a table, using the .import command. Everything is OK, except for the fact that it seems to treat clearly numeric values as text. For instance, conditions such as WHERE field > 4 are always met. I did not specify datatypes when I created the table, but this doesn’t seem to matter when small tables are created.

Any advice would be welcome. Thanks!

Edit/conclusion: It turns out some of the values in my CSV file were blanks. I ended up solving this by being a bit less lazy and declaring the datatypes explicitly.

Answer 1

The way SQLite handles types is described on this page: http://www.sqlite.org/datatype3.html

In particular:

Under circumstances described below,
the database engine may convert values
between numeric storage classes
(INTEGER and REAL) and TEXT during
query execution.

Section 3.4 (Comparison Example) should give you concrete examples, which are likely to explain the problem you have. This is probably this example:

-- Because column "a" has text affinity, numeric values on the
-- right-hand side of the comparisons are converted to text before
-- the comparison occurs.
SELECT a < 40,   a < 60,   a < 600 FROM t1;
0|1|1

To avoid the affinity to be guessed, you can use CAST explicitly (see section 3.2 too):

SQLite may attempt to convert values
between the storage classes INTEGER,
REAL, and/or TEXT before performing a
comparison. Whether or not any
conversions are attempted before the
comparison takes place depends on the
affinity of the operands. Operand
affinity is determined by the
following rules:

• An expression that is a simple reference to a column value has the
  same affinity as the column. Note that
  if X and Y.Z are column names, then +X
  and +Y.Z are considered expressions
  for the purpose of determining
  affinity.
• An expression of the form “CAST(expr AS type)” has an affinity
  that is the same as a column with a
  declared type of “type”.
• Otherwise, an expression has NONE affinity.

Here is another example:

CREATE TABLE test (value TEXT);
INSERT INTO test VALUES(2);
INSERT INTO test VALUES(123);
INSERT INTO test VALUES(500);

SELECT value, value < 4 FROM test;
2|1
123|1
500|0

It’s likely that the CSV import create columns of affinity TEXT.