Question

I’ve imported a spreadsheet of U.S. Government statistics into a mysql table with the aim of working through a tutorial.

However, it would appear that the U.S. Government has changed its formatting so that some of numerical figures are wrapped in quote marks, turning them into strings.

For example, this is the format of the tab-separated data that I’m supposed to have downloaded:

CN010010 01 001 “Autauga County, AL” 2005 23831 23061 770 3.2

However, it looks like this: CN010010 01 001 “Autauga County, AL” 2005 “23,831 ” “23,061 ” 770 3.2

As a result the two key columns of data (the 23831 and 23061 bits) that I want to import as integers are registering as 0 – presumably because it doesn’t meet the data type.

What’s the best solution for resolving this problem, now and in the future?

Thanks in advance.

Answer 1

The “comma” might cause problem. Are you using load data infile for this?

Assuming table definition:

CREATE TABLE `t` (
  `code` varchar(255) DEFAULT NULL,
  `state` char(2) DEFAULT NULL,
  `city` char(3) DEFAULT NULL,
  `name` varchar(255) DEFAULT NULL,
  `year` int(11) DEFAULT NULL,
  `pop1` int(11) DEFAULT NULL,
  `pop2` int(11) DEFAULT NULL,
  `diff` int(11) DEFAULT NULL,
  `ratio` float DEFAULT NULL
)

The following would import the file:

load data local infile "test.txt"
into table t
columns terminated by '\t'
optionally enclosed by '"'
(code, state, city, name, year, @var1, @var2, diff, ratio)
set pop1=replace(@var1,",", ""),
    pop2=replace(@var2, ",", "");

would insert the following row:

 code: CN010010
state: 01
 city: 001
 name: Autauga County, AL
 year: 2005
 pop1: 23831
 pop2: 23061
 diff: 770
ratio: 3.2