Question

I’m trying to load different data, from different files, into multiple columns in MySQL. I’m not a big database guy, so maybe I have my data structured wrong. 🙂

Here’s how I have it set up:

DATABASE: mydb
TABLE:    aixserver1
COLUMNS:  os, hostname, num_users, num_groups, pkg_epoch

shown from mysql:

+---------------+-----------+------+-----+-------------------+----------------+
| Field         | Type      | Null | Key | Default           | Extra          |
+---------------+-----------+------+-----+-------------------+----------------+
| id            | int(11)   | NO   | PRI | NULL              | auto_increment |
| cur_timestamp | timestamp | NO   |     | CURRENT_TIMESTAMP |                |
| pkg_epoch     | int(11)   | NO   |     | NULL              |                |
| os            | char(5)   | YES  |     | NULL              |                |
| hostname      | char(40)  | YES  |     | NULL              |                |
| num_users     | int(10)   | YES  |     | NULL              |                |
| num_groups    | int(10)   | YES  |     | NULL              |                |
+---------------+-----------+------+-----+-------------------+----------------+

So basically I want to populate pkg_epoch, os, hostname, num_users and num_groups into the database. The data I want to load is inside 5 flat files on the server. I’m using ruby to load the data.

My question is how do I load all these values from those files into my table at once. If I do my inserts one at a time, then the other records become NULL. I.E, I load data into just the hostname column, and all the other columns become NULL for that row.

What am I missing? 🙂

Answer 1

You can do this a couple ways but the trick is to use a variable placeholder. Here is an example if you used the database’s LOAD DATA function:

LOAD DATA INFILE '/PATH/TO/FILE' IGNORE INTO TABLE tableName FIELDS TERMINATED BY '\t' LINES  
TERMINATED BY '\r' (@skip, @skip, @skip, login_name, pwd, @skip, @skip, @skip, @skip, @skip, first_name, last_name);

You see I just set a variable @skip or @anything for the fields I don’t want to include in the database and name the columns that I do want.

---

I can get you halfway there with this but am uncertain best approach if you build your own loader with Ruby. I would suggest you retrieve the file and let MySQL import using LOAD DATA as it’ll be very performant and you can use trick above.