Question

Background:
I was asking a question on stack overflow regarding creating tables on the fly where this conversation ensued:

This smells like a terrible idea! In fact, it smells just like this one.
What in the world do you want to use this for? – deceze

---

@deceze: very true, However, How else would you store the contents of these CSV files.
They must be stored in mysql for indexing.
The only solid fact about them is that they all have a mobile column with a standard format.
The CSV can have an arbitrary amount of columns with an arbitrary amount of rows.
They can (with no exaggeration) range from a single row, 35 column csv to an 80k row single column CSV. I am open to other ideas. – Hailwood

---

There are many solutions for this, from attribute-value schemas to
JSON storage and NoSQL storage. Open a
new question about it. Whatever you do
though, don’t dynamically create
tables! – deceze

Question:
So my question is,
What would you say is the best way to store this data?
Are you in agreement with deceze about not creating dynamic tables?

Answer 1

A very simple schema for storing arbitrarily long records is this:

Table `records`
===============
id
created
... other meta data ...

Table `record_attributes`
=====================
id
record_id
attribute
value

So a CSV record can be stored like this:

"Foo","Bar","Baz"     // column names
42,"Lorem","Jerry"    // first record
7,"Ipsum","Tom"       // second record
...

records(1, '2010-1-17', ...)
record_attributes(1, 1, 'Foo', 42)
record_attributes(2, 1, 'Bar', 'Lorem')
record_attributes(3, 1, 'Baz', 'Jerry')

records(2, '2010-1-17', ...)
record_attributes(4, 2, 'Foo', 7)
record_attributes(5, 2, 'Bar', 'Ipsum')
record_attributes(6, 2, 'Baz', 'Tom')

An alternative is to store the record data as JSON packed blob in a single column. If you don’t need to search for the data, this is the most compact way, albeit not very RDBMS.

The best fit is probably a NoSQL database, if you have that option.