Fairly new to relational database and to sqlite. But I’m curious what’s the best way to represent list of reals, per item/row, in SQLite.
I have several items X; contains 100s-100,000s items. Each item x in X has one core attribute, which is a list of float pairs (e.g. {{100,1.2}, {101.1, 234.3}, … , {20000, 90}}; usually contains 10s-1000s pair of floats).
My first idea is to use blob. But it makes is difficult to read these pairs of floats (using some generic sqlite viewer, which I’m not aware of yet). I’d also like to query by the values of these float pairs. Is there a way to parse blobs and use them during a query? I also suspect there will be issues with big/little-endian binary encoding with blobs.
x = [prim_key, blob]
Second idea is to represent them as a string. This makes it easier to read, but probably less compact.
x = [prim_key, “100, 1.2, 101.1, 234.3, … “]
A third is to “normalize” and represent these floats in a separate table, with reference to the primary key. This seems awkward, slower at gathering these floats for a particular x, and space wasteful as each float pair must be associated with the referencing primary key values. But this will allow me to make queries using the float pairs.
x_float_pairs = [prim_key_ref, 100, 1.2; prim_key_ref, 101.1, 234.3; …]
A fourth is to create a table to contain the pair of floats per item x; probably the worst idea right?
Any suggestions? Thank you in advance.
If you’re committed to using SQLLite I would tend to go with the normalized approach. It is very quick to look up rows when you have an index. That is the paradigm which is supported by SQL databases.
If you have a bit more flexibility then you might think about using blob storage which would allow you to retrieve the rows as one set. Without knowing more about how you’re planning on accessing the data it is difficult to recommend a strategy which will work well in all cases.