Question

I am creating a database for my company that will store many different types of information. The categories are Brightness, Contrast, Chromaticity, ect. Each category has a number of data points which my company would like to start storing.

Normally, I would create a table for each category which would store the corresponding data. (This is how I learned to do it). However, Sometimes these categories have “sub-data” which would change the number of fields required in each table.

My question is then how do people handle the inconsistency of data when structuring their databases? Do they just keep adding more tables for extra data or is it something else altogether?

Answer 1

There are a few (and thank goodness only a few) unbendable rules about relational database models. One of those is, that if you don’t know what to store, you have a hard time storing it. Chances are, you’ll have an even harder time retrieving it.

That said, the reality of business rules is often less clear cut than the ivory tower of database design. Most importantly, you might want or even need a way to introduce a new property without changing the schema.

Here are two feasable ways to go at this:

1. Use a datastore, that specializes in loose or inexistant schemas
   (NoSQL and friends). Explaining this in detail is a subject of a CS
   Thesis, not a stackoverflow answer.
2. My recommendation: Use a separate properties table – here is how
   this goes:

Assuming for the sake of argument, your products allways have (unique string) name, (integer) id, brightness, contrast, chromaticity plus sometimes (integer) foo and (string) bar, consider these tables

CREATE TABLE products (
  id INT PRIMARY KEY AUTO_INCREMENT,
  name VARCHAR(50) NOT NULL,
  brightness INT,
  contrast INT,
  chromaticity INT,
  UNIQUE INDEX(name)
);

CREATE TABLE properties (
  id INT PRIMARY KEY AUTO_INCREMENT,
  name VARCHAR(50) NOT NULL,
  proptype ENUM('null','int','string') NOT NULL default 'null',
  UNIQUE INDEX(name)
);

INSERT INTO properties VALUES
  (0,'foo','int'),
  (0,'bar','string');

CREATE TABLE product_properties (
  id INT PRIMARY KEY AUTO_INCREMENT,
  products_id INT NOT NULL,
  properties_id INT NOT NULL,
  intvalue INT NOT NULL,
  stringvalue VARCHAR(250) NOT NULL,
  UNIQUE INDEX(products_id,properties_id)
);

now your “standard” properties would be in the products table as usual, while the “optional” properties would be stored in a row of product_properties, that references the product id and property id, with the value being in intvalue or stringvalue.

Selecting products including their foo if any would look like

SELECT 
  products.*,
  product_properties.intvalue AS foo
FROM products
  LEFT JOIN product_properties 
    ON products.id=product_properties.product_id 
    AND product_properties.property_id=1

or even

SELECT 
  products.*,
  product_properties.intvalue AS foo
FROM products
  LEFT JOIN product_properties 
    ON products.id=product_properties.product_id 
  LEFT JOIN properties 
    ON product_properties.property_id=properties.id
WHERE properties.name='foo' OR properties.name IS NULL

Please understand, that this incurs a performance penalty – in fact you trade performance against flexibility: Adding another property is nothing more than INSERTing a row in properties, the schema stays the same.