Question

A while back, I learned a little bit about big O notation and the efficiency of different algorithms.

For example, looping through each item in an array to do something with it

foreach(item in array)
    doSomethingWith(item)

is an O(n) algorithm, because the number of cycles the program performs is directly proportional to the size of the array.

What amazed me, though, was that table lookup is O(1). That is, looking up a key in a hash table or dictionary

value = hashTable[key]

takes the same number of cycles regardless of whether the table has one key, ten keys, a hundred keys, or a gigabrajillion keys.

This is really cool, and I’m very happy that it’s true, but it’s unintuitive to me and I don’t understand why it’s true.

I can understand the first O(n) algorithm, because I can compare it to a real-life example: if I have sheets of paper that I want to stamp, I can go through each paper one-by-one and stamp each one. It makes a lot of sense to me that if I have 2,000 sheets of paper, it will take twice as long to stamp using this method than it would if I had 1,000 sheets of paper.

But I can’t understand why table lookup is O(1). I’m thinking that if I have a dictionary, and I want to find the definition of polymorphism, it will take me O(logn) time to find it: I’ll open some page in the dictionary and see if it’s alphabetically before or after polymorphism. If, say, it was after the P section, I can eliminate all the contents of the dictionary after the page I opened and repeat the process with the remainder of the dictionary until I find the word polymorphism.

This is not an O(1) process: it will usually take me longer to find words in a thousand page dictionary than in a two page dictionary. I’m having a hard time imagining a process that takes the same amount of time regardless of the size of the dictionary.

tl;dr: Can you explain to me how it’s possible to do a table lookup with O(1) complexity?

(If you show me how to replicate the amazing O(1) lookup algorithm, I’m definitely going to get a big fat dictionary so I can show off to all of my friends my ninja-dictionary-looking-up skills)

EDIT: Most of the answers seem to be contingent on this assumption:

You have the ability to access any page of a dictionary given its page number in constant time

If this is true, it’s easy for me to see. But I don’t know why this underlying assumption is true: I would use the same process to to look up a page by number as I would by word.

Same thing with memory addresses, what algorithm is used to load a memory address? What makes it so cheap to find a piece of memory from an address? In other words, why is memory access O(1)?

Answer 1

You should read the Wikipedia article.

But the essence is that you first apply a hash function to your key, which converts it to an integer index (this is O(1)). This is then used to index into an array, which is also O(1). If the hash function has been well designed, there should only be one (or a few items) stored at each location in the array, so the lookup is complete.

So in massively-simplified pseudocode:

ValueType array[ARRAY_SIZE];

void insert(KeyType k, ValueType v)
{
    int index = hash(k);
    array[index] = v;
}

ValueType lookup(KeyType k)
{
    int index = hash(k);
    return array[index];
}

Obviously, this doesn’t handle collisions, but you can read the article to learn how that’s handled.

Update

To address the edited question, indexing into an array is O(1) because underneath the hood, the CPU is doing this:

    ADD index, array_base_address -> pointer
    LOAD pointer -> some_cpu_register

where LOAD loads data stored in memory at the specified address.

Update 2

And the reason a load from memory is O(1) is really just because this is an axiom we usually specify when we talk about computational complexity (see http://en.wikipedia.org/wiki/RAM_model). If we ignore cache hierarchies and data-access patterns, then this is a reasonable assumption. As we scale the size of the machine,, this may not be true (a machine with 100TB of storage may not take the same amount of time as a machine with 100kB). But usually, we assume that the storage capacity of our machine is constant, and much much bigger than any problem size we’re likely to look at. So for all intents and purposes, it’s a constant-time operation.