Question

I’m trying to experiment with Stephan Marslands 2D Self-Organzing Map (In conjunction with Principal Components Analysis)

import som as sm
#put it into som
def som_algorithm(inputs,nIterations,x=0,y=0):
    print "Running Self-Organizing Map!"
    if x == 0 and y == 0:
        y = len(inputs[0])
        x = len(inputs)
    print "x axis is " + str(x) + " / y axis is " + str(y)
    som = sm.som(x,y,inputs)
    som.somtrain(inputs, nIterations)
    print som.somfwd(inputs)

som_algorithm(training, 100)

And based on my data set after preprocessing, produces this error

Running Self-Organizing Map!
x axis is 600 / y axis is 173
Traceback (most recent call last):
  File "\source\bag_of_words.py", line 112, in <module>
som_algorithm(training, 100)
  File "\source\bag_of_words.py", line 87, in som_algorithm
som = sm.som(x,y,inputs)
  File "\som\__init__.py", line 44, in __init__
self.mapDist = zeros((self.x*self.y,self.x*self.y))
MemoryError

Currently my Y represents the amount of elements per array (array dimension contained within the larger dimension) and my X represents the larger dimension containing smaller array dimensions. What could be causing this memory error?

Answer 1

The code is trying to create an array of zeros with shape x*y by x*y. Thats 10774440000 elements. Each value in the array is 8 bytes (double precision). That’s over 80 gigabytes. You are simply running out of memory.