Question

I have constructed two dictionary that contains histogram values of images. Each dictionary has file name of the image file as the key and the list of three one dimension vectors put together as its values.

Example: {'someFileName.jpg' : ['forecolor=2,3,5,5,6','edge=2,4,5','texture=5,4,3']}

Here is an actual representation of one of my dictionaries:

Dictionary1

{'/Users/images/Transcend-8GB-Class-10-SDHC-Flash-Memory-Card.jpg': ['fcolor=2,4,14,5,0,0,0,0,0,0,0,0,0,0,12,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,6,0,0,0,0,1,0,0,0,0,0,0,0,0,0,20,9,0,0,0,2,2,0,0,0,0,0,0,0,0,0,13,6,0,0,0,1,0,0,0,0,0,0,0,0,0,0,10,8,0,0,0,1,2,0,0,0,0,0,0,0,0,0,17,17,0,0,0,3,6,0,0,0,0,0,0,0,0,0,7,5,0,0,0,2,0,0,0,0,0,0,0,0,0,0,4,3,0,0,0,1,1,0,0,0,0,0,0,0,0,0,6,6,0,0,0,2,3', 'edge=1,252,1,32,124,194,63,252,67,15,240,1,7,244,66,47,0,192,63', 'texture=1,78,27,37,13,6,6,7,78']}

Dictionary2

{'/Users/images/kodax-camera-M531.jpg': ['fcolor=2,74,6,20,30,1,2,0,1,0,0,0,1,3,2,0,0,0,0,0,1,1,1,0,0,2,0,0,0,2,2,0,0,0,0,0,2,2,1,0,0,5,0,0,0,1,4,0,0,0,0,0,2,2,1,0,0,1,0,0,0,3,1,0,0,0,0,0,1,1,0,0,0,3,0,0,0,1,2,0,0,0,0,0,2,2,1,0,0,4,0,0,0,0,5,0,0,0,0,0,2,1,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0', 'edge=1,4,1,88,128,22,8,39,25,142,230,226,31,60,64,255,252,12,76', 'texture=1,15,32,31,28,19,16,12,98']}

My end goal is to pass two of these dictionaries to a method and actually run cosign value,

example: So each dictionary has list as its value, so for each dictionary key i want to do vector multiplication among dictionary1’s key1, velu1 with dictionary2 key1, value1,

I have the vector multiplication function so what I am trying to figure out how to properly iterate, I was thinking along the line of using a yield function but it didn’t really work when I tried. This is what I have so far:

def cosignSimilarity(image1VectorDict, image2VectorDict):
    for image1Key, image2Value in image1VectorDict.iteritems():
        print image1Key
        for aValue in image1Value:
            print aValue
            for image2Key, image2Value in image2VectorDict.iteritems():
                for eValue in image2Value:
                    print aValue
                    print "\n"
                    print eValue

FYI: I am not asking help on the cosign calculation.

This is how my current code is spitting the data out if I can isolate key to key from one dictionary to another then I can do the rest such as calculating cosine values.

   First  Dictionary
    {'/Users/test/Transcend-8GB-Class-10-SDHC-Flash-Memory-Card.jpg': ['fcolor=2,4,14,5,0,0,0,0,0,0,0,0,0,0,12,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,6,0,0,0,0,1,0,0,0,0,0,0,0,0,0,20,9,0,0,0,2,2,0,0,0,0,0,0,0,0,0,13,6,0,0,0,1,0,0,0,0,0,0,0,0,0,0,10,8,0,0,0,1,2,0,0,0,0,0,0,0,0,0,17,17,0,0,0,3,6,0,0,0,0,0,0,0,0,0,7,5,0,0,0,2,0,0,0,0,0,0,0,0,0,0,4,3,0,0,0,1,1,0,0,0,0,0,0,0,0,0,6,6,0,0,0,2,3', 'edge=1,252,1,32,124,194,63,252,67,15,240,1,7,244,66,47,0,192,63', 'texture=1,78,27,37,13,6,6,7,78']}
    ------------------
Second Dictionary
    {'/Users/test/kodax-camera-M531.jpg': ['fcolor=2,74,6,20,30,1,2,0,1,0,0,0,1,3,2,0,0,0,0,0,1,1,1,0,0,2,0,0,0,2,2,0,0,0,0,0,2,2,1,0,0,5,0,0,0,1,4,0,0,0,0,0,2,2,1,0,0,1,0,0,0,3,1,0,0,0,0,0,1,1,0,0,0,3,0,0,0,1,2,0,0,0,0,0,2,2,1,0,0,4,0,0,0,0,5,0,0,0,0,0,2,1,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0', 'edge=1,4,1,88,128,22,8,39,25,142,230,226,31,60,64,255,252,12,76', 'texture=1,15,32,31,28,19,16,12,98']}
    ++++++++++++++++++
    /Users/test/Transcend-8GB-Class-10-SDHC-Flash-Memory-Card.jpg
    fcolor=2,4,14,5,0,0,0,0,0,0,0,0,0,0,12,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,6,0,0,0,0,1,0,0,0,0,0,0,0,0,0,20,9,0,0,0,2,2,0,0,0,0,0,0,0,0,0,13,6,0,0,0,1,0,0,0,0,0,0,0,0,0,0,10,8,0,0,0,1,2,0,0,0,0,0,0,0,0,0,17,17,0,0,0,3,6,0,0,0,0,0,0,0,0,0,7,5,0,0,0,2,0,0,0,0,0,0,0,0,0,0,4,3,0,0,0,1,1,0,0,0,0,0,0,0,0,0,6,6,0,0,0,2,3
    fcolor=2,4,14,5,0,0,0,0,0,0,0,0,0,0,12,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,6,0,0,0,0,1,0,0,0,0,0,0,0,0,0,20,9,0,0,0,2,2,0,0,0,0,0,0,0,0,0,13,6,0,0,0,1,0,0,0,0,0,0,0,0,0,0,10,8,0,0,0,1,2,0,0,0,0,0,0,0,0,0,17,17,0,0,0,3,6,0,0,0,0,0,0,0,0,0,7,5,0,0,0,2,0,0,0,0,0,0,0,0,0,0,4,3,0,0,0,1,1,0,0,0,0,0,0,0,0,0,6,6,0,0,0,2,3


    fcolor=2,74,6,20,30,1,2,0,1,0,0,0,1,3,2,0,0,0,0,0,1,1,1,0,0,2,0,0,0,2,2,0,0,0,0,0,2,2,1,0,0,5,0,0,0,1,4,0,0,0,0,0,2,2,1,0,0,1,0,0,0,3,1,0,0,0,0,0,1,1,0,0,0,3,0,0,0,1,2,0,0,0,0,0,2,2,1,0,0,4,0,0,0,0,5,0,0,0,0,0,2,1,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0
    fcolor=2,4,14,5,0,0,0,0,0,0,0,0,0,0,12,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,6,0,0,0,0,1,0,0,0,0,0,0,0,0,0,20,9,0,0,0,2,2,0,0,0,0,0,0,0,0,0,13,6,0,0,0,1,0,0,0,0,0,0,0,0,0,0,10,8,0,0,0,1,2,0,0,0,0,0,0,0,0,0,17,17,0,0,0,3,6,0,0,0,0,0,0,0,0,0,7,5,0,0,0,2,0,0,0,0,0,0,0,0,0,0,4,3,0,0,0,1,1,0,0,0,0,0,0,0,0,0,6,6,0,0,0,2,3


    edge=1,4,1,88,128,22,8,39,25,142,230,226,31,60,64,255,252,12,76
    fcolor=2,4,14,5,0,0,0,0,0,0,0,0,0,0,12,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,6,0,0,0,0,1,0,0,0,0,0,0,0,0,0,20,9,0,0,0,2,2,0,0,0,0,0,0,0,0,0,13,6,0,0,0,1,0,0,0,0,0,0,0,0,0,0,10,8,0,0,0,1,2,0,0,0,0,0,0,0,0,0,17,17,0,0,0,3,6,0,0,0,0,0,0,0,0,0,7,5,0,0,0,2,0,0,0,0,0,0,0,0,0,0,4,3,0,0,0,1,1,0,0,0,0,0,0,0,0,0,6,6,0,0,0,2,3


    texture=1,15,32,31,28,19,16,12,98
    edge=1,252,1,32,124,194,63,252,67,15,240,1,7,244,66,47,0,192,63
    edge=1,252,1,32,124,194,63,252,67,15,240,1,7,244,66,47,0,192,63


    fcolor=2,74,6,20,30,1,2,0,1,0,0,0,1,3,2,0,0,0,0,0,1,1,1,0,0,2,0,0,0,2,2,0,0,0,0,0,2,2,1,0,0,5,0,0,0,1,4,0,0,0,0,0,2,2,1,0,0,1,0,0,0,3,1,0,0,0,0,0,1,1,0,0,0,3,0,0,0,1,2,0,0,0,0,0,2,2,1,0,0,4,0,0,0,0,5,0,0,0,0,0,2,1,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0
    edge=1,252,1,32,124,194,63,252,67,15,240,1,7,244,66,47,0,192,63


    edge=1,4,1,88,128,22,8,39,25,142,230,226,31,60,64,255,252,12,76
    edge=1,252,1,32,124,194,63,252,67,15,240,1,7,244,66,47,0,192,63


    texture=1,15,32,31,28,19,16,12,98
    texture=1,78,27,37,13,6,6,7,78
    texture=1,78,27,37,13,6,6,7,78


    fcolor=2,74,6,20,30,1,2,0,1,0,0,0,1,3,2,0,0,0,0,0,1,1,1,0,0,2,0,0,0,2,2,0,0,0,0,0,2,2,1,0,0,5,0,0,0,1,4,0,0,0,0,0,2,2,1,0,0,1,0,0,0,3,1,0,0,0,0,0,1,1,0,0,0,3,0,0,0,1,2,0,0,0,0,0,2,2,1,0,0,4,0,0,0,0,5,0,0,0,0,0,2,1,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0
    texture=1,78,27,37,13,6,6,7,78


    edge=1,4,1,88,128,22,8,39,25,142,230,226,31,60,64,255,252,12,76
    texture=1,78,27,37,13,6,6,7,78


    texture=1,15,32,31,28,19,16,12,98

Obviously as you can see I am spitting out way to many repetition of same value

These are the actual dictionaries that I am dealing with:

Dictionary 1:

{'/Users/test/Transcend-8GB-Class-10-SDHC-Flash-Memory-Card.jpg': ['fcolor=2,4,14,5,0,0,0,0,0,0,0,0,0,0,12,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,6,0,0,0,0,1,0,0,0,0,0,0,0,0,0,20,9,0,0,0,2,2,0,0,0,0,0,0,0,0,0,13,6,0,0,0,1,0,0,0,0,0,0,0,0,0,0,10,8,0,0,0,1,2,0,0,0,0,0,0,0,0,0,17,17,0,0,0,3,6,0,0,0,0,0,0,0,0,0,7,5,0,0,0,2,0,0,0,0,0,0,0,0,0,0,4,3,0,0,0,1,1,0,0,0,0,0,0,0,0,0,6,6,0,0,0,2,3', 'edge=1,252,1,32,124,194,63,252,67,15,240,1,7,244,66,47,0,192,63', 'texture=1,78,27,37,13,6,6,7,78']}

Dictionary 2:

{'/Users/test/kodax-camera-M531.jpg': ['fcolor=2,74,6,20,30,1,2,0,1,0,0,0,1,3,2,0,0,0,0,0,1,1,1,0,0,2,0,0,0,2,2,0,0,0,0,0,2,2,1,0,0,5,0,0,0,1,4,0,0,0,0,0,2,2,1,0,0,1,0,0,0,3,1,0,0,0,0,0,1,1,0,0,0,3,0,0,0,1,2,0,0,0,0,0,2,2,1,0,0,4,0,0,0,0,5,0,0,0,0,0,2,1,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0', 'edge=1,4,1,88,128,22,8,39,25,142,230,226,31,60,64,255,252,12,76', 'texture=1,15,32,31,28,19,16,12,98']}

I have lamba function

cosinLamba = lambda a, b : round(NP.inner(a, b)/(LA.norm(a)*LA.norm(b)), 3)

I want to iterate over dictionary 1 and dictionary 2 and get the fcolor value of dictionary1 'fcolor=2,4,14,5,0,0,0,0,0,0,0,0,0,0,12,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,6,0,0,0,0,1,0,0,0,0,0,0,0,0,0,20,9,0,0,0,2,2,0,0,0,0,0,0,0,0,0,13,6,0,0,0,1,0,0,0,0,0,0,0,0,0,0,10,8,0,0,0,1,2,0,0,0,0,0,0,0,0,0,17,17,0,0,0,3,6,0,0,0,0,0,0,0,0,0,7,5,0,0,0,2,0,0,0,0,0,0,0,0,0,0,4,3,0,0,0,1,1,0,0,0,0,0,0,0,0,0,6,6,0,0,0,2,3'

and fcolor value of dictionary2

'fcolor=2,74,6,20,30,1,2,0,1,0,0,0,1,3,2,0,0,0,0,0,1,1,1,0,0,2,0,0,0,2,2,0,0,0,0,0,2,2,1,0,0,5,0,0,0,1,4,0,0,0,0,0,2,2,1,0,0,1,0,0,0,3,1,0,0,0,0,0,1,1,0,0,0,3,0,0,0,1,2,0,0,0,0,0,2,2,1,0,0,4,0,0,0,0,5,0,0,0,0,0,2,1,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0'

Send them to my lamba function cosinLamba(valu1, value2) value1 and value 2 are strings and thats why I have saved them in my dictionary these as values. And I want to do for fcolor, texture, edge all the vectors that I have stored for given picture in each dictionary.

Answer 1

you could start by changing your representation to:

{'someFileName.jpg' : {'forecolor': [2,3,5,5,6],'edge': [2,4,5],'texture':[5,4,3]}}

Or

{('someFileName.jpg', 'forecolor'): [2,3,5,5,6],
 ('someFileName.jpg', 'edge'): [2,4,5],
 ('someFileName.jpg', 'texture'):[5,4,3]}

For example, to get corresponding lists for the 1st case:

from itertools import product

# pair info for each image with info of every image from another dictionary
for (fn1, d1), (fn2,d2) in product(dict1.iteritems(), dict2.iteritems()):
    for property_, list_value in d1.iteritems():
        compute_cosine_similarity(list_value, d2[property_])

Using your representation with list of string it looks like:

from itertools import product

# pair info for each image with info of every image from another dictionary
for (fn1,lst1), (fn2,lst2) in product(dict1.iteritems(), dict2.iteritems()):
    # assume all lists has the same order of elements
    for string_value1, string_value2 in zip(lst1, lst2):
        compute(string_value1, string_value2)

You shouldn’t store numbers as a list of ascii strings. If you need to save memory you could use numpy arrays. cosinLamba already accepts them.

from collections import namedtuple
import numpy as np

Info = namedtuple('Info', 'forecolor edge texture')

dict1 = {'someFileName.jpg': Info(np.array([...], dtype=np.uint8),
                                  np.array([...], dtype=np.uint8),
                                  np.array([...], dtype=np.uint8))}

The code to call cosine_similarity() is exactly the same as for your representation.