Question

I am trying to get data from a nested java script structure. I want to use json.loads() for this. However, I am getting an error saying "No JSON object could be decoded"

Following is the code that I have tried out and the java script structure:
Code

page_us = urllib2.urlopen('http://www.verizonwireless.com/smartphones-2.shtml')

soup_us = BeautifulSoup(page_us)
scripts_us = soup_us.findAll('script')

script=[]
for s in scripts_us:
    if s.string and "$j('#module_1_Tile" in s.string:
        script.append(s.text.split('data')[1].replace("\n","").replace("(","").replace(")","").replace(";","").replace("\t",""))

Data Structure

script[1] = u'{"phones":{"id5986":{"id":"5986","rating":"stars_4","colorName":"White","colorCode":"#FFFFFF","capacity":"16 GB","price":"$149.99","fullPrice":"$599.99","addToCartQty":"0","image":"http://s7.vzw.com/is/image/VerizonWireless/Motorola%5Fdroid%5Frazr%5Fhd%5Fwhite?$device%2Dmed$","ATCST":"submitThisPhone","MAST":"false","CIL":"0","IRURL":"https://preorder.verizonwireless.com/iconic/","BAGX":"false","priceRange":"150","rating":"4","OOS":"","freeShipping":"freeOvernightShippingHTML","bagxGetPhone":"","badges":{"lteBadge","vzwExclusiveBadge","globalReadyBadge"},"vPrice":"$221.96","vFullPrice":"$671.96","vBundleName":"DROID RAZR HD by Motorola in White Bluetooth&reg Pack","vBundleImage":"http://s7.vzw.com/is/image/VerizonWireless/moto%5Fdroid%5Frazr%5Fhd%5Fwht%5Fbluetooth%5Fvirt%5Fbndl?$device%2Dmed$","vBundleDescription":"<ul><li>Bluetooth&reg Headset</li><li>Clear Hard Cover</li><li>Vehicle Charger</li></ul>"},"id5985":{"id":"5985","rating":"stars_4_5","colorName":"Black","colorCode":"#000000","capacity":"16 GB","price":"$149.99","fullPrice":"$599.99","addToCartQty":"0","image":"http://s7.vzw.com/is/image/VerizonWireless/Motorola%5Fdroid%5Frazr%5Fhd%5Fblack?$device%2Dmed$","ATCST":"submitThisPhone","MAST":"false","CIL":"0","IRURL":"https://preorder.verizonwireless.com/iconic/","BAGX":"false","priceRange":"150","rating":"4_5","OOS":"","freeShipping":"freeOvernightShippingHTML","bagxGetPhone":"","badges":{"lteBadge","vzwExclusiveBadge","globalReadyBadge"},"vPrice":"$221.96","vFullPrice":"$671.96","vBundleName":"DROID RAZR HD by Motorola Bluetooth&reg Pack","vBundleImage":"http://s7.vzw.com/is/image/VerizonWireless/moto%5Fdroid%5Frazr%5Fhd%5Fblk%5Fbluetooth%5Fvirt%5Fbndl?$device%2Dmed$","vBundleDescription":"<ul><li>Bluetooth&reg Headset</li><li>Silicone Cover</li><li>Vehicle Charger</li></ul>"}},"options":{"colorName":"Black","colorCode":"#000000","capacity":"16 GB"},"info":{"brand":"486","os":"10351","features":{502,569,501,568,247,318,85,503,497,431,458,11,49,150,44,17,141,145,165,20,58,15,24,172,186,184,159,187,185,199,156,249,157,189,142,168,211,13,188,239,14,167,321,41,25,357,443,441,442,444,459,418,416,12,413,5,61,7,446,504,362,573,202,522,"hasVB"},"priceRange":"150","phoneId":"id5985","ATCST":"submitThisPhone","MAST":"false","CIL":"0","IRURL":"https://preorder.verizonwireless.com/iconic/","bagxGetPhone":"","BAGX":"false"}}'

json.loads works fine for script[0], but gives the above error on script[1]. Please let me know what am I missing over here. Thanks.

Answer 1

This section has curly braces instead of square brackets –

{"lteBadge","vzwExclusiveBadge","globalReadyBadge"}

Should be –

["lteBadge","vzwExclusiveBadge","globalReadyBadge"]

You can use Python to help you identify these errors –

>>> from ast import literal_eval
>>> # Break it into lines to check location of error
>>> s = data_structure.replace(',', ',\n')
>>> literal_eval(s)