I have this huge database of information that I need to cut down on size as it’s pretty big and very taxing to load up. What is the best way to go about ‘zipping’ this up and then expanding it using javascript?
Are there any tools that can do this for me? (Dean Edward’s isn’t packed enough, and gzip isn’t available)
I’m looking for something that makes use of all the allowed javascript variables and looks for patterns (which there are many), similar to the way zip compression works.
Here’s a sample of how my database is structured (the final result need to be in this format):
var crops = {
Jan1: {
Apple: [1995, 1996, 1998, 2003, 2007, 2008, 2009],
Banana: [1995, 1998, 2001, 2002, 2003, 2007, 2008, 2009],
Cantaloupe: [2000, 2003, 2004, 2005, 2006, 2007],
Date: [1995, 1998, 2001, 2002, 2003, 2004, 2007],
Elephant: [1995, 2002, 2004, 2007, 2009],
Fig: [1996, 2003, 2004, 2005, 2006, 2007],
Grape: [1999, 2000, 2001, 2004, 2006, 2008],
Honeydew: [2001, 2005, 2008],
Ita: [1997, 2000, 2001, 2002, 2004, 2005],
Jasmine: [1997, 1998, 2002, 2004, 2005, 2007, 2008],
Kiwi: [1997, 1998, 2003],
Lemon: [1995, 1997, 2001, 2003, 2004, 2007, 2009],
Melon: [1995, 1997, 2000, 2003, 2005],
Nectarine: [1996, 1998, 2000, 2002, 2007],
Orange: [1996, 1998, 2000, 2003, 2007],
Pineapple: [1996, 1997, 1999, 2000, 2002, 2003, 2005, 2007, 2008, 2009],
Quince: [1995, 1997, 1999, 2000, 2001, 2002, 2003, 2004, 2007, 2008],
Raspberry: [1995, 1996, 1997, 1998, 2000, 2001, 2002, 2007],
Strawberry: [2002, 2003, 2006, 2008],
Tangerine: [1995, 1997, 1998, 1999, 2006, 2008],
Ugli: [1997, 2001, 2002, 2006, 2009],
Vanilla: [1998, 1999, 2001, 2002, 2004, 2005],
Watermelon: [1997, 1998, 2001, 2002, 2003, 2005],
Xigua: [1996, 2001, 2002, 2003, 2004, 2005, 2006],
Yali: [1997, 1998, 1999, 2000, 2001, 2002, 2005, 2006, 2009],
Zucchini: [2001, 2002]
},
Jan2: {
Banana: [2005, 2009],
Cantaloupe: [2006],
Grape: [1997, 2005, 2006, 2007],
Honeydew: [1999],
Ita: [2001, 2002],
Jasmine: [1999],
Kiwi: [2008, 2009],
Nectarine: [2001],
Orange: [1997, 2008],
Quince: [2005],
Raspberry: [2002],
Strawberry: [2004],
Vanilla: [1995, 1999],
Watermelon: [1997, 2002],
Xigua: [2006, 2007],
Yali: [2001, 2005],
Zucchini: [2004]
},
Jan3: {
Apple: [1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2009],
Banana: [1997, 1998, 2001, 2006],
Cantaloupe: [1995, 1996, 1997, 1998, 1999, 2000, 2002, 2006, 2008],
Date: [1995, 1998, 1999, 2000, 2001, 2005, 2007, 2008, 2009],
Elephant: [1996, 1997, 1998, 2000, 2002, 2003, 2004, 2005, 2006, 2007],
Fig: [1998, 1999, 2000, 2002, 2003, 2009],
Grape: [1997, 1998, 2002, 2005, 2006, 2007],
Honeydew: [1995, 1996, 2000, 2001, 2002, 2003, 2005, 2006, 2007, 2008],
Ita: [1995, 1996, 2000, 2001, 2004, 2005, 2008, 2009],
Jasmine: [1995, 1999, 2002, 2007, 2008],
Kiwi: [1996, 2003, 2006, 2008, 2009],
Lemon: [1995, 1996, 1999, 2002, 2006, 2007, 2009],
Melon: [1997, 1999, 2002, 2005, 2008],
Nectarine: [1998, 2000, 2002, 2004, 2005, 2008],
Orange: [1995, 1997, 2000, 2002, 2005, 2007, 2009],
Pineapple: [1998, 2001, 2002, 2004, 2005, 2007, 2009],
Quince: [1995, 1997, 2000, 2002, 2004, 2005, 2006, 2007, 2009],
Raspberry: [1995, 1996, 1997, 1998, 2002, 2004, 2007, 2008],
Strawberry: [1996, 1997, 1999, 2000, 2006],
Tangerine: [1999, 2001, 2004, 2007, 2008],
Ugli: [1995, 1997, 1998, 1999, 2000, 2006, 2007, 2009],
Vanilla: [1996, 1997, 2003, 2006, 2008],
Watermelon: [1995, 1997, 1999, 2002, 2003, 2005],
Xigua: [1996, 1997, 1999, 2001, 2004, 2006, 2008, 2009],
Yali: [1996, 1998, 2001, 2003, 2004, 2005, 2007, 2008],
Zucchini: [1997, 2003, 2006, 2007, 2009]
},
Jan4: {
Apple: [2000, 2002, 2003],
Cantaloupe: [2009],
Elephant: [2003],
Fig: [2001, 2008],
Honeydew: [2001],
Pineapple: [2004],
Quince: [1999],
Ugli: [2009],
Vanilla: [2005],
Yali: [2007]
},
Feb1: {
Apple: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
Banana: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2008, 2009],
Cantaloupe: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Date: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2004, 2005, 2006, 2007, 2008, 2009],
Elephant: [1996, 1997, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
Fig: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Grape: [1995, 1996, 1997, 1998, 1999, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Honeydew: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Ita: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
Jasmine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Kiwi: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Lemon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Melon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Nectarine: [1995, 1996, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Orange: [1995, 1996, 1997, 1998, 1999, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Pineapple: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Quince: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2004, 2005, 2006, 2007, 2008],
Raspberry: [1995, 1996, 1997, 1998, 1999, 2000, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Strawberry: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2004, 2005, 2006, 2007, 2009],
Tangerine: [1995, 1996, 1997, 1998, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008],
Ugli: [1996, 1997, 1998, 1999, 2000, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Vanilla: [1995, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008, 2009],
Watermelon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2004, 2006, 2007, 2009],
Xigua: [1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Yali: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2005, 2006, 2007, 2008, 2009],
Zucchini: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009]
},
Feb2: {
Apple: [1995, 1996, 1998, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Banana: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2004, 2005, 2006, 2008, 2009],
Cantaloupe: [1995, 1996, 1997, 1998, 1999, 2001, 2004, 2005, 2006, 2007],
Date: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
Elephant: [1996, 1997, 1998, 1999, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Fig: [1995, 1996, 1997, 1998, 1999, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Grape: [1995, 1996, 1998, 1999, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Honeydew: [1995, 1996, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Ita: [1995, 1997, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Jasmine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2004, 2005, 2006, 2007, 2008],
Kiwi: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2006, 2007, 2009],
Lemon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2004, 2005, 2006, 2007, 2008],
Melon: [1995, 1996, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Nectarine: [1995, 1996, 1997, 1998, 1999, 2000, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Orange: [1995, 1996, 1997, 1998, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Pineapple: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
Quince: [1995, 1996, 1997, 1999, 2000, 2001, 2002, 2003, 2006, 2007, 2008, 2009],
Raspberry: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Strawberry: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
Tangerine: [1996, 1997, 1998, 2000, 2001, 2002, 2003, 2004, 2005, 2008, 2009],
Ugli: [1995, 1997, 1998, 1999, 2000, 2001, 2003, 2006, 2008, 2009],
Vanilla: [1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2008],
Watermelon: [1997, 1999, 2000, 2001, 2002, 2003, 2004, 2007, 2009],
Xigua: [1995, 1996, 1997, 1999, 2000, 2001, 2002, 2005, 2006, 2007, 2008, 2009],
Yali: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2006, 2007, 2008, 2009],
Zucchini: [1995, 1997, 1998, 1999, 2001, 2003, 2006, 2007, 2008, 2009]
},
Feb3: {
Apple: [1996, 1997, 2003, 2004],
Banana: [1998],
Cantaloupe: [1996, 1997, 1999, 2000],
Date: [1996, 1997, 2006],
Elephant: [2004],
Fig: [1995, 1996, 2001, 2006, 2008],
Grape: [2003],
Honeydew: [2004],
Ita: [1997, 2009],
Jasmine: [1999, 2004, 2006, 2008],
Kiwi: [1999, 2002, 2003, 2006],
Lemon: [1995, 1997, 2004],
Melon: [2008],
Nectarine: [1995, 1998, 2005],
Orange: [2003, 2009],
Pineapple: [1995, 2001],
Quince: [1995, 1996, 2000, 2004, 2005, 2006],
Raspberry: [1996, 1999, 2000, 2003, 2009],
Strawberry: [1995, 1998, 2001, 2004, 2008],
Tangerine: [1996, 1998, 2002],
Ugli: [2008],
Vanilla: [1996, 2007, 2009],
Watermelon: [2006, 2008],
Xigua: [1998, 1999, 2002, 2005, 2009],
Yali: [1996, 2000, 2001, 2002, 2004],
Zucchini: [1998, 2000, 2003, 2007]
},
Feb4: {
Apple: [1995, 1996, 1997, 1998, 2000, 2002, 2003, 2006, 2009],
Banana: [1995, 1996, 1997, 2000, 2002, 2004, 2005, 2007, 2008, 2009],
Cantaloupe: [1995, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008],
Date: [1995, 1997, 1998, 2000, 2001, 2002, 2004, 2005, 2008, 2009],
Elephant: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2006, 2007, 2008, 2009],
Fig: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2005, 2007, 2008, 2009],
Grape: [1995, 1996, 1997, 1998, 2003, 2005, 2006, 2007, 2008],
Honeydew: [1995, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
Ita: [1995, 1997, 1999, 2000, 2001, 2006, 2007, 2008, 2009],
Jasmine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008, 2009],
Kiwi: [1995, 1996, 1997, 1999, 2000, 2002, 2003, 2004, 2005, 2007, 2008],
Lemon: [1995, 1996, 1998, 1999, 2000, 2001, 2003, 2005, 2006, 2007, 2008, 2009],
Melon: [1995, 1996, 1998, 2000, 2002, 2003, 2005, 2006, 2007, 2008],
Nectarine: [1996, 1997, 1998, 2000, 2003, 2004, 2006, 2007, 2008, 2009],
Orange: [1996, 1998, 1999, 2000, 2001, 2002, 2003, 2006, 2008],
Pineapple: [1995, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
Quince: [1995, 1996, 1997, 1998, 1999, 2000, 2002, 2004, 2005, 2007, 2009],
Raspberry: [1997, 2002, 2004, 2005, 2006, 2007, 2008, 2009],
Strawberry: [1995, 1996, 1997, 1998, 2000, 2001, 2004, 2007, 2008],
Tangerine: [1997, 1998, 1999, 2000, 2002, 2003, 2004, 2005, 2006, 2007, 2008],
Ugli: [1995, 1996, 1997, 1998, 1999, 2001, 2003, 2007, 2008, 2009],
Vanilla: [1997, 1998, 1999, 2000, 2001, 2002, 2003, 2005, 2006, 2007, 2008, 2009],
Watermelon: [1995, 1996, 1997, 1998, 1999, 2001, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Xigua: [1996, 1997, 1998, 2000, 2003, 2005, 2007, 2008, 2009],
Yali: [1995, 1997, 1998, 2005, 2007, 2008],
Zucchini: [1995, 1996, 1997, 2000, 2003, 2005, 2006, 2007, 2008, 2009]
},
Mar1: {
Apple: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Banana: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Cantaloupe: [1995, 1996, 1997, 1998, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Date: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Elephant: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Fig: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Grape: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Honeydew: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Ita: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Jasmine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Kiwi: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Lemon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Melon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Nectarine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2004, 2005, 2006, 2007, 2008, 2009],
Orange: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Pineapple: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Quince: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Raspberry: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008],
Strawberry: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Tangerine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Ugli: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Vanilla: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Watermelon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Xigua: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Yali: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
Zucchini: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009]
}
/* Much much more */
}
You need to compact your data. You have 12 months, (say) 20 fruits, and a range of years that, presumably, has a lower limit. Store the data like this:
Each array element is:
If you need it in long form to deal with, you can convert it easily with a reference.
1 = Apple, 2= Banana, …
1 = Jan, 2 = Feb, …
1 = 1995, 2= 1996, …