Question

I wonder if working with arrays (or objects) which contain about 100 000 elements (properties) can cause performance or memory problems in browsers when frequently access them using indexOf, slice etc. Are there some recommendations for working with big arrays in modern browsers?

My particular case. I have the following structure:

tack01: [array of elements 10 000 in average]
…
tack0n: [array of elements 10 000 in average]

tracks average amount is 10.

element looks like {id: “xa432fds”, some properties}

During the runtime I need to access any of the element knowing providing it’s id.

If I use this structure without transformation I need to perform search throught all tracks and use indexOf to find element with Id.

So I deside to create an index object which has following structure:
indexObj = {id1: reference to element with id1, id2: reference to element with id2}

to access a certain element I just need to access indexObj[id], is it right solution for my case?

All this should be performed on the client side.

Answer 1

It’s a very broad question.

I’d say the primary recommendation would be to really deeply understand what you’re working with. Arrays in JavaScript aren’t really arrays at all, they’re objects with all the normal plumbing of JavaScript normal objects. Array indexes aren’t numbers, aren’t offsets into some memory table*; they’re string keys in a dictionary-like map. Once you embrace the fact that arrays are just objects, it may open up new ways of structuring or accessing your data such that you can avoid expensive operations like indexOf.

(* Barring JavaScript engine optimizations, of course. And engines do optimize.)

---

Update: Looking at your edit, yes, transforming the data so that you can look up tracks by using their id as a property name (indexObj[id]) is what I’d recommend. Then, instead of the expensive linear search required by indexOf, you get the benefit of the JavaScript engine’s handling of property names, which will typically be a much more efficient lookup (b-trees and/or hash structures, etc.).

Once you’ve created your indexed version, if you can release the array version, the memory consumed by the array and its property names ("0", "1", etc.) can be eligible for reclamation, which may be useful (your individual tracks will remain in memory because you’re referencing them from the indexed structure).