Question

I’ve got an array of words I need to sort by frequency. Before I do that, I need to strip out words like ‘the,’ ‘it,’ etc (anything less than three letters, really), as well as all numerals and any words beginning with # (the array of words is pulled from Twitter, although the example below is just a random paragraph from Wikipedia).

I can remove one word, but have been going crazy trying to remove more than one, or a range. Any suggestions? Thank you!

http://jsfiddle.net/9NzAC/6/

HTML:

<div id="text" style="background-color:Teal;position:absolute;left:100px;top:10px;height:500px;width:500px;">
Phrenology is a pseudoscience primarily focused on measurements of the human skull, based on the concept that the brain is the organ of the mind, and that certain brain areas have localized, specific functions or modules. The distinguishing feature of phrenology is the idea that the sizes of brain areas were meaningful and could be inferred by examining the skull of an individual.
</div>

JS:

//this is the function to remove words
<script type="text/javascript">
    function removeA(arr){
        var what, a= arguments, L= a.length, ax;
        while(L> 1 && arr.length){
            what= a[--L];
            while((ax= arr.indexOf(what))!= -1){
                arr.splice(ax, 1);
            }
        }
            return arr;
        }
</script>

//and this does the sorting & counting
<script type="text/javascript">
    var getMostFrequentWords = function(words) {
        var freq={}, freqArr=[], i;

        // Map each word to its frequency in "freq".
            for (i=0; i<words.length; i++) {
            freq[words[i]] = (freq[words[i]]||0) + 1;
        }

        // Sort from most to least frequent.
            for (i in freq) freqArr.push([i, freq[i]]);
            return freqArr.sort(function(a,b) { return b[1] - a[1]; });
        };

        var words = $('#text').get(0).innerText.split(/\s+/);

        //Remove articles & words we don't care about.
        var badWords = "the";
            removeA(words,badWords);
        var mostUsed = getMostFrequentWords(words);
        alert(words);

</script>

Answer 1

Instead of removing from the original array, just push to a new one, it’s simpler, and it’ll make your code shorter and more readable.

var words = ['the', 'it', '12', '#twit', 'aloha', 'hello', 'bye']
var filteredWords = []

for (var i = 0, l = words.length, w; i < l; i++) {
    w = words[i]
    if (!/^(#|\d+)/.test(w) && w.length > 3)
        filteredWords.push(w)
}

console.log(filteredWords) // ['aloha', 'hello']

Demo: http://jsfiddle.net/VcfvU/