Question

I m working on a application which is a service. I am receiving a request object and I need to pass this object through set of filters and return the response. There are about 10 filters I need to pass the object through.

Currently the application is doing a sequential search on every filter as follows:

public List<Element) FilterA(Request request){
  for(Element element in items)
  {
     // compare element to request object elements   
     // there are different field checking per object
  }
}

So there is FilterB, FilterC etc. they are all done in similar fashion, within for loops different fields are being compared.

Can this be done via hashset? or Binary search?

Or is there an efficient algorithm. Essentially I d like to improve the O(n) to something less.

Answer 1

If you have n lists and f filters there are bascially only two approaches: iterate through the list and apply each filter to each individual element (keep it if it passes all of them, remove it otherwise); or do what you’re doing now and let each filter iterate over the entire list. Both have a worst-case complexity of O(n*f), assuming O(1) element removal (I recommend using a LinkedList to achieve this, copy the contents to one if necessary).

You can really only improve upon this complexity by utilising properties of your input. Maybe you can combine multiple filters into one (when they’re range checks, for instance) or maybe taking one element from the list will also result in the removal of others. Also, if you can guess which filters will probably remove more elements it will pay off to run these first.

So yeah, it really depends on what kind of stuff you’re filtering and what your filters look like. In the most general case you can’t win much (as long as you’re already using lists from which you can remove elements in O(1) time) but you might gain something if you take knowledge of your input into account.