Question

Is anyone aware of a nearest neighbor algorithm implemented in Python that can be updated incrementally? All the ones I’ve found, such as this one, appear to be batch processes. Is it possible to implement an incremental NN algorithm?

Answer 1

I think the problem with incremental construction of a KD-tree or KNN-tree is, as you’ve alluded to in a comment, that the tree will eventually become unbalanced and you can’t do simple tree rotation to fix balance problems and keep consistency. At the minimum, the re-balancing task is not trivial and one would definitely not want to do it at each insertion. Often, one will choose to build a tree with a batch method, insert a bunch of new points and allow the tree to become unbalanced up to a point, and then re-balance it.

A very similar thing to do is to build the data structure in batch for M points, use it for M’ points, and then re-build the data structure in batch with M+M’ points. Since re-balancing is not normal, fast algorithm we are familiar with for trees, rebuilding is not necessarily slow in comparison and in some cases can be faster (depending on how the sequence of the points entering your incremental algorithm).

That being said, the amount of code you write, debugging difficulty, and the ease of others’ understanding of your code can be significantly smaller if you take the rebuild approach. If you do so, you can use a batch method and keep an external list of points not yet inserted into the tree. A brute force approach can be used to ensure none of these is closer than the ones in the tree.

Some links to Python implementations/discussions are below, but I haven’t found any that explicitly claim to be incremental. Good luck.

http://www.scipy.org/Cookbook/KDTree

http://cgi.di.uoa.gr/~compgeom/pycgalvisual/kdppython.shtml

http://sites.google.com/site/mikescoderama/Home/kd-tree-knn

http://en.wikipedia.org/wiki/Kd-tree

Note: My comments here apply to high-dimensional spaces. If you’re working in 2D or 3D, what I’ve said may not be appropriate. (If you working in very high dimensional spaces, use brute force or approximate nearest neighbor.)