To customize the default blue gradient highlight style I made a subclass of NSOutlineView and overrode the method -highlightSelectionInClipRect, like so:
- (void)highlightSelectionInClipRect:(NSRect)theClipRect
{
NSRange aVisibleRowIndexes = [self rowsInRect:theClipRect];
NSIndexSet *aSelectedRowIndexes = [self selectedRowIndexes];
NSInteger aRow = aVisibleRowIndexes.location;
NSInteger anEndRow = aRow + aVisibleRowIndexes.length;
for (int aRow; aRow < anEndRow; aRow++) {
if([aSelectedRowIndexes containsIndex:aRow]) {
// draw gradient
}
}
}
This works fine, but sometimes the background is not drawn. On the screenshot below you can see how the selection highlight is not drawn when clicking on the first item after the last one was selected.

It seems as if this only happens if the new selected item is not directly beneath or above the old selected one. Selecting five items in the order 1-2-3-4-5-4-3-2-1 always draws the appropriate background, anything else (e.g. 1-2-5) not.
Why is this happening? If you need any more details I will be glad to add some more code, but at the meantime I have no clue where to search for this behaviour.
here is my (very simple) solution using blocks: