Question

if ((catA & maskB) != 0 && (catB & maskA) != 0)

It is in Box2d’s manual: 6.2, and is used to check if two objects should collide (after filtering)

Answer 1

catA is a bit field of collision categories for object A
maskA is a bit field of categories that object A can collide with.

For example:

catA  = 100010000010010 // Object A is in 4 collision categories
maskA = 001010000000000 // Object A can collide with too different categories

catB  = 000010001000001 // Object B is in 3 collision categories
maskB = 100000000000000 // Object B can only collide with the category represented by the highest bit

catA & maskB means the bits that are 1 in both catA and maskB, so 1000000000000000. It’s not 0, because Object B can collide with objects in the highest bit, and object A has that bit set.

catB & maskA means the bits that are 1 in both catB and maskA, so 0000100000000000. It’s also not zero since Object A can collide with objects in the 5th highest bit category, and Object B is in that category.

So the two objects can collide.