Question

My teacher said that Cohen Sutherland algorithm http://en.wikipedia.org/wiki/Cohen%E2%80%93Sutherland_algorithm cannot be used with concave polygon. I asked him why and he could not give me an answer(I don’t why hew works here).

So I ask You: Why Cohen-Sutherland algorithm is not appliable on concave polygon? What will go wrong?

Answer 1

From the Wikipedia article:

The algorithm includes, excludes or partially includes the line based
on where:

• Both endpoints are in the viewport region (bitwise OR of endpoints == 0): trivial accept.
• …

Take a look at the (informal) definition of convexity:

an object is convex if for every pair of points within the object, every point on the straight line segment that joins them is also within the object.

Your teacher may either be talking about scanlines generated from convex polygons, or lines as a special case of a convex polygon.

If the polygon is not convex, concave regions may be drawn as filled in based on the endpoints, as any point along the line is assumed to be part of the polygon. Similar assumptions apply for the other bullet points in the article.