I’m moving a character (ellipsoid) around in my physics engine. The movement must be constrained by the static geometry, but should slide on the edges, so it won’t be stuck.
My current approach is to move it a little and then push it back out of the geometry. It seems to work, but I think it’s mostly because of luck. I fear there must be some corner cases where this method will go haywire. For example a sharp corner where two walls keeps pushing the character into each other.
How would a “state of the art” game engine solve this?
I have not developed a state of the art game engine, but I once wrote a racing game where collision was simply handled by reversing the simulation time and calculate where the edge was crossed. Then the car was allowed to bounce back into the game field. The penalty was that the controls was disabled until the car stopped.
So my suggestion is that you run your physics engine to calculate exactly where the edge is hit (it might need some non-linear equation solving approach), then you change your velocity vector to either bounce off or follow the edge.
In the case of protecting against corner cases, one could always keep a history of the last valid position within the game and state of the physics engine. If the game gets stuck, the simulation can be restarted from that point but with a different condition (say by adding some randomization to the internal parameters).