Question

I’m using iPhone ARToolkit and I’m wondering how it works.

I want to know how with a destination location, a user location and a compass, this toolkit can know it user is looking to that destination.

How can I know the maths behind this calculations?

Answer 1

The maths that AR ToolKit uses is basic trigonometry. It doesn’t use the technique that Thomas describes which I think would be a better approach (apart from step 5. See below)

Overview of the steps involved.

The iPhone’s GPS supplies the device’s location and you already have the coordinates of the location you want to look at.

First it calculates the difference between the latitude and the longitude values of the two points. These two difference measurements mean you can construct a right-angled triangle and calculate what angle from your current position another given position is. This is the relevant code:

- (float)angleFromCoordinate:(CLLocationCoordinate2D)first toCoordinate:(CLLocationCoordinate2D)second {
    float longitudinalDifference = second.longitude - first.longitude;
    float latitudinalDifference = second.latitude - first.latitude;
    float possibleAzimuth = (M_PI * .5f) - atan(latitudinalDifference / longitudinalDifference);
    if (longitudinalDifference > 0) return possibleAzimuth;
    else if (longitudinalDifference < 0) return possibleAzimuth + M_PI;
    else if (latitudinalDifference < 0) return M_PI;

    return 0.0f;
}

At this point you can then read the compass value from the phone and determine what specific compass angle(azimuth) your device is pointing at. The reading from the compass will be the angle directly in the center of the camera’s view. The AR ToolKit then calculates the full range of angle’s currently displayed on screen as the iPhone’s field of view is known.

In particular it does this by calculating what the angle of the leftmost part of the view is showing:

double leftAzimuth = centerAzimuth - VIEWPORT_WIDTH_RADIANS / 2.0;

if (leftAzimuth < 0.0) {
    leftAzimuth = 2 * M_PI + leftAzimuth;
}

And then calculates the right most:

double rightAzimuth = centerAzimuth + VIEWPORT_WIDTH_RADIANS / 2.0;

if (rightAzimuth > 2 * M_PI) {
    rightAzimuth = rightAzimuth - 2 * M_PI;
}

We now have:

1. The angle relative to our current position of something we want to display
2. A range of angles which are currently visible on the screen

This is enough to plot a marker on the screen in the correct position (kind of…see problems section below)

It also does similar calculations related to the devices inclination so if you look at the sky you hopefully won’t see a city marker up there and if you point it at your feet you should in theory see cities on the opposite side of the planet. There are problems with these calculation in this toolkit however.

The problems…

Device orientation is not perfect

The value I’ve just explained the calculation of assumes you’re holding the device in an exact position relative to the earth. i.e. perfectly landscape or portrait. Your user probably won’t always be doing that. If you tilt the device slightly your horizon line will no longer be horizontal on screen.

The earth is actually 3D!

The earth is 3-dimensional. Few of the calculations in the toolkit account for that. The calculations it performs are only really accurate when you’re pointing the device towards the horizon.

For example if you try to plot a point on the opposite side of the globe (directly under your feet) this toolkit behaves very strangely. The approach used to calculate the azimuth range on screen is only valid when looking at the horizon. If you point your camera at the floor you can actually see every single compass point. The toolkit however, thinks you’re still only looking at compass reading ± (width of view / 2). If you rotate on the spot you’ll see your marker move to edge of the screen, disappear and then reappear on the other side. What you would expect to see is the marker stay on screen as you rotate.

The solution

I’ve recently implemented an app with AR which I initially hoped AR Toolkit would do the heavy lifting for me. I came across the problems just described which aren’t acceptable for my app so had to roll my own.

Thomas’ approach is a good method up to point 5 which as I explained above only works when pointing towards the horizon. If you need to plot anything outside of that it breaks down. In my case I have to plot objects that are overhead so it’s completely unsuitable.

I addressed this by using OpenGL ES to plot my markers where they actually are in 3D space and move the OpenGL viewport around according to readings from the gyroscope while continuously re-calibrating against the compass. The 3D engine handles all the hard work of determining what’s on screen.

Hope that’s enough to get you started. I wish I could provide more detail than that but short of posting a lot of hacky code I can’t. This approach however did address both problems described above. I hope to open source that part of my code at some point but it’s very rough and coupled to my problem domain at the moment.