Question

I would like to start developing for iOS. Coming from an Android developing background, I know that the more types of devices you can get your hands on, the better testing will be, as all devices have wildly different specs, and what may work perfectly in your test device may not even run in another one, let alone look good.

I know that testing on the actual device is very important, as there are many limitations on what you can test on an emulator, so I’ve decided to get an actual device.

However, there are also tons of devices available in the iOS world! There’s the iPod touch, the iPad and the iPhone, each in several different generations and configurations (8GB version vs 16GB version, WiFi version, 3G version, etc.). Not also the screen sizes, but also the aspect ratio is very different across devices, and also the included sensors.

I think that getting an application to run in varied devices should not be difficult, but is it necessary to actually test on all the device types you plan to support? Apple is not renown for its low price, and I would like to keep the starting costs as low as possible.

So, to conclude: Is it necessary/recommended to test on as many device types as you can in the iOS development world?

A small clarification: I’m specially asking if it is possible for there to be compatibility issues related to a specific device/family-of-devices that I would not be able to catch either by testing on the emulator nor a totally different device.

Answer 1

Generally speaking, the major differences in capabilities between testing on the simulator and testing on a device are:

• The simulator does not use exactly the same sandboxing as the device. So, for instance, if your provisioning profile is missing your Passbook credentials, this problem will show up on a physical device but not on the simulator.
• The simulator doesn’t generally support GPS, multitouch, push notifications, Bluetooth, and some other specific features.
• On a non-retina display, the simulator view for an iPhone 5+ or (especially!) a retina iPad will be nigh unusable at 100% because its size will exceed the size of your screen.
• There are a few, very rare, crashes that occur only on the simulator and a few that occur only on the device.
• The simulator does not always support the earliest iOS versions your app supports. For instance, the current version of Xcode (which you must use if you want to build for the latest iOS version) only has simulators from 5.0+ available.
• Certain profiling with Instruments is, as far as I can tell, only available in the Simulator.

Now, in my specific case, I try to test on one of each screen resolution I support and one of each major OS version I support.

This boils down to the following array of test devices:

• (480×320) iPhone 3GS running 4.3.3
• (1136×640) iPod 5gen running the latest 6.x
• (960×640) iPhone 4S running 6.0
• (1024×768) iPad 1st gen running 5.0
• (2048×1536) iPad 4th gen running the latest 6.x

Note that the iPad mini is the same resolution as the iPad 1st gen.

(My choices are skewed towards later iOS versions since I like to implement integration with all of Apple’s snazzy optional features as they roll them out. It would probably be a more balanced assortment if one of the 6.x devices were running 5.1 instead.)

If you don’t need to support 4.x, I would personally advise against it, since iTunes Connect no longer collects crash reports for it and the simulator no longer offers it. Of course, only you can decide whether you really need to or not, and if you do, focus a lot of your testing there as Xcode does not warn you if you are using APIs that were only introduced in 5.0, which will crash any device running 4.x.

Please note that there are ways to (with significant preparation) downgrade the version of iOS on a device, so if you really want to test more versions than you have devices for, you can (with a lot of effort). But you’re probably better off cultivating a strong pool of beta testers for this, anyway.