The iOS beta-testing process sucks. It sucks for developers and it sucks for testers. TestFlight is an attempt to fix that.
Here’s what usually happens when a developer wants to get a new version of their app into the hands of testers:
- The developer emails the .ipa app package and the .mobileprovision file to the tester;
- The tester drags these to iTunes;
- The tester syncs their device with iTunes.
In 1., the sucky part is that most developers have to manage the testers manually, adding them to and removing them from an Address Book group or some other mailing list. There are mailing-list apps which make this step bearable.
2. isn’t exactly backbreaking work, but you can only do it if you’re at your syncing computer. Many people - myself included - sync with their desktop Mac, not their laptop. If a new version of the app is sent out while you’re on the road, you can’t update.
3. is slow. Really slow. Depending on what you’ve done with your device (and your iTunes library) since the last sync, this could take fifteen minutes. Not a huge deal if you do it once, but if you do it frequently…
Which is an important point: the more hurdles in this process, the less likely the tester is to bother to update. I’m currently behind on two apps I’m testing; maybe I’m lazy and evil, but in that I’m probably not much different than other unpaid testers.
Back in June, I flew to WWDC just as The Incident (the app I was working on) was shaping up into a fully playable game. I was obviously excited to show it off, but the timing ended up being slightly wonky: there was a serious bug in the last version I had, and Matt was about to add significant new features. He did this a few days into the conference, and I was cursing my luck. I even pondered syncing with somebody else’s laptop just to get this important build.
My luck quickly changed: my friends at 23 Divide were just beginning to work on an app called TestFlight, which sought to make possible the impossible: you could update ad hoc app builds over the air, with one tap. Let me clarify this immediately: TestFlight does not require a jailbroken device, shady private-API voodoo, or even an installed app. Standing in a crowded bar in SoMa, I just tapped a button and saw the app begin to download and install. Holy crap!
Since then, Matt and I have been very happily using TestFlight to manage beta builds. It’s made it not only possible, but quick and easy to send super-small updates, even for debugging; this would have been wildly impractical before.
Sadly, another thing that’s happened since is that Apple has rejected the TestFlight app from the App Store. This app is not strictly required for the update system to work, but it makes it a bit sleeker - it can collect new users’ UDIDs and send out push notifications when updates are available. The rejection was vague and misleading. Officially, the app “contains insufficient functionality”; unofficially, Apple doesn’t want to see beta-assist tools. They’d rather write their own, which would be fair if Apple’s current process didn’t, well, suck.
But that’s not the end of TestFlight. They are now signing up for their beta which will most likely be web-based; the magic one-tap update is still there. Beyond that, the service should streamline what happens before and after the builds as well: signing up new testers, grouping them for A/B testing and different tiers of builds, managing tester feedback, and hopefully much more.
So far it’s been an invaluable tool. See for yourself!
