Question

I work on a vertical-market Mac application that uses a USB dongle to make sure users have paid for it. It’s expensive enough, and in enough demand, that “black hats” have tried to crack the dongle scheme, so the app checks the executable and key resource files at runtime, and if something has been tampered with, the program won’t run.

With PKI (Public Key Infrastructure) based code signing becoming more commonplace in the Mac world, I’m considering switching to using it to do this runtime verification, which would have the nice side effect of making Gatekeeper happy.

However, Apple’s interest is very different from mine. Their focus is on making the user happy, so if an app has an incorrect signature, Mac OS X will simply ask the user if they want to run it anyway. My focus is on thwarting crackers, so if my app has an incorrect signature, I simply don’t want it to do anything that’s useful to an end-user.

So I want my app to be able to validate its own executable and resources, using Apple’s signatures, at runtime.

Also, from what I’ve read of the libraries offered Mac OS X for doing validation, they simply give a “yes” or “no” answer to requests to validate an executable. It strikes me that this is susceptible to “black hat” attack in a number of ways – for instance, one could simply replace Apple’s tools with ones that always say, “yes, this is valid”, either in the system directories or by changing the search path for those tools. So I think it might be a good idea to build the complete set of signature validation code into my app.

So I have a few questions:

1. What PKI libraries/APIs are available to let an app validate its own executable and resources which have been signed using Apple’s codesign system?
2. Do these libraries have source code available?
3. Are my security concerns about using the PKI libraries Apple ships with Mac OS X valid, or are they safer to use than I think?
4. Does anybody have experience with this kind of thing they’d be willing to share? Are there other gotchas or tips?

Thanks in advance for any help with this.

Answer 1

To answer #1 and #2 above, Apple has open-sourced its code for creating and verifying digital signatures as libsecurity_codesigning. So a developer could build that into their app to let it validate its own signature.

Alternatively, MacOS’s version of this library can be used by an app – but the API is private, so that’s a gotcha. There’s a brief discussion at this link.