do not step out
12/Mar/2008 | 03:52

Encouraging Failure: A Brief Tutorial

Dave Jewell has written an article over at RegDeveloper regarding Mac OS X preferences windows. He titled it "A Better Way to Build OS X Preferences". I hoped that with such a title, the article was going to be insightful and informative. Instead, I found ignorance and arrogance posing as a qualified advice.

Or as I said to a friend, I'm calling this guy the fuck out.

#import <GenesisKit/GenesisKit.h>

In the beginning, there were some developers working day and night to ship an operating system. This operating system included some modules—frameworks, as it were—that were created to provide common, useful functionality. Along the way, the frameworks changed frequently in both how they worked and how they were supposed to be used. There came a point in the development of the frameworks where the engineers plotted out what parts they could let everyone use, including third-party developers, and which parts they'd have to keep to themselves.

The problem, they realized, was three-fold:

1. If Feature X of a framework was still changing internally, they couldn't give it out. They knew that if Feature X's implementation was in too volatile of a state, they may introduce bugs for anyone using it.

2. Volatility exists outside of the implementation of Feature X. Once they released a public API for using Feature X, they couldn't just change it up later or yank it completely without causing problems for everyone involved. Especially third-party developers, who'd be mighty pissed, and rightly so. Once they put forth a public API, they knew they'd have to support it. In that respect, it needed to be well-designed and well-tested.

3. Once these public APIs were settled upon, they would need to be well-documented. This is no easy task, and it's extremely important. During documentation of public APIs, it is often discovered that a piece of the public API didn't work well, or something important was missing. Now they'd have to start part of the cycle over again. And then come back to finish, review, edit, review, and publish the documentation.

This is a pretty big order.

You're wondering what any of this process has to do with the article. It all ties together, baby. Don't move.


Since we have a good picture of what went on behind the scenes, let's take a look at who's going to be using these frameworks. At the most basic level, there are four kinds of mindsets when it comes to developers and private APIs:

A. "I will not use any private APIs, period. They're completely unsupported, undocumented, and I can't rely on them. I need my code to be reliable and future-proof, and if I can't find a way to do something with a public API, I won't do that something."

B. "I really don't like using private APIs. They're unreliable, undocumented, and I can't rely on them. I prefer my code to be reliable and future-proof, so I try to avoid them. Once in a while, I find something that I need to do that can't be done with a public API, so I weigh the importance of what I need to do against the dangers of using things the developers didn't want me using. If I don't really need to, I won't; but I won't rule it out."

C. "I prefer to ship code written with public APIs, but I love exploring private stuff. It's a hell of a lot of fun to dig around inside the wiring and put together neat shit I wouldn't normally be able to implement. Sometimes the product of this ends up in an app I release, but I try to keep it from happening too much because I can't predict when something's going to change."

D. "Private APIs are just features like anything else, but secret. There's nothing wrong with using these even though they're undocumented and we have to figure out how they work. The API developers wanted to keep all the toys to themselves, but I'll do whatever I think is best. It's not really going to cause a problem."

Take a strong guess as to which kind of developer the author is. To demonstrate how I arrived at this conclusion, let's do some surgery.

Coming Together to Pull Things Apart

We're going to take what we've learned so far and apply it to the RegDeveloper article. Deep breath.

Here, on a regular basis, Dave will introduce you to unknown and undocumented aspects of the Foundation and AppKit class libraries that Apple has, er, neglected to tell you about...

Think back to the section that described the development of the operating system and its public and private APIs. Now reread this quote and tell me if you think Apple "neglected to tell you about" something that they don't think is ready for public, widespread use. Sounds much less secretive now, doesn't it?

The author is just setting up a justification for bad advice, it seems.

In my opinion, there's no sense in re-inventing the wheel if Apple has already done the job for you.

This is the second in Dave Jewell's Series of Bad Assumptions. Apple hasn't done the job for you already; they've done the job in a way that isn't ready for other people to use. They have deemed it not to contain unshippable issues, but it has either not been finalized or documented well enough to be used in real, shipping third-party applications. Or both. There's also a really strong chance they know about quirks that they haven't worked out, which makes the API unsuitable for mass adoption.

Over the coming months you'll find there are an amazing number of undocumented wheels within the Apple libraries that let you get on with writing great applications without having to reinvent what Apple have already done.

Undocumented "wheels"—which is a misleading way of saying "shit that wasn't designed for you to use"—usually have nothing to do with writing great applications. The two are mutually exclusive concepts when the mass of public APIs are as rich and powerful as they are. We're not talking about things that are essential to creating good software being hidden from developers.

Alternative thought: Fuck, he's going to keep recommending people use private APIs in shipping applications. And green developers are going to listen to him.

They are just waiting for you to set 'em spinning!

No, they're really not. That's the point of being private.

At this point, a few folks will doubtless be sharpening their quill pens (or even wooden stakes!) and preparing to chastise me for using undocumented features.

Using a private method once in a while is common practice in software development, though usually not recommended; commandeering an entire private CLASS is almost always frowned upon. And when what you want to do is so trivial as what we're about to encounter, it's silly to go out of your way to dig into private APIs to accomplish your task unless you're just tinkering for the sake of curiosity.

I'm not chastising anyone for using private shit, but I am taking the author to task for encouraging people to ship applications to people that needlessly make use of PRIVATE FUCKING UNDOCUMENTED FUCKING UNRELIABLE FUCKING STUFF.

That's true, but I'd offer a couple of counter arguments in return.

I braced for the bullshit excuses and pressed on . . .

First, Leopard (OS X 10.5) has only recently been released, and is likely to be good for several years - it's very unlikely that minor system updates would impact what I'm describing here.

Is Dave Jewell privvy to the past, present, and future engineering plans of the AppKit team? No? Then perhaps he should pretending he's some kind of authority on what private APIs will not undergo unannounced changes. They're private, they're 100% subject to change, and private changes don't get announced.

And if they do, I'll obviously make every effort to bring you a work around.

Excellent idea. Should I set up a POP forwarder to your e-mail address so you can also handle bug reports and incident support from people using my application when one of the private APIs changes in an update? And will you help implement and support the "workaround"? How soon can I expect a workaround?

I mean, if you're gonna push someone down this path, you'd better not ditch them in the woods when they bump into a tree. It's almost like . . . having to support an API you encourage people to use. Sounds familiar, yeah?

Next, and more importantly, you can spend more time making your great Mac application even greater - and surely that's what we'd all like to see, including Apple. Right, Apple?

Apple wants to see developers make great applications. They don't want developers making applications that break because their engineers need to change something internally and no one was supposed to have been relying on what got changed in the first place.

This is also how internal API engineering gets jammed up sometimes. When it becomes apparent that third-party developers have come to rely on some undocumented private API, it makes decisions based around that API more difficult. In a perfect world, whatever people were using would've been public anyway. But things are rarely that simple in the real world.

You'll also note that evil-looking call to _nsBeginNSPSupport. This is a fairly recent addition by Apple that is designed, I suspect, to prevent third party use of NSPreferences. Without a preceding call to this extern void routine, the call to super init will return nil.

1. Developers who have used this private API in the past have been bitten by a change to the API. I'm shocked.

2. It's extremely easy to find function calls buried in libraries. Apple knows people will find things that they go searching for. As a result, the addition of a private function like _nsBeginNSPSupport() does not mean that Apple's trying to prevent third-party use. It's too easy to find, and people will find it, as the article shows. What the article doesn't show, however, is what evidence is behind this "suspicion" other than the same shortsightedness from the rest of the article.

3. A corollary of #2: "Evil-looking"? Are you serious? Painting things in a malicious light does not make your article any better.

Remember, an API often remains private because it's volatile or hasn't been documented to give developers the information they need to use it properly or debug problems encountered while using it.

This Could Have All Been Avoided

I sludged through the article and realized something:

This guy wrote two pages about how a developer could use a completely private system to do something as trivial as making a preferences window for his or her application. That's pretty weak. Not only is it weak, but it's very lazy thinking.

It's just a preferences window. With the exception of the individual preference bindings, it's an extremely generic thing. A better solution would be to write solid, supported, portable code using documented APIs, with the intent of reusing the generic preferences window base you built.

This is exactly what Apple did with NSPreferences and NSPreferencesModule. They have their own simple interface for creating and managing preferences windows. It's not that hard to use public APIs to build your own generic preferences classes like these. Once you've built it, you can reuse it. Maybe Apple will finalize a public version of what they're doing for themselves, and that'd be great, but they aren't exactly withholding the secrets of the atom here.

The article suggests that you should hack around with Apple's private, unsupported APIs instead of letting work get in the way of building your next great application. I posit that a better article would've taught you how to build your own reusable preferences window construction kit just like Apple.