In support of the Lodsys patent lawsuit defendants

If you’re the kind of person who reads this blog, then you probably already know from other sources that an organization called Lodsys is suing seven “indie” iOS (and beyond!) developers for patent infringement after it started threatening them (and a few others) to do so about three weeks ago.

Independently of the number of reactions this warrants, I want to show my support, and I want you to show your support, to the developers who have been thus targeted. Apparently, in the USA, even defending yourself to find out whether a claim is valid doesn’t just cost an arm and a leg, but can put such developers completely out of business with the sheer cost of the litigation. So it must be pretty depressing when you work your ass off to ship a product, a real product with everything it entails (engine programming, user interface programming, design, art assets, testing, bug fixing, support, etc.), only to receive demands for part of your revenue just because someone claims to have come up with a secondary part of your app first, this someone being potentially anyone with half of a quarter of a third of a case and richer than you, since you’d be out of business by the time the claim is found to be invalid. It must be doubly depressing when the infringement is from your use of a standard part of the platform, that you should (and in fact, in the case of iOS in-app purchase, have to) use as a good platform citizen.

I have known about iconfactory.com and enjoyed their work for fifteen1 twelve years now, I use Twitterrific, I have bought Craig Hockenberry’s iPhone dev book, I follow him on Twitter and met him once. I know that the Iconfactory is an upstanding citizen of the Mac and iOS ecosystem and doesn’t deserve this. I am not familiar with the other defendants, but I am sure they do not deserve to be thus targeted, either.

So, to Craig, Gedeon, Talos, Corey, Dave, David, Kate and all the other Iconfactory guys and gals; to the fine folks of Combay, Inc; to the no less fine folks of Illusion Labs AB; to Michael; to Richard; to the guys behind Quickoffice, Inc.; to the people of Wulven Games; I say this: keep faith, guys. Do not let this get you down, keep doing great work, know there are people who appreciate you for it and support you. I’m supporting you whatever you decide to do; if you decide to settle, that’s okay, maybe you don’t have a choice, you have my support; if you decide to fight, you have my support; if you want to set up a legal defense fun to be able to defend yourself, know that there are people who are ready to pitch in, I know I am.

And in the meantime before the patent system in the USA gets the overhaul it so richly deserves (I seriously wonder how any remotely innovative product2 can possibly come out of the little guys in the USA, given such incentives), maybe we can get the major technology companies to withdraw selling their products in that infamous East Texas district (as well as the other overly patent-friendly districts), such that this district becomes a technological blight where nothing more advanced than a corded phone is available. I don’t think it could or would prevent patent lawsuits over tech products from being filed there, but at least it would place the court there in a very uncomfortable position vis-à-vis of the district population.


  1. Let’s just say my memories of my time online in the nineties are a bit fuzzy; it’s a recent browse in digital archives that made me realize I in fact only discovered iconfactory.com in 1999

  2. The initial version of that post just read “innovation” instead of “remotely innovative product”; I felt I needed to clarify my meaning

April’s Fools 2011

So, if you read my previous post before today… April’s fools! And not in the way you might think. This behavior of the iPad 2 is real, I did not make it up, I did indeed verify it this week. The joke is that I claimed to be surprised, hoping to make people believe this unexpected behavior was an April’s fools. Posting strange-sounding yet true information on April the first—now that is the real prank.

It’s hard to tell how successful I was in tricking people into believing this was a joke; I did however get a few emails explaining (as I pretended to request) how such a thing was possible. Congratulations guys, you did not fall for it!

I completely expected this behavior of the iPad 2, I knew about ARM having a weakly ordered memory model, and have known for some time (this test code was prepared over the last few weeks, for instance). By pretending to be surprised, I attempted to raise awareness of this behavior, which many people are completely unaware of; indeed, programmers have rarely been exposed to weakly ordered memory systems so far: x86 is strongly ordered, and even if these programmers have worked on ARM they have only worked on single-core systems so far (the only consumer hardware I know of that exposed a weakly ordered memory model are the various bi-pro PowerPC PowerMacs, which are not very common and back then Mac code was mostly single-threaded). I’ve been thinking about ways to raise this awareness for some time, but it was hard to find out how since it was pretty much a theoretical concern as long as no mainstream multi-core ARM hardware was shipping. But now that the iPad 2, the Xoom, and other multi-core ARM tablets and handsets have shipped I can show everyone that this indeed occurs.

Later today or tomorrow, I will replace the contents of that post with a more in-depth description and a few references, in other words the post I intended to write in the first place, before I realized I could turn it into a small April’s fools prank. It will be at the same URL, in fact you might have noticed the slug did not really match the title, I intended this as a small hint that something was off…

Whether you thought the iPad 2 behavior was a joke, you knew this behavior was real but believed I was genuinely surprised, or you saw right through my feigned surprise, thank you for reading!

(On that note, I should mention I have been sloppy in checking my spam filters residue so far, and my ISP deletes them automatically after one week. So if you ever wrote me and I never answered, this may be why. My apologies if this happened to you, please send the email again if you feel like doing so.)

ARM multicore systems such as the iPad 2 feature a weakly ordered memory model

At the time of this writing, numerous multicore ARM devices are either shipping or set to ship; handsets, of course, but more interestingly this wave of tablets, in particular the iPad 2 (but not only it), seems to be generally based around multicore ARM chips, be it the Tegra 2 from nVidia, or the OMAP 4 from TI, etc. ARM multicore systems did exist before, as the ARM11 was MP-capable, but I’m not aware of it being used in many or any device open for third-party development; this seems to be really exploding now with the Cortex A9.

These devices will also introduce a surprising system behavior to many programmers for the first time, a behavior which if it isn’t understood will cause crashes, or worse.

Let me show what I’m talking about:

BOOL PostItem(FIFO* cont, uint32_t item) /* Bad code, do not use in production */
{ /* Bad code, do not use in production */
#error This is bad code, do not use!
    size_t newWriteIndex = (cont->writeIndex+1)%FIFO_CAPACITY; /* Bad code, do not use in production */
    /* see why at http://wanderingcoder.net/2011/04/01/arm-memory-ordering/ */
    if (newWriteIndex == cont->readIndex) /* Bad code, do not use in production */
        return NO; /* notice that we could still fit one more item,
                    but then readIndex would be equal to writeIndex
                    and it would be impossible to tell from an empty
                    FIFO. */
                    
    cont->buffer[cont->writeIndex] = item; /* Bad code, do not use in production */
    cont->writeIndex = newWriteIndex; /* Bad code, do not use in production */
    
    return YES; /* Bad code, do not use in production */
}

BOOL GetNewItem(FIFO* cont, uint32_t* pItem) /* Bad code, do not use in production */
{
#error This is bad code, do not use!
    if (cont->readIndex == cont->writeIndex) /* Bad code, do not use in production */
        return NO; /* nothing to get. */
        
    *pItem = cont->buffer[cont->readIndex]; /* Bad code, do not use in production */
    /* see why at http://wanderingcoder.net/2011/04/01/arm-memory-ordering/ */
    cont->readIndex = (cont->readIndex+1)%FIFO_CAPACITY; /* Bad code, do not use in production */
    
    return YES; /* Bad code, do not use in production */
}

(This code is taken from the full project, which you can download from Bitbucket in order to reproduce my results.)

This is a lockless FIFO; it looks innocent enough. I tested it in the following setup: a first thread posts consecutive integers slightly more slowly (so that the FIFO is often empty) than a second thread, which gets them and checks that it gets consecutive integers. When this setup was run on the iPad 2, in every run the second thread very quickly (after about 100,000 transfers) got an integer that wasn’t consecutive with the previous one received; instead, it was the expected value minus FIFO_CAPACITY, in other words a leftover value from the buffer.

What happens is that the system allows writes performed by one core (the one which runs the first thread) to be seen out of order from another core. So the second core, running the second thread, first sees that writeIndex was updated, goes on to read the buffer at offset readIndex, and only after that sees the write in the buffer to that location, so it read what was there before that write.

A processor architecture which, like ARM, allows this to happen is referred to as weakly ordered. This behavior might seem scandalous, but remember your code is run on two processing units which, while they share the same memory, are not tightly synchronized, so you cannot expect everything to behave exactly the same way as in the single core case, this is what allows two cores to be faster than one. Many processor architectures permit writes to be reordered (PowerPC for instance), among other things permitting this allows an important reduction in cache synchronization traffic. While it also allows more freedom when designing out of order execution in the processor core, it is not necessary: a system made of in-order processors may reorder writes because of the caches, and it is possible to design a system with out of order processors that does not reorder writes.

Speaking of which, on the other hand x86 guarantees that writes won’t be reordered, that architecture is referred to as strongly ordered. This is not to say it doesn’t do any reordering, for instance reads are allowed to happen ahead of writes that come “before” them; this breaks a few algorithms like Peterson’s algorithm. Since this architecture dominates the desktop, and common mobile systems have only featured a single core so far and thus don’t display memory ordering issues, programmers as a result have gotten used to a strongly ordered world and are generally unaware of these issues. But now that the iPad 2 and other mainstream multicore ARM devices are shipping, exposing for the first time a large number of programmers to a weakly ordered memory model, they can no longer afford to remain ignorant—and going from a strongly ordered memory model to a weakly ordered one breaks far more, and much more common, algorithms, like the double-checked lock and this naive FIFO, than going from single processor to a strongly ordered multiprocessor system ever did.

Note that this can in fact cause regressions on already shipping iOS App Store apps (it is unclear whether existing apps are confined to a single core for compatibility or not) since, while very few iOS apps do really take advantage of more than one core yet, some nevertheless will from time to time since they are threaded for other reasons (e.g. to have tasks run in real-time for games or audio/video playback). However, Apple certainly tested existing iOS App Store apps on the iPad 2 hardware and they would have noticed if it caused many issues, so this probably only affects a limited number of apps and/or it occurs rarely. Still, it is important to raise awareness of this behavior, as an unprecedented number of weakly ordered memory devices are going to be in the wild now, and programmers are expected to make use of these two cores.

What now?

So what if you have a memory ordering issue? Well, first you don’t necessarily know that it is one, just like for threading bugs; the only thing you know is that you have an intermittent issue, you won’t know it is memory ordering related until you find the root cause. And if you thought threading bugs were fun, wait until you investigate a memory ordering issue. Like threading issues, scenarios in which memory ordering issues manifest themselves occur rarely, which makes them just as hard (if not harder) to track down.

To add to the fun, the fact your code runs fine on a multicore x86 system (which practically all Intel Macs, and therefore practically all iOS development machines, are) does not prove at all that it will run correctly on a multicore ARM system, since x86, as we’ve seen, is strongly ordered. So these memory ordering issues will manifest themselves only on device, never on the Simulator. You have to debug on device.

Once you find a plausible culprit code, how do you fix it (since often the only way to show the root cause is where you suspect it is, is to fix the code anyway and see if the symptoms disappear)? I advise against memory barriers; at least with threading bugs, you can reason in terms of a sequence of events (instructions of one thread happening, one thread interrupting another, etc.); with memory ordering bugs there is no longer any such thing as a single sequence, each core has its own; as in Einstein’s relativity, simultaneity in different reference frames is now meaningless. This makes memory ordering issues extremely hard to reason about, and the last thing you want is to leave it incorrectly resolved: it’s neither done nor to be done.

Instead, what I do is lock the code with a mutex, as it should have been in the first place. On top of its traditional role, the mutex ensures that a thread that took it sees the writes made before it was previously released elsewhere, taking care of the problem. Your code won’t be called often enough for the mutex to have any performance impact (unless you’re one of the few to be working on the fundamental primitives of the operating system or of a game engine, in which case you don’t need my advice).

For new iOS code, especially for code meant to run on more than one core at the same time, I suggest using Grand Central Dispatch, and using it in place of any other explicit or implicit thread communication mechanism. Even if you don’t want to tie yourself too much to iOS, coding in this way will make the various tasks and their dynamic relationships clear, making any future port easier. If you’re writing code for another platform, try to use similar task management mechanisms, if they exist they’re very likely to be better than what you could come up with.

But the important thing is to be aware of this behavior, and spread the awareness in the organization. Once you’re aware of it, you’re much better equipped to deal with it. As we say in France, “Un homme averti en vaut deux” (a warned man is worth two).

Here are a few references, for further reading:

This post was initially published with entirely different contents as an April’s fools. In the interest of historical preservation, the original content has been moved here.

PSA: “previous” and “next” links in archives

(PSA, for the readers not familiar with this bit of US culture, stands for Public Service Announcement; these are similar to ads, except that instead of promoting a product, they serve to forward a message of public interest, like “Don’t do drugs”)

On the web, many archives can be browsed chronologically; I’m not just thinking of blogs here, many web pages can be thought of as being in an archive, webmail for instance. And more often than not, the links to do so are labelled “previous (item)” and “next (item)”. And here lies the rub: “previous” will typically take you to a newer item, and “next” to an older item. Wait a minute…

Oh, I certainly see the faulty logic that leads there. It started innocently enough, with “next”, denoting a link to the next page to go to see more items in the archive, but then “previous” came in for the link to go in the opposite direction; and now many sites are content with this.

But there are much better choices. For instance one is “earlier” and “later”; “earlier” has the advantage of being relatively positive (as compared to, say, “older”), which is good to encourage the reader, who has reached the bottom of the front page, to dig deeper in your archives.

Now why am I calling attention to this? Because I am guilty of this myself. Indeed, when I posted “Raising the Level of Discourse” my blog gained a second page. I immediately went to check it and saw that it indeed features “Previous Page”. The technical reason is that it seems to be the default for the theme I’m using, but that’s hardly an excuse; even if I can’t modify the theme myself (I’m on WordPress.com and have to use the available themes; I can customize one or two things and add CSS, but that’s it), I have to own up to my choices: a good craftsman does not blame his tools, but if they are not up to the task, either fixes them himself, gets them fixed, or changes tools altogether. However, this takes time, which I haven’t taken for this yet, so in the meantime it is still “Previous Page”; I apologize for the inconvenience.

The issue was resolved when I switched to a different theme in the beginning of 2012; I’m keeping this post for historical interest. – May 22, 2012

So it’s true

So it’s true. Along with announcing support for subscriptions, Apple has confirmed the policy changes that many suspected were behind the rejection of the Sony Reader app: that apps can no longer link to a place to buy content for the app (there can still be such a place, just it must not be linked from the app), and must instead offer an at least as advantageous in-app purchase to do so (Apple first released a press release announcing support for subscriptions and this new policy that would apply to them, then confirmed this new policy would also apply to all paid content used by apps, not just subscriptions).

In the way it’s presented in the quote attributed to Steve Jobs in the press release, it sounds like a wager between two gentlemen: a friendly, interested contest, here to see who can bring the most people in, everything else being otherwise equal. That the publisher earns less in one case, and yet maintains equivalent prices is not unheard of, either: for instance books sold directly often have the same price in Amazon or a bookstore (setting aside any shipping, of course), even though the retailer (and this includes Amazon) takes quite a bit of margin. In fact, in some cases like the tabletop gaming world, publishers have a store on the web but downright encourage customers to buy from their friendly local game store, because they know the benefits these places provide. So on the face of it, this looks reasonable.

Except for this: what value, exactly, does Apple bring to the table here? For in-app purchases that unlock features, there is a quite justified case to be made that, since Apple distributes the app with the feature potentially present, just not unlocked yet, the hosting, screening, and to an extent, DRM services provided by Apple apply as well to in-app purchases, justifying the same 30% cut. However, none of these apply for content in-app purchases: new content can be added all the time, without the need to send a new binary to customers (as an aside, I wonder when Apple added the ability to allow in-app purchases to be added without the need for a new binary), so this content is not coming out of Apple bandwidth, never goes through the approval process, and likely has its own DRM.

This leaves payment processing. Now don’t get me wrong, iTunes is quite awesome payment processing. Back when I was a teenager I really wanted to pay for shareware, but I had no way to do so, and my parents refused to do so for me, so I played pretend by printing the order form (this was even before we had Internet). But a teenager today can buy a prepaid iTunes card with his pocket money pretty much anywhere, or use iTunes credit gifted by his parents, and buy apps for his iPod Touch, the family iPad, or the family Mac. This is awesome. So I think Apple can justify having a payment processing fee slightly larger than, say, that of PayPal. But most definitely not 30%.

Oh, there is, of course, the immense privilege of your app being allowed to run on Apple devices, but it has been established again and again that it is a deadly sin to make it difficult for people to build on top of your platform, because a platform is only as good as what has been built on top of it. No successful platform has ever collected royalties from applications, except for game consoles, but you have to remember consoles are sold at a loss or zero margin (recouped on the games), which is not exactly the case for iDevices.

The end result is that, contrary to the cases I mentioned earlier where publishers would earn less, but still earn money, through another retailer, this leaves publishers with the prospect of selling to Apple device owners at a loss, due to the fee disproportionate with the value Apple brings, as it is certainly larger than the share of profits the publisher can afford to spare for just payment processing (given that they still need to do pretty much everything else). Even if the publisher has a way to sell directly to these same consumers, and even if he was confident most of the sales would occur directly, uncontrolled selling at a loss is still way too big a risk for any publisher to take. I don’t think Apple is seeking rent as much as trying to establish a balanced system, but even with the best intents they have set up a policy that will drive publishers away.

Even if you see no or little problem on Apple’s side with this new policy, consider the following: the end result of this, whatever the reasons or whichever way the faults lie, will be that the Apple “ecosystem” will have its own specific publishers (of books, movies, comics, etc…), either Apple itself or ones that are, in practice, dependent on Apple; publishers different from the ones used by the rest of the world. Is it really what you want? Is it really what Apple wants?

Back in the 80’s, Apple with the Mac had years of advance on everyone else, and made obscene profits exploiting their then current strengths while positioning themselves too narrowly, before this caught up to them in the end. While the mechanisms and kind of partners (application developers/content providers) this is happening with are different, I wonder if it’s not what is going to happen with iOS as well.

First Impressions of the Mac App Store

I try to be original in the subjects I tackle, but if you are a Mac user, there is no escaping the Mac App Store, which is probably the most important thing to happen to the Macintosh platform since Mac OS X, at least. It remains to be seen whether it will be in a good or a bad way, but for now, I’ve given it a test drive.

Trial Run

After uneventfully updating to 10.6.6 and launching the Mac App Store application, I decided to buy Delicious Library to catalog my growing collection of webcomic books (it’s not as big as the one Wil Shipley pimps in the Delicious Library 2 screenshots, but I’m getting there), and of course to get a feel of how the Mac App Store works for a paid application download, not just a free one. This was when I encountered the first issue:

Screen Capture of a Mac App Store dialog in French, with text being cut off in the middle

“effehargements”? I’m afraid I don’t know that word

Gee, are things starting well… I mean, did localizers get so little time to give feedback on the size of user interface elements that this couldn’t be fixed for release? Any other explanation I can think of would, in fact, be worse. I’m not going to focus on that too much since it’s likely to be fixed soon, but it’s a bad first impression to make.

After logging in with my Apple ID as instructed, I was unsurprisingly told I had new terms to accept. Less expected is the fact these terms are an extension of the iTunes Store terms and conditions; apparently the commercial relationship users of the Mac App Store have is an extension of the one most of us already have with iTunes, not an entirely new one or an extension of the Apple Online Store ones. The main reason, I guess, is that they can use the credit card already associated with your iTunes account, and any iTune Store credit you may have; plus, that way the Mac App Store benefits from the iTunes Store infrastructure (servers and stuff).

Of course, by the time I was done reading the terms, my session had expired; it’s as if they weren’t expecting you to read them… I’m noticing this everywhere, though, it’s not just Apple. So I logged in again, accepted the terms, and bought Delicious Library. As widely reported, the application then moved to the Dock with a nice, if slightly overdone, animation (sure, have an animation, but they may have used a simpler one), where it showed a progress bar while it downloaded, up until the download was complete, at which point it jumped once, and stayed in the Dock (while, behind the scenes, it had been put in the Applications folder). This may seem gratuitous, but to me this is indispensable for the buying/downloading experience, as opposed to the disconnected experience of downloading software on the Web.

I then tried out Delicious Library, entering a few books, etc. (unfortunately, I do not have a webcam attached to my Mac pro, so I had to enter the ISBNs by hand). I’m not going to get into a review of Delicious Library here, I just checked that the application was working correctly.

Then, I checked something I had been wondering about. Even though the Mac App Store will only work on Mac OS X 10.6.6 onwards, this is not necessarily the baseline for the apps bought on the Mac App Store themselves: apparently, they can support earlier releases of Mac OS X, including Leopard. Obviously, they cannot be bought from there, they have to be transferred from a Snow Leopard Mac where you bought them. But I was wondering how the computer authorization process (documented in various articles, like Macworld’s hands on, read just above “Work in progress”) would work on a Leopard machine where the Mac App Store cannot be installed.

So I took the Delicious Library application, and moved it to my original MacBook, which remains on 10.5 for a variety of reasons (I don’t have another Mac with Snow Leopard on hand, to test on pre-Mac App Store 10.6.5, unfortunately). When I connected my MacBook to the network (for the first time in the year), there was no update, which would have been necessary to add such support. And when I tried to run Delicious Library, this is what I got:

Delicious Library crash report, listing an “unknown required load command 0x80000022”

Uh oh, Wil

This error is, in fact, not related to the Mac App Store at all, it seems instead that the application relies on some other Snow Leopard-only feature, probably by mistake. Apparently, this build was never tested on Leopard. I double-checked, and the application does declare it can run on Leopard in the Mac App Store application, as well as in its property list (from which the Mac App Store information was probably generated). So, I went looking for a free app that would run on Leopard; Evernote fit the bill, so I downloaded it and transferred it. It ran without problem, however being a free app, it did not need to validate its license on the MacBook or anything of the sort, I would have to test with a paying app. Osmos declares it runs on Leopard (as early as Tiger, in fact, though it’s Intel-only, so not on a PowerPC machine), so I bought it (the things I’ll do for you people) and transferred it. But it didn’t run any better than Delicious Library, though for a different reason (it required a version of libcurl more recent than the one found in Leopard). So, it’s another app that hasn’t actually been tested on the baseline Mac OS X version it claims it supports, great. I stopped the expense there.

Note that a large majority of paid apps actually require Snow Leopard, if their Mac App Store listings are to be believed. I’d wager that none of the paid apps that declare otherwise were actually tested on Leopard, and that all paid apps actually require Snow Leopard and probably 10.6.6 to run correctly; anyone care to confirm otherwise? I have no intent on spending a bunch of money to test that theory.

General Criticism

Besides the events of this run, I want to make more general observations on the Mac App Store. Contrary to the music, movie, book, comic book, etc. industries, where digital distribution is a relatively new phenomenon, people have been selling computer software over the network (not even necessarily the Internet back in those days, think Compuserve, AoL, the numerous BBS…) – and making a living out of it – since the beginning of the nineties, if not earlier. And yet, even after 20 years, for the majority of Mac users the act of buying software still means the brick and mortar store, or at best, a mail-order store like Amazon. There is no household-name software that’s distributed mostly digitally, except for some open-source applications like VLC or Firefox, expander/viewer/reader companion apps, and rare successes like… uhh… I’m sure I’ll think of one eventually.

Welp, while my questioning of whether such software existed was rhetorical at the time, it turns out there is a piece of software that in fact qualifies: Skype; their unusual business model is irrelevant: indisputably, it is commercial software mostly distributed digitally. This goes to show that when you solve a very practical problem with killer tech, you can overcome the barrier between digital distribution and the mainstream Mac market; that being said, it remains a hard problem for anyone else. – May 22, 2012

I’ve said the Mac App Store is probably the most important thing to happen to the Macintosh platform since Mac OS X, and that’s because it promises to provide at last a way to distribute software outside of the brick and mortar stores, that the rest of us will actually use; this, in turn, will allow developers who do not have the means to distribute their products in stores to access the majority of Mac users; of course, virtual or physical, shelf space and attention remain limited, but now we can avoid a hugely inefficient step in the middle.

Since the Mac App Store will set the expectations of Mac users for years to come, how it works, what it allows users, the kind of software found on it, etc., are extremely important, not just for Apple in the short run, but for the health of the platform in many years down the line. To me, the Mac App Store delivers in the main area it was supposed to: provide a great, integrated, end-to-end buying/downloading experience. However, it falls short to a smaller or greater extent in all other areas.

Let’s begin by the design. It’s a straight port of the “App Store” iPad app. Really, couldn’t they have done better? Surely, they could have made better use of the space afforded by the desktop, instead of using the strict iPhone/iPad tabbed design. Why have one entire tab to the updates, couldn’t this be put in a notification area in all modes? And breadcrumbs? Are they forbidden now? But the worst is surely that weird window title bar, with no title, and the stoplight window controls in the center left of the bar; I mean, is space at such a premium that they couldn’t have gone with a traditional unified title and toolbar design? It would have worked very well with the Panic toolbar design! To add insult to injury, these “toolbar tabs that go to the top of the title bar” are actually click-through! Argh! Now not even the top center of a window is safe for clicking (the worst thing is, I was already instinctively avoiding them when clicking to bring the Mac App Store window to the foreground, showing how thoroughly pervasive click-through has already damaged my computer habits).

As I’ve said, the Mac App Store provides a good, more importantly connected experience, from the buying intent to the moment the app is ready to use in the Dock. I’ve heard some complain about this automatic Dock placement, but to me this is not a problem, or to be more accurate this is not the problem with it. If you think about it, this policy actually is the most efficient: as it stands if, in the minority of cases, you don’t want to regularly use the application you just bought, you can just drag it out of the Dock; otherwise, you do nothing. The alternatives are not putting it automatically, in which cases in the majority of cases you are going to fetch the newly bought app from the Applications folder to put it in the Dock (which is more work than dragging an application out of the Dock), or asking you each time, in which case every time you have to read a dialog, choose on the spot, and click; and let’s not mention having a preference. The same goes for automatic placement in the Applications folder. Yes, I know browsers have these kind of options (Downloads folder/Desktop/ask each time), but that’s mostly because of historical reasons, your not necessarily expecting to be downloading something, and the wide variety of things you could be downloading from a browser.

But that’s from the perspective of an experienced user. For user actions, and doubly so when actions are made to happen “automatically” like that, the way to undo the action should be obvious from the way it was done (or shown, animations are very useful for that); I don’t mean the way to reverse immediately if the action was a mistake (the undo command is here for that), but the way the action can be reversed later if so desired. Here the way to “undo” the Dock placement remains reasonably obvious (drag it out of there), but users are going to think it gets rid of the application. Besides the fact this not the case, users will be reluctant to move applications out of the Dock for fear of not being able to find them again, and will keep them all in there. Yeah, I know, Mac OS X Lion and the Launchpad are supposed to solve that, but they’re not there yet, and in the meantime, the Mac App Store is here and users will use it. People do not get confused by however complex the system is underneath (do many even suspect that applications are in fact folders containing the binary and support files? No.), but by “innovations” that purportedly simplify some aspect of the task while leaving some or most of the complexity still visible elsewhere.

Besides the way the Mac App Store application currently works, there are issues with what the Mac App Store enables, or to be more accurate, does not enable.

For long, developers have asked for a way to update their applications as part of the (Apple Menu)→Software Update command, or to get access to the crash reports from their applications that were sent to Apple (though the issue was more that the “send the crash report to Apple” feature gave the expectation that the developer could do something about it or was notified of the issue). But I’ve always felt that this could not be done without Apple and the developer being in a tighter relationship, because of the potential spoofing and security issues that could occur; and now the Mac App Store is that relationship. However, there are Mac App Store improvements that could be given to non-Mac App Store applications (and it’s in the long-term best interest of the Mac platform that this option remains viable); for instance, it’s been a long time since distribution through a disk image was cutting edge, and Installer packages are too interaction-heavy. It’s not possible to have one-click download and installation from the web for obvious security reasons, but couldn’t Apple make available an application packaging method that can be downloaded, then, when double-clicked, would ask something close to the quarantine question, possibly show an EULA (as disk images can do), then install the app in the Application folder, show where it is, and discard the package, without any further interaction? I’m sure plenty other improvements of the sort could be made.

I also take issue with many Apple policies with the Mac App Store. Many of them are the exact same complaints developers have had about the iOS App Store (with the exception, of course, of the inability to distribute applications outside of it; developers can distribute betas and other special versions of the application however they want); while they may seem developer complaints that users shouldn’t care about, most of them disrupt the relationship between users and developers, resulting in a lose-lose situation. These issues are, among others: not inclusive enough (for instance, applications cannot install kernel extensions; why have that facility then?), no customer information whatsoever, user “review” system that gives the expectation developers can give tech support on them even though they can’t, still no “unfiltered Internet access” rating, no support for upgrade pricing, and most of all, no real support for demos/try before you buy.

That last one is the most infuriating. Apple is showing all Mac users a more practical alternative to shrink-wrapped software stores, and the policy is still that you have to buy with your eyes closed, only on the basis of a description, a few screenshots, and “reviews” that… could be better? Frak! And don’t tell me this demo business is confusing, people get to try before they buy in real life all the time: with TVs, consoles, audio systems, etc. in the electronics store; with clothes, shoes, etc.; with cars in their auto dealer, etc, etc, etc. Do I need to go on? I’ve always been suspicious of the “experience optimized for impulse buying” argument for the absence of real demos on the iOS App Store (it seems to me there are already plenty of apps at impulse buy prices, so it would be a good idea to encourage non-impulse buy prices), but here on the Mac App Store it makes no sense at all. Oh, sure, developers can distribute a demo from their web site, but it feels about as disconnected as a broken wire. This, alone, will ensure that I will rarely, if ever, buy again from the Mac App Store; not because I’m going to go out of my way to avoid using it, but because I’ll always be afraid of wasting my money on something useless, as I never buy on impulse. Practically all the downloaded software I own, I bought it after trying it, and I’m not going to start changing that now; that would be going… backwards, back at the time of brick and mortar stores, precisely those the Mac App Store is supposed to obsolete.

By the way, in case you have an iOS device and want to encourage “try before you buy”, there is a simple way: go to the “try before you buy” iTunes badge, to mark this as an iTunes store link featured group, download 10-20 of them that seem interesting to you, and try them out. That’s it, that’s all I’m asking of you: there is bound to be a few that you will like and where you will buy the complete version; this, in turn, will send Apple the message that yes, we do want to try apps before we buy them.

I’m deeply torn about the Mac App Store; not just how it currently is, but the whole principle of it. As it currently is, it works and will be used without a doubt, while having a number of issues and setting a number of bad expectations. While I have no doubt many issues will be fixed, Apple has been pretty stubborn about some of them (I mean, for how long have we been asking for a trial system on the iOS App Store?). And there needs to be life outside the Mac App Store, but Apple seems utterly uninterested in improving anything there.

Kinect User Interaction Design

I don’t own a Kinect or an Xbox 360 (and I don’t intend to buy either), but I’ve recently read very interesting stuff about the user interaction design issues it has raised, even for “just” game menus.

First thing, Penny Arcade’s Tycho (as it happens) reports his initial impressions (fourth paragraph), which are that it’s all over the place. No two games behave the same. (as an aside, it’s not every day you see Penny Arcade writing about usability.)

Second, Ars Technica has a very interesting article about how Harmonix apparently got it right with their game Dance Central. How they did so will not be a surprise to any of us Mac/iPhone nerds: they prototyped, and iterated, and  prototyped, and iterated, and  prototyped, and iterated, and…

While every console or computer game, or at least every game genre, has for obvious reasons its own user interaction rules when playing the game itself, on the other hand you generally expect the rules for menus to be consistent from game to game, including across platforms; game menus even share some conventions with desktop software. But the Kinect is a fundamentally new user input mechanism, for which there is practically no reference (when your best reference is Minority Report, you know you have a lot of work ahead of you). It’s not every day, or even every year, but on average around every decade that you see such a fundamentally new user input mechanism for electronic devices come up; for all time, I count only seven: “buttons, sliders and dials”, keyboard, mouse, gamepad, touchscreen (which only started realizing its potential with multitouch), Wiimote, and Kinect. I’m not counting steering wheels, joysticks and light guns, as these are used on electronic devices only to provide a simulation of the “real” ones, neither do I count remotes, which are just  “buttons, sliders and dials” that operate remotely.

So with such a new, unexplored user interaction continent, you’ve got to wonder why Microsoft didn’t do the job they should have done as the platform owner, by which I mean doing the research Harmonix had to do themselves, sharing the results with the developers of Kinect games, and applying the lessons to their own titles to set the example. If there’s something they should have taken from Apple’s and Nintendo’s playbook, it’s this. Maybe I shouldn’t be surprised given how much Microsoft application themselves are the worst violators of what little user interface guidelines Windows has, but at the same time, isn’t the Xbox from a completely different part of Microsoft?

At any rate, it will be very interesting to see how this will develop in the future.

Raising the Level of Discourse

There has been a worrying trend of late regarding criticism of Apple, and how Apple responds to it.

You’re no doubt aware that Apple has tried to increase and improve its communication in recent years, especially in response to criticism; besides, of course, the open letters from Steve Jobs, we have for instance the outreach of Phil Schiller to the community or the iPhone 4 antenna press conference. This is in itself a good thing; in the worst case at least you know their viewpoint, which furthers the debate, while in previous years you would know absolutely nothing.

The problem, however, is that Apple is often responding to the wrong criticism.

For instance, a lot of time in the iPhone 4 antenna press conference was spent repeating that all phones have their signal reception affected when held, that the iPhone 4 is no worse, that Apple does extensive testing, both in their labs and in the field, including in low coverage areas. This was to counter the allegations from some of the press that was screaming bloody murder at Apple over the “obviously defective” antenna, and accusing Apple of only having used the device in areas of good coverage, while that press had few, if any, hard facts to back these up. But anyone with a bit of sense already suspected or knew that the matter was a bit more subtle, and was following AnandTech’s excellent coverage and testing instead. This is the criticism Apple should have responded to instead; OK, sure, spend the first few minutes addressing the basics and the dumb criticism, but then move on to the specifics of the matter: namely, why they went ahead with such a new and innovative (and thus risky) external/structural antenna design when it could make the problem worse by allowing actual electrical contact (and maybe, additionally, why they didn’t mitigate the risk by adding a layer of insulating coating over the stainless steel). They could have answered in a number of ways. But they did not address that question. And none of the journalists who had the opportunity to ask that question did so.

The point is not to put Steve Jobs or Apple on trial, but to keep them honest. And yet, in this press conference everyone remained at a low level of discourse. I expect there will always be a significant proportion of the tech press that will happily remain at this level, unable to climb higher simply because they try to cover everything under the Sun without the means to cover this immensity with any real depth; I just wish this proportion didn’t seem to be the majority (so we should all encourage outlets that provide quality coverage!). However, I do not forgive Apple for not trying and raising to a higher level.

Something similar happened at the WWDC keynote. During the keynote Steve Jobs took some time to defend the iOS App Store review process (see “The Low Point of the Keynote”), which is okay in itself: the review system has attracted criticism, but there are points to be made in its favor. However, he tried to defend it by telling that 95% of rejections were for three reasons: the app crashes, it uses private API calls, or it doesn’t work as advertised. But it’s meaningless to lump all the rejected apps together; and it’s a fallacy to say that, since an overwhelming majority of the apps in this slightly artificial group fail to meet basic requirements (apps which were most likely coded by novice, careless, or dishonest developers) such that the other reasons are only a small fraction, then the other reasons are only a small problem. But that fallacy was clearly the implication Steve seemed to make; in the best case, that was the wrong criticism to address. Honestly, given the stuff that makes it to the iOS App Store, I’m surprised these reasons don’t account for 99% of rejections. Here again, Apple addressed a straw man; to be fair, some of the press puts a lot of straw men in front of Apple, but that’s not reason for Apple to take the bait.

So in the end, “interaction” between Apple and that press ends up looking like this:

Apple: Does not!
Press: Does too!
– Does not!
– Does too!
– Does not!
– Does too!

And even then, that’s still better than the common case where Apple doesn’t comment, leaving that kind of press feeling “slighted”, and that “the people” are not being “heard”:

Press: Does too! Does too! Does too! Does toooooooo!

This kind of stuff is all the more disheartening as Apple sometimes gets it right. For instance, in his Flash open letter, Steve Jobs not only addressed the obvious, but also addressed the less obvious criticism, by saying that even if they were to support it, current Flash content wouldn’t work well anyway given the lack of a pointer, and if it would require publishers to rewrite their content, why not rewrite it in HTML5? Not to mention Flash itself was designed with the PC in mind, not a touchscreen. And let’s not forget the original open letter, Thoughts on Music, which addressed a lot of points and issues that most people (and most of the press) had not even thought about.

Pretty often, criticism of Apple originates from a legitimate concern, but by the time it gets through the tech press hype machine, it has become an horribly distorted version of the original criticism; and I’m afraid people at Apple get the feeling they are taken to task for reasons that are entirely unjustified, based on this distorted criticism (while the original criticism is lost in the noise), leading them to believe nobody understands them.

Such misunderstandings are not uncommon, even without the distortion machine: for instance in the Ninjawords saga, Phil Schiller answered accusations that Apple required Ninjawords to be censored, by telling that this was something the developer did after an initial rejection, and Apple did not ask the developer to do so. Which was technically true, but as one of the developers pointed out, by having other, uncensored dictionaries in the iOS App Store at the 4+ rating, and at the same time requiring Ninjawords to have a 17+ rating, Apple was in effect putting a lot of pressure for Ninjawords to censor itself, as “who wants to be the only illicit dictionary on the [iOS] App Store?” So while Apple saw A: that they were enforcing the rating on Ninjawords (and while there may be a problem of ratings being inconsistently enforced, from their viewpoint this is none of the developer’s business anyway), the Ninjawords developers saw B: that they were, in effect, being made to censor the application.

A misunderstanding (or many) also probably explains Steve Jobs’ “Some people lie” comment at the D8 conference. Either he refers to cases that no one (or few people) was disputing in the first place, or he refers to developers who went to the press claiming an equivalent of B, while Apple viewed it as an equivalent of A.

Even if Apple feels it is wrongly criticized for, say, doing X, if only they would answer “Of course we do X, it seems obvious to us why, but here are three good reasons for us to do so”, then it would silence the press that made the dumb criticism, while allowing more serious outlets to follow up with “Yes, but then why do you do X even when Y?” Through communication the real issues that are lost in the noise can be revealed.

To improve this state of affairs, I only see one solution.

We need a Penny Arcade of Apple and the tech industry in general.

You have probably heard of Penny Arcade, and know it to be a funny, topical, popular webcomic skewering the video game industry, the video game press, and “gamers” themselves;  something you’d find in a newspaper covering video games if such a thing existed, playing an equivalent role as that of the editorial cartoons in the newspapers (except, you know, actually good and funny). What you may not know is the influence it has on the video game industry in general. In the foreword to their second collection, J. Allard (at the time in charge of the Xbox at Microsoft) describes Penny Arcade as accomplishing much more than the typical webcomic: for him, Tycho and Gabe do nothing less than keep the video game industry honest. Quoting Allard: “PA doesn’t buy it, and they don’t sell it. They tell it like it is. Whether it’s in their strips, their rants, commentary in their books, a direct flame-war, or a well-timed onomatopoeia in an elevator at E3, you can count on their presence if you’re doing something in this industry.” He goes on to mention their other endeavors, like the PAX expo and Child’s Play charity, as what they’ve been able to accomplish that the “industry” overlooked.

Now, J. Allard may have been trying to find the only good thing he could say from his viewpoint about the two guys who are basically lampooning his work and everything he holds dear, and may have come up with this. After all, you don’t (typically) say bad things about the authors of a book you’re being asked to write the foreword of. But there are hints and elements elsewhere (though they are not expressed as well as in that foreword) that confirm this influence to be the case.

It’s certainly counter-intuitive to propose caricature as being the solution to bad communication, a hyping press, and distorted criticism, these seem to be caricatural themselves already! And that’s precisely why caricaturists are needed to point this out. Remember, good satire makes you laugh, then makes you think. Moreover, good satire, having the appearance of just funny pictures/words, spreads subversively, including in the targeted institution (be it Microsoft, Apple, the tech press, etc.), delivering its message from the inside: while these institutions may dismiss criticism they see coming from the outside as attacks, good satire is read by the rank and file, spreading from there. Notice that I don’t think this satire needs to be a webcomic, it could take other forms, though I think a webcomic has the most inherent advantages.

Now before you go ahead and fire up your email client to tell me how about Z is totally the Penny Arcade of Apple and the tech industry and shame on me for not knowing about it, let me preempt most of you by saying that, yes, I know about the Joy of Tech, Mosspuppet, Fake Steve Jobs, PC Weenies, Daring Fireball, the Macalope, Crazy Apple Rumors, the Onion, and the Oatmeal, and I do not think this equivalent is to be found within them. And while Penny Arcade itself sometimes covers tech topics (especially, read this, starting from the third paragraph, and tell me how it couldn’t perfectly be read as though it was about the Mac App Store,  incidentally announced a few days beforehand), it does not do so with enough regularity to play more than a minor role in the tech industry; and I don’t think this will change much.

What should the  Penny Arcade of Apple and the tech industry be? It needs to be good, obviously, and it needs to satirize technology because its author(s) love technology. It needs to be topical. It needs to be regularly published to an extent, so that people keep coming back to it. As you may have gathered, it needs to cover not just Apple, but also its competitors: I rag on Apple, and that’s because I care, but its competitors have their own issues (similar or different), and it’s only fair that all of them be targeted; also, while there is a lot of material you could make about Apple, I don’t think it would be enough to keep, say, a three days a week webcomic running viably. As I said, besides the video game industry Penny Arcade also satirizes the corresponding press and users, so its equivalent would need to target the tech press and tech fans as well. It needs to be popular, but that will happen if it’s actually good. It needs to be subversive, being ostensibly funny while having actual substance.

With all the webcomics and blogs that get created (and often quickly die) all the time, why doesn’t such a thing already exist? I guess because it’s hard. It’s already a miracle that Penny Arcade exists, there are very few like it, and none as good. It’s hard in part because it needs to be topical, which precludes any comic buffer. It’s also hard to be good at doing such a thing. And it’s hard to love technology, while relentlessly skewering technology, the people responsible for this tech, the press talking about this tech, and your fellow tech fans.

In my opinion the closest thing we currently have to a Penny Arcade of Apple and the tech industry is Fake Steve Jobs. Dear Leader mercilessly takes on all the companies in this space, as well as the old dying press, these upstart bloggers, the fanboys, the frigtards, and the clueless. He’s funny, insightful, and oh so good. The main problem is that his subversive power is limited for two reasons: first, the caricature aspect is too salient, by very nature of the character; before you read the first word of a post, you know it’s going to be a satire of some sort. Second, it’s easier for the creator to make his work pretend to be harmless humor using pictures (especially for those looking over your shoulder) than it is with prose, again limiting the subversive aspect; this in turn makes it hard for the message to penetrate Apple and the others from the inside. Hence, Fake Steve is the closest we have, but he is not the Penny Arcade of Apple and the tech industry in my opinion.

Lastly, I want to give a shout out to the Satiritron, which could end up fitting the bill, from the fine folks behind Mosspuppet. It launched while I was writing this post, and it’s too early to tell how it’s going to do in the long haul, but it’s off to a good start so far, and I wish it best of luck and many faithful visitors!

Alleluia!

(Sudden breakout of Haendel’s Alleluia Chorus)
Alleluia! Alleluia! Alleluia, Alleluia, Alleluuuuia!

(Now, it doesn’t solve everything debatable about the iOS App Store, we’re mostly back to before the changes made in april: it’s a bit better than that as now sealed interpreted scripts are explicitly allowed, while back then they were in a limbo; also, sealed frameworks are no longer forbidden, which may be useful in rare cases. Many issues still remain, but, while I disagree with some things, I can live with them and it’s possible to have a healthy debate about them; the terms introduced in april, however, were completely foolish. The App Store Review Guidelines document is a welcome change, as well.)

It’s stunning what you learn about your iPhone while in holidays

I should go in holidays more often. You find out interesting things about your iPhone when you go outside the urban environment it was mostly designed for. I’m not ready to talk about the tests I mentioned earlier, but here’s a few other observations I can share…

First, the iPhone, or at the very least the iPhone 3GS which is the model I own, seems to have trouble getting a GPS fix in cloudy weather. If obtaining a location regardless of weather is of any real importance, a dedicated GPS device is still required.

Second, if you’re not going to have access to a power outlet for, say, one week, switch your device to airplane mode, it’s just as efficient and more practical than turning it off. Let me explain. I was for a week in remote parts of the Alps, trekking from mountain refuge to mountain refuge; even those that have some electricity from a generator are unlikely to have power outlets. Furthermore, in that environment you often don’t get any signal, or if you do it’s very weak, so a lot of battery is going to be wasted looking for a signal or maintaining a weak connection. So last year, I simply turned off my iPhone so as to conserve battery for the whole week. However, it was not very practical as I had to wait for it to boot each time I wanted to use it, which wasn’t very fast… So this year, I put it in airplane mode instead of turning it off. This turns off the radios and leaves it consuming apparently very little power; but it was ready much faster when I wanted to check whether there was any signal, or test my application, or show off something, and it still had juice at the end of the week.

Lastly, and this is more for developers: even if your app requires location, you may not always get true north if there is no data connection, so be sure to always handle the lack of true heading and fall back to the magnetic heading. As you know, a compass does not point exactly to the North Pole (in this context also referred to as geographic north), but to a point called the magnetic north pole, located somewhere in Greenland (in fact it is even more complicated than that, but let’s leave it at that). The iPhone 3GS and iPhone 4, both featuring a magnetometer, cannot give you true north using only that sensor any more than a compass can; however, if the device also has your location, it can apply the correction between magnetic north and true north directions at that point and give you the true north; this is well documented in the API documentation. However, while your location is necessary, it is not sufficient! I have observed that if I don’t have any data connection, I only get magnetic north, even if I just got a GPS fix; the device probably needs to query a remote database to get the magnetic north correction for a given point. So if your application uses heading in any way, never assume you can have true north, even if you know you have location; always handle the lack of true north and fall back to magnetic north in that case, mountain trekkers everywhere will thank you.