Apple to phase out usage of Imagination Technologies GPU in iOS devices

Big news dropped recently: via Daring Fireball, we learn that Apple notified Imagination Technologies that they would no longer be using their products in new iPhone, iPad or iPod Touch designs within a 15 to 24 months timeframe.

For some time already, the GPU has been the biggest driver and bottleneck of iOS performance, if not since the beginning, at least starting with the iPad and Retina devices, compounded when iPads became Retina themselves: iOS SoCs have been characterized for some time as being bandwidth monsters (relatively to mobile devices), most of it connected to the GPU so that it can feed the screen pixels. It is the GPU which is mostly responsible for scrolling smoothness, for the amount of layers you can have on screen before performance takes a dive, for the performance of games, etc. The improvement of CPU performance, comparatively, improves the iOS experience much less (in the browser, mostly). If you’ve been curious enough to look at CPU teardowns of iPhones, for instance here for the iPhone 7, you know the GPU can take as much space as the multiple CPU cores, and for iPads a truly outrageous amount of silicon surface is taken by the GPU alone. And you are more than aware of Apple’s reliance on graphical effects (not just partial transparency, but also now translucency, blurs, etc.) in the iOS interface, all of which are generated by the GPU. So the GPU on iPhones and iPads has strategic importance.

If you need a refresher, Apple has been using PowerVR GPUs from Imagination ever since the original iPhone. More than that, though, it is the only outside technology (and a significant one, at that) that is and has always been an explicit dependency for iOS apps: readers of this blog don’t need to be reminded of Apple’s insistence to own every single aspect of the iOS platform (if you missed the previous episodes, most of it is in my iPhone shenanigans category) so as not to let anyone (Microsoft, Adobe, whoever) get leverage over them, but graphical technologies have been a notable exception, being more than mere software. For instance, while Apple uses OpenGL ES, and now Metal, to abstract away the GPU, a number of PowerVR-specific extensions have always been available and Apple encouraged their use. Even if Apple has recently tried to wean their developers away from these extensions, and stopped advertising to developers the GPUs as being PowerVR products (starting with the A7/iPhone 5S, if I recall correctly), iDevices are still using Imagination products, and PVRTC, as in PowerVR Texture Compression format, textures are still a common sight in the bundle of iOS games and other apps, for instance.

So the first challenge here is the dependencies on these extensions. I don’t see Apple getting developers to make such a transition so quickly, especially as the first devices without Imagination tech are going to be available 12 months before the deadline (the iOS product lines have become too complex to perform the hardware transition all at once), which would leave developers 3 to 12 months to transition… So most likely, Apple is going to have to support those, and this is going to expose them to intellectual properties issues (patents or otherwise). Besides the extensions developers explicitly use, there are all the performance aspects and tradeoffs specific to PowerVR that iOS games have unwittingly become dependent upon (e.g. whether to use complex geometry or compensate with shaders, how to best obtain some effects, etc.), which Apple would have to best reproduce, or at least not regress on, in a new GPU.

And even if they started from a blank slate when it came to third-party software, Apple has many technological challenges to overcome. Much like audio and video codecs, graphical processing technologies are patented to the hilt; but contrary to audio/video codecs, there is no FRAND licensing, no patent pool, or single licensing counter for GPU tech; instead, existing GPU companies live in an uneasy truce, given they are all exposed to each other’s patents. And mobile GPUs are a particular breed within this universe, with adapted techniques to live in such constraints, like Tile-Based Deferred Rendering (present in all PowerVR GPUs). Apple has managed to build its own CPU with great success, so I have little doubt that they will manage to develop their own GPU, especially given their expertise in SoC design as well. But I also see patent royalty payments in Apple’s future.

So what does this mean for iOS developers? For now, nothing. There is nothing to justify scrambling to remove any PowerVR dependency at this point, and it’s pointless to second-guess the performance characteristics of these future Apple GPUs. Best to wait for Apple to come forward. But there is some transition ahead, because at least some long-held assumptions about how iPhone graphics work are going to be challenged when the new Apple GPU will eventually appear. If anything, I’m surprised for such a glaring externality in the iOS platform to have managed to remain for so long, and it will be interesting to see how this will play out and how Apple will manage any necessary transition.

See also: Ryan Smith’s take at AnandTech, a reference.

APFS’s “Bag of Bytes” Filenames (Michael Tsai – Blog)

I have sooooooooo many questions. I mean, first I have the same ones as Michael, but on top of that:

  • “bag of bytes”, but I hope at least that the file name, even if not normalized, is guaranteed to be valid UTF-8, right? Right? Right?
  • In some circumstances, it is possible for the user to type the beginning of a file name to select or at least winnow the file selection; is there going to be guidance on how to perform this?
  • Sorting file names for display. Oh, the fun we shall have with sorting. Again, will guidance/a standard function be provided?
  • Normally this should result in less issues for software that wrote a file name with any valid UTF-8 string, then expects a file with that exact name to be in the directory listing, as it will be the case at least more often (I must admit I don’t fully understand the issue that led to the Apple response in the first place, though I understand even less the Apple response). However, when performing manipulations with NSString/NSURL/Swift String, do those preserve composition enough that developers can rely on them for that?

Now, granted, I know two people this will make happy (or, OK, less unhappy)…

EDIT: One additional data point about this, is that in a similar situation, even Apple doesn’t get it right (coincidentally, fixed in Safari 10.1 and iOS 10.3). Let me tell you, this issue was a bear to isolate.

I admit:

  • I have no idea where this was in Safari, though it is safe to say Apple has responsibility for that code,
  • Safari is already compensating for invalid data, the URL should be properly escaped in the first place, and
  • this is when using HTTP, not the filesystem.

Nevertheless, this shows Apple themselves sometimes get it wrong and normalize strings in a way that causes issues because the underlying namespace has a dumb byte string for key. So if they can get it wrong, then third-party developers will need all the help they can get to get it right.

EDIT: New info, in that there will be a case-insensitive variant for the Mac, which will also behave differently for normalization.

I think “normalization-preserving, but not normalization-sensitive” means that (like HFS+ on the Mac, unlike APFS on iOS) you cannot have multiple files whose names differ only in normalization. And you can look up a file using the “wrong” normalization and still find it. Additionally, beyond what HFS+ offers, if you create a file and then read the directory contents, you’ll see the filename listed using the same normalization that you used.

This is my interpretation as well.

Curtain update

I took advantage of the recent update to JPS to experiment a bit with Curtain. I significantly retooled it towards one goal: separate the generation of the deployment package from the deployment itself.

While the initial version of Curtain benefitted from many influences, one I completely forgot to take into account was Alex Papadimoulis’ teachings, more specifically those about release management and database changes. Especially the commandment that builds be immutable and to make sure that what gets deployed on production is the same thing that got deployed on the earlier environments.

When I recently re-read those two articles for inspiration at work, I thought: “Uh, oh.”

Indeed, with Curtain the deployment process is not only a function of the revision that we ultimately want there, but also of what was previously there, in order to support proper rollover of resources (itself necessary because of offline support). And as originally designed, Curtain would just adapt its deployment to what was previously there, which means that, if I wasn’t careful and did not double check that staging was properly rolled back to what is present in production (which let’s admit, we’ve all done at some point), then the Curtain deployment to staging would not be representative of the eventual deployment to production. Oops.

So Curtain has been updated to, rather than perform the deployment itself, instead generate a package containing the generated files; this package doubles as a Python script which, when invoked, will perform all the deployment steps to the target of choice. The script itself is dumb and takes no decisions, such that it can be invoked multiple times and perform always the same job, but it also checks prior to operating that the data previously present corresponds to the expectations it was generated with. That way, we can use the same script multiple times, once on staging and once on production, and be certain that the two deployments will be the same. And Alex will be happy.

One more thing. In my initial post, I also completely forgot to mention another influence: Deployinator. Many aspects of Curtain come from Deployinator: deployment as a single operation, deploying assets as a layer separate from code, and versioning these assets as part of the URL, etc. The lessons from Deployinator were so obvious to me that it did not even occur to me to mention where they came from. That omission has now been repaired.

Simple File Cache: improve the performance of FileReader in the browser

When was the last time you obtained a 10x (ten times, 1000%) performance gain with a single improvement?

Not recently, I bet. Most optimizations work incrementally, eking out 3% here, 2% there, and only achieve an observable effect by iterating many such optimization steps. Even algorithmic improvements, such as replacing an O(n²) algorithm by an O(n·log(n)) one, typically get you on the order of 3 or 4 times performance improvement, at least on the data sizes in typical use at the time the improvement is made. So let me tell you how I improved performance of JPS, my web app to apply IPS patches, tenfold.

Once upon a time…

Soon after the initial public version of JPS, I started working on support of another format that (among other processing) requires the CRC32 of the whole file to be obtained, which is best done in blocks of, say, 1024 bytes rather than reading from the file byte by byte. Given my prior experiences, I dreaded the performance penalty from having to (re-)visit every single byte of the file, but it turned out to perform surprisingly well. Why couldn’t I get the same performance when processing IPS files?

So as a proof of concept I started developing a layer that would read from the file in blocks of 4096 bytes, then serve read requests from the loaded data whenever possible, entirely in JavaScript. In other words, a cache. Writing a cache is something you always end up learning in any Computer Science curriculum, and you always wonder why, given that it seems so simple and obvious it need not be taught, and simultaneously is something the platform will provide anyway (especially as modern caches tend to be very complex beasts, what with replacement policies, cache invalidation, and so forth). And Mac OS X, on which I develop, aggressively caches filesystem reads at every level already. Writing my own cache for file reads seemed too obvious to be something worth doing.

Photo of the Mont Blanc, lighted with sunset light

From now on, my longer posts will have random photos from my various trips inserted to serve as breathers. This is the Mont Blanc, lighted by sunset light.

As a way to test this anyway, I wrote the dumbest file cache you could possibly imagine: there is only one cache bucket, it can only be loaded from whole block-aligned ranges in the file, with the result that a number of requests, e.g. those that cross block-aligned boundaries, or those that load from the remainder of the file that can’t form a whole block, have to sidestep the cache and be served from the file separately. Furthermore, JavaScript Blobs are supposed to be immutable, so I did not need to worry about invalidating my cache when the underlying storage changed. Even then, this was a not a trivial thing: the asynchronous nature of the browser file reading API meant the cache had to provide an asynchronous API itself and maintain a “todo list” of read operations being processed.

And now I turn on the cache, and measure the performance improvement… and files that used to take Chrome 50 seconds to process now take 5 seconds! (Chrome being my reference browser for development of JPS). And the 10x factor is consistent, applying over various source files, often turning the processing time into “too short to measure”, and over various platforms: the same files which took around 200 seconds on Chrome for Android now take 20 (and the behavior of desktop Chrome on Windows was the same as on Mac OS X). Similar improvements could be observed with desktop Firefox, with processing times going from 20 seconds to 2 seconds.

Wow.

I reported these findings on the Chromium discussion forums (Chrome being the worst offender), because surely that meant something was wrong with Chrome somewhere. However, not much came out of it, so I decided to productize the cache so as to deploy these performance improvements in production.

From proof of concept to production-worthy code

The proof of concept assumed that, for every read operation except from the first it could just append a new read request from the client to its todo list, and once control would bubble up back to the cache code, the request could be served there if it was in cache. That worked in most cases, at least enough to get performance measurements; but in some cases, a new request would be logged from code that was not called from a callback from our cache, so it would never bubble up back to our code and never be served, and the pump would drain.

Photo of a young Ibex

A young ibex.

Easy enough, I thought, I will get rid of the todo list and instead I will always defer processing by calling setTimeout(,O).

That worked.

But it was slow. Even slower than without the cache.

Turns out, the overhead of calling setTimeout(,O) and getting called back by it was killing this solution. What to do, what to do, what to do? Back to the drawing board, I came up with the solution: reinstate the todo list, and use it, but only if we can tell for sure that we are within code that is being called by cache code — which entails keeping track of that information. If we are not within code that is being called by cache code, only then use setTimeout(,O). That managed to both work in all cases and with good performance.

And then I also had to support aborting requests, adding a number of unit tests, fix a few bugs… and then it was done.

Photo of the Grandes Jorasses

The Grandes Jorasses.

What have we learned?

  • Don’t diss CS or the CS curriculum. You never know when what you learn there might turn out to be useful.
  • Sometimes the obvious solution is the right one.
  • The source of slowness isn’t reading files, per se, but rather the shocking overhead of calling a Web API and getting called back by it (whether it be FileReader or setTimeout(,O)), which by my estimates is around 2 ms for each such operation with Chrome on a modern desktop machine. This is crazy. Other browsers (with the exception of Internet Explorer/Edge, which I have not been able to test) fare better, but still have enough overhead that you have to wonder what is going on in there.

Get the code

I set up a specific project for the cache code: you can get the code on BitBucket, and I also published it on NPM as simple-file-cache. It is free to use and modify (under the terms of the BSD license). If you find it useful, I request that you consider donating to the ACLU and the UNHCR, however.


P.S.: While I’ve got your attention, I’m happy to report that JPS will soon support Safari, as this browser is finally about to get support for the download attribute and downloading blobs, normally as part of Safari 10.1, which is meant to arrive with Mac OS X 10.12.4. Being able to be used on a stock install of Mac OS X will be a huge milestone for JPS and the viability of web apps in general as a way to circumvent Developer ID and Gatekeeper.

In-app purchases are in need of reform

The common wisdom with Apple, especially when it comes to explaining the unusual and apparently limiting ways they introduce features, is that to better serve the user they introduce features that solve the user need in a specific way for each task, instead of providing a generic, unrestricted feature that may not provide an optimal user experience.

At least, that’s how I have seen it expressed, e.g. in one Jesper post:

I am more out of my depth here, but just applying the output to what we know of the process, I think the iOS group sees files as something you are under pressure to manage. In particular, it sees files for everything as a generic solution, and by applying Apple philosophy, it thinks that most of the problems that can be solved using files and applications are instead better solved in a task-specific way for each task.

(post which you may remember from our exchanges on the lack of a document filing system on iOS)

This applies very well to iOS multitasking, as well: instead of just allowing apps to run unconditionally in the background, Apple provided ways to fulfill (practically) each user need in a specific way, and grant background execution privileges commensurately with the need: frozen but no background execution in the general case, background execution for a limited time in the “complete a task” case to e.g. complete an upload, background execution only as long as audio is played for the “play audio while doing something else” case, etc. List which Apple has expanded a few years later with new specific privileges, which shows a willingness to revisit initial restrictions.

So I have to wonder why Apple is not applying this principle to in-app purchases. Currently, it is a generic feature that does not provide an optimal user experience for a variety of user needs:

  • digital content purchases (ebooks, comics, etc.)
  • apps that are downloaded for free with limited features for trial purposes, with a one-time fee to buy the app and get the full functionality (known to old-timers like me as the shareware model)
  • games with a base scenario supplemented by substantial expansions (think StarCraft/StarCraft Brood War)
  • games with more discrete, non-recurring downloadable content (extra weapons, extra maps, etc.)
  • apps with extra functionality obtainable through in-app purchase
  • coin-operated games or with consumables (ammunition, smurfberries, boosters, gems, etc…)

Yes, the purchase experience per se is optimized for each user need, by virtue of each app managing entirely that experience; where this is not is for the other places where in-app purchases have an impact, such as the top grossing list. In particular, information in the iOS App Store about presence of in-app purchases, and how many/how expensive they are, is a completely generic solution to many specific problems, and in way which is not very transparent, to say the least.

This results in warped incentives for app developers, which you probably know about already since Apple has gotten in hot water in the press for those, especially the matter with children buying smurfberries amounting hundreds of dollars or more (which they’ve been able to do while under the timer, initiated by the initial purchase, where the Apple ID password is not prompted for). Apple has fixed the most egregious issues, for instance by having separate timers for the initial iOS App Store download and for in-app purchases, but the fundamental incentive of appearing as an ordinary game, then tempting the user with “boosters” to get him out of a bind, or even possibly get him addicted to these boosters, remains1.

Apple has more recently improved the situation, by changing the language when obtaining free apps (which now reads “Get” rather than “Free”), including those with in-app purchases, and by featuring games that you Pay Once and Play, i.e. without in-app purchases, and while this is a step in the right direction, this is far from sufficient as this excludes games like Monument Valley that feature a single, consistent expansion, and everyone (Apple included, since they featured Monument Valley in the WWDC intro video) wants to encourage apps like Monument Valley.

What can be done?

So what can be done? I think the most important is not to prohibit anything outright, because there may always be a legitimate use for a particular in-app purchase pattern. For instance, long ago, way before there even was an iPhone, I remember reading an article bemoaning that arcade games (back, you know, when arcade games mattered) were ported to consoles without any adaptation; that is, when the arcade version would prompt for a quarter after a game over, the port would simply allow unlimited continues, which sometimes would make it absurdly easier. And the article imagined potential solutions, one of which was a system by which the player on his home console would actually pay 25¢ whenever he would continue that way, that would be wired somehow to the game publisher, which sounded completely outlandish at the time. Not so outlandish now, eh?

But whatever is allowed, what matters is that the user is properly informed when he installs the app.

So the solution I propose is to keep in-app purchases as the common infrastructure behind the scenes, but for the iOS App Store to present each app in a specific way for each use case:

  • First, of course, apps (free or paid) without any in-app purchase, featured as they are currently.

  • Then, apps that you can try before buying. Those would be listed among paid apps, with a price tag that is the unlock price, but with a mention that you can try them out for free; and those would have two buttons rather than “Get”: something like “Try for free” and “Buy outright”, so that you could save yourself the trouble of going through the in-app purchase process if you know the app already and know you need it.

  • Then we would have apps, typically games, with a discrete and limited number of “tiers”. They would be listed among paid apps with a price tag which is the first tier; and in the page for the app, the tiers would be shown in a clear way (instead of this meaningless ranking of in-app purchases), e.g. as a series of “expansion” elements which visually combine, with each the name and price, as in:

    /-------------------\--------------\
    | StarCraft          \ Brood War    \
    | $20                / +$10         /
    \-------------------/--------------/
    

    or even:

    /-------------------\------------------\-------------------------\------------\----------------------\
    | World of Warcraft  \ Burning Crusade  \ Wrath of the Lich King  \ Cataclysm  \ Hey why not Narnia?  \
    | $30                / +$15             / +$15                    / +$15       / +$15                 /
    \-------------------/------------------/-------------------------/------------/----------------------/
    

    (Maybe with a shape that less suggests an arrow, but you get the drift)

  • Then apps that have unlockable features in a more complicated structure, but with no “ammunition” in-app purchase (what Apple refers to as a “Consumable” in-app purchase). Those would just have their initial price, then a ranking of these in-app purchases in a way close with what is done currently, but a “maximum cost” which is the price of obtaining all of them would also be shown as an indication.

  • Then apps with content in-app purchases, such as Comixology before it removed them. For those, there would be no such “maximum cost”, because no one is going to buy the whole catalog.

  • And lastly, apps that do have “ammunition” in-app purchases. These would be listed with a special price tag mentioning no specific cost, and the page for the app would have the button say, not “Free”, not “Get”, but “Install coin-operated machine” or some such that makes it clear you would be inviting on your device a box that belongs to the app developer and has a slot that takes money and directly sends it there, because that is what these apps are. Such a decision wouldn’t be popular with many app developers, but Apple has shown itself willing to take decisions that don’t sit well with developers when they sincerely think they are acting for the benefit of the consumer, for instance when Apple still doesn’t allow paid upgrades.

  • And we would also have apps using recurring subscriptions, about which I don’t have much of an opinion so far.

Building on these distinctions, more changes would be possible, for instance there could be separate top grossing lists, one for each category, which would avoid legitimate hits of the first categories from being drowned by the eternally grossing coin-operated machines of the App Store.

There you have it. At any rate, even if there could be completely different ways to go about it, it is certainly an area of the iOS App Store that could use some improvement (Mr. Schiller, if you’re listening…), having barely changed for so long without any of Apple’s apparent philosophy of “Let’s replace this confusing, generic solution by a number of specific solutions designed for each task”.


  1. In fact, given the similarities with gambling, I can’t exclude for these boosters-laden games to be eventually regulated as such

Re: Nintendo and iOS games

While there would be plenty to say on the technical (say, heterogenous multiprocessing in the iPhone 7) or tech-related (say, Apple’s transition away from the 3.5 mm audio jack) announcements from the latest Apple announcement event, I want to focus today on one that has been less talked about (relatively). Which is the commitment from Nintendo on smartphone games, materialized, and how, by Shigeru Miyamoto’s appearance on the event to present a Super Mario game for the iPhone.

Miyamoto-san’s appearance is not that big a deal, per se; or at least that is what my head says (he has appeared as a guest in non-Nintendo productions before), because, look, for someone like me who grew up on Apple hardware and the NES, seeing Miyamoto-san and Apple together in some official fashion is like some sort of childhood dream come true. Nintendo being willing to show what is arguably their most iconic character starring a game on Apple hardware is, however, a big deal whether you look at it from the viewpoint of your adult self or your 10-year-old self.

One thing I was particularly interested in was the angle, namely, how they would justify it being on a handset by taking advantage of something they couldn’t do on their own hardware (a discussion you may remember my old post, from when everyone in the Apple community seemed to have an opinion on what Nintendo should be doing); turns out, it’s one-handed operation and quick start, quick stop interactions. I never saw it coming, but makes every bit of sense: I triple dog dare you to play anything on any Nintendo handheld one-handed (well, maybe WarioWare Twisted, which you may remember from my earlier post), and while the DS (and later devices) goes to sleep when closed and can be resumed quickly, most of the time this is not really conductive to such gameplay; which is fine: this is one of the reasons I still play on my DS during my commute, because I have an uninterrupted 30-minute stretch in it where I’d rather play something “meaty”. However, I indeed never take it out while waiting for the bus.

And on that matter, while the aim is not to make a score sheet of what I got right or wrong back then, I have to admit that I was wrong that Apple would not be going to bend the rules for Nintendo: Apple introduced an interesting feature on the iOS App Store specifically for Super Mario Run. Indeed, even though the game is not out yet there is already a page for it on the iOS App Store, where instead of the “get” button you have a “notify” button. I have no doubt this is going to be extended to other developers in the future, but for now it’s exclusive to Super Mario Run.

I’ll also note that this kind of smaller-scale project fits well with Miyamoto-san’s role at Nintendo, where a few years ago he changed position to focus on more experimental projects rather than head the blockbuster game releases.

While previously Nintendo’s commitment to smartphone games was questioned even with the DeNa partnership, then Pokémon Go (which many considered as not being “real” Nintendo games, an assessment I do not share, but what do I know?), now with Shigeru Miyamoto’s appearance and Super Mario Run there is no doubt on Nintendo’s commitment, it will be hard for them to turn back on that. And who knows, maybe at some point Nintendo will make that WarioWare for iOS based on Apple nostalgia games I expected back then

Do not retroactively change the pistol emoji

Apple: don’t retroactively change the pistol emoji. Just don’t. The costs far outweigh the benefits, and even if you’re successful, it will come back in another way, so we will be back to square one anyway (except we will not have recovered the costs).

When I first heard of the change, I was already skeptical, and after pondering it some more, I have reason to think the benefits are not worth the costs.

To begin with, by doing it this way Apple makes the change retroactive. Any piece of text (email, text message, blog post, article, photo caption, or of course tweet) with a pistol emoji has now had its meaning retroactively changed when viewed on the latest iOS 10 beta. This change does not just affect newly received messages: any time the pistol emoji was used in the last few years will be affected by this change.

Besides personal usage, this will represent an issue for researchers studying past texts. Tweets get archived, you know (even if these efforts still can’t be accessed). Will researchers who study these archives have to use special software to render the pistol emoji from texts pre-iOS 10 as a revolver, from texts from 2017 on as a water pistol, and from the intermediate period as something else to signal the ambiguity?

Even accounting just for immediate message interchange, the drawbacks of the semantic change during the transition period may kill the idea: Jeremy Burge mentioned one, but problems also exist the other way round: people sending the pistol emoji but the recipient interpreting it as merely being a water pistol and not taking them seriously.

It has been noticed that (at least up until recently) Microsoft has been using a futuristic/toy gun glyph to represent the pistol emoji without causing the same kind of reactions. This is worth noting as an interesting piece of context, however I don’t feel it constitutes precedent, as the Microsoft glyph still represents a lethal weapon, if a fantastic one, so I see it more as a stylistic variant (of which there are many of this caliber between emoji typefaces, be it with this glyph or others) of the same semantic base. And Microsoft has limited impact in this domain, anyway.

Besides, this sets a dangerous precedent, because if Apple can unilaterally force everyone to change the meaning in this way of one Unicode character, what’s to stop them from doing it again? Even with the best intentions of the world, to circumvent in this way the Unicode consortium is probably too much power given to one particular vendor, be it Apple or any other.

But then (in case that was not reason enough) another reason came to mind, and I started performing research, which very quickly bore fruit.

If you’ve ever read comics of the French-Belgian tradition (and even a few others), you are undoubtedly familiar with the graphical symbols used to represent swearing. And I have no doubt that they will all someday be able to be represented as part of text; most of them are in the emoji repertoire anyway, and it’s only a matter of when, not if, the few remaining ones will be standardized. And guess what I quickly found in the handful of such comics I have on hand?

Exerpt of a comic page, with in one panel a character using symbol swearing

(from a Les Tuniques Bleues book, “Mariage à Fort Bow”, page 24)

This is not rare; of course you’re not going to find any in, say, Astérix, but in anything thematically appropriate it’s going to be found. And you can’t retroactively change that. You just can’t.

Having the pistol emoji as such in Unicode for the purposes of symbol swearing will also be useful to “type” it so that it can be rendered as such by a lettering typeface. For instance, Blambot (if you’ve been reading a webcomic in the last few years, and it’s not hand lettered, then it’s most likely using a Blambot typeface) has a typeface for symbol swearing, called Potty Mouth BB, and yes, it does contain a pistol as part of its repertoire. Currently the font ”cheats” and uses ordinary letters to allow you to type up these symbols, much like the Symbol font of old, but at some point a Unicode update to support them will inevitably happen. And if by that point the original PISTOL emoji’s meaning will have been successfully watered down, the Unicode consortium will have no choice but to add a new REAL PISTOL or some such emoji to support the actual pistol in these typefaces. And the gun that you thought you had chased will have come back through the window.

So removing the gun emoji from the iOS keyboard would be fine. But don’t change it. Unless you want to go against every gun representation in the world, then good luck to you.

Looking back on WWDC 2016

Now that the most important Apple release of WWDC has been dealt with, we can cover everything else. I haven’t followed as closely as previous years (hence no keynote reactions on Twitter), but to me here is what stands out.

The Apple App Stores policy announcements

As seen at Daring Fireball for instance, Apple briefed the press on many current and coming improvements to the Apple App Stores (iOS, Mac, tvOS, watchOS). This actually happened ahead of WWDC, but is part of the package. There are a lot of good things, such as for instance the first acceptance that Apple isn’t entitled over the whole lifetime of an app to 30% of any purchase where the buying intent originated from the app with the 85/15 split instead of 70/30 for subscriptions after the first year. However, none of this solves the lack of free trials: if only subscription apps can have free trials, then thanks, but no thanks. I want to both try before I buy and avoid renting my software, and I don’t think subscriptions make sense for every app anyway, so improvements and clarifications (e.g. indication of whether the app is “pay once and play” or ”shareware” or ”coin-op machine”) to apps using non-recurring payment options would be welcome (more on that in a later post). Also, while those apply to the Mac App Store as well, this one will need more specific improvements to regain credibility. I don’t have much of an opinion on the new search ad system.

The new Apple File System (APFS for short)

Apple announced a new filesystem, and to say that it has, over the years, accumulated a lot of pent-up expectations to fulfill would be the understatement of the year. I can’t speak for everyone, but each year N after the loss of ZFS my reaction was “Well, they did not announce anything this year, it’s likely because they only started on year N-1 and can’t announce it yet because they can’t develop such a piece of software in a yearly release cycle, so there is not use complaining about it as it could be already started, and will show up for year N+1.” Repeat every year. So while I can scarcely believe the news that development of APFS only started in 2014, at the same time I’m not really surprised by it.

I haven’t been able to try it out, unfortunately, but from published information these are the highlights. This is as compared to ZFS because ZFS is the reference that the Mac community has studied extensively back when Apple was working on a ZFS port in the open.

What we’ll get from APFS that we hoped to have with ZFS:

  • A modern, copy-on-write filesystem. By itself, this doesn’t do much, but this is the indispensable basis for everything else:
  • Snapshots, or if you prefer, read-only clones of the filesystem as a whole. Probably the most important feature, by itself it alone would justify the investment of a new filesystem to replace HFS+.

    While the obvious use case is backups, particularly with Time Machine, it is not necessarily in the way you think. Currently, when Time Machine backs up a volume, it has to contend with it being in use, and potentially being modified, while it is being backed up; if it was required to freeze a volume while backing it up, you wouldn’t be able to use it during that time and, as a result, you would back up much less often and that would defeat most of the purpose of Time Machine. So Time Machine has no choice but to read a volume while it is being modified, and as a result may not capture a consistent view of the filesystem! Indeed, if two files are modified at the same time, but one was read by Time Machine before the modification and the other after, on the backup the saved filesystem will have one file without the modification and the other with, which has not been the state of the filesystem you intended to back up at any point in time. In fact, this may mean the data is lost if you have to reload from that backup in case neither half can work with the other as a result.

    Instead, with APFS the backup application will be able to create a snapshot, which is a constant time operation (i.e. does not depend on how much data the volume contains) and results in no additional space being taken, at least initially, then can copy from that snapshot, while the filesystem is in use and being modified, and be confident that it is capturing a consistent view of the filesystem, regardless of where the data is being saved (it could be to an HFS+ drive!). Once the copy is over, the snapshot can be harvested to make sure no additional space is used beyond that needed by the live data. Of course, this will also allow, by using multiple snapshots, to more efficiently determine what changed from last time, and with APFS on the backup drive as well the backup application will be able to save space on the backup drive, in particular not taking up space for redundancies the source APFS drive knows about already. But snapshots on the APFS source drive will mean that, after 10 years, Time Machine will finally be safe: this is a correctness improvement, not merely a performance (faster backups and/or taking less space) one.

  • Real protection in the face of crashes and power loss events. HFS+ had some of that with its journal, but it only protected metadata and came with a number of costs. APFS will make sure its writes and other filesystem updates are “crash-safe”.
  • I/O prioritization. A filesystem does not exist merely as a layout of the data on disk, but also as a kernel module that has in-memory state (mostly cache) that processes filesystem requests, and the two are generally tied. I/O prioritization, some level of it at least, will allow some more urgent requests (to load data for an interactive action for instance) to “jump the queue” ahead of background actions (e.g. reads by a backup utility), all the while keeping the filesystem view consistent (e.g. a read after a write to the same file has to see the file as modified, so it can’t just naively jump over the corresponding write).
  • Multithreaded. In the same vein of improvements to the tied filesystem kernel module, this will allow to better serve different processes or threads that read and write from independent parts of the filesystem, especially if multiple cores are involved. HFS+, having been designed at the time of single-processor, single-threaded machines, requires centralized, bottleneck locks and is inefficient for multithreaded use cases.
  • File and directory hierarchy clones. Contrary to snapshots, clones are writable and are copied to another place in the directory hierarchy (while snapshots are filesystem-wide and exist in a namespace above the filesystem root). The direct usefulness is less clear, but it could be massively useful as infrastructure used by specialized apps, version control notably (both for work areas and repositories).
  • Logical volume management. Apple calls this “space sharing”, but it’s really the possibility to make “super folders” by making them their own filesystem in the same partition, and allows this super folder to have different backup behavior for instance.
  • Sparse files. Might as well have that, too.

What APFS will provide beyond ZFS, btrfs, etc. features:

  • Encryption as a first class feature. Full disk and per-file encryption will be integrated in the filesystem and provided by a common encryption codebase, not as layers above or below the filesystem and with two separate implementations. This also means files that are encrypted per-file will be able to be cloned, snapshotted, etc. without distinction from their unencrypted brethren.
  • Scalability down to the watch. ZFS never scaled down very well, in particular when it comes to small RAM amounts.

What we hoped to have with ZFS, but won’t get from APFS:

  • Crazy ZFS-like scalability. For instance, APFS has 64-bit nodes, not 128-bit. This is probably not unreasonable on Apple’s part.
  • RAID integration as part of the filesystem. APFS can work atop a software or hardware RAID in traditional RAID configurations (RAID-0, RAID-1, RAID-10, RAID-5, etc.), but always as a separate layer. APFS does not provide anything like RAID-Z or any other solution to the RAID-5 write hole. That is worth a mention, though I have no idea whether this is a need Apple should fulfill.
  • Deduplication. This is more generally useful to save space than clones or sparse files, but is also probably only really useful for enterprise storage arrays.

What is unclear at this point, either from the current state or because Apple may or may not add it by the time it ships:

  • Whether APFS will checksum data, and thus guarantee end-to-end data integrity. Currently it seems it doesn’t, but it checksums metadata, and has extensible data structures such that the code could trivially be extended to checksum all data while remaining backwards compatible. I don’t know why Apple does not have that turned on, but I beg them to do so, given the ever-increasing amounts of data we store on disks and SSD and their decreasing reliability (e.g. I have heard of TLC flash being used in Apple devices); we need to know when data becomes bad rather than blindly using it, which is the first step to try and improve storage reliability.
  • Whether APFS is completely transaction-based and always consistent on-disk. Copy-on-write filesystems generally are, but being copy-on-write is not sufficient by itself, and the existence of a fsck_apfs suggests that APFS isn’t always consistent on-disk, because otherwise it would not need a FileSystem Consistency checK. Apple claims writes and other filesystem updates will be “crash-safe”, but the guarantees may be lower than a fully transactional FS.
  • Whether APFS containers will be able to be extended after the fact with an additional partition (from another disk, typically), possibly even while the volumes in it are mounted. APFS support for JBOD, and the fact APFS lazily initializes its data structures (saving initialization time when formatting large disks), suggest it, and it would be undeniably useful, but it is still unknown at this time.
  • Whether APFS will be composition-preserving when it comes to file names. It will, certainly, be insensitive to composition differences in file names, like HFS+; however HFS+ goes one step further and normalizes the composition of file names, which ends up making the returned file name byte string different from what was provided at file creation, which itself subtly trips up some software like version control (via Eric Sink), and which is probably the specific behavior that led Linux founder Linus Torvalds to proclaim that HFS+ was “complete and utter crap”; see also this (latter via the Accidental Tech Podcast guys, who had the same Unicode thoughts as I did). Won’t you make Linus happy now by at least preserving composition, Apple? This is your opportunity!
  • Whether APFS uses B+trees. I know, this is an implementation detail, but it’d be neat if Apple could claim to have continuously been using B-/+trees of either kind for their storage for the last 30 years and counting.

For a more in-depth look at what we know so far about APFS, the best source by all accounts is Adam Leventhal’s series of posts.

Apple File Protocol deprecation

Along with APFS, Apple announced it would not be able to be served over AFP, only SMB (Windows file sharing), and AFP was thus deprecated. This raises the question over whether SMB is at parity with AFP: last I checked (but it was some time ago), AFP was still superior when it came to:

  • metadata and
  • searching

But I have no doubt that, whatever feature gap is left between SMB and AFP (if there is even one left), Apple will make sure it is closed before APFS ships, just like Apple made sure Bonjour had feature parity with AppleTalk before stopping support for AppleTalk.

Playgrounds on iOS

I’m of two minds about this one. I’ve always found Swift playgrounds to be a great idea. To give you an idea, back in the day when the only computer in the house was an Apple ][e, I did not yet know how to code, but I knew enough syntax that my father had set up a program that would, in a loop, plot the result of an expression over a two-axis system, and I would only have to change the line containing the expression, with the input variable being conveniently x, and the output, y; e.g. to plot the result of squaring x, I would only have to enter1:

60 y = x*x

run the program, and away I went. It was an interesting lesson when, due to my limited understanding of expressions, specifically that they are not equations, I once wrote:

60 2y = x+4

Which resulted in the same thing as I previously plotted, because this command actually modified line 602 (beyond the end of the loop)… good times.

Anyway, Swift playgrounds, which automatically plot the outcome of expressions run multiple times in a loop for instance, and even more so on iPad where you have the draggable loop templates and other control structure templates, provide the necessary infrastructure program out of the box, and learners will be able to experiment and visualize what they are doing in autonomy.

These playgrounds will be able to be shared, but when I hear some people compare this to the possibilities of Hypercard stacks, I don’t buy it. There is nothing for a user to do with these playgrounds, the graphic aspect is only a visualization (and why does it need to be so elaborate? This is basically Logo, you don’t need to make it look like a Monument Valley that would not even be minimalistic); even if the user can enter simple commands, it always has to start back from the beginning when you change the code (which is not a bad thing mind you, but shows even the command area isn’t an interactive interface). You can’t interact with these creations. Sharing these is like sharing elaborate Rube Goldberg constructions created in The Incredible Machine: it’s fun, and it’s not entirely closed as the recipient can try and improve on it, but except watching it play there is nothing for the recipient to do without understanding the working of the machine first.

Contrast that with Hypercard, in which not only you set up an actual interface, but what you’d code was handlers for actions coming from the interface, and not a non-interactive automaton. This also means that it was much less of a jump to go from there to an actual app, especially one using Cocoa: it’s fundamentally just a bunch of handlers attached to a user interface. It’s a much bigger jump when all you’re familiar with is playgrounds or even command-line programs, because it’s far from obvious how to go from there to something interactive. Seriously, I’m completely done with teaching programming by starting with command-line apps. It needs to die. What I’d like to see Apple try on the iPad is something inspired by the old Currency Converter tutorial (unfortunately gone now), where you’d create a simple but functional app that anyone could interact with.

Stricter Gatekeeper

…speaking of sharing your programming creations. I’m hardly surprised. This shows web apps is definitely the future of tinkerer apps.


  1. In Apple II Basic, you’d enter a line number then a statement, and that would replace the line in the saved program by the one you just entered. Code editors have improved a bit since then.

RIP, QuickTime for Windows

As you may have heard, Apple will no longer provide fixes for QuickTime for Windows, not even for two released security vulnerabilities (this post is a sort of PSA, as well: if for some reason you have QuickTime for Windows, uninstall it now). I wonder why anyone refers to QuickTime for Windows as being deprecated, as deprecated technologies don’t receive updates or fixes except for critical issues: the correct term for the no-fixes-at-all situation is unsupported; for all intents and purposes, QuickTime for Windows is dead. And while this has been coming for some time, this doesn’t make these news any less sad; so today, let us remember QuickTime for Windows.

While I think it existed earlier in some form, the real beginning for QuickTime for Windows was with QuickTime 3.0, which had feature parity with the MacOS version — imagine that! I know little about how it fared at that time, since my usage of Windows machines was limited; I only know that a number of game developers adopted it, eager for an acceptable media playback solution (e.g. for cutscenes): a number of games had you install QuickTime for Windows (bundled on the game CD) in order to run. Also, QuickTime for Windows came with an implementation of a subset of the Mac toolbox (though with some differences, e.g. file name length), which helped with the port of some Mac games to Windows.

Then, some of you might not have really known that time, so you have to take my word for the fact that, before YouTube in 2005-2006, there was no universal standard for distributing video online; but QuickTime with its browser plugin was the closest we had. So people were posting videos in QuickTime format (e.g. this Apple switch ad campaign parody); this did not support Linux or Unix, and Windows users were a bit reluctant to install QuickTime, but this was miles better than any alternative such as Windows media which, if it was supported at all on the Mac, was always incredibly crappy.

QuickTime served also, back then, as the basis for media playback of iTunes for Windows, which itself was the indispensable tool for allowing anyone (not just Mac owners) to own an iPod, then later on an iPhone. For those purposes and many others, QuickTime for Windows carried the burden of making sure many Apple initiatives were at least viewable outside of just Macs, playing no small part in keeping Apple relevant for all these years. QuickTime for Windows was the symbol of Apple’s leadership in multimedia, and everything it allowed legitimized the Mac and Apple including for die-hard Windows users in a way that is impossible to overstate.

For instance, back when I worked at NXP Software, QuickTime Player was the standard test for determining whether a movie file was correctly formatted (among other reasons because we were working with 3GPP media files, whose format, like that of MPEG4 media files, was derived from the QuickTime movie format): if a file generated by our media recorder had an issue with QuickTime Player, which was necessarily on Windows (we did not use Macs, at least not before we developed iPhone apps), then there was a bug in our media recorder. This made for a fun investigation when I tried to understand a bug that turned out to actually be in QuickTime!

As far as users go, the average user now has a number of alternatives, starting with VLC, but there are a number of people working on Windows in media and media-related industries who will miss having a reference media player on their machine (iTunes’ just not the same thing). However, software developers who were still building against the QuickTime SDK and relying on QuickTime being installed on Windows should have seen it coming for some time: the writing has been on the wall for QuickTime for Windows since QuickTime X in 2009, when there was no corresponding update on the Windows side, which stayed on QuickTime 7. I have not used Windows machines for media work for some time, and I missed the event when iTunes for Windows become independent of QuickTime, so this personally caught me a bit by surprise nevertheless.

So long, QuickTime for Windows. We’ll miss you.

The Stela comics app

Stela is a new comics app for smartphones (iOS-only at the time of this writing), but it works nothing like, say, Comic Chameleon (which presents existing webcomics with a phone-adapted navigation) or Comixology (which presents comics you’d find in stores as digital products, with a phone-adapted navigation when not running on a tablet). Rather, once you use it it becomes clear Stela’s purpose is to publish comics that embrace the 5 centimeters (that’s about 2 inches, for the metrically-challenged) width of today’s smartphone screens1.

These are comics that are native to that world: the panels are only as wide as the screen (nary a vertical gutter in sight) and can only extend vertically, but they can do so as much as desired because they are read by vertical scrolling. A panel may not necessarily fit on a screen (at least on an iPhone 5/5S/SE; I haven’t checked on the larger models)! An iPhone 5 screenful is a common size, but most of these comics have widely varying panels sizes, and anyway have conversations for instance that extend over multiple screenfuls: they don’t follow a pattern of identically-sized pages. The result is a very fluid flow and a reading experience that is meant to be fast.

The essence of most iPhone apps since the beginning, as best seen for instance with Twitter clients, is of a (potentially long) scrolling list of items (our friend the UITableView), with more or less drilldown or navigation between these lists. Stela is the comics embodiment of that2, and it’s very addictive.

The comics are updated chapter by chapter (which make for checkpoints as well); the economic model is that the first chapter of each story is free, and you can get a subscription (using Apple’s in-app subscription system) to read after that. It is a single subscription global to the app, not per-series, so it works a bit like an anthology series. Comics are always loaded from the network, which bothers me a little: there is no way to preload while on WiFi to avoid eating into your phone data allotment, and no way to read at all if you are off the network. iPod Touches exist, you know.

The comics themselves are of good quality, and I enjoyed the series I read, though many are still developing their story (eagerly waiting for the next chapter of Crystal Fighters for instance) and it’s a bit early to tell how they will turn out.

Either way, whether you’re from my usual audience of iOS app developers, and/or involved in comics, or neither, check it out, you’re bound to find some interesting lessons in this experiment in comics and app design.

~ Reactions ~

Over at Fleen, Gary Tyrrell cautions that, since it’s subscription-based, your access to the content will only last as long as you keep paying for it (I specifically allowed him to quote from this post as much as he wanted). It’s absolutely worth noting; maybe I’ve just become blasé to such things.


  1. The app works natively on iPad, but the comics are just scaled up, which makes for funnily huge lettering.
  2. For instance, images are loaded dynamically and present a spinner if you scroll too fast before they have had time to load, as is traditional in iPhone apps: prioritize the flow, even if that means betraying some implementation realities.