What the Joy of Tech means to me

I have a confession to make.

At the risk of ruining my credibility both as a webcomic specialist and as a long-time member of the Mac community, I have to admit I only discovered the Joy of Tech at the occasion of this Foxtrot strip paying homage to webcomics. “The Ecstasy of Tech? I get the others, but what could this possibly refer to?”

Of course, I found out soon enough, and I’ve been following the Joy of Tech ever since. Through the various news around Apple, or around the tech industry in general, or even completely unrelated matters, they are here to bring a little joy to our lives. I have used one of their comics once while paying homage to Fake Steve, and while I also wrote they come close to, but aren’t, the Penny Arcade of Apple and the Tech industry, that is only because, in my opinion, Nitrozac and Snaggy are too damn nice for their work to be considered satirical…

Nevertheless, the Joy of Tech is performing a duty that I haven’t seen anyone else fulfill: topical humor on Apple and the tech industry in general. On that, they are pretty much the only game in town, which means they get drafted (with little attribution…) whenever media is looking for humoristic commentary, especially of the graphical kind, on tech events. Take, for instance, the time France 5 (French public TV channel) used them to illustrate the acquisition of Instagram (I was the one who tipped off Nitrozac and Snaggy about it). And yet, whenever I stroll back the memory lane and browse old JoT strips, remarkably they hold up much better than, say, Penny Arcade strips do; and to me it’s because they are not just about the immediate event at hand, but more generally tell things about us Mac and tech aficionados.

Nitrozac and Snaggy are at a difficult time in their careers. If you’ve ever appreciated what they do, consider contributing to their Patreon or otherwise supporting them in some fashion. Thank you.

Support “The Secret History of Mac Gaming”

I have always had an inkling that Mac gaming must have influenced gaming in general, if only because of the many user interface innovations the Macintosh pioneered in the marketplace: they are bound to have allowed game designers to innovate in turn, building upon the Mac interface itself. However, I have not come across many mentions of such an influence in practice, in no small part because of the long-standing disparagement and prejudice against Mac gaming in general, which resulted in this not being reported on and studied in the way it should have.

But Richard Moss will be doing just that with his book project, The Secret History of Mac Gaming. And he needs your help making it a reality; granted, it has funded by now, but it is not too late to bring your contribution and make it even better, I just did. Please help show how the Mac did matter in the history of gaming.

(via Michael Tsai)

Looking back on WWDC 2016

Now that the most important Apple release of WWDC has been dealt with, we can cover everything else. I haven’t followed as closely as previous years (hence no keynote reactions on Twitter), but to me here is what stands out.

The Apple App Stores policy announcements

As seen at Daring Fireball for instance, Apple briefed the press on many current and coming improvements to the Apple App Stores (iOS, Mac, tvOS, watchOS). This actually happened ahead of WWDC, but is part of the package. There are a lot of good things, such as for instance the first acceptance that Apple isn’t entitled over the whole lifetime of an app to 30% of any purchase where the buying intent originated from the app with the 85/15 split instead of 70/30 for subscriptions after the first year. However, none of this solves the lack of free trials: if only subscription apps can have free trials, then thanks, but no thanks. I want to both try before I buy and avoid renting my software, and I don’t think subscriptions make sense for every app anyway, so improvements and clarifications (e.g. indication of whether the app is “pay once and play” or ”shareware” or ”coin-op machine”) to apps using non-recurring payment options would be welcome (more on that in a later post). Also, while those apply to the Mac App Store as well, this one will need more specific improvements to regain credibility. I don’t have much of an opinion on the new search ad system.

The new Apple File System (APFS for short)

Apple announced a new filesystem, and to say that it has, over the years, accumulated a lot of pent-up expectations to fulfill would be the understatement of the year. I can’t speak for everyone, but each year N after the loss of ZFS my reaction was “Well, they did not announce anything this year, it’s likely because they only started on year N-1 and can’t announce it yet because they can’t develop such a piece of software in a yearly release cycle, so there is not use complaining about it as it could be already started, and will show up for year N+1.” Repeat every year. So while I can scarcely believe the news that development of APFS only started in 2014, at the same time I’m not really surprised by it.

I haven’t been able to try it out, unfortunately, but from published information these are the highlights. This is as compared to ZFS because ZFS is the reference that the Mac community has studied extensively back when Apple was working on a ZFS port in the open.

What we’ll get from APFS that we hoped to have with ZFS:

  • A modern, copy-on-write filesystem. By itself, this doesn’t do much, but this is the indispensable basis for everything else:
  • Snapshots, or if you prefer, read-only clones of the filesystem as a whole. Probably the most important feature, by itself it alone would justify the investment of a new filesystem to replace HFS+.

    While the obvious use case is backups, particularly with Time Machine, it is not necessarily in the way you think. Currently, when Time Machine backs up a volume, it has to contend with it being in use, and potentially being modified, while it is being backed up; if it was required to freeze a volume while backing it up, you wouldn’t be able to use it during that time and, as a result, you would back up much less often and that would defeat most of the purpose of Time Machine. So Time Machine has no choice but to read a volume while it is being modified, and as a result may not capture a consistent view of the filesystem! Indeed, if two files are modified at the same time, but one was read by Time Machine before the modification and the other after, on the backup the saved filesystem will have one file without the modification and the other with, which has not been the state of the filesystem you intended to back up at any point in time. In fact, this may mean the data is lost if you have to reload from that backup in case neither half can work with the other as a result.

    Instead, with APFS the backup application will be able to create a snapshot, which is a constant time operation (i.e. does not depend on how much data the volume contains) and results in no additional space being taken, at least initially, then can copy from that snapshot, while the filesystem is in use and being modified, and be confident that it is capturing a consistent view of the filesystem, regardless of where the data is being saved (it could be to an HFS+ drive!). Once the copy is over, the snapshot can be harvested to make sure no additional space is used beyond that needed by the live data. Of course, this will also allow, by using multiple snapshots, to more efficiently determine what changed from last time, and with APFS on the backup drive as well the backup application will be able to save space on the backup drive, in particular not taking up space for redundancies the source APFS drive knows about already. But snapshots on the APFS source drive will mean that, after 10 years, Time Machine will finally be safe: this is a correctness improvement, not merely a performance (faster backups and/or taking less space) one.

  • Real protection in the face of crashes and power loss events. HFS+ had some of that with its journal, but it only protected metadata and came with a number of costs. APFS will make sure its writes and other filesystem updates are “crash-safe”.
  • I/O prioritization. A filesystem does not exist merely as a layout of the data on disk, but also as a kernel module that has in-memory state (mostly cache) that processes filesystem requests, and the two are generally tied. I/O prioritization, some level of it at least, will allow some more urgent requests (to load data for an interactive action for instance) to “jump the queue” ahead of background actions (e.g. reads by a backup utility), all the while keeping the filesystem view consistent (e.g. a read after a write to the same file has to see the file as modified, so it can’t just naively jump over the corresponding write).
  • Multithreaded. In the same vein of improvements to the tied filesystem kernel module, this will allow to better serve different processes or threads that read and write from independent parts of the filesystem, especially if multiple cores are involved. HFS+, having been designed at the time of single-processor, single-threaded machines, requires centralized, bottleneck locks and is inefficient for multithreaded use cases.
  • File and directory hierarchy clones. Contrary to snapshots, clones are writable and are copied to another place in the directory hierarchy (while snapshots are filesystem-wide and exist in a namespace above the filesystem root). The direct usefulness is less clear, but it could be massively useful as infrastructure used by specialized apps, version control notably (both for work areas and repositories).
  • Logical volume management. Apple calls this “space sharing”, but it’s really the possibility to make “super folders” by making them their own filesystem in the same partition, and allows this super folder to have different backup behavior for instance.
  • Sparse files. Might as well have that, too.

What APFS will provide beyond ZFS, btrfs, etc. features:

  • Encryption as a first class feature. Full disk and per-file encryption will be integrated in the filesystem and provided by a common encryption codebase, not as layers above or below the filesystem and with two separate implementations. This also means files that are encrypted per-file will be able to be cloned, snapshotted, etc. without distinction from their unencrypted brethren.
  • Scalability down to the watch. ZFS never scaled down very well, in particular when it comes to small RAM amounts.

What we hoped to have with ZFS, but won’t get from APFS:

  • Crazy ZFS-like scalability. For instance, APFS has 64-bit nodes, not 128-bit. This is probably not unreasonable on Apple’s part.
  • RAID integration as part of the filesystem. APFS can work atop a software or hardware RAID in traditional RAID configurations (RAID-0, RAID-1, RAID-10, RAID-5, etc.), but always as a separate layer. APFS does not provide anything like RAID-Z or any other solution to the RAID-5 write hole. That is worth a mention, though I have no idea whether this is a need Apple should fulfill.
  • Deduplication. This is more generally useful to save space than clones or sparse files, but is also probably only really useful for enterprise storage arrays.

What is unclear at this point, either from the current state or because Apple may or may not add it by the time it ships:

  • Whether APFS will checksum data, and thus guarantee end-to-end data integrity. Currently it seems it doesn’t, but it checksums metadata, and has extensible data structures such that the code could trivially be extended to checksum all data while remaining backwards compatible. I don’t know why Apple does not have that turned on, but I beg them to do so, given the ever-increasing amounts of data we store on disks and SSD and their decreasing reliability (e.g. I have heard of TLC flash being used in Apple devices); we need to know when data becomes bad rather than blindly using it, which is the first step to try and improve storage reliability.
  • Whether APFS is completely transaction-based and always consistent on-disk. Copy-on-write filesystems generally are, but being copy-on-write is not sufficient by itself, and the existence of a fsck_apfs suggests that APFS isn’t always consistent on-disk, because otherwise it would not need a FileSystem Consistency checK. Apple claims writes and other filesystem updates will be “crash-safe”, but the guarantees may be lower than a fully transactional FS.
  • Whether APFS containers will be able to be extended after the fact with an additional partition (from another disk, typically), possibly even while the volumes in it are mounted. APFS support for JBOD, and the fact APFS lazily initializes its data structures (saving initialization time when formatting large disks), suggest it, and it would be undeniably useful, but it is still unknown at this time.
  • Whether APFS will be composition-preserving when it comes to file names. It will, certainly, be insensitive to composition differences in file names, like HFS+; however HFS+ goes one step further and normalizes the composition of file names, which ends up making the returned file name byte string different from what was provided at file creation, which itself subtly trips up some software like version control (via Eric Sink), and which is probably the specific behavior that led Linux founder Linus Torvalds to proclaim that HFS+ was “complete and utter crap”; see also this (latter via the Accidental Tech Podcast guys, who had the same Unicode thoughts as I did). Won’t you make Linus happy now by at least preserving composition, Apple? This is your opportunity!
  • Whether APFS uses B+trees. I know, this is an implementation detail, but it’d be neat if Apple could claim to have continuously been using B-/+trees of either kind for their storage for the last 30 years and counting.

For a more in-depth look at what we know so far about APFS, the best source by all accounts is Adam Leventhal’s series of posts.

Apple File Protocol deprecation

Along with APFS, Apple announced it would not be able to be served over AFP, only SMB (Windows file sharing), and AFP was thus deprecated. This raises the question over whether SMB is at parity with AFP: last I checked (but it was some time ago), AFP was still superior when it came to:

  • metadata and
  • searching

But I have no doubt that, whatever feature gap is left between SMB and AFP (if there is even one left), Apple will make sure it is closed before APFS ships, just like Apple made sure Bonjour had feature parity with AppleTalk before stopping support for AppleTalk.

Playgrounds on iOS

I’m of two minds about this one. I’ve always found Swift playgrounds to be a great idea. To give you an idea, back in the day when the only computer in the house was an Apple ][e, I did not yet know how to code, but I knew enough syntax that my father had set up a program that would, in a loop, plot the result of an expression over a two-axis system, and I would only have to change the line containing the expression, with the input variable being conveniently x, and the output, y; e.g. to plot the result of squaring x, I would only have to enter1:

60 y = x*x

run the program, and away I went. It was an interesting lesson when, due to my limited understanding of expressions, specifically that they are not equations, I once wrote:

60 2y = x+4

Which resulted in the same thing as I previously plotted, because this command actually modified line 602 (beyond the end of the loop)… good times.

Anyway, Swift playgrounds, which automatically plot the outcome of expressions run multiple times in a loop for instance, and even more so on iPad where you have the draggable loop templates and other control structure templates, provide the necessary infrastructure program out of the box, and learners will be able to experiment and visualize what they are doing in autonomy.

These playgrounds will be able to be shared, but when I hear some people compare this to the possibilities of Hypercard stacks, I don’t buy it. There is nothing for a user to do with these playgrounds, the graphic aspect is only a visualization (and why does it need to be so elaborate? This is basically Logo, you don’t need to make it look like a Monument Valley that would not even be minimalistic); even if the user can enter simple commands, it always has to start back from the beginning when you change the code (which is not a bad thing mind you, but shows even the command area isn’t an interactive interface). You can’t interact with these creations. Sharing these is like sharing elaborate Rube Goldberg constructions created in The Incredible Machine: it’s fun, and it’s not entirely closed as the recipient can try and improve on it, but except watching it play there is nothing for the recipient to do without understanding the working of the machine first.

Contrast that with Hypercard, in which not only you set up an actual interface, but what you’d code was handlers for actions coming from the interface, and not a non-interactive automaton. This also means that it was much less of a jump to go from there to an actual app, especially one using Cocoa: it’s fundamentally just a bunch of handlers attached to a user interface. It’s a much bigger jump when all you’re familiar with is playgrounds or even command-line programs, because it’s far from obvious how to go from there to something interactive. Seriously, I’m completely done with teaching programming by starting with command-line apps. It needs to die. What I’d like to see Apple try on the iPad is something inspired by the old Currency Converter tutorial (unfortunately gone now), where you’d create a simple but functional app that anyone could interact with.

Stricter Gatekeeper

…speaking of sharing your programming creations. I’m hardly surprised. This shows web apps is definitely the future of tinkerer apps.


  1. In Apple II Basic, you’d enter a line number then a statement, and that would replace the line in the saved program by the one you just entered. Code editors have improved a bit since then.

Review: App Review Guidelines: The Comic Book

The review for this Wednesday is for an unexpected, shall we say, release: it doesn’t appear to have been solicited through Diamond1 beforehand, and so the first comic book coming from Apple Inc. as a publisher, at least first in recent history, came as a complete surprise to everyone. It was released at the same time as many news from Apple, so it took me a bit of time to notice it, then get to it.

Before we begin, if you’ve followed this blog for a bit, you might have noticed I have a bit of a thing for comics, be it in previous posts or the comicroll and the pull list in the sidebar; or maybe you’ve been following some of my other endeavors or follow me on Twitter and have been left with little doubt that I do read and enjoy comics very much. So this is where I’m coming from on comics in general.

I also have a lot of appreciation more specifically for comics as teaching aids: it is to me a very suitable medium for teaching, and there is a lot of unjustified prejudice against this art form as being not for serious purposes, whatever that means. This is completely wrong, as can show the generally cheesy, but not bad teaching comics I read as a child, and it goes for grownups too, as the cartoons from Larry Gonick show (a nice trove of which can be found here, thanks Jeff), or more recently those Dante Shepherd is commissioning with a dedicated grant: 1, 2, 3, 4, 5 and 6 (so far); hat tip to Fleen. So this comic from Apple could, if well done, help with general understanding of what they are trying to accomplish with these guidelines.

I also understand that, as a developer who has followed Apple’s policies relatively well and have some expertise in interpreting them, and who reads a few specialists in Apple kremlinology, I may not actually be in the target audience. I have little doubt that the app review team and DTS are interacting daily with many, many developers who discover the guidelines when their app gets rejected for violating them and/or have a very incomplete picture of the whole of the guidelines and/or are are very stubborn about what they think their “rights” are; this comic is probably intended for them. Lastly, the link to this comic has been provided to me by people I trust, and it is hosted in a CDN domain that Apple uses for a variety of developer-related resources (e.g. Swift blog post images), so I have little reason to doubt its authenticity.

Get on with it!

Ok, ok. This comic is actually sort of an anthology, split in five parts, and the first is:

Safety

With art by Mark Simmons. In a setting and style reminiscent of Jack Kirby’s cosmic works (New Gods in particular), we find the hidden son of Flash and the Silver Surfer as the hero of this story, in which he has to cruise through space, avoiding a number of hazards, after he encounters some sort of Galactus-like planet eater. Will he succeed in time?

I found the story rather hard to follow, no doubt due to the unfamiliar setting, and had to reread it a few times to make sure I hadn’t missed anything; beyond that, the art serves its purpose, unfortunately the tests clearly isn’t here to support it.

Performance

With art by Ile Wolf and Luján Fernández. In a more playful style, two schoolchildren in uniform are battling using Pokémon/Digimon/kaiju (circle as appropriate), and the battle appears to have grown out of control. The situation is dramatic, and it’s not sure there is anything that can stop them.

At least here any ambiguity as to the situation is intentional, but even then it’s hard to take it seriously when the text (speech or narration) takes you out of the climax; not everyone can be a Stan Lee and add text after-the-fact that works well with such a story. And while the conclusion of “Safety” in part explains its title, I can’t help but think its hero would have been more appropriate to star in the “Performance” section.

Business

With art by Shari Chankhamma. A more intimate setting with interesting art where we follow the growth of a boy through times good and bad, but always in the same place: the barbershop he patronizes.

Maybe the most interesting of the stories in this anthology, and it’s too bad they couldn’t come up with text that was to the level: either do away with it, or hire better writers! Who edited this stuff?

Design

With art by Ben Jelter. Foraging in a post-apocalyptic wasteland, with an art style to match, a boy locates and manages to repair a robot who may or may not be related to Wall-e and Eve.

It’s a section for which developers for Apple platforms have understandably high expectations, but I don’t know if they’ll be met with the robot design, or with the art in general, which is nothing special. The less said about the text, the better.

Legal

With art by Malcolm Johnson. A noir/private eye story, all in greyscale, and interestingly starring a woman.

The art style is surprising in a good way for such a story, but it does not do a very good job of carrying the story, and as we’ve seen, no point in counting on the text for that either. At least this one has more relationship with its claimed subject matter than the others do.

Conclusion

What… in… the… ever-loving… frick? This comic may have the dimensions and approximate page count of a comic issue, but is, to be blunt, a crushing disappointment. Its only point, it turns out, is to put pictures which tell their own stories around the exact words of the official document, without any attempt at adaptation, or even just, say, recontextualization of the guidelines as an exchange between two characters. These words don’t benefit in any way from being told there. Meanwhile, the pictures just follow their own scenarios and tell their own stories without any consideration for what is supposedly spoken in the bubbles: there is no correspondence either thematic or in pace between the events depicted and the words you can read. There is no teaching benefit whatsoever to these comics, and no way I see anyone at any knowledge level benefit from reading it, let alone be enlightened as to the profound meaning of the guidelines. It’s as if bubbles were randomly placed, linked so that each would overflow into the next, then the text of the guidelines was just dumped into them. This shows better than anything I have previously seen that sequential art is more than the sum of pictures and text.

Verdict: download it, but don’t read it, and only use it in a few years to remind your interlocutor who works at Apple that this has been a real thing that Apple has released, in order to embarrass him.

App Review Guidelines: The Comic Book
Price: 0¢ (digital only)
Publisher: Apple
Words: Apple
Cover illustration: Dailen Ogden
Illustrations: Mark Simmons, Ile Wolf, Luján Fernández, Shari Chankhamma, Ben Jelter, and Malcolm Johnson


  1. Diamond is the only distributor to comic book stores in North America, and comics appear in its catalog a few months before being available, in case you’re not familiar with that aspect of the comics industry.

On the recent Apple top management adjustments

Michael Tsai summed up my thoughts exactly on the Schiller side of the announcements: I share his reservations about Schiller, but I indeed can’t complain when the various groups that interact with developers, whose lack of coordination I previously listed as being part of the problem, are now under a single top executive (with the exception of APIs, still the responsibility of Craig Federighi, of course).

So I thought I’d also mention the promotion to top leadership of Johny Srouji, promotion which to me represents the rise of semiconductor engineering inside Apple. Far from shedding this skill (as it might have appeared to some at the time of the Intel transition), Apple doubled down on it: in the domain of system glue (chipsets), peripheral controllers, sensors, etc., but also going as far as to design its own processors, both at the RTL level (with the acquisition of PA Semi) and at the RTL to mask translation level (with the acquisition of Intrisity); something Apple never did (as far as we know, anyway) for 68k, PowerPC, or any other processor they used prior to ARM. With impressive results, particularly now with the iPad pro.

The Mac App Store and long-term app preservation

I am fortunate enough not to have apps on the Mac App Store, and I have bought few enough apps on it (for reasons I previously exposed) that I initially missed the meltdown, due to the store, of many apps bought there. This is not an outage, in the sense that an outage implies the user is aware on some level on being dependent on an online resource; this is worse. This is not just unacceptable: this is a fundamental violation of the trust that both app developers and customers have placed in Apple, namely that bought, installed and compatible apps would keep working (short of any dramatic action taken for consumer protection so that they would not, such as revoking the certificate of a malicious developer).

Worse, this has implications beyond the Mac App Store per se. As you know, Apple is reserving many APIs related to online services even in a remote fashion to Mac App Store apps: even when there is a non-Mac App Store version of the app available, it cannot make use of iCloud (is there a typo version of “revealing tongue slips”? Because I initially typed ”iCould”…) or Apple Maps. So, in turn, how am I supposed to trust iCloud or Apple Maps, if I am not sure I can run any app that can access it? As if these services did not already have a reputation…

But even more troubling are the implications for long-term usage and preservation of software and it data. The consumer issue of not being able to trust that a purchased app will keep running even when nothing else changes is bad enough (you could set back the system clock, but how realistic is that, even in an unconnected system? You would no longer be able to trust the creation or modification dates of any of your documents, for a start); but the implications on being unable to preserve running software on a cultural level is frightening. Even more so for the documents with proprietary formats created by that software. I’ve been following with interest the initiatives of Jason Scott in that area, I am definitely down with the need to preserve this software and data, not just for ourselves, but the future generations. And the Mac App Store (and the iOS App Store, the only difference being that we have not had any fire drill on that side. Yet.) is “not helping”. To put it mildly, because this blog tries to be family friendly.

I initially though there was no DRM component to this story: certificates, “damaged apps”, that sounded like code signature infrastructure, in other words protection of the consumer against malware, something that the user can disable (ostensibly, at least). But when I tried to convince my Mac to run this app as an unsigned app, I encountered what is extremely likely to be the store DRM: I initially got the “your app was bought on another machine” message, so I tried deleting the receipt, but then I got the dreaded “app damaged” message, at which point I removed the signature. But no way: in that case, what happens is that the app does not launch either, with the console printing:

13/11/15 15:36:23,608 com.apple.launchd.peruser.502: ([0x0-0x2cc2cc].com.tapbots.TweetbotMac[9317]) Exited with code: 173
13/11/15 15:36:23,663 storeagent: Unsigned app (/Applications/Tweetbot.app).

Since I removed the MAS receipt, how is storeagent getting involved? Probably in order to decode the app DRM, and as you see it refuses to do so due to the app being now unsigned. So now we have DRM preventing us from running our legitimately bought software. I have kept a pristine copy of the app in a safe location to make further attempts, but the only way I can see is to create a new root CA which I install on the machine as a trusted root, and redo the signing chain, and even that might not work if the DRM is somehow tied to the signature chain.

I was already wary of buying apps without trials; this event guarantees that I will never buy anything else from the Mac App Store (and will try to obtain direct licenses of apps already bought there). No direct version of your app? You don’t get my business. I would delete the Mac App Store app if I could. Apple could change my mind by providing verifiable commitments on the ability to disengage the signature checks and in operational service levels, and even then… Furthermore, Apple owes an apology to all the app developers who trusted them with the Mac App Store and who had a long day (and will continue to have long days) of customer support entirely due to Apple’s incompetence.

Apple later on sent emails to developers to explain themselves on the issue; I will count that as the aforementioned apology. I don’t mind that they took a few business days to react, as they themselves had to figure out what the problems were from the multiple reports; I do mind that, operationally, they allowed developers and themselves to be caught flat-footed in the first place: why isn’t there anyone at Apple checking that a sample of Mac App Store apps do run on a machine with the time perpetually set to one month in the future? Still, I guess I’m glad we got an answer in the first place. — November 19, 2015

~ Reactions ~

Rainer Brockerhoff, besides presenting some investigations and corrections, took some issue with my investigation methods, and we started exchanging in the comments there. Don’t miss it.

Thoughts on developer presentation audiences

So after the WWDC 2015 keynote, reading Dr. Drang (via Six Colors) and generally agreeing with his take sparked some reflexions. In particular, as a software developer myself, a side reflexion about what (if anything) is particular about software developer audiences, so that people like Drake and Jimmy Iovine, in the unlikely case they read this, don’t think of software developers as a mean crowd; and who knows, it could be applicable to other show-biz types in case they present at events like Build or Google I/O.

But to being with, whose bright idea was it, honestly, to have Apple Music be the “one more thing” of a WWDC keynote? Especially of a keynote that was one of the longest, if not the longest, in recent history (I’d check, but http://www.apple.com/apple-events/ is currently redirecting me to http://www.apple.com/live/2015-june-event/)? Only one of those (“one more thing” or “WWDC keynote”) would have been fine, but not both. You can have, at the end of a long presentation, at a time the attention of the crowd (which, if it needs to be reminded, was up very early and spent a lot of time in line, because otherwise you end up in an overflow room) may be waning, a “one more thing” about something outside of the theme, for instance new hardware announcement, on condition that this announcement relieves some pain points (e.g. new hardware that makes a previously impossible combination now possible, easing the life of developers who use one and develop for the other) or otherwise has elements that can spark the audience’s specific interest so that you can be guaranteed some cheers and applauses and keep the crowd interested. Apple Music, even if it materializes as a good product, does not have that.

Don’t get me wrong, software developers like music just as much as the next guy. And heck, we’ve seen worse, including at WWDC or iPhone SDK events (Farmville, anyone?). In fact, software developers are not a tough crowd; they will almost alway give at least polite applause when cued: I remember the Safari kickoff presentation at WWDC 2010, with an audience therefore presumably dominated by web developers, and Safari extensions ended up being introduced, with one of the presenters presenting… how they ported their ad blocking extension to Safari. To an audience at least in part making a living (directly or indirectly) from web advertising. Even then, he got polite applause at the end of his presentation, like everyone else. And outside of very specific, preexisting situations (it was at Macworld, but could just as well have happened at WWDC) software developers will never boo a presenter offstage. Why is that? Well, an important part is that software developers have been in the presenter’s shoes before; not to this scale, most likely, but they know it’s a tough part, either to have a demo that works (hence why the applause even for incremental features that were even seen on other platforms before), or worse, if there is no demo, to be able to convey the importance of the software you are talking about without being able to show it. And even if the presenter is mediocre, contrary to a mediocre artist, software developers know that applauding the presenter will not make him stay longer, and his script will end soon enough anyway, so might as well politely applaud.

Software developers, as tech enthusiasts, are more generally interested in anything that moves the state of the art forward, even if it has little relation with any technology they will actually make use of in their job, on condition they can see what is new or specific about it; or at least, they want to be able to take the announcement apart, as we’re going to see. Plus they are heavy users of the platforms they are developing for, so anything that makes user’s lives easier makes theirs easier, too, and they will react to that.

But software developers are also a wary bunch. All of them have been burned before; doesn’t matter whether they trusted a company they shouldn’t have or whether they couldn’t have foreseen anything but were betrayed, they all got a past experience of betting on something (a platform, an API, a service, a tool, etc.) and losing their bet. So in presentations they don’t want to be given dreams of an ideal future where the product magically does what we expect of it, but rather they want demos, or at the very least material claims that can be objectively evaluated as soon as anything concrete is provided. Triply so for Internet services. Everything else is just a setup to get to a demo, as far as a software developer is concerned. Also, as a result software developers take apart everything that is being said to try and figure how it works, in particular to foresee any eventual limitation; yes, to an extent it does take out the magic to dissect everything that way, but remember that for software developers this is a matter of survival. Again, triply so for Internet services. Do I need to remark that the Apple Music presentation, even the demo from Eddy Cue, provided little in the way of these material claims? He did show how the user interacts with the service, but, being an Internet service, this does not show how it works, really. Also, this means that the crowd may be too busy trying to make sense of what you are saying to react to your quick quips.

Software developers are also extremely good at math and mental arithmetic. It goes with the job. They will double-check everything you throw at them, live, so don’t ever expect to be able to assert claims that don’t literally add up.

As with any recurring event (as opposed to, say, a concert date, where this is less the case), there is also a lot of lore and unsaid things that are nevertheless known from both the regular presenters and the audience. If you’re not a regular presenter, it’s not something you can tap into (so yes, you will be at a disadvantage from the regular presenters), but you better be briefed about those to avoid running into them by accident. I remember a high school incident where I had an exchange with a classmate that the class couldn’t miss: it was about silex/stone blades, and I can’t remember what the root of the problem was, but I was countering that this was no way to build a hatchet that could chop down trees, to which my classmate countered that chopping down trees was not the guys’ aim. I think I let him have the point at the time; this was presumptuous of him to assume this, but at the same time it was presumptuous for me to assume this use case, this was just something I came up with as a use case for a hatchet at the top of my head. Fast forward a few months, but still in the same year, the class was making an outside trip to a place where they studied stone age tooling, and during his presentation the guy explained that following the methods from these times: preparing and carving the stone, pairing it with a wooden handle and attaching with then-current methods, he got a tool that worked very well, taking as an example that he had been able to chop down a small tree with it.

After maybe a beat, the whole class erupted in laughter.

My classmates had clearly not forgotten; the poor presenter was all “Was it something I said?” (I was tempted reach out to him, shake his hand and thank him profusely), and our teacher fortunately came to his rescue by promising to tell him later. There was of course no way he could have been briefed for this, but this is not the case for an event such as WWDC.

And, as a “one more thing”, I think I should close with a mention of the other WWDC keynote audience, those who watch the live stream, and react on Twitter instead. This is not exactly a developer audience, but pretty close. However, the reactions tend to be all over the place, in particular you always get the complaint (that ends up trending each time!) that no new hardware (or no new hardware the poster was interested in) was announced, even though this is a silly expectation to have: you can always hope for some, mind you, but given there is always a lot to say about future OS updates at WWDC, especially now with three major Apple platforms, it’s better for Apple to make these announcements at other times. So it’s hard to get a feel for the WWDC live stream audience.

WWDC 2015 Keynote not-quite-live tweeting

(Times are GMT+2)

  • 10:14 PM – 8 Jun 2015: Looking at the previewed Mac OS X improvement, I think it’s too bad @siracusa is not going to be reviewing them (but it’s his call) #WWDC15
  • 10:16 PM – 8 Jun 2015: Speaking of @siracusa , I’d almost prefer for multitasking dividers not to be repositionable. #positioning #OCD #WWDC15
  • 10:19 PM – 8 Jun 2015: Metal on the Mac: “Of course this means war” #OpenGL #WWDC15
  • 10:21 PM – 8 Jun 2015: At long last we have search in third-party apps on iOS! (viz. http://wanderingcoder.net/2012/01/20/ios-document-filing/ ) #WWDC15
  • 10:23 PM – 8 Jun 2015: (Around the 37 minute mark): It’s funny, because I’m actually exercising as I watch the keynote stream and take these notes. #WWDC15
  • 10:24 PM – 8 Jun 2015: Siri does still rely on network services, so it can’t all “stay on the device”… #WWDC15
  • 10:27 PM – 8 Jun 2015: (Around the 43 minute mark): the Apple guys have turned into Stanley Yankeeball http://crazyapplerumors.com/2006/10/10/another-mac-publication-changes-its-name/ (minus the Stanley) #WWDC15
  • 10:33 PM – 8 Jun 2015: These improvements to notes might be good, or might turn it into a mess(is it a word processor? For structured text? Something else?)#WWDC15
  • 10:35 PM – 8 Jun 2015: Mapping exits of tube stations is great, not even all of the transit systems’ dedicated apps do so. #WWDC15
  • 10:37 PM – 8 Jun 2015: Since it’s only in select countries at first, the new news app is more than just an aggregator and probably has some editorial. #WWDC15
  • 10:39 PM – 8 Jun 2015: Keyboard gestures for editing are great, but are they like cursor keys (more accurate) or more like mouse movement? #WWDC15
  • 10:40 PM – 8 Jun 2015: Yes! Yes! Yes! Yes! Split screen multitasking on iPad! Amply justifies upgrading to the Air 2. #WWDC15
  • 10:41 PM – 8 Jun 2015: I don’t know how practical multi-touch on multiple apps is, but it sure rocks. #WWDC15
  • 10:42 PM – 8 Jun 2015: New low power mode is the battery equivalent of low memory warnings. #WWDC15
  • 10:43 PM – 8 Jun 2015: Apple game development frameworks still aren’t credible as long as Apple is not dogfooding them. We want Apple-made games! #WWDC15
  • 10:45 PM – 8 Jun 2015: With Home Kit through iCloud, better hope that iCloud is secure… (or that this particular part can be disabled). #WWDC15
  • 10:46 PM – 8 Jun 2015: About Swift: open source is nice, standardization would be nicer. Yes, Objective-C isn’t a standard, but C and C++ are. #WWDC15
  • 10:47 PM – 8 Jun 2015: With iOS9 still supporting the iPad 2, get ready to have to support ARMv7 and the Cortex A9 for some time (it’s not hard, mind you). #WWDC15
  • 10:48 PM – 8 Jun 2015: Can’t really comment on watchOS improvements, since I don’t know much about what it currently does anyway. #WWDC15
  • 10:51 PM – 8 Jun 2015: With native Apple Watch apps, get ready for a “Benchmarking on your wrist” post from @chockenberry as soon as watchOS 2.0 lands. #WWDC15
  • 10:52 PM – 8 Jun 2015: (around the 1:40 mark): wasn’t expecting them to be ready to demo the new watchOS features live so soon after Apple Watch release. #WWDC15
  • 10:54 PM – 8 Jun 2015: (around the 1:41 mark): Kevin Lynch was tethered by the wrist during the Apple Watch demo. Is that punishment for Flash? #WWDC15
  • 10:56 PM – 8 Jun 2015: I was even less expecting them to have a new watch OS beta ready today, 6 weeks after the Apple Watch release. #WWDC15
  • 10:57 PM – 8 Jun 2015: Between Jimmy Iovine and the two women (sorry ladies, I did not write down your names), many new presenters, that’s great. #WWDC15
  • 10:58 PM – 8 Jun 2015: Interesting that they would present Apple Music at WWDC, would appear more fitting for an iPhone or music event. #WWDC15
  • 10:59 PM – 8 Jun 2015: I am more interested in music I can keep, though global radio is interesting. #WWDC15
  • 11:01 PM – 8 Jun 2015: Nothing has really replaced the records stores so far when it comes to music discovery. Will Apple Music do better than Ping? #WWDC15
  • 11:02 PM – 8 Jun 2015: With the news app and Apple Music, Apple is doing more editorial/curation than they ever did. #WWDC15
  • 11:03 PM – 8 Jun 2015: I won’t comment on Apple Pay until it reaches France. #WWDC15
  • 11:04 PM – 8 Jun 2015: Sure, you can ask Siri for the music used in Selma, but she’s no Shazam. #WWDC15
  • 11:05 PM – 8 Jun 2015: After the demo, my feeling of Apple Music is: Netflix for music. Android support is interesting… #WWDC15
  • 11:06 PM – 8 Jun 2015: Again, interesting to have a live performance at WWDC, rather than at an iPhone or music event. #WWDC15
  • 11:09 PM – 8 Jun 2015: And that’s it for the #WWDC15 keynote comments. Now back to notifying of new http://wanderingcoder.net/ posts.
  • 8:53 AM – 9 Jun 2015: Some more post-sleep #WWDC15 thoughts before returning to normal:
  • 8:56 AM – 9 Jun 2015: First, there was no homage or reference (that I could spot) in the keynote to @Siracusa and his Mac OS X reviews, I’m disappointed. #WWDC15
  • 9:01 AM – 9 Jun 2015: Second, maybe it’s just me, but I get the impression the keynote is less and less for developer-level features. #WWDC15
  • 9:13 AM – 9 Jun 2015: Third, no free Apple Music tier means people won’t get the impression this is music they can access forever. #WWDC15
  • 9:15 AM – 9 Jun 2015: Fourth and I’ll be done: with Apple global radio, what happens to iTunes Radio? #WWDC15

Thank you, Mr. Siracusa

Today, I learned that John Siracusa had retired from his role of writing the review of each new Mac OS X release for Ars Technica. Well, review is not quite the right word: as I’ve previously written when I had the audacity to review one of his reviews, what are ostensibly articles reviewing Mac OS X are, to my mind, better thought of as book-length essays that aim to vulgarize the progress made in each release of Mac OS X. They will be missed.

It would be hard for me to overstate the influence that John Siracusa’s “reviews” have had on my understanding of Mac OS X and on my writing; you only have to see the various references to John or his reviews I made over the years on this blog (including this bit…). In fact, the very existence of this blog was inspired in part by John: when I wrote him with some additional information in reaction to his Mac OS X Snow Leopard review, he concluded his answer with:

You should actually massage your whole email into a blog post [of] your own.  I’d definitely tweet a link to it! 🙂

to which my reaction was:

Blog? Which blog? On the other hand, it’d be a good way to start one
Hmm

Merely 4 months later, for this reason and others, this blog started (I finally managed to drop the information alluded to in 2012; still waiting for that tweet 😉 ).

And I’ll add that his podcasting output may dwarf his blogging in volume, but, besides the fact I don’t listen to podcasts much, I don’t think they really compare, mostly because podcasts lack the reference aspect of his Mac OS X masterpieces due to the inherent limitations of podcasts (not indexed, hard to link to a specific part, not possible to listen in every context, etc.). But, ultimately, it was his call; as someone, if I remember well, commented on the video of this (the actual video has since gone the way of the dodo): “Dear John, no pressure. Love, the Internet”. Let us not mourn, but rather celebrate, from the Mac OS X developer preview write-ups to the Mac OS X 10.10 Yosemite review, the magnum opus he brought to the world. Thank you, Mr. Siracusa.

MPW on Mac OS X

From Steven Troughton-Smith (via both Michael Tsai and John Gruber) comes the news of an MPW compatibility layer project and how to use it to build code targeting Classic Mac OS and even Carbonized code from a Mac OS X host, including Yosemite (10.10). This is quite clever, and awesome news, as doing so was becoming more and more complicated, and in practice required keeping one ore more old Macs around.

Back in the days of Mac OS X 10.2-10.4, I toyed with backporting some of my programming projects, originally developed in Carbon with Project Builder, to MacOS 9, and downloaded MPW (since it was free, and CodeWarrior was not) to do so. The Macintosh Programmer’s Workshop was Apple’s own development environment for developing Mac apps, tracing its lineage from the Lisa Programmer’s Workshop, which was originally the only way to develop Mac apps (yes, in 1984 you could not develop Mac software on the Mac itself). If I recall correctly, Apple originally had MPW for sale, before they made it free when it could no longer compete with CodeWarrior. You can still find elements from MPW in the form of a few tools in today’s Xcode — mostly Rez, DeRez, GetFileInfo and SetFile. As a result, I do have some advice when backporting code from Mac OS X to MacOS 9 (and possibly earlier, as Steven demonstrated).

First, you of course have to forget about Objective-C, forget about any modern Carbon (e.g. HIObject, though the Carbon Event Manager is OK), forget about Quartz (hello QuickDraw), forget about most of Unix, though if I recall correctly the C standard library included with MPW (whose name escapes me at the moment) does have some support beside the standard C library, such as open(), read(), write() and close(). Don’t even think about preemptive threads (or at least, ones you would want to use). In fact, depending on how far back you want to go, you may not have support for what you would not even consider niceties, but were actually nicer than what came before; for instance, before Carbon, a Mac app would call WaitNextEvent() in a loop to sleep until the next event that needed processing, and then the app would have to manually dispatch it to the right target, including switching on the event type, performing hit testing, etc.: no callback-based event handing! But WaitNextEvent() itself did not appear until System 7, if I recall correctly, so if you want to target System 6 and earlier, you have to poll for events while remembering to yield processing time from time to time to drivers, to QuickTime (if you were using it), etc. The same way, if you want to target anything before MacOS 8 you cannot use Navigation Services and instead have to get yourself acquainted with the Standard File Package… FSRefs are not usable before MacOS 9, as another example.

When running in MacOS 9 and earlier, the responsibilities of your code also considerably increase. For instance, you have to be mindful of your memory usage much more than you would have to in Mac OS X, as even when running with virtual memory in MacOS 9 (something many users disabled anyway) your application only has access to a small slice of address space called the memory partition of the application (specified in the 'SIZE' resource and that the user can change): there is only one address space in the system which is partitioned between the running apps; as a result memory fragmentation becomes a much more pressing concern, requiring in practice the use of movable memory blocks and a number of assorted things (move high, locking the block, preallocating master pointers, etc.). Another example is that you must be careful to leave processor time for background apps, even if you are a fullscreen game: otherwise, for instance if iTunes is playing music in the background, it will keep playing (thanks to a trick known as “interrupt time”)… until the end of the track, and become silent from then on. Oh, and did I mention that (at least before Carbon and the Carbon Event Manager) menu handling runs in a closed event handling loop (speaking of interrupt time) that does not yield any processing time to your other tasks? Fun times.

Also, depending again on how far back you want to go, you might have difficulty using the same code in MacOS 9 and Mac OS X, even with Carbon and CarbonLib (the backport of most of the Carbon APIs to MacOS 9 as a library, in order to support the same binary and even the same slice running on both MacOS 9 and Mac OS X). For instance, if you use FSSpec instead of FSRef in order to run on MacOS 8, your app will have issues on Mac OS X with file names longer than were possible on MacOS 9; they are not fatal, but will cause your app to report the file name as something like Thisisaverylongfilena#17678A… not very user-friendly. And the Standard File Package is not supported at all in Carbon, so you will have to split your code at compile time (so that the references to the Standard File Package are not even present when compiling for Carbon) and diverge at runtime so that when running in System 7 the app uses the Standard File Package, and when running in MacOS 8 and later it uses Navigation Services, plus the assorted packaging headaches (e.g. using a solution like FatCarbon to have two slices, one ppc that links to InterfaceLib, the pre-Carbon system library, linking weakly to the Navigation Services symbols, and one ppc that links to CarbonLib and only runs on Mac OS X).

You think I’m done? Of course not, don’t be silly. The runtime environment in MacOS 9 is in general less conductive to development than that of Mac OS X: the lack of memory protection not only means that, when your app crashes, it is safer to just reboot the Mac since it may have corrupted the other applications, but also means you typically do not even know when your code, say, follows a NULL pointer, since that action typically doesn’t fault. Cooperative multitasking also means that a hang from your app hangs the whole Mac (only the pointer is still moving), though that can normally be solved by a good command-alt-escape… after which it’s best to reboot anyway. As for MacsBug, your friendly neighborhood debugger… well, for one, it is disassembly only, no source. But you can handle that, right?

It’s not that bad!

But don’t let these things discourage you from toying with Classic MacOS development! Indeed, doing so is not as bad as you could imagine from the preceding descriptions: none of those things matter when programming trivial, for fun stuff, and even if you program slightly-less-than-trivial stuff, your app will merely require a 128 MB memory partition where it ought to only take 32 MB, which doesn’t matter in this day and age.

And in fact, it is a very interesting exercise because it allows a better understanding of what makes the Macintosh the Macintosh, by seeing how it was originally programmed for. So I encourage you all to try and play with it.

For this, I do have some specific advice about MPW. For one, I remember MrC, the PowerPC compiler, being quite anal-retentive for certain casts, which it just refuses to do implicitly: for instance, the following code will cause an error (not just a warning):

SInt16** sndHand;
sndHand = NewHandle(sampleNb * sizeof(SInt16));

You need to explicitly cast:

SInt16** sndHand;
sndHand = (Sint16**)NewHandle(sampleNb * sizeof(SInt16));

It is less demanding when it comes to simple casts between pointers. Also, even though it makes exactly no difference in PowerPC code, it will check that functions that are supposed to have a pascal attribute (supposed to mark the function as being called using the calling conventions for Pascal, which makes a difference in 68k code), typically callbacks, do have it, and will refuse to compile if this is not the case.

If you go as far back as 68k, if I remember correctly int is 16 bit wide in the Mac 68k environment (this is why SInt32 was long up until 64-bit arrived: in __LP64__ mode SInt32 is int), but became 32 bit wide when ppc arrived, so be careful, it’s better not to use int in general.

QuickDraw is, by some aspects, more approachable that Quartz (e.g. no object to keep track of and deallocate at the end), but on the other hand the Carbon transition added some hoops to jump through that makes it harder to just get started with it; for instance something as basic as getting the black pattern, used to ensure your drawing is a flat color, is described in most docs as using the black global variable, but those docs should have been changed for Carbon: with Carbon GetQDGlobalsBlack(&blackPat); must be used to merely get that value. Another aspect which complicates initial understanding is that pre-Carbon you would just directly cast between a WindowPtr, (C)GrafPtr, offscreen GWorldPtr, etc., but when compiling for Carbon you have to use conversion functions, for instance GetWindowPort() to get the port for a given window… but only for some of those conversions, the others just being done with casts, and it is hard to know at a glance which are which.

When it came to packaging, I think I got an app building for classic MacOS relatively easily with MPW, but when I made it link to CarbonLib I got various issues related to the standard C library, in particular the standard streams (stdin, stdout and stderr), and I think I had to download an updated version of some library or some headers before it would work and I could get a single binary that ran both in MacOS 9 and natively on Mac OS X.

Also, while an empty 'carb' resource with ID 0 does work to mark the application as being carbonized and make it run natively on Mac OS X, you are supposed to instead use a 'plst' resource with ID 0 and put in there what you would put in the Info.plist if the app were in a package. Also, it is not safe to use __i386__ to know whether to use framework includes (#include <Carbon/Carbon.h>) or “flat” includes (#include <Carbon.h>); typically you’d use something like WATEVER_USE_FRAMEWORK_INCLUDES, which you then set in your Makefile depending on the target.

Lastly, don’t make the same mistake I originally did: when an API asks for a Handle, it doesn’t just mean a pointer to pointer to something, it means something that was specifically allocated with NewHandle() (possibly indirectly, e.g. with GetResource() and loaded if necessary), so make sure that is what you give it.

I also have a few practical tips for dealing with Macs running ancient system software (be they physical or emulated). Mac OS X removed support for writing to an HFS (as opposed to HFS+) filesystem starting with Mac OS X 10.6, and HFS is the only thing MacOS 8 and earlier can read. However, you can still for instance write pre-made HFS disk images to floppy discs with Disk Utility (and any emulator worth its salt will allow you to mount disk images inside the emulated system), so your best bet is to use a pre-made image to load some essential tools, then if you can, set up a network connection (either real or emulated) and transfer files that way, making sure to encode them in MacBinary before transfer (which I generally prefer to BinHex); unless you know the transfer method is Mac-friendly the whole way, always decode from MacBinary as the last step, directly from the target. Alternately, you can keep around a Mac running Leopard around to directly write to HFS floppies, as I do.

Okay, exercise time.

If you are cheap, you could get away with only providing a 68k build and a Mac OS X Intel build (except neither of these can run on Leopard running on PowerPC…). So the exercise is to, on the contrary, successfully build the same code (modulo #ifdefs, etc.) for 68k, CFM-PPC linking to InterfaceLib, CFM-PPC linking to CarbonLib, Mach-o Intel, Mach-o 64-bit PPC, and Mach-o 64-bit Intel (a Cocoa UI will be tolerated for those two) for optimal performance everywhere (ARM being excluded here, obviously). Bonus points for Mach-o PPC (all three variants) and CFM-68k. More bonus points for gathering all or at least most of those in a single obese package.

Second exercise: figure out the APIs which were present in System 1.0 and are supported in 64-bit on Mac OS X. It’s a short list, but I know for sure it is not empty.

References

Macintosh C Carbon: besides the old Inside Mac books (most of which can still be found here), this is how I learned Carbon programming back in the day.

Gwynne Raskind presents the Mac toolbox for contemporary audiences in two twin articles, reminding you in particular to never neglect error handling, you can’t get away with it when using the toolbox APIs.