Archive for the ‘Operating Systems’ Category

Connecting Mavericks to a Freebox: Oh the pain!

I’m very frustrated. Today, I wasted basically two or three hours fighting unreliable software implementing one of the most basic features in the world of networking, namely file sharing. I ought to be simple, it used to work, but if my experience is representative, it’s complexly broken nowadays. Grenouille bouillie.

What I tried to do is not that complicated. We have a Freebox at home, it’s basically a DSL modem with many features, including the possibility to act as a NAS. So I connected my external drives, and tried to connect to them with my Macs.

It worked. I was happy. I started copying my files around. I noticed that it was very reactive. For example, I could eject a disk from the NAS user interface (a web GUI that is relatively well designed), and instantly, the file server would restart. I saw a message on the Macs saying that the server had shut down, and a couple of seconds later, I was back in business.

Then something happened, and everything stopped working at once.

It’s frustrating, because I know exactly what I did at that moment. I reformatted a disk with the NAS user interface. That disk was initially formatted as HFS+, which the NAS would expose as read only. So I reformatted it as Ext4.

And suddenly, the file server stopped working, even when I removed the disk in question. That leads me to believe I changed something else without knowing it. Maybe the GUI changed some configuration behind my back? I have no idea. All I know is that I spent the last two hours trying to understand how to revert my configuration so that I would be able to share disks again.

Symptoms: I connect to the NAS, and it refuses to show my disks. If I disable Mac sharing (AFP) and only enable PC sharing (SMB), then I can connect to the NAS, but somewhat unreliably.

I suspect the problem is on the Mavericks side of things, because connexions to another NAS I have at home are equally flaky. One minute it transfers dozen of files per second, the next it’s as if I was writing to a floppy disk. Transfers from other protocols (e.g. using a web browser) are fast and reliable, so I don’t think it’s a Wifi or network issue.

How annoying.


Paul Graham recommends doing things that don’t scale

As usual, Paul Graham writes an interesting piece about startups. He recommends doing things that don’t scale. Thinking like a big company is a sure way to fail. It’s a reassuring piece for the startup creator that I am, because at Taodyne, we are indeed in this phase where you do everything yourself and you’d need 48 hours a day to do the basics. Good to know that the solution to this problem is to keep working.

Connect this to the survivor bias. This is a very serious cognitive bias, which makes us look only at the survivors, at the planes who return from combat, at the successful entrepreneurs. Because we don’t look at the dead startups or planes that were shot down, we build our statistics on a biased sample. As a result, we make incorrect assumptions. For example, if the planes that return have mostly been shot in the tail and wings, you might deduce that this is where planes are being shot at, so that’s the parts you need to protect, when in reality what this proves is that these are the parts that don’t prevent a plane from returning when shot. Very useful.

Last interesting link of the day is the discussion about bullying on the Linux Kernel Mailing List (LKML). Sarah Sharp, a female Intel engineer, stands up to Linus Torvalds and asks him to stop verbal abuse. It’s an interesting conflict between very smart people. To me, there’s a lot of cultural difference at play here (one of the main topics of Grenouille Bouillie). For example, I learned from Torvalds what Management by Perkele means. On one side, it’s legitimate for Sarah to explain that she is offended by Linus’ behavior. On the other hand, it’s legitimate for Linus to keep doing what works.

Sarah reminds me of a very good friend of mine and former colleague, Karen Noel, a very sharp engineer who joined me on the HPVM project and taught me everything I forgot about VMS. Like Sarah, Karen was willing to stand up her ground while remaining very polite.

Apple backups and RAID are not reliable

You’d think that if you use RAID1 and multiple redundant, distributed backups with hourly backups, daily backups, etc, you’d be safe? Think again. If your backup software lies to you, you may not realize it until it’s way too late. If you RAID software does not deem it worthy to mention that a disk failed, what good is it?

Read more…

Everything is broken and no one cares

February 10, 2013 1 comment

Everything is broken and no one cares

This post from Dear Apple is just so true, and so clearly on topic for Grenouille Bouillie!

Have we reached the point in complexity where we can’t make good quality products anymore? Or is that some kind of strategic choice?

The original post is mostly about Apple products, but the same is true with Linux, with Android, with Windows.

Here is my own list of additional bugs, focusing on those that can easily be reproduced:

  1. Open a file named X in any of the new Apple applications, those without Save As. Open another file named Y. Save Y as X. Beachball. For every application. Worse yet, since applications often remember which windows were open, you get the beachball again when you reopen the application. It takes another force quit for the application to (fortunately) offer to not reopen the windows.
  2. A relatively well known one now: Type F i l e : / / / in practically any OSX application. Without the spaces. Hang or assert depending on your luck.
  3. Use a stereoscopic application like Tao Presentations ( Activate stereoscopy. Switch spaces or unplug an external monitor. Kernel panic or hang to be expected. Go tell to your customers that the kernel panic is Apple’s fault, not ours…
  4. If you backup over the network, set your computer to sleep after say 1 hour while on power. Change your disk enough that the backup takes more than one hour. Backup disk will come up as corrupt after a couple of days, and OSX will suggest you start a new one (and the cycle will repeat).
  5. Use the “Share” button. It takes forever to show up the window (like 2-3 seconds in general on my 2.6GHz quad-core i7 with 8GB of RAM). Since what I type generally begins with an uppercase letter, I usually prepare myself by having the finger on the shift key. But to that stupid animation framework, “shift” means “slow animation down so that Steve can demo it”. Steve is dead, but the “shift” behavior is still there.

I’ll keep updating this list as more come to mind. Add your own favorite bugs in the comments.

First update (Feb 13, 2013):

  1. Safari often fails to refresh various portions of the screen. Visible in particular when used in combination with Redmine. This used to be very annoying, but it has gotten much better in more recent updates of Safari.
  2. iTunes 11 no longer has Coverflow. It was a neat way to navigate in your music, which wasn’t even the default, why remove it?
  3. Valgrind on OSX 10.8 is completely broken. I have no idea what’s wrong, but it’s a pretty useful tool for developers, and Apple has nothing in its own development tools that is even remotely close.
  4. “Detect displays” is gone, both from the Monitors control panel and from the Monitors menu icon. Combine that with the fact that OSX 10.8, unlike its predecessors, sometimes totally fails to detect that you unplug a monitor. And you find yourself with windows stuck on a screen that is no longer there…
  5. That little Monitor menu icon used to be quite handy, e.g. to select the right resolution when connecting to an external projector for the first time. Now, it’s entirely useless. It only offers mirroring, fails to show up 90% of the time when there is a possibility to do mirroring, shows up when mirroring is impossible (e.g. after you disconnected the projector). It used to be working and useful, it’s now broken and useless. What’s not to love?
  6. Contacts used to have a way for me to format phone numbers the way I like. That’s gone. Now I have to accept the (broken) way it formats all phone numbers for me.
  7. I used to be able to sync between iPhone and Contacts relatively reliably. Now, if there’s a way to remove a phone number, I’ve not found it. Old numbers I removed keep reappearing at the next sync, ensuring that I never know which of the 2, 3 or 4 phone numbers I have is the not dead one.
  8. Still in Contacts, putting Facebook e-mail addresses as the first choice for my contacts? No thanks, it was heinous enough that Facebook replaced all genuine email addresses with aliases. But having that as the first one that pops up is really annoying.
  9. Now fixed, but in the early 10.8, connecting a wired network when I also had Wifi on the same network would not give me higher speed. It would just drop all network connectivity.

Updated February 28th after restoring a machine following a serious problem:

  1. Time machine restores are only good if your target disk is at least as big. But with Apple’s recent move to SSD, this may no longer be affordable to you. In my case, I’d like to squeeze 1TB of data into 512G. Time machine does not give me the level of fine-grained control I’d need to restore what I really need. So I need to try and do it manually, which is a real pain.
  2. Calendar sync is a real mess. Restoring calendars from a backup is worse.
  3. Spaces? Where are my good old spaces? Why is it I had spaces on the original machine, no longer have them, and find myself unable to say “I want 6 spaces” or to setup keyboard shortcuts for them as they used to be.


Is it normal to wait for your computer? Why should I wait 5 seconds when I click on a menu? Why does it sometimes take half a minute to open a new document? Developers, optimize your code, if only as a matter of public service! What about making it a New Year resolution?

Why is my Mac laptop slower than my iPad?

Apple cares about iPad performance

Apple cares about iPad performance

I have a serious issue with the fact that on a laptop with 8G of RAM, 1TB of hard disk, a quad-core 2GHz i7, I spend my time waiting. All the time. For long, horribly annoying pauses.

Just typing these few paragraphs had Safari go into “pause” twice. I type something and it takes ten seconds or so with nothing showing up on screen, and then it catches up. Whaaaaat? How did programmers manage to write code so horribly that a computer with a quad-core 2.6GHz i7 can’t even keep up with my typing? Seriously? The Apple II, with its glorious 1MHz 8-bit 6502 never had trouble keeping up, no matter how fast I typed. Nor did Snow Leopard, for that matter…

Even today, why is it that I always find myself waiting for my Mac as soon as I have 5 to 10 applications open, when a poor iPad always feel responsive even with 20 or 30 applications open at the same time? Aren’t we talking about the same company (Apple)? About the same core operating system (Darwin being the core of both iOS and OSX)? So what’s the difference?

The difference, folks, is optimizations. Code for iOS is tuned, tight, fit. Applications are programmed with severe hardware limitations in mind. The iPad, for instance, is very good at “pausing” applications that you are not using and recalling them quickly when you switch to them. Also, most applications are very careful in their use of resources, in particular memory and storage. Apple definitely cares about the performance of the iPad. There was a time the performance of the Mac mattered as well, but that was a long time ago.

Boiled frog syndrome : we slowly got used to desktops or laptops being slower than tablets, but it’s just plain stupid.

Lion and Mountain Lion are Dog Slow

It's obvious why they called it Lion...

It’s obvious why they called it Lion…

I’ve been running every single version of MacOSX since the Rhapsody days. Up until Snow Leopard, each release was a definite improvement over the previous version. Lion and Mountain Lion, on the other hand, were a severe step backwards…

Lion and Mountain Lion were not just loaded with features I didn’t care about (like crippling my address book with Facebook email addresses), they didn’t just break features I relied on on a daily basis (like full screen applications that works with multiple monitors, or RSS feeds). They were slow.

We are not talking about small-scale slowness here. We are talking about molasses-fed slugs caught in a tar pit, of lag raised to an art form, of junk code piling up at an industrial scale, of inefficiency that makes soviet car design look good in comparison.

And it’s not just me. My wife and my kids keep complaining that “the machine lags”. And it’s been the case with every single machine I “upgraded” to Lion or Mountain Lion. To the point where I’m not upgrading my other machines anymore.

In my experience, the core issue is memory management. OSX Lion and Mountain Lion are much worse than their predecessors at handling multiple programs. On OSX, the primary rule of optimization seems to be “grab 1GB of memory first, ask questions later.” That makes sense if you are alone: RAM is faster than disk, by orders of magnitude, so copying stuff there is a good idea if you use it frequently.

But if you share the RAM with other apps, you may push those other apps away from memory, a process called “paging“. Paging depends very largely on heuristics, and has major impact on performance. Because, you see, RAM is faster than disk, by orders of magnitude. And now, this plays against you.

Here is an example of a heuristic that I believe was introduced in Lion: the OS apparently puts aside programs that you have not been using for a long while. A bit like an iPad, I guess. On the surface, this seems like a good idea. If you are not using them, free some memory for other programs. But this means that if I go away from my laptop and the screen saver kicks in, it will eat all available RAM and push other programs out. When I log back in… I have 3GB of free RAM and a spinning beach ball. Every time. And even if the screensaver does not run, other things like backupd (the backup daemon) or Spotlight surely will use a few gigabytes for, you know, copying files, indexing them, stuff.

Boiled frog syndrome : we slowly got used to programs using thousands of Mac128K worth of memory to do simple things like running a screensaver. It’s preposterous.

Tuning memory management is very hard

Virtual Memory is complicated

Virtual Memory is complicated

The VM subsystem, responsible for memory management, was never particularly good in OSX. I remember a meeting with an Apple executive back in the times OSX was called Rhapsody. Apple engineers were all excited about the new memory management, which was admittedly an improvement over MacOS9.

I told the Apple person I met that I could crash his Mac with 2 minutes at the keyboard, doing only things a normal user could do (i.e. no Terminal…) He laughed at me, gave me his keyboard and refused to even save documents. Foolish, that.

I went to the ancestor of, opened a document, clicked on “Zoom” repeatedly until the zoom factor was about 6400% or so. See, in these times, the application was apparently allocating a buffer for rendering that was growing as you zoomed. The machine crawled to a halt, as it started paging these gigabytes in and out just to draw the preview on the screen. “It’s dead, Jim“, time to reboot with a long, hard and somewhat angry press on the Power button.

That particular problem was fixed, but not the underlying issue, which is a philosophical decision to take control away from users in the name of “simplicity“. OS9 allowed me to say that an App was supposed to use 8M of RAM. OSX does not. I wish I could say: “Screen Saver can use 256M of RAM. If it wants more, have it page to disk, not the other apps.” If there is a way to do that, I have not found it.

Boiled frog syndrome : we have slowly been accustomed by software vendors to give away control. But lack of control is not a feature.

Faster machines are not faster

A 1986 Mac beats a 2007 PC

A 1986 Mac beats a 2007 PC

One issue with poor optimizations is that faster machines, with much faster CPUs, GPUs and hard disks, are not actually faster to perform the tasks the user expects from them, because they are burdened with much higher loads. It’s as if developers always stopped at the limit of what the machine can do.

It actually makes business sense, because you get the most of your machine. But it also means its easy to push the machine right over the edge. And more to the point, an original 1986 Mac Plus will execute programs designed for it faster than a 2007 machine. I bet this would still hold in 2013.

So if you have been brainwashed by “Premature optimization is the root of all evil“, you should forget that. Optimizing is good. Optimize where it matters. Always. Or, as a colleague of mine once put it, “belated pessimization is the leaf of no good.”

Boiled frog syndrome : we have slowly been accustomed to our machines running inefficient code. But inefficiency is not law of nature. Actually, in the natural world, inefficiency gets you killed. So…


Apple started decaying before Steve Jobs’ death

Sad MacMany have wondered what would happen to Apple after Steve Job’s death. I’m afraid things started to go south at Infinite Loop long before Steve passed away. Case in point: Mac OSX Lion, which I think is the worst version of MacOSX ever (and I’ve used all of them since Rhapsody).

What’s wrong with MacOSX Lion?

While there are a number of relatively useful features in OSX Lion, like being able to resize a window from all sides (granted, not exactly a new feature in the computer world), the general philosophy of that OS seems to be “We know better“. A computer company that thinks it knows better than me how I should use my own computer? Let’s not get used to it.

I will illustrate this with three real-life cases:

  • Ten minutes to reboot on a Core i7 laptop is not cool.
  • How I came to positively hate the mandatory auto-save feature.
  • The sad story of Quit, Select All, Undo and Close Window.

There are a few other smaller cases that I will brush on quickly at the end. Like the broken replacement for good ol’ Save-As, the stereoscopy crashes, the mysterious unimprovements to Spaces, the Screen Saver of Doom…

Ten minutes to reboot is not cool

Many know the great story of Steve Jobs telling an early Mac engineer that making Macs boot faster would save lives. This lesson seems to have been forgotten these days.

This morning, I had a kernel panic in Lion (a not so uncommon occurrence, sadly). So I was forced to reboot. And what happened next prompted me to write this blog entry. Crashing is enough of a waste of time. But then, MacOS X Lion aggravated that by reloading every single tiny window I happened to have open at the time of the crash. And not letting me do anything in the meantime, because you see, it was busy, it had better things to do than even letting me quit an application.

Being able to quit an application is what I took as an indication that the system was done booting. It’s as good a measure as any, since if you can’t quit an application, you can’t do much else. And it took more than 10 minutes for me to be able to quit Firefox: I booted the machine at 7:02 (according to uptime), Firefox accepted to quit at 7:14.

In the meantime, OSX Lion had reloaded, for my own good:

  1. Mail, with 7 windows
  2. Pages, with 4 documents
  3. Numbers, with 6 documents
  4. Keynote, without any document open, but hey, what’s wrong with launching it anyway?
  5. Terminal, with 2 windows, one of them was running a build. There’s a severe bug in OSX Lion restore-everything-at-reboot-time functionality: it didn’t restart my build!
  6. Safari, with 8 web sites, including two with videos I had already seen.
  7. Firefox, with 2 web sites, which I certainly didn’t wan tot re-open since they were payments.
  8. iTunes (which helpfully started downloading new contents)

While the system attempts to “please” you in some demented sense of “pleasing”, there’s very little you can do but wait. Actually, you need to do a little more than that, because the machine will occasionally ask for passwords or pop up some dialog box. And it does so in such a random fashion that even reading mail is difficult. All the more so because the machine is so busy re-indexing its Spotlight database and downloading iTunes contents you really don’t care about right now that everything crawls.

Even switching windows is difficult, even borderline hazardous. You think you brought up one window, but then the system shows another one right at the moment you click or close something or do something dangerous, and bam, the one and only window you didn’t want to close vanishes from the screen!

What is so infuriating about this incredibly stupid behavior of OSX Lion is that practically every single time I rebooted my machine, I unchecked that little box asking if I want to re-open my windows when I log back in. Can’t OSX Lion get the hint? If it’s smart enough to save my windows at the time of a kernel panic (of all times), can’t it save a little preference like “I don’t want you to re-open windows at boot time, ever”, without forcing me to resort to command line hacks.

Yes, I know how to fix it. It’s a script like this one:

echo "#!/bin/bash" > /tmp/
echo "rm /Users/*/Library/Preferences/ByHost/*" >> /tmp/
mv /tmp/ /usr/bin/
chmod +x /usr/bin/
defaults write LoginHook /usr/bin/

Having to resort to something like is really annoying. And if Apple really didn’t want to store the “reopen windows” user choice in preferences, then it shouldn’t be a check-box. It should have been a separate action button, just like “Shutdown” and “Restart”.

I’m clearly not the only one who dislikes that features. If you look up on the web, you’d be hard pressed to find any site that explains how great that feature is. Instead, you’ll find dozen of places telling you how to disable it. So it’s a useless feature compounded with a bad UI made more annoying by a blatant disrespect for user preferences. That seems to be the general theme for changes in OSX Lion. Let’s not get used to it.

How I came to hate the mandatory auto-save feature

Another feature that follows the exact same pattern is the mandatory auto-save feature in applications like Pages, Numbers, Keynote, etc. What this feature does looks good on paper. It helpfully saves things for you at regular interval. Since Apple implemented for Time Machine a relatively nice way to version files, Apple used that to offer a kind of per-document Time Machine. Isn’t that a great idea?

The problem with that auto-save is that it doesn’t scale, and that there is no way to turn it off, even temporarily. So here is what happened to me once. I was animating an event, and for some reason, they decided to use my laptop as the main machine connected to the projector. So they gave me this 150 pages Keynote made by copy-pasting together a dozen or so slide decks. So far, so good.

Then, various people started coming to me asking if they could change a word here, copy a new slide there, etc. Guess what: it took over one minute to save the 150 pages document. My guess is that Keynote uses a brain-dead algorithm to decide when to auto-save, something like “if something changed and if the last auto-stave started more than 30 seconds ago”. Just a wild guess. What I observed, though, is that it doesn’t check that after you do an operation, but before executing the next operation you request.

All these harebrained design decisions blend together in a perfectly distasteful mix. You hit a key. Keynote shows your keypress. You hit another key. Keynote detects it should auto-save. The save takes more than one minute. The “saving takes a long time” progress dialog shows up and eats the key you typed! So you need to type again, very fast. But usually you don’t succeed. Same with mouse clicks. You send mouse clicks that get eaten by the stupid “Please wait while I’m saving” progress dialog. Who decided to call this a “progress” dialog? It’s not progress!

Anyway, after a very painful 15 minutes trying to make this work, here is how I ended up doing things: I started another Keynote instance, edited slides one at a time there, and once it was done, I would copy things back in the original Keynote document. But talk about a counter-productive exercise fighting a badly designed UI that won’t accept my preferences (namely: I don’t want to auto-save. Period. I hit Command-S when I’m happy. I know what I’m doing.)

Quit, Select All, Close and Undo

The third scenario I came to hate in OSX Lion is much more specific. See, I’m French, so I often have to type French text. And I’m a coder, so I often have to write English text or computer code. At some point in my past, I started taking the habit of typing French text in the native French keyboard, which is AZERTY, and English text in QWERTY

I don’t think many other people do this, but to me, that means faster typing in both cases. In English, I can type all the wonderful special characters used in code, like [ and ]. In French, I have easy access to all the wonderful àccénts that pepper our language (and, by the way, in defense of Jean Dujardin, “Putain” is more like an accent than a swear word in French; it doesn’t really mean “Whore” anymore than, say, “OK” means “all correct” or “gay” means “in a good mood”). A and Qs flip automatically when I’m typing French. If anything, that shows how flexible the human brain is.

There’s just one little problem with that clever scheme: keyboard shortcuts. It so happens that Command-Q (Quit), Command-A (Select All), Command-Z (Undo) and Command-W (Close window) are some of the most frequently used shortcuts of all. And unfortunately, they flip places when I switch languages. And unlike complete words, which my brain has “short-circuited” to the correct keyboard layout depending on language, they have no context, no language associated with them. So often, I want to Undo, and instead I Close the Window.

So what does this have to do with OSX Lion, you may ask? Well, the auto-save feature has, for me, a very nasty side effect. If a document has been modified, OSX Lion no longer asks if you want to close it or save it. That dialog box that used to pop was my saving grace in the old days. If I hit Command-W instead of Command-Z, then the dialog box would pop up, I’d hit ESC and hit Command-Z. No harm done. Nowadays, my window vanishes, and with it, all my undo history. In other words, at the exact time I want to undo something, MacOSX Lion finds a way to erase the entire undo history!

Of course, Apple engineers know better. Their reasoning must have been that the auto-save feature is like a kind of persistent undo to disk. Unfortunately, auto-save and Time Machine are nothing like undo. Undo remembers the actions. Auto-save remembers the result. So Time Machine is way way slower to activate, and it makes it harder to find where a document actually changed.

So a couple of times a day, I close a document by mistake, and I swear. Nothing has been closer to turining my Macbook into some unidentified flying object than this feature. At some point, I’ll find the time to disable that auto-save feature as well, in a way that doesn’t break other things.

Then, there’s the small stuff

OSX Lion’s disregard for users’ taste and preferences permeates throughout. It looks more like a design philosophy than an accident.

There’s the odd reversal of the trackpad scrolling behavior. This one at least can be configured. But either you stick with the default, and you are backwards each time you return to a Windows, Linux, older MacOSX machine. Or you change it, and you are backwards each time you return to a Lion box that is configured with the default.

There’s the new technique for “Save As”. It used to be “Save As”, Command-Shift-S, select the new file name (defaults to the document’s directory). Basically, one keystroke. Now, it is “Duplicate” (no keyboard shortcut), close the old window, Save (which now defaults to the Desktop rather than where the original document was), find the original location, save. So you have replaced one keyboards shortcut with 4 to a good dozen clicks depending on where your original document was. And “Duplicate” seems to use more memory and take more time than Save As (maybe it saves the document somewhere?) The benefits? Hmmm. I don’t see any, it seems like a less intelligent way to do the same thing as before.

Stereoscopy is a minor nuisance to the majority of people. But it turns out Taodyne, my company, produces a 3D presentation software. Something like Flash blending with Avatar. One of the ways we generate stereoscopic images is with the OpenGL Quad buffer support. It was broken in 10.6.1: when our application ran, switching spaces would kill the window server. I reported it to Apple, it was fixed in 10.6.2. It was broken again in Lion (10.7.0). Only this time the crash is a random kernel panic or system freeze, a bit more serious. I reported it three times to Apple. It’s still there.

Overall, Spaces and the Window server are nowhere as good as they used to be. When you switch spaces, it used to be smooth. You used to have a single desktop background. Now, it’s not smooth. Sometimes, windows won’t drag from one space to another. Spaces can be “out of order” (i.e. the number keyboard shortcuts no longer correspond to the logical layout of the spaces). The windows that MacOSX Lion insists on reopening at startup don’t reopen in their original space. And so on.

It’s more than just spaces. For example, using full screen app mode on dual-screen setups makes one of the monitors become unusable (covered with a oh-so-nice Lionesque background). Try watching a DVD full screen on one monitor while you do something on the other. Worked like a charm in Snow Leopard, impossible to do in Lion. Try putting the DVD player full screen on your TV while the menu is on the main screen of your laptop (just because most HD TVs overscan, so putting the menu there means you don’t see it). Nope, Lion knows better, it will bring the full screen picture where the menu is, not when the window is. That also worked in Snow Leopard.

And there’s another one of my favorites, the Screen Saver of Doom. Some screen savers in OSX Lion consume all the memory they can. They evict all the useful stuff away. So when it’s time to log back in, you see your keystrokes show up. one. at. a. time. And by the time you logged back in, you see that 5G of your 8G of RAM are now free. For the next few minutes, every single application you will try to use will page stuff back in and be extremely unresponsive. That too is a nuisance. Let’s not get used to it.


OSX Snow Leopard was a lean and mean operating system, just like its feline counterpart. OSX Lion takes on its role model as well: it’s big, heavy, slow, lazy, and it doesn’t care about you a tiny bit.

Let’s just hope that Apple’s next OSX version, Mountain Lion, will not be smaller, less powerful but more dangerous than its predecessor… Unfortunately, I won’t hold my breath based on what we can gather from Apple’s sneak peek Making it more difficult to download third-party applications in the name of security? A chat application? Twitter support? A features page that has so little to show that it needs to boast on the US web site about stuff designed specifically for China? Seriously?

I still want to believe that Apple will soon focus again on making their OS lean, mean and efficient like it used to be.

Stereoscopy: What works, what doesn’t

In the past months, we have been looking for a relatively cheap way to present the output of a 3D application using stereoscopy. Our objective was to see how an application could generate stereoscopic output using commercially-available low-cost hardware. Our expectation was that a budget 3D laptop and a budget 3D projector would make for a budget 3D presentation solution.

The solution that ended up working was a total surprise to us, so I thought I’d share…

Budget 3D laptop and projector

We recently purchased two pieces of equipment with a “3D” sticker on them. One was an ACER Aspire 5740D laptop, the other was an Optoma DW318 projector. Both can be considered entry level hardware, around $700 each with glasses.

This family of Acer laptops uses a polarized display and passive glasses. Even and odd scan-lines are polarized differently, meaning that you get half the vertical resolution when using stereoscopy. Native resolution is 1366×768. It comes equipped with an ATI Radeon 4570 and some 3D software from TriDef. It is also covered with at least 5 stickers and littered with various nagware (software based on the “we’ll bug you until you pay” school of design) and other software nuisances, almost guaranteeing a supremely annoying unpacking experience.

The Optoma is labelled as “3D-ready”. It uses active glasses based on Texas Instrument’s DLP-Link technology. Native resolution is 1280×800. The active glasses mean that you get the full resolution with 120Hz frame-sequential input, with only a barely noticeable decrease in frame rate (60Hz for each eye). It can also reportedly take 60Hz field-sequential input up to 480i, but we didn’t test that. There is practically no manual (at least as far as 3D is concerned), and the manual that came with the glasses didn’t help much either (I’m still not sure I know how to switch the glasses off).

No OpenGL stereoscopy on Windows

Our test application uses standard OpenGL quad-buffer support. We need OpenGL because we want the application to run on Macs and Linux machines as well, not just Windows. So DirectX is a no-go. However, we also evaluated how things worked with DirectX just in case we had no other choice.

With the Acer, the ATI “Catalyst” drivers simply tell us that there is no OpenGL quad buffer support. As soon as we try to use the buffers, we get an error (1282: Invalid operation). This is a disappointment, as quad-buffer support for DirectX was one of the advertised features for recent Catalyst drivers. Apparently, this only means that a third-party driver, either from iZ3D or TriDef, can silently convert non-stereoscopic games into stereoscopic games.

It makes sense from a business point of view, since all the majority of users will care about is 3D games, which mostly use DirectX nowadays. But if you want to programmatically produce stereoscopic 3D-accelerated output using OpenGL, which should theoretically be simpler, it just doesn’t work out of the box.

I spent a little bit of time trying to find drivers from iZ3D or TriDef that would support OpenGL quad-buffer on this machine. No such luck.

  • The iZ3D setup page prominently shows “OpenGL QB driver for iZ3D”. However, when I installed the driver, it didn’t work for me. Only DirectX would show up. It took me a while to see this little note somewhere:

Note! OpenGL QB is for 32-bit systems only and can not be run properly on 64-bit systems.

You guessed it: my system is 64-bit, so no OpenGL for me.

  • The experience with the TriDef setup was exactly similar. At first, it looks promising: “The DDD TriDef Visualizer Program is ideal for anyone interested in enabling their application for output to stereoscopic displays.” That is, until you click on the link to get the SDK, only to be told:

PLEASE NOTE: TriDef Visualizer OpenGL SDK has been RETIRED and is no longer available for sale.

If someone at Microsoft has been tasked with the job of killing OpenGL on Windows, it looks like they have been doing a rather good job. In any case, we were unable to find any combination of drivers on 64-bit Windows that would allow us to use 3D stereoscopy from OpenGL. If you know of any combination that works, please comment here.

Working around lackluster OpenGL support

The structure of the screen on the Acer is really simple. Every other line on the screen is polarized differently. So in order to produce a stereoscopic image, you don’t really need a driver. All you need is the good old OpenGL stencil buffer, as explained here. You then render one eye with all even lines masked out, and then another eye with all odd lines masked out.

This worked well, and we were soon able to get some basic stereoscopy working. It only took a couple of hours of coding. I only wish this coding had not been necessary on a machine that came littered with 3D stickers and pictures of 3D pirates on the box!

Then, we could start experimenting with stereoscopic rendering and judge by ourselves how the effect played out. Unfortunately, while it worked to some extent, it was not entirely satisfying…

The drawbacks of interlaced stereoscopy

The most annoying issue with this setup, as we quickly discovered, is that you lose half of the vertical resolution. It may not seem like much, but this actually makes text totally unreadable. “Who needs text in a stereoscopic display?“, you may wonder. Well, Google Earth, for one, uses a lot of text. And this is practically the only non-game application I managed to get working on the Acer with the built-in software.

The problem is that the Windows user interface itself uses a fair amount of text and small drawings. Since the glasses actually block every other line, here is what your desktop looks like when you wear the glasses:

Windows 7 desktop, as seen through passive glasses

Windows 7 desktop, as seen through passive glasses

The effect is actually much more annoying than that, because what your left eye and your right eye see is different. I just can’t stand looking at my desktop like this for long. So what happens is that you spend your time putting the glasses on, then removing them, then putting them back on. After a short while, the game stops being funny and you wish you had bought active glasses instead. This probably doesn’t impact gamers much, but in our case, it is almost a deal breaker…

No stereoscopy on Windows with a low-cost projector

In order to address this issue, our next experiment was with the Optoma DW318 projector. I got this projector from Saturn at a bargain price. The vendor was actually quite honest that they had not tried 3D and didn’t know if it worked. Also, this projector is marketed as “3D-ready”, so I wasn’t entirely sure that we’d get anywhere. But we naively hoped that if we connected a “3D-ready” projector to a laptop that has stereoscopic 3D support as its main selling point, we stood a decent change of getting a stereoscopic image on the projector.

Boy! Were we naive last week!

If the built-in stereoscopic software on the Acer laptop has any kind of support for the Optoma projector, it is rather well hidden. I tried various options, but as far as I can tell, stereoscopy on the Acer is meant to be restricted to the built-in screen. That’s rather odd! What would you think of a color laptop than can only supply black-and-white pictures to external displays?

I tried supplementing the anemic built-in software. Downloading drivers from iZ3D, I was able to get a 3D image from their test and setup application. But still, I was not able to get a stereoscopic image from Google maps on the projector, however, something that works on the built-in screen of the Acer laptop.

The overall feeling is that this stuff was rushed out of the door before it’s really ready.

The Good Surprise: on a Mac, It Just Works!

We also have Apple Macbook Pro notebooks and Linux laptops and virtual machines. During testing, Linux didn’t do much better than Windows. However, OpenGL on the Mac accepted quad-buffer (stereoscopic) mode. Compared to Windows, the primary difference was that we did not get the 0×502 = 1282 error (OpenGL invalid operation) when selecting the back-left or back-right buffers. Tweaking a bit, we had gotten what looked like a stereoscopic output on the built-in display of the Mac laptops or on external displays, with fuzzy blurry images.

Of course, without glasses, that’s all these were: fuzzy blurry images, not that useful unless you can blink really really fast.

Actually, everything was not all smooth and fuzzy at first. OpenGL was apparently doing something smart there, but we were clearly pushing it a bit. For example, if running an OpenGL application under the debugger and putting the laptop to sleep, MacOSX would die a little too often for my taste (like: almost every single time). Another interesting issue was that if you had any stereoscopic application running, switching desktops with Spaces would just kill the window server (the MacOSX blue screen of death). That was really annoying. I filed a couple of bug reports with Apple. Stereoscopy was apparently working to some extent, but it still made for a very un-Apple dangerous user experience.

That was last month, when MacOSX 10.6.4 was all the rage. Imagine my surprise when 10.6.5 came out. All the issues I had were fixed with 10.6.5. And this certainly took some work from Apple engineers. Now, when you switch spaces with a stereoscopic application (and only a stereoscopic application), something happens that you can notice visually, almost as if some part of the window server was rebooted or something. It’s just weird, but at least, it’s just solid now. Kudos, Apple for fixing something that could easily have passed for a corner case with exactly one user in the world.

Anyway, I figured that it was worth trying to hook a Macbook Pro to the projector and see what we got. And IT JUST WORKED! We got perfect, high quality stereoscopic images the first time we tried.

Or almost.

How can you make glasses complicated?

Clearly, I had two pictures on the screen. And when I wore the glasses, I had only one picture. So something was working. But the result was still disappointing. It looked boringly flat, nothing like what I expected.

I tried several settings on the projector, checked that 3D was enabled in the firmware menu. I adjusted the eye distance to try to increase the depth. The effect of increasing eye distance was clearly visible with the naked eye, as the left and right pictures became more distant from one another. But with the glasses, still no feeling of depth.

Until I tried to switch off the glasses while wearing them, using the little red button. And I noticed that the picture moved. It was still flat, but it had moved in front of me. Hmmm?!?

So I tried again to see if the picture would move again. And I almost fell on my back. The effect was intense, in no small part because I had pushed the eye distance so high. A few adjustments later, I was back to the original setting, and I had true, breathtaking stereoscopy.

Guess what: some bozo working for Optoma or whoever designed the glasses thought it was a good idea to have three settings: left eye only, right eye only and stereoscopic. I’m sure there’s a good reason for that, but as far as I can tell, the manual only talks about on and off settings.

Trying Windows on a Macintosh

Once we knew that the Macs could drive the Optoma projctor in stereoscopic mode, we thought we also had a Windows solution. See, all recent Macs can run Windows using Bootcamp. We figured that it was a simple matter of switching to Windows, installing the latest Nvidia drivers, and boom.

Well, not quite.

First, Nvidia won’t let us install drivers on a Mac. We are supposed to ask Apple. Of course, there are workarounds. So quickly, we were able to get the latest drivers to install on the Mac. These drivers that are ostensibly designed for stereoscopy. But it still doesn’t work with the Optoma projector.

See, Nvidia has their own little 3D project called 3Dvision. You’d think that since 3Dvision is about supporting stereoscopy, we stood a chance of making it work with an Optoma stereoscopic projector.

And you’d be wrong. It doesn’t work. With OpenGL, you get the same dreaded error 1282.

According to some posts on the Internet, it used to work relatively well with much older Nvidia drivers and cards. But apparently, Nvidia is now trying to leverage their position as a key provider of graphics chips to force third parties to “certify” projectors. The Optoma HD67 is certified and reportedly recognized by 3Dvision drivers. The DW318 is not certified, and therefore not recognized.

If this is a trick to try and force me to buy Nvidia glasses, it’s not just a cheap trick, it’s a stupid trick. The Nvidia solution uses a small infrared emitter. If I’m projecting on a large screen, I doubt this emitter would be good enough to cover a medium-sized room. I suspect that the technique used in DLP-Link, which uses data sent on the screen by the projector, is much better suited to sharing with a large audience.

Anyway, the very same Optoma projector just works on the very same Macbook pro when running MacOSX instead of Windows 7. So this is clearly a software block from Nvidia rather than some subtle hardware limitation. If you are the Nvidia or Microsoft bozo who is responsible for this fiasco, here is a hint: why don’t you get it fixed?

Conclusion: Don’t Trust Marketing

To summarize my experience:

  • The Acer 5740DG offers some basic stereoscopy, but support for OpenGL is underwhelming. OpenGL stereoscopy won’t work with the pre-installed 64-bit version of Windows 7. This Acer laptop is designed primarily for games, although it may be a bit underpowered for that purpose. For any windowed 3D application, or any application that shows text, dialog boxes or other small elements, the stereoscopic experience is highly frustrating because you don’t see half of the screen while wearing the glasses.
  • The Optoma DW318 is a very capable 3D projector if you can make it work in your configuration. The picture is actually much better with the glasses in stereoscopy mode, with rich deep blacks and well balanced colors. Without the glasses, the picture looks a bit washed out. Still, it’s a decent value, and a good way to demo stereoscopy to more than one person. I only wish they didn’t charge $100 or so for a pair of cheap plastic glasses with $0.0027 worth of electronics components in them.
  • Nvidia and ATI are both equally incompetent at delivering OpenGL stereoscopy on Windows 7. Part of it may be Microsoft pushing DirectX, but I believe the driver vendors are equally to blame. I couldn’t get OpenGL stereoscopy to work whether with ATI or Nvidia, whether with built-in screens or external projectors.
  • MacOSX 10.6.5 has full support for OpenGL stereoscopy, and when connected to a 3D-ready projector like the Optoma DW318, it just works the way it should. This is advertised nowhere, which makes it even sweeter.

In short, those who talk the most about 3D (Nvidia, Acer) are those who deliver the least. By contrast, Apple, who doesn’t say a word about stereoscopy, delivers flawless OpenGL stereoscopy support as far as I can tell. And Optoma, who labels its DW318 with a rather understated “3D-ready”, has made the process of projecting in 3D both inexpensive and painless.

From language to platform…

September 19, 2010 10 comments

Reading this article comparing applications platform, I couldn’t help but think: “nothing new under the sun”.

History is repeating itself

In the 1980′s, a war raged between programmers. The reason for the war? Which programming language to use for applications. There were many candidates, compiled languages such as C or Pascal, interpreted languages such as BASIC, and “bizarre” languages such as Lisp, Prolog or Smalltalk. For all the uncertainty, things were not really open for discussion: each side knew they were absolutely right. Total war! Example: Turbo Pascal vs. QuickC.

In the 1990′s, a war raged between programmers. The reason for the war? Which operating system to use for applications. There were many candidates, text-based DOS, elegant or copycat graphical user interfaces such as Macintosh or Windows, as well as “bizarre” server operating systems such as Unix, Novell Netware or Linux. For all the uncertainty, things were not really open for discussion: each side knew they were right, and they knew their side had to win for them to make money. Total war! Example: Apple suing Microsoft over the Graphical User Interface.

In the 2000′s, a war raged between programmers. The reason for the war? Which execution environment to use for application? There were many candidates: client side native environments such as Windows or MacOS, browser-based virtual machines such as Java, as well as bizarre models such as VMware-style virtual machines, massively distributed systems (aka “the cloud”) or web services. For all the uncertainty, things were not really open for discussion. Each side preferred their model, and they knew that the financial stakes were high. Total war! Example: Sun suing Microsoft repeatedly over Java.

In the 2010, a war rages between programmers. The reason for the war? Which platform to use for applications. There are many candidates: desktop clients that won’t die, the iOS and Android mobile platforms, as well as “bizarre” platforms such as Facebook, Google Wave, Twitter, even good old TVs. For all the uncertainty, things were not really open for discussion. Each side has to be compatible and integrate with everybody else, but the financial stakes are mind-boggling. Total war! Example: the iTunes Ping – Facebook integration debacle.

The open platform wins

Of course, one can draw many conclusions from this parallel. For example, that programmers didn’t really take to heart the Make Love, Not War slogan, and instead would rather rally to “make war, not love” cries. Cheap shot! Cheap shot!

More seriously, I find it more interesting to observe who won over time. And in each case, it looks to me like the “more open” side won:

  • Pascal was criticized as being too closed compared to C. Ultimately, C won over Pascal. As an aside, the article “Why Pascal is not my favourite programming language” seems hard to find on Google these days. I hope it’s not slowly disappearing from the Internet…
  • The Macintosh was perceived as less open than Windows. Ultimately, Windows won over the Macintosh.
  • Sun positioned Java as “write once, run anywhere”, making it the open alternative to closed systems such as Windows or Macintosh. Customers came to associate distributed systems with freedom. Perception matters here, because if you think about it, you own Google Docs contents much less than documents on your hard disk. Still, the cloud and distributed system came out as the winners.
  • Finally, there is a strong perception that the iPhone is closed and Android is open. There are enough privacy concerns about how the “closed” Facebook to spur an open-source alternative, Diaspora.

Features are less important

The result that the more open approach wins in the long run is not entirely intuitive. Naively, one is tempted to believe that platform control is important, that features and quality will help secure a position in the long run. And often, these things do wonders at the beginning.

But look who had the head start and the better features:

  • Pascal was published slightly before C (1970 vs. 1972), and most people at the time found it cleaner.
  • The Macintosh operating system appeared before Windows, and at least until Windows 95, was significantly more polished, on the user experience front as well as on the features it offered programmers.
  • Native operating systems had more features and higher performance than Java-style or VMware-style virtual machines.

The same holds true for the internal architecture, for the overall design, for the ease of integration, even for the pace at which things changed. Arguably, for example, Pascal changed more rapidly than C, Turbo Pascal introducing for example instant compilation, modules, graphics and what we would call an IDE today.

Keep it open!

If you build a system for developers today, you can make short term wins if you design it well, make it easy to extend, and secure a few control points. In the long run, however, what matters is whether your system will be perceived as “open”.

LeMag IT article about HPVM

Sorry for my non-French readers, here is an article from LeMag IT about HPVM, who several colleagues forwarded me.

This is also probably the first time someone links to the Taodyne web site.

Welcome to the readers of LeMag IT who decided to follow the link to my blog…

HP DE200C Digital Entertainment Center

I finally took some time to put back together my good old trusted HP DE200C Digital Entertainment Center, and I thought I would share the result:

In my opinion, this is a good illustration of the kind of innovation open-source brings. The software stack is pure free and open-source software. That allowed me to add my two cents contribution (driver for the Vacuum Fluorescent Display, support for the remote-control and front-panel keys).

But equally important, it shows the kind of innovative hardware that Hewlett-Packard was putting together back in 2001, seven years ago now… Kudos to Lee Devlin and other people who gave us this great toy.

The source code for my little contributions can be found at the Noritake VFD driver, the LCDproc changes and the Linux kernel changes. I had some trouble pushing the git repository for the last one, so here is a patch against 2.6.26, which may be easier for most.


Get every new post delivered to your Inbox.

Join 365 other followers

%d bloggers like this: