Grenouille Bouillie

Switching to Wordpress?

2010-07-25T16:11:00.002+01:00

As my post yesterday alluded to, I'm switching to Wordpress (at least for the time being). The entire blog has been transferred over there thanks to the import mechanism.

I invite my reader(s) to try the new site and tell me what they think. I believe that for users, Wordpress is a big win, but your opinion matters. Please leave comments on either side, I will be monitoring both. I will, however, be posting mostly on the Wordpress side.

Hello World - Switching to Wordpress?

2010-07-24T16:04:00.004+01:00

This is a copy of my first blog post created on Wordpress

Following a comment I made regarding the controversy about the applicability to Wordpress themes of GNU GPL (the license under which Wordpress is distributed), I was invited to switch to Wordpress by its creator. It's hard to say no... So I created a Wordpress account and a blog. Wordpress creates a first blog entry for you. The title of this first post is "Hello World!". For programmers, "Hello World" is a very special thing. It is this small little program that prints "Hello World" on the screen. In C, it looks like this:

#include <stdio.h>
int main(int argc, char *argv[])
{
    printf("Hello world!\n");
}

The output of this wonderful little program on the console is the following:

Hello world!

To achieve the same result with my own programming language, XL, you'd write something like:

use XL.UI.CONSOLE
WriteLn "Hello world!"

The Hello World program is a small benchmark of what a programming language can do. For example, the team behind Google's Go programming language used Hello World to highlight their support for internationalization.

A 3D "Hello World!" in 23 lines of code...

So with such an exciting blog post title, it's only fair that I would use this post to show what Hello World looks like in the stuff we are developing at Taodyne:

The video is a bit skippy, but it looks smoother in real life. Apparently, there's something wrong in the combination of grabbing the screen with Snapz Pro X, creating a movie with iMovie and publishing on YouTube. Don't ask me why, I'm just the computer guy...

The XL axioms: reconciling Lisp and C++ hackers

2010-06-19T23:19:00.006+01:00

Re-reading Paul Graham, specifically The Hundred-Year language helped me understand that I need to write down the axioms of XL. I need to describe XL's equivalent of lambda-calculus for Lisp. If there is such a thing.

Lisp inventions

In Revenge of the Nerds, Paul Graham makes a list of the 9 things that Lisp invented

Conditionals
Function types (i.e. functions as first-class items)
Recursion
Dynamic typing
Garbage collection
Programs composed of expressions (no statements)
A symbol type
A notation for code using trees of symbols and constants (programs as data).
The whole language there all the time (no distinction between load time, compile time and run time).

Of these, many became part of mainstream languages over time. Some remain the exception rather than the norm.

What makes XL unique

Later in that same essay, however, Paul Graham unknowingly explains that makes XL unique in my opinion (emphasis mine):

Macros (in the Lisp sense) are still, as far as I know, unique to Lisp. This is partly because in order to have macros you probably have to make your language look as strange as Lisp. It may also be because if you do add that final increment of power, you can no longer claim to have invented a new language, but only a new dialect of Lisp.

What XL demonstrates is precisely that you can have Lisp-style macros (and in fact build a whole language on macros) without the hairy syntax. Here is how you use 'if then else' in XL (whether the imperative variant XL1 or the functionalish variant XLR):

    if A < 0 then
        write "A is negative"
    else
        write "A is positive or null"

This code should look really familiar to users of other languages, I would hope. Yet under the hood, it corresponds exactly to a simple parse tree similar to what Paul Graham refers to in his point #6. Here is what that parse tree will look like if you expand it:

% ./xl -parse ifthenelse.xl -style debug -show
(infix else
 (infix then
  (prefix
   if
   (infix <
    A
    0))
  (block indent
   (prefix
    write
    "A is negative")))
 (block indent
  (prefix
   write
   "A is positive or null")))

The big difference with Lisp is that Lisp, as Paul Graham points it out, has no parser. XL has a simple parser, but a parser nonetheless.

Axiom #1: A simple parse tree improves code readability

This leads to axiom #1 of XL: you can parse text with 8 node types or less. The 8 node types currently found in XLR are:

Integer: Numbers like 1234
Real: Numbers like 3.1415
Text: Literals such as "ABC" or 'X'
Names and symbols: Names contain letters and digits, like HelloWorld or X0, symbols contain other characters such as ; or ->
Prefix: Adjacent nodes where the first one defines the operation, such as sin x or -3.
Postfix: Adjacent nodes where the second one defines the operation, such as 3! (factorial of 3) or 24in (24 inches)
Infix: Adjacent nodes separated by a name or symbol which defines the operation, such as 3+4 or X and Y
Block: A node surrounded by separators, such as (24-3) or the second half of the prefix A[3].

This can be reduced to less than 8 relatively easily: the real and integer types could be merged into a "number" type, the prefix and postfix could be merged into a "positional" operator, which could be made more general, for instance allowing "above" and "below" notations.

In practice however, the choice currently implemented in XL is a practical trade-off. For instance, the natural computer representation of integers and real numbers are different, so it makes sense to treat them as separate from the beginning.

Of note, in XL, indentation is represented by a block, and line separators are just another kind of infix.

The benefit of this representation compared to the Lisp representation is that it is only marginally more complex (4 constructed node types instead of one, the list), but allows us to parse things much closer to the way humans do. For instance, XL will parse 2+3*6 the right way.

Axiom #2: Programs can be evaluated with macros

Despite what Paul Graham says, Lisp really has two "modes" of operation: macros/compile-time and evaluation. The distinction is subtle: Macros (and compilers) evaluate one program to transform another program, whereas evaluation evaluates the program itself.

In the XL1 compiler (the XL imperative language), the distinction is more marked than in Lisp, whereas in XLR (the XL runtime language), it is less clearly visible. Let me illustrate with an example of each.
In XL1, you would define a recursive factorial as follows:

function Factorial(N : integer) return integer written N! is
    if N = 0 then
        return 1
    else
        return N * (N-1)!

That example looks really similar to what you would write in C or Ada or whatever other imperative language you can think of. But the parse tree is there, not too far. One example is the notation written N!. This allows the compiler to recognize that (N-1)! is really a call to Factorial. This is called expression reduction. This is already quite useful, but basically it's a compile-time only thing.

Under the hood, however, the XL1 compiler uses the exact same mechanism to recognize (N-1)! and to recognize if-then-else. The exact same macro rewrite mechanism is at play. In fact, you can extend the compiler easily with your own "macros", so that for example you could write d/dx(sin x) and have the compiler rewrite it for you as cos x. This particular differentiation plug-in has been in the compiler for several years now.

In short, the language looks imperative, but it's implemented as a big set of macros.

By contrast, in XLR, you would define the same function as:

0! -> 1
N! -> N * (N-1)!

The code is much shorter, because XLR basically has only one built-in operator, which is the macro rewrite operator ->. XLR is little more than a big macro pre-processor. Now, if XLR happens to dynamically generate machine code using LLVM to evaluate this quickly enough, it's just an optimization, an evaluation strategy.

As a result, in XLR, evaluation and macros are practically the same thing, even more so than in Lisp in my opinion.

Axiom #3: C+ hackers and Lisp hackers are both right

Depending on whether your favorite language is C or Lisp, Ada or Haskell, Java or Python, you may prefer the first or the second approach. It's really a matter of taste, and after years of healthy competition, the two camps are not any closer to being reconciled

What XL brings to the table is a single technology that demonstrably can do both.

Axiom #4: It works, almost

Well, today, XL does neither too well, mostly because the technology is not finished. It has never stopped making progress in the years since the project began, though. If you want to contribute, go to the XL web site

XL: Advancing the state of programming languages

2010-06-19T13:42:00.012+01:00

This is the 200th post on this blog!

The WGP2010 workshop was a good occasion for me to write a short summary of where we stand with XL. With the creation of Taodyne, I'm spending much more time working on XL and with XL. This may not be immediately visible, thought, because XL is only a tool to achieve Taodyne's objectives, it is not a goal in itself.

The article's title is "Eliminating Newspeak in Programming language". Here is the abstract:

Programming languages provide us with numerous tools. But like the fictional language Newspeak in George Orwell's 1984, they also restrict what we can say, and in doing so, they shackle our minds. As a result, historical programming language all became an economic dead weight as soon as the hardware, fueled by Moore's law, passed them by.

XL is a programming language designed to get rid of this Newspeak in programming. Its primary focus is to help programmers add their own concepts and their own notations to the language. To validate this approach, many of the traditional features that XL provides are constructed, not built in the compiler. The associated methodology is called ``concept programming''. It focuses on the transformation of ideas into programs.

Writing an article about XL made me realize two things and a half:

XL remains novel and relevant today. Ten years after I first shared code, the problems that XL addresses are still there, the solutions are still nowhere seen in other languages.
On the other hand, the language never caught on. But then, the article, if accepted, will be the first one I ever wrote about XL for academic circles. From that point of view, it's good to have left HP.
XL is still evolving, and the compilers are still far from being finished... Shame on me.

So I think that XL is really capable of advancing the state of programming languages, but I need to put some additional muscle to explain what it is. Or as a friend reminded me yesterday: "It goes without saying, but it goes better saying it."

From concept to code

The first key idea behind XL is that the role of a programming language is to help us transform ideas into code. The idea seems simple enough... until you dig deeper and realize just how hard this is. This presentation is a good starting point...

What does it mean to you, to transform ideas into code? How well do the existing languages do that for you? Do you sometimes feel impaired in the way to write your code? Are there things that you simply can't say with your favorite programming language? Do you have funny horror stories about saying one thing and having the compiler understand an entirely different thing?

Linus Torvalds on religion

2010-06-18T06:35:00.002+01:00

Linus Torvalds, original creator of Linux, writes in his blog:

One of them starts to seriously talk about praying demons away, and then after the prayer has driven the demon out of the person, you have to support the person so that the demon doesn't come back. And nobody laughs at him.

Seriously? What year is it again? I'm pretty sure they didn't have Costco foodcourts in the middle ages, but maybe there was some time warping going on.

What the hell is wrong with people?

There is a large number of comments (169 to this date). Very few of them are critical of Linus at the beginning, but then a few start voicing a different opinion. And Linus Torvalds answers with gems like:

And yes, it is acceptable to ridicule people for their beliefs. If you meet a grown person who believes in the easter bunny, in santa claus, or in the tooth fairy, they're damn well fair game.

Seriously, what's wrong with Linus Torvalds?

If someone doesn't know something, you educate them. You don't run back home and cowardly post blog entries mocking your "opponent" to a friendly crowd! This is really not Linus Torvalds' most glorious moment...

What I find more annoying is just how religiously illiterate Torvalds is, and how that doesn't stop him from believing he knows everything about everything. He is not alone in his belief that to be religious, you have to be stupid.

Facebook privacy policy

2010-05-20T13:28:00.002+01:00

This might be interesting for folks using Facebook...

Patents considered harmful...

2010-05-01T18:10:00.004+01:00

In the wake of the HP's acquisition of Palm, Technologizer ran a story about some Palm patents showing Palm gizmos that could have happened. Except that they didn't.

Something bothers me about all these patents: the majority of them describe items that couldn't possibly have been built at the time the patent was granted. Video-conferencing on a Palm Pilot? Seriously?

To me, this begs the question of whether it is sufficient to describe something to be able to patent it, irrespective of whether you have the technical capability to built it. If so, then StarTrek writers probably have patents on a number of useful technologies.

If you don't have a safeguard against "innovation" that is little more than a wild idea about something that might exist some day, then you open the doors for patent trolls, since they don't need to ability to build the stuff, only the ability to talk about it.

Any engineer will tell you: anybody can have a good idea. It's building it that is hard. I believe that patents were intended to promote real innovation, not just random ideas.

Dark matter as a proof of E.T. intelligence?

2010-04-30T18:23:00.001+01:00

Here is a random thought... What if dark matter was a sign of intelligent extra-terrestrial life?

The idea is simply that a Type III civilization on the Kardashev scale would control the flows of energy escaping their galaxy. Many solar systems (the inhabitable or useful ones) would end up with mechanisms such as Dyson spheres, therefore lowering the amount of energy escaping these system, to the point where we would no longer be able to identify such systems as stars.

I don't know if the idea has any merit, but a quick Google search shows that I'm not the first one to have it...

Where are the hackers gone?

2010-04-21T08:32:00.005+01:00

Hackers, originally, didn't mean the young kid who breaks into a computer, but the raving geniuses who built the stuff that we take for granted today. Think "Dilbert", think über-geek.

This leads to the question: what did all these folks become, 25 years after the computer revolution began? Wired answers the question:

But it was the hackers themselves who inspired a generation of programmers, thinkers, and entrepreneurs — and not just fellow techies. Everyone who has ever used a computer has benefited. The Internet itself exists thanks to hacker ideals — its expansion was lubricated by a design that enabled free access. The word hacker entered the popular lexicon, although its meaning has changed: In the mid-’80s, following a rash of computer break-ins by teenagers with personal computers, true hackers stood by in horror as the general public began to equate the word — their word — with people who used computers not as instruments of innovation and creation but as tools of thievery and surveillance. The kind of hacker I wrote about was motivated by the desire to learn and build, not steal and destroy. On the positive side of the ledger, this friendly hacker type has also become a cultural icon — the fuzzy, genial whiz kid who wields a keyboard to get Jack Bauer out of a jam, or the brainy billionaire in a T-shirt — even if today he’s more likely to be called a geek.

In the end, some of them became rich, some became famous, some are still hacking in relative obscurity... I feel really fortunate to have seen these very early days when 8 bits and 16K of memory were considered quite respectable.

The Good and Bad of DRM defense

2010-04-21T08:13:00.006+01:00

Here is the right way to tell people about DRM and freedom. I'm sorry I can't embed it here, YouTube says "Embedding disabled by request" (who talks about DRM...) You should also read the EFF original story.

Unfortunately, the freedom fighters are not always that good. Here is an entirely wrong way to attack the problem. Hint: don't watch that movie, it's boring. Instead of making simple points in a humorous way, we get a lot of talking heads, all on the same side of the story except for a few "bad guys". Only the analogy with music remotely hits a chord in that whole movie. Yawn...

Don't get fooled, though. There is a war going on, a war to grab land and power, a war to get more control. That war is currently being played on your iPhone by Apple, on your PS3 by Sony. But it's all about denying you rights that you have and that you should fight to keep. Gizmodo says it quite well:

Virtual Windows

2010-04-15T08:28:00.002+01:00

I think that this is really cool:

Of course, at the moment, only one viewer is allowed to enjoy the benefits of the technology, and he/she/it has to wear some special device that tells the windows where she/it/he is.

Steve Jobs decided to knife the baby (oh no, not again!)

2010-04-11T19:53:00.013+01:00

There has been some stir around a change in the software development kit for the iPhone. See the Tao Effect Blog for an exchange between an iPhone developer and Steve Jobs on the topic.

Apple decided to change the wording of the license to forbid use of non-approved development tools. I believe that Steve Jobs is dead wrong on this one. Our own application requires an interpreter, which was pretty much already verboten. The new language in the SDK makes it even worse, blocking some workarounds we could have considered. Well, there is no way I would go for a subpar solution and substandard user experience just because Steve Jobs happens to have a beef with Adobe at the moment. So we will skip the iPhone platform entirely.

I can only hope that our application will one day matter enough that people will make it a factor in their choice of mobile phone. But even if it doesn't (quite likely), I can't imagine that we are alone in this situation. It only takes a few killer apps that can't reach the iPhone due to Apple's control-freakness to tip the balance away to some other platform.

Steve Jobs should know, of all people. Apple was once so dominant in the field of personal computers it could laugh at IBM, much like the iPhone in smartphones today. Well, guess what? A few years down the road, IBM's presence in the Macintosh core market was no longer laughing matter.

The parallel is striking. Back then, Steve Jobs wanted complete control on the Macintosh, to the point of not allowing expansion slots. The iPhone has no expansion of any kind, no SD card, nothing, except for an Apple-proprietary connector. The Macintosh unique software would not be licensed to anybody. The iPhone OS only powers the iPhone. To program a Macintosh, you had to buy a Lisa. To program an iPhone you need a Macintosh. Apple told you what was OK and not OK on their platform (Inside Macintosh User Interface Guidelines), and still does (SDK license agreement and AppStore approval process). Microsoft got advance notice of the Macintosh, and they loved it. Google got advanced notice of the iPhone, and they loved it.

But Microsoft didn't want to become Steve Jobs' puppet, and after a while they walked, something which Jobs took as a betrayal. Years later, Google didn't want to become Steve Jobs' puppet either, and they too walked. And Steve Jobs felt betrayed again. Twenty five years ago, Microsoft ended up developing their own competing platform. Steve Jobs saw that as stealing their IP, and sued. Today, Google ended up developing their own competing platform, and Steve Jobs is suing (HTC).

I can't help but wonder if there's something that I'm missing. Steve Jobs must be smarter than this, after all he's contributed to our daily lives. Yet history is repeating itself. Steve Jobs seems to believe that he's entitled not just to benefit from his own creativity (perfectly legitimate) but also to control and limit the creativity of others (perfectly not legitimate). That's quite unfortunate, because the iPhone rocks and keeps getting better, like the original Macintosh did. Still, if the iPhone in a few years turns into a marginal platform with a shrinking market share, Steve Jobs will only need a mirror to look at the person to blame. No amount of suing will repair the predictable result of his own mistakes.

Nothing is irreversible at this point, though. Steve Jobs is smart, and the team at Apple is top-notch. Maybe someone at Apple can convince Steve Jobs that he should return to what he does best, winning by delivering insanely great products, rather than be at his worst by delivering inanely closed products. Please Steve, for the good of your own company, get rid of paragraph 3.3.1 in the iPhone SDK, and then work a plan to attack Android where it would hurt them, e.g. by licensing iPhoneOS (gasp) and allowing other teams to not just use your device but improve it (re-gasp). Otherwise, it's safe to predict that in a few years, I will be using an Android phone and regret the good old times when the iPhone had a shot, much like I was forced to use Windows for most of my professional life and hated it.

Eating my own dog food

2010-03-11T17:36:00.007+01:00

This post was edited from within Tao, for the first time. This is looking really cool, but I can't really tell why just yet. It just makes me happy to see that we are making very significant progress.

More interesting general news: code bubbles are an interesting new paradigm for IDEs. It's a really nice looking user interface, both fancy and apparently usable. Worth taking a look at, IMHO.

Now, I wonder why the Java programmers get all the fun and interesting tools... I have a theory that maybe they need so many tools because programming in Java is more difficult than in most other languages...

The Reinvigorated Programmer

2010-03-10T23:02:00.005+01:00

Thanks to Slashdot, I just came across a blog I did not know, The Reinvigorated Programmer. The last few posts I read there were all quite good:

The value of BASIC as a first programming language dwells into interesting educational aspects of a much criticized language. To his argument, I'd add that anybody who learned BASIC had to learn how to do interesting programs in 48K at best, but sometimes as little as 1K (like on the ZX-81)
Whatever happened to programming, quickly followed by a redux, discusses how programming has changed into a job where you mostly assemble software components. I'd say it's about time, but it's true that assembling components is sometimes less fun than creating your own.

In any case, all the posts I read interested me, despite being illustrated by strange pictures of cold dead fish or cold dead humans. Well, the writing is good, and at least the guy understood that a few pictures here and there help.

Added to the side links.

Making the best of your brain

2010-03-09T06:49:00.003+01:00

A truly amazing Google Talk presented by David Rock, author of the book Your Brain at Work. And if the title of the book has at least two meanings, it's definitely intentional.

Why Go isn't my favourite programming language

2010-03-07T10:11:00.034+01:00

In case you are wondering, the title of this post is a nod to Brian Kernighan's Why Pascal isn't my favourite programming language. I'm only showing my age here, not trying to compare myself to Kernighan :-)

Summary

There has been a lot of buzz recently about the Go programming language. There was an article on Slashdot recently, videos of Google talks on YouTube (slides), the works. And of course, there is the Go web site itself. The YouTube video below has been viewed 183,000 times as of this writing, so the roll-out campaign had some success...

Of particular interest are the minds and forces behind the language. The initial contributors are Robert Griesemer, Ken Thompson, and Rob Pike, and the project was developed at Google over a time period of two years. With such a pedigree, expectations are high. And that's probably why I'm a little disappointed with what was presented so far. It's not that it's a bad language, it's more like as so-what language. There's nothing I find really compelling about it. In particular, I didn't find anything that would make me significantly revisit my own pet programming language, XL.

The point of views below are indeed largely based on my own experience designing and implementing a programming language. So I would say that they are both educated and biased. Keep this in mind as you read on. Also, I tend to point out the negative aspects because the campaign discussed above did a rather good job at highlighting the strong points of Go.

First impression

My initial impression of the Go programming language, based on the material above, is that it has a few serious flaws:

No generics: Go is yet another language where generics will be bolted-on as an afterthought. As a direct consequence, Go includes many constructs (including maps or goroutines) that in my opinion rightfully belong to a library. This is important for programmers: the choices of the Go designers might not match the needs of the Go programmers. Worse yet, if you have some reason to use some kind of fancy container, Go is not the right language to implement it.
Not suitable for what I call "Systems": The Go designers seem to consider that "Systems" equates writing applications such as web browsers. To me, "systems" implies that you can write programs that may talk directly to the hardware (as in operating systems or real-time systems). At first glance, Go seems ill-suited for such tasks. In particular, it does not just provides a concurrency or memory management model, but actively enforces it at the language level. In other words, if you want to write a real-time system in Go, you first need to implement the runtime support for goroutines or Go garbage collection. I ran into this problem in the early days of HPVM: implementing a C++ runtime on bare metal proved complicated enough that I had to rewrite my C++ binary translator in plain old C.
Orwellian Newspeak: The two examples above illustrate how, all too often, Go "knows what is good for you" and won't let you deviate from the party line. This goes as far as imposing where the braces go for rather bogus syntactic reasons, deciding when you can use upper-case letters, or formatting the code automatically when you submit it!

These three aspects of the design mean that I probably won't use the language seriously any time soon. I can't do meta-programming well because of the first issue. I can't do system software because of the second issue. And if I want to explore, I will be constrained in what I can do because of the third one. These issues may not mean as much to the majority of programmers, though. Just because Go is not a language for me doesn't mean that it won't work for you. Your mileage may vary.

In any case, there are a number of aspects to the Go language worth discussing, if only because of the publicity it received.

Stated Objectives

The objectives of the team in creating the language are, according to the slides:

New
Experimental
Concurrent
Garbage-collected
Systems
Language

In my humble opinion, these objectives lack a clear sense of direction and purpose. They include mere facts ("New"), low-level implementation details ("Garbage-collected"), usage models ("Systems") and actual design objectives ("Concurrent" and maybe "Experimental"). Specifically, "New" might be intended to mean innovative, but while there's little debate that Go is new, I didn't find much in it that wasn't already hammered elsewhere. "Experimental" should probably mean that it explores a few radical ideas, but it could also simply reveal that it's not fully cooked yet. Unfortunately, radical ideas are exactly what I didn't find.

Concurrency is the most interesting of these objectives. It is not exactly a novel idea, but there is still a lot of progress to be made in how languages support concurrency. Still, it's hard for me to see what Go adds compared to industrial languages like Erlang. Sure, Rob Pike demonstrated 100,000 goroutines completing in 1s. At first sight, that might seem impressive. But according to this source, someone ran 20 million processes in Erlang as early as 2005. And apparently, run-time performance at the time were similar: 5.3μs + 6.3μs times 100,000 would be 1.16 second, very comparable to what Rob Pike demonstrated on presumably faster hardware.

The language prominently features garbage collection. Garbage collection for memory today is practically a given. You can't appeal to Java programmers without it. But collecting garbage is not about just memory objects. In real programs, there are many other forms of garbage to recycle: temporary files, open files, locks, threads, open network connections, ... Furthermore, the requirements placed on the garbage collector may vary from application to application. Providing a garbage collector by default is good, providing one that is implemented in the library, that you can taylor to your needs and [gasp] apply to non-memory objects would be so much better... In short, is Go's garbage collection worth the prominent position that its designers gave to it in the presentations? I don't think so personally.

Implicit Objectives

It's even more interesting to analyze the implicit objectives of the language. I could find at least three such objectives that, while not explicitly stated, seemed quite important to the design team.

Compilation speed: There were many demos of how fast Go compiles, and it's featured prominently on the front page of the web site, so this seems to be a rather important objective to the Go design team.
Simplicity: Although it is being described as "slightly bigger than C", Go is clearly intended to remain simple, with a simplified syntax, no parsing ambiguities, ...
Programmer comfort: Rob Pike stressed how the language is designed to make it easy to write tools, to compose software (there's a rather nice dependency checking mechanism), to reduce typing, and so on. And of course, not waiting for the compiler is comfortable too.

All these are rather noble objectives. But they also stress the kind of trade-offs that were made in the design of the language. For example, compilation speed does matter, nobody can dispute that. But in exchange for that speed, we pay 10% in execution speed, and more importantly we lose a number of features I consider essential for productivity, like templates. This may not matter much to software consumers, however. It is true that more and more, we simply compile someone else's code. In that case, compilation speed is practically the only metric that matters.

Tipping the balance the other way, why not push the reasoning to the limit, and ditch compilation entirely? A lot of recent work has been in the field of just-in-time compilers. LLVM has shown that it's possible to dynamically generate high-quality code in a portable way. The XLR runtime component of XL is now using LLVM, so that you can execute XLR programs (that is, the run-time language) without any explicit compilation. Compilation does happen, but entirely transparently, on the fly, as you execute the program. In that scenario, some heavier compilation remains possible once the program runs, to get better optimizations, faster execution or tighter verification of the code.

Conclusion

In conclusion, Go shows how difficult it is to design an innovative programming language for today's programming world. Many of the choices made by the design team seem rather old-fashioned to me. Go didn't rattle my brain the way Haskell, Erlang or Ada did in their time.

But ultimately, the bottom line for me is this: Go seems to be a solution in search of a problem.

Dark matter, the modern aether

2010-03-05T17:33:00.016+01:00

Today, my 16-year old son asked me what dark matter was. I was surprised that he would even have heard about dark matter, but it turns out that even junior science magazines talk about the search for dark matter these days. I must say that I'm not too happy about that. The junior science article, like many other, present dark matter practically as a fact.

The reason this makes me rather nervous is because of the rather obvious parallel with aether. Just like the luminiferous aether, dark matter is something that was postulated when no physical evidence justified it, in order to preserve existing theory.

Those of you who were already dabbling in physics during the 1850s¹ may recall that luminiferous aether was hardly a ridiculous idea at the time. Aether was very simply the medium carrying light waves, much like air or water carry sound waves. It was initially postulated by Isaac Newton to explain things like refraction. According to Wikipedia, Augustin Fresnel proposed in 1818 a theory of light as a transverse wave in aether. To quote Wikipedia, from this point on, no one even seems to question its existence. In other words, the existence of aether was postulated in order to preserve the existing theory of waves. All existing waves required a medium, such as air or water, therefore it was natural to assume that light waves also needed a medium to carry them.

The key point to remember here is that the brightest minds of the time did not question aether at all. Some of them, like Newton or Fresnel invented it. Later, the vast majority of scientists were busy trying to refine the concept to make it work. Yet today, luminiferous aether is seen as the canonical example of an obsolete physics theory. Einstein's relativity made the very notion of aether not just useless, but actually wrong. Relativity simplified things by removing the need for a system of coordinates that would be special, but this simplification meant that aether could not exist, because otherwise aether itself would have defined a system of coordinates that was unique.

Back to dark matter. We find ourselves in a similar situation today. There's something about the universe that we very plainly, very visibly do not understand. The original problem, as identified by Fritz Zwicky, was that galaxies do not spin the way they should according to our best theory of gravitation, general relativity. They behave as if there was more matter in them than we can see.

The operational keyword here is as if. At the moment, we really have no idea whether it's the theory of gravitation that is flawed, or whether there really is 95% of the universe's mass that we can't detect. Talking about "dark matter" is choosing one option over the other. It's pretending that we know, when in reality we still lack a model that really explains all the evidence. In my humble opinion, the jury is still out on what this model will look like.

In short, I'm unhappy about references to dark matter made as if it was a settled topic, a known, validated scientific fact on a par with photons or Pluto. Maybe the problem is with the terminology. Talking about dark matter rather than, say, "gravitational anomaly in galaxies" (GAG) is a good way to preserve the illusion that we know what we are talking about. It makes it sound real. But just because we gave it a fancy name doesn't make it more real than aether or the tooth fairy.

Let's be humble and honestly face the simple fact that our model of mass and gravitation breaks down in face of quite a bit of physical evidence. We find ourselves in the situation of physicists in 1850 whose aether-based theories predicted phenomena like aether drag and aether wind, which experiments repeatedly didn't find. It's exciting, it's fun. It's a good thing for physics, because it means there is something new to be found.

Note 1: My editor tells me it's considered bad taste to live past 150 on this planet. My apologies to those of my readers I might have offended...

LeMag IT article about HPVM

2010-03-04T21:04:00.006+01:00

Sorry for my non-French readers, here is an article from LeMag IT about HPVM, who several colleagues forwarded me.

This is also probably the first time someone links to the Taodyne web site.

Welcome to the readers of LeMag IT who decided to follow the link to my blog...

ACM Sigplan Workshop on Generic Programming

2010-03-02T18:46:00.000+01:00

I received a call for papers for the 6th ACM SIGPLAN Workshop on Generic Programminng.

Goals of the workshop

Generic programming is about making programs more adaptable by making them more general. Generic programs often embody non-traditional kinds of polymorphism; ordinary programs are obtained from them by suitably instantiating their parameters. In contrast with normal programs, the parameters of a generic program are often quite rich in structure; for example they may be other programs, types or type constructors, class hierarchies, or even programming paradigms.

Generic programming techniques have always been of interest, both to practitioners and to theoreticians, and, for at least 20 years, generic programming techniques have been a specific focus of research in the functional and object-oriented programming communities. Generic programming has gradually spread to more and more mainstream languages, and today is widely used in industry. This workshop brings together leading researchers and practitioners in generic programming from around the world, and features papers capturing the state of the art in this important area.

We welcome contributions on all aspects, theoretical as well as practical, of

polytypic programming,
programming with dependent types,
programming with type classes,
programming with (C++) concepts,
generic programming,
programming with modules,
meta-programming,
adaptive object-oriented programming,
component-based programming,
strategic programming,
aspect-oriented programming,
family polymorphism,
object-oriented generic programming,
and so on.

More information can be found at the original web site.

Goodbye HP, Hello Taodyne

2010-01-22T15:44:00.024+01:00

After almost 16 years working for HP, almost half of which on HPVM, I have decided to do something else. Please welcome my new little company, Taodyne, into the fierce world of software development startups. Taodyne is supported and funded by Incubateur PACA-Est.

For me, this is a pretty major change. So it was not an easy decision to make. Leaving behind the HPVM team, in particular, was tough. Still, the decision was the right one for a couple of reasons.

First, I had to face the fact that HP is not trying too hard to retain its employees. To be honest, in my case, they actually offered something at the last minute. But it was way too late. The decision had already been made over one year ago. There was no going back, as the creation of the new company commits other people who I could not simply leave behind.
However, the primary reason for leaving HP was my desire to invent new stuff, something that I felt was less and less possible at HP. Don't get me wrong, it's still possible to "invent" at HP, but you have to do it on demand, it has to fit in the corporate agenda, to belong to the corporate portfolio. Nobody could invent a totally new business like the Inkjet or the calculator at HP nowadays. Well, at least I could not (and I tried).

The tipping point for me was probably when one of my past managers told me that I should feel content with my contributions to HPVM, that I "had been lucky" to have this as the "high point to my career". That manager probably meant his remark as a compliment. But I personally think sweat had more to do with the success of HPVM than luck, and I'm not sure how to take the "high point part" ("from there, you can only go down"?)

Ultimately, the choice was relatively obvious, if hard to make. Simply put, I didn't have a dream job at HP anymore, nor any hope that things would improve. I was working long hours (meetings from 5PM to 1AM 5 times a week, anyone?), more or less officially had two jobs for only one pay, only to be told year after year that the only way forward was down... I owed it to my family to look for a way out.

So really, for me, HPVM was hopefully not the high point in a career, it was hopefully just one step along the way. Taodyne is the next one. Let's hope I still have something to offer. When we deliver our first product, I hope that you will agree that I took the right decision. In the meantime, please check out our progress on the XLR web site. Update: This entry was shortened compared to the original to remove useless material.

Incubation - Accepted

2009-12-08T16:54:00.002+01:00

Today was the real presentation to Incubateur PACA Est. The whole team was present. The presentation went mostly smoothly, except for one "Ooops" moment where my presentation crashed on me :-( But why should something that we rehearsed just a couple of days before successfully work smoothly a second time? Murphy's law practically forbids it...

Incubation - Dry run talk

2009-12-04T16:35:00.007+01:00

Today, I gave a dry-run talk about our project in front of representatives of "Incubateur PACA Est". I think it went relatively well. We got interesting feedback from all the participants.

Incubation is a very interesting approach, where the French state has put in place special structures intended to help young companies succeed. The reasoning is that most companies fails for bad reasons, and might not have failed if given just enough help. This is particularly true for innovative companies, where the founders generally have a technical background, but no real knowledge of how to run a company. Incubation provides them with a safe and motivating environment, including consulting, basic amenities (e.g. network access), and frequent contacts with other people running into the same kind of issues. Not to mention some financial help.

The "real" talk is on December 8th.

Welcome Marie...

2009-05-28T15:37:00.000+01:00

Today, a little baby girl is born in our family. After three boys, I'm so thrilled! Now, don't be surprised if the blog becomes much less active. I won't have much sleep for a while.

HP Pay Cuts - an unfair act of economic opportunism and greed.

2009-03-11T14:38:00.002+01:00

Damian Saunders writes:

HP’s CEO, Mark Hurd announced, on the 20th February that he would be implementing a company wide cut in pay for all employees. Starting with a reduction in his own salary by 20%, followed by senior executives who would take a drop between ten and fifteen percent, regular employees 5 percent and exempt employees 2.5 percent. All this in reaction to a 13.5 percent fall in the company’s first quarter profit.

[...] First we need to put Mark Hurd’s 20% salary cut into perspective, remember he is only taking a cut to his base salary ($1,450,000) which amounts to a $290,000 drop. Seems quite reasonable until you examine the following, publicly available, information. Mark Hurd’s total compensation in 2008 was $42,514,524 [...] a 68% increase in the total package from 2007 to 2008. [...] So, the question is; what’s the significance of his stated 20% cut in base salary? I would suggest next to nothing. [...] Put bluntly, 6 people at the top of the HP pyramid accounted for $142,774,325 in compensation in 2008 alone. That is an obscene amount of money.

[...] Let’s look at the plight of the HP employee. The first thing we have to consider is that, unlike Mark Hurd, a 5% cut in salary is in fact a 5% cut in total compensation. Someone on a salary of $65,000 would be losing $3250 per year before tax, or $270.00 per month. Some would say this is a small price to pay for keeping your job but I think holding that gun to an employee’s head is outright exploitation and can not be condoned, especially when they have already been exploited enough for the sake of high profit margins and Mark Hurd’s stellar career performance. Ask a majority of HP employees about their current remuneration and you will be lifting a rock that you don’t want to look under.

I recommend you read the rest of the post, and spend some time browsing through the many comments. You will get a pretty good idea of the morale of HP employees these days.

HP DE200C Digital Entertainment Center

2008-12-22T10:41:00.008+01:00

I finally took some time to put back together my good old trusted HP DE200C Digital Entertainment Center, and I thought I would share the result:

In my opinion, this is a good illustration of the kind of innovation open-source brings. The software stack is pure free and open-source software. That allowed me to add my two cents contribution (driver for the Vacuum Fluorescent Display, support for the remote-control and front-panel keys).

But equally important, it shows the kind of innovative hardware that Hewlett-Packard was putting together back in 2001, seven years ago now... Kudos to Lee Devlin and other people who gave us this great toy.

The source code for my little contributions can be found at http://repo.or.cz: the Noritake VFD driver, the LCDproc changes and the Linux kernel changes. I had some trouble pushing the git repository for the last one, so here is a patch against 2.6.26, which may be easier for most.

Science and the fear of innovation

2008-11-28T17:54:00.003+01:00

Michael Nielsen writes:

The disincentives facing scientists have led to a ludicrous situation where popular culture is open enough that people feel comfortable writing Pokemon reviews, yet scientific culture is so closed that people will not publicly share their opinions of scientific papers. Some people find this contrast curious or amusing; I believe it signifies something seriously amiss with science, something we need to understand and change.

The whole article is worth reading. It suggests the same kind of innovation I am hoping for in science. Right now, scientists often congratulate themselves for open science. But as Michael and others point out, arXiv is not exactly a model of openness, at least relative to standards such as open-source software development.

Enough hot air...

2008-11-28T07:17:00.007+01:00

I just came across this article on Advogato:

Once upon a time in the not-too-distant past, a hacker I know blogged about using object-oriented C to implement a lightweight imitation of some of C++'s features for his latest project; almost immediately, somebody saw fit to reward this charming piece of acceptably self-congratulatory writing with a stern and quite public deconstruction. Does this scene seem familiar? Why does this keep happening? And what, if anything, can we do about it? We can hardly hope to appease all of hackerdom's malcontent — but we can at least try to avoid stepping on each other's toes.

Amen to that. Remember, a few days ago when I wrote an article indicating that I agreed with an earlier post by Eric Raymond. Only to discover, a couple of weeks later when I returned to read Raymond's blog that he had a new piece that fell just short of calling me a "blithering idiot".

Had I done anything to merit that? Apparently, my sin was to remark that unless you are famous like Eric Raymond, it's hard to make a living purely from open-source, and that consequently, genuine open-source innovation is hard to find. Instead, I think that innovation generally attributed to "open-source" is still very much driven by corporate interests. I have personal reasons to hold that particular opinion (having put an innovative programming language in the open-source domain, being offered to work on open-source project multiple times, but never on my own project...) I find it surprising that it deserved being attacked by Eric Raymond so unnecessarily, even less so when I agreed so much with what he had written...

There is a psychological explanation

I read recently an explanation for that frequent behavior on the net. Apparently, the mechanisms that we use to throttle and moderate our social interactions are based very largely on visual cues, and they are very complex (they show up quite late in the human development, typically maturing in one's late teens). When we don't have these visual cues, our brain's moderation system doesn't quite work as it should, neither on the sender's nor on the receiver's side.

So, based on this explanation, it is likely that Eric Raymond read much more of an attack in my original post than I intended, whereas if we had been speaking face to face, he might have seen various expressions on my face that might have convinced some part of his brain that I was not that critical of open-source, that I was not implying open-source folks cannot innovate, that I was not downplaying his own intelligence.

Conversely, when he started writing, he used very scalding words like "blithering idiot" (even if he downplays that initial statement a few words later with "reasonably bright"). It is unlikely that he would have used such words in a face-to-face discussion, if only because our brains know very well how quickly a bad choice of words can lead to a non-verbal response or even physical harm... So when you talk to someone, you rarely say to anybody "you are an idiot", even if you really believe it.

... or is there?

When I first read that psychological explanation, it convinced me almost entirely. Since then, though, I noticed something interesting. Letters written on paper tend to be very polite, very nice.

So it seems that something else than just "not seeing the other guy" is at play. Writings between scientists of the early twentieth century, for example, are sometimes heated discussions between people who often squarely fall into the "genius" category. Yet I don't remember any "flame", any "hot air". Maybe that's just because I'm not familiar enough with these writings.

But the other possibility is that our modern society doesn't value politeness as much as it used to...

Stability and innovation

2008-11-27T10:44:00.005+01:00

Two days ago, I attended a conference in Paris on the future of virtualization in mission-critical environments. There was a presentation from Intel about the roadmap for Itanium and virtualization.

Stability vs. Innovation

Two things in this presentation reminded me of what Martin Fink calls the Unix paradox:

Intel pointed out that Itanium is mission-critical, so they tend to be more conservative. For example, they use processes for Itanium that have already been proven on the x86 side. Like for Unix, there is a similar paradox for mission-critical processors.
The cost equation is very different for mission-critical systems. For commodity hardware, the acquisition cost tends to dominate your thinking; for mission-critical hardware, it's the (potential) cost of losing the system that drives you. So whereas in volume systems, you are ready to pay more for innovation, e.g. features or performance, in mission-critical systems, it's for stability that you pay more.

Following the recent discussion with Eric Raymond, I thought that was yet another interesting angle about what "innovation" means to different people.

One step at a time, or "TIC-TAC-TOC"

Intel also reminded us of the TIC-TOC model they now use to release CPUs:

TIC: change the process on a stable micro-architecture
TOC: change the micro-architecture on a stable process

I think that a similar approach applies to how our customers want to upgrade their mission-critical software, something that I would call TIC-TAC-TOC:

TIC: Change the infrastructure (e.g. machines, disks), keep OS and applications the same
TAC: Change the applications, keep infrastructure and OS the same
TOC: Change the OS, keep infrastructure and applications the same

Customers may, at their discretion, decide to do multiple steps at the same time. For example, they may use an infrastructure change as an opportunity to also upgrade their OS and applications. But as a vendor, we should be careful not to force them to de-stabilize more than one thing at once. It should be their choice, not ours.

When innovation is the problem

Historically, HP has been good at this. TIC: You update from PA-RISC to Itanium, and you can still run the same OS, still run your PA-RISC applications. TAC: Upgrade the applications, keeping everything else the same, and you get a healthy speed boost. TOC: Upgrade to 11iv3, another speed boost; Install HP Integrity Virtual Machines and you get the latest in virtualization features, even on 2002-vintage Itanium hardware. As far as I know, you can't virtualize a POWER4, and you can't get Live Partition Mobility on a POWER5 system.

But TIC-TAC-TOC is not a perfect solution. That model is painless for customers only if we can convince them to stay reasonably current in two out of three dimensions at any given time. That model breaks down for a customer who runs HP-UX 10.20 on PA-RISC and obsolete applications. Such customers feel left behind, and the leap of faith to move to current technology is so big that they are an easy prey for competitors.

So here is my interpretation of Martin Fink's Unix paradox:

Stability + Innovation = Disruption

How to solve that equation is left as an exercise for the reader :-)

Genuine open-source innovation is hard to find...

2008-11-24T15:36:00.004+01:00

Eric Raymond responded to one of my earlier posts. It was not one of his best days:

The easy, cheap reply would be to write the author off as a blithering idiot who has failed to notice that his entire environment has been drastically reshaped by open-source innovation, and the proof slaps him in the face every time he looks at a browser. The easy, cheap reply would be to write the author off as a blithering idiot who has failed to notice that his entire environment has been drastically reshaped by open-source innovation, and the proof slaps him in the face every time he looks at a browser. But, in fact, I think he (and others like him) are not idiots; they are reasonably bright people making a couple of serious and identifiable errors in their reasoning about open source, closed source, and innovation.

Uh?

Eric Raymond makes a fool of himself with this post. To begin with, he quotes only a fraction of my “screed”, later makes fun of “M’sieu de Dinechin”, and barely avoids calling me a blithering idiot. I’m honored he later revises his judgment and ups me to “reasonably bright”…

Yet for all the name calling, ESR totally misses my point: developers need to eat, and corporations provide vast majority of the necessary funds, even to open-source contributors like ESR or myself. My XL work is all open-source, but I would not have been able to afford it without a regular source of income. That’s my point, and if ESR wrote a single word to address it, I didn’t see it.

As for claiming that open-source build the world-wide-web, sorry Eric, but that’s bollocks and you know it. I could not even find any evidence that Berners-Lee’s browser, WorldWideWeb, was open-source at the time. What everyone knows is that Berners-Lee worked for CERN at the time, in other words he had a stable revenue, and that only reinforces my point. The browser that ignited the web was not Berners-Lee’s (nobody could afford the incredibly expensive and closed-source NeXT machines that ran it) but Mosaic. And as I pointed out in my “screed”, the source code of Mosaic was public, but not open-source by any standard definition. More importantly, it is silly to ignore all the corporate contributions that made the web what it is today, from Netscape to Microsoft to Cisco to fiber-optics to ISPs…

It is legitimate to say that “Tim Berners-Lee invented the world-wide-web”. I think it is even OK to say that hackers built it. But that’s a far cry from “open-source built it”, which seems to be what ESR would want us to believe…

Ultimately, I’d say that Eric’s error number zero is to confuse “ideas” with “innovation”. Innovation is ideas made real.

Factor, an interactive extensible programming language

2008-10-30T15:22:00.003+01:00

An interesting overview of Factor, an interactive and extensible programming language:

The language is available here.

Cool high-level virtualization video

2008-10-30T14:59:00.002+01:00

A good explanation of what my work is about:

Why writing proprietary software is not fun

2008-10-30T11:18:00.008+01:00

Eric Raymond explained why he hates proprietary software. And the reason is not with the software itself, but with how it is being written:

In that world, the working programmer’s normal experience includes being forced to use broken tools for political reasons, insane specifications, and impossible deadlines. It means living in Dilbert-land, only without the irony. It means sweating blood through the forehead to do sound work only to have it trashed, mangled, or buried by people who couldn’t write a line of code to save their lives.

If you love programming, trying to do work you can be proud of in this situation is heartbreaking. You know you could do better work if they’d just goddamn give you room to breathe. But there’s never time to do it right, and always another brain-dead idea for a feature nobody will ever actually use coming at you from a marketing department who thinks it will look good on the checklist. Long days, long nights and at the end of it all some guy in a suit owns all that work, owns the children of your mind, owns a piece of you.

Raymond concludes with:

I will have no part of helping it do to the young, malleable, innocent programmers of today and tomorrow what was done to me and my peers.

Because two decades later, my scars still ache.

Unfortunately, based on my own experience, I must admit that he is right on the mark. Writing software in a corporate environment is often painful: artificial deadlines, artificial constraints, artificial priorities, artificial pressure. But it has gotten a lot worse in the past decade, at least in the company I work for.

Why it's worse today than Raymond remembers

In today's cost-saving business environment, there is even more than what Raymond identified based on an experience that is over two decades old now. These days, the tools are not just broken for political reasons, more and more they are broken for cost reasons.

My team and I work on several locations, in the US and in Europe. Three of these locations host our servers. On these three locations, in the past month alone, we had no less than six electrical outages (!), five of which were unplanned, and no location was immune. You'd think that having backup power is really basic. But in one case, the backup generator itself was so old and so broken that it prevented power from being restored when the grid came back up! We also lost networking or primary infrastructure tools (e.g. bug tracking system or e-mail) on at least five occasions. These are not accessories, these are all critical tools that directly impact our work. If electricity or network or mail is down, there isn't much you can do to develop software.

Meanwhile, the release schedule itself tightens, for us it's now two releases a year. The team shrinks despite the increasing amount of work: we lost key contributors and we don't even try to replace them. Servers break down because they are too old: my most recent machine was obsoleted in 2004, and it's a leftover from some other team. Travel is restricted and inconvenient: I once had to wait 12 hours in New-York on my way back from California, which practically doubled the length of an already exhausting trip; the corporate travel booking system chose that flight to save $28, which was not very smart when you know how much two meals in New-York cost...

And to top it off, we have all the small daily frustrations. Coffee machines are not repaired, or worse yet are removed, with explanations that a colleague called "soviet style communication", like "this coffee pot is not being used" (talk to the guy who's been filling it three-five times a day). Work PCs are replaced with the lowest-end model, and are already obsolete when you get them: you give employees what customers wouldn't buy anymore. Signs are posted in the restrooms asking us to be "green" and limit water usage, but the nearby toilets have been leaking for months, not to mention that they are clogged too. Raises are harder and harder to get, bonuses are recomputed with new formulas that make them smaller each year...

This is why I wrote recently to some of my colleagues that "developing software for this company is like being asked to win a marathon while wearing ski boots and carrying three people"... which my colleagues apparently did not think was even strong enough, since the first response was "... on one leg".

Incompetent bosses were replaced by powerless bosses

Raymond talks about incompetent bosses, and this is still the picture given by Dilbert. I was lucky enough that I did not have that many incompetent bosses. Sure, they all made a mistake here or there, but who doesn't. On the other hand, my bosses appeared to be less and less empowered, and more and more trapped into a system that dictates what they can do and cannot do.

For example, "standard" applications and processes have been made mandatory in the name of cost savings. It has become more and more difficult to not be punished for maintaining local applications that do what you need, something now called "shadow IT". And too bad if the standard, centralized applications lack the capacity or features, if they don't scale, if they are hosted on servers that are too small, if there is practically no redundancy. In the name of cost savings, you accept that there will be several outages per month.

Again, what is really frightening these days is that you can talk to your manager about that problem, and he will talk to his manager, and so on, yet nothing will ever change. You never get any feedback like "we heard you, we will make this and that change". Instead, what you get is top-down self-congratulatory message explaining that our IT is now so good we could practically run another company with it! In short, whatever you ask your boss, chances are he can't do a thing about it, and chances are you won't get it.

Competing

How can it be bad to reduce costs? Running a company is, after all, all about competing: competing for customers, competing for best costs, competing for highest revenues, competing for employees. So saving costs seems like a good way to get a competitive advantage.

But the key asset of a company is its employees. Everything else is really just support, tools to do the job. If you consider employees as an asset, you ultimately win, because employees work together to deliver great products. If you start considering them as a liability, as a cost center, as something that you need to eliminate rather than as something you want to optimize, you might get some short-term gains out of it. But I believe that in the long run, you can only lose. The only reason this strategy is so popular today is because most CxOs get more benefit from boosting short-term profits than they could from building a sustainable business model. It is more profitable to "cash out" on the accumulated assets of an existing big company than to solidify these assets for the future.

This is why right now, what I just described is too often the norm in the good old corporations, but not in companies still driven by their original founders. The founders of a company tend to have the same pride for their creation as Raymond has for his software. It is not a matter of scale: even large companies like Google still get it. Some corporations used to get it until their founders were replaced. But I think that we have enough evidence to know that companies can deliver a lot of good products and shareholder value while treating their employees really well.

Open-source vs. Corporate?

There is still one point where I differ slightly from Raymond's point of view. I am not sure that there is anything mandatory about corporations crushing software developers. There is enough difference between one corporation and the next, enough difference even over the lifetime of a single corporation to believe that treating employees well has little relationship with how you distribute software.

On the contrary, I think that corporations who nurture their employees can provide the best possible environment to develop software, including FOSS. They can provide money, resources, financial safety that makes it easier to concentrate on the work, a sense of purpose or direction (like Mark Shuttleworth is trying to do with Canonical and Ubuntu). Linus Torvalds, the icon of free software developers, has worked for various corporations. An alternative is foundations. In all cases, you can only work on free software if you have enough money to buy a computer.

Where Raymond is right is that open-source software gives developers a whole lot more freedom and control about what will happen with the software. All the work that I put in a product once called "TS/5470 - ECUTEST", a world-class real-time measurement and control software, was lost when Agilent (HP at the time) decided to shut it down. Nowadays, you can barely find this on Google. It's too bad, it was really useful. Before even being released, it found bugs in every single piece of car electronics we tested with it, including production ones that had been running car engines for years. Even today, I think there is still nothing like it. But as far as I know, it's lost.

Where FOSS falls short

Still, FOSS is not the ultimate solution. It is generally very good at replicating infrastructure and commodity software, where cost becomes marginal. It is not so good at innovating. I can't think of any FOSS innovation similar in scope and impact to the iPhone, Google or Mosaic (which was a proprietary program, even if the source was available).

Even Linux, the poster child of FOSS software, a very innovative platform these days, started as a copycat of a proprietary product (Unix), and started making real progress only with the help of corporations. And as I wrote several years ago, the OS itself is a commodity:

The OS itself will probably fade into the background where it belongs. You don't care much about the OS of a Palm Pilot or a network appliance or an ATM, and you shouldn't. The OS would probably have disappeared from the public consciousness five years ago, weren't it for Microsoft's insistence on making it its main source of profit

This is exactly where Linux stands today: it is most successful in applications where you don't see it (e.g. cell phones or appliances.)

What I'd like to see happen is genuine open-source innovation. But I'm afraid this cannot happen, because real innovation requires a lot of money, and corporations remain the best way to fund such innovation, in general with high hopes to make even more money in return.

We all need to eat

This is personal experience too. In the past year, I have been contacted by three companies to develop open-source software. But it was always working on their own agenda. None of them offered to work on my personal open-source project. And that's the real problem. If, as Raymond points out, the pride you may have about the children of your minds matters that much (and clearly, it does matter to me), do I really win by leaving a product I invented for one I did not invent, even if it's an open-source one?

Eric Raymond may have a second income that allows him to do what he wants. I personally don't have this luxury. I would like nothing better than to work full time XL, on creating the most advanced programming language there is, but this is not going to happen. Unless someone, maybe me, suddenly realizes how much money they could make out of it, and decided to fund a company or to add this to an existing company's projects.

My point is that, in the end, corporations fund innovation based on their own objectives. And in the end, we all need to eat, we all need someone to pay us. It's not that different in the open-source world, except maybe for a few lucky stars that are about as representative of the open-source community as Bill Gates is representative of the corporate programmer.

Virtual machines and scalability

2008-10-17T06:25:00.079+01:00

I already pointed out many problems regarding the comparison of virtual machines recently posted by IBM. But there is one topic which I thought required a separate post, namely scalability.

What is scalability?

Simply put, scalability is the ability to take advantage of having more CPUs, more memory, more disk, more bandwidth. If I put two CPUs to the task, do I get twice the power? It is not generally true. As Fred Brooks said, no matter how many women you put to the task, it still takes nine months to make a baby. On the other hand, with nine women, you can make nine babies in nine months. In computer terminology, we would say that making babies is a task where throughput (the total amount of work per time unit) scales well with the number of baby-carrying women, whereas latency (the time it takes to complete a request) does not.

Computer scalability is very similar. Different applications will care about bandwidth or latency in different ways. For example, if you connect to Google Maps, latency is the time it takes for Google to show the map, but in that case (unlike for pregnant women), it is presumably improved because Google Maps sends many small chunks of the map in parallel.

I have already written in an earlier post why I believe HP has a good track record with respect to partitioning and scalability.

Scalability of virtual machines

However, IBM has very harsh words against HP Integrity Virtual Machines (aka HPVM), and describes HPVM scalability as a "downside" of the product:

The downside here is scalability. With HP's virtual machines, there is a 4 CPU limitation and RAM limitation of 64GB. Reboots are also required to add processors or memory. There is no support for features such as uncapped partitions or shared processor pools. Finally, it's important to note that HP PA RISC servers are not supported; only Integrity servers are supported. Virtual storage adapters also cannot be moved, unless the virtual machines are shut down. You also cannot dedicate processing resources to single partitions.

I already pointed out factual errors in every sentence of this paragraph. But scalability is a more subtle problem, and it takes more explaining just to describe what the problems are, not to mention possible solutions... What matters is not just the performance of a single virtual machine when nothing else is running on the system. You also care about performance under competition, about fairness and balance between workloads, about response time to changes in demand.

The problem is that these are all contradictory goals. You cannot increase the performance of one virtual machine without taking something away from the others. Obviously, the CPU time that you give to one VM cannot be given to another one at the same time. Similarly, increasing the reactivity to fast-changing workloads also increases the risk of instability, as for any feedback loop. Finally, in a server, there is generally no privileged workload, which makes the selection of the "correct" answers harder to make than for workstation virtualization products.

Checkmark features vs. usefulness

Delivering good VM performance is a complex problem. It is not just a matter of lining up virtual CPUs. HPVM implements various safeguards to help ensure that a VM configuration will not just run, but run well. I don't have as much experience with IBM micro-partitions, but it seems much easier to create configurations that are inefficient by construction. What IBM calls a "downside" of HPVM is, I believe, a feature.

Here is a very simple example. On a system with, say, 4 physical CPUs, HPVM will warn you if you try to configure more than 4 virtual CPUs:

bash-2.05b# hpvmmodify -P vm7 -c 8
HPVM guest vm7 configuration problems:
    Warning 1: The number of guest VCPUS exceeds server's physical cpus.
    Warning 2: Insufficient cpu resource for guest.
These problems may prevent HPVM guest vm7 from starting.
hpvmmodify: The modification process is continuing.

It seems like a sensible thing to do. After all, if you only have 4 physical CPUs, you will not get more power by adding more virtual CPUs. There are, however, good chances that you will get less performance, in any case where one virtual CPU waits on another. Why? Because you increased the chances that the virtual CPU you are waiting on is not actually running at the time you request its help, independently of the synchronization mechanism that you are using. So instead of getting a response in a matter of microseconds (the typical wait time for, say, spinlocks), you will get it in a matter of milliseconds (the typical time slice on most modern systems).

Now, the virtual machine monitor might be able to do something smart about some of the synchronization mechanisms (notably kernel-level ones). But there are just too many ways to synchronize threads in user-space. In other words, by configuring more virtual CPUs than physical CPUs, you simply increased the chances of performing sub-optimally. How is that a good idea?

IBM doesn't seem to agree with me. First, in their article, they complain about HP vPars implementing a similar restriction: The scalability is also restricted to the nPartition that the vPar is created on. Also, the IBM user-interface lets you create micro-partitions that have too many virtual CPUs with nary a peep. You can create a micro-partition with 16 virtual CPUs on a 2-way host, as illustrated below. Actually, 16 virtual CPUs is almost the maximum on a two way host for another reason: there is a minimum of 0.1 physical CPU per virtual CPU in the IBM technology, and 16 * 0.1 is 1.6, which only leaves a meager 0.4 CPU for the virtual I/O server.

The problem is that no matter how I look at it, I can't imagine how it would be a good thing to run 16 virtual CPUs on a 2-CPU system. To me, this approach sounds a lot like the Fantasia school of scalability. If you remember, in that movie, Mickey Mouse plays a sorcerer apprentice who casts a spell so that his broom will do his chores in his stead. But things rapidly get wildly out of control. When Mickey tries to ax the brooms to stop the whole thing, each fragment rapidly grows back into a full grown broom, and things go from bad to worse. CPUs, unfortunately, are not magic brooms: cutting a CPU in half will not magically make two full-size CPUs.

Performing well in degraded configurations

Now, I don't want you to believe that I went all defensive because IBM found a clever way to do something that HPVM can't do. Actually, even if HPVM warns you by default, you can still force it to start a guest in such a "stupid" configuration, using the -F switch of hpvmstart. And it's not like HPVM systematically performs really badly in this case either.

For example, below are the build times for a Linux 2.6.25 kernel in a variety of configurations.

4-way guest running on a 4-way host, 5 jobs
[linux-2.6.26.5]# gmake -j5
real    5m25.544s
user    18m46.979s
sys     1m41.009s
8-way guest running on a 4-way host, 9 jobs
[linux-2.6.26.5]# time gmake -j9
real    5m38.680s
user    36m23.662s
sys     3m52.764s
8-way guest running on a 4-way host, 5 jobs
[linux-2.6.26.5]# time gmake -j5
real    5m35.500s
user    22m25.501s
sys     2m6.003s

As you can verify, the build time is almost exactly the same, whether the guest has 4 our 8 virtual CPUs. As expected, the additional virtual CPUs do not bring any benefit. In that case, the degradation exists, but it is minor. It is however relatively easy to build cases where the degradation would be much larger. Another observation is that running only enough jobs to keep 4 virtual CPUs busy actually improves performance: less time is spent for the virtual CPUs to wait on one another.

So, why do we even test such configurations or allow them to run, then? Well, there is always the possibility that a CPU goes bad, in which case the host operating system is most likely to disable it. When that happens, we may end up running with an insufficient number of CPUs. Even so, this is no reason to kill the guest. We still want to perform as well as we can, until the failed CPU is replaced with a good one.

In short, I think that HPVM is doing the right thing by telling you if you are about to do something that will not be efficient. However, in case you found yourself in that situation due to some unplanned event, such as a hardware failure, it still does the hard work to keep you up and running with the best possible performance.

Remaining balanced and fair

There is another important point to consider regarding the performance of virtual machines. You don't want virtual machines to just perform well, you also care a lot about maintaining balance between the various workloads, both inside the virtual machine itself, and between virtual machines. This is actually very relevant to scalability, because multi-threaded or multi-processor software often scales worse when some CPUs run markedly slower than others.

Consider for example that you have 4 CPUs, and divide a task into four approximately equal-sized chunks. The task will only complete when all 4 sub-tasks are done. If one CPU is significantly slower, all other CPUs will have to wait for it. In some cases, such as ray-tracing, it may be easy enough for another CPU to pick up some of the extra work. For other more complicated algorithms, however, the cost of partitioning may be significant, and it may not pay off to re-partition the task in flight. And even when re-partitioning on the fly is possible, software is unlikely to have implemented it if it did not bring any benefit on non-virtual hardware.

Loading virtual machines little by little...

In order to get a better feeling for all these points, readers are invited to do the following very simple experiment with their favorite virtual machine technology. To maximize chances that you can run the experiment yourself, I will not base my experiment on some 128-way machine with 1TB of memory running 200 16-way virtual machines or anything über-expensive like that. Instead, I will consider the simplest of all cases involving SMP guests: two virtual machines VM1 and VM2, each with two virtual CPUs, running concurrently on a 2-CPU machine. What could possibly go wrong with that? Nowadays, this is something you can try on most laptops...

The experiment is equally simple. We will simply incrementally load the virtual machines with more and more jobs, and see what happens. When I ran the experiment, I used a simple CPU spinner program written in C that counts how many loops per second it can perform. The baseline, which I will refer to as "100%", is the number of iterations that the program makes on a virtual machine, with another virtual machine sitting idle. This is illustrated below, with the process Process 1 running in VM1, colored in orange.

CPU 1	CPU 2
Process 1	Idle

Now, let's say that you start another identical process in VM2. The ideal situation is that one virtual CPU for each virtual machine gets loaded at 100%, so that each process gets a 100% score. In other words, each physical CPU is dedicated to running a virtual CPU, but the virtual CPUs belong to different virtual machines. The sum of the scores is 200%, which is the maximum you can get on the machine, and the average is 100%. This is both optimal and fair. As far as I can tell, both HPVM and IBM micro-partitions implement this behavior. This is illustrated below, with VM1 in orange and VM2 in green.

CPU 1	CPU 2
Process 1	Process 2

However, this behavior is not the only choice. Versions of VMware up to version 3 used about a mechanism called co-scheduling, where all virtual CPUs must run together. As the document linked above shows, VMware was boasting about that technique, but the result was that as soon as one virtual CPU was busy, the other physical CPU had to be reserved as well. As a result, in our little experiment, each process would get 50% of its baseline, not 100%. This approach is fair, but hardly optimal since you waste half of the available CPU power. VMware apparently chose that approach to avoid dealing with the more complicated cases where one virtual CPU would wait for another virtual CPU that was not running at the time.

CPU 1	CPU 2
Process 1	Idle
Idle	Process 2

Now, let's fire a second process in VM1. This is where things get interesting. In that situation, VM1 has both its virtual CPUs busy, but VM2 has only one virtual CPU busy, the other being idle. There are many choices here. One is to schedule the two CPUs of VM1, then the two CPUs of VM2 (even if one is idle). This method is fair between the two virtual machines, but it reserves a physical CPU for an idle virtual CPU half of the time. As a result, all processes will get a score of 50%. This is fair, but suboptimal, since you get a total score of 150% when you could get 200%.

CPU 1	CPU 2
Process 1	Process 3
Process 2	Idle

In order to optimize things, you have to take advantage of that 'idle' spot, but that creates imbalance. For example, you may want to allocate CPU resources as follows:

CPU 1	CPU 2
Process 1	Process 2
Process 1	Process 3

This scenario is optimal, since the total CPU bandwidth available is 200%, but it is not fair: process 1 now gets twice as much CPU bandwidth as processes 2 and 3. In the worst case, the guest operating system may end up being confused by what is going on. So one solution is to balance things out over longer periods of time:

CPU 1	CPU 2
Process 1	Process 2
Process 1	Process 3
Process 3	Process 2

This solution is again optimal and fair: process 1, 2 and 3 each get 66% of a CPU, for a total of 200%. But other important performance considerations come into play. One is that we cannot keep all processes on a single CPU. Keeping processes bound to a given CPU improves cache and TLB usage. But in this example, one of the processes (at least) will have to jump from one CPU to the other, even if the guest operating system thinks that it's bound to a single CPU.

Another big downside as far as scalability is concerned is with respect to inter-process communication. If processes 1 and 3 want to talk to one another in VM1, they can do so without waiting only half of the time, since during the other half, the other CPU is actually running a process that belongs to another virtual machine. A consequence is that the latency of this inter-process communication increase very significantly. As far as I can tell, this particular problem is the primary issue with the scalability of virtual machines. VMware tried to address it with co-scheduling, but we have seen why it is not a perfect solution. Statistically speaking, adding virtual CPUs increases the chances that the CPU you need will not be running, in particular when other virtual machines are competing for CPU time.

Actual scalability vs. simple metrics

This class of problems is the primary reason why HPVM limits the size of virtual machines. It is not that it doesn't work. There are even workloads that scale really well, Linux kernel builds or ray-tracing being good examples. But even workloads that scale OK with a single virtual machine will no longer scale as well under competition. Again, virtual machine scalability is nowhere as simple as "add more virtual CPUs and it works".

This is not just theory. We have tested the theory. Consider the graph below, which shows the results of the same benchmark run into a variety of configurations. The top blue line, which is almost straight, is perfect scalability, which you practically get on native hardware. The red line is HPVM under light competition, i.e. with other virtual machines running but mostly idle. In that case, HPVM scales well up to 16-way. The blue line below is HPVM under heavy competition. If memory serves me right, the purple line is fully-virtualized Xen... under no competition.

In other words, if HPVM supports 8 virtual CPUs today, it is because we believe that we can maintain useful scalability on demanding workloads and even under competition. We know, for having tested and measured it, that going beyond 8-way will usually not result in increased performance, only in increasing CPU churn.

One picture is worth 2¹⁰ words

As we have shown, making the right decisions for virtual machines is not simple. Interestingly, even the very simple experiment I just described highlights important differences between various virtual machine technologies. After launching 10 processes on each guest, here is the performance of the various processes when running under HPVM. In that case, the guest is a Linux RedHat 4 server competing against an HP-UX partition running the same kind of workload. You can see the Linux scheduler granting time to all processes almost exactly fairly, although there is some noise. I suspect that this noise is the result of the feedback loop that Linux puts in place to ensure fairness between processes. By contrast, here is how AIX 6.1 performs when running the same workload. As far as I can tell, IBM implements what looks like a much simpler algorithm, probably without any feedback loop. It's possible that there is an option to enable fair share scheduling on AIX (I am much less familiar with that system, obviously). The clear benefit is that it is very stable over time. The downside is that it seems quite a bit unfair compared to what we see in Linux. Some processes are getting a lot more CPU time than others, and this over an extended period of time (the graph spans about 5 minutes). The result shown in the graphs is actually a combination of the effect of the operating system and virtual machine schedulers. In the case of IBM servers, I must say that I'm not entirely sure about how the partition and process schedulers interact. I'm not even sure that they are distinct: partitions seem to behave much like processes in AIX. In the case of HPVM, you can see the effect of the host HP-UX scheduler on the total amount allocated to the virtual machine.

Conclusion

Naturally, this is a very simple test, not even a realistic benchmark. It is not intended to demonstrate the superiority of one approach over the other. Instead, it demonstrates that virtual machine scalability and performance is not as simple as counting how many CPUs your virtual machine software can allocate. There are large numbers of complicated trade-offs, and what works for one class of workloads might not work so well for others.

I would not be surprised if someone shows benchmarks where IBM scales better than HPVM. Actually, it should be expected: after all, micro-partitions are almost built into the processor to start with; the operating system has been rewritten to delegate a number of tasks to the firmware; AIX really runs paravirtualized. HPVM, on the other hand, is full virtualization, which implies higher virtualization costs. It doesn't get any help from Linux or Windows, and only very limited help from HP-UX. So if anything, I expect IBM micro-partitions to scale better than HPVM.

Yet I must say that my experience has not confirmed that expectation. In the few tests I made, differences, if any, were in HPVM's favor. Therefore, my recommendation is to benchmark your application and see which solution performs best for you. Don't let others tell you what the result should be, not even me...

IBM's comparison of virtual machines...

2008-10-16T21:53:00.025+01:00

IBM just posted a comparison of virtual machines that I find annoyingly flawed. Fortunately, discussion on OSnews showed that technical people don't buy this kind of "data". Still, I thought that there was some value in pointing out some of the problems with IBM's paper.

nPartitions are tougher than logical partitions

What HP calls 'nPartitions' are electrically isolated partitions. IBM correctly indicates that nPartitions offers true electrical isolation. The key benefit, using IBM's own words, is that nPartitions allow you to service one partition while others are online. You can for example extract a cell, replace or upgrade CPUs or memory, and re-insert the cell without downtime.

The problem is when IBM states that this is similar to IBM logical partitioning. In reality, electrically-isolated partitions are tougher. Like on IBM systems, CPUs can be replaced in case of failure. But you can replace entire cells without shutting down the system, contrary to IBM's claim that systems require a reboot when moving cells from one partition to another.

Finally, IBM remarks that Another downside is that entry-level servers do not support this technology, only HP9000 and Integrity High End and Midrange servers. Highly redundant, electrically isolated partitions have a cost, so they don't make so much sense on entry level systems. But it's not a downside of HP that they only have partitioning technologies similar to IBM's on entry level systems, it's a downside of IBM not to have anything similar to nPartitions on their high-end servers.

Integrity servers support more operating systems

IBM writes: It's important to note that while nPartitions support HP-UX, Windows®, VMS, and Linux, they only do so on the Itanium processor, not on the HP9000 PA Risc architecture. It is true that HP (Microsoft, really) doesn't support Windows on PA-RISC, a long obsolete architecture, but what is more relevant is that IBM doesn't support Windows on POWER even today. This is the same kind of spin as for electrically-isolated partitions: IBM tries to divert attention from a missing feature in their product line by pointing out that there was a time when HP didn't have the feature either.

IBM writes that Partition scalability also depends on the operating system running in the nPartition,. Again, this is true... on any platform. But the obvious innuendo here is that the scalability of Integrity servers is inconsistent across operating systems. A quick visit to the TPC-H and and TPC-C result pages will demonstrate that this is false. HP posts world-class results with HP-UX and Windows, and other Itanium vendors demonstrate the scalability of Linux on Itanium.

By contrast, the Power systems from IBM or Bull all run AIX. And if you want to talk about scalability, IBM as of today has no 30TB TPC-H results, and it often takes several IBM machines to compete with a single HP system.

Regarding scalability of other operating systems, HP engineers have put a lot of effort early into Linux scalability. I would go as far as saying that 64-way scalability of Linux is probably something that HP engineers tested seriously before anybody else, and they got some recognition for this work.

HP offers both isolation and flexibility

IBM also tries to minimize the flexibility of hard partitions. If this is electrically isolated, it can't be flexible, right? That's exactly what IBM wants you to believe: They also do not support moving resources to and from other partitions without a reboot.

However, HP has found a clever way to provide both flexibility and electrical isolation. With HP's Instant Capacity (iCap), you can move "right to use" CPUs around. On such a high-end machine, it is a good idea to have a few spare CPUs, either in case of CPU failure or in case of a sudden peak in demand. HP recognized that, so you can buy machines with CPUs installed, but disabled. You pay less for these CPUs, you do not pay software licenses, and so on.

The neat trick that iCap enables is to transfer a CPU from one nPartition to another without breaking electrical isolation: you simply shut it down in one partition, which gives you the right to use it in another partition without paying anything for it. All this can be done without any downtime, and again, it preserved electrical isolation between the partitions.

Shared vs. dedicated

Regarding virtual partitions, IBM is quick to point out the "drawbacks": What you can't do with vPars is share resources, because there is no virtualized layer in which to manage the interface between the hardware and the operating systems. Of course, you cannot do that with dedicated resources on IBM systems either, that's the point of having a partitioning system with dedicated resources! If you want shared resources, HP offers virtual machines, see below. So when IBM claims that This is one reason why performance overhead is limited, a feature that HP will market without discussing its clear limitations, the same also holds for dedicated resources on IBM systems.

What is true is that IBM has attempted to deliver in a single product what HP offers in two different products, virtual partitions (vPars) and virtual machines (HPVM). The apparent benefit is that you can configure systems that have a mix of dedicated and shared resources. For example, you can have in the same partition some disks directly attached to a partition like they would be in HP's vPars, and other disks served through the VIO in a way similar to HPVM. In reality, though, the difference between the way IBM partitions and HPVM virtual machines work is not that big, and if anything, is going to diminish over time.

A more puzzling statement is that The scalability is also restricted to the nPartition that the vPar is created on. Is IBM really presenting as a limitation the fact that you can't put more than 16 CPUs in a partition on a 16-CPU system? So I tested that idea. And to my surprise, IBM does indeed allow you to create for example a system with 4 virtual CPUs on a host with 2 physical CPUs. Interestingly, I know how to create such configurations with HPVM, but I also know that they invariably run like crap when put under heavy stress, so HPVM will refuse to run them if you don't twist its arm with some special "force" option. I believe that it's the correct thing to do. My quick experiments with IBM systems in such configurations confirms my opinion.

IBM also comments that There is also limited workload support; resources cannot be added or removed. Resources can be added or removed to a vPar, or moved between vPars, so I'm not sure what IBM refers to. Similarly, when IBM writes: Finally, vPars don't allow you to share resources between partitions, nor can you dynamically allocate processing resources between partitions, I invite readers to check the transcript of a 3-years old demo to see for how long (at least) that statement has been false...

Virtual Machines

Finally, IBM also gives false or outdated information regarding HP Integrity Virtual Machines, aka HPVM (not "IVM", which is an IBM product.)

HPVM now supports 8 virtual CPUs, not 4, and it has always been able to take advantage of the much large numbers of physical CPUs HP-UX supports (128 today). Please note that 8 virtual CPUs is what is supported, not what works (the red line in the graph shows 16-way scalability of an HPVM virtual machine under moderate competition using a Linux scalability benchmark; the top blue line is the scalability on native hardware; the bottom curves are HPVM under maximal competition and Xen under no competition.)

IBM also states that Reboots are also required to add processors or memory. , but this is actually not very different from their own micropartitions. In a Linux partition with a kernel supporting CPU hotplug, for example, you can disable CPU3 under HPVM by typing echo 0 > /sys/devices/system/cpu/cpu3/online, just like you would on a real system (on HP-UX, you would use hpvmmgmt -c 3 to keep only 3 CPUs). HPVM also features dynamic memory in a virtual machine, meaning that you can add or remove memory while the guest is running. You need to reboot a virtual machine only to change the maximum number of CPUs or the maximum amount of memory, but then that's also the case on IBM systems: to change the maximum number of CPUs, you need to modify a "profile", but to apply that profile, you need to reboot. However, IBM micro-partitions have a real advantage (probably not for long) which is that you can boot with less than the maximum, whereas HPVM requires that the maximum resources be available at boot time.

IBM wants you to believe that There is no support for features such as uncapped partitions or shared processor pools. In reality, HPVM has always worked in uncapped mode by default. Until recently, you needed HP's Global Workload Manager to enable capping, because we thought that it was a little too cumbersome to configure manually. Our customers told us otherwise, so you can now directly create virtual machines with capped CPU allocation. As for shared processor pools, I think that this is an IBM configuration concept that is entirely unnecessary on HP systems, as HPVM computes CPU placement dynamically for you. Let me know if you think otherwise...

The statement IBM made that Virtual storage adapters also cannot be moved, unless the virtual machines are shut down. is also untrue. All you need is to use hpvmmodify to add or remove virtual I/O devices. There are limits. For example, HPVM only provisions a limited number of spare PCI slots and busses, based on how many devices you started with. So if you start with one disk controller, you might have an issue going to the maximum supported configuration of 158 I/O devices without rebooting the partition. But if you know ahead of time that you will need that kind of growth, we even provide "null" backing stores that can be used for that purpose.

But the core argument IBM has against HP Virtual Machines is: The downside here is scalability. Scalability is a complex enough topic that it deserves a separate post, with some actual data...

Preserving the investment matters

Let me conclude by pointing out an important fact: HP has a much better track record at preserving the investment of their customers. For example, on cell-based systems, HP allowed mixed PA-RISC and Itanium cells to facilitate the transition.

This is particularly true for virtual machines. HPVM is software, that HP supports on any Integrity server running the required HP-UX (including systems by other vendors). By contrast, IBM micropartitions are firmware, with a lot of hardware dependency. Why does it matter? Because firmware only applies to recent hardware, whereas software can be designed to run on older hardware. As an illustration, my development and test environments consists in severely outdated zx2000 and rx5670 (the old 900MHz version). These machines were introduced in 2002 and discontinued in 2003 or 2004. In other words, I develop HPVM on hardware that was discontinued before HPVM was even introduced...

By contrast, you cannot run IBM micro-partitions on any POWER-4 system, and you need new POWER-6 systems in order to take advantage of new features such as Live Partition Mobility (the ability to transfer virtual machines from one host to another). HP customers who purchased hardware five years ago can use it to evaluate the equivalent HPVM feature today, without having to purchase brand new hardware.

Update: About history

The IBM paper boasts about IBM's 40-plus year history of virtualization, trying to suggest that they have been in the virtualization market longer than anybody else. In reality, the PowerVM solution is really recent (less than 5 years old). Earlier partitioning solutions (LPARs) were comparable to HP's vPars, and both technologies were introduced in 2001.

But a friend and colleague at HP pointed out that a lot of these more modern partitioning technologies actually originated at Digital Equipment, under the name OpenVMS Galaxy. And the Galaxy implementation actually offered features that have not entirely been matched yet, like APIs to share memory between partitions.

What is your dangerous idea?

2008-10-15T06:25:00.028+01:00

I've just finished reading John Brockman's What is your Dangerous Idea?. This book is a collection of short essays (108 of them, I believe) by various authors selected by the Edge foundation. This makes it a book that is difficult to summarize or even analyze, but I'll try. If you want to read the essays on-line, you can find them on the Edge site.

An impressionist painting of today's intellectual landscape

One thing can be said upfront: this is a book worth reading, even if you are unlikely to like all of it. It's worth reading, because it exposes the viewpoint of various smart thinkers on a large number of topics. You are unlikely to like all of it, because in this long list of idea, one is bound to trigger disagreement, angst, anger, or some other kind of negative sentiment.

The book covers a large range of topics, largely circling around science (physics, biology, neurophysiology, computer science), with peeks at sociology, religion or morality. Trying to talk about each and every individual idea would be quite pointless. It's clear that the "Dangerous idea" question leaves things quite open, and so there is no real central theme. Some ideas appear quite insightful, others frightening, other depressing, some even seem clearly wrong or overly simplistic to me. But the general impression remains that of an impressionist painting of today's intellectual landscape: it's made of many little strokes, yet you can see trends and general shapes emerging from all these dots.

The dangerous uniformity of bland gray matter

Unfortunately, compared to real impressionists, the picture surprisingly lacks color and intensity, in my humble opinion. This was really the first impression I had after the few chapters of the book: "Hey, why are they all saying the same thing?" Now, to be honest, some of that may be a result of the subtle editorial work of John Brockman, who did a remarkable job of grouping writers who happened to be covering a similar theme together, which seems to give the book a kind of gentle flow.

But still, from such an honorable audience, I expected a lot more: more fire, more contrast, more debate, more disagreement. Oh, sure, many of the ideas are bound to shock a large fraction of the readers, in particular in the US, by claiming for example that we have no soul (John Horgan), that the probability of God is small (Philip W. Anderson), even that science must destroy religion (Sam Harris), that the fight against global warming is lost (Paul C.W. Davies), and so on.

However, while all these thinkers might liken themselves more or less consciously to Galileo or Darwin for their revolutionary ideas, for going against majority thinking, they are still nowhere close to these role models, because they largely agree among themselves and with their peers. The true honor of Galileo or Louis Pasteur was to go against the scientific establishment of the time. By contrast, What is your Dangerous Idea shows an unfortunate uniformity in the thinking of our present scientific establishment.

To illustrate this, it is sufficient to take the first three essays in the print edition: We Have No Souls (John Horgan), The Rejection of Soul (Paul Bloom), and The Evolution of Evil(David Buss). All three talk about a topic that is central to religious thinking, and all three go straight against conventional religious wisdom. Worse yet, they present their ideas under the authority of science, as if the ideas they present were scientific.

For example, John Horgan writes:

The dangerous (probably true) idea I'd like to dwell on is that we humans have no souls.

Paul Bloom confirms:

If what you mean by "soul" is something immaterial and immortal, something that exists independently of the brain, then souls do not exist.

and David Buss, on a different topic, similarly defines evil in a way that entirely ignores traditional morality:

At a rough approximation, we view as evil people who inflict massive evolutionary fitness costs on us, our families, or our allies.

Now, maybe you think that this is just the theme for essays at the beginning of the book? Not really. There is even an essay dedicated to that same ever-present idea, This Is All There Is, written by Robert R. Provine. And in one of the last essays on an unrelated topic, What We Know May Not Change Us (link apparently broken), Barry C. Smith finds it useful to begin his work by echoing exactly the same sentiment:

Human beings, like everything else, are part of the natural world. The natural world is all there is.

This would not be so dramatic if there was, in all this loud concert of consensual voices, at least one lone dissenter daring to proclaim a dangerous idea that would go against that flow, something like: Science has nothing to say about religion, or There may be something else after all. This would be the expected high-contrast picture of healthy scientific skepticism. Instead, what we have is bland gray matter, pun intended.

Somehow this all reminds me of a cartoon by French cartoonist Sempé, which unfortunately I can't find on the web, where dozens of characters in a building were all saying I'm fortunate not to have seen that happen in my lifetime, but I'm afraid that you kids will live in a world where everybody believes the same thing. Hilarious, and so true...

How could you tell if you are in the Matrix?

The irony is that there is, actually, one essay that explains very well why the general consensus expressed by all these thinkers is, in reality, illogical, and why it should quickly appear untenable to such bright minds. The essay I'm referring to is: We Are All Virtual, by Clifford Pickover. In it, Pickover argues that there are more chances for us to be virtual than real. The idea is that we are building ever more realistic virtual worlds, and that at one point, we will certainly be able to make them appear as realistic to us as our own dreams. Since there will be more such virtual environments than real ones, chances are that you are currently in one.

I should know about virtualization. As the original designer of HP Integrity Virtual Machines, HP's virtualization technology for servers, I am acutely aware that it's possible to fool the most "reality-aware" form of software, the only form of software that routinely interacts with real, physical devices, namely operating systems. In other words, we can make an operating system such as Linux, Windows, HP-UX or OpenVMS believe that it's running on some real hardware, with real disks, real network cards, a real console, and so on, when in reality, everything is pure software. This is why I'm often describing my own job by saying that I'm coding The Matrix.

Is there any reason the brain's "software" can't be fooled that way? If there is one, I don't know it. It is certainly not widely known. Consequently, claims that there is nothing beyond natural reality, beyond the sum of our sense, are vacuous at best and fraudulent at worst. They are a belief, not science. Worse yet, from a scientific point of view, these claims fly in the face of simple technological and scientific evidence, from virtual machines to the psychology of dreams.

Ernst Pöppel also makes a similar point about the limits of what we can know in How Can I Trust, in the Face of So Many Unknownables (the on-line version differs somewhat from the print version). So that viewpoint is somewhat common, and as well it should. But to get back to my initial comment about uniformity of thought, what I find unfortunate is that both Pöppel and Pickover carefully wrap this simple point in a consensual, conflict-avoiding shroud of anything-but-religion. In today's scientific community, it is apparently acceptable to write We are all virtual, but Gödel forbids that you'd write Chances are that the natural world is not all there is... even if the two sentences are essentially equivalent

My dangerous idea: there is a supernatural world

Now, from the reasoning above, you probably understand that what I truly believe is that we really can't tell if there is something beyond the natural world or not. This is not exactly new philosophy. After all, it's quite similar to Pascal's Wager. But the obvious next question is: is there any way to tilt the balance one way or another?

In order to address this question, I need to point out that everything we "know" is really something we believe more strongly than the rest. Pöppel's essay doesn't say anything else, actually. In order to "know", we must trust in a number of things, in general simply because we have been told about them, or because our senses report them. Sometimes, this approach leads us to hold as self-evident beliefs that are spectacularly false. But in general, it is a good working model we can use to drive our own actions. Trust in our own beliefs is the main driver that guides everyone of our actions.

Now, where do religions come from? They come from a testimony. According to the scriptures common to the majority of religious people on Earth today, someone once told a guy named Moses that reports of his nonexistence were vastly exaggerated. As I wrote earlier in this blog, I am that I am is actually quite profound: someone claims to exist without the need for anything else. This is definitely not the case for us: can we ascertain that we can exist without a brain or a physical body? Someone out there, allegedly, once said that He can.

Everything considered, monotheist religions are nothing but faith in such testimony. Unlike the characterization of religion given incorrectly at various places in Dangerous Idea, truly religious people do not claim that they know God, only that they know God spoke to us (Catholics often refer to God as "The Word" for that very reason.) In that respect, there is very little difference between believing that God exists and believing that George Bush exists, there is little difference between believing that God wants us to love one another and believing that George Bush wants to eradicate the axis of evil. In both cases, it's mostly hearsay, testimony from someone who heard about something that someone else once heard, or indirect testimony through the Bible or newspapers.

Answering the Question Personally

There is one big difference, though. At least in the Catholic Faith (I'm not too sure about others), it is possible to speak to God directly, something called "prayer". All things considered, this is probably much easier than speaking to George Bush directly. What the Church tells us is that we have some inner sense that allows us to listen to God. It is more subtle than hearing or seeing. It takes some faith to believe that it works. But thousands of people speak to God today.

To them, saying that God doesn't exist is like saying that their parents don't exist. Try this with your friends: "I'm sorry, Sir, but my computer has no trace of your father, so he doesn't exist." Of course, many believers will acknowledge that their faith may be a fluke in their brain, that the answers to their prayers may just be chance and random coincidence.

But in that respect, if we are all virtual, George Bush may also be a fluke in my brain for all I know, and the fact that my wife cooks a delicious dinner may also be a pure coincidence totally unrelated to her love for me. Still, in both cases, scientists beloved Ockam's razor recommends that I act under the hypothesis that both George Bush and my wife's love are "real", whatever that word means. And believing in God is not more stupid than this belief, no matter what too many scientists in Dangerous Idea proclaim.

So here is why the idea that there is a supernatural reality is dangerous: if this person named God is real, and if He is who He claims to be, then we'd better listen carefully. After all, His message is not so bad, it's all about love. Aren't we lucky not to be in a world where the creator constantly yells "How can I make you miserable, you unworthy pieces of carbon-based junk"? Granted, the message is all garbled and difficult to comprehend. But then, my virtual machines probably also have a very poor understanding of my "reality".

Other gems in the book

I can't finish this review without mentioning that there are a number of essays in this book that stand out for a reason or another. The usual suspects talk about physics (e.g. Leonard Suskind, Lee Smolin, Brian Greene and Carlo Rovelli), and defend their now relatively well known core ideas with the usual brio.

As usual, I find Carlo Rovelli to be the most insightful of the bunch, pointing out that somehow, the two core discoveries of twentieth century physics have not yet been assimilated by most, including by physicists:

Many of today's audacious scientific speculations - about extra dimensions, the multiverse and hte like - are not only unsupported experimentally, but also quite often formulated within a worldview that has not even fully digested quantum mechanics and relativity

Unfortunately, for most readers, it may not be entirely clear what he means by that, even if he does a good job at explaining it in the rest of the essay. Rovelli's core idea, I believe, is that Relativity demonstrated that the notion of "fixed background" where events play out is a persistent illusion, one that is appropriate from our low-energy, low-speed environment, but totally wrong in general. As Brian Greene explains in one of his books, if you walk towards Andromeda or away from it, the "now" in the Andromeda galaxy shifts backward or forward by a few thousand years...

Yet, what Rovelli complains about is that some theories today are still written on some absolute background. This is certainly not the case for my own pet theory, which removes yet another "absolute background" (in my case, absolute variables such as x, t or m, as opposed to relative measurements of the corresponding physical entities, length, time or mass). But in general, I believe that he has a point. It's a shame that relativity or quantum mechanics are still presented as dangerous ideas.

Computer geeks rule!

But the pearl of wisdom in the whole book, in my opinion, goes to Kai Krause for the strangely titled Anty Gravity: Chaos theory in an All-To-Practical Sense, probably in large part because it feels so "original" compared to all other essays. The Anty spelling is not a typo: it's both in the title and in the contents table, and the essay does talk about ants. Kai's essay is about what he calls "Super Individualism", the need for every single one of us (myself included) to feel unique and special, and how this goes against the need of the many:

What if each ant suddenly wants to be the queen? What if soldiering and nest building and cleaning chores are just not cool enough anymore? If AntTV shows them, every day, nothing but un-Ant behavior...?

He concludes with:

Next year, let's ask for good ideas, not dangerous ones. Really, practical, serious, good ideas, like: "What is the most immediate positive global impact of any kind that can be achieved within one year?"

And when I read that, I was proud to be a computer geek: in this large collection of essays from a large pool of bright minds, the one that really stood out from the rest was from a fellow geek...

Links and Links

2008-10-14T06:26:00.004+01:00

Here is a semi-random collection of links about various topics I have covered on this blog at some point in time:

John Timmer on Ars Technica discusses how Scientific publishing might create a winner's curse, according to a recently published paper. The economic angle should not have us forget that about the bigger problem of information overload and intelligent content filtering (which anonymous review certainly isn't).
Machines have never been so close to fooling humans and passing the Turing test. But then, conversations held over chat always seem a little bit "disconnected", without real substance, even with real humans. Fooling 30% of the audience is a good first step, but that's definitely not enough.
NeuroSky and Square Enix announced the first "brainwave-controlled game". Much like Alpha Waves (in my biased opinion the first real 3D game) looks quite crude today, these brainwave-controlled games are probably only the very beginning...
There has been a lot of buzz about the Microsoft Touchless SDK, but watching the videos, one can't help but notice a rather blatant rip-off of CamSpace...
Debate about C++ rages on, with Eric Raymond announcing an upcoming article about C++ considered harmful, to which Indogo Jo replies that C++ is the ugly useful language. It's interesting that Eric Raymond's earlier post is a cristicism of the Unix Hater's Handbook which, I think, applies fittingly well to the little he showed of the C++ article...

The new C++ standard

2008-09-26T10:10:00.025+01:00

DevX.com has a special report on the new C++ standard, currently referred to as C++ 0x because it was due sometime between 2000 and 2009. Well, the standard was quite late compared to early expectations, so an easy joke was that we might end up with 0xA or 0xB, the C++ hexadecimal notation for 10 or 11. Ultimately, chances are that the standard will make it for 2009, so we will probably refer to it as C++ 09...

This new iteration of the language is of interest to all programmers, because it brings a number of major changes to one of the most popular programming languages today, and one that is already very complex (and therefore hard to extend). But for me in particular, it is all the more interesting to consider how various "innovations" in that new standard compare to features that play the same role in XL.

Concepts?

One of the major features in C++ is concepts. DevX has a dedicated article about concepts. In short, concepts in C++ are a way to describe categories of templates, and to help the compiler figure out what the programmer intended for a given template. This new aspect of the language makes it easier to define a real contract between the users of a template and its implementers.

C++ concepts, however, are somewhat annoying to me. One reason is that XL has been for a long time based on an approach that I dubbed concept programming. Concept programming, in the XL sense, is about the relationship between concepts that exist only in our head, and concept representations that exist in the computer. The key idea is to make sure that implementations look and feel like the concepts they represent.

One key consequence of that idea is that a programming language should comfortably support arbitrary concepts, not some finite set (e.g. functions or objects), because the set of concepts we manipulate is not a-priori limited. This is the key reason so much effort was put into making XL extensible.

To summarize, "concepts" in XL are only very remotely related to "concepts" in C++, although, arguably, the XL usage of the word is closer to the standard meaning.

XL generic validation = C++ concepts

Many aspects of XL are a direct consequence of the concept programming design philosophy. For example, XL implemented, since at least 2002, the idea that one can describe how a generic type can be used. This feature is called generic validation in XL terminology. I invite the reader to compare the XL implementation of a minimum function with the C++-with-concepts implementation of the same. This should convince you that the two ideas are basically almost identical.

So where are the differences between C++ concepts and XL generic validation? One of them is how the contract is being specified. In C++, you specify the kind of operators and functions that define the concept. For example, you would write something like the following to indicate that a min function requires a less-than operator:

concept LessThanComparable<typename T> {
  bool operator<(const T& x, const T& y);
}

template<typename T>
requires LessThanComparable<T>
const T& min(const T& x, const T& y) {
  return x < y? x : y;
}

In XL, by contrast, you give an example of code that has to compile with the generic type you want to validate. For example, in XL, you would write something like:

generic type ordered where
    A, B : ordered
    Test : boolean := A < B

function Min (X, Y : ordered) return ordered is
    if X < Y then
        return X
    else
        return Y

Now, as you can see from this simple example, a significant difference is that XL considers the validation to be tied to a generic type, which can then be used to declare a function like Min directly. In other words, since you declared that ordered is generic, Min becomes implicitly generic. By contrast, in C++, LessThanComparable is a kind of predicate that applies to template classes, so you need one additional "connection" using the require statement, to let the compiler relate the T in the definition of min with the T in LessThanComparable. As a result, the C++ code for that example is more verbose and more convoluted. This becomes more visible as the code becomes more complex.

Another drawback is that the C++ concept specification as written doesn't work for, say, int because the less-than operator in that case doesn't have the right signature. So you need an additional concept_map in that case, making the code even more verbose, as shown below:

concept_map LessThanComparable<int> { }

One benefit of the C++ approach, however, is that the specification of the concept makes it easier to validate early that the implementation actually doesn't require anything besides what is declared in the concept. For example, if the body of min attempts to refer to an operator that is not present in the concept specification, the compiler may detect this. Doing this with the kind of specification given in XL is much more complicated. I am considering various ways to fix this problem, which is much easier in XL since practically nobody uses it yet.

Multitasking and Threads

C++ 0x also adds standard support for threads. In my opinion, it is ironic that they manage to shoe-in support for a thread model that is so "last century". Today, the difficult problem is not threading on a SMP system, but threading on non-uniform architectures, for example threading between a CPU and a GPU, or between the components of a Cell microprocessor, or threads that cooperate on machines with different architectures across the Internet.

This kind of problem is much more complicated, and is already, to some extent, solved by other languages such as Java or Erlang.

At this point, XL has little to offer in that space, because what is needed is not coded yet. However, I am confident that XL's extensibility will make it easy to implement not one, but a multitude of tasking models. Among the top candidates are rendez-vous based mechanisms similar to Ada, message-passing protocols similar to Erlang, or data-driven parallelism similar to several functional languages. Stay tuned.

Variadic templates

C++ 0x will, at long last, implement variadic templates. This feature will make it possible to write functions that take a variable number of arguments, yet are type-safe.

This is, again, something that existed in XL since 2001 or earlier. You can see that XL implementation of the Max function takes advantage of this feature.

The C++ implementation is more complete, however, as it makes it possible to create not just variadic functions, but also variadic classes. This is something that is planned, but not currently implemented in XL.

Range-based iterations

A new range-based iteration mechanism was also added to C++ 0x. XL has a more general form of iteration, that already covers this specific case. Here is for example how for loops are declared in XL:

iterator IntegerIterator(
    var It : integer;
    Low, High : integer
  ) written It in Low..High is
    It := Low
    while It <= High loop
        yield
        It := It + 1

The notation It in Low..High is how you will invoke the iterator, and the yield statement in the iterator is where the body of the loop will go. The usage of the iterator is very natural:

for I in 1..5 loop
    for J in 1..I loop
        WriteLn "I=", I, " and J=", J

The benefit of this more general approach is that you can for example define two-variable iterators:

iterator MatrixIterator (
    var I : integer; LI, HI : integer;
    var J : integer; LJ, HJ : integer
  ) written I,J in [LI..HI, LJ..HJ] is
    I := LI
    while I <= HI loop
        J := LJ
        while J <= HJ loop
            yield
            J := J + 1
        I := I + 1

for A, B in [3..5, 7..9] loop
    WriteLn "A=", A, " and B=", B

You can also define iterators over any kind of data structure, using any syntax you need for this particular data structure.

Constant Expressions

C++ 0x introduces the notion of generalized constant expression. This makes it possible to declare functions that the compiler will be able to evaluate at compile time.

Once again, the XL approach is very different. The XL compiler has various phases, implemented as "plug-ins" for the compiler. One of them deals with constant folding (i.e. evaluation of constant expressions). Here is an example showing how to compute factorials at compile-time using that technique.

The XL pre-processor also makes it easy to implement compile-time assertions, something that is also a new feature of C++ 0x. The XL implementation, however, will automatically optimize a static assertion if it can evaluate the argument at compile time, instead of requiring a specific keyword.

Conclusion

C++ is an extremely complex language, and extending it took a lot of effort. Many of the new features have already existed in XL for a while, and are much easier to implement. However, the implementation in C++ points out some weaknesses in the way things are currently done in XL, something that is fortunately still easy to change that early in the language's life.

Multiple operating systems on a single tomato

2008-09-13T08:56:00.005+01:00

If you want to learn something about virtualization, this video is not necessarily the most extensive technical explanation you might get, but it's short and it's funny. And Debbie is really our program manager.

There are many other videos here

It's alive!

2008-09-10T19:39:00.007+01:00

Every single blog or web site that vaguely talks about physics is going to tell you about the LHC today... And nobody else has found the Higgs Boson yet... As Bee noticed, this has some people frightened.

There is no reason to worry, most physicists would tell you, the LHC is orders of magnitude to puny to be really dangerous... To create a really dangerous black-hole machine, physicsists would really need a few trillion dollars more... Or whatever the unit is above trillion...

Still, if we were going into a singularity, would we even see it?

Really bad math in a sci-fi book...

2008-09-08T08:01:00.010+01:00

I have just been reading Alien Embassy by Ian Watson (in the French translation published by Presses Pocket). This is a somewhat esoteric book, and I can't say that I'm really fond of that genre. But it is an interesting story, it's well written, and I enjoyed it.

Howewer, something really puzzled me, and it has to do with what seems to be the deep (mis)understanding of relatively fundamental mathematics, namely irrational or imaginary numbers, somewhere in the first third of the book.

Irrational numbers

Unfortunately, I can't quote the original text, only back-translate from French, which may be hard to map on the original text. It reads something like that (I'd appreciate if some reader could give me the actual original text):

- He's talking about mathematics, whispered the voice of Klimt. Irrational numbers are numbers like pi, the constant ratio between the circumference and the diameter of a circle, you must know that.

- That's about twenty two seventh, I added; I knew at least that!

- It's a very important number. Without it, geometry could not exist, Klimt commented. It represents a true geometrical relation. Pi appears as soon as you draw a circle. Yet it is totally irrational. There is no explanation to the sum "tweny two seventh". You can divide twenty two by seventh as long as you want, you will never get a really definitive answer. [...]

This is bogus twice.

First, the author reasons about 22/7 instead of reasoning about pi. The two numbers are really not that close. To the fourth decimal, 22/7 is 3.1428 whereas pi is 3.1415. It's really bizarre to talk about the irrational nature of pi by using the different number 22/7 that just happens to share 3 digits with it...
But second, and more importantly, irrational in mathematics means precisely that the number cannot be identified to the ratio of any two integers. That seems to have eluded the author entirely, as he seems to think that 22/7 itself is irrational, and that "irrational" denotes a number with an infinite number of decimals, like 1/3 = 0.33333...

Imaginary numbers

There is a similar issue a couple of pages later:

- From what I know, our scientists consider mathematics as imaginary dimensions, said Klimt - Imaginary dimensions? the Azuran fluttered. Imaginary? Ah, but that's where you got it wrong. These other dimensions are all but imaginary. They really exist. [...]

Here, multiple layers of a very strange understanding of mathematics and physics mix up. I don't think any scientist ever considered mathematics as a whole to be imaginary dimensions. Mathematics use symbols and relations between symbols, and I think that most mathematicians would consider the mathematics we can ever talk about as a countable set. There are imaginary numbers, a terminology that refers to a class of complex numbers like i that have a negative square (e.g. i²=-1).

Where imaginary dimensions show up is for example in some explanations of Einstein's special relativity, where time is considered as an imaginary dimension, as opposed to three real spatial dimensions. This is a mathematical trick to account for the form of distance in space-time, which has three squares with the same sign and one with a different sign (ds² = − (dt²) + dx² + dy² + dz²).

Why does it matter?

Why bother about two little errors in what is, after all, simply intended to be entertainment? Well, it shows the limits of terminology when we cross domain boundaries. What seems to have confused the author in both cases is that a terminology in mathematics has a very precise meaning that happens to be really far from the everyday meaning of the same word.

The problem is that this gave the author, and possibly his readers, a false sense of understanding. Curiously, this echoes in my mind a number of issues I had with popular science books, some of which were written by well known names in physics. And it is also a problem with the operating systems people use on computers all the time: we tend to forget that a "desktop" or a "window" has nothing to do with the common acceptation of the term, which may confuse beginners quite a bit.

I'm not sure there is much to do about it, though, but to gently correct the mistakes when you see them. After all, short of inventing new terminology all the time (like "quarks" or "widgets"), we just have to proceed by analogy (e.g. "mouse", "complex numbers") and then stick with that wording. We consciously forget that the word is the same, the concepts end up being quite different in our brain.

But this has implication in another area I am interested in, concept programming. We give names to concepts, but these names overlap. Our brains know very well how to sort it out, but only after we have been trained. As a result, the big simplification I was expecting from concept programming might not happen, after all...

Page charges...

2008-09-08T07:54:00.002+01:00

Part of the problem I ran into for getting a physics article published was that you have to pay to get published. This is something that physicists usually accept, because they don't directly pay for it: their institution does. For me, this was a problem, though: I could hardly justify to my family to pay a few hundred bucks or more just for the "vanity" of getting published, even less so when Internet gives me a perfectly good outlet to reach a wider audience at no incremental cost.

But I just discovered that there is another category of people that is hurt by these page charges: retired physicists... Retired = can't publish? That's odd...

Self-evident?

2008-09-07T17:43:00.012+01:00

There are a number of things that we have heard so often that we simply take them for granted. They become absolute truths. One of these is that we have five senses. This is so "true" that we use an expression like The Sixth Sense to refer to some extraordinary sense, typically extra-sensory perception (e.g. telepathy, or in the case of the movie, the ability to see dead people.)

And then today, for some reason, I started counting my senses. Sight, that's one. Hearing, that's two. Touch, that's three. Smell, that's four. Taste, that's five. Good so far. And then, I thought: "but my sense of balance, where does it fit? Is that touch?" And then "What about hot and cold?" That can't be touch: I don't need to touch something to feel that it's hot, the radiations it emits are enough...

That got me thinking. How many senses do we really possess?

It turns out that this "truth" that there are five senses is entirely wrong. According to the Wikipedia article on senses, we have no less than 11 different senses:

The traditional five senses are sight, hearing, touch, smell, taste: a classification attributed to Aristotle. Humans also have at least six additional senses (a total of eleven including interoceptive senses) that include: nociception (pain), equilibrioception (balance), proprioception & kinesthesia (joint motion and acceleration), sense of time, thermoception (temperature differences), and in some a weak magnetoception (direction).

You may draw your own conclusion... Mine is this: some conceptions may be very widespread, and yet entirely wrong when give even a moderate amount of thought. A corollary is that our species seems to be very uncritical of what we hear, of what is asserted by some authority. It took me nearly 40 years to "discover" that we did not have five senses. Clearly, I had all the knowledge it takes to know it, or at least discover it, yet it had never materialized in my brain, for some reason. In retrospect, that makes it much easier for me to understand very negative reactions to innovation or "thinking different" I have witnessed in the past.

But here is the real question that follows from this experience: how much knowledge do we bury under conventional wisdom?

Slashdot on the relevance of academic journals

2008-06-09T13:02:00.007+01:00

I've written a few times about peer reviews, in particular advocating non-anonymous peer reviews. The idea is not popular.

That's why I was so interested to see an article on Slashdot asking if academic journals are still relevant. Do they still ensure quality? Are they nimble enough? Can they compete with the information glut on the Internet? All these are really relevant questions. One reader expresses some of my own thoughts about anonymous peer reviews:

There are certainly parts of the peer review process that are less than ideal--reviewers don't take the time to understand what they're reviewing, or they have an emotional reaction to something that seems to undercut their fond hopes for how something will turn out and make stupid, picky attacks on a paper, or they realize that they're about to get scooped and so ask for every pedantic little thing so they gain more time for their own work. But even with these flaws, the process does a pretty good job at rejecting junk; it just rejects a little too much non-junk, too, or at least makes the process more painful than necessary.

Interestingly, Slashdot itself has some of the properties one might want in a modern-age on-line academic journal: quick to respond to news, filtering data with the help of its readers, offering support for anonymous as well as non-anonymous comments (and favoring non-anonymous ones slightly). In that sense, it is also similar to Wikipedia, or in a different direction, the ArXiv, which both share much of these properties.

Yet all these examples also show the limits of the system. They are not recognized as offering the highest quality information (even if, in reality, they do most of the time), because bad information is shoved through them all the time and there is no guarantee that what you read will not be garbage. And, paradoxically, while they are openly democratic by construction, these outlets give enormous power (censorship, favoring certain view, or even rewriting contents) to a small circle of individuals.

So the question is: how would you fix these flaws? I have ideas, but I'm interested in hearing yours.

The Singularity has already happened...

2008-06-04T06:56:00.009+01:00

IEEE Spectrum has a special report about the Singularity, that point in our future where predictions fall apart because major technical changes make any extrapolation we may make based on today's trends essentially obsolete. Even the New-York Times has an article, entitled The Future Is Now? Pretty Soon, at Least, which quickly brushes up some of the ideas.

The special issue in IEEE is more extensive. There are many interesting articles. In one of them, Ray Kurzweil, arguably the inventor of the concept of Singularity, debates with Neil Gershenfeld, and Vernor Vinge shares what he sees as the signs of the Singularity.

One important point, I believe, is that "there will be a singularity at time t" is a proposition that might depend on the time it's being enunciated. It seems very likely to me that when you are in the middle of a singularity, you have no idea that it's there. That's why I am a bit wary of the use of a singular noun, the singularity, when I think really that there have been many singularities over the course of history.

How could someone from the middle-age, for example, predict the structure of a society after motorized personal transportation became not only possible, but mainstream and relatively cheap (I know, I know, gas prices...)? In other words, seen from the middle-age, the invention of the automobile or, even more so, the airplane, were singularities that might be predicted (e.g. by Leonardo da Vinci), but whose impact on society was really difficult to grasp. The same is true for remote communication, from the telephone to television to the Internet.

Now, one singularity is somewhat special, and it's when we started building enhancements to our intelligence, and not just our physical abilities. That's the very definition Vernor Vinge uses when he writes:

I think it's likely that with technology we can in the fairly near future create or become creatures of more than human intelligence. Such a technological singularity would revolutionize our world, ushering in a posthuman epoch.

But that already happened. The first modest pocket calculators enabled computations so complex that they completely changed the course of engineering. Any engineer with a calculator has "more than human intelligence", for he can compute faster than any human being without a calculator can. It's only recently that we redefined intelligence to exclude the ability to perform computations, and the only reason we did that is because computers were so much better at it than we are.

So that's my personal view on that question: the most important singularity, the one that Ray Kurzweil sees sometime in the future, has already happened, and we are right in the middle of seeing its effects.

Inside a TRS-80 model 100

2008-06-04T06:28:00.005+01:00

PC World opens the guts of a TRS-80 model 100, a vintage computer that was one of the first truly portable computers. Unfortunately, that's not one I have in my collection, so if you happen to have one... There is also a link in this story to the most collectible PCs of all time, and it turns out I have only three of them, not counting pieces of some as yet unidentified Cray which I doubt is a Cray 1.

I remember seeing the TRS-80 model 100, and beeing unimpressed. What made it so popular among journalists, the set of built-in applications, to some extent lowered its value to young geeks. To me at the time, it looked way too much like a largely oversized business thingie. I was much more impressed by the Canon XO-7 at the time. It was not quite as "big" in terms of features (who needs 32K of memory or all these built-in applications?), but it could be connected to a TV and had this very cool plotter.

Back to the TRS model 100, I think that the most interesting part of the story is that this is the last time Bill Gates wrote a significant fraction of the software for a prodduct. And you can hear from the way he describes it that he was really excited about the business uses, about what you could do with the product. Of course, that software crashed from time to time...

Another thing to remember in these days of "green" is that this machine ran for 20 hours on 4 standard AA batteries!

Palais de la Découverte...

2008-05-28T20:30:00.002+01:00

The Palais de la Découverte in Paris is one of these lovely spots where science and fun mix. There is a a rumor that the French government might be planning to close it down (sign the petition to save the place).

This is too bad: this may be the only place in the world that has an entire room that is dedicated to a number that is almost, but not quite, entirely unlike π. More specifically, a mathematician named William Shanks spent 20 years of his life computing 707 decimals of π, but only 528 were correct. In 1937, the bogus result was cast in stone in the Palais de la Découverte. The mistake was only discovered in 1945.

In any case, if you think a place like this is worth saving, just click here.

Where did the HP Way go?

2008-05-24T08:25:00.056+01:00

Recently, I discussed with some HP colleagues about the old "HP Way". This happens a lot, actually. I'd say that this is a topic of discussion during lunch at HP maybe once a week, in one form or another.

Employees who were at Hewlett-Packard before the merger with Compaq, more specifically before Carly Fiorina decided to overhaul the corporate culture, will often comment about the "good old days". Employees from companies that HP acquired later, most notably Compaq or DEC, are obviously much less passionate about the HP Way, but they generally show some interest if only because of the role it used to play in making HP employees so passionate about their company.

Oh, look, the HP Way is gone!

One thing I had noticed was that the "HP Way" was nowhere to be found on any HP web site that I know of. It is not on the corporate HP History web site, nor does a search for "HP Way" on that site get any meaningful result. It's possible that there is a better search string that would get the result, it may even be somewhere I did not look, but my point is that it's not very easy to find. (Update: Since one reader got confused, I want to make clear that I'm looking for the text of the HP Way, the description of the values that used to be given to employees, which I quote below. I am not looking for the words "HP Way", which are present at a number of places.)

Contrast this with the About HP corporate page in 1996, and what do you see here as the last link? Sure thing: the HP way is prominently displayed as an essential component of the HP culture. Every HP employee was "brainwashed" with the HP Way from his or her first day in the company. No wonder that years later, they still ask where it's gone. Back then, the HP way even had its own dedicated web page.

Did someone rewrite HP corporate history?

So the fact that there is no reference to it anywhere on today's corporate web site seems odd. It almost looks like history has been rewritten. It get the same feeling when I enter the HP building in Sophia-Antipolis. Why? Because there are two portraits in the lobby: Bill Hewlett and Dave Packard. That building was initially purchased and built by Digital Equipment Corporation. If any picture should be there, it should show Ken Olsen, not Hewlett or Packard, nor Rod Canion for that matter.

I would personally hate to have built a company that left the kind of imprint in computer history DEC left, only to see it vanish from corporate memory almost overnight... Erasing the pictures of the past sounds much more like Vae Victis or the work of George Orwell's Minitrue than the kind of fair and balanced rendition of corporate history you would naively expect from a well-meaning corporate communication department.

Google can't find the HP Way either...

But the truth is, I don't think there is any evil intended here. One reason is that Google sometimes has troube finding the HP Way too. Actually, your mileage may vary. I once got a link on the HP alumni as the second result. But usually, you are much more likely to find Lunch, the HP way, an extremely funny story for those who were at HP in those days (because it is sooo true).

As the saying goes, "never ascribe to malice that which is adequately explained by studidity". As an aside, this is sometimes attributed to Napoleon Bonaparte, but I can't find any French source that would confirm it. I think that the closest confirmed equivalent from Bonaparte would be "N'interrompez jamais un ennemi qui est en train de faire une erreur" (never interrupt an enemy while he's making a mistake), but that is not even close.

Back on topic, I'm tempted to think that it's simply a case where the HP Way was no longer considered relevant, and nobody bothered to keep a tab on it in the HP corporate web site. As a result, when looking for "HP Way" on the web today, it's become much easier to find highly critical accounts of HP than a description of what the HP Way really represented. Obviously, that can't be too good for HP's image...

Where can we find the HP Way today?

To finally find a reference to the HP way as I remembered it being described to HP employees, I had to search Google for a comparison with Carly Fiorina's "Rules of the garage". And I finally found a tribute to Bill Hewlett that quotes both texts exactly as I remembered them.

As the link to the historical HP way page on the HP web site shows, another option is to use the excellent Wayback Machine to look at the web they way it used to be at some point in the past. But that's something you will do only if you remember that there once was something to be searched for. Again, the point here is not that you cannot find it, it's that finding the original HP Way has become so much more difficult...

The original HP Way: It's all about employees

The original HP Way was not so much about a company as it was about its employees:

The HP Way
We have trust and respect for individuals.
We focus on a high level of achievement and contribution.
We conduct our business with uncompromising integrity.
We achieve our common objectives through teamwork.
We encourage flexibility and innovation.

There are only five points and very few words. It's a highly condensed way to express the corporate policy, which trusts the employees to understand not just the rules, but most importantly their intent and spirit. There is no redundancy, each point is about a different topic. These rules have been written by engineers for engineers. It's almost a computer program...

The rules of the garage: What was that all about?

By contrast, the so-called "Rules of the Garage" introduced by Carly Fiorina, look really weak:

Rules of the Garage
Believe you can change the world.
Work quickly, keep the tools unlocked, work whenever.
Know when to work alone and when to work together.
Share - tools, ideas.
Trust your colleagues.
No politics. No bureaucracy. (These are ridiculous in a garage.)
The customer defines a job well done.
Radical ideas are not bad ideas.
Invent different ways of working.
Make a contribution every day.
If it doesn't contribute, it doesn't leave the garage.
Believe that together we can do anything.

To me, it sounds much more like the kind of routine you'd give to little kids in kindergarden... It's not precise, highly redundant. More importantly, the relationship between the company and the employee is no longer bi-directional as it was in the HP Way. Notice how "we" changed into "you" except in the last one. If I were cynical, I'd say that this new set of rules is: "you do this, and we'll get the reward"... Isn't that exactly what happened in the years that followed?

The Rules of the Garage did not last long. They quietly went the way of the dodo, but I don't think it was ever intended for them to last decades as the HP Way had. Instead, I believe that the problem was to evict the HP Way without giving the impression that nothing replaced it. But in reality, nothing replaced the HP Way: the Rules of the Garage were essentially empty, and after the Rules of the Garage, there was nothing...

How is that relevant today?

Despite its long absence on the HP Corporate web site, the HP Way is still seen as the reference for a successful corporate culture nowadays. It's widely recognized that HP and the HP culture ignited the Silicon Valley. There is a good reason for that: highly creative people are what makes this economy thrive. See Phil McKinney's ideas on the creative economy to see just how relevant this remains today. But guess what: creativity is motivated by the confident belief that there will be a reward.

The HP Way was about the various aspects of that reward: respect, achievement, integrity, teamwork, innovation. I can't think of much beyond that in terms of recognition. I can't think of better reasons to work hard. Now, I don't care much about calling it the HP Way, but I do care about these values being at the core of what my company does. This is not by accident if the top technology companies in the world, including a large fraction of the Silicon Valley, have applied the HP Way in one form or another. It's not charity, it's simply the most efficient way to do business when the majority of your employees are highly creative individuals.

With all the respect that I have for HP's current management, as far as corporate culture is concerned, they could still learn a thing of two from such history-certified business geniuses as Hewlett and Packard. About eight years after having been actively erased from corporate communication, the HP Way is still very much being talked about; it is still regarded as a reference. Maybe that's a sign that there is something timeless about it...

Update: HP Way 2.0?

Today, Google "HP Way" and HP's corporate values show as the second entry. No keyword stuffing here (I checked), but Google apparently decided that the page was relevant to the topic somehow. It's a good thing: the keywords in HP's corporate objectives are much closer to the old HP Way than to the Rules of the Garage. The five original keywords, respect, achievement, integrity, teamwork, innovation, are all there. New keywords have been added, including agility or passion. But the style is the same, very terse, very dense. The HP web site labels this as "our shared values", but it wouldn't be unfair to call it "HP Way 2.0".

The next step is to make employees (and from there, outside observers) really believe that these values are back. People outside HP may not realize just how hard it was to turn HP around. Reorganizations were frequent. A number of good people, colleagues and friends, lost their jobs. This left scars. Many employees don't feel valued or safe anymore. Many will no longer believe that "trust or respect for individuals" applies to them. This can be fixed, and for the long term of HP, this has to be fixed. Not because HP should be a charity, but precisely because the HP Way is what made HP such a successful business.

Il est dans le caractère français d'exagérer,
de se plaindre et de tout défigurer
dès qu'on est mécontent.
Napoléon Bonaparte

The woes of the "One laptop per child" project

2008-05-14T20:22:00.006+01:00

Ivan Krstić writes in Sic Transit Gloria Laptopi about the woes of the One Laptop Per Child (OLPC) project.

The whole essay is a bit long, but definitely worth reading. It goes through the history of the OLPC project (including its roots in early experiments), through musings about the best choice of operating system, to suggestions on how to move the project forward successfully, after what appears to have been a severe crisis.

No matter what, Krstić is right that the whole experience will not have been in vain. But it's too bad that a project like this can die for purely political reasons. On one hand, OLPC could not have seen the light of day without the efficient support of someone like Nicholas Negroponte. On the other hand, if we are to trust Krstić earlier essay, Things to remember when reading news about OLPC, he's now almost a liability to the project:

To those on the outside and looking in: remember that, though he takes the liberty of speaking in its name, Nicholas is not OLPC. OLPC is Walter Bender, Scott Ananian, Chris Ball, Mitch Bradley, Mark Foster, Marco Pesenti Gritti, Mary Lou Jepsen, Andres Salomon, Richard Smith, Michael Stone, Tomeu Vizoso, John Watlington, Dan Williams, Dave Woodhouse, and the community, and the rest of the people who worked days, nights, and weekends without end, fighting like warrior poets to make this project work. Nicholas wasn’t the one who built the hardware, or wrote the software, or deployed the machines. Nicholas talks, but these people’s work walks.

Makes you wonder who really "invented" the OLPC... Two earlier posts may be relevant to this topic:

Another earlier discussion of Group Dynamics, which seems to apply quite well here
Maybe we should hear the other side of the story.

The Perimeter Institude hit by the Slashdot effect

2008-05-13T19:44:00.008+01:00

The Slashdot effect is what happens to a web site when it is referenced on Slashdot. Many web sites are unable to handle the load correctly, as the very large readership of Slashdot tries to connect to the linked site.

I thought that the Slashdot effect was more or less a thing of the past, but this just happened to the Perimeter Institute (PI), following the publication of a post entitled "Lectures on the Frontier of Physics". Currently, the PI web site only displays the following:

Thank you for visiting Perimeter Institute.

The website is under heavy load at present due to the popularity of announcements regarding the appointment of Neil Turok as Executive Director of PI, as well as awareness of PI's online public lecture series - as reported on Slashdot and other sources.

Please visit again when traffic is back to normal.

Thank you.

Hopefully, traffic will soon return to normal, because the agenda looked quite promising. The Slashdot story cites:

Presentations include Neil Turok's 'What Banged?,' John Ellis with 'The Large Hadron Collider,' Nima Arkani-Hamed with 'Fundamental Physics in 2010,' Paul Steinhardt with 'Impossible Crystals,' Edward Witten with 'The Quest for Supersymmetry,' Seth Lloyd with 'Programming the Universe,' Anton Zeilinger with 'From Einstein to Quantum Information,' Raymond Laflamme with 'Harnessing the Quantum World,' and many other talks. The presentations feature a split-screen presentation with the guest speaker in one frame and their full-frame graphics in the other.

Can we improve HTTP?

This kind of experience is a reminder that HTTP is a really simple protocol, one where no attempt whatsoever is made to offer some kind of proximity caching. I wonder if it's possible to retrofit P2P proximity caching technologies into the more bare-bones HTTP? Does anybody know of any such research? The idea, obviously, would be to have nodes that are closer to the client act as content proxies, thereby offloading the original server.

The problem, of course, is that much of the content on the Internet is dynamic and hard to cache. That's the reason it's an interesting question :-)

Too much travel...

2008-05-12T02:32:00.007+01:00

There is such a thing as too much travel, but I will certainly not complain about the sightseeing.

HP TechCon 2008

2008-05-07T13:55:00.037+01:00

I just came back from HP TechCon 2008, Hewlett-Packard's internal conference for technologists, which was held in Boston this year.

HP TechCon is something serious. It works like a real conference: you have to submit papers to be invited, there is a formal selection process, and technologists end up presenting their work to a wide audience.

The two differences with other conferences are what makes it very interesting:

it's internal to HP, so we get to see and talk about stuff that is confidential or really far out.
It's all about HP, which is a big company, so there are a lot of different topics. Most conferences are really monolithic by comparison.

Meeting great people

Obviously, there is a lot I can't talk about. But I can at least say that it was the occasion for me to meet many amazing people. Below are some public links telling you more about some people I met there:

Phil McKinney is the VP and CTO of HP's Personal Systems Group. He has a blog, a Facebook profile, a Wikipedia entry, publishes podcasts, has more than 500 LinkedIn connections, and even a Google Phil entry on his web pages. Recommended reading.

Exploring the web around Phil's blog and comments, I discovered a number of things, like mscape and the HP Game On series of ads. And for serious gamers, the Blackbird 002 game system (adding "002" to the name is, by itself, a really cool piece of serious geekery.)

Duncan Stewart is a physics researcher at HP Labs. You need to add "memristor" to Google Duncan, because there are other people with the same name. Occasional readers of my blog may know that physics is a topic I'm quite interested in. He's one of the researchers behind the memristor, which I will talk about below.

There are a number of pages about Duncan Stewart, including this one on the HP Labs web site. I really enjoyed talking to him.

The memristor

In case you did not hear about it, the memristor is the fourth passive device, after the resistor, the capacitor and the inductor. But it would be surprising if you had not heard about it. The number of papers and articles on such a recent topic is really amazing. Here is a link at HP Labs, the article in Nature that I think started it all, or the Wikipedia entry.

To explain what the memristor is, a little hydraulic analogy is in order. As you know, a good way to think about electricity is to see "voltage" as the height of water (the pressure, really), and "current' as the flow of water.

A resistor is like a grid or something that blocks the flow: to get more water to flow through (more current), you will need a higher level of water (more voltage). This is expressed as Ohm's law , .
A capacitor is like a tank, where inbound current elevates the level of water, and outbound current depletes the contents of the tank and therefore the level of water. Therefore, it relates a change in voltage to a current, , which you can also write as
An inductor is like a heavy paddle wheel in a current, which prevents it from changing quickly. In that case, changes in current are related to the height of water: if you try to reverse the current for example, water will accumulate until the paddle wheel changes direction. This is traditionally expressed as , but you can also write it as
Finally, a memristor is like a gravel-filled pipe near a constriction. If flow brings the gravel towards the constriction, the gravel blocks the pipe and the resistance to flow will increase. On the other hand, if the flow brings the gravel away from the constriction, water can flow freely. This relates a change in current to a change in voltage, which you can write as .

The equations show why this is the "fourth passive element": if you consider current and voltage, and accumulated current and voltage, there are four ways to combine them.

Fine, so... what do you do with it?

So... why does it matter, what's the big deal? Well, very simply put, the memristor looks, among other things, like a very cheap and dense way to build memory devices. There are still a good number of questions to be addressed before this unseats Flash memory and other persistent storage, and to be honest, there is even a small chance that it never will (the hard disk is still with us, no matter how many times its death was predicted.)

There are many other applications as well. One of them, "artificial intelligence", looks a bit overhyped to me at the moment. What seems true, however, is that memristors might enable cheap neural-like circuits to be mass-produced efficiently.

Conclusion

This was overall a great TechCon. Still, I'm glad to be back, it was exhausting.

Added an Amstrad CPC-464 to my collection

2008-05-01T21:28:00.008+01:00

My brother Matthieu just offered me an Amstrad CPC-464 for my computer collection.

Writing this, I realized that this may be the first time that I actually mention this collection on the blog. I have about 25 computers, and about the same number of calculators, the vast majority more or less in working order. I have to write more or less, unfortunately, because it's not infrequent for one of these old machines to break.

It works, but it's so slow

Anyway, the Amstrad is in perfect working order, as you can see on the picture above. I had forgotten just how fast these machines booted, and just how slow the BASIC was after that. Old memories kicked in, as I tried to practice the horrid program editor of these machines "EDIT 30" would bring a line editor for line 30. You could also move the cursor around, and it looked like you were editing, but you really weren't. You had to use the magic "COPY" key to copy one character at a time for editing. Totally idiotic if you look at it from today's perspective.

My kids were unimpressed. We tried to load a few games, but it was really boring, even to me. Load the cassette tape, wait 5 minutes for something to load, that asks you whether you want to play part 1 or part 2 or part 3 of the game, then load again for 5 minutes, and so on. Today's kids just have no idea how lucky they are to have instant access to a gigantic library of games from the Internet. But even if I grew up with this kind of boxes, I must admit that my memory of how bad this was had somewhat faded.

At some point, I tried to write a little program to draw a Lissajous curve. This was the kind of thing that we thought were cool at the time. Except that it took minutes to draw the curve! I simply had forgotten that. My oldest son, Tanguy, looked at this, and while the curve interested him, I can't say that the BASIC moved him a bit. We ended up discussing Lissajous curves on the remarkable Apple Grapher application. So much for old times...

On the lookout for other machines...

At some point, I have to spend a little time on e-Bay trying to look for the missing pieces in the collection, but for the moment, I'm simply looking at random, and picking up things when I think they are interesting. Right now, one of the few pieces left that I'd really like to have is a TI-99/4A. That's just because I spent so many hours programming it at a friend's house as a kid...

Antisocial punishment

2008-04-29T07:12:00.008+01:00

A recent article in Science documents what the authors call "antisocial punishment". Specifically, they demonstrated that society punishes those who contribute the least or try to abuse the system, but also that there is a tendency to punish those who contribute the most.

This will not come as a surprise to anybody who remembers Honore de Balzac stating that Behind every great fortune, there is a crime. For the curious, the actual quotation in French is from Le Père Goriot:

Le secret des grandes fortunes sans cause apparente est un crime oublié, parce qu' il a été proprement fait.

which I could translate approximately into:

The secret of great fortunes without apparent cause is a forgotten crime, because it was nicely done

This reminds me of the debate about whether Poincaré or Einstein invented relativity. According to Jules Leveugle, this was nothing less than a "german conspiracy" to promote "german science" at the expense of Poincaré. I don't have enough data at that point. But even if Leveugle seems to have some information to back his claims, he asserts a number of things about the motivations of people a century ago that are quite hard to prove one way or another. Anyway, if the conspiracy theorists are right on this one, Poincaré would have been a victim of exactly the kind of antisocial punishment the Science article demonstrates...

As an aside, while everybody nowadays agrees that Poincaré published the core of special relativity and even invented the term "relativity" shortly before Einstein, Leveugle's assertions that Poincaré also invented general relativity before Einstein is much less widely believed. He mentions some document having been more or less willfully damaged by later scientists, this becomes a little bit harder to prove. In any case, I could not find references on the web.

André Vignon, beloved grandfather, passed away

2008-04-26T18:59:00.002+01:00

My grandfather, André Vignon, passed away today, at the age of 96. This remarkable man had 13 children, more than 50 grandchildren, and currently just shy of 80 grand-grandchildren. He was awarded the Croix de Guerre 39-45, but he was so modest that almost nobody in the family can tell you why. For years, he was the CEO of Favrichon, a company specialized in healthy foods, particularly cereals. We grew up testing the prototypes of various cereal products, and I suppose that our enthusiasm for this or that new flavor decided its fate.

It's very hard to describe what a loss this is to the whole family. We all grew up among dozens of cousins. It always felt perfectly normal to me to go on vacation during summer time with half a dozen cousins my age to play with, until I realized much later in life just how exceptional this experience was. Love was completely central to this family, and years later, we still see each others regularly and with great pleasure.

Big families: recommended...

Pictures for the family

MacOSX Leopard: disappointed...

2008-04-20T18:25:00.003+01:00

I finally purchased MacOSX Leopard. For a long time, I had purchased every single update of MacOSX as soon as I could, because they were generally worth it. This was the first time I had some second thoughts. There were a number of mixed reviews over the net, like the excellent Ars Technica review.

The main problems that these reviews were reporting were drops in looks and usability. Looks: folder icons that look bad, inconsistent shadows, translucent menu bar, overly bright window widgets, and so on. Usability: initially, folders placed in the dock would show as "stacks", in other words a big pile of stuff, and fan out in a way that made it quite hard to pick up anything in the folder. Overall, the new OS was also reported to be much more resource hungry than the older ones, not a big surprise here...

To me, it was annoying to have a trade-off between features I'd get, like Time Machine or Parental Controls, and features I'd lose (something Apple does not advertise much), like Classic (the environment to run MacOS9 applications). In previous releases, there was a net gain in functionality, but the loss of Classic was a pretty big deal to me. In particular, I wanted to have Parental Controls on the kids Mac, an old Dual G4 which contains tons of MacOS9 games.

But a few weeks ago, having learned that 10.5.2 was finally giving users options about the dock icons and menu bar, I thought that it was safe to jump, at least for my Intel-based Powerbook, which can't run Classic anyways, and which was acting weird lately. In particular, it jut can't run straight any time I have run any virtual machine with Parallels Desktop. That was most likely a problem with Parallels, not with Apple, but I thought that the latest Apple OS might help. Another problem I kept hitting was that the machine would lock-up when I tried to unlock the screen saver. Hoping to get rid of these annoyances, I went ahead, purchased it and installed it.

The first thing I noticed is that my machine had become quite slower than before. Starting applications, in particular, seems to take quite a bit longer. This is particularly noticeable at login time. I now need something like one minute to log-in, which is too long for my taste. Unfortunately, the problems I had before were both still there: crashes or hangs after running Parallels, and even without running it, I still have the "black screen of death" way too often when I try to get out of the screen saver.

So I decided to try it on another machine, see if the experience would be any better: on the kids computer. Here, the experience is nothing but miserable. First, the installer forced my screen resolution to 640x480 on a flat-panel LCD that normally runs at 1280x1024. No big deal, it's just the installer, right? Wrong! The screens do not fit at that resolution, so to get anything you have to hit "tab" at random and hope to hit the "Continue" button with the space bar (you can't reach it with the mouse, since it's out of the screen...)

Once you went through the whole installation routine, you are greeted with a screen that looks like this:

That's right: the menu bar is a bright pink, and the background image is pink colored more or less at random. Performance is abysmal, my kids had to downgrade the various tunable parameters of World of Warcraft by a couple of notches. And I got at least one lock-up.

The upgrade process is not smooth either. To install 6 updates on the G4 took me 3 or 4 attempts. It seems to install, but after the install-and-reboot, it still wants to install the same thing. On the Intel PowerBook, I have been unable to update so far, it tells me that it "cannot write to /". Repairing disk permissions failed. One of my disks, that reads fine with 10.4, appears as "unrepairable" with 10.5. Granted, it was probably damaged in some way, but I'd like to avoid losing 350GB of data, please?

So, overall, MacOSX 10.5 is a disappointment. Apple is probably focusing a lot of energy on the iPhone right now... Or something else happened. Regardless, the quality of that product is not what I am ready to pay for.

Update: I had the same kind of color problem on my MacBook with an external HP monitor. The colors just looked way off, and trying to adjust the ColorSync profile only made things worse. This monitor used to work really well with this machine. What's going on here?

Oh, the joy of not being remote...

2008-04-12T08:05:00.004+01:00

Following my CGO talk, I went to visit my colleagues in Nashua, New-Hampshire.

Nashua ZKO site shutting down

The so-called ZKO building is a historical landmark in the history of DEC. For the old-timers in computer science, @zko.dec.com was a pretty healthy thing to have in your e-mail address... Its walls are layered with pictures of early VMS luminaries. The Integrity VM project was fortunate to inherit some of this historical expertise that DEC (don't say "Compaq" to these folks...) had built in operating systems. We have in the team people who not only know what TOPS-10 and TOPS-20 were, but actually wrote or maintained parts of it.

But after so many years, Hewlett-Packard will be closing ZKO soon, and transferring people to another HP building in Marlborough. As a result, many of my colleagues will have to move to new offices, and others decided to work from home. So I thought this was a good time to visit and meet the team in one place, while this was still possible.

Working remotely is hard

I have strong reservations about working from home or remotely, however. This is much harder than corporations seem to think. I've been doing it occasionally for almost 5 years now. Of course, part of the problem is that the HP VPN is sized for people doing PowerPoint and Outlook e-mail. It's easily overloaded when too many people at once try to do something interactive, like a Remote Desktop or VNC session. So whenever I try to work from home, two days out of three, I end up returning to the office to be able to work in decent conditions. I hope that our team will not be impacted the same way.

There is also a whole dimension of face-to-face interaction that no amount of phone conversation or chat or e-mail can compensate for. Face to face, you become friends. Over e-mail, you are at best acquaintances. This is something that the proponents of teleworking fail to realize: just how many friends did you make over e-mail? how many over a beer, or a coffee, or just chatting face to face?

Another factor in my case is time zone differences. When the Nashua team starts working, it's 3PM in France. When the Cupertino team starts working, it's 6PM. I have regular meetings after 6PM two or three days a week, and a weekly meeting on Thursdays that finishes at 9PM and is killing me. No amount of technology really helps with that.

In any case, I was so happy to spend some quality time with the team. I met for the first time several people I had been working with for years! This was a treat.

Book review: The trouble with physics

2008-04-10T07:14:00.006+01:00

Until now, I had not read Lee Smolin's famous The Trouble with Physics. Having recently written a rather negative review about a book written by someone who can't stand Smolin, I thought that I should read both sides.

A pleasure to read

Lee Smolin writes very well, and it is clear why his book had a lot of success. He presents a simple thesis, and backs it with a well-constructed reasoning illustrated with dozens of examples. Like any good book, there are dozens of sub-plots, twists and turns, recurring themes and characters, but more importantly, what makes this book interesting is, very simply, that it has a point. Lee Smolin seems to have a single and clear objective with this book, which is to explain why physics research is no longer working as well as it should. And to be honest, I find the point pretty convincing myself. It's hard to not have a deep sense of sadness when a researcher like Smolin begins a book with "we have failed", but to reuse one of his expressions, this has the ring of truth...

One interesting twist and recurring theme is Smolin's own role in this grand saga, how he views himself in the small community of hard core physicists. Clearly, Smolin is an insider, but a relatively atypical one. I would say that the reason is because he recognizes failure early, and is ready to switch horses and try something else when failure happens. Smolin also makes an interesting distinction between seers, who invent new techniques, and physicists who are more adept at applying existing techniques. It is clear that he sees himself more as a seer, and that he believes that we are at a turn where seers are what physics science needs the most. One of the key subplots is how today's academia favors technicians much more than seers. Smolin advocates for academia to be more open to ideas that seem strange to the mainstream.

Limits of the book

However, this is also the limit of the book in my view. Reading it, one cannot escape a feeling that Smolin essentially argues in favor of himself and his own research. I do not believe that he's being dishonest at all, but rather that it somewhat distorts his judgement and weakens his argument. For example, he argues that scientists should have an open mind and be ready to find ideas where they don't expect it. But he immediately slashed that openness with a note at the end of the book that, naturally, you have to be serious and that you can't extend the privilege to people without a PhD in physics and other crackpots. In that case, how is that different qualitatively from Lubos Motl's argument that physics should be closed to anybody not doing string theory? The only difference in "open mindedness" seems to be a quantitative one, where the bar is set.

History records a number of people who contributed science simply because they loved it or had a talent for it. Education often came after their major insight. We all know that Einstein did not have an academic career when he first published his special relativity papers, but he's hardly the exception among those that Smolin calls seers. Ramanujan, for example, did not have any high-level formal education. This is a bit of an extreme case, naturally, and I would not go as far as saying that you should avoid high-level education. But what this proves (one example being enough for this proof) is that there are people without a PhD who made major contributions, and consequently, that Smolin's proposal to put the limit there is inconsistent with the rest of his argument about welcoming ideas in physics.

Information overload: the big missing topic

Furthermore, I believe that he fails to say anything about another major issue physicists face, the "information glut", and which most physicists would recognize in the Sokal affair or the Bogdanov affair. The problem is that there is simply too much to read, including e-mails, books, blogs, articles, and no human being can be on top of everything nowadays. In an ideal world, Smolin would have time to make an informed opinion about everything there is out there, including ideas from people who don't have a PhD. In an ideal world, Smolin would be able to teach those people who got it wrong why they got it wrong, and maybe to pick up the occasional gem. But this is not an ideal world. Setting the bar at the PhD level is the simplest kind of filter you can put in place to reduce the information flow to manageable levels.

Specialization, this trend in academia that Smolin dislikes because it doesn't favor seers like him, is just another simple-minded attempt to reduce the amount of data. When Lubos Motl calls "crackpot" anybody who isn't interested in string theory, he has, in a sense, the same objective as Smolin, which is to reduce his world view to manageable amounts of data.

What I would try if I had time...

This is the reason I think that it's now time to put some technology in the mix. It's time to move beyond arXiv and e-mail and newsgroups. It's time to create some tool to help scientists find the data they need. For this to work, it has to be based on social networks (where others with similar tastes help you filter things). I've a pretty good idea of what it would look like, but so little time... And we are back to square one: time, bandwidth, information overload, these are the problems. PhDs aren't.

CGO / EPIC-7

2008-04-06T18:44:00.005+01:00

Today, I gave a talk about Integrity Virtual Machines at the EPIC-7 workshop of the CGO-2008 conference. The audience was smaller than I hoped, but quite interesting. This was a good occasion to make new contacts, and to renew contacts with old friends. After 7 years working on Integrity VM, this was actually the first time I talked publicly about it outside of HP (although Todd Kjos gave a talk at Gelato in April 2007).

There were a number of other interesting talks. Rohit Bhatia presented Tukwila, the next generation Itanium. I did not learn much during this talk, but it was interesting to see the Intel take on this processor (I usually only hear the HP side of the story). This somewhat renewed my confidence about this platform. It's a very interesting processor for a programmer, but that alone has never been a guarantee of success. Clearly, the initial expectations for Itanium were set a little bit too high, but it's interesting to see that Intel doesn't give up. And hearing them boast about Tukwila as "the fastest processor in the industry" opens some perspectives.

One thing that I'm still wondering about is how we can explain customers why Itanium will stick at relatively modest number of cores (e.g. four), whereas Larrabee will feature a much larger number (16 or 24 according to Wikipedia). Hmmm...

Parallelism: the new new thing

2008-03-28T21:39:00.003+01:00

Bill McColl writes about parallelism and how this is the next big thing. I agree, and I would add that what makes it newly new is its non-homogeneous nature. We have examples of this with the Cell microprocessor and its coprocessing elements. But a network of heterogeneous machines such as cell phones are another instance.

And guess what?

Only around 1% of the world’s software developers have any experience of parallel programming. The percentage that has significant experience in the design, analysis and implementation of highly parallel systems is perhaps as low as 0.01% (1 in 10,000).

Hyper-V: Linux or not Linux?

2008-03-23T10:03:00.005+01:00

A recent column by Mitchell Ashley argues that Microsoft's upcoming Hyper-V virtualization platform (formerly known as Viridian) "leaves out Linux in the cold", because it only supports SuSE Linux and not the bigger contenders like RedHat and Ubuntu.

I believe that Mitchell Ashley misses two important points in his analysis:

The US market, where RedHat and Ubuntu dominate, is not the market where Microsoft has the most trouble with Windows. In Europe and Asia, their dominance is not as clear. SuSE is a key player in Europe, in particular in Germany, and these are also the locations where governments threaten to standardize on "open platforms". So instead of focusing on markets where it has little to gain, Microsoft may be after markets where Windows is threatened.
The relative cost of software has gone up tremendously, and now is the majority of the purchase cost of any IT infrastructure. Long gone are the days when IBM simply gave the software for free when you purchased its hardware. So Microsoft may be playing catch-up, but as long as they can offer deals you can't refuse regarding the licenses of Microsoft Windows (e.g. it's much cheaper to run 4 Windows VMs under Hyper-V than under VMware), they have the possibility to tie rocks to the other guys' ankles...

On a different topic, one of the comments suggests that SuSE only runs thanks to a binary-only kernel module. That would prove interesting if this is indeed the case. While binary kernel modules have been used for specific proprietary hardware such as 3D graphics cards, I don't think it's ever been the case before that you needed one for the kernel itself.

If it's some kind of paravirtualization or acceleration as I suspect (another comment about someone running other kernels tends to confirm that viewpoint), then it's a bit different. But if you need some proprietary binary simply to run Linux, I believe that this will cause some backlash from the Free Software community.

Developing the security mindset

2008-03-21T15:02:00.001+01:00

An interesting commentary on Wired about the security mindset:

SmartWater is a liquid with a unique identifier linked to a particular owner. "The idea is for me to paint this stuff on my valuables as proof of ownership," I wrote when I first learned about the idea. "I think a better idea would be for me to paint it on your valuables, and then call the police."

Superlanguages

2008-03-15T11:12:00.002+01:00

Christopher Diggins talks about superlanguages to describe more or less what I tried to achieve with XL.

While they are related, language-oriented programming and concept programming are not exactly the same thing. Concept programming uses the techniques of language-oriented programming, but the objective is different (although it may be implicit to most practitioners of LOP). In concept programming, what really matters is how the code looks, and how that relates to the way we think about it and represent the related concepts outside of the programming community.

That implies reuse of LOP techniques, because you cannot represent arbitrary concepts without a good dose of metaprogramming. But the reverse is not true. You can use LOP techniques without caring at all about how this "looks and feels". Lisp is a good example of that: in Lisp, nobody cares if you write (+ 1 2) instead of 1+2. That's why the "pseudo-metrics" in concept programming are a truly distinguishing feature.

There can be only one...

2008-03-13T23:05:00.007+01:00

There has to be a last guy.

Yesterday, the last French soldier in World War I, Lazare Ponticelli, died at age 110.

In a totally different context, I find myself feeling a little bit like the "last guy standing" today. Todd Kjos found some new technical challenge to tackle and decided to move on. He was one of the three fathers of HPVM, and his contributions were invaluable.

Nothing lasts forever. We knew this was coming. Still, this is a sad day. What else to expect from a 13^th... But it was a privilege working with guys like Todd and Jonathan Ross.

Multicore panic...

2008-03-13T22:02:00.005+01:00

According to this article:

Chuck Moore, an AMD senior fellow, made the case for the shift to a new software model based on heterogeneous collections of cores optimized for various tasks. He suggested computers should be more like cellphones, using a variety of specialty cores to run modular software scheduled by a high-level applications programming interface.

I second that entirely. When silicon is cheap, you tend to specialize. Today, we specialize entire machines. This is why it is OK to have in any cell phone today the power of a good top-of-the-line personal computer of the early to mid 1990s. Specialized designs can be much faster and economical than specialized ones. But I'm not sure that we'll ever see thousands and thousands of pre-wired cores. The reason is that we are beginning to understand how to build customized floating-point circuits on a per-application basis (If the name in these papers rings a bell, it's because the guy writing them is my brother...)

But even if you have a plethora of cores, the problem still remains of how to develop for these cores:

The only emerging consensus seems to be that multicore computing is facing a major crisis. In a recent EE Times article titled 'Multicore puts screws to parallel-programming models', AMD's Chuck Moore is reported to have said that 'the industry is in a little bit of a panic about how to program multicore processors, especially heterogeneous ones.

That's where something like XL, concept programming or Intentional Programming may turn out to be indispensable. Why? Because these new techniques allow programmer to describe custom conceps, not just obsolete CPU models of the 1970s as we still find them in C and C++ ("memory accesses are cheap", "every CPU knows about the C operators and only them", and so on).

Brain scanner

2008-03-08T10:10:00.005+01:00

Another progress made on brain scanning technology. I predict that in a few years, the interface to computers will probably no longer use keyboards as much. But programming such interfaces will take some innovative programming techniques.

Carlo Rovelli's "dialogue" on LQG and strings

2008-03-08T09:34:00.005+01:00

You may remember that I recently read L'equation Bogdanov, and did not find it very good.

So it felt refreshing to come across a more interesting discussion of these problems, namely a dialogue written by Carlo Rovelli. Carlo Rovelli is one of the leading researchers in loop quantum gravity, that Lubos Motl so incorrectly describes as trying to reinvent aether with "atoms of space". The dialogue is not new, but it was apparently refreshed in February 2008, despite what the ArXiv web site indicates.

I won't take sides between string theory and loop quantum gravity. I'm not entirely comfortable in either approach, which may be because I don't know enough (that's clearly what Lubos Motl thinks), or maybe because the questions I'm interested in are not resolved by either.

Still, I feel conceptually slightly closer to LQG. For example, through a different line of reasoning, my approach also suggests that there is no "background", i.e. some sort of fixed space-time where you play physical events. The existence of such a background (and of properties I really cannot subscribe to, like the fact that it is continuous) are, to the best of my knowledge, essential for string theory.

And obviously, on the form, I much prefer the calm, short, yet substantiated and verifiable approach to the debate that Carlo Rovelli gives us here to the endless, abusive approach Lubos Motl represents. Of course, that means that Carlo Rovelli is not as loud, and possibly is heard less. But he's the smart guy.

The number you can't compute...

2008-03-05T19:39:00.009+01:00

One of my respected colleagues at Hewlett-Packard is a guy named Patrick Demichel. He's one of the "titans", having held various world records on the number of digits of π and things like that.

But a couple of weeks ago, he talked to me about something really intriguing. During a conference on high-performance computing, he told me about a number he had computed, and that even if he was not sure that the value was right, he considered it highly unlikely that anybody could prove him wrong, because there weren't enough atoms in the universe to do so. How does that sound?

Well, it turns out that he was talking about the Skewes numbers. More specifically, he has computed the first Skewes number to be most probably equal to 1.397162914×10³¹⁶. See another reference in Wolfram's MathWorld. You can find the complete presentation of that research here. The general idea is that we have something that is extremely difficult to compute, so we compute it at reduced precision, skipping through large ranges of numbers. In doing so, we do a probabilistic estimate of the risk that we miss the right values. It's really low, to the point that finding a counter example would take more computations than what can have been done in the entire universe since the Big Bang. However, we clearly have not tested all possible values (which would take even more power).

OK, in reality, this depends on the behavior of the Zeta function, more specifically on the Riemann hypothesis. So there is still an infinitesimal chance that all this huge computing power went to waste, and that 1.397162914×10³¹⁶ is actually not the first Skewes number. Time will tell...

L'equation Bogdanov: The Salvador Dali school of physics

2008-03-05T06:06:00.063+01:00

Two days ago, I stumbled across L'equation Bogdanov at the local bookstore in Grasse, France. That was a surprise, since that particular store is where I go to buy gifts or kids school books. It is generally not too strong on science books. That means that the book is intended for a rather wide distribution. Lubos Motl, writing an introductory book? About the Bogdanovs? That was sure to pique my curiosity!

Well, call me biased, but...

If you occasionally read my blog, you may know that I have reasons to not really like Motl too much, after he wrote a rather silly and inflamatory column about one of my posts. If you have some time, read my post, then Lubos', and try to find any correlation between the two. For example, Lubos' first point presents me as supporting the idea that "The goal of science is to wait for a "new Einstein" or a savior," when my own text precisely criticizes Lee Smolin's famous question Why is there no new Einstein. I'm still wondering how he could attribute me any of the ideas he seems to claim were mine in his ten bullets list...

The first pages of the book left me with the same kind of feeling, the feeling you get when looking at a picture from Salvador Dali, that some people care more about their interpretation of beauty than about mundane things like reality or truth. It sure makes for great art, impossible visions of what could almost be, stuff that is almost, but not quite, entirely unlike tea. But in science, I don't think that it has as much value. I will illustrate this shortly.

Still, I refrained from writing this post immediately after reading the introduction (an ode to Lubos the Great that is in itself "worth its weight in peanuts", as we say in French). Instead, I chose to remain focused and read the book to the end. Although, for the first time in years, I read a book while scribbling all over it with a pencil. Ultimately, reading it through was the right thing to do, as some parts of the book are not entirely worthless.

However, if you are interested in something else than Lubos' passionate yet sterile debate about strings vs. the rest of the universe, I unfortunately have to recommend spending your 19€ elsewhere. And if Lubos' point of view interests you, there isn't much in the book that you won't find on his blog. But since I realize I'm biased, I'll try to support this opinion with facts. And I'll try to keep these facts simple and verifiable, including by the "layman".

The good

There are some relatively good things in this book if you are willing to sift through mud. The mandatory review of physics history in chapter 2 is much less intertwined with the primary topic of the book than, say, in Brian Greene's The Fabric of the Cosmos. But it has the redeeming quality that it shows the relationships between various physicists, illustrating "standing on the shoulders of giants" better than many other "histories of physics".

Chapter 6, "The strange adventure of the Bogdanovs", is probably the most interesting part in the book. It appeared slightly less surrealist than the rest of the work. It essentially argue that the Bogdanovs did not deserve the attacks they received, because even if their work is hard to follow (Lubos himself grants that he had trouble following it), their efforts seem a genuine attempt to contribute to physics rather than a malicious attempt to play some elaborate hoax on physicists. I'll refer the reader to what has been written on the subject. The Bogdanov affair, as it is now called, is a complex case of the sociology of science, and tempers certainly flared more than they should have.

Anyway, this chapter is especially good coming from Lubos, as it illustrates that he can sometimes show some balance and moderation in his writings.

The mediocre

Unfortunately, these few nuggets are hard to find in a book that is, overall, mediocre, mostly because Lubos never seems to have decided who the target audience was. Is this an introductory book intended for laymen, as seems to be indicated by multiple footnotes like on page 61, explaining what a wave is, or a very vague explanation of what complex numbers are on page 156? Actually, Lubos himself calls his book the "100th vulgarization book on supergravity" on page 165...

But then, if it's an introductory book, the order is all wrong for this target audience. Lubos talk about topics such as black holes entropy and horizons on page 55 and 56, and again about information loss in black holes on page 89, and at several other places. And then, on page 125, a footnote finally tells us what a black hole is. Huh? Ah but wait, there's another such explanation on page 75! Confusing enough?

Similarly, Lie groups are introduced by a footnote on page 29 that finds it useful to illustrate this with "the Lorentz group SO(3,1)" and "SU(2)"... How can the layman have any idea what SO(3,1) or SU(2) might be? If at least there was a forward-reference to the page 165, where Lubos attempts to explain the importance of symmetries and Noether's theorem, but nope... One last example: why explain complex numbers on page 156 if the introduction insists on "imaginary time measured by imaginary numbers" and the lack of total order on the complex plane on... page 14!?!

And if the book is for a wide audience, some topics are pretty advanced for someone who would not know what waves or complex number are. For instance, how can such a person feel about the argument given on page 102, that there's a problem in LQG because spectra of surface operators are not gauge invariant? Seriously?

The book also does not do a very good job at explaining anything. I invite the interested readers to contrast Lubos' explanation of the importance of correlated systems and (even if the name is not given) the EPR paradox on page 67 with that given by Brian Greene around pages 107-109 of "The Fabric of the Cosmos", and you will understand what I mean.

The bad

So in general, the book does a very poor job addressing the layman. For someone with a little bit more knowledge, it's irritating to see various comments that are either overly simplistic or just plain wrong. For instance, on page 157, another footnote argues that you need imaginary numbers to build a circuit breaker!

All too often, a moderately educated person like myself might expect to learn something, only to realize that he's been fooled once more. For example, on page 156, there is a footnote on Hopf algebra, that doesn't even begin to explain what they are, but basically tells us that Hopf played music with Einstein! Similarly, on page 103, the footnote about the hamiltonian constraint only vaguely tells us what the Hamiltonian is, but nothing about the constraint which seems to be the heart of the discussion. Fooled again!

And then, there are way too many plain errors. There's one I can't help but laugh about, because of Motl's insistence on calling me a "French programmer" in his blog, apparently with the intent to ridicule my ability to say anything about physics. On page 103, in a discussion about separable and non-separable Hilbert spaces, Lubos states that a bit can hold 256 values. Well, "everyone" knows that a bit contains 2 values, usually represented as 0 and 1, and that it takes 8 bits to make a byte, which does indeed represent 2⁸=256 values. But then, being able to work on computers is apparently a bad thing for Motl, who describes Peter Woit as being merely "in charge of computer systems" at Columbia University, on page 33...

That's not the only such major error. On page 55, he writes that if you ever see a cup of cold coffee warm up while the table cools down, you can immediately call the French Academy of Science. Well, maybe I should, because that's exactly what happens whenever the table is hotter than the coffee cup. I did not add the word "cold" before "coffee", Lubos did, but I strongly suspect he intended to write "hot coffee". On page 72, Lubos states that Einstein should in no way be held responsible for nuclear weapons. This is simply not true: Einstein's letters were highly influential on the decision to develop the first A-bomb, irrespective of Einstein's later regrets (he called this his "greatest mistake"). There is a mention of "the european GPS and Galileo" on page 77 (Galileo is the european GPS). On page 157, Lubos apparently attributes the invention of complex numbers to Hero of Alexandria, which is stretching the truth, to say the least.

This leaves the impression of a book that was barely proofread, an impression in stark contrast with the surprising self-promoting tidbit Lubos gives us on page 122, that for his greatest pleasure, he had reported 120 errors in a book by Joe Polchinski, who he essentially describes as a failed perfectionist. Instead of embarassing Polchinski, Lubos might have wanted to spend a little more time improving the quality of his own writings!

The ugly

But the worst aspect of the book is, without a doubt, that it seems to be a barely disguised excuse to attack other physicists and indulge in more of the sterile debate that opposes Lubos Motl and people like Peter Woit or Lee Smolin. Living in France, I hoped I might have a chance to evade that dispute. Too bad. Most of chapter 4, for example, is dedicated to this. This would be legitimate if it was on topic. But it's not, and Motl instead resorts to sneak tactics, name calling, all techniques that made him a persona non grata in so many places.

Let me put it this way: I like the Bogdanov brothers, not for their self-promotion or for their theories, but because I loved their TV shows. And I feel almost sorry for them that Lubos Motl used their names and pictures. With the Bogdanovs on the front cover, the book is almost guaranteed to sell at least a little in France, since the Bogdanov entertained so many of today's adults back when they were kids.

But for what? A book in which Lubos Motl talks less about the Bogdanovs than about the alleged damage that Lee Smolin or Peter Woit made to physics, and why all these folks are (in Motl's view) idiots. I mean: who cares? If a book was ever going to restore the scientific credit of the Bogdanovs, that's certainly not it, and it's too bad, because on that point, Lubos might be right. But if he wanted to support to the Bogdanovs, he would have been well inspired to focus on their work and very temporarily put aside his personal griefs, at least for the time it took him to write the book. But even that was apparently too much to ask!

Update: For the french readers who would be tempted to believe the arguments of Lubos Motl that loop quantum gravity is reintroducing aether, there's an excellent answer to this very question from Prof. Rovelli (in French) in this conference. The question is asked from the audience at 1h08m15s into the video.

All kinds of invalid proofs

Another thing that I find particularly ugly is the vast collection of invalid techniques of proof Professor Lubos managed to accumulate in a single book. This is particularly annoying for someone who, to do his job well, is supposed to be capable of some seriously solid reasoning. But even the average "layman" is going to be annoyed by all the bogus arguments. Here are a few examples:

On page 28, we have a pretty long proof by eminent authority, citing names like André Lichnerowicz (and, as if this was not enough, Élie Cartan and Sophus Lie) to give credit to the Bogdanovs. I do not dispute that having a guy like Lichnerowicz recommend the Bogdanovs to their advisor, Moshe Flato, is probably a good sign for them. But you have to remember that this is not enough, in particular in a French context where both Lichnerowicz and the Bogdanovs were celebrities. This reverence to names is spread throughout, including in the way he calls the everyday scale of things the "Planck-Einstein scale", by averaging the "Planck scale" of very small things with the "Einstein scale" of very big stuff.
On page 94, a proof by partial enumeration, namely that candidate theories fall into the theories that have not been proven and the theories that have been proven false. This eliminates the most important category, theories where people are still debating. And whether Lubos likes it or not, both string theory and the "atoms of space theories" (to use his bizarre terminology) fall into that category more than in others.
On page 97, we have both a proof by mutual reference (since Lubos cites his own blog to make a point), and a good case of proof by vehement assertion (quoted directly from the blog, rather than translated back from French):
First of all, Lee reveals his intense hostility against all of modern physics, not just string theory. He believes that quantum mechanics must be wrong at some fundamental level and many people should try to prove it. He also believes that the attempts to falsify the theory of relativity are among the most important topics to work on.
This attack is all the more surprising because Motl himself writes on page 83, about general relativity: "the existence of infinities reminds us that we may not have learned our lesson well, and then suggests that either we asked the wrong question, or our theory is wrong. He then follows on to point similar problems in quantum mechanics. So how is that different from this bad attitude Lee Smolin allegedly has towards "modern physics"? Are the two guys in violent agreement here or what?
On page 111, we have a proof by appeal to intuition, where we are supposed to believe that LQG is an invention, as opposed to a discovery, making the LQG researchers roughly comparable to a Thomas Edison, complete with a footnote about who Thomas Edison is, in case anybody on Earth doesn't know! So... LQG is much like Thomas Edison's inventions? Says who? Why? How?

These are only examples. Finding more is left as an exercise for the reader...

True lies

If this was not enough to completely bury that book, there is more, unfortunately. There are various statements in the book that cannot be mere errors, but can only be considered willful lies. The best illustration is a quote on page 151. In the context, the quote, from a referees, is clearly intended to give some credence to the Bogdanov's idea that a topological field theory is just the thing to describe the early universe. Here is Lubos Motl's version:

I can accept that in the limit of infinite temperature, contact can be made with a topological phase of some field theory

Doesn't this sound as if the guy agreed with the Bogdanovs? Well, fortunately, I have a relatively good memory, and even if the quote is in French in the book, I remembered reading that sentence, so it was just a matter of tracking it down. You can find the complete referee report, helpfully not provided by Lubos Motl, here. And here is the non-truncated version, which you probably would agree sounds much less supportive of the Bogdanovs:

(5) I can accept that in the limit of infinite temperature, contact can be made with a topological phase of some field theory (the type of field theory needs to be elaborated on however). The crucial question, however, is how does the initial topological phase break down to a universe we see today. It would be of great interest if the authors' could at least worry about this issue.

And the little things

Is there more? Well, I think that the rest goes without saying, knowing Motl. There are insults, for example on page 134, where Motl says that he spoke with Lee Smolin numerous times, and that they always had "interesting and peaceful discussions" (sic!), but that "each time we were getting close to the answer to a crucial question, his spirit evaporated in the clouds". Uh oh! I picked up this one because I thought it was funny, but there are others.

There's also a rather usual dose of self-promotion (and Lubos is in good company with the Bogdanovs here, if I may give a personal opinion). On page 105, Lubos calls himself a messiah. On page 134, he compares himself to Einstein, specifically referring to himself as another guy from the patent office. On page 143, he mentions the IQ of the "Zweistein", which they allege is about 210, in a sentence where he presents himself as the new Max Planck. Literally, it reads: By the way, if you bought this book in the hope that a new Max Planck - me in that case - would present the two new Einsteins (the "Zweistein" if you will), well, too bad". So much modesty in so little space!

Getting physics out of the hole

Let me finish this review on a slightly less negative note. On page 188, there is a nice long note about the right processes to follow to identify truly innovative ideas in physics. It's actually not from Motl, he simply quotes Dr Osher Doctorow, but there are interesting things in there. One I found particularly interesting, is that true creativity is identified by looking at folks who do not just follow, but go one step ahead of the pack. OK, I realize it sounds obvious, but the point being made is really that it's hard for the pack to judge those who are already ahead. This is one reason why I advocate a more "open-source" approach to peer reviews.

This question interests me. Readers of this blog, if there is any, may know that I have my own pet theory, and that even if I believe it's dead simple compared to the kind of ideas the Bogdanovs are working on, I still seem to have an extraordinarily hard time getting the point across to any physicist. For example, if Lubos seems to have no problem with the idea that near the singularity, the description of space-time is entirely topological (i.e. there is no "distance"), or even that space-time reduces to "pure information", whatever that means (page 25), or that there is a lower physical limit to distance (the Planck scale), he still insists that any theory where space-time is not continuous is absurd, for instance on page 102. In my experience, that kind of "historical" position is frequent among physicists.

I hold the exactly opposite view. If there is a minimum length in physics, and if all our physical measurements give results that are not even countable, but actually finite, don't we need to build that into physics? Isn't there a clear contradiction between "continuous" and "smallest physical distance"? In other words, if it's truly a continuum, shouldn't we be able to find a physical meaning for a distance of 10^-250m? That question is essentially the foundation of my "theory".

A non-local theory of creativity

But since I'm an outsider, I keep asking myself the question: do my ideas make sense, or am I just delusional. In other words, do I have problems getting the point across because I'm ahead of the pack, or because I'm lost in the woods? As Lubos points out, it's easy to tell in retrospect, but when you are in the middle, it's much harder. Related question: assuming my ideas do make sense, do I stand a chance of getting the point across someday? Lubos argues on pages 197 and 198 that being an outsider is not necessarily a bad thing. He also reminds us, on pages 177-179, that big guys like Einstein or Heisenberg, had to fight initially. Of course, it would be better if this call for an open-minded approach to physics was not so clearly contradicted by Lubos' attitude...

Anyway, Lubos and I share at least two things: the same birthdate, and a desire to contribute something to physics. Of course, it's obvious that just thinking you have something to say is not enough. To paraphrase the Calvin cartoon characters, "You know Einstein had bad grades in math? Well, mine are even worse!" So it's entirely possible, and not even improbable, that my ideas about physics are simply bogus. However, for the moment, I'm still waiting for any solid rebuttal of what I already wrote. From what I can tell, it looks more like I'm in the "nobody cares" category rather than in the "patently idiotic" category. Ah, delusion... That's what keeps me alive.

Actually, another thing that gives me hope is that, in my own field, I think it is fair to say that I have shown some creativity, both when I was young and more recently. Maybe, just maybe, having shown some creativity in one field means you might be able to show the same creativity at another place. And if I have beaten someone like John Carmack in the race to the first 3D videogame, maybe I can beat someone like Lubos Motl in the race to the next big idea in physics...

The KISS principle (Keep It Simple, Stupid!)

But the key thing in my ideas which, I think, is in sharp contrast with Lubos' approach to physics, the key thing that differentiates the Salvador Dali school of physics in which Lubos excels from the Thomas Edison school of physics that I'd much rather belong to, it's the clarity of the questions and answers.

As Einstein once said: Make everything as simple as possible, but not simpler. A good test is: do your questions and answers make sense to the layman? Einstein or Feynman were really good at that game. It's a point I already made earlier on this blog. In the present case, I think it strongly speaks in my favor:

Lubos asks questions like "can the signature of the metric fluctuate around the Planck scale", or "are homology cycles in the moduli space of gravitational instantons the right way to represent observables at the beginning of the universe". He answers things like: "who can say this won't be the right answer in 2030" (page 144), and he calls that "research". I'm really tempted to call that a proof by obfuscation.
At least from my own biased point of view, my own questions seem much simpler. Things like: "why would an invert square law like gravitation or electromagnetism remain an inverse square law when we change the definition of distance" (specifically, from measuring it with solid rods to measuring it with light waves). And my answer is: "because we calibrate the two definitions to match", which raises another question: "does that calibration hold for all values, at all scales". No part of these questions or answers is something that the laymen can't understand. I'm not trying to obscure things behind layers and layers of jargon.

Now, I believe that my questions and answers may hold a key to putting physics back in shape in the coming century. And I believe that Lubos' questions and answers only confuse things a little further. Again, that's the difference between the Salvator Dali and the Thomas Edison schools of physics. Of course, I may be wrong, but I'm afraid you'll have to prove it.

A nice graph...

2008-03-03T10:11:00.003+01:00

It's not often that I post about my work, in part because this is a personal blog, in part because I don't want to leak any information that would benefit our competitors, or be incorrectly interpreted by our customers.

But the graph on the right is an exception, because I was really happy to get that result. I am not really at liberty to tell what the graph represents, except that it's a performance benchmark testing the behavior of HP Integrity VM and comparing it against a native machine and two variants of a competiting product generally praised (including inside HP) for its superior performance. Well, Integrity VM is the set of reddish curves, which means we behave pretty well, and that makes me quite happy. Hey, sometimes killing hype with hard data is all it takes to make my day...

Concept Programming Article

2008-02-18T18:42:00.005+01:00

There is an article about concept programming at El Register. Not only that, but Phil Manchester did a pretty good job of summarizing our conversation. I thought I was speaking too quickly, and I sort of expected him to call back and ask a few questions. Since he had not done so, I thought he had simply given up on the idea for the moment.

Some comments are interesting. I'll start with the positive ones, it's good for the morale. James Anderson wrote:

I read the article expecting the usual Computer Science giberish and the usual calling old ideas with new names.

But actually I was rather impressed. First of there is a real appreciation of what was good in C, Algol, Pascal and a serious attempt to hold on to the good stuff and improve it. Also a serious look at what went wrong with the C++ Java etc.

The resulting XL language looks nice and whats more it looks usable.

The bit people seem to have trouble getting thier head around is the generics and the IMHO very wonderful "written as" syntax. But generics are a mine field anyway and the XL approach is definately better than the C++ or Java (C++ generic are unreadable and undebugable, my forst impresion of java genrerics is that you end up with more code than if you wrote a method for each signature!). And the "written as" is something completely new (at least to me!).

Indeed, I believe that I invented the written keyword in XL (of course, I might be wrong). And of course, I hope that it's useful. For those who don't know, it's a generalized form of operator overloading, which lets you define for example stuff like A+B*C and "reduce it" to a function call. Hence the official name, "expression reduction".

Naturally, not all comments were just as nice. But for the bad publicity, you'll have to follow the link... ;-)

Also of note: a link to a language called iolanguage, which one of the comments claimed is similar to XL. But as far as I can tell, this is certainly not a concept-oriented language.

Peer reviews...

2008-02-18T18:17:00.003+01:00

Bee has posted a number of comments on peer reviews lately, notably this one, which reports about a survey made among scientists. The summary is that the majority feel that peer reviews are necessary, but that the majority also would like to see them improve (notably regarding timing).

Another nice link I found on BackReaction was this presentation. Food for thought.

An intelligent review of the TIM

2008-02-17T13:47:00.030+01:00

An anonymous reader wrote an interesting comment about my own pet theory (which I called theory of incomplete measurements, or TIM for short). This anonymous comment exemplifies both what I like and what I dislike about physics discussions these days:

On one hand, it gives honest feedback, both positive and negative, and this hints at the kind of exchange of ideas that I enjoy when discussing with knowledgeable people. For instance, the anonymous reader cites a number of relevant sources, and makes a number of intelligent objections to my work.
On the other hand, I cannot imagine any good reason why this reader would choose to remain anonymous, only bad ones. Too much work? The risk of backlash if anybody knew this person dedicated any time to fringe physics? A fear of true, bi-directional discussion? Whatever his reasons were, I suspect that I would not find it admirable.

I first wrote a quick reply, but I think that this comment demands a more prominent answer.

Below are excerpts from the anonymous reader's comment, which I will hereby refer to as "Dr. Anonymous", with my thoughts (not necessarily answers) inserted.

A theory, or just a descriptive framework?

1. The title would be better as "A general framework for describing physical measurements", as this is all that is provided. Theories are rather more specific than what is given, and make predictions.

The obvious implication here is that my choice of title is a bit too grandiose and that the article delivers too little to merit such a title. But is it really true that the article only provides a framework for describing physical measurements? Is it true that it makes no predictions?

Granted, I never intended the TIM to be some theory of everything. What it is is a theory of measurements. What you need for this is to make predictions about measurements, not about gravitation or bananas. Otherwise, it would be called a theory of gravitation or a banana theory... However, with respect to measurements, the TIM does a number of predictions, starting I believe with equation (9) in the article. Most of these predictions may be qualitative or physical instead of mathematical or numerical. But that does not make them any less predictive, and it does not make them any less falsifiable, which is the most important thing for progress in science.

In particular, I can think of at least three predictions that are apparently suprising, if not shocking, to most physicists who commented about my paper. Specifically, according to the TIM:

All our measurements of space-time can be reduced to properties of electromagnetic interactions (roughly, counting wave-fronts of electromagnetic waves).
It does not make sense to normalize the wave-function on a continuum ranging from "minus infinity" to "plus infinity", the correct way being to normalize it on a discrete sets of points that are in a time-like connected region. In particular, there is no such thing as an "instantaneous collapse" of the wave function.
The laws of general relativity and quantum mechanics are not absolute, but depend on specific choices of physical measurement. In particular, two measurements with distinct resolutions may give distinct metrics. This is a generalization of the "scale relativity principle" dear to Laurent Nottale.

theories can either can be fitted or not be fitted into the particular framework given.

Isn't it useful enough to have a relatively simple framework where both general relativity and quantum mechanics can be fitted?

Other frameworks are often less general, including, for example, 'quantum' logic (measurements represented by propositions), W*-algebras (measurements represented by algebras with a 'complete' set of idempotent elements), and convex set theory (states represented by convex linear measures) - see, eg, Hans Primas, "Chemistry, quantum mechanics and reductionism" (Springer), and Stanley Gudder's "Stochastic quantum mechanics", for overviews. However, more general approaches, at similar levels to your own, are the operational frameworks of Guenther Ludwig and of Charlie Randall and David Foulis - see, eg, the references at http://plato.stanford.edu/entries/qt-quantlog/ .

All these are interesting references. I've only scanned through those available on the web. But the main articles of Randall and Foulis for example require a subscription that I don't have.

Personal gratification, or improvement of the collective understanding?

2. The paper is sincerely and honestly written, and often quite well written, and clearly the ideas have provided you with a personally satisfactory way of thinking about physical phenomena.

It is true that I was looking for an understanding of physics I would be satisfied with. But having made some progress, I hope to offer a way of thinking that is also satisfactory to others. It's a worthy goal too. Last time I checked, there were over 2 million Google hits on "quantum measurement problem". That can only mean that I'm not alone being dissatisfied with the "mainstream" way of thinking about physical phenomena.

However, there is often little more than this to be achieved by embedding known theories into a particular framework unless they (i) suggest better theories, and/or (ii) allow better 'understanding' of known theories.

The TIM does suggest better theories. It suggests a theory of relativity that would explicitly incorporate an additional parameter specifying how the measurement is done. This is pretty much what Nottale's scale relativity or doubly-special relativity boil down to when looked at from a TIM point of view. The TIM also suggests an extended quantum mechanics where space-time coordinates do not commute and are discretized. Aren't these suggestions enough for starters?

More imporantly in my view, the TIM also allows better understanding of known theories. Or, more precisely, it provided me with a so far more satisfactory illusion of better understanding, but as I wrote in my reply, it may just be that ignorance is bliss... Anyway, if I understood things correctly, the TIM shows for example that the "collapse of the wave function" is a necessary consequence of the fact that we demand repeatability from our measurements. One of the meanings of "theory of incomplete measurements" is that it suggests a theory for when measurements do not collapse instantaneously, or perfectly. That is yet another "better theory" that the TIM suggests.

Point (i) is the important one, as point (ii) is quite subjective - for example, one person's interpretation of QM is very often another person's anathema!

I would argue that "one person's interpretation of QM is very often another person's anathema" is a bad thing. Isn't it embarassing for physics that QM is not defined sharply enough to leave room for multiple conflicting interpretations? After all, there is very little room for interpretation in Newton's second law, or in Ohm's law. So why do we need an interpretation of quantum mechanics at all?

Here, I apparently disagree somewhat strongly with Dr. Anonymous. Point (ii) is probably the most important one for quantum mechanics, because QM is not clearly understood yet, and that ought to be fixed. The TIM might be a step in the right direction, or maybe not. But at least, it's an attempt to address this problem.

Dr. Anonymous is in very good company here. The problem of unambiguously interpreting quantum mechanics has resisted physicists for 80 years. So now it's more commonly qualified of "philosophical" or "uninteresting". In other words it became what Douglas Adams called Somebody Else's Problem... Well, maybe it does not interest mainstream physicists. Maybe, actually, you need to be able to ignore the problem to become a mainstream physicist. But it definitely interests me. And it is definitely relevant to modern physics.

Since no better theories are in fact suggested, the content of the paper has only subjective significance - this is not necessarily a bad thing, but it does make it more of interest to philosophers rather than to mainstream physicists such as myself. Similar comments apply to the operational frameworks mentioned above.

I explained why I disagree with the assertion that no better theories are suggestd. But it is interesting that the final objective of all this discussion is to kick the TIM outside of "mainstream physics" into the lower realm of "philosophical interest only" with severely restricted "subjective significance only". This is a pretty good way to invalidate the whole thing.

But the TIM has a physical significance that I think is not so easily dismissed. It shows that practically any equation in quantum mechanics or general relativity is only an approximation. And it tells physicists how to fix that. That's a pretty broad challenge, and you cannot address this challenge by saying that the problem is only subjective. It is not a subjective issue that our laws of physics were not rewritten when we changed the definition of the metre from a reference solid to some distance travelled by light during such and such time. Why not? What makes a 1/r^2 law relative to a solid rod remain a 1/r^2 law relative to a ray of light? Seriously, is this a subjective question?

Is it OK to disagree with Einstein on general relativity?

3. I have little to remark on the relativity sections, as my only disagreements are at the 'philosophical' level - eg, I would have given Einstein a little more credit for physical justification of GR (principle of equivalence, and the recognition of curvature effects in SR from a mass rotating about a fixed point);

Actually, I do give credit to Einstein for these "physical justification", only to explain why I believe they are not valid. So, despite a lot of admiration for Einstein, I simply point out a couple of logical flaws in the reasoning. It's not flaws in the mathematical results, fortunately, but conclusions that do not follow from the initial statements.

The recognition of curvature effects in special relativity from a mass rotating about a fixed point is the easiest to explain. If a mass rotates about a fixed point, there will be a contraction along the path it follows, but not along the radius joining the mass to the central point. In other words, the ratio of circumference to radius is not 2pi, it's a little less. See this page for a more modern discussion than my reference [12].

My objection to this is that you define a 3D surface that is curved, namely the 4D helicoid corresponding to the area that the radius covers over time. But we all know that you can find a 2D surface, like an helicoid or a sphere, in a 3D space, without necessarily inducing a "curvature effect" on the 3D space. In other words, the rotating mass in special relativity is not sufficient to imply a 4D non-euclidean geometry.

Again, there is nothing "philosophical" about this. It's a discussion of whether there is a flaw of logic in the original reasoning, as I think there is, or not.

writing down the local Minkowski metric or geodesic equation does not give them any physical significance per se;

That is true. In the context where they are written in the TIM, the only point is to connect the traditional notation to the TIM notation, and to show why they are compatible.

the two-slit experiment is a poor justification for curved spacetime

It is part of a series of observations to show that light rays are a poor foundation to build an euclidean geometry on. I think that it is a better justification to say: "we don't know how to build an euclidean geometry because the closest we have is not euclidean in this and that case" than to incorrectly infer that a 3D curved surface implies a 4D curved space-time. But that's just the biased opinion of someone who saw a logical flaw in the generally accepted justification, and spent some serious time finding a justification he thought might be less flawed to salvage the beautify theory of general relativity.

and the existence of the Planck length could just as well be used to argue against instead of for scale invariance of physical laws.

This is how it is used in the TIM. If there is a scale-invariant constant in physics, a scale-invariant physics requires a pretty serious rethinking, similar to what Laurent Nottale attempted with his scale relativity. Again, this is a pretty serious issue that one cannot just paper over with a "philosophical question - dismissed" sticker... :-)

But the majority of "mainstream physicists" are just not ready to make their space-time fractal, as this kind of theory apparently demands. And they are probably right that a lot of assertions in the current presentation of scale relativity sound ad-hoc, that the mathematics is still a bit inconsistent and immature, and so on. But that only means the solution is not completely satisfying yet, certainly not that the problem is not there. In my opinion, this issue is a thorn in XXI^st century physics about as painful as the constancy of the celerity of light implied by Maxwell's equations was for XIX^th century physics. Which is a pretty long-winded way of saying it's worth solving.

Linearity and complex Hilbert spaces

The objections Dr. Anonymous makes to the quantum mechanics part are by far the most interesting ones. They are also the most technical ones, and it's possible that my answers will be unconvincing, if only by lack of space. Expect more iterations here, and possibly some serious changes in the paper to make things more precise or easier to understand.

4. I think that the quantum sections are a little misleading to a careless or non-expert reader(though not intentionally so!), both in terminology and presentation

It is definitely not my intention to mislead. I think that a reason they may be misleading is because it might seem like I attempt to deduce all of quantum mechanics, when in reality there are things I know I deduced, and others I simply borrow because we know from quantum mechanics that they work. The line is a bit blurred, not the least because I tried to push as much as I could into the "demonstrated" half, and sometimes failed (for the moment at least).

the arguments given to introduce linear operators and wavefunctions and the like have little substance,

This is pretty vague, but fortunately specific examples are given later, which I will address then. I acknowledge that the TIM allows only to derive certain properties of QM, but not all of them. What is missing may simply be because I don't know how to derive it, because it derives from other reasons, or because it's not necessary for a working physics. I found cases in each category.

Linearity or real-valuedness appear to be not necessary for a working physics, because the TIM highlights counter-examples. So they are considered as added hypotheses that make some cases easier to solve. For example, the "principle of superposition" is demoted to a simple "hypothesis of superposability" which, when applicable, makes more theorems and results applicable.
The evolution law, for example the Schrödinger equation, is something that I have not entirely been able to derive. There are multiple derivations of Schrödinger, but none of them seems to apply too well in the TIM. I have, however, been able to offer a quick qualitative "sanity test", which requires an ad-hoc, but not unreasonable hypothesis. So that's all there is in the TIM article, and I would certainly understand if Dr. Anonymous or anybody else qualified this section of "having little substance", because it does.
The collapse of the wave function, on the other hand, can entirely be deduced from the need to have repeatable measurements. I have several lines of reasoning that all lead to the same result, so I'm pretty convinced that this is solid.

and do not lead even to the usual Hilbert space description.

By this, I think that Dr. Anonymous means primarily that only a real-valued HIlbert space is implied, not a complex-valued one. Since she makes the objection more explicit below, I'll address it there. Another possible interpretation is that it's always a finite-dimension Hilbert space, even for space-time coordinates. But this is intentional.

Now, if the TIM is not deriving the "correct" Hilbert spaces, isn't it easy to show that it makes incorrect predictions? In other words, instead of not being a theory, it would simply be a wrong theory. So these little differences are what makes things interesting and challenging.

The best one can say is that standard QM fits into your general framework, but that very little of it is implied by this framework.

If it were only the collapse of the wave function, it would already be an interesting result. But I think that more is implied, specifically practically all the axioms but the evolution law. If it's not implied by my reasoning, it is important to tell me why not. The specific remarks that follow are not entirely convicing to me.

Probabilities and evolution

4a. One can certainly, and trivially, rewrite measurement probability distributions p=(p1, p2, ..., pn), which live on the n-simplex, as "unit vectors", psi=(psi1, psi2, ...,psin), which live on the surface of an n-ball (where psi1=sqrt[p1], etc).

This rewrite, made in section 4.1 of the TIM, is indeed mathematically "trivial", but a lot is trivial in hindsight. The really difficult question is: if it's so trivial, why aren't we taught more often that the state vector in quantum mechanics is the only form a state describing predictions of future measurement results can take? It's possible someone else wrote it before me, but I know this is not something I've been taught, and I tried to find references to such a result after discovering it. Honestly, it was so "trivial" I thought impossible no one else would have made this observation. But it is still not a standard staple in QM teaching.

However, there is no reason whatsoever in the TIM framework to postulate that measurements and evolution are linear with respect to psi, rather than being linear with respect to p (or indeed with respect to any any other function g=(g1, g2, ..., gn) of the probabilities - eg, g1=exp[p1]).

Indeed. This is why it is not postulated in the TIM.

For measurements, it is only observed that if a measurement is real valued, and if psi is the probability vector as defined in the TIM, then one can define an operator that 1) is linear in psi, 2) has the measurement results as eigenvalues with specific vectors as eigenvectors, and 3) can be used to compute the right expectation value. So from these properties, one has to deduce that it's the "observable" of quantum mechanics.

For the evolution, I have to introduce it ad-hoc, and I think it is pretty clear in the article. I can't say that I'm happy with it, and I honestly don't have a satisfying answer to offer yet.

And it is merely ex post facto analysis in the light of QM to suggest such postulates are even reasonable!

It is normal to use "ex post facto analysis in the light of QM" to find the results of QM, just like it is reasonable to use "ex post facto analysis in the light of metrology" to find a definition of time that is based on the TIM, but fits our standard definition of time pretty accurately. So this is not cheating, this is simply verifying that QM "fits".

One needs something else to 'derive' a physical significance for linearity in psi (eg, Kaehler manifolds).

Here, I'm really lost. I don't see how Kaehler manifolds can give a physical significance to anything. The physical significance for linearity in psi is that if you construct a linear operator on psi with the right eigenvalues and eigenvectors, it trivially allows you to compute the right expectation value. What else do you need?

In the absence of this something else, it would seem far more reasonable, for example, to postulate that the evolution and operators should be linear with respect to the probability vector p !

I don't think so. To get the eigenvectors, you need the nxn matrix. Then, to get the right expectation value (i.e. average observed value), it cannot be linear in p.

Complex or real Hilbert spaces?

4b. Even when one formally writes down a vector psi as above, it only inhabits a real Hilbert space, rather than the complex Hilbert space needed for QM (eg, to explain the physical double-slit experiment, the Aharanov-Bohm effect, the singlet state, etc).

The argument that complex Hilbert spaces are needed for QM is a frequent one. If that is true, if QM probabilities are somehow special and dependent on complex numbers, how can we get a "double slit experiment", i.e. interference, with classical waves over water? Only classical probabilities are at play in that case.

In reality, complex numbers are only a tool in the computation of probability, to sum probabilities that are not independent from one another. In the water interference scenario, like in the double slit experiments, the Aharonov-Bohm effect or the singlet state, you have probabilities that are "entangled". It's simpler to explain with the water waves, but it's fundamentally the same mathematics in all cases. The waves can propagate through two slits in a wall, but they come from the same source. The height of water at one point (hence the probability of presence of water particles) can be computed by summing the contribution of the wave coming from one slit and from the other. But these contributions are not independent from one another. They are always at a constant phase from one another, which for water waves is a function of the difference in distance between the point where you compute the probability and each of the two slits.

I tried to illustrate how complex numbers emerge in the subsections "Trajectory measurement" and "Normalization of the wave function" in the TIM paper, but to be honest, these are some of the sections I consider the least satisfying of the whole article... It attempts to demonstrate that having a field of "probability of presence" is the correct way to represent the predictions about an experiment where the question is "where is a single particle", but it does not make a very good case that this is the same thing as the wave function in traditional quantum mechanics.

Hence, while the formulas written down "look" quantum mechanical, much more is needed.

A little more may be needed in terms of explanations. But f the formulas look quantum mechanical and behave like quantum mechanical formulas, why can't I legitimately use the quantum mechanical toolset to solve them? And if the predictions and the formulas are quantum mechanical, why is "much more" needed?

Can one use an operator notation for a non-injective and non-linear operator?

4c. The notation M^|psi> for general nonlinear operators, and M^|psi>=m|psi> for eigenvectors, is misleading. One does not have, for example, the property

(M^ + N^) |psi> = M^|psi> + N^|psi>

in general, even though the notation suggests it.

The TIM article is very careful to not use the property you cite, and goes to extraordinary lengths to explain that it does not hold in the general case. That's probably one of the key point of the whole article. Isn't it a bit disingenuous to say that the notation suggests something when the text explicitely says the opposite?

The "hat-M" notation is an intermediate step in reaching the "inverse-hat-M" or "check-M" notation, which does have the property you cite. It would be very confusing to change the notation meaning "apply an operator" just because some operators are linear and some are not, don't you think? It's a bit as if I insisted that the notation f(x) implies a smooth function because I'm used to sin(x) or cos(x), and then required that you use a different notation for any discontinuous functions...

It would be more accurate to represent nonlinear operators as functions mapping the sphere to itself, eg, M(psi), and to refer to vectors satisfying the property M(psi) = m psi as 'fixed points up to scaling' or similar.

It would not really be more "accurate", it would be a change of notation that would obscure the fact that hat-M and check-M are cousins, hat-M being constructed to be a linear operator based on the eigenvectors (or fixed points) identified with hat-M. My notation would only be inaccurate if I did not carefully explain that the notation "hat-M" is used for a non-linear operator, or if I used properties that implicitly require linearity. I do not think that I do either.

As I point out in my short answer, in the general case, M(psi) would be even worse as a choice of notation, because it implies that hat-M is a function, and in the most general case hat-M is not injective, something which the text also insists on. We have a mathematical object which is a "random function with fixed points", and this strange thing will do strange things to your notation. The best I can do is to take the closest notation, and the closests / simplest notation for what I wanted to express is the operator notation.

Collapse limited by the speed of light, or normalization limited by the speed of light?

4d. The QM of entangled systems (more than one particle) is not easily explainable by postulating that collapse is limited to the speed of light, and does not appear to be compatible with recent experiments by Gisin et al (eg, http://prola.aps.org/abstract/PRA/v63/i2/e022111 and http://lists.paleopsych.org/pipermail/paleopsych/2006-April/005093.html,).

The article does not postulate that collapse is limited to the speed of light. It only focuses on how to normalize the wave function, and it points out that collapse is not instantaneous, which is not the same thing. Our computation of probabilities must obey constraints set of the experiment. One of these constraints is that we cannot collect data or setup entanglement between particles faster than the speed of light. Consequently, the space-time points that are summed in a single probability normalization may not be simultaneous in a given reference frame. The experiments listed would only tend to prove that this is the correct approach, not to invalidate it.

For example, suppose Alice and Bob live in spacelike separated regions and share a singlet state (or ensemble thereof), and each decides to measure spin in directions a or b, at random. The probability for Bob to measure spin -up is 1/2 in any run. Now, if collapse is at the speed of light, then his result cannot depend on whether Alice has found spin up or spin down for her measurement - it can only depend on local variables in his vicinity. But in fact Alice and Bob find perfect correlations when they happen to make the same choice of measurement direction - which cannot be explained via such local variables (Bell's theorem). So, whatever entanglement is, its effects cannot easily be explained, and indeed are exacerbated in strangeness, by restricting collapse to the speed of light.

First, there is a somewhat ambiguous use of Bell's theorem in your sentence. Bell's theorem does not prove that there are no hidden variables in quantum mechanics, it shows how to detect if there are any. Only experiments using Bell's theorem such as those of Alain Aspect have led us to think that there are no hidden variables by applying the theorem. I think that this is what you meant...

Second, in your experiments, Alice and Bob's particle can only be entangled if we made them entangled to start with. For example, they can be the result of two particles generated by the same event. These particles then travel at most at the speed of light. So entanglement itself is carried at most at the speed of light.

More to the point, when the experiment is performed, no correlation can be established until the data of both Alice and Bob can be collected in a single point. All the TIM is suggesting is that the normalization condition, if seen as a sum of probabilities being one, can only be collected at that point, where both Alice and Bob are in the past light cone. What will matter is that Alice and Bob's events are in the past light cone. It does not matter if they occured at the "same time" or not, which is good, because the notion of "same time" is quite obsolete in relativity. One of the experiments I found funny, because it's done in such a way that both Alice and Bob think they made the experiment first. Well, it does not really matter, they both did it in the past for the guy collecting the data!

The final word...

I apologise for corresponding anonymously, but after a friend pointed me to your paper and said that you were keen for feedback, I thought this might be helpful as a once-off thing.

That makes me sad. It all started as a helpful and intelligent discussion. But all of a sudden, it's no longer a discussion, it's just a one-way, once-off "thing", a quick attempt to bring the poor fool out of his erroneous ways before moving on to more important "things".

Oh well... Better having anonymous discussions than no discussion at all... :-)

Young talent

2008-01-27T14:37:00.000+01:00

We have a singer at our parish who, two weeks ago, performed a music I really liked. So I asked her what it was, planning to order a CD or something like that. It turns out that she had composed it. I'm not alone liking it, my kids keep humming it, even as I type this.

It turns out that she put a few videos on YouTube. Here is the music we heard that Sunday (but the video clearly doesn't do it justice):

Here is another one in French that I liked:

Enjoy!

Do-it-yourself triod

2008-01-08T13:15:00.001+01:00

Remember the good old vacuum tubes? Here is a video showing how to build one yourself, given enough skills and patience...

Fabrication d'une lampe triode
Uploaded by F2FO

The future of user interfaces

2007-11-28T07:40:00.000+01:00

Smashingmagazine offers a good summary of future user interfaces.

Blog-o-pictures

2007-11-27T12:49:00.000+01:00

I found this blog thanks to this entry about Russian retro-futuristic art.

But truth be told, I prefer some older posts even better.

Also, one about Buran, the ill-fated russian space shuttle...

Funny science quote of the day

2007-11-22T05:45:00.000+01:00

I found this piece funny, in particular:

[Funding a particle accelerator is] not a trivial sum of money for a lot of people, but it's a drop in the bucket compared to the $1,500 per head price tag of the Iraq war to date. And building a particle accelerator doesn't require waterboarding anybody.

(Well, strictly speaking, neither does the Iraq war, and I can't say with certainty what might happen if Alberto Gonzales were put in charge of ILC construction...)

and

this kind of ultra-hot, ultra-dense stupidity can only be achieved by colliding at least two forms of idiocy at speeds approaching that of light

Concept Programming Redux

2007-11-17T21:25:00.000+01:00

Based on feedback from many people (most notably Daveed Vandevoorde and Lionel Schaffhauser), I posted a new presentation about Concept Programming. This should make the ideas much easier to grasp.

During the discussion, Daveed Vandevoorde pointed me to a very interesting paper about memory transactional models, which he used as an argument in favor of functional programming. Recommended reading, even if I don't think that this is a point for functional programming at all myself (what is really being used is the very strong type system of Haskell).

Lionel Schaffhauser pointed out a number of presentation-related issues, which prompted me to change the order of the slides and add a whole section about the "Maximum" example.

As a reminder, there is also an older presentation about XL itself.

See announcement on the XL web site

Airbus construction

2007-11-17T20:55:00.000+01:00

You may recently have heard that an Airbus A340 was recently damaged during ground testing. Well, it's surprising this does not happen more often. Below is a video showing just how complex the construction of an A340 is:

E8 theory...

2007-11-15T20:01:00.000+01:00

Garett Lisi recently posted An Exceptionally Simple Theory of Everything on ArXiv. Don't expect the article to really be "exceptionally simple", the title is really a pun on the fact that E8 is the largest exceptional simple Lie group. There's nothing really simple about all this, except that it's a beautiful form of symmetry.

This work was quickly picked up all over the place, for example at Backreaction, where a good discussion followed. It even made it to Slashdot, where whatever discussion was typical Slashdot ("it's funny, laugh!" :-). Not useful, but entertaining.

Unfortunately, it also made it to less reputable science outlets. My favorite blogger made predictable comments about it being stupidly wrong, rejoicing in an arXiv demotion from hep-th ("professional") to gen-ph ("laymen's fantasies"):

Update I: the preprint was re-classified from the professional hep-th archive to gen-ph, general physics, an archive mostly dedicated to laymen's fantasies. Thanks God. Comment for general readers: this preprint is of course not peer-reviewed and probably won't get published anywhere.

That demotion may very well be Motl's own fantasy, as the preprint still appears as hep-th for me. In any case, you can compare Lubos' "professional" piece of work to, say, John Baez's reaction to the exact same event. True, Baez also shows some skepticisim, which I will share for the moment, but he remains healthily neutral, whereas Motl "exploded in laughter" at the first equation in the paper. More importantly, Baez, unlike Motl, starts looking at similar work, like earlier attempts at unifying forces using SU(5). That's the right way to go. Even if the paper is totally bogus (which I don't think it is, more likely to be subtly bogus), you still have to prove it, which Lubos Motl spectacularly failed to do as usual.

So... Why be skeptical? Simply because that's not the first time folks look at E8, and even if Garrett Lisi's approach is innovative, it may still very well be entirely wrong, until experiments show the predicted additional particles. But at least, Garrett is ready to take that risk, and that's worth a lot in my book. And, if nothing else, he provided us with a very beautiful video:

So at this time, what I find a source of concern is not Garrett's article itself, it's all the publicity around it. Early publicity for something as "grandiose" as a "theory of everything" can prove fatal to the researcher if he turns out to be wrong, or to not make as much progress as expected. That's in no small part what happened to Laurent Nottale. And just like for Nottale, that would be a shame if Garrett Lisi ended up doing nothing more than surfing in Hawaii...

And I also believe that Garrett Lisi's work is absolutely not in conflict with my own ideas, nor any step in that direction. In other words, the paper I submitted still shows, in my opinion, just how far Garrett's work is from a "theory of everything".

The other side

2007-11-14T23:42:00.000+01:00

Funny that I should, twice in the same day, read a story about what the other side has to say about one of Richard Stallman's many gripes:

The first one is Rob Pike's recollection of a protest against one of his software patents.
The other is Dan Weinreb's rebuttal of Stallman's story on Symbolics and LMI.

Selling: From complex to simple

2007-11-14T23:33:00.000+01:00

An interesting thought from a friend of mine over lunch a couple of days ago (that's my rewrite, so errors are mine).

Hi-tech companies start with the complex, high value add, because it means high differentiation. That's where engineers shine. But as their market expands, marketing becomes more and more important. And marketing hates complexity, because it's hard to sell. So as the company grows, it natually swings towards commodity markets, not because they bring more money, but because they are simpler to sell.

Simple. Right, or wrong?

Growing a language

2007-11-14T21:26:00.000+01:00

I recently re-discovered an excellent paper I had not read in a long time, Growing a languag by Guy Steele. Highly recommended.

Concept Programming presentation posted

2007-11-10T12:17:00.000+01:00

I finally posted a slideset about concept programming that had been rotting on my hard disk for way too long... (also posted here.) This gave me an opportunity to re-read older articles on these topics, one I had written for Dr Dobb's Journal, and another one written by Martin Heller (Mr. Computer Language Person) for Byte. They are still pretty accurate.

What is concept programming?

Concept programming is a new programming paradigm which is intended to reduce the gap between the concepts in your brain and their representation in computer code. The method is detailed in the slides, so I won't repeat it here. There is also a very simplistic overview at Wikipedia.

These techniques apply to many programming languages. I used them personally for assembly language in HP Integrity VM. But just like you can do object-oriented programming more easily in C++ than in C, it's easier to use a language designed specifically for it. The slides also give a few examples of how the concept programming philosophy impacted the XL language design.

Making Things Happen - Getting Help

Concept programming has a pretty long history. I'm not sure when I invented the term exactly, but I'd say before 2003 (I'm pretty sure it was explicit on the "new" XL web site). But this is too valuable to be left dormant like this. I would so much like to spend more time on it, but... too... many... things... at once.

This is the reason I am trying to get some help getting this stuff done. It actually proves difficult. You would think this is easy: the ideas are pretty clear and easily explained, a lot of code is already available. So what's the problem? The problem is getting people to listen at all, just like in that story of that world-class musician posing as a street musician. If you don't remember the results: practically nobody listened to him.

Well, just like that musician, concept programming and XL will require more than talent and technique. They are now in need of some kind of advertising, evangelism, call it what you want. If you think that you can help promoting these simple ideas about programming, let me know...

Grenouille Bouillie

Switching to Wordpress?

Hello World - Switching to Wordpress?

A 3D "Hello World!" in 23 lines of code...

The XL axioms: reconciling Lisp and C++ hackers

Lisp inventions

What makes XL unique

Axiom #1: A simple parse tree improves code readability

Axiom #2: Programs can be evaluated with macros

Axiom #3: C+ hackers and Lisp hackers are both right

Axiom #4: It works, almost

XL: Advancing the state of programming languages

From concept to code

Linus Torvalds on religion

Facebook privacy policy

Patents considered harmful...

Dark matter as a proof of E.T. intelligence?

Where are the hackers gone?

The Good and Bad of DRM defense

Virtual Windows

Steve Jobs decided to knife the baby (oh no, not again!)

Eating my own dog food

The Reinvigorated Programmer

Making the best of your brain

Why Go isn't my favourite programming language

Summary

First impression

Stated Objectives

Implicit Objectives

Conclusion

Dark matter, the modern aether

LeMag IT article about HPVM

ACM Sigplan Workshop on Generic Programming

Goals of the workshop

Goodbye HP, Hello Taodyne

Incubation - Accepted

Incubation - Dry run talk

Welcome Marie...

HP Pay Cuts - an unfair act of economic opportunism and greed.

HP DE200C Digital Entertainment Center

Science and the fear of innovation

Enough hot air...

There is a psychological explanation

... or is there?

Stability and innovation

Stability vs. Innovation

One step at a time, or "TIC-TAC-TOC"

When innovation is the problem

Genuine open-source innovation is hard to find...

Factor, an interactive extensible programming language

Cool high-level virtualization video

Why writing proprietary software is not fun

Why it's worse today than Raymond remembers

Incompetent bosses were replaced by powerless bosses

Competing

Open-source vs. Corporate?

Where FOSS falls short

We all need to eat

Virtual machines and scalability

What is scalability?

Scalability of virtual machines

Checkmark features vs. usefulness

Performing well in degraded configurations

Remaining balanced and fair

Loading virtual machines little by little...

Actual scalability vs. simple metrics

One picture is worth 210 words

Conclusion

IBM's comparison of virtual machines...

nPartitions are tougher than logical partitions

Integrity servers support more operating systems

HP offers both isolation and flexibility

Shared vs. dedicated

Virtual Machines

Preserving the investment matters

Update: About history

What is your dangerous idea?

An impressionist painting of today's intellectual landscape

The dangerous uniformity of bland gray matter

How could you tell if you are in the Matrix?

One picture is worth 2¹⁰ words