blogs

Donnie, I am very aware of Spider's Chinstrap. I've spent quite some time discussing the concept of a "binary Gentoo" with him, both on IRC and in real life.

As he pointed out to me then, compiling a package set is really the least of your worries when building a binary distro. The result should be an integrated whole, and when you install a package you expect it to Just Work(tm). Here, there are many devils, all hiding in the details.

An obvious one, is the fixation of USE flags. In a binary distro, it is customary to split one project into smaller "sub" packages. A trivial example is ncurses, which is usually split into ncurses-lib and ncurses-dev (or libncurses5, libncurses5-dev on Debian). The -dev package contains the static library version, API man pages and the headers files. This makes a lot of sense in a binary distro, because most of the time, the users won't compile their own programs based on the installed packages and would rather not keep the -dev part around.

In Gentoo, we don't have the required kind of meta information lying around to be able make such a split. We could apply some heuristics -- picking out the static libraries and the header files would be easy, but separating the API man pages from "regular" man pages is less trivial. Some packages come with scripts or programs which are only useful when you want to compile against the package, such as a foo-config, telling you the location of header and library files. Such files should also go into the -dev package, but this fileset is impossible to automatically deduce.

A more tricky situation happens if you look at global USE flags. In a way, our global USE flags allow you set a global property of your system, such as "I want xine support". (Of course, you can override this on a per-package basis if you really want to). In a binary distro, you want to capture this variability by having a subpackage providing just this, for every package which as optional xine support. As an example, in Ubuntu, there is the amarok package (a media player) that has optional xine support. If you want xine, you need to merge the amarok-xine package. In Gentoo, we have a xine USE flag on media-sound/amarok itself. In general, will be very difficult to automate the generation of these extra subpackages based on USE flags on the main package, but at the same time, the user expects this kind of configuration granularity in his system: I certainly won't be bothered to use a system where they have pre-selected a least common denominator in all cases.

That being said, perhaps we can at least get automated, up-to-date (unoffical!) GRP sets when Spunkbox (a Gentoo tinderbox implementation) is installed on Patrick's German überkluster.

2 - 4. of May, the Sixth Stratego User Days were held at Utrecht University. This year, attendees came from The Netherlands, Norway, UK, France and Russia, all presenting their neat projects based on Stratego.

The first day was basically a tutorial on Stratego, for newcomers. I didn't attend. The second day started off with Martin presenting the state of Stratego/XT. He gave a pretty decent overview of where we are, and where we are headed.

The other presentations that I found in particular interesting that day was the series of talks held by the EPITA people. Very inspiring. Especially their work on the C++ frontend (Transformers) is of relevance to us, since we're in need of a new C++ frontend to plug into CodeBoost. Transformers is based on SGLR with a highly ambiguous SDF grammar. The resulting parse is disambiguated using a basic attribute grammar system written in Stratego. The grammar definition is annotated with the disambiguating attribute programs.

They had also come up with a sort of AST conversion scheme, and made a converter from their AST (which is pretty tied to the C++ grammar) into the CodeBoost AST (which is a lot more high level). This will save us from a lot of work if (when?) we move over to their frontend.

The final day, the EPITA people presented detection of code smells and how it can work as a guide for (semi-)automatic refactoring. This is also something I find to be highly interesting, but I can't say I have the time to investigate this in detail at the present time.

As for myself, I was fortunate enough to have material for three presentations this year. The first one was Combining Aspect-Oriented and Strategic Programming, which was basically a rerun of my presentation in Japan. The second was a crackpot idea we've been exploring a bit the last few months, that we call Clone, Adapt and Improve. The presentation by itself doesn't make much sense without the accompanying paper, which is still under revision. The final presentation was Spoofax: A Stratego Editor -- a brief overview of the current state of the Spoofax Editor.

The wiki page for SUD'05 contains (will contain) all the presentations

My laptop had a severe case of file system corruption a little over three weeks ago, and I had a trip Japan coming up in few days, so a decent reinstall of Gentoo from scratch was not going to be possible.

Looking around for alternatives which were quick to install and had a suitable selection of packages, I decided to revert to Debian -- sort of.

I've been following the Ubuntu distribution for quite some time now (quite some internet time, that is), and from the outside, it looks a like a pretty neat distro. It's being advertised as a good all-round distro, but with special attention to desktop users. Too many bad memories about GNOME 2.2, 2.6 and 2.8 have slowly but forcefully pushed me into the KDE camp, so the recent release of Kubuntu and KDE 3.4 was the feature that closed the deal.

Due to circumstance, installation was extremely tricky. I did not have any external media on my machine, and only a half-working Gentoo distribution. So this is what I did:

First, I disabled my 1GB swap partition (hda2), and installed VMWare. Then I configured VMWare get physical access to hda2. I downloaded the latest Kubuntu ISO, and booted it inside VMWare, installing a minimal text-mode setup on hda2. Then I added an entry in my grub.conf on hda1, so that I may boot into my newly installed OS. After this, I rebooted into Kubuntu with init=/bin/bash, wiped my Gentoo drive on /dev/hda3, reformatted it and copied kubuntu from hda2 to hda3, fixed up the boot record again, and rebooted one final time into Kubuntu. From there, I just apt-get'ed my way into a full system.

In retrospect, the installation procedure must by all accounts be rather simple. It required only two reboots -- less than a typical Windows install when applying the service packs -- and was done completely without external media.

There are numerous aspects of the (K)Ubuntu distribution that I found very appealing:

• As it's based on Debian, their package system is familiar and stable.
• Compared to Debian (and Gentoo), the Ubuntu package repository is tiny, but by adding universe and multiverse to /etc/apt/sources.list, it becomes respectable.
• As with Debian and Gentoo, every package is just one command away, and they even have a decent graphical package handler for most basic stuff.
• Unlike (my experience with) Debian, the packages are actually up to date, even bleeding edge in may cases. This is a very welcome change. My original reason for defecting to Gentoo from Debian was just this acute lack of up-to-dateness (even unstable seemed outdated at the time).
• In a real pinch, you can even add Debian repositories to your sources.list.

Overall, after spending three weeks and going half way around the world with this distro, I must say I'm pretty happy with it. As Linux distros go, it's definitely in the class of "sucks less".

However, if I were a regular user, I'd be very stuck very soon, as there's still a lot of tweaking necessary to get everything working somewhat decently. Examples abound:

The dhclient utility, when run without any parameters, will look for IPs on all available interfaces, a nice feature. Unfortunately, it will also listen on the loopback interface, determine that there's no IP coming, and remove the existing IP (127.0.0.1) from that interface. As you can imagine, cripping lo like this leads to a lot of peculiar behaviour in the system, such as printing not working, since cupsd on localhost:678 can no longer be reached. Of course I reported this.

On the topic of cups, it is set up not to auto-discover printers. This is the same as in Debian. I can certainly agree with the security aspects of this, but as a user expecting a proper plug-n-play desktop, the default settings are a bit on the safe side for my tastes.

Another issue -- and this is the kicker -- is that APM no longer works. That is, by disabling ACPI and enabling APM, I actually get suspend and resume, but on resume, the kernel oopses, complains about a preemption error, then proceeds to spit out an endless (presumably, I haven't verified yet) list of IDE errors of type SectorIdNotFound.

Donning my dumbfounded user hat, I could kick back and say confusedly "but it worked in Gentoo!". But I decided to bring over the 2.6.11-gentoo-r6 kernel, to perform a closer comparison. When I compiled it with the same config as the Ubuntu kernel (2.6.10-34), it gave the same results, i.e. oops and IDE problems. Hmm, strange. Then I reconfigured the kernel from scratch, and still the same result. Then I rebooted the machine with init=/bin/sh, loaded the apm module and suspend/resume. Still SectorIdNotFound. At this point, I invoked "but it worked in Gentoo!" and still hope for a kernel update.

Another thing that bit me pretty hard was the DPI difference between KDE and GNOME apps. This may just be because I decided to apt-get my system and didn't use the nice and friendly installer all the way. Whatever it is, it shows up (as it always does) with GNOME showing correct font size when X is configured with DPI 96, while KDE shows a correct font size when X is configured to 75. Nothing a little tweaking can't get around, but you'd think this had been settled for you in a desktop distro?

When most essentials were dealt with, I decided to play my music. I mean, my legitimate backup of my music. The music that I also have to play on small, flimsy audio players which only accept WMA, WAV and MP3. Yes, music in MP3 format. Oh, you don't do that? Because it's patent encumbered? But it's not! There are no software patents in Norway (yet). Still, they robbed me of my choice, I had to punt and add yet another external source to my sources.list, and then it almost works. Except for mplayer crashing even more frequently than it does on Gentoo.

Same deal with Java. You have to add ftp://neacm.fe.up.pt/pub/ubuntu-java/, and then you don't even get Eclipse. Being part of the Java herd, I guess I'm overly biased on this point, so I'll just stop complaining and rather sigh once, demonstratively.

In general, the (sometimes to small collection of) projects which have have been packaged for Ubuntu work nicely, with some rather strange exceptions where application behaviour is not exactly what I'm used to. For example, Firefox does not accept one throwing URLs at it (by pasting a URL into the HTML widget), making it ever so cumbersome to work with. OpenOffice pops up a query every time it wants to auto save. If you are unlucky enough to be making a selection and holding your left mouse button at the same, time, it will crash. In the course of preparing a set of slides for a recent conference, I was attacked by this four times in less than an hour, so in practice it's not a minor issue. Speaking of minor issues, Eclipse M6 (binary from eclipse.org) shows a lot of strange rendering bugs.

In summary, I don't consider any of these issues to be blockers, except for the APM problem, but that is more of a kernel "feature", I believe. I must confess that I have found the experience of using a different distro rather refreshing. There have been times when I've been far less than happy with Gentoo, and thought that if I just moved to another distro, all would be well (greenpasturitis). In a sense, it's reassuring -- at least for Gentoo -- to see that this is a fallacy.

Popped over to Japan for a week now in the spring, to present the paper I wrote with Eelco on AspectStratego (see the research page) at RULE 2005. As usual, RULE was cohosted with RTA, and both conferences were held in Nara, Japan.

I flew via Hong Kong and landed in Osaka before I made my way to Nara using the railways. As the weather was rather cold in The Netherlands when I left, I was completely maldressed when I arrived in the rather warm spring (ooh, cherry blossoms). I got all kinds of weird looks on the train, probably accompanied by puzzled thoughts about this "crazy gaijin".

On my second day, I met with Mamoru Komachi who's a fellow Gentoo developer. Mamoru showed me around the Nara prefecture. Nara city was one of the first capitals of Japan. Mamoru also took me to Asuka, which is the birthplace of the Japanese civilization. The map said "Ruins on Asuka", but as everything in Japan is mostly built from wood, there wasn't much of a ruin left.

The conference itself was rather nice. Phil Wadler held the RTA keynote and Thierry Coquand held the RULE (join with TLCA) keynote. Of the papers presented, I think the paper by Tankut Aktemur and Sam Kamin on Mumbo was the most interesting one for the work we do. That is not to say that many of the more theoretically oriented presentations weren't intriguing.

I released an update to the Stratego/XT editor plugin part of Spoofax today. This new version adds simple syntax highlighting support for SDF files.

I am still in the process of rewriting the Stratego parser to enable code completion and outlines. Further, I've started thinking about how to couple the editor with the compiler to map back Stratego compiler errors. I think this would easy as π if strc were to emit the errors as an ATerm.

All of this is slow going, as I have more pressing matters to finish.

To test the plugin provide the URL
http://www.ii.uib.no/~karltk/spoofax/
to the Eclipse Update Manager.

An enticing screenshot:

I'm currently roaming about in the Kansai region of Japan. It's wonderful, so I won't spend too much time blurting inanely about the Japanese experience, but rather spend the time on exploring.

(entirely raw, straight off my CF card)

Apart from doing research on and writing the documentation for our April Fools joke (GeNToo), I've not had too much time for Gentoo lately.

This week I've been playing catch-up -- as usual -- and finally sat down to update the Eclipse ebuilds. In my absence, both 3.0.2 and 3.1M6 were released upstream.

For a few days, it seemed that we would not be able to provide 3.0.2 at all, as it was (re)designed to require a full Gecko SDK installed, and would not compiled against Mozilla. One of our users managed to find a semi-decent workaround, see #86850. It allows the 3.0.2 release to be compiled against Mozilla again, and gives partial support for the embedded (Mozilla) browser. It may be that my memory is confuzzled by using the 3.1 series for so long, but it appears that that the javadoc view in 3.0.2 is text-only (no fancy html formatting).

As for M6, I have finally been able to construct a working ebuild for it. The Eclipse guys sure like to move things around in their build system, so I decided to revise the entire ebuild. The biggest change in M6 from a packaging standpoint, is that each plugin may (and should) now be a self-contained .jar file. It may even contain platform native .so files. On plugin activation, these will be extracted to the default configuration directory (On Gentoo, /usr/lib/eclipse-3.1/configuration). If that fails (say, because of permissions), they will be extracted to a directory deep inside ~/.eclipse.

From a deployment perspective, this may be wise, but for packaging it on Gentoo, it has the following drawbacks:

• If you run Eclipse as root, it will mess around in /usr/lib/eclipse-3.1, and create files there which Portage cannot remove at uninstallation time (as they are extracted only at runtime, Portage doesn't know anything about them). Obviously, this is butt-ugly and very Windowsy.
• The lack of information about the .sos will break revdevp-rebuild, since revdep-rebuild will not be able to determine the correct package to recompile for these rogue .sos.
• I am not certain what happens to the old .so when you recompile Eclipse with a different set of USE flags and restart the platform, say you removed Mozilla support. From my inital tests, it appears that they will just keep floating around in the configuration directory.

There are a few workarounds for this mess that I am investigating, but none have so far been very appealing. We provide M6 pretty much as-is, and await feedback from our bright and savvy users:)

As promised, the preliminary draft of the paper Combining Aspect-Oriented and Strategic Programming by myself and Eelco Visser is now available.

Abstract

Properties such as logging, persistence, debugging, tracing, distribution, performance monitoring and exception handling occur in most programming paradigms and are normally very difficult or even impossible to modularize with traditional modularization mechanisms because they are cross-cutting. Recently, aspect-oriented programming has enjoyed recognition as a practical solution for separating these concerns. In this paper we describe an extension to the Stratego term rewriting language for capturing such properties. We show our aspect language offers a concise, practical and adaptable solution for dealing with unanticipated algorithm extension for forward data-flow propagation and dynamic type checking of terms. We briefly discuss some of the challenges faced when designing and implementing an aspect extension for and in a rule-based term rewriting system.