Tech Preview: Sabayon on ARMv7

One week ago, the BeagleBone I ordered from Tigal.com eventually landed on my desk. As you may know, it’s an ARMv7a OMAP device with 256Mb RAM, USB 2.0 and FastEthernet 10/100. The board doesn’t come with HDMI output but daughter boards are expected to be shipped soon it seems.

It ships with Angstrom Linux, an embedded distribution that is no way close to Gentoo, at least in my opinion. I found it kinda broken and slow. opkg, the package manager, is a nightmare. I had no choice, that thing had to be dumped ASAP. And that’s what I did.

First thing I did was copycating the Angstrom kernel configuration by copying /proc/config.gz in a safe place and starting to merge the Beagleboard kernel tree into my Linux 3.1 branch. This of course means that the AM335x is not yet supported by the vanilla kernel. Last I heard is that there are plans to merge the patches during the 3.3 merge window… You can find two ebuilds in the “sabayon-distro” overlay: sys-kernel/beagleboard-sources and sys-kernel/linux-beagleboard, providing sources and binaries respectively.
Introducing ARM support in sabayon-kernel.eclass (the eclass that builds kernel binaries using genkernel) was quite straightforward, it now builds uImages directly!

The boot strategy works like this: u-boot.img searches /boot/uImage into the root filesystem (ext4 doesn’t seem to work with my image). In our case, /boot/uImage is a symlink pointing to a versioned file (the one installed by sys-kernel/linux-beagleboard). You can manage the symlink using eselect-uimage, from the “sabayon” overlay and shipped with the Sabayon images already. This means that you can change the boot kernel at runtime without even touching the boot partition!

The second thing was setting up a chroot, both Entropy build chroot (for pushing out binary packages to the armv7l repo) and “image” chroot (the one from where images are generated) using qemu-user to emulate armv7l. In order to be able to prepare disk images using loop devices, I also completely rewrote the famous “mkcard.txt” script, dropping bc dependency (hey, bash can do math already!!). You can find it here, as well as molecules that we use to build the ARMv7a images for Beagle{Bone,Board}.

If you are interested in knowing more about how I managed to get Sabayon on ARM, have a look at the “Hitchhikers Guide to the BeagleBone” on our wiki.

I uploaded the Sabayon ARMv7 images on our mirrors this morning, under the iso/daily directory, in different sizes (depending on your MMC card size): 4GB, 8GB, 16GB.

a75fd2a7d9cae17762034c5c049a08fc Sabayon_Linux_DAILY_armv7a_Base_16GB.img.xz

0e17081050fa19c7f769318e3235ebaa Sabayon_Linux_DAILY_armv7a_Base_4GB.img.xz

56606bc906715cdebce02611ae03285c Sabayon_Linux_DAILY_armv7a_Base_8GB.img.xz

Installing them onto your MMC card is as easy as running:

xzcat <image file>.xz > /dev/sdX

Where /dev/sdX is your memory card device (might be mmcblk0).
They come in different sizes, make sure to match the advertised image size with your MMC device one.

If you have 32GB or 64GB MMCs you have two choices: either use the 16GB version and create a separate partition later or take the bootfs and rootfs images (they come in different sizes but the content is the same) from the same dir:

93960b28cde8dde51f2d29cc0c76f6bb Sabayon_Linux_DAILY_armv7a_Base_16GB.img.bootfs.tar.xz

e85fe8d344dacf79eec94562a59c6750 Sabayon_Linux_DAILY_armv7a_Base_16GB.img.rootfs.tar.xz

93960b28cde8dde51f2d29cc0c76f6bb Sabayon_Linux_DAILY_armv7a_Base_4GB.img.bootfs.tar.xz

1a6bce6f585d52f2b50806bd2bd69578 Sabayon_Linux_DAILY_armv7a_Base_4GB.img.rootfs.tar.xz

93960b28cde8dde51f2d29cc0c76f6bb Sabayon_Linux_DAILY_armv7a_Base_8GB.img.bootfs.tar.xz

13b2a47a88c55c8692ce61fc2fd42022 Sabayon_Linux_DAILY_armv7a_Base_8GB.img.rootfs.tar.xz

This way you can create your own partition layout and then unpack the content into the respective partitions. So easy. No grub nor MBR nightmare!
If you are as lazy as me, here is the download link to the directory (using the GARR mirror). But you are encouraged to use our download page to find the mirror closest to you.

The root password is: root. The OS is set to automatically boot and start eth0 and sshd (so you can connect to it via ssh). During the first boot, there is a script that configures some stuff and reboots your device automatically (so on the first boot it will just reboot). If you want to change the locale, edit /etc/env.d/02locale and run locale-gen !. The System is already configured to allow serial login (at least this works on the BeagleBone out of the box).

That’s it. You have a great distro (Gentoo) with a great Package Manager, Entropy (Sabayon) in a credit-card sized device.
If you appreciate our efforts towards the ARM architecture, please consider to donate us either hardware or MONEY to buy it! (yeah, we can’t just handle money, we always need money!).
If you’re a developer and interested in ARM stuff, why don’t you join us? We can improve both Gentoo and Sabayon together!

Please note: I only have a BeagleBone for now thus I wasn’t able to test out the images on the BeagleBoard. Moreover, the boot partition contains Beagle* related boot binaries that won’t work on other OMAP devices out of the box (but still, we provide split boot and root filesystem images).

Please note 2: I consider this a tech preview because at the time of writing, we only have 400 binary packages available for install. You can browse them using http://packages.sabayon.org.

DONATE US !

Top-ix.org is supporting Sabayon

I am very excited to inform you guys, that the number of production servers powering the whole Sabayon Linux infrastructure has grown once more.

This time it’s all about the Top-ix Consortium which is going to host our main Entropy and ISO mirror very very soon (the migration is in process right now). The same Consortium kindly donated a powerful Intel Xeon Quad Core system with 4GB RAM, quite enough for the expected load. Thanks to this, we’ll be able to reinforce pkg.sabayon.org and distfiles.sabayon.org aiming to make our users happier.

So, thanks to Top-ix and their Development Program. But still, don’t forget to donate us, we need your support!

Splitting the Sabayon Overlay

After 2 months of hard work, Enlik and I (but mainly him) completed the split of the Sabayon Portage overlay. But what is it about?
It all started when Pacho Ramos kindly asked us to separate ebuilds that could be upstreamed (towards Gentoo) and ebuilds that only contained Sabayon business logic.
Since our overlay was quite fat, it really took us several weeks to accomplish the request… {omg here}

After the split, we ended up having: overlays/for-gentoo.git and overlays/sabayon.git git repos containing respectively the “sabayon” and “sabayon-distro” layman repositories.
So, layman -fa sabayon will now pull the for-gentoo.git overlay, which contains new cool stuff geared towards Gentoo users as well.
And, as said above, layman -fa sabayon-distro will give you the rest of our ebuild stuff.

Now it’s time to (slowly) “monetize” the split helping out upstream with patches and version bumps!

Eit: the stupid package tracker, reinvented

If you are maintaining either a Community Repository or a full blown one (inside a chroot, etc), please go ahead reading this post, important changes are going to happen very soon.

As the average FLOSS developer does, one morning a few months ago I woke up and decided that I needed to break stuff. Three months later, it turned out I fully achieved this goal, for your great joy.

Eit is going to ease your life as Entropy repository maintainer (and packages cook!) in a very dramatic way. Largely inspired to the “Git-way of doing things” and to its awesomeness, it represents a complete rewrite of the “Community Repositories/Entropy Server” functionalities exposes by “equo community”, “reagent” and “activator”.

It will be available starting from Entropy 1.0_rc60 and will completely replace the old tools and their syntax carrying out a lot of minor improvements as well.
Please note that Community Repositories maintainers, shall need to set “community-mode = enable” inside /etc/entropy/server.conf (see server.conf.example for more info) in order to make Entropy Server behave correctly.

How the syntax changed actually? Just a few examples below.

eit status [<repository>], shows repository status, unstaged packages, packages ready to push, etc.
eit commit [<repository>], commits unstaged packages to given repository (or the current one).
eit checkout <repository>, switches to the given Entropy repository.
eit add [--to <repository>] <package> [<package> ...], add the given unstaged packages to repository (current repository if none given).
eit push [<repository>] push staged changes to the remote repository.

This is just a subset of all the commands available, but they’re enough to start using Eit. “Top-level commands” are implemented as plugins to allow maximum modularity and 3rd-party extensions. Just subclass eit.commands.command.EitCommand and implement the respective methods.

Angelbird Wings x4, 32GB SSD donated

People at Angelbird and Cloudspecialists.net were kind enough to send us a Wings x4 PCI-Express SATA card with super-fast 32GB SSD memory onboard and space for 4 2.5″ SSD drives. Besides being a simple/cheapo SATA card, it seems to work great under Linux (which is worth mentioning), the controller is the Marvell 88SX7042 and supports up to 4 2.5″ SATA drives.
Mounting /var/tmp/portage on it is giving us a nice I/O boost.

Thanks Angelbird!

AMD Catalyst 11.9, complete disaster?

Let me explain my feelings a bit deeper. I do really appreciate the effort AMD is putting in providing high quality GPU drivers (in terms of performance and power consumption) for our beloved Operating System.

But this time I feel quite depressed. We were delaying Sabayon 7 just because we wanted to ship with working AMD drivers, and make GNOME3 users happy. Today, it turned out this is not going to be the case: x11-drivers/ati-drivers-11.9 seems to be a complete disaster, people still reporting (also been able to reproduce them) GNOME3 rendering issues (flickering), KWin and other random crashes (regressions!) as well. Yes, even KDE is crashy on fglrx now.

AMD is unlikely going to fix the situation in reasonable time (we must release Sabayon 7 asap!), so, like it or not, we are forced to revert back to 11.8 and ship our new ISO images with GNOME3 Shell not working on AMD GPU systems.

We are very sorry.

Entropy API Tutorial #4: inverse dependencies

Before moving on to other Entropy codebase areas, I need to also mention inverse dependencies. The same are used for calculating a removal schedule basing on currently installed packages. Let’s say you want to remove app-foo/bar, are there packages needing it? If so, are there packages needing packages needing it? And so on… If you know something about Algorithms, you can easily understand that the whole thing is just a topological sort out of a DAG (actually package dependencies unfortunately come with cycles, so, even if it wouldn’t make much sense to have dep A pulling dep B itself pulling dep A, this is what we have in real life, for reasons out of the scope of this blog post…).

But let’s keep things simple, especially if you’re new to these algorithmic concepts (scratch them!).
First of all we need to match a package name in the installed packages repository, then query Entropy Client asking for a removal schedule, and perhaps execute the actual removal.

import os
from entropy.client.interfaces import Client
entropy_client = Client()

installed_repository = entropy_client.installed_repository()
# let’s match app-foo/bar out of the installed packages repository
package_id, pkg_rc = installed_repository.atomMatch(“app-foo/bar”)
if pkg_rc != 0:
    print(“cannot find app-foo/bar”)
    entropy_client.shutdown()
    raise SystemExit(1)

# now ask Entropy Client to build up a sorted removal list
# in case of system packages, if system_packages keyword arg
# is True (see API documentation), DependenciesNotRemovable
# exception is raised. But we’re not going to discuss these
# details here for brevity.
packages_ids = entropy_client.get_removal_queue([package_id])

# now let’s remove them from the Live filesystem!
exit_st = os.EX_OK
for package_id in packages_ids:
    pkg = entropy_client.Package()
    pkg.prepare([package_id], “remove”)
    rc = pkg.run()
    pkg.kill()
    if rc != os.EX_OK:
        exit_st = rc
        break
entropy_client.shutdown()
raise SystemExit(exit_st)

That is pretty much it!

kernel.org abbreviated is k.o.

Not that I dislike it. I mean, I always thought that the release process was just a crazy mix of different flavours of madness. Especially with the stable repo, containing “facepalm”-like fixes (hey, this is software engineering, not rocket science — but still!).

After a full month, git.kernel.org (both http and git ports), http://www.kernel.org, are still down, and I’m starting to get really bored. Linus temporarily (permanently?) moved his git repo to github.com in order to keep 3.1 development going and this seems to have worked for a good cut of consumers. But how about the other 100+ repos that were sitting on git.kernel.org? They’re not even accessible in read-only.

Being the Linux Kernel a vital part of the whole Linux-based FLOSS world, how can we accept a 1-month downtime? Why isn’t there anybody out there starting to raise the voice? That is completely ridiculous. While I understand the QA and Security teams involved in bisecting the kernel.org logs etc, I cannot really accept that git.kernel.org is still down, for the following reasons:

  • git repositories themselves haven’t been harmed (as they said — not my own words), so git.k.o could be restored at least to a read-only state without being much scared
  • ssh keys could be revoked for everybody and their access limited to git pushing

Can’t wait to ditch my frustration seeing kernel.org back to life.

Sunday as first weekday, how can glibc be wrong?

It all happened after Sabayon bug 2699, and I am still laughing so hard.
Everybody in Italy knows at least two things: who is Silvio Berlusconi and what is the first weekday. I cannot guarantee on the rest, though.
But of course, glibc asserts that the second pillar of the Italian knowledge is just… wrong. For it and Ulrich Drepper the first weekday is Sunday and no matter what the fuck you think, say, write: as long as you don’t have the proof of it being Monday, you will live with the bug.

It seems that drepper has more power than our Prime Minister and his escorts all together when it’s about picking the first weekday! woot!

Entropy API Tutorial #3: installing packages

Basing on my previous Entropy API Tutorial #2 regarding how to calculate package dependencies through the Entropy Python API, let’s move forward and see how to actually install packages from a repository.

Let’s say we have a list called “package_matches” containing all the package matches we want to install (dependencies have been expanded already). At this point we’re ready to ask Entropy to fetch from mirrors and merge them into the live filesystem. The following example is very simple. For the same reason, it doesn’t cover all the functionalities, such as, for example, multi-fetching or fetching of package source code.

import os
from entropy.client.interfaces import Client

entropy_client = Client()
# this is out package_matches list (fake content)
package_matches = [(12345, 'sabayon-weekly'),
    (12343, 'sabayon-weekly')]

# first fetch, then merge
exit_st = os.EX_OK
for pkg_match in package_matches:
    pkg = entropy_client.Package()
    pkg.prepare(pkg_match, “fetch”, {})
    rc = pkg.run()
    if rc != os.EX_OK:
        exit_st = rc
        break
if exit_st != os.EX_OK:
    entropy_client.shutdown()
    raise SystemExit(exit_st)

# now merge into our live filesystem
for pkg_match in package_matches:
    pkg = entropy_client.Package()
    pkg.prepare(pkg_match, “install”, {})
    rc = pkg.run()
    if rc != os.EX_OK:
        exit_st = rc
        break

entropy_client.shutdown()
raise SystemExit(exit_st)

Some curiosities. For http{s,}://, ftp{s,}://, file://, Entropy uses Python urllib2 with custom User-Agent. For rsync:// Entropy uses the external rsync executable.
As some of you pointed out, exit statuses are a heritage of early Entropy codebase stages and are going to be replaced by exceptions in future. Entropy Client refactoring is going to happen in a few months. But for now, enjoy your hacking ;-).

In the next tutorial, before starting to show you the power of Entropy Services, I’ll continue my excursus through Entropy Client, focusing on metadata retrieval from repositories.

hello, twitter

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 584 other followers

del.icio.us

Follow

Get every new post delivered to your Inbox.

Join 584 other followers