Top-ix.org is supporting Sabayon

I am very excited to inform you guys, that the number of production servers powering the whole Sabayon Linux infrastructure has grown once more.

This time it’s all about the Top-ix Consortium which is going to host our main Entropy and ISO mirror very very soon (the migration is in process right now). The same Consortium kindly donated a powerful Intel Xeon Quad Core system with 4GB RAM, quite enough for the expected load. Thanks to this, we’ll be able to reinforce pkg.sabayon.org and distfiles.sabayon.org aiming to make our users happier.

So, thanks to Top-ix and their Development Program. But still, don’t forget to donate us, we need your support!

Splitting the Sabayon Overlay

After 2 months of hard work, Enlik and I (but mainly him) completed the split of the Sabayon Portage overlay. But what is it about?
It all started when Pacho Ramos kindly asked us to separate ebuilds that could be upstreamed (towards Gentoo) and ebuilds that only contained Sabayon business logic.
Since our overlay was quite fat, it really took us several weeks to accomplish the request… {omg here}

After the split, we ended up having: overlays/for-gentoo.git and overlays/sabayon.git git repos containing respectively the “sabayon” and “sabayon-distro” layman repositories.
So, layman -fa sabayon will now pull the for-gentoo.git overlay, which contains new cool stuff geared towards Gentoo users as well.
And, as said above, layman -fa sabayon-distro will give you the rest of our ebuild stuff.

Now it’s time to (slowly) “monetize” the split helping out upstream with patches and version bumps!

Eit: the stupid package tracker, reinvented

If you are maintaining either a Community Repository or a full blown one (inside a chroot, etc), please go ahead reading this post, important changes are going to happen very soon.

As the average FLOSS developer does, one morning a few months ago I woke up and decided that I needed to break stuff. Three months later, it turned out I fully achieved this goal, for your great joy.

Eit is going to ease your life as Entropy repository maintainer (and packages cook!) in a very dramatic way. Largely inspired to the “Git-way of doing things” and to its awesomeness, it represents a complete rewrite of the “Community Repositories/Entropy Server” functionalities exposes by “equo community”, “reagent” and “activator”.

It will be available starting from Entropy 1.0_rc60 and will completely replace the old tools and their syntax carrying out a lot of minor improvements as well.
Please note that Community Repositories maintainers, shall need to set “community-mode = enable” inside /etc/entropy/server.conf (see server.conf.example for more info) in order to make Entropy Server behave correctly.

How the syntax changed actually? Just a few examples below.

eit status [<repository>], shows repository status, unstaged packages, packages ready to push, etc.
eit commit [<repository>], commits unstaged packages to given repository (or the current one).
eit checkout <repository>, switches to the given Entropy repository.
eit add [--to <repository>] <package> [<package> ...], add the given unstaged packages to repository (current repository if none given).
eit push [<repository>] push staged changes to the remote repository.

This is just a subset of all the commands available, but they’re enough to start using Eit. “Top-level commands” are implemented as plugins to allow maximum modularity and 3rd-party extensions. Just subclass eit.commands.command.EitCommand and implement the respective methods.

Angelbird Wings x4, 32GB SSD donated

People at Angelbird and Cloudspecialists.net were kind enough to send us a Wings x4 PCI-Express SATA card with super-fast 32GB SSD memory onboard and space for 4 2.5″ SSD drives. Besides being a simple/cheapo SATA card, it seems to work great under Linux (which is worth mentioning), the controller is the Marvell 88SX7042 and supports up to 4 2.5″ SATA drives.
Mounting /var/tmp/portage on it is giving us a nice I/O boost.

Thanks Angelbird!

AMD Catalyst 11.9, complete disaster?

Let me explain my feelings a bit deeper. I do really appreciate the effort AMD is putting in providing high quality GPU drivers (in terms of performance and power consumption) for our beloved Operating System.

But this time I feel quite depressed. We were delaying Sabayon 7 just because we wanted to ship with working AMD drivers, and make GNOME3 users happy. Today, it turned out this is not going to be the case: x11-drivers/ati-drivers-11.9 seems to be a complete disaster, people still reporting (also been able to reproduce them) GNOME3 rendering issues (flickering), KWin and other random crashes (regressions!) as well. Yes, even KDE is crashy on fglrx now.

AMD is unlikely going to fix the situation in reasonable time (we must release Sabayon 7 asap!), so, like it or not, we are forced to revert back to 11.8 and ship our new ISO images with GNOME3 Shell not working on AMD GPU systems.

We are very sorry.

Entropy API Tutorial #4: inverse dependencies

Before moving on to other Entropy codebase areas, I need to also mention inverse dependencies. The same are used for calculating a removal schedule basing on currently installed packages. Let’s say you want to remove app-foo/bar, are there packages needing it? If so, are there packages needing packages needing it? And so on… If you know something about Algorithms, you can easily understand that the whole thing is just a topological sort out of a DAG (actually package dependencies unfortunately come with cycles, so, even if it wouldn’t make much sense to have dep A pulling dep B itself pulling dep A, this is what we have in real life, for reasons out of the scope of this blog post…).

But let’s keep things simple, especially if you’re new to these algorithmic concepts (scratch them!).
First of all we need to match a package name in the installed packages repository, then query Entropy Client asking for a removal schedule, and perhaps execute the actual removal.

import os
from entropy.client.interfaces import Client
entropy_client = Client()

installed_repository = entropy_client.installed_repository()
# let’s match app-foo/bar out of the installed packages repository
package_id, pkg_rc = installed_repository.atomMatch(“app-foo/bar”)
if pkg_rc != 0:
    print(“cannot find app-foo/bar”)
    entropy_client.shutdown()
    raise SystemExit(1)

# now ask Entropy Client to build up a sorted removal list
# in case of system packages, if system_packages keyword arg
# is True (see API documentation), DependenciesNotRemovable
# exception is raised. But we’re not going to discuss these
# details here for brevity.
packages_ids = entropy_client.get_removal_queue([package_id])

# now let’s remove them from the Live filesystem!
exit_st = os.EX_OK
for package_id in packages_ids:
    pkg = entropy_client.Package()
    pkg.prepare([package_id], “remove”)
    rc = pkg.run()
    pkg.kill()
    if rc != os.EX_OK:
        exit_st = rc
        break
entropy_client.shutdown()
raise SystemExit(exit_st)

That is pretty much it!

kernel.org abbreviated is k.o.

Not that I dislike it. I mean, I always thought that the release process was just a crazy mix of different flavours of madness. Especially with the stable repo, containing “facepalm”-like fixes (hey, this is software engineering, not rocket science — but still!).

After a full month, git.kernel.org (both http and git ports), http://www.kernel.org, are still down, and I’m starting to get really bored. Linus temporarily (permanently?) moved his git repo to github.com in order to keep 3.1 development going and this seems to have worked for a good cut of consumers. But how about the other 100+ repos that were sitting on git.kernel.org? They’re not even accessible in read-only.

Being the Linux Kernel a vital part of the whole Linux-based FLOSS world, how can we accept a 1-month downtime? Why isn’t there anybody out there starting to raise the voice? That is completely ridiculous. While I understand the QA and Security teams involved in bisecting the kernel.org logs etc, I cannot really accept that git.kernel.org is still down, for the following reasons:

  • git repositories themselves haven’t been harmed (as they said — not my own words), so git.k.o could be restored at least to a read-only state without being much scared
  • ssh keys could be revoked for everybody and their access limited to git pushing

Can’t wait to ditch my frustration seeing kernel.org back to life.

Sunday as first weekday, how can glibc be wrong?

It all happened after Sabayon bug 2699, and I am still laughing so hard.
Everybody in Italy knows at least two things: who is Silvio Berlusconi and what is the first weekday. I cannot guarantee on the rest, though.
But of course, glibc asserts that the second pillar of the Italian knowledge is just… wrong. For it and Ulrich Drepper the first weekday is Sunday and no matter what the fuck you think, say, write: as long as you don’t have the proof of it being Monday, you will live with the bug.

It seems that drepper has more power than our Prime Minister and his escorts all together when it’s about picking the first weekday! woot!

Entropy API Tutorial #3: installing packages

Basing on my previous Entropy API Tutorial #2 regarding how to calculate package dependencies through the Entropy Python API, let’s move forward and see how to actually install packages from a repository.

Let’s say we have a list called “package_matches” containing all the package matches we want to install (dependencies have been expanded already). At this point we’re ready to ask Entropy to fetch from mirrors and merge them into the live filesystem. The following example is very simple. For the same reason, it doesn’t cover all the functionalities, such as, for example, multi-fetching or fetching of package source code.

import os
from entropy.client.interfaces import Client

entropy_client = Client()
# this is out package_matches list (fake content)
package_matches = [(12345, 'sabayon-weekly'),
    (12343, 'sabayon-weekly')]

# first fetch, then merge
exit_st = os.EX_OK
for pkg_match in package_matches:
    pkg = entropy_client.Package()
    pkg.prepare(pkg_match, “fetch”, {})
    rc = pkg.run()
    if rc != os.EX_OK:
        exit_st = rc
        break
if exit_st != os.EX_OK:
    entropy_client.shutdown()
    raise SystemExit(exit_st)

# now merge into our live filesystem
for pkg_match in package_matches:
    pkg = entropy_client.Package()
    pkg.prepare(pkg_match, “install”, {})
    rc = pkg.run()
    if rc != os.EX_OK:
        exit_st = rc
        break

entropy_client.shutdown()
raise SystemExit(exit_st)

Some curiosities. For http{s,}://, ftp{s,}://, file://, Entropy uses Python urllib2 with custom User-Agent. For rsync:// Entropy uses the external rsync executable.
As some of you pointed out, exit statuses are a heritage of early Entropy codebase stages and are going to be replaced by exceptions in future. Entropy Client refactoring is going to happen in a few months. But for now, enjoy your hacking ;-).

In the next tutorial, before starting to show you the power of Entropy Services, I’ll continue my excursus through Entropy Client, focusing on metadata retrieval from repositories.

Yet another GNOME/GDM 3 idiocy: dropping ~/.dmrc support

I am so sorry I have to blog about GNOME3 once again. But this time, as distributor, I feel really upset.

As long as we had GNOME2 in repositories, we stuck to gdm-2.20, the last GDM release providing complete theming support and all the bells and whistles a complete Login Manager should have.

Unfortunately, we had to move to the somewhat incomplete thing GNOME3 is, and the same happened to GDM. No more xml-based theming support (unlike current KDM 4.7), no more fancy features but also, no more ~/.dmrc handling.

For those who don’t know what .dmrc is, in short, it’s a place where the default xsession can be declared. This is really important at provisioning/boot time, especially when you want to load non-standard xsessions (perhaps: XBMC, in our Media Center boot profile).

Funny enough, this is still the suggested method at GNOME. And this is a de-facto standard. Any DM out there supports ~/.dmrc.

But it turns out GNOME developers are smarter than any other in deciding that standards (even de-facto ones) just suck and are for noobs. In fact, they happily decided to “fuck the user is stupid” (yeah, it’s a verb) and implement a GNOME-centric dbus-based service (oh MY GOD!) called: accountsservice just to mangle with DM default user settings, like language and the default XSession. How can a GNOME-related tool not really considered by the rest of the world, be hosted at freedesktop.org is a mystery.

Anyway, you may think “ok, let’s use this thing then”. And here comes the even more fun part. There is no tool in gdm nor in accountsservice to do that. How comes? They moved a functionality somewhere else, dropped compatibility and killed its usefulness completely? Hello? Is there anybody at GNOME?

Creating a custom configuration file is not working, due to a bug in GDM/AccountsService, which always resets the default xsession at every boot. Doing the whole thing through dbus at boot time is just ridiculous and kills the performance so bad. Now you may wonder how distros tried to cope with it. The answer is right there in the open: GNOME Bugzilla. Oh, look, it’s yet another rotting bug from 2009.

WTF GNOME guys? Are you done breaking stuff? (don’t get me started with notification-daemon…). Sick of all this.

hello, twitter

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 680 other followers

del.icio.us

Follow

Get every new post delivered to your Inbox.

Join 680 other followers