Amazon EC2, kernels and S3-backed images

Some time ago I started playing with Amazon EC2 (I love working at lightstreamer.com) trying to get Sabayon running in there as part of the process of learning how to make EC2-ready images. It ended up being relatively easy and the changes required have been integrated into our beloved distribution and upstreamed to Gentoo (genkernel.git), even thanks to Rich and his guidance through the platform via G+.

So, for S3-backed AMIs (the Amazon Machine Images), what is required is a Xen domU kernel inside a simple ext3/4 (or another filesystem supported by Amazon) filesystem image. Sounds easy isn’t it?

Since I wanted to provide pre-made kernel binaries through Entropy, the very first thing to do was to make genkernel able to compile Xen-based kernels. Due to build system interface changes, you cannot just “make bzImage”, the same target doesn’t actually exist. So, I jumped in and fixed genkernel in a way that is possible to override the build target for the kernel image through –kernel-target=.

Once fixed that, I moved to kernel sources and binaries ebuilds, and produced: sys-kernel/xen-dom0-sources, sys-kernel/xen-domU-sources, sys-kernel/linux-xen-dum0, sys-kernel/linux-xen-domU. Respectively Xen dom0/domU kernel sources (provided with kernel configuration that works out of the box on Amazon EC2, of course just the domU one) and dom0/domU kernel binaries. The same are available on our Portage overlay in ebuild form (for Gentoo users woot) and on Sabayon Entropy repositories as install-and-go packages.

The final step was to write a molecule for Molecule.
Molecule is an image build tool I wrote in 2009 (sick of dev-util/catalyst!), that makes possible to build different kinds of images (ISO, tarballs, filesystem images, etc — the whole interface is pluggable) through a specification file (the molecule). I am proud to say that Molecule has played a key role in latest Sabayon success by making the problem of building ISO images as easy as spawning a command. Back on topic, that’s what I did here and here.

I haven’t hooked the EC2 molecules into our automated nightly image build system yet. But if you are interested in cooking one, the procedure is pretty simple. Just clone molecules.git into /sabayon, install dev-util/molecule (it’s in Portage and Entropy), download “Sabayon_Linux_SpinBase_DAILY_amd64.iso” from one of our mirrors and place it into /sabayon/iso, and type (for amd64):

# molecule /sabayon/molecules/sabayon-amd64-spinbase-amazon-ami-template.spec

This will generate an S3-backed AMI for Amazon Ec2. You will have to pick the right kernel AKI (just take the one for x86_64 and S3 and you’re set) and use the EC2 AMI tools for uploading it into Amazon S3.

If you browse through all the available molecules, you will realize that you could virtually build all the Sabayon images yourself like we do on our ISO build server housed at unimib.it. Ain’t that something?

About lxnay

the creator of Sabayon Linux, Entropy Package Manager {Eit, Equo, Rigo}, Molecule release media buildsystem, Matter Portage buildbot/tinderbox and only God knows what else...

2 responses to “Amazon EC2, kernels and S3-backed images

  1. gluni

    This is probably not the right place for this kind of suggestion, but since you where talking about the Molecule features:
    Why not add a functionality à la “remastersys” from ubuntu, that allows to make a snapshot of a fully configured system (including home and all settings and installed packages) and then makes a bootable iso out of this? So far only remastersys of ubuntu and the linux-live scripts of Salix have this feature. Maybe this would complete Moelcule even more.

    I know im gone get a beating for posting this here, but then, whats another beating among the many ones i got so far…

  2. It should not require much work, since there is already a plugin doing “chroot to iso”. We’ll see in future😉

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

hello, twitter

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 585 other followers

del.icio.us

%d bloggers like this: