Building your Pelican website with schroot

Lately I moved my blog to Pelican. I really like how simple and flexible it is. So in this post I'd like to highlight one particular aspect of my Pelican's workflow: how to setup a Debian-based environment to build your Pelican's website, and how to leverage Pelican's Makefile to transparently use this build environment. Overall, this post has more to do with the Debian tooling, and little with Pelican.

Introduction

First thing first, why would you setup a build environment for your project?

Imagine that you run Debian stable on your machine, then you want to build your website with a fancy theme, that requires the latest bleeding edge features from Pelican. But hey, in Debian stable you don't have these shiny new things, and the version of Pelican you need is only available in Debian unstable. How do you handle that? Will you start messing around with apt configuration and pinning, and try to install an unstable package in your stable system? Wrong, please stop.

Another scenario, the opposite: you run Debian unstable on your system. You have all the new shiny things, but sometimes an update of your system might break things. What if you update, and then can't build your website because of this or that? Will you wait a few days until another update comes and fixes everything? How many days before you can build your blog again? Or will you dive in the issues, and debug, which is nice and fun, but can also keep you busy all night, and is not exactly what you wanted to do in the first place, right?

So, for both of these issues, there's one simple answer: setup a build environment for your project. The most simple way is to use a chroot, which is roughly another filesystem hierarchy that you create and install somewhere, and in which you will run your build process. A more elaborate build environment is a container, which brings more isolation from the host system, and many more features, but for something as simple as building your website on your own machine, it's kind of overkill.

So that's what I want to detail here, I'll show you the way to setup and use a chroot. There are many tools for the job, and for example Pelican's official documentation recommends virtualenv, which is kind of the standard Python solution for that. However, I'm not too much of a Python developer, and I'm more familiar with the Debian tools, so I'll show you the Debian way instead.

Version-wise, it's 2019, we're talking about Pelican 4.x, if ever it matters.

Create the chroot

To create a basic, minimal Debian system, the usual command is debootstrap. Then in order to actually use this new system, we'll use schroot. So be sure to have these two packages installed on your machine.

sudo apt install debootstrap schroot

It seems that the standard location for chroots is /srv/chroot, so let's create our chroot there. It also seems that the traditional naming scheme for these chroots is something like SUITE-ARCH-APPLICATION, at least that's what other tools like sbuild do. While you're free to do whatever you want, in this tutorial we'll try to stick to the conventions.

Let's go and create a bookworm chroot:

SYSROOT=/srv/chroot/bookworm-amd64-pelican
sudo mkdir -p ${SYSROOT:?}
sudo debootstrap --variant=minbase bookworm ${SYSROOT:?}

And there we are, we just installed a minimal Debian system in $SYSROOT, how easy and neat is that! Just run a quick ls there, and see by yourself:

ls ${SYSROOT:?}

Now let's setup schroot to be able to use it. schroot will require a bit of a configuration file that tells it how to use this chroot. This is where things might get a bit complicated and cryptic for the newcomer.

So for now, stick with me, and create the schroot config file as follow:

cat << EOF | sudo tee /etc/schroot/chroot.d/bookworm-amd64-pelican.conf
[bookworm-amd64-pelican]
users=$LOGNAME
root-users=$LOGNAME
source-users=$LOGNAME
source-root-users=$LOGNAME
type=directory
union-type=overlay
directory=/srv/chroot/bookworm-amd64-pelican
EOF

Here, we tell schroot who can use this chroot ($LOGNAME, it's you), as normal user and root user. We also say where is the chroot directory located, and that we want an overlay, which means that the chroot will actually be read-only, and during operation a writable overlay will be stacked up on top of it, so that modifications are possible, but are not saved when you exit the chroot.

In our case, it makes sense because we have no intention to modify the build environment. The basic idea with a build environment is that it's identical for every build, we don't want anything to change, we hate surprises. So we make it read-only, but we also need a writable overlay on top of it, in case some process might want to write things in /var, for example. We don't care about these changes, so we're fine discarding this data after each build, when we leave the chroot.

And now, for the last step, let's install Pelican in our chroot:

schroot -c source:bookworm-amd64-pelican -u root -- \
  bash -c "apt update && apt install --yes make pelican && apt clean"

In this command, we log into the source chroot as root, and we install the two packages make and pelican. We also clean up after ourselves, to save a bit of space on the disk.

At this point, our chroot is ready to be used. If you're new to all of this, then read the next section, I'll try to explain a bit more how it works.

A quick introduction to schroot

In this part, let me try to explain a bit more how schroot works. If you're already acquainted, you can skip this part.

So now that the chroot is ready, let's experiment a bit. For example, you might want to start by listing the chroots available:

$ schroot -l
chroot:bookworm-amd64-pelican
source:bookworm-amd64-pelican

Interestingly, there are two of them... So, this is due to the overlay thing that I mentioned just above. Using the regular chroot (chroot:) gives you the read-only version, for daily use, while the source chroot (source:) allows you to make persistent modifications to the filesystem, for install and maintenance basically. In effect, the source chroot has no overlay mounted on top of it, and is writable.

So you can experiment some more. For example, to have a shell into your regular chroot, run:

$ schroot -c chroot:bookworm-amd64-pelican

Notice that the namespace (eg. chroot: or source:) is optional, if you omit it, schroot will be smart and choose the right namespace. So the command above is equivalent to:

$ schroot -c bookworm-amd64-pelican

Let's try to see the overlay thing in action. For example, once inside the chroot, you could create a file in some writable place of the filesystem.

(chroot)$ touch /var/tmp/this-is-an-empty-file
(chroot)$ ls /var/tmp
this-is-an-empty-file

Then log out with <Ctrl-D>, and log in again. Have a look in /var/tmp: the file is gone. The overlay in action.

Now, there's a bit more to that. If you look into the current directory, you will see that you're not within any isolated environment, you can still see all your files, for example:

(chroot)$ pwd
/home/arno/my-pelican-blog
(chroot)$ ls
content  Makefile  pelicanconf.py  ...

Not only are all your files available in the chroot, you can also create new files, delete existing ones, and so on. It doesn't even matter if you're inside or outside the chroot, and the reason is simple: by default, schroot will mount the /home directory inside the chroot, so that you can access all your files transparently. For more details, just type mount inside the chroot, and see what's listed.

So, this default of schroot is actually what makes it super convenient to use, because you don't have to bother about bind-mounting every directory you care about inside the chroot, which is actually quite annoying. Having /home directly available saves time, because what you want to isolate are the tools you need for the job (so basically /usr), but what you need is the data you work with (which is supposedly in /home). And schroot gives you just that, out of the box, without having to fiddle too much with the configuration.

If you're not familiar with chroots, containers, VMs, or more generally bind mounts, maybe it's still very confusing. But you'd better get used to it, as virtual environment are very standard in software development nowadays.

But anyway, let's get back to the topic. How to make use of this chroot to build our Pelican website?

Chroot usage with Pelican

Pelican provides two helpers to build and manage your project: one is a Makefile, and the other is a Python script called fabfile.py. As I said before, I'm not really a seasoned Pythonista, but it happens that I'm quite a fan of make, hence I will focus on the Makefile for this part.

So, here's how your daily blogging workflow might look like, now that everything is in place.

Open a first terminal, and edit your blog posts with your favorite editor:

$ nano content/bla-bla-bla.md

Then open a second terminal, enter the chroot, build your blog and serve it:

$ schroot -c bookworm-amd64-pelican
(chroot)$ make html
(chroot)$ make serve

And finally, open your web browser at http://localhost:8000 and enjoy yourself.

This is easy and neat, but guess what, we can even do better. Open the Makefile and have a look at the very first lines:

PY?=python3
PELICAN?=pelican

It turns out that the Pelican developers know how to write Makefiles, and they were kind enough to allow their users to easily override the default commands. In our case, it means that we can just replace these two lines with these ones:

PY?=schroot -c bookworm-amd64-pelican -- python3
PELICAN?=schroot -c bookworm-amd64-pelican -- pelican

And after these changes, we can now completely forget about the chroot, and simply type make html and make serve. The chroot invocation is now handled automatically in the Makefile. How neat!

Maintenance

So you might want to update your chroot from time to time, and you do that with apt, like for any Debian system. Remember the distinction between regular chroot and source chroot due to the overlay? If you want to actually modify your chroot, what you want is the source chroot. And here's the one-liner:

schroot -c source:$PROJECT -u root -- \
  bash -c "apt update && apt --yes dist-upgrade && apt clean"

If one day you stop using it, just delete the chroot directory, and the schroot configuration file:

sudo rm /etc/schroot/chroot.d/bookworm-amd64-pelican.conf
sudo rm -fr /srv/chroot/bookworm-amd64-pelican

And that's about it.

Last words

The general idea of keeping your build environments separated from your host environment is a very important one if you're a software developer, and especially if you're doing consulting and working on several projects at the same time. Installing all the build tools and dependencies directly on your system can work at the beginning, but it won't get you very far.

schroot is only one of the many tools that exist to address this, and I think it's Debian specific. Maybe you have never heard of it, as chroots in general are far from the container hype, even though they have some common use-cases. schroot has been around for a while, it works great, it's simple, flexible, what else? Just give it a try!

It's also well integrated with other Debian tools, for example you might use it through sbuild to build Debian packages (another daily task that is better done in a dedicated build environment), so I think it's a tool worth knowing if you're doing some Debian work.

That's about it, in the end it was mostly a post about schroot, hope you liked it.