House cleaning with Perl 6

2018-09-28 14:45:00

One of the standard features of content management systems is automatic scaling of images to the size required by the website layout. Our CMS does so on the first request for a certain size of a certain image. We have over 1400 customers and provide common content like tax related news. With so many shared images, having a centralized cache for scaled images becomes a really good idea and is easy to do using content addressing in the form of SHA hashes.

Of course there are probably as many cache implementations as there are CMS. Caches are simple beasts. You give them a key and they give you the data. Many implementations store the data in files in some directory structure which is appropriate for image data. Our use case is a bit special though, because we actually don't want the data. We would much rather have the file name to hand it off to the frontend webserver. That's why we implemented caching and fetching ourselves. But how do you keep the cache from growing indefinitely?

We just wrote a little daemon that watches the cache directory and limits its size. It's a simple "whenever a new file enters the cache, if the total size exceeds the configured limit, delete the least recently used files" algorithm. Reacting to events is inherently asynchronous and Perl 6 seemed the natural choice:

react {
    whenever $ -> $change {
        my $path = $change.path.IO;
        if $path.e { # could be deleted after event was generated
            my $size = $path.s;
            if $size { # could get notified before any data is written
                %!files{$change.path} =$size, :age(now));
    whenever signal(SIGINT) {

The actual program of course needs some more scaffolding like reading the existing cache entries on startup and the 4 line CacheEntry class, so it comes at a total of 68 lines of code. 5 of those lines are for reading the configuration of our Perl 5 based content management system:

use lib:from<Perl5> $*PROGRAM.parent.parent.add('lib').Str; 
use ZMS::Configuration:from<Perl5>;
my $config =;
my $cache_root = $config.get(|<Controller::Static cache_root>).IO;
my $cache_size = $config.get(|<Controller::Static cache_size>);$cache_root, :$cache_size).run;

Using a "best of both worlds" approach, we have Perl 5 code render our customers websites and Perl 6 code to clean up after it.

Production on Perl 6

2018-01-23 09:20:00

Half a life time ago, moritz++ asked me to blog about how we use Perl 6 in production after I mentioned that we went a very traditional way to introduce a Perl in a company. A Perl because almost all our production code is written in Perl 5. So where does Perl 6 enter the picture?

A little over a year ago we turned our single image servers into a kvm based virtualization cluster with one VM per service for the security benefit of well separated systems. The systems running in those VMs are configured by puppet but we also wanted the VMs themselves to be automatically created by puppet. Our sysadmins started writing bash scripts for that job but suffice to say, my programmer eyes could not stand the code. So I showed my admins some Perl 6 and it soon became apparent how well suited the language is for writing sysadminy scripts. A great example is my personal favourite feature: MAIN subs.

sub MAIN(Str :$vm-name!, Int :$ram!, Str :$disk-space!, Int :$vcpus!, Bool :$public)

With just this line we not only get automatic a fully automated usage text:

 create_vm.pl6 --vm-name=<Str> --ram=<Int> --disk-space=<Str> --vcpus=<Int> [--public]

but also argument parsing and validation. Did you notice that it expects just a number for RAM while disk space accepts a string? No need to guess which argument expects a unit suffix and which doesn't. Of course, tasks processing our DHCP configuration to find the highest unused IP address in the server range have always been home territory for Perl and Perl 6 is no exception.

Whenever we need to host a new service, a Perl 6 script on the admin's machine creates the configuration and puppet will run another Perl 6 script to actually provision storage space, create the drbd device and finally copy our master image and prepare it for its first run. So it's fair to say that Perl 6 is an essential part at the very heart of our operations.

Just as Perl 5 set out to ease the life of system administrators, Perl 6 is your friend when you need it. It not only helps with technical challenges of automation, but also with the social side by giving your users help, preventing mistakes and by helping to keep your scripts maintainable.

Perl Toolchain Summit 2017

2017-05-15 11:24:00

This is my first blog post, ever. I've had interesting things to talk about before, but they never seemed important enough to actually decide on where to put my blog.

I'm currently on my way home from the Perl Toolchain Summit in Lyon, France. Looking at the beautiful alps through the aircraft's window, I can't help but feel highly enthused. Though the view is probably only responsible for a small part of this feeling. I've been part of this unique conference before, two years ago, when it was still called Perl QA hackathon. Back then I got my invitation for the virtue of having written Inline::Perl5 which is the bridge between Perl 6 and Perl 5 code. Ever since I started writing that I've made it my mission to connect the Perl 5 and Perl 6 worlds, which over the long years of Perl 6's development have become more and more separated.

Back then, two years ago, though it felt like Perl 6 people were highly welcomed at the hackathon, my attempts at bringing Perl 6 into discussions about the Perl toolchain have not exactly been met with enthusiastic responses. I did have very interesting and even productive chats with individuals like ANDK, but it was quite clear that the toolchainers as a group were concerned with Perl 5 issues. This is quite understandable as Perl 6 as a language had not seen its first stable release back then and for people outside the core team it was probably hard to believe that it ever would be.

This year the vibe was completely different. I came to the summit with a very concrete TODO list focused on making Perl 6 the language with the easiest to package module ecosystem. What I wouldn't have dreamed of was Perl 5 toolchain people approaching us with questions on what they could do to integrate Perl 6 into services like CPAN Testers. At the very first daily standup, lizmat expressed an urgent desire to have Perl 6 modules hosted on the CPAN infrastructure. This was followed by an, I dare say, unprecedented collaboration between Perl 5 and Perl 6 people on resolving the remaining issues that stood in the way of a smooth transition.

To avoid any misunderstanding, there had been done a metric ton of work before that by both 5ers and 6ers to make this possible. It took years to get all the many involved pieces into place to get us to this point. What I felt was very different this year was that when lizmat told us her wish, it was like it's the most natural thing in the world that we simply get it done once and for all. There was neither the slightest hesitation nor worry. ANDK, who is known for his tireless and amazing job at keeping PAUSE - the underlying pillar of the CPAN ecosystem - stable and running, even took the risk of having to make emergency fixes rather than missing the opportunity of getting the job done this weekend.

Garu worked countless hours with lizmat and ugexe on getting reports on Perl 6 modules into CPAN Testers. Hearing of our plan to work on a declarative build system and native dependencies, several people came to us with advice, examples of prior art and maybe helpful code. I think it's fair to say that this summit took collaboration to a whole new level. For this and lots of other things, I can't help but feel deeply grateful. This is as it should be. It hasn't always been easy in the years before, but what family doesn't have its problems? Surely the important bit is, that we are back together. We talk, we collaborate, we try to help each other.  On that note, I do hope, that the work we've done on handling native dependencies will be useful not only for Perl 6 installers and packagers, but will also somehow make it back into the Perl 5 toolchain. It feels like I owe the awesome people at the summit and elsewhere at least a good and honest attempt.

With a short night behind me and a descent towards Vienna in front of me, I'm gonna end now, knowing, that I am part of this truly epic family. Can there be a better start into a new week?


Of course, my thanks go out to the sponsors of the Perl Toolchain Summit, who shall take a lot of credit for ensuring that the Perl community has a bright future:, ActiveState, Atikon, cPanel, FastMail, MaxMind, Perl Careers, MongoDB, SureVoIP, Campus Explorer, Bytemark, CAPSiDE, Charlie Gonzalez, Elastic, OpusVL, Perl Services, Procura, XS4ALL, Oetiker+Partner.

Managed by CiderCMS