raku at Monterey Docks (part II)

Title: Building on fabled Cannery Row, Monterey, California – Wikimedia Commons

Recap

In Part I of this post, I started on my macOS tool chain rebuild, I want this to work for all my languages (Python, Perl, JavaScript, Raku and so on including specific compiler build versions and combinations such as Inline::Perl5) … coming off the disappointing Apple induced cliff edge for M1 machines upgrading from Big Sur to Monterey OS versions.

In that thrilling episode, I turned the corner and was starting to make progress – seeing how I can get the speed benefits of M1 and apply them to my raku projects:

  • Path 1: Basic Module Test with JJ’s excellent raku-test on Docker Desktop
  • Path 2: Run any Raku script with JJ’s excellent alpine-raku on Docker Desktop
  • Path 3: Run my existing jupyter notebook projects based on Brian Duggan’s p6-jupyter-kernel Dockerfile on Docker Desktop
  • Get all of this working from macOS zsh and Comma IDE Docker plugin

BUT – as mentioned and commented, this way I have to install all my module dependencies on every test run … surely there must be a better, faster way… 

Context

Now before I dig in, I must say that I was feeling a bit lonely out here with my opinion that macOS Monterey is a sign that Apple is defocusing on the wider developer community …. then I found this comment that cheered my up a bit:

smoldesu 5 months ago | root | parent | prev | next [–]

It was impressive back when OSX debuted, but now it feels like Apple is lacking in both regards. Their shell utilities are all horribly outdated/mismatched, and their UI design took a nosedive with Big Sur. Modern MacOS reminds me of what Ubuntu felt like 10 years ago: it’s a confused and scared operating system that doesn’t quite know where to go from here. I mean, look at what Monterrey introduced; basically better Facetime and some new wallpapers. I’m getting the feeling that Apple has painted themselves into a corner here. They spent the past 2 decades focusing on vertical growth, only to discover that the next 50 years are going to be ruled by better interop and protocols.

https://news.ycombinator.com/item?id=27537040

And, to be fair, I think that good support for VMs is probably a fork in the road for all desktop OS providers … then (spoiler alert) a decent VM side-load means that the general (ie. non Apple) developer can have complete control and Apple can build their wall higher and need to pander to fewer requirements (goodbye perl 5 on Mac?)

Vector One – Docker Desktop & AWS Ubuntu

False Steps

First, I naively decided to add a couple of lines to the raku-test Dockerfile cloned from JJ’s repo:

RUN zef install XML::Writer SVG SVG::Plot

and then rebuild from the Dockerfile pwd with:

docker build -t new-test .

But, this did not work via the zsh CLI provided by Docker Desktop on macOS:

  • Docker Desktop on macOS (either Intel or M1)
  • ubuntu:latest on Docker Desktop on macOS (--platform=ARM64)

Then, I took out my changes and just tried to clone and run an unchanged alpine-raku:

  • docker build -t alpine-raku . on M1 (Version moar 2021.09 is broken.) [Intel]
  • docker build -t alpine-raku ./2021.04 on M1 () [Intel]
  • docker build -t alpine-raku ./2021.04 on M1 (Version moar-2021.04 is broken)

Hmmm, I dug a bit deeper into the CI commands in JJ’s github repo and realised that he was building these variants of alpine-raku on ubuntu …. so while alpine-raku is a fantastic lightweight container image for test, right now alpine-raku can’t be build on Docker Desktop on macOS Monterey M1. (Mumble mumble bloody bleeding edge.)

SO – I needed to get a clean ubuntu build environment – with a workflow that goes:

  • Clone and build my alpine-raku variants on a pure ubuntu server
  • Push to a docker repo online
  • Pull and run on macOS Docker Desktop

Rent a Build

Right now I am on the free 5 day trial of the subscription based “Run Docker on Ubuntu 20.04 with latest Ubuntu Image” by Code Creator (just search for this AMI in the AWS marketplace)

  1. first blind alley was to try and build rakubrew from scratch (ie. run the base alpine-raku Dockerfile) as this would fail to build with make/echo errors
  2. then I realised that I can just use the derivative raku-test Dockerfile and embed the additional zef install lines in here – BINGO!

Now all I have to do is prebuild the variants of zef installs that I need, tag these new Docker images and push to the Docker image repository.

Here’s an example Dockerfile (see each module eg. Physics::Unit ):

FROM jjmerelo/alpine-raku:latest
LABEL version="6.0.2" maintainer="JJ Merelo <jjmerelo@GMail.com>"

ARG DIR="/test"
USER root

# Add raku-physics-unit dependencies
RUN zef install SVG::Plot

# Set up dirs
RUN mkdir $DIR && chown raku $DIR
COPY --chown=raku test.sh /home/raku
VOLUME $DIR
WORKDIR $DIR

# Change to non-privileged user
USER raku

# Will run this
ENTRYPOINT ["/home/raku/test.sh"]

Here’s my CLI incantation to do the build:

git clone https://github.com/p6steve/raku-Physics-Unit.git

sudo docker build -t arp-unit-deps .
sudo docker run -t -v /home/ubuntu/raku-Physics-Unit:/test arp-unit-deps --verbose
sudo docker tag arp-unit-deps p6steve/alpine-raku-physics:arp-unit-deps
sudo docker push p6steve/alpine-raku-physics:arp-unit-deps

And to run it on the macOS Docker Desktop client:

docker pull p6steve/alpine-raku-physics:arp-unit-deps
docker run -t -v //path-to/raku-Physics-Measure:/test p6steve/alpine-raku-physics:arp-unit-deps --verbose

Here’s the list of images that I built – in each case, the image is built with the module dependencies since it is used for faster testing of the module…

  • raku-Physics-Units (ie Physics-Units deps)
  • raku-Physics-Measure (ie Physics-Measure deps)
  • raku-Physics-Constants (ie Physics-Constants deps)
  • raku-Physics (full install for downstream code)

These are all now built and pushed to https://hub.docker.com/r/p6steve/alpine-raku-physics – enjoy!

Vector Two: Dump Docker Desktop

So – this has proven the point and determined a general need for a clean ubuntu build environment for adjustments to my images, I felt that I was limited by going to AWS and a paid build for this step and still felt unnecessarily hemmed in and slowed down in by the macOS Docker Desktop confection.

A bit more research turned up vftools … and this excellent recipe from Jan Mikeš… take a look to see how Jan describes 40x load speed improvements (from 160s to 3.7s).

Here’s what it has provided for me (ymmv):

  • VM in ~/ubuntu-vm
  • Docker installed on VM
  • VM hostname is ubuntu
  • You can ssh root@ubuntu without password
  • You can ssh ubuntu without password
  • Global start-ubuntu command on mac
  • Backup at ~/ubuntu-vm/disk-backup.img.tar.gz

So now I can start my ubuntu-vm with docker in the morning and reach for something like:

docker pull --platform linux/arm64 rakudo-star
docker run -it --entrypoint sh -l -c rakudo-star
raku

Line 2 of this takes about 1s to get a raku REPL prompt. Yessssss!

So now I am off to scratch my head with:

  • What are the docker images I want to compose and build for my dev needs
  • How to adjust Comma IDE to run / ssh / utilise this CLI

Please do leave any feedback and comments (click here & scroll down) …

~p6steve

Published by librasteve