Raku Performance and Physics::Unit

I have been able to spend some time on the Physics::Unit module over the holidays and to expunge some of the frustrations that have crept in regarding the compile times of raku.

The basic problem I have been wrestling with is the desire to express physical SI units using the raku custom postfix operator mechanism without having to wait for 10 mins for raku to compile my module.

This is the story of how judicious design, lazy execution, trial & error and the raku power tools got from 10 mins+ to under 13 secs!

Let’s start by looking at the sunlit uplands. Imagine a raku which provides a simple and intuitive tool for scientists and educators to perform calculations that automatically figure out what the physical units are doing.

Something like this:

use Physics::Constants;
use Physics::Measure :ALL;

$Physics::Measure::round-to = 0.01;

my \λ = 2.5nm; 
my \ν = c / λ;  
my \Ep = ℎ * ν;  

say "Wavelength of photon (λ) is " ~λ;              #2.5 nm
say "Frequency of photon (ν) is " ~ν.norm;          #119.92 petahertz 
say "Energy of photon (Ep) is " ~Ep.norm;           #79.46 attojoule

Now, before you ask, yes – this is real code that works and compiles in a few seconds. It uses the latest version Physics::Measure module which in turn uses the Physics::Units module. Let me point out a few cool things about how raku’s unique combination of features is helping out:

  • unicode to keep familiar symbols such as λ (lambda) and ν (nu)
  • variable names without the $ sigil to keep equations clean
  • Physics::Constants – for c (speed of light) and ℎ (Planck’s constant)
  • Physics::Measure :ALL to imports all the SI unit postfix operators
  • postfix:<nm> to do Measure.new( value => 2.5, unit => ‘nanometre’ )
  • does Measure math with custom ‘/’ and ‘*’ operators
  • knows that a Frequency class type takes SI unit hertz
  • knows that an Energy class type takes SI unit joule
  • can normalise a Measure object and round the output

So – how can it be that hard? Well the devil is in the little word all [the SI unit postfix operators]. Consider this table:

So we have 27 units and 20 prefixes – that’s, err, 540 combinations. And you want me to import all of these into my namespace. And you want me to have a library of 540 Physics::Unit types that get loaded when I use the postfix. Have you thought this through!!??

So – by way of sharing the pain of this part of my raku Physics::Journey – here are the lessons I have learned to optimise my module code:

Attempt 1 – Ignore It

My first instinct was to punt on the issue. The original perl5 Physics::Unit module allows coders to specify a unit type via a string expression – something like this:

my $u2 = GetUnit( 'kg m^2 / s^2' );

Anyway I knew I would need unit expressions to cope with textual variants such as ‘miles per hour’ or ‘mph’, or ‘m/s’, ‘ms^-1’, ‘m.s-1’ (the SI derived unit representation) or ‘m⋅s⁻¹’ (the SI recommended string representation, with superscript powers). So a new unit expression parser was built into Physics::Unit from the start with raku Grammars. However, it became apparent that saying:

my $l = Length.new( value => 42, units => 'yards' );

Is a pretty long-winded way to enter each measurement. Still, this was a cool way to apply (and for me to learn) raku Grammars and Actions which has resulted in a flexible, human-friendly unit expression slang as a built-in piece of the Physics::Unit toolkit.

Attempt 2 – Working but Dog Slow

So far, my Physics::Unit module would happily take a unit string, parse it with the UnitGrammar and create a suitable instance of a Unit object. Something like this:

Unit.new( factor => 0.00016631, offset => 0, 
    defn => 'furlong / fortnight', 
    type => Speed, dims => [1,0,-1,0,0,0,0,0], 
    dmix => ("fortnight"=>-1,"furlong"=>1).MixHash, names => ['ff'] );

This user-defined object is generated by iterating over it’s roots (e.g.) 1 fortnight => 2 weeks => 14 days => 336 hours => 2,016 mins => 120,960 secs (thus the factor attribute). More than 270 built in unit and prefix definitions – covering SI, US (feet, inches), Imperial (pints, gallons) and so on. And the .in() method is used for conversions. [There does not seem much point in a Unit library unless it can support common usage such as mph and conversion between this and the formal SI units.]

But, now I come to implement my postfix operators – then I need to pass 540 definitions to the Grammar on first compilation and it needs to build 540 object instances. Welcome to 10 mins+ compile times.

Before I go too far with this critique – I would like to note a couple of very important caveats:

  1. “So finally, we have an actual Perl 6 that can compete with all those languages too, at least in terms of features. One of the ways it does not yet compete with Perl 5 is in the area of performance, but the first half of this year we’ve already got it running nearly twice as fast as it was at Christmas. We’ve still got a lot of headroom for optimization. But Perl 6 has its eye on surpassing all those other languages eventually. Perl has always been about raising the bar, then raising it again, and again. ” Larry Wall on Slashdot in 2016 … and optimisations and enhancements are coming all the time.
  2. Raku recompilation is a very big speed multiplier – even with 30 min compile times, the precompiled binary loaded and ran in about 12 seconds.
  3. Personally I echo the view that raku is a very high level language and that the ensuing programmer productivity benefits in precision and expression outweigh a few seconds of compile time. I am confident that the hardware will continue to improve and will soon eliminate any noticeable delay – for example the recent Apple M1 launch.

Attempt 3 – Stock Unit Blind Alley

Sooo – the third attempt to combine the desired features and reasonable speed was to pre-generate the 540 units as literals – “Stock” units. So the code could be run in “dump” mode to generate the unit literals using the Grammar and store to a text file, then paste them back into the module source so that in the released version they are just read in via a “fast start” mode.

By reusing the same Grammar for pre-generation and on the fly generation of user-defined Units, this eliminated any potential compatibility issues. I chose not to compromise with any of the MONKEY-SEE-NO-EVAL for module security and code integrity reasons.

Performance improvements were dramatic. By bypassing the Grammar step, first compile times came down to ~340s and precompile start times to under 3s. Nevertheless, I was still unhappy to release a module with such a slow first compile time and searched for a better design.

Attempt 4 – Lazy Instantiation

On the fourth pass, I decided to re-factor the module with lazy Unit instantiation. Thus the unit definitions are initialised as hash data maps, but only a handful of objects are actually created (the base units and prefixes).

Then, when a new object is called for, it is generated “lazily” on demand. Even in an extreme case such as the ‘furlong / fortnight’ example, only O(10) objects are created.

By eliminating the Stock units, this reduced the module source by 2000+ lines (540 x 4 lines per object literal). Performance improved substantially again – this time about 60s first compile and 2.6s precomp start times.

However, the Physics::Measure code still had to embody the postfix operators and to export them to the user program. Thus 540 lines to individually compile. Each postfix declaration like this:

sub postfix:<m> ( Real:D $x ) is export { do-postfix( $x, 'm' ) }

Attempt 5 – UNIT::EXPORT package

Even better, I could learn from the excellent raku documentation again – this time to discover UNIT::EXPORT that gave me a good start to produce a programmatic export all 540 postfixes in just 6 lines of code. Goodbye boilerplate!

my package EXPORT::ALL {
  for %affix-by-name.keys -> $u {
    OUR::{'&postfix:<' ~ $u ~ '>'} := 
                    sub (Real:D $x) { do-postfix($x,"$u") };
  }   
}

This had the additional performance boost – the final numbers below…

Attempt 6 – Selective Import

Finally, a word on selective import. Prior to Attempt 5, I experimented with labelling the less common units (:DEFAULT, :electrical, :mechanical, :universal and :ALL if you are interested). But even by reducing :DEFAULT to only 25% of the total, this did not reduce the first compile time measurably. I interpret this as the compiler needing to process all the lines even if the import labels are not specified by the user program.

But with the package approach, Physics::Measure :ALL will export all of the SI units. Just drop the :ALL label if you want even more speed and plan to go without the postfix operators.

Final Outcome

So, the latest speed measurements (on my low spec 1.2GHz/8GB laptop) are:

# use Physics::Measure;      ...10s first-, 1.2s pre- compiled
# use Physics::Measure :ALL; ...13s first-, 2.8s pre- compiled

YMMV!

~p6steve (pronounced p-sics)

Published by librasteve