After some excellent
feedback
my previous post on coordinating multiple
grammars in Raku, I realized that I'd had fundamentally the wrong mental model of regexes in
Raku(do). This flawed model hadn't stopped me from using regexes, but was nevertheless wrong –
and it prevented me from correctly grasping some of the more complex behaviors involved in
grammars. Now that I've corrected that misconception, everything makes so much more sense!
In this post, I'm going to briefly present the flawed view I had (hopefully in enough of an outline
that you'll understand where I was coming from but not be tempted into the same confusion) and then
present the correct (or at least, less wrong) mental model. Exploring this model will lead us to a
significantly deeper understanding of how Raku's regexes work under the hood (or, at least, deeper
than I had a week ago; YMMV, as they say). Next, I'll explain how this new understanding allowed me
to build a trait – on that I believe will make composing multiple grammars much easier. And,
finally, I'll quickly walk you through the ~100 lines of code involved in implementing that trait.
Raku's grammars aren't always the right tool for the job
– but when they are, they're so powerful that they feel almost like cheating. And one of the
not-so-secret weapons that gives them that power is the ability to specify an action
object to specify parse-time behavior. (Methods on this
object are called whenever a token with the same name in the grammar matches, which lets you
manipulate the syntax tree as you build it and avoids having to navigate a (potentially very deep!)
match tree after parsing is complete.)
Over the past few weeks, there's been a fair
bit
of
discussion
about different ways to combine
multiple grammars. Most notably, Mike Clark
had an excellent blog post titled Multiple Co-operating Grammars in
Raku. In that post, Mike showed
two ways to combine two grammars – a simple way for times when you're fine with the grammars sharing
an action object, and a more complex way to let each grammar have its own actions. And then,
building on that post, Matthew Stuckwisch (guifa) opened a Problem Solving
issue discussing different ways Raku could
one day add syntax that better supports combining grammars (maybe in version 6.f).
Now, both of those posts were very interesting. But, being lazy, I wasn't too sure about Mike's
second approach
– it seemed like a lot of work, and a lot of bookkeeping. And, being impatient, I also wasn't too
keen on waiting for a syntax-level solution in v6.f. After all, we haven't even finalized a
release date for v6.e!
So I started thinking to myself, "self, would there be a way to do this with roles somehow?". And,
after a bit of playing around, I replied in the affirmative. The rest of this post shows an example
of using a Subgram role to easily combine multiple grammars, each with its own action object.
Brains sure are funny things. Or at least mine is; maybe I shouldn't speak for the rest of you.
Last night, I posted a few thoughts on pattern matching in Raku. A bit later, I saw a reply to that post suggesting that it'd be nice to add better pattern matching syntax with a Slang. I responded by saying that it probably wouldn't require a Slang, but that I'm not typically a fan of changing basic syntax purely for convenience (when Raku already gives us so much).
I didn't give the matter much more thought, at least consciously. But my subconscious mind must have been noodling around with the idea because, somehow, I woke up this morning absolutely convinced of three things.
A module should add support for better pattern matching.
It could do so with a regular sub, not a Slang or term.
Writing that sub would be easy.
And, for once at least, that all turned out to be true. Here's the function I came up with:
Raku has extremely strong support for pattern matching in function/method signatures – you can match on literals, types, names, or pretty much anything at all and can conveniently destructure the value you're matching on into a set of variables that fit your needs.
But Raku also has a second type of pattern matching (or at least something very much like pattern matching): the ~~smartmatch operator powered by the .ACCEPTS method. This form of matching is also very convenient; is has a slightly different use case from matching on a signature, but it's no less powerful on the whole. And, when it fits, it can be an even better/more lightweight solution to the same set of problems. In fact, I'd bet that when (which is powered by this sort of matching) is one of the keywords that shows up most often in my Raku code.
The problem
Since these two forms of pattern matching are different, there are some problems that are easier to solve with signature matching and others that are easier to solve with smartmatching. Fortunately, Raku makes it very easy to add smartmatching into a signature – you can easily smartmatch in a where clause, for example.
I've typically found going the other direction a bit more cumbersome, however. Consider the following code:
Lisp is famous for having pretty much no syntax. Structure and Interpretation of Computer Programs – arguably the most well known intro to programming in Lisp – presents pretty much the entirety of the syntax in its first hour. And that's by no means the only thing SICP does in that hour.
Raku, on the other hand, has a bit more syntax.
Ok, that's an understatement. Raku is syntactically maximalist to exactly the same degree that Lisps are syntactically minimalist. Forget “syntax that fits on a postcard”; Raku's syntax struggles to fit on an A4 sheet of paper. Raku has the type of syntactic riches that inspire Rakoons to classify its operators into beautiful (though now sadly dated) Periodic Tables.
When organizing a program of any size, you'll obviously need to break your code up into smaller chunks. Often, it makes sense for these chunks to be factored out into their own functions – especially if they're good candidates for reuse. But it's possible to take this urge to factor out too far and it's often a better call to leave the code inline. Doing so makes it clear that the code isn't being reused anywhere else, and keeps your program's control flow more linear.
Indeed, John Carmack wrote an influential post a few years ago describing how he has shifted from a coding style that looked a lot like
I'm not trying to argue that this style is always better, but it is a good option to have.
When organizing my code like that, though, I don't tend to use // MinorFunction1 comments. Instead, I'm more likely to write a Label, so it'd look more like minor_function:. You might benefit from using labels too, if your language includes them – and if it's at all a C/Algol family language, it probably does. Labels are supported in at least JavaScript, C/C++, Perl, golang, and my own language of choice, Raku.
As I write this, I've now had my Keyboardio Atreus for a full six weeks and have been
using it pretty much exclusively that whole time. After a month and a half, I've given it
a fair shake and can give you my full impressions. And my full impression is that it is
amazing.
Atreus background
Before I tell you what I love about it, what is the Atreus? Well, it's a keyboard. It's
a tiny keyboard.
More specifically, it's a keyboard with 44 keys – compared with 84 on my ThinkPad
keyboard, and a full 108 on a traditional desktop keyboard. How does it get by with so
few keys? More on that in a minute; for now, just focus on how small the thing is,
without needing to cramp any of the individual keys in the least.
In part 1 and part 2 I discussed my
personal take on a Raku manifesto:
Expressive code over uniform code
Rewarding mastery over ease of learnability
Powerful code over unsurprising code
Individual productivity over large-group productivity
In those posts, I explained how Raku prioritizes each of the values on the left over each of
the values on the right. I also explained that, in my view, the final value pair –
prioritizing individual productivity over large-group productivity – is the most fundamental
value driving Raku's design. All the other sets of priorities in my Raku manifesto play a
supporting role in prioritizing individual productivity over large-group productivity. In
this post, I explain why I'm convinced that Raku made the right call.
Why Raku is correct to prioritize individual productivity
I mentioned earlier that language designers have recognized the tension between individual
productivity and large group productivity. If you look at what has been written on the
topic – all the way from the original Programming-in-the-Large Versus
Programming-in-the-Small
paper through today – you might get the impression that small scale programming is a solved
problem and that the only interesting/meaningful question is how we increase the
productivity of large software teams. Certainly many of the programming languages that have
gained popularity recently seem to reflect that view:
Language
Developer
Purpose
Golang
Google
build large software projects
Rust
Mozilla
build large software projects
TypeScript
Microsoft
build large software projects
Hask
Facebook
build large software projects
Looking at that list, you might conclude that we clearly need a programming language for
large projects and that older languages do fine for smaller groups.
Individual productivity over large-group productivity
In part 1, I discussed how and why I believe that Raku values expressive code over uniform
code and rewarding mastery over providing a fast learning curve. Now,
we're ready to move on to the third value pair.
Powerful code over unsurprising code
The principle of least
astonishment is a
fundamental maxim of computer science that states code is far clearer when its behavior can
be easily predicted; code is far more head-scratchingly confusing when it triggers spooky
action at a
distance.
Rust provides a particularly clear example of this principle in action: Rust doesn't let you
override existing operators for existing types; it doesn't let you declare a function
without fully defining the types; it doesn't let you have default arguments; it doesn't let
you call a macro without using the ! character that sets that call apart from function
calls. Rust is powerful enough that it could let you break any of these rules, but it
doesn't – enforcing those rules keeps the language less surprising, and thus makes it much
easier to reason about.
And Rust is hardly alone in this regard. Indeed, the majority of programming languages
don't allow operator overloading and
even fewer allow the programmer to define new operators. Put differently, most languages
are willing to deny programmers the considerable power of user-defined operators to keep the
language less surprising.
How Raku values unsurprising code
I've already mentioned one way Raku helps prevent nasty surprises: eliminating the
action-at-a-distance variables that were so sinful in Perl 5. Raku has also thoroughly
embraced lexical scoping; even when you modify Raku's very syntax, your changes will be
limited to your current lexical block. And, as I mentioned in part
1, Raku strongly supports both object-oriented and functional
programming – each of which, in its own way, promotes predictable code.
Individual productivity over large-group productivity
I'm not so sure that I'm a fan of agile software development (at least as it's commonly
practiced). But I am sure that the Agile Manifesto got one
thing really right: It not only stated what values were important but also stated which
values (though important!) could be sacrificed. That is, the Agile Manifesto was explicit
about what tradeoffs it was willing to make.
That's crucial because it's really easy to select some nice-sounding phrases and label
them as "priorities" – who wouldn't agree that "readability
counts"? But it's much harder to pick out
important values that you're willing to sacrifice. And, because it's harder, it's also
much more revealing. You'll often learn a lot more from knowing a project's non-goals
than from knowing its goals.
In that spirit, I'd like to present a similar manifesto for the Raku programming
language. Note: I said "a" manifesto, not "the" manifesto for
Raku. I'm speaking only for myself; I'm sure many others would include different
dichotomies on their version of a Raku manifesto. Additionally, this is very much a first
draft. I've put considerable thought into this but haven't yet discussed my views broadly.
I also artificially limited myself to four pairs of values (following the form set by the
Agile Manifesto), and I could easily imagine changing my mind. If you disagree or just have
a different perspective, I'd love to hear your thoughts on the
r/rakulang
thread for this post on the #raku IRC channel.
Before we get started, I want to remind you of a statement from the Agile Manifesto:
That is, while there is value in the items on the right, we value the items on the left more.
I feel the same way. All the items on the right side of the chart up above (the items that
come after "over") are still really important. So, in this series of posts, I'm going to
walk through the four value pairs in my Raku manifesto. I'll discuss the first two pairs in
this post, and the next two in the second post. For each pair, I'll say why the value on
the right side is important, and what Raku does to support it. Then I'll say why, in my
view, the value on the left is even more important (for the language Raku is trying to
be, anyway), and how Raku prioritizes it in ways that require sacrificing the value on the
right. After discussing all four value pairs, I'll close with a post on why I believe that
Raku's decision to prioritize the values on the left is absolutely the correct one.