At Applifting, we often strive to utilize our digital craftsmanship to create not just working code but good code. This is usually what brings happiness to our developers—the ability to make use of their deep knowledge and let their creativity solve problems while developing a good environment for others to work on the same code. We value a fair mix of maintainability and elegance. We prefer not to rely on tricks nor producing write-only code but finding clever ways to make development easier. This also extends beyond the confines of Applifting, as one of our tribes, DX Heroes, specifically focuses on improving developer experience.
We use a range of languages in our projects, but today, I want to talk about TypeScript. When I was offered to work in TypeScript, I immediately fell in love with its implementation of a type system. It made my head fill with thoughts of all the borderline insane types we could craft to make entire classes of errors a thing of the past. This comes at the cost of having to manage those type definitions, of course, but we still have tests. Strong types are complementary here, allowing tests to focus more on actual features and less on ensuring that we use our internal APIs correctly.
Creating a language parser
Having experienced Haskell at university, having written some template magicks in C++, and in general being interested in higher-order types, I felt more than comfortable experimenting in TypeScript. As I descended into the depths of development, I was tasked with producing a parser for a domain-specific language–Comlink–that we were also developing for our client Superface. This language was simple enough (and the project was lenient enough) to create our own recursive descent parser, to define a lexer and its tokens, and to then start defining syntax rules. Afterwards came rule combinators, conceptually simple "or", "and", "optional", "repeat", etc. As the complexity was rising, it became clear that it was imperative to create comprehensive typing for our code to keep our sanity at reasonable levels. And we got more focused tests as a bonus. It allowed us to handle local complexity better by constraining global complexity—by checking correct integration at the type level.
Consider the following type definitions, which imply a very simple toy language, with a construct like foo = 1 + bar - baz. We also assume equal operator precedence.
The syntax rule for Assignment could be written in plain English: identifier and "=" and (number or identifier) and optional repeat (operator and (number or identifier)).
Ideally, we’d like our programmatic definition of this rule to be very similar so that the intuition is easy to transfer. We would also like to have generic and correctly typed partial rules. Having to define each rule individually, manually assign types, and compose them together would be overwhelming. Instead, we want a simple API with automatic type resolution. TypeScript gives us just that.
Note that since we are still in a dynamic language, we don’t have to worry as much about the practicality of the implementation. We are always scoped to a single function, so we can afford to use tricks such as the any type and as casts to convince the compiler that we know what we are doing. We localize complex code so that all other code can be made simpler, and the types check the usage.
And now we have everything to define the rules themselves. Composability comes by binding commonly partial rules to variables. In the code block above, we also defined some convenience in terms of filtering Operators and the map function, but these could just as easily be achieved by subclassing. The following example includes types explicitly, but the only requirements are return types when mapping. All other types are correctly inferred by TypeScript, especially for partial rules.
And there we have it. We now have all the types needed to comfortably call ruleAssignment.match(stream) on a token stream and get an Assignment node for our toy language. If we ever need to change the definition of the assignment rule, we can do so knowing that the compiler will guide us, derive correct types, and show sensible errors. We have improved the developer experience by utilizing the power of the language.
Mr. President, I have types on my hands
You can find a working implementation in this TypeScript playground, and the real project is open source, so you can see the full implementation of the actual parser on GitHub. However, it is more complex because it deals with error reporting, has more token types, and includes an embedded scripting language. It is still in development, so some code distilled in this article might actually be cleaner than the one in the project. Iterative development is often like that, which is why it is so beneficial to have both an expressive typing system and strong testing coverage to ensure the quality of our product.
After all, quality is always on our minds. It takes time, so we continuously hone our skills through self education and by having the freedom to explore advanced language features. The acquired knowledge is valued, and we are incentivized to pass it on inside the company. That way, we get to learn not just the standard way of doing things, but we also get to experiment and innovate, which helps to see the problem at hand from a different point of view. If the level of technological know-how we gain at Applifting is something that interests you, check out our open positions and see if there is anything that catches your eye. In the next article, I will follow up with another type construct from the same project, which pushes the boundaries of how far is too far. Stay tuned!
