Originally posted to the ChiselStrike blog on Medium.com. [github] [discord]

Oh, language wars. If there is anything more fulfilling than fighting over editors and IDEs, that’s fighting over programming languages. That’s why at any given day, right below your usual post about Bitcoin and Web3, you will find a post about how Jane the Developer moved from language A to B, or B to C (Err… C is an actual language, but alas).

Gosh, who am I to point fingers. When I started my Rust career, I did that myself, comparing my decade-long experience with C++ with that breath of fresh air also known as Rust.

What’s common in all of those posts is that at the end of the day, despite all languages being theoretically equivalent, in practice they encroach in specific problem domains for practical reasons. It would be surprising to find someone writing modern user interfaces in C outside of Guantanamo Bay, and equally as surprising to find anyone writing an OS kernel in Python.

Because programmers tend to specialize, language comparisons tend to form their cliques as well. So you move between C++ and Rust, Ruby and PHP, but moving across cliques is far less common. Almost like moving to a new country on the other side of the planet with a completely new culture, as opposed to a new city a couple of miles away.

So I figured I’d share my own notes from my journey as a systems-level Rust programmer to an unlikely contender: Typescript!

Why the <expletive> would you do that?

The first question on anyone’s mind should be why would I do this, especially since I’ve done so much work in low level storage systems and databases before.

As it is known to some, I have recently founded ChiselStrike. ChiselStrike is a modern take on what something like Firebase might look like if designed in 2022 (making it trivial for app developers to just host their backends as a side effect). We’re abstracting the entire backend at the high level, providing it entirely as a consumable service. Uniquely — we abstract away most of the common elements of backend development (like Databases and authentication). So developers just express what they desire from their backend in clean, pure Typescript, and we take care of the rest. Why Typescript? Because (if counted together with Javascript) it reigns supreme with application developers.

So naturally, we wanted to write all the database and low level code in Rust, but at the same time all the APIs and user-visible interactions would happen in Typescript. The moment this choice was made, my fate was sealed: I had to go learn and master Typescript.

What I liked about Typescript

The first thing I loved about Typescript, compared to Rust, is just how expressive it is. In Rust, handling things like strings is an absolute pain. It might very well be better than C and C++, but that doesn’t mean much.

Functions can take multiple types with a very neat syntax: a: T1 | T2 . In Rust, you have to use enums. The Rust way does have the advantage to force you to check all possible variants, but understanding that Typescript is a higher level language, the convenience of just expressing type optionality is convenient, and that syntax is very pleasant in my opinion.

Objects are natively JSON and that is… surprisingly handy? It felt weird at first, but after less than a day I already started to miss that convenience when going back to Rust and being forced to serialize things back and forth.

Typescript is also very async-friendly, and everything is single threaded. This one is a fun one: I guess most people would consider this a disadvantage, but having worked with two thread-per-core frameworks in both C++ and Rust, I like how Typescript fits that model well.

In ChiselStrike, for example, the http server (written in Rust, using hyper) is a Thread-per-Core server. We do that by passing the SO_REUSEPORT flag when accepting the connection, and then essentially as many executors as we have threads. Inside each thread, there is a v8 instance. Since http requests are independent, this works just fine. Boom! Thread-per-Core!

Typescript works with async code really well, and it is really easy to produce high performance architectures like Thread-per-Core.

Another big advantage of Typescript is that it unleashes something that is pretty hard to get with anything else: an ability to work transparently both in the browser and in the server. The ecosystem is also pretty impressive. With over 1.3M npm packages available, orders of magnitude more than crates.io.

What I hated about Typescript

My previous impression of the Javascript/Typescript ecosystem was that it was a bit of a mess. I had heard the classic thing about the lack of native integer types making type coercion have surprising results. And by and large it is all true. In particular, even though I knew this theoretically, I still ended up with buggy code. Blame it on lack of experience if you will, but I still reserve the right to hate it!

I wanted to find if a particular http response was successful, so “anything in the 200s”. How did I write?

const success = req.ret / 100 == 2

This works if the return code is 200, but not if it is 201 or higher. I’ve been since informed by my very patient Typescript-expert friends that the usual way to write this in Typescript is with if statements testing for two bounds, like if ret ≥ 200 && ret < 300 so if you’re are a long-time Typescript developer thinking I am obviously stupid right now, I swear to you that this makes total sense in the minds of anyone coming from Rust and C++. For two reasons:

When you are used to dealing with low-level code, you learn to fear the processor ‘s branch predictor a bit and it is common to avoid branches when possible (although in this case it would likely predict well). Although division is usually not the fastest processor operation either, at least it is consistent, and in many cases you can rewrite them with shifts and bitwise operations (not in this case), so it is a natural way for us to do it.
100 is an integer, so is 201. In any of these other languages 201 / 100 is 2, because that’s how integer division works. But in Typescript, 201 / 100 = 2.01 , and the code above has to rewritten as:

const success = (req.ret / 100 | 0) == 2

I spent 10 years writing Kernel code, where as a rule of thumb you can’t even use the floating point unit. And in every other language I’ve had to interact, integers are the default. So this was really annoying. And sure, you get used to it, but those are my notes, from my journey and this is annoying to me! =)

But if I had to choose the thing I disliked the most… that would be the module system. Seriously, I don’t even know where to start. Maybe this makes sense for someone dealing with this ecosystem for many years, but the whole thing with CommonJS, ESM, and all the different standards is just beyond comprehension. It reminds me of the Python2 vs Python3 debacle, which is in my humble opinion probably the biggest disaster in our industry as a whole.

Although we love Deno, it certainly didn’t help my overall impression that Deno wants imports to be an actual URL including file extensions, while tscwill scream if the extension is actually there.

Now, once we established that this is bad, and just accept it as a fact of life, I still expected that to show up mostly in obscure modules here and there for most modern stuff.

But to my surprise, even fetch presented me with issues. To begin with, I was surprised to even need a module to do that. It feels like the most basic thing I’d want to do in that environment. The npm page for node-fetch opens with:

“Instead of implementing XMLHttpRequest in Node.js to run browser-specific Fetch polyfill, why not go from native http to fetch API directly?”

Polyfills are a word that you never hear about outside of this ecosystem… for describing something that happens a lot outside of this ecosystem!

First, let me tell you that as a foreigner in Typescript land, my brain translated that phrase to “F*** You”. With a big capital F. Later when I found out about polyfills, that made a bit more sense, but alas: node-fetch , a package with 13 million weekly downloads, is an ESM-only module. Much like with Python2 and Python3, the choice of one system versus the other is usually already there, so you end up stuck. And sure, the npm page for node-fetch itself claims that there is an alternate version of the package that works with CommonJS, but the whole thing is just so frustrating.

Lastly, I actually found an interesting design decision, at first, that Typescript is really just a bunch of annotations on top of Javascript. I love backwards compatibility and interoperability, so learning that filled me with joy.

However, calling those things “types” is a bit of a stretch. Consider the following two types:

type First = { a: string }
type Second = { a: string }

A function that accepts the first, will gladly accept the second. In Rust, I grew accustomed to defining wrappers on POD types, which is handy when you have a function taking many parameters at the same type and want to defensively make sure that you will never switch their order. For example, you could write:

fn my_func(first: FirstBool, second: SecondBool)

A bit of proc macros here, some type coercion there, and you don’t even need to have any significant boilerplate for that.

In Typescript (should we call it Shapescript?) this is just impossible, and that’s really frustrating.

One breath of fresh air in that direction is DeepKit. Although compile-time types are still the same as before, DeepKit preserves the type information at runtime. When we were considering how our translation layer should look like at ChiselStrike, we wanted to express uniqueness with something like this:

foo: Unique<string>,

This is completely impossible because Unique doesn’t mean anything in Typescript. We ended up using decorators instead. DeepKit allows us to write things like:

foo: string & Unique

While I prefer the <> syntax, at this point this really is just personal inertia. I am really excited about DeepKit, and following them closely to see what kind of new worlds they will enable.

Is that really an exclusive choice?

At the end of the day, the biggest surprise for me, is that this wasn’t really a transition. Like an immigrant that keeps their old home and visit often, I was pleasantly surprised when I looked around and realized that my life was now expressed in both Typescript and Rust.

We chose Deno as our runtime, which not only is written in Rust, allowing us to fix bugs and improve it easily, but also allows us to move things easily between the two worlds.

Although there is some cost of serialization, it is relatively easy to call Rust functions from Typescript through Deno, and that has been serving us well.

#[op]
fn op_chisel_relational_query_create(
    op_state: &mut OpState,
    op_chain: QueryOpChain,
    context: ChiselRequestContext,
) -> Result<ResourceId> {
    let query_plan = QueryPlan::from_op_chain(
        &RequestContext::new(
            current_policies(op_state),
            current_type_system(op_state),
            context,
        ),
        op_chain,
    )?;
    create_query(op_state, query_plan)

All in all, this was a surprisingly pleasant journey, that reminded me of previous times in my life when I moved countries. A bit of culture shock at first, but it now feels like I have two homes instead of one!

If you liked this article, check out ChiselStrike’s Github repo, where lots of those concepts are put in practice. Also feel free to join our Discordcommunity!

#Rust

Leveraging Turso’s embedded replicas in real world applications

Notes from my journey from Rust to … TypeScript????