The History of Async Rust

I'm writing this post because I've seen a lot of backlash and misunderstanding regarding async/await in Rust, both from a syntactic perspective but also direct criticisms of the implementation itself. I think it's important to understand the reasons behind the implementation details to understand why async Rust exists the way it does. It's certainly not perfect, and not the only way to implement async in modern languages, but my hope is that you'll understand why it had to be implemented in Rust the way that it was.

Introduction

The async/await syntax was merged into the Rust language in May 2018 after a massive effort by many brilliant people, introduced by RFC #2394 and RFC #2395, which was later revised in RFC #2418.

Asynchronous programming is an architecture that allows a process to initialize a long-running operation that doesn't block other operations from running at the same time. Non-blocking async is often implemented using the async/await syntactic pattern, which allows asynchronous functions to be defined in a similar way to synchronous functions. The details of the syntax vary slightly between languages, but almost all implementations use the notion of a "future" or a "promise" which are a proxy for the result of an async process until it resolves.

Async/await in rust is implemented using a mechanism called cooperative scheduling; there's another multi-threading approach called preemptive scheduling. The basic difference between the two is that in the preemptive model, the OS thread scheduler can step in and hand control from one thread to another at any time, such that tasks can be forcibly suspended. In the cooperative model, once a thread is given control, it continues to run until it explicitly yields control, or until it blocks.

A naive approach to developing an application that handles multiple tasks concurrently is by creating a new thread for each task. While this method is fine when dealing with a small number of tasks, it becomes problematic as the number of tasks increases. The more threads created, the greater the strain on system resources.

The underlying mechanism for asynchronous computation in Rust is the notion of a "task", which are lightweight threads of execution, such that many tasks can be cooperatively scheduled onto a single operating system thread.

The async/await syntax in Rust provides the following mechanism for function definitions:

async fn function(argument: &str) -> usize {
     // ...
}

This provides an equivalent definition to:

fn function<'a>(argument: &'a str) -> _Anonymous<'a, usize> {
     // ...
}

As we'll see later, the goal of this syntax is for the compiler to transform async functions into functions that evaluate to futures so that users don't have to write futures themselves.

pub trait Future {
    /// The type of value produced on completion.
    type Output;

    fn poll(self: PinMut<Self>, cx: &mut task::Context) -> Poll<Self::Output>;
}

In order to cooperatively schedule tasks, blocking tasks schedule themselves for later wakeup by the executor that's driving the future to completion. This wakeup by the executor is called polling, and it always returns a Poll value:

pub enum Poll<T> {
    /// Represents that a value is immediately ready.
    Ready(T),
    /// Represents that a value is not ready yet.
    Pending,
}

Futures do nothing unless polled by an executor - and there are many in the Rust ecosystem - but the most popular executor is Tokio.

Towards External Iterators

The origins of async rust seem to trace back to a 2013 mailing list post on external iterators by Daniel Micay. The relationship between external iterators and futures in Rust is deeply connected, with roots in the evolution of Rust's ownership and borrowing model.

External iterators integrated perfectly with Rust's ownership and borrowing model, with external iterators being compiled into structs that encapsulate the state of iteration. This encapsulation not only facilitated the maintenance of safe references to the data structures being traversed but also enhanced compilation efficiency.

Back to the Future

This same principle was later applied to futures, with Rust's type system ensuring that asynchronous computations can be composed and executed while maintaining memory safety. By conceptualizing asynchronous operations as state machines, Rust enables these operations to be polled for their readiness. This model eliminates the need for callbacks and unnecessary memory allocations, leading to more efficient and safer asynchronous code.

This representation of async processes as state machines led to one of the big criticisms of async in rust: the language chose a readiness model instead of a completeness model. The readiness (or demand-driven) approach essentially tells a process "let me know when you're ready to do this work," as opposed to the completeness (or callback) approach which says "let me know when you're done with this work."

The Development of the Async/Await Syntax

From 2017 to 2019, the development of the async/await syntax was led by Boats. Until that point, async/await was implemented as a macro library by Crichton. The macro implementation led to some gnarly errors if a user referenced a future state that was held over an await point, because futures could not hold a reference to its own state while awaiting.

The goal of the futures project was to be able to transform async functions into functions that evaluate to futures to avoid users having to write these themselves. One of the issues Boats and the team ran into was that the compilation of futures to state machines was a "perfectly-sized stack" - that is, that the stack didn't need to be resizeable. But the state machine was represented as a struct, and structs in Rust are safe to move, even though there's no need to move a future while it's being executed.

The final attempt at a solution introduced what we now know as the Pin type, which allowed enforcing that an encapsulated object not be moveable.

---

Footnotes

There is a difference between a future and a promise - a future is considered a read-only view of a variable, whereas a promise is a writable container that sets the value of the future.

Monomorphization is a compile-time process where the compiler creates a different copy of the code of a generic function for each concrete type that's needed.