@without-boats
Publisher
60
Posts
In July, I described a way to make pinning more ergonomic by integrating it more fully into the language. Last week, I develoepd that idea further with the notion of UnpinCell: a wrapper type that lets a user take an &pin mut UnpinCell<T> and produce an &mut T, similar to how other cells let a us...
A variation on my previous design for pinned places has occurred to me that would be more consistent with Rust’s existing feature set.
In the previous post, I described the goal of Rust’s Pin type and the history of how it came to exist. When we were initially developing this API in 2018, one of our explicit goals was the limit the number of changes we would make to Rust, because we wanted to ship a “minimum viable product” of a...
The Pin type (and the concept of pinning in general) is a foundational building block on which the rest of the the Rust async ecosystem stands. Unfortunately, it has also been one of the least accessible and most misunderstood elements of async Rust. This post is meant to explain what Pin achieve...
This post is meant as an explainer about how substructural type theory can be applied in programming language design. Terms like “substructural type theory” tend to scare and confuse programmers who don’t write Haskell on the weekends, so one thing programming language designers should do when th...
In a high-level language, the programmer is deprived of the dangerous power to update his own program while it is running. Even more valuable, he has the power to split his machine into a number of separate variables, arrays, files, etc.; when he wishes to update any of these he must quote its na...
For the past few months I’ve been mulling over some things that Russell Johnston made me realize about the relationship between effect systems and coroutines. You can read more of his thoughts on this subject here, but he made me realize that effect systems (like that found in Koka) and coroutine...
One problem with the design of async Rust is what do about async clean-up code. Consider that you have a type representing some object or operation (like an async IO handle) and it runs clean up code when you are done using it, but that clean up code itself is also non-blocking and could yield co...
In my previous post, I wrote about the distinction between “multi-task” and “intra-task” concurrency in async Rust. I want to open this post by considering a common pattern that users encounter, and how they might implement a solution using each technique.
In the early-to-mid 2010s, there was a renaissance in languages exploring new ways of doing concurrency. In the midst of this renaissance, one abstraction for achieving concurrent operations that was developed was the “future” or “promise” abstraction, which represented a unit of work that will m...
Last week, Tyler Mandry published an interesting post about a problem that the Rust project calls “Barbara battles buffered streams.” Tyler does a good job explaining the issue, but briefly the problem is that the buffering adapters from the futures library (Buffered and BufferUnordered) do not i...
When we developed the Pin API, our vision was that “ordinary users” - that is, users using the “high-level” registers of Rust, would never have to interact with it. We intended that only users implementing Futures by hand, in the “low-level” register, would have to deal with that additional compl...
I wanted to follow up my previous post with a small note elaborating on the use of coroutines for asynchrony and iteration from a more abstract perspective. I realized the point I made about AsyncIterator being the product of Iterator and Future makes a bit more sense if you also consider the “ba...
In my previous post, I said that the single best thing the Rust project could do for users is stabilize AsyncIterator. I specifically meant the interface that already exists in the standard library, which uses a method called poll_next. Ideally this would have happened years ago, but the second b...
Four years ago today, the Rust async/await feature was released in version 1.39.0. The announcement post says that “this work has been a long time in development – the key ideas for zero-cost futures, for example, were first proposed by Aaron Turon and Alex Crichton in 2016”. It’s now been longer...
Async/await syntax in Rust was initially released to much fanfare and excitement. To quote Hacker News at the time:
I want to address a controversy that has gripped the Rust community for the past year or so: the choice by the prominent async “runtimes” to default to multi-threaded executors that perform work-stealing to balance work dynamically among their many tasks. Some Rust users are unhappy with this dec...
I want to wrap up my consideration of the idea of adding new auto traits to Rust with some notes from a conversation I had with Ariel Ben-Yehuda.
In my previous post, I described the idea of using an edition mechanism to introduce a new auto trait. I wrote that the compiler would need to create an “unbreakable firewall” to prevent using !Leak types from the new edition with code from the old edition that assumes values of all types can be...
In Rust, there are certain API decisions about what is and isn’t sound that impact all Rust code. That is, a decision was made to allow or not allow types which have certain safety requirements, and now all users are committed to that decision. They can’t just use a different API with different r...
One of the most famous anecdotes that forms the basis of the United States’ political self-identity is the story of an interaction between Benjamin Franklin and Elizabeth Willing Powel after the Constitutional Convention of 1787, which established the United States’ present form of government. Po...
I have been devoting a lot of my free time in the past month to thinking about structured concurrency, and a blog post about that is coming soon, but first I want to revisit iterators and generators.
This is the first post in a series of posts about concurrency in Rust, and the different APIs that can exist to support it. Unlike my recent series on control-flow effects, this series isn’t driving toward any particular vision of what I think the Rust project should do. Instead, I am just trying...
One of the main emphases of my recent posts has been that I believe shipping generators would solve a lot of user problems by making it easy to write imperative iterative code, and especially to make that iterative code interact well with asynchrony and fallibility as well. One thing that frustra...
In a previous post, I established the notion of “registers” - code in Rust can be written in different registers, and it’s important to adequately support all registers. I specifically discussed the low-level interface of the AsyncIterator trait, about which there is currently a debate. The inter...
The previous two posts in this series tried to discuss the design of Rust through the lens of some higher level language concepts:
This is the second post in an informal series commenting on the design of async Rust in 2023. In my previous post, after a discussion of the “registers” in which control-flow effects could be handled in Rust, I promised to turn my attention to recent proposals around a concept called “keyword gen...
It’s been nearly two and half years since I was an active contributor to the Rust project. There are some releases that I’ve been very excited about since then, and I’m heartened by Niko’s recent blog post emphasizing stability and polish over grand new projects. But I’ve also felt a certain appr...
In the previous post in this series, I wrote about how the core state machine of ringbahn is implemented. In this post I want to talk about another central concept in ringbahn: “drivers”, external libraries which determine how ringbahn schedules IO operations over an io-uring instance.
A bit over a year ago, I wrote some notes on a “smaller Rust” - a higher level language that would take inspiration from some of Rust’s type system innovations, but would be simpler by virtue of targeting a domain with less stringent requirements for user control and performance. During my time o...
Today I made a new release of the iou library, which contains idiomatic Rust bindings to the liburing library. This library allows users to manipulate the new io-uring interface for asynchronous IO on Linux. For more context, you can read my previous post on the first release of iou last year.
I’ve just released a new crate called propane, which is a library for writing generator functions. It can only run on nightly:
It’s hard to believe that its been more than 3 years since I opened RFC 2000, which defined the const generics for Rust. At the same time, reading the RFC thread, there’s also been a huge amount of change in this area: for one thing, at the time the RFC was written, const fns weren’t stable, and...
Last time I wrote about ringbahn, a safe API for using io-uring from Rust. I wrote that I would soon write a series of posts about the mechanism that makes ringbahn work. In the first post in that series, I want to look at the core state machine of ringbahn which makes it memory safe. The key typ...
While implementing ringbahn, I introduced at least two bugs that caused memory safety errors, resulting in segfaults, allocator aborts, and bizarre undefined behavior. I’ve fixed both bugs that I could find, and now I have no evidence that there are more memory safety issues in the current codeba...
This is just a note on getting the best performance out of an async program.
In my previous post, I discussed the new io-uring interface for Linux, and how to create a safe API for using io-uring from Rust. In the time since that post, I have implemented a prototype of such an API. The crate is called ringbahn, and it is intended to enable users to perform IO on io-uring...
Last fall I was working on a library to make a safe API for driving futures on top of an an io-uring instance. Though I released bindings to liburing called iou, the futures integration, called ostkreuz, was never released. I don’t know if I will pick this work up again in the future but several...
In a previous post, I shortly discussed the concept of “effects” and the parallels between them. In an unrelated post since then, Yosh Wuyts writes about the problem of trying to write fallible code inside of an iterator adapter that doesn’t support it. In a previous discussion, the users of the...
I’ve long been a proponent of having some sort of syntax in Rust for writing functions which return results which “ok-wrap” the happy path. This is has also always been a feature with very vocal, immediate, and even emotional opposition from many of our most enthusiastic users. I want to write, i...
About two and a half years ago I wrote a Rust library called failure, which quickly became one of the most popular error handling libraries in Rust. This week, its current maintainer decided to deprecate it, a decision I strongly support. This week, I also released a new and very different error-...
This is just a post about something that grinds my gears a bit more than it reasonably should: I think the habit of applying for CVEs for Rust (and Rust ecosystem libraries) is silly at best and harmful at worst. I think it muddies the waters about what a vulnerability is, and paints an overly ne...
I’ve just released a new crate called waitmap. This is a concurrent hash map (built on top of dashmap) intended for use as a concurrency primitive with async/await. It extends the API of dashmap by having an additional wait method.
One of the big sources of difficulty on the async ecosystem is spawning tasks. Because there is no API in std for spawning tasks, library authors who want their library to spawn tasks have to depend on one of the multiple executors in the ecosystem to spawn a task, coupling the library to that ex...
Today I’m releasing a library called iou. This library provides idiomatic Rust bindings to the C library called liburing, which itself is a higher interface for interacting with the io_uring Linux kernel interface. Here are the answers to some questions I expect that may provoke.
The first version of async/await syntax is in the beta release, set to be shipped to stable in 1.39 on November 7, next month. There are a wide variety of additional features we could add to async/await in Rust beyond what we’re shipping in that release, but speaking for myself I know that I’d li...
Many people who use Rust for a bit - especially those who like the language but do not fall in love with it - feel a sense that there must be a smaller, simpler variation on the same theme which would maybe be a little less powerful, but would also be much easier to use. I agree with them, but I...
In my previous post I said that the lang team would be making our final decision about the syntax of the await operator in the May 23 meeting. That was last Thursday, and we did reach a decision. In brief, we decided to go forward with the preliminary proposal I outlined earlier: a postfix dot sy...
The idea of a zero cost abstraction is very important to certain programming languages, like Rust and C++, which intend to enable users to write programs with excellent performance profiles with relatively little effort. Since this idea is fundamental to the design of Rust and my work, I want to...
This is an announcement regarding the resolution of the syntax for the await operator in Rust. This is one of the last major unresolved questions blocking the stabilization of the async/await feature, a feature which will enable many more people to write non-blocking network services in Rust. Thi...
The biggest unresolved question regarding the async/await syntax is the final syntax for the await operator. There’s been an enormous amount of discussion on this question so far; a summary of the present status of that discussion and the positions within the language team is coming soon. Right n...
This is my second post on the design of generators. In the first post, I outlined what an MVP of the feature would look like. In this post, I want to take a look at the first design issue for the feature: how it integrates with the ? operator.
We’re still not finished with the design of async/await, but it’s already become clear that it’s time to get the next phases of the feature into the pipeline. There are two extensions to the minimal async/await feature we’ve currently got that seem like the clear high priority:
In the previous post, I provided a lot of background on what the waker API is trying to solve. Toward the end, I touched on one of the tricky problems the waker API has: how do we handle thread safety for the dynamic Waker type? In this post, I want to look at that in greater detail: what we’ve b...
Work on supporting async/await in Rust continues to progress rapidly. I’m hoping to write a retrospective on everything that happened in 2018 in a few weeks. Right now we’re closing in on an important milestone: stabilizing the futures API that will be used to interact programmatically with async...
We all know that classic aphorism: Year comes to an end, Rust blog post press send. This is mine.
This blog post is about a project called Romio that I’ve been working on over the past two or three weeks. Romio is a port of a small part of the Tokio project to the newer futures APIs.
One thing we’ve left as an unresolved question so far in the matter of async/await syntax is the exact final syntax for the await operation. In the current implementation, awaits are written using a compiler plugin:
Rust 2018 is almost out the door, but there is one big decision the language team has yet to make. It has to do with the modules and paths system, so of course it is a very easy decision that no one has a strong opinion about. ;-)
