Rust 2018: maybe don't be too stable

I initially did not want to write a post with what I want and foresee for Rust in 2018, because I'm already very happy with it! I have spent more than 4 years tinkering with the language, experimenting, and I love the freedom I get when playing with low level stuff. In those 4 years, I discovered a wonderful, welcoming community and made some awesome friends. So, yes, I'm happy with Rust as it is :)

But some of the recent #Rust2018 posts made me react a bit. I'm interested in learning what other people see in Rust, so I read almost all of them, and there's an easy trend to follow. Rust should be stabilized. Rust should be boring and safe. Crates should be stabilized. We should have definitive crates for some purposes like HTTP clients or async programming.
This is not surprising, since there's already been a lot of focus on stability in 2017, with the impl period, the merge of the Rust epochs RFC, and the fact that more and more companies start relying on Rust.
We want Rust to be appealing to (big(ger)) companies, and to that end we need good compatibility between Rust versions, a high quality ecosystem of crates that work on stable Rust versions. We want newcomers to have a well prepared toolbox for their first projects.

Before that stabilization goal appeared, Rust looked a bit chaotic, with new features coming every 6 weeks, new crates popping up here and there, people hacking something quickly and publishing it the next minute. And this is something I love about this language.
People try stuff, cargo lets them publish it easily, Rust makes sure it's running smoothly. Sure, there's a lot of redundant crates, most of them are far from the big "1.0 stable" target, but it's fine.
This language and its community are full of that unabashed optimism that makes newcomers go "hey, should I really try to write my own kernel? OF COURSE I SHOULD". Should I try to make cool stuff with Web Assembly while it barely landed in nightly? YESSSSSS
I have seen over and over shitposting on twitter that ends up with people hacking on a cool new project. I have seen people publish a crate competing with another well known one, that will then send a PR for their idea to the bigger crate the next day.
I am overly enthusiastic about this, to the point that opening /r/rust often feels like Christmas: what new toys will we get today?

So, to be clear, I am all for getting more stuff stable. We need a stable, asynchronous hyper. We need futures to work. We need impl trait and various other Rust features that will appear in the following months or years. What we do not need is the attitude that wants everything to crystallize.
How many times have I seen people criticising the "yet another" asynchronous IO/command line argument system/web framework/parser, with the usual arguments that this is lost focus, redundant, that why didn't they try to do that in $BIG_PROJECT. This is fine.
Go on, make other parser libraries to compete with nom, keep me on my toes. Try other approaches than tokio. Test different approaches to writing web applications.

The underlying idea for me is that Rust is still incredibly young, extremely enthusiastic, and we still don't fully know how to write Rust. So, yes, we need some parts of Rust to stabilize, but we must balance that with its movement. What is stable and "the way we do things" now might not be the way to go in a year or so.

Let people experiment and lose focus. Keep hacking on cool stuff.

Adventures in logging

After working on the Sōzu HTTP reverse proxy for a while, I came up with an interesting approach to logging. Now why would I come up with my own logger, when there are existing solutions in Rust? Mainly, log and slog. That logging library grew up from testing things out with log, and changing requirements along the way.

Beginning with log and env_logger

Like a lot of other Rust developers, I started out with log and env_logger:


#[macro_use]
extern crate log;
extern crate env_logger;

fn main() {
env_logger::init().unwrap();

info!("starting up");
}

It's nice and easy: every library that depends on log will use that same set of logging macros (error, info, warn, debug, trace) that will use whatever global logger was defined. Here we use env_logger to define one.

env_logger is useful because it can apply a filter to the log, from an
environment variable:

[code lang=text]
# will show the logs above info level
RUST_LOG=info ./main

# will show the logs above info level, but also logs above debug level
# for the dependency 'dep1'
RUST_LOG=info,dep1=debug ./main
[/code]

You can also define the filter by module or apply a regular expression.

Custom formatter

env_logger allows you to build your own log formatter. This feature is especially important for me, as I like to add metadata to my logs.

Defining a custom formatter with env_logger is quite straightforward:

[code lang=text]
let format = |record: &LogRecord| {
format!("{} - {}", record.level(), record.args())
};

let mut builder = LogBuilder::new();
builder.format(format).filter(None, LogLevelFilter::Info);

if env::var("RUST_LOG").is_ok() {
builder.parse(&env::var("RUST_LOG").unwrap());
}

builder.init().unwrap();
[/code]

It is easily combined with the filtering and usage of the RUST_LOG environment variable.

Where things get annoying: reducing allocations

If you take a look at env_logger, you'll realize that it will allocate a String for every log line that will be written, using a formatting closure.

Let's get one thing out of the way first: I completely agree with the idea you should not try to optimize stuff too much. But I'm in the case of a networking component that will handle a lot of traffic. I had debugging sessions where I generated tens of gigabytes of logs in a few seconds, and needed almost all of them, to debug async IO issues. In those cases, the time spent allocating and deallocating log lines becomes relevant.

So, how would I get a custom log formatter that does not allocate much? As it turns out, when you tell log to use your logger with log::set_logger, it requires something that implements Log. The logger's log method receives a LogRecord, a structure that's created on the fly from LogLocation, LogMetadata and