2014-03-13

Co.Labs

Under The Hood Of Mozilla's New Multi-Core Browser And The Open Source Language That Powers It

The next-generation web engine from Mozilla is called Servo, and it's built in a language called Rust which Mozilla specially developed for performance.



Everyone knows Mozilla is a force for open source. But a lesser known story is their advancements in a revolutionary new open source language called Rust, which is what enables Mozilla's futuristic new browser to take advantage of computers with multi-core processors and machines with more than one processor.

The Rust language will power Mozilla's new browser, Servo, and its big selling point is efficiency. Because C++ crashes when it runs into memory allocation issues, it weakens any browser that uses the language. Mozilla designed Rust to be superior to C++ this way, more easily isolating tasks and promote a process known as "work stealing," which is when tasks from an overloaded processor are shifted over to another one.

"Rust is targeted at C++ developers, and Mozilla has many of those. We do not expect it to take long for an experienced C++ hacker to acclimatize to Rust. So far initial experiences have shown that to be the case," says Brendan Eich, chief technology officer and senior vice president of Engineering at Mozilla.

Mozilla has no immediate plans to release Servo, since much of its success relies on Rust’s completion. Rust will be nearing a 1.0 release over the course of this year, but until then, it is in the hands of its myriad contributors.

And those are good hands to be in. "Umbrella Security Labs has been using Rust in production to do reliable, real-time processing of DNS information. The Skylight service also uses Rust to implement its Ruby on Rails application monitor," says Eich.

A More Efficient Browser

It is true that most browsers, including Mozilla Firefox, eventually patched up their browsers to incorporate the multi-thread programming that leverages our computer’s multi-processors. But their architectures are so convoluted as a result of the incremental changes that they are difficult to modify.

Servo won’t have this issue. Mozilla is building Servo’s browser stack architecture from scratch.

Most of the hiccups that a browser runs into when loading a webpage are related to how it sequences the priority of its page elements. Conventionally, elements are loaded in a hierarchy so that they don’t affect each other’s positioning on the page.

But with the help of Rust, Servo can pre-identify those page elements that don’t conflict in size and positioning and lay them out at the same time, in parallel.

In total, six Mozilla engineers are working on Servo almost full-time, with other part-timers in the organization. Over a dozen engineers at Samsung do another chunk of the work, and the number of contributors is growing.

Not surprisingly, Mozilla plans to widen their outreach and training to contributors as Rust matures. And in what might interest those volunteers, Mozilla is now hiring three more paid engineers to work on Servo.

The Rust Subreddit

Free-moving contributors from outside Mozilla most notably occupy the Rust subreddit. Run by Rust enthusiasts since 2010, it is a one-stop source of real-time updates on changes, issues, and even meeting minutes from Mozilla’s in-house Rust team.

"The Rust subreddit tends to be where news items of interest to the Servo and Rust community are posted or announcements of interesting new projects," says Eich.

Reddit user Azth founded the Rust Subreddit on December 2nd, 2010, though current co-moderator Ben Striegel says, "he seems to be more of a mildly interested outsider than an active contributor." To be sure, the subreddit had almost no activity in the first years following its inception.

Striegel had been following Rust developments since late 2011 and focused on removing undesirable and outdated features from the Rust compiler that takes Rust from source code to a bona fide programming language. But by the end of 2012, he saw that Rust’s contributor community was outgrowing the existing mailing list and IRC channels, both of which Mozilla runs.

At first, Striegel thought about creating a dedicated website to host a forum but ultimately realized that Rust needed more reach than an insular site could provide.

"I decided that we needed maximum visibility for faster growth, and it came down to Reddit or Google+. Though I wasn't exactly thrilled with either site in general, I decided I was more put-off by Google+'s real-names policy than by Reddit's typical anarchy," says Striegel.

So, in October 2012, Striegel asked Azth to appoint him as a moderator. Almost immediately, Striegel aggressively advertised the Rust subreddit in places where he thought Rust threads might pop up, like on the Programming subreddit and Hacker News. Rust’s subreddit eventually grew to a point where Striegel had to appoint another moderator.

To make people’s roles more clear on Rust’s subreddit, Striegel created three Reddit-specific tags called "flairs": the "contributor" flair, the "supercontributor" flair, and the "Mozilla" flair. While the "Mozilla" flair is reserved for Mozilla employees, the "contributor" and "supercontributor" flairs denote non-Mozilla Rust supporters.

Now, there are 33 people on the subreddit with either a "contributor" or the more elitist "supercontributor" flair and nine with the "Mozilla" flair.

Open Communication Philosophy

Although Reddit is an unofficial communication channel at Mozilla, it is nonetheless a valuable one, "especially given how much work is done by people who aren't employees of Mozilla," says Eich.

That goes for any of the other communication channels that Mozilla operates on. It contributes heavily to Hacker News and StackOverflow, where Mozilla is now discussing plans to create a section dedicated only to Rust. And anything that shows up on its official broadcast channels, GitHub, IRC, and email on Google Groups, is available online right away.

"Almost all the communication is publicly visible without any delay. We do this because that is the way Mozilla works, and because with a distributed team, it's the only way to keep everyone in the loop. This means you can see projects at every stage of development, well before they are ready for launch," says Eich.

"Most of the real work happens on GitHub. We use email for discussions that are too large for GitHub or where broad reach is desirable. IRC is for more casual discussions and Q&A about various things," adds Eich.

If you were to peruse each of these channels on any given day, you would run across a few of the same people participating in discussions across each space. "This structure isn't too different than most other open source projects, except perhaps that Rust's large community gives rise to things like the Rust Reddit," says Eich.

Through its online presence, Mozilla gives generous guidance to its contributors, or the "Mozilla community" as they are called. And every now and then it sparks the right flame to get the broader community moving along.

In a message exchange over Reddit Striegel wrote to me, "I spend a lot of time trawling websites looking for mentions of Rust (such as monitoring the "rust" tag on Stack Overflow and the "#rustlang" tag on Twitter), answering questions, correcting misinformation, and providing historical context. I also lurk in over a dozen programming language-related IRC channels on irc.freenode.net, listening for mentions of Rust in order to get a pulse on how we are perceived in the broader community."

If star Mozilla contributors like the Rust subreddit’s Striegel are any evidence, it seems that Mozilla has found the right tack when it comes to creating a community movement.

[Image: Flickr user rbrwr]




Add New Comment

5 Comments

  • No doubt, I know Rust has a strong marketing lead, but what does it have to offer over other multicore concurrent focused languages like golang?

  • In addition to Ted's comments, I would also add that Rust's concurrency features aren't baked in. The language semantics provide the building blocks for creating safe, statically checked concurrency features, like those in the standard library. However if those are not suited to the specific project they can be custom built. For example Rust's lightweight tasks are great for applications like web servers, but Servo has implemented its own highly optimized work queue to serve its particular performance demands. This kind of low level control combined with compile time safety simply does not exist in other languages like Go, Erlang, or C++.

  • Ted Mielczarek

    Rust has a few features that make it more suitable than Go for this purpose--the ability to program without a garbage collector, for one (although the option is available) means managing memory usage more carefully is possible. Additionally, Rust's type system is designed without null values, rendering a large class of programming bugs impossible. Finally, Rust does not have shared state in its concurrency model, meaning that programmers aren't forced to deal with the same old concurrency bugs as they have in C.