This Week in Redox 27

By goyox86 on Thursday, July 27, 2017

This is the 27th post of a series of blog posts tracking the development and progress of Redox, the Rust operating system. If you want to know more about Redox in general, visit our Github page.

(edited by goyox86)

PSA

If you have any questions, ideas, or are curious about Redox, we recommend joining #redox on irc.mozilla.org, our Discourse forum or you can get an invite to our chat by sending an email request to info@redox-os.org.

What’s new in Redox?

TL;DR

Welcome to the 27th edition of “This Week in Redox”!

As always, we have a bunch of good news to share with you. So lets get the ball rolling!

This week, one of the biggest changes is that we started to track the Cargo.lock file of most of Redox crates in Git as an attempt to improve the reproducibility of builds.

@jackpot51 did a bunch of work in the kernel in order to add support for process groups. Result: as of this week, you can send signals to process groups!

There was a also a small change on the kernel making an error to pass an non-empty buffer to dup in schemes, basically, after this change you will get EINVAL if you do so. @jackpot51 also prevented handling of nested signals.

The streak on Ion seems like a never ending one! They continue to ship new features and improvements at a steady (and rather fast!) pace. This week @jackpot51 simplified signal handling by adding an atomic bitmask of pending signals and removing the extra signal handling thread. Work by @mmstick on the mdBook based user manual continues, @huntergoldstein added support to allow builtins to be executed as part of pipelines, and made the RawFDs now managed by forked process when possible, along some other bug-fixes. Also in Ion, @bblancha implemented associative arrays, the and and or builtins and he added an option to the drop builtin for dropping arrays. Mr @drosseau added support for stepped ranges while @memoryleak47 implemented command line options for popd.

There was work done on moving the OS related #[cfg] functions to sys by @bb010g and @pithonsmear who added reverse, to_lowercase and to_uppercase to Ion’s word expansion parser.

The XHCI driver got a lot of love by @jackpot51 with a big general refactoring plus the addition of runtime registers, tests for the TLB, a revamp of the events mechanism, fixes of the mapping size and the addition of a lot of debugging. It looks very cool now!

Redoxfs saw the birth of directory symbolic links, a fix for readdir when directories don’t fit in the buffer and a FUSE update allowing setting mtime to earlier times. All of those by @ids1024.

On the TFS department we have for the first time in a while new contributors! This work includes @m4b’s usage of debug_map in chashmap enabling maps pretty printing, @cedenday’s remotion of specks dependency on std and the fix for an infinite loop in chashmap’s scan() by @memoryleak47. Well done people!

@ticki continues improving TFS’s test suite along with an interesting work in reserving reserved special pointers for conc::hazard::State to avoid overlapping with pointers used in Protect. Also, if you are an compression algorithm aficionado you might want to look at @ticki’s notes on zmicro, as they were updated this week.

In coreutils land @ids1024 allowed directory as second argument to ln and implemented octal escapes in tr.

Continuing his last week’s work on userutils @goyox86 made revisions to getty, passwd, login and su in order to update them to coreutils conventions and new APIs while Mr @jackpot51 made updates to getty in order to use PTY to provide line control for raw consoles and vesad.

We end this week’s tour on the cookbook which saw the addition of a status.sh script for checking git modifications, an update.sh one to, well, update everything and the corresponding updates to cook.sh caused by Cargo.lock being now tracked in version control. All of that by @jackpot51.

@ids1024 added recipes for GNU grep, diffutils and made a patch in the git recipe to use ; as PATH separator.

Enjoy the rest, and see you next week!

Kernel

@jackpot51 Made changes to make an error to supply a dup buffer to schemes that do not handle it. Details here and here.
@jackpot51 Added group id (pgid) to processed. Details here.
@jackpot51 Implemented sending signals to process groups and modified the maximum contexts number fit inside isize. Details here.
@jackpot51 Prevented nested signals and fixed a check for PID > 0. Details here.
@ids1024 Stripped extra slashes from the path in processes. Details here.
@jackpot51 Checked Cargo.lock in into source control. Details here.
@jackpot51 Updated the paging module to new Rust nightly API. Details here.

Ion

Ion is a shell for UNIX platforms, and is the default shell in Redox. It is still a work in progress, but much of the core functionality is complete. It is also currently significantly faster than Bash, and even Dash, making it the fastest system shell to date.

@pithonsmear Added reverse, to_lowercase and to_uppercase. Details here.
@jackpot51 Added support for setpgid and tcsetpgrp to Redox. Also, updated to use a customised termion to get correct sizes on Redox. Details here.
@bblancha Added an option to the drop builtin for dropping arrays. Details here.
@jackpot51 Simplified signal logic so that it is safe and single-threaded. Details here.
@mmstick Implemented a README for mdBook Manual. Details here.
@huntergoldstein Made a fix to ignore SIGTTOU before setting process group. Details here.
@bb010g Moved OS #[cfg] functions to sys. Details here.
@bblancha Implemented a simple form of associative arrays. Details here.
@llambda Made Ion compile on Darwin. Details here.
@fengalin Fixed the compilation on Redox. Details here.
@nivkner Included whitespace in alias builtin arguments. Details here.
@memoryleak47 Implemented popd command line options. Details here.
@huntergoldstein Added support to allow builtins to be executed as part of pipelines. Details here.
@mmstick Fixed quote terminator. Details here.
@drosseau Added support for stepped ranges. Details here.
@jackpot51 Fixed the build on Redox. Details here.
@mmstick Added rustfmt config. Details here.
@mmstick Create PID variable to store shell’s PID. Details here.
@mmstick Did some updates to the manual. Details here and here.
@huntergoldstein Made a change so RawFDs are now managed by forked process, not shell::pipe::builtin, when appropriate. Details here.
@drosseau Fixed some range syntax bugs. Details here.
@huntergoldstein Made a fix so expand_process now does not strip newlines for quoted instances. Details here.
@bblancha Implemented the and and or builtins. Details here.
@jackpot51 Simplified cross platform functions. Details here.

Drivers

@jackpot51 Added a change to return error when dup buffer is not empty. Details here and here.
@jackpot51 Added runtime registers and testing for TLB in the XHCI driver. Details here.
@jackpot51 Fixed mapping size of XHCI, and added more debugging. Details here and here.
@jackpot51 Refactored events mechanism on the XHCI driver. Details here.
@jackpot51 Added Cargo.lock to version control. Details here.
@jackpot51 Did a big refactoring of the XHCI driver. Details here and here.

Redoxfs

@jackpot51 Added a change to return error when dup buffer is not empty. Details here and here.
@ids1024 Made a fix for readdir when directory does not fit in buffer. Details here.
@ids1024 Made a change to FUSE in order to allow setting mtime to earlier time. Details here.
@ids1024 Implemented directory symlinks. Details here.

TFS

TFS is a modular, fast, and feature rich next-gen file system, employing modern techniques for high performance, high space efficiency, and high scalability.

@ticki Add a debug assertion against null pointers and one pointers in conc::garbage::Garbage. Details here.
@m4b Made a change in chashmap to use debug_map enabling pretty printing requests from user. Details here.
@m4b Fixed a stack overflow in debug prints for guards in chashmap. Details here.
@ticki Bumped speck to 1.1.0. Details here.
@ticki Reserved special pointers for conc::hazard::State to avoid overlapping with pointers used in Protect. Details here.
@ticki Removed warnings from conc in release mode. Details here.
@ticki Added some notes on the ideas for the zmicro compression algorithm. Details here.
@ticki Prevented certain tests in conc from messing with each other by having separated states. Details here.
@ticki Added tests on-exit garbage collection in conc. Details here.
@ticki Set down MAX_NON_FREE_HAZARDS in conc::local to avoid unnecessarily high memory usage. Details here.
@ticki Added new tests for drops in conc::Atomic with the overwrite drops. Details here.
@ticki Added tests for conc::local’s hazard clearing behaviour. Details here.
@memoryleak47 Fixed an infinite loop in chashmap’s scan(). Details here.
@ticki Fixed an unsoundness of the internal conc API. Details here.
@cedenday Removed the dependency of speck on std. Details here.

Coreutils

@jackpot51 Added Cargo.lock to version control. Details here and here.
@ids1024 Allowed directory as second argument to ln. Details here.
@ids1024 Implemented octal escapes in tr. Details here.

Userutils

@goyox86 Made some improvements to getty. Details here.
@jackpot51 Fixed getty spawning code. Details here.
@jackpot51 Made updates to gettyin order to use PTY to provide line control for raw consoles and vesad. Details here.
@jackpot51 Added Cargo.lock to version control. Details here and here.
@goyox86 Revisiting passwd to update it to coreutils conventions and new APIs. Details here.
@goyox86 Revisiting su to update it to coreutils conventions and new APIs. Details here.
@goyox86 Revisiting login to update it to coreutils conventions and new APIs. Details here.

Netstack

@jackpot51 Added a change to return error when dup buffer is not empty. Details here.
@jackpot51 Added Cargo.lock to version control. Details here and here.

Netutils

@jackpot51 Added Cargo.lock to version control. Details here and here.

Orbital

@jackpot51 Added Cargo.lock to version control. Details here and here.

Cookbook

The cookbook the collection of package recipes of Redox.

@jackpot51 Added a status.sh script for checking git modifications. Details here,here and here.
@jackpot51 Added a update.sh script. Details here.
@jackpot51 Updated the ion recipe to use a custom branch with simpler signal handling and afterwards switched back to master as the simpler signals work was mainstreamed into Ion. Details here and here respectively.
@ids1024 Added a recipe for GNU grep. Details here and here.
@ids1024 Made a patch in the git to use ; as PATH separator. Details here.
@ids1024 Added a recipe for diffutils. Details here.
@jackpot51 Made a change to cook.sh so it updates the source and not the build directory as Cargo.lock is now committed. Details here.
@jackpot51 Made a change to update.sh to check for source and not for build as Cargo.lock is now committed. Details here.

Handy links

New contributors

Since the list of contributors are growing too fast, we’ll now only list the new contributors. This might change in the future.

Sorted in alphabetical order.

Agoston Szepessy 🎂
Brayden Banks 🎂
Bryan Blanchard 🎂
Danny 🎂
fengalin 🎂
Fredrik 🎂
friend 🎂
garasubo 🎂
Grant Miner 🎂
m4b 🎂
memoryleak47 🎂
Michel Boaventura 🎂
Niv 🎂
Richard Palethorpe 🎂

If I missed something, feel free to contact me (goyox86) or send a PR to Redox website.