GSoC Project: Making Redox Self-hosting, Status Report 4

By ids1024 on Tuesday, August 15, 2017

This is a continuation of status report 1, status report 2, and status report 3.

Backtraces

I found the lack of Rust backtraces on Redox inconvenient for an issue I was debugging, so I decided to try to get them working. So I fixed them in Rust PR #43635. It was fairly simple, but I had to figure out various issues. I had to modify Rust to build libbacktrace on Redox, and update the config.sub in libbacktrace to support Redox (that’s an autotools thing). It needed to link against libgcc to get the unwinding symbols, and I needed to copy the backtrace code from the Unix backend to the Redox one, and implement get_executable_filename using Redox’s APIs.

With that, Rust backtraces are now functional on Redox.

Uutils

Uutils is a work in progress Rust implementation of Unix coreutils. Redox has been using it’s own coreutils, but some of the uutils ones are better. I’ve added uutils to the default Redox install, and added symlinks for some of the utilities, namely expr, install, and mktemp, which Redox did not have, and chmod, env, and ls, which had fewer features than the uutils versions.

My goal here has mainly been to get autotools-based builds working, which requires various coreutils utilities, and the uutils versions have helped.

I’ve also had to send fixes to uutils: the chmod fix mentioned in my last report, a patch making ls work on Redox, and adding Redox support to mktemp and install.

Additionally, I came across a few bugs in uutils, and submitted fixes; ls -t was backwards, cp -r was misbehaving, and verbose mode wasn’t working in install.

Those uutils utilities also required Redox patches to atty, termsize, tempfile, and rust-pretty-bytes.

Autotools, and the Cookbook

I have gotten basic autotools builds to work; although it requires export LD=ld (the detection code requires an absolute path, and a Redox path starting with file: isn’t recognized as absolute). Some things fail to build due to other issues, including other issues related to Redox’s schemes.

One annoying thing with autotools is that config.guess needs to have code for the host system. I sent a patch upstream, so in the future software should contain the new config.guess, but for now we need to override this.

./configure scripts triggered a weird bug in redoxfs, where large numbers of unlinks resulted in corruption. That was hard to debug, but eventually I found the cause and send a PR.

To get the cookbook to work, I ported bash, which it currently requires, although it will likely use Ion in the future. I implemented the F_DUPFD flag of fcntl, which bash and dash use; I had a hack in dash, but that was rather ugly so I implemented it properly.

For the cookbook to be able to extract source tarballs, I ported xz/liblzma (there doesn’t yet seem to be a pure Rust library that supports .xz files), and added gzip and xz extraction support to tar, along with verbose mode; I later added --directory and --strip-components support, which cook.sh also uses, as well as bzip2 extraction, which also pulls in C code for the time being. I also made tar set file times, and ported patch to make the cookbook work. And I modified Redox’s wget utility to match the standard command line.

I have successfully built packages for lua and nasm (the latter of which uses autotools) using the cookbook from inside Redox, although I’m currently using a couple changes that haven’t been merged into uutils yet, and export LD=ld is still needed as mentioned above. The native gcc toolchain seems to be working quite nicely, although the scripting around it can be complicated.

Other fixes

I fixed a bug in redoxfs fuse mounts with large directories, and made it posible to set mtime to an earlier time in fuse. I fixed directory symlinks, which I somehow forgot about when I originally implemented symlinks. Redox was not matching POSIX behavior for file descriptors; when dup is called, the new fd should still refer to the same “file description”, which Redox did not have a concept of. I modified the kernel to match POSIX behavior.

I changed Redox’s target triple (for the gcc toolchain) from x86_64-elf-redox to x86_64-unknown-redox, with a PR in libc and the cookbook. The Rust toolchain was already *-unknown-*. Frankly it doesn’t make much sense (has the person building the toolchain forgotten who they are?), but it is conventional.

Rustc

I’ve tried to merge upstream changes into Redox’s native rustc, but it seems it is now pulling in jobserver-rs, so it has the same issue as cargo. Both should work once interruptible system calls are implemented (i.e. making read (and other calls) return EINTR early when a signal handler runs).

Pthreads

Pthreads (the POSIX threads library) is not really related to self-hosting, but quite important for porting C software. I have started porting threads-emb to Redox; in theory all that’s left is implementing semaphores and cancellation, but those are somewhat awkward, and other issues may appear in testing.