Bochs on Redox OS (and QEMU - Part 2)

By Enygmator on

Prologue

Hey there everyone! I’m Enygmator and I’m back!

TLDR; The technical stuff starts from the Emulators on Redox section.


For those of you who don’t know me (or my role in Redox), I’d like to introduce myself as the RSoC (Redox Summer of Code) 2021 student who worked on porting QEMU to Redox OS.

This post brings you some updates on the progress I’ve made on that front (QEMU on Redox) and another juicy undertaking of mine - It’s BOCHS on Redox y’all! 😀

I’m writing this post as an UPDATE to my previous post (refer post - RSoC 2021: QEMU on Redox OS - Part 1) that I wrote last year, as part of RSoC 2021, where I promised an update (which is long due). This post is going to be more straightforward and technical than the last one, so if you aren’t fully aware of QEMU, OSes, emulators, userspace program porting, etc, you should definitely read that post. I explain from the basic in simple language, before getting more technical.
You should also read it if you’re interested in how I started out porting QEMU to Redox OS.

It has been one heck of a year and despite my preoccupation with my undergraduate degree (BTech - CSE, 2023), I was able to put in some amount of time after the duration of RSoC 2021, to further develop the project I was working on (QEMU, and later Bochs). I worked on the stuff mentioned in this post a while back, but am writing this post only now (procrastination). This post is not a part of either RSoC 2021 or RSoC 2022, just an update to spice things up.

BTW, I’m currently working on Virtualization on Redox OS as part of RSoC 2022.
Yep! Virtualization support is coming to Redox OS soon! The feature is called Revirt and the project plan and implementation details are documented in the two posts - “Revirt - Virtualization on Redox OS” AND “RSoC 2022: Revirt-U - Part 1”.


Updates

If I make any updates to this post (which is sometimes helpful for people reading it in the future), I shall make note of it here and st the same time, scratch out older content and replace it with something like “UPDATE1: …”.



Emulators on Redox

Emulators are applications that provide ISA-level virtualization (one among other types of virtualization). If an emulator emulates an ISA which is the same as the ISA of the hardware the emulator is running on, it is also known as Hardware-level Virtualization (example: Bochs)
Though, do note that ‘Hardware-level virtualization’ is not the same as ‘Hardware-assisted virtualization’. The latter is a special type of the former, where ‘hardware-assisted’ means ISA-support (hardwired) for virtualization at the ‘hardware-level’


Having emulators like QEMU and Bochs successfully run on Redox OS, helps realize the support (tooling and ecosystem) required for self-hosting, and at the same time enabling support for specific use cases for Redox OS.

Self-hosting isn’t a long way off on Redox, and tooling around a new software/hardware that makes it easier to work with pretty much decides how successful the software/hardware will be. (as suggested by Linus Torvalds, where he said that he loves x86 more than arm because of the amazing ecosystem around x86 that has been built over the past decades).

While I was attempting to port QEMU, I thought of looking into porting Bochs too, given that its codebase looked much simpler (given that it only emulates x86 and some peripherals). Here, I write about both those attempts of mine (and their results).

QEMU

You can find all the work I did at enygmator/qemu GitLab Repo - qemu_for_redox branch.

QEMU has its own build system based on MAKEFILES and a meta-build system based on meson that is capable of compiling to Linux, MacOS and Windows targets. I created a new target called redox, which disables all features of QEMU by default and also the tests in meson.build. Then I manually enable only a subset of the features (for qemu-system-x86_64) to run.

Using preprocessor checks like #if defined(__redox__), some of the alternative code was written.

Build, make and install scripts were configured in recipe.sh. It also had to be modified to recursively link the dependencies of each dependency statically (using LD_FLAGS).
TODO: I need to figure out a way to make that happen automatically based on pkg-config info.


Result

With a required number of features enabled (only to successfully build the emulator for x86_64-softmmu), QEMU is successfully compiling for Redox OS (including the static dependencies).

TODO: But it fails to execute - most likely because I may have written some incorrect placeholder code causing it to go into an infinite loop. Fixing this should be easier now that I have spent time understanding the kernel internals better and can use debuggers to track code execution.

I’ll have an update on QEMU when qemu-system-x86_64 runs for the first time.


TLDR; Working on the code

The recipe.sh file is here - on the qemu_on_redox branch of enygmator/cookbook repo.

Feature configuration is done in the recipe.sh file, which is used to build the QEMU package for Redox, which can later be installed on Redox. The recipe.sh file currently depends on the QEMU source being located at try_redox/redox_apps/qemu-7.0.0 (where the Redox OS repo is located at try_redox/redox). This is used to run the VPATH build (done by commands in recipe.sh).

If you want complete instructions on compiling qemu-7.0.0 on Redox OS, you’ll find them in the README.md, here - on the qemu_on_redox branch of enygmator/cookbook repo. DO contact me if you have questions, suggestions or face issues. I may be able to help. (or you can file an issue in one of repo links - cookbook - GitLab Issues or qemu - GitLab Issues)

Bochs

Bochs 2.7 seemed like a good emulator to port, as it is so much more lighter than QEMU, due to the fact that it only emulates x86 ISA with a certain number of peripherals, and doesn’t support KVM either. So, I started to work on porting it here - on the bochs_on_redox branch of enygmator/bochs on GitLab.

Bochs uses Makefile for it’s build system, and similar to QEMU, it uses the VPATH build method (which ensures that all build files are worked on in a separate directory that you can manually specify, so that the source repo isn’t touched in any manner at all).

My experience with porting QEMU (incompletely) came in very handy, as within 6 hours I actually was able to get Bochs to statically compile! albeit with some ‘extra’ features turned off. There were some #include changes with respect to the SDL and SDL2 libraries and some ‘tricky’ method to force bochs to compile statically (since it’s build system was not doing it correctly). You can view the changes in the repo link above.

I then configured the make procedure and install sections of the recipe and booted into Redox and ran bochs with a super minimal “bootloader + kernel” that I had written back in 2020 (my first “OS project” 😊) loaded onto a ‘hard drive file’ and configured bochs emulator using a bochs.src bochs configuration file…. and it ran! You can see “stage 2” of my kernel in the below image:

Bochs GUI running on Redox

At a later point, I’ll enable and work on more features to make bochs work better. Though, the priority will always be QEMU as it is so much more important in terms of common/industrial usage and the functionality it offers, and a priority for emulation and Hardware-assisted Virtualization (which I’m working on as part of RSoC 2022).


Known Issues

  1. There seems to be some problem in the framebuffer mapping when the SDL2 window resizes, causing things to look “skewed”. (I fixed it in the image above by manually changing the mapping 😅)
  2. Bochs also crashes when the window size in increased by a huge amount.

TLDR; Working on the code

The recipe.sh file is here - on the bochs_on_redox branch of enygmator/cookbook on GitLab.

Feature configuration is done in the recipe.sh file, which is used to build the Bochs package for Redox, which can later be installed on Redox. The recipe.sh file currently depends on the Bochs source being located at try_redox/redox_apps/bochs (where the Redox OS repo is located at try_redox/redox). This is used to run the VPATH build (done by commands in recipe.sh).

If you want complete instructions on compiling bochs on Redox OS, you’ll find them in the README.md, here - on the bochs_on_redox branch of enygmator/bochs on GitLab. DO contact me if you have questions, suggestions or face issues. I may be able to help. (or you can file an issue in one of repo links - enygmator/cookbook - GitLab Issues or enygmator/bochs - GitLab Issues)



Epilogue

While writing this post, I made some updates to the previous post. While reading that, I cringed many times looking at some glaring errors I had made in understanding a few concepts (that I now understand much better) or the method by which I did something. But I guess that’s what learning and growth are about - every time you look back at your previous work, you think “I sure was an amateur back then! 😣”. Some day, I’ll look at this post and realize the same thing.

Contributing to Redox OS’s future is a lot of fun. I learnt about the different ways in which build systems are configured/constructed (and am already using them in my projects 🙂) in addition to cross-compilation and lots of other tid-bits. I want to thank those who have guided me in the Redox OS mattermost chat during RSoC 2021 and later (omac777, a4ldo2 and of course jackpot51).

I shall soon write my next post (as an RSoC 2022 student), so hang on tight people!

Until later!

If you’ve reached till this point, thank you for reading. I shall be coming back sometime next month(UPDATE1) with more exciting updates!

Until then, keep well.
Bye! 💕💕💕


You can find me on: