Add book

book/.gitignore

@@ -0,0 +1 @@
+book

book/book.toml

@@ -0,0 +1,6 @@
+[book]
+authors = ["Matt Staveley-Taylor"]
+language = "en"
+multilingual = false
+src = "src"
+title = "mlibc User Guide"

book/src/README.md

@@ -0,0 +1,5 @@
+# Introduction
+
+Welcome to the [`mlibc`](https://github.com/managarm/mlibc) User Guide.
+
+[`mlibc`](https://github.com/managarm/mlibc) is a fully featured C standard library designed with portability in mind. It provides a clean syscall abstraction layer for new operating system ports to plug into. This guide will explain how to integrate a new OS / syscall backend and get your first program up and running.

book/src/SUMMARY.md

@@ -0,0 +1,11 @@
+# Summary
+
+[Introduction](README.md)
+
+- [Adding a new OS port](porting/intro.md)
+    - [Kernel prerequisites](porting/kernel_prerequisites.md)
+    - [Choosing a compiler](porting/choosing_a_compiler.md)
+    - [Implementing sysdeps](porting/implementing_sysdeps.md)
+    - [Upstreaming your port](porting/upstreaming.md)
+    - [Next steps](porting/next_steps.md)
+- [Adding a new ISA port]()

book/src/porting/choosing_a_compiler.md

@@ -0,0 +1,26 @@
+# Choosing a compiler
+
+To compile [`mlibc`](https://github.com/managarm/mlibc) and any userspace programs which link against it, you'll need a suitable compiler. Roughly speaking you have two choices:
+
+1. **Build a full [OS Specific Toolchain](https://wiki.osdev.org/OS_Specific_Toolchain).**
+
+    With this, you'll have a compiler that implicitly links against (m)libc, so we recommend doing this _after_ you have a basic port working.
+
+1. **Use a generic ELF toolchain.**
+
+    If compiling your kernel with a generic toolchain like `x86_64-unknown-elf-gcc` or `riscv64-elf-gcc`, you may re-use the same compiler for this.
+
+    Note that when compiling userspace programs, you must manually provide a number of command line arguments (such as the location of your compiled libc and any headers).
+
+
+For [`mlibc-demo-os`](https://github.com/64/mlibc-demo-os), we'll rely on a generic `riscv64-elf-gcc` toolchain provided by our distro.
+
+## Creating a Meson cross file
+
+[`mlibc`](https://github.com/managarm/mlibc) uses a build system called [Meson](https://mesonbuild.com/). When cross-compiling, you must tell meson about your compiler and target via a [_cross file_](https://mesonbuild.com/Cross-compilation.html).
+
+For [`mlibc-demo-os`](https://github.com/64/mlibc-demo-os), our cross file looks like this:
+
+```toml
+{{#include ../../../ci/demo.cross-file}}
+```

book/src/porting/implementing_sysdeps.md

@@ -0,0 +1,148 @@
+# Implementing your sysdeps
+
+## Telling mlibc about your sysdeps
+
+In mlibc's top level `meson.build` file, you'll see a large `if`/`else` chain that selects the right sysdeps subdirectory based on the system name. We'll add ourselves here:
+```meson
+{{#include ../../../meson.build:demo-sysdeps}}
+    # ...
+```
+
+Now, create the `sysdeps/demo` folder. At minimum it should contain a `meson.build` file where we'll declare all the `.cpp` files and select header files to install:
+
+```meson
+{{#include ../../../sysdeps/demo/meson.build:sources-and-includes}}
+```
+
+The `sysdep_supported_options` tells mlibc which 'options' your sysdeps supports. For example, the `unistd.h` header is only available when the POSIX option is enabled.
+
+## Preparing to build mlibc
+
+Now, from a fresh clone of `mlibc` you can run:
+
+```
+$ meson setup --cross-file=path/to/your.cross-file -Ddefault_library=static -Dbuildtype=debug build
+```
+
+The `-Ddefault_library=static` tells meson to only produce a statically linked library (`libc.a`). We *strongly* recommend getting statically linked binaries to work before dynamically linked ones.
+
+Now run `ninja -C build` to start the build. You'll likely get a large number of compilation errors at this point, and we'll work on fixing these in the next sections.
+
+## Selecting ABI headers
+
+Many of the compile errors are because we need to tell mlibc about the ABI between our sysdeps and kernel. For example, we must define the layout of `struct stat`.
+
+We _strongly_ recommend re-using an existing ABI where possible rather than creating your own, because many programs assume a particular ABI and will fail to compile or subtly break if you choose a different one.
+
+For the demo OS, we'll symlink each ABI header to Linux's definition in `sysdeps/linux/include/abi-bits`. Then, we'll add the following to our `meson.build`:
+
+```meson
+{{#include ../../../sysdeps/demo/meson.build:abi-includes}}
+```
+
+This is not an exhaustive list of ABI headers but should be enough to get started.
+
+## Implementing (S)crt1
+
+Next, we must provide a definition of the ELF entrypoint (traditionally named `_start`). Each ISA tends to require its own definition written in assembly; for example RISC-V targets must initialise the `gp` register before jumping to C++.
+
+Traditionally the file that defines `_start` is called `crt1.S` (or `Scrt1.S` for position independent executables). This produces an object file which has to be linked into each application.
+
+Part of configuring an [OS Specific Toolchain](https://wiki.osdev.org/OS_Specific_Toolchain) is [specifying the location of `(S)crt1.o`](https://wiki.osdev.org/OS_Specific_Toolchain#Start_Files_Directory) so that it linked automatically, but generic ELF targets must link it explicitly.
+
+We recommend copying `(S)crt1.S` from an existing target like Linux. Then, we'll compile it by adding the following to our `meson.build`:
+
+```
+{{#include ../../../sysdeps/demo/meson.build:crt1}}
+```
+
+## Implementing a C++ entry point
+
+Now, we'll implement the C++ entry point that we call from `crt1.S`. 
+
+```cpp
+{{#include ../../../sysdeps/demo/entry.cpp}}
+```
+
+The call to `__dlapi_enter` is used to perform initialisation in statically linked executables (but is a no-op in dynamically linked ones). For example, any global constructors in the program will be called from here.
+
+The `LibraryGuard` is used to perform initialisation of `libc` itself like parsing the environment and arguments from the kernel. Putting this code in a global constructor ensures that it's called at the right time relative to other pieces of initialisation.
+
+## Performing system calls
+
+The final piece of infrastructure we require is the ability to invoke system calls.
+
+For example, on RISC-V a system call is invoked via the `ecall` instruction and requires putting arguments in specific registers, which requires a bit of (inline) assembly. We recommend copying this glue from an existing target.
+
+For the demo OS, this is provided by `syscall.cpp` and `include/bits/syscall.h`.
+
+## Implementing sysdeps
+
+Finally we're ready to implement the actual sysdep functions. For a basic statically-linked hello world program, you'll need to provide definitions of the following sysdep functions:
+
+- `mlibc::sys_libc_panic`
+- `mlibc::sys_libc_log`
+- `mlibc::sys_isatty`
+- `mlibc::sys_write`
+- `mlibc::sys_tcb_set`
+- `mlibc::sys_anon_allocate`
+- `mlibc::sys_anon_free`
+- `mlibc::sys_seek`
+- `mlibc::sys_exit`
+- `mlibc::sys_close`
+- `mlibc::sys_futex_wake`
+- `mlibc::sys_futex_wait`
+- `mlibc::sys_read`
+- `mlibc::sys_open`
+- `mlibc::sys_vm_map`
+
+Note that many of these functions are declared as weak symbols. You _must_ include the relevant headers (e.g `<mlibc/all-sysdeps.hpp>`) before providing definitions otherwise these won't be emitted as weak symbols and you'll see strange errors.
+
+Most sysdep functions return an integer error code (0 for success, -E for errors) and return data via out parameters. Note that sysdeps shouldn't set errno directly - mlibc will set it from the error code you return.
+
+As a general strategy, it's a good idea to stub whatever's required to make things compile, and then add proper implementations later. For example:
+
+```cpp
+{{#include ../../../sysdeps/demo/sysdeps.cpp:stub}}
+
+namespace mlibc {
+int sys_close(int fd) { STUB(); }
+}
+```
+
+## Compiling a test program
+
+At this point, you should be able to compile and link mlibc itself. Congratulations!
+
+Now we'll compile a simple hello world program that we can run on our kernel:
+
+```bash
+# Install mlibc to a temporary directory
+DESTDIR=/tmp/mlibc-install ninja -C build install
+
+# Some targets require an implementation of libgcc (if you see linker errors below).
+# We recommend using cc-runtime: https://github.com/osdev0/cc-runtime
+
+riscv64-elf-gcc \
+    -static -nostdinc -nostdlib -g \
+    -I /tmp/mlibc-install/include \
+    /tmp/mlibc-install/lib/crt1.o \
+    hello_world_program.c \
+    /tmp/mlibc-install/lib/libc.a \
+    /path/to/cc-runtime.a \
+    -o hello_world_program
+```
+
+## Troubleshooting
+
+Something not working? Here are some common issues to look out for:
+
+- Your kernel isn't loading the ELF file correctly. Double check that you're loading the contents from file to the right addresses and zeroing the uninitialised portions.
+- Your `sys_anon_allocate` function is broken. Try commenting out `sys_anon_free` and see if that helps. Try printing the addresses to see if you're getting unique allocations.
+- You're pushing the arguments/environment/auxiliary vectors to the stack wrong. Double check you're pushing in the right order and everything is properly aligned.
+- You're not saving and restoring the userspace state properly when a syscall happens.
+- Your `syscall` glue is passing arguments in the wrong registers. Double check that the kernel and userspace agree on the order of arguments.
+
+When stuck, we recommend using GDB to step through the program.
+
+If all else fails, feel free to hop in [the Managarm Discord server](https://discord.gg/7WB6Ur3) and ask for help in `#mlibc-dev`.

book/src/porting/intro.md

@@ -0,0 +1 @@
+# Adding a new OS port

book/src/porting/kernel_prerequisites.md

@@ -0,0 +1,11 @@
+# Kernel prerequisites
+
+Before you attempt to port [`mlibc`](https://github.com/managarm/mlibc), ensure your kernel supports these things:
+- Writing text to the screen or a serial port
+- Basic paging and virtual memory operations
+- Loading an ELF program, mapping a stack and jumping to the entrypoint
+- Handling syscalls from userspace
+
+For this port, we'll use [`mlibc-demo-os`](https://github.com/64/mlibc-demo-os), a small RISC-V kernel which supports the minimum functionality required for a mlibc 'hello world' program. Refer to this if you get stuck!
+
+Note that a filesystem and storage drivers are not strictly necessary. In [`mlibc-demo-os`](https://github.com/64/mlibc-demo-os), we `include_bytes!` the contents of the user-space program into the kernel's executable. To support multiple files, you could include a trivial read-only filesystem like `tar` instead.

book/src/porting/next_steps.md

@@ -0,0 +1,3 @@
+# Next steps
+
+- implement more sysdeps, enable more options

book/src/porting/upstreaming.md

@@ -0,0 +1,9 @@
+# Upstreaming your port
+
+Once your port is reasonably stable, feel free to submit it to us upstream. This ensures that your sysdeps won't be broken by any internal refactorings.
+
+Though we won't be able to test your kernel on our CI, we require you add your port to the 'Compile sysdeps' GitHub action which checks that compilation succeeds.
+
+It's a good idea to include a `.clang-format` file so that any changes we make to your code will be formatted to your liking.
+
+See the [pull request adding Astral sysdeps](https://github.com/managarm/mlibc/pull/1136) for an example to follow.

meson.build

@@ -259,11 +259,11 @@ elif host_machine.system() == 'astral'
 	internal_conf.set10('MLIBC_MAP_DSO_SEGMENTS', true)
 	internal_conf.set10('MLIBC_MAP_FILE_WINDOWS', true)
 	subdir('sysdeps/astral')
+# ANCHOR: demo-sysdeps
 elif host_machine.system() == 'demo'
-	rtld_include_dirs += include_directories('sysdeps/demo/include')
-	libc_include_dirs += include_directories('sysdeps/demo/include')
 	subdir('sysdeps/demo')
 else
+# ANCHOR_END: demo-sysdeps
 	error('No sysdeps defined for OS: ' + host_machine.system())
 endif
 

sysdeps/demo/meson.build

@@ -1,3 +1,4 @@
+# ANCHOR: sources-and-includes
 sysdep_supported_options = {
 	'posix': false,
 	'linux': false,
@@ -8,13 +9,17 @@ sysdep_supported_options = {
 rtld_sources += files(
 	'sysdeps.cpp',
 )
+rtld_include_dirs += include_directories('sysdeps/demo/include')
 
 libc_sources += files(
 	'entry.cpp',
 	'sysdeps.cpp',
 	'syscall.cpp',
 )
+libc_include_dirs += include_directories('sysdeps/demo/include')
+# ANCHOR_END: sources-and-includes
 
+# ANCHOR: abi-includes
 if not no_headers
 	install_headers(
 		'include/abi-bits/auxv.h',
@@ -41,7 +46,9 @@ if not no_headers
 		follow_symlinks: true
 	)
 endif
+# ANCHOR_END: abi-includes
 
+# ANCHOR: crt1
 if not headers_only
 	crt = custom_target('crt1',
 		build_by_default: true,
@@ -52,3 +59,4 @@ if not headers_only
 		install_dir: get_option('libdir')
 	)
 endif
+# ANCHOR_END: crt1

sysdeps/demo/sysdeps.cpp

@@ -9,11 +9,13 @@
 #define SYS_WRITE 1
 #define SYS_MMAP 2
 
+// ANCHOR: stub
 #define STUB()                                                                 \
   ({                                                                           \
-    __ensure(!"STUB_ONLY function was called");                                \
+    __ensure(!"STUB function was called");                                     \
     __builtin_unreachable();                                                   \
   })
+// ANCHOR_END: stub
 
 namespace mlibc {
 
@@ -35,7 +37,7 @@ int sys_isatty(int fd) {
 
 int sys_write(int fd, void const *buf, size_t size, ssize_t *ret) {
   *ret = syscall(SYS_WRITE, fd, buf, size);
-  return ret == 0 ? 0 : -1;
+  return *ret >= 0 ? 0 : -1;
 }
 
 int sys_tcb_set(void *pointer) {
@@ -55,7 +57,9 @@ int sys_anon_allocate(size_t size, void **pointer) {
 
 int sys_anon_free(void *, unsigned long) { return 0; }
 
-int sys_seek(int fd, off_t offset, int whence, off_t *new_offset) { return 0; }
+int sys_seek(int fd, off_t offset, int whence, off_t *new_offset) {
+  return ESPIPE;
+}
 
 void sys_exit(int status) {
   syscall(SYS_EXIT, status);