In the last section a method to communicate between threads
was implemented (in rendezvous.rs), and used to add a
read syscall so that user programs can get keyboard input.
The way this works is: When a key is pressed whichever thread is
running (thread A) is interrupted, and the keyboard handler calls
send on the rendezvous. In most cases there is already a thread
waiting (thread B), so send returns the Box<Thread> corresponding
to thread B, which the keyboard handler passes to
process:schedule_thread. The keyboard handler then returns to
thread A, and a little time later the timer interrupt stops thread A
and switches context to thread B.
What’s happened to thread B is curious: It called the kernel with a
syscall, but returns from that call with an iret from the timer
interrupt handler. This shows that our sysret and interrupt handlers
treat contexts consistently, and that we can use iret to restart any
thread. Note that the other way around doesn’t work: We can’t
interrupt a thread and then return to it with a sysret instruction
because sysret uses the rcx and r11 registers. The interrupted
thread would, as far as it is concerned, suddenly have two of its
registers unexpectedly modified.
The easier part to speed up is removing the hlt instruction in the
sys_receive function (syscalls.rs). If the calling thread blocks
(waiting for a message) then all we need to do is find the next
thread, and then jump into the middle of the interrupt_wrap! macro
(interrupts.rs), as if we had returned from the timer interrupt.
We can do that by just copying the assembly starting pop r15 and
ending iret, in a new function launch_thread() in interrupts.rs:
pub fn launch_thread(context_addr: usize) -> ! {
unsafe {
asm!("mov rsp, rdi", // Set the stack to the Context address
"pop r15",
"pop r14",
"pop r13",
"pop r12",
"pop r11",
"pop r10",
"pop r9",
"pop r8",
"pop rbp",
"pop rsi",
"pop rdi",
"pop rdx",
"pop rcx",
"pop rbx",
"pop rax",
"sti", // Enable interrupts
"iretq",// Interrupt return
in("rdi") context_addr,
options(noreturn));
}
}This takes the address of the Context as input, sets the stack pointer
(rsi) to that address, pops the registers and then returns via
iret. That iret restores the instruction pointer, RFLAGS, CS and
SS registers because the Context contains an exception frame even if
it was created in a syscall.
Now in syscalls.rs the sys_receive() function can be
modified, to first get the next thread from the scheduler,
and then call launch_thread to run it:
use crate::interrupts; // New
fn sys_receive(context_ptr: *mut Context, handle: u64) {
...
if !returning {
let new_context_addr =
process::schedule_next(context_ptr as usize);
interrupts::launch_thread(new_context_addr);
}
}That’s it! The schedule_next() function already handles changing page tables,
the kernel stack in the TSS, and the CURRENT_THREAD.
… or almost it. One thing to watch out for is that the
launch_thread process never returns. Even though it’s marked as
no-return (! return type), the compiler doesn’t insert code to drop
variables in the current scope. That means that the Rendezvous handle
rdv which is an Arc<RwLock<Rendezvous>> is not dropped, the Arc
strong count is not decreased but increases every time this
launch_thread path through sys_receive is followed. This will lead
to a memory leak as the Rendezvous will not be free’d when all
processes holding it exit. To fix this we can do:
use core::mem::drop;
...
if !returning {
drop(rdv); // new
let new_context_addr =
process::schedule_next(context_ptr as usize);
interrupts::launch_thread(new_context_addr);
}The implementation of drop is quite neat: It’s just
pub fn drop<T>(_x: T) { }so the ownership of rdv is transferred to this function
and then dropped.
The keyboard interrupt handler uses the x86-interrupt calling
convention, which saves us some work but doesn’t capture a context in
the same way as our timer interrupt or syscall handlers. To switch to
the thread which receives the keyboard message, rather than returning
to the interrupted thread, we need to get that context.
Fortunately we already have a macro that will do this for us:
interrupt_wrap!. In interrupts.rs replace:
extern "x86-interrupt" fn keyboard_interrupt_handler(
_stack_frame: InterruptStackFrame)with
interrupt_wrap!(keyboard_handler_inner => keyboard_interrupt_handler);
extern "C" fn keyboard_handler_inner(context_addr: usize)
-> usize {
...
0 // New
}and everything should still work as before. The handler returns 0 so the stack isn’t modified and it returns to the original thread.
Then we can change the end of this function to decide whether to return to the interrupted thread, or schedule another:
...
let next_context = if returning {context_addr} else {
// Schedule a different thread to run
process::schedule_next(context_addr)
};
unsafe {
PICS.lock()
.notify_end_of_interrupt(InterruptIndex::Keyboard.as_u8());
}
next_context
}This should work if you type at a reasonable speed, but if you mash the keyboard you’ll find a page fault with error code USER_MODE | INSTRUCTION_FETCH. By adding print statements you can see that this is happening because of this sequence of events:
- Keyboard interrupts thread 1, it’s context is written to the keyboard interrupt handler stack (GDT index 0). Control is passed to thread 2 which was waiting.
- Thread 2 is interrupted before it can call sys_receive again. It’s context is written to the keyboard handler stack, overwriting thread 1’s context.
- Soon thereafter thread 1 is run again. Unfortunately its context still points to the keyboard interrupt stack, which has been overwritten by thread 2’s context, so now has the wrong instruction pointer.
The fix is quite simple: In gdt.rs change KEYBOARD_INTERRUPT_INDEX
from 0 to 1, so it’s the same as the timer interrupt index and is
unique to each thread. We need to remember to use the stack at GDT
index 1 for any interrupt where we might switch contexts. The page
fault handler is ok (for now) because it either returns to the same
thread (e.g on-demand paging), or the thread will be stopped and not
restarted.
We now have a user space program that can quite efficiently receive input from the keyboard via messaging. In the next section we’ll enable user programs to send messages to write to the screen.