How are System Calls implemented in gem5 x86 Full System mode?

[ballsmahoney]

Hi Ciro,

I am confused how system calls are called and returned in Full System and was wondering what is the intercepting point? More specifically, where does gem5 pass the system call handling to the kernel, and where does it receive the system call's result? For example, in x86 if FullSystemInt = 1, it just has sysenter() for the opcode that normally opens up the SE system mode handler:

       0x4: decode FullSystemInt {
            0: SyscallInst::sysenter('xc->syscall(&fault)',
                                     IsSyscall, IsNonSpeculative,
                                     IsSerializeAfter);
            default: sysenter();
        }

Thank you for your amazing guide. It has been extremely helpful.

[cirosantilli]

Hi,

Full system just simulates everything exactly like in hardware, unlike SE which fakes all of kernelland and manually reimplements all syscalls.

Therefore, in fullsystem userland calls an interrupt, and then this is intercepted by the guest kernel code (normally a real llinux kernel image given to gem5 with gem5.opt fs.py --kernel) which had previously setup a handler. Then the kernel image runs the syscall code exactly as it would in real hardware.

So the only thing fullsystem does of relevance for syscalls is make interrupts do exactly what they would do in real hardware, which is basically jump to the handler location.

So the only different thing would be to understand the gem5 code that generates interrupts, and the x86 specification of how handlers are setup.

[ballsmahoney]

Thank you for the reply. Most of the simulator code regarding System Calls are for SE mode. And x86 does not treat system calls like faults or hardware interrupts, correct? I believe the following routines in x86 assembly perform the system call return. However, I was wondering if there is any part of the C++/python interface that intercepts the system call returns, since from the look of the below code it does not. Is my understanding correct, that these things are taken care of in assembly? Is the best way to create an intercepting point by creating my own fault/interrupt and adding some lines of assembly to execute the respective routine?

add 
def macroop SYSRET_TO_64
{
    # All 1s.
    limm t1, "(uint64_t)(-1)", dataSize=8

    rdval t3, star
    srli t3, t3, 48, dataSize=8
    ori t3, t3, 3, dataSize=1

    # Set rflags to r11 with RF and VM cleared.
    limm t4, "~(RFBit | VMBit)", dataSize=8
    and t4, t4, r11, dataSize=8
    wrflags t4, t0

    # Set up CS.
    addi t4, t3, 16, dataSize=8
    wrsel cs, t4
    wrbase cs, t0, dataSize=8
    wrlimit cs, t1, dataSize=4
    # Not writable, read/execute-able, not expandDown,
    # dpl=3, defaultSize=0, long mode
    limm t4, ((3 << 0)  | (0  << 2)  | (0 << 3)   | \
              (1 << 4)  | (0  << 5)  | (1 << 6)   | \
              (1 << 7)  | (10 << 8)  | (0 << 12)  | \
              (1 << 13) | (0  << 14) | (1 << 15)), dataSize=8
    wrattr cs, t4

    # Only the selector is changed for SS.
    addi t4, t3, 8, dataSize=8
    wrsel ss, t4

    # Set the RIP back.
    wrip rcx, t0, dataSize=8
};

[cirosantilli]

And x86 does not treat system calls like faults or hardware interrupts, correct?

I think they do, I think x86 syscall instruction instruction calls an interrupt, and sysret leaves it. You could try to intercept FS interrupts from the simulator, but differentiating between syscalls interrupts and non syscalls interrupts would require further inspecting the state of registers/memory. Likely doable.

[winterNan]

I think in 64-bit mode syscall instruction does not generate a software interrupt. However to do this, the OS should set up the star and lstar MSRs during booting up. Such registers can only be modified by ring-0 code.

So to the question, the gem5 code in system_calls.py will be translated in the constructor of SYSCALL_64 (exists in decoder-ns.cc.inc), which one-to-one maps the assembly implementing the behavior of SYSCALL_64 into gem5 micro-ops.

So there is no software interrupt generated during syscall. What gem5 does is to go through the instructions created by SYSCALL_64 (the constructor), which is further determined by system_calls.py (<- this is implemented according to the behavior of the syscall instruction.)

The legacy way which indeed generates software interrupt (SYSCALL_LEGACY) is not implemented in 64-bit mode. And the switching between SYSCALL_64 and SYSCALL_LEGACY depends on the instruction mode.

Hope my understanding can help.