Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

c18n: Export c18n statistics to procstat(1) and file (supersedes #2084) #2079

Merged
merged 4 commits into from
Jun 4, 2024

Conversation

dpgao
Copy link
Contributor

@dpgao dpgao commented Apr 6, 2024

Edit: The c18n statistics part of this PR is a bit stalled, so I pulled out the interrupt-safe changes to #2090 which will hopefully get merged soon.

This PR builds upon #2012 and #2032 and the real content is in the very last commit entitled c18n: Rework implementation to be interrupt-safe. This is not meat to be merged but is a stable implementation needing feedback. I do hope it can be merged in the next release if time permits.

This commit completely refactors the trampoline and how stack switching works. The purecap and benchmark ABI implementations now both use a dedicated register to store the trusted stack (ddc and rddc respectively). This makes the trampolines look identical (modulo register names) on both ABIs. No metadata recording the current top of the stack is stored at the bottom of each compartment's stack. Instead, the stack lookup table now stores that information.

The signal handling mechanism has been rewritten to handle (rare) cases where c18n code, in particular trampolines, is interrupted. All c18n code paths that could be interrupted have been audited and it is believed that they can all be handled correctly, although testing for that is hard.

This PR modifies #2080 to export a slightly different set of statistics:

  • Number of compartments
  • Number of compartment stacks
  • Number of trampolines
  • Number of trampoline pages
  • Memory space taken up by c18n data structures
  • Number of compartment switches so far

It allows exposes environment variable LD_COMPARTMENT_STATS that allows a struct containing the above statistics to be exported to a file.

@dpgao dpgao force-pushed the c18n-nobot branch 6 times, most recently from 874a94c to 0ecef41 Compare April 10, 2024 17:01
@dpgao dpgao changed the title c18n: [WIP] Do not store stack metadata at its bottom c18n: Rework implementation to be interrupt-safe Apr 10, 2024
@dpgao dpgao force-pushed the c18n-nobot branch 2 times, most recently from 9680cbd to 773358e Compare April 11, 2024 17:14
@dpgao dpgao force-pushed the c18n-nobot branch 2 times, most recently from e195871 to c0da4f1 Compare April 19, 2024 15:45
libexec/rtld-elf/rtld_c18n.c Show resolved Hide resolved
sys/cheri/cheri.h Outdated Show resolved Hide resolved
sys/kern/kern_proc.c Outdated Show resolved Hide resolved
sys/kern/kern_proc.c Outdated Show resolved Hide resolved
struct proc *p;
struct cheri_c18n_info info;
int error;
void *buffer;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you initialize this to NULL you don't need two labels for the exit path.

sys/sys/imgact.h Outdated Show resolved Hide resolved
sys/kern/kern_proc.c Show resolved Hide resolved
sys/sys/elf_common.h Outdated Show resolved Hide resolved
sys/sys/sysctl.h Outdated Show resolved Hide resolved
lib/libprocstat/libprocstat.c Outdated Show resolved Hide resolved
@dpgao dpgao force-pushed the c18n-nobot branch 6 times, most recently from b4b5479 to 8c7be11 Compare April 24, 2024 14:23
Comment on lines 387 to 399
/*
* Error handling here is wrong. If ENOEXEC, really want to print
* output indicating no information, which this function signature
* doesn't currently support. This is because the process probably
* simply doesn't have c18n in use
*/
name[0] = CTL_KERN;
name[1] = KERN_PROC;
name[2] = KERN_PROC_C18N;
name[3] = kp->ki_pid;
error = sysctl(name, nitems(name), *pp, lenp, NULL, 0);
if (error != 0 && errno != ESRCH && errno != EPERM &&
errno != ENOEXEC) {
warn("sysctl(kern.proc.c18n)");
goto out_free;
}
if (error != 0)
goto out_free;
return (0);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rwatson Do we need to fix the error handling here?

sys/kern/kern_proc.c Show resolved Hide resolved
@dpgao dpgao changed the title c18n: Rework implementation to be interrupt-safe c18n: [DRAFT] Rework implementation to be interrupt-safe Apr 24, 2024
@dpgao dpgao changed the title c18n: [DRAFT] Rework implementation to be interrupt-safe c18n: Export c18n statistics to procstat(1) and file (superseced #2084) Jun 3, 2024
Copy link
Member

@brooksdavis brooksdavis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to include the environment variable version once procstat is available?

Please fix the "line over 80 characters" style warnings.

sys/sys/proc.h Outdated Show resolved Hide resolved
sys/kern/sys_process.c Outdated Show resolved Hide resolved
@dpgao
Copy link
Contributor Author

dpgao commented Jun 3, 2024

Does it make sense to include the environment variable version once procstat is available?

Yes. The env var version allows the stats to persist after the process exits, which will be useful for benchmarks.

Copy link
Contributor

@gvnn3 gvnn3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nits mostly.

lib/libprocstat/libprocstat.c Show resolved Hide resolved
lib/libprocstat/libprocstat.h Outdated Show resolved Hide resolved
libexec/rtld-elf/rtld_c18n.c Show resolved Hide resolved
dpgao and others added 4 commits June 4, 2024 20:09
Exposes LD_COMPARTMENT_STATS that exports a set of
compartmentalisation-related statistics to a user-specified file.
Co-authored-by: Dapeng Gao <dapeng.gao@cl.cam.ac.uk>
@dpgao dpgao merged commit 97df9be into dev Jun 4, 2024
7 checks passed
@dpgao dpgao deleted the c18n-nobot branch June 4, 2024 19:11
@@ -400,6 +400,13 @@ TRAMP(tramp_update_fp_untagged)
clrtag c29, TRUSTED_STACK_C
TRAMPEND(tramp_update_fp_untagged)

TRAMP(tramp_count_entry)
1: ldr c24, #0 /* To be patched at runtime */
stadd w25, [c24]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does this w25 come from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the gctag x25, c30 above and we can assume that it is always 1. This is done to save instructions and assumes that the program is well-behaved.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Firstly, these things have to be commented. There are so many of these implicit dependencies floating around within and across trampoline fragments that it's really hard to keep track of them all, and it's just asking for someone to break this in future by using a different register for the GCTAG (which would be reasonable, because who would expect it to be used many instructions later?). Secondly, having an untagged return capability seems a totally valid thing to do if you're making a call to a function that won't return. I don't think you can currently easily end up with it from a compiler, but I see no reason why it wouldn't be valid.

@dpgao dpgao changed the title c18n: Export c18n statistics to procstat(1) and file (superseced #2084) c18n: Export c18n statistics to procstat(1) and file (supersedes #2084) Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants