Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate docs for core and alloc locally #129

Closed
wants to merge 3 commits into from

Conversation

wusyong
Copy link

@wusyong wusyong commented Mar 20, 2021

Resolve #97
There are several places in core and alloc using allow(missing_doc) lint.
And this is not compatible with -Fmissing-docs.
So I moved the lint to individual crates.

@alex alex requested a review from ojeda March 20, 2021 14:11
@ojeda
Copy link
Member

ojeda commented Mar 20, 2021

I think we still want to forbid not having docs everywhere (i.e. instead of writing it in the code), so we could use deny instead of forbid or, even better, keep the flag everywhere except for core and alloc.

rust/Makefile Outdated
rustdoc: rustdoc-module rustdoc-compiler_builtins rustdoc-kernel
rustdoc: rustdoc-core rustdoc-module rustdoc-compiler_builtins rustdoc-alloc rustdoc-kernel

rustdoc-core: private skip_clippy = 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skip_clippy is not used for rustdoc. Did you add it for a particular reason?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I didnt aware that. Fixed it!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to say sorry! Thanks for working on this! :-)

@ojeda
Copy link
Member

ojeda commented Mar 20, 2021

I tested this, and it looks beautiful having core in there, and it does not take long to build, which is great.

However, many links inside core and alloc stop working because they assume std is available locally (e.g. see core::mem::transmute). Generating std wouldn't help because some go to the nomicon and other places, and we don't really want to regenerate everything on our end.

So I am not sure what to do. I think it is better that links work vs. not supporting offline use. We could hack the links manually, though, but not sure how reliable that would end up being, and targeting the official webpage means it is one less thing to worry about, at least for the moment.

Thoughts?

@wusyong
Copy link
Author

wusyong commented Mar 20, 2021

I think this is more like upstream issues. Tested a bit, and I also found there are some would link to online docs (core::array for example) and some wouldn't.
I'm not so sure about this behavior either. On the bright side, we will have local docs for core and alloc. But with some broken links though.

@ojeda
Copy link
Member

ojeda commented Mar 20, 2021

Thinking about this a bit more, I would prefer that all the links work for the moment.

In particular, since this will be shown to kernel developers new to Rust, they may be confused if they have never seen the standard library docs -- they may just think they are broken (rightfully so).

In other words, I would prefer saying "we don't have offline support for core and alloc (yet)" vs. saying "yeah, they are broken, please use your favorite search engine to reach the same page in the official docs and then click on the link read it there, sorry about that".

Furthermore, if they actually need to go to the official page to read those docs anyway, they will need to be online anyway, so...

I'm sorry because the PR is fine otherwise, and we will end up doing it at some point, but I think it would be a worse UX at the moment :-/

@ojeda
Copy link
Member

ojeda commented Mar 20, 2021

Another point is that, if I manage to upload this to kernel.org, then I expect many people will likely read it there vs. generating them on their own, so they would need online support anyway.

@wusyong
Copy link
Author

wusyong commented Mar 21, 2021

Yeah, I tend to agree on this. I'll close this PR and let the docs stay as is.

@wusyong wusyong closed this Mar 21, 2021
@wusyong wusyong deleted the core_alloc_docs branch March 21, 2021 03:24
@ojeda
Copy link
Member

ojeda commented Mar 21, 2021

Thanks a lot for your work -- I am sorry we couldn't merge it. I will write what we found in #97 for the future.

ojeda pushed a commit that referenced this pull request Jan 22, 2024
Like commit 1cf3bfc ("bpf: Support 64-bit pointers to kfuncs")
for s390x, add support for 64-bit pointers to kfuncs for LoongArch.
Since the infrastructure is already implemented in BPF core, the only
thing need to be done is to override bpf_jit_supports_far_kfunc_call().

Before this change, several test_verifier tests failed:

  # ./test_verifier | grep # | grep FAIL
  #119/p calls: invalid kfunc call: ptr_to_mem to struct with non-scalar FAIL
  #120/p calls: invalid kfunc call: ptr_to_mem to struct with nesting depth > 4 FAIL
  #121/p calls: invalid kfunc call: ptr_to_mem to struct with FAM FAIL
  #122/p calls: invalid kfunc call: reg->type != PTR_TO_CTX FAIL
  #123/p calls: invalid kfunc call: void * not allowed in func proto without mem size arg FAIL
  #124/p calls: trigger reg2btf_ids[reg->type] for reg->type > __BPF_REG_TYPE_MAX FAIL
  #125/p calls: invalid kfunc call: reg->off must be zero when passed to release kfunc FAIL
  #126/p calls: invalid kfunc call: don't match first member type when passed to release kfunc FAIL
  #127/p calls: invalid kfunc call: PTR_TO_BTF_ID with negative offset FAIL
  #128/p calls: invalid kfunc call: PTR_TO_BTF_ID with variable offset FAIL
  #129/p calls: invalid kfunc call: referenced arg needs refcounted PTR_TO_BTF_ID FAIL
  #130/p calls: valid kfunc call: referenced arg needs refcounted PTR_TO_BTF_ID FAIL
  #486/p map_kptr: ref: reference state created and released on xchg FAIL

This is because the kfuncs in the loaded module are far away from
__bpf_call_base:

  ffff800002009440 t bpf_kfunc_call_test_fail1    [bpf_testmod]
  9000000002e128d8 T __bpf_call_base

The offset relative to __bpf_call_base does NOT fit in s32, which breaks
the assumption in BPF core. Enable bpf_jit_supports_far_kfunc_call() lifts
this limit.

Note that to reproduce the above result, tools/testing/selftests/bpf/config
should be applied, and run the test with JIT enabled, unpriv BPF enabled.

With this change, the test_verifier tests now all passed:

  # ./test_verifier
  ...
  Summary: 777 PASSED, 0 SKIPPED, 0 FAILED

Tested-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Consider generating docs for core and alloc too
2 participants