Add RISC-V support #272

DaniAffCH · 2022-08-30T11:18:20Z

I added RISC-V architecture support on linux by parsing /proc/cpuinfo. It has been tested using QEMU emulator.
I have done the following points:

features detection (i kept it as a single letter, is it fine or do i have to replace it with its description?)
microarchitecture detection (i'm not sure if it's correct, so check it carefully)
vendor detection
Tests have been added as well

… main

toor1245 · 2022-08-30T15:37:43Z

CMakeLists.txt

@@ -49,6 +49,7 @@ set(PROCESSOR_IS_ARM FALSE)
 set(PROCESSOR_IS_AARCH64 FALSE)


@gchatelet, should we update the Bazel file in this PR?

src/impl_riscv_linux.c

toor1245 · 2022-08-30T16:14:46Z

src/impl_riscv_linux.c

+
+#include "cpuinfo_riscv.h"
+
+////////////////////////////////////////////////////////////////////////////////


better to move duplicate logic to a generic place for other operating systems

At the moment there isn't a duplicate logic except for the GetRiscvInfo() function (just a few lines of code). Do I nevertheless have to move it to another file?

yes, but the next developer who will add support for another OS should rewrite/move your common logic. See #268, #204

include/internal/hwcaps.h

src/impl_riscv_linux.c

toor1245 · 2022-08-30T18:27:02Z

include/cpuinfo_riscv.h

+    char          uarch[64];  // 0 terminated string
+    char          vendor[64]; // 0 terminated string


I think we should consider API that will be consistent with other operating systems, what about FreeBSD and possible future operating systems? Can we get this info?

I'm not sure about other os compatibility. The more general way would be using misa instruction but as we have seen in #247 it's not possible unless you have machine privileges. Do you have any suggestion? We could define custom codes for each uarch and vendor and fill them accordingly with informations retrieved from the os

I mean that we should define only common info, and avoid features which on certain operating systems will return empty results.

for example, if both Linux and FreeBSD return hart from the cpuinfo file, then we can add to our functionality, but if only one Linux/FreeBSD returns some info and the other doesn't provide that info we should avoid it.

include/cpuinfo_riscv.h

test/cpuinfo_riscv_test.cc

michael-roe

I think your parser of the RISC-V ISA string is incorrect, because the ISA string can contain multi-letter extensions, such as "_zbc". Simply searching for single letters within the ISA string won't work, you need to parse it to find out where the multi-letter extensions begin.

DaniAffCH · 2022-08-31T19:36:33Z

Ok @michael-roe, Do I have to support these extensions as well?

michael-roe · 2022-08-31T19:56:05Z

Ok @michael-roe, Do I have to support these extensions as well?

Probably ought to support all the user space ones. The supervisor state ones (starting with S) are less relevant to user space programs that call cpu_features, so can probably leave those out.

toor1245 · 2022-08-31T21:51:52Z

src/impl_riscv_linux.c

+      value.size-=2;
+      for (size_t i = 0; i < RISCV_LAST_; ++i){
+        tmp_sv.ptr = kCpuInfoFlags[i];
+        tmp_sv.size = strlen(kCpuInfoFlags[i]);


Why are you using libc functions like strlen, strcpy? According to documentation cpu_features is suitable for implementing fundamental libc functions like malloc, memcpy, and memcmp. Hence we use our implementation, see src/copy.inl, src/equals.inl.

@gchatelet, it seems I misunderstood. Is this the right point? Since I found several codes where memmove and strlen are used:
https://github.com/google/cpu_features/blob/main/src/stack_line_reader.c#L59
https://github.com/google/cpu_features/blob/main/include/internal/string_view.h#L47
https://github.com/google/cpu_features/blob/main/src/filesystem.c#L46
https://github.com/google/cpu_features/blob/main/src/filesystem.c#L57

It used to be strongly enforced but it is not anymore as it is hard to prevent the compiler from generating calls to these functions. That is, even if the code contains no call to C memory functions, the compiler may decide to generate them (e.g., https://godbolt.org/z/h5KKM7MEj)

should we provide info to REAMDE that even with compiler flags such as freestanding no-stdlib no-builtin there is a possibility that the compiler will generate C functions?

Yep I think so, here is an example where the compiler generates a call even with all the compiler flags https://godbolt.org/z/rTTfzEq15

In fact, compilers require some C functions to exist.

src/impl_riscv_linux.c

gchatelet · 2022-09-19T10:24:44Z

src/impl_riscv_linux.c

+  StringView key, value;
+  if (CpuFeatures_StringView_GetAttributeKeyValue(line, &key, &value)) {
+    if (CpuFeatures_StringView_IsEquals(key, str("isa"))) {
+      StringView tmp_sv;


if(CpuFeatures_StringView_StartsWith(value, "rv") == false) continue;

if(!CpuFeatures_StringView_StartsWith(value, "rv")) continue;?

gchatelet · 2022-09-19T10:25:25Z

src/impl_riscv_linux.c

+    if (CpuFeatures_StringView_IsEquals(key, str("isa"))) {
+      StringView tmp_sv;
+      value.ptr+=2; // ignore 'rv' prefix at the beginning
+      value.size-=2;


value = CpuFeatures_StringView_PopFront(value, 2)

gchatelet · 2022-09-19T10:28:02Z

src/impl_riscv_linux.c

+      value.ptr+=2; // ignore 'rv' prefix at the beginning
+      value.size-=2;
+      for (size_t i = 0; i < RISCV_LAST_; ++i){
+        tmp_sv.ptr = kCpuInfoFlags[i];


StringView tmp_sv = str(kCpuInfoFlags[i]);

gchatelet · 2022-09-19T10:29:39Z

src/impl_riscv_linux.c

+        tmp_sv.ptr = kCpuInfoFlags[i];
+        tmp_sv.size = strlen(kCpuInfoFlags[i]);
+        kSetters[i](&info->features,
+                    CpuFeatures_StringView_IndexOf(value, tmp_sv) != -1);


extract a variable:

bool is_set = CpuFeatures_StringView_IndexOf(value, tmp_sv) != -1; kSetters[i](&info->features, is_set);

gchatelet · 2022-09-19T10:38:05Z

src/impl_riscv_linux.c

+      value.size-=2;
+      for (size_t i = 0; i < RISCV_LAST_; ++i){
+        tmp_sv.ptr = kCpuInfoFlags[i];
+        tmp_sv.size = strlen(kCpuInfoFlags[i]);


It used to be strongly enforced but it is not anymore as it is hard to prevent the compiler from generating calls to these functions. That is, even if the code contains no call to C memory functions, the compiler may decide to generate them (e.g., https://godbolt.org/z/h5KKM7MEj)

gchatelet · 2022-09-19T10:45:06Z

src/impl_riscv_linux.c

+}
+
+#endif  //  defined(CPU_FEATURES_OS_LINUX) || defined(CPU_FEATURES_OS_ANDROID)
+#endif  // CPU_FEATURES_ARCH_RISCV


Add an empty line at EOF

gchatelet · 2022-09-19T10:48:40Z

src/utils/list_cpu_features.c

@@ -408,6 +413,12 @@ static Node* CreateTree(void) {
  AddMapEntry(root, "microarchitecture",
              CreateString(strings.type.base_platform));
  AddFlags(root, &info.features);
+#elif defined(CPU_FEATURES_ARCH_RISCV)
+  const RiscvInfo info = GetRiscvInfo();
+  AddMapEntry(root, "arch", CreateString("risc-v"));


With enough time people will rely on this string. Are we sure we want "risc-v"?
Alternative naming scheme I can think of :

"riscv"

"riscv-32"/ "riscv-64" / "riscv-128"

the second naming scheme seems better to me since this functionality is implemented with GetRiscvInfo and more precise arch detection compared to riscv, but I also don't mind "riscv"

As an afterthought, it seems that the 32 / 64 / 128 address space is really seen as a feature so we could stick to "risc-v". As you said the information exists elsewhere. I don't mind either, I just wanted to check whether we could regret this choice later on.

gchatelet · 2022-09-19T10:49:57Z

test/cpuinfo_riscv_test.cc

+TEST(CpuinfoRiscvTest, KendryteFromCpuInfo) {
+    ResetHwcaps();
+    auto& fs = GetEmptyFilesystem();
+    fs.CreateFile("/proc/cpuinfo", R"(hart  : 0


put hart : 0 on a new line for visibility

gchatelet · 2022-09-19T10:51:06Z

test/cpuinfo_riscv_test.cc

+}
+
+}  // namespace
+}  // namespace cpu_features


new line at EOF

gchatelet · 2022-09-19T10:54:36Z

src/impl_riscv_linux.c

+      StringView tmp_sv;
+      value.ptr+=2; // ignore 'rv' prefix at the beginning
+      value.size-=2;
+      for (size_t i = 0; i < RISCV_LAST_; ++i){


The complexity here is O(N^2)
I'd rather scan the characters after "rv" and check if the character corresponds to one of the riscv extensions.

I just read the comment from @michael-roe that the extensions can be strings and not single letters so I take my previous comment back.
Do we have a strong guarantee that no extension string can be embedded in another extension string ? If not, we cannot simply look at CpuFeatures_StringView_IndexOf(...) != -1 as we would take an extension for another one.

toor1245 · 2022-09-19T17:04:47Z

include/cpuinfo_riscv.h

+    RISCV_32,
+    RISCV_64,
+    RISCV_128,


is it right that we mixed isa extensions and arch to RiscvFeaturesEnum should we separate them?

Wikipedia separates them into base and extension, I think we should do the same (see my comment on l.40)

toor1245 · 2022-09-19T17:37:20Z

README.md

+| Linux   | yes² | yes¹    | yes¹    | yes¹    | yes¹    | yes¹    |
+| MacOs   | yes² | N/A     | not yet | N/A     | no      | N/A     |
+| Windows | yes² | not yet | not yet | N/A     | N/A     | N/A     |
+| FreeBSD | yes² | not yet | not yet | not yet | not yet | N/A     |


FreeBSD - not yet

gchatelet · 2022-09-20T08:39:49Z

include/cpuinfo_riscv.h

+    RISCV_32,
+    RISCV_64,
+    RISCV_128,


Wikipedia separates them into base and extension, I think we should do the same (see my comment on l.40)

gchatelet · 2022-09-20T08:42:10Z

include/cpuinfo_riscv.h

+CPU_FEATURES_START_CPP_NAMESPACE
+
+typedef struct {
+    int riscv32 : 1;  // Is a 32 bit architecture


// Is a 32 bit architecture AFAIU this is more about address space

gchatelet · 2022-09-20T08:52:52Z

include/cpuinfo_riscv.h

+    int m : 1;        // Integer Multiplication and Division
+    int v : 1;        // Vector Operations
+    int q : 1;        // Quad Precision Floating Point
+} RiscvFeatures;


As discussed lower in this file I think we should make a distinction between base and extensions.
https://en.wikipedia.org/wiki/RISC-V#ISA_base_and_extensions

I was tempted to create two Features structs (i.e., BaseRiscvFeatures and ExtRiscvFeatures) but this will not play nicely with the current introspection facilities.

Let's embed the distinction in the name instead.
I see two paths forward : use the names as they are defined in the wikipedia page or prefix them with base_ (std_?) and ext_.

I think I prefer sticking to the original names (RVWMO, RV32I, RV32E, RV64I, RV128I for base and M,A,F,D,Zicsr,Zifencei,G,Q,L,C,B,J,T,P,V,K,N,H,S,Zam,Ztso for extensions) as long as riscv commits to this naming scheme.

Thoughts?

I agree with you to stick naming according to riscv documentation, thus we eliminate the confusion with riscv32/riscv64/riscv128, also we can add a comment with the isa definition (base/ext):

typedef struct { int rvmwo : 1; // Base ISA. Weak Memory Ordering int rv32i: 1; // Base ISA. Base Integer Instruction Set, 32-bit ... int zicsr: 1; // Extension ISA. Control and Status Register (CSR) ... } RiscvFeatures;

gchatelet · 2022-09-20T08:59:55Z

src/impl_riscv_linux.c

+      StringView tmp_sv;
+      value.ptr+=2; // ignore 'rv' prefix at the beginning
+      value.size-=2;
+      for (size_t i = 0; i < RISCV_LAST_; ++i){


I just read the comment from @michael-roe that the extensions can be strings and not single letters so I take my previous comment back.
Do we have a strong guarantee that no extension string can be embedded in another extension string ? If not, we cannot simply look at CpuFeatures_StringView_IndexOf(...) != -1 as we would take an extension for another one.

gchatelet · 2022-09-20T09:13:01Z

src/impl_riscv_linux.c

+      value.size-=2;
+      for (size_t i = 0; i < RISCV_LAST_; ++i){
+        tmp_sv.ptr = kCpuInfoFlags[i];
+        tmp_sv.size = strlen(kCpuInfoFlags[i]);


Yep I think so, here is an example where the compiler generates a call even with all the compiler flags https://godbolt.org/z/rTTfzEq15

In fact, compilers require some C functions to exist.

gchatelet · 2022-09-20T09:16:36Z

src/utils/list_cpu_features.c

@@ -408,6 +413,12 @@ static Node* CreateTree(void) {
  AddMapEntry(root, "microarchitecture",
              CreateString(strings.type.base_platform));
  AddFlags(root, &info.features);
+#elif defined(CPU_FEATURES_ARCH_RISCV)
+  const RiscvInfo info = GetRiscvInfo();
+  AddMapEntry(root, "arch", CreateString("risc-v"));


As an afterthought, it seems that the 32 / 64 / 128 address space is really seen as a feature so we could stick to "risc-v". As you said the information exists elsewhere. I don't mind either, I just wanted to check whether we could regret this choice later on.

gchatelet · 2022-11-09T13:56:35Z

Hi @DaniAffCH are you still interested in contributing to this branch?
I can take it over if you want.

DaniAffCH · 2022-11-10T09:44:13Z

Hi @gchatelet, yes I'm still interested in contributing to this project but I have a really busy period, so I'm unable to fix the PR issues, I'll do it as soon as I'm free though. If you want to keep the PR list clean feel free to close my PR and I'll open it again when I'm ready.

gchatelet · 2022-11-10T10:55:29Z

Hi @gchatelet, yes I'm still interested in contributing to this project but I have a really busy period, so I'm unable to fix the PR issues, I'll do it as soon as I'm free though. If you want to keep the PR list clean feel free to close my PR and I'll open it again when I'm ready.

Ok no worries. I've started #287 based on this PR to retain your original work (I hope you don't mind!). I'm now working on fixing the comments we've made. Thx to @Mizux we also have a running CI for RISC-V 💪.

DaniAffCH · 2022-11-10T11:24:13Z

Hi @gchatelet, yes I'm still interested in contributing to this project but I have a really busy period, so I'm unable to fix the PR issues, I'll do it as soon as I'm free though. If you want to keep the PR list clean feel free to close my PR and I'll open it again when I'm ready.

Ok no worries. I've started #287 based on this PR to retain your original work (I hope you don't mind!). I'm now working on fixing the comments we've made. Thx to @Mizux we also have a running CI for RISC-V muscle.

Ok, good job! I can probably be back active in 2 weeks. Please keep track of the problems you have fixed (or I'm gonna figure it out from your commits).

gchatelet · 2023-01-13T10:38:29Z

Merged as part of #287 . Thx a lot @DaniAffCH for the initial implementation and research, this helped !

DaniAffCH added 14 commits July 26, 2022 15:26

added some missing amd uarch

333fa0b

removed llano uarch

3d56e46

removed llano definition

891a2f6

moved comment

3087eec

Removed duplicate of google#256

b9e5202

Merge branch 'google:main' into main

ec352d3

RISC-V feature detection

e72f7ec

Merge branch 'main' of https://github.com/DaniAffCH/cpu_features into…

ae1a9be

… main

Merge branch 'google:main' into main

f52986d

Merge branch 'google:main' into main

169b600

Architecture detection, vendor detection and automated tests

930a761

Merge branch 'google:main' into main

e77cbfe

Update README.md

f3bc4d7

Merge branch 'google:main' into main

79b489d