fix: build on risc-v #9215

XieJiSS · 2021-11-25T07:08:36Z

Patch is modified from ~~https://reviews.llvm.org/file/data/du5ol5zctyqw53ma7dwz/PHID-FILE-knherxziu4tl4erti5ab/file~~

Tested on Arch Linux riscv64gc (qemu)

UPDATE: Seems like the above link is broken, so I tried to search for a link pointing to the original merge request. It turned out to me that the LLVM guys are cherry-picking from google/benchmark, and the upstream should be this:

https://github.com/google/benchmark/blob/808571a52fd6cc7e9f0788e08f71f0f4175b6673/src/cycleclock.h#L190

Patch is modified from https://reviews.llvm.org/file/data/du5ol5zctyqw53ma7dwz/PHID-FILE-knherxziu4tl4erti5ab/file

facebook-github-bot · 2021-11-25T07:08:40Z

Hi @XieJiSS!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

facebook-github-bot · 2021-11-25T08:25:06Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

adamretter · 2021-11-25T10:20:00Z

@XieJiSS have you considered also sending this fix upstream to the Toku code base? It might help us in future if we ever want to pull in changes from there too

XieJiSS · 2021-11-25T11:39:27Z

@adamretter Sure, I'll do so ASAP.

XieJiSS · 2021-11-25T12:22:33Z

@adamretter I've filed a new PR regarding this to upstream.

percona/PerconaFT#461 (toku_time is part of the TokuDB which is now PerconaFT)

adamretter · 2021-11-26T15:30:51Z

@XieJiSS I would like to give this a test, I can get an emulated RiscV system up under QEMU. I am not sure of the best operations to test this. I am afraid my knowledge of assembly coding is very cursory. Do you have suggestions for how I might test this?

XieJiSS · 2021-11-27T06:33:09Z

@adamretter If by stating "test" you mean to test the assembly codes, how about creating a new .c file and paste toku_time.h#L125..L160 into it? AFAIK gcc is now usable on risc-v platforms.

Download the archriscv package, unzip. (see the wget shell command below)
create the test.c and edit it: vim ./test.c (see https://gist.github.com/XieJiSS/18acd12685ecc0913d43d6bcfb3baefd)
sudo vim path/to/archriscv/root/test.c
systemd-nspawn
compile & run on both platforms, check output

# append these 2 lines to /etc/pacman.conf
[archlinuxcn]
Server = https://repo.archlinuxcn.org/$arch

# This is an Arch Linux x86_64 server

# These two packages are in [archlinuxcn]
sudo pacman -S qemu-user-static binfmt-qemu-static

# An rv64gc image. If root password is needed in this image, it should be: sifive
wget -c https://archriscv.felixc.at/images/archriscv-20210601.tar.zst
sha512sum archriscv-20210601.tar.zst
# 6f012a169fe6f1ea15aeb3283091466e7992f78d823951ee2170940fa030e7fa2394aee11bf67c29943d21579ab42d2262a3d5ca973b5de8be779f338ba1dd44  archriscv-20210601.tar.zst
mkdir ~/archriscv
sudo bsdtar -xvf archriscv-20210601.tar.zst -C archriscv

vim ./test.c  # edit file
sudo cp ./test.c ~/archriscv/root/

gcc ./test.c
./a.out

Then, enter the archriscv container:

sudo systemd-nspawn -D ~/archriscv/ --machine archriscv -a -U

gcc ./test.c
./a.out

My test result is:

17433587739862234
17433592096901640

XieJiSS · 2021-12-17T03:15:11Z

Hi @adamretter , is there anything I can help regarding the test?

adamretter · 2021-12-17T06:34:18Z

I've had trouble running Ubuntu riscv qemu. Seems there are some bugs in the vmbuilder scripts, I will dig out the details and paste here later

adamretter · 2021-12-17T08:24:09Z

@XieJiSS So I am using a Ubuntu 20.4 system to try and install a RISC-V Qemu VM for testing your code. However, I have been unable to get the RISC-V VM successfully installed.

I am basically following these instructions:

Install QEMU Support for RISC-V on the host

$ apt-get install qemu-system-misc opensbi u-boot-qemu qemu-utils

Get the Ubuntu 20.04 LTS (focal) image for SiFive Umatched RISC-V

$ wget https://cdimage.ubuntu.com/releases/20.04.3/release/ubuntu-20.04.3-preinstalled-server-riscv64+unmatched.img.xz
$ xz -dk ubuntu-20.04.3-preinstalled-server-riscv64+unmatched.img.xz

Create a RISC-V VM

$ apt-get install virtinst
$ virt-install --debug --name riscv --memory 4096 --arch riscv64 --machine sifive_u --vcpus 4 --os-variant  ubuntu20.04 --boot loader=/usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf,kernel=/usr/lib/u-boot/qemu-riscv64_smode/uboot.elf --import --disk /tmp/ubuntu-20.04.3-preinstalled-server-riscv64+unmatched.img,format=raw,bus=virtio

However, I cannot start the VM, and end up with what I think is the same bug as this: https://www.spinics.net/lists/virt-tools/msg14419.html

XieJiSS · 2021-12-18T04:46:17Z

Hi @adamretter , looks like currently the libvirt bug is difficult to deal with.

Here at PLCTLab, we provides an open CI infrastructure for RISC-V porting ( https://ci.rvperf.org ). Would it be possible for you to test facebook/rocksdb on the CI? We already have nodejs RISC-V CI hosted on this CI (ref nodejs/node#37856).

Another choice is that we can provide SSH access to SiFive unmatched boards, so that you can test the code on (physical) RISC-V boards.

I'll also have a look on how to get RISC-V qemu toolchain work successfully on Ubuntu.

XieJiSS · 2021-12-18T04:55:18Z

And I can't figure out why the code format CI test is failing, make format told me that Nothing needs to be reformatted

adamretter · 2021-12-18T10:57:25Z

@XieJiSS I could add RocksDB to your CI as an experimental target, but how do I sign-up for that.
Also if you can provide access to a RISC-V vm, then I could use that, and even produce some experimental release builds of RocksJava for RISC-V if that is of interest?

XieJiSS · 2021-12-19T15:42:52Z

Hi @adamretter , we are happy to see RocksDB being added to rvperf CI. I can create a job by opening a PR at https://github.com/plctlab/riscv-ci (which, when merged, will automatically add a job to the CI), but I may need help with creating a working build script (cross-build might be needed) on ubuntu 20.04 (the CI host os). I'll discuss this with my mentor asap.

As for the SSH access to SiFive unmatched boards, I'll add an account for you in (I hope) a week or so. It depends on the efficiency of the internal workflow at PLCT. Is it ok to use your pub keys retrieved from https://github.com/adamretter.keys ?

adamretter · 2021-12-20T11:49:19Z

@XieJiSS Yes the 2nd SSH key will work fine thanks.

XieJiSS · 2021-12-26T15:08:10Z

@adamretter Hi, I've sent you an email with ssh access related information.

adamretter · 2022-01-07T22:00:07Z

@XieJiSS Thanks for access to the RISCV system, I have managed to test your code there and it appears to work.

Where above you say:

I've filed a new PR regarding this to upstream.

Would you mind inserting a link to that PR in that message, so we have a record of it?

XieJiSS · 2022-01-23T13:46:16Z

@adamretter Sure!

XieJiSS · 2022-02-10T06:46:10Z

@adamretter Hi Adam, is there anything else I need to do to get this PR merged? Thanks!

facebook-github-bot · 2022-02-11T13:04:03Z

@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-02-16T06:37:47Z

@XieJiSS has updated the pull request. You must reimport the pull request before landing.

XieJiSS · 2022-02-16T06:38:52Z

Oops... I've just noticed that the clang format CI check is failing, so I re-formatted the code as the linter advised. Hope I didn't break anything

facebook-github-bot · 2022-02-18T12:22:53Z

@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Summary: Related to: #9215 * Adds build_detect_platform support for RISCV on Linux (at least on SiFive Unmatched platforms) This still leaves some linking issues on RISCV remaining (e.g. when building `db_test`): ``` /usr/bin/ld: ./librocksdb_debug.a(memtable.o): in function `__gnu_cxx::new_allocator<char>::deallocate(char*, unsigned long)': /usr/include/c++/10/ext/new_allocator.h:133: undefined reference to `__atomic_compare_exchange_1' /usr/bin/ld: ./librocksdb_debug.a(memtable.o): in function `std::__atomic_base<bool>::compare_exchange_weak(bool&, bool, std::memory_order, std::memory_order)': /usr/include/c++/10/bits/atomic_base.h:464: undefined reference to `__atomic_compare_exchange_1' /usr/bin/ld: /usr/include/c++/10/bits/atomic_base.h:464: undefined reference to `__atomic_compare_exchange_1' /usr/bin/ld: /usr/include/c++/10/bits/atomic_base.h:464: undefined reference to `__atomic_compare_exchange_1' /usr/bin/ld: /usr/include/c++/10/bits/atomic_base.h:464: undefined reference to `__atomic_compare_exchange_1' /usr/bin/ld: ./librocksdb_debug.a(memtable.o):/usr/include/c++/10/bits/atomic_base.h:464: more undefined references to `__atomic_compare_exchange_1' follow /usr/bin/ld: ./librocksdb_debug.a(db_impl.o): in function `rocksdb::DBImpl::NewIteratorImpl(rocksdb::ReadOptions const&, rocksdb::ColumnFamilyData*, unsigned long, rocksdb::ReadCallback*, bool, bool)': /home/adamretter/rocksdb/db/db_impl/db_impl.cc:3019: undefined reference to `__atomic_exchange_1' /usr/bin/ld: ./librocksdb_debug.a(write_thread.o): in function `rocksdb::WriteThread::Writer::CreateMutex()': /home/adamretter/rocksdb/./db/write_thread.h:205: undefined reference to `__atomic_compare_exchange_1' /usr/bin/ld: ./librocksdb_debug.a(write_thread.o): in function `rocksdb::WriteThread::SetState(rocksdb::WriteThread::Writer*, unsigned char)': /home/adamretter/rocksdb/db/write_thread.cc:222: undefined reference to `__atomic_compare_exchange_1' collect2: error: ld returned 1 exit status make: *** [Makefile:1449: db_test] Error 1 ``` Pull Request resolved: #9366 Reviewed By: jay-zhuang Differential Revision: D34377664 Pulled By: mrambacher fbshipit-source-id: c86f9d0cd1cb0c18de72b06f1bf5847f23f51118

See: https://bugs.gentoo.org/834855#c3 Related upstream PR: facebook/rocksdb#9215 Bug: https://bugs.gentoo.org/834855 Signed-off-by: Yun Pan <dinoallo@netc.it>

See: https://bugs.gentoo.org/834855#c3 Related upstream PR: facebook/rocksdb#9215 Bug: https://bugs.gentoo.org/834855 Signed-off-by: Yun Pan <dinoallo@netc.it> Signed-off-by: Yixun Lan <dlan@gentoo.org>

playground · 2022-03-24T01:31:26Z

How can I install and test nodejs on risc-v Fedora?

curl -sL https://rpm.nodesource.com/setup_16.x | sudo -E bash - 

## Installing the NodeSource Node.js 16.x repo...


## Inspecting system...

+ rpm -q --whatprovides redhat-release || rpm -q --whatprovides centos-release || rpm -q --whatprovides cloudlinux-release || rpm -q --whatprovides sl-release || rpm -q --whatprovides fedora-release
+ uname -m

## You don't appear to be running a supported machine architecture: riscv64. Please contact NodeSource at https://github.com/nodesource/distributions/issues if you think this is incorrect or would like your architecture to be considered for support. ```

playground · 2022-03-24T01:32:39Z

How can I install and test nodejs on risc-v Fedora?

curl -sL https://rpm.nodesource.com/setup_16.x | sudo -E bash - 

## Installing the NodeSource Node.js 16.x repo...


## Inspecting system...

+ rpm -q --whatprovides redhat-release || rpm -q --whatprovides centos-release || rpm -q --whatprovides cloudlinux-release || rpm -q --whatprovides sl-release || rpm -q --whatprovides fedora-release
+ uname -m

## You don't appear to be running a supported machine architecture: riscv64. Please contact NodeSource at https://github.com/nodesource/distributions/issues if you think this is incorrect or would like your architecture to be considered for support. ```

XieJiSS · 2022-03-24T12:10:37Z

Hi @playground, this is a bit offtopic, but AFAIK you need to grab the nodejs 16.14.+ source code and compile it manually:

$ sudo dnf install python3 gcc-c++ make python3-pip
$ curl -O https://nodejs.org/dist/v16.14.2/node-v16.14.2.tar.xz
$ tar -xf node-v16.14.2.tar.xz
$ cd node-v16.14.2
$ ./configure
$ make CFLAGS="-fno-strict-aliasing $CFLAGS" CXXFLAGS="-fno-strict-aliasing $CXXFLAGS" -j4

Let's stop here and kindly keep the thread clean :-)

siying · 2022-05-06T23:41:07Z

@hermanlee @lth any comment on this?

facebook-github-bot · 2022-05-17T21:39:05Z

@XieJiSS has updated the pull request. You must reimport the pull request before landing.

riversand963

Thanks @XieJiSS for the PR and @adamretter for verifying it.

facebook-github-bot · 2022-05-17T21:43:44Z

@riversand963 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

fix: build on risc-v

55b049f

Patch is modified from https://reviews.llvm.org/file/data/du5ol5zctyqw53ma7dwz/PHID-FILE-knherxziu4tl4erti5ab/file

XieJiSS mentioned this pull request Nov 25, 2021

addpkg: rocksdb felixonmars/archriscv-packages#407

Merged

facebook-github-bot added the CLA Signed label Nov 25, 2021

XieJiSS mentioned this pull request Nov 25, 2021

Add support for s390x and risc-v percona/PerconaFT#461

Open

adamretter mentioned this pull request Jan 7, 2022

Improve build detect for RISCV #9366

Closed

format: edit risc-v asm code for linter

1e730ed

alexfanqi mentioned this pull request Apr 11, 2022

Add riscv64 support JuliaLinearAlgebra/libblastrampoline#68

Merged

siying requested a review from riversand963 May 6, 2022 23:40

Merge branch 'main' into patch-1

983423e

riversand963 approved these changes May 17, 2022

View reviewed changes

facebook-github-bot closed this in 8b1df10 May 18, 2022

XieJiSS deleted the patch-1 branch June 5, 2022 14:17

XieJiSS mentioned this pull request Oct 29, 2022

fix: SIGILL caused by rdcycle prohibited by kernel on RISC-V #10901

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: build on risc-v #9215

fix: build on risc-v #9215

XieJiSS commented Nov 25, 2021 •

edited

Loading

facebook-github-bot commented Nov 25, 2021

facebook-github-bot commented Nov 25, 2021

adamretter commented Nov 25, 2021

XieJiSS commented Nov 25, 2021

XieJiSS commented Nov 25, 2021 •

edited

Loading

adamretter commented Nov 26, 2021

XieJiSS commented Nov 27, 2021 •

edited

Loading

XieJiSS commented Dec 17, 2021

adamretter commented Dec 17, 2021

adamretter commented Dec 17, 2021

XieJiSS commented Dec 18, 2021

XieJiSS commented Dec 18, 2021

adamretter commented Dec 18, 2021

XieJiSS commented Dec 19, 2021 •

edited

Loading

adamretter commented Dec 20, 2021

XieJiSS commented Dec 26, 2021

adamretter commented Jan 7, 2022 •

edited

Loading

XieJiSS commented Jan 23, 2022 •

edited

Loading

XieJiSS commented Feb 10, 2022

facebook-github-bot commented Feb 11, 2022

facebook-github-bot commented Feb 16, 2022

XieJiSS commented Feb 16, 2022 •

edited

Loading

facebook-github-bot commented Feb 18, 2022

playground commented Mar 24, 2022

playground commented Mar 24, 2022

XieJiSS commented Mar 24, 2022

siying commented May 6, 2022

facebook-github-bot commented May 17, 2022

riversand963 left a comment

facebook-github-bot commented May 17, 2022

fix: build on risc-v #9215

fix: build on risc-v #9215

Conversation

XieJiSS commented Nov 25, 2021 • edited Loading

facebook-github-bot commented Nov 25, 2021

Action Required

Process

facebook-github-bot commented Nov 25, 2021

adamretter commented Nov 25, 2021

XieJiSS commented Nov 25, 2021

XieJiSS commented Nov 25, 2021 • edited Loading

adamretter commented Nov 26, 2021

XieJiSS commented Nov 27, 2021 • edited Loading

XieJiSS commented Dec 17, 2021

adamretter commented Dec 17, 2021

adamretter commented Dec 17, 2021

XieJiSS commented Dec 18, 2021

XieJiSS commented Dec 18, 2021

adamretter commented Dec 18, 2021

XieJiSS commented Dec 19, 2021 • edited Loading

adamretter commented Dec 20, 2021

XieJiSS commented Dec 26, 2021

adamretter commented Jan 7, 2022 • edited Loading

XieJiSS commented Jan 23, 2022 • edited Loading

XieJiSS commented Feb 10, 2022

facebook-github-bot commented Feb 11, 2022

facebook-github-bot commented Feb 16, 2022

XieJiSS commented Feb 16, 2022 • edited Loading

facebook-github-bot commented Feb 18, 2022

playground commented Mar 24, 2022

playground commented Mar 24, 2022

XieJiSS commented Mar 24, 2022

siying commented May 6, 2022

facebook-github-bot commented May 17, 2022

riversand963 left a comment

Choose a reason for hiding this comment

facebook-github-bot commented May 17, 2022

XieJiSS commented Nov 25, 2021 •

edited

Loading

XieJiSS commented Nov 25, 2021 •

edited

Loading

XieJiSS commented Nov 27, 2021 •

edited

Loading

XieJiSS commented Dec 19, 2021 •

edited

Loading

adamretter commented Jan 7, 2022 •

edited

Loading

XieJiSS commented Jan 23, 2022 •

edited

Loading

XieJiSS commented Feb 16, 2022 •

edited

Loading