Skip to content
/ xilinx-ethash Public
forked from Ed-Yang/xilinx-ethash

Run ethash opencl kernel on Xilinx's Alveo U50

2 stars 12 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

developeralgo8888/xilinx-ethash

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
.vscode
.vscode
 
 
ethash @ e4a15c3
ethash @ e4a15c3
 
 
test
test
 
 
xleth
xleth
 
 
.gitignore
.gitignore
 
 
.gitmodules
.gitmodules
 
 
CMakeLists.txt
CMakeLists.txt
 
 
README.md
README.md
 
 
bin2h.cmake
bin2h.cmake
 
 

Repository files navigation

Xilinx Ethash Test

This project is created for testing the Ethminer's ethash opencl kernel on Xilinx Alveo U50 PCIe card.

  • Testing Environment

    CPU: Intel(R) Core(TM) i5-4460  CPU @ 3.20GHz
RAN: 16GB
OS: Ubuntu 18.04.5 LTS
Xilinx: Alveo U50/Vitis 2020.2

  • Alveo U50 information:

    $ sudo /opt/xilinx/xrt/bin/xbmgmt flash --scan
    Card [0000:01:00.0]
    Card type:		u50
    Flash type:		SPI
    Flashable partition running on FPGA:
        xilinx_u50_gen3x16_xdma_201920_3,[ID=0xf465b0a3ae8c64f6],[SC=5.0]
    Flashable partitions installed in system:	
        xilinx_u50_gen3x16_xdma_201920_3,[ID=0xf465b0a3ae8c64f6],[SC=5.1.7]

  • Source tree:

    xilinx-ethash
├── CMakeLists.txt
├── README.md
├── test
│   └── main.cpp <-- main program
└── xleth
    ├── config
    │   ├── connectivity_u50.ini <-- U50 HBM settings
    │   └── xrt.ini
    ├── kernel
    │   ├── ethash-01112021.cl <-- new ethminer opencl kernel (Xilinx build failed)
    │   ├── ethash-08222020.cl <-- original ethminer opencl kernel (base)
    │   └── ethash.cl <-- from ethminer with minor modification
    ├── src
    │   ├── xleth.cpp <-- eth opencl library
    │   └── xleth.hpp <-- eth opencl header
    └── xclbin
        ├── ethash.hw.xclbin <-- binary stream used for U50
        └── ethash.sw_emu.xclbin  <-- binary stream used for software emulation

    Clone source:

    git clone --recursive https://github.com/Ed-Yang/xilinx-ethash.git

Questions

  • How comes running in Alveo U50 is slower than software emulation ?

    Summary:

    Cards DAG generation (epoch=0)
    Xilinx Alveo U50 sw_emu 588.30 seconds
    Xilinx Alveo U50 PCIe 1709.76 seconds
    AMD RX560 6.00 seconds

  • Does Xilinx's OpenCL support atomic_inc or alternative function ?

Build Xilinx binary stream

  • Setup build environment

    Open a terminal and run:

    source /tools/Xilinx/Vitis/2020.2/settings64.sh
source /opt/xilinx/xrt/setup.sh
export XILINX_VITIS=/tools/Xilinx/Vitis/2020.2
    export PATH=$PATH:/opt/xilinx/xrt/bin
export XILINX_XRT=/opt/xilinx/xrt

  • Build binary stream for software emulation

    cd ./xleth
make build TARGET=sw_emu DEVICE=xilinx_u50_gen3x16_xdma_201920_3

  • Build binary stream for U50

    cd ./xleth
make build TARGET=hw DEVICE=xilinx_u50_gen3x16_xdma_201920_3

    Output binary stream will be copied to xclbin directory

Packages

OpenCL:

```shell
sudo apt install -y ocl-icd-opencl-dev opencl-headers gdb
```

CMake

```shell
# upgrade openssl
wget wget https://www.openssl.org/source/openssl-1.1.1j.tar.gz
tar xvf openssl-1.1.1j.tar.gz 
cd openssl-1.1.1j/
./config
make
make install

# upgrade CMake (need new openssl)
wget https://github.com/Kitware/CMake/releases/download/v3.19.5/cmake-3.19.5.tar.gz
tar xvf cmake-3.19.5.tar.gz
cd cmake-3.19.5
./bootstrap
make
sudo make install
```

Build host applicatopm

Build ethash library

Open a terminal with CMake version > 3.13.

```shell
mkdir -p ./ethash/build
cd ./ethash/build
cmake ..
make
```

Build application

```shell
mkdir -p ./build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug
make
```

Run application - Xilinx

  • Setup environment

    source /tools/Xilinx/Vitis/2020.2/settings64.sh
source /opt/xilinx/xrt/setup.sh
export XILINX_VITIS=/tools/Xilinx/Vitis/2020.2

    Usage:

    ./build/xleth <epoch> <Xilinx|AMD> <kernel-file> [quiet]

Run software emulation

  • Setup emulation mode

    export XCL_EMULATION_MODE=sw_emu

  • Generate emconfig.json

    The file "emconfig.json" must be located at the same directory as executable "xleth".

    cd ./build
/tools/Xilinx/Vitis/2020.2/bin/emconfigutil -f xilinx_u50_gen3x16_xdma_201920_3

Run on software emulation

Run:

Difficulty: 1000000

```shell
./build/xleth 0 Xilinx ./xleth/xclbin/ethash.sw_emu.xclbin 
```

Output:

```shell
-----------------------------------------------
Loading OpenCL kernel ...
-----------------------------------------------
Got platform : Xilinx
Found Platform
Platform Name: Xilinx
Trying to program device xilinx_u50_gen3x16_xdma_201920_3
INFO: Reading ./xleth/xclbin/ethash.sw_emu.xclbin
Loading: './xleth/xclbin/ethash.sw_emu.xclbin'
Device program successful.
DEV: Global mem size  8 GB
DEV: Max alloc size   4096 MB
DEV: Max W Group size 4294967295
DEV: Max compute unit 2
-----------------------------------------------
Generating DAG ...
-----------------------------------------------
DAG: generating for epoch 0 ...
DAG: epoch 0 lightSize 16776896 dagSize 1073739904
DAG: item          0 chunk 1280000, took  43.83s
DAG: item    1280000 chunk 1280000, took  44.76s
DAG: item    2560000 chunk 1280000, took  44.80s
DAG: item    3840000 chunk 1280000, took  45.03s
DAG: item    5120000 chunk 1280000, took  43.59s
DAG: item    6400000 chunk 1280000, took  44.01s
DAG: item    7680000 chunk 1280000, took  45.99s
DAG: item    8960000 chunk 1280000, took  44.74s
DAG: item   10240000 chunk 1280000, took  43.54s
DAG: item   11520000 chunk 1280000, took  44.33s
DAG: item   12800000 chunk 1280000, took  46.63s
DAG: item   14080000 chunk 1280000, took  45.99s
DAG: item   15360000 chunk 1280000, took  45.53s
DAG: took 588.30 seconds.
-----------------------------------------------
Searching ...
-----------------------------------------------
seed     : 0000000000 00000000 00000000 00000000 00000000 00000000 00000000 000000
header   : 0000000000 00000000 00000000 00000000 00000000 00000000 00000000 000000
boundary : 0000002af3 1dc46118 73bf3f70 834acdae 9f0f4f53 4f5d6058 5a5f1c1a 3ced1b
Search: target 0x2af31dc461
Search: start nonce 0, hash count 0
Search: found, startNonce 0, gid 3664863, hashCount 0 abort 0
-----------------------------------------------
Check solution ...
-----------------------------------------------
Sol: nonce    : 3664863
Sol: mix_hash : 09f1a1567c a8f6fbbf 6e8c6793 a6474814 7e0d9998 e2e21821 2bb05bff 99360a
Sol: valid.
```

Run on Alveo U50

Run:

Difficulty: 1000000

```shell
./build/xleth 0 Xilinx ./xleth/xclbin/ethash.hw.xclbin
```

Output:

```shell
-----------------------------------------------
Loading OpenCL kernel ...
-----------------------------------------------
Got platform : Xilinx
Found Platform
Platform Name: Xilinx
Trying to program device xilinx_u50_gen3x16_xdma_201920_3
INFO: Reading ./xleth/xclbin/ethash.hw.xclbin
Loading: './xleth/xclbin/ethash.hw.xclbin'
Device program successful.
DEV: Global mem size  0 GB
DEV: Max alloc size   4096 MB
DEV: Max W Group size 4294967295
DEV: Max compute unit 2
-----------------------------------------------
Generating DAG ...
-----------------------------------------------
DAG: generating for epoch 0 ...
DAG: epoch 0 lightSize 16776896 dagSize 1073739904
DAG: item          0 chunk 1280000, took 130.49s
DAG: item    1280000 chunk 1280000, took 130.38s
DAG: item    2560000 chunk 1280000, took 130.39s
DAG: item    3840000 chunk 1280000, took 130.40s
DAG: item    5120000 chunk 1280000, took 130.40s
DAG: item    6400000 chunk 1280000, took 130.38s
DAG: item    7680000 chunk 1280000, took 130.35s
DAG: item    8960000 chunk 1280000, took 130.37s
DAG: item   10240000 chunk 1280000, took 130.38s
DAG: item   11520000 chunk 1280000, took 130.36s
DAG: item   12800000 chunk 1280000, took 130.39s
DAG: item   14080000 chunk 1280000, took 130.39s
DAG: item   15360000 chunk 1280000, took 130.41s
DAG: took 1709.76 seconds.
-----------------------------------------------
Searching ...
-----------------------------------------------
seed     : 0000000000 00000000 00000000 00000000 00000000 00000000 00000000 000000
header   : 0000000000 00000000 00000000 00000000 00000000 00000000 00000000 000000
boundary : 0000002af3 1dc46118 73bf3f70 834acdae 9f0f4f53 4f5d6058 5a5f1c1a 3ced1b
Search: target 0x2af31dc461
Search: start nonce 0, hash count 0
Search: found, startNonce 0, gid 3664863, hashCount 0 abort 0
-----------------------------------------------
Check solution ...
-----------------------------------------------
Sol: nonce    : 3664863
Sol: mix_hash : 09f1a1567c a8f6fbbf 6e8c6793 a6474814 7e0d9998 e2e21821 2bb05bff 99360a
Sol: valid.
```

Run on AMD RX560

Run:

Difficulty: 2000000

```shell
./build/xleth 0 AMD ./xleth/kernel/ethash.cl
```

Output:

```shell
-----------------------------------------------
Loading OpenCL kernel ...
-----------------------------------------------
Got platform : AMD Accelerated Parallel Processing
Found Platform
Platform Name: AMD Accelerated Parallel Processing
Trying to program device Baffin
Device program successful.
KNL: L_WORKSIZE 128
KNL: MULTIPLIER 65535
KNL: G_WORKSIZE 16384
KNL: FASTEXIT   1
DEV: Global mem size  3 GB
DEV: Max alloc size   3256 MB
DEV: Max W Group size 256
DEV: Max W Item size  1024/1024/1024
DEV: Max compute unit 14
-----------------------------------------------
Generating DAG ...
-----------------------------------------------
DAG: generating for epoch 0 ...
DAG: epoch 0 lightSize 16776896 dagSize 1073739904
DAG: item          0 chunk 1280000, took   0.70s
DAG: item    1280000 chunk 1280000, took   0.40s
DAG: item    2560000 chunk 1280000, took   0.40s
DAG: item    3840000 chunk 1280000, took   0.40s
DAG: item    5120000 chunk 1280000, took   0.40s
DAG: item    6400000 chunk 1280000, took   0.40s
DAG: item    7680000 chunk 1280000, took   0.40s
DAG: item    8960000 chunk 1280000, took   0.40s
DAG: item   10240000 chunk 1280000, took   0.40s
DAG: item   11520000 chunk 1280000, took   0.40s
DAG: item   12800000 chunk 1280000, took   0.40s
DAG: item   14080000 chunk 1280000, took   0.40s
DAG: item   15360000 chunk 1280000, took   0.40s
DAG: took   5.99 seconds.
-----------------------------------------------
Searching ...
-----------------------------------------------
seed     : 0000000000 00000000 00000000 00000000 00000000 00000000 00000000 000000
header   : 0000000000 00000000 00000000 00000000 00000000 00000000 00000000 000000
boundary : 0000001579 8ee2308c 39df9fb8 41a566d7 4f87a7a9 a7aeb02c 2d2f8e0d 1e768d
Search: target 0x15798ee230
Search: start nonce            0, hash count 65535
Search: start nonce      8388480, hash count 65535
Search: start nonce     16776960, hash count 65535
Search: start nonce     25165440, hash count 65535
Search: start nonce     33553920, hash count 65535
Search: start nonce     41942400, hash count 65535
Search: start nonce     50330880, hash count 65535
Search: start nonce     58719360, hash count 65535
Search: start nonce     67107840, hash count 65535
Search: start nonce     75496320, hash count 65535
Search: start nonce     83884800, hash count 65535
Search: start nonce     92273280, hash count 65535
Search: start nonce    100661760, hash count 65535
Search: start nonce    109050240, hash count 65535
Search: start nonce    117438720, hash count 65535
Search: start nonce    125827200, hash count 65535
Search: start nonce    134215680, hash count 65535
Search: start nonce    142604160, hash count 65535
Search: start nonce    150992640, hash count 65535
Search: start nonce    159381120, hash count 65535
Search: start nonce    167769600, hash count 65535
Search: start nonce    176158080, hash count 65535
Search: start nonce    184546560, hash count 65535
Search: start nonce    192935040, hash count 65535
Search: start nonce    201323520, hash count 17742

Search: found, startNonce 201323520, gid 2264112, hashCount 17742 abort 1
-----------------------------------------------
Check solution ...
-----------------------------------------------
Sol: nonce    : 203587632
Sol: mix_hash : 0c004dd83c 00b67765 a09bddb1 ecc14561 f006778b 85bb61f5 74fa6e7d ecd29b
Sol: valid.
Sol: Hash rate 10.47 Mh 
```

Reference

Xilinx

Ethash

About

Run ethash opencl kernel on Xilinx's Alveo U50

Resources

Readme
Activity

Stars

2 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • C 53.4%
  • C++ 27.0%
  • Makefile 14.3%
  • CMake 5.3%