Merge branch 'master' into riscv-vector-isa-simx

README.md

@@ -1,6 +1,30 @@
 # Vortex GPGPU
 
-Vortex is a full-stack open-source RISC-V GPGPU.
+Vortex is a full-stack open-source RISC-V GPGPU. Vortex supports multiple **backend drivers**, including our C++ simulator (simx), an RTL simulator, and physical Xilinx and Altera FPGAs-- all controlled by a single driver script. The chosen driver determines the corresponding code invoked to run Vortex. Generally, developers will prototype their intended design in simx, before completing going forward with an RTL implementation. Alternatively, you can get up and running by selecting a driver of your choice and running a demo program.
+
+## Website
+Vortex news can be found on its [website](https://vortex.cc.gatech.edu/)
+
+## Citation
+```
+@inproceedings{10.1145/3466752.3480128,
+	author = {Tine, Blaise and Yalamarthy, Krishna Praveen and Elsabbagh, Fares and Hyesoon, Kim},
+	title = {Vortex: Extending the RISC-V ISA for GPGPU and 3D-Graphics},
+	year = {2021},
+	isbn = {9781450385572},
+	publisher = {Association for Computing Machinery},
+	address = {New York, NY, USA},
+	url = {https://doi.org/10.1145/3466752.3480128},
+	doi = {10.1145/3466752.3480128},
+	abstract = {The importance of open-source hardware and software has been increasing. However, despite GPUs being one of the more popular accelerators across various applications, there is very little open-source GPU infrastructure in the public domain. We argue that one of the reasons for the lack of open-source infrastructure for GPUs is rooted in the complexity of their ISA and software stacks. In this work, we first propose an ISA extension to RISC-V that supports GPGPUs and graphics. The main goal of the ISA extension proposal is to minimize the ISA changes so that the corresponding changes to the open-source ecosystem are also minimal, which makes for a sustainable development ecosystem. To demonstrate the feasibility of the minimally extended RISC-V ISA, we implemented the complete software and hardware stacks of Vortex on FPGA. Vortex is a PCIe-based soft GPU that supports OpenCL and OpenGL. Vortex can be used in a variety of applications, including machine learning, graph analytics, and graphics rendering. Vortex can scale up to 32 cores on an Altera Stratix 10 FPGA, delivering a peak performance of 25.6 GFlops at 200 Mhz.},
+	booktitle = {MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture},
+	pages = {754–766},
+	numpages = {13},
+	keywords = {reconfigurable computing, memory systems., computer graphics},
+	location = {Virtual Event, Greece},
+	series = {MICRO '21}
+}
+```
 
 ## Specifications
 
@@ -30,12 +54,14 @@ Vortex is a full-stack open-source RISC-V GPGPU.
 - `ci`: Continuous integration scripts.
 - `miscs`: Miscellaneous resources.
 
-## Build Instructions
-More detailed build instructions can be found [here](docs/install_vortex.md).
+## Quick Start
+If you are interested in a stable release of Vortex, you can download the latest release [here](https://github.com/vortexgpgpu/vortex/releases/latest). Otherwise, you can pull the most recent, but (potentially) unstable version as shown below. The following steps demonstrate how to build and run Vortex with the default driver: SimX. If you are interested in a different backend, look [here](docs/simulation.md).
+
 ### Supported OS Platforms
 - Ubuntu 18.04, 20.04, 22.04, 24.04
 - Centos 7
 ### Toolchain Dependencies
+The following dependencies will be fetched prebuilt by `toolchain_install.sh`.
 - [POCL](http://portablecl.org/)
 - [LLVM](https://llvm.org/)
 - [RISCV-GNU-TOOLCHAIN](https://github.com/riscv-collab/riscv-gnu-toolchain)
@@ -105,4 +131,4 @@ echo "source <build-path>/ci/toolchain_env.sh" >> ~/.bashrc
 ```sh
 ./ci/blackbox.sh --app=demo --debug=3
 ```
-- For additional information, check out the /docs.
+- For additional information, check out the [documentation](docs/index.md)

ci/install_dependencies.sh

@@ -31,7 +31,7 @@ check_gcc_version() {
 apt-get update -y
 
 # install system dependencies
-apt-get install -y build-essential valgrind libstdc++6 binutils python3 uuid-dev ccache
+apt-get install -y build-essential valgrind libstdc++6 binutils python3 uuid-dev ccache cmake
 
 # Check and install GCC 11 if necessary
 if check_gcc_version; then

ci/regression.sh.in

@@ -163,8 +163,9 @@ cache()
     CONFIGS="-DL1_LINE_SIZE=$XSIZE -DLMEM_DISABLE" ./ci/blackbox.sh --driver=simx --app=sgemmx
 
     # test cache ways
-    CONFIGS="-DICACHE_NUM_WAYS=8 -DDCACHE_NUM_WAYS=8" ./ci/blackbox.sh --driver=rtlsim --app=sgemmx
-    CONFIGS="-DICACHE_NUM_WAYS=8 -DDCACHE_NUM_WAYS=8" ./ci/blackbox.sh --driver=simx --app=sgemmx
+    CONFIGS="-DICACHE_NUM_WAYS=1 -DDCACHE_NUM_WAYS=1" ./ci/blackbox.sh --driver=rtlsim --app=sgemmx
+    CONFIGS="-DICACHE_NUM_WAYS=4 -DDCACHE_NUM_WAYS=8" ./ci/blackbox.sh --driver=rtlsim --app=sgemmx
+    CONFIGS="-DICACHE_NUM_WAYS=4 -DDCACHE_NUM_WAYS=8" ./ci/blackbox.sh --driver=simx --app=sgemmx
 
     # test cache banking
     CONFIGS="-DLMEM_NUM_BANKS=4 -DDCACHE_NUM_BANKS=1" ./ci/blackbox.sh --driver=rtlsim --app=sgemmx
@@ -174,11 +175,17 @@ cache()
     CONFIGS="-DDCACHE_NUM_BANKS=2" ./ci/blackbox.sh --driver=rtlsim --app=sgemmx
     CONFIGS="-DDCACHE_NUM_BANKS=2" ./ci/blackbox.sh --driver=simx --app=sgemmx
 
+    # replacement policy
+    CONFIGS="-DDCACHE_REPL_POLICY=0" ./ci/blackbox.sh --driver=rtlsim --app=sgemmx
+    CONFIGS="-DDCACHE_REPL_POLICY=1" ./ci/blackbox.sh --driver=rtlsim --app=sgemmx
+    CONFIGS="-DDCACHE_REPL_POLICY=2" ./ci/blackbox.sh --driver=rtlsim --app=sgemmx
+
     # test writeback
-    CONFIGS="-DDCACHE_WRITEBACK=1" ./ci/blackbox.sh --driver=rtlsim --app=mstress
-    CONFIGS="-DDCACHE_WRITEBACK=1" ./ci/blackbox.sh --driver=simx --app=mstress
-    CONFIGS="-DSOCKET_SIZE=1 -DDCACHE_WRITEBACK=1 -DL2_WRITEBACK=1 -DL3_WRITEBACK=1" ./ci/blackbox.sh --driver=rtlsim --cores=2 --clusters=2 --l2cache --l3cache --app=mstress
-    CONFIGS="-DSOCKET_SIZE=1 -DDCACHE_WRITEBACK=1 -DL2_WRITEBACK=1 -DL3_WRITEBACK=1" ./ci/blackbox.sh --driver=simx --cores=2 --clusters=2 --l2cache --l3cache --app=mstress
+    CONFIGS="-DDCACHE_WRITEBACK=1 -DDCACHE_DIRTYBYTES=0 -DDCACHE_NUM_WAYS=4" ./ci/blackbox.sh --driver=rtlsim --app=mstress
+    CONFIGS="-DDCACHE_WRITEBACK=1 -DDCACHE_DIRTYBYTES=1 -DDCACHE_NUM_WAYS=4" ./ci/blackbox.sh --driver=rtlsim --app=mstress
+    CONFIGS="-DDCACHE_WRITEBACK=1 -DDCACHE_NUM_WAYS=4" ./ci/blackbox.sh --driver=simx --app=mstress
+    CONFIGS="-DSOCKET_SIZE=1 -DL2_WRITEBACK=1 -DL3_WRITEBACK=1" ./ci/blackbox.sh --driver=rtlsim --cores=2 --clusters=2 --l2cache --l3cache --app=mstress
+    CONFIGS="-DSOCKET_SIZE=1 -DL2_WRITEBACK=1 -DL3_WRITEBACK=1" ./ci/blackbox.sh --driver=simx --cores=2 --clusters=2 --l2cache --l3cache --app=mstress
 
     # cache clustering
     CONFIGS="-DSOCKET_SIZE=4 -DNUM_DCACHES=4 -DNUM_ICACHES=2" ./ci/blackbox.sh --driver=rtlsim --app=sgemmx --cores=4 --warps=1 --threads=2

configure

@@ -65,7 +65,7 @@ copy_files() {
                     filename_no_ext="${filename%.in}"
                     dest_file="$dest_dir/$filename_no_ext"
                     mkdir -p "$dest_dir"
-                    sed "s|@VORTEX_HOME@|$SCRIPT_DIR|g; s|@XLEN@|$XLEN|g; s|@TOOLDIR@|$TOOLDIR|g; s|@OSVERSION@|$OSVERSION|g; s|@INSTALLDIR@|$PREFIX|g; s|@CURRENTDIR@|$CURRENT_DIR|g" "$file" > "$dest_file"
+                    sed "s|@VORTEX_HOME@|$SOURCE_DIR|g; s|@XLEN@|$XLEN|g; s|@TOOLDIR@|$TOOLDIR|g; s|@OSVERSION@|$OSVERSION|g; s|@INSTALLDIR@|$PREFIX|g; s|@BUILDDIR@|$CURRENT_DIR|g" "$file" > "$dest_file"
                     # apply permissions to bash scripts
                     read -r firstline < "$dest_file"
                     if [[ "$firstline" =~ ^#!.*bash ]]; then
@@ -169,8 +169,8 @@ fi
 SUBDIRS=("." "!ci" "!perf" "hw*" "kernel*" "runtime*" "sim*" "tests*")
 
 # Get the directory of the script
-SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
+SOURCE_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
 
-THIRD_PARTY_DIR=$SCRIPT_DIR/third_party
+THIRD_PARTY_DIR=$SOURCE_DIR/third_party
 
-copy_files "$SCRIPT_DIR" "$CURRENT_DIR"
+copy_files "$SOURCE_DIR" "$CURRENT_DIR"

docs/contributing.md

@@ -1,18 +1,37 @@
-# Contributing to Vortex on Github
+# Contributing to Vortex
 
-## Github Details
-- There are two main repos, `vortex` (public, this one) and `vortex-dev` (private)
-- todo: Most current development is on `vortex`
-- If you have a legacy version of `vortex`, you can use the releases branch or tags to access the repo at that point in time
+## Github
+Vortex uses Github to host its git repositories.
+There are a lot of ways to use the features on Github for collaboration.
+Therefore, this documentation details the standard procedure for contributing to Vortex.
+Development of Vortex is consolidated to this repo, `vortex` and any associated forks.
+Previously, there was active work done on a private repo named `vortex-dev`.
+`vortex-dev` has officially been deprecated and fully merged into this public repo, `vortex`.
+If you are returning to this project and have legacy versions of Vortex, you can use the releases branches to access older versions.
 
 ## Contribution Process
-- You should create a new branch from develop that is clearly named with the feature that you want to add
-- Avoid pushing directly to the `master` branch instead you will need to make a Pull Request (PR)
-- There should be protections in place that prevent pushing directly to the main branch, but don't rely on it
-- When you make a PR it will be tested against the continuous integration (ci) pipeline (see `continuous_integration.md`)
-- It is not sufficient to just write some tests, they need to be incorporated into the ci pipeline to make sure they are run
-- During a PR, you might receive feedback regarding your changes and you might need to make further commits to your branch
+In an effort to keep `vortex` organized, permissions to directly create branches and push code has been limited to admins.
+However, contributions are strongly encouraged and keep the project moving forward! Here is the procedure for contributing:
 
+1. Create a fork of `vortex`
+2. In your fork, create a branch from `master` that briefly explains the work you are adding (ie: `develop-documentation`)
+3. Make your changes on the new branch in your fork. You may create as many commits as you need, which might be common if you are making multiple iterations
+4. Since you are the owner of your fork, you have full permissions to push commits to your fork
+4. When you are satisfied with the changes on your fork, you can open a PR from your fork using the online interface
+5. If you recently made a push, you will get automatically get a prompt on Github online to create a PR, which you can press
+6. Otherwise, you can go to your fork on Github online and manually create a PR (todo)
+(todo): how to name and format your PR, what information you should add to the PR, does not need to be too strict if you are attending the weekly meetings*
+7. Github uses the following semantics: `base repository` gets the changes from your `head repository`
+8. Therefore, you should set the `base repository` to `vortexgpgpu/vortex` and the `base` branch to `master` since the master branch is protected by reviewed PRs.
+9. And you should assign the `head repository` to `<your-github-username>/vortex` (which represents your fork of vortex) and the `base` branch to the one created in step 2
+10. Now that your intended PR has been specified, you should review the status. Check for merge conflicts, if all your commits are present, and all the modified files make sense
+11. You can still make a PR if there are issues in step 10, just make sure the structure is correct according to steps 7-9
+12. Once the PR is made, the CI pipeline will run automatically, testing your changes
+13. Remember, a PR is flexible if you need to make changes to the code you can go back to your branch of the fork to commit and push any updates
+14. As long as the `head repository`'s `base` branch is the one you edited, the PR will automatically get the most recent changes
+15. When all merge conflicts are resolved, changes are made, and tests pass you can have an admin merge your PR
 
-## Creating  and Adding Tests
-see `testing.md`
+## What Makes a Good Contribution?
+- If you are contributing code changes, then review [testing.md](./testing.md) to ensure your tests are integrated into the [CI pipeline](continuous_integration.md)
+- During a PR, you should consider the advice you are provided by your reviewers. Remember you keep adding commits to an open PR!
+- If your change aims to fix an issue opened on Github, please tag that issue in the PR itself

docs/environment_setup.md

@@ -1,16 +1,19 @@
 # Environment Setup
-These instructions apply to the development vortex repo using the updated toolchain. The updated toolchain is considered to be any commit of `master` pulled from July 2, 2023 onwards. The toolchain update in question can be viewed in this [commit](https://github.com/vortexgpgpu/vortex-dev/commit/0048496ba28d7b9a209a0e569d52d60f2b68fc04). Therefore, if you are unsure whether you are using the new toolchain or not, then you should check the `ci` folder for the existence of the `toolchain_prebuilt.sh` script. Furthermore, you should notice that the `toolchain_install.sh` script has the legacy `llvm()` split into `llvm-vortex()` and `llvm-pocl()`.
 
+These instructions apply to the development vortex repo using the updated toolchain. The updated toolchain is considered to be any commit of `master` pulled from July 2, 2023 onwards. The toolchain update in question can be viewed in this [commit](https://github.com/vortexgpgpu/vortex-dev/commit/0048496ba28d7b9a209a0e569d52d60f2b68fc04). Therefore, if you are unsure whether you are using the new toolchain or not, then you should check the `ci` folder for the existence of the `toolchain_prebuilt.sh` script. Furthermore, you should notice that the `toolchain_install.sh` script has the legacy `llvm()` split into `llvm-vortex()` and `llvm-pocl()`.
 
 ## Set Up on Your Own System
-The toolchain binaries provided with Vortex are built on Ubuntu-based systems. To install Vortex on your own system, [follow these instructions](install_vortex.md).
 
+The toolchain binaries provided with Vortex are built on Ubuntu-based systems. To install Vortex on your own system, [follow these instructions](install_vortex.md).
 
 ## Servers for Georgia Tech Students and Collaborators
+
 ### Volvo
+
 Volvo is a 64-core server provided by HPArch. You need valid credentials to access it. If you don't already have access, you can get in contact with your mentor to ask about setting your account up.
 
 Setup on Volvo:
+
 1. Connect to Georgia Tech's VPN or ssh into another machine on campus
 2. `ssh volvo.cc.gatech.edu`
 3. Clone Vortex to your home directory: `git clone --recursive https://github.com/vortexgpgpu/vortex.git`
@@ -19,21 +22,24 @@ Setup on Volvo:
 6. Run a test program: `./ci/blackbox.sh --cores=2 --app=dogfood`
 
 ### Nio
+
 Nio is a 20-core desktop server provided by HPArch. If you have access to Volvo, you also have access to Nio.
 
 Setup on Nio:
+
 1. Connect to Georgia Tech's VPN or ssh into another machine on campus
 2. `ssh nio.cc.gatech.edu`
 3. Clone Vortex to your home directory: `git clone --recursive https://github.com/vortexgpgpu/vortex.git`
 4. `source /opt/set_vortex_env_dev.sh` to set up the necessary environment variables.
 5. `make -s` in the `vortex` root directory
 6. Run a test program: `./ci/blackbox.sh --cores=2 --app=dogfood`
 
-
 ## Docker (Experimental)
+
 Docker allows for isolated pre-built environments to be created, shared and used. The emulation mode required for ARM-based processors will incur a decrease in performance. Currently, the dockerfile is not included with the official vortex repository and is not actively maintained or supported.
 
 ### Setup with Docker
+
 1. Clone repo recursively onto your local machine: `git clone --recursive https://github.com/vortexgpgpu/vortex.git`
 2. Download the dockerfile from [here](https://github.gatech.edu/gist/usubramanya3/f1bf3e953faa38a6372e1292ffd0b65c) and place it in the root of the repo.
 3. Build the Dockerfile into an image: `docker build --platform=linux/amd64 -t vortex -f dockerfile .`