diff --git a/README.md b/README.md index 5b62f5e48..3366433a5 100755 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ A pipeline framework for developing video and image processing applications. Sup Learn more about ApraPipes here https://apra-labs.github.io/ApraPipes. ## Build status -Aprapipes is automatically built and tested on Ubuntu (18.04 and 20.04), Jetson Boards (Jetpack 4.4) and Windows (11) x64 Visual Studio 2017 Community. +Aprapipes is automatically built and tested on Ubuntu (18.04 and 20.04), Jetson Boards (Jetpack 4.4) and Windows (11) x64 Visual Studio 2019 Community. |OS|Version|With Cuda|Tests|Status| |--|-------|---------|------|------| |Windows|2019|No|[![Test Results](https://gist.githubusercontent.com/kumaakh/f80af234a4aabedc69af3ee197f66944/raw/badge_Windows.svg)](https://gist.githubusercontent.com/kumaakh/f80af234a4aabedc69af3ee197f66944/raw/badge_Windows.svg)|[![CI-Win-NoCUDA](https://github.com/Apra-Labs/ApraPipes/actions/workflows/CI-Win-NoCUDA.yml/badge.svg)](https://github.com/Apra-Labs/ApraPipes/actions/workflows/CI-Win-NoCUDA.yml)| @@ -40,22 +40,24 @@ Aprapipes is automatically built and tested on Ubuntu (18.04 and 20.04), Jetson
Requirements + ### Prerequisites + + ### Visual Studio + * Install Visual Studio 2019 Community + * Install Desktop development C++ + * .NET Desktop development + * Universal Windows Development Platform + ### Cuda * Create an account on developer.nvidia.com if you're not already a member. Note : Otherwise the next step will show HTTP 404/403 error. * Windows 10/11 : [Cuda Toolkit 10.2](https://developer.nvidia.com/cuda-10.2-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal) or [CUDA Toolkit 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive?target_os=Windows). ### Cudnn * Download [Cudnn](https://developer.nvidia.com/rdp/cudnn-archive#a-collapse765-102) and extract files where cuda is installed. Note: Please be aware that this process requires some effort. Here are the necessary steps: - * Download the correct zip file matching your cuda version. _Do not download the exe/installer/deb package._ - * Windows: - * Download [this file](https://developer.nvidia.com/compute/cudnn/secure/8.3.2/local_installers/10.2/cudnn-windows-x86_64-8.3.2.44_cuda10.2-archive.zip). - - ### Prerequisites - * Install Visual Studio 2019 Community - * Install Desktop development C++ - * .NET Desktop development - * Universal Windows Development Platform - + * Download the correct zip file matching your cuda version. _Do not download the exe/installer/deb package._ + * Windows: + * Download [this file](https://developer.nvidia.com/compute/cudnn/secure/8.3.2/local_installers/10.2/cudnn-windows-x86_64-8.3.2.44_cuda10.2-archive.zip). + * Clone with submodules and LFS. ``` git clone --recursive https://github.com/Apra-Labs/ApraPipes.git @@ -66,9 +68,9 @@ Aprapipes is automatically built and tested on Ubuntu (18.04 and 20.04), Jetson
Build - ### Build Without Cuda Open PowerShell as an administrator and execute the following commands + ### Build Without Cuda If your windows system does not have an NVIDIA GPU use this script ``` build_windows_no_cuda.bat @@ -77,7 +79,10 @@ Aprapipes is automatically built and tested on Ubuntu (18.04 and 20.04), Jetson ``` build_windows_cuda.bat ``` - + ### To Build With Documentation + ``` + build_windows_cuda.bat --build-doc + ```
@@ -114,6 +119,8 @@ Aprapipes is automatically built and tested on Ubuntu (18.04 and 20.04), Jetson
Requirements + ### Prerequisites + ### Cuda * Create an account on developer.nvidia.com if you're not already a member. Note : Otherwise the next step will show HTTP 404/403 error. * Ubuntu 18.04/20.04: @@ -125,7 +132,8 @@ Aprapipes is automatically built and tested on Ubuntu (18.04 and 20.04), Jetson * Linux: * Download [this file](https://developer.nvidia.com/compute/cudnn/secure/8.3.2/local_installers/10.2/cudnn-linux-x86_64-8.3.2.44_cuda10.2-archive.tar.xz) - ### Prerequisites + * Clone with submodules and LFS. + ``` git clone --recursive https://github.com/Apra-Labs/ApraPipes.git ``` @@ -141,11 +149,15 @@ Aprapipes is automatically built and tested on Ubuntu (18.04 and 20.04), Jetson ### Build Without Cuda If your windows system does not have an NVIDIA GPU use this script ``` - sudo ./build_linux_no_cuda.sh + ./build_linux_no_cuda.sh ``` ### Build With Cuda ``` - sudo ./build_linux_cuda.sh + ./build_linux_cuda.sh + ``` + ### To Build With Documentation + ``` + ./build_linux_cuda.sh --build-doc ``` Build can take ~2 hours depending on the machine configuration. @@ -200,11 +212,14 @@ Aprapipes is automatically built and tested on Ubuntu (18.04 and 20.04), Jetson ``` chmod +x build_jetson.sh ``` - * ApraPipes builds CUDA version on Jerson Boads. + * ApraPipes builds CUDA version on Jetson Boards. ``` - sudo ./build_jetson.sh + ./build_jetson.sh + ``` + * To Build With Documentation + ``` + ./build_jetson.sh --build-doc ``` - Build can take ~12 hours on Jetson Nano. Note: Jetson build can also be done using Ubuntu 18.04 x86_64 Laptop via cross compilation.
@@ -290,13 +305,10 @@ This build will be fairly fast (~10 mins) as entire vcpkg cache comes down with git submodule update --init --recursive ``` ## Update Documentation - After making changes to the documentation located in the /docs/source folder, it's essential to regenerate the documentation by following the provided steps. Once regenerated, commit the new content to ensure the latest documentation is up-to-date. +To update documentation, refer to Documentation Guidelines in the [Contribution-Guidelines](https://github.com/Apra-Labs/ApraPipes/wiki/Contribution-Guidelines). ### To regenerate documentation +Run, +``` +./build_documentation.sh ``` -To build docs -apt-install get python-sphinx -pip install sphinx-rtd-theme -cd docs -make html -``` \ No newline at end of file diff --git a/build_jetson.sh b/build_jetson.sh index 000e8e434..ddc05079d 100755 --- a/build_jetson.sh +++ b/build_jetson.sh @@ -1,4 +1,4 @@ -apt-get install clang-format +sudo apt-get install clang-format clang-format -style=llvm -dump-config > .clang-format if ! command -v pip &> /dev/null; then # If pip is not available, download and install pip @@ -9,10 +9,44 @@ pip install pre-commit pre-commit install chmod +x build_scripts/build_dependencies_jetson_cuda.sh -./build_scripts/build_dependencies_jetson_cuda.sh +sudo ./build_scripts/build_dependencies_jetson_cuda.sh -chmod +x build_documentation.sh -./build_documentation.sh +if nvcc --version; then + USER_NAME=$(whoami) + TARGET_USER="$USER_NAME" + TARGET_HOME=$(eval echo ~$TARGET_USER) + + # Append lines to the target user's ~/.bashrc + if ! grep -qxF 'export VCPKG_FORCE_SYSTEM_BINARIES=1' $TARGET_HOME/.bashrc; then + echo 'export VCPKG_FORCE_SYSTEM_BINARIES=1' | sudo -u $TARGET_USER tee -a $TARGET_HOME/.bashrc + echo "VCPKG_FORCE_SYSTEM_BINARIES flag added in .bashrc" + else + echo "VCPKG_FORCE_SYSTEM_BINARIES flag already exists in .bashrc" + fi + + if ! grep -qxF 'export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}' $TARGET_HOME/.bashrc; then + echo 'export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}' | sudo -u $TARGET_USER tee -a $TARGET_HOME/.bashrc + echo "CUDA Binary Path added to .bashrc" + else + echo "CUDA Binary Path already exists in .bashrc" + fi + + if ! grep -qxF 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' $TARGET_HOME/.bashrc; then + echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' | sudo -u $TARGET_USER tee -a $TARGET_HOME/.bashrc + echo "CUDA Library Path added to .bashrc" + else + echo "CUDA Library Path already exists in .bashrc" + fi + + echo "Appended paths to ~/.bashrc and saved changes." + source ~/.bashrc + echo "Reloaded ~/.bashrc" +fi + +if [[ $1 == "--build-doc" ]]; then + chmod +x build_documentation.sh + ./build_documentation.sh +fi cd vcpkg ./bootstrap-vcpkg.sh diff --git a/build_linux_cuda.sh b/build_linux_cuda.sh index a90acf238..f37842c01 100755 --- a/build_linux_cuda.sh +++ b/build_linux_cuda.sh @@ -1,4 +1,4 @@ -apt-get install clang-format +sudo apt-get install clang-format clang-format -style=llvm -dump-config > .clang-format if ! command -v pip &> /dev/null; then # If pip is not available, download and install pip @@ -11,8 +11,55 @@ pre-commit install chmod +x build_scripts/build_dependencies_linux_cuda.sh sudo ./build_scripts/build_dependencies_linux_cuda.sh -chmod +x build_documentation.sh -./build_documentation.sh +if ! sudo nvcc --version &>/dev/null; then + USER_NAME=$(whoami) + cudnn_archives="/home/$USER_NAME/Downloads/cudnn-*.tar.xz" + + for archive in $cudnn_archives; do + if [ -e "$archive" ]; then + extracted_folder="/home/$USER_NAME/Downloads/$(basename "$archive" .tar.xz)" + if [ ! -d "$extracted_folder" ]; then + echo "Extracting $archive..." + tar xf "$archive" -C "/home/$USER_NAME/Downloads/" + else + echo "Archive already extracted: $extracted_folder" + fi + fi + done + + echo "Copying files..." + sudo cp -r /home/$USER_NAME/Downloads/cudnn-*/include/* /usr/local/cuda/include/ + sudo cp -r /home/$USER_NAME/Downloads/cudnn-*/lib/* /usr/local/cuda/lib64/ + + TARGET_USER="$USER_NAME" + TARGET_HOME=$(eval echo ~$TARGET_USER) + + # Append lines to the target user's ~/.bashrc + if ! grep -qxF 'export PATH=/usr/local/cuda/bin:${PATH}' $TARGET_HOME/.bashrc; then + echo 'export PATH=/usr/local/cuda/bin:${PATH}' | sudo -u $TARGET_USER tee -a $TARGET_HOME/.bashrc + echo "CUDA Binary Path added to .bashrc" + else + echo "CUDA Binary Path already exists in .bashrc" + fi + + if ! grep -qxF 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:${LD_LIBRARY_PATH}' $TARGET_HOME/.bashrc; then + echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:${LD_LIBRARY_PATH}' | sudo -u $TARGET_USER tee -a $TARGET_HOME/.bashrc + echo "CUDA Library Path added to .bashrc" + else + echo "CUDA Library Path already exists in .bashrc" + fi + + # Reload .bashrc + source ~/.bashrc + + echo "Appended line to ~/.bashrc and saved changes." + echo "Reloaded ~/.bashrc" +fi + +if [[ $1 == "--build-doc" ]]; then + chmod +x build_documentation.sh + ./build_documentation.sh +fi cd vcpkg ./bootstrap-vcpkg.sh diff --git a/build_linux_no_cuda.sh b/build_linux_no_cuda.sh index 6b9e2d129..5da2ce429 100755 --- a/build_linux_no_cuda.sh +++ b/build_linux_no_cuda.sh @@ -1,4 +1,4 @@ -apt-get install clang-format +sudo apt-get install clang-format clang-format -style=llvm -dump-config > .clang-format if ! command -v pip &> /dev/null; then # If pip is not available, download and install pip @@ -9,13 +9,15 @@ pip install pre-commit pre-commit install chmod +x build_scripts/build_dependencies_linux_no_cuda.sh -./build_scripts/build_dependencies_linux_no_cuda.sh +sudo ./build_scripts/build_dependencies_linux_no_cuda.sh chmod +x base/fix-vcpkg-json.sh ./base/fix-vcpkg-json.sh true false false -chmod +x build_documentation.sh -./build_documentation.sh +if [[ $1 == "--build-doc" ]]; then + chmod +x build_documentation.sh + ./build_documentation.sh +fi cd vcpkg ./bootstrap-vcpkg.sh diff --git a/build_scripts/build_dependencies_jetson_cuda.sh b/build_scripts/build_dependencies_jetson_cuda.sh index 19e10df4e..2cdd90949 100644 --- a/build_scripts/build_dependencies_jetson_cuda.sh +++ b/build_scripts/build_dependencies_jetson_cuda.sh @@ -30,15 +30,4 @@ if [ ! -d "/usr/local/cuda/include" ] || [ ! -d "/usr/local/cuda/lib64" ]; then exit 1 fi -if nvcc --version; then - TARGET_USER="$SUDO_USER" - TARGET_HOME=$(eval echo ~$TARGET_USER) - echo 'export VCPKG_FORCE_SYSTEM_BINARIES=1' | sudo -u $TARGET_USER tee -a $TARGET_HOME/.bashrc - echo 'export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}' | sudo -u $TARGET_USER tee -a $TARGET_HOME/.bashrc - echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' | sudo -u $TARGET_USER tee -a $TARGET_HOME/.bashrc - echo "Appended paths to ~/.bashrc and saved changes." - source ~/.bashrc - echo "Reloaded ~/.bashrc" -fi - echo "Dependencies verified and installed successfully." diff --git a/build_scripts/build_dependencies_linux_cuda.sh b/build_scripts/build_dependencies_linux_cuda.sh index 4b3ee1d9d..1580d3ad8 100755 --- a/build_scripts/build_dependencies_linux_cuda.sh +++ b/build_scripts/build_dependencies_linux_cuda.sh @@ -47,33 +47,4 @@ if [ ! -d "/usr/local/cuda/include" ] || [ ! -d "/usr/local/cuda/lib64" ]; then exit 1 fi -if ! nvcc --version &>/dev/null; then - userName=$(whoami) - cudnn_archives="/home/$userName/Downloads/cudnn-*.tar.xz" - - for archive in $cudnn_archives; do - if [ -e "$archive" ]; then - echo "Extracting $archive..." - tar xf "$archive" -C /home/$userName/Downloads/ - fi - done - - echo "Copying files..." - cp -r /home/$userName/Downloads/cudnn-*/include/* /usr/local/cuda/include/ - cp -r /home/$userName/Downloads/cudnn-*/lib/* /usr/local/cuda/lib64/ - - TARGET_USER="$SUDO_USER" - TARGET_HOME=$(eval echo ~$TARGET_USER) - - # Append lines to the target user's ~/.bashrc - echo 'export PATH=/usr/local/cuda/bin:${PATH}' | sudo -u $TARGET_USER tee -a $TARGET_HOME/.bashrc - echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:${LD_LIBRARY_PATH}' | sudo -u $TARGET_USER tee -a $TARGET_HOME/.bashrc - - # Reload .bashrc - source $TARGET_HOME/.bashrc - - echo "Appended line to ~/.bashrc and saved changes." - echo "Reloaded ~/.bashrc" -fi - -echo "Dependencies verified and installed successfully." \ No newline at end of file +echo "Dependencies verified and installed successfully." diff --git a/build_scripts/build_dependencies_linux_no_cuda.sh b/build_scripts/build_dependencies_linux_no_cuda.sh index 7526f808e..7e401a9f7 100644 --- a/build_scripts/build_dependencies_linux_no_cuda.sh +++ b/build_scripts/build_dependencies_linux_no_cuda.sh @@ -19,8 +19,8 @@ done # If there are missing dependencies, install them if [ "${#missing_dependencies[@]}" -gt 0 ]; then echo "Installing missing dependencies..." - apt-get update -qq - apt-get -y install "${missing_dependencies[@]}" + apt-get update -qq + apt-get -y install "${missing_dependencies[@]}" fi # Install Meson if not present diff --git a/build_scripts/build_dependencies_windows_cuda.ps1 b/build_scripts/build_dependencies_windows_cuda.ps1 index ff5218659..0e8ab88d8 100644 --- a/build_scripts/build_dependencies_windows_cuda.ps1 +++ b/build_scripts/build_dependencies_windows_cuda.ps1 @@ -22,10 +22,14 @@ if (-not (Test-Path "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\$cudaVer } else { $userName = $env:UserName $zipFilePath = "C:\Users\$userName\Downloads\cudnn-*.zip" - - Write-Host "Extracting zip file..." - - Expand-Archive -Path $zipFilePath -DestinationPath C:\Users\$userName\Downloads\ -Force + $extractedPath = "C:\Users\$userName\Downloads\cudnn-*\" + + if (-not (Test-Path $extractedPath)) { + Write-Host "Extracting zip file..." + Expand-Archive -Path $zipFilePath -DestinationPath "C:\Users\$userName\Downloads\" -Force + } else { + Write-Host "Already extracted files found." + } Write-Host "Copying files..." Copy-Item -Path "C:\Users\$userName\Downloads\cudnn-*\include\*.h" -Destination "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\$cudaVersion\include\" -Recurse diff --git a/build_windows_cuda.bat b/build_windows_cuda.bat index 0fd221876..8c98fd8d9 100644 --- a/build_windows_cuda.bat +++ b/build_windows_cuda.bat @@ -4,8 +4,10 @@ cd %batdir%/build_scripts powershell -nologo -executionpolicy bypass -File build_dependencies_windows_cuda.ps1 cd .. -@echo off -sh .\build_documentation.sh +IF "%1"=="--build-doc" ( + @echo off + sh .\build_documentation.sh +) @echo off set batdir=%~dp0 @@ -16,17 +18,15 @@ call bootstrap-vcpkg.bat vcpkg.exe integrate install cd .. - - SET VCPKG_ARGS=-DENABLE_CUDA=ON -DENABLE_WINDOWS=ON -DENABLE_LINUX=OFF -DCMAKE_TOOLCHAIN_FILE=../vcpkg/scripts/buildsystems/vcpkg.cmake -A x64 ../base mkdir _build cd _build cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo %VCPKG_ARGS% -cmake --build . +cmake --build . --config RelWithDebInfo cd .. rem goto :EOF mkdir _debugbuild cd _debugbuild cmake -DCMAKE_BUILD_TYPE=Debug %VCPKG_ARGS% -cmake --build . \ No newline at end of file +cmake --build . --config Debug \ No newline at end of file diff --git a/build_windows_no_cuda.bat b/build_windows_no_cuda.bat index a4077c902..abc4ba1d1 100644 --- a/build_windows_no_cuda.bat +++ b/build_windows_no_cuda.bat @@ -10,8 +10,10 @@ cd %batdir%/base powershell -nologo -executionpolicy bypass -File fix-vcpkg-json.ps1 -removeCUDA cd .. -@echo off -sh .\build_documentation.sh +IF "%1"=="--build-doc" ( + @echo off + sh .\build_documentation.sh +) cd vcpkg call bootstrap-vcpkg.bat @@ -21,11 +23,11 @@ cd .. mkdir _buildNoCuda cd _buildNoCuda cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_CUDA=OFF -DENABLE_WINDOWS=ON -DENABLE_LINUX=OFF -DCMAKE_TOOLCHAIN_FILE=../vcpkg/scripts/buildsystems/vcpkg.cmake -A x64 ../base -cmake --build . +cmake --build . --config RelWithDebInfo cd .. rem goto :EOF mkdir _debugbuildNoCuda cd _debugbuildNoCuda cmake -DCMAKE_BUILD_TYPE=Debug -DENABLE_CUDA=OFF -DENABLE_WINDOWS=ON -DENABLE_LINUX=OFF -DCMAKE_TOOLCHAIN_FILE=../vcpkg/scripts/buildsystems/vcpkg.cmake -A x64 ../base -cmake --build . \ No newline at end of file +cmake --build . --config Debug \ No newline at end of file