Skip to content

Commit

Permalink
build: Build system overhaul (#1686)
Browse files Browse the repository at this point in the history
This PR introduces a new CMake-based build system for ares.

Broadly, the goals of this build system are to:

* Modernize the build process somewhat, and make ares more approachable
for new developers
* Improve ares's dependency handling
* Expand and standardize platform/compiler/architecture support for ares
* Improve project code quality
* Improve build performance
* Lay groundwork for easier support of more platforms in the future

There are a few notable changes that this PR brings about. Briefly:

* Dynamic dependencies (librashader, MoltenVK, SDL, slang-shaders) are
no longer built in-tree
* Build system generation and building now take place as two separate
steps

Other notes:

* The legacy build system is deprecated, but is planned to still
function until such time as it is removed in a later PR

# New Features

* ares can now generate projects for Xcode and Visual Studio 17 2022.
Xcode is the recommended generator on macOS, supporting universal
binaries, advanced profiling and GPU debugging. Visual Studio projects
can be configured with either Clang-CL or MSVC, and may be generated for
either x86_64 or arm64.

* On macOS and Windows, dynamic dependencies are now built by
[ares-deps](https://github.com/jcm93/ares-deps) and fetched at
configure-time. This is to reduce build requirements for development,
define explicit versioning for dependencies, reduce build times
generally, and streamline CI workflows.

* Builds can now be configured through the default Windows command line
rather than requiring an MSYS2/MinGW shell.

* Ccache is also now universally supported to improve build performance.
Compile times should be further reduced with the addition of
precompilation of certain headers.

* A variety of other build system generators are now supported on all
platforms, including multi-configuration Ninja and applicable types of
Makefiles.

* Compiler diagnostic flags with clang have been standardized across all
platforms where possible[^1]. Currently, diagnostic flags have rough
parity with the old build system, but one goal of the new build system
is to introduce more strict compiler settings, to eliminate certain
classes of bugs and generally increase code quality.

# Supported Platforms

ARM64 and x86_64 architectures are officially supported across macOS,
Windows, Linux, and BSD[^1]. clang, Clang-CL, gcc, and MSVC are all
supported on applicable platforms. MSYS2/MinGW is supported along with
configuration and builds via the default Windows shell.

32-bit builds have not been tested.

# Current Status

Builds across all three of macOS, Windows, Linux, and BSD are generally
expected to work. Detailed build instructions can be found on the new
[ares wiki](https://github.com/ares-emulator/ares/wiki). CI workflows
are present and implemented but are still a work in progress. More
complete documentation is also a work in progress.

#### macOS

* Xcode, Ninja, Ninja Multi-Config, and Makefile generators are
functional
* Unlike Windows, multi-architecture builds are supported; pass
`-DCMAKE_OSX_ARCHITECTURES` as appropriate.
* Builds should generate a fully functional and relocatable app bundle
within the build directory, with relative rpaths; `install` rules are
placeholders.

#### Windows

* Visual Studio 17 2022, Ninja, Ninja Multi-Config, and Makefile
generators are functional
* Only single-architecture builds are currently supported
* Similarly to macOS, the build step generates a fully functional and
relocatable rundir with placeholder `install` commands.

#### Linux / BSD

* Unlike on macOS and Windows, it is required that the developer have
packages installed for libraries ares links against. A list of required
and optional libraries is found in the README.
* Build will output to a staging directory `rundir`; ares only uses
system libraries so this rundir should be fully relocatable
* `install` will perform a standard installation as CMake deems
appropriate for your distribution; you may pass the `--prefix` argument
as needed
* Maintainers may package as appropriate; a new Flatpak may be created
as part of this process, but has not yet

> [!IMPORTANT]
> These tasks should be completed before this PR moves out of the draft
state:
> 
> * ~~CI workflows~~ - incomplete
> * ~~BSD support~~
> * ~~Windows SDL added to ares-deps~~
> * ~~ares-deps should probably be migrated to the ares-emulator
organization~~
> * More documentation
> * Miscellaneous cleanup, fixing issues found in testing, incorporating
general feedback

Nevertheless, now that builds are functioning on all three platforms, I
wanted to get this PR out there in a draft state to solicit testing and
feedback, as well as to give Linux package maintainers a head start in
terms of packaging.

> [!NOTE]
> While the PR is in a draft state, there may be breaking changes,
things may temporarily stop functioning, commits may be squashed, etc.
Thank you for testing and please report any issues you encounter!

* More detailed documentation on the internals of the build system is
available
[here](https://github.com/jcm93/ares/blob/cmake/cmake/README.md).

### Miscellaneous changes/points

* .gitignore is reorganized
* [gersemi](https://github.com/BlankSpruce/gersemi) is used for CMake
formatting, hence the presence of a `.gersemirc` file
* SDL on Windows is now a dynamic dependency.
* macOS asset files are reworked into .xcassets archives
* new macOS local build script introduced
* CI will include debug symbols for easier crash debugging

[^1]: Indicates something that might not be finished yet, or is subject
to change

---------

Co-authored-by: invertego <invertego@users.noreply.github.com>
Co-authored-by: Stefan Schlosser <bsdcode@disroot.org>
Co-authored-by: John Tur <john-tur@outlook.com>
  • Loading branch information
4 people authored Dec 31, 2024
1 parent ecdce5e commit 8433666
Show file tree
Hide file tree
Showing 143 changed files with 8,901 additions and 160 deletions.
11 changes: 11 additions & 0 deletions .gersemirc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/BlankSpruce/gersemi/master/gersemi/configuration.schema.json

color: false
definitions: [ares/CMakeLists.txt]
line_length: 120
indent: 2
list_expansion: favour-inlining
quiet: false
unsafe: false
warn_about_unknown_commands: false
workers: 10
25 changes: 25 additions & 0 deletions .github/scripts/build_macos.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env sh
set -euo pipefail

# configure
cmake --preset $TARGET_PRESET

# change into the build directory
pushd build

# build; pipe output to render warnings neatly
xcodebuild -configuration RelWithDebInfo \
DEBUG_INFORMATION_FORMAT="dwarf-with-dsym" \
2>&1 | xcbeautify --renderer github-actions

popd

# package debug symbols

# move dependency dSYMs to build output directory
ditto .deps/ares-deps-macos-universal/lib/*.dSYM build/desktop-ui/RelWithDebInfo

mkdir build/dsyms

# ares.app.dSYM already exists alongside ares.app; package it with the rest
mv build/desktop-ui/RelWithDebInfo/*.dSYM build/dSYMs/
24 changes: 24 additions & 0 deletions .github/scripts/build_windows.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/usr/bin/env sh
set -euo pipefail

if [ "$CROSS_COMPILE" = true ]; then
cmake --preset $NATIVE_PRESET -B build_native
pushd build_native
# build sourcery natively so it may be invoked during cross-compilation
cmake --build . --target sourcery --config RelWithDebInfo
popd
fi

cmake --preset $TARGET_PRESET
pushd build
cmake --build . --config RelWithDebInfo

if [ "$CROSS_COMPILE" = true ]; then
cp ../.deps/ares-deps-windows-arm64/lib/*.pdb desktop-ui/rundir/
else
cp ../.deps/ares-deps-windows-x64/lib/*.pdb desktop-ui/rundir/
fi

mkdir PDBs
mv desktop-ui/rundir/*.pdb PDBs/
popd
46 changes: 46 additions & 0 deletions .github/scripts/package_artifacts.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
set -eu
case ${GITHUB_REF} in
refs/tags/*) suffix="-${GITHUB_REF#refs/tags/}" ;;
refs/heads/master) suffix="-nightly" ;;
*) suffix="" ;;
esac

srcdir="${GITHUB_WORKSPACE}/src"
bindir="${GITHUB_WORKSPACE}/bin"

# Hack: Workaround for GitHub artifacts losing attributes.
chmod +x ${bindir}/ares-macos-universal/ares.app/Contents/MacOS/ares

for package in macos-universal
do
mkdir "${package}"
cd "${package}"

# Package ares.
outdir=ares${suffix}
mkdir ${outdir}
mkdir ${outdir}-dSYMs
cp -ar ${bindir}/ares-${package}-dSYMs/*.dSYM ${outdir}-dSYMs
cp -ar ${bindir}/ares-${package}/*.app ${outdir}
zip -r ../ares-${package}.zip ${outdir}
zip -r ../ares-${package}-dSYMs.zip ${outdir}-dSYMs

cd -
done

for package in windows-x64 windows-clang-cl-x64 windows-clang-cl-arm64
do
mkdir "${package}"
cd "${package}"

# Package ares.
outdir=ares${suffix}
mkdir ${outdir}
mkdir ${outdir}-PDBs
cp -ar ${bindir}/ares-${package}-PDBs/*.pdb ${outdir}-PDBs
zip -r ../ares-${package}-PDBs.zip ${outdir}-PDBs
rm -rf ${bindir}/ares-${package}/PDBs
cp -ar ${bindir}/ares-${package}/* ${outdir}
zip -r ../ares-${package}.zip ${outdir}
cd -
done
180 changes: 180 additions & 0 deletions .github/workflows/build_new.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
name: Build (New)
on:
push
permissions:
contents: write
jobs:
build:
strategy:
fail-fast: false
matrix:
program:
- ares
config:
- RelWithDebInfo
platform:
- name: windows-x64
os: windows-latest
compiler: clang++
shell: 'msys2 {0}'
msystem: clang64
install: mingw-w64-clang-x86_64-clang
target-cmake-preset: windows-ci-mingw-native
- name: windows-clang-cl-x64
os: windows-latest
windres: rc
shell: bash
target-cmake-preset: windows-ci-cl-native
- name: windows-clang-cl-arm64
os: windows-latest
windres: rc
shell: bash
target-cmake-preset: windows-ci-cl-cross
native-cmake-preset: windows-ci-cl-native
- name: windows-msvc-x64
os: windows-latest
windres: rc
shell: bash
target-cmake-preset: windows-ci-msvc-native
- name: windows-msvc-arm64
os: windows-latest
windres: rc
shell: bash
target-cmake-preset: windows-ci-msvc-cross
native-cmake-preset: windows-ci-msvc-native
- name: macos-universal
os: macos-15
compiler: clang++
shell: sh
install: cmake ccache xcbeautify
target-cmake-preset: macos-ci-universal
name: ${{ matrix.program }}-${{ matrix.platform.name }}
runs-on: ${{ matrix.platform.os }}
defaults:
run:
shell: ${{ matrix.platform.shell }}
steps:
- name: Install MSYS2 Dependencies
if: matrix.platform.shell == 'msys2 {0}'
uses: msys2/setup-msys2@v2
with:
msystem: ${{ matrix.platform.msystem }}
install: make git cmake ccache ninja ${{ matrix.platform.install }}
- name: "macOS: Import Certificate"
if: runner.os == 'macOS'
uses: apple-actions/import-codesign-certs@v3
with:
p12-file-base64: ${{ secrets.MACOS_CERTIFICATE_DATA }}
p12-password: ${{ secrets.MACOS_CERTIFICATE_PASSPHRASE }}
- name: Checkout source code
uses: actions/checkout@v4
- name: Install macOS Dependencies
if: runner.os == 'macOS'
run: |
brew install ${{ matrix.platform.install }}
- name: "Build: Windows"
if: runner.os != 'macOS'
run: .github/scripts/build_windows.sh
env:
CROSS_COMPILE: ${{ matrix.platform.native-cmake-preset != '' }}
NATIVE_PRESET: ${{ matrix.platform.native-cmake-preset }}
TARGET_PRESET: ${{ matrix.platform.target-cmake-preset }}
- name: "Build: macOS"
if: runner.os == 'macOS'
run: .github/scripts/build_macos.sh
env:
TARGET_PRESET: ${{ matrix.platform.target-cmake-preset }}
MACOS_CERTIFICATE_NAME: ${{ secrets.MACOS_CERTIFICATE_NAME }}
MACOS_NOTARIZATION_TEAMID: ${{ secrets.MACOS_NOTARIZATION_TEAMID }}
MACOS_KEYCHAIN_NAME: ares-macos-keychain
MACOS_KEYCHAIN_PASSWORD: ${{ secrets.MACOS_KEYCHAIN_PASSWORD }}
- name: "Windows: sign"
if: runner.os == 'Windows' && (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/'))
uses: DanaBear/code-sign-action@98c79121b376beab8d6a9484f445089db4461bca
with:
certificate: '${{ secrets.WIN_CERTIFICATE }}'
password: '${{ secrets.WIN_PASSWORD }}'
certificatesha1: '${{ secrets.WIN_CERTHASH }}'
certificatename: '${{ secrets.WIN_CERTNAME }}'
folder: 'build/desktop-ui/rundir/'
timestampUrl: 'http://timestamp.digicert.com'
- name: "macOS: notarize"
if: runner.os == 'macOS' && (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/'))
run: |
ditto -c -k --keepParent ${{ github.workspace }}/build/desktop-ui/RelWithDebInfo/ares.app /tmp/ares.zip
xcrun notarytool submit /tmp/ares.zip --apple-id "$MACOS_NOTARIZATION_USERNAME" --password "$MACOS_NOTARIZATION_PASSWORD" --team-id "$MACOS_NOTARIZATION_TEAMID" --wait
xcrun stapler staple ${{ github.workspace }}/build/desktop-ui/RelWithDebInfo/ares.app
env:
MACOS_NOTARIZATION_USERNAME: ${{ secrets.MACOS_NOTARIZATION_USERNAME }}
MACOS_NOTARIZATION_PASSWORD: ${{ secrets.MACOS_NOTARIZATION_PASSWORD }}
MACOS_NOTARIZATION_TEAMID: ${{ secrets.MACOS_NOTARIZATION_TEAMID }}
- name: Upload Build (Windows)
if: runner.os != 'macOS'
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.program }}-${{ matrix.platform.name }}
path: build/desktop-ui/rundir/*
- name: Upload Build (macOS)
if: runner.os == 'macOS'
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.program }}-${{ matrix.platform.name }}
path: build/desktop-ui/RelWithDebInfo/
- name: Upload Debug Symbols (Windows)
if: runner.os != 'macOS'
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.program }}-${{ matrix.platform.name }}-PDBs
path: build/PDBs/*
- name: Upload Debug Symbols (macOS)
if: runner.os == 'macOS'
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.program }}-${{ matrix.platform.name }}-dSYMs
path: build/dSYMs/*

release:
if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
needs:
- build
steps:
- uses: actions/checkout@v4
with:
path: 'src'
- name: Download Artifacts
uses: actions/download-artifact@v4
with:
path: 'bin'
- name: Package Artifacts
run: src/.github/scripts/package_artifacts.sh
- name: Check Release Tag ☑️
id: check
run: |
if [[ ${GITHUB_REF_NAME} == master ]];
then
echo "tag=nightly" >> $GITHUB_OUTPUT
now=$(date +'%Y-%m-%d')
versionName="nightly ${now}"
echo "versionName=${versionName}" >> $GITHUB_OUTPUT
else
echo "tag=${GITHUB_REF_NAME}" >> $GITHUB_OUTPUT
echo "versionName=${GITHUB_REF_NAME}" >> $GITHUB_OUTPUT
fi
- name: Create Release 🛫
id: create_release
uses: softprops/action-gh-release@c062e08bd532815e2082a85e87e3ef29c3e6d191
with:
draft: ${{ github.ref != 'refs/heads/master' }}
tag_name: ${{ steps.check.outputs.tag }}
name: ares ${{ steps.check.outputs.versionName }}
files: |
${{ github.workspace }}/ares-macos-universal.zip
${{ github.workspace }}/ares-macos-universal-dSYMs.zip
${{ github.workspace }}/ares-windows-x64.zip
${{ github.workspace }}/ares-windows-x64-PDBs.zip
${{ github.workspace }}/ares-windows-clang-cl-x64.zip
${{ github.workspace }}/ares-windows-clang-cl-x64-PDBs.zip
${{ github.workspace }}/ares-windows-clang-cl-arm64.zip
${{ github.workspace }}/ares-windows-clang-cl-arm64-PDBs.zip
54 changes: 38 additions & 16 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,18 +1,40 @@
*.moc
*.user
*.xcuserdata
.vs/
.vscode/
.idea/
cmake-*/
obj/
out/
obj-amd64/
obj-arm64/
out-amd64/
out-arm64/
# Exclude everything
/*

# Except for default project files
!/.github
!/ares
!/desktop-ui
!/hiro
!/cmake
!/libco
!/mia
!/nall
!/ruby
!/scripts
!/tests/i8080
!/thirdparty
!/tools
!/GNUMakefile
!/LICENSE
!/README.md
!.editorconfig
!.gitignore
!.gitattributes
!.gersemirc
!deps.json
!CMakeLists.txt
!CMakePresets.json

thirdparty/SDL/SDL
thirdparty/SDL/libSDL2-2.0.0.dylib
macos-xcode/
.swiftpm
*.xcodeproj
desktop-ui/out
desktop-ui/obj
ruby/video/metal/shaders.metallib
ruby/video/metal/shaders.ir
ares/ares/resource/resource.cpp
ares/ares/resource/resource.hpp
hiro/resource/resource.cpp
hiro/resource/resource.hpp
desktop-ui/resource/resource.cpp
desktop-ui/resource/resource.hpp
51 changes: 51 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
cmake_minimum_required(VERSION 3.28...3.30)

include("${CMAKE_CURRENT_SOURCE_DIR}/cmake/common/bootstrap.cmake" NO_POLICY_SCOPE)

project(ares VERSION ${ARES_VERSION_CANONICAL})

include(compilerconfig)
include(defaults)
include(helpers)

add_subdirectory(cmake)

add_subdirectory(thirdparty)

add_subdirectory(nall)
add_subdirectory(libco)
add_subdirectory(ruby)
add_subdirectory(hiro)

# gersemi: off
set(
ARES_CORES
a26 fc sfc sg ms md ps1 pce ng msx cv myvision gb gba ws ngp spec n64
CACHE INTERNAL LIST
)
# gersemi: on

add_subdirectory(ares)
add_subdirectory(mia)
add_subdirectory(desktop-ui)

option(ARES_BUILD_OPTIONAL_TARGETS "Include supplemental tools and tests" OFF)

if(ARES_BUILD_OPTIONAL_TARGETS)
add_subdirectory(tests/i8080)
if(NOT OS_WINDOWS AND NOT OS_MACOS)
add_subdirectory(tools/genius)
else()
target_disable_subproject(genius "genius (database editor)")
endif()
add_subdirectory(tools/mame2bml)
else()
target_disable_subproject(i8080 "i8080 processor test harness")
target_disable_subproject(mame2bml "mame2bml (MAME manifest converter)")
target_disable_subproject(genius "genius (database editor)")
endif()

add_subdirectory(tools/sourcery)

message_configuration()
return()
Loading

0 comments on commit 8433666

Please sign in to comment.