Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update hictk to v2.0.0 #50881

Merged
merged 48 commits into from
Oct 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
bdf70eb
Bump version
robomics Sep 23, 2024
60e7a38
Update version range
robomics Sep 23, 2024
3150e32
Bugfix
robomics Sep 23, 2024
1dcaa51
Drop trasitive dependencies
robomics Sep 23, 2024
fbb65b7
Debugging
robomics Sep 23, 2024
49637e6
Debugging
robomics Sep 23, 2024
9e0d730
Bugfix
robomics Sep 23, 2024
1665615
Debugging
robomics Sep 23, 2024
93279b3
Debugging
robomics Sep 23, 2024
c73e633
Bugfix
robomics Sep 23, 2024
8a68637
Debugging
robomics Sep 23, 2024
c6c0383
Install all test deps using conda
robomics Sep 24, 2024
9da0827
Debugging
robomics Sep 24, 2024
03b0bb0
Debugging
robomics Sep 24, 2024
f0db1f6
Bugfix
robomics Sep 24, 2024
a24a32b
Bugfix
robomics Sep 24, 2024
cbb0958
Bugfix
robomics Sep 24, 2024
bd58e46
Bugfix
robomics Sep 24, 2024
e4c94cd
Update doc URL
robomics Oct 8, 2024
fec89b0
Update URL
robomics Oct 8, 2024
081aaae
Fix tests
robomics Oct 8, 2024
ba1f9be
Update patch
robomics Oct 8, 2024
9fc9793
Debugging
robomics Oct 8, 2024
e45b934
What the hell is eating all the memory!?
robomics Oct 8, 2024
c3b74c6
Debugging
robomics Oct 8, 2024
47404a0
Debugging
robomics Oct 8, 2024
c4e46aa
Debugging
robomics Oct 8, 2024
0a405cd
Debugging
robomics Oct 8, 2024
b75838d
Remove debug code
robomics Oct 8, 2024
16b01e7
Try to see if clang is less memory-hungry
robomics Oct 8, 2024
8fb03a2
Bugfix
robomics Oct 8, 2024
c75b713
Can we dare using clang 19?
robomics Oct 8, 2024
983b0bd
Update c_stdlib pinning
robomics Oct 8, 2024
42714a1
Switch back to clang 18
robomics Oct 8, 2024
769b4b6
Try to see if we can afford to turn on parallel compilation
robomics Oct 8, 2024
9e348a0
Debugging
robomics Oct 8, 2024
7c4a761
Try to use clang 19 one more time
robomics Oct 8, 2024
7030c47
Revert "Try to use clang 19 one more time"
robomics Oct 8, 2024
a43dd22
Run unit tests with a deterministic order
robomics Oct 8, 2024
4347965
Update URL
robomics Oct 8, 2024
e648f2e
Fix macOS builds
robomics Oct 8, 2024
7b49337
Actually, let's try a different solution first
robomics Oct 8, 2024
947d8ab
Revert "Actually, let's try a different solution first"
robomics Oct 8, 2024
81abbe1
Bugfix
robomics Oct 9, 2024
ed1ca0e
Debugging
robomics Oct 9, 2024
1e1c082
Update URL
robomics Oct 14, 2024
a14129a
Update patch
robomics Oct 14, 2024
a99416d
Update URL
robomics Oct 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 16 additions & 8 deletions recipes/hictk/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

export CONAN_NON_INTERACTIVE=1

export CMAKE_BUILD_PARALLEL_LEVEL=1 # ${CPU_COUNT}
export CMAKE_BUILD_PARALLEL_LEVEL=${CPU_COUNT}
export CTEST_PARALLEL_LEVEL=${CPU_COUNT}

if [[ ${DEBUG_C} == yes ]]; then
Expand All @@ -19,23 +19,31 @@ trap "rm -rf '$scratch'" EXIT

declare -a CMAKE_PLATFORM_FLAGS
if [[ ${HOST} =~ .*darwin.* ]]; then
export MACOSX_DEPLOYMENT_TARGET=10.15 # Required to use std::filesystem
# https://conda-forge.org/docs/maintainer/knowledge_base/#newer-c-features-with-old-sdk
export CXXFLAGS="${CXXFLAGS} -D_LIBCPP_DISABLE_AVAILABILITY"
CMAKE_PLATFORM_FLAGS+=(-DCMAKE_OSX_SYSROOT="${CONDA_BUILD_SYSROOT}")
conan_profile='apple-clang'

# https://github.com/conda/conda-build/issues/4392
for toolname in "otool" "install_name_tool"; do
tool=$(find "${BUILD_PREFIX}/bin/" -name "*apple*-$toolname")
mv "${tool}" "${tool}.bak"
ln -s "/Library/Developer/CommandLineTools/usr/bin/${toolname}" "$tool"
done
robomics marked this conversation as resolved.
Show resolved Hide resolved
else
CMAKE_PLATFORM_FLAGS+=(-DCMAKE_TOOLCHAIN_FILE="${RECIPE_DIR}/cross-linux.cmake")
conan_profile='gcc'
conan_profile='clang'
fi

# Remember to update these profiles when bioconda's compiler toolchains are updated
mkdir -p "$CONAN_HOME/profiles/"
ln -s "${RECIPE_DIR}/conan_profiles/$conan_profile" "$CONAN_HOME/profiles/$conan_profile"

# Remove unnecessary dependencies from conanfile.txt
patch conanfile.py < "${RECIPE_DIR}/conanfile.py.patch"
# Remove unnecessary dependencies from conanfile.py
patch conanfile.Dockerfile.py < "${RECIPE_DIR}/conanfile.Dockerfile.py.patch"

# Install header-only deps
conan install conanfile.py \
conan install conanfile.Dockerfile.py \
--build="*" \
-pr:b "$conan_profile" \
-pr:h "$conan_profile" \
Expand All @@ -52,6 +60,7 @@ cmake -DCMAKE_BUILD_TYPE="$CMAKE_BUILD_TYPE" \
-DBUILD_SHARED_LIBS=ON \
-DENABLE_DEVELOPER_MODE=OFF \
-DHICTK_ENABLE_TESTING=ON \
-DHICTK_ENABLE_FUZZY_TESTING=OFF \
-DHICTK_BUILD_EXAMPLES=OFF \
-DHICTK_BUILD_BENCHMARKS=OFF \
-DHICTK_BUILD_TOOLS=ON \
Expand All @@ -66,10 +75,9 @@ cmake -DCMAKE_BUILD_TYPE="$CMAKE_BUILD_TYPE" \
cmake --build build/

ctest --test-dir build/ \
--schedule-random \
--output-on-failure \
--no-tests=error \
--timeout 100
--timeout 180

cmake --install build/

Expand Down
8 changes: 8 additions & 0 deletions recipes/hictk/conan_profiles/clang
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=17
compiler.libcxx=libstdc++11
compiler.version=18
os=Linux
66 changes: 66 additions & 0 deletions recipes/hictk/conanfile.Dockerfile.py.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
diff --git a/conanfile.Dockerfile.py b/conanfile.Dockerfile.py
index c9e6d177..b190d2df 100644
--- a/conanfile.Dockerfile.py
+++ b/conanfile.Dockerfile.py
@@ -36,26 +36,10 @@ class HictkConan(ConanFile):

def requirements(self):
self.requires("bshoshany-thread-pool/4.1.0#be1802a8768416a6c9b1393cf0ce5e9c")
- self.requires("bzip2/1.0.8#d00dac990f08d991998d624be81a9526")
- self.requires("cli11/2.4.2#1b431bda2fb2cd3efed633899abcd8cc")
self.requires("concurrentqueue/1.0.4#1e48e1c712bcfd892087c9c622a51502")
- self.requires("fast_float/6.1.5#e067b96a6271d1b4c255858ca9805bdd")
- self.requires("fmt/11.0.2#5c7438ef4d5d69ab106a41e460ce11f3", force=True)
- self.requires("hdf5/1.14.4.3#df1467d7374938c231edbe10e83f2bb4", force=True)
- self.requires("highfive/2.10.0#3d1bd25944a57fa1bc30a0a22923d528")
- self.requires("libarchive/3.7.6#11e70b88b334f684eb9f6b65f287c81e")
- self.requires("libdeflate/1.22#f95aebe763153ccbc4cc76c023e42e5a")
- self.requires("lz4/1.10.0#68a01ece147a441b463d8cefea68d555", force=True)
- self.requires("lzo/2.10#5725914235423c771cb1c6b607109b45")
- self.requires("nlohmann_json/3.11.3#45828be26eb619a2e04ca517bb7b828d")
self.requires("parallel-hashmap/1.4.0#36ac84df77219748440cdb0f23624d56")
self.requires("readerwriterqueue/1.0.6#aaa5ff6fac60c2aee591e9e51b063b83")
self.requires("span-lite/0.11.0#519fd49fff711674cfed8cd17d4ed422")
- self.requires("spdlog/1.14.1#972bbf70be1da4bc57ea589af0efde03")
- self.requires("tomlplusplus/3.4.0#85dbfed71376fb8dc23cdcc0570e4727")
- self.requires("xz_utils/5.4.5#b885d1d79c9d30cff3803f7f551dbe66")
- self.requires("zstd/1.5.6#afefe79a309bc2a7b9f56c2093504c8b", force=True)
- self.requires("zlib/1.3.1#f52e03ae3d251dec704634230cd806a2")

def validate(self):
if self.settings.get_safe("compiler.cppstd"):
@@ -64,34 +48,3 @@ class HictkConan(ConanFile):
def configure(self):
if self.settings.compiler in ["clang", "gcc"]:
self.settings.compiler.libcxx = "libstdc++11"
-
- self.options["fmt"].header_only = True
- self.options["hdf5"].enable_cxx = False
- self.options["hdf5"].hl = False
- self.options["hdf5"].threadsafe = False
- self.options["hdf5"].parallel = False
- self.options["highfive"].with_boost = False
- self.options["highfive"].with_eigen = False
- self.options["highfive"].with_opencv = False
- self.options["highfive"].with_xtensor = False
- self.options["libarchive"].with_acl = False
- self.options["libarchive"].with_zlib = True
- self.options["libarchive"].with_bzip2 = True
- self.options["libarchive"].with_libxml2 = False
- self.options["libarchive"].with_expat = False
- self.options["libarchive"].with_iconv = False
- self.options["libarchive"].with_acl = False
- self.options["libarchive"].with_pcreposix = False
- self.options["libarchive"].with_cng = False
- self.options["libarchive"].with_nettle = False
- self.options["libarchive"].with_openssl = False
- self.options["libarchive"].with_libb2 = False
- self.options["libarchive"].with_lz4 = True
- self.options["libarchive"].with_lzo = True
- self.options["libarchive"].with_lzma = True
- self.options["libarchive"].with_zstd = True
- self.options["libarchive"].with_mbedtls = False
- self.options["libarchive"].with_xattr = False
- self.options["libarchive"].with_pcre2 = False
- self.options["spdlog"].header_only = True
- self.options["zstd"].build_programs = False
90 changes: 0 additions & 90 deletions recipes/hictk/conanfile.py.patch

This file was deleted.

8 changes: 8 additions & 0 deletions recipes/hictk/conda_build_config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
c_compiler: # [linux]
- clang # [linux]
c_compiler_version: # [linux]
- 18 # [linux]
cxx_compiler: # [linux]
- clangxx # [linux]
cxx_compiler_version: # [linux]
- 18 # [linux]
63 changes: 20 additions & 43 deletions recipes/hictk/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
{% set name = "hictk" %}
{% set version = "1.0.0" %}
{% set sha256 = "e337f52658a257eb6265e211dd3cef6b73db40bdc5b5a6433b47ce4faa6006fb" %}
{% set version = "2.0.0" %}
{% set sha256 = "714db992756867dd885581b58e61bad190cbd0f118168de69bc9714b18f9ad2f" %}

package:
name: {{ name|lower }}
version: {{ version }}

build:
number: 2
number: 0
run_exports:
- {{ pin_subpackage('hictk', max_pin='x') }}

Expand All @@ -22,24 +22,28 @@ requirements:
- cmake >=3.25
- conan >=2
- make
- catch2 >=3.5
- cli11 >=2.3
- catch2 >=3.7
- cli11 >=2.4
- eigen >=3.4
- fast_float >=5
- fmt >=10
- highfive >=2.8
- spdlog >=1.12
- fast_float >=6
- fmt >=11
- highfive >=2.9
- nlohmann_json >=3.11
- tomlplusplus >=3.3
- spdlog >=1.14
robomics marked this conversation as resolved.
Show resolved Hide resolved

host:
- hdf5 >=1.12
- libarchive >=3
- libarrow >=16
- libdeflate
- libdeflate >=1
- zstd >=1.5

run:
- hdf5 >=1.12
- libarchive >=3
- libarrow >=16
- libdeflate
- libdeflate >=1
- zstd >=1.5

about:
Expand All @@ -48,45 +52,18 @@ about:
license_file: LICENSE
summary: Blazing fast toolkit to work with .hic and .cool files
dev_url: https://github.com/paulsengroup/{{ name }}
doc_url: https://github.com/paulsengroup/{{ name }}#readme
doc_url: https://{{ name }}.readthedocs.io/en/stable/

test:
requires:
- python >=3.11,<3.13
- pip
- curl
- cooler>=0.9.3
- perl-digest-sha1
- xz
- zstd
source_files:
- cmake/FetchTestDataset.cmake
- test/scripts/check_test_files_exist.sh
- test/scripts/compare_matrix_files.sh
- test/scripts/compare_plain_files.sh
- test/scripts/compare_weights.py
- test/scripts/hictk_balance_ice.sh
- test/scripts/hictk_balance_scale.sh
- test/scripts/hictk_balance_vc.sh
- test/scripts/hictk_convert_hic2cool.sh
- test/scripts/hictk_convert_cool2hic.sh
- test/scripts/hictk_dump_cells.sh
- test/scripts/hictk_dump_normalizations.sh
- test/scripts/hictk_dump_resolutions.sh
- test/scripts/hictk_dump_chroms.sh
- test/scripts/hictk_dump_bins.sh
- test/scripts/hictk_dump_gw.sh
- test/scripts/hictk_dump_cis.sh
- test/scripts/hictk_dump_trans.sh
- test/scripts/hictk_dump_balanced.sh
- test/scripts/hictk_fix_mcool.sh
- test/scripts/hictk_load_coo.sh
- test/scripts/hictk_load_bg2.sh
- test/scripts/hictk_load_4dn.sh
- test/scripts/hictk_merge.sh
- test/scripts/hictk_rename_chromosomes.sh
- test/scripts/hictk_validate.sh
- test/scripts/hictk_zoomify.sh
- test/scripts/nproc.sh
- test/scripts/readlink.sh
- test/scripts/shuffle.sh
- test/integration/
commands:
- hictk --help
- hictk --version
Expand Down
Loading