Sandbox friendly build #92

npatsakula · 2023-10-13T11:55:28Z

I've made some effort to simplify cmake machinery and make build sandbox-friendly (now it is not changing source directory), now this crate can be build with NIX Flake.

Any feedback appreciable!

tazz4843

Currently late in the day where I am, will test and possibly merge in the morning.

sys/build.rs

tazz4843 · 2023-10-14T15:43:11Z

I don't run Windows so I can't help much with the failing check unfortunately.

tazz4843

Besides the failing checks looks good to me.

GitHub doesn't appear to be letting me restart the checks this time around, perhaps force push once more?

npatsakula · 2023-10-14T15:53:49Z

I don't run Windows so I can't help much with the failing check unfortunately.

Don't worry, I found symbols search path at whisper.cpp CI. Looks like it should work now (last time only rustfmt was broken).

npatsakula · 2023-10-15T14:04:30Z

Hello @tazz4843! Can I ask you to run CI pipeline?

tazz4843 · 2023-10-15T14:31:51Z

GitHub doesn't appear to be letting me run the checks this time around, perhaps force push once more?

* CMake machinery simplified. * Build artifacts not stored in source directories anymore.

tazz4843 · 2023-10-15T15:02:02Z

Can't get this to compile locally:

memory 19GiB/62GiB at 10-15 09:01:26 exit 101 took 6s156ms
on git sandbox_friendly is pkg v0.9.0-rc.2 via rs v1.75.0-nightly
niko@niko-desktop whisper-rs > cargo run --example basic_use
   Compiling whisper-rs-sys v0.7.3 (/tmp/whisper-rs/sys)
The following warnings were emitted during compilation:

warning: Unable to generate bindings: clang diagnosed error: wrapper.h:1:10: fatal error: 'whisper.h' file not found
warning: Using bundled bindings.rs, which may be out of date

error: failed to run custom build command for `whisper-rs-sys v0.7.3 (/tmp/whisper-rs/sys)`

Caused by:
  process didn't exit successfully: `/tmp/whisper-rs/target/debug/build/whisper-rs-sys-c3f9ea4e385de8cf/build-script-build` (exit status: 101)
  --- stdout
  cargo:rustc-link-lib=dylib=stdc++
  cargo:rerun-if-changed=wrapper.h
  cargo:rerun-if-env-changed=TARGET
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS
  cargo:warning=Unable to generate bindings: clang diagnosed error: wrapper.h:1:10: fatal error: 'whisper.h' file not found

  cargo:warning=Using bundled bindings.rs, which may be out of date
  CMAKE_TOOLCHAIN_FILE_x86_64-unknown-linux-gnu = None
  CMAKE_TOOLCHAIN_FILE_x86_64_unknown_linux_gnu = None
  HOST_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_x86_64-unknown-linux-gnu = None
  CMAKE_GENERATOR_x86_64_unknown_linux_gnu = None
  HOST_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_x86_64-unknown-linux-gnu = None
  CMAKE_PREFIX_PATH_x86_64_unknown_linux_gnu = None
  HOST_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_x86_64-unknown-linux-gnu = None
  CMAKE_x86_64_unknown_linux_gnu = None
  HOST_CMAKE = None
  CMAKE = None
  running: cd "/tmp/whisper-rs/target/debug/build/whisper-rs-sys-f7e57ece2896f8a9/out/build" && CMAKE_PREFIX_PATH="" "cmake" "-Wdev" "--debug-output" "/tmp/whisper-rs/target/debug/build/whisper-rs-sys-f7e57ece2896f8a9/out/whisper.cpp/" "-DBUILD_SHARED_LIBS=OFF" "-DWHISPER_ALL_WARNINGS=OFF" "-DWHISPER_ALL_WARNINGS_3RD_PARTY=OFF" "-DWHISPER_BUILD_TESTS=OFF" "-DWHISPER_BUILD_EXAMPLES=OFF" "-DCMAKE_INSTALL_PREFIX=/tmp/whisper-rs/target/debug/build/whisper-rs-sys-f7e57ece2896f8a9/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_ASM_COMPILER=/usr/bin/cc" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON"
  Running with debug output on.

  --- stderr
  wrapper.h:1:10: fatal error: 'whisper.h' file not found
  CMake Warning:
    Ignoring extra path from command line:

     "/tmp/whisper-rs/target/debug/build/whisper-rs-sys-f7e57ece2896f8a9/out/whisper.cpp/"


  CMake Error: The source directory "/tmp/whisper-rs/target/debug/build/whisper-rs-sys-f7e57ece2896f8a9/out/whisper.cpp" does not appear to contain CMakeLists.txt.
  Specify --help for usage, or press the help button on the CMake GUI.
  thread 'main' panicked at /home/niko/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

  command did not execute successfully, got: exit status: 1

  build script failed, must exit now
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

npatsakula · 2023-10-15T15:36:02Z

Can't get this to compile locally:

memory 19GiB/62GiB at 10-15 09:01:26 exit 101 took 6s156ms
on git sandbox_friendly is pkg v0.9.0-rc.2 via rs v1.75.0-nightly
niko@niko-desktop whisper-rs > cargo run --example basic_use
   Compiling whisper-rs-sys v0.7.3 (/tmp/whisper-rs/sys)
The following warnings were emitted during compilation:

warning: Unable to generate bindings: clang diagnosed error: wrapper.h:1:10: fatal error: 'whisper.h' file not found
warning: Using bundled bindings.rs, which may be out of date

error: failed to run custom build command for `whisper-rs-sys v0.7.3 (/tmp/whisper-rs/sys)`

Caused by:
  process didn't exit successfully: `/tmp/whisper-rs/target/debug/build/whisper-rs-sys-c3f9ea4e385de8cf/build-script-build` (exit status: 101)
  --- stdout
  cargo:rustc-link-lib=dylib=stdc++
  cargo:rerun-if-changed=wrapper.h
  cargo:rerun-if-env-changed=TARGET
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS
  cargo:warning=Unable to generate bindings: clang diagnosed error: wrapper.h:1:10: fatal error: 'whisper.h' file not found

  cargo:warning=Using bundled bindings.rs, which may be out of date
  CMAKE_TOOLCHAIN_FILE_x86_64-unknown-linux-gnu = None
  CMAKE_TOOLCHAIN_FILE_x86_64_unknown_linux_gnu = None
  HOST_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_x86_64-unknown-linux-gnu = None
  CMAKE_GENERATOR_x86_64_unknown_linux_gnu = None
  HOST_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_x86_64-unknown-linux-gnu = None
  CMAKE_PREFIX_PATH_x86_64_unknown_linux_gnu = None
  HOST_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_x86_64-unknown-linux-gnu = None
  CMAKE_x86_64_unknown_linux_gnu = None
  HOST_CMAKE = None
  CMAKE = None
  running: cd "/tmp/whisper-rs/target/debug/build/whisper-rs-sys-f7e57ece2896f8a9/out/build" && CMAKE_PREFIX_PATH="" "cmake" "-Wdev" "--debug-output" "/tmp/whisper-rs/target/debug/build/whisper-rs-sys-f7e57ece2896f8a9/out/whisper.cpp/" "-DBUILD_SHARED_LIBS=OFF" "-DWHISPER_ALL_WARNINGS=OFF" "-DWHISPER_ALL_WARNINGS_3RD_PARTY=OFF" "-DWHISPER_BUILD_TESTS=OFF" "-DWHISPER_BUILD_EXAMPLES=OFF" "-DCMAKE_INSTALL_PREFIX=/tmp/whisper-rs/target/debug/build/whisper-rs-sys-f7e57ece2896f8a9/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_ASM_COMPILER=/usr/bin/cc" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON"
  Running with debug output on.

  --- stderr
  wrapper.h:1:10: fatal error: 'whisper.h' file not found
  CMake Warning:
    Ignoring extra path from command line:

     "/tmp/whisper-rs/target/debug/build/whisper-rs-sys-f7e57ece2896f8a9/out/whisper.cpp/"


  CMake Error: The source directory "/tmp/whisper-rs/target/debug/build/whisper-rs-sys-f7e57ece2896f8a9/out/whisper.cpp" does not appear to contain CMakeLists.txt.
  Specify --help for usage, or press the help button on the CMake GUI.
  thread 'main' panicked at /home/niko/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

  command did not execute successfully, got: exit status: 1

  build script failed, must exit now
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Have you downloaded submodules?

git submodule update --init --recursive

I tested this setup in few environments: OS X M1, X86 and ARM servers.

tazz4843 · 2023-10-15T15:39:07Z

I always forget to clone the submodules, thanks for reminding me. Seems to work now.

tazz4843 · 2023-10-15T15:39:46Z

Thanks for the PR!

npatsakula force-pushed the sandbox_friendly branch from ff7a744 to aba2d3b Compare October 13, 2023 14:17

tazz4843 requested changes Oct 14, 2023

View reviewed changes

sys/build.rs Outdated Show resolved Hide resolved

sys/build.rs Outdated Show resolved Hide resolved

npatsakula force-pushed the sandbox_friendly branch 5 times, most recently from 38ef3e6 to 938ecbf Compare October 14, 2023 15:34

npatsakula force-pushed the sandbox_friendly branch 2 times, most recently from a668d3d to 5b3bdca Compare October 14, 2023 15:49

tazz4843 approved these changes Oct 14, 2023

View reviewed changes

[*] Sandbox-friendly build implemented.

0b5a74a

* CMake machinery simplified. * Build artifacts not stored in source directories anymore.

npatsakula force-pushed the sandbox_friendly branch from 5b3bdca to 0b5a74a Compare October 15, 2023 14:34

tazz4843 merged commit 9ec0fd6 into tazz4843:master Oct 15, 2023
13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sandbox friendly build #92

Sandbox friendly build #92

npatsakula commented Oct 13, 2023 •

edited

Loading

tazz4843 left a comment

tazz4843 commented Oct 14, 2023

tazz4843 left a comment •

edited

Loading

npatsakula commented Oct 14, 2023 •

edited

Loading

npatsakula commented Oct 15, 2023

tazz4843 commented Oct 15, 2023

tazz4843 commented Oct 15, 2023

npatsakula commented Oct 15, 2023

tazz4843 commented Oct 15, 2023 •

edited

Loading

tazz4843 commented Oct 15, 2023

Sandbox friendly build #92

Sandbox friendly build #92

Conversation

npatsakula commented Oct 13, 2023 • edited Loading

tazz4843 left a comment

Choose a reason for hiding this comment

tazz4843 commented Oct 14, 2023

tazz4843 left a comment • edited Loading

Choose a reason for hiding this comment

npatsakula commented Oct 14, 2023 • edited Loading

npatsakula commented Oct 15, 2023

tazz4843 commented Oct 15, 2023

tazz4843 commented Oct 15, 2023

npatsakula commented Oct 15, 2023

tazz4843 commented Oct 15, 2023 • edited Loading

tazz4843 commented Oct 15, 2023

npatsakula commented Oct 13, 2023 •

edited

Loading

tazz4843 left a comment •

edited

Loading

npatsakula commented Oct 14, 2023 •

edited

Loading

tazz4843 commented Oct 15, 2023 •

edited

Loading