onnxruntime: add 1.14.1

[mayeut]

Specify library name and version: onnxruntime/1.14.1

Fixes #4806
Fixes #16699
Closes #16842

This is similar to #16842 but I already had some work done before that PR was created and thought I'd share the state of said work.

Differences with #16842:

• nothing is downloaded, only conan packages are used. This comes with some caveats:
  • mac armv8 will fail because eigen 3.4.0 has a bug (onnxruntime pins a git commit before 3.4.0, the not released 3.4.1 has a fix for this), c.f. eigen(3.4.0): Fix vectorized reductions compilation errors for Eigen::half #16854
  • Windows will most likely fail (pending wil: recipe added #16701 & update of the requirement/patch)
  • unit tests can't be built without onnx sources (requires a proto definition from those sources, not patched)
• no options are implemented yet. only with_xnnpack option is implemented.

It's been built and tested on mac arm64 & linux x86_64 (gcc-12).
Unit tests were built with a specific patch (adding the missing onyx proto & patching tests for the location) & run successfully.

conan v2 won't build without #16703 EDIT: now available

I also opened microsoft/onnxruntime#15323 upstream to get some sense of what part of the patch could be upstreamed. EDIT the PR was merged in onnxruntime/main for a future version, I'll try to see if I can squeeze other changes as well.

Like @gmeeker in #16842, I don't really have time to finish this recipe (and in my case, I don't have the skills to do so), so any help is appreciated.

---

• I've read the contributing guidelines.
• I've used a recent Conan client version close to the currently deployed.
• I've tried at least one configuration locally with the conan-center hook activated.

[github-actions]

Hooks produced the following warnings for commit 183c477

onnxruntime/1.14.1

post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './csharp/sample/InferenceSample/Microsoft.ML.OnnxRuntime.InferenceSample.Forms.Android/Microsoft.ML.OnnxRuntime.InferenceSample.Forms.Android.csproj' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.
post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './csharp/sample/InferenceSample/Microsoft.ML.OnnxRuntime.InferenceSample.NetCoreApp/Microsoft.ML.OnnxRuntime.InferenceSample.NetCoreApp.csproj' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.
post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './csharp/sample/InferenceSample/Microsoft.ML.OnnxRuntime.InferenceSample.Forms.iOS/Microsoft.ML.OnnxRuntime.InferenceSample.Forms.iOS.csproj' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.
post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/flash_attention_causal/fmha_v2_flash_attention_fp16_Causal_0_80_sm86.cubin.cc' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.
post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/flash_attention/fmha_v2_flash_attention_fp16_128_16_S_16_sm80.cubin.cc' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.
post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './onnxruntime/test/testdata/transform/propagate_cast/matmul_add_transpose_inputs_transpose_product_cast_product_cast_input2_cast_sum.onnx' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.

[github-actions]

Hooks produced the following warnings for commit 183c477

onnxruntime/1.14.1

post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './csharp/sample/InferenceSample/Microsoft.ML.OnnxRuntime.InferenceSample.Forms.Android/Microsoft.ML.OnnxRuntime.InferenceSample.Forms.Android.csproj' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.
post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './csharp/sample/InferenceSample/Microsoft.ML.OnnxRuntime.InferenceSample.NetCoreApp/Microsoft.ML.OnnxRuntime.InferenceSample.NetCoreApp.csproj' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.
post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './csharp/sample/InferenceSample/Microsoft.ML.OnnxRuntime.InferenceSample.Forms.iOS/Microsoft.ML.OnnxRuntime.InferenceSample.Forms.iOS.csproj' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.
post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/flash_attention_causal/fmha_v2_flash_attention_fp16_Causal_0_80_sm86.cubin.cc' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.
post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/flash_attention/fmha_v2_flash_attention_fp16_128_16_S_16_sm80.cubin.cc' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.
post_source(): WARN: [SHORT_PATHS USAGE (KB-H066)] The file './onnxruntime/test/testdata/transform/propagate_cast/matmul_add_transpose_inputs_transpose_product_cast_product_cast_input2_cast_sum.onnx' has a very long path and may exceed Windows max path length. Add 'short_paths = True' in your recipe.

[Hopobcn]

BTW if you don't have time to iterate over this recipe i'm able to help during this week.

[elejke]

C:\J\w\prod\BuildSingleReference\s\827467\1\src\onnxruntime\core\platform\windows\env.cc(36,10): fatal error C1083: Cannot open include file: 'wil/Resource.h': No such file or directory [C:\J\w\prod\BuildSingleReference\s\827467\1\build\onnxruntime_common.vcxproj]

waiting #16701 to be merged

@prince-chrismc fyi

[github-actions]

Hooks produced the following warnings for commit 640b652

onnxruntime/1.14.1

post_package(): WARN: [MISSING SYSTEM LIBS (KB-H043)] Library '.\bin\onnxruntime.dll' links to system library 'shlwapi' but it is not in cpp_info.system_libs.

[Hopobcn]

Tried building with this command conan create . --name onnxruntime --version 1.14.1 -o "*/*:shared=True" -u --build=missing and configuration:

Profile host:
[settings]
arch=armv8
build_type=Release
compiler=apple-clang
compiler.cppstd=gnu17
compiler.libcxx=libc++
compiler.version=14
os=Macos
[options]
*/*:shared=True

Profile build:
[settings]
arch=armv8
build_type=Release
compiler=apple-clang
compiler.cppstd=gnu17
compiler.libcxx=libc++
compiler.version=14
os=Macos

And the build succeeds... so either it affects only apple-clang 13, or it's a c3i issue. Conan2 builds are optional anyways no? For me I would merge it already.

[mayeut]

conan v2 now fails with something different... unrelated to the change.

[Hopobcn]

Conan v2 failure just means date/3.0.1:167cc04113817160bfdf222914651da0997a93a4 is not available in CCI.

@prince-chrismc last MR for date recipe generated v2 packages #16709 (comment) https://github.com/conan-io/conan-center-index/commits/master/recipes/date/all/conanfile.py why aren't they available now?

[mayeut]

Conan v2 failure just means date/3.0.1:167cc04113817160bfdf222914651da0997a93a4 is not available in CCI.

@Hopobcn, I do understand that but builds weren't failing for Linux on conan v2 before.

e.g. in build 21, which was passing for that configuration, we get:

date/3.0.1#da64a2838b57529a13c1451b8e1be7e5 - Downloaded (conan-center)
...
date/3.0.1#da64a2838b57529a13c1451b8e1be7e5:789e77da5f55303c795884aea8afdebb109117e5#64ba707ffefe87640d1616bb1dddecca - Download (conan-center)

& in build 23/24, the same config is not passing with:

date/3.0.1#da64a2838b57529a13c1451b8e1be7e5 - Downloaded (conan-center)
...
date/3.0.1: Checking 9 compatible configurations:
date/3.0.1: 'd9b4230e293b329b2750786c3a13900034605b94': compiler.cppstd=11
date/3.0.1: '7cba268ff100b1aac578449d56b01e879d00f289': compiler.cppstd=gnu11
date/3.0.1: 'c014423a8a683043163a60eef892d510269cc28a': compiler.cppstd=14
date/3.0.1: '54960fd4130c0912fee4835d1c33073a5a66601e': compiler.cppstd=gnu14
date/3.0.1: '9f205fbe0cdd1824412c15cc755dee9e36d71d44': compiler.cppstd=17
date/3.0.1: 'dc71cc4c85095f2c4042cec1227a14444c431dcd': compiler.cppstd=20
date/3.0.1: '836f31b5d4931183feb8466f9d53c7d35420054f': compiler.cppstd=gnu20
date/3.0.1: '667f21c2d01814053ee554bfed3cae7ef1068cd7': compiler.cppstd=23
date/3.0.1: 'cf1f0d4368c2316dc224a68a17aa4df5f3404341': compiler.cppstd=gnu23
...
date/3.0.1#da64a2838b57529a13c1451b8e1be7e5:167cc04113817160bfdf222914651da0997a93a4 - Missing

@prince-chrismc,
Has the package gone missing ? (I don't see a package for compiler.cppstd=gnu17 in the check for compatible configuration, is that normal ?)
Has something changed in conan itself leading to this change in behavior ?
Something else ?

[mayeut]

closing/reopening to see if #17163 helped with conan v2

[mayeut]

closing/reopening to see if #17163 helped with conan v2

It did, yeah
With the don't link with boost commit, we now have a Conan v2 builds as well.

[prince-chrismc]

v2 is missing 🤔

[mayeut]

v2 is missing 🤔

I don't why it did not appear in the PR but it was built on c3i: https://c3i.jfrog.io/c3i/misc-v2/logs/pr/16849/25/onnxruntime/1.14.1/summary.json

[conan-center-bot]

Conan v1 pipeline ✔️

All green in build 26 (71659f42e60c476de1659e88ec512ba0f7621236):

• onnxruntime/1.14.1@:
  All packages built successfully! (All logs)

---

Conan v2 pipeline ❌

Note: Conan v2 builds may be required once they are on the v2 ready list

The v2 pipeline failed. Please, review the errors and note this will be required for pull requests to be merged in the near future.

See details:

Failure in build 26 (71659f42e60c476de1659e88ec512ba0f7621236):

• onnxruntime/1.14.1@:
  CI failed to create some packages (All logs)

  Logs for packageID 838b1fb699e581a135152df5c0386e1496cd9e7f:

  [settings]
  arch=x86_64
  build_type=Release
  compiler=apple-clang
  compiler.cppstd=gnu17
  compiler.libcxx=libc++
  compiler.version=13
  os=Macos
  [options]
  */*:shared=True
  

  [...]
  -- Conan: Component target declared 'absl::check'
  -- Conan: Component target declared 'absl::die_if_null'
  -- Conan: Component target declared 'absl::log_flags'
  -- Conan: Target declared 'abseil::abseil'
  -- Conan: Including build module from '/Users/jenkins/w/prod-v2/BuildSingleReference/p/absei2e24122a9b6a6/p/lib/cmake/conan_trick/cxx_std.cmake'
  -- Conan: Component target declared 'date::date-tz'
  -- Conan: Target declared 'date::date'
  -- Conan: Component target declared 'CURL::libcurl'
  -- Conan: Target declared 'ZLIB::ZLIB'
  -- Conan: Target declared 're2::re2'
  -- Conan: Component target declared 'onnx_proto'
  -- Conan: Component target declared 'onnx'
  -- Conan: Target declared 'onnx::onnx'
  -- Conan: Component target declared 'protobuf::libprotobuf'
  -- Conan: Component target declared 'protobuf::libprotoc'
  -- Conan: Target declared 'protobuf::protobuf'
  -- Conan: Including build module from '/Users/jenkins/w/prod-v2/BuildSingleReference/p/proto1765feb0b77ef/p/lib/cmake/protobuf/protobuf-generate.cmake'
  -- Conan: Including build module from '/Users/jenkins/w/prod-v2/BuildSingleReference/p/proto1765feb0b77ef/p/lib/cmake/protobuf/protobuf-module.cmake'
  -- Conan: Including build module from '/Users/jenkins/w/prod-v2/BuildSingleReference/p/proto1765feb0b77ef/p/lib/cmake/protobuf/protobuf-options.cmake'
  -- Conan: Component target declared 'flatbuffers::libflatbuffers'
  -- Conan: Target declared 'flatbuffers::flatbuffers_shared'
  -- Conan: Including build module from '/Users/jenkins/w/prod-v2/BuildSingleReference/p/flatbed8dbf9d77e82/p/lib/cmake/FlatcTargets.cmake'
  -- Conan: Including build module from '/Users/jenkins/w/prod-v2/BuildSingleReference/p/flatbed8dbf9d77e82/p/lib/cmake/BuildFlatBuffers.cmake'
  -- Conan: Target declared 'cpuinfo::cpuinfo'
  -- Conan: Component target declared 'nsync::nsync_c'
  -- Conan: Component target declared 'nsync::nsync_cpp'
  -- Conan: Target declared 'nsync::nsync'
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /Users/jenkins/w/prod-v2/BuildSingleReference/cci-2f4a050b/recipes/onnxruntime/all/test_package/build/apple-clang-13-x86_64-gnu17-release
  
  onnxruntime/1.14.1 (test package): Running CMake.build()
  onnxruntime/1.14.1 (test package): RUN: cmake --build "/Users/jenkins/w/prod-v2/BuildSingleReference/cci-2f4a050b/recipes/onnxruntime/all/test_package/build/apple-clang-13-x86_64-gnu17-release" -- -j4
  [ 50%] Building CXX object CMakeFiles/test_package.dir/test_package.cpp.o
  [100%] Linking CXX executable test_package
  [100%] Built target test_package
  
  
  ======== Testing the package: Executing test ========
  onnxruntime/1.14.1 (test package): Running test()
  onnxruntime/1.14.1 (test package): RUN: ./test_package
  dyld: Symbol not found: _iconv
    Referenced from: /usr/lib/libarchive.2.dylib
    Expected in: /Users/jenkins/w/prod-v2/BuildSingleReference/p/libic50650dc30c43d/p/lib/libiconv.2.dylib
   in /usr/lib/libarchive.2.dylib
  /bin/sh: line 1: 31519 Abort trap: 6           ./test_package
  
  ERROR: onnxruntime/1.14.1 (test package): Error in test() method, line 27
  	self.run(bin_path, env="conanrun")
  	ConanException: Error 134 while executing
  

---

^{Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.}

[brian-pieces]

Thanks for this! Huge help to have ORT on conan. I was able to successfully build, link to, and test ORT but I get the same warnings at runtime as microsoft/onnxruntime#8556: Schema error: Trying to register schema with name ... (domain: version: 1) from file ... line ..., but it is already registered from file ... line ...

I'm guessing it has something to do with ONNX_DISABLE_STATIC_REGISTRATION?