Nighly feedstock build failed

[github-actions]

Nightly feedstock build failure for tiledbfeedstock_CI at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=4&_a=summary

[github-actions]

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

[KiterLuc]

@teo-tsirpanis something is wrong with the windows build... I think this is related to your standalone test PR from yesterday?

[teo-tsirpanis]

I don't think so; that PR did not change anything that would affect the Conda build.

[teo-tsirpanis]

I reran the Windows job and it failed again. CI fails after vcpkg has ran, when the main CMake build tries to find the compiler:

CMake Error at D:/bld/tiledb_1715871109896/_build_env/Library/share/cmake-3.29/Modules/CMakeDetermineCCompiler.cmake:49 (message):
  Could not find compiler set in environment variable CC:

  cl.exe.
-- Configuring incomplete, errors occurred!

@jdblischak any ideas?

[jdblischak]

I suspect it is a conda problem. There were no commits to TileDB on May 15th, so the successful build on Tuesday night was built from the same commit as the failed build last night: TileDB-Inc/TileDB@42dbf36

[jdblischak]

The most recent updates to vc-feedstock and cmake-feedstock happened days ago

[github-actions]

Nightly feedstock build failure for tiledbfeedstock_CI at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=4&_a=summary

[github-actions]

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

[jdblischak]

The win-64 build is still failing and I don't know why :-(

[github-actions]

Nightly feedstock build failure for tiledbfeedstock_CI at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=4&_a=summary

[github-actions]

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

[github-actions]

Nightly feedstock build failure for tiledbfeedstock_CI at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=4&_a=summary

[github-actions]

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

[KiterLuc]

@jdblischak Still no clue what's happening here? @teo-tsirpanis any thoughts?

[teo-tsirpanis]

No idea either. 😕

[KiterLuc]

Wasn't there a cmake update recently? Can we submit any branch to the pipeline so that we can debug the issue?

[github-actions]

Nightly feedstock build failure for tiledbfeedstock_CI at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=4&_a=summary

[github-actions]

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

[KiterLuc]

This isn't windows, but the error and timing make me think something might have been removed from the windows image as well? actions/runner-images#9866

[ihnorton]

The conda build activation script sets

2024-05-20T01:35:48.9267510Z %SRC_DIR%>set "CC=cl.exe" 

so maybe the image has taken visual c++ out of the default path (in which case we need to call vcvars), or maybe this is related to actions/runner-images#9888 (azure images are same/derived from GHA).

[jdblischak]

This win-64 build problem has become more urgent. It is blocking the migration PRs for libgoogle-cloud 2.24 and aws-sdk-cpp
1.11.329 to both main and 2.22.x (conda-forge/tiledb-feedstock#285, conda-forge/tiledb-feedstock#286, conda-forge/tiledb-feedstock#287, conda-forge/tiledb-feedstock#288).

Even worse, tiledbsoma-feedstock now requires that tiledb-feedstock be migrated to libgoogle-cloud 2.24 (TileDB-Inc/tiledbsoma-feedstock#155)

[ihnorton]

@jdblischak we re-render in the nightly script, right? If this was an image issue I would expect to see a lot more chatter on the github images repo (aside from the one I linked above), which makes me thinkg these feedstocks could somehow be out of date.

[jdblischak]

or maybe this is related to actions/runner-images#9888 (azure images are same/derived from GHA).

I think this is the most promising lead so far. Recall that we opted in to using vs2022 for tiledb-feedstock, whereas most conda-forge feedstocks are still using vs2019. Also see conda-forge/vc-feedstock#73 (comment)

The PR below just updated vc-feedstock to build cl.exe 19.38.33135 🤞

conda-forge/vc-feedstock#76

[jdblischak]

we re-render in the nightly script, right?

Yes, we always rerender the nightly-build branch

If this was an image issue I would expect to see a lot more chatter on the github images repo (aside from the one I linked above), which makes me thinkg these feedstocks could somehow be out of date.

My current understanding is that this is only affecting conda-forge feedstocks that have opted in to using vs2022 instead of vs2019, which from a quick search is only 7 feedstocks

[jdblischak]

For future reference:

• The unexpected win-64 build failure was caused by GitHub updating the Visual Studio stack in their Windows Server 2022 image (which is used not only in GitHub Actions but also in Azure, where the feedstock build runs). This happened 5 days ago in Windows Server 2022 (20240514) Image Update actions/runner-images#9853 (xref: Activation script does not pick up VSINSTALLDIR on GitHub runners conda-forge/vc-feedstock#78)
• Fixed by MSVC 14.38.33135 conda-forge/vc-feedstock#76 and Update to VS2022 update 9 conda-forge/vc-feedstock#77