py3-tensorflow-data-validation/1.17.0 package update

[octo-sts]

[octo-sts]

🛑 Build Failed: Compilation

ERROR: /root/.cache/bazel/_bazel_root/79a1bfc8c8b5b6a6d226b38d072f165b/external/com_google_zetasql/zetasql/common/match_recognize/BUILD:21:11: Compiling zetasql/common/match_recognize/nfa.cc failed: (Exit 1): gcc failed: error executing command (from target @com_google_zetasql//zetasql/common/match_recognize:nfa)

Build Details

Category	Details
Build System	Bazel
Failure Point	Compiling zetasql/common/match_recognize/nfa.cc

Root Cause Analysis 🔍

The compilation fails because of a type error in the hash policy traits. The specific error is '(absl::lts_20240722::container_internal::TypeErasedApplyToSlotFn<absl::lts_20240722::hash_internal::Hashzetasql::functions::match_recognize::NFAState, zetasql::functions::match_recognize::NFAState> == 0)' is not a constant expression. This is likely due to incompatibilities between the NFAState class and the hash implementation in Abseil.

---

🔍 Build failure fix suggestions

Found similar build failures that have been fixed in the past and analyzed them to suggest a fix:

Suggested Changes

File: 0001-fix-build.patch

• addition (Create or modify the existing patch file)
  Replacement:

diff --git a/tensorflow_data_validation/workspace.bzl b/tensorflow_data_validation/workspace.bzl
index xxxxxxx..yyyyyyy 100644
--- a/tensorflow_data_validation/workspace.bzl
+++ b/tensorflow_data_validation/workspace.bzl
@@ -xx,xx @@ def tf_data_validation_workspace():
     # Add explicit definition for Abseil version compatibility
     native.new_local_repository(
         name = "com_google_absl",
         path = "",
         build_file_content = """  
# Workaround for Abseil compatibility issues
package(default_visibility = ["//visibility:public"])
cc_library(
    name = "container_compatibility_fixes",
    hdrs = ["absl_fixes.h"],
)
         """,
     )

Content:

Add a patch to fix the NFAState hash implementation compatibility issue with Abseil

• addition (Create a new file absl_fixes.h)
  Replacement:

#ifndef ABSL_FIXES_H_
#define ABSL_FIXES_H_

#include <functional>

namespace zetasql {
namespace functions {
namespace match_recognize {

// Ensure NFAState has proper hash function implementation
struct NFAStateHasher {
  std::size_t operator()(const NFAState& state) const {
    // Implement a simple hash function that works with the current Abseil version
    return std::hash<std::string>{}(state.ToString());
  }
};

} // namespace match_recognize
} // namespace functions
} // namespace zetasql

#endif // ABSL_FIXES_H_

Content:

Create a header file to fix the hash function implementation

• addition (Add to the patch file)
  Replacement:

diff --git a/external/com_google_zetasql/zetasql/common/match_recognize/nfa.cc b/external/com_google_zetasql/zetasql/common/match_recognize/nfa.cc
index xxxxxxx..yyyyyyy 100644
--- a/external/com_google_zetasql/zetasql/common/match_recognize/nfa.cc
+++ b/external/com_google_zetasql/zetasql/common/match_recognize/nfa.cc
@@ -xx,xx @@ namespace match_recognize {
 
// Update the hash implementation for NFAState to be compatible with newer Abseil
#include "absl_fixes.h"

// Replace the problematic hash definition with a custom one
typedef absl::flat_hash_set<NFAState, NFAStateHasher> NFAStateSet;

Content:

Modify the nfa.cc file to use the custom hasher

File: pipeline

• modification (environment.contents.packages)
  Original:

build-base

Replacement:

gcc-10 g++-10 build-base

Content:

Update the environment to use gcc-10 as mentioned in the changelog

• addition (environment.environment)
  Original:

JAVA_HOME: /usr/lib/jvm/java-11-openjdk

Replacement:

JAVA_HOME: /usr/lib/jvm/java-11-openjdk
    CC: gcc-10
    CXX: g++-10

Content:

Add environment variables to use gcc-10

• addition (Add before the Python Build step)
  Replacement:

  - name: Configure Bazel
    runs: |
      echo 'build --cxxopt="-std=c++17"' >> .bazelrc
      echo 'build --host_cxxopt="-std=c++17"' >> .bazelrc

Content:

Add bazel options to work around the compilation issue

Click to expand fix analysis

Analysis

The build failure is related to a compilation error in the ZetaSQL component when building py3-tensorflow-data-validation. The error specifically occurs in the file zetasql/common/match_recognize/nfa.cc and involves a type error in the hash policy traits. The error message indicates an issue with the NFAState class and the hash implementation in Abseil.

According to the upstream changelog, there was a recent update to ZetaSQL (v2024.11.1) and related dependencies. The error likely stems from incompatibilities between the updated ZetaSQL code and the Abseil library version being used. The changelog also mentions updating the GCC version to gcc-10, which suggests that newer C++ features or standards might be required.

Click to expand fix explanation

Explanation

The build failure is happening because of an incompatibility between the hash function implementation for the NFAState class in ZetaSQL and the newer version of Abseil library. The error message specifically mentions a problem with the type erasure in the hash policy traits, indicating that the hash function for NFAState is not properly defined for the current Abseil version.

The suggested fix addresses this issue through several approaches:

1. Creating a custom hash implementation: We're adding a custom NFAStateHasher class that provides a compatible hash function for NFAState. This avoids the problems with Abseil's internal hash implementation.

2. Updating the compiler to gcc-10: As mentioned in the changelog, the project now requires gcc-10. Using an older compiler might cause compatibility issues with C++17 features that the code might be using.

3. Adding C++17 support: By explicitly specifying C++17 standard for the compiler through Bazel options, we ensure that modern C++ features required by the updated ZetaSQL and Abseil libraries are available.

The patch creates a compatibility layer that bridges the gap between the NFAState class implementation and the hashing requirements of the newer Abseil library. By providing a custom hasher, we avoid the internal type erasure mechanism of Abseil that's causing the current error. The toString() method is commonly available on such objects, making it a reliable way to generate a hash value, even though it might not be the most efficient approach.

This solution should allow the code to compile successfully while maintaining compatibility with the updated ZetaSQL and Abseil libraries.

Click to expand alternative approaches

Alternative Approaches

• Instead of creating a custom hasher, modify the NFAState class directly to specialize the Abseil hash implementation. This would require adding a friend function or specializing the absl::Hash template for NFAState.
• Downgrade the ZetaSQL version to an earlier version that's compatible with the current build environment. This is less ideal as the changelog specifically mentions updating to v2024.11.1.
• Use the Bazel sandbox environment instead of directly installing dependencies via apk. This would give more control over the specific versions of libraries used during the build.
• Apply compiler flags to disable specific warnings or errors that might be causing the build failure. This could include -fpermissive to allow certain constructs that might be rejected by stricter compiler versions.

---

Was this comment helpful? Please use 👍 or 👎 reactions on this comment.

[debasishbsws]

Current failure is mostly released to abseil/abseil-cpp#1634

For a temporary workaround, I used clang for sanitizer builds.

But changing the config to use clang by these change causes another set of error

diff --git a/py3-tensorflow-data-validation.yaml b/py3-tensorflow-data-validation.yaml
index 7b415f870..af642e056 100644
--- a/py3-tensorflow-data-validation.yaml
+++ b/py3-tensorflow-data-validation.yaml
@@ -30,6 +30,9 @@ environment:
       - build-base
       - busybox
       - ca-certificates-bundle
+      - clang-16
+      - clang-16-dev
+      - lld
       - numpy
       - openjdk-11
       - patch
@@ -43,6 +46,10 @@ environment:
       - wolfi-base
   environment:
     JAVA_HOME: /usr/lib/jvm/java-11-openjdk
+    CC: clang
+    CXX: clang++
+    LD: lld
+    BAZEL_LINKOPTS: "-fuse-ld=lld"
 
 pipeline:
   - uses: git-checkout
@@ -55,6 +62,16 @@ pipeline:
     with:
       patches: 0001-fix-build.patch
 
+  - name: Create .bazelrc for Clang
+    runs: |
+      cat > .bazelrc << 'EOF'
+      build --action_env=CC=clang
+      build --action_env=CXX=clang++
+      build --linkopt=-fuse-ld=lld
+      build:opt --copt=-march=native
+      build:opt --copt=-O3
+      EOF
+
   - name: Python Build
     runs: python setup.py bdist_wheel

ERROR:

2025/06/16 17:20:23 WARN [967 / 2,499] Compiling absl/status/statusor.cc [for tool]; 0s processwrapper-sandbox ... (14 actions, 13 running)
2025/06/16 17:20:24 WARN ERROR: /root/.cache/bazel/_bazel_root/79a1bfc8c8b5b6a6d226b38d072f165b/external/com_google_zetasql/zetasql/public/BUILD:148:11: Compiling zetasql/public/cycle_detector.cc failed: (Exit 1): clang-16 failed: error executing command (from target @com_google_zetasql//zetasql/public:cycle_detector) /usr/bin/clang-16 -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 ... (remaining 28 arguments skipped

And somehow everything is connected to setasql 😢