Skip to content

Commit

Permalink
2.13 update (#117)
Browse files Browse the repository at this point in the history
* Refreshed to 2.13

* Fixed on x86 with cuda 12

* Updated dist info patch and versions of meta package

* Fixed TF 2.13 on Power

* Added patch

* Fixed build with cuda 11.8

* Added filter for cuda 11.8 specific patch

* Fixes for s390x

* Use LIMIT_BUILD_RESOURCES to change different bazel jobs number

* Fix pip check failure due to typing extensions

---------

Co-authored-by: Deepali Chourasia <deepch23@in.ibm.com>
  • Loading branch information
Nishidha and cdeepali authored Aug 25, 2023
1 parent 690c446 commit 9b6388b
Show file tree
Hide file tree
Showing 10 changed files with 244 additions and 145 deletions.
1 change: 0 additions & 1 deletion buildscripts/set_tensorflow_bazelrc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,6 @@ build --define=PREFIX="$SYSTEM_LIBS_PREFIX"
build --define=LIBDIR="$SYSTEM_LIBS_PREFIX/lib"
build --define=INCLUDEDIR="$SYSTEM_LIBS_PREFIX/include"
build --define=tflite_with_xnnpack="$XNNPACK_STATUS"
build --define=tflite_kernel_use_xnnpack="$XNNPACK_STATUS"
build --copt="-DEIGEN_ALTIVEC_ENABLE_MMA_DYNAMIC_DISPATCH=$USE_MMA"
build --strip=always
build --color=yes
Expand Down
5 changes: 2 additions & 3 deletions meta_recipe/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# Recipe to build Tensorflow meta package
{% set build_version = "2.12.0" %}
{% set estimator_version = "2.12" %}
{% set build_version = "2.13.0" %}

package:
name: tensorflow{{ '' if build_type == 'cuda' else '-cpu' }}
Expand All @@ -16,7 +15,7 @@ requirements:
- tensorflow-base {{ build_version }}
- _tensorflow_select {{ tensorflow_select_version }}
- tensorboard =={{ tensorboard }}
- tensorflow-estimator =={{ estimator_version }}
- tensorflow-estimator =={{ tf_estimator }}
- keras =={{ keras }}
- tensorflow-io-gcs-filesystem {{ tensorflow_io_gcs_filesystem }} [not s390x]
- python {{ python }}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,28 +1,31 @@
From 1f5407515474febdf82341fe89e074cd2d9cd0fd Mon Sep 17 00:00:00 2001
From bdc8e1b77be2300e4452b2d5dc57ddd1aadf38b0 Mon Sep 17 00:00:00 2001
From: Nishidha Panpaliya <npanpa23@in.ibm.com>
Date: Fri, 16 Jun 2023 15:33:33 +0000
Subject: [PATCH] Updated Dist info
Date: Wed, 23 Aug 2023 05:52:41 +0000
Subject: [PATCH] Updated Dist Info of TF

---
tensorflow/tools/pip_package/setup.py | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
tensorflow/tools/pip_package/setup.py | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/tensorflow/tools/pip_package/setup.py b/tensorflow/tools/pip_package/setup.py
index d087ae2a671..072fee45c1f 100644
index e77e6f81449..7c6a133d168 100644
--- a/tensorflow/tools/pip_package/setup.py
+++ b/tensorflow/tools/pip_package/setup.py
@@ -92,7 +92,6 @@ REQUIRED_PACKAGES = [
# TODO(b/239052279): replace with external dependency on JAX repo once JAX
# no longer relies on TF.
'jax >= 0.3.15',
@@ -89,7 +89,6 @@ REQUIRED_PACKAGES = [
'gast >= 0.2.1, <= 0.4.0',
'google_pasta >= 0.1.1',
'h5py >= 2.9.0',
- 'libclang >= 13.0.0',
# TODO(b/263178356): numpy 1.24 breaks TF's tests
'numpy >= 1.22, <1.24',
'numpy >= 1.22, <= 1.24.3',
'opt_einsum >= 2.3.2',
@@ -112,8 +111,8 @@ REQUIRED_PACKAGES = [
# TODO(b/266362323): wrapt==1.15.0rc0 incompatible with TF 2.12.0 RC0 (and
# nightly, but works with TF 2.11)
'wrapt >= 1.11.0, <1.15',
'packaging',
@@ -97,10 +96,10 @@ REQUIRED_PACKAGES = [
'setuptools',
'six >= 1.12.0',
'termcolor >= 1.1.0',
- 'typing_extensions>=3.6.6,<4.6.0',
+ 'typing_extensions>=3.6.6',
'wrapt >= 1.11.0',
- 'tensorflow-io-gcs-filesystem >= 0.23.1;platform_machine!="arm64" or ' +
- 'platform_system!="Darwin"',
+ 'tensorflow-io-gcs-filesystem >= 0.23.1;platform_machine!="arm64" and ' +
Expand All @@ -31,5 +34,5 @@ index d087ae2a671..072fee45c1f 100644
# BoringSSL support.
# See https://github.com/tensorflow/tensorflow/issues/17882.
--
2.34.1
2.40.1

108 changes: 26 additions & 82 deletions recipe/0316-s390x-fixes.patch
Original file line number Diff line number Diff line change
@@ -1,20 +1,17 @@
From 46843484c96d61eb8600f9d5dd52118fd262bb10 Mon Sep 17 00:00:00 2001
From: Nishidha Panpaliya <npanpa23@in.ibm.com>
Date: Thu, 21 Jul 2022 14:07:55 +0000
Subject: [PATCH] s390x fixes
From b9be8d10fb71a3974d6cde8581c469e95af9cf7c Mon Sep 17 00:00:00 2001
From: Deepali Chourasia <deepch23@in.ibm.com>
Date: Mon, 14 Aug 2023 10:29:58 +0000
Subject: [PATCH] S390x fixes

---
tensorflow/core/api_def/base_api/api_def_Bitcast.pbtxt | 4 ++--
.../core/grappler/optimizers/arithmetic_optimizer_test.cc | 4 ++--
tensorflow/python/eager/backprop_test.py | 4 ++--
tensorflow/python/framework/tensor_util_test.py | 2 +-
tensorflow/python/tfcompile_wrapper.cc | 4 +++-
tensorflow/stream_executor/tpu/c_api_conversions.cc | 8 ++++----
tensorflow/stream_executor/tpu/c_api_conversions.h | 4 ++--
7 files changed, 16 insertions(+), 14 deletions(-)
tensorflow/core/api_def/base_api/api_def_Bitcast.pbtxt | 4 ++--
.../core/grappler/optimizers/arithmetic_optimizer_test.cc | 4 ++--
tensorflow/python/eager/backprop_test.py | 4 ++--
tensorflow/python/tfcompile_wrapper.cc | 5 +++--
4 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/tensorflow/core/api_def/base_api/api_def_Bitcast.pbtxt b/tensorflow/core/api_def/base_api/api_def_Bitcast.pbtxt
index 4a606566eb5..408aba6d99b 100644
index 23c29c04683..e24f77d38c8 100644
--- a/tensorflow/core/api_def/base_api/api_def_Bitcast.pbtxt
+++ b/tensorflow/core/api_def/base_api/api_def_Bitcast.pbtxt
@@ -47,8 +47,8 @@ tf.Tensor([0. 1. 1.], shape=(3,), dtype=float32)
Expand All @@ -27,12 +24,12 @@ index 4a606566eb5..408aba6d99b 100644
+ [ 63 128 0 0]], shape=(3, 4), dtype=uint8)

*NOTE*: Bitcast is implemented as a low-level cast, so machines with different
endian orderings will give different results.
endian orderings will give different results. A copy from input buffer to output
diff --git a/tensorflow/core/grappler/optimizers/arithmetic_optimizer_test.cc b/tensorflow/core/grappler/optimizers/arithmetic_optimizer_test.cc
index a3d8d226504..68711c74bb9 100644
index bd10921cb87..07487a5f2b7 100644
--- a/tensorflow/core/grappler/optimizers/arithmetic_optimizer_test.cc
+++ b/tensorflow/core/grappler/optimizers/arithmetic_optimizer_test.cc
@@ -715,7 +715,7 @@ TEST_F(ArithmeticOptimizerTest, TrivialSumsSimple) {
@@ -717,7 +717,7 @@ TEST_F(ArithmeticOptimizerTest, TrivialSumsSimple) {
ASSERT_EQ(new_const->input_size(), 1);
EXPECT_EQ(new_const->input(0), "^x");
EXPECT_EQ(new_const->attr().at("value").tensor().tensor_content(),
Expand All @@ -41,7 +38,7 @@ index a3d8d226504..68711c74bb9 100644

const NodeDef* new_mul = node_map.GetNode(optimized_mul_name);
ASSERT_NE(new_mul, nullptr);
@@ -762,7 +762,7 @@ TEST_F(ArithmeticOptimizerTest, TrivialSumsSimpleWithControlDep) {
@@ -764,7 +764,7 @@ TEST_F(ArithmeticOptimizerTest, TrivialSumsSimpleWithControlDep) {
ASSERT_EQ(new_const->input_size(), 1);
EXPECT_EQ(new_const->input(0), "^x");
EXPECT_EQ(new_const->attr().at("value").tensor().tensor_content(),
Expand All @@ -51,10 +48,10 @@ index a3d8d226504..68711c74bb9 100644
const NodeDef* new_mul = node_map.GetNode(optimized_mul_name);
ASSERT_NE(new_mul, nullptr);
diff --git a/tensorflow/python/eager/backprop_test.py b/tensorflow/python/eager/backprop_test.py
index 0db892e3fff..62273b50c80 100644
index 230da35ad08..9f33ab8ccdf 100644
--- a/tensorflow/python/eager/backprop_test.py
+++ b/tensorflow/python/eager/backprop_test.py
@@ -1851,7 +1851,7 @@ class JacobianTest(test.TestCase):
@@ -1869,7 +1869,7 @@ class JacobianTest(test.TestCase):

theoretical, numerical = gradient_checker_v2.compute_gradient(
def_function.function(_inner), [array_ops.ones([10, 4, 4, 1])])
Expand All @@ -63,7 +60,7 @@ index 0db892e3fff..62273b50c80 100644

@def_function.function
def _outer():
@@ -1862,7 +1862,7 @@ class JacobianTest(test.TestCase):
@@ -1880,7 +1880,7 @@ class JacobianTest(test.TestCase):
return tape.gradient(y, x)

self.assertAllClose(array_ops.reshape(numerical, [-1]),
Expand All @@ -72,32 +69,20 @@ index 0db892e3fff..62273b50c80 100644

@test_util.run_in_graph_and_eager_modes
def test_indexed_slices(self):
diff --git a/tensorflow/python/framework/tensor_util_test.py b/tensorflow/python/framework/tensor_util_test.py
index edb79a8874d..b196bcb1b3c 100644
--- a/tensorflow/python/framework/tensor_util_test.py
+++ b/tensorflow/python/framework/tensor_util_test.py
@@ -229,7 +229,7 @@ class TensorUtilTest(test.TestCase, parameterized.TestCase):
"""
dtype: DT_HALF
tensor_shape { dim { size: 2 } }
- tensor_content: "\000I\000M"
+ tensor_content: "I\000M\000"
""", t)

a = tensor_util.MakeNdarray(t)
diff --git a/tensorflow/python/tfcompile_wrapper.cc b/tensorflow/python/tfcompile_wrapper.cc
index c8818309919..b02ba0036e1 100644
index 71e302f6321..3b3ac74910e 100644
--- a/tensorflow/python/tfcompile_wrapper.cc
+++ b/tensorflow/python/tfcompile_wrapper.cc
@@ -15,6 +15,7 @@ limitations under the License.
@@ -14,7 +14,7 @@ limitations under the License.
==============================================================================*/

#include <string>
-
+#include "llvm/Support/Host.h"
#include "pybind11/cast.h"
#include "pybind11/pybind11.h"
#include "pybind11/pytypes.h"
@@ -45,7 +46,8 @@ PYBIND11_MODULE(_pywrap_tfcompile, m) {
#include "pybind11/cast.h" // from @pybind11
#include "pybind11/pybind11.h" // from @pybind11
#include "pybind11/pytypes.h" // from @pybind11
@@ -45,7 +45,8 @@ PYBIND11_MODULE(_pywrap_tfcompile, m) {
flags.graph = std::move(graph);
flags.config = std::move(config);
flags.target_triple = std::move(target_triple);
Expand All @@ -107,47 +92,6 @@ index c8818309919..b02ba0036e1 100644
flags.target_features = std::move(target_features);
flags.entry_point = std::move(entry_point);
flags.cpp_class = std::move(cpp_class);
diff --git a/tensorflow/stream_executor/tpu/c_api_conversions.cc b/tensorflow/stream_executor/tpu/c_api_conversions.cc
index 24c2565e76e..68c5a128745 100644
--- a/tensorflow/stream_executor/tpu/c_api_conversions.cc
+++ b/tensorflow/stream_executor/tpu/c_api_conversions.cc
@@ -171,8 +171,8 @@ static void CreateVectorBase(const absl::Span<Src> src, DstList* dst) {
static void CreateVector(const absl::Span<const int64_t> src, Int64List* dst) {
return CreateVectorBase<const int64_t, int64_t, Int64List>(src, dst);
}
-void CreateVector(const absl::Span<const float> src, FloatList* dst) {
- return CreateVectorBase<const float, float, FloatList>(src, dst);
+void CreateVector(const absl::Span<const double> src, FloatList* dst) {
+ return CreateVectorBase<const double, double, FloatList>(src, dst);
}
static void CreateVector(const absl::Span<const bool> src, BoolList* dst) {
return CreateVectorBase<const bool, bool, BoolList>(src, dst);
@@ -210,8 +210,8 @@ static absl::Span<const int64_t> MakeSpan(const Int64List& src_list) {
return MakeSpanBase<int64_t, int64_t, Int64List>(src_list);
}

-absl::Span<const float> MakeSpan(const FloatList& src_list) {
- return MakeSpanBase<float, float, FloatList>(src_list);
+absl::Span<const double> MakeSpan(const FloatList& src_list) {
+ return MakeSpanBase<double, double, FloatList>(src_list);
}
static absl::Span<const bool> MakeSpan(const BoolList& src_list) {
return MakeSpanBase<bool, bool, BoolList>(src_list);
diff --git a/tensorflow/stream_executor/tpu/c_api_conversions.h b/tensorflow/stream_executor/tpu/c_api_conversions.h
index 0c4fed1771c..9ae98a70e57 100644
--- a/tensorflow/stream_executor/tpu/c_api_conversions.h
+++ b/tensorflow/stream_executor/tpu/c_api_conversions.h
@@ -35,8 +35,8 @@ limitations under the License.
// XLA/StreamExecutor data structures.
namespace ApiConverter {

-absl::Span<const float> MakeSpan(const FloatList& src_list);
-void CreateVector(const absl::Span<const float> src, FloatList* dst);
+absl::Span<const double> MakeSpan(const FloatList& src_list);
+void CreateVector(const absl::Span<const double> src, FloatList* dst);
void Destroy(FloatList* float_list);

// se::DeviceMemoryBase
--
2.34.1
2.40.1

32 changes: 0 additions & 32 deletions recipe/0320-Fixed-for-cuda-11.2.patch

This file was deleted.

29 changes: 29 additions & 0 deletions recipe/0321-Update-cudnn-frontend-for-cuda-12.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
From 02f61f4714b0ad82cbf9b2f37e447124b0f54cda Mon Sep 17 00:00:00 2001
From: Nishidha Panpaliya <npanpa23@in.ibm.com>
Date: Wed, 12 Jul 2023 10:38:13 +0000
Subject: [PATCH] Update cudnn frontend for cuda 12

---
tensorflow/workspace2.bzl | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tensorflow/workspace2.bzl b/tensorflow/workspace2.bzl
index da9295adaba..86a3df2fe37 100644
--- a/tensorflow/workspace2.bzl
+++ b/tensorflow/workspace2.bzl
@@ -174,9 +174,9 @@ def _tf_repositories():
name = "cudnn_frontend_archive",
build_file = "//third_party:cudnn_frontend.BUILD",
patch_file = ["//third_party:cudnn_frontend_header_fix.patch"],
- sha256 = "bfcf778030831f325cfc13ae5995388cc834fbff2995a297ba580d9ec65ca3b6",
- strip_prefix = "cudnn-frontend-0.8",
- urls = tf_mirror_urls("https://github.com/NVIDIA/cudnn-frontend/archive/refs/tags/v0.8.zip"),
+ sha256 = "d8dba9e2607a0c256aa8eacb45b39986ab6f3f24a4d431d4397047a3cb0cd4fb",
+ strip_prefix = "cudnn-frontend-0.9",
+ urls = tf_mirror_urls("https://github.com/NVIDIA/cudnn-frontend/archive/refs/tags/v0.9.zip"),
)

tf_http_archive(
--
2.40.1

82 changes: 82 additions & 0 deletions recipe/0322-Fix-cpuinfo.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
From ba2c5f49780a4d7c0094d864aafafff2eaf8c2a1 Mon Sep 17 00:00:00 2001
From: Nishidha Panpaliya <npanpa23@in.ibm.com>
Date: Tue, 18 Jul 2023 12:12:18 +0000
Subject: [PATCH] Fix cpuinfo and downgrade boringssl

---
tensorflow/workspace2.bzl | 7 +++--
third_party/0001-Support-for-ppc64le.patch | 36 ++++++++++++++++++++++
2 files changed, 40 insertions(+), 3 deletions(-)
create mode 100644 third_party/0001-Support-for-ppc64le.patch

diff --git a/tensorflow/workspace2.bzl b/tensorflow/workspace2.bzl
index da9295adaba..527794eab57 100644
--- a/tensorflow/workspace2.bzl
+++ b/tensorflow/workspace2.bzl
@@ -166,6 +166,7 @@ def _tf_repositories():
tf_http_archive(
name = "cpuinfo",
strip_prefix = "cpuinfo-3dc310302210c1891ffcfb12ae67b11a3ad3a150",
+ patch_file = ["//third_party:0001-Support-for-ppc64le.patch"],
sha256 = "ba668f9f8ea5b4890309b7db1ed2e152aaaf98af6f9a8a63dbe1b75c04e52cb9",
urls = tf_mirror_urls("https://github.com/pytorch/cpuinfo/archive/3dc310302210c1891ffcfb12ae67b11a3ad3a150.zip"),
)
@@ -564,10 +565,10 @@ def _tf_repositories():

tf_http_archive(
name = "boringssl",
- sha256 = "9dc53f851107eaf87b391136d13b815df97ec8f76dadb487b58b2fc45e624d2c",
- strip_prefix = "boringssl-c00d7ca810e93780bd0c8ee4eea28f4f2ea4bcdc",
+ sha256 = "534fa658bd845fd974b50b10f444d392dfd0d93768c4a51b61263fd37d851c40",
+ strip_prefix = "boringssl-b9232f9e27e5668bc0414879dcdedb2a59ea75f2",
system_build_file = "//third_party/systemlibs:boringssl.BUILD",
- urls = tf_mirror_urls("https://github.com/google/boringssl/archive/c00d7ca810e93780bd0c8ee4eea28f4f2ea4bcdc.tar.gz"),
+ urls = tf_mirror_urls("https://github.com/google/boringssl/archive/b9232f9e27e5668bc0414879dcdedb2a59ea75f2.tar.gz"),
)

# Note: if you update this, you have to update libpng too. See cl/437813808
diff --git a/third_party/0001-Support-for-ppc64le.patch b/third_party/0001-Support-for-ppc64le.patch
new file mode 100644
index 00000000000..65c914ea873
--- /dev/null
+++ b/third_party/0001-Support-for-ppc64le.patch
@@ -0,0 +1,36 @@
+From def97b3f85d390edce6d1b8cfa2ab1f49c1f66a6 Mon Sep 17 00:00:00 2001
+From: Nishidha Panpaliya <npanpa23@in.ibm.com>
+Date: Tue, 18 Jul 2023 12:11:01 +0000
+Subject: [PATCH] Port on ppc64le
+
+---
+ BUILD.bazel | 6 ++++++
+ 1 file changed, 6 insertions(+)
+
+diff --git a/BUILD.bazel b/BUILD.bazel
+index 0c92064..d30f075 100644
+--- a/BUILD.bazel
++++ b/BUILD.bazel
+@@ -113,6 +113,7 @@ cc_library(
+ ":linux_mips64": COMMON_SRCS + LINUX_SRCS,
+ ":linux_riscv64": COMMON_SRCS + LINUX_SRCS,
+ ":linux_s390x": COMMON_SRCS + LINUX_SRCS,
++ ":linux_ppc64le": COMMON_SRCS + LINUX_SRCS,
+ ":macos_x86_64": COMMON_SRCS + X86_SRCS + MACH_SRCS + MACH_X86_SRCS,
+ ":macos_arm64": COMMON_SRCS + MACH_SRCS + MACH_ARM_SRCS,
+ ":windows_x86_64": COMMON_SRCS + X86_SRCS + WINDOWS_X86_SRCS,
+@@ -239,6 +240,11 @@ config_setting(
+ values = {"cpu": "s390x"},
+ )
+
++config_setting(
++ name = "linux_ppc64le",
++ values = {"cpu": "ppc"},
++)
++
+ config_setting(
+ name = "macos_x86_64",
+ values = {
+--
+2.40.1
+
--
2.40.1

Loading

0 comments on commit 9b6388b

Please sign in to comment.