tensorflow 2.11 #286

ngam · 2022-11-21T05:10:55Z

Linux + CUDA -- http://anaconda.org/cftf-linux/ -- log.txt
Linux + CPU -- http://anaconda.org/cftf-linux/-- log.txt
OSX x86 -- https://anaconda.org/pkgw-forge/ -- logs.txt
OSX arm -- https://anaconda.org/pkgw-forge/ -- logs.txt

Included migrations

openssl3
grpc 1.51

Checklist

Used a personal fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

conda-forge-linter · 2022-11-21T05:11:02Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe:

It looks like the 'tensorflow-base' output doesn't have any tests.

ngam · 2022-11-21T05:11:07Z

@conda-forge-admin, please rerender.

…nda-forge-pinning 2022.11.20.19.10.04

ngam · 2022-11-21T05:20:05Z

Status:

--

- ~waiting on https://github.com/conda-forge/keras-feedstock/pull/61~ - looks like cuda112-py311 is failing, we will *not* migrate py311 for now - we are *not* migrating openssl3

Artifacts & logs:

…nda-forge-pinning 2022.11.22.12.08.38

ngam · 2022-11-23T03:02:57Z

macos runs failing with "snappy.h not found", likely related to this commit: tensorflow/tensorflow@05ad750

Fixed locally, update PR 7f20e5a.

Test failures on linux-64 cpu builds: (fixed in 7f20e5a)

/home/conda/feedstock_root/build_artifacts/tensorflow-split_1669167736163/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include/tensorflow/core/framework/numeric_types.h:24:10: fatal error: tensorflow/tsl/framework/numeric_types.h: No such file or directory
   24 | #include "tensorflow/tsl/framework/numeric_types.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Tests failed for libtensorflow_cc-2.11.0-cpu_hf8653da_0.conda - moving package to /home/conda/feedstock_root/build_artifacts/broken

xref:
tensorflow/tensorflow@d5446e7
tensorflow/tensorflow@6117aa1

Likely, this line needs to be changed to reflect new changes (Eigen --> Eigen & tsl?): https://github.com/ngam/tensorflow-feedstock/blob/4c529f97eacc3f8e185c1376865fe9c21b022e27/recipe/build.sh#LL188

fixed!

ngam · 2022-11-23T19:38:03Z

@xhochy how can I build tensorflow osx-arm64 natively on M1 using this recipe? It is cross-compiled as is, and I cannot remember what changes you made that made it cross-compile (it used to be native).

Wildly enough, for now this recipe can only compile for osx-arm64 if I am inside a conda osx-64 env, to then cross-compile for osx-arm64. Even crazier, it completely fails for osx-64. It always fails if I start from an osx-arm64 env. The errors I get are somewhat incoherent, except for the case when starting with osx-arm64 env to compile osx-64 (it complains about x86_64 stuff)

ngam · 2022-12-03T00:37:15Z

I cannot build the osx ones (something is wrong with my local setup). Others are welcome to try. Sorry.

hmaarrfk · 2022-12-03T14:28:46Z

can you build anything from osx? maybe even x86?

ngam · 2022-12-03T15:35:00Z

No, for some reason, I cannot build anything. It is surprising because it is failing much earlier than the CI... However, my machines are arm64, so I assume that might be it...

ngam · 2022-12-05T05:49:27Z

If nobody attempts to build, I will have another closer look next week. There also seems to be some effort to change how they build osx (especially to open-source Metal plug). I will also update this to include openssl3 before we merge.

h-vetinari · 2022-12-06T00:21:39Z

If nobody attempts to build, I will have another closer look next week.

Should we attempt to migrate libgrpc 1.50 in the meantime (I could do it)?

1.51 is already out (we could even migrate to that¹), and tensorflow is one of the trickiest ones to get on even footing w.r.t. grpc, making it hard to install with other packages.

seems we don't need to keep a version "in reserve" for the privatization stuff, as the release quite frequently... ↩

ngam · 2022-12-06T00:23:41Z

Yes, we can tinker around. Also please know that I can readily and quickly build the linux stuff at least through the end of Dec (I will lose access to a lot of computing resources on Dec 31 😢)

h-vetinari · 2022-12-06T19:51:32Z

Grpc PR arrived: #289

Should be easy to include. :)

Pity to hear you'll lose access to those compute resources, it was very helpful to have you building/debugging things so regularly ♥️

ngam · 2022-12-06T19:55:00Z

Fingers crossed I will have access to even better systems (maybe even including some of these alt linux arches we struggle with...) 😉

(It will just take a bit of time to get everything set up... so hopefully not more than ~60-day interruption!)

h-vetinari · 2022-12-09T06:22:23Z

@conda-forge-admin, please rerender

…nda-forge-pinning 2022.12.08.16.56.34

ngam · 2022-12-09T18:27:22Z

Will attempt building this again over the weekend

hmaarrfk · 2022-12-10T16:14:03Z

So flatbuffers 22 was released yesterday on conda-forge breaking the so name.

pkgw · 2023-01-08T18:57:25Z

My anaconda account pkgw-forge now contains all six macOS builds: OpenSSL3 × { Python 3.8, 3.9, 3.10 } × { osx-64, osx-arm64 }. Here are the build logs:

logs.zip

I never got the cross-builds to pass the testing phase due to cross-execution issues but that doesn't mean that there's anything wrong with the builds. (I think I must be doing something wrong with setting up the cross build invocation ... my last error was with the imports: test step which mambabuild must be smart enough to realize won't work on a cross environment, no?)

hmaarrfk · 2023-01-08T19:58:20Z

the build locally command should skip the tests right?

Sorry i just got back from travels, i'll move the builds over.

hmaarrfk · 2023-01-08T21:12:53Z

I've tested python 3.9 and cuda, as well as CPU on linux using my workflow. seems to work ok. i'm going to upload all. Thanks for the hard work.

I seemed to have a problem with openssl3+cuda on linux though. I can't seem to point it down.

In order to build the subpackages that have Python dependencies, they need to have Python and cross-python as build deps. tensorflow-estimator further needs to have tensorflow-base as a build dep because its build process needs to `import tensorflow` in order to generate API bindings, and in the cross-python setup the build-arch package is required to provide binary Python modules for import.

conda-forge-webservices · 2023-01-08T21:31:01Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe:

It looks like the 'tensorflow-base' output doesn't have any tests.

hmaarrfk · 2023-01-08T21:33:31Z

Ok, all got uploaded. Thanks all. Onto the next stuff.

Seems like grpc 1.51 also got folded in.

h-vetinari · 2023-01-08T21:36:21Z

Woohoo, thanks everyone for all the hard work! 🥳

Seems like grpc 1.51 also got folded in.

Yes, that was explicitly a goal.

hmaarrfk · 2023-01-08T21:37:55Z

Yes, that was explicitly a goal.

It really became hard to follow what got "done" especially since the migration file got deleted.

Ultimately, i checked a few logs.

h-vetinari · 2023-01-08T21:45:40Z

It really became hard to follow what got "done" especially since the migration file got deleted.

That's because the migration got closed in the meantime (it was clear that the next TF build would contain it), and it's right there in the diff (for the config yamls). But happy to answer any questions about these things next time.

hmaarrfk · 2023-01-08T22:55:55Z

i think what might help as these CFEP03 builds get done is if somebody wanted to keep the top level comment updated with highlights such as this one.

Thanks for help on all of these difficult builds!

ngam · 2023-01-08T23:43:29Z

Thanks all, especially @pkgw and as always @hmaarrfk and @h-vetinari for keeping track of everything. Hopefully things are going to be easier in the next releases. Thanks!!!

version(s) update(s)

616fb7e

MNT: Re-rendered with conda-build 3.23.1, conda-smithy 3.22.0, and co…

1bce903

…nda-forge-pinning 2022.11.20.19.10.04

ngam closed this Nov 21, 2022

ngam reopened this Nov 21, 2022

ngam and others added 2 commits November 22, 2022 12:11

Delete python311.yaml

fa74ea2

MNT: Re-rendered with conda-build 3.23.1, conda-smithy 3.22.0, and co…

4c529f9

…nda-forge-pinning 2022.11.22.12.08.38

add tsl to includes

7f20e5a

ngam and others added 3 commits December 8, 2022 01:39

libgrpc151

bda76ff

openssl3

2d8278a

fix migrator files

fe8a8f5

conda-forge deleted a comment from github-actions bot Dec 9, 2022

MNT: Re-rendered with conda-build 3.23.3, conda-smithy 3.22.1, and co…

ca707b3

…nda-forge-pinning 2022.12.08.16.56.34

Pin flatbuffers

d40e27e

pkgw added 3 commits January 8, 2023 16:30

Require Bazel <6

e92b24a

Deselect test commands that won't work for cross builds

a3b2c3d

hmaarrfk marked this pull request as ready for review January 8, 2023 21:31

hmaarrfk requested review from FarhanTejani, ghego, hajapy, jschueller, njzjz, waitingkuo, wolfv and xhochy as code owners January 8, 2023 21:31

hmaarrfk merged commit 0422d85 into conda-forge:main Jan 8, 2023

This was referenced Jan 8, 2023

Rebuild for openssl3 #287

Closed

Rebuild for libgrpc151 #289

Closed

tensorflow v2.11.0 #285

Closed

tensorflow v2.10.1 #283

Closed

h-vetinari mentioned this pull request Jan 8, 2023

Timeline for closing out OpenSSL 3 migration / dropping OpenSSL 1.1.1 conda-forge/conda-forge-pinning-feedstock#3838

Closed

28 tasks

h-vetinari mentioned this pull request Jan 26, 2023

Something is causing conflicts on OSX #298

Closed

ngam mentioned this pull request Mar 17, 2023

Rebuild for jpegturbo #303

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tensorflow 2.11 #286

tensorflow 2.11 #286

ngam commented Nov 21, 2022 •

edited by hmaarrfk

Loading

conda-forge-linter commented Nov 21, 2022

ngam commented Nov 21, 2022 •

edited

Loading

ngam commented Nov 21, 2022 •

edited

Loading

ngam commented Nov 23, 2022 •

edited

Loading

ngam commented Nov 23, 2022 •

edited

Loading

ngam commented Dec 3, 2022 •

edited

Loading

hmaarrfk commented Dec 3, 2022

ngam commented Dec 3, 2022

ngam commented Dec 5, 2022

h-vetinari commented Dec 6, 2022

ngam commented Dec 6, 2022 •

edited

Loading

h-vetinari commented Dec 6, 2022

ngam commented Dec 6, 2022

h-vetinari commented Dec 9, 2022

ngam commented Dec 9, 2022

hmaarrfk commented Dec 10, 2022

pkgw commented Jan 8, 2023

hmaarrfk commented Jan 8, 2023

hmaarrfk commented Jan 8, 2023

conda-forge-webservices bot commented Jan 8, 2023

hmaarrfk commented Jan 8, 2023

h-vetinari commented Jan 8, 2023

hmaarrfk commented Jan 8, 2023

h-vetinari commented Jan 8, 2023

hmaarrfk commented Jan 8, 2023

ngam commented Jan 8, 2023

tensorflow 2.11 #286

tensorflow 2.11 #286

Conversation

ngam commented Nov 21, 2022 • edited by hmaarrfk Loading

conda-forge-linter commented Nov 21, 2022

ngam commented Nov 21, 2022 • edited Loading

ngam commented Nov 21, 2022 • edited Loading

Status:

ngam commented Nov 23, 2022 • edited Loading

ngam commented Nov 23, 2022 • edited Loading

ngam commented Dec 3, 2022 • edited Loading

hmaarrfk commented Dec 3, 2022

ngam commented Dec 3, 2022

ngam commented Dec 5, 2022

h-vetinari commented Dec 6, 2022

Footnotes

ngam commented Dec 6, 2022 • edited Loading

h-vetinari commented Dec 6, 2022

ngam commented Dec 6, 2022

h-vetinari commented Dec 9, 2022

ngam commented Dec 9, 2022

hmaarrfk commented Dec 10, 2022

pkgw commented Jan 8, 2023

hmaarrfk commented Jan 8, 2023

hmaarrfk commented Jan 8, 2023

conda-forge-webservices bot commented Jan 8, 2023

hmaarrfk commented Jan 8, 2023

h-vetinari commented Jan 8, 2023

hmaarrfk commented Jan 8, 2023

h-vetinari commented Jan 8, 2023

hmaarrfk commented Jan 8, 2023

ngam commented Jan 8, 2023

ngam commented Nov 21, 2022 •

edited by hmaarrfk

Loading

ngam commented Nov 21, 2022 •

edited

Loading

ngam commented Nov 21, 2022 •

edited

Loading

ngam commented Nov 23, 2022 •

edited

Loading

ngam commented Nov 23, 2022 •

edited

Loading

ngam commented Dec 3, 2022 •

edited

Loading

ngam commented Dec 6, 2022 •

edited

Loading