Make TensorFlow Lite available as Swift Package Manager package

[lothrop]

Please make sure that this is a feature request. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template

System information

• TensorFlow version (you are using): Lite 2.3.0
• Are you willing to contribute it (Yes/No): yes

Describe the feature and the current behavior/state.
Right now, TensorFlow Lite is available as a CocoaPod or as source for iOS developers. CocoaPods cannot be used from inside a Swift Package Manager (SPM) package. Since SPM is the official package manager from Apple and the way forward, it would be nice to have TensorFlow Lite available as an SPM package or an XCFramwork, in addition to the current CocoaPods distribution.

Will this change the current api? How?
No

Who will benefit with this feature?
iOS developers who are using SPM for module manager or who are not using CocoaPods anymore.

Any Other info.
I have tried creating my own SPM package or XCFramework from the source but could not get it to work because of the native code. But I'm confident it should be possible.

[mrosales]

This isn't a solution, but I spent a fair amount of time digging into this trying to find a workaround. This is a little brain dump of my progress.

I was able to get a custom build of TFLite built and packaged as an XCFramework in a private SPM module, but it required a handful of hacks to get working properly. I'm not really comfortable with bazel though, so I did all of this with writing a bash script over a handful of bazel operations.

My workflow is:

1. Build a device static framework with

bazel build --config=ios --ios_multi_cpus="armv7,arm64" //tensorflow/lite/experimental/ios:TensorFlowLiteC_framework
# copy output file to temporary build dir

2. Build a simulator static framework with

bazel build --config=ios --ios_multi_cpus="i386,x86_64" //tensorflow/lite/experimental/ios:TensorFlowLiteC_framework
# copy output file to temporary build dir

3. Write some Info.plist files into the framework folders since bazel seems to omit them and Xcode will silently fail to install your app on a device if they are missing.

4. Create an xcframework with something like

xcodebuild -create-xcframework \
   -output "${OUTPUT}" \
   -framework "${BUILD_DIR}/dev/TensorFlowLiteC.framework" \
   -framework "${BUILD_DIR}/sim/TensorFlowLiteC.framework

5. At this point, I made a sample private git repo that vendored the swift code and the TensorFlowLiteC.xcframework file and then added a Package.swift file to make it work as a source for SPM. The instructions on the apple developer site give some helpful instructions for shipping a binary framework with SPM.

One limitation is that the xcframeworks will require you to manually disable the iOS simulator for arm64 architecture because tensorflow (rooted in a problem in bazel-core) assumes that arm64 is always an ios device, not a simulator. See the bug I filed here for details: bazelbuild/rules_apple#980

To "disable the iOS simulator for arm64", you need to set the following Xcode build setting:

EXCLUDED_ARCHS[sdk=iphonesimulator*] = arm64

[jdduke]

@yyoon and @teijeong have we done any investigation into supporting the Swift Package Manager?

[yyoon]

We haven't done any investigation, and I'll need to learn more about Swift Package Manager to understand how much effort it would be required to support it.

cc/ @morganchen12 who might have some experience about this.

[morganchen12]

Probably the most challenging part of adding SPM support is you will need to add SPM support for all of your dependencies. SPM packages are also defined per-repository, so one Package.swift file at the root of the tensorflow repo would be responsible for representing Tensorflow, TensorFlowLiteSwift, TensorFlowLiteObjC, TensorFlowLiteC, and any other modules that are buildable in a Swift target.

It looks like TensorFlowLiteSwift and TensorFlowLiteObjC have very shallow dependency graphs, so adding SPM support should be straightforward.

[greenzeal]

Today I hit the error building the framework with Xcode 12.3 Building for iOS Simulator, but the linked and embedded framework 'ITLogin.framework' was built for iOS + iOS Simulator. Which was working with Xcode 12.2 With Xcode 12.3 it is not a feature anymore, it is a bugfix.
It is time to move to XCFrameworks. Lots of big and small companies are already doing this.

https://gist.github.com/evnik introduced conversion script as a temporary solution: https://gist.github.com/evnik/6762d5c3a4b21f61f13b100e03b62c38

[yyoon]

Yeah, I think XCFramework makes more sense than Swift Package Manager approach, as XCFramework supports packaging compiled binaries as we currently do today.

As far as I can tell, CocoaPods supports XCFrameworks already, but bazel still doesn't support building for iOS arm64 simulator (for M1-based macs) nor building XCFrameworks natively. We might need to do some manual work in the interim, with something like the script @greenzeal linked.

@greenzeal In what exact situation were you seeing that specific error? Can you be a bit more specific and help me reproduce that issue? What does your project structure look like?

[greenzeal]

@yyoon
I am trying to create my framework, but have a dependency on TensorFlowLiteC.framework.
to reproduce:

• Xcode 12.3
• Having a TensorFlowLiteC.framework embedded
• Build for iOS Simulator with release configuration

The compiler will complain like this: Building for iOS Simulator, but the linked and embedded framework 'ITLogin.framework' was built for iOS + iOS Simulator.

Another simple way to reproduce:

• Xcode 12.3
• create dummy Xcode project
• pod init
• add to pods pod 'TensorFlowLiteObjC' (it will download its dependency TensorFlowLiteC.framework)
• pod install
• Build for iOS Simulator with release configuration

I tried different ways to find a solution like building the Bazel and then TensorFlowLiteC from the source, but all lead to that error.
Besides tried to build TensorFlowLiteC normally and then using the patch mentioned here after with lipo added third architecture to simulator build in the XCFramework.

I would really appreciate if anyone has a workaround. Or maybe it's possible to build TensorFlowLiteC.framework using Xcode (xcodebuild)?

[greenzeal]

btw also I will be happy if there is a way, even a manual one. And will be happy to try to automate it and share it here.

[yyoon]

@greenzeal I could create a dummy project as you described and confirmed that I could reproduce.

Do you really need to build your framework for arm64 simulator?
If not, you could go to the target build settings, and add arm64 as an excluded architecture for Any iOS Simulator SDK for both debug and release configs.

If you have a pressing need to build for arm64 simulator, you might want to try this:
bazelbuild/rules_apple#980 (comment)

This would let you build a TensorFlowLiteC slice for arm64 simulator (haven't had a chance to try it myself). Then, you could add the build output to the "Frameworks and Libraries" list of your framework project.

[greenzeal]

Excluding arm64 is a temporary option indeed.

I had tried with a patch, but had no luck yet to make it work. Will try again definitely.

[ebraraktas]

What is the current situation of the issue? @yyoon @teijeong @jdduke

I have tried to create TensorFlowLiteC.xcframework following the gist shared by @greenzeal (message) and steps described by @mrosales . I have created TensorFlowLiteC Swift Package and TensorFlowLite Swift Package (which depends on the former, TensorFlowLiteC) and successfully built, run and archived my app. However, it has failed at Upload to App Store stage (you can find error logs at the end). It would be nice to have official TensorFlowLiteC.xcframework to depend on. Most of the popular packages such as firebase have SPM support for their binaryTargets.

I have used official TensofFlowLiteC.framework distributed in Cocoapods and lipo to create simulator and ios frameworks, and added an Info.plist having relevant fields for a framework. Note that I had to separate TensorFlowLiteC.xcframework to its own package to work around some build issues on Xcode.

Errors

I have tried several workarounds to eliminate some errors like first two below.

I applied the solution in swift forums to eliminate:

ERROR ITMS-90680: "Invalid directory. The bundle Payload/MyApp.app/PlugIns/TensorFlowLiteC.framework is not contained in a correctly named directory. It should be under "Frameworks"."

and it seems worked out.

And, I am not sure if it is relevant but I tried to set ENABLE_BITCODE=NO for my project to get rid of the error below:

ERROR ITMS-90635: "Invalid Mach-O Format. The Mach-O in bundle "MyApp.app/Frameworks/TensorFlowLiteC.framework" isn’t consistent with the Mach-O in the main bundle. The main bundle Mach-O contains arm64(bitcode), while the nested bundle Mach-O contains arm64(machine code). Verify that all of the targets for a platform have a consistent value for the ENABLE_BITCODE build setting."

But as you can see below, now I have something similar and several others:

ERROR ITMS-90635: "Invalid Mach-O Format. The Mach-O in bundle "MyApp.app/Frameworks/TensorFlowLiteC.framework" isn’t consistent with the Mach-O in the main bundle. The main bundle Mach-O contains arm64(machine code), while the nested bundle Mach-O contains arm64(machine code). Verify that all of the targets for a platform have a consistent value for the ENABLE_BITCODE build setting."
ERROR ITMS-90171: "Invalid Bundle Structure - The binary file 'MyApp.app/Frameworks/TensorFlowLiteC.framework/TensorFlowLiteC' is not permitted. Your app can’t contain standalone executables or libraries, other than a valid CFBundleExecutable of supported bundles. Refer to the Bundle Programming Guide at https://developer.apple.com/go/?id=bundle-structure for information on the iOS app bundle structure."
ERROR ITMS-90124: "The binary is invalid. The executable 'MyApp.app/Frameworks/TensorFlowLiteC.framework/TensorFlowLiteC' has type 'OBJECT' that is not valid. Only 'EXECUTE' is permitted."
ERROR ITMS-90125: "The binary is invalid. The encryption info in the LC_ENCRYPTION_INFO load command is either missing or invalid, or the binary is already encrypted. This binary does not seem to have been built with Apple's linker."
ERROR ITMS-90210: "Missing load commands. The executable at 'MyApp.app/Frameworks/TensorFlowLiteC.framework' does not have the necessary load commands. Try rebuilding the app with the latest Xcode version. If you are using third party development tools, contact the provider."```

[jdduke]

Also looping in @miaout17

[sjang42]

Any progress here? Is there nobody using TensorFlowLiteC.framework on m1 simulator now?

[leviyehonatan]

i am having the same issues and i do not manage with the offered work arounds, would appreciate any direct help of how to currently solve this problem of using TensorFlowLiteSwift as a dependency! help!

[Edgarchuk]

Is there any progress?

[adammasyk]

+1

[wcappel]

+1

[arctop-sk]

@pkgoogle @yishuangP This is absolutly nonsensical.

A feature request that has been open for 3 years, isn't even a big feature or a development of any sort of core functionality, but rather just a bit of putting together an automated script and a wrapper, uploading it here, and releasing us all from these workarounds.

There are a dozen of engineers here, that with the proper access and support will HAPPILY do the work and contribute. In fact some of us have already spent hours trying to work this for the good of the community.

Furthermore, the lack of communication of something that is trivial, with anyone from the team not having updated or participated in this thread for almost a full year, is downright unacceptable.

[teonicel]

Maybe this can help you https://github.com/kewlbear/TensorFlowLiteC

[yomw]

@pkgoogle @yishuangP Is there a way to get an official response from the team? Even just to say "Sorry we won't do anything for the time being" (though an ETA would be much appreciated). This issue has been tagged with "Awaiting tensorflower" for 4 months now...

[imyrvold]

+1

[teonicel]

If you manage to export the current pod as .xcframework making it a SPM is very easy

[arctop-sk]

If you manage to export the current pod as .xcframework making it a SPM is very easy

It is very easy to roll out your own SPM.

Problem is that won't correctly sign and you cannot deploy that to the App Store.

Works great in debug but useless in actual apps.

[jnelle]

+1

[mariomastrandrea-poli]

+1

[tareksabry1337]

Is there any hope to have SPM anytime in the future?

[tareksabry1337]

I gave up on TensorFlow creating SPM, I created my own SPM repo that passes code signing issues in App Store and uploads just fine https://github.com/tareksabry1337/TensorFlowLiteC

[yomw]

@tareksabry1337 Could you elaborate on how you did it? Maybe the guys here will try and implement it if they have the solution (if anyone at tensorflow actually still reads this thread - cc @yishuangP)

[tareksabry1337]

@yomw Everything is in the repo, it's an automated GitHub action that builds static variant of TensorFlowLiteC framework, and modifies TensorFlowLiteCCoreML build script to build a static variant too, both for device, building simulator variant from these scripts fails for some obscure reason that I don't understand, to fix that I install the version from Cocoapods and copy over the simulator slice and then combine both slices (device static slice / simulator dynamic slice) and create an XCFramework that works just fine.

[Mr-Goldberg]

+1

[arcslash]

+1

[Bryanlin920616]

+1

[itjustcrashed]

+54

this is turning into the GNOME gitlab

[vijaytholpadi]

+1

Can we get an update on this please?

It's been quite a while since SPM has been out.

[r-a-o]

Can we PLEASE add SPM support already?!

Since everyone is now starting to use SPM and dropping cocoapods, this will really help all Apple devs.

(Its been 4years since the issue was opened and we're all willing to help & get this done!)

Thanks!

Attn: @pkgoogle @yyoon @yishuangP

[vijaytholpadi]

Literally having to choose CoreML because of this limitation.

SPM support is more critical to iOS ecosystem than it might seem.

CocoaPods has gone into maintenance mode earlier this year and we will only see more people move to SPM from here on as that's the only option around with an actual feature and maintenance roadmap.

[mmaetzler]

I wonder if this SPM issue is part of the reason why there is no SPM yet? I think it would be impossible at the moment to bundle the binary frameworks inside the SPM because of that issue.

[tareksabry1337]

@mmaetzler I literally packaged TensorFlow under SPM on a fork and it's currently in app that's on the AppStore, I can't fathom why such simple request is being completely ignored.

[mmaetzler]

@tareksabry1337 I did the same. I have an app submitted to the store with my own (internal) tflite SPM. But I was not able to create an SPM with TensorFlowLiteSelectOps because the binary size requires us to use git lfs and SPM does not support git lfs files. So if your model requires a good chunk of the select ops, then you are out of luck.

[r-a-o]

@pkgoogle Continuing the conversation here as you suggested, #78787 (comment)

Could you please tell us if we can expect this to be resolved in the next 6 months? Its crazy that after 4 years and so many developers requesting & waiting for so long, there is no clarity. There must be some valid reason for this, right?
If there is a reason, please let us know so we don't waste each other's time.
Thanks.

[pkgoogle]

HI @lothrop,

Thanks for raising this issue. Are you aware of the migration to LiteRT? This transition is aimed at enhancing our project's capabilities and providing improved support and focus for our users. As we believe this issue is still relevant to LiteRT we are moving your issue there. Please follow progress here.

Let us know if you have any questions. Thanks.