Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Xcode 12.5+ build of iPhone Simulator on Apple M1 #32284

Closed
wants to merge 2 commits into from

Conversation

barbieri
Copy link
Contributor

Summary

Since Apple released its own silicon M1, an ARM64, the react-native build is broken or at least not as effective as it should.

This PR stops excluding arm64 simulator (this is not needed on the M1 neither on Intel devices) and removes the problematic $(TOOLCHAIN_DIR)/usr/lib/swift-5.0/$(PLATFORM_NAME) from LIBRARY_SEARCH_PATHS, since on Xcode 12.5 and 13.0 this folder contains only i386/x86_64 binaries and will fail compilation.

Instead this PR forces $(SDKROOT)/usr/lib/swift while it removes the incorrect directory. Ideally we could just remove LIBRARY_SEARCH_PATHS altogether if $(inherited) and $(SDKROOT)/usr/lib/swift were the only entries, but it would require us a newer CocoaPods, since that was fixed with 1.11 (see CocoaPods/CocoaPods@6985cbf). Since we don't enforce that, lets keep the $(SDKROOT)/usr/lib/swift and call it done.

Last but not least, deprecate the __apply_Xcode_12_5_M1_post_install_workaround() as it's not needed anymore, at least with recent versions of the dependencies, no patching is required with RCT-Folly, neither we need to force IPHONEOS_DEPLOYMENT_TARGET=11.0

Changelog

[iOS] [Fixed] - Xcode 12.5+ build of iPhone Simulator on Apple M1
[iOS] [Changed] - Do not exclude the arm64 iphonesimulator
[iOS] [Deprecated] - __apply_Xcode_12_5_M1_post_install_workaround()

Test Plan

  • Build packages/rn-tester on M1 and see it still works properly
  • Run pod install on x86_64 and arm64 (m1) and see the project.pbxproj is not changed

References:

@facebook-github-bot facebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Shared with Meta Applied via automation to indicate that an Issue or Pull Request has been shared with the team. labels Sep 28, 2021
@mikehardy
Copy link
Contributor

@hramos has enabled "commitly" releases on CircleCI (like nightly builds, but for every commit) and perhaps he could enable it for all branches or PRs ? Otherwise it will be difficult for me to test this

An example:
https://app.circleci.com/pipelines/github/facebook/react-native/10500/workflows/e371829f-6c5f-4621-bd6b-2e693806f19a/jobs/219350/artifacts

It appears that the core of it is all in the text file scripts/react_native_pods.rb though? (along with a tweak to the pbxproj that will be manual) https://github.com/facebook/react-native/pull/32284/files?authenticity_token=ySqltbmTwcBjXyrBTB3DwWdpYfJMHVTq2LrWsgPqdei2gfnF0fHR2Xm8kWwhZIhXqo4KKmSArLWVpk9JXcUONQ%3D%3D&file-filters%5B%5D=.pbxproj&file-filters%5B%5D=.rb#diff-adcf572f001c2b710d14f409c14763f1a50b08369b3034548f1602685d21f67f

If you ran npx patch-package react-native on your local rn-tester installation (after running ./gradlew clean in android so it's a clean package) and attach that here I think it would be more test-able by people to see if it worked for them

@analysis-bot
Copy link

analysis-bot commented Sep 28, 2021

Platform Engine Arch Size (bytes) Diff
android hermes arm64-v8a 7,713,347 +0
android hermes armeabi-v7a 7,242,182 +0
android hermes x86 8,132,954 +0
android hermes x86_64 8,098,354 +0
android jsc arm64-v8a 9,632,691 +0
android jsc armeabi-v7a 8,549,051 +0
android jsc x86 9,646,592 +0
android jsc x86_64 10,255,678 +0

Base commit: f3fe7a0

@barbieri
Copy link
Contributor Author

CI Failures:

  • Windows: unrelated 503
  • iOS: possible related __swift_FORCE_LOAD_$_swiftCompatibilityDynamicReplacements, I'll take a look and update the PR

@analysis-bot
Copy link

Platform Engine Arch Size (bytes) Diff
ios - universal n/a --

Base commit: f3fe7a0

@barbieri
Copy link
Contributor Author

While it passed most tests, I'll do some improvements to react_native_pods.rb as some people seems to have extra quotes and those must be fixed as well.

Remove __apply_Xcode_12_5_M1_post_install_workaround() usage and make
that function empty.

Instead just force `$(SDKROOT)/usr/lib/swift` into the
LIBRARY_SEARCH_PATHS, removing any
`$(TOOLCHAIN_DIR)/usr/lib/swift-5.0/$(PLATFORM_NAME)` as that contains
only i386+x86_64 in iphonesimulator, failing the build for M1 arm64
simulators.
The "project.pbxproj" file was being updated by team members running
on x86_64 and arm64 as the file was being changed based on their
hw.optional.arm64 value.

Xcode 12.4 and on (at least, couldn't test on older versions) properly
handle builds on both x86_64 and arm64.
@barbieri
Copy link
Contributor Author

@mikehardy see if the following patch helps. I did it on top of 0.65.1, save as patches/react-native+0.65.1.patch so npx patch-package works:

diff --git a/node_modules/react-native/scripts/react_native_pods.rb b/node_modules/react-native/scripts/react_native_pods.rb
index 3e79bd6..ad619f8 100644
--- a/node_modules/react-native/scripts/react_native_pods.rb
+++ b/node_modules/react-native/scripts/react_native_pods.rb
@@ -123,20 +123,49 @@ def exclude_architectures(installer)
     .uniq{ |p| p.path }
     .push(installer.pods_project)
 
-  arm_value = `/usr/sbin/sysctl -n hw.optional.arm64 2>&1`.to_i
-
   # Hermes does not support `i386` architecture
   excluded_archs_default = has_pod(installer, 'hermes-engine') ? "i386" : ""
 
   projects.each do |project|
     project.build_configurations.each do |config|
-      if arm_value == 1 then
-        config.build_settings["EXCLUDED_ARCHS[sdk=iphonesimulator*]"] = excluded_archs_default
-      else
-        config.build_settings["EXCLUDED_ARCHS[sdk=iphonesimulator*]"] = "arm64 " + excluded_archs_default
+      config.build_settings["EXCLUDED_ARCHS[sdk=iphonesimulator*]"] = excluded_archs_default
+    end
+
+    project.save()
+  end
+end
+
+def fix_library_search_paths(installer)
+  def fix_config(config)
+    lib_search_paths = config.build_settings["LIBRARY_SEARCH_PATHS"]
+    if lib_search_paths
+      if lib_search_paths.include?("$(TOOLCHAIN_DIR)/usr/lib/swift-5.0/$(PLATFORM_NAME)") || lib_search_paths.include?("\"$(TOOLCHAIN_DIR)/usr/lib/swift-5.0/$(PLATFORM_NAME)\"")
+        # $(TOOLCHAIN_DIR)/usr/lib/swift-5.0/$(PLATFORM_NAME) causes problem with Xcode 12.5 + arm64 (Apple M1)
+        # since the libraries there are only built for x86_64 and i386.
+        lib_search_paths.delete("$(TOOLCHAIN_DIR)/usr/lib/swift-5.0/$(PLATFORM_NAME)")
+        lib_search_paths.delete("\"$(TOOLCHAIN_DIR)/usr/lib/swift-5.0/$(PLATFORM_NAME)\"")
+        if !(lib_search_paths.include?("$(SDKROOT)/usr/lib/swift") || lib_search_paths.include?("\"$(SDKROOT)/usr/lib/swift\""))
+          # however, $(SDKROOT)/usr/lib/swift is required, at least if user is not running CocoaPods 1.11
+          lib_search_paths.insert(0, "$(SDKROOT)/usr/lib/swift")
+        end
       end
     end
+  end
 
+  projects = installer.aggregate_targets
+    .map{ |t| t.user_project }
+    .uniq{ |p| p.path }
+    .push(installer.pods_project)
+
+  projects.each do |project|
+    project.build_configurations.each do |config|
+      fix_config(config)
+    end
+    project.native_targets.each do |target|
+      target.build_configurations.each do |config|
+        fix_config(config)
+      end
+    end
     project.save()
   end
 end
@@ -147,6 +176,7 @@ def react_native_post_install(installer)
   end
 
   exclude_architectures(installer)
+  fix_library_search_paths(installer)
 end
 
 def use_react_native_codegen!(spec, options={})
diff --git a/node_modules/react-native/template/ios/HelloWorld.xcodeproj/project.pbxproj b/node_modules/react-native/template/ios/HelloWorld.xcodeproj/project.pbxproj
index acb72e4..4aee980 100644
--- a/node_modules/react-native/template/ios/HelloWorld.xcodeproj/project.pbxproj
+++ b/node_modules/react-native/template/ios/HelloWorld.xcodeproj/project.pbxproj
@@ -435,8 +435,8 @@
 					"$(inherited)",
 				);
 				LIBRARY_SEARCH_PATHS = (
+					"\"$(SDKROOT)/usr/lib/swift\"",
 					"\"$(TOOLCHAIN_DIR)/usr/lib/swift/$(PLATFORM_NAME)\"",
-					"\"$(TOOLCHAIN_DIR)/usr/lib/swift-5.0/$(PLATFORM_NAME)\"",
 					"\"$(inherited)\"",
 				);
 				MTL_ENABLE_DEBUG_INFO = YES;
@@ -492,8 +492,8 @@
 					"$(inherited)",
 				);
 				LIBRARY_SEARCH_PATHS = (
+					"\"$(SDKROOT)/usr/lib/swift\"",
 					"\"$(TOOLCHAIN_DIR)/usr/lib/swift/$(PLATFORM_NAME)\"",
-					"\"$(TOOLCHAIN_DIR)/usr/lib/swift-5.0/$(PLATFORM_NAME)\"",
 					"\"$(inherited)\"",
 				);
 				MTL_ENABLE_DEBUG_INFO = NO;

@fkgozali
Copy link
Contributor

Thanks a lot for the fix @barbieri! Just so I understand this: we need this PR if we want to stay with CocoaPods 1.10.x, or we could require upgrade (in main and next release) to CocoaPods 1.11.x and this PR is no longer necessary?

@fkgozali
Copy link
Contributor

@mikehardy

@hramos has enabled "commitly" releases on CircleCI (like nightly builds, but for every commit) and perhaps he could enable it for all branches or PRs ? Otherwise it will be difficult for me to test this

He's landing that support as we speak, so should be available later today hopefully.

@facebook-github-bot
Copy link
Contributor

@fkgozali has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@barbieri
Copy link
Contributor Author

@fkgozali this works and is required on both 1.10 and 1.11.

If we force 1.11 we could do a different fix_library_search_paths() implementation: instead of lib_search_paths.insert(0, "$(SDKROOT)/usr/lib/swift") we could just delete the LIBRARY_SEARCH_PATHS altogether IF AND ONLY IF all that is there are a subset of the following lines. (Note we must respect this condition as users may have manually added other libraries in there, in such case they should be kept intact)

  • $(TOOLCHAIN_DIR)/usr/lib/swift/$(PLATFORM_NAME)
  • $(SDKROOT)/usr/lib/swift
  • $(inherited)

In every case, we MUST remove $(TOOLCHAIN_DIR)/usr/lib/swift-5.0/$(PLATFORM_NAME) (note the -5.0 in there) because on M1 Apple did a mistake and only provides i386 and x86_64:

$ file /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift-5.0/iphonesimulator/libswiftFoundation.dylib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift-5.0/iphonesimulator/libswiftFoundation.dylib: Mach-O universal binary with 2 architectures: [i386:Mach-O dynamically linked shared library i386] [x86_64]
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift-5.0/iphonesimulator/libswiftFoundation.dylib (for architecture i386):	Mach-O dynamically linked shared library i386
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift-5.0/iphonesimulator/libswiftFoundation.dylib (for architecture x86_64):	Mach-O 64-bit dynamically linked shared library x86_64

The difference in CocoaPods is that 1.11 was fixed to automatically link with Swift if any of the dependencies uses that (ie: YogaKit does), then with 1.11 we do NOT need LIBRARY_SEARCH_PATHS set in the user project because Pods/Target Support Files/Pods-${APP_NAME}/Pods-${APP_NAME}.debug.xcconfig (and .release) will have it properly set. Older versions we must do that ourselves.

@fkgozali
Copy link
Contributor

@barbieri Thanks for the explanation. I'm trying to figure out if:

  • we should land this PR as is right now
  • we should prepare for 1.11.x upgrade soon
  • we should have another PR on top of 1.11.x upgrade so that main branch will be on the cleanest setup possible

I think we should proceed in that order. Now, whether this fix should be considered for 0.65.1 or 0.66.1 (patched releases), we'll discuss that separately. I don't think we need to block on this to make 0.66.0 stable though. cc @lunaleaps

@barbieri
Copy link
Contributor Author

@fkgozali I just got my M1, but it seems it's worth to require the following tools and versions:

  • ruby 2.7.4 (ex: via rbenv), because the 2.6.3 that comes with macOS Big Sur is a universal build and with that I found many pitfalls, such as the Gemfile.lock containing weird PLATFORMS entries. I also noticed problems with ffi and illegal instructions... which I did not debug, but seems it was lost which module to use (x86_64 or arm64). Going with 2.7.4 from rbenv it works like a charm, full arm64... no rosetta2 emulation is needed. (While at that, I recommend using PLATFORMS: ruby in the Gemfile.lock, otherwise it keeps changing with macOS version and architecture)
  • bundler 2.2.28 (no further comments, but I'm using this one)
  • CocoaPods 1.11.2 as it contains CocoaPods/CocoaPods@6985cbf and possibly other commits to improve Xcode and M1 usage.
  • node 16 (or 15), since that is supported natively in macOS/M1 and runs much faster than legacy 12 (all of those managed by nvm). Note that @types/react-native clashes with that since AbortSignal and AbortController are already defined by @types/node-16.

@fkgozali
Copy link
Contributor

Re tooling upgrades: we can track that separately for main and next releases, but probably not worth bringing those upgrades requirements to 0.65, 0.66 for now.

@mikehardy
Copy link
Contributor

mikehardy commented Sep 28, 2021

re: tooling, does react-native really get into what ruby a user is using? That said, seems like just strongly recommending a non-Rosetta ruby execution environment is the goal, and detecting it is possible (snippets on stackoverflow, and my tested version below).

For cocoapods, I think enforcing 1.11.x is a pretty easy requirement for the next release unless I'm missing something, isn't switching to it from 1.10 trivial? I don't recall any breaking changes? (I just scanned their releases list again, I can't see any)


Bash arch detection, exits on Rosetta while wagging a finger at you - ruby syntax port is an exercise for the reader 😅

  # We do not want to run under Rosetta 2, brew doesn't work and compiles might not work after
  arch_name="$(uname -m)"
  if [ "${arch_name}" = "x86_64" ]; then
    if [ "$(sysctl -in sysctl.proc_translated)" = "1" ]; then
      echo "Running on Rosetta 2"
      echo "This is not supported. Run \`env /usr/bin/arch -arm64 /bin/bash --login\` then try again"
      exit 1
    else
      echo "Running on native Intel"
    fi
  elif [ "${arch_name}" = "arm64" ]; then
    echo "Running on ARM"
  else
    echo "Unknown architecture: ${arch_name}"
  fi

@fkgozali
Copy link
Contributor

For cocoapods, I think enforcing 1.11.x is a pretty easy requirement for the next release unless I'm missing something, isn't switching to it from 1.10 trivial?

Yeah, that's right, the only question is do we want to put that line in 0.66.0 release note or wait for later release.

@mikehardy
Copy link
Contributor

For cocoapods, I think enforcing 1.11.x is a pretty easy requirement for the next release unless I'm missing something, isn't switching to it from 1.10 trivial?

Yeah, that's right, the only question is do we want to put that line in 0.66.0 release note or wait for later release.

Well, it's not a requirement for rc4 so I wouldn't add it now. If you merge this as is, still doesn't need it. If we go for a cleaner version then and only then it's needed if I understand correctly. So for me, later release (0.67)

I'm responding fast just because I happen to be paying attention about this right now, I don't feel super strongly about it and I'm just expressing opinion, to be clear. I do have an opinion but at the end of the day if it builds for me from npx react-native init on an M1 I'm happy :-)

@fkgozali
Copy link
Contributor

I'm gonna land this as is to main, per the convo above.

@mikehardy
Copy link
Contributor

Ah cool, then with the commitlies everyone interested can start playing with it with a notice to #ios perhaps, so we can replace my smelly batch o' hacks with this cleaner approach with confidence :-)

@barbieri
Copy link
Contributor Author

re: tooling, does react-native really get into what ruby a user is using? That said, seems like just strongly recommending a non-Rosetta ruby execution environment is the goal, and detecting it is possible (snippets on stackoverflow, and my tested version below).

Since it does pod install by default, I'd say so. Actually I'd suggest start shipping a Gemfile + Gemfile.lock in the template project, with every pod execution prefixed with bundle exec pod ... so we're sure it's using the local version. More on that, we could use the deployment flag or --path .bundler to force local installation (and simplify the react-native init, removing that prompt "where to get cocoapods?").

For cocoapods, I think enforcing 1.11.x is a pretty easy requirement for the next release unless I'm missing something, isn't switching to it from 1.10 trivial? I don't recall any breaking changes? (I just scanned their releases list again, I can't see any)

Agreed. I don't think there is any breaking change, but needs more testing to confirm (as usual).

As for your script to detect rosetta, that makes sense.

@barbieri barbieri deleted the fix/xcode12.5-and-m1 branch September 29, 2021 12:33
facebook-github-bot pushed a commit that referenced this pull request Sep 30, 2021
Summary:
* Remove left over from a1c445a#commitcomment-57240925
* Add a warning if running with Rosetta2 as per #32284 (comment)

## Changelog

[iOS] [Fixed] - Removed __apply_Xcode_12_5_M1_post_install_workaround
[iOS] [Changed] - Warn if Rosetta2 is being used (x86_64 on arm64)

Pull Request resolved: #32296

Test Plan: Build on macOS Apple devices without any warnings during `pod install`

Reviewed By: RSNara

Differential Revision: D31291567

Pulled By: fkgozali

fbshipit-source-id: 65e54507dedcdba39c1b441aad85e940eedc8b52
@mikehardy
Copy link
Contributor

@barbieri

@fkgozali I just got my M1, but it seems it's worth to require the following tools and versions:

  • ruby 2.7.4 (ex: via rbenv), because the 2.6.3 that comes with macOS Big Sur is a universal build and with that I found many pitfalls, such as the Gemfile.lock containing weird PLATFORMS entries. I also noticed problems with ffi and illegal instructions... which I did not debug, but seems it was lost which module to use (x86_64 or arm64). Going with 2.7.4 from rbenv it works like a charm, full arm64... no rosetta2 emulation is needed. (While at that, I recommend using PLATFORMS: ruby in the Gemfile.lock, otherwise it keeps changing with macOS version and architecture)

Having experienced this ruby version spec / Gemfile stuff in the template for the last couple months, I have to say it adds friction to my workflows. Specifically if you do npx react-native init and you have current stable ruby (3.1.1, not 2.7.x) the react-native CLI sees the underlying version mismatch as an error, assumes cocoapods isn't installed and hangs waiting for interactive prompt.

The workaround for me is to add --skip-install to every init in my scripts, then immediately delete all the ruby version pinning stuff, then run yarn and npx pod-install.

If I could propose deleting all of that ruby version thing from the template I'd be a happy person. Then if cocoapods was going to fail, it would fail anyway, but if it was going to work (with a newer version) at least it would actually work which would be nice...

@barbieri
Copy link
Contributor Author

@mikehardy deleting is okay, but would make life to "grant the tools are all known to work" harder.

However, since 2.7.4 is not pre-installed in the M1, I'd be all for go with 3.1.x... I used 2.7.4 at the time because it was more well known and widely used (ie: in CI).

@mikehardy
Copy link
Contributor

Any solution that does not allow for friction-less compatibility with future workings is a bug in my opinion
Cocoapods does not handle .ruby-version with a >=, so I'm not sure it's a tractable problem.
My test case for solution that is friction-less is:

  • rvm install (latest version, whatever it is)
  • npx react-native init TestProject

That's literally it, that should work :-)

If there was some way to have an open-ended range so we could enforce minimum versions that's great but constraining max is the issue for me

@mikehardy
Copy link
Contributor

doctor checks can do things more finely I think, by the way - we could enforce a range there if we determine that there is some problem with current stable interop with cocoapods (that did happen with ruby-3.0.0 initially, it took a little while for cocoapods to work well with it)

@barbieri
Copy link
Contributor Author

with each solution comes a problem, some people use rvm, others asdf, rbenv... Other than that, ok.

The idea was to help unify, but we can just document/recommend and let people execute the installations. I'd still provide a .bundle/config as it would remove the need to sudo during the installation, keeping everything local (similar to node_modules/)

@mikehardy
Copy link
Contributor

with each solution comes a problem, some people use rvm, others asdf, rbenv... Other than that, ok.

Exactly - for that reason the find-node.sh script was also retired, something intended to help caused more trouble than it's worth. We just need to assume node is installed and callable and move on, so find-node went away

Similarly, something that restricts ruby is causing friction. I'm agnostic on the .bundle/config, my use case is as stated, so if it causes no friction with rvm installing 3.1.1, works for me... but in general I think it's okay to focus on doctor, let it verify compatible ruby / cocoapods install and prompt people to install them if not

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Shared with Meta Applied via automation to indicate that an Issue or Pull Request has been shared with the team.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Xcode 12.5 troubleshooting guide (RN 0.61/0.62/0.63/0.64)
5 participants