[BUG] Missing support for handling productRef file references

[davidohyer]

Describe the bug
When using Swift Packages in a project, running this tool crashes due to the library being unsure of how to handle productRef file references. Below is an example line from a .pbxproj file that contains Swift Packages:

58AFD9DF24D2077900331519 /* Core in Frameworks */ = {isa = PBXBuildFile; productRef = 58AFD9DE24D2077900331519 /* Core */; };

And here is the trace error:

Traceback (most recent call last):
  File "modpbx.py", line 57, in <module>
    resultClasses = project.remove_group_by_id(classesGroup.get_id(), True)
  File "/usr/local/lib/python3.7/site-packages/pbxproj-2.9.0-py3.7.egg/pbxproj/pbxextensions/ProjectGroups.py", line 65, in remove_group_by_id
  File "/usr/local/lib/python3.7/site-packages/pbxproj-2.9.0-py3.7.egg/pbxproj/pbxextensions/ProjectGroups.py", line 70, in remove_group_by_id
  File "/usr/local/lib/python3.7/site-packages/pbxproj-2.9.0-py3.7.egg/pbxproj/pbxextensions/ProjectFiles.py", line 317, in remove_file_by_id
AttributeError: 'PBXBuildFile' object has no attribute 'fileRef'

System information

1. pbxproj version used: latest code from master
2. python version used: 3.7.6
3. Xcode version used: 11.5

To Reproduce
Steps to reproduce the behavior:

1. Add a Swift Package to your project
2. Add the framework to the targets frameworks area
3. Run the following script:

#! /usr/bin/env python3
import os
from pbxproj import XcodeProject
from pbxproj.pbxextensions.ProjectFiles import *

from pbxproj.pbxextensions.ProjectFiles import FileOptions
ProjectFiles._FILE_TYPES[u'.icalls'] = (u'library.icalls', u'PBXResourcesBuildPhase')
del ProjectFiles._FILE_TYPES['.h'] # to prevent header files from being attached to a

project = XcodeProject.load('./App/App.xcodeproj/project.pbxproj')

rootGroup = project._get_parent_group( None )

unityGroup = None
unityGroups = project.get_groups_by_name('Unity', rootGroup)

if unityGroups.__len__() > 0:
	unityGroup = unityGroups[0]

classesGroups = project.get_groups_by_name('Classes', unityGroup)
classesGroup = None

if classesGroups.__len__() > 0:
	classesGroup = classesGroups[0]

librariesGroups = project.get_groups_by_name('Libraries', unityGroup)
librariesGroup = None

if librariesGroups.__len__() > 0:
	librariesGroup = librariesGroups[0]

if classesGroup is not None:
	resultClasses = project.remove_group_by_id(classesGroup.get_id(), True)

Expected behavior
Remove the specified groups.

[github-actions]

This issue has become stale, the required information has not been provided and it is been marked for closure in the next 5 days

[kronenthaler]

Thanks @davidohyer for this report. Could you add a small sample project to ease the debugging?

[github-actions]

This issue has become stale, the required information has not been provided and it is been marked for closure in the next 10 days

[github-actions]

This issue has become stale, the required information has not been provided and it is been marked for closure in the next 10 days

[kowaalczyk]

I ran into the same issue when trying to add 3rd party frameworks to a project.

It seems that build files for dependencies added via Swift Package Manager have a productRef attribute instead of fileRef attribute:

These attributes are defined in XCSwiftPackageProductDependency section:

I'm currently trying to programatically add compiled libtorch c++ files as framework dependencies, following this tutorial, so this is hardly a minimal example but to create one:

1. Create a new project in Xcode (ios app)
2. Select the project in left sidebar, select application target, go to "Frameworks, Libraries and Embedded Content"
3. Click "+" -> "Add other" -> "Add package dependency" -> paste https://github.com/soto-project/soto (or any other repo using Swift Package Manager)

This project will have the XCSwiftPackageProductDependency section and productRef attributes for SotoS3 package dependency, which make the following code fail:

file = Path("my-custom-libtorch") / "install" / "lib" / "libtorch_cpu.a"
opts = FileOptions(
    create_build_files=True,
    ignore_unknown_type=False,
    embed_framework=True,
    code_sign_on_copy=True,
)
project.add_file(str(file), parent=frameworks, force=False, file_options=opts)

Error happens in the _filter_target_without_path function, line 181 and as far as I understand the intent of the function, it can be easily fixed by wrapping that line in:

if hasattr(build_file, "fileRef"):
    ...

By adding a new file we cannot break targets that use productRef as they don't reference any files, so this fix works correctly.

In my case, adding force=True solved the issue but more people will surely run into it as swift pm gets more popular - if you're reading this I hope it helps.

System info:

• XCode 12.4
• macOS 11.2.3
• Python 3.9.2
• pbxproj 3.2.1

[kronenthaler]

Thanks @kowaalczyk for the detailed info. This should help to investigate the issue further.

I don't think that simply wrapping the breaking call with a hasattr function will solve the underlying issue. That some buildFile no don't have the fileRef but a productRef and we should be able to deal with them or at least be able to differenciate the usages and implications of it.

Good to know that force=True help you out, that might give me extra pointers!

[imeteora]

@kronenthaler It looks like fileRef equals to get_id(). Am I right?

[JohanAlbrectsen]

Also experiencing this issue.

[krs1w]

Thanks @kowaalczyk for the detailed info. This should help to investigate the issue further.

I don't think that simply wrapping the breaking call with a hasattr function will solve the underlying issue. That some buildFile no don't have the fileRef but a productRef and we should be able to deal with them or at least be able to differenciate the usages and implications of it.

Good to know that force=True help you out, that might give me extra pointers!

You are right, fixing that one line won't solve the problem entirely.
We are experiencing the same problem when calling remove_file_by_id leading to a crash here:

mod-pbxproj/pbxproj/pbxextensions/ProjectFiles.py

Line 328 in bc75ef2

if build_file.fileRef == file_ref.get_id():

Are there any plans on implementing the support for productRef?
I guess swiftPM is fairly matured now and the number of usages in projects will only rise.

One workaround would be to remove the package(s), alter the other files and re-add the package(s) again. For this to work we would need a remove_package function which currently does not exist, you can only add packages.

Unfortunately the workaround mentioned by @kowaalczyk does not work for us, as we are running our script repeatedly to keep everything made/changed in the build chain up to date in the Xcode project. force=True leads to duplicated files (frameworks & libs in our case) on each run of the script. The workaround for this issue would then be to remove the duplicates which in turn fails with the crash mentioned above.

Anyway, thanks for the great work so far! Made our lives a lot easier;)