Refactor Podfile DSL

[fabiopelosin]

This is a living comment

TODO

• Workspace blocks
• Project blocks
• Target blocks
• Build configuration blocks
• Support for the specification of dependencies at any level
• Backward compatibility adjustments
• Deprecation notices
• Tests
• Support abstract targets
• Support inheritance
• Support headers only inheritance
• JSON serialization
• Accept paths as the argument for the workspace and the xcodeproj.
• Support build configurations at the project and at the workspace level.

Questions

Related issues

Discussion

• Podfile DSL is backwards CocoaPods#1922 Podfile DSL is backwards
• Deprecate implicit target definition of Podfile CocoaPods#1243 Deprecate implicit target definition of Podfile
• Clean up Podfile syntax CocoaPods#840 Clean up Podfile syntax

Blocked issues

• Add support for the specification of dependencies by configuration #52 Add support for the specification of dependencies by configuration

[fabiopelosin]

The current direction is to support the following syntax:

workspace "MyWorkspace" do
  project "MyProject" do
    target "Tests" do
      build_configuration "Development" do
        pod 'Specta', '~> 0.2.1'
      end
    end
  end
end

[orta]

[fabiopelosin]

Q1 Currently the workspace attribute accepts a name, should we switch to only the name?

Which one of the following syntaxes do we want to support?

A1.1

project "MyProject" do
end

No support for paths, name only.

A1.2

project "path/to/MyProject" do
end

Support for paths, no extension.

A1.3

project "path/to/MyProject.xcodeproj" do
end

Support for paths with extension.

[fabiopelosin]

Q2 Is there any interest in supporting build configurations at the project and at the workspace level?

The question is wether the following makes any sense?

workspace "MyWorkspace" do
  project "MyProject" do
    build_configuration "Debug" do
      pod 'Tweaks'
    end

    target 'Name' do
    end

    target 'Name2' do
    end
  end
end

In this scenario all the targets of the projects will have Tweaks enabled in the Debug build configuration. Same logic for the workspace.

[fabiopelosin]

@orta Thanks man, I need it!

[alloy]

Q1 Currently the workspace attribute accepts a name, should we switch to only the name?

Is there a big problem with supporting all those options? (Although maybe A3 could be dropped.)

Q2 Is there any interest in supporting build configurations at the project and at the workspace level?

Yes.

[fabiopelosin]

Q1 Currently the workspace attribute accepts a name, should we switch to only the name?

Is there a big problem with supporting all those options? (Although maybe A3 could be dropped.)

No, just double checking that it made sense.

[fabiopelosin]

@kylef is using clever solution to define common dependencies among targets: https://github.com/kylef/KFData/blob/master/Tests/Podfile#L5-9. I'm hence wondering if a proper DSL attribute is desirable.

Regarding the inheritance of the headers only (useful for bundled targets: i.e. tests), a dedicated attribute is definitely needed.

[AliSoftware]

Regarding the new syntax, are we all in phase with the following meaning of the different scopes?

workspace "MyWorkspace" do
  pod 'A' # Will be added in all the targets of all the projects in the workspace
          # (except any "Pods" project) ?! (unlikely to be useful, right?)
  project "MyProject" do
    pod 'B' # Will be added to all the targets of this project (and all their build configuration)
    build_configuration "Debug" do
      pod 'C' # Will be added to all the targets of this project, but only in their "Debug" configuration
    end

    target 'Name' do
       pod 'D' # Will only be added to the target "Name" but in all the build configs
       build_configuration "Debug" do
         pod 'E' # Will only be added to target "Name" and only in its "Debug" configuration
       end
    end
  end
end

The biggest question is for the case of pod 'A':

1. If we want to be consistant throughout the DSL, every pod declared in a higher level would mean that it applies to all the sublevels (declared at the "project" level ➡️ will apply to all targets of this project, …)
2. But if we still want the newcomers to CP to be able to simply write nothing more that the single pod 'A' line in their Podfile (and no workspace/project/target attribute) and expect it to work, especially only add the pod in the app target and not in the test target, this solution won't work.

I personally prefer solution 1, namely force even newcomers to CP to explicitly declare on which target the pods must be added, and totally dropping the "implicit target" rule. It seems we agree on this (according to #1243) but I just wanted things to be clear on this new syntax and the fact that we totally drop the implicit thingy.

[AliSoftware]

Regarding inheritance, the question is still open:

The solution of @kylef is clever, but a lot of CP users don't know a thing about Ruby.

• Either we explicitly tell users in the documentation that defining a ruby function to group common pods is the official solution, giving them some examples so that even non-ruby users understand how to do it. This has the advantage of giving much more flexibility
• Or we add an explicit attribute in the DSL that does the same, just in order to facilitate the understanding of the Podfile for non-ruby users. The drawback with this solution is that it will probably lead to add much more in the DSL than we could initially expect: we will start to wonder if we need to introduce the concept of abstract_target, (and why not abstract configurations and abstract projects too?), what are the consequences in complex cases like the fact that targets would also already inherit pods declared at the project level, etc… I believe it may lead to a lot of confusion, mixing all those inheritance mechanisms possibilities.

---

💡 Maybe, instead of relying on inheritance (and introducing the abstract targets concept and all), a solution could be to be able to declare an block for an array of targets instead of just one?

For example, imagine we have a project with 2 app targets (say one "Lite" version and one "Pro" version of our app) and a 3rd target being the Unit Tests targets. Then we could have:

target "MyApp-Pro" do
  # pods that are ONLY used for MyApp-ProVersion
end
targets ["MyApp-Lite", "MyApp-Pro", "MyAppTests"] do
  # pods that are common to MyApp-Lite, MyApp-Pro AND MyAppTests
end
targets ["MyApp-Lite", "MyApp-Pro"] do
  # pods that are common to MyApp-Lite AND MyApp-Pro but NOT MyAppTests
end
target "MyAppTests" do
  # pods that are ONLY used by "MyAppTests"
end

Of course, same idea applies to configuration(s) and other similar attributes in the new DSL.

---

If we add some read-only attributes to the Project objects, like all_app_targets, all_tests_targets, all_targets, etc, we can even imagine sthg as flexible as this:

project "MyProject" do |proj|
  targets proj.all_app_targets do
    # pods that are common to MyApp-Lite AND MyApp-Pro but NOT MyAppTests
  end
  target proj.all_tests_targets do
    # pods that are ONLY used by "MyAppTests"
  end
end

[fabiopelosin]

I want to highlight a point as a consequence of the discussion generated by CocoaPods/CocoaPods#840 (comment).

If the workspace and the project can be implicit. The following means link every target of the user project with this dependency:

pod 'ARAnalytics'

This is something which I really like, but is a different behaviour which could break existing setups (if the project had two targets, the second one would not have linked with the previous ones).

I also agree with the point of @AliSoftware (CocoaPods/CocoaPods#840 (comment)) that the need of DSL attribute is mooted by the possibility of specifying dependencies at the project and at the workspace level.

[fabiopelosin]

Re @AliSoftware (btw thanks for the feedback! Very appreciated)

---

I don't like the solution of specifying an array of targets, and I think that it could be confusing and suboptimal in the long run.
I thought find very intriguing the idea of differentiating between target and test_target with the later meaning (just import the headers of the inherited deps but don't link agains them – maybe a more appropriate name would be bundled_target).

/c @alloy

[AliSoftware]

Actually the idea of the array of targets was inspired by the dependency declaration of the Rakefile format (task 'Name' => ['Dep1','Dep2'] do), and I thought, instead of using it for dependencies, maybe using it to directly declare everything.

The typical example that I have in mind is some project of mine that is declined in multiple app targets (a white-labelled product with 5 or 6 variants, one per customer, for example) and one test target. Of course I will have common pods which must be added to all my app targets, but not in the test target. And I want to avoid repeating all the pods in each of those 5 or 6 app targets.

[EDIT] To add a bit more complexity (in the sake of trying to cover all possible use cases however complex they can become), let's say that apps 3 and 5 are for premium clients and thus need one additional pod to add some specific functionality to those app variants. So all app targets are not strictly the same… and let's see how we can cover that case too

For this kind of case — which is fairly common (when you have mulitple app targets, you generally link most of the pods with all of those targets), here are the solutions I imagine

Solution 1 : allow target(s) to accept an array

targets p.all_app_targets do
   pod 'A'
   pod 'B'
end
targets %w(app3 app5) do
   pod 'ComponentForProUsersOnly'
end

Of course the target 'X' do using a simple string would still be valid, so the simple syntax is still valid.
💡 Also note that in this solution (and the followings) I assume that targets can be addressed multiple times in the Podfile (in the above example app3 is in both arrays passed to both targets attribute), which should be accepted and not be considered an error in the DSL/syntax, so that we can still be flexible in that regard.

Solution 2: Use native ruby code to loop thru the targets

#p.all_app_targets.each do |name|
%w(app1 app2 app3 app4 app5).each do |name|
   target name do
      pod 'A'
      pod 'B'
   end
end
%w(app3 app5).each do |pro|
   target pro do
      pod 'ComponentForProUsersOnly'
   end
end

Or course if you don't agree to add the all_app_targets function to the Project object, we could still use a manually written array %w(app1 app2 app3 app4 app5) instead, but this means that the user will have to edit its Podfile each time s/he adds a target (new variation for his/her white-labelled product) to the project

➡️ Drawback: non-ruby users will never think of this solution (but we can hint them to it in the documentation), and it is not really a DSL if we have to throw some ruby and mix the CocoaPods-DSLanguage and the ruby Language…

Solution 3: Use a ruby function to "copy/paste" common pods

def common_pods
   pod 'A'
   pod 'B'
end
def proapp_pods
   pod 'ComponentForProUsersOnly'
end

target 'app1' do
   common_pods
end
target 'app2' do
   common_pods
end
target 'app3' do
   common_pods
   proapps_pods
end
target 'app4' do
   common_pods
end
target 'app5' do
   common_pods
   proapps_pods
end

➡️ This still needs us to repeat the target XXX do ... end block for each app target, which is a pain and way too much verbose IMHO. Still needs some native ruby, but less complex to understand for non-ruby users, and less perturbing the DSL (still feels like a simple language like everything else in the Podfile syntax, less complex than the Array#each and the %w() which may seem obscure for strangers to ruby)

Solution 4: Introduce the concept of abstract targets

This is actually the same as Solution 3 but with a dedicated abstract_target keyword to help define the function and inherit_from keyword to apply it. Still too much verbose when we have too much targets.

Solution 5: Use inspiration from Rakefile

Even if we still have to add the abstract_target concept, the (multiple-)inheritance (or mixin concept) could simply be a parameter of the target attribute:

abstract_target 'Common' do
   pod 'A'
   pod 'B'
end
abstract_target 'ProApp' do
   pod 'ComponentForProUsersOnly'
end

target 'app1' => 'Common' do
  # pods only in app1 but not in other targets
end
target 'app2' => 'Common' do
  # pods only in app2 but not in other targets
end
target 'app3' => ['Common', 'ProApp'] do
  # pods only in app3 but not in other targets
end
target 'app4' => 'Common' # it should be acceptable not to have any block when there is nothing additional to do
target 'app5' => ['Common', 'ProApp'] # likewise, no block_given? when nothing to do

➡️ Still very verbose when we have a lot of targets.
➡️ We need to introduce the concept of abstract_target (which I'm not very fond of; maybe pod_group would be a better-fitting name?)

Other

Any ideas of alternate solutions for this typical usage example?

[AliSoftware]

I thought find very intriguing the idea of differentiating between target and test_target with the later meaning (just import the headers of the inherited deps but don't link agains them – maybe a more appropriate name would be bundled_target).

Have to admit I didn't understand your point/idea here?

[fabiopelosin]

Premise: I'm not sure of how typical this case could be with up to 6 targets sharing the same reps (I think that usually is 2 and rarely 3)

My comment was about this proposal:

project 'MyProject' do
   pod 'A'
   pod 'B'

  test_target 'Tests' do
    # This target is equivalent to the current `:exclusive => true`.
    # In other words has only the headers of pod A and pod B visible
    # but it doesn't link against them.
  end
end

In the above all the targets app1-5 will link against the pods A & B.

---

Note that your proposal p.all_app_targets requires some form of logic to discern from app targets to test targets, and the only way I can think of is using a dedicated attribute like test_targets

[AliSoftware]

Agreed about your solution to use pod 'XXX'at the root level and use :exclusive => true for the test target. I always forget about this "exclusive" attribute.

But let's say then that half of the pods are common to all 5 targets and the other half only common to some of them… what then?

---

Note: I edited my previous comment to throw some more complexity in my example use case ("Pro Apps" for 2 of the 5 targets), to add to the debate about how the new DSL could address such case.

I agree that this may seem a complex and far-fetched example, but the aim here is to see if the future DSL can handle such complex examples, and to experience/stress-test this new DSL 😉

[AliSoftware]

Note that your proposal p.all_app_targets requires some form of logic to discern from app targets to test targets, and the only way I can think of is using a dedicated attribute like test_targets

Xcode can do the distinction, so I'm sure we could find a way for the xcodeproj gem to find out too 😉 … For example in the project.pbxproj, each PBXNativeTarget has a productType attribute which is one of com.apple.product-type.application, com.apple.product-type.bundle.unit-test, com.apple.product-type.library.static, etc. The idea was to create as many read-only attributes as there are product types, to easily address in the Podfile all targets of a given product type at once.

[AliSoftware]

What bother me with the :exclusive => true / test_target approach is that this is a "negation"/"blacklist"/"exception" approach, instead of a whitelist approach.
We say "all the targets have those pods" then only later "oh wait, no, except this one, and this one"… it feels like we make rules with exceptions instead of explicit rule declaration (and for consistency and clarity I personally prefer explicitly declaring which pod goes in which explicit (group of) targets, instead of the exceptions/blacklist approach — but maybe I'm the only one)

[fabiopelosin]

I see your point, however there another consideration to take into account... test_target is not about blacklisting, but about defining which targets should only have the headers accessible (I generally want the headers accessible in a test target, then the fact that I use and import them is another issue).

[AliSoftware]

Can you elaborate why you could possibly need headers to pods you don't link against? Why would you need to access, from your test target, to header of pods that are only in the app target?

If the pod is not linked against the target, you won't be able to call it anyway, so what is the point of having the headers/API of the pod there? (Except if you tell Xcode to use your app to host your test bundle ("General" tab of the test target > dropdown menu "Target" in the right pane > choose "YourApp" instead of "None") and thus the libraries will be linked against the app thus code could be called from the tests, but in case the app is the host for the test target, Xcode will build the app as an implicit dependency first and the headers will be in the Products Directory anyway when building the test target then… so we would be able to import them from the tests, right?)

Or am I missing something?

[AliSoftware]

By the way, your last comments imply that future test_target attribute would prevent the target to inherit pods declared in higher levels (behavior like the current :exclusive => true), but that's not always true (in fact in most of my projects that's not the case).

For example I have many projects where a lot of common pods are used both in the app and in my unit tests. For example if I use AFNetworking to perform network requests both in my app and in my unit tests that test my webservice API. Or when I use MagicalRecord to perform CoreData operations in my app, but also use it in my tests to validate that my code/framework that is filling my CoreData base (from the WebService JSON responses for example) does it properly and ends up with the expected NSManagedObjects…

For all those cases I don't want my test target to be "exclusive", on the contrary I want it to include the same pods (and possibly more) than my app target, so I would rely on the inheritance mechanism of the new DSL, declaring my pods at the higher level (project level) in my Podfile. But according to what you say regarding your suggested test_target keyword, this won't work like this and there won't be inheritance for this particular case, leading to confusion.

Sure if I really want my test target to inherit pods from the higher level I would be able to use target 'MyAppTests' instead of test_target 'MyAppTests', but:

• when users see that the DSL have an attribute named test_target they will assume given its name that it must be used each time you use a test target (namely they will assume that it is expected to be used and that target 'MyAppTests' is invalid/not expected)
• the name is not explicit enough to let users know that this implies that pods of higher levels won't be inherited

---

I believe that the main problem here is the vocabulary: the term chosen qualifies what type of target it is (test target vs. app target) but the effect it has is on how the target handles inheritance (inherit parent scope or not) which is not semantically related.

Frankly I really rather prefer keeping the :exclusive => true keyword existing in the current DSL which is much more adapted and much more clear about the meaning and consequence on the behavior. We can still have non-test targets which could need to be exclusive, and test targets that need to be non-exclusive, which are both valid and plausible usages.

---

PS: here are some 🍻 to help you go thru all that reading I give to you… sorry for my very long messages and the headaches I may have given you 😄

[fabiopelosin]

If the pod is not linked against the target, you won't be able to call it anyway, so what is the point of having the headers/API of the pod there? (Except if you tell Xcode to use your app to host your test bundle ("General" tab of the test target > dropdown menu "Target" in the right pane > choose "YourApp" instead of "None")

Which is the default behaviour of Xcode and not an exceptional case.

... and thus the libraries will be linked against the app thus code could be called from the tests, but in case the app is the host for the test target, Xcode will build the app as an implicit dependency first

In this case thus we should avoid double linking. Moreover access to the headers of the dependencies might be used for stubbing or setting up the test environment in other forms.

...and the headers will be in the Products Directory anyway when building the test target then… so we would be able to import them from the tests, right?)

I'm not sure what you are referring to, but my understanding is that the headers search paths must be set in the xcconfig of the test target, otherwise the build will fail.

[fabiopelosin]

We should support test targets hosted in another target and not. I agree that the nomenclature test_target is weak (I hinted about it in a previous comment... see bundled_target), however I think that we should find a new term that will supersede exclusive => true. For two reasons:

• It is an option and we are moving away from them.
• It is pretty misleading as people use in hosted test targets only because CocoaPods makes all the headers visible by default to all the integrated targets (and we can't fix this behaviour until we don't fix the DSL).

PS: here are some 🍻 to help you go thru all that reading I give to you… sorry for my very long messages and the headaches I may have given you 😄

🍻 I appreciate some good feedback!

[AliSoftware]

I'm not sure what you are referring to, but my understanding is that the headers search paths must be set in the xcconfig of the test target, otherwise the build will fail.

Actually I just realized that it seams that CocoaPods does not correctly export the libPods* headers, using the misleading-named "Copy Headers" build phase instead of the "Copy Files" build phase.

All the details about correct Static Libraries configuration are explained here in this Apple TechNote. As you can see, Apple discourage the usage of the "Copy Headers" phase and suggest using "Copy Files" phase with a subpath of include/${PRODUCT_NAME}.

As a consequence of this, every library that copy/exports its headers this way make those headers available to other dependent targets, as the headers are copied in the "Products Directory" and are then automatically accessible using #import <ProductName/HeaderName>. The ${PRODUCTSDIR}/include/** directory seems to be implicitly added to the Headers Search Path by Xcode itself.

[fabiopelosin]

Whoa I wasn't aware of that... Thanks man! This works very well the only drawback is that you need to specify the import like the following:

#import <Pods-SAMPLE-AFNetworking/AFNetworking.h>

I have created a sample proj on https://github.com/irrationalfab/CocoaPodsCore81

@alloy what is your take?

[AliSoftware]

Cool glad I taught you something interesting :) Actually the name to use in the #import is <Pods-SAMPLE-AFNetworking/xxx.h> of course because we used include/${PRODUCT_NAME} as the destination folder, but I suppose that if we explicitly use include/AFNetworking we could then do #import <AFNetworking/AFNetworking.h> and fall back to the actual usage we always used, and have a clean prefix then, right?

My dream would be that when Apple & LLVM finally officialize their support for @import for third-party headers too, we can easily migrate to @import AFNetworking or @import AFNetworking.AFHTTPRequestOperation (see here for more info — but it starts to be out of scope of the DSL issue)

[fabiopelosin]

Nice! I'm of the opinion that we should switch to this behaviour... @alloy I would like your confirmation because I recall that we experimented with the Copy Files phase and if I recall correctly we abandoned it, but I can't remember why.

[alloy]

@irrationalfab I’m confused. Last time I checked we don’t actually do anything with headers, we just set the search paths to Pods/Headers/NAME ?

@AliSoftware / @irrationalfab In the past we did copy and I experiment with both types of phases. At that time we concluded ‘copy headers’ is broken and we switched to ‘copy files’.

[fabiopelosin]

@alloy We create the symlinks and set the header search paths. However with @AliSoftware approach we could skip the creation of the symlinks and the set of the search paths in the xcconfig. However I recall experimenting as well and bing bitten by them, but maybe was an error on my side. My conclusion is that we should experiment again if we don't recall the reason why they were not working... and if there is one document it.

[fabiopelosin]

@AliSoftware I toyed around with modules as soon as they where announced however I think that we should not rely on them as they are not officially supported and provide limited benefit respect the risk. However if things change I think that we should switch to them asap.

[alloy]

However with @AliSoftware approach we could skip the creation of the symlinks and the set of the search paths in the xcconfig.

@irrationalfab The ticket is too noisy for me to focus on just that. Can you please point me to where this specifically is discussed?

However I recall experimenting as well and bing bitten by them, but maybe was an error on my side. My conclusion is that we should experiment again if we don't recall the reason why they were not working...

No it was definitely not an error on our side, somebody specifically asked an Apple engineer and I documented it in the Xcodeproj API that was used to add source files (and which would add headers to the copy phase).

Having said that, we should most definitely re-check such issues after each big Xcode release :)

[fabiopelosin]

@alloy I recall some similar discussion... the @AliSoftware suggestion is detailed here #81 (comment)

[AliSoftware]

@AliSoftware I toyed around with modules as soon as they where announced however I think that we should not rely on them as they are not officially supported and provide limited benefit respect the risk.

I totally agree and was not suggesting to migrate to them — as they are not ready to be used yet for 3rdP libs — but just pointing out that one of the numerous advantages of following Apple's recommandation of using the "Copy Files" phase instead of "Copy Headers" is that it will also be compatible with the @import/modules syntax when they will be officially supported in the (far?) future.

Being able to use the same syntax (whether it is #import <Framework/Header.h> or @import Framework) is also nice as it abstracts the fact that the pods are used the same way as 1st-party frameworks. The user can transparently consider that AFNetworking, MagicalRecord, CoreGraphics and UIKit are all frameworks and are to be imported with the same syntax, regardless of whether they are Apple 3rdP frameworks, so I find it nice and convenient!

[alloy]

@irrationalfab I still don’t see it. We don’t use any of the Xcode ‘copy’ phases for the headers, because we don’t copy them at all. Instead we symlink them during the CP install process. So it’s still completely unclear to me what issue that we have this would solve, which means it might not be an efficient thing to spend time on.

[AliSoftware]

@alloy Here is what we are talking about:

[alloy]

@AliSoftware Thanks!

So we are using the copy phase after all. I think we should simply keep doing what we do atm (set the header search path to Pods/Headers) and remove those copy phases from the targets. Unless I’m overlooking some reason why it’s better to use the copy phases?

[fabiopelosin]

The copy phase is not needed and can be removed if needed iirc (is there just to properly setup the target). The benefit of @AliSoftware approach is that we would delegate to Xcode the responsibility of making headers visible... in detail the advantages are:

• proper visibility of headers which would work with hosted targets (currently CP makes all headers visible to all targets for this purpose and should be fixed)
  • no need of a DSL syntax to specify that a target wants the headers visible of another one.
  • no need to fix headers visibility
• less code for us
• less pollution in the Pods dir

[fatuhoku]

+1 for abstract targets. It should do more than the def ... approach (approach 3).

[segiddins]

Closing due to extreme age.