lein 2.9.2 regression: default profiles not removed from pom/jar/uberjar

[bsless]

Initial debugging steps
Before creating a report, especially around exceptions being thrown when running Leiningen, please check if the error still occurs after:

• Updating to using the latest released version of Leiningen (lein upgrade).
• Moving your ~/.lein/profiles.clj (if present) out of the way. This contains third-party dependencies and plugins that can cause problems inside Leiningen.
• Updating any old versions of plugins in your project.clj, especially if the problem is with a plugin not working. Old versions of plugins like nREPL and CIDER (as well as others) can cause problems with newer versions of Leiningen.

Describe the bug
Leiningen does not clear default profiles settings when executing tasks pom/jar/uberjar

To Reproduce
Steps to reproduce the behavior:

• in an empty dir run lein new dummy
• add to defproject: :profiles {:foo {:dependencies [[criterium "0.4.6"]]} :dev [:foo]}
• run lein pom
• see dev dependency in pom:

    <dependency>
      <groupId>criterium</groupId>
      <artifactId>criterium</artifactId>
      <version>0.4.6</version>
    </dependency>

Actual behavior
A dev dependency was transitively pulled in

Expected behavior
I expect from the documentation that everything brought in by :dev to be removed

Link to sample project
https://gist.github.com/bsless/e1f17bf8cfd1968f783c2d9b545e30ea

Environment

• Leiningen Version: happens on 2.9.2 and every version after, does not happen on 2.9.1
• Leiningen installation method: manual
• JDK Version: openjdk version "1.8.0_275"
  OpenJDK Runtime Environment (build 1.8.0_275-8u275-b01-0ubuntu1~20.04-b01)
  OpenJDK 64-Bit Server VM (build 25.275-b01, mixed mode)
• OS: Ubuntu 20.4

Additional context
This worked for every version of lein I tried before 2.9.2

[glts]

Perhaps the recent doc in 8fb39db helps, at least as a work-around? :profiles {:foo ^{:pom-scope :dev} {:dependencies [[criterium "0.4.6"]]}}

[ptaoussanis]

Hi folks,

Just confirming that I'm also running into this problem with Leiningen 2.9.5 on Java 1.8.0_275 OpenJDK 64-Bit Server VM.

Here's another minimal reproducible project in case it's helpful:

(defproject com.taoensso.tests/clojars-test "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.10.1"]]
  :profiles
  {:ring {:dependencies [[ring "1.8.2"]]}
   :dev  [:ring]})

Running lein deploy clojars with this results in the :dev dependencies being transitively included in Clojars:

Nothing in my lein profiles, no plugins.

Running on macOS Big Sur 11.1.

Thanks!

[technomancy]

@glts I did some bisecting and this was caused by your changes in #2586 ; can you take a look?

[glts]

@technomancy all right, will check

[ptaoussanis]

Much thanks to you both!! (And for all your work generally on Leiningen!) 🙏 🙏

[glts]

The change in #2586 was actually really tiny (apart from the minor refactoring), and you can bring back the old behaviour simply by commenting one form:

diff --git a/leiningen-core/src/leiningen/core/project.clj b/leiningen-core/src/leiningen/core/project.clj
index bce9ba90..7b3f55fc 100644
--- a/leiningen-core/src/leiningen/core/project.clj
+++ b/leiningen-core/src/leiningen/core/project.clj
@@ -904,7 +904,7 @@
 (defn- apply-profile-meta [default-meta profile]
   (cond-> profile
     (or (map? profile)
-        (composite-profile? profile))
+        #_(composite-profile? profile))
     (vary-meta (fn [m] (merge default-meta m)))
     (map? profile) set-dependencies-pom-scope))
 

Unfortunately the profiles code is super complicated and I don't (yet) see why it impacted pom/jar/uberjar in the way it did. I will try to find out more, not sure when.

[joelittlejohn]

Correct me if I'm wrong, but this seems like a very serious issue. It means for a couple of years now Leiningen has been silently including all dev dependencies in the pom of Clojure library releases. I guess this means an increasing number of projects on clojars.org now have dev dependencies listed in the pom and therefore these dev dependencies are being pulled into applications when the library is used.

This seems a lot more of a serious problem than #2132, which makes me wonder, should we comment out the call to composite-profile? as @glts has suggested above, and reopen #2132?

Sorry if I have misunderstood the scope of this issue. I didn't notice the problem until an eagle-eyed user told me that a new release I had made included some dev dependencies. I'm concerned about how many projects this will affect.

[technomancy]

No, you're right; this is a more serious problem than it looked. I was hoping we'd hear back with more about how this is related to the other change, but I'd much rather have the other thing broken than this. If you want to submit a patch with the change suggested above along with a test for the broken composite pom behavior that'd be fantastic; otherwise I will get to this next week or so.

[glts]

@joelittlejohn @technomancy I cannot spend time on this, so restoring the old behaviour is fine with me.

Of course, if someone is motivated and wants to take this opportunity to look into this in depth, that would be great too.

[joelittlejohn]

@glts As you say above, it's a tricky area. I think I'll try to work out if we can fix this properly. Clearly there is some additional check we should make, but not sure what that is right now. I will have a look.

[technomancy]

Took care of this and added a test so it doesn't have a regression. Will see if we can do something about the other issue in a way that doesn't break this one.

[d-t-w]

I have an issue that is derivative of this one (I think) whereby if the dev profile is itself a composite then it breaks other profiles when creating an uberjar in strange ways. I'll provide a reproducer next week if that's of interest.

Basically it boils down to the dev profile not being removed from uberjar but the behaviour slightly more odd because the composite nature of dev.

I'm guessing your fix above probably wraps that up as well.

[technomancy]

I'll provide a reproducer next week if that's of interest.

Honestly I'm more interested in knowing whether you can reproduce the problem on the latest dev version; please let me know about that. If it still happens, then a repro case would be helpful, yeah, but if you haven't tried it yet there's no reason to bother with a repro.