Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing the gradle plugin #92

Closed
bbaudry opened this issue May 26, 2021 · 38 comments
Closed

Testing the gradle plugin #92

bbaudry opened this issue May 26, 2021 · 38 comments

Comments

@bbaudry
Copy link
Member

bbaudry commented May 26, 2021

@ABHAY0O7 can you implement a test harness for the gralde plugin, using the small gradle projects that you initiated with issue #91?

@ABHAY0O7
Copy link
Contributor

@ABHAY0O7 can you implement a test harness for the gralde plugin, using the small gradle projects that you initiated with issue #91?

@bbaudry @tdurieux
Yes, I can implement this.
But, there are few things that I want to convey to you :

  • Firstly, the code that I have written until now (Gradle plugin demonstration #89 ) is at this stage: -

    1. It analyzes the Gradle project (with its sub-projects if any) and divides the dependencies into three different categories usedDirectArtifacts, usedTransitiveArtifacts, and unusedDirectArtifacts.
  • Now, to test the above-mentioned part, first, I need to implement the required (or say must) plugin and the task class like mojo in maven. (Just for testing purposes on a local system) (require at least half or 1 week time from now).

  • Next, If the results of the tests are not in favor then I have to debug it and make it do work, which I wanted.

  • After this, I will move forward to divide the dependencies from 3 to 6 categories (required goal) and testing them.

  • After successfully testing the above tasks, I will try to make the plugin highly configurable like ignoring scopes or ignoring a particular dependency.

  • After successfully completing the plugin implementation, I will give it a finishing touch with some tests and documentation and will move to my next GSoC task (multi-module support for the maven plugin).

Also, if you allow me, then I want to update my timeline here ie. I will start working on the Gradle plugin from now and only after successfully completing this task, I will discuss with you my approach to the next GSoC task.

So this is my documented plan for the plugin task (as I mentioned in my proposal).
If require any amendments, then I welcome you otherwise I can start my work.

Thank you.
Abhay

@tdurieux
Copy link
Contributor

tdurieux commented Jun 5, 2021

Hello, could you try to explain by message first? I am a busy week I am not sure that we will find a time for a meeting.

@ABHAY0O7
Copy link
Contributor

ABHAY0O7 commented Jun 6, 2021

Hello, could you try to explain by message first? I am a busy week I am not sure that we will find a time for a meeting.

Ok, no problem,
I will mention my issue in this thread by tomorrow.

@ABHAY0O7
Copy link
Contributor

ABHAY0O7 commented Jun 7, 2021

Hello @bbaudry @tdurieux
Earlier I was facing 3 problems, but now, I have solved two of them.
I am currently working on the third problem and I may solve this one too.
If I solve this one, then I will push these commits on my current open PR, and if not then I will convey it to you.(max 2 days)

Thank you

@tdurieux
Copy link
Contributor

tdurieux commented Jun 9, 2021

Perfect!

@ABHAY0O7
Copy link
Contributor

ABHAY0O7 commented Jun 9, 2021

Hello @bbaudry @tdurieux
These are the things that I have done in my recent commits on #93
What new :

  • Removed configuration manual settings (configure it directly from code)
  • Added test for the empty project.
  • Include an extra portion of Unresolved dependencies (this will work only when Gradle is unable to resolve a dependency to get its transitive artifacts.)
  • Removed debugging part from the results(not from code)

Problem:

  • Unable to automate task copy dependencies locally.

What I tried to solve this problem:

  • I write the task in the build.gradle using BufferedWriter and then build the project again. With the help of this I am able to add a task in the project that will copy the dependencies locally but when I use this task through code , then it behaves abnormally.
  • The (may)cause for this abnormality is that the build process of a project takes time at least 5 sec or depends on project size and your PC too, but the code execution that is running simultaneously is very fast, so it crosses that part of the code where I used the task command, well before the build finishes and add the required task to the project.

If you got the problem then please suggest to me some solutions regarding this.
If you not then I can only explain it in a meet via screen sharing.

NOTE: To test the plugin results again use the method that I mentioned in my PR #93

Thank you

@tdurieux
Copy link
Contributor

From what I understand copyDependenciesLocally take the classpath and copy all the jar into a specific folder. It does not seem to be something that is hard to recreate. You can already get the classpath and then you can just copy everything that is in the classpath into a specific folder.

Is there a reason why you could not do this?

@ABHAY0O7
Copy link
Contributor

Is there a reason why you could not do this?

The only reason behind this was, I didn't know that it takes classpath.
But, thanks for your valuable help, now the plugin is executing correctly.
I have implemented the proposed way of copy dependencies, I will push my recent commits with some tests in a day or two.
And then I will proceed to make it highly configurable.

Thank you

@ABHAY0O7
Copy link
Contributor

Hello, @tdurieux @bbaudry,
For the last two days, my laptop's battery got damaged, so I was unable to work on the project.
I don't even have a phone, so I was unable to inform you as my ID is only open on my laptop.
But, now everything is OK, and soon I will push my recent work on the open PR.

Thank you

@tdurieux
Copy link
Contributor

No worry, I hope that the damage was not too big. Ping me when you think that your PR is ready.

@ABHAY0O7
Copy link
Contributor

Hello @tdurieux
I have recently pushed a commit on the open PR
What new:

  • Automated task copyDependenciesLocally with the help of the existing code more specific JarUtils, also reduces the task execution time.
  • Added 3 more tests.
  • Some refactoring in GradleProjectDependencyAnalysis.

I think up to now the plugin is working fine and I think now I should move forward to make it configurable.
Waiting for your feedback...

Thank you

@ABHAY0O7
Copy link
Contributor

Hello @tdurieux @bbaudry
I need some help...
Currently, I am stuck, I am not getting a way so that I can make the plugin configurable.
I know what to do but I am not getting a way of doing it correctly.
I have searched on the internet about it, but I am unable to find anything solid.
Could you please help me in getting out of this situation by referencing some blogs or articles or anything else that can demonstrate how to add extra parameters to a task using Gradle API?

Thank you

@tdurieux
Copy link
Contributor

tdurieux commented Jun 24, 2021

Are you trying to do be able to add optional properties to the plugin, like: https://github.com/riiid/gradle-github-plugin/tree/63be285ea8de3be6d65d86462c5e1ed09ea1a1e1?

This project is relatively simple you can follow what they are doing, the properties are registered here: https://github.com/riiid/gradle-github-plugin/blob/63be285ea8de3be6d65d86462c5e1ed09ea1a1e1/src/main/groovy/co/riiid/gradle/GithubPlugin.groovy#L11 and defined here: https://github.com/riiid/gradle-github-plugin/blob/63be285ea8de3be6d65d86462c5e1ed09ea1a1e1/src/main/groovy/co/riiid/gradle/GithubExtension.groovy

Does it help you?

Side note: when I am struggling with a specific task that I expect someone already did, I look on github for projects or API usage. Generally, it helps me to solve my problem.

@ABHAY0O7
Copy link
Contributor

ABHAY0O7 commented Jun 24, 2021

Side note: when I am struggling with a specific task that I expect someone already did, I look on GitHub for projects or API usage. Generally, it helps me to solve my problem.

This side note helps me a lot.
I looked over some other projects on GitHub, watched some video tutorials on YouTube, and finally found the correct solution to my problem.
Currently, I have successfully configured the isIgnoreTest parameter.
Now I am moving to add some more parameters.
Soon, I will update my work with some new tests.

Thank you
Abhay

@ABHAY0O7
Copy link
Contributor

Hii @tdurieux @bbaudry
In the previous comment, I claimed that I have successfully configured the parameter isIgnoreTest, but that was a misconception from my side. When I was using that parameter to test the plugin, I commented out all the existing test code written for the project, and in the result, the plugin shows all the direct test dependency as UNUSED DIRECT and I thought that it is because of the parameter isIgnoreTest, but these dependencies are actually not used in our project. So, that's was a very poor misconception from me and I really apologize for that.

But, now I have tried so many different different ways to add parameters, but nothing has worked as I expected.
Also, I tried the way that followed in your reference, but I think here groovy and java is making some difference i.e. the code which is written in the reference can not be written in java.

So, please suggest to me something that I can do here.

Thank you

@tdurieux
Copy link
Contributor

I am not sure to understand what is exactly the problem:

if you setup your class depcleanGradleTask like this:

package se.kth.depclean;

import java.util.Locale;

import org.gradle.api.Action;
import org.gradle.api.DefaultTask;
import org.gradle.api.Project;
import org.gradle.api.tasks.TaskAction;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class depcleanGradleTask extends DefaultTask {

    public static final Logger log = LoggerFactory.getLogger(depcleanGradleTask.class);

    boolean isIgnoreTest = false;

    @TaskAction
    public void printDependencies() {
        System.out.println("isIgnoreTest: " + isIgnoreTest);
        log.info("Debloating starts...".toUpperCase(Locale.ROOT));

        Action<Project> defaultAction = new depcleanGradleAction();
        getProject().allprojects(defaultAction);

        log.info("Debloating ends.".toUpperCase(Locale.ROOT));
    }

}

and in depclean-gradle-plugin/src/Test/resources/all_dependencies_unused/build.gradle you have

buildscript{
  repositories {
      mavenLocal()
      repositories { flatDir name: 'libs', dirs: "../../../../build/libs" }
      dependencies{
        classpath 'se.kth.castor:depclean-gradle-plugin:1.0-SNAPSHOT'
      }
  }
}

plugins {
    id 'java'
    id 'maven-publish'
}
apply plugin: 'se.kth.castor.depclean-gradle-plugin'
debloat.isIgnoreTest = true

repositories {
    mavenCentral()
    mavenLocal()
    jcenter()
}

dependencies {
    implementation 'com.fasterxml.jackson.core:jackson-databind:2.12.2'
}

group = 'org.foo.bar'
version = '1.0.0-SNAPSHOT'
description = 'foobar'
//java.sourceCompatibility = JavaVersion.VERSION_1_8


tasks.withType(JavaCompile) {
    options.encoding = 'UTF-8'
}

and you execute the debloat plugin on depclean-gradle-plugin/src/Test/resources/all_dependencies_unused you will see that isIgnoreTest: trueis printed.

Does it help you?

@ABHAY0O7
Copy link
Contributor

Hii @tdurieux
I got the concept you explained above.
Previously I was using @Input & @Optional annotations which was creating some kind of unstability.
But now, I also found a way of adding extra configuration to the task.

  1. I created a class named DepcleanGradlePluginExtension to add extra configuration directly to the project.
package se.kth.depclean;

public class DepcleanGradlePluginExtension {

    public boolean isIgnoreTest = false;

    public boolean isIgnoreTest() {
        return isIgnoreTest;
    }

    public boolean skipDepclean = false;

    public boolean isSkipDepclean() {
        return skipDepclean;
    }

}
  1. Add it to the project's configuration.
package se.kth.depclean;

import org.gradle.api.Plugin;
import org.gradle.api.Project;
import org.jetbrains.annotations.NotNull;

public class depcleanGradlePlugin implements Plugin<Project> {

    String taskName = "debloat";

    @Override
    public void apply(@NotNull Project project) {

        project.getExtensions().create("depclean", DepcleanGradlePluginExtension.class);
        createTask(project);
    }

    public void createTask(Project project) {
        depcleanGradleTask task = project.getTasks().create(taskName, depcleanGradleTask.class);
        task.setGroup("group");
        task.setDescription("Print outcome.");
    }
}

  1. Using it in the depcleanGradleAction by creating an extension object. Just add these lines of code at the beginning of the execute() method of depcleanGradleAction.
        DepcleanGradlePluginExtension extension = project.getExtensions().getByType(DepcleanGradlePluginExtension.class);
        boolean isIgnoreTest = extension.isIgnoreTest();
        boolean skipDepClean = extension.isSkipDepclean();

        if (skipDepClean) {
            printString("Skipping DepClean plugin execution");
            return;
        }
  1. Now use it from your build.gradle directly. i.e. just include this extra section in your build.gradle
depclean {
    skipDepclean = true
}

And this is the result exactly as I expected.


> Task :debloat
Skipping DepClean plugin execution

BUILD SUCCESSFUL in 2s
1 actionable task: 1 executed

I have seen some projects on Github which use both ways.
Now, please can you suggest to me which method I should prefer?
Also, it will be helpful for me if you can also elaborate that what is the exact difference between these two methods? 😄

Thank you
Abhay

@tdurieux
Copy link
Contributor

tdurieux commented Jul 1, 2021

It looks perfectly fine to me to use the way you described.

@ABHAY0O7
Copy link
Contributor

ABHAY0O7 commented Jul 4, 2021

Hello @tdurieux @bbaudry
I have recently pushed a commit that marks the completion of 95% of the plugin deployment.
What's new

  • Added extra configuration to the plugin like ignoreTest to make it flexible.
  • Fix the logging messages.

Also since I am heading towards the end of this task, I want to end it professionally, so please suggest to me the default task name that I should use for the plugin like in maven we use this long command se.kth.castor:depclean-maven-plugin:2.0.2-SNAPSHOT:depclean, Also when you will be reviewing then I will be glad if you suggest me some changes in the plugin, classes, method or field names.

One more thing, I have not implemented some extra configuration similar to createPomDebloated or createResultJson, the reason behind this is that if we include these parameters in our configuration then the plugin will take lots of time to generate them. See a report here #79, it shows all. So, a developer will rarely like to generate a new debloated file by giving a lot of time, rather than this he will prefer manual removing of debloated dependencies which takes a little bit of time after analyzing the report. I also want your and the community's feedback regarding this.

PR #93 is completely open for review.
Please update me soon with some feedback or suggestions.
Until then I will continue with some documentation and refactoring.

Thank you
Abhay

@tdurieux
Copy link
Contributor

tdurieux commented Jul 9, 2021

creating the pom should be extremely fast, I don't see a reason why it would be slow, for the JSON, it depends of the size of the project. it can potentially take a lot of time for big projects.

@tdurieux
Copy link
Contributor

tdurieux commented Jul 10, 2021 via email

@ABHAY0O7
Copy link
Contributor

Hii @tdurieux
Sorry for not responsive for the last 4 days, I was a little bit busy in my college vivas and also was trying to configure out how can I implement the createDebloatedBuild parameter.
I searched over the internet for how to use model and any Writer object in Gradle to write a new build.gradle file, and sadly I didn't find anything that could help me.
But now I have an idea for this which is:
We know that the difference between the debloated build.gradle and the existing build.gradle file is only inside the dependencies portion i.e.

dependencies {
       // project's dependencies
}

So what we can do is that instead of creating a whole new build.gradle file, we can just create a debloated-dependencies.gradle file using BufferedWriter & Reader and if one wants to use it then (s)he can specify this file in its original build.gradle file to apply the project's dependencies.

Example: https://github.com/yoomoney-gradle-plugins/check-dependencies-plugin/blob/d2186b1244d2ffbecf19f2a45ecfe8ff50bd4601/build.gradle#L7

Also, its generation will be very fast as we will be dealing only with the dependencies section of the project.
I am not sure that whether this will gonna work, because there are some problems regarding this like,
Recently, in Gradle 7.0.0 and above there are some configurations (scopes) removed by Gradle (See here) but when we use the getConfiguration method on any artifact then it may return those removed configurations, so I have to deal with them i.e. mentioning the dependencies in its right configuration.

If I will able to conquer all the problems then this way of creating this parameter will definitely work.
Also, I want to know your thoughts about this too.
If there are any mistakes or any other way of doing this in a more perfect way then please guide me through it.

Thank you
Abhay

@ABHAY0O7
Copy link
Contributor

Hii @tdurieux @bbaudry,
I have now successfully implemented the createBuildDebloated parameter and wrote a proper functional test that ensures its proper working.
I used BufferWriter & PrintWriter to write debloated-dependencies.gradle file with some new programming logic, which if anyone wants can directly use from his build.gradle file as I mentioned earlier in the previous comment.
Now, you can review it and if there are any problems then let me know.
Also after your review, if all goes well, then I want to discuss the remaining parameter and tasks.

Thank you

@ABHAY0O7
Copy link
Contributor

Hello, @tdurieux @bbaudry,
As I am a little bit behind as compare to my timeline, I thought that I should continue.
I want to say that, for the last parameter to be implemented, the problem that I am facing is to get the output of the task
gradle dependency to a file, but for now there seems to be no method to do that. But, I am trying to find a solution for that and I will also request to you, please help me with this.
But if I didn't found any way soon, then I will implement that parameter later(Postpone), and for now, I have to move to my second task.

Waiting for your fast response...
Best
Abhay

@tdurieux
Copy link
Contributor

tdurieux commented Jul 19, 2021

What are you trying to achieve with gradle dependency since you already get the list of dependencies programmatically?

@ABHAY0O7
Copy link
Contributor

What are you trying to achieve with gradle dependency since you already get the list of dependencies programmatically?

I want to generate the complete dependency tree of the project which can be done using the cmd gradle dependencies and also want the result of this task to a file because later It will be used in getting nodes from the tree.

@ABHAY0O7
Copy link
Contributor

Hello @tdurieux @bbaudry
I apologize for this delay in the completion of this task, this week was quite busy for me. I was learning about Github Actions, was giving my end semester exams, and was also finding a correct way to add the test to Github Actions, but since the project is now a multi-build project so, it was quite hard to set up continuous integration to a module instead of project.

But finally, with the help of one of my friends, we found a way to add module tests to CI.
But now, when I set up the stable CI, Github actions are failing the tests. 😞
This is because there was a bug in my already written tests, the bug is quite difficult to explain in the comments, but I am finding a way to debug them. So, it will take around 1 more day.

Also, I want to update you that my final semester exams are over (22 - 25) and I am getting a 10 day leave from college, so I will be providing as maximum as possible hours to the project. 😄 👍🏽

Best Abhay

@ABHAY0O7
Copy link
Contributor

Hii @tdurieux @bbaudry
I have successfully debugged the test and they are working fine now.
I have also included the Gradle build in the CI, but I excluded the test for now as they are failing on Github actions but they are running successfully on the local system. This is because I implemented functional tests using Gradle runner, which is not supported by Github actions, see here. This is a bug that is still open to be debugged, so we have to wait to add tests to CI until Gradle doesn't get rid of this bug.

You can review the modified tests and workflow files.
Thank you

@ABHAY0O7
Copy link
Contributor

ABHAY0O7 commented Aug 6, 2021

Hii @bbaudry @tdurieux
It's been a long time since I made the last PR, but the debugging was very difficult for me this time.
As you advised that these files can be very useful sometimes in data analysis, so finally, I found a way to generate a proper JSON and CSV file. 😄
Also, the time in producing these files is not that much which I thought, it is approx one minute. 🚀
I also added a test for the JSON parameter, now you can review the PR (#100) and can merge all the PRs according to their numbers.
Also, while merging there may be some merge conflicts, so just ping me if there are any.

Until then I will be carrying on with some more debugging and testing. 😃
Best,
Abhay

@Amishakumari544
Copy link

Hello everyone , I am Amisha pursuing Btech in computer Science I am looking to work on this project can anyone tell me how to start ? @ABHAY0O7 can you please guide me?

@ABHAY0O7
Copy link
Contributor

ABHAY0O7 commented Aug 11, 2021

Hello @tdurieux @bbaudry
I hope you are having a nice time.
As my first task of creating depclean-gradle-plugin is already completed, so I moved to the second one, and I had implemented support for multi-module-analysis.
It is not so perfect but still, it will prevent the failure of building the project.
What I have implemented :

  1. Suppose this maven-based java project to be the targeted one (link), it has three modules in which module-3 depends on module-2.
  2. Now the thing is that the dependencies commons-codec:commons-codec:1.15 & com.jcabi:jcabi-manifests:1.1 are declared in module-2 but are not used in module-2, so depclean (old version) report these dependencies bloated.
  3. But in module module-3 which depends on module-2 uses its dependencies commons-codec:commons-codec:1.15 & com.jcabi:jcabi-manifests:1.1 and the depclean (old version) will report these two dependencies as debloated.
  4. See this whole report generated by depclean for this project.
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] abhayPlugin                                                        [pom]
[INFO] abhay-module-1                                                     [jar]
[INFO] abhay-module-2                                                     [jar]
[INFO] abhay-module-3                                                     [jar]
[INFO] 
[INFO] -----------------------< org.abhay:abhayPlugin >------------------------
[INFO] Building abhayPlugin 1.0-SNAPSHOT                                  [1/4]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] --- depclean-maven-plugin:2.0.2-SNAPSHOT:depclean (default-cli) @ abhayPlugin ---
-------------------------------------------------------
[INFO] Starting DepClean dependency analysis
[INFO] Skipping because packaging type pom.
[INFO] 
[INFO] ----------------------< org.abhay:abhay-module-1 >----------------------
[INFO] Building abhay-module-1 1.0-SNAPSHOT                               [2/4]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- depclean-maven-plugin:2.0.2-SNAPSHOT:depclean (default-cli) @ abhay-module-1 ---
-------------------------------------------------------
[INFO] Starting DepClean dependency analysis
[WARNING] Dependencies were not copied locally
-------------------------------------------------------
 D E P C L E A N   A N A L Y S I S   R E S U L T S
-------------------------------------------------------
USED DIRECT DEPENDENCIES [0]: 
USED INHERITED DEPENDENCIES [0]: 
USED TRANSITIVE DEPENDENCIES [0]: 
POTENTIALLY UNUSED DIRECT DEPENDENCIES [0]: 
POTENTIALLY UNUSED INHERITED DEPENDENCIES [0]: 
POTENTIALLY UNUSED TRANSITIVE DEPENDENCIES [0]: 
[INFO] 
[INFO] ----------------------< org.abhay:abhay-module-2 >----------------------
[INFO] Building abhay-module-2 1.0-SNAPSHOT                               [3/4]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- depclean-maven-plugin:2.0.2-SNAPSHOT:depclean (default-cli) @ abhay-module-2 ---
-------------------------------------------------------
[INFO] Starting DepClean dependency analysis
[WARNING] Dependencies were not copied locally
-------------------------------------------------------
 D E P C L E A N   A N A L Y S I S   R E S U L T S
-------------------------------------------------------
USED DIRECT DEPENDENCIES [1]: 
	com.fasterxml.jackson.core:jackson-core:2.12.2:compile (size unknown)
USED INHERITED DEPENDENCIES [0]: 
USED TRANSITIVE DEPENDENCIES [0]: 
POTENTIALLY UNUSED DIRECT DEPENDENCIES [3]: 
	commons-codec:commons-codec:1.15:compile (size unknown)
	com.jcabi:jcabi-manifests:1.1:compile (size unknown)
	commons-io:commons-io:2.8.0:compile (size unknown)
POTENTIALLY UNUSED INHERITED DEPENDENCIES [0]: 
POTENTIALLY UNUSED TRANSITIVE DEPENDENCIES [2]: 
	org.slf4j:slf4j-api:1.7.5:compile (size unknown)
	com.jcabi:jcabi-log:0.14:compile (size unknown)
[INFO] 
[INFO] ----------------------< org.abhay:abhay-module-3 >----------------------
[INFO] Building abhay-module-3 1.0-SNAPSHOT                               [4/4]
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] Could not transfer metadata org.abhay:abhay-module-2:1.0-SNAPSHOT/maven-metadata.xml from/to example-repo (C:\Users\ABHAY SINGH/.m2/repository): Cannot access C:\Users\ABHAY SINGH/.m2/repository with type default using the available connector factories: BasicRepositoryConnectorFactory
[WARNING] Failure to transfer org.abhay:abhay-module-2:1.0-SNAPSHOT/maven-metadata.xml from C:\Users\ABHAY SINGH/.m2/repository was cached in the local repository, resolution will not be reattempted until the update interval of example-repo has elapsed or updates are forced. Original error: Could not transfer metadata org.abhay:abhay-module-2:1.0-SNAPSHOT/maven-metadata.xml from/to example-repo (C:\Users\ABHAY SINGH/.m2/repository): Cannot access C:\Users\ABHAY SINGH/.m2/repository with type default using the available connector factories: BasicRepositoryConnectorFactory
[INFO] 
[INFO] --- depclean-maven-plugin:2.0.2-SNAPSHOT:depclean (default-cli) @ abhay-module-3 ---
-------------------------------------------------------
[INFO] Starting DepClean dependency analysis
[WARNING] Dependencies were not copied locally
-------------------------------------------------------
 D E P C L E A N   A N A L Y S I S   R E S U L T S
-------------------------------------------------------
USED DIRECT DEPENDENCIES [1]: 
	org.abhay:abhay-module-2:1.0-SNAPSHOT:compile (size unknown)
USED INHERITED DEPENDENCIES [0]: 
USED TRANSITIVE DEPENDENCIES [3]: 
	commons-codec:commons-codec:1.15:compile (size unknown)
	com.fasterxml.jackson.core:jackson-core:2.12.2:compile (size unknown)
	com.jcabi:jcabi-manifests:1.1:compile (size unknown)
POTENTIALLY UNUSED DIRECT DEPENDENCIES [0]: 
POTENTIALLY UNUSED INHERITED DEPENDENCIES [0]: 
POTENTIALLY UNUSED TRANSITIVE DEPENDENCIES [3]: 
	org.slf4j:slf4j-api:1.7.5:compile (size unknown)
	com.jcabi:jcabi-log:0.14:compile (size unknown)
	commons-io:commons-io:2.8.0:compile (size unknown)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for abhayPlugin 1.0-SNAPSHOT:
[INFO] 
[INFO] abhayPlugin ........................................ SUCCESS [  0.878 s]
[INFO] abhay-module-1 ..................................... SUCCESS [  1.306 s]
[INFO] abhay-module-2 ..................................... SUCCESS [  1.285 s]
[INFO] abhay-module-3 ..................................... SUCCESS [  1.249 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.841 s
[INFO] Finished at: 2021-08-12T17:17:08+05:30
[INFO] ------------------------------------------------------------------------
  1. Now what will happen, after seeing that the dependencies commons-codec:commons-codec:1.15 & com.jcabi:jcabi-manifests:1.1 are not used in module-2, so he will remove them from it which will result in project builds failure as module-3 which requires these two dependencies, will not find these dependencies in module-2.
  2. So, I propose (implemented) an extra section for these kinds of dependencies whose debloat status depends on other modules. Look in this report, how I implemented that.
  3. Depclean report (new version)
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] abhayPlugin                                                        [pom]
[INFO] abhay-module-1                                                     [jar]
[INFO] abhay-module-2                                                     [jar]
[INFO] abhay-module-3                                                     [jar]
[INFO] 
[INFO] -----------------------< org.abhay:abhayPlugin >------------------------
[INFO] Building abhayPlugin 1.0-SNAPSHOT                                  [1/4]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] --- depclean-maven-plugin:2.0.2-SNAPSHOT:depclean (default-cli) @ abhayPlugin ---
-------------------------------------------------------
[INFO] Starting DepClean dependency analysis
[INFO] Skipping because packaging type pom.
[INFO] 
[INFO] ----------------------< org.abhay:abhay-module-1 >----------------------
[INFO] Building abhay-module-1 1.0-SNAPSHOT                               [2/4]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- depclean-maven-plugin:2.0.2-SNAPSHOT:depclean (default-cli) @ abhay-module-1 ---
-------------------------------------------------------
[INFO] Starting DepClean dependency analysis
[WARNING] Dependencies were not copied locally
-------------------------------------------------------
 D E P C L E A N   A N A L Y S I S   R E S U L T S
-------------------------------------------------------
USED DIRECT DEPENDENCIES [0]: 
USED INHERITED DEPENDENCIES [0]: 
USED TRANSITIVE DEPENDENCIES [0]: 
POTENTIALLY UNUSED DIRECT DEPENDENCIES [0]: 
POTENTIALLY UNUSED INHERITED DEPENDENCIES [0]: 
POTENTIALLY UNUSED TRANSITIVE DEPENDENCIES [0]: 
[INFO] 
[INFO] ----------------------< org.abhay:abhay-module-2 >----------------------
[INFO] Building abhay-module-2 1.0-SNAPSHOT                               [3/4]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- depclean-maven-plugin:2.0.2-SNAPSHOT:depclean (default-cli) @ abhay-module-2 ---
-------------------------------------------------------
[INFO] Starting DepClean dependency analysis
[WARNING] Dependencies were not copied locally
-------------------------------------------------------
 D E P C L E A N   A N A L Y S I S   R E S U L T S
-------------------------------------------------------
USED DIRECT DEPENDENCIES [1]: 
	com.fasterxml.jackson.core:jackson-core:2.12.2:compile (size unknown)
USED INHERITED DEPENDENCIES [0]: 
USED TRANSITIVE DEPENDENCIES [0]: 
POTENTIALLY UNUSED DIRECT DEPENDENCIES [3]: 
	commons-codec:commons-codec:1.15:compile (size unknown)
	com.jcabi:jcabi-manifests:1.1:compile (size unknown)
	commons-io:commons-io:2.8.0:compile (size unknown)
POTENTIALLY UNUSED INHERITED DEPENDENCIES [0]: 
POTENTIALLY UNUSED TRANSITIVE DEPENDENCIES [2]: 
	org.slf4j:slf4j-api:1.7.5:compile (size unknown)
	com.jcabi:jcabi-log:0.14:compile (size unknown)
[INFO] 
[INFO] ----------------------< org.abhay:abhay-module-3 >----------------------
[INFO] Building abhay-module-3 1.0-SNAPSHOT                               [4/4]
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] Could not transfer metadata org.abhay:abhay-module-2:1.0-SNAPSHOT/maven-metadata.xml from/to example-repo (C:\Users\ABHAY SINGH/.m2/repository): Cannot access C:\Users\ABHAY SINGH/.m2/repository with type default using the available connector factories: BasicRepositoryConnectorFactory
[WARNING] Failure to transfer org.abhay:abhay-module-2:1.0-SNAPSHOT/maven-metadata.xml from C:\Users\ABHAY SINGH/.m2/repository was cached in the local repository, resolution will not be reattempted until the update interval of example-repo has elapsed or updates are forced. Original error: Could not transfer metadata org.abhay:abhay-module-2:1.0-SNAPSHOT/maven-metadata.xml from/to example-repo (C:\Users\ABHAY SINGH/.m2/repository): Cannot access C:\Users\ABHAY SINGH/.m2/repository with type default using the available connector factories: BasicRepositoryConnectorFactory
[INFO] 
[INFO] --- depclean-maven-plugin:2.0.2-SNAPSHOT:depclean (default-cli) @ abhay-module-3 ---
-------------------------------------------------------
[INFO] Starting DepClean dependency analysis
[WARNING] Dependencies were not copied locally
-------------------------------------------------------
 D E P C L E A N   A N A L Y S I S   R E S U L T S
-------------------------------------------------------
USED DIRECT DEPENDENCIES [1]: 
	org.abhay:abhay-module-2:1.0-SNAPSHOT:compile (size unknown)
USED INHERITED DEPENDENCIES [0]: 
USED TRANSITIVE DEPENDENCIES [3]: 
	commons-codec:commons-codec:1.15:compile (size unknown)
	com.fasterxml.jackson.core:jackson-core:2.12.2:compile (size unknown)
	com.jcabi:jcabi-manifests:1.1:compile (size unknown)
POTENTIALLY UNUSED DIRECT DEPENDENCIES [0]: 
POTENTIALLY UNUSED INHERITED DEPENDENCIES [0]: 
POTENTIALLY UNUSED TRANSITIVE DEPENDENCIES [3]: 
	org.slf4j:slf4j-api:1.7.5:compile (size unknown)
	com.jcabi:jcabi-log:0.14:compile (size unknown)
	commons-io:commons-io:2.8.0:compile (size unknown)

-------------------------------------------------------
[INFO] DEPENDENT MODULES FOUND
Due to dependent modules, the debloated result of some dependencies from previous modules has been changed now.
The dependency-module details of such dependencies with the new results are as follows :

	1) ModuleCoordinates : org.abhay:abhay-module-2:1.0-SNAPSHOT
	   DependencyCoordinates : com.jcabi:jcabi-manifests:1.1:compile
	   OldType : unusedDirect
	   NewType : usedDirect

	2) ModuleCoordinates : org.abhay:abhay-module-2:1.0-SNAPSHOT
	   DependencyCoordinates : commons-codec:commons-codec:1.15:compile
	   OldType : unusedDirect
	   NewType : usedDirect

-------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for abhayPlugin 1.0-SNAPSHOT:
[INFO] 
[INFO] abhayPlugin ........................................ SUCCESS [  0.878 s]
[INFO] abhay-module-1 ..................................... SUCCESS [  1.306 s]
[INFO] abhay-module-2 ..................................... SUCCESS [  1.285 s]
[INFO] abhay-module-3 ..................................... SUCCESS [  1.249 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.841 s
[INFO] Finished at: 2021-08-12T17:17:08+05:30
[INFO] ------------------------------------------------------------------------
  1. Now, in the new feature, after the analysis of module-3 there is an extra section for dependencies whose debloated status has been changed. This will tell the user, not to exclude these dependencies from their respective pom.xml, which will prevent build failures.
  2. This feature does not analyze the bulk of data to generate a report (maybe a better way), but it serves the purpose which was required.

The implementation of the above feature is here #101. You can review it and provide me the feedback as soon as possible.

Thank you

@tdurieux
Copy link
Contributor

Nice idea.
Do you think it is possible to add a summary report at the end of the built of the three modules?
And this summary could also contain the information from (DEPENDENT MODULES FOUND)

@ABHAY0O7
Copy link
Contributor

Do you think it is possible to add a summary report at the end of the built of the three modules?

Sorry to say, but to my knowledge, this is not possible and actually, this was the whole problem for me, just think, if I have a way of adding the summary report at the end of the build, then instead of the summary report, I will prefer the bulk analysis of the result of all modules generated by depclean and then the problem of multi-module analysis would never arrive, but the things didn't work this way. I had already spent a lot of time researching this.

How do they work?
The mojo considers every module as an individual project, so whatever we write inside the execute() method, will be executed by every module in the order provided by the maven reactor. So, we can collect the data from every module and can manipulate it at the end of every module which I already did here #101, but not at the end of the whole project. 😢

Also, I think the only solution to it is inside the maven reactor codebase because the work performed by the reactor can only be possible by the bulk analysis of modules, not individual analysis. Unfortunately, I didn't find the codebase and logic behind the working of the maven reactor, but I am researching this, and maybe in the future I will be able to implement this summary report too. 😄

Until then I will try to add some new tests. 👨🏽‍💻

Thank you
Abhay

@ABHAY0O7
Copy link
Contributor

Hello everyone, I am Amisha pursuing Btech in Computer Science I am looking to work on this project can anyone tell me how to start? @ABHAY0O7 can you please guide me?

And if you don't mind, please answer her too.
And If you are busy, then provide me this wonderful opportunity of guiding new contributors. 😃

@ABHAY0O7
Copy link
Contributor

Hello @tdurieux @bbaudry
Since today is the last day of the GSoC coding period, I want you to review and merge all of my pending PRs so that I can include them in my final report.
Also, if possible I will be sending more PRs of some new tests or updates in README, so please try to review and merge them too before the deadline. 😅

Thank you
Abhay

@Amishakumari544
Copy link

Hello everyone, I am Amisha pursuing Btech in Computer Science I am looking to work on this project can anyone tell me how to start? @ABHAY0O7 can you please guide me?

And if you don't mind, please answer her too.
And If you are busy, then provide me this wonderful opportunity of guiding new contributors. 😃

Hello @tdurieux @bbaudry
Since today is the last day of the GSoC coding period, I want you to review and merge all of my pending PRs so that I can include them in my final report.
Also, if possible I will be sending more PRs of some new tests or updates in README, so please try to review and merge them too before the deadline. 😅

Thank you
Abhay

It would be great if you help me .

@tdurieux
Copy link
Contributor

Hello @tdurieux @bbaudry
Since today is the last day of the GSoC coding period, I want you to review and merge all of my pending PRs so that I can include them in my final report.
Also, if possible I will be sending more PRs of some new tests or updates in README, so please try to review and merge them too before the deadline. 😅

Thank you
Abhay

Sorry @ABHAY0O7, I was on holiday without my laptop. I am reviewing and merging your PRs.

@ABHAY0O7
Copy link
Contributor

ABHAY0O7 commented Aug 27, 2021

It would be great if you help me.

Hello @Amishakumari544,
Sorry for the delay,
The project is written completely in Java, and if you know it then you can start by adding java-docs, refactoring, readme update, or maybe some new tests.
Make sure to visit here : A Comprehensive Study of Bloated Dependencies in the Maven Ecosystem.
Try to understand the project first, then it will be easier to contribute.
You can refer to these links for information about Gradle & Maven plugins.
Maven-api
Gradle-api

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants