Add provenance metadata support #5

lakshya8066 · 2022-08-15T11:18:27Z

Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
Ensure that the pull request title represents the desired changelog entry
Please describe what you did
This PR adds support for generating Provenance v0.2 metadata for each step of the Jenkins build.
Link to relevant issues in GitHub or Jira
Link to relevant pull requests, esp. upstream and downstream changes
Ensure you have provided tests - that demonstrates feature works or fixes the issue

lakshya8066 · 2022-08-21T18:13:27Z

@SantiagoTorres This is the work so far, can you take a look? :)

adityasaky · 2022-08-29T02:51:16Z

src/main/java/io/jenkins/plugins/intotorecorder/InTotoWrapper.java

+        setProvenanceMaterials(InTotoWrapper.collectArtifacts(this.cwd));
+
+        //setting up Invocation
+        this.invocation = new BuildInvocation();


Why can't this be locally scoped?

I'm guessing it's because of setProvenanceMaterials etc but I suspect it can be cleaner.

Are you referring to the BuildInvocation object?
I think there is no need as such for it to be global, we can make it local as well.

Yep, I think I agree. The same probably goes for the metadata object as well?

Yes, I'll do that

SantiagoTorres

There are some minor nits that I'd love to address :)

SantiagoTorres · 2022-09-06T20:48:24Z

src/main/java/io/jenkins/plugins/intotorecorder/InTotoWrapper.java

+        setBuilder(initialEnvironment);
+
+        //setting up materials
+        setProvenanceMaterials(InTotoWrapper.collectArtifacts(this.cwd));


You probably want to reuse the retval on 210 so you don't make two consecutive calls to the same

SantiagoTorres · 2022-09-06T20:50:59Z

src/main/java/io/jenkins/plugins/intotorecorder/InTotoWrapper.java

+        invocation.setEnvironments(initialEnvironment);
+        try {
+            invocation.setConfigSource(initialEnvironment);
+        } catch (NoSuchAlgorithmException e) {


Not entirely sure why setConfigSource throws a NSAE. isn't this for bouncycastle/dsse signature algorithms?

Ah it's my bad, I used bouncycastle to get the sha256 hash of the git commit and that threw NSAE.
But yes, now the try-catch block is not needed anymore. Will update it...

SantiagoTorres · 2022-09-06T20:51:44Z

src/main/java/io/jenkins/plugins/intotorecorder/InTotoWrapper.java

+            return environment;
+        }
+
+        public void setConfigSource(EnvVars initialEnvironment) throws NoSuchAlgorithmException{


See comment above. Who is throwing this? uriDigest?

It does not, will remove throws NoSuchAlgorithmException

SantiagoTorres · 2022-09-06T20:55:29Z

src/main/java/io/jenkins/plugins/intotorecorder/InTotoWrapper.java

+    private void setProvenanceMaterials(HashMap<String, ArtifactHash> results) {
+
+        HashMap<String, ArtifactHash> materials =new HashMap<String, ArtifactHash>();
+        materials=link.excludeArtifactsByPattern(results,null);


Hmm, this logic is somewhat convoluted here. You call collectArtifacts twice, in one you implicitly use it here, the other you pass as a parameter. Am I understanding this right? I think we could simplify this code by:

reusing the link as we are doing

not passing the results parameter.

Iterate over th elements on the link itself (not sure why are we filtering)

Form the arraylist using the iterator

Yes, you are right, this seems more straightforward. Will implement this...

src/main/java/io/jenkins/plugins/intotorecorder/InTotoWrapper.java

lakshya8066 · 2022-09-12T06:38:42Z

Is it a good idea to remove the import statements that are not used?

adityasaky · 2022-10-21T16:50:12Z

@lakshya8066 is this ready for a review again?

lakshya8066 · 2022-10-23T07:09:21Z

@adityasaky Yes, the updates are ready for review.

adityasaky

I think this is mostly in good shape with some TODOs we should ticketize separately. We should open a ticket for Rekor support and another for Archivist support.

src/main/java/io/jenkins/plugins/intotorecorder/InTotoWrapper.java

lakshya8066 · 2022-11-06T09:00:27Z

Can we pass in a list of user controlled environment variables to exclude after running through some of the defaults here?

@adityasaky By user controlled, do you mean the user running the plugin through Jenkinsfile?
In that case, we can take in the env variables that need to be excluded from the metadata as input in the Jenkinsfile.

lakshya8066 · 2022-11-07T12:33:56Z

There is a ticked opened for Rekor already, so opened a ticked for Archivist only.

adityasaky · 2022-11-07T12:35:29Z

By user controlled, do you mean the user running the plugin through Jenkinsfile? In that case, we can take in the env variables that need to be excluded from the metadata as input in the Jenkinsfile.

Yes but on second thought, probably okay to open a ticket for a feature request. Probably a good first issue.

lakshya8066 · 2022-11-07T12:57:15Z

Yes but on second thought, probably okay to open a ticket for a feature request. Probably a good first issue.

I am working on this, but yeah I think let's get this PR merged and I can push a patch for this afterward. Is that okay?

adityasaky · 2022-11-07T15:06:41Z

Sounds good, can we still open a ticket for it? You can probably submit that patch directly to jenkinsci as well :)

lakshya8066 · 2022-11-07T17:05:59Z

Sure!

adityasaky · 2022-11-23T16:53:25Z

src/main/java/io/jenkins/plugins/intotorecorder/InTotoWrapper.java

+        @Override
+        public String invoke(File f, VirtualChannel channel) {
+            Gson gson = new Gson();
+            System.out.println(this.ProvenanceData);


Suggested change

System.out.println(this.ProvenanceData);

adityasaky

I think we can merge this with in-toto/in-toto-java#40 after the print statement nit.

lakshya8066 · 2022-11-28T05:43:38Z

@adityasaky This should be ready to merge right?

adityasaky · 2022-11-28T14:06:02Z

We want to cut a release of -java and bump up the version here, yes?

lakshya8066 · 2022-11-30T09:56:03Z

Ah yes, correct. Will update the version here once -java is released...

adityasaky · 2022-12-11T23:07:55Z

Thanks for your work, @lakshya8066!

Add provenance metadata support

adityasaky requested review from adityasaky and SantiagoTorres August 23, 2022 13:07

adityasaky reviewed Aug 29, 2022

View reviewed changes

lakshya8066 changed the title ~~(WIP) Add provenance metadata support~~ Add provenance metadata support Sep 5, 2022

SantiagoTorres requested changes Sep 6, 2022

View reviewed changes

adityasaky requested review from SantiagoTorres and adityasaky October 26, 2022 19:54

adityasaky reviewed Nov 1, 2022

View reviewed changes

src/main/java/io/jenkins/plugins/intotorecorder/InTotoWrapper.java Show resolved Hide resolved

src/main/java/io/jenkins/plugins/intotorecorder/InTotoWrapper.java Show resolved Hide resolved

src/main/java/io/jenkins/plugins/intotorecorder/InTotoWrapper.java Outdated Show resolved Hide resolved

adityasaky reviewed Nov 23, 2022

View reviewed changes

adityasaky approved these changes Nov 23, 2022

View reviewed changes

lakshya8066 force-pushed the add-provenance branch from a70bb26 to fd18d7f Compare November 23, 2022 19:17

added SLSA Provenance Support

87bbdd7

lakshya8066 force-pushed the add-provenance branch from fd18d7f to 87bbdd7 Compare December 10, 2022 18:06

adityasaky approved these changes Dec 11, 2022

View reviewed changes

adityasaky merged commit ccabbc7 into in-toto:master Dec 11, 2022

adityasaky added a commit to adityasaky/in-toto-jenkins-plugin that referenced this pull request Dec 12, 2022

Merge pull request in-toto#5 from lakshya8066/add-provenance

4f0c06a

Add provenance metadata support

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add provenance metadata support #5

Add provenance metadata support #5

lakshya8066 commented Aug 15, 2022

lakshya8066 commented Aug 21, 2022

adityasaky Aug 29, 2022

adityasaky Aug 29, 2022

lakshya8066 Aug 29, 2022

adityasaky Aug 29, 2022

lakshya8066 Aug 30, 2022

SantiagoTorres left a comment

SantiagoTorres Sep 6, 2022

SantiagoTorres Sep 6, 2022

lakshya8066 Sep 8, 2022

SantiagoTorres Sep 6, 2022

lakshya8066 Sep 8, 2022

SantiagoTorres Sep 6, 2022

lakshya8066 Sep 8, 2022

lakshya8066 commented Sep 12, 2022

adityasaky commented Oct 21, 2022

lakshya8066 commented Oct 23, 2022

adityasaky left a comment

lakshya8066 commented Nov 6, 2022 •

edited

Loading

lakshya8066 commented Nov 7, 2022

adityasaky commented Nov 7, 2022 •

edited

Loading

lakshya8066 commented Nov 7, 2022

adityasaky commented Nov 7, 2022

lakshya8066 commented Nov 7, 2022

adityasaky Nov 23, 2022

adityasaky left a comment

lakshya8066 commented Nov 28, 2022

adityasaky commented Nov 28, 2022

lakshya8066 commented Nov 30, 2022

adityasaky commented Dec 11, 2022

Add provenance metadata support #5

Add provenance metadata support #5

Conversation

lakshya8066 commented Aug 15, 2022

lakshya8066 commented Aug 21, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SantiagoTorres left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lakshya8066 commented Sep 12, 2022

adityasaky commented Oct 21, 2022

lakshya8066 commented Oct 23, 2022

adityasaky left a comment

Choose a reason for hiding this comment

lakshya8066 commented Nov 6, 2022 • edited Loading

lakshya8066 commented Nov 7, 2022

adityasaky commented Nov 7, 2022 • edited Loading

lakshya8066 commented Nov 7, 2022

adityasaky commented Nov 7, 2022

lakshya8066 commented Nov 7, 2022

Choose a reason for hiding this comment

adityasaky left a comment

Choose a reason for hiding this comment

lakshya8066 commented Nov 28, 2022

adityasaky commented Nov 28, 2022

lakshya8066 commented Nov 30, 2022

adityasaky commented Dec 11, 2022

lakshya8066 commented Nov 6, 2022 •

edited

Loading

adityasaky commented Nov 7, 2022 •

edited

Loading