Issues with private registry github app

[BenjaminDecreusefond]

Feedback

Hi!

We are trying to set up the private registry with a github app and we are facing issue regarding the app.
We followed this https://docs.terrakube.io/user-guide/vcs-providers/github-app to create the VCS provider with the following parameter

• We named the app Terrakube (OrganizationName)
• Set the home page url to our api url endpoint
• set the correct privilege for the app
• generated the private key and put it into terrakube with the correct pkcs8 format
• and allowed it on all our repos

we then followed this https://docs.terrakube.io/user-guide/private-registry/publishing-private-modules to set up the module. We set the following parameter

• github url to the repo that stores the modules
• set a prefix tag for the repo
• base path for the module is /modules/mymodule/
  When we validate the config it creates the module with the version already present, however when we create a new tag version it is never taken into account by the module and never appear in the UI and we keep having this log in the API logs

2024-10-18T18:18:00.242Z  INFO 1 --- [ryBean_Worker-2] o.terrakube.api.rs.module.GitTagsCache   : vcs using GITHUB
2024-10-18T18:18:00.242Z ERROR 1 --- [ryBean_Worker-2] o.t.a.plugin.scheduler.module.CacheJob   : Updating module index for Weezevent/database/aws
2024-10-18T18:18:00.242Z ERROR 1 --- [ryBean_Worker-2] o.t.a.plugin.scheduler.module.CacheJob   : Cannot invoke "org.terrakube.api.rs.vcs.GitHubAppToken.getToken()" because "gitHubAppToken" is null

It is very strange and we have trouble identifying where it could come from ?
Do you have any clue for us please ?

Regards ! :)

[alfespa17]

Did you install the app in your github organization?

[BenjaminDecreusefond]

Yep we did ! We give it privileges to all repositories

[alfespa17]

Did you use command for the private key?

openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in my-terrakube-app.private-key.pem -out pkcs8.key

[BenjaminDecreusefond]

yep we did ! and put the output key from pkcs8.key into the terrakube VCS provider

[alfespa17]

Can you create a workspace using the github app and run a plan just to check if the connection is working correctly?

If the connection is working propertly it should download the github repository content

[BenjaminDecreusefond]

I just created a VCS workspace, set default path as / in the repository, there is no terraform at the root of the repository but I suppose it tells us that the vcs connection is working correctly ?

[alfespa17]

I just created a VCS workspace, set default path as / in the repository, there is no terraform at the root of the repository but I suppose it tells us that the vcs connection is working correctly ?

Use some folder that has some terraform file, run a plan and it should fail but you should be able to see some terraform related error because of some missing parameters related or something like that.

That should be enough to see if the connection is cloning the repository correctly

[BenjaminDecreusefond]

Just performed what you asked ! I have no issues and the plan display successfully !

[alfespa17]

Can you try creating a module again? From the above it looks like the github app token is working correctly

[BenjaminDecreusefond]

The creation of the module works fine ! When I create it it sees all tags however, when i create a new one after the module creation on terrakube, it seems like terrakube never sees it :/

[alfespa17]

I think maybe Terrakube generates the github token correctly when you create a workspace, after creating one workspace your modules creation should work.

I guess there is some logic missing to generate the token if you create a module right after the VCS connection is created

[alfespa17]

Just to clarify @BenjaminDecreusefond , module creation is working fine after you tested the workspace creation, right?

[BenjaminDecreusefond]

The module creation was working fine before the workspace creation. The issue that I had and that i still have is that now, my module is created on Terrakube with all tags present on the repository. However, if now I push a new tag, Terrakube will never see and don't update the tag list.
Maybe I can try using a oauth app ?

[alfespa17]

XD now I get it sorry

You won't be able to see new tags immediately there is an internal job that refresh tags every five or three minutes

[BenjaminDecreusefond]

Oh !

It's been 4 hours 😅

[alfespa17]

It is related to this issue from some time ago

#479 (comment)

[BenjaminDecreusefond]

Do you think the error

2024-10-18T20:06:00.181Z  INFO 1 --- [ryBean_Worker-9] o.terrakube.api.rs.module.GitTagsCache   : vcs using GITHUB
2024-10-18T20:06:00.182Z ERROR 1 --- [ryBean_Worker-9] o.t.a.plugin.scheduler.module.CacheJob   : Updating module index for Weezevent/certificate/aws
2024-10-18T20:06:00.182Z ERROR 1 --- [ryBean_Worker-9] o.t.a.plugin.scheduler.module.CacheJob   : Cannot invoke "org.terrakube.api.rs.vcs.GitHubAppToken.getToken()" because "gitHubAppToken" is null
2024-10-18T20:06:00.182Z  INFO 1 --- [ryBean_Worker-9] o.terrakube.api.rs.module.GitTagsCache   : vcs using GITHUB
2024-10-18T20:06:00.182Z ERROR 1 --- [ryBean_Worker-9] o.t.a.plugin.scheduler.module.CacheJob   : Updating module index for Weezevent/database/aws
2024-10-18T20:06:00.182Z ERROR 1 --- [ryBean_Worker-9] o.t.a.plugin.scheduler.module.CacheJob   : Cannot invoke "org.terrakube.api.rs.vcs.GitHubAppToken.getToken()" because "gitHubAppToken" is null

could have an incidence ?
It seems like it keeps the module from updating ?

[alfespa17]

Hey @stanleyz do you have any idea about this one? it looks related to the changes related to github app in this class

terrakube/api/src/main/java/org/terrakube/api/rs/module/GitTagsCache.java

Line 181 in 14a48ca

gitHubAppToken.getToken());

It looks like a bug but I am not sure

[BenjaminDecreusefond]

Happy to help if needed ! :)

[alfespa17]

Happy to help if needed ! :)

By the way using oAuth app should work for you as a work around for now

[BenjaminDecreusefond]

Yep, thank you I think that's what we're going to do for now !

[BenjaminDecreusefond]

Great ! I can confirm you it works like a charm with Oauth app ! thanks you very much !

[BenjaminDecreusefond]

Btw not sure if it is normal but during debugging I tried to click on the edit client button in the VCS providers tab of organization but it seems like nothing happens. I check in the network tab and when I click the button no call are being made so I wonder if this button has been implemented ?

this one ! :)

[alfespa17]

Btw not sure if it is normal but during debugging I tried to click on the edit client button in the VCS providers tab of organization but it seems like nothing happens. I check in the network tab and when I click the button no call are being made so I wonder if this button has been implemented ? this one ! :)

FYI #1096

[stanleyz]

Hey @stanleyz do you have any idea about this one? it looks related to the changes related to github app in this class

terrakube/api/src/main/java/org/terrakube/api/rs/module/GitTagsCache.java

Line 181 in 14a48ca

gitHubAppToken.getToken());

It looks like a bug but I am not sure

looks like so, will take a look later.

[BenjaminDecreusefond]

Btw not sure if it is normal but during debugging I tried to click on the edit client button in the VCS providers tab of organization but it seems like nothing happens. I check in the network tab and when I click the button no call are being made so I wonder if this button has been implemented ? this one ! :)

FYI #1096

Ah ok I didn't know sorry !