Skip to content

Gitops runtime troubleshooting #1250

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Apr 22, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions _data/argohub-home-content.yml
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,8 @@
localurl: /gitops/gitops-runtimes/runtime-install-with-existing-argo-cd/
- title: Install GitOps Runtime with new Argo CD
localurl: /gitops/gitops-runtimes/hybrid-gitops-helm-installation/
- title: Troubleshooting Runtime installation
localurl: /gitops/gitops-runtimes/runtime-troubleshooting/


- title: Administration
Expand Down
2 changes: 2 additions & 0 deletions _data/argohub-nav.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@
url: "/hybrid-gitops-helm-installation"
- title: Configure GitOps Runtime
url: "/runtime-configuration"
- title: Troubleshoot installation
url: "/runtime-troubleshooting"
- title: Runtime values file validation
url: "/gitops-values-yaml-validation"
- title: Ingress and service-mesh access modes
Expand Down
2 changes: 2 additions & 0 deletions _data/home-content.yml
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,8 @@
localurl: /docs/installation/gitops/runtime-install-with-existing-argo-cd/
- title: Install GitOps Runtime with new Argo CD
localurl: /docs/installation/gitops/hybrid-gitops-helm-installation/
- title: Troubleshooting Runtime installation
localurl: /docs/installation/gitops/runtime-troubleshooting/


- title: Administration
Expand Down
4 changes: 3 additions & 1 deletion _data/nav.yml
Original file line number Diff line number Diff line change
Expand Up @@ -610,7 +610,9 @@
- title: Install GitOps Runtime with existing Argo CD
url: "/runtime-install-with-existing-argo-cd"
- title: Install GitOps Runtime with new Argo CD
url: "/hybrid-gitops-helm-installation"
url: "/hybrid-gitops-helm-installation"
- title: Troubleshoot installation
url: "/runtime-troubleshooting"
- title: On-premises GitOps Runtime
url: "/on-prem-gitops-runtime-install"
- title: Runtime values file validation
Expand Down
8 changes: 4 additions & 4 deletions _docs/installation/gitops/gitops-values-yaml-validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,10 @@ global:
{% endhighlight %}

## Disable validation for custom/fine-grained Git tokens
When using token with custom scopes, or GitHub's fine-grained tokens (currently not officially supported by Codefresh), _skip token validation_ to avoid validation failures during installation.
When using token with custom scopes, or GitHub's fine-grained tokens (currently not officially supported by Codefresh), _for Runtime version v0.18.0 or lower, skip token validation_ to avoid validation failures during installation.

Add the `skipGitPermissionValidation` flag to your `values.yaml` file:

* Add the `skipGitPermissionValidation` flag to your `values.yaml` file:

```yaml
app-proxy:
Expand All @@ -105,6 +106,5 @@ where:
* `<version>` is the version of the runtime to install.


## Install first GitOps Runtime in account
If you are installing the first GitOps Runtime in your Codefresh account, follow the installation wizard for guided instructions.


49 changes: 23 additions & 26 deletions _docs/installation/gitops/runtime-troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ description: "Review how to resolve issues during Runtime installation"
toc: true
---


## Troubleshooting Runtime installation
This article describes potential issues you may encounter when installing the GitOps Runtime, whether you're using an existing Argo CD instance or setting up a new one.

## Error: Job has reached the specified backoff limit
Before initiating the installation, Codefresh automatically validates the `values.yaml` file to verify that the supplied values are correct.
This error appears when you copy and run the Install Runtime command in the Runtime Installation wizard.
Before starting the installation, GitOps Cloud automatically validates the `values.yaml` file to ensure the supplied values are correct.

### Possible cause
Validation errors in your `values.yaml` file.
Expand All @@ -31,18 +33,18 @@ Validation errors in your `values.yaml` file.


## Error: failed converting helm release <runtime-name> to GitOops;...ISC repo not initialized
This error displayed when you try to Configure the Runtime as Argo CD Application.
This error appears when you try to Configure the Runtime as Argo CD Application in the Runtime Installation wizard.

### Possible cause
The Shared Configuration Repository (`ISC`) has been created but is not yet initialized.
The Shared Configuration Repository (`ISC`) has been created but is not yet initialized.

### Resolution
Wait a few seconds, and try again.


## Runtime timeout errors
One of these errors indicating a timeout:
* `Error: "unable to initialise Codefresh Client", "error": "secrets \"codefresh-token\" not found"`
One of these errors appear in the UI indicating a timeout after completing Runtime installation and configuration in the Runtime Installation wizard:
* `Error: "unable to initialize Codefresh Client", "error": "secrets \"codefresh-token\" not found"`
* For tunnel-based access mode, the `codefresh-tunnel-client` pod fails after installation.
* Pods failed error

Expand All @@ -68,45 +70,40 @@ The `--wait` flag in Install Runtime command controls how long the installation


## Invalid Git token error
This error appears in the UI after Runtime installation when all Runtime components including the Application Proxy (app-proxy) are up and running.

### Possible cause
Your token is invalid because of missing scopes
Your token is [invalid because of missing scopes](#check-token-scopes)
OR
You are using a fine-grained token or one with custom scopes
You are using a [fine-grained token or one with custom scopes](#skip-validation-for-fine-grained-github-tokens)

### Resolution

##### Check token scopes
1. Check the scopes for the Runtime token you provided during installation.
If you provided the same token as both the Runtime and user token, make sure the token has the additional scopes required.
1. If needed, generate a new PAT (Personal Access Token) with the correct scopes.

**GitHub Git Runtime token**
{: .table .table-bordered .table-hover}
| Type | Required scopes |
| -------- | ----------------- |
| **Classic** |{::nomarkdown}<ul><li><code class="highlighter-rouge">repo</code></li><li><code class="highlighter-rouge">admin:repo_hook</code></li></ul>{:/}|
|**Fine-grained** (personal or group-based) |{::nomarkdown}<ul><li>Repository access: <code class="highlighter-rouge">All repositories</code> or <code class="highlighter-rouge">Only select repositories</code> including all repos that Argo CD syncs from</li><li>Repository permissions:<ul><li>Administration: <code class="highlighter-rouge">Read and write</code></li><li>Commit statuses: <code class="highlighter-rouge">Read and write</code></li><li>Contents: <code class="highlighter-rouge">Read and write</code></li><li>Metadata: <code class="highlighter-rouge">Read-only</code></li><li>Pull requests: <code class="highlighter-rouge">Read and write</code></li><li>Webhooks: <code class="highlighter-rouge">Read and write</code></li></ul></li></ul>{:/}|

{: .table .table-bordered .table-hover}
| Type | Required scopes |
| -------- | ----------------- |
| **Classic** | {::nomarkdown}<ul><li><code class="highlighter-rouge">repo</code></li><li><code class="highlighter-rouge">admin:repo_hook</code></li></ul>{:/} |
| **Fine-grained** (personal or group-based) | {::nomarkdown}<ul><li>Repository access: <code class="highlighter-rouge">All repositories</code> or <code class="highlighter-rouge">Only select repositories</code> including all repos that Argo CD syncs from
</li><li>Repository permissions:<ul><li>Administration: <code class="highlighter-rouge">Read and write</code></li><li>Commit statuses: <code class="highlighter-rouge">Read and write</code></li><li>Contents: <code class="highlighter-rouge">Read and write</code></li><li>Metadata: <code class="highlighter-rouge">Read-only</code></li><li>Pull requests: <code class="highlighter-rouge">Read and write</code></li><li>Webhooks: <code class="highlighter-rouge">Read and write</code></li></ul></li</ul>{:/} |
| **Fine-grained** (personal or group-based) | {::nomarkdown}<ul><li>Repository access: <code class="highlighter-rouge">All repositories</code> or <code class="highlighter-rouge">Only select repositories</code> including all repos that Argo CD syncs from</li><li>Repository permissions: <ul><li>Administration: <code class="highlighter-rouge">Read and write</code></li><li>Commit statuses: <code class="highlighter-rouge">Read and write</code></li><li>Contents: <code class="highlighter-rouge">Read and write</code></li><li>Metadata: <code class="highlighter-rouge">Read-only</code></li><li>Pull requests: <code class="highlighter-rouge">Read and write</code></li><li>Webhooks: <code class="highlighter-rouge">Read and write</code></li></ul></li></ul>{:/} |


<br>

**GitHub Git user token**
{: .table .table-bordered .table-hover}
| Type | Required scopes |
| ------------ | ----------------- |
| **Classic** |{::nomarkdown}<ul><li><code class="highlighter-rouge">repo</code></li></ul>{:/} |
|**Fine-grained** |{::nomarkdown}<ul><li>Repository access: <code class="highlighter-rouge">All repositories</code> or <code class="highlighter-rouge">Only select repositories</code></li><li>Repository permissions:<ul><li>Contents: <code class="highlighter-rouge">Read and write</code></li><li>Metadata: <code class="highlighter-rouge">Read-only</code></li></ul></li></ul>{:/}|


{: .table .table-bordered .table-hover}
| Type | Required scopes |
| Type | Required scopes |
| -------- | ----------------- |
| **Classic** | {::nomarkdown}<ul><li><code class="highlighter-rouge">repo</code></li><li><code class="highlighter-rouge">admin:repo_hook</code></li></ul>{:/} |
| **Fine-grained** (personal or group-based) | {::nomarkdown}<ul><li>Repository access: <code class="highlighter-rouge">All repositories</code> or <code class="highlighter-rouge">Only select repositories</code> including all repos that Argo CD syncs from</li><li>Repository permissions:<ul><li>Administration: <code class="highlighter-rouge">Read and write</code></li><li>Commit statuses: <code class="highlighter-rouge">Read and write</code></li><li>Contents: <code class="highlighter-rouge">Read and write</code></li><li>Metadata: <code class="highlighter-rouge">Read-only</code></li><li>Pull requests: <code class="highlighter-rouge">Read and write</code></li><li>Webhooks: <code class="highlighter-rouge">Read and write</code></li></ul></li></ul>{:/} |
| **Fine-grained** (personal or group-based) | {::nomarkdown}<ul><li>Repository access: <code class="highlighter-rouge">All repositories</code> or <code class="highlighter-rouge">Only select repositories</code> including all repos that Argo CD syncs from</li><li>Repository permissions: <ul><li>Administration: <code class="highlighter-rouge">Read and write</code></li><li>Commit statuses: <code class="highlighter-rouge">Read and write</code></li><li>Contents: <code class="highlighter-rouge">Read and write</code></li><li>Metadata: <code class="highlighter-rouge">Read-only</code></li><li>Pull requests: <code class="highlighter-rouge">Read and write</code></li><li>Webhooks: <code class="highlighter-rouge">Read and write</code></li></ul></li></ul>{:/} |




<!--- ##### Ensure token formatting
Expand Down Expand Up @@ -184,6 +181,6 @@ global:
...
```


## Add Git Source failure
TBD
## Related articles
[Install GitOps Runtime with existing Argo CD]({{site.baseurl}}/docs/installation/gitops/runtime-install-with-existing-argo-cd/)
[Install GitOps Runtime with new Argo CD]({{site.baseurl}}/docs/installation/gitops/hybrid-gitops-helm-installation/)
1 change: 1 addition & 0 deletions _gitops/gitops-runtimes/runtime-troubleshooting.md
1 change: 1 addition & 0 deletions assets/js/src/argohub-redirect-mapping.json
Original file line number Diff line number Diff line change
Expand Up @@ -118,5 +118,6 @@
"/docs/installation/gitops/runtime-argocd-admin-api-token/": "/gitops/gitops-runtimes/runtime-argocd-admin-api-token/",
"/docs/installation/gitops/runtime-install-ingress-service-mesh-access-mode/": "/gitops/gitops-runtimes/runtime-install-ingress-service-mesh-access-mode/",
"/docs/promotions/getting-started/": "/gitops/promotions/getting-started/",
"/docs/installation/gitops/runtime-troubleshooting/": "/gitops/gitops-runtimes/runtime-troubleshooting/",
"/docs/products/promotion-concurrency/": "/gitops/products/promotion-concurrency/"
}