Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Main develop sync bd5118 #5817

Merged
merged 43 commits into from
Sep 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
779e913
migration number changes (#5692)
prakash100198 Aug 15, 2024
4f04d6b
refrain from checkin autoscalingCheckBeforeTrigger for virt clus (#5696)
prakash100198 Aug 20, 2024
2e58e77
fix: Decode secret fix on add update oss (#5695)
prakash100198 Aug 20, 2024
bf23515
saving pco concurrency case handled (#5688)
prakash100198 Aug 20, 2024
694831c
fix: script for pipelineStageStepVariable, making input value and def…
prakash100198 Aug 21, 2024
3e31f49
fix: ea fixes for helm app (#5708)
RajeevRanjan27 Aug 21, 2024
8de88d7
Revert "fix: ea fixes for helm app (#5708)" (#5713)
RajeevRanjan27 Aug 22, 2024
378c2d9
fix: SkipCiBuildCachePushPull code incorporated with minor refac in h…
prakash100198 Aug 22, 2024
827608f
migration syn with ent (#5718)
prkhrkat Aug 23, 2024
5f43eb2
doc: Edit Deployment Chart Schema (#5735)
ashokdevtron Aug 23, 2024
16d01d6
doc: Redirection of old entry in gitbook.yaml (#5738)
ashokdevtron Aug 23, 2024
d816dee
docs: added Documentation for Air-Gapped Installation (#5360)
badal773 Aug 23, 2024
26784d5
feat: Env description handling (#5744)
kripanshdevtron Aug 27, 2024
e677fbd
misc: Main sync rc - branch update (#5753)
kartik-579 Aug 28, 2024
8a61bac
doc: Update prerequisites of code-scan (#5625)
bhushan-nemade-dt Aug 28, 2024
6da544f
fix: ci patch rbac for branch update (#5759)
Ash-exp Aug 28, 2024
09946c2
feat: Added basic auth support for servicemonitor (#5761)
pawan-mehta-dt Aug 29, 2024
80f0758
fix: Bitnami chart repo tls issue (#5740)
akshatsinha007 Aug 29, 2024
7ee4a32
doc: Cosign plugin doc (#5665)
bhushan-nemade-dt Aug 29, 2024
99d10f5
fix: check rbac on env if envName is present (#5765)
prakash100198 Aug 29, 2024
f1a50b1
doc: CraneCopy plugin doc (#5658)
bhushan-nemade-dt Aug 30, 2024
3ef2b96
doc: Devtron CD Trigger Plugin doc (#5747)
bhushan-nemade-dt Aug 30, 2024
3888a41
doc: DockerSlim plugin doc (#5660)
bhushan-nemade-dt Aug 30, 2024
a625e7e
doc: Devtron Job Trigger Plugin doc (#5742)
bhushan-nemade-dt Aug 30, 2024
ff89a26
fix: scan tool active check removed (#5771)
kripanshdevtron Aug 30, 2024
5170040
feat: Docker pull env driven (#5767)
prakash100198 Aug 30, 2024
c66ccf5
fix: panic handlings and argocd app delete stuck in partial stage (#5…
Ash-exp Aug 30, 2024
4296366
feat: plugin creation support (#5630)
prakash100198 Sep 2, 2024
47843d9
Revert "feat: plugin creation support (#5630)" (#5778)
prakash100198 Sep 2, 2024
fd90dfb
fix: unimplemented cluster cron service (#5781)
Ash-exp Sep 2, 2024
1540271
fix: sql injection fixes (#5783)
kripanshdevtron Sep 2, 2024
ba02845
doc: Vulnerability Scanning Plugin doc (#5722)
bhushan-nemade-dt Sep 3, 2024
02f4a1b
docs: Jira plugins doc (Validator + Updater) (#5709)
ashokdevtron Sep 3, 2024
a6a2ae2
add basic auth and tls for sm (#5789)
pawan-mehta-dt Sep 3, 2024
654ba93
docs: added commands enable ingress during helm installation (#5794)
badal773 Sep 4, 2024
0e16daf
Revamped + Restructured Ingress Setup Doc (#5798)
ashokdevtron Sep 4, 2024
d4bd272
modifying route (#5799)
badal773 Sep 4, 2024
43ba232
fix: cron status update refactoring (#5790)
Ash-exp Sep 4, 2024
be9d553
docs: modified the anchorlink in ingress.md (#5800)
badal773 Sep 4, 2024
1e0af22
query param split (#5801)
Shivam-nagar23 Sep 4, 2024
8f92d3f
fix: upgraded to /argo-cd/v2 v2.9.21 (#5758)
prkhrkat Sep 5, 2024
bd51187
fix: Ea rbac fixes (#5813)
kripanshdevtron Sep 9, 2024
c25bd75
Merge branch 'develop' into main-develop-sync-9sep
kartik-579 Sep 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions api/cluster/EnvironmentRestHandler.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import (
"net/http"
"strconv"
"strings"
"sync"
"time"

k8s2 "github.com/devtron-labs/common-lib/utils/k8s"
Expand Down Expand Up @@ -513,9 +514,8 @@ func (impl EnvironmentRestHandlerImpl) GetEnvironmentConnection(w http.ResponseW
responseObj.ClusterReachable = false
}
//updating the cluster connection error to db
mapObj := map[int]error{
clusterBean.Id: err,
}
mapObj := &sync.Map{}
mapObj.Store(clusterBean.Id, err)
impl.environmentClusterMappingsService.HandleErrorInClusterConnections([]*request.ClusterBean{clusterBean}, mapObj, true)
common.WriteJsonResp(w, nil, responseObj, http.StatusOK)
}
1 change: 1 addition & 0 deletions api/cluster/wire_cluster.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import (
var ClusterWireSet = wire.NewSet(
repository.NewClusterRepositoryImpl,
wire.Bind(new(repository.ClusterRepository), new(*repository.ClusterRepositoryImpl)),
cluster.NewClusterServiceImpl,
cluster.NewClusterServiceImplExtended,
wire.Bind(new(cluster.ClusterService), new(*cluster.ClusterServiceImplExtended)),

Expand Down
1 change: 1 addition & 0 deletions api/k8s/application/k8sApplicationRestHandler.go
Original file line number Diff line number Diff line change
Expand Up @@ -559,6 +559,7 @@ func (handler *K8sApplicationRestHandlerImpl) GetPodLogs(w http.ResponseWriter,
common.WriteJsonResp(w, err, nil, http.StatusBadRequest)
return
}
handler.logger.Infow("get pod logs request", "request", request)
handler.requestValidationAndRBAC(w, r, token, request)
lastEventId := r.Header.Get(bean2.LastEventID)
isReconnect := false
Expand Down
3 changes: 0 additions & 3 deletions api/k8s/wire_k8sApp.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,4 @@ var K8sApplicationWireSet = wire.NewSet(
informer.NewGlobalMapClusterNamespace,
informer.NewK8sInformerFactoryImpl,
wire.Bind(new(informer.K8sInformerFactory), new(*informer.K8sInformerFactoryImpl)),

cluster.NewClusterCronServiceImpl,
wire.Bind(new(cluster.ClusterCronService), new(*cluster.ClusterCronServiceImpl)),
)
9 changes: 6 additions & 3 deletions cmd/external-app/wire_gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,11 +146,14 @@
* [DockerSlim](user-guide/plugins/docker-slim.md)
* [GoLang-migrate](user-guide/plugins/golang-migrate.md)
* [Jenkins](user-guide/plugins/jenkins.md)
* [Jira Issue Validator](user-guide/plugins/jira-validator.md)
* [Jira Issue Updater](user-guide/plugins/jira-updater.md)
* [K6 Load Testing](user-guide/plugins/k6-load-testing.md)
* [Pull images from container repository](user-guide/plugins/pull-images-from-container-repository.md)
* [Semgrep](user-guide/plugins/semgrep.md)
* [SonarQube](user-guide/plugins/sonarqube.md)
* [SonarQube v1.1.0](user-guide/plugins/sonarqube-v1.1.0.md)
* [Vulnerability Scanning](user-guide/plugins/vulnerability-scanning.md)


## Resources
Expand Down
153 changes: 140 additions & 13 deletions docs/setup/install/ingress-setup.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,144 @@
# Ingress Setup

After Devtron is installed, Devtron is accessible through service `devtron-service`.
If you want to access Devtron through ingress, edit `devtron-service` and change the loadbalancer to ClusterIP. You can do this using `kubectl patch` command:
## Introduction

If you wish to use [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) as a means to access the Devtron services available in your cluster, you can configure it either during the installation or after the installation of Devtron.

Refer the section relevant to you:
* [During Devtron Installation](#enabling-ingress-during-devtron-installation)
* [After Devtron Installation](#configuring-ingress-after-devtron-installation)

If you have successfully configured Ingress, refer [Post Ingress Setup](#enable-https-for-devtron).

---

## Enabling Ingress during Devtron Installation

If you are installing Devtron, you can enable Ingress either via [set flag](#using-set-flag) or by using [ingress-values.yaml](#using-ingress-values.yaml) to specify the desired Ingress settings.

### Using set flag

You can use the `--set` flag to specify the desired Ingress settings.

Here, we have added 5 configurations you can perform depending on your requirements:
* [Only Basic Configuration](#only-basic-configuration)
* [Configuration Including Labels](#configuration-including-labels)
* [Configuration Including Annotations](#configuration-including-annotations)
* [Configuration Including TLS Settings](#configuration-including-tls-settings)
* [Comprehensive Configuration](#comprehensive-configuration)

#### Only Basic Configuration

To enable Ingress and set basic parameters, use the following command:

```bash
helm install devtron devtron/devtron-operator -n devtroncd \
--set components.devtron.ingress.enabled=true \
--set components.devtron.ingress.className=nginx \
--set components.devtron.ingress.host=devtron.example.com
```

#### Configuration Including Labels

To add labels to the Ingress resource, use the following command:

```bash
helm install devtron devtron/devtron-operator -n devtroncd \
--set components.devtron.ingress.enabled=true \
--set components.devtron.ingress.className=nginx \
--set components.devtron.ingress.host=devtron.example.com \
--set components.devtron.ingress.labels.env=production
```

#### Configuration Including Annotations

To add annotations to the Ingress resource, use the following command:

```bash
helm install devtron devtron/devtron-operator -n devtroncd \
--set components.devtron.ingress.enabled=true \
--set components.devtron.ingress.className=nginx \
--set components.devtron.ingress.host=devtron.example.com \
--set components.devtron.ingress.annotations."kubernetes\.io/ingress\.class"=nginx \
--set components.devtron.ingress.annotations."nginx\.ingress\.kubernetes\.io\/app-root"="/dashboard"
```

#### Configuration Including TLS Settings

To configure TLS settings, including `secretName` and `hosts`, use the following command:

```bash
helm install devtron devtron/devtron-operator -n devtroncd \
--set components.devtron.ingress.enabled=true \
--set components.devtron.ingress.className=nginx \
--set components.devtron.ingress.host=devtron.example.com \
--set components.devtron.ingress.tls[0].secretName=devtron-tls \
--set components.devtron.ingress.tls[0].hosts[0]=devtron.example.com
```

#### Comprehensive Configuration

To include all the above settings in a single command, use:

```bash
helm install devtron devtron/devtron-operator -n devtroncd \
--set components.devtron.ingress.enabled=true \
--set components.devtron.ingress.className=nginx \
--set components.devtron.ingress.host=devtron.example.com \
--set components.devtron.ingress.annotations."kubernetes\.io/ingress\.class"=nginx \
--set components.devtron.ingress.annotations."nginx\.ingress\.kubernetes\.io\/app-root"="/dashboard" \
--set components.devtron.ingress.labels.env=production \
--set components.devtron.ingress.pathType=ImplementationSpecific \
--set components.devtron.ingress.tls[0].secretName=devtron-tls \
--set components.devtron.ingress.tls[0].hosts[0]=devtron.example.com
```


### Using ingress-values.yaml

As an alternative to the [set flag](#using-set-flag) method, you can enable Ingress using `ingress-values.yaml` instead.

Create an `ingress-values.yaml` file. You may refer the below format for an advanced ingress configuration which includes labels, annotations, secrets, and many more.

```yml
components:
devtron:
ingress:
enabled: true
className: nginx
labels: {}
# env: production
annotations: {}
# nginx.ingress.kubernetes.io/app-root: /dashboard
pathType: ImplementationSpecific
host: devtron.example.com
tls: []
# - secretName: devtron-info-tls
# hosts:
# - devtron.example.com
```

Once you have the `ingress-values.yaml` file ready, run the following command:

```bash
helm install devtron devtron/devtron-operator -n devtroncd --reuse-values -f ingress-values.yaml
```

---

## Configuring Ingress after Devtron Installation

After Devtron is installed, Devtron is accessible through `devtron-service`. If you wish to access Devtron through ingress, you'll need to modify this service to use a ClusterIP instead of a LoadBalancer.

You can do this using the `kubectl patch` command:

```bash
kubectl patch -n devtroncd svc devtron-service -p '{"spec": {"ports": [{"port": 80,"targetPort": "devtron","protocol": "TCP","name": "devtron"}],"type": "ClusterIP","selector": {"app": "devtron"}}}'
```

After this, create ingress by applying the ingress yaml file.
You can use [this yaml file](https://github.com/devtron-labs/devtron/blob/main/manifests/yamls/devtron-ingress.yaml) to create ingress to access Devtron:
Next, create ingress to access Devtron by applying the `devtron-ingress.yaml` file. The file is also available on this [link](https://github.com/devtron-labs/devtron/blob/main/manifests/yamls/devtron-ingress.yaml). You can access Devtron from any host after applying this yaml.

```yaml
```yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
Expand Down Expand Up @@ -49,9 +177,9 @@ spec:
pathType: ImplementationSpecific
```

You can access Devtron from any host after applying this yaml. For k8s versions <1.19, [apply this yaml](https://github.com/devtron-labs/devtron/blob/main/manifests/yamls/devtron-ingress-legacy.yaml):
For k8s versions < 1.19, [apply this yaml](https://github.com/devtron-labs/devtron/blob/main/manifests/yamls/devtron-ingress-legacy.yaml):

```yaml
```yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
Expand Down Expand Up @@ -79,7 +207,7 @@ spec:

Optionally, you also can access Devtron through a specific host by running the following YAML file:

```yaml
```yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
Expand Down Expand Up @@ -119,9 +247,11 @@ spec:
pathType: ImplementationSpecific
```

---

## Enable HTTPS For Devtron

Once ingress setup for devtron is done and you want to run Devtron over `https`, you need to add different annotations for different ingress controllers and load balancers.
Once Ingress setup for Devtron is done and you want to run Devtron over `https`, you need to add different annotations for different ingress controllers and load balancers.

### 1. Nginx Ingress Controller

Expand Down Expand Up @@ -175,7 +305,4 @@ In case of AWS application load balancer, the following annotations need to be a
```
For an Ingress resource to be observed by AGIC (Application Gateway Ingress Controller) must be annotated with kubernetes.io/ingress.class: azure/application-gateway. Only then AGIC will work with the Ingress resource in question.

> Note: Make sure NOT to use port 80 with HTTPS and port 443 with HTTP on the Pods.



> Note: Make sure NOT to use port 80 with HTTPS and port 443 with HTTP on the Pods.
57 changes: 57 additions & 0 deletions docs/user-guide/plugins/jira-updater.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Jira Issue Updater

## Introduction
The Jira Issue Updater plugin extends the capabilities of Devtron CI by allowing updates to Jira issues directly from the pipeline. It can add build pipeline status and docker image ID as a comment on Jira tickets, keeping the issue tracking synchronized with your CI processes.

### Prerequisites

- A Jira account with the necessary [API access](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/#Create-an-API-token).
- The API credentials (username, password, and base URL) for your Jira instance. Obtain the API credentials from your Jira admin if required.
- A pull request raised with your Git provider. Title of pull request must contain the Jira ID.
- Jira Issue (e.g., REDOC-12)
- Webhook added to the git repository. [Click here](https://docs.devtron.ai/usage/applications/creating-application/workflow/ci-pipeline#configuring-webhook) to know more.

---

## Steps

1. On the **Edit build pipeline** page, go to the **Post-Build Stage**.
2. Click **+ Add task**.
3. Select **Jira Issue Updater** from the list of plugins.
* Enter a task name (mandatory).
* Optionally, enter a description.
* Provide values for the input variables.

| Variable | Format | Description |
| ------------------------ | ------ | --------------------------------------------------------- |
| JiraUsername | String | Your Jira username (e.g., johndoe@devtron.ai) |
| JiraPassword | String | Your Jira API token provided by the Jira admin |
| JiraBaseUrl | String | The base URL of your Jira instance (e.g., https://yourdomain.atlassian.net/) |
| UpdateWithDockerImageId | Bool | Set to `true` to include the Docker Image ID in the update |
| UpdateWithBuildStatus | Bool | Set to `true` to include the build status in the update |

* `Trigger/Skip Condition` allows you to set conditions under which this task will execute or be skipped.
* `Pass/Failure Condition` allows you define conditions to determine if the build passes or fails based on the Jira update.

4. Go to the **Build Stage**.

5. Select **Pull Request** in the **Source Type** dropdown.

6. Use filters to fetch only the PRs matching your regex. Here are few examples:
* **Title** can be a regex pattern (e.g., `^(?P<jira_Id>([a-zA-Z0-9-].*))`) to extract the Jira ID from the PR title. Only those PRs fulfilling the regex will be shown for image build process.
* **State** can be `^open$`, where only PRs in open state will be shown for image build process.

7. Click **Update Pipeline**.

---

## Results

![Figure 1: Build Log](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/plugins/jira/jira-updater-log.jpg)

![Figure 2: Comments added by the Plugin on the Jira Issue](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/plugins/jira/jira-updater.jpg)





54 changes: 54 additions & 0 deletions docs/user-guide/plugins/jira-validator.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Jira Issue Validator

## Introduction
The Jira Issue Validator plugin extends the filtering capabilities of the Devtron CI and lets users perform validation based on Jira Ticket ID status. This plugin ensures that only builds associated with valid Jira tickets are executed, improving the accuracy of the CI process.

### Prerequisites

- A Jira account with the necessary [API access](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/#Create-an-API-token).
- The API credentials (username, password, and base URL) for your Jira instance. Obtain the API credentials from your Jira admin if required.
- A pull request raised with your Git provider. Title of pull request must contain the Jira ID.
- Jira Issue (e.g., REDOC-12)
- Webhook added to the git repository. [Click here](https://docs.devtron.ai/usage/applications/creating-application/workflow/ci-pipeline#configuring-webhook) to know more.

---

## Steps

1. On the **Edit build pipeline** page, go to the **Pre-Build Stage** (or Post-Build Stage).
2. Click **+ Add task**.
3. Select **Jira Issue Validator** from the list of plugins.
* Enter a task name (mandatory).
* Optionally, enter a description.
* Provide values for the input variables.

| Variable | Format | Description |
| -------------- | ------ | --------------------------------------------------------- |
| JiraUsername | String | Your Jira username (e.g., johndoe@devtron.ai) |
| JiraPassword | String | Your Jira API token provided by the Jira admin |
| JiraBaseUrl | String | The base URL of your Jira instance (e.g., https://yourdomain.atlassian.net) |

* `Trigger/Skip Condition` allows you to set conditions under which this task will execute or be skipped.
* `Pass/Failure Condition` allows you to define conditions that determine whether the build passes or fails based on Jira validation.

4. Go to the **Build Stage**.

5. Select **Pull Request** in the **Source Type** dropdown.

6. Use filters to fetch only the PRs matching your regex. Here are few examples:
* **Title** can be a regex pattern (e.g., `^(?P<jira_Id>([a-zA-Z0-9-].*))`) to extract the Jira ID from the PR title. Only those PRs fulfilling the regex will be shown for image build process.
* **State** can be `^open$`, where only PRs in open state will be shown for image build process.

7. Click **Update Pipeline**.

---

## Results

**Case 1**: If Jira issue exists and the same is found in the PR title

![Figure 1: Jira Issue Match](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/plugins/jira/jira-issue-validator.jpg)

**Case 2**: If Jira issue is not found

![Figure 2: Error in Finding Jira Issue](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/plugins/jira/issue-validation-failed.jpg)
Loading
Loading