Skip to content

Commit 03d60d2

Browse files
Fix broken links (#1066)
* fix links * fix links * fix links * fix links * fix links * fix links * fix links * fix links
1 parent ae79ac7 commit 03d60d2

File tree

104 files changed

+286
-429
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

104 files changed

+286
-429
lines changed

docs/.pages

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@ nav:
33
- Engineering Fundamentals Checklist: engineering-fundamentals-checklist.md
44
- The First Week of an ISE Project: the-first-week-of-an-ise-project.md
55
- Who is ISE?: ISE.md
6-
- Contributing: contributing.md
76
- Agile Development: agile-development
87
- Automated Testing: automated-testing
98
- CI/CD: CI-CD

docs/CI-CD/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# Continuous Integration and Continuous Delivery
22

3-
[**Continuous Integration (CI)**](continuous-integration.md) is the engineering practice of frequently committing code in a shared repository, ideally several times a day, and performing an automated build on it. These changes are built with other simultaneous changes to the system, which enables early detection of integration issues between multiple developers working on a project. Build breaks due to integration failures are treated as the highest priority issue for all the developers on a team and generally work stops until they are fixed.
3+
[**Continuous Integration (CI)**](./continuous-integration.md) is the engineering practice of frequently committing code in a shared repository, ideally several times a day, and performing an automated build on it. These changes are built with other simultaneous changes to the system, which enables early detection of integration issues between multiple developers working on a project. Build breaks due to integration failures are treated as the highest priority issue for all the developers on a team and generally work stops until they are fixed.
44

55
Paired with an automated testing approach, continuous integration also allows us to also test the integrated build such that we can verify that not only does the code base still build correctly, but also is still functionally correct. This is also a best practice for building robust and flexible software systems.
66

7-
[**Continuous Delivery (CD)**](continuous-delivery.md) takes the **Continuous Integration (CI)** concept further to also test deployments of the integrated code base on a replica of the environment it will be ultimately deployed on. This enables us to learn early about any unforeseen operational issues that arise from our changes as quickly as possible and also learn about gaps in our test coverage.
7+
[**Continuous Delivery (CD)**](./continuous-delivery.md) takes the **Continuous Integration (CI)** concept further to also test deployments of the integrated code base on a replica of the environment it will be ultimately deployed on. This enables us to learn early about any unforeseen operational issues that arise from our changes as quickly as possible and also learn about gaps in our test coverage.
88

99
The goal of all of this is to ensure that the main branch is always shippable, meaning that we could, if we needed to, take a build from the main branch of our code base and ship it on production.
1010

docs/CI-CD/continuous-integration.md

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,8 @@ An automated build should encompass the following principles:
4444
### Code Style Checks
4545

4646
- Code across an engineering team must be formatted to agreed coding standards. Such standards keep code consistent, and most importantly easy for the team and customer(s) to read and refactor. Code styling consistency encourages collective ownership for project scrum teams and our partners.
47-
- There are several open source code style validation tools available to choose from ([code style checks](https://github.com/checkstyle/checkstyle), [StyleCop](https://en.wikipedia.org/wiki/StyleCop)). The [Code Review recipes section](../code-reviews/recipes/README.md) of the playbook has suggestions for linters and preferred styles for a number of languages.
48-
- Your code and documentation should avoid the use of non-inclusive language wherever possible. Follow the [Inclusive Linting section](recipes/inclusive-linting.md) to ensure your project promotes an inclusive work environment for both the team and for customers.
47+
- There are several open source code style validation tools available to choose from ([code style checks](https://github.com/checkstyle/checkstyle), [StyleCop](https://en.wikipedia.org/wiki/StyleCop)). The [Code Review recipes section](../code-reviews/recipes/) of the playbook has suggestions for linters and preferred styles for a number of languages.
48+
- Your code and documentation should avoid the use of non-inclusive language wherever possible. Follow the [Inclusive Linting section](./recipes/inclusive-linting.md) to ensure your project promotes an inclusive work environment for both the team and for customers.
4949
- We recommend incorporating security analysis tools within the build stage of your pipeline such as: code credential scanner, security risk detection, static analysis, etc. For Azure DevOps, you can add a security scan task to your pipeline by installing the [Microsoft Security Code Analysis Extension](https://secdevtools.azurewebsites.net/#pills-onboard). GitHub Actions supports a similar extension with the [RIPS security scan solution](https://github.com/marketplace/actions/rips-security-scan).
5050
- Code standards are maintained within a single configuration file. There should be a step in your build pipeline that asserts code in the latest commit conforms to the known style definition.
5151

@@ -59,7 +59,7 @@ An automated build should encompass the following principles:
5959

6060
### DevOps Security Checks
6161

62-
- Introduce security to your project at early stages. Follow the [DevSecOps section](dev-sec-ops/README.md) to introduce security practices, automation, tools and frameworks as part of the CI.
62+
- Introduce security to your project at early stages. Follow the [DevSecOps section](./dev-sec-ops/README.md) to introduce security practices, automation, tools and frameworks as part of the CI.
6363

6464
## Build Environment Dependencies
6565

@@ -68,7 +68,7 @@ An automated build should encompass the following principles:
6868
- We encourage maintaining a consistent developer experience for all team members. There should be a central automated manifest / process that streamlines the installation and setup of any software dependencies. This way developers can replicate the same build environment locally as the one running on a CI server.
6969
- Build automation scripts often require specific software packages and version pre-installed within the runtime environment of the OS. This presents some challenges as build processes typically version lock these dependencies.
7070
- All developers on the team should be able to emulate the build environment from their local desktop regardless of their OS.
71-
- For projects using VS Code, leveraging [Dev Containers](../developer-experience/devcontainers.md) can really help standardize the local developer experience across the team.
71+
- For projects using VS Code, leveraging [Dev Containers](../developer-experience/devcontainers-getting-started.md) can really help standardize the local developer experience across the team.
7272
- Well established software packaging tools like Docker, Maven, npm, etc should be considered when designing your build automation tool chain.
7373

7474
### Document Local Setup
@@ -172,7 +172,6 @@ Implementing schema validation is divided in two - the generation of the schemas
172172
There are two options to generate a schema:
173173

174174
- [From code](https://json-schema.org/implementations.html#from-code) - we can leverage the existing models and objects in the code and generate a customized schema.
175-
176175
- [From data](https://json-schema.org/implementations.html#from-data) - we can take yaml/json samples which reflect the configuration in general and use the various online tools to generate a schema.
177176

178177
### Validation

docs/CI-CD/dev-sec-ops/azure-devops-service-connection-security.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ Since Service Connections can have a lot of permissions in the external service,
5151
5252
To prevent accidental mis-use of Service Connections there are several checks that can be configured. These checks are configured on the Service Connection itself and therefore can only be configured by the owner or administrator of that Service Connection. A user of a certain YAML Pipeline cannot modify these checks since the checks are not defined in the YAML file itself.
5353
Configuration can be done in the Approvals and Checks menu on the Service Connection.
54-
![ApprovalsAndChecks](images/approvals-and-checks.png)
54+
![ApprovalsAndChecks](./images/approvals-and-checks.png)
5555
5656
### Branch Control
5757
@@ -64,4 +64,4 @@ With Branch Control in place, in combination with Branch Protections, it is not
6464
6565
> **Note:** When setting a wildcard for the Allowed Branches, anyone could still create a branch matching that wildcard and would be able to use the Service Connection. Using [git permissions](https://learn.microsoft.com/en-us/azure/devops/repos/git/require-branch-folders#enforce-permissions) it can be configured so only administrators are allowed to create certain branches, like release branches.*
6666
67-
![BranchControl](images/branch-control.png)
67+
![BranchControl](./images/branch-control.png)

docs/CI-CD/dev-sec-ops/secrets-management/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -188,4 +188,4 @@ The following steps lay out a clear pathway to creating new secrets and then uti
188188

189189
### Validation
190190

191-
Automated [credential scanning](credential_scanning.md) can be performed on the code regardless of the programming language.
191+
Automated [credential scanning](./credential_scanning.md) can be performed on the code regardless of the programming language.

docs/CI-CD/gitops/deploying-with-gitops.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
GitOps simply allows faster deployments by having git repositories in the center offering a clear audit trail via git commits and no direct environment access. Read more on [Why should I use GitOps?](https://www.gitops.tech/#why-should-i-use-gitops)
1010

1111
The below diagram compares traditional CI/CD vs GitOps workflow:
12-
![push based vs pull based deployments](images/GitopsWorflowVsTraditionalPush.jpg)
12+
![push based vs pull based deployments](./images/GitopsWorflowVsTraditionalPush.jpg)
1313

1414
## Tools for GitOps
1515

docs/CI-CD/gitops/github-workflows.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,19 @@
22

33
A workflow is a configurable automated process made up of one or more jobs where each of these jobs can be an action in GitHub. Currently, a YAML file format is supported for defining a workflow in GitHub.
44

5-
Additional information on GitHub actions and GitHub Workflows in the links posted in the [references](#references) section below.
5+
Additional information on GitHub actions and GitHub Workflows in the links posted in the [resources](#resources) section below.
66

77
## Workflow per Environment
88

99
The general approach is to have one pipeline, where the code is built, tested and deployed, and the artifact is then promoted to the next environment, eventually to be deployed into production.
1010

1111
There are multiple ways in GitHub that an environment setup can be achieved. One way it can be done is to have one workflow for multiple environments, but the complexity increases as additional processes and jobs are added to a workflow, which does not mean it cannot be done for small pipelines. The plus point of having one workflow is that, when an artifact flows from one environment to another the state and environment values between the deployment environments can be passed easily.
1212

13-
![Workflow-Designs-Dependent-Workflows](images/Workflow-Designs-Dependent-Workflows.png)
13+
![Workflow-Designs-Dependent-Workflows](./images/Workflow-Designs-Dependent-Workflows.png)
1414

1515
One way to get around the complexity of a single workflow is to have separate workflows for different environments, making sure that only the artifacts created and validated are promoted from one environment to another, as well as, the workflow is small enough, to debug any issues seen in any of the workflows. In this case, the state and environment values need to be passed from one deployment environment to another. Multiple workflows also helps to keep the deployments to the environments independent thus reducing the time to deploy and find issues earlier than later in the process. Also, since the environments are independent of each other, any failures in deploying to one environment does not block deployments to other environments. One tradeoff in this method, is that with different workflows for each environment, the maintenance increases as the complexity of workflows increase over time.
1616

17-
![Workflow-Designs-Independent-Workflows](images/Workflow-Designs-Independent-Workflows.png)
17+
![Workflow-Designs-Independent-Workflows](./images/Workflow-Designs-Independent-Workflows.png)
1818

1919
## Resources
2020

docs/CI-CD/gitops/secret-management/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ All the below tools share the following:
6464
- Easily scalable with multi-cluster and larger teams
6565
- Both solutions support either Azure Active Directory (Azure AD) [service principal](https://learn.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals) or [managed identity](https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview) for [authentication with the Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/authentication).
6666

67-
For secret rotation ideas, see [Secrets Rotation on Environment Variables and Mounted Secrets](secret-rotation-in-pods.md)
67+
For secret rotation ideas, see [Secrets Rotation on Environment Variables and Mounted Secrets](./secret-rotation-in-pods.md)
6868

6969
For how to authenticate private container registries with a service principal see: [Authenticated Private Container Registry](#authenticated-private-container-registry)
7070

docs/CI-CD/gitops/secret-management/azure-devops-secret-management-per-branch.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ When using [Azure DevOps Pipelines](https://azure.microsoft.com/en-us/services/d
66

77
- *Pipeline variables are global shared state.* This can lead to confusing situations and hard to debug problems when developers make concurrent changes to the pipeline variables which may override each other. Having a single global set of pipeline variables also makes it impossible for secrets to vary per environment (e.g. when using a branch-based deployment model where 'master' deploys using the production secrets, 'development' deploys using the staging secrets, and so forth).
88

9-
A solution to these limitations is to manage secrets in the Git repository jointly with the project's source code. As described in [secrets management](README.md), don't check secrets into the repository in plain text. Instead we can add an encrypted version of our secrets to the repository and enable our CI/CD agents and developers to decrypt the secrets for local usage with some pre-shared key. This gives us the best of both worlds: a secure storage for secrets as well as side-by-side management of secrets and code.
9+
A solution to these limitations is to manage secrets in the Git repository jointly with the project's source code. As described in [secrets management](./README.md), don't check secrets into the repository in plain text. Instead we can add an encrypted version of our secrets to the repository and enable our CI/CD agents and developers to decrypt the secrets for local usage with some pre-shared key. This gives us the best of both worlds: a secure storage for secrets as well as side-by-side management of secrets and code.
1010

1111
```sh
1212
# first, make sure that we never commit our plain text secrets and generate a strong encryption key

docs/CI-CD/recipes/ci-with-jupyter-notebooks.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@ This document aims to automate this process in Azure DevOps, so the DSs don't ne
1010
A Data Science repository has this folder structure:
1111

1212
```bash
13-
1413
.
1514
├── notebooks
1615
│   ├── Machine Learning Experiments - 00.ipynb
@@ -22,7 +21,6 @@ A Data Science repository has this folder structure:
2221
   ├── Machine Learning Experiments - 01.py
2322
   ├── Machine Learning Experiments - 02.py
2423
   └── Machine Learning Experiments - 03.py
25-
2624
```
2725

2826
The python files are needed to allow Pull Request reviewers to add comments to the notebooks, they can add comments

docs/CI-CD/recipes/github-actions/runtime-variables/README.md

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,13 @@ We assume that you, as a CI/CD engineer, want to inject environment variables or
1616

1717
Many integration or end-to-end workflows require specific environment variables that are only available at runtime. For example, a workflow might be doing the following:
1818

19-
![Workflow Diagram](images/workflow-diagram.png)
19+
![Workflow Diagram](./images/workflow-diagram.png)
2020

2121
In this situation, testing the pipeline is extremely difficult without having to make external calls to the resource. In many cases, making external calls to the resource can be expensive or time-consuming, significantly slowing down inner loop development.
2222

2323
Azure DevOps, as an example, offers a way to define pipeline variables on a manual trigger:
2424

25-
![AzDo Example](images/AzDoExample.PNG)
25+
![AzDo Example](./images/AzDoExample.PNG)
2626

2727
GitHub Actions does not do so yet.
2828

@@ -83,8 +83,6 @@ jobs:
8383
run: echo "Flag is available and true"
8484
```
8585
86-
Available as a .YAML [here](examples/commit-example.yaml).
87-
8886
Code Explanation:
8987
9088
The first part of the code is setting up Push triggers on the working branch and checking out the repository, so we will not explore that in detail.
@@ -153,7 +151,7 @@ Including the Variable
153151

154152
2. This triggers the workflow (as will any push). As the `[commit var]` is in the commit message, the `${COMMIT_VAR}` variable in the workflow will be set to `true` and result in the following:
155153

156-
![Commit True Scenario](images/CommitTrue.PNG)
154+
![Commit True Scenario](./images/CommitTrue.PNG)
157155

158156
Not Including the Variable
159157

@@ -167,7 +165,7 @@ Not Including the Variable
167165

168166
2. This triggers the workflow (as will any push). As the `[commit var]` is **not** in the commit message, the `${COMMIT_VAR}` variable in the workflow will be set to `false` and result in the following:
169167

170-
![Commit False Scenario](images/CommitFalse.PNG)
168+
![Commit False Scenario](./images/CommitFalse.PNG)
171169

172170
## PR Body Variables
173171

@@ -211,8 +209,6 @@ jobs:
211209
run: echo "Flag is available and true"
212210
```
213211

214-
Available as a .YAML [here](examples/pr-example.yaml).
215-
216212
Code Explanation:
217213

218214
The first part of the YAML file simply sets up the Pull Request Trigger. The majority of the following code is identical, so we will only explain the differences.
@@ -256,15 +252,15 @@ There are many real world scenarios where controlling environment variables can
256252

257253
Developer A is in the process of writing and testing an integration pipeline. The integration pipeline needs to make a call to an external service such as Azure Data Factory or Databricks, wait for a result, and then echo that result. The workflow could look like this:
258254

259-
![Workflow A](images/DevAWorkflow.png)
255+
![Workflow A](./images/DevAWorkflow.png)
260256

261257
The workflow inherently takes time and is expensive to run, as it involves maintaining a Databricks cluster while also waiting for the response. This external dependency can be removed by essentially mocking the response for the duration of writing and testing other parts of the workflow, and mocking the response in situations where the actual response either does not matter, or is not being directly tested.
262258

263259
### Skipping Long CI processes
264260

265261
Developer B is in the process of writing and testing a CI/CD pipeline. The pipeline has multiple CI stages, each of which runs sequentially. The workflow might look like this:
266262

267-
![Workflow B](images/DevBWorkflow.png)
263+
![Workflow B](./images/DevBWorkflow.png)
268264

269265
In this case, each CI stage needs to run before the next one starts, and errors in the middle of the process can cause the entire pipeline to fail. While this might be intended behavior for the pipeline in some situations (Perhaps you don't want to run a more involved, longer build or run a time-consuming test coverage suite if the CI process is failing), it means that steps need to be commented out or deleted when testing the pipeline itself.
270266

docs/CI-CD/recipes/github-actions/runtime-variables/examples/commit-example.yaml

Lines changed: 0 additions & 28 deletions
This file was deleted.

0 commit comments

Comments
 (0)