Source code and documentation for the BSides Orlando 2022 (11/18 - 11/19) talk: DevSecOps on a $ store budget
Security tooling can be expensive, very expensive. Never fear though, there are a multitude of cheap and open source options out there. You too can build a robust DevSecOps pipeline on a dollar store budget.
Covering everything from open source SAST tools to free secret scanning and Infrastructure-as-Code audits, this talk walks you through options to build out CI/CD pipelines that help to secure your code base without taking out a second mortgage.
Some basic knowledge of DevOps and CI/CD would be useful but is not mandatory.
This talk will walk the audience through how to build a variety of simple CI/CD pipelines with Jenkins and GitHub Actions. The CI/CD pipeline will demonstrate how to incorporate a number of security and code analysis tools including:
-
Cloc - What’s in the repository?
-
Checkov - Bridgecrews free IaC scanner
-
PHPMetrics - considering Cyclomatic Complexity from a security perspective
-
Tortufo - GoDaddy’s free Secrets Scanning tool
-
Git-secrets - Prevent your AWS secrets from hitting the repository
-
CodeQL and Dependabot - free scanning of open source repos in GitHub
The talk will then wrap up by looking at Horusec an open source SAST platform that incorporates a variety of security and linting tools in a container based environment.
An example PHP application using Terraform Infrastructure-as-Code targeting an AWS environment will be used to demo the DevSecOps process to the audience. This code can be downloaded during the talk to allow participants to follow along in a hands on fashion.
-
Introduction
-
What is DevSecOps
-
Commercial tools aren't the only option
-
Jenkins overview
-
A walk through our example repository
-
Scanning a repository with cloc
-
Reviewing Terraform with Checkov
-
Auditing PHP with PHPMetrics
-
Scan for secrets using Tortufo
-
Pre-commit hooks with git-secrets
-
Open source Shift Left with GitHub
-
Wrapping up with Horusec
-
Closing statement
-
QA
The following guide provides instructions for setting up the Jenkins CI/CD pipeline for this DevSecOps project. Depending on the environment you chose to use for running Jenkins, you will need to install/setup a number of prequisite services prior to building your pipeline. For example when setting up the cloud infrastructure you will need to ensure you have Terraform installed locally, and if you wish to map a domain to Jenkins, ensure this is registered in Route 53.
Each section will provide a guide on the pre-requisites and the steps to get up and running.
To kick the the tires on the application and the pipeline you can install Jenkins locally.
Jenkins install instructions can be found at: https://www.jenkins.io/doc/book/installing/
The guide below covers local setup on personal devices such as Windows and Mac laptops.
If you want to follow along locally you will need:
-
Jenkins setup locally
-
A forked copy of the course code (this repository)
-
A bridgecrew account - you can set this up with your GitHub user: https://www.bridgecrew.cloud/
In addition to the above, there will be some local OS specific requirements, which are covered under the relevant section below.
Make sure you have brew installed on your Mac and that you have updated it recently.
With brew in place we can then beging by adding Jenkins.
Install the LTS version:
brew install jenkins-lts
Once installed, start it up e.g.
brew services start jenkins-lts
Navigate to:
http://localhost:8080
You will now be prompted to enter the default password. This will be stored in a location shared with you on the webpage and will be in a format similar to:
/Users/<user>/.jenkins/secrets/initialAdminPassword
Using this password you should now be logged in, and can configure Jenkins, change the password and get started.
The default plugin installation option presented initially should be sufficent for this talk, bar one missing plugin you will need. The default plugins installation wizard will also allow you to setup a new admin user.
Once this is complete, you will need to add the afore mentioned missing plugin, which is the Docker pipeline one.
Navigate to your Jenkins management view:
Dashboard > Manage Jenkins
From here select Plugin Manager.
From the list of Available plugins select Docker Pipeline and install this.
Dashboard > Manage Jenkins > Manage Plugins > Available (tab) > docker-workflow.
You should now be all set to start building CI/CD pipelines in Jenkins.
TBD
TBD
For users who wish to build clloud based infrastructure that can be shared among team members, these instructions guide you through deploying the Terraform code for setting up the environment, using Packer to create a pre-baked AMI and finally integrating your Jenkins cloud hosted environment with GitHub.
Currently AWS is the only cloud vendor supported, but the code could easy by expanded to support Microsoft Azure and GCP.
The AWS installation instructions walk you through how to setup an environment in AWS to hosted Jenkins. This includes:
-
Building a Jenkins AMI with baked in Docker support, which can be deployed on an EC2 instance
-
Terraform code that builds out:
a. The VPC and S3 buckets for storing state
b. Subnets (Private and Public)
c. Bastion host and security group
d. Jenkins servers and security group
e. ELB and security group
-
Manually adding in your domain name and certificate (and instructions on disabling this feature if you don't want to use it)
At the completion of the infrastructure setup you will have a fully built Jenkins CI/CD environment that can be easily added to or destroyed via Terraform.
To follow along in the cloud, you will need:
-
An AWS account
-
A forked copy of the source code
-
A bridgecrew account - you can set this up with your GitHub user
-
A domain name/subdomain configured as a Hosted zone in your AWS account. A number of services offer free domain registration or domains for as little as $0.01. Note: if you do not want to setup a domain and SSL cert, and map these to the load balancer, instructions will be provided on disabling this portion of the Terraform code.
Follow the steps here to manually create a subdomain:
https://aws.amazon.com/premiumsupport/knowledge-center/create-subdomain-route-53/
Then add the SSL cert like this:
-
Request a certificate
-
Add the reference to DNS
-
Copy the Cert ARN. You wll need to add this to your .tfsecrets file in the next stage
-
Cert can now be hooked up to the ELB via Terraform
A separate README is provided with a detailed description of the Terraform code and what it is does.
From a high-level the Terraform files are responsible for building:
-
The S3 bucket for storing state in AWS
terraform > tf-state -
The Jenkins server and associated infrastructure
terraform > network-infra -
The Jenkins AMI
packer -
The PHP server for running our vulnerable webapp.
terraform > app-server
In order to run the Terraform scripts make sure you have Terraform installed on the device you will be executing them from.
Instructions for Mac, Windows and Linux are located here: https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli
You will also need Packer installed to create the AMI. Instructions for this are located at: https://developer.hashicorp.com/packer/tutorials/docker-get-started/get-started-install-cli
Once installation is complete you are ready to begin.
The Terraform scripts will reference the AMI that you have Jenkins installed on. Included in this repository is a simple Packer configuration that will build the AMI and store it in your AWS account.
In order for this to work you will need to have a VPC already in place. This creates something of a chicken and egg problem, as in order to create the VPC for Jenkins, you need the AMI reference.
The easiest way to do this is to create a VPC manually through the web console, and then point Packer to it. For further information on this, please refer to: $
AWS Marketplace > Discover products
Use the Search AWS Marketplace products field to search for Jenkins.
There are multiple options, many of which would be very cheap or practically free for testing these instructions out.
Alternatively, you could create an AMI manually by installing Jenkins on an EC2 instance and taking an AMI snapshot.
For further instructions on this, refer to: https://www.jenkins.io/doc/tutorials/tutorial-for-installing-jenkins-on-AWS/
Search results
Follow the instructions at: https://github.com/moduslabs/dollarstore-devsecops-bsides-orlando-2022/blob/master/terraform/README.md then return back here.
(Optional) Next add the load balancer to the domain:
https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-elb-load-balancer.html
-
Make sure you used a forked version of this repository and cloned it (you'll need it again shortly)
-
Login to Jenkins
-
Call your pipeline bsidesorlando2022
-
Description - add a description of your choosing
-
Under pipeline select
Pipeline script from SCM -
Select
Gitfrom SCM and add the repo location. Note, if you have a private fork you will need to enter login credentials -
Add the path to the Jenkinsfile
-
Save the file.
-
Build Now -
Expect to see a few errors, as we need to generate an API key for Checkov.
Start with adding in an API key.
Add the Env var to Jenkins:
This project is parameterized
Select: Credentials parameter
Create a Jenkins credential:
Jenkins Credentials Provider: Jenkins
-
Domain, leave default
-
Kind - secret text
-
Scope: Global
-
Secret - paste API key
-
ID:
checkov-api-key -
Description - say what it is
Pop up closes, set the Default Value to the BSides API Key
Edit the Jenkins file and update the Checkov call to include your API key environment var name if you called it something different.
Run build with Parameters.
View Console log, should see some errors. We need to fix these!
Cloc can be used to ascertain what files existing in a rpeository. This can be a great and quick way of looking for things that shouldn't be in the repostiroy, and could contain security breaches. Examples include:
-
Word docs with passwords
-
Spreadsheets with passwords
-
Binary files with hard coded secrets
When this stage executes, you can see in the Jenkins console the findings.
Will see errors in Terraform files.
These are:
.
They are fixed by doing the following:
https://phpmetrics.org/index.html
If you installed PHPMetrics locally, we also installed composer alongside PHPMetrics, if you are using the AWS environment option, we have a container in place that executes PHPMetrics.
The key difference between the local and cloud installation is the Installation step, which isn't required in Jenkinsfile.aws since the container is leveraged.
Execute via Pipeline against target code base.
https://tartufo.readthedocs.io/en/stable/
tartufo.toml
Runs in pipeline or locally as pre-commit hook.
We are going to run the basic version without the supplementary container tools from inside Jenkins.
The instructions for this are located at: https://docs.horusec.io/docs/pt-br/tutorials/how-to-use-horusec-without-docker/
The supported scans are: https://docs.horusec.io/docs/pt-br/cli/analysis-tools/open-source-horusec-engine/
Based off of a simple tutorial here: https://code.tutsplus.com/tutorials/how-to-build-a-simple-rest-api-in-php--cms-37000
To add to this project please fork the repository and create a pull request.
Modus Create is a digital product consultancy. We use a distributed team of the best talent in the world to offer a full suite of digital product design-build services; ranging from consumer facing apps, to digital migration, to agile development training, and business transformation.
This project is part of Modus Labs.
This project is MIT licensed.