Skip to content

Commit

Permalink
Merging preview to ITHC (#324)
Browse files Browse the repository at this point in the history
* Sidm 3294 mfa e2e tests (#256)

* add mfa otp login tests

* add missing file

* fix typo

* add missing code

* fix test failures

* add more waits

* add more waits

* add exiplit wait

* add block policy scenario

* Sidm 3296 waf password configure test (#259)

* special character password test

* added password characters

* changed password characters

* redued wait time

* SIDM-3127 Remove “Is there a problem with this page?” element. (#257)

* feat(SIDM-3128-survey): add pageurl parameter to smartsurvey link (#258) (#268)

* feat(make): add port-forward and force-update-pods (#269)

* feat(SIDM-2040-pw): update password reset messages (#266)

* feat(SIDM-2040-pw): update password reset messages:
- blacklisted: Your password is too easy to guess
- containing personal info: Do not include your name or email in your password
- did not match rules: Your password didn't have all the required characters

* feat(SIDM-3410-ips): filter out internal ips from policy valuation (#270) (#271)

* feat(SIDM-3410-ips): filter out internal ips from policy valuation

* feat(SIDM-3410-ips): simplify and merge methods

* feat(SIDM-3410-ips): simplify regex

* feat(SIDM-3410-ips-preview): remove filter pattern in preview (#272)

* Updating with for suggestions (#273)

* Updating with for suggestions

* Need to escape the dot

* fix(vnet_private_ip_pattern): update escape syntax (#281)

* feat(SIDM-3441-sso): Policy eval: remove bearer auth token (#282)

* feat(SIDM-3437-redir): login/mfa: redirecting using slash (#280)

* feat(SIDM-3410-fix-def): use same escaping as preview which is tested (#285)

* This should fix the ArrAffinity token problem (#278)

* feat(SIDM-2040-pw-create): update create pwd (#279)

* feat(SIDM-2040-pw): update create pwd

* feat(SIDM-2040-pw-register): update create pwd on register user to handle FR errors:
- blacklisted password
- password contains personal info

* add password validation tests (#289)

* feat(SIDM-3128-surveyfix): fix survey url and allow only client_id param (#290)

* SIDM-3397 Expired Reset Password link. (#288)

* SIDM-3397 Expired Reset Password link.

* SIDM-3397 Add test.

* SIDM-3397 Fix after merge.

* Code review adjustements.

* Don't show the hyperlink when the data is not provided.

* SIDM-3499 Upgrade insights agent (#292)

* SIDM-3499 Upgrade Application Insights Agent.

* Attempt to make one of the tests less flaky.

* Fix the web config file.

* fix(docker): update dockerfile from base and use app insights 2.5.1

* Add the agent.

* Fix.

* SIDM-3499 Adjust link caption. (#293)

* Adjust link caption.

* Adjust link caption.

* Rename message names.

* Update contact us details. (revert of the revert) (#294)

* login with spaces in user email (#296)

* SIDM-3411 Springboot whitelabel error (#295)

* Add an AppErrorController.

* Add a redirect.

* Add a generic error jsp.

* Review fixes.

* Duplicated code fix (#291)

* user eal hmcts policyset in the tests (#298)

* Sidm 3530 illegal chars (#299)

* Add illegal characters validation.

* Add the message.

* Improve test coverage and enhance the for loop.

* Update working hours. (#301)

* SIDM 3487 suspended users login message (#303)

* SIDM-3487 Incorrect error message when suspended users try to log in.

* Empty commit

* Trigger a build.

* Upgrade tomcat minor version.

* Adjust the code.

* Revert "Adjust the code."

This reverts commit 3a1f4cd

* SIDM-3591 Update Contact Us page with email details for SSCS. (#304)

* Bringing all project dependencies in line for 1.5.1. (#309)

* Bringing all project dependencies in line for 1.5.1.

* Fixing a test

Co-authored-by: NikolaNaydenov <47004340+NikolaNaydenov@users.noreply.github.com>

* Sidm 3557 nightly functional tests (#305)

* updated code

* added both smoke and functuonal tests to  ightly pipeline

* add reset password with diff case email test (#310)

* Moving the sonar setup here to be in line with idam-api (#311)

Co-authored-by: NikolaNaydenov <47004340+NikolaNaydenov@users.noreply.github.com>

* Sidm 3118 fix flapping tests (#312)

* fix flapping tests

* fix flapping tests

* update wait time in the tests

* fix review comment

* Eliminating vulnerabilities

* fix unknown char in email links (#317)

* SIDM-3511 - Update chart-java release 2.16.0 (#318)

* build(chart-java): update chart-java release to 2.16.0

* refactor(cicd): remove deprecated enableDockerBuild()

* feat(staging deployment): add aat values for helm

* feat(sidm-3483-fr6): 6.5 web-public (#315) (#321)

* feat(sidm-3483-fr6): update test regex to extract activation parssword

* feat(sidm-3483-fr6): password reset working but had to change some of the codecepts waits

* feat(sidm-3483-fr6): add 2s wait after Sign In to fix flappy test

* feat(sidm-3483-fr6): remove waits because it was fixed by Shravs changes on codecept conf

* chore(ase asp): disable legacy deployments (#323)

Co-authored-by: Shravan Mechineni <shravanmechineni5@gmail.com>
Co-authored-by: sudhasane <vanisekhar75@gmail.com>
Co-authored-by: dfourn <dpatynski@gmail.com>
Co-authored-by: tbamido <50667636+tbamido@users.noreply.github.com>
Co-authored-by: Henry Dobson <henrydobson@me.com>
Co-authored-by: NikolaNaydenov <47004340+NikolaNaydenov@users.noreply.github.com>
  • Loading branch information
7 people authored Feb 14, 2020
1 parent b0eeb2e commit 0c66817
Show file tree
Hide file tree
Showing 45 changed files with 773 additions and 356 deletions.
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
ARG APP_INSIGHTS_AGENT_VERSION=2.4.0
ARG APP_INSIGHTS_AGENT_VERSION=2.5.1

FROM hmctspublic.azurecr.io/base/java:openjdk-8-distroless-1.1
FROM hmctspublic.azurecr.io/base/java:openjdk-8-distroless-1.4

LABEL maintainer=IDAM \
owner="HM Courts & Tribunals Service"
Expand All @@ -14,7 +14,7 @@ ENV SERVER_PORT=8080

ADD --chown=hmcts:hmcts build/libs/idam-web-public.war \
lib/AI-Agent.xml \
lib/applicationinsights-agent-2.4.0.jar /opt/app/
lib/applicationinsights-agent-2.5.1.jar /opt/app/

CMD ["-Dspring.profiles.active=docker,local", "idam-web-public.war"]

Expand Down
3 changes: 2 additions & 1 deletion Jenkinsfile_CNP
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,10 @@ static LinkedHashMap<String, Object> secret(String secretName, String envVar) {

withPipeline(type, product, component) {
loadVaultSecrets(secrets)
enableDockerBuild()
enableSlackNotifications('#idam_tech')
installCharts()
enableAksStagingDeployment()
disableLegacyDeployment()

// AKS Callbacks
before('akschartsinstall') {
Expand Down
68 changes: 54 additions & 14 deletions Jenkinsfile_nightly
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
#!groovy

properties([
pipelineTriggers([cron('10 21 * * *')]),
pipelineTriggers([cron('10 21 * * *')]),

parameters([
parameters([

string(name: 'URL_TO_TEST', defaultValue: 'https://idam-web-public.aat.platform.hmcts.net', description: 'The URL you want to run these tests against'),
string(name: 'URL_TO_TEST', defaultValue: 'https://idam-web-public-idam-preview.service.core-compute-idam-preview.internal', description: 'The URL you want to run these tests against'),
string(name: 'API_URL_TO_TEST', defaultValue: 'https://idam-api-idam-preview.service.core-compute-idam-preview.internal', description: 'The API URL you want to run these tests against '),
])
])

Expand All @@ -17,21 +18,60 @@ def product = "idam"

def component = "web-public"

def secrets = [
'idam-idam-preview': [
secret('smoke-test-user-username', 'SMOKE_TEST_USER_USERNAME'),
secret('smoke-test-user-password', 'SMOKE_TEST_USER_PASSWORD'),
secret('notify-api-key', 'NOTIFY_API_KEY')
]
]

static LinkedHashMap<String, Object> secret(String secretName, String envVar) {
[$class : 'AzureKeyVaultSecret',
secretType : 'Secret',
name : secretName,
version : '',
envVariable: envVar
]
}

withNightlyPipeline(type, product, component) {

after('DependencyCheckNightly') {
env.TEST_URL = params.URL_TO_TEST
enableSecurityScan()

sh "./gradlew --no-daemon --init-script init.gradle pitest"
env.TEST_URL = params.URL_TO_TEST

env.IDAMAPI = params.API_URL_TO_TEST

loadVaultSecrets(secrets)

enableFullFunctionalTest(200)

archiveArtifacts '**/build/reports/pitest/**/*'
after('fullFunctionalTest') {

withSonarQubeEnv("SonarQube") {
sh "./gradlew --info sonarqube"
}
sh "./gradlew smoke"

archiveArtifacts '**/build/test-results/**/*'

publishHTML target: [
allowMissing : true,
alwaysLinkToLastBuild: true,
keepAll : true,
reportDir : "output",
reportFiles : "idam-web-public-e2e-result.html",
reportName : "IDAM Web Public E2E smoke tests result"
]

sh "./gradlew functional"

archiveArtifacts '**/build/test-results/**/*'

publishHTML target: [
allowMissing : true,
alwaysLinkToLastBuild: true,
keepAll : true,
reportDir : "output",
reportFiles : "idam-web-public-e2e-result.html",
reportName : "IDAM Web Public E2E functional tests result"
]
}


}

85 changes: 63 additions & 22 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,26 +1,42 @@
.DEFAULT_GOAL := all
CHART := idam-web-public
RELEASE := ${CHART}-pr-207
RELEASE := ${CHART}-pr-${PR}
NAMESPACE := idam
TEST := ${RELEASE}
TEST := ${RELEASE}-test-service
ACR := hmctspublic
ACR_SUBSCRIPTION := DCD-CNP-DEV
ACR_SUBSCRIPTION := DCD-CNP-Dev
AKS_RESOURCE_GROUP := cnp-aks-rg
AKS_CLUSTER := cnp-aks-cluster

HELM_INSTALLED := $(command -v helm)
UNAME := $(uname)

# Usage example: make <command> PR='123'

setup:
az account set --subscription ${ACR_SUBSCRIPTION}
az configure --defaults acr=${ACR}
az acr helm repo add
az aks get-credentials --resource-group ${AKS_RESOURCE_GROUP} --name ${AKS_CLUSTER}
- @if [ -z "${HELM_INSTALLED}" ] && [[ "${UNAME}" == 'Darwin' ]]; then \
brew install helm ; \
elif $$(helm version | grep -q 'v2'); then \
brew upgrade helm ; \
fi
@az account set --subscription ${ACR_SUBSCRIPTION}
@az configure --defaults acr=${ACR}
@az acr helm repo add
@az aks get-credentials --resource-group ${AKS_RESOURCE_GROUP} --name ${AKS_CLUSTER}
- @if [ ! -d $${HOME}/.helm ]; then \
helm init --client-only ; \
fi

clean:
- helm delete --purge ${RELEASE} || echo "Release not found"
- for i in $$(kubectl -n chart-tests get rs -o name | grep ${RELEASE}); do \
kubectl -n chart-tests delete $${i} --grace-period=0 --force ; \
- @helm uninstall --namespace ${NAMESPACE} ${RELEASE}
- @for i in $$(kubectl -n ${NAMESPACE} get deploy -o name | grep ${RELEASE}); do \
kubectl -n ${NAMESPACE} delete $${i} --grace-period=0 --force ; \
done
- @for i in $$(kubectl -n ${NAMESPACE} get rs -o name | grep ${RELEASE}); do \
kubectl -n ${NAMESPACE} delete $${i} --grace-period=0 --force ; \
done
- for i in $$(kubectl -n ${NAMESPACE} get pod -o name | grep ${RELEASE}); do \
kubectl -n ${NAMESPACE} delete $${i} --grace-period=0 --force ; \
- @for i in $$(kubectl -n ${NAMESPACE} get pod -o name | grep ${RELEASE}); do \
kubectl -n ${NAMESPACE} delete $${i} --grace-period=0 --force ; \
done

update:
Expand All @@ -30,28 +46,53 @@ lint:
helm lint charts/${CHART}

template:
helm template charts/${CHART}
helm template ${RELEASE} --set "java.releaseNameOverride=${RELEASE}" --namespace ${NAMESPACE} charts/${CHART}

dry-run:
helm install charts/${CHART} --name ${RELEASE} --namespace ${NAMESPACE} -f ci-values.yaml --dry-run --timeout 30 --atomic
helm install ${RELEASE} --set "java.releaseNameOverride=${RELEASE}" --namespace ${NAMESPACE} --dry-run --timeout 30s --atomic charts/${CHART}

deploy:
helm install charts/${CHART} --name ${RELEASE} --namespace ${NAMESPACE} --wait --timeout 30
helm install ${RELEASE} --set "java.releaseNameOverride=${RELEASE}" --set "java.replicas=1" --namespace ${NAMESPACE} --wait --timeout 8m charts/${CHART}

test:
helm test charts/${RELEASE}
helm test ${RELEASE} --namespace ${NAMESPACE}

force-update-pods:
@kubectl -n ${NAMESPACE} scale --current-replicas=2 --replicas=0 deploy/idam-api
@kubectl -n ${NAMESPACE} patch deploy idam-api -p '{"spec":{"template":{"spec":{"containers":[{"name":"idam-api", "imagePullPolicy": "Always"}]}}}}'
@kubectl -n ${NAMESPACE} scale --current-replicas=0 --replicas=2 deploy/idam-api
@kubectl -n ${NAMESPACE} scale --current-replicas=2 --replicas=0 deploy/idam-web-public
@kubectl -n ${NAMESPACE} patch deploy idam-web-public -p '{"spec":{"template":{"spec":{"containers":[{"name":"idam-web-public", "imagePullPolicy": "Always"}]}}}}'
@kubectl -n ${NAMESPACE} scale --current-replicas=0 --replicas=2 deploy/idam-web-public
@kubectl -n ${NAMESPACE} scale --current-replicas=1 --replicas=0 deploy/idam-web-admin
@kubectl -n ${NAMESPACE} patch deploy idam-web-admin -p '{"spec":{"template":{"spec":{"containers":[{"name":"idam-web-admin", "imagePullPolicy": "Always"}]}}}}'
@kubectl -n ${NAMESPACE} scale --current-replicas=0 --replicas=1 deploy/idam-web-admin
@echo Done

logs:
@echo "Use the spacebar to page and 'q' to exit."
@sleep 2
@kubectl -n ${NAMESPACE} logs $$(kubectl -n ${NAMESPACE} get deployments -o name | grep ${RELEASE} | awk NR==1) | more

# make port-forward PR='257'
port-forward:
@echo -e "Killing kubectl pids on 8080.\nStarting port-forward.\nCtrl-C to exit."
@kill $$(lsof -i tcp:8080 | grep kubectl | awk '{print $$2}') 2>&1 || echo 'No processes to kill.'
@sleep 1
@kubectl -n ${NAMESPACE} port-forward deployment/${RELEASE} 8080:8080 &
@open 'http://localhost:8080/login?client_id=test-public-service&redirect_uri=https://test-public-service.com'

show-dep:
kubectl -n ${NAMESPACE} describe $$(kubectl -n ${NAMESPACE} get deployments -o name | grep ${RELEASE})
deployment:
@kubectl -n ${NAMESPACE} describe $$(kubectl -n ${NAMESPACE} get deployments -o name | grep ${RELEASE})

show-pod:
kubectl -n ${NAMESPACE} describe $$(kubectl -n ${NAMESPACE} get pods -o name | grep ${RELEASE})
pods:
@kubectl -n ${NAMESPACE} describe $$(kubectl -n ${NAMESPACE} get pods -o name | grep ${RELEASE})

events:
kubectl -n ${NAMESPACE} describe $$(kubectl -n ${NAMESPACE} get pods -o name | grep ${RELEASE}) | grep -A 3 Events
@kubectl -n ${NAMESPACE} describe $$(kubectl -n ${NAMESPACE} get pods -o name | grep ${RELEASE}) | grep -A 3 Events

redeploy: clean deploy

all: setup update clean lint deploy test

.PHONY: setup clean lint deploy test all
.PHONY: setup clean lint deploy test all
33 changes: 9 additions & 24 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ import java.util.stream.Collectors
plugins {
id 'java'
id 'jacoco'
id 'io.spring.dependency-management' version '1.0.8.RELEASE' apply false
id 'io.spring.dependency-management' version '1.0.9.RELEASE' apply false
id 'org.owasp.dependencycheck' version '5.1.1'
id 'org.sonarqube' version '2.6.2'
id 'org.springframework.boot' version '2.1.6.RELEASE' apply false
id 'org.springframework.boot' version '2.2.4.RELEASE' apply false
id 'com.gorylenko.gradle-git-properties' version '1.4.21'
id "info.solidsoft.pitest" version "1.3.0"
id 'pmd'
Expand All @@ -31,8 +31,8 @@ allprojects {
sourceCompatibility = 1.8
targetCompatibility = 1.8

def idamBomVersion = '1.9.0'
ext['tomcat.version'] = '9.0.22'
def idamBomVersion = '1.9.7'
ext['tomcat.version'] = '9.0.30'

dependencyManagement {
imports {
Expand Down Expand Up @@ -65,7 +65,8 @@ allprojects {
implementation group: 'org.springframework.boot', name: 'spring-boot-starter-actuator'
implementation group: 'org.springframework.boot', name: 'spring-boot-starter-web'
implementation group: 'org.springframework.boot', name: 'spring-boot-starter-security'
implementation (group: 'org.springframework.cloud', name: 'spring-cloud-starter-netflix-zuul') {
// TODO: remove version once 2.2.2.RELEASE is out
implementation (group: 'org.springframework.cloud', name: 'spring-cloud-starter-netflix-zuul', version: '2.2.1.RELEASE') {
exclude(module: 'rxnetty-contexts')
exclude(module: 'rxnetty-servo')
exclude(module: 'rxnetty')
Expand All @@ -79,8 +80,7 @@ allprojects {

implementation group: 'javax.servlet', name: 'jstl'
implementation group: 'javax.json', name: 'javax.json-api'
// TODO remove this hardcoded version when all of jackson > 2.9.9
implementation group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.9.9.3'
implementation group: 'com.fasterxml.jackson.core', name: 'jackson-databind'
implementation group: 'org.apache.httpcomponents', name: 'httpclient'
implementation group: 'org.apache.httpcomponents', name: 'httpcore'
implementation group: 'org.apache.commons', name: 'commons-text'
Expand All @@ -99,7 +99,7 @@ allprojects {

testAnnotationProcessor("org.projectlombok:lombok")

testImplementation group: 'org.mockito', name: 'mockito-core', version: '2.8.9'
testImplementation group: 'org.mockito', name: 'mockito-core'
testImplementation group: 'org.springframework.boot', name: 'spring-boot-devtools'
testCompile(group: 'org.springframework.boot', name: 'spring-boot-starter-test') {
exclude(module: 'commons-logging')
Expand Down Expand Up @@ -171,7 +171,6 @@ def listFiles(String pattern) {
sonarqube {
properties {
property "sonar.projectName", "SIDAM-WEB-PUBLIC"
property "sonar.jacoco.reportPath", "${listFiles('**/test.exec')}"
property "sonar.exclusions", "**/uk/gov/hmcts/reform/idam/web/config/properties/*.java," +
"**/uk/gov/hmcts/reform/idam/web/model/*.java," +
"**/uk/gov/hmcts/reform/idam/web/helper/MvcKeys.java," +
Expand All @@ -180,6 +179,7 @@ sonarqube {
property "sonar.host.url", "https://sonar.reform.hmcts.net/"
property "sonar.pitest.mode", "reuseReport"
property "sonar.pitest.reportsDirectory", "build/reports/pitest"
property "sonar.coverage.jacoco.xmlReportPaths", "${jacocoTestReport.reports.xml.destination.path}"
}
}

Expand Down Expand Up @@ -208,21 +208,6 @@ jacocoTestReport {
html.enabled = true
html.destination = file("${buildDir}/reports/jacoco")
}

additionalSourceDirs = files(sourceSets.main.allSource.srcDirs)
sourceDirectories = files(sourceSets.main.allSource.srcDirs)
classDirectories = files(sourceSets.main.output)
println 'class directories: ' + classDirectories.asPath
afterEvaluate {
classDirectories = files(classDirectories.files.collect {
fileTree(dir: it,
exclude: ['**/uk/gov/hmcts/reform/idam/web/config/properties/*',
'**/uk/gov/hmcts/reform/idam/web/model/*',
'**/uk/gov/hmcts/reform/idam/web/helper/MvcKeys**',
'**/uk/gov/hmcts/reform/idam/web/Application**',
'**/*Exception**'])
})
}
}

test.finalizedBy jacocoTestReport
Expand Down
2 changes: 1 addition & 1 deletion charts/idam-web-public/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v1
appVersion: "1.0"
description: A Helm chart for HMCTS Reform IDAM Web Public
name: idam-web-public
version: 0.2.2
version: 0.2.3
maintainers:
- name: Amido Reform SIDAM Team
email: reform.idam@HMCTS.NET
Expand Down
2 changes: 1 addition & 1 deletion charts/idam-web-public/requirements.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
dependencies:
- name: java
version: ~2.11.1
version: ~2.16.0
repository: '@hmctspublic'
6 changes: 6 additions & 0 deletions charts/idam-web-public/values.aat.template.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
java:
releaseNameOverride: ${SERVICE_NAME}
image: ${IMAGE_NAME}
ingressHost: ${SERVICE_FQDN}
ingressIP: ${INGRESS_IP}
consulIP: ${CONSUL_LB_IP}
2 changes: 1 addition & 1 deletion charts/idam-web-public/values.preview.template.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
java:
releaseNameOverride: ${SERVICE_NAME}
image: ${IMAGE_NAME}
replicas: 2
replicas: 1
ingressHost: ${SERVICE_FQDN}
ingressIP: ${INGRESS_IP}
consulIP: ${CONSUL_LB_IP}
7 changes: 6 additions & 1 deletion charts/idam-web-public/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,9 @@ java:
devmemoryRequests: '512Mi'
devcpuRequests: '1000m'
devmemoryLimits: '1024Mi'
devcpuLimits: '2500m'
devcpuLimits: '2500m'

global:
tenantId: "531ff96d-0ae9-462a-8d2d-bec7c0b42082"
environment: preview
enableKeyVaults: true
6 changes: 3 additions & 3 deletions codecept.conf.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ exports.config = {
bootstrap: false,
helpers: {
Puppeteer: {
//show: true,
// show: true,
url: TestData.WEB_PUBLIC_URL,
waitForTimeout: 60000,
waitForAction: 100,
getPageTimeout: 90000,
waitForAction: 2000,
getPageTimeout: 20000,
chrome: {
//args: ["--proxy-server=" + process.env.PROXY_SERVER],
ignoreHTTPSErrors: true
Expand Down
Binary file removed lib/applicationinsights-agent-2.4.0.jar
Binary file not shown.
Binary file added lib/applicationinsights-agent-2.5.1.jar
Binary file not shown.
Loading

0 comments on commit 0c66817

Please sign in to comment.