-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OC-926: Make ECS task run incremental ARI import #747
base: main
Are you sure you want to change the base?
Conversation
return { | ||
importAllDepartments: !!allDepartmentsArg, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These were actually always setting the property to true, so I changed the way they're set.
build: | ||
context: . | ||
command: | ||
["/bin/sh", "-c", "/app/infra/docker/ariImportRunner/local-ari-import.sh"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Override the container's command locally - only difference is to change the report format to a file because the container can't send an email.
@@ -19,8 +19,8 @@ resource "aws_codebuild_project" "deploy-docker-image" { | |||
|
|||
environment { | |||
compute_type = "BUILD_GENERAL1_SMALL" | |||
image = "aws/codebuild/standard:5.0" | |||
type = "LINUX_CONTAINER" | |||
image = "aws/codebuild/amazonlinux-aarch64-standard:3.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The codepipeline to build the image wasn't working until I changed this to build with an ARM based environment.
openssl \ | ||
openssl-dev \ | ||
libc6-compat | ||
RUN git clone https://github.com/JiscSD/octopus /app |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How come you're cloning the repo in the Dockerfile? This is already in the repo, so things could just be copied in
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's true, I'll try it with a copy
@@ -1,3 +1,4 @@ | |||
# syntax=docker/dockerfile:1.7-labs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haven't seen this before, what does it do?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was needed to use the --exclude flag with COPY.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# syntax=docker/dockerfile:1.7-labs
FROM public.ecr.aws/docker/library/node:20-alpine
RUN apk add \
ca-certificates \
curl \
gnupg \
git \
openssl \
openssl-dev \
libc6-compat
WORKDIR /app
COPY package.json .
RUN npm i
COPY . .
CMD ["npm", "run", "ariImport", "--", "dryRun=false", "reportFormat=email"]
Alternative way of doing it, removes the mkdir command, WORKDIR does this for you. Splits things into multiple steps so you don't have to redo npm i
everytime a file changes. Also seems to be better to set context root to the api folder, and only copy over the api stuff rather than everything (the above example assumes this change).
The purpose of this PR was to alter the existing proof-of-concept ECS task to run the ARI import process. This involved:
Tested on int environment by manually changing an ARI publication's title, then running the task via its API endpoint and confirming that the script ran, updated the title back to how it should be, and reported the results appropriately.
Acceptance Criteria:
The ECS task runs an incremental ARI import successfully.
Checklist:
Tests:
E2E
API
Screenshots:
UpdatedAt field is changed in the DB at the point the script was run and corrected the publication title.
Canonical DOI of publication is updated by the script after updating the publication title.
Publication has been reindexed in opensearch, and appears at the top because it has the most recent updatedAt value.
Task log
Output email sent to slack channel