Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publication chain for Feast fronts #1565

Merged
merged 27 commits into from
May 7, 2024
Merged

Publication chain for Feast fronts #1565

merged 27 commits into from
May 7, 2024

Conversation

fredex42
Copy link
Contributor

@fredex42 fredex42 commented Apr 24, 2024

Note Requires https://github.com/guardian/editorial-tools-platform/pull/770 as well

What's changed?

Previously, the Editions workflow only targeted one product - the Editions app. This meant outputting "standard" issue fronts into S3 from where they are picked up by the Editions app backend.

Now, we are extending the UI to use the same process for the Feast app. This has a different requirement; the front must have its format converted into the form that the Feast app expects and it needs to be sent over to the Feast app backend via SNS.

Another difference is in "proofing". This is an integral part of the Editions publication process, but is not (currently) supported by the Feast app. In-app support is necessary for it to work. An extra flag has therefore been added to the Edition object, requiresProofing, which tells both frontend and backend if the Proofing stage is required.

  • When this is true (always the case for Editions app; the logic here sets the flag to true if the template is defined in the list of Editions app templates) then the UI shows the Proof button, proofing is required to publish and the Publish action publishes the last proofed version.
  • When this is false (currently true for non-Editions app templates) then the UI does not show the Proof button and the proofing process is short-circuited to directly output the current state of the Edition.
  • There is therefore no change in the Editions app workflow, and the Proofing stage is effectively hidden for Feast

This PR makes the relevant changes, by extracting the generic parts of the publication chain into Traits, which are then extended by concrete implementations for both Editions and Feast.

How will this data be used?

See guardian/recipes-backend#36

How to test in CODE

  • Create a new Feast edition
  • Add some recipes to the All Recipes and Meat-Free fronts
  • Hit Check to ensure that it's all ready to go
  • Hit Publish. Make a mental note of the time. You should see the status is Started (currently this won't change as the feedback loop is not present)
  • Log into the CMS Fronts account via AWS console
  • Open up the SQS console
  • Go to the topic feast-facia-test. This is subscribed to the SNS topic that the app writes to
  • Click Send and Receive Messages
  • In the lower pane, click Poll for messages
  • You should see a message appear with a timestamp about when you hit 'Publish'
  • Open it up to inspect. Drag the text box so you can see the whole thing
  • You should see your front as Feast-style data, double-encoded as a JSON object within the Message field. Something like this:
  "Message" : "{\"all-recipes\":[{\"id\":\"f36ee25d-54f5-4b63-9998-df8cb424f13f\",\"title\":\"Dish of the day\",\"body\":\"\",\"items\":[{\"recipe\":{\"id\":\"14049060\"}}]},{\"id\":\"fa9d34d5-ffa4-4caf-9647-2e785f0f7a89\",\"title\":\"Collection 2\",\"body\":\"\",\"items\":[{\"recipe\":{\"id\":\"14002695\"}},{\"recipe\":{\"id\":\"13891262\"}}]},{\"id\":\"6ab6b4a9-f02a-435c-999c-96145eb383e7\",\"title\":\"Collection 3\",\"body\":\"\",\"items\":[{\"recipe\":{\"id\":\"14077819\"}}]}],\"meat-free\":[{\"id\":\"45be31c5-931c-4c85-a77d-86ebff3eaf0a\",\"title\":\"Dish of the day\",\"body\":\"\",\"items\":[{\"recipe\":{\"id\":\"13898146\"}},{\"recipe\":{\"id\":\"13891262\"}}]},{\"id\":\"1e9e56ea-cc91-4ac6-8847-11fcf2b6c091\",\"title\":\"Collection 2\",\"body\":\"\",\"items\":[{\"recipe\":{\"id\":\"13726752\"}},{\"recipe\":{\"id\":\"14048612\"}}]}]}",

Note the Feast-style formatting, article internal IDs in place of recipe IDs

Implementation notes

  • The EditionsPublishing class (containing generic publication code) has been renamed to simply Publishing
  • EditionsPublishing used to directly reference EditionsBucket instances for pushing to Preview and Live locations. EditionsBucket now inherits from a PublicationTarget trait, in common with the new FeastPublicationTarget implementation
  • A new PublicationTargetWithTransform layer is also put in, that represents any potential publication target which needs to have the data converted from the Fronts format into 'something' else
  • The transform is implemented for Feast (though this is a dummy implementation at present, because the relevant data is not yet in the upstream Fronts tool)

Held over for future PRs

  • It seems strange to have strongly-typed output methods for PublicationTargets alongside one which just takes a String (which has been JSON-encoded by the caller). Changing this is out of scope for this PR, however.

Checklist

General

  • 🤖 Relevant tests added
  • ✅ CI checks / tests run locally
  • 🔍 Checked on CODE

Client

  • 🚫 No obvious console errors on the client (i.e. React dev mode errors)
    • Note, there is a warning on initial load of editorial fronts 404 accessing https://fronts.code.dev-gutools.co.uk/undefined. This is also present on main branch so leaving it out of scope here.
  • 🎛️ No regressions with existing user interactions (i.e. all existing buttons, inputs etc. work)
  • 📷 Screenshots / GIFs of relevant UI changes included

@fredex42 fredex42 requested a review from a team as a code owner April 24, 2024 16:20
@fredex42 fredex42 marked this pull request as draft April 24, 2024 16:20
@fredex42 fredex42 marked this pull request as ready for review April 29, 2024 13:06
@fredex42 fredex42 force-pushed the ag/feast-publication branch 2 times, most recently from 9c0db8b to 2c92681 Compare May 1, 2024 10:41
Copy link
Contributor

@jonathonherbert jonathonherbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing this afternoon – a few comments on the code as I do that!

app/controllers/EditionsController.scala Show resolved Hide resolved
app/controllers/EditionsController.scala Outdated Show resolved Hide resolved
app/services/editions/publishing/Publishing.scala Outdated Show resolved Hide resolved
conf/logback.xml Show resolved Hide resolved
@jonathonherbert
Copy link
Contributor

Tested in CODE, I can see publish messages on the queue 🎉

Before we merge: there's an odd error when editing collections in Editions App issues: S3 is spitting out errors at preview, giving 'request too big'. I've deployed main and don't see these errors. Happy to help investigate tomorrow.

@fredex42
Copy link
Contributor Author

fredex42 commented May 2, 2024

S3 errors now appear to be fixed

Copy link
Contributor

@jonathonherbert jonathonherbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested proofing and publishing again against Editions in CODE, LGTM 👍

@fredex42 fredex42 merged commit 9c4cca1 into main May 7, 2024
6 checks passed
@fredex42 fredex42 deleted the ag/feast-publication branch May 7, 2024 13:50
@prout-bot
Copy link

Seen on PROD (merged by @fredex42 8 minutes and 48 seconds ago) Please check your changes!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants