Local content item support

[KludgeKML]

⚠️ This repo is Continuously Deployed: make sure you follow the guidance ⚠️

What

Adds more structured support for loading content items from the local machine. By setting an environment variable and putting a content item in a particular directory, it will be loaded from that file instead of from a call to the content store.

Why

There's currently a restricted version of this code working for Landing Pages (it only loads the blocks from the fixture, but includes fake values for the rest of the content item). This scaffolding has proved useful during Landing Pages development, but it's limited in scope and only works for Landing Pages. By replacing it with a more robust option, we can improve developer experience in the absence of a local version of content store, and make it easier to demo experimental content items on Heroku.

https://trello.com/c/FytCkByy/383-improve-contentitem-offline-loading

How

We add a new singleton class (ContentItemLoader) that is responsible for calls to GdsApi.content_store.get_content - this allows us to have a cache of items (simplifying the current Format Constraint code, which caches responses in the request env field), and to have a single point where those calls can be intercepted and swapped out for local code if the required environment variable is set. Setting ALLOW_LOCAL_CONTENT_ITEM_OVERRIDE to true will cause the loader to look in /config/local-content-items/path/to/the/item for any call to the content store (falling back to an actual call if the file isn't present).

See Also:

• Add ALLOW_LOCAL_CONTENT_ITEM_OVERRIDE=true to all frontend apps except -lite govuk-docker#793. (adds the env var to govuk-docker frontend apps)

[leenagupte]

I like the idea in principle. I have a few inline comments that I think need to be fixed before this will work.

[leenagupte]

It'd be good if there was an accompanying change in govuk-docker that sets this.

[KludgeKML]

alphagov/govuk-docker#793

[leenagupte]

Thanks for adding the govuk-docker PR. This env var should also be added to the --live option in the startup script.

[KludgeKML]

Done (and added it to app.json)

[leenagupte]

This doesn't look right. There isn't actually a class called ContentItemLoaderGdsApi is there?

The tests are passing, so either this is fine somehow, or this bit of code is not being tested.

[KludgeKML]

Yeah, not sure what was going on here. I've fixed it now!

[leenagupte]

I had a bit more time to review, so had a closer look at the ContentItemLoader class and added a few inline suggestions for refactoring it.

There's also a request for some tests around the scaffolding if this is going to become a permanent feature.

There's nothing major though and the overall approach is great 🎉

[leenagupte]

Thanks for adding the govuk-docker PR. This env var should also be added to the --live option in the startup script.

[leenagupte]

It occurs to me that you're doing this check twice. Once here and once in load_local_file.

It probably means that this class could do with a refactor.

Maybe split out the if clause in the load method and start from there?

e.g.

    def load(base_path)
      cache[base_path] ||= if use_local_file? && yaml_file?(base_path)
                             load_yaml_file(base_path)
                           elsif use_local_file? && json_file?(base_path)
                             load_json_file(base_path)
                           else
                             begin
                               GdsApi.content_store.content_item(base_path)
                             rescue GdsApi::HTTPErrorResponse, GdsApi::InvalidUrl => e
                               e
                             end
                           end
    end

Then you wouldn't need to redo the checks in the load_file methods.

It'd also allow you to split up the load_local_file method into the bits that load the file and create the gds-api response

    def load_yaml_file(base_path)
      file_name = yaml_filename(base_path)
      Rails.logger.debug("Loading content item #{base_path} from #{file_name}")
      contents = YAML.load(File.read(file_name)).to_json

      gds_api_response(contents)
    end

    def load_json_file(base_path)
      file_name = json_filename(base_path)
      Rails.logger.debug("Loading content item #{base_path} from #{file_name}")
      contents = File.read(file_name)

      gds_api_response(contents)
    end

    def gds_api_response(body)
      GdsApi::Response.new(OpenStruct.new(code: 200, body:, headers: { cache_control: "max-age=0, public", expires: "" }))
    end

I think also that limiting the use of local in the method names will make them a bit shorter and easier to read.

[leenagupte]

All of the methods apart from load should be made private as they're not being used or tested elsewhere.

[leenagupte]

It's great that you've fixed this. I think it would be good to add a test for this scaffolding too.

[KludgeKML]

The good news is that the existing request tests for landing_page already cover the old scaffolding (tests written... checks blame ... one @leenagupte)

[leenagupte]

They can't have been thorough enough because nothing caught the attempted call to ContentItemLoaderGdsApi.content_store.content_item(request.path).to_h which doesn't exist!