Allow disabling cache in OpenApiSpex.Plug.PutApiSpec

[hauleth]

Caching can be problematic in development and cleaning configuration after each recompilation can be troublesome.

Additionally such behaviour can be problematic when doing hot upgrades.

One possible solution is to store mod.module_info(:md5) along the cached data and drop everything on hash change.

[hauleth]

My current approach to that is to use persistent_term (OTP 21.2+ only) and @on_load callback in my controllers:

defmodule KokonRest.SpecCache do
  @moduledoc false

  @type operation_lookup :: %{optional(String.t) => OpenApiSpex.Operation.t()}
  @type spec :: %{
    spec: OpenApiSpex.OpenApi.t(),
    operation_lookup: operation_lookup()
  }

  @spec get_spec(module()) :: spec()
  def get_spec(mod) do
    case :persistent_term.get(__MODULE__, %{}) do
      %{^mod => spec} ->
        spec

      map ->
        spec = build_lookup(mod)

        :persistent_term.put(__MODULE__, Map.put(map, mod, spec))

        spec
    end
  end

  @spec clean() :: :ok
  def clean do
    if :persistent_term.erase(__MODULE__),
      do: :logger.info("Clearing spec cache")

    :ok
  end

  @spec build_lookup(module()) :: spec()
  defp build_lookup(mod) do
    spec = mod.spec()

    operation_lookup =
      for {_name, item} <- spec.paths,
          %OpenApiSpex.Operation{} = operation <- Map.values(item),
          into: %{},
          do: {operation.operationId, operation}

    %{spec: spec, operation_lookup: operation_lookup}
  end
end

And in my API controllers I have:

@on_load :__clean_cache__

def __clean_cache__, do: KokonRest.SpecCache.clean()

And it seems to work right now, even with Phoenix.CodeReload.

[dmt]

👍

I'm finding development pretty challenging at the moment. It seems essentially impossible to change a schema without also renaming it.

Maybe I'm missing something. What is the benefit of persisting this between restarts? Could this at least be a configuration option?

[hauleth]

@dmt this do not persist schema between restarts. This caches the resulting schema to not recalculate it between each request (pretty handy), but on the other hand it prevents any hot-reloads and live development (especially when used together with OpenApiSpex.Plug.CastAndValidate. My current approach, that can be seen in comment above, is to use persistent_term and clean it on each module reload. Alternative to that could be launching ETS table to cache builds that would be cleaned on given modules reload (like shown).

[dmt]

Thanks. I see now that my issues must be coming from somewhere else. I'll need to investigate more when I have the time.

[hauleth]

Another idea I have is to check if file (for example) priv/schema.json (format could be different, for example we could use ETF) and if it is present, then compile function directly, if not present, then generate schema on the fly. But to be honest the ETS/persistent_term seems simpler.

[moxley]

@hauleth

My current approach to that is to use persistent_term (OTP 21.2+ only)

I'm not sure you're aware, but we recently changed PutApiSpec to use persistent_term when it's available.

https://github.com/open-api-spex/open_api_spex/blob/master/lib/open_api_spex/plug/put_api_spec.ex#L53-L65

[hauleth]

@moxley it still do not saves the problem (oh and by the way, you do not need whole separate module, just if would be enough), it just changes the engine that will be used for caching.

[moxley]

@hauleth

it still do not saves the problem

Right, I'm aware of that. But you may want to merge that with your solution, since there is duplication involved.

[mbuhot]

Should be resolved by #262 in version 3.8.0