Return just the latest version of the packages if all=true is not set when proxy mode is enabled

[mrodm]

When using proxy mode, currently if all=true is set just the latest packages are returned.

In this PR, it is updated search endpoint to return all packages in case all=true is not set. It joins the packages returned locally plus the ones returned by the proxy EPR.

In case all=true is set, it will return just the latest packages resulting from merging the packages found locally plues the ones returned by the proxy EPR.

Examples:

• Running without any local package

 $ curl -s "http://localhost:8080/search?all=true&prerelease=true" | jq -r '.[]|"\(.name) - \(.version)"' | wc -l
4542
 $ curl -s "https://epr.elastic.co/search?all=true&prerelease=true" | jq -r '.[]|"\(.name) - \(.version)"' | wc -l
4542

 $ curl -s "http://localhost:8080/search?prerelease=true" | jq -r '.[]|"\(.name) - \(.version)"' | wc -l
240
12:38:21 mariorodriguez@atlantis:~/Coding/work/integrations/packages/elastic_package_registry @ main - 27f40f6c9/256e01144 * (git) 
 $ curl -s "https://epr.elastic.co/search?prerelease=true" | jq -r '.[]|"\(.name) - \(.version)"' | wc -l
240

 $ curl -s "http://localhost:8080/search?all=true&package=elastic_package_registry&prerelease=true" | jq -r '.[]|"\(.name) - \(.version)"' | wc -l
7
 $ curl -s "https://epr.elastic.co/search?all=true&package=elastic_package_registry&prerelease=true" | jq -r '.[]|"\(.name) - \(.version)"' | wc -l
7

• Running with 2 new local elastic_package_registry packages:

 $ curl -s "http://localhost:8080/search?all=true&prerelease=true" | jq -r '.[]|"\(.name) - \(.version)"' | wc -l
4544

 $ curl -s "https://epr.elastic.co/search?all=true&prerelease=true" | jq -r '.[]|"\(.name) - \(.version)"' | wc -l
4542

 $ curl -s "http://localhost:8080/search?all=true&package=elastic_package_registry&prerelease=true" | jq -r '.[]|"\(.name) - \(.version)"' | wc -l
9

 $ curl -s "https://epr.elastic.co/search?all=true&package=elastic_package_registry&prerelease=true" | jq -r '.[]|"\(.name) - \(.version)"' | wc -l
7

How to test

1. Build a package-registry docker image from this branch:

docker build -t docker.elastic.co/package-registry/package-registry:v1.21.0-rc1 .

2. To test without local packages:
   1. Run locally Elastic Package Registry configured with the proxy remote:

      docker run --rm -p 8080:8080 -e EPR_LOG_LEVEL=debug -e EPR_FEATURE_PROXY_MODE=true -e EPR_PROXY_TO=https://epr.elastic.co -it docker.elastic.co/package-registry/package-registry:v1.21.0-rc1

3. To test with local packages:
   1. Build some local packages from the integrations repository:
      1. Update manifest and changelog from a package
      2. Execute elastic-package build
      3. Folder where packages have been built needs to be used in the following command. Example: /home/mariorodriguez/Coding/work/integrations/build/packages/
   2. Run locally Elastic Package Registry configured with the proxy remote:

      docker run --rm -p 8080:8080 -e EPR_LOG_LEVEL=debug -e EPR_FEATURE_PROXY_MODE=true -e EPR_PROXY_TO=https://epr.elastic.co  -v /home/mariorodriguez/Coding/work/integrations/build/packages/:/packages/package-registry -it docker.elastic.co/package-registry/package-registry:v1.21.0-rc1

To test if local EPR returns the same exact packages, this script can be run:

#!/bin/bash

HOSTS="http://localhost:8080 https://epr.elastic.co"

QUERIES="all=true all=true&prerelease=true prerelease=true experimental=true"

for url in ${HOSTS}; do
    for query in ${QUERIES}; do
         echo "${url} ${query}"
         curl -s "${url}/search?${query}" | wc -l
    done
done

Relates #873

[jsoriano]

Good catch!

[jsoriano]

Do we have tests covered by this? I would be interested on seeing that we don't duplicate packages, but packages.Join should take care of this.

[mrodm]

There were no tests for this proxy mode in search endpoint.

I've added a test here 54d32e7 using proxy mode using all=true and not using it.

[jsoriano]

👍