Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spec tests need to be version-locked on 1.5 branch #4824

Closed
schlessera opened this issue Jun 9, 2020 · 1 comment · Fixed by #4825
Closed

Spec tests need to be version-locked on 1.5 branch #4824

schlessera opened this issue Jun 9, 2020 · 1 comment · Fixed by #4825
Labels
Optimizer Testing Issues related with Unit, E2E, Smoke, and other testing requirements/needs
Milestone

Comments

@schlessera
Copy link
Collaborator

Bug Description

We are currently always pulling the latest spec files from the [ampproject/amp-toolbox](https://github.com/ampproject/amp-toolbox project) to run the language-agnostic spec tests against our optimizer implementation.

This also happens in the separate 1.5 branch that we have open for bug fixes. However, while this causes the 1.5 branch to always run against the latest spec file tests, this branch does not get the latest updates to the Optimizer that would fix the code to make updated tests pass if behavior has changed upstream.

Expected Behaviour

The 1.5 branch (and any future release branch that is split off of develop) should be version-locked into a specific set of spec files to test against that is known to work.

Steps to reproduce

  1. Check out commit 6f0ed6f
  2. Run composer install within lib/optimizer folder
  3. Run composer test within lib/optimizer folder

Screenshots

Image 2020-06-09 at 6 25 13 AM


Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

Implementation brief

QA testing instructions

Demo

Changelog entry

@schlessera schlessera added Optimizer Testing Issues related with Unit, E2E, Smoke, and other testing requirements/needs labels Jun 9, 2020
@schlessera schlessera added this to the v1.5.4 milestone Jun 9, 2020
@westonruter
Copy link
Member

Fixed by #4825 and #4826.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Optimizer Testing Issues related with Unit, E2E, Smoke, and other testing requirements/needs
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants