Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revisit PHP representation of AMP validator spec #2769

Closed
westonruter opened this issue Jul 8, 2019 · 3 comments
Closed

Revisit PHP representation of AMP validator spec #2769

westonruter opened this issue Jul 8, 2019 · 3 comments
Assignees
Labels
Enhancement New feature or improvement of an existing one P2 Low priority Sanitizers WS:Perf Work stream for Metrics, Performance and Optimizer

Comments

@westonruter
Copy link
Member

The AMP validator spec is converted from protoascii into PHP via the bin/amphtml-update.py Python script. This file tries to only extract the information that is needed for the tag-and-attribute sanitizer, but still, it is large: 451KB—so large in fact that it crashes phpcs: #2767 (comment). Just requiring this file incurs memory usage of +4MB. We could look at splitting up file by tag so that we only load the data for tags that are actually encountered.

Stepping back a bit further, we should also consider whether the Python script is the best way to extract the validator spec into PHP. It turns out that @fstanis has worked on making available the validator spec in JSON format: ampproject/amphtml#22528. This would mean at the very least that we could rewrite the spec extraction logic in PHP (or even JS) instead of Python (which has the required protobuf library, though there is probably a PHP protobuf library that could have been used instead). In any case, it is much more comfortable to work with JSON than protoascii, as long as there is no loss of fidelity in the conversion, which is done in validator/validator_gen_js.py:

  rules = validator_pb2.ValidatorRules()
  text_format.Merge(open(specfile).read(), rules)
  out.append(json_format.MessageToJson(rules))

Another benefit here is this would avoid us having to download the entire amphtml repo, since the entire spec in JSON format is always available at https://cdn.ampproject.org/v0/validator.json

This JSON file is only ~250KB as opposed to an archive export of the amphtml repo which is 100MB+.

@westonruter westonruter added this to the v2.0 milestone Jul 8, 2019
@fstanis
Copy link

fstanis commented Jul 8, 2019

If you go for a JS solution, ampproject/amp-toolbox#377 might be of interest.

It's meant to be a general purpose querying library for the validator rules. It'll be minimal at first, but you're free to extend it to include the functionality you need.

@schlessera
Copy link
Collaborator

Related #3730

@amedina amedina removed this from the v2.0 milestone Mar 31, 2020
@amedina amedina added the P2 Low priority label May 14, 2020
@kmyram kmyram added the WS:Perf Work stream for Metrics, Performance and Optimizer label Aug 5, 2020
@westonruter
Copy link
Member Author

Implemented in ampproject/amp-toolbox-php#100

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or improvement of an existing one P2 Low priority Sanitizers WS:Perf Work stream for Metrics, Performance and Optimizer
Projects
None yet
Development

No branches or pull requests

6 participants