Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added validation for all types of cfbs.json files and all fields #167

Merged
merged 64 commits into from
Dec 13, 2023
Merged

Added validation for all types of cfbs.json files and all fields #167

merged 64 commits into from
Dec 13, 2023

Conversation

olehermanse
Copy link
Member

@olehermanse olehermanse commented Dec 7, 2023 •
edited
Loading

No description provided.

@olehermanse
Moved module validation function into separate function
ba903ee 
Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Removed unused import
bed074d 
Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Renamed CFBSIndexError to CFBSValidationError
1b91d1f 
Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Changed to double quotes in validation errors to match JSON syntax
c0e1eca 
Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Refactored module alias validation to prepare for further changes
58434eb 
Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Added separate validation for required fields
2ded2a5 
Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Simplified validation functions now that required fields are handled …
0032e32 
…separately

Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Changed all validation functions which only care about a single modul…
3a82a9d 
…e object to only take that module as an argument

Only "dependencies" field needs to reference other modules in the dict.

Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Added basic validation of module names to shared validation function
40d3b2f 
Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
cfbs validate now more thoroughly validates various cfbs.json files
ba7f69d 
Changed so the same module validation logic is used for the
different keys ("build", "provides", "index"), and for the
different types of cfbs.json files ("policy-set", "module", "index").

Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
validate.py: Small formatting improvements
a8c1cc7 
Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Added warnings when cfbs.json files have unknown fields
045c906 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Removed assertions which are no longer valid
6addb4c 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Fixed typo in required fields logic
f180fef 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
validate.py: Fixed typo in alias validation
9d5a224 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Added raw_data property for working with the actual JSON data of a CF…
0f4649a 
…BSJson

Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Fixed issue with typechecking for index and provides fields
fa1a66b 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Fixed checking of aliases
ee8aac6 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Made it possible to only pass message to CFBSValidationError
77a320a 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Updated validation log messages for being used more generally
9deb960 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Prepared CFBSValidationError to be used with an index instead of a name
f656cfe 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Added validation for top level keys of cfbs.json
a36219c 
Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Removed a lot of the special handling for validation of index files
fe5d7de 
We're now using the same functions for validating all cfbs.json files,
including the index file.

Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Added "alias" as a known module key so it's recognized in validation
eb19557 
Ticket: CFE-4060
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Fixed the warning message for unknown keys
48218aa 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Validation: Added checking to keep required_fields in sync with globa…
7011dc3 
…l lists of accepted fields

Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Put the index in place of module name when module name is missing
d5e4659 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Improved validation error messages by catching the exception
d532da9 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Fixed validation error message for missing index
165437c 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Fixed validation error message for a required field missing inside mo…
02c0334 
…dules

Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse olehermanse removed the request for review from vpodzime December 8, 2023 16:43
@olehermanse olehermanse added the WIP Work in progress label Dec 8, 2023
larsewi
larsewi reviewed Dec 9, 2023
View reviewed changes
cfbs/cfbs_config.py Outdated Show resolved Hide resolved
olehermanse and others added 9 commits December 12, 2023 13:03
@olehermanse
Updated backwards compatibility tests to more strict validation
5f576e3 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse @larsewi
Applied suggestion from @larsewi's code review
c25dc57 
Co-authored-by: Lars Erik Wik <53906608+larsewi@users.noreply.github.com>
@olehermanse
Fixed 2 issues with looking for alias targets / dependencies
83fb681 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Added validation test for aliases
ee2de69 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
JSON.md: Removed trailing whitespace
6973e02 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Fixed validation for number of arguments for "run" build steps
af9210a 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Fixed validation for looking for dependencies of modules in "build" a…
81c4877 
…rray

Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Updated many "subdirectory" fields to conform with the new stricter v…
6d67490 
…alidation

Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
A tribute to Python 3.5
d5656f7 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse
Fixed validation for when "key" is not required in module input list …
8a7ca28 
…subtypes

Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse olehermanse changed the title Added a lot more validation for cfbs.json files Added validation for all types of cfbs.json files and all fields Dec 12, 2023
@olehermanse olehermanse removed the WIP Work in progress label Dec 12, 2023
craigcomstock
craigcomstock reviewed Dec 12, 2023
View reviewed changes
Copy link
Contributor

@craigcomstock craigcomstock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after 5f576e3 looks good 👍

tests/test_validate_index_alias.py Outdated Show resolved Hide resolved
craigcomstock
craigcomstock reviewed Dec 12, 2023
View reviewed changes
Copy link
Contributor

@craigcomstock craigcomstock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few warning message suggestions. Looks good.

cfbs/validate.py Outdated Show resolved Hide resolved
cfbs/validate.py Outdated Show resolved Hide resolved
cfbs/validate.py Outdated Show resolved Hide resolved
cfbs/validate.py Outdated Show resolved Hide resolved
olehermanse and others added 2 commits December 12, 2023 17:18
@olehermanse
test_validate_index_alias.py: Fixed name of one of the tests / functions
ba459ce 
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
@olehermanse @craigcomstock
Improved some validation warning messages
edd4a7b 
Code review suggestions from @craigcomstock

Co-authored-by: Craig Comstock <craig.comstock@northern.tech>
Signed-off-by: Ole Herman Schumacher Elgesem <ole.elgesem@northern.tech>
larsewi
larsewi approved these changes Dec 12, 2023
View reviewed changes
cfbs/validate.py Outdated Show resolved Hide resolved
@olehermanse @larsewi
Fixed typo / spacing by @larsewi
0e23a7b 
Co-authored-by: Lars Erik Wik <53906608+larsewi@users.noreply.github.com>
@olehermanse olehermanse merged commit 3e5b263 into cfengine:master Dec 13, 2023
7 checks passed
@olehermanse olehermanse deleted the validation branch December 13, 2023 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@larsewi larsewi larsewi approved these changes

@vpodzime vpodzime Awaiting requested review from vpodzime

@craigcomstock craigcomstock Awaiting requested review from craigcomstock

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@olehermanse @craigcomstock @larsewi