Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Pydantic for validation and serialisation #1661

Merged
merged 67 commits into from
Jan 19, 2024
Merged
Changes from 1 commit
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
f3acf72
WIP: Use Pydantic for context settings
JimMadge Nov 7, 2023
12688ea
WIP: Fix ContextSettings tests
JimMadge Nov 7, 2023
562a6b4
Fix context command tests
JimMadge Nov 7, 2023
026f14a
Ensure selected context is defined
JimMadge Nov 7, 2023
aed1a86
Remove schema typings
JimMadge Nov 7, 2023
1abe112
Add test for invalid YAML
JimMadge Nov 13, 2023
7f314d5
Add test for YAML not being a dict
JimMadge Nov 13, 2023
d22bbd1
Add file not found test
JimMadge Nov 13, 2023
502fe5b
Fix linting errors
JimMadge Nov 13, 2023
9093b97
Add Pydantic to linting environment
JimMadge Nov 14, 2023
d374b1c
WIP: Introduce annotated types for validation
JimMadge Nov 15, 2023
7907e0c
WIP: Add Typer validators factory
JimMadge Nov 15, 2023
36f0e1c
Validate Context and ContextSettings on assignment
JimMadge Nov 15, 2023
1136b49
WIP: rewrite Config (and related) as BaseModel
JimMadge Nov 20, 2023
515668e
Update Context tests
JimMadge Nov 21, 2023
a741542
Remove ConfigSectionContext
JimMadge Nov 21, 2023
acb01b8
Improve design of config sections
JimMadge Nov 21, 2023
e7ca180
Add missing tests
JimMadge Nov 22, 2023
e8e2e5b
Add basic tests for Config
JimMadge Nov 22, 2023
8f98a9e
Add to_yaml method
JimMadge Nov 23, 2023
bde5fd4
Add upload and from remote tests
JimMadge Nov 23, 2023
e908111
Run lint:fmt
JimMadge Nov 23, 2023
e44acd6
Tidy Config
JimMadge Nov 23, 2023
6344a92
Correct sre method, add tests
JimMadge Nov 23, 2023
bf9435e
Dynamically derive config items from context
JimMadge Nov 24, 2023
c0002f6
Dynamically construct ConfigSectionTags
JimMadge Nov 24, 2023
bd358dc
Replace tags to_dict with model_dump
JimMadge Nov 24, 2023
e6eb86f
Add missing type annotation
JimMadge Nov 27, 2023
09e8427
Correct config attributes
JimMadge Nov 27, 2023
adbb464
Add missing properties to context
JimMadge Nov 27, 2023
c040e47
Update minimum Python to 3.11
JimMadge Nov 28, 2023
ba05dd7
Correct typer validator factory
JimMadge Nov 29, 2023
e1a0264
Support optional args in typer validator factory
JimMadge Nov 29, 2023
7a0cab8
Move Pulumi encryption key to Config property
JimMadge Nov 28, 2023
4727c32
Add template method
JimMadge Nov 28, 2023
36d05f3
Add config template command
JimMadge Nov 28, 2023
681379e
Add config upload command
JimMadge Nov 29, 2023
82a8d17
Add config show command
JimMadge Nov 29, 2023
b25f977
Run lint:fmt
JimMadge Nov 29, 2023
3235983
Remove dependency on config for context
JimMadge Nov 29, 2023
db7855a
Use new Config in deploy commands
JimMadge Nov 30, 2023
5fe21f8
Update Pulumi version draft README
JimMadge Nov 30, 2023
e6fb404
Use new config in teardown commands
JimMadge Dec 12, 2023
56ec257
Rename context infrastructure class
JimMadge Dec 12, 2023
9fd210c
Correct references
JimMadge Dec 12, 2023
829d7ac
Use new Config in admin commands
JimMadge Dec 12, 2023
16e73ff
Add help text for config show command
JimMadge Dec 14, 2023
c9ceb39
Fix import
JimMadge Dec 15, 2023
9b91daf
Allow no selected context
JimMadge Dec 15, 2023
9cea20e
Add test for constructor with no selected context
JimMadge Dec 15, 2023
c2f3de5
Add assert context tests
JimMadge Dec 15, 2023
4d291a1
Add test for selecting no context
JimMadge Dec 15, 2023
9321eb8
Add update test for no selected context
JimMadge Dec 15, 2023
8576f50
Add test for removing selected context
JimMadge Dec 15, 2023
3486091
Allow showing no selected context
JimMadge Dec 15, 2023
512f386
Add test for available with no selected context
JimMadge Dec 15, 2023
430926b
Add fqdn validation
JimMadge Jan 15, 2024
7d6c50d
Fix invalid fqdn in tests
JimMadge Jan 15, 2024
3990fdc
Fix test failing due to env variable
JimMadge Jan 15, 2024
26e64e0
Merge pull request #1691 from alan-turing-institute/selected
JimMadge Jan 15, 2024
625eb47
Merge pull request #1674 from alan-turing-institute/config
JimMadge Jan 15, 2024
1493dcb
Run lint:fmt
JimMadge Jan 15, 2024
50d83a9
Print Ruff version in CI
JimMadge Jan 15, 2024
672d7df
Fix linting errors
JimMadge Jan 15, 2024
3b5d552
Add fqdn typings
JimMadge Jan 16, 2024
2b9b268
Add typing hint
JimMadge Jan 16, 2024
ed4ec99
Update data_safe_haven/README.md
JimMadge Jan 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
WIP: Fix ContextSettings tests
JimMadge committed Nov 15, 2023
commit 12688ea160973f6343a2f811fef4ba271aeb20af
9 changes: 3 additions & 6 deletions data_safe_haven/config/context_settings.py
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@
)

from pathlib import Path
from typing import Any, ClassVar
from typing import ClassVar

import yaml
from pydantic import BaseModel, Field, ValidationError
@@ -136,11 +136,8 @@ def from_yaml(cls, settings_yaml: str) -> ContextSettings:
try:
return ContextSettings.model_validate(settings_dict)
except ValidationError as exc:
cls.logger.error(f"{exc.error_count()} errors found in context settings:")
for error in exc.errors():
cls.logger.error(f"{error['msg']} at '{'->'.join(error['loc'])}'")
msg = f"Could not load context settings.\n{exc}"
raise DataSafeHavenConfigError(msg) from exc
raise DataSafeHavenParameterError(msg) from exc

@classmethod
def from_file(cls, config_file_path: Path | None = None) -> ContextSettings:
@@ -165,5 +162,5 @@ def write(self, config_file_path: Path | None = None) -> None:
config_file_path.parent.mkdir(parents=True, exist_ok=True)

with open(config_file_path, "w", encoding="utf-8") as f_yaml:
yaml.dump(self.model_dump(), f_yaml, indent=2)
yaml.dump(self.model_dump(by_alias=True), f_yaml, indent=2)
self.logger.info(f"Saved context settings to '[green]{config_file_path}[/]'.")
24 changes: 17 additions & 7 deletions tests_/config/test_context_settings.py
Original file line number Diff line number Diff line change
@@ -41,23 +41,33 @@ def context_yaml():

@fixture
def context_settings(context_yaml):
return ContextSettings(yaml.safe_load(context_yaml))
return ContextSettings.from_yaml(context_yaml)


class TestContextSettings:
def test_constructor(self, context_yaml):
settings = ContextSettings(yaml.safe_load(context_yaml))
settings = ContextSettings(
selected="acme_deployment",
contexts={
"acme_deployment": Context(
name="Acme Deployment",
admin_group_id="d5c5c439-1115-4cb6-ab50-b8e547b6c8dd",
location="uksouth",
subscription_name="Data Safe Haven (Acme)",
)
},
)
assert isinstance(settings, ContextSettings)

def test_missing_selected(self, context_yaml):
context_yaml = "\n".join(context_yaml.splitlines()[1:])

with pytest.raises(DataSafeHavenParameterError) as exc:
ContextSettings(yaml.safe_load(context_yaml))
assert "Missing Key: 'selected'" in exc

def test_settings(self, context_settings):
assert isinstance(context_settings.settings, dict)
ContextSettings.from_yaml(context_yaml)
assert "Could not load context settings" in exc
assert "1 validation error for ContextSettings" in exc
assert "selected" in exc
assert "Field required" in exc

def test_selected(self, context_settings):
assert context_settings.selected == "acme_deployment"