Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Snapshot parsing during migration #3022

Merged
merged 1 commit into from
Aug 18, 2024

Conversation

izeigerman
Copy link
Member

@izeigerman izeigerman commented Aug 18, 2024

Fix for:

>>> from sqlmesh.core.context import Context
>>>
>>> ctx = Context()
>>> ctx.migrate()
Traceback (most recent call last):
  File "/Users/georgesittas/Code/tobiko/sqlmesh/sqlmesh/core/state_sync/engine_adapter.py", line 1218, in migrate
    self._migrate_rows(promoted_snapshots_only)
  File "/Users/georgesittas/Code/tobiko/sqlmesh/sqlmesh/core/state_sync/engine_adapter.py", line 1317, in _migrate_rows
    snapshot_mapping = self._migrate_snapshot_rows(snapshots_to_migrate)
  File "/Users/georgesittas/Code/tobiko/sqlmesh/sqlmesh/core/state_sync/engine_adapter.py", line 1462, in _migrate_snapshot_rows
    _visit(root_snapshot_id, {})
  File "/Users/georgesittas/Code/tobiko/sqlmesh/sqlmesh/core/state_sync/engine_adapter.py", line 1380, in _visit
    snapshot = parsed_snapshots[snapshot_id]
  File "/Users/georgesittas/Code/tobiko/sqlmesh/sqlmesh/core/state_sync/engine_adapter.py", line 1762, in __getitem__
    snapshot = self.get(snapshot_id)
  File "/Users/georgesittas/Code/tobiko/sqlmesh/sqlmesh/core/state_sync/engine_adapter.py", line 1753, in get
    self._parsed_snapshots[snapshot_id] = Snapshot.parse_obj(raw_snapshot)
  File "/Users/georgesittas/Code/tobiko/sqlmesh/sqlmesh/utils/pydantic.py", line 181, in parse_obj
    super().model_validate(obj) if PYDANTIC_MAJOR_VERSION >= 2 else super().parse_obj(obj)  # type: ignore
  File "/Users/georgesittas/Code/tobiko/sqlmesh/.venv/lib/python3.10/site-packages/pydantic/main.py", line 503, in model_validate
    return cls.__pydantic_validator__.validate_python(
pydantic_core._pydantic_core.ValidationError: 1 validation error for Snapshot
updated_ts
  Field required [type=missing, input_value={'name': '"db"."sqlmesh_e...913', 'migrated': False}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
    ```

@izeigerman izeigerman requested a review from a team August 18, 2024 19:00
@@ -73,7 +73,7 @@ def migrate(state_sync, **kwargs): # type: ignore
"name": name,
"identifier": identifier,
"version": version,
"snapshot": snapshot,
"snapshot": json.dumps(parsed_snapshot),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of this the existing test didn't catch the issue.

@izeigerman izeigerman merged commit 1756ca0 into main Aug 18, 2024
21 checks passed
@izeigerman izeigerman deleted the fix-snapshot-parsing-when-migrating-rows branch August 18, 2024 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants