Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix restore of parametrized views #201

Merged
merged 4 commits into from
Jan 4, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ build: install-deps ch_backup/version.txt
all: build lint test-unit test-integration

.PHONY: lint
lint: install-deps isort black codespell ruff pylint mypy bandit
lint: build isort black codespell ruff pylint mypy bandit

.PHONY: isort
isort: install-deps
Expand All @@ -75,7 +75,7 @@ ruff: install-deps
${TEST_ENV} ruff check $(SRC_DIR) $(TESTS_DIR)

.PHONY: pylint
pylint: install-deps
pylint: build
${TEST_ENV} pylint $(SRC_DIR)
${TEST_ENV} pylint --disable=missing-docstring,invalid-name $(TESTS_DIR)

Expand Down
5 changes: 4 additions & 1 deletion ch_backup/logging.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,10 @@ def configure(config_loguru: dict) -> None:
handler["filter"] = make_filter(name)
loguru_handlers.append(handler)

logger.configure(handlers=loguru_handlers, activation=[("", True)])
logger.configure(
handlers=loguru_handlers, # type: ignore
activation=[("", True)],
)

# Configure logging.
logging.basicConfig(handlers=[InterceptHandler()], level=0)
Expand Down
8 changes: 5 additions & 3 deletions ch_backup/logic/table.py
Original file line number Diff line number Diff line change
Expand Up @@ -843,12 +843,14 @@ def _restore_table_objects(

@staticmethod
def _restore_table_object(
context: BackupContext, db: Database, table: Table
context: BackupContext,
db: Database,
table: Table,
) -> None:
try:
if (
table.is_merge_tree()
or table.is_view()
or table.is_materialized_view()
or table.is_external_engine()
or table.is_distributed()
):
Expand All @@ -867,7 +869,7 @@ def _restore_table_object(
context.ch_ctl.create_table(table)
except Exception as e:
logging.debug(
f"Both table `{db.name}`.`{table.name}` restore methods failed. Removing it. Exception: {e}"
f"Failed to restore table `{db.name}`.`{table.name}`. Removing it. Exception: {e}"
)
if table.is_dictionary():
context.ch_ctl.drop_dictionary_if_exists(table)
Expand Down
2 changes: 1 addition & 1 deletion images/clickhouse/entrypoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

# In our test by default ch nodes have different root paths in the zookeeper(`/clickhouse01/` and `/clickhouse02/`).
# So they are not connected to each other.
# If you need the same path for nodes, use `step_enable_shared_zookeeper_for_clickhouse` step to override configs.
# If you need the same path for nodes, use `step_enable_shared_zookeeper_for_clickhouse` step to override configs.
client.ensure_path("/{{ instance_name }}")
client.ensure_path("/{{ conf.zk.shared_node }}")

Expand Down
5 changes: 5 additions & 0 deletions tests/integration/features/table_engines.feature
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,7 @@ Feature: Backup of tables with different engines and configurations
And we got same clickhouse data at clickhouse01 clickhouse02

@view
@require_version_23.3
Scenario: Create backup containing views
Given ClickHouse settings
"""
Expand Down Expand Up @@ -215,6 +216,10 @@ Feature: Backup of tables with different engines and configurations
CREATE LIVE VIEW test_db.live_view
AS SELECT n, n * n AS "n2"
FROM test_db.table_01;

CREATE VIEW test_db.parametrized_view
AS WITH {a:UInt32} AS a, {b:UInt32} AS b
SELECT a + b
"""
When we create clickhouse01 clickhouse backup
Then we got the following backups on clickhouse01
Expand Down
Loading