Skip to content

Conversation

TCeason
Copy link
Collaborator

@TCeason TCeason commented Aug 28, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

This PR add Plan::SecureFilter, a new operator that automatically applies row access policies defined in table metadata.

  1. Policy Enforcement: During query binding, if a table has a row_access_policy, a SecureFilter is automatically inserted directly above the table's Scan operator. This ensures the policy is applied early and specifically to that table.
  2. Stats Privacy: For SecureFilter, the optimizer will hide column-level statistics (e.g., histograms, NDV). This prevents accidental leakage of sensitive data patterns, even during query planning.
  3. Scoped Application: SecureFilter is marked as non-movable; optimizer rules will not push it above joins or merge it into generic filters. This guarantees the policy stays tied to its specific table.
  4. Runtime Filtering: At execution, SecureFilter evaluates its policy expression for each row, filtering out non-compliant rows, similar to a regular Filter.

Note:

Need to disable enable_planner_cache. Because stmt maybe same but table meta is different(add or drop a row access policy).

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Aug 28, 2025
This PR add Plan::SecureFilter, a new operator that automatically applies row access policies defined in table metadata.

1.  **Policy Enforcement:** During query binding, if a table has a `row_access_policy`, a `SecureFilter` is automatically inserted directly above the table's `Scan` operator. This ensures the policy is applied early and specifically to that table.
2.  **Stats Privacy:** For `SecureFilter`, the optimizer will **hide column-level statistics** (e.g., histograms, NDV). This prevents accidental leakage of sensitive data patterns, even during query planning.
3.  **Scoped Application:** `SecureFilter` is marked as non-movable; optimizer rules will *not* push it above joins or merge it into generic filters. This guarantees the policy stays tied to its specific table.
4.  **Runtime Filtering:** At execution, `SecureFilter` evaluates its policy expression for each row, filtering out non-compliant rows, similar to a regular `Filter`.
@TCeason TCeason marked this pull request as draft September 1, 2025 08:52
@TCeason TCeason marked this pull request as ready for review September 1, 2025 09:34
@TCeason TCeason requested a review from b41sh September 1, 2025 09:34
@TCeason TCeason merged commit 029101a into databendlabs:main Sep 2, 2025
90 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants