Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wip scope populator #3965

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

Conversation

jamesstarr
Copy link
Contributor

No description provided.

Adding SqlQueryScopes to encapsulate the data for resolving
SqlNode to SqlValidatorScope and SqlValidatorNamespaces.  Moving
the api for resolving scopes from SqlValidator to SqlQueryScopes.
… using

a generic Fuction in SqlValiator.Config.
Adding NamespaceBuilder to formalize the api for customizing/reusing
namespaces downstream.
@julianhyde
Copy link
Contributor

Looks pretty good.

I'm tempted to make the following next steps:

  • Rework it to make SqlValidator implement ScopeMap, and reduce the amount of noise. (I take your point that maybe the interfaces should be decoupled eventually.)
  • Make ScopeMapImpl package-private
  • Make ScopeMapImpl it immutable (with the mutable bits in a builder)
  • Address the Checkerframework issues (yes they are painful, but they generally point to real problems in lifecycle)

…is allows

for overriding validateFeature.

Removing unused getter from ValidatorAggStuff.
@jamesstarr
Copy link
Contributor Author

jamesstarr commented Sep 17, 2024

Looks pretty good.

I'm tempted to make the following next steps:

  • Rework it to make SqlValidator implement ScopeMap, and reduce the amount of noise. (I take your point that maybe the interfaces should be decoupled eventually.)
  • Make ScopeMapImpl package-private
  • Make ScopeMapImpl it immutable (with the mutable bits in a builder)
  • Address the Checkerframework issues (yes they are painful, but they generally point to real problems in lifecycle)

@julianhyde, the ScopeMap interface is already read-only, which provides most of the clarity we gain from immutability. The additional clarity from introducing an explicit immutable version of ScopeMap is marginal, while the complexity introduced is significant. This is due to your desire for a redelegation layer in SqlValidator, the existing circular dependencies between ScopeMap, namespaces, and SqlValidator, and the non-trivial methods in ScopeMapImpl, which need to support downstream overriding while not have code duplication. In addition, the scopeMap reference would have to mutate within SqlValidatorImpl, which would create a complex lifecycle and essentially negate the intended clarity.

Allowing downstream projects to override ScopeMapImpl is why it's currently package-scoped. Moreover, several methods in ScopeMap and ScopeMapImpl are public or protected, so we'd need to provide an API for overriding those methods moving forward, unless we plan to offer an upgrade path for projects that have customized them.

Given the complexity of these interactions and the time it would take to implement, I’m not willing to introduce an immutable version of ScopeMap at this time. However, the work on the ScopeMap interface will make it easier to add an immutable version later, once the circular dependencies are resolved.

I can add a redegalation layer to SqlValidator, but please be very explicit on why you want this, because I do not understand. All it seems to do is adding 200 lines of boilerplate while making api opaque and creating future work.

Copy link

@jamesstarr
Copy link
Contributor Author

@julianhyde Any thoughts on wether this is worth going forward?

@jamesstarr
Copy link
Contributor Author

@julianhyde Would it be ok if asked @mihaibudiu to review this?

@mihaibudiu
Copy link
Contributor

I can try to review it, but @julianhyde knows at least 10^6 times more about this software than I do

@jamesstarr
Copy link
Contributor Author

@mihaibudiu Lets wait for @julianhyde blessing to go forward before. I do not want to waste your time.

@mihaibudiu
Copy link
Contributor

Reading code is a great way of learning about how the system works, and since I only have a vague understanding of the validator, it could be useful to me. The main problem is that I couldn't come up with most arguments that @julianhyde made, because I don't understand many of the reasons behind the current design and their implications.

Copy link

github-actions bot commented Dec 5, 2024

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 90 days if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@calcite.apache.org list. Thank you for your contributions.

@github-actions github-actions bot added the stale label Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants