importccl: lift computed column check to input converter #55845
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
|
That was fast! I'm curious if you see a way to get rid of https://github.com/cockroachdb/cockroach/pull/55845/files#diff-e56c3f528820bd6c95bc632c127d4ef1b5ae752a8accb667690a5e07c7c9b25eR137 as well. What are the cases where |
|
We expect that case when there are no "targetted" columns. In #55846 is built on top of this, and I think that should do the trick. I took a couple of quick profiles and it looks like that change got rid of VisibleColumns from the Row() profile as expected. |
pbardea
force-pushed
the
import-only-into
branch
2 times, most recently
from
92d82a2 to
555457d
Compare
pbardea
changed the title
pbardea
force-pushed
the
import-only-into
branch
from
555457d to
007bc36
Compare
dt
approved these changes
Oct 26, 2020
The current implementation of checking for validating the number of rows for computed columns for in non-IMPORT INTO backups is inefficient. This commit moves the check from being performed on every row to only being performed once per import. Release note: None
pbardea
force-pushed
the
import-only-into
branch
from
007bc36 to
caaf981
Compare
This commit moves the computation of finding the expected number of data columns to the creation of the input converter, rather than re-calculating it on every row. The memory footprint of loading all of the visible columns for every row was noticeable. Release note (performance improvement): CSV imports should now be slightly faster.
pbardea
force-pushed
the
import-only-into
branch
from
caaf981 to
6a9eaf3
Compare
|
TFTR! |
|
Build succeeded: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current implementation of checking for validating the number of rows
for computed columns for in non-IMPORT INTO backups is inefficient. This
commit moves the check from being performed on every row to only being
performed once per import.
Fixes #55842.
Release note: None