Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tianmu): support in-completed column data during load data. (#1209) #1273

Merged
merged 1 commit into from
Feb 20, 2023

Conversation

lujiashun
Copy link

@lujiashun lujiashun commented Feb 4, 2023

Summary about this PR

Issue Number: close #1209

  1. support few column;
  2. support too many column;
  3. support the data of column is not completed,such as
    3.1 such as start with the enclosed char, finished without enclosed char;
    3.2 without line-termination string, but reach to the end of file;

Tests Check List

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Changelog

  • New Feature
  • Bug Fix
  • Performance Improvement
  • Build/Testing/CI/CD
  • Documentation
  • Not for changelog (changelog entry is not required)

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features

@lujiashun lujiashun marked this pull request as draft February 4, 2023 01:49
@mergify
Copy link
Contributor

mergify bot commented Feb 4, 2023

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

@mergify mergify bot added the PR-feature feature for pull request label Feb 4, 2023
@Nliver Nliver added this to the stonedb_5.7_v1.0.3 milestone Feb 4, 2023
@lujiashun lujiashun force-pushed the feat-1209-stonedb5.7 branch 4 times, most recently from bf09580 to 66920b9 Compare February 4, 2023 04:20
@codecov
Copy link

codecov bot commented Feb 4, 2023

Codecov Report

Base: 43.12% // Head: 43.28% // Increases project coverage by +0.16% 🎉

Coverage data is based on head (7849009) compared to base (6a21184).
Patch coverage: 100.00% of modified lines in pull request are covered.

❗ Current head 7849009 differs from pull request most recent head 3908f50. Consider uploading reports for the commit 3908f50 to get more accurate results

Additional details and impacted files
@@                 Coverage Diff                 @@
##           stonedb-5.7-dev    #1273      +/-   ##
===================================================
+ Coverage            43.12%   43.28%   +0.16%     
===================================================
  Files                 1830     1830              
  Lines               396321   396166     -155     
===================================================
+ Hits                170897   171485     +588     
+ Misses              225424   224681     -743     
Impacted Files Coverage Δ
storage/tianmu/loader/parsing_strategy.h 100.00% <ø> (ø)
storage/tianmu/loader/load_parser.cpp 64.28% <100.00%> (-5.63%) ⬇️
storage/tianmu/loader/parsing_strategy.cpp 70.90% <100.00%> (+3.54%) ⬆️
storage/tianmu/core/mi_rough_sorter.cpp 0.00% <0.00%> (-98.08%) ⬇️
storage/tianmu/core/joiner_hash_table.cpp 0.00% <0.00%> (-91.73%) ⬇️
storage/tianmu/core/mi_rough_sorter.h 0.00% <0.00%> (-81.25%) ⬇️
storage/tianmu/core/joiner_hash_table.h 0.00% <0.00%> (-69.24%) ⬇️
storage/tianmu/core/joiner_hash.cpp 0.00% <0.00%> (-67.65%) ⬇️
storage/tianmu/core/joiner_hash.h 0.00% <0.00%> (-50.00%) ⬇️
storage/tianmu/loader/rejecter.cpp 9.09% <0.00%> (-50.00%) ⬇️
... and 107 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@lujiashun lujiashun force-pushed the feat-1209-stonedb5.7 branch 2 times, most recently from e675776 to 5047242 Compare February 6, 2023 11:20
@lujiashun lujiashun force-pushed the feat-1209-stonedb5.7 branch 4 times, most recently from 7849009 to facc44b Compare February 7, 2023 11:25
@lujiashun lujiashun marked this pull request as ready for review February 7, 2023 14:00
Copy link
Collaborator

@DandreChen DandreChen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lujiashun lujiashun force-pushed the feat-1209-stonedb5.7 branch from a699249 to 3862080 Compare February 20, 2023 07:31
…neatom#1209)

[summary]
1. support few column;
2. support too many column;
3. support the data of column is not completed,such as
3.1 such as start with the enclosed char, finished without enclosed char;
3.2 without line-termination string, but reach to the end of file;
@lujiashun lujiashun force-pushed the feat-1209-stonedb5.7 branch from 3862080 to 3908f50 Compare February 20, 2023 09:36
@mergify mergify bot merged commit a1d0ad2 into stoneatom:stonedb-5.7-dev Feb 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR-feature feature for pull request
Projects
Development

Successfully merging this pull request may close these issues.

bug: LOAD data statement with LINES TERMINATED return error
4 participants