Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 1227 - Allow materialisation of df_representatives with no _ suffix #1228

Merged
merged 1 commit into from
May 10, 2023

Conversation

RobinL
Copy link
Member

@RobinL RobinL commented May 10, 2023

Bug reported here:
#1227

The final table is called (templated name):
__splink__df_representatives
whereas intermedaite tables are called e.g.
__splink__df_representatives_1
__splink__df_representatives_2 etc.

This modifies the regex so __splink__df_representatives is also materialised

@RobinL RobinL linked an issue May 10, 2023 that may be closed by this pull request
@github-actions
Copy link
Contributor

Test: test_2_rounds_1k_duckdb

Percentage change: -16.0%

date time stats_mean stats_min commit_info_branch commit_info_id machine_info_cpu_brand_raw machine_info_cpu_hz_actual_friendly commit_hash
849 2022-07-12 18:40:05 1.89098 1.87463 splink3 c334bb9 Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz 2.7934 GHz c334bb9
1617 2023-05-10 12:03:49 1.6375 1.57496 (detached head) 52544b3 Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz 2.0951 GHz 52544b3

Test: test_2_rounds_1k_sqlite

Percentage change: -9.5%

date time stats_mean stats_min commit_info_branch commit_info_id machine_info_cpu_brand_raw machine_info_cpu_hz_actual_friendly commit_hash
851 2022-07-12 18:40:05 4.32179 4.25898 splink3 c334bb9 Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz 2.7934 GHz c334bb9
1619 2023-05-10 12:03:49 3.87473 3.85272 (detached head) 52544b3 Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz 2.0951 GHz 52544b3

Click here for vega lite time series charts

@RobinL RobinL merged commit f1236ff into master May 10, 2023
@RobinL RobinL deleted the 1227-df_representatives-not-materialised-in-spark branch August 12, 2024 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

df_representatives not materialised in Spark
2 participants