Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: change private attr and step getter #82

Merged
merged 66 commits into from
Oct 30, 2024
Merged

refactor: change private attr and step getter #82

merged 66 commits into from
Oct 30, 2024

Conversation

mikita-sakalouski
Copy link
Contributor

Description

Adjust Output class and fix getting of attributes from Step classes.

Related Issue

#81

Motivation and Context

Get correct behavior for class attributes and also raise Exception in proper manner

How Has This Been Tested?

Existing tests

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

mikita-sakalouski and others added 30 commits August 9, 2024 12:23
@mikita-sakalouski
fix: work in progress
2e86208
@mikita-sakalouski
Merge remote-tracking branch 'origin/main' into 33-feature-ensure-tha…
0aa6a53 
…t-we-can-support-dbr-143lts
@mikita-sakalouski
feat: add delta test skipping for 3.5
a9fe361
@mikita-sakalouski
refactor: update DataFrame imports to use koheesio.spark across the c…
f2ab79f 
…odebase
@mikita-sakalouski
chore: update dependencies and improve SQL step handling
f3e6014
@dannymeijer
Added show_string utility
f24ed3e
@mikita-sakalouski
refactor: replace spark_minor_version with SPARK_MINOR_VERSION consta…
ecbdc9a 
…nt for consistency
@dannymeijer
few more fixes
18ff011
@dannymeijer
Merge remote-tracking branch 'origin/33-feature-ensure-that-we-can-su…
8a84038 
…pport-dbr-143lts' into 33-feature-ensure-that-we-can-support-dbr-143lts
@dannymeijer
few more fixes
6a291b2
@mikita-sakalouski
refactor: update utility functions and improve test assertions for cl…
3bfe84a 
…arity
@dannymeijer
few more fixes
a3a5ad6
@dannymeijer
Down to the last 36 tests to fix
31bb7d7
@dannymeijer
fix typo
c837cd4
@mikita-sakalouski
refactor: streamline imports in row_number_dedup.py for clarity
3b09e5e
@mikita-sakalouski
refactor: enhance BoxCsvFileReader to use pandas for CSV parsing
ac9eee5
@dannymeijer
last 24
b360d8e
@dannymeijer
fix formatting
cfab89f
@dannymeijer
one more test
891c7f5
@dannymeijer
17 more remaining
349770c
@dannymeijer
Last 21
1b47c75
@dannymeijer
Last 21
4c93701
@dannymeijer
Last 20
1a77512
@dannymeijer
EOD
224c0cc
@mikita-sakalouski
fix: refactor connect types
42b7d86
@mikita-sakalouski
fix: improve session handling and type annotations in connect_utils a…
0c3b4f1 
…nd delta readers
@mikita-sakalouski
fix: improve tests
586d76a
@dannymeijer
snowflake refactoring (95% done)
46a18ca
@mikita-sakalouski
fix: adjust imports os spark connect
16290e3
@mikita-sakalouski
Merge branch '33-feature-ensure-that-we-can-support-dbr-143lts' of pe…
fd415ad 
…rsonal.github.com:Nike-Inc/koheesio into 33-feature-ensure-that-we-can-support-dbr-143lts
@mikita-sakalouski mikita-sakalouski requested a review from a team as a code owner October 23, 2024 14:24
@mikita-sakalouski mikita-sakalouski self-assigned this Oct 23, 2024
@mikita-sakalouski mikita-sakalouski added the bug Something isn't working label Oct 23, 2024
@mikita-sakalouski mikita-sakalouski linked an issue Oct 23, 2024 that may be closed by this pull request
[BUG] Step get attribute is passing on missed attributes and shadow excpetions #81
Open
dannymeijer
dannymeijer approved these changes Oct 23, 2024
View reviewed changes
Copy link
Member

@dannymeijer dannymeijer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@mikita-sakalouski mikita-sakalouski changed the title refactor: change private attr and output getter refactor: change private attr and step getter Oct 23, 2024
Base automatically changed from 33-feature-ensure-that-we-can-support-dbr-143lts to release/0.9 October 29, 2024 21:14
dannymeijer
dannymeijer requested changes Oct 30, 2024
View reviewed changes
src/koheesio/models/reader.py Outdated Show resolved Hide resolved
src/koheesio/spark/transformations/drop_column.py Outdated Show resolved Hide resolved
src/koheesio/spark/writers/delta/stream.py Outdated Show resolved Hide resolved
@dannymeijer dannymeijer merged commit 90e6462 into release/0.9 Oct 30, 2024
12 checks passed
@dannymeijer dannymeijer deleted the hotfix/step_output branch October 30, 2024 08:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dannymeijer dannymeijer dannymeijer approved these changes

Assignees

@mikita-sakalouski mikita-sakalouski

Labels
bug Something isn't working
Projects
Koheesio
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] Step get attribute is passing on missed attributes and shadow excpetions
2 participants
@mikita-sakalouski @dannymeijer