Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Stream map __alias__ is lost after first schema for a stream is processed #1521

Closed
1 task
edgarrmondragon opened this issue Mar 22, 2023 · 0 comments · Fixed by #1524
Closed
1 task
Assignees
Labels
kind/Bug Something isn't working valuestream/SDK

Comments

@edgarrmondragon
Copy link
Collaborator

Singer SDK Version

0.22.0

Is this a regression?

  • Yes

Python Version

NA

Bug scope

Mapping (stream maps, flattening, etc.)

Operating System

NA

Description

Reported by a user in Slack:

Within the current Singer SDK version, the PluginMapper class in mapper.py has this function register_raw_stream_schema where the code is 'popping' values from stream definition. I think the intention was the for loop here would copy the stream definition (stream_def), but that is not the case.

sdk/singer_sdk/mapper.py

Lines 668 to 670 in aee57cc

for stream_map_key, stream_def in self.stream_maps_dict.items():
stream_alias: str = stream_map_key
source_stream: str = stream_map_key

Within the for loop, the following is later executed, and it is altering the stream definition:

sdk/singer_sdk/mapper.py

Lines 699 to 707 in aee57cc

if MAPPER_SOURCE_OPTION in stream_def:
source_stream = stream_def.pop(MAPPER_SOURCE_OPTION)
if source_stream != stream_name:
# Not a match
continue
if MAPPER_ALIAS_OPTION in stream_def:
stream_alias = stream_def.pop(MAPPER_ALIAS_OPTION)

When I altered the for loop locally to make a copy of the stream definition, the alias configuration was maintained and we got the expected outcome.

This can be fixed by copying the stream_def dict.

Code

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/Bug Something isn't working valuestream/SDK
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant