Skip to content
This repository has been archived by the owner on Sep 23, 2024. It is now read-only.

Pipeline fails if file format is in a separate database #304

Open
gnilrets opened this issue Aug 19, 2022 · 0 comments
Open

Pipeline fails if file format is in a separate database #304

gnilrets opened this issue Aug 19, 2022 · 0 comments
Labels
bug Something isn't working

Comments

@gnilrets
Copy link

Describe the bug
I created a file format called PROD_RAW.PUBLIC.MELTANO_TARGET_SNOWFLAKE. However, I'm trying to load data into a development database called DEV_STERLINGP_RAW. I'm getting a "file format not found" error when I try to execute a pipeline, despite my user having USAGE permissions on this target.

To Reproduce
Steps to reproduce the behavior:

  1. Create file format in a database that is different from the target database
  2. Run the target
  3. See error target_snowflake.exceptions.FileFormatNotFoundException: Named file format not found: PROD_RAW.PUBLIC.MELTANO_TARGET_SNOWFLAKE

Expected behavior
Target should be able to find file formats in different databases.

Your environment

  • Version of target: v2.2.0
  • Version of python 3.8

Additional context
Looks like the target is stripping the database and schema name when it's looking for the file format. Presumably the connection is using the target database as default. Fixing this should be as simple as getting the db.schema and using SHOW FILE FORMATS LIKE '{file_format_name}' in schema {file_format_db_schema}

@gnilrets gnilrets added the bug Something isn't working label Aug 19, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant