Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace print with log at different levels, and add the option to mask AWS credentials #50

Merged
merged 1 commit into from
Jul 2, 2020

Conversation

yaojiach
Copy link
Collaborator

@yaojiach yaojiach commented Jul 2, 2020

Replaced print with logging. User can set the logging level. Default setting ('info' level), will keep the current behavior.
Added feature for users to decide if they want to mask/show AWS credentials when related information is printed. New default behavior is that credentials will be masked.

Examples

pr.set_log_level('info')

This will print everything but AWS credentials will be masked:

copy "table"
from 's3'
delimiter ','
ignoreheader 1
csv quote as '"'
dateformat 'auto'
timeformat 'auto'
access_key_id '********'
secret_access_key '********'    
;

pr.set_log_level('info', mask_secrets=False)

Same as current behavior, AWS credentials will be printed as is.

copy "table"
from 's3'
delimiter ','
ignoreheader 1
csv quote as '"'
dateformat 'auto'
timeformat 'auto'
access_key_id 'AWS_KEY'
secret_access_key 'AWS_SECRET'    
;

pr.set_log_level('error')

This will only log errors when they occur.

@agawronski
Copy link
Owner

Thanks man!

@agawronski agawronski merged commit f649e38 into master Jul 2, 2020
@yaojiach yaojiach mentioned this pull request Jul 2, 2020
@agawronski
Copy link
Owner

Version 2.0.4 is up on PyPi, thanks again dude.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Don't log AWS Secrets when running COPY commands verbose flag + clean up temp files on S3
2 participants