Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace Python print statements with logging to StreamHandler #2319

Closed
jim-sheldon opened this issue Dec 9, 2021 · 1 comment
Closed

Replace Python print statements with logging to StreamHandler #2319

jim-sheldon opened this issue Dec 9, 2021 · 1 comment
Assignees
@iamleeg @jim-sheldon
Labels
Bug Infrastructure Issues related to AWS/EKS/Docker etc P1: Launch blocker Needs fixing before we launch, schedule some time to investigate & fix Tech Debt
Milestone
Leonidas

Comments

@jim-sheldon
Copy link
Collaborator

Calling print in Python does not automatically write its output to stdout; when a script runs in Docker and hangs on a line, it can result in a timeout with incomplete (even no) output written to Cloudwatch logs.
We should use logging instead of print statements, and we need to use a StreamHandler to ensure each log statement gets flushed from the buffer and made visible in Cloudwatch.

https://docs.python.org/3/library/logging.handlers.html#streamhandler
@jim-sheldon jim-sheldon added P0: Critical path blocker Blocking testing or the launch critical path, drop everything to investigate & fix Infrastructure Issues related to AWS/EKS/Docker etc Bug Tech Debt labels Dec 9, 2021
@jim-sheldon jim-sheldon added this to the Lewis milestone Dec 9, 2021
@jim-sheldon jim-sheldon self-assigned this Dec 9, 2021
abhidg added a commit that referenced this issue Dec 13, 2021
@abhidg
geocoding: Return None if tile ID not found in admins collection
26b4c20 
* Possible fix for #1877
* Add logging using StreamHandler #2319
abhidg added a commit that referenced this issue Dec 13, 2021
@abhidg
geocoding: Return None if tile ID not found in admins collection
78f68f2 
* Possible fix for #1877
* Add logging using StreamHandler #2319
abhidg added a commit that referenced this issue Dec 13, 2021
@abhidg
geocoding: Return None if tile ID not found in admins collection
65c84dd 
* Possible fix for #1877
* Add logging using StreamHandler #2319
abhidg added a commit that referenced this issue Dec 13, 2021
@abhidg
geocoding: Return None if tile ID not found in admins collection
309a993 
* Possible fix for #1877
* Add logging using StreamHandler #2319
iamleeg pushed a commit that referenced this issue Dec 16, 2021
@abhidg @iamleeg
geocoding: Return None if tile ID not found in admins collection
3c63250 
* Possible fix for #1877
* Add logging using StreamHandler #2319
@joe-brilliant joe-brilliant added P1: Launch blocker Needs fixing before we launch, schedule some time to investigate & fix and removed P0: Critical path blocker Blocking testing or the launch critical path, drop everything to investigate & fix labels Jan 11, 2022
@joe-brilliant joe-brilliant modified the milestones: Lewis, Leonidas Feb 1, 2022
@iamleeg iamleeg self-assigned this Feb 15, 2022
@iamleeg
Copy link
Contributor

iamleeg commented Feb 15, 2022

Looks like there's still stuff to do in ingestion, I'll take that on.

iamleeg added a commit that referenced this issue Feb 15, 2022
@iamleeg
Replace print with logging where it makes sense #2319
8cbdeaa
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Correct use of logging in England parser #2319
df300f7
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Replace logger.warn with logger.warning throughout #2319
6a5d9ff
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Fix broken call to logger #2319
2c5fe56
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Redirect logger to stdout in test #2319
593a53c
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
define a local logging module to configure loggers uniformly #2319
21f9e8f
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Use shared logger definition everywhere #2319
c648835
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Test removal of import workaround. Tests still passed locally... #2319
8568238
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Test removal of import workaround. Tests still passed locally... #2319
889a1be
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Remove the imports hack in the rest of the parsers #2319
7dc4822
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Replace print with logging where it makes sense #2319
08dcc55
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Correct use of logging in England parser #2319
b17fee9
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Replace logger.warn with logger.warning throughout #2319
0526c80
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Fix broken call to logger #2319
cdd24fc
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Redirect logger to stdout in test #2319
8fcc726
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
define a local logging module to configure loggers uniformly #2319
88a6491
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Use shared logger definition everywhere #2319
9a97a26
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Test removal of import workaround. Tests still passed locally... #2319
4e8bf40
iamleeg added a commit that referenced this issue Feb 16, 2022
@iamleeg
Remove the imports hack in the rest of the parsers #2319
29ca1b4
@iamleeg iamleeg closed this as completed Feb 16, 2022
abhidg added a commit that referenced this issue Feb 18, 2022
@abhidg
Revert "Test removal of import workaround. Tests still passed locally…
f44aa1a 
…... #2319"

This reverts commit 4e8bf40.
abhidg added a commit that referenced this issue Feb 18, 2022
@abhidg
Revert "Remove the imports hack in the rest of the parsers #2319"
cb4232f 
This reverts commit 29ca1b4.
@abhidg abhidg mentioned this issue Feb 18, 2022
Merged
abhidg added a commit that referenced this issue Feb 18, 2022
@abhidg
Revert "Test removal of import workaround. Tests still passed locally…
c0a2126 
…... #2319"

This reverts commit 4e8bf40.
abhidg added a commit that referenced this issue Feb 18, 2022
@abhidg
Revert "Remove the imports hack in the rest of the parsers #2319"
a23f6d1 
This reverts commit 29ca1b4.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@iamleeg iamleeg

@jim-sheldon jim-sheldon

Labels
Bug Infrastructure Issues related to AWS/EKS/Docker etc P1: Launch blocker Needs fixing before we launch, schedule some time to investigate & fix Tech Debt
Projects
None yet
Milestone
Leonidas
Development

No branches or pull requests
3 participants
@iamleeg @jim-sheldon @joe-brilliant