Python 3.8 missing library anticrlf

[windhamwong]

The python script using the anticrlf library:
https://github.com/apache/incubator-heron/blob/2190502da0ad723db86a13216f5d9acd0b4c6474/heron/shell/src/python/handlers/downloadhandler.py

It is breaking the whole thing with error:

[2021-12-09 07:37:40 +0000] [INFO]: heron-shell-0 stdout:
[2021-12-09 07:37:50 +0000] [INFO]: Running heron-shell-0 process as ./heron-core/bin/heron-shell --port=6004 --log_file_prefix=log-files/heron-shell-0.log --secret=biohacker-leslie74e3139c-cc07-40d3-83b8-a42a458582fd
[2021-12-09 07:37:50 +0000] [INFO]: Logging pid 225 to file heron-shell-0.pid
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout: Traceback (most recent call last):
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/heron/.pex/unzipped_pexes/df2f773e4ac2c0ebc390f03014ddd6abc6f6b98b/.bootstrap/pex/pex.py", line 476, in execute
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/heron/.pex/unzipped_pexes/df2f773e4ac2c0ebc390f03014ddd6abc6f6b98b/.bootstrap/pex/pex.py", line 401, in _wrap_coverage
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     return runner(*args)
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/heron/.pex/unzipped_pexes/df2f773e4ac2c0ebc390f03014ddd6abc6f6b98b/.bootstrap/pex/pex.py", line 432, in _wrap_profiling
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     return runner(*args)
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/heron/.pex/unzipped_pexes/df2f773e4ac2c0ebc390f03014ddd6abc6f6b98b/.bootstrap/pex/pex.py", line 532, in _execute
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     return self.execute_entry(self._pex_info.entry_point)
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/heron/.pex/unzipped_pexes/df2f773e4ac2c0ebc390f03014ddd6abc6f6b98b/.bootstrap/pex/pex.py", line 679, in execute_entry
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     return self.execute_module(entry_point, alter_sys)
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/heron/.pex/unzipped_pexes/df2f773e4ac2c0ebc390f03014ddd6abc6f6b98b/.bootstrap/pex/pex.py", line 691, in execute_module
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     runpy.run_module(module_name, run_name="__main__", alter_sys=alter_sys)
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/usr/lib/python3.8/runpy.py", line 207, in run_module
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     return _run_module_code(code, init_globals, run_name, mod_spec)
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     _run_code(code, mod_globals, init_globals,
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     exec(code, run_globals)
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/heron/.pex/unzipped_pexes/df2f773e4ac2c0ebc390f03014ddd6abc6f6b98b/heron/shell/src/python/main.py", line 30, in <module>
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     from heron.shell.src.python import handlers
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/heron/.pex/unzipped_pexes/df2f773e4ac2c0ebc390f03014ddd6abc6f6b98b/heron/shell/src/python/handlers/__init__.py", line 19, in <module>
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     from .downloadhandler import DownloadHandler
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:   File "/heron/.pex/unzipped_pexes/df2f773e4ac2c0ebc390f03014ddd6abc6f6b98b/heron/shell/src/python/handlers/downloadhandler.py", line 26, in <module>
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout:     import anticrlf
[2021-12-09 07:37:51 +0000] [INFO]: heron-shell-0 stdout: ModuleNotFoundError: No module named 'anticrlf'

[nicknezis]

It looks like we added anticrlf to heron-shell for security reasons in this PR. #3718 But then removed it here 10 days ago: #3746 Is the desire to have the library used? The fix might be to add the proper Python dependency (as opposed to reverting the security fix). Looks like this is the library. I'll try adding to the BUILD file as a dependency and test. https://pypi.org/project/logging-formatter-anticrlf/#history What do you guys think? Just wanted to make sure we're all on the same page with this.

…

On Fri, Dec 10, 2021 at 6:05 AM Josh Fischer ***@***.***> wrote: Closed #3745 <#3745> via #3746 <#3746>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3745 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABMNKXULWLU5S4E7LW4JODUQHNGVANCNFSM5JVY44KA> .

[nicknezis]

I fixed the issue in this PR. https://github.com/apache/incubator-heron/pull/3749/files Nick On Sat, Dec 18, 2021 at 6:36 PM Nicholas Nezis ***@***.***> wrote:

…

It looks like we added anticrlf to heron-shell for security reasons in this PR. #3718 But then removed it here 10 days ago: #3746 Is the desire to have the library used? The fix might be to add the proper Python dependency (as opposed to reverting the security fix). Looks like this is the library. I'll try adding to the BUILD file as a dependency and test. https://pypi.org/project/logging-formatter-anticrlf/#history What do you guys think? Just wanted to make sure we're all on the same page with this. On Fri, Dec 10, 2021 at 6:05 AM Josh Fischer ***@***.***> wrote: > Closed #3745 <#3745> via > #3746 <#3746>. > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#3745 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AABMNKXULWLU5S4E7LW4JODUQHNGVANCNFSM5JVY44KA> > . >