Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve the stdout parsing #174

Merged
merged 11 commits into from
Sep 8, 2022
Merged

improve the stdout parsing #174

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
e089eeb
ignore urls without ports and use urllib to get it
keewis Aug 18, 2022
3204783
add an additional log file with contamination for testing
keewis Aug 20, 2022
9b202da
Merge branch 'main' into parse-stdout
keewis Aug 20, 2022
95016a6
extend the regular expression
keewis Aug 20, 2022
ebf1047
use urllib.parse.parse_qs instead of manually splitting the query
keewis Aug 20, 2022
2d04a4a
tighten the regex a bit
keewis Aug 20, 2022
875d4f4
back to just the filter by port
keewis Aug 22, 2022
a0e3e7b
Merge branch 'main' into parse-stdout
andersy005 Aug 23, 2022
20f82fc
also expect the port to be a int in the parse_log_file test
keewis Aug 24, 2022
b32aa99
use put_file instead of echo + redirect
keewis Aug 24, 2022
e6803d4
Merge branch 'main' into parse-stdout
andersy005 Sep 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions jupyter_forward/helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,11 +73,13 @@ def parse_stdout(stdout: str) -> dict[str, str]:
)
for url in urls:
url = url.strip()
if '127.0.0.1' not in url:
result = urllib.parse.urlparse(url)
hostname, port = result.netloc.split(':')
if 'token' in result.query:
token = result.query.split('token=')[-1].strip()
result = urllib.parse.urlparse(url)
if result.hostname != '127.0.0.1' and result.port:
hostname = result.hostname
port = result.port

params = urllib.parse.parse_qs(result.query)
token = params.get('token', [None])[0]
break
return {'hostname': hostname, 'port': port, 'token': token, 'url': url}

Expand Down
2 changes: 1 addition & 1 deletion setup.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ ignore =
max-line-length = 100
max-complexity = 18
select = B,C,E,F,W,T4,B9
extend-ignore = E203,E501,E402,W605
extend-ignore = E203,E501,E402,W605,W503
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't need that one anymore, but note that that warning is incompatible with black


[isort]
known_first_party=jupyter_forward
Expand Down
3 changes: 3 additions & 0 deletions tests/misc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,6 @@
'http://eniac01:59628/?token=Loremipsumdolorsitamet',
'or http://127.0.0.1:59628/?token=Loremipsumdolorsitamet',
]
sample_log_file_contents_with_contamination = [
'{"type":"info","data":"Visit https://yarnpkg.com/en/docs/cli/config for documentation about this command."}'
] + sample_log_file_contents
7 changes: 3 additions & 4 deletions tests/test_core.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,13 +178,12 @@ def test_prepare_batch_job_script(runner):
def test_parse_log_file(runner):
runner._set_log_directory()
runner._set_log_file()
runner.run_command(f"echo '{sample_log_file_contents[0]}' > {runner.log_file}")
for line in sample_log_file_contents[1:]:
runner.run_command(f"echo '{line}' >> {runner.log_file}")
content = ''.join(f'{line}\n' for line in sample_log_file_contents)
runner.put_file(runner.log_file, content)
out = runner._parse_log_file()
assert out == {
'hostname': 'eniac01',
'port': '59628',
'port': 59628,
'token': 'Loremipsumdolorsitamet',
'url': 'http://eniac01:59628/?token=Loremipsumdolorsitamet',
}
Expand Down
13 changes: 11 additions & 2 deletions tests/test_helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

import jupyter_forward.helpers

from .misc import sample_log_file_contents
from .misc import sample_log_file_contents, sample_log_file_contents_with_contamination


@pytest.mark.parametrize(
Expand All @@ -15,7 +15,16 @@
sample_log_file_contents,
{
'hostname': 'eniac01',
'port': '59628',
'port': 59628,
'token': 'Loremipsumdolorsitamet',
'url': 'http://eniac01:59628/?token=Loremipsumdolorsitamet',
},
),
(
sample_log_file_contents_with_contamination,
{
'hostname': 'eniac01',
'port': 59628,
'token': 'Loremipsumdolorsitamet',
'url': 'http://eniac01:59628/?token=Loremipsumdolorsitamet',
},
Expand Down