Skip to content

Commit

Permalink
fix: removes 64-char limit for url path & query
Browse files Browse the repository at this point in the history
- removes IDNA 64-char limit for url path & query
- validates absurd quoted url: `http://-.~_!$&'()*+,;=:%40:80%2f::::::@example.com`
- updates dependencies and changelog
- bumps patch version

**Related items**

*Issues*

- Closes #257
  • Loading branch information
yozachar committed Apr 10, 2023
1 parent 75cda81 commit eadbb22
Show file tree
Hide file tree
Showing 6 changed files with 71 additions and 69 deletions.
10 changes: 10 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# Changelog

## 0.21.1 (2023-04-10)

- fix: `source .venv/bin/activate` before build by @joe733 in [#260](https://github.com/python-validators/validators/pull/260)
- fix: id-token write permission at job level by @joe733 in [#261](https://github.com/python-validators/validators/pull/261)
- feat: docs can be built with both sphinx & mkdocs by @joe733 in [#262](https://github.com/python-validators/validators/pull/262)
- fix: improves build process by @joe733 in [#263](https://github.com/python-validators/validators/pull/263)
- fix: removes 64-char limit for url path & query by @joe733 in [#264](https://github.com/python-validators/validators/pull/264)

**Full Changelog**: [0.21.0...0.21.1](https://github.com/python-validators/validators/compare/0.21.0...0.21.1)

## 0.21.0 (2023-03-25)

- feat: add build for pypi workflow by @joe733 in [#255](https://github.com/python-validators/validators/pull/255)
Expand Down
94 changes: 37 additions & 57 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 6 additions & 6 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "validators"
version = "0.21.0"
version = "0.21.1"
description = "Python Data Validation for Humans™"
authors = ["Konsta Vesterinen <konsta@fastmonkeys.com>"]
license = "MIT"
Expand Down Expand Up @@ -28,16 +28,16 @@ include = ["CHANGES.md", "docs/*", "docs/validators.1", "validators/py.typed"]
python = "^3.8"

[tool.poetry.group.dev.dependencies]
tox = "^4.4.8"
tox = "^4.4.11"

[tool.poetry.group.docs.dependencies]
mkdocs = "^1.4.2"
mkdocs-material = "^9.1.4"
mkdocs-material = "^9.1.6"
mkdocstrings = { extras = ["python"], version = "^0.20.0" }
pyaml = "^21.10.1"

[tool.poetry.group.hooks.dependencies]
pre-commit = "^3.2.1"
pre-commit = "^3.2.2"

[tool.poetry.group.sast.dependencies]
bandit = "^1.7.5"
Expand All @@ -48,14 +48,14 @@ myst-parser = "^1.0.0"
pypandoc-binary = "^1.11"

[tool.poetry.group.tests.dependencies]
pytest = "^7.2.2"
pytest = "^7.3.0"

[tool.poetry.group.type-lint-format.dependencies]
black = "^23.1.1"
flake8 = "^5.0.4"
flake8-docstrings = "^1.7.0"
isort = "^5.12.0"
pyright = "^1.1.301"
pyright = "^1.1.302"

[build-system]
requires = ["poetry-core"]
Expand Down
2 changes: 2 additions & 0 deletions tests/test_url.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@
"http://президент.рф/",
"http://10.24.90.255:83/",
"https://travel-usa.com/wisconsin/旅行/",
"http://:::::::::::::@exmp.com",
"http://-.~_!$&'()*+,;=:%40:80%2f::::::@example.com",
# when simple_host=True
# "http://localhost",
# "http://localhost:8000",
Expand Down
2 changes: 1 addition & 1 deletion validators/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,4 @@
"fi_ssn",
)

__version__ = "0.20.0"
__version__ = "0.21.1"
20 changes: 15 additions & 5 deletions validators/url.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# -*- coding: utf-8 -*-

# standard
from urllib.parse import urlsplit
from urllib.parse import urlsplit, unquote
from functools import lru_cache
import re

Expand All @@ -24,7 +24,15 @@ def _username_regex():

@lru_cache
def _path_regex():
return re.compile(r"^[\/a-zA-Z0-9\-\.\_\~\!\$\&\'\(\)\*\+\,\;\=\:\@\%]+$", re.IGNORECASE)
return re.compile(
# allowed symbols
r"^[\/a-zA-Z0-9\-\.\_\~\!\$\&\'\(\)\*\+\,\;\=\:\@\%"
# emoticons / emoji
+ r"\U0001F600-\U0001F64F"
# multilingual unicode ranges
+ r"\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF]+$",
re.IGNORECASE,
)


@lru_cache
Expand Down Expand Up @@ -52,7 +60,9 @@ def _validate_auth_segment(value: str):
if not value:
return True
if (colon_count := value.count(":")) > 1:
return False
# everything before @ is then considered as a username
# this is a bad practice, but syntactically valid URL
return _username_regex().match(unquote(value))
if colon_count < 1:
return _username_regex().match(value)
username, password = value.rsplit(":", 1)
Expand Down Expand Up @@ -103,9 +113,9 @@ def _validate_optionals(path: str, query: str, fragment: str):
"""Validate path query and fragments."""
optional_segments = True
if path:
optional_segments &= bool(_path_regex().match(path.encode("idna").decode("utf-8")))
optional_segments &= bool(_path_regex().match(path))
if query:
optional_segments &= bool(_query_regex().match(query.encode("idna").decode("utf-8")))
optional_segments &= bool(_query_regex().match(query))
if fragment:
optional_segments &= all(char_to_avoid not in fragment for char_to_avoid in ("/", "?"))
return optional_segments
Expand Down

0 comments on commit eadbb22

Please sign in to comment.