Skip to content

Add tomli_loads benchmark #200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Apr 7, 2023
1 change: 1 addition & 0 deletions pyperformance/data-files/benchmarks/MANIFEST
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ sqlglot <local>
sqlite_synth <local>
sympy <local>
telco <local>
tomli_loads <local>
tornado_http <local>
unpack_sequence <local>
unpickle <local:pickle>
Expand Down
343,022 changes: 343,022 additions & 0 deletions pyperformance/data-files/benchmarks/bm_tomli_loads/data/tomli-bench-data.toml

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from urllib.request import urlopen
import json
import toml

BASE_URL = "https://api.github.com/repos/python/cpython/pulls?per_page=1000&state=all"

def main():
all_issues = []
for page in range(1, 11):
with urlopen(f"{BASE_URL}&page={page}") as response:
issues = json.loads(response.read())
if not issues:
break
all_issues.extend(issues)
print(f"Page: {page} Total Issues: {len(all_issues)}")
with open("issues.toml", "w") as f:
f.write(toml.dumps({"data": all_issues}))

if __name__ == "__main__":
main()
10 changes: 10 additions & 0 deletions pyperformance/data-files/benchmarks/bm_tomli_loads/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[project]
name = "pyperformance_bm_tomli_loads"
requires-python = ">=3.8"
dependencies = ["pyperf", "tomli"]
urls = {repository = "https://github.com/python/pyperformance"}
dynamic = ["version"]

[tool.pyperformance]
name = "tomli_loads"
tags = "serialize"
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
tomli==2.0.1
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"""
Benchmark ``loads()`` function of the ``tomli`` module
on a large TOML file of GitHub's real world data generated by
the ``generate_data.py`` script.

It heavily exercises string operations such as concatenation,
subscripting and iteration.

Author: Kumar Aditya
"""

from pathlib import Path

import pyperf
import tomli

DATA_FILE = Path(__file__).parent / "data" / "tomli-bench-data.toml"

def bench_tomli_loads(loops: int) -> float:
data = DATA_FILE.read_text('utf-8')
range_it = range(loops)
t0 = pyperf.perf_counter()
for _ in range_it:
tomli.loads(data)
return pyperf.perf_counter() - t0

if __name__ == "__main__":
runner = pyperf.Runner()
runner.metadata['description'] = "Benchmark tomli.loads()"
runner.bench_time_func('tomli_loads', bench_tomli_loads)