Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix garbage collection #1119

Merged
merged 40 commits into from
Jul 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
a9a10af
Make creation of aligned mapping lazy
ivirshup Sep 4, 2023
ca0759a
Use weakref for filemanager
ivirshup Sep 4, 2023
92615d8
Merge branch 'main' into remove-backrefs
ilan-gold Jun 3, 2024
cb53b77
(chore): add benchmark
ilan-gold Jun 3, 2024
2df03b5
(fix): use right `gen_adata` function
ilan-gold Jun 3, 2024
38d218f
(fix): change name
ilan-gold Jun 3, 2024
a9eba7a
(chore): fewer runs
ilan-gold Jun 3, 2024
be8ae55
(fix): benchmark test name
ilan-gold Jun 4, 2024
82fc74c
Merge branch 'main' into remove-backrefs
ilan-gold Jun 4, 2024
e0bff0e
(fix): return `cls` for `None` obj
ilan-gold Jun 4, 2024
ccbdaf3
Merge branch 'remove-backrefs' of https://github.com/ivirshup/anndata…
ilan-gold Jun 4, 2024
6ed9963
(fix): try `track_peakmem`
ilan-gold Jun 4, 2024
ce4e2d8
(fix): remove `track_` name
ilan-gold Jun 5, 2024
0c9e563
(fix): docs
ilan-gold Jun 5, 2024
c971dfb
(fix): do custom peakmem track
ilan-gold Jun 5, 2024
9e08151
(fix): `n` -> `runs`
ilan-gold Jun 5, 2024
c771079
(fix): comment
ilan-gold Jun 5, 2024
9d3caad
(fix): `vals` typing for aligned mappings
ilan-gold Jun 5, 2024
eeb8865
(fix): don't use mutable argument
ilan-gold Jun 5, 2024
5267c2d
(fix): change profiler name to `track_peakmem_garbage_collection`
ilan-gold Jun 5, 2024
13d134a
(chore): delete rogue comment
ilan-gold Jun 5, 2024
4dec616
clarify reference use
flying-sheep Jun 7, 2024
5361188
consistency
flying-sheep Jun 7, 2024
b61549d
remove boilerplate
flying-sheep Jun 7, 2024
81d5933
Merge branch 'main' into remove-backrefs
ilan-gold Jun 7, 2024
1a85497
add types, undo moves
flying-sheep Jun 7, 2024
f750954
Add type hints for properties
flying-sheep Jun 7, 2024
ab4cd79
Merge branch 'main' into remove-backrefs
ilan-gold Jun 18, 2024
cf4377c
Merge branch 'main' into pr/ivirshup/1119
flying-sheep Jun 21, 2024
48ae69d
fmt
flying-sheep Jul 2, 2024
480b0a4
cleanup
flying-sheep Jul 2, 2024
24b89a0
more fmt
flying-sheep Jul 4, 2024
05ebee9
dedupe and test
flying-sheep Jul 4, 2024
866abbd
Simplify copy
flying-sheep Jul 4, 2024
3595197
Merge branch 'main' into pr/ivirshup/1119
flying-sheep Jul 8, 2024
015d6ad
docs and typing
flying-sheep Jul 22, 2024
3a1a007
fix parent_mapping type
flying-sheep Jul 22, 2024
73fc94c
Fix I
flying-sheep Jul 22, 2024
ba61e2b
fix docs
flying-sheep Jul 22, 2024
e138fc9
fix 3.9
flying-sheep Jul 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions benchmarks/benchmarks/anndata.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
from __future__ import annotations

import tracemalloc

import numpy as np

from .utils import gen_adata


class GarbargeCollectionSuite:
runs = 10

# custom because `memory_profiler` is a line-by-line profiler (also: https://github.com/pythonprofilers/memory_profiler/issues/402)
def track_peakmem_garbage_collection(self, *_):
def display_top(snapshot, key_type="lineno"):
snapshot = snapshot.filter_traces(
(
tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
tracemalloc.Filter(False, "<unknown>"),
)
)
top_stats = snapshot.statistics(key_type)
total = sum(stat.size for stat in top_stats)
return total

total = np.zeros(self.runs)
tracemalloc.start()
for i in range(self.runs):
data = gen_adata(10000, 10000, "X-csc") # noqa: F841
snapshot = tracemalloc.take_snapshot()
total[i] = display_top(snapshot)
tracemalloc.stop()
return max(total)
4 changes: 2 additions & 2 deletions docs/_templates/autosummary/class.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
.. autosummary::
:toctree: .
{% for item in attributes %}
~{{ fullname }}.{{ item }}
~{{ name }}.{{ item }}
{%- endfor %}
{% endif %}
{% endblock %}
Expand All @@ -26,7 +26,7 @@
:toctree: .
{% for item in methods %}
{%- if item != '__init__' %}
~{{ fullname }}.{{ item }}
~{{ name }}.{{ item }}
{%- endif -%}
{%- endfor %}
{% endif %}
Expand Down
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
# default settings
templates_path = ["_templates"]
html_static_path = ["_static"]
source_suffix = [".rst", ".md"]
source_suffix = {".rst": "restructuredtext", ".md": "markdown"}
master_doc = "index"
default_role = "literal"
exclude_patterns = [
Expand Down
Loading
Loading