Skip to content

Remove unused config options #806

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,12 @@
# serve to show the default value.

# If your extensions are in another directory, add it here. If the directory
# is relative to the documentation root, use os.path.abspath to make it
# is relative to the documentation root, use Path.absolute to make it
# absolute, like shown here.
# sys.path.append(os.path.abspath('some/directory'))
# sys.path.append(str(Path("some/directory").absolute()))

import os
import inspect
import sys
import pytensor

Expand Down Expand Up @@ -236,11 +237,9 @@ def find_source():
obj = sys.modules[info["module"]]
for part in info["fullname"].split("."):
obj = getattr(obj, part)
import inspect
import os

fn = inspect.getsourcefile(obj)
fn = os.path.relpath(fn, start=os.path.dirname(pytensor.__file__))
fn = Path(inspect.getsourcefile(obj))
fn = fn.relative_to(Path(__file__).parent)
source, lineno = inspect.getsourcelines(obj)
return fn, lineno, lineno + len(source) - 1

Expand Down
104 changes: 0 additions & 104 deletions doc/library/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -510,116 +510,12 @@ import ``pytensor`` and print the config variable, as in:

Removing these asserts can speed up execution.

.. attribute:: config.dnn__enabled

String value: ``'auto'``, ``'True'``, ``'False'``, ``'no_check'``

Default: ``'auto'``

If ``'auto'``, automatically detect and use
`cuDNN <https://developer.nvidia.com/cudnn>`_ when it is available.
If cuDNN is unavailable, do not raise an error.

If ``'True'``, require the use of cuDNN. If cuDNN is unavailable, raise an error.

If ``'False'``, neither use cuDNN nor check if it is available.

If ``'no_check'``, assume cuDNN is present and that the versions between the
header and library match.

.. attribute:: config.dnn__include_path

Default: ``include`` sub-folder in CUDA root directory, or headers paths defined for the compiler.

Location of the cuDNN header.

.. attribute:: config.dnn__library_path

Default: Library sub-folder (``lib64`` on Linux) in CUDA root directory, or
libraries paths defined for the compiler.

Location of the cuDNN library.

.. attribute:: config.conv__assert_shape

If ``True``, ``AbstractConv*`` :class:`Op`\s will verify that user-provided shapes
match the run-time shapes. This is a debugging option, and may slow down
compilation.

.. attribute:: config.dnn.conv.workmem

Deprecated, use :attr:`config.dnn__conv__algo_fwd`.


.. attribute:: config.dnn.conv.workmem_bwd

Deprecated, use :attr:`config.dnn__conv__algo_bwd_filter` and
:attr:`config.dnn__conv__algo_bwd_data` instead.

.. attribute:: config.dnn__conv__algo_fwd

String value:
``'small'``, ``'none'``, ``'large'``, ``'fft'``, ``'fft_tiling'``,
``'winograd'``, ``'winograd_non_fused'``, ``'guess_once'``, ``'guess_on_shape_change'``,
``'time_once'``, ``'time_on_shape_change'``.

Default: ``'small'``

3d convolution only support ``'none'``, ``'small'``, ``'fft_tiling'``, ``'guess_once'``,
``'guess_on_shape_change'``, ``'time_once'``, ``'time_on_shape_change'``.

.. attribute:: config.dnn.conv.algo_bwd

Deprecated, use :attr:`config.dnn__conv__algo_bwd_filter` and
:attr:`config.dnn__conv__algo_bwd_data` instead.

.. attribute:: config.dnn__conv__algo_bwd_filter

String value:
``'none'``, ``'deterministic'``, ``'fft'``, ``'small'``, ``'winograd_non_fused'``, ``'fft_tiling'``, ``'guess_once'``,
``'guess_on_shape_change'``, ``'time_once'``, ``'time_on_shape_change'``.

Default: ``'none'``

3d convolution only supports ``'none'``, ``'small'``, ``'guess_once'``,
``'guess_on_shape_change'``, ``'time_once'``, ``'time_on_shape_change'``.

.. attribute:: config.dnn__conv__algo_bwd_data

String value:
``'none'``, ``'deterministic'``, ``'fft'``, ``'fft_tiling'``, ``'winograd'``,
``'winograd_non_fused'``, ``'guess_once'``, ``'guess_on_shape_change'``, ``'time_once'``,
``'time_on_shape_change'``.

Default: ``'none'``

3d convolution only supports ``'none'``, ``'deterministic'``, ``'fft_tiling'``
``'guess_once'``, ``'guess_on_shape_change'``, ``'time_once'``,
``'time_on_shape_change'``.

.. attribute:: config.magma__enabled

String value: ``'True'``, ``'False'``

Default: ``'False'``

If ``'True'``, use `magma <http://icl.cs.utk.edu/magma/>`_ for matrix
computations.

If ``'False'``, disable magma.

.. attribute:: config.magma__include_path

Default: ``''``

Location of the magma headers.

.. attribute:: config.magma__library_path

Default: ``''``

Location of the magma library.

.. attribute:: config.ctc__root

Default: ``''``
Expand Down
5 changes: 2 additions & 3 deletions doc/library/d3viz/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,9 @@ visualize it with :py:func:`pytensor.printing.pydotprint` as follows:
.. code:: python

from pytensor.printing import pydotprint
import os
from pathlib import Path

if not os.path.exists('examples'):
os.makedirs('examples')
Path("examples").mkdir(exist_ok=True)
pydotprint(predict, 'examples/mlp.png')


Expand Down
2 changes: 1 addition & 1 deletion doc/troubleshooting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ PyTensor/BLAS speed test:

.. code-block:: bash

python `python -c "import os, pytensor; print(os.path.dirname(pytensor.__file__))"`/misc/check_blas.py
python $(python -c "import pathlib, pytensor; print(pathlib.Path(pytensor.__file__).parent / 'misc/check_blas.py')")

This will print a table with different versions of BLAS/numbers of
threads on multiple CPUs. It will also print some PyTensor/NumPy
Expand Down
3 changes: 1 addition & 2 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ dependencies:
- compilers
- numpy>=1.17.0,<2
- scipy>=0.14,<1.14.0
- filelock
- filelock>=3.15
- etuples
- logical-unification
- miniKanren
Expand All @@ -27,7 +27,6 @@ dependencies:
- coveralls
- diff-cover
- mypy
- types-filelock
- types-setuptools
- pytest
- pytest-cov
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ dependencies = [
"setuptools>=59.0.0",
"scipy>=0.14,<1.14",
"numpy>=1.17.0,<2",
"filelock",
"filelock>=3.15",
"etuples",
"logical-unification",
"miniKanren",
Expand Down
8 changes: 3 additions & 5 deletions pytensor/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@
# Set a default logger. It is important to do this before importing some other
# pytensor code, since this code may want to log some messages.
import logging
import os
import sys
from functools import singledispatch
from pathlib import Path
from typing import Any, NoReturn, Optional

from pytensor.version import version as __version__
Expand All @@ -52,10 +52,8 @@ def disable_log_handler(logger=pytensor_logger, handler=logging_default_handler)

# Raise a meaningful warning/error if the pytensor directory is in the Python
# path.
rpath = os.path.realpath(__path__[0])
for p in sys.path:
if os.path.realpath(p) != rpath:
continue
rpath = Path(__file__).parent.resolve()
if any(rpath == Path(p).resolve() for p in sys.path):
raise RuntimeError("You have the pytensor directory in your Python path.")

from pytensor.configdefaults import config
Expand Down
38 changes: 16 additions & 22 deletions pytensor/compile/compiledir.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
"""

import logging
import os
import pickle
import shutil
from collections import Counter
Expand Down Expand Up @@ -33,12 +32,11 @@
If there is no key left for a compiled module, we delete the module.

"""
compiledir = config.compiledir
for directory in os.listdir(compiledir):
for directory in config.compiledir.iterdir():
try:
filename = os.path.join(compiledir, directory, "key.pkl")
filename = directory / "key.pkl"

Check warning on line 37 in pytensor/compile/compiledir.py

View check run for this annotation

Codecov / codecov/patch

pytensor/compile/compiledir.py#L37

Added line #L37 was not covered by tests
# print file
with open(filename, "rb") as file:
with filename.open("rb") as file:
try:
keydata = pickle.load(file)

Expand Down Expand Up @@ -79,7 +77,7 @@
"the directory containing it."
)
if len(keydata.keys) == 0:
shutil.rmtree(os.path.join(compiledir, directory))
shutil.rmtree(directory)

Check warning on line 80 in pytensor/compile/compiledir.py

View check run for this annotation

Codecov / codecov/patch

pytensor/compile/compiledir.py#L80

Added line #L80 was not covered by tests

except (EOFError, AttributeError):
_logger.error(
Expand Down Expand Up @@ -117,11 +115,11 @@
big_key_files = []
total_key_sizes = 0
nb_keys = Counter()
for dir in os.listdir(compiledir):
filename = os.path.join(compiledir, dir, "key.pkl")
if not os.path.exists(filename):
for dir in config.compiledir.iterdir():
filename = dir / "key.pkl"

Check warning on line 119 in pytensor/compile/compiledir.py

View check run for this annotation

Codecov / codecov/patch

pytensor/compile/compiledir.py#L119

Added line #L119 was not covered by tests
if not filename.exists():
continue
with open(filename, "rb") as file:
with filename.open("rb") as file:
try:
keydata = pickle.load(file)
ops = list({x for x in flatten(keydata.keys) if isinstance(x, Op)})
Expand All @@ -134,15 +132,11 @@
{x for x in flatten(keydata.keys) if isinstance(x, CType)}
)
compile_start = compile_end = float("nan")
for fn in os.listdir(os.path.join(compiledir, dir)):
if fn.startswith("mod.c"):
compile_start = os.path.getmtime(
os.path.join(compiledir, dir, fn)
)
elif fn.endswith(".so"):
compile_end = os.path.getmtime(
os.path.join(compiledir, dir, fn)
)
for fn in dir.iterdir():
if fn.name == "mod.c":
compile_start = fn.stat().st_mtime

Check warning on line 137 in pytensor/compile/compiledir.py

View check run for this annotation

Codecov / codecov/patch

pytensor/compile/compiledir.py#L137

Added line #L137 was not covered by tests
elif fn.suffix == ".so":
compile_end = fn.stat().st_mtime

Check warning on line 139 in pytensor/compile/compiledir.py

View check run for this annotation

Codecov / codecov/patch

pytensor/compile/compiledir.py#L139

Added line #L139 was not covered by tests
compile_time = compile_end - compile_start
if len(ops) == 1:
table.append((dir, ops[0], types, compile_time))
Expand All @@ -153,7 +147,7 @@
(dir, ops_to_str, types_to_str, compile_time)
)

size = os.path.getsize(filename)
size = filename.stat().st_size

Check warning on line 150 in pytensor/compile/compiledir.py

View check run for this annotation

Codecov / codecov/patch

pytensor/compile/compiledir.py#L150

Added line #L150 was not covered by tests
total_key_sizes += size
if size > max_key_file_size:
big_key_files.append((dir, size, ops))
Expand Down Expand Up @@ -239,8 +233,8 @@
"""
subdirs = []
others = []
for f in os.listdir(config.base_compiledir):
if os.path.isdir(os.path.join(config.base_compiledir, f)):
for f in config.base_compiledir.iterdir():
if f.is_dir():
subdirs.append(f)
else:
others.append(f)
Expand Down
5 changes: 3 additions & 2 deletions pytensor/compile/compilelock.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import os
import threading
from contextlib import contextmanager
from pathlib import Path

import filelock

Expand Down Expand Up @@ -35,7 +36,7 @@ def force_unlock(lock_dir: os.PathLike):
Path to a directory that was locked with `lock_ctx`.
"""

fl = filelock.FileLock(os.path.join(lock_dir, ".lock"))
fl = filelock.FileLock(Path(lock_dir) / ".lock")
fl.release(force=True)

dir_key = f"{lock_dir}-{os.getpid()}"
Expand Down Expand Up @@ -72,7 +73,7 @@ def lock_ctx(

if dir_key not in local_mem._locks:
local_mem._locks[dir_key] = True
fl = filelock.FileLock(os.path.join(lock_dir, ".lock"))
fl = filelock.FileLock(Path(lock_dir) / ".lock")
fl.acquire(timeout=timeout)
try:
yield
Expand Down
3 changes: 2 additions & 1 deletion pytensor/compile/profiling.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
import time
from collections import Counter, defaultdict
from contextlib import contextmanager
from pathlib import Path
from typing import Any

import numpy as np
Expand All @@ -34,7 +35,7 @@ def extended_open(filename, mode="r"):
elif filename == "<stderr>":
yield sys.stderr
else:
with open(filename, mode=mode) as f:
with Path(filename).open(mode=mode) as f:
yield f


Expand Down
Loading
Loading