Skip to content

Commit

Permalink
[batch] Fix batch local mode in jupyter notebooks (#13614)
Browse files Browse the repository at this point in the history
The `LocalBackend` recently was changed to include a call to
`async_to_blocking` but does not apply `nest_asyncio`, so using the
local backend in a jupyter notebook breaks. I lifted the application of
`nest_asyncio` from the `ServiceBackend` to the super class. Should we
add a test that runs in its own python session to make sure that this is
caught?
  • Loading branch information
daniel-goldstein authored Sep 14, 2023
1 parent c95f2d1 commit ccb4a94
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 8 deletions.
3 changes: 0 additions & 3 deletions hail/python/hail/docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,3 @@ Indices and tables
* :ref:`genindex`

If you would like to refer to our Hail v0.1 (deprecated) docs, please view `Hail 0.1 docs </docs/0.1/index.html>`_



7 changes: 2 additions & 5 deletions hail/python/hailtop/batch/backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,12 @@ class Backend(abc.ABC, Generic[RunningBatchType]):
"""
Abstract class for backends.
"""

_closed = False

def __init__(self):
self._requester_pays_fses: Dict[GCSRequesterPaysConfiguration, RouterAsyncFS] = {}
import nest_asyncio # pylint: disable=import-outside-toplevel
nest_asyncio.apply()

def requester_pays_fs(self, requester_pays_config: GCSRequesterPaysConfiguration) -> RouterAsyncFS:
try:
Expand Down Expand Up @@ -486,11 +487,7 @@ def __init__(
gcs_requester_pays_configuration: Optional[GCSRequesterPaysConfiguration] = None,
gcs_bucket_allow_list: Optional[List[str]] = None,
):

import nest_asyncio # pylint: disable=import-outside-toplevel

super().__init__()
nest_asyncio.apply()

if len(args) > 2:
raise TypeError(f'ServiceBackend() takes 2 positional arguments but {len(args)} were given')
Expand Down

0 comments on commit ccb4a94

Please sign in to comment.