Skip to content

_get_scheduler() exception if dask.multiprocessing missing #3401

Closed

Description

@johnomotani

These lines were recently changed in #3358

pass
if actual_get is dask.multiprocessing.get:
return "multiprocessing"
else:
return "threaded"

If the 'cloudpickle' package is not installed, then dask.multiprocessing is not available. The try/except that used to be wrapped around if actual_get is dask.multiprocessing.get meant that _get_scheduler() worked in that case, returning "threaded" (I assume this was the expected behaviour). After #3358, _get_scheduler() raised an AttributeError: module 'dask' has no attribute 'multiprocessing' until I installed 'cloudpickle'.

Suggest either reverting the changes that removed the try/except or making 'cloudpickle' a dependency.


To reproduce:

  1. check 'cloudpickle' is not installed, but 'dask' is
  2. execute the following commands
>>> import xarray
>>> xarray.backends.api._get_scheduler()

Expected result: "threaded"
Actual result:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-20da238796b7> in <module>
----> 1 xarray.backends.api._get_scheduler()

~/.local/lib/python3.6/site-packages/xarray/backends/locks.py in _get_scheduler(get, collection)
     87         pass
     88 
---> 89     if actual_get is dask.multiprocessing.get:
     90         return "multiprocessing"
     91     else:

AttributeError: module 'dask' has no attribute 'multiprocessing'

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions