xr.dot requires equal indexes (join="exact") #3694

Activity

1. dcherian

   changed the title [-]xr.dot requires equal dimensions[/-] [+]xr.dot requires equal indexes (join="exact")[/+] on Jan 14, 2020
2. fujiisoup commented on Jan 14, 2020

   

   fujiisoup

   on Jan 14, 2020

   Member

   More actions

   I have no strong opinion, but if most of the arithmetic in xarray uses join='inner', then it would be nicer to do so here too.
3. shoyer commented on Jan 15, 2020

   

   shoyer

   on Jan 15, 2020

   Member

   More actions

   You could dig through the original PRs to be sure, but I think we mostly picked join='exact' out of an abundance of caution. In principle I think it would be reasonable to change it, there is a pretty good case that (d1 * d2).sum() and d1 @ d2 should be consistent.
4. mathause commented on Jan 16, 2020

   

   mathause

   on Jan 16, 2020

   CollaboratorAuthor

   More actions

   I started with a PR using join = OPTIONS["arithmetic_join"] but then I realized that dot does not support skipna. Thus join="left" join="right", and join="outer" returns NaN for nonequal coords, which I think defeats the purpose. I can

   1. Use OPTIONS["arithmetic_join"] anyway
   2. Only support join="inner" (i.e. hard-code it in the call to apply_ufunc)
   3. try to implement skipna for dot (in a separate PR)
   4. Other?

   (3) is certainly the largest change but may be as easy as da.fillna(0.). Thoughts?

   import numpy as np
   import xarray as xr
   
   d1 = xr.DataArray([2, 3, 5, np.NaN])
   d2 = xr.DataArray([2, 3, 5, 7])
   
   xr.dot(d1, d2) # -> NaN
   
   xr.dot(d1.fillna(0.), d2) # -> 38
   (d1 * d2).sum() # -> 38

   I use this at:
   

   xarray/xarray/core/weighted.py

   Lines 142 to 149 in 3cc00c1

   	# need to mask invalid DATA as `dot` does not implement skipna
   	if skipna or (skipna is None and da.dtype.kind in "cfO"):
   	da = da.fillna(0.0)
   	# `dot` does not broadcast arrays, so this avoids creating a large
   	# DataArray (if `weights` has additional dimensions)
   	# TODO: maybe add fasttrack (`(da * weights).sum(dims=dim, skipna=skipna)`)
   	return dot(da, self.weights, dims=dim)
5. mathause commented on Jan 16, 2020

   

   mathause

   on Jan 16, 2020

   Last edited by mathause

   CollaboratorAuthor

   More actions

   On second thought, even with implemented skipna the result will not differ between join="inner", join="left" join="right", and join="outer"; i.e.:

   with xr.set_options(arithmetic_join="outer"):
       print((d1 * d2).sum()) # -> 38

   So I guess (2) is fine.
6. mathause commented on Jan 16, 2020

   

   mathause

   on Jan 16, 2020

   CollaboratorAuthor

   More actions

   Sorry for the noise, I do live thinking...

   ---

   However, if I do (2) there is a different behavior for (d1 * d2).sum() and d1 @ d2 in one case:

   with xr.set_options(arithmetic_join="exact"):
       xr.dot(d2, d3) # -> array(8)
       (d2 * d3).sum() # -> ValueError

   So?

   join = OPTIONS["arithmetic_join"]
   if join not in ["exact", "inner"]:
       join = "inner"
7. fujiisoup

   closed this as completedin #3699on Jan 20, 2020