You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I just want to propose an alternative for discussion. I'm not sure if it's doable, but given that we have control over array-api-compat, could we have array_namespace return an object that explicitly derives from an ABC ArrayNamespace?
Then you have perfect runtime check-ability, and you don't have to worry about keeping array-api-compat in sync with array-api-typing?
I just want to propose an alternative for discussion. I'm not sure if it's doable, but given that we have control over array-api-compat, could we have array_namespace return an object that explicitly derives from an ABC ArrayNamespace?
A nominal approach like that will only work if you're using array-api-compat. And in an ideal scenario, there wouldn't be any need for array-api-compat. So for a complete solution, I think that structural typing (i.e. "static duck typing" with Protocols) is the way to go here.
And in an ideal scenario, there wouldn't be any need for array-api-compat
Is this even realistic for any backend? Aren't we patching all of them?
So for a complete solution, I think that structural typing (i.e. "static duck typing" with Protocols) is the way to go here.
TBH, I'm not even convinced of this since the Array API will have different versions, and each backend may have itself a different version. Therefore, you're going to have a very hard designing a protocol that matches every backend you happened to import.
On the other hand, if you had a particular version of array-api-compat, you can have it generate the appropriate shims to each of your backends so that they all match a given version of the Array API.
Is this even realistic for any backend? Aren't we patching all of them?
I might be wrong here, but I think that NumPy 2.3.0 is pretty close to fully supporting v2024.
But I don't think that requiring array-api-compat as a "middle-man" here is a good idea. The relevant libraries generally speaking care about performance. And I can imagine that wrapping call to array-api functionality using array-api-compat can have a pretty big impact.
TBH, I'm not even convinced of this since the Array API will have different versions, and each backend may have itself a different version. Therefore, you're going to have a very hard designing a protocol that matches every backend you happened to import.
The plan is to publish different array-api-typing releases that target individual array-api versions. We'll start with 2021, and then work our way up. We'll use a versioning scheme that includes the array-api version, and our own patch version number, in case of bugfix backports and such.
On the other hand, if you had a particular version of array-api-compat, you can have it generate the appropriate shims to each of your backends so that they all match a given version of the Array API.
I think we'll be able to get this all working without code generation :)
In jorenham/optype#25 I wrote down how I think we'll can do that on the static typing side. But I want to condense it into something that will resembles a plan for array-api-typing, once I have the bandwidth.
But I don't think that requiring array-api-compat as a "middle-man" here is a good idea. The relevant libraries generally speaking care about performance. And I can imagine that wrapping call to array-api functionality using array-api-compat can have a pretty big impact.
Can you explain in which way you can prevent compat from being a middleman? Isn't that the way in which you get array namespaces? If people want to use numpy or jax.numpy, they're still free to do that and they'll have the annotations from those libraries. If people use array_namespace(...), then they'll get xpt.ArrayNamespace, and they'll get those annotations, right?
The plan is to publish different array-api-typing releases that target individual array-api versions. We'll start with 2021, and then work our way up. We'll use a versioning scheme that includes the array-api version, and our own patch version number, in case of bugfix backports and such.
For any version of array-api-typing, is it not that case that:
an old version of jax.numpy may not match the modern protocol (for example, due to an added method), and
a newer version of jax.numpy may not match an older protocol (for example, due to a changed signature)
Therefore, even if you have clearly versioned xpt, the user still has to match it to numpy and jax versions (or whatever library they're using). So the user has to do that manually in pyproject?
If the user gets it wrong, the protocols won't match and there may be type errors, right?
I think we'll be able to get this all working without code generation :)
Oh that would be great. Looking forward to seeing it!
Can you explain in which way you can prevent compat from being a middleman?
I can, but that's a very long answer, as that would basically require me to explain the entire plan I have in mind (and implementing it is probably faster), but I plan to do this soon (see #20 (comment)). So I'll leave answering this untill then, if you don't mind.
Isn't that the way in which you get array namespaces? If people want to use numpy or jax.numpy, they're still free to do that and they'll have the annotations from those libraries. If people use array_namespace(...), then they'll get xpt.ArrayNamespace, and they'll get those annotations, right?
The array_namespace(a) function is indeed a way of accessing the array-api functionality. Another way would be to use a.__array_namespace__() directly, which would give you the same result. So the library that provides this namespace, could annotate the __array_namespace__() return type using array-api-typing, without having to use aray-api-compat. If they don't then the user could use array-api-typing themselves to e.g. typing.cast this unknown return type to the appropriate structural type provided by array-api-typing. In both cases, there is no need for array-api-compat.
(Oh haha this accidentally also answers your first question, so it wasn't as complicated as I thought)
For any version of array-api-typing, is it not that case that:
an old version of jax.numpy may not match the modern protocol (for example, due to an added method), and
a newer version of jax.numpy may not match an older protocol (for example, due to a changed signature)
Therefore, even if you have clearly versioned xpt, the user still has to match it to numpy and jax versions (or whatever library they're using). So the user has to do that manually in pyproject?
If the user gets it wrong, the protocols won't match and there may be type errors, right?
As far as I know, the array API versions are backwards compatible with each other. So if you use different libraries with different array-api versions, then the lowest one should be used to pin array-api-typing.
Activity
jorenham commentedon Jun 13, 2025
I prefer
-> TypeIs[...]functions like #19 as part of the public API.There's also a secret option number three: jorenham/optype#252. It's not for the faint of heart, though (see e.g. https://github.com/jorenham/optype/blob/c48c16c1e0dea630d4d99509e0f911a55e627808/optype/_core/_just.py#L93-L134).
nstarman commentedon Jun 14, 2025
Then maybe we should standardize the naming scheme?
jorenham commentedon Jun 14, 2025
Yea good idea
NeilGirdhar commentedon Jun 14, 2025
I just want to propose an alternative for discussion. I'm not sure if it's doable, but given that we have control over
array-api-compat, could we havearray_namespacereturn an object that explicitly derives from an ABCArrayNamespace?Then you have perfect runtime check-ability, and you don't have to worry about keeping array-api-compat in sync with array-api-typing?
jorenham commentedon Jun 15, 2025
A nominal approach like that will only work if you're using
array-api-compat. And in an ideal scenario, there wouldn't be any need forarray-api-compat. So for a complete solution, I think that structural typing (i.e. "static duck typing" withProtocols) is the way to go here.NeilGirdhar commentedon Jun 15, 2025
Is this even realistic for any backend? Aren't we patching all of them?
TBH, I'm not even convinced of this since the Array API will have different versions, and each backend may have itself a different version. Therefore, you're going to have a very hard designing a protocol that matches every backend you happened to import.
On the other hand, if you had a particular version of array-api-compat, you can have it generate the appropriate shims to each of your backends so that they all match a given version of the Array API.
jorenham commentedon Jun 15, 2025
I might be wrong here, but I think that NumPy 2.3.0 is pretty close to fully supporting v2024.
But I don't think that requiring
array-api-compatas a "middle-man" here is a good idea. The relevant libraries generally speaking care about performance. And I can imagine that wrapping call to array-api functionality using array-api-compat can have a pretty big impact.The plan is to publish different
array-api-typingreleases that target individual array-api versions. We'll start with 2021, and then work our way up. We'll use a versioning scheme that includes the array-api version, and our own patch version number, in case of bugfix backports and such.I think we'll be able to get this all working without code generation :)
In jorenham/optype#25 I wrote down how I think we'll can do that on the static typing side. But I want to condense it into something that will resembles a plan for array-api-typing, once I have the bandwidth.
NeilGirdhar commentedon Jun 15, 2025
Can you explain in which way you can prevent compat from being a middleman? Isn't that the way in which you get array namespaces? If people want to use
numpyorjax.numpy, they're still free to do that and they'll have the annotations from those libraries. If people usearray_namespace(...), then they'll getxpt.ArrayNamespace, and they'll get those annotations, right?For any version of array-api-typing, is it not that case that:
Therefore, even if you have clearly versioned xpt, the user still has to match it to numpy and jax versions (or whatever library they're using). So the user has to do that manually in
pyproject?If the user gets it wrong, the protocols won't match and there may be type errors, right?
Oh that would be great. Looking forward to seeing it!
jorenham commentedon Jun 18, 2025
I can, but that's a very long answer, as that would basically require me to explain the entire plan I have in mind (and implementing it is probably faster), but I plan to do this soon (see #20 (comment)). So I'll leave answering this untill then, if you don't mind.
The
array_namespace(a)function is indeed a way of accessing the array-api functionality. Another way would be to usea.__array_namespace__()directly, which would give you the same result. So the library that provides this namespace, could annotate the__array_namespace__()return type usingarray-api-typing, without having to usearay-api-compat. If they don't then the user could usearray-api-typingthemselves to e.g.typing.castthis unknown return type to the appropriate structural type provided byarray-api-typing. In both cases, there is no need forarray-api-compat.(Oh haha this accidentally also answers your first question, so it wasn't as complicated as I thought)
As far as I know, the array API versions are backwards compatible with each other. So if you use different libraries with different array-api versions, then the lowest one should be used to pin
array-api-typing.