cuSparseLt python bindings & performance questions

Hi all,

it would be great to have some python bindings for cuSparseLt, because it'll be a while until pytorch supports this for all dtypes, especially low-precision such as int8
Using the C++ API, I'm only getting about 30% more speed for sparse int8 x int8 compared to dense torch._int_mm. Is that expected? I would have expected more given the hardware claims of twice the speed.

I'm pretty sure int8xint8 isn't bandwidth-limited on any modern GPU, is it?

@dxqb

Thanks for the request. We don’t currently plan to provide official Python wrappers. We will re-evaluate based on demand and resourcing.
What's your problem dimensions and GPU? 30% could be reasonable for small problems. For large problems it could be up to 70%. 2x is the theoretical speedup and in practice it's almost impossible to achieve it.

Thanks!
I don't have it in front of me, but I usually test with a sequence len of a few thousand and a feature dim of 3072. So probably something like
(2048, 3072) x (3072, 3072)

Please share the GPU you are using.

I'm pretty sure int8xint8 isn't bandwidth-limited on any modern GPU, is it?
Whether it's bandwidth bound or not mainly depends on the problem sizes rather than data time.

Navigation Menu

Repository navigation

cuSparseLt python bindings & performance questions #301

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

cuSparseLt python bindings & performance questions #301

Description

Activity

j4yan commented on Jan 7, 2026

dxqb commented on Jan 7, 2026

j4yan commented on Jan 9, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions