Always killed

[HelloWorldLTY]

Hi, thanks for your great work. I wonder if it is possible to provide some guidance about memory trade-off setting. I intend to allcoate a KAN with 3072 as input dimension and 2000 as output dimension. I always get killed. Normal MLP does not meet such error. Thanks a lot

[unrealwill]

When written naively, this layer consume a lot of memory.
If you use einsum as explained in code comment, it should use less memory.
Fusing with KeOps lazy tensor, is an alternative left to the reader.

The optimal memory usage is zero extra memory beyond the parameters (coeff size = 2inputdimoutputdim*gridsize) , inputs and outputs, which can be obtained by manually fusing :

https://github.com/GistNoesis/FusedFourierKAN

The code is not yet optimized, and painful to install, so performance may not be as fast as some code that use more memory.

Some other project https://github.com/Blealtan/efficient-kan maybe more user friendly with respect to memory.

[HelloWorldLTY]

Thanks a lot. I will try other versions.