RNN-style train and eval for S4/S4D

[mingweima]

Excellent idea and great paper!
Could you please provide a concrete example on how to both train and eval using the stateful RNN version of S4/S4D? I only find an evaluation example in the SaShiMi code but I have not found an example for training.
Thank you!

[albertfgu]

You can find functionality in v1 of this codebase, by passing an initial state into the forward pass of the S4 module. Unfortunately this functionality has been discontinued because it is a non-trivial technical addition that is difficult to maintain and has not been published yet. We are thinking of putting up a short technical report with the details and adding official support for this.

If you really need this functionality, you can modify it from v1 of this codebase. Alternatively you can modify the RNN mode (with the step function), but this could be very slow.

[mingweima]

Thanks！

[albertfgu]

If you haven't seen it yet, this functionality has been re-introduced and improved: see the README.

As I mentioned previously, this functionality is non-trivial and unpublished, and we would appreciate you sending a private correspondence if it ends up being important for a project.