Fix adapter in streaming_forward

[marcoyang1998]

Fix for #1553.

[marcoyang1998]

To finetune a causal zipformer with adapter, just specify --causal 1 when using finetune.py.

The trained model can be exported using the following command (you can adjust the chunk size and left context when exporting the model):

./zipformer_adapter/export.py \
    --exp-dir ./zipformer_adapter/exp_causal1_giga_finetune1_adapters1_adapter_dim16 \
    --epoch 4 \
    --avg 1 \
    --use-averaged-model 1 \
    --chunk-size 16 \
    --left-context-frames 128 \
    --causal 1 \
    --tokens icefall-asr-librispeech-streaming-zipformer-2023-05-17/data/lang_bpe_500/tokens.txt \
    --use-adapters 1 \
    --adapter-dim 16 \
    --jit 1