No need to copy tokens

[howard0su]

No description provided.

[slaren]

Ironically with N=1, doing this will move more data than the memcpy (a 64-bit pointer instead of just a 32-bit value).

[howard0su]

If we can expose api to create tensor with data, then this will save additional cycles to allocate data of tensor.

[howard0su]

If we can expose api to create tensor with data, then this will save additional cycles to allocate data of tensor.

Ironically with N=1, doing this will move more data than the memcpy (a 64-bit pointer instead of just a 32-bit value).

Overhead is more than the data but also memcpy? I think the intention is not perf, but we should not do the copy here.

[ggerganov]

Redirecting ggml_tensor.data is not a good thing to do. mmap changes set a precedent, but we should avoid it in general