Are we loading each source file twice on startup?

[lnicola]

Will post a trace later on, but it seems we're loading every file two times into the VFS on startup. Avoiding that might be nice.

[lnicola]

I got some statistics. On my branch, we're loading 4401 distinct files. We load (set) rust-analyzer/crates/proc-macro-test/imp/src/lib.rs 5 times, 203 files 3 times, 4228 files 2 times and 9 files just one time (some source code generated by build scripts).

The (de-duplicated) code takes 59 MB uncompressed and 18.5 MB compressed with LZ4.

When typing one character, we're reading and setting the text once (to update it), then reading it two more times. I don't remember exactly, but I'm a little surprised it's not cached in salsa.

vfs-trace.zip

[petr-tik]

thanks for providing a trace file - I found another file that is loaded even more times.

sqlite> SELECT path, count(*) as cnt from trace GROUP BY path ORDER BY cnt DESC LIMIT 10;
Projects/rust-analyzer/crates/proc-macro-test/imp/src/lib.rs|7
Projects/rust-analyzer/xtask/src/main.rs|5
Projects/rust-analyzer/lib/lsp-server/src/lib.rs|5
Projects/rust-analyzer/lib/lsp-server/examples/goto_def.rs|5
Projects/rust-analyzer/lib/la-arena/src/lib.rs|5
Projects/rust-analyzer/crates/vfs/src/lib.rs|5
Projects/rust-analyzer/crates/vfs-notify/src/lib.rs|5
Projects/rust-analyzer/crates/tt/src/lib.rs|5
Projects/rust-analyzer/crates/toolchain/src/lib.rs|5
Projects/rust-analyzer/crates/text-edit/src/lib.rs|5

I would like to take a stab at it. Can you please tell me how you collected this trace and recommended the best way to setup a test feedback loop for this?

[lnicola]

Hi @petr-tik and sorry for the poor info and delay 😔. You're marked as busy, but I'll answer now anyway.

I added some tracing at https://github.com/lnicola/rust-analyzer/tree/vfs-log. RA_LOG=vfs=debug RA_LOG_FILE=ra.log code . should work (there'll be one message from vfs-notify, I have no idea how to exclude it).

My gut feeling is that this is related to the project loading and reloading (crates/rust-analyzer/src/reload.rs) code. It's pretty hard to follow and I think there's a bit of duplication there.

[lnicola]

Just to be clear, this might not be a problem. In any case, we detect that the files didn't change and ignore it, so we're just paying for the IO. But I wanted to be sure it works as intended.

The other thing is that storing the file contents in the VFS is mostly redundant, since we put everything into salsa anyway (set_file_text).

[Veykril]

It's not quite redundant, the salsa copy is normalized iirc while for the lsp document change notification we need the unnormalized files. Though maybe we can be smart in some way to get around that requirement

[Veykril]

The cause for this seems to be us sending two set_config calls which will re-request loading all files apparently. The second one probably comes from us switching workspaces once build scripts are built.