Measure performance of indexer(s)

[radeksimko]

Background

The LS (re)indexes files at various stages, as necessary per LSP:

• initialize
• textDocument/didOpen
• textDocument/didChange
• textDocument/didChangeWatchedFiles
• workspace/didChangeWorkspaceFolders

The first one in particular is likely the most impactful on resource usage as it likely to index deeper hierarchy with many modules, the other ones are more likely to work with much more limited scope of 1 or few modules only.

We currently run benchmarks of various modules in

terraform-ls/.github/workflows/benchmarks.yml

Lines 36 to 44 in 2738c6c

	-
	name: Run benchmarks
	id: bench
	run: |
	go test ./internal/langserver/handlers \
	-bench=InitializeFolder_basic \
	-run=^# \
	-benchtime=60s \
	-timeout=60m | tee ${{ runner.temp }}/benchmarks.txt

This however only tells us how some particular publicly available modules perform. Users may or may not use these modules, or any number of any modules and combination of those. To understand the real impact on real end-users we need some way of measuring the performance at runtime (as opposed to in isolation).

Measuring any aggregate performance impact - e.g. by logging total time spent indexing via initialize is currently not possible as indexing happens asynchronously. This includes both the filesystem walking and the actual indexing jobs on discovered modules.

We have a mechanism for tracking job IDs in the walker:

terraform-ls/internal/walker/walker.go

Lines 197 to 201 in 2738c6c

	ids, err := w.walkFunc(ctx, dir)
	if err != nil {
	w.collectError(fmt.Errorf("walkFunc: %w", err))
	}
	w.collectJobIds(ids)

but this isn't suitable for runtime (yet) - it's only used in tests. In order to use it at runtime we'd need to track those IDs per walked path.

Proposal

• state: Introduce new PathState: PathStateWalked for WalkerPath and reflect it in all methods

  terraform-ls/internal/state/walker_paths.go

  Lines 32 to 35 in c5a2090

  	const (
  	PathStateQueued PathState = iota
  	PathStateWalking
  	)

• state: Introduce new field to WalkerPath - JobIds job.IDs to track job IDs
• state: Introduce new field to WalkerPath - RootDir document.Dir to track the original root dir
• state: Introduce (*WalkerPathStore).WaitForRootDirWalked(dir) (job.IDs, error) to wait until all paths with the given RootDir are walked and return job IDs to wait for
• state: Introduce new argument to (*WalkerPathStore).EnqueueDir(dir, cleanupMode) - cleanupMode CleanupMode to enable delaying of walked dir removal until it was processed by WaitForDir() or WaitForRootDir()
  • state: Reflect cleanup mode in (*WalkerPathStore).waitForDir()
  • state: Reflect cleanup mode in (*WalkerPathStore).WaitForRootDirWalked()
  • walker: Reflect cleanup mode - i.e. avoid removing paths if cleanup mode is CleanupAfterWait
• langserver/handlers: Launch a go routine as part of initialize handler to WaitForDir() and also wait for jobs and then log time elapsed between enqueuing and finishing wait

type CleanupMode uint

const (
	CleanupAfterWalk CleanupMode = iota
	CleanupAfterWait
)