feat(linter/plugins): implement SourceCode#getAllComments

[lilnasy]

• Towards Linter plugins: Add comments-related APIs #14564

Future PRs

• look into RawTransferData { program.comments, comments } duplication.
• oxlint's interface Program doesn't include the comments type.
• lazy deserialization
• remove comment.parent
• UTF8 to UTF16 conversion on the JS side
• correct start/end offsets
• getCommentsBefore
• getCommentsAfter
• getCommentsInside
• commentsExistBetween

[graphite-app]

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• 0-merge - adds this PR to the back of the merge queue
• hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

[Copilot]

Pull Request Overview

This PR implements the SourceCode#getAllComments method for linter plugins by enabling comment serialization in the AST and exposing comments through the JavaScript plugin API.

Key changes:

• Enables comment serialization in AST Program nodes by removing the #[estree(skip)] attribute
• Updates TypeScript type definitions to include Comment interface exports
• Implements the getAllComments() method in the SourceCode API
• Adds comprehensive test coverage for the new functionality

Reviewed Changes

Copilot reviewed 10 out of 24 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
crates/oxc_ast/src/ast/js.rs	Removes estree skip attribute to enable comment serialization in Program nodes
crates/oxc_ast/src/serialize/mod.rs	Adds comments field serialization to Program ESTree output
napi/parser/src/raw_transfer.rs	Refactors comment handling to avoid unnecessary memory operations and use CloneIn trait
tasks/ast_tools/src/generators/typescript.rs	Updates TypeScript definitions to export Comment interface alongside Span
apps/oxlint/src-js/plugins/source_code.ts	Implements getAllComments method to return ast.comments array
apps/oxlint/test/fixtures/getAllComments/*	Adds comprehensive test fixture with plugin, test files, and expected output
apps/oxlint/test/e2e.test.ts	Adds end-to-end test for getAllComments functionality

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[codspeed-hq]

CodSpeed Performance Report

Merging #14589 will not alter performance

_{Comparing lilnasy:comment-related-apis-1 (09bd314) with main (463be73)¹}

Summary

✅ 37 untouched

Footnotes

1. No successful run was found on main (9e54002) during the generation of this report, so 463be73 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[overlookmotel]

Thanks for jumping on this!

This is a little tricky. Raw transfer deserializer is used in both oxc-parser and oxlint, but we want them to do different things. I think probably:

• In parser, we don't want to add comments as a field on program (so as to avoid a breaking change).
• In Oxlint, we do (because ESLint does).
• In parser, comments has to be deserialized eagerly (because the buffer that AST is in may be reused for another file after parsing the current one).
• In Oxlint, we may want to make comments a lazy getter, to avoid deserializing them unless user accesses them.

Hmm... let me think about this a little.

Context: The slowest part of raw transfer deserializer is deserializing strings, because it often involves a call into C++ (TextDecoder#decode). Comments are often long strings, so if we can avoid deserializing them unless they're needed, that'd be a win.

[Copilot]

Pull Request Overview

Copilot reviewed 9 out of 19 changed files in this pull request and generated 2 comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[overlookmotel]

Sorry I'm slow to come back. Here's what I think we should do:

• Leave the parser alone, only add comments to program in Oxlint.
• Not worry about lazily deserializing comments for now - we can try that in a follow-up.

We should be able to do that as follows:

Alter codegen

Here's the code which generates different variants of the deserializer for parser and oxlint:

oxc/tasks/ast_tools/src/generators/raw_transfer.rs

Lines 212 to 262 in 9e54002

	// Create deserializers with various settings, by setting `IS_TS`, `RANGE`, `LOC`, `PARENT`
	// and `PRESERVE_PARENS` consts, and running through minifier to shake out irrelevant code
	struct VariantGen {
	variant_paths: Vec<String>,
	}
	impl VariantGenerator<5> for VariantGen {
	const FLAG_NAMES: [&str; 5] = ["IS_TS", "RANGE", "LOC", "PARENT", "PRESERVE_PARENS"];
	fn variants(&mut self) -> Vec<[bool; 5]> {
	let mut variants = Vec::with_capacity(9);
	for is_ts in [false, true] {
	for range in [false, true] {
	for parent in [false, true] {
	self.variant_paths.push(format!(
	"{NAPI_PARSER_PACKAGE_PATH}/generated/deserialize/{}{}{}.js",
	if is_ts { "ts" } else { "js" },
	if range { "_range" } else { "" },
	if parent { "_parent" } else { "" },
	));
	variants.push([
	is_ts, range, /* loc */ false, parent,
	/* preserve_parens */ true,
	]);
	}
	}
	}
	self.variant_paths.push(format!("{OXLINT_APP_PATH}/src-js/generated/deserialize.js"));
	variants.push([
	/* is_ts */ true, /* range */ true, /* loc */ true,
	/* parent */ true, /* preserve_parens */ false,
	]);
	variants
	}
	fn pre_process_variant<'a>(
	&mut self,
	program: &mut Program<'a>,
	flags: [bool; 5],
	allocator: &'a Allocator,
	) {
	if flags[2] {
	// `loc` enabled
	LocFieldAdder::new(allocator).visit_program(program);
	}
	}
	}

• Add "COMMENTS" to end of const FLAG_NAMES.
• Set that flag to false for all the parser deserializers (variants.push(...) in the triple-nested loop).
• Set it to true for the linter deserializer (the single variants.push(...) at the end).

Alter deserializer code template for Program

oxc/crates/oxc_ast/src/serialize/mod.rs

Lines 123 to 173 in 9e54002

	#[estree(raw_deser = "
	const start = IS_TS ? 0 : DESER[u32](POS_OFFSET.span.start),
	end = DESER[u32](POS_OFFSET.span.end);
	const program = parent = {
	type: 'Program',
	body: null,
	sourceType: DESER[ModuleKind](POS_OFFSET.source_type.module_kind),
	hashbang: null,
	start,
	end,
	...(RANGE && { range: [start, end] }),
	...(PARENT && { parent: null }),
	};
	program.hashbang = DESER[Option<Hashbang>](POS_OFFSET.hashbang);
	const body = program.body = DESER[Vec<Directive>](POS_OFFSET.directives);
	body.push(...DESER[Vec<Statement>](POS_OFFSET.body));
	if (IS_TS) {
	let start;
	if (body.length > 0) {
	const first = body[0];
	start = first.start;
	if (first.type === 'ExportNamedDeclaration' || first.type === 'ExportDefaultDeclaration') {
	const { declaration } = first;
	if (
	declaration !== null && declaration.type === 'ClassDeclaration'
	&& declaration.decorators.length > 0
	) {
	const decoratorStart = declaration.decorators[0].start;
	if (decoratorStart < start) start = decoratorStart;
	}
	}
	} else {
	start = end;
	}
	if (RANGE) {
	program.start = program.range[0] = start;
	} else {
	program.start = start;
	}
	}
	if (PARENT) parent = null;
	program
	")]
	pub struct ProgramConverter<'a, 'b>(pub &'b Program<'a>);

• Add ...(COMMENTS && { comments: DESER[Vec<Comment>](POS_OFFSET.comments) }), after hashbang: null.

Codegen will set COMMENTS to false in all the deserializers except the linter one, and then minifier will shake out the dead code.

What else?

Because we're not altering the parser, that should sidestep problem of conformance tests failing.

Next steps:

• Comments will have an extraneous parent field. We'll need to remove it.
• Translate start / end of comments from UTF-8 bytes (Rust) to UTF-16 chars (JS), same as we do for rest of the AST (in Linter::run_external_rules in oxc_linter, see napi/parser).
• Add tests for correctness of comment start / end in files containing unicode characters.
• Implement the rest of the comment-related APIs in SourceCode.

We can deal with all of those in follow-up PRs.

How does that sound?

[lilnasy]

That sounds great! Thanks for the guidance, it didn't take me long to get everything up and running after it.

[overlookmotel]

Looks good!

I pushed a couple of commits - I hope you don't mind, I find it often saves time vs going back-and-forth on review. One just reverts changes to parser that we no longer need, and the other is total style nit. The latter is not at all important, and really I should have left it alone - I'm sorry, there's some tedious fastidiousness in me which I can't entirely control.

Merging now. Great!