Incorrect .rs files interpretation in language activity

[Pandicon]

I am currently mainly working in Rust and I was wondering why the "Language activity" section was saying I was using RenderScript (see screenshot). I recenly looked into it and found out that both Rust and RenderScript use the .rs file extension. Both the "Most used languages" and "Recently used languages" sections interpret them correctly as Rust source files.
I tried to alias RenderScript to Rust via plugin_languages_aliases: RenderScript:Rust, but it didn't affect the "Language activity" section so I undid it to not confuse me later. I am using the GitHub actions version using lowlighter/metrics@latest.
Is there a way to fix it/at least alias it? The files include all sorts of Rust things (pub, fn, use...) and are usually at least 100 lines long, so it shouldn't be an issue of not enough data.

[spenserblack]

Probably related to #1047 and #551.

Basically, like you said, RenderScript and Rust share the same extension, .rs. When metrics (via linguist-js) can't tell which language it is, it defaults to RenderScript. This happens because metrics breaks down your commits into patch files.

metrics/source/plugins/habits/index.mjs

Lines 42 to 55 in 552c2a2

	//Retrieve edited files and filter edited lines (those starting with +/-) from patches
	console.debug(`metrics/compute/${login}/plugins > habits > loading patches`)
	const patches = [
	...await Promise.allSettled(
	commits
	.flatMap(({payload}) => payload.commits)
	.filter(({author}) => data.shared["commits.authoring"].filter(authoring => author?.login?.toLocaleLowerCase().includes(authoring) || author?.email?.toLocaleLowerCase().includes(authoring) || author?.name?.toLocaleLowerCase().includes(authoring)).length)
	.map(async commit => (await rest.request(commit)).data.files),
	),
	]
	.filter(({status}) => status === "fulfilled")
	.map(({value}) => value)
	.flatMap(files => files.map(file => ({name: imports.paths.basename(file.filename), patch: file.patch ?? ""})))
	.map(({name, patch}) => ({name, patch: patch.split("\n").filter(line => /^[+]/.test(line)).map(line => line.substring(1)).join("\n")}))

Even if the file as a whole contains enough Rust-like code to be picked up by heuristics, it's likely that the patches won't. This might relate to your commit style, too: big monolithic commits might result in more accurate language metrics than small, atomic commits (though the latter is much better IMO).

[spenserblack]

The analyzer is supposed to fetch your .gitattributes, so that might be a bug.

metrics/source/plugins/habits/index.mjs

Line 2 in 552c2a2

import { recent as recent_analyzer } from "./../languages/analyzers.mjs"

metrics/source/plugins/habits/index.mjs

Line 106 in 552c2a2

const {total, stats} = await recent_analyzer({login, data, imports, rest, account}, {days, load: from || 1000, tempdir: "habits"})

metrics/source/plugins/languages/analyzers.mjs

Line 165 in 552c2a2

await imports.fs.writeFile(imports.paths.join(path, directory, ".gitattributes"), await imports.fetch(`https://raw.githubusercontent.com/${repo}/${branch}/.gitattributes`).then(response => response.text()).catch(() => ""))

[lowlighter]

Not sure why the .gitattributes isn't loaded correctly 😕

But I think one way to solve this issue may be to fetch the whole file to perform github-linguist analysis on it, and only count lines/bytes from the patch itself so in theory the language should be detected more accurately with more context while the stats remain the same

[Pandicon]

Even after I pushed a 4.77kB file with multiple struct definitions and impl blocks for them, nothing was reflected in the stats. I don't know how the patches work, but I suppose it is the added/removed highlighting GitHub shows. If that is the case, in the commit the patch should contain the whole file, including a lot of clearly Rust stuff (use, struct, impl, ...). Not sure how helpful this will be, but if I understand how the patches work, I feel like fetching the whole file may not really help.

[spenserblack]

@lowlighter The patch's lines start with - or +, right?

@Pandicon That can be why use, impl, etc. aren't picked up by the heuristics.
The current regex matches on a line like use, but not a line like +use.

[lowlighter]

The patch's lines start with - or +, right?

Yeah, patches looks like this:

(e.g. 3d1d5ed)

It's basically what it output by a git diff

The + part is supposed to be trimmed by this line:

metrics/source/plugins/languages/analyzers.mjs

Line 143 in 8e38168

.map(({name, directory, patch, repo}) => ({name, directory: `${repo.replace(/[/]/g, "@")}/${directory}`, patch: patch.split("\n").filter(line => /^[+]/.test(line)).map(line => line.substring(1)).join("\n")}))

But added lines are not trimmed (it just remove the + char), so that'd why it probably doesn't match heuristics since they have the ^ in the regex with no whitespaces