Fix heuristic for extension .yy (JSON vs Yacc)

[DecimalTurn]

Description

Closes #6517
496k+ files affected

Implements fix suggested by @lildude and backed by @RobQuistNL in #6517 (comment).

See this comment for the approach used: #6976 (comment)

Checklist:

• I am fixing a misclassified language
  • I have included a new sample for the misclassified language:
    • Sample source: https://github.com/Ttanasart-pt/Pixel-Composer/blob/363a1dec53f6785ee58a7b3f3b8407a48dcdd2c2/scripts/VCT/VCT.yy
    • Sample License: MIT license
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

[lildude]

Can we have a new sample and updated heuristic test, if necessary, for this too please.

[DecimalTurn]

Can we have a new sample and updated heuristic test, if necessary, for this too please.

Sure. I've added a sample JSON file with the .yy extension.

The test doesn't need any changes:

linguist/test/test_heuristics.rb

Lines 1087 to 1093 in 39fd5e9

	def test_yy_by_heuristics
	assert_heuristics({
	"JSON" => all_fixtures("JSON", "*.yy"),
	"Yacc" => all_fixtures("Yacc", "*.yy")
	})
	end
	end

[lildude]

I was referring to adding a new GMStudio sample that matches the heuristic change.

[DecimalTurn]

Those .yy files are produced by GMStudio, but they are (should be) classified as JSON by Linguist. That's why I've added the sample in the JSON folder.

The file added matches the heuristic added with the following line:
"resourceType":"GMScript",

[lildude]

🤦 Whoops. I missed that.

[lildude]

LGTM. Thanks.

Note: this PR will not be merged until close to when the next release is made. See here for more details.

[DecimalTurn]

🤦 Whoops. I missed that.

No worries. Let me know if you want me to fix the generated detection as well or if we can keep that for another PR.

linguist/lib/linguist/generated.rb

Lines 672 to 683 in 916bd8f

	# Internal: Is this a generated Game Maker Studio (2) metadata file?
	#
	# All Game Maker Studio 2 generated files will be JSON, .yy or .yyp, and have
	# a part that looks like "modelName: GMname" on the 3rd line
	#
	# Return true or false
	def generated_gamemakerstudio?
	return false unless ['.yy', '.yyp'].include? extname
	return false unless lines.count > 3
	return lines[2].match(/\"modelName\"\:\s*\"GM/) ||
	lines[0] =~ /^\d\.\d\.\d.+\|\{/
	end

[lildude]

Yeah, probably best to fix that too. Lets do it in this PR.

[DecimalTurn]

Should be all done now.
Note that I've had to relax the constraint that the property has to be on the 3rd line. There seems to be 50k+ files of GMStudio generated JSON where the old modelName property appears after the 3rd line, so even for GMStudio before v2.3, that constraint was too restrictive.

[DecimalTurn]

After looking a little more at the syntax for Yacc, I realize that a Yacc file cannot start with a { or [. It has to start with a declaration which begins with a %. This means that we could simply use the heuristic that we used for JSON from here (checking that the first non-whitespace character is either { or [):

linguist/lib/linguist/heuristics.yml

Lines 353 to 354 in 39fd5e9

	- language: JSON
	pattern: '\A\s*[{\[]'

And for the generated condition inside generated.rb , we could simply have something like:

      return lines.first(3).join('').match?(/^\s*[{\[]/) ||
             lines[0] =~ /^\d\.\d\.\d.+\|\{/

Thoughts?

---

Corresponding search query: Still 600K files.

[RobQuistNL]

That is actually a much better approach in my opinion :+) Good find! I've seen one other .yy syntax that I don't think is either GM or Yacc, but since this setup only has these 2 included I think this might be a very clean and good solution.

JSON + .yy extension = GM
No JSON + .yy extension = Yacc

[DecimalTurn]

We should be all good now with the latest changes bb34e54 (#6976).

I've seen one other .yy syntax that I don't think is either GM or Yacc

Good catch! This actually seems like an unusual Yacc file that skips the directives section going against the convention/documentation, so it would still be correct to categorize them as Yacc technically.

However, for this to be a problem, those unconventional Yacc files would need to start with a {, but GitHub Search tells me the number of files where this happens is likely in the single digits as it's struggling to find more than 3.

[RobQuistNL]

LGTM!

[lildude]

LGTM. Thanks.

Note: this PR will not be merged until close to when the next release is made. See here for more details.