Different font rendering between Unix and macOS

[neustaTerasa]

I tried the golden toolkit and it works great on my machine. I'm working on a MacBook with macOS 10.15.6.

Our CI is running on a unix machine and the golden tests always fail. On the first look the screens look the same. If you look carefully, you can see little differences around texts. Obviously the anti aliasing works diffently.

If my colleague creates the goldens on his unix machine, they match on the CI.

I already tried to turn off the font smoothing in the macOS system settings. Then the rendered text is looking diffently than with smoothing. But also still differently from unix.

Is there anything we can do about? Or can you fix that in the toolkit?

[tsimbalar]

FYI : the usual way is to disable Golden Tests completely on platforms that are not the ones the developers usually work on :-/

you can do it like so :

flutter_glove_box/packages/golden_toolkit/test/flutter_test_config.dart

Lines 15 to 28 in fcbd020

	Future<void> main(FutureOr<void> Function() testMain) async {
	return GoldenToolkit.runWithConfiguration(
	() async {
	await loadAppFonts();
	await testMain();
	},
	config: GoldenToolkitConfiguration(
	// Currently, goldens are not generated/validated in CI for this repo. We have settled on the goldens for this package
	// being captured/validated by developers running on MacOSX. We may revisit this in the future if there is a reason to invest
	// in more sophistication
	skipGoldenAssertion: () => !Platform.isMacOS,
	),
	);
	}

Another alternative is to use the "Ahem" font which shows opaque black blocks instead of the real font ...

[fatherOfLegends]

@neustaTerasa This is a Flutter issue (flutter/flutter#17483) that said on my team we all use macs for development so we skip golden assertions on when tests are not running on macs like the example @tsimbalar shows. We then have a mac in our CI pipelines that runs only the golden tests.

[neustaTerasa]

Ok, I tried that and it works, we also have a Mac runner in our CI. Not an ideal solution because we also have colleagues that are working on unix machines, but at the current project all members have macs.
Thank you.

[patmuk]

The closed flutter issue (flutter/flutter#17483) unfortunately does not fix the font difference issue between MacOS and linux. Is there an idea to solve this issue? Is there a workaround some team practices? We unfortunately have Macs, Linux boxes and even Windows machines for our developers in our team.

[coreysprague]

@patmuk As fas as I am aware, there are only 2 real choices:

1. have developers regenerate goldens locally, and have all developers on the same OS (can optionally validate via CI if running on the same OS) the benefit of this approach is that developers can leverage goldens as a tool during their development process
2. have CI be the source of truth and pick an OS -- have a CI command to generate the new golden references and commit them, and have CI run tests against the current goldens (This is what the Flutter team does). I think this is probably the best setup, but requires a lot more orchestration in your automation pipeline. the benefit is it supports multiple development environments, but it requires developers to lean on CI for validation

[patmuk]

@coreysprague wow, that was a fast reply! Many thanks!
Sounds reasonable.