This repository was archived by the owner on Feb 25, 2025. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 6k
[Impeller] Adds golden image tests. #40366
Merged
Merged
Changes from all commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
d6d5695
Added golden image tests to impeller
gaaclarke b17819e
started writing a digest
gaaclarke 345d019
added docstrings
gaaclarke 93725ba
added harvester
gaaclarke aca9c66
updated readme
gaaclarke 1ec9eb3
added logger wrapper
gaaclarke d079bd2
made the tests only run on mac
gaaclarke 3141ca1
updated the run_tests.py script
gaaclarke f8c4431
added readme
gaaclarke 0a6757f
made sure that the golden image tests are only compiled for mac
gaaclarke b881ff5
updated licenses
gaaclarke 28ef8ff
lint fixes for run_tests.py
gaaclarke e9e796a
updated licenses again
gaaclarke 9d902bb
more lints
gaaclarke e87078c
unused import
gaaclarke 83c93fe
sorted imports
gaaclarke 5735ac3
turned on the golden tests by default
gaaclarke 1f863c9
fixed remote execution
gaaclarke 3f00b5d
turned the tests off for the engine runs, going to specify it in the …
gaaclarke 7d0058b
added goldctl dependency to "Mac host engine"
gaaclarke d0c5781
tried switching to engine_v2 to run the golden tests
gaaclarke 386ddb4
tried adding the dependency to the drone
gaaclarke bfa2200
tried flipping the image in ciimage since it wasn't working when writ…
gaaclarke 8267da7
switched to sending the filename to skia gold client, since the code …
gaaclarke 79cc6fd
zach feedback 1
gaaclarke 02b9f03
updated licenses golden
gaaclarke File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| # Copyright 2013 The Flutter Authors. All rights reserved. | ||
| # Use of this source code is governed by a BSD-style license that can be | ||
| # found in the LICENSE file. | ||
|
|
||
| import("//flutter/common/config.gni") | ||
| import("//flutter/impeller/tools/impeller.gni") | ||
|
|
||
| if (is_mac) { | ||
| test_fixtures("impeller_golden_tests_fixtures") { | ||
| fixtures = [] | ||
| } | ||
|
|
||
| impeller_component("impeller_golden_tests") { | ||
| target_type = "executable" | ||
|
|
||
| testonly = true | ||
|
|
||
| sources = [ | ||
| "golden_digest.cc", | ||
| "golden_digest.h", | ||
| "golden_tests.cc", | ||
| "main.cc", | ||
| "metal_screenshot.h", | ||
| "metal_screenshot.mm", | ||
| "metal_screenshoter.h", | ||
| "metal_screenshoter.mm", | ||
| "working_directory.cc", | ||
| "working_directory.h", | ||
| ] | ||
|
|
||
| deps = [ | ||
| ":impeller_golden_tests_fixtures", | ||
| "//flutter/impeller/aiks", | ||
| "//flutter/impeller/playground", | ||
| "//flutter/impeller/renderer/backend/metal:metal", | ||
| "//third_party/googletest:gtest", | ||
| ] | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Impeller Golden Tests | ||
|
|
||
| This is the executable that will generate the golden image results that can then | ||
| be sent to Skia Gold vial the | ||
| [golden_tests_harvester]("../golden_tests_harvester"). | ||
|
|
||
| Running these tests should happen from | ||
| [//flutter/testing/run_tests.py](../../testing/run_tests.py). That will do all | ||
| the steps to generate the golden images and transmit them to Skia Gold. If you | ||
| run the tests locally it will not actually upload anything. That only happens if | ||
| the script is executed from LUCI. | ||
|
|
||
| Example invocation: | ||
|
|
||
| ```sh | ||
| ./run_tests.py --variant="host_debug_unopt_arm64" --type="impeller-golden" | ||
| ``` | ||
|
|
||
| Currently these tests are only supported on macOS and only test the Metal | ||
| backend to Impeller. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| // Copyright 2013 The Flutter Authors. All rights reserved. | ||
| // Use of this source code is governed by a BSD-style license that can be | ||
| // found in the LICENSE file. | ||
|
|
||
| #include "impeller/golden_tests/golden_digest.h" | ||
|
|
||
| #include <fstream> | ||
|
|
||
| namespace impeller { | ||
| namespace testing { | ||
|
|
||
| GoldenDigest* GoldenDigest::instance_ = nullptr; | ||
|
|
||
| GoldenDigest* GoldenDigest::Instance() { | ||
| if (!instance_) { | ||
| instance_ = new GoldenDigest(); | ||
| } | ||
| return instance_; | ||
| } | ||
|
|
||
| GoldenDigest::GoldenDigest() {} | ||
|
|
||
| void GoldenDigest::AddImage(const std::string& test_name, | ||
| const std::string& filename, | ||
| int32_t width, | ||
| int32_t height) { | ||
| entries_.push_back({test_name, filename, width, height}); | ||
| } | ||
|
|
||
| bool GoldenDigest::Write(WorkingDirectory* working_directory) { | ||
| std::ofstream fout; | ||
| fout.open(working_directory->GetFilenamePath("digest.json")); | ||
| if (!fout.good()) { | ||
| return false; | ||
| } | ||
|
|
||
| fout << "[" << std::endl; | ||
| bool is_first = true; | ||
| for (const auto& entry : entries_) { | ||
| if (!is_first) { | ||
| fout << "," << std::endl; | ||
| is_first = false; | ||
| } | ||
| fout << " { " | ||
| << "\"testName\" : \"" << entry.test_name << "\", " | ||
| << "\"filename\" : \"" << entry.filename << "\", " | ||
| << "\"width\" : " << entry.width << ", " | ||
| << "\"height\" : " << entry.height << " " | ||
| << "}"; | ||
| } | ||
| fout << std::endl << "]" << std::endl; | ||
|
|
||
| fout.close(); | ||
| return true; | ||
| } | ||
|
|
||
| } // namespace testing | ||
| } // namespace impeller |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| // Copyright 2013 The Flutter Authors. All rights reserved. | ||
| // Use of this source code is governed by a BSD-style license that can be | ||
| // found in the LICENSE file. | ||
|
|
||
| #pragma once | ||
|
|
||
| #include <string> | ||
| #include <vector> | ||
|
|
||
| #include "flutter/fml/macros.h" | ||
| #include "flutter/impeller/golden_tests/working_directory.h" | ||
|
|
||
| namespace impeller { | ||
| namespace testing { | ||
|
|
||
| /// Manages a global variable for tracking instances of golden images. | ||
| class GoldenDigest { | ||
| public: | ||
| static GoldenDigest* Instance(); | ||
|
|
||
| void AddImage(const std::string& test_name, | ||
| const std::string& filename, | ||
| int32_t width, | ||
| int32_t height); | ||
|
|
||
| /// Writes a "digest.json" file to `working_directory`. | ||
| /// | ||
| /// Returns `true` on success. | ||
| bool Write(WorkingDirectory* working_directory); | ||
|
|
||
| private: | ||
| FML_DISALLOW_COPY_AND_ASSIGN(GoldenDigest); | ||
| GoldenDigest(); | ||
| struct Entry { | ||
| std::string test_name; | ||
| std::string filename; | ||
| int32_t width; | ||
| int32_t height; | ||
| }; | ||
|
|
||
| static GoldenDigest* instance_; | ||
| std::vector<Entry> entries_; | ||
| }; | ||
| } // namespace testing | ||
| } // namespace impeller | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| // Copyright 2013 The Flutter Authors. All rights reserved. | ||
| // Use of this source code is governed by a BSD-style license that can be | ||
| // found in the LICENSE file. | ||
|
|
||
| #include "gtest/gtest.h" | ||
|
|
||
| #include <sstream> | ||
|
|
||
| #include "impeller/aiks/canvas.h" | ||
| #include "impeller/entity/contents/conical_gradient_contents.h" | ||
| #include "impeller/geometry/path_builder.h" | ||
| #include "impeller/golden_tests/golden_digest.h" | ||
| #include "impeller/golden_tests/metal_screenshot.h" | ||
| #include "impeller/golden_tests/metal_screenshoter.h" | ||
| #include "impeller/golden_tests/working_directory.h" | ||
|
|
||
| namespace impeller { | ||
| namespace testing { | ||
|
|
||
| namespace { | ||
| std::string GetTestName() { | ||
| std::string suite_name = | ||
| ::testing::UnitTest::GetInstance()->current_test_suite()->name(); | ||
| std::string test_name = | ||
| ::testing::UnitTest::GetInstance()->current_test_info()->name(); | ||
| std::stringstream ss; | ||
| ss << "impeller_" << suite_name << "_" << test_name; | ||
| return ss.str(); | ||
| } | ||
|
|
||
| std::string GetGoldenFilename() { | ||
| return GetTestName() + ".png"; | ||
| } | ||
|
|
||
| bool SaveScreenshot(std::unique_ptr<MetalScreenshot> screenshot) { | ||
| if (!screenshot || !screenshot->GetBytes()) { | ||
| return false; | ||
| } | ||
| std::string test_name = GetTestName(); | ||
| std::string filename = GetGoldenFilename(); | ||
| GoldenDigest::Instance()->AddImage( | ||
| test_name, filename, screenshot->GetWidth(), screenshot->GetHeight()); | ||
| return screenshot->WriteToPNG( | ||
| WorkingDirectory::Instance()->GetFilenamePath(filename)); | ||
| } | ||
| } // namespace | ||
|
|
||
| class GoldenTests : public ::testing::Test { | ||
| public: | ||
| GoldenTests() : screenshoter_(new MetalScreenshoter()) {} | ||
|
|
||
| MetalScreenshoter& Screenshoter() { return *screenshoter_; } | ||
|
|
||
| private: | ||
| std::unique_ptr<MetalScreenshoter> screenshoter_; | ||
| }; | ||
|
|
||
| TEST_F(GoldenTests, ConicalGradient) { | ||
| Canvas canvas; | ||
| Paint paint; | ||
| paint.color_source_type = Paint::ColorSourceType::kConicalGradient; | ||
| paint.color_source = []() { | ||
| auto result = std::make_shared<ConicalGradientContents>(); | ||
| result->SetCenterAndRadius(Point(125, 125), 125); | ||
| result->SetColors({Color(1.0, 0.0, 0.0, 1.0), Color(0.0, 0.0, 1.0, 1.0)}); | ||
| result->SetStops({0, 1}); | ||
| result->SetFocus(Point(180, 180), 0); | ||
| result->SetTileMode(Entity::TileMode::kClamp); | ||
| return result; | ||
| }; | ||
| paint.stroke_width = 0.0; | ||
| paint.style = Paint::Style::kFill; | ||
| canvas.DrawRect(Rect(10, 10, 250, 250), paint); | ||
| Picture picture = canvas.EndRecordingAsPicture(); | ||
| auto screenshot = Screenshoter().MakeScreenshot(std::move(picture)); | ||
| ASSERT_TRUE(SaveScreenshot(std::move(screenshot))); | ||
| } | ||
| } // namespace testing | ||
| } // namespace impeller |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No action needed, but why isn't this just included in the file itself? Or is this for a human verifying to make sure something didn't go wrong if the file is suddenly smaller than expected or something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Storing it in the image file would require a library to parse the image to get the width and height in the harvester. Since we already know it here, it was more convenient to report it. Our dart instances in the engine have a limited set of packages available so they might not even have the ability to easily parse image headers.