Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for High-FPS Readings in Flashlight for Devices Over 60 FPS #316

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

MalcolmTomisin
Copy link

@MalcolmTomisin MalcolmTomisin commented Aug 25, 2024

Description

This PR introduces a new feature to Flashlight that extends support for devices capable of displaying frame rates (FPS) higher than 60Hz. It addresses this issue

Key Changes:

High-FPS Support: Adjustments have been made to ensure that FPS readings above 60Hz are correctly captured and displayed by the flashlight tool.
Data-model: A DeviceSpecs type has been created, including the refresh rate into the Results type didn't seem like an optimal way to model the results of profiling.
Testing: Added unit tests to verify the functionality of the adb command for fps information.
Minor: Extracting socket events into enums

Challenges

There's a lot of prop drilling going on, perhaps, including the refresh rate into the results type could have avoided all the prop drilling, looking forward to comments on that.

Additional Notes

  • This update is backward-compatible with devices that have a refresh rate of 60Hz or below.
  • No significant impact on existing functionality or performance.
  • Further improvements could come through a DeviceSpecs type. Adding device specs of the device being profiled could be helpful metadata to add to reports.

Screenshots

image
image

Copy link

netlify bot commented Aug 25, 2024

Deploy Preview for flashlight-docs ready!

Name Link
🔨 Latest commit 3fe02ab
🔍 Latest deploy log https://app.netlify.com/sites/flashlight-docs/deploys/66ef4e4fcbbc860008a1a886
😎 Deploy Preview https://deploy-preview-316--flashlight-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@@ -12,7 +12,11 @@ Time taken to run the test.
Can be helpful to measure Time To Interactive of your app, if the test is checking app start for instance.
Average FPS
60 FPS
Frame Per Second. Your app should display 60 Frames Per Second to give an impression of fluidity. This number should be close to 60, otherwise it will seem laggy.
Frame Per Second. Your app should display
60
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Head scratching moment why this change happened

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did the snapshot update come from this commit specifically?

@MalcolmTomisin MalcolmTomisin marked this pull request as ready for review August 25, 2024 21:57
Copy link
Member

@Almouro Almouro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @MalcolmTomisin !

First of all, apologies for the delay answering and thanks so much for submitting this great PR!! ❤️
This has been an open issue for some time now, and it's so cool to see someone spending a lot of time to fix it! 🙏

I've added some comments here and there, but I have to ask: what is the reasoning behind not adding the refresh rate in the results object?
It seems to me that adding the device specs directly there would simplify the code.

Also flashlight relies in a lot of place on a simple json file being the source of truth for a report. For instance, flashlight test outputs a json that can be used by flashlight report, and flashlight measure can also output one. flashlight cloud also has a similar mechanism.
So putting the device specs out of it might make it tricky to have for instance:

  • flashlight test recognizing 120FPS and outputting results in json file
  • flashlight report displaying the report for the json file, taking into account that the json file was a result for a 120FPS phone

I'll be away again for 10 days without access to computer, but I'm very happy to discuss this by message over Discord/Twitter/... if you want to get this in as soon as possible 🤞

@@ -12,7 +12,11 @@ Time taken to run the test.
Can be helpful to measure Time To Interactive of your app, if the test is checking app start for instance.
Average FPS
60 FPS
Frame Per Second. Your app should display 60 Frames Per Second to give an impression of fluidity. This number should be close to 60, otherwise it will seem laggy.
Frame Per Second. Your app should display
60
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did the snapshot update come from this commit specifically?

@MalcolmTomisin
Copy link
Author

MalcolmTomisin commented Sep 6, 2024

Hi @MalcolmTomisin !

First of all, apologies for the delay answering and thanks so much for submitting this great PR!! ❤️ This has been an open issue for some time now, and it's so cool to see someone spending a lot of time to fix it! 🙏

I've added some comments here and there, but I have to ask: what is the reasoning behind not adding the refresh rate in the results object? It seems to me that adding the device specs directly there would simplify the code.

Also flashlight relies in a lot of place on a simple json file being the source of truth for a report. For instance, flashlight test outputs a json that can be used by flashlight report, and flashlight measure can also output one. flashlight cloud also has a similar mechanism. So putting the device specs out of it might make it tricky to have for instance:

  • flashlight test recognizing 120FPS and outputting results in json file
  • flashlight report displaying the report for the json file, taking into account that the json file was a result for a 120FPS phone

I'll be away again for 10 days without access to computer, but I'm very happy to discuss this by message over Discord/Twitter/... if you want to get this in as soon as possible 🤞

I’ve considered adding the refreshRate to the results JSON, but I wasn’t sure how to structure it in a way that maintains backward compatibility, I also want to avoid duplicating the refreshRate across multiple report arrays. You had suggested adding it to both TestCaseResult and AveragedTestCaseResult, but since the refreshRate wouldn’t change during a profiling session, I felt it might be redundant.

Additionally, I’m thinking about including device specifications in the reports, which I believe would be a significant benefit for transparency when reporting performance data.
What do you recommend?

@Almouro
Copy link
Member

Almouro commented Sep 6, 2024

We could add a deviceSpecs field to both TestCaseResult and AveragedTestCaseResult
Yeah I agree it's a bit weird to have it in both, AveragedTestCaseResult is a bit of a weird object at the moment to be fair and already duplicates the name of the report for instance. It would just get copied over right here https://github.com/bamlab/flashlight/blob/main/packages/core/reporter/src/reporting/averageIterations.ts#L76

We could make the field accept an undefined value for backwards compatibility.

What do you think?

@MalcolmTomisin
Copy link
Author

We could add a deviceSpecs field to both TestCaseResult and AveragedTestCaseResult Yeah I agree it's a bit weird to have it in both, AveragedTestCaseResult is a bit of a weird object at the moment to be fair and already duplicates the name of the report for instance. It would just get copied over right here https://github.com/bamlab/flashlight/blob/main/packages/core/reporter/src/reporting/averageIterations.ts#L76

We could make the field accept an undefined value for backwards compatibility.

What do you think?

Sounds fine to me.

Use regular expression to find all matches of fps details from the adb command response. A contrived algorithm was used previously, relying on different regex constructs to isolate sections of the adb response strings such as strings that had the following format - "refreshRate %d", "fps=%d". The "refreshRate" string was dropped for the more common "fps".

chore(profiler): update base types with new method

Adding detectDeviceRefreshRate method to the base and sub classes. An implementation templation that can be adopted across multiple platforms to provide GPU refresh rate of the devices being profiled
chore: use device specs type

A new device spec has been created for use to model device spec relevant to performance

test: update snapshots and test cases
…ADB command

refactor: remove unnecessary catch block
Some devices come with the capabilities of switching between 60,90, 120 fps. The renderFrameRate reports the currently configured refresh rate.
For consistency, refresh rate is added to the DeviceSpecs property of the TestCaseResult and AveragedTestCaseResult

chore(test): update snapshots

refactor: remove unneeded changes

refactor: use logger for error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants