Skip to content

Commit c61fd7b

Browse files
lukehbBelchy06github-actions[bot]mcottontensor
authored
Feature: Dump latency stats to CSV | Re-ship: 4.27 browser send offer support (#647)
* Add 4.27 support back for latency testing * Save webrtc stats to csv and auto download * Deep copy stats object * Update record parsing logic to handle missing field when generating the csv * Update session test to handle additional latency stats * Fixing path setup in the windows platform scripts. (#594) (#595) (cherry picked from commit 152abd6) Co-authored-by: mcottontensor <80377552+mcottontensor@users.noreply.github.com> (cherry picked from commit 1dedc87) * Fixing more path issues in platform_scripts (#604) * feat: Adding script tests where the scripts are run multiple times. This tests that the scripts still work when things have already been installed. * fix: Fixing typo in github action. * gimme a break. * Adding mutliple script runs for other OSs * Testing node on the runner * Trying to figure out what node is doing what. * Trying to change the way node version is detected. * Messy * The additions to the script really didn't catch anything. Could be worth revisiting how we handle the local node install. * Removing debug print. (cherry picked from commit 2321fa9) * refactor: Better handling of node in Linux scripts. (#607) Updated the way the bash script sets up the environment for node. Node will only be installed if a version meeting the minimum requirements is not found. The new environment should result in a more robust environment setup. (cherry picked from commit f27bace) * Fixing a small issue with how path was setup previously. (#611) If the node install was installed locally previously the script would not pick that up and try to install again. It would fail because the directory already exists and then the path would not get properly setup for the local install of node. This change sets PATH to point to the local install anyway. If it doesn't exist it doesn't matter, if it does exist we use the local install. (cherry picked from commit 00aaa52) * Added disable button when session test is running * Added new flag ?LatencyCSV to enable the session test and its UI * Added changeset for 4.27 support and session test feature --------- Co-authored-by: Will Belcher <william.belcher@xa.epicgames.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mcottontensor <80377552+mcottontensor@users.noreply.github.com>
1 parent 1ba1bf5 commit c61fd7b

File tree

10 files changed

+334
-6
lines changed

10 files changed

+334
-6
lines changed

.backportrc.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"repoOwner": "EpicGamesExt",
33
"repoName": "PixelStreamingInfrastructure",
4-
"targetBranchChoices": ["master", "UE5.2", "UE5.3", "UE5.4", "UE5.5", "UE5.6", "LatencyTest"],
4+
"targetBranchChoices": ["master", "UE5.2", "UE5.3", "UE5.4", "UE5.5", "UE5.6"],
55
"autoMerge": true,
66
"autoMergeMethod": "squash"
77
}

.changeset/weak-files-wait.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
'@epicgames-ps/lib-pixelstreamingfrontend-ui-ue5.6': minor
3+
'@epicgames-ps/lib-pixelstreamingfrontend-ue5.6': minor
4+
---
5+
6+
## Latency Session Test and dump to csv
7+
8+
Added a new feature to run a variable length latency test session (e.g. a 60s window)
9+
and dump that stats from the session to two .csv files:
10+
11+
1. latency.csv - Which contains the video timing stats
12+
2. stats.csv - Which contains all WebRTC stats the library currently tracks
13+
14+
To enable the latency session test use the flag/url parameter ?LatencyCSV
15+
to enable this feature (by default it is disabled and not UI-configurable).
16+
17+
To use this latency session test feature:
18+
19+
1. Navigate to http://localhost/?LatencyCSV
20+
2. Open the stats panel and click the "Run Test" button under the "Session Test" heading.
21+
22+
## 4.27 support restored
23+
24+
Re-shipped UE 4.27 support by restoring the ?BrowserSendOffer flag.
25+
It was found useful to support running this latency session test against UE 4.27
26+
for internal historical testing so support for connecting to this version has been restored.
27+
28+
To connect to a 4.27 project:
29+
30+
1. Navigate to http://localhost/?BrowserSendOffer
31+
2. Connect (warning: this option is not compatible with all newer UE versions)

Frontend/library/src/Config/Config.ts

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ export class Flags {
3636
static WaitForStreamer = 'WaitForStreamer' as const;
3737
static HideUI = 'HideUI' as const;
3838
static EnableCaptureTimeExt = 'EnableCaptureTimeExt' as const;
39+
static BrowserSendOffer = 'BrowserSendOffer' as const;
40+
static LatencyCSV = 'LatencyCSV' as const;
3941
}
4042

4143
export type FlagsKeys = Exclude<keyof typeof Flags, 'prototype'>;
@@ -580,6 +582,32 @@ export class Config {
580582
)
581583
);
582584

585+
this.flags.set(
586+
Flags.BrowserSendOffer,
587+
new SettingFlag(
588+
Flags.BrowserSendOffer,
589+
'Browser send offer (4.27 ONLY)',
590+
'Browser will initiate the WebRTC handshake by sending the offer to the streamer (4.27 ONLY)',
591+
settings && Object.prototype.hasOwnProperty.call(settings, Flags.BrowserSendOffer)
592+
? settings[Flags.BrowserSendOffer]
593+
: false,
594+
useUrlParams
595+
)
596+
);
597+
598+
this.flags.set(
599+
Flags.LatencyCSV,
600+
new SettingFlag(
601+
Flags.LatencyCSV,
602+
'Export Latency CSV',
603+
'Shows a button in the stats panel that allows to run a latency test and export the results to a CSV file.',
604+
settings && Object.prototype.hasOwnProperty.call(settings, Flags.LatencyCSV)
605+
? settings[Flags.LatencyCSV]
606+
: false,
607+
useUrlParams
608+
)
609+
);
610+
583611
/**
584612
* Numeric parameters
585613
*/

Frontend/library/src/PeerConnectionController/AggregatedStats.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,10 @@ export class AggregatedStats {
3434
constructor() {
3535
this.inboundVideoStats = new InboundVideoStats();
3636
this.inboundAudioStats = new InboundAudioStats();
37+
this.candidatePairs = new Array<CandidatePairStats>();
3738
this.datachannelStats = new DataChannelStats();
39+
this.localCandidates = new Array<CandidateStat>();
40+
this.remoteCandidates = new Array<CandidateStat>();
3841
this.outboundVideoStats = new OutboundRTPStats();
3942
this.outboundAudioStats = new OutboundRTPStats();
4043
this.remoteOutboundAudioStats = new RemoteOutboundRTPStats();

Frontend/library/src/WebRtcPlayer/WebRtcPlayerController.ts

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -206,8 +206,11 @@ export class WebRtcPlayerController {
206206
this.handleIceCandidate(iceCandidateMessage.candidate);
207207
});
208208
this.protocol.transport.addListener('open', () => {
209-
const message = MessageHelpers.createMessage(Messages.listStreamers);
210-
this.protocol.sendMessage(message);
209+
const BrowserSendOffer = this.config.isFlagEnabled(Flags.BrowserSendOffer);
210+
if (!BrowserSendOffer) {
211+
const message = MessageHelpers.createMessage(Messages.listStreamers);
212+
this.protocol.sendMessage(message);
213+
}
211214
this.reconnectAttempt = 0;
212215
this.isReconnecting = false;
213216
});
@@ -1151,6 +1154,19 @@ export class WebRtcPlayerController {
11511154
/* RTC Peer Connection on Track event -> handle on track */
11521155
this.peerConnectionController.onTrack = (trackEvent: RTCTrackEvent) =>
11531156
this.streamController.handleOnTrack(trackEvent);
1157+
1158+
const BrowserSendOffer = this.config.isFlagEnabled(Flags.BrowserSendOffer);
1159+
if (BrowserSendOffer) {
1160+
// If browser is sending the offer, create an offer and send it to the streamer
1161+
this.sendrecvDataChannelController.createDataChannel(
1162+
this.peerConnectionController.peerConnection,
1163+
'cirrus',
1164+
this.datachannelOptions
1165+
);
1166+
this.sendrecvDataChannelController.handleOnMessage = (ev: MessageEvent<ArrayBuffer>) =>
1167+
this.handleOnMessage(ev);
1168+
this.peerConnectionController.createOffer(this.sdpConstraints, this.config);
1169+
}
11541170
}
11551171

11561172
/**

Frontend/ui-library/src/Application/Application.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,7 @@ export class Application {
130130

131131
if (isPanelEnabled(options.statsPanelConfig)) {
132132
// Add stats panel
133-
this.statsPanel = new StatsPanel(options.statsPanelConfig);
133+
this.statsPanel = new StatsPanel(options.statsPanelConfig, this.stream.config);
134134
this.uiFeaturesElement.appendChild(this.statsPanel.rootElement);
135135
}
136136

Frontend/ui-library/src/Config/ConfigUI.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,8 @@ export class ConfigUI {
141141
psSettingsSection,
142142
this.textParametersUi.get(TextParameters.SignallingServerUrl)
143143
);
144+
if (isSettingEnabled(settingsConfig, Flags.BrowserSendOffer))
145+
this.addSettingFlag(psSettingsSection, this.flagsUi.get(Flags.BrowserSendOffer));
144146
if (isSettingEnabled(settingsConfig, OptionParameters.StreamerId))
145147
this.addSettingOption(
146148
psSettingsSection,
Lines changed: 227 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
// Copyright Epic Games, Inc. All Rights Reserved.
2+
3+
import {
4+
AggregatedStats,
5+
LatencyInfo,
6+
Logger,
7+
SettingNumber
8+
} from '@epicgames-ps/lib-pixelstreamingfrontend-ue5.6';
9+
import { SettingUINumber } from '../Config/SettingUINumber';
10+
11+
/**
12+
* Session test UI elements and results handling.
13+
* Creates a button to start the test and collects stats and latency info during the test.
14+
* After the test is finished, it generates CSV files for stats and latency info.
15+
* The test runs for a specified time frame, which can be set in the UI.
16+
*/
17+
export class SessionTest {
18+
_rootElement: HTMLElement;
19+
_latencyTestButton: HTMLInputElement;
20+
_testTimeFrameSetting: SettingNumber<'TestTimeFrame'>;
21+
22+
isCollectingStats: boolean;
23+
24+
records: AggregatedStats[];
25+
latencyRecords: LatencyInfo[];
26+
27+
constructor() {
28+
this.isCollectingStats = false;
29+
}
30+
31+
/**
32+
* Make the elements for the session test: e.g. button and test time input.
33+
*/
34+
public get rootElement(): HTMLElement {
35+
if (!this._rootElement) {
36+
this._rootElement = document.createElement('section');
37+
this._rootElement.classList.add('settingsContainer');
38+
39+
// make heading
40+
const heading = document.createElement('div');
41+
heading.id = 'latencyTestHeader';
42+
heading.classList.add('settings-text');
43+
heading.classList.add('settingsHeader');
44+
this._rootElement.appendChild(heading);
45+
46+
const headingText = document.createElement('div');
47+
headingText.innerHTML = 'Session Test';
48+
heading.appendChild(headingText);
49+
50+
// make test results element
51+
const resultsParentElem = document.createElement('div');
52+
resultsParentElem.id = 'latencyTestContainer';
53+
resultsParentElem.classList.add('d-none');
54+
this._rootElement.appendChild(resultsParentElem);
55+
56+
this._testTimeFrameSetting = new SettingNumber(
57+
'TestTimeFrame',
58+
'Test Time Frame',
59+
'How long the test runs for (seconds)',
60+
0 /*min*/,
61+
3600 /*max*/,
62+
60 /*default*/,
63+
false
64+
);
65+
const testTimeFrameSetting = new SettingUINumber(this._testTimeFrameSetting);
66+
resultsParentElem.appendChild(testTimeFrameSetting.rootElement);
67+
resultsParentElem.appendChild(this.latencyTestButton);
68+
}
69+
return this._rootElement;
70+
}
71+
72+
public get latencyTestButton(): HTMLInputElement {
73+
if (!this._latencyTestButton) {
74+
this._latencyTestButton = document.createElement('input');
75+
this._latencyTestButton.type = 'button';
76+
this._latencyTestButton.value = 'Run Test';
77+
this._latencyTestButton.id = 'btn-start-latency-test';
78+
this._latencyTestButton.classList.add('streamTools-button');
79+
this._latencyTestButton.classList.add('btn-flat');
80+
81+
this._latencyTestButton.onclick = () => {
82+
this.records = [];
83+
this.latencyRecords = [];
84+
this.isCollectingStats = true;
85+
this._latencyTestButton.disabled = true;
86+
this._latencyTestButton.value = 'Running...';
87+
Logger.Info(`Starting session test. Duration: [${this._testTimeFrameSetting.number}]`);
88+
setTimeout(() => {
89+
this.onCollectingFinished();
90+
this._latencyTestButton.disabled = false;
91+
this._latencyTestButton.value = 'Run Test';
92+
}, this._testTimeFrameSetting.number * 1000);
93+
};
94+
}
95+
return this._latencyTestButton;
96+
}
97+
98+
public handleStats(stats: AggregatedStats) {
99+
if (!this.isCollectingStats) {
100+
return;
101+
}
102+
103+
const statsCopy = structuredClone(stats);
104+
this.records.push(statsCopy);
105+
}
106+
107+
public handleLatencyInfo(latencyInfo: LatencyInfo) {
108+
if (!this.isCollectingStats) {
109+
return;
110+
}
111+
112+
const latencyInfoCopy = structuredClone(latencyInfo);
113+
this.latencyRecords.push(latencyInfoCopy);
114+
}
115+
116+
private onCollectingFinished() {
117+
this.isCollectingStats = false;
118+
Logger.Info(`Finished session test`);
119+
120+
this.generateStatsCsv();
121+
this.generateLatencyCsv();
122+
}
123+
124+
private generateStatsCsv() {
125+
const csvHeader: string[] = [];
126+
127+
this.records.forEach((record: AggregatedStats) => {
128+
for (const i in record) {
129+
const obj: {} = record[i as never];
130+
131+
if (Array.isArray(obj)) {
132+
for (let j = 0; j < obj.length; j++) {
133+
const arrayVal = obj[j];
134+
for (const k in arrayVal) {
135+
if (csvHeader.indexOf(`${i}.${j}.${k}`) === -1) {
136+
csvHeader.push(`${i}.${j}.${k}`);
137+
}
138+
}
139+
}
140+
} else if (obj instanceof Map) {
141+
for (const j in obj.keys()) {
142+
const mapVal = obj.get(j);
143+
for (const k in mapVal) {
144+
if (csvHeader.indexOf(`${i}.${j}.${k}`) === -1) {
145+
csvHeader.push(`${i}.${j}.${k}`);
146+
}
147+
}
148+
}
149+
} else {
150+
for (const j in obj) {
151+
if (csvHeader.indexOf(`${i}.${j}`) === -1) {
152+
csvHeader.push(`${i}.${j}`);
153+
}
154+
}
155+
}
156+
}
157+
});
158+
159+
let csvBody = '';
160+
this.records.forEach((record) => {
161+
csvHeader.forEach((field) => {
162+
try {
163+
csvBody += `"${field.split('.').reduce((o, k) => o[k as never], record)}",`;
164+
} catch (_) {
165+
csvBody += `"",`;
166+
}
167+
});
168+
csvBody += `\n`;
169+
});
170+
171+
const file = new Blob([`${csvHeader.join(',')}\n${csvBody}`], { type: 'text/plain' });
172+
const a = document.createElement('a');
173+
const url = URL.createObjectURL(file);
174+
a.href = url;
175+
a.download = 'stats.csv';
176+
document.body.appendChild(a);
177+
a.click();
178+
setTimeout(function () {
179+
document.body.removeChild(a);
180+
window.URL.revokeObjectURL(url);
181+
}, 0);
182+
}
183+
184+
private generateLatencyCsv() {
185+
const csvHeader: string[] = [];
186+
187+
this.latencyRecords.forEach((record) => {
188+
for (const i in record) {
189+
const obj = record[i as never];
190+
191+
if (typeof obj === 'object') {
192+
for (const j in obj as object) {
193+
if (csvHeader.indexOf(`${i}.${j}`) === -1) {
194+
csvHeader.push(`${i}.${j}`);
195+
}
196+
}
197+
} else if (csvHeader.indexOf(`${i}`) === -1) {
198+
csvHeader.push(`${i}`);
199+
}
200+
}
201+
});
202+
203+
let csvBody = '';
204+
this.latencyRecords.forEach((record) => {
205+
csvHeader.forEach((field) => {
206+
try {
207+
csvBody += `"${field.split('.').reduce((o, k) => o[k as never], record)}",`;
208+
} catch (_) {
209+
csvBody += `"",`;
210+
}
211+
});
212+
csvBody += `\n`;
213+
});
214+
215+
const file = new Blob([`${csvHeader.join(',')}\n${csvBody}`], { type: 'text/plain' });
216+
const a = document.createElement('a');
217+
const url = URL.createObjectURL(file);
218+
a.href = url;
219+
a.download = 'latency.csv';
220+
document.body.appendChild(a);
221+
a.click();
222+
setTimeout(function () {
223+
document.body.removeChild(a);
224+
window.URL.revokeObjectURL(url);
225+
}, 0);
226+
}
227+
}

0 commit comments

Comments
 (0)