Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update reach-calculator.html #191

Merged
merged 3 commits into from
Mar 22, 2024
Merged

Update reach-calculator.html #191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
97249c5
Update reach-calculator.html
tagawa Mar 22, 2024
cee7f1d
Added media queries to reach-calculator.html
tagawa Mar 22, 2024
e4f095f
Added link from homepage to reach calculator
tagawa Mar 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ <h2>Other</h2>
<li><a href='./viewport/'>Viewport test cases</a></li>
<li><a href='./windows-browser/script-injection/index.html'>Windows Browser: MainWorld Script injection tests</a></li>
<li><a href='./windows-browser/script-injection/secureWorld_index.html'>Windows Browser: SecureWorld Script injection tests</a></li>
<li><a href='./tools/reach-calculator.html'>Site Breakage Reach Calculator</a></li>
</ul>

</body>
Expand Down
233 changes: 224 additions & 9 deletions tools/reach-calculator.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,17 @@

<style>
body {
padding: 0;
padding: 1em 0;
}
h2 {
margin-top: 0;
padding-top: 0;
}
#container {
max-width: 600px; /* Set the maximum width of the element to 600px */
width: 100%; /* Allow the element to expand to the full width of its container */
margin: 0 auto; /* Auto margins on left and right will center the element */
max-width: 600px;
padding-bottom: 1em;
width: 100%;
}
button {
background: #fff;
Expand Down Expand Up @@ -55,18 +60,81 @@
input[type="number"] {
width: 8em;
}
abbr {
cursor: help;
}
hr {
margin: 1em 0;
}
#results {
font-weight: bold;
}
#faq {
margin-top: 1em;
}
details {
margin-bottom: 1em;
}
summary {
cursor: pointer;
font-size: larger;
font-weight: bold;
}
details h3 {
font-size: larger;
}
dl dt {
float: left;
margin-bottom: 0;
margin-right: 1em;
width: auto;
}
dl dd {
clear: right;
margin-bottom: 0;
margin-left: 0;
width: auto;
}
details .answer {
font-weight: normal;
padding-left: 4em;
}
details .answer::before {
content: "> ";
}
math {
display: block;
margin-bottom: 1em;
}
mrow {
margin-bottom: 0.5em;
}

/* Styles for wide screens */
@media screen and (min-width: 800px) {
#container {
max-width: 1000px;
padding-top: 1em;
}
#form, #faq {
display: inline-block;
width: 50%;
}
#form {
float: left;
}
#faq {
margin-top: 0;
padding-left: 2em;
}
}
</style>
</head>
<body class="domain">
<div id="container">
<h2>Site Breakage Reach Calculator</h2>

<div id="form">
<div id="options">
<button id="domainButton" class="tablink">Domains</button>
<button id="apiButton" class="tablink">APIs/Features</button>
Expand All @@ -86,21 +154,168 @@ <h2>Site Breakage Reach Calculator</h2>
</div>

<div>
<label><abbr title="Further reduction assumption, e.g. visiting 2+ pages, particular version, etc.">[Optional] Reduction assumption (%):<br>
<input type="number" id="reductionFactorInput" min="0" max="100"></abbr></label><br>
<label>[Optional] <abbr title="Should the reach of this breakage be reduced? This should be based on data if available, or your best estimate.">Reduction assumption (%):<br>
<input type="number" id="reductionFactorInput" min="0" max="100"></abbr></label>
</div>

<div>
<label for="confidenceInput">Additional unaccounted-for breakage?</label><br>
<abbr title="How confident are you that this breakage is happening on other domains or affects other APIs?">Additional unaccounted-for breakage?</abbr><br>
<label><input type="radio" id="low" name="confidenceInput" value="low" checked> Low confidence</label><br>
<label><input type="radio" id="medium" name="confidenceInput" value="medium"> Medium confidence</label><br>
<label><input type="radio" id="high" name="confidenceInput" value="high"> High confidence</label><br>
<label><input type="radio" id="high" name="confidenceInput" value="high"> High confidence</label>
</div>

<hr>
<div id="results"></div>
<div>(Use this and the Severity Assessment Matrix to get a priority level.)</div>
<div>(Use this and the <a href="https://duckduckgo-my.sharepoint.com/:x:/p/lmattei/EcV1df19DeVMssJDG1YRcnABHgQpzVZHfchesN-AZj2VcA?e=Nyn7mu" target="_blank">Severity Assessment Matrix</a> to get a priority level.)</div>
</div>

</div><div id="faq">
<h2>FAQ</h2>

<details>
<summary>How do I use this?</summary>
<p>This tool is to calculate the reach of breakage issues, i.e. roughly the percentage of users affected on a particular platform. When combined with a severity level, you can look up a priority level for the issue in the <a href="https://duckduckgo-my.sharepoint.com/:x:/p/lmattei/EcV1df19DeVMssJDG1YRcnABHgQpzVZHfchesN-AZj2VcA?e=Nyn7mu" target="_blank">Severity Assessment Matrix</a>.</p>
</details>

<details>
<summary>What is "reduction assumption"?</summary>
<p>Should the reach of this breakage be reduced? This should be based on data if available, or your best estimate. Examples of when this could apply:</p>
<ul>
<li>A specific platform version, e.g. only Big Sur users, which might account for 5% of all macOS users.</li>
<li>A specific user action, e.g. only when users pause a video, which might be 10% of all video plays.</li>
</ul>
</details>

<details>
<summary>What is "unaccounted-for breakage"?</summary>
<p>How confident are you that this breakage is happening on other domains or affects other APIs?</p>
</details>

<details>
<summary>What if multiple domains are broken?</summary>
<p>There are a couple of ways to handle this:</p>
<ul>
<li>Use the calculator for the highest ranked domain you're aware of, then set "unaccounted-for breakage" to medium or high confidence.</li>
<li>Use the calculator for each broken domain, set the "unaccounted-for breakage" to low confidence, then sum the resulting percentages.</li>
</ul>
</details>

<details>
<summary>What are some example calculations?</summary>
<p>
<h3>Single domain example: example.com broken, only happens after 2+ clicks</h3>
<dl>
<dt>Domain rank:</dt><dd>171</dd>
<dt class="answer">Initial % of all page loads:</dt><dd>0.0337%</dd>
<dt>Further reduction assumption:</dt><dd>50%</dd>
<dt class="answer">Updated % of page loads:</dt><dd>0.0168%</dd>
<dt>Additional unaccounted-for breakage:</dt><dd>High confidence</dd>
<dt class="answer">Final % of page loads:</dt><dd>1.68%</dd>
</dl>

<h3>Single API example: WebShare API not supported</h3>
<dl>
<dt>API usage share:</dt><dd>0.07%</dd>
<dt class="answer">Initial % of all page loads:</dt><dd><a href="https://chromestatus.com/metrics/feature/timeline/popularity/2737" target="_blank">0.07%</a></dd>
<dt>Further reduction assumption:</dt><dd>[None]</dd>
<dt class="answer">Updated % of page loads:</dt><dd>0.07%</dd>
<dt>Additional unaccounted-for breakage:</dt><dd>Low confidence</dd>
<dt class="answer">Final % of page loads:</dt><dd>0.07%</dd>
</dl>

<h3>Extreme example 1: Google is broken for everyone</h3>
<dl>
<dt>Domain rank:</dt><dd>1</dd>
<dt class="answer">Initial % of all page loads:</dt><dd>21.352%</dd>
<dt>Further reduction assumption:</dt><dd>[None]</dd>
<dt class="answer">Updated % of page loads:</dt><dd>21.352%</dd>
<dt>Additional unaccounted-for breakage:</dt><dd>Low confidence</dd>
<dt class="answer">Final % of page loads:</dt><dd>21.352%</dd>
</dl>

<h3>Extreme example 2: Videos not playing on my personal blog </h3>
<dl>
<dt>Domain rank:</dt><dd>1,000,000</dd>
<dt class="answer">Initial % of all page loads:</dt><dd>0.00011%</dd>
<dt>Further reduction assumption:</dt><dd>10%</dd>
<dt class="answer">Updated % of page loads:</dt><dd>0.000011%</dd>
<dt>Additional unaccounted-for breakage:</dt><dd>High confidence</dd>
<dt class="answer">Final % of page loads:</dt><dd>0.0011%</dd>
</dl>
</p>
</details>

<details>
<summary>How is the page load percentage calculated from domain rank?</summary>
<p>The paper "<a href="https://dl.acm.org/doi/10.1145/3517745.3561418" target="_blank">A World Wide View of Browsing the World Wide Web</a>", written by researchers at Google and Stanford University, presents data from "several hundred million global users" revealing the following traffic statistics:</p>
<ul>
<li>The top website has 17% of traffic</li>
<li>The top 6 sites have 25% of traffic</li>
<li>The top 100 sites have 40% of traffic</li>
<li>The top 10K sites have 70% of traffic</li>
<li>The top 1M sites have 95%+ of traffic</li>
</ul>
<p>This is a logarithmic increase which, when plotted, gives us the following trend line formula:</p>

<math xmlns="http://www.w3.org/1998/Math/MathML">
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hah this is massively above and beyond here! I just meant copying that comment into a <!--- tag. Anyway all good!

<mrow>
<mi>f</mi>
<mo>(</mo>
<mi>x</mi>
<mo>)</mo>
<mo>=</mo>
<mn>0.0577045083370069</mn>
<mo>&times;</mo>
<mi>ln</mi>
<mo>(</mo>
<mi>x</mi>
<mo>)</mo>
<mo>+</mo>
<mn>0.154434582485181</mn>
</mrow>
<mrow>
<msup>
<mi>R</mi>
<mn>2</mn>
</msup>
<mo>=</mo>
<mn>0.9978</mn>
</mrow>
</math>

<p>Note that's a cumulative page load percentage &mdash; the running total for all domain ranks. To get individual domain ranks we can calculate the page load difference for each domain rank. I did this for the top 300 domains which can them be plotted to give another trend line formula:</p>

<math xmlns="http://www.w3.org/1998/Math/MathML">
<mrow>
<mi>f</mi>
<mo>(</mo>
<mi>x</mi>
<mo>)</mo>
<mo>=</mo>
<mn>0.0687189317758425</mn>
<mo>&times;</mo>
<msup>
<mi>x</mi>
<mrow>
<mo>-</mo>
<mn>1.03435633987409</mn>
</mrow>
</msup>
</mrow>
<mrow>
<msup>
<mi>R</mi>
<mn>2</mn>
</msup>
<mo>=</mo>
<mn>0.997267490393083</mn>
</mrow>
</math>

<p>These are the constants that this calculator uses.</p>
</details>
</div>
</div>

<script>
Expand Down
Loading