diff --git a/index.html b/index.html index 8bd039c..1de345f 100644 --- a/index.html +++ b/index.html @@ -117,6 +117,7 @@

Other

  • Viewport test cases
  • Windows Browser: MainWorld Script injection tests
  • Windows Browser: SecureWorld Script injection tests
    • +
  • Site Breakage Reach Calculator
    • diff --git a/tools/reach-calculator.html b/tools/reach-calculator.html index 5854a3a..70bbb91 100644 --- a/tools/reach-calculator.html +++ b/tools/reach-calculator.html @@ -12,12 +12,17 @@

    Site Breakage Reach Calculator

    +
    @@ -86,21 +154,168 @@

    Site Breakage Reach Calculator

    -
    +
    -
    + Additional unaccounted-for breakage?


    -
    +
    - +
    -
    (Use this and the Severity Assessment Matrix to get a priority level.)
    +
    (Use this and the Severity Assessment Matrix to get a priority level.)
    + +
    +

    FAQ

    + +
    + How do I use this? +

    This tool is to calculate the reach of breakage issues, i.e. roughly the percentage of users affected on a particular platform. When combined with a severity level, you can look up a priority level for the issue in the Severity Assessment Matrix.

    +
    + +
    + What is "reduction assumption"? +

    Should the reach of this breakage be reduced? This should be based on data if available, or your best estimate. Examples of when this could apply:

    + +
    + +
    + What is "unaccounted-for breakage"? +

    How confident are you that this breakage is happening on other domains or affects other APIs?

    +
    + +
    + What if multiple domains are broken? +

    There are a couple of ways to handle this:

    + +
    + +
    + What are some example calculations? +

    +

    Single domain example: example.com broken, only happens after 2+ clicks

    +
    +
    Domain rank:
    171
    +
    Initial % of all page loads:
    0.0337%
    +
    Further reduction assumption:
    50%
    +
    Updated % of page loads:
    0.0168%
    +
    Additional unaccounted-for breakage:
    High confidence
    +
    Final % of page loads:
    1.68%
    +
    + +

    Single API example: WebShare API not supported

    +
    +
    API usage share:
    0.07%
    +
    Initial % of all page loads:
    0.07%
    +
    Further reduction assumption:
    [None]
    +
    Updated % of page loads:
    0.07%
    +
    Additional unaccounted-for breakage:
    Low confidence
    +
    Final % of page loads:
    0.07%
    +
    + +

    Extreme example 1: Google is broken for everyone

    +
    +
    Domain rank:
    1
    +
    Initial % of all page loads:
    21.352%
    +
    Further reduction assumption:
    [None]
    +
    Updated % of page loads:
    21.352%
    +
    Additional unaccounted-for breakage:
    Low confidence
    +
    Final % of page loads:
    21.352%
    +
    + +

    Extreme example 2: Videos not playing on my personal blog

    +
    +
    Domain rank:
    1,000,000
    +
    Initial % of all page loads:
    0.00011%
    +
    Further reduction assumption:
    10%
    +
    Updated % of page loads:
    0.000011%
    +
    Additional unaccounted-for breakage:
    High confidence
    +
    Final % of page loads:
    0.0011%
    +
    +

    +
    + +
    + How is the page load percentage calculated from domain rank? +

    The paper "A World Wide View of Browsing the World Wide Web", written by researchers at Google and Stanford University, presents data from "several hundred million global users" revealing the following traffic statistics:

    + +

    This is a logarithmic increase which, when plotted, gives us the following trend line formula:

    + + + + f + ( + x + ) + = + 0.0577045083370069 + × + ln + ( + x + ) + + + 0.154434582485181 + + + + R + 2 + + = + 0.9978 + + + +

    Note that's a cumulative page load percentage — the running total for all domain ranks. To get individual domain ranks we can calculate the page load difference for each domain rank. I did this for the top 300 domains which can them be plotted to give another trend line formula:

    + + + + f + ( + x + ) + = + 0.0687189317758425 + × + + x + + - + 1.03435633987409 + + + + + + R + 2 + + = + 0.997267490393083 + + + +

    These are the constants that this calculator uses.

    +
    +