Figure 2. HTTP/2 usage by request. (Source: HTTP Archive)
-
The results show that HTTP/2 usage is now the majority protocol-an impressive feat just 4 short years after formal standardization! Looking at the breakdown of all HTTP versions by request we see the following:
@@ -232,9 +231,7 @@
Adoption of HTTP/2
-
Figure 3. HTTP version usage by request.
-
Figure 3 shows that HTTP/1.1 and HTTP/2 are the versions used by the vast majority of requests as expected. There is only a very small number of requests on the older HTTP/1.0 and HTTP/0.9 protocols. Annoyingly, there is a larger percentage where the protocol was not correctly tracked by the HTTP Archive crawl, particularly on desktop. Digging into this has shown various reasons, some of which can be explained and some of which can't. Based on spot checks, they mostly appear to be HTTP/1.1 requests and, assuming they are, desktop and mobile usage is similar.
Despite there being a little larger percentage of noise than we'd like, it doesn't alter the overall message being conveyed here. Other than that, the mobile/desktop similarity is not unexpected; HTTP Archive tests with Chrome, which supports HTTP/2 for both desktop and mobile. Real-world usage may have slightly different stats with some older usage of browsers on both, but even then support is widespread, so we would not expect a large variation between desktop and mobile.
@@ -282,9 +279,7 @@
Adoption of HTTP/2
-
Figure 4. HTTP version usage for home pages.
-
It is still interesting to look at home pages only to get a rough figure on the number of sites that support HTTP/2 (at least on their home page). Figure 4 shows less support than overall requests, as expected, at around 36%.
HTTP/2 is only supported by browsers over HTTPS, even though officially HTTP/2 can be used over HTTPS or over unencrypted non-HTTPS connections. As mentioned previously, hiding the new protocol in encrypted HTTPS connections prevents networking appliances which do not understand this new protocol from interfering with (or rejecting!) its usage. Additionally, the HTTPS handshake allows an easy method of the client and server agreeing to use HTTP/2.
@@ -329,9 +324,7 @@
Adoption of HTTP/2
-
Figure 5. HTTP version usage for HTTPS home pages.
-
The web is moving to HTTPS, and HTTP/2 turns the traditional argument of HTTPS being bad for performance almost completely on its head. Not every site has made the transition to HTTPS, so HTTP/2 will not even be available to those that have not. Looking at just those sites that use HTTPS, in Figure 5 we do see a higher adoption of HTTP/2 at around 55%, similar to the percent of all requests in Figure 2.
We have shown that browser support for HTTP/2 is strong and that there is a safe road to adoption, so why doesn't every site (or at least every HTTPS site) support HTTP/2? Well, here we come to the final item for support we have not measured yet: server support.
@@ -407,9 +400,7 @@
Adoption of HTTP/2
-
Figure 6. Servers used for HTTP/2.
-
Nginx provides package repositories that allow ease of installing or upgrading to the latest version, so it is no surprise to see it leading the way here. Cloudflare is the most popular CDN and enables HTTP/2 by default, so again it is not surprising to see it hosts a large percentage of HTTP/2 sites. Incidently, Cloudflare uses a heavily customised version of nginx as their web server. After those, we see Apache at around 20% of usage, followed by some servers who choose to hide what they are, and then the smaller players such as LiteSpeed, IIS, Google Servlet Engine, and openresty, which is nginx based.
What is more interesting is those servers that that do not support HTTP/2:
@@ -484,9 +475,7 @@
Adoption of HTTP/2
-
Figure 7. Servers used for HTTP/1.1 or lower.
-
Some of this will be non-HTTPS traffic that would use HTTP/1.1 even if the server supported HTTP/2, but a bigger issue is those that do not support HTTP/2 at all. In these stats, we see a much greater share for Apache and IIS, which are likely running older versions.
For Apache in particular, it is often not easy to add HTTP/2 support to an existing installation, as Apache does not provide an official repository to install this from. This often means resorting to compiling from source or trusting a third-party repository, neither of which is particularly appealing to many administrators.
@@ -553,9 +542,7 @@
Adoption of HTTP/2
-
Figure 8. Percentage installs of each server used to provide HTTP/2.
-
It's clear that Apache and IIS fall way behind with 18% and 14% of their installed based supporting HTTP/2, which has to be (at least in part) a consequence of it being more difficult to upgrade them. A full operating system upgrade is often required for many servers to get this support easily. Hopefully this will get easier as new versions of operating systems become the norm.
Figure 9. TCP connections per page. (Source: HTTP Archive)
-
HTTP Archive measures the number of TCP connections per page, and that is dropping steadily as more sites support HTTP/2 and use its single connection instead of six separate connections.
@@ -575,7 +561,6 @@
Impact of HTTP/2
Figure 10. Total requests per page. (Source: HTTP Archive)
-
Bundling assets to obtain fewer requests was another HTTP/1.1 workaround that went by many names: bundling, concatenation, packaging, spriting, etc. This is less necessary when using HTTP/2 as there is less overhead with requests, but it should be noted that requests are not free in HTTP/2, and those that experimented with removing bundling completely have noticed a loss in performance. Looking at the number of requests loaded per page over time, we do see a slight decrease in requests, rather than the expected increase.
This low rate of change can perhaps be attributed to the aforementioned observations that bundling cannot be removed (at least, not completely) without a negative performance impact and that many build tools currently bundle for historical reasons based on HTTP/1.1 recommendations. It is also likely that many sites may not be willing to penalize HTTP/1.1 users by undoing their HTTP/1.1 performance hacks just yet, or at least that they do not have the confidence (or time!) to feel that this is worthwhile.
@@ -613,9 +598,7 @@
HTTP/2 Push
-
Figure 11. Sites using HTTP/2 push.
-
@@ -643,9 +626,7 @@
HTTP/2 Push
-
Figure 12. How much is pushed when it is used.
-
These stats show that the uptick of HTTP/2 push is very low, most likely because of the issues described previously. However, when sites do use push, they tend to use it a lot rather than for one or two assets as shown in Figure 12.
-
Figure 14. HTTP/2 prioritization support in common CDNs.
-
Figure 14 shows that a fairly significant portion of traffic is subject to the identified issue, totalling 26.82% on desktop and 27.83% on mobile. How much of a problem this is depends on exactly how your page loads and whether high priority resources are discovered late or not for your site.
-
Figure 3. Top 10 most popular third party domains.
-
-
-
@@ -392,9 +388,7 @@
Providers
-
Figure 4. Top 10 most popular third party requests.
-
Resource Types
The resource type breakdown of third-party content also lends insight into how third-party code is used across the web. While first-party requests are 56% images, 23% script, 14% CSS, and only 4% HTML, third-party requests skew more heavily toward script and HTML at 32% script, 34% images, 12% HTML, and only 6% CSS. While this suggests that third-party code is less frequently used to aid the design and instead used more frequently to facilitate or observe interactions than first-party code, a breakdown of resource types by party status tells a more nuanced story. While CSS and images are dominantly first-party at 70% and 64% respectively, fonts are largely served by third-party providers with only 28% being served from first-party sources. This concept of usage patterns is explored in more depth later in this chapter.
diff --git a/src/tools/generate/generate_chapters.js b/src/tools/generate/generate_chapters.js
index 713a2627688..0967e6d1f9f 100644
--- a/src/tools/generate/generate_chapters.js
+++ b/src/tools/generate/generate_chapters.js
@@ -45,6 +45,7 @@ const parse_file = async (markdown) => {
body = generate_figure_ids(body);
body = wrap_tables(body);
+ body = body.replace(/
<\/p>/g,""); //tempm fix to remove extra paragraph tags created by wrap_tables
const toc = generate_table_of_contents(body);
const m = converter.getMetadata();
From a5eeda2dec827f31861c859e338dd29ec2fa5341 Mon Sep 17 00:00:00 2001
From: Mike Geyser
Date: Sun, 10 Nov 2019 08:51:43 +0000
Subject: [PATCH 2/4] Added a regex script that removes unnecessary markup.
---
src/templates/en/2019/chapters/http2.html | 21 -------------------
src/templates/en/2019/chapters/markup.html | 2 --
.../en/2019/chapters/third-parties.html | 6 ------
src/tools/generate/generate_chapters.js | 2 ++
.../generate/remove_unnecessary_markup.js | 10 +++++++++
5 files changed, 12 insertions(+), 29 deletions(-)
create mode 100644 src/tools/generate/remove_unnecessary_markup.js
diff --git a/src/templates/en/2019/chapters/http2.html b/src/templates/en/2019/chapters/http2.html
index c5a58f83769..ae482b5c4e5 100644
--- a/src/templates/en/2019/chapters/http2.html
+++ b/src/templates/en/2019/chapters/http2.html
@@ -182,7 +182,6 @@
Adoption of HTTP/2
Figure 2. HTTP/2 usage by request. (Source: HTTP Archive)
-
The results show that HTTP/2 usage is now the majority protocol-an impressive feat just 4 short years after formal standardization! Looking at the breakdown of all HTTP versions by request we see the following:
@@ -232,9 +231,7 @@
Adoption of HTTP/2
-
Figure 3. HTTP version usage by request.
-
Figure 3 shows that HTTP/1.1 and HTTP/2 are the versions used by the vast majority of requests as expected. There is only a very small number of requests on the older HTTP/1.0 and HTTP/0.9 protocols. Annoyingly, there is a larger percentage where the protocol was not correctly tracked by the HTTP Archive crawl, particularly on desktop. Digging into this has shown various reasons, some of which can be explained and some of which can't. Based on spot checks, they mostly appear to be HTTP/1.1 requests and, assuming they are, desktop and mobile usage is similar.
Despite there being a little larger percentage of noise than we'd like, it doesn't alter the overall message being conveyed here. Other than that, the mobile/desktop similarity is not unexpected; HTTP Archive tests with Chrome, which supports HTTP/2 for both desktop and mobile. Real-world usage may have slightly different stats with some older usage of browsers on both, but even then support is widespread, so we would not expect a large variation between desktop and mobile.
@@ -282,9 +279,7 @@
Adoption of HTTP/2
-
Figure 4. HTTP version usage for home pages.
-
It is still interesting to look at home pages only to get a rough figure on the number of sites that support HTTP/2 (at least on their home page). Figure 4 shows less support than overall requests, as expected, at around 36%.
HTTP/2 is only supported by browsers over HTTPS, even though officially HTTP/2 can be used over HTTPS or over unencrypted non-HTTPS connections. As mentioned previously, hiding the new protocol in encrypted HTTPS connections prevents networking appliances which do not understand this new protocol from interfering with (or rejecting!) its usage. Additionally, the HTTPS handshake allows an easy method of the client and server agreeing to use HTTP/2.
@@ -329,9 +324,7 @@
Adoption of HTTP/2
-
Figure 5. HTTP version usage for HTTPS home pages.
-
The web is moving to HTTPS, and HTTP/2 turns the traditional argument of HTTPS being bad for performance almost completely on its head. Not every site has made the transition to HTTPS, so HTTP/2 will not even be available to those that have not. Looking at just those sites that use HTTPS, in Figure 5 we do see a higher adoption of HTTP/2 at around 55%, similar to the percent of all requests in Figure 2.
We have shown that browser support for HTTP/2 is strong and that there is a safe road to adoption, so why doesn't every site (or at least every HTTPS site) support HTTP/2? Well, here we come to the final item for support we have not measured yet: server support.
@@ -407,9 +400,7 @@
Adoption of HTTP/2
-
Figure 6. Servers used for HTTP/2.
-
Nginx provides package repositories that allow ease of installing or upgrading to the latest version, so it is no surprise to see it leading the way here. Cloudflare is the most popular CDN and enables HTTP/2 by default, so again it is not surprising to see it hosts a large percentage of HTTP/2 sites. Incidently, Cloudflare uses a heavily customised version of nginx as their web server. After those, we see Apache at around 20% of usage, followed by some servers who choose to hide what they are, and then the smaller players such as LiteSpeed, IIS, Google Servlet Engine, and openresty, which is nginx based.
What is more interesting is those servers that that do not support HTTP/2:
@@ -484,9 +475,7 @@
Adoption of HTTP/2
-
Figure 7. Servers used for HTTP/1.1 or lower.
-
Some of this will be non-HTTPS traffic that would use HTTP/1.1 even if the server supported HTTP/2, but a bigger issue is those that do not support HTTP/2 at all. In these stats, we see a much greater share for Apache and IIS, which are likely running older versions.
For Apache in particular, it is often not easy to add HTTP/2 support to an existing installation, as Apache does not provide an official repository to install this from. This often means resorting to compiling from source or trusting a third-party repository, neither of which is particularly appealing to many administrators.
@@ -553,9 +542,7 @@
Adoption of HTTP/2
-
Figure 8. Percentage installs of each server used to provide HTTP/2.
-
It's clear that Apache and IIS fall way behind with 18% and 14% of their installed based supporting HTTP/2, which has to be (at least in part) a consequence of it being more difficult to upgrade them. A full operating system upgrade is often required for many servers to get this support easily. Hopefully this will get easier as new versions of operating systems become the norm.
Figure 9. TCP connections per page. (Source: HTTP Archive)
-
HTTP Archive measures the number of TCP connections per page, and that is dropping steadily as more sites support HTTP/2 and use its single connection instead of six separate connections.
@@ -575,7 +561,6 @@
Impact of HTTP/2
Figure 10. Total requests per page. (Source: HTTP Archive)
-
Bundling assets to obtain fewer requests was another HTTP/1.1 workaround that went by many names: bundling, concatenation, packaging, spriting, etc. This is less necessary when using HTTP/2 as there is less overhead with requests, but it should be noted that requests are not free in HTTP/2, and those that experimented with removing bundling completely have noticed a loss in performance. Looking at the number of requests loaded per page over time, we do see a slight decrease in requests, rather than the expected increase.
This low rate of change can perhaps be attributed to the aforementioned observations that bundling cannot be removed (at least, not completely) without a negative performance impact and that many build tools currently bundle for historical reasons based on HTTP/1.1 recommendations. It is also likely that many sites may not be willing to penalize HTTP/1.1 users by undoing their HTTP/1.1 performance hacks just yet, or at least that they do not have the confidence (or time!) to feel that this is worthwhile.
@@ -613,9 +598,7 @@
HTTP/2 Push
-
Figure 11. Sites using HTTP/2 push.
-
@@ -643,9 +626,7 @@
HTTP/2 Push
-
Figure 12. How much is pushed when it is used.
-
These stats show that the uptick of HTTP/2 push is very low, most likely because of the issues described previously. However, when sites do use push, they tend to use it a lot rather than for one or two assets as shown in Figure 12.
-
Figure 14. HTTP/2 prioritization support in common CDNs.
-
Figure 14 shows that a fairly significant portion of traffic is subject to the identified issue, totalling 26.82% on desktop and 27.83% on mobile. How much of a problem this is depends on exactly how your page loads and whether high priority resources are discovered late or not for your site.
-
Figure 3. Top 10 most popular third party domains.
-
-
-
@@ -392,9 +388,7 @@
Providers
-
Figure 4. Top 10 most popular third party requests.
-
Resource Types
The resource type breakdown of third-party content also lends insight into how third-party code is used across the web. While first-party requests are 56% images, 23% script, 14% CSS, and only 4% HTML, third-party requests skew more heavily toward script and HTML at 32% script, 34% images, 12% HTML, and only 6% CSS. While this suggests that third-party code is less frequently used to aid the design and instead used more frequently to facilitate or observe interactions than first-party code, a breakdown of resource types by party status tells a more nuanced story. While CSS and images are dominantly first-party at 70% and 64% respectively, fonts are largely served by third-party providers with only 28% being served from first-party sources. This concept of usage patterns is explored in more depth later in this chapter.
diff --git a/src/tools/generate/generate_chapters.js b/src/tools/generate/generate_chapters.js
index 713a2627688..01c838a378d 100644
--- a/src/tools/generate/generate_chapters.js
+++ b/src/tools/generate/generate_chapters.js
@@ -8,6 +8,7 @@ const { generate_table_of_contents } = require('./generate_table_of_contents');
const { generate_figure_ids } = require('./generate_figure_ids');
const { generate_sitemap } = require('./generate_sitemap');
const { wrap_tables } = require('./wrap_tables');
+const { remove_unnecessary_markup } = require('./remove_unnecessary_markup');
const converter = new showdown.Converter({ tables: true, metadata: true });
converter.setFlavor('github');
@@ -45,6 +46,7 @@ const parse_file = async (markdown) => {
body = generate_figure_ids(body);
body = wrap_tables(body);
+ body = remove_unnecessary_markup(body);
const toc = generate_table_of_contents(body);
const m = converter.getMetadata();
diff --git a/src/tools/generate/remove_unnecessary_markup.js b/src/tools/generate/remove_unnecessary_markup.js
new file mode 100644
index 00000000000..d0ce8c8ec52
--- /dev/null
+++ b/src/tools/generate/remove_unnecessary_markup.js
@@ -0,0 +1,10 @@
+/*
+ TODO: This is a workaround for the unnecessary markup that's being generated
+ by JSDOM. Once that has been fixed, this can hopefully be removed.
+ (See #412)
+*/
+const remove_unnecessary_markup = (html) => html.replace(/
<\/p>/g, '');
+
+module.exports = {
+ remove_unnecessary_markup
+};
From 7aa2b4618320194fb36cc05bda736ff1c93990b2 Mon Sep 17 00:00:00 2001
From: Barry
Date: Sun, 10 Nov 2019 08:51:57 +0000
Subject: [PATCH 3/4] Fix typo in comment
---
src/tools/generate/generate_chapters.js | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/tools/generate/generate_chapters.js b/src/tools/generate/generate_chapters.js
index 0967e6d1f9f..fe751ad02c1 100644
--- a/src/tools/generate/generate_chapters.js
+++ b/src/tools/generate/generate_chapters.js
@@ -45,7 +45,7 @@ const parse_file = async (markdown) => {
body = generate_figure_ids(body);
body = wrap_tables(body);
- body = body.replace(/
<\/p>/g,""); //tempm fix to remove extra paragraph tags created by wrap_tables
+ body = body.replace(/