diff --git a/06-connection-problems.adoc b/06-connection-problems.adoc index 1b73e0e..1b7f0d7 100644 --- a/06-connection-problems.adoc +++ b/06-connection-problems.adoc @@ -66,42 +66,67 @@ ISP (Internet Service Provider) blocked a domain, or a keyword in the request/re Squirrels attacked the data center? [quote,Rich Miller,"Surviving Electric Squirrels and UPS Failures, 2012, Data Center Knowledge"] +-- "A frying squirrel took out half of our Santa Clara data center two years back," Christian said, noting squirrels' propensity to interact with electrical equipment, with unfortunate results. If you enter “squirrel outage” in either Google News or Google web search, you'll find a lengthy record of both recent and historic incidents of squirrels causing local power outages. +-- Ships trash an undersea cables by dropping anchor right on The Internet?! // TODO Stock photo liven it up a bit? https://www.istockphoto.com/photo/underwater-fiber-optic-cable-on-ocean-floor-gm1362710800-434533439 -== Coding Defensively +== Defensive Code -Going over the wire is fraught with danger, and it gets worse the farther a -request and response have to travel. A common theme "in this book will be: avoid -making network calls when possible, expect failure, and make sure your user -interface degrades to vary levels, so failures are clear. No white screens of -death! +All of these problems are going to cause the happy path to get messy. -=== Detecting Success or Failure +Let's harden our code one step at a time. -Failed to connect? Try it again, and if it fails a few times maybe show -something to the user explaining that their Internet is down. +[sidebar] +Inspiration for these code examples was taken from Umar Hansa's brilliant article https://web.dev/fetch-api-error-handling/[Implement error handling when using the Fetch API]. -It's important to make sure that no single part of any client -application _requires_ a connection to leave that state. Often I have -seen client applications submit a form, hide the form they just -submitted, fail to make the connection, and as they were expecting a -positive or negative JSON response in a specific structure, in order to -dictate showing the form again or progressing, they end up with a blank -screen. +[,js] +---- +include::code/ch06-connection-problems/02-catch-fetch-errors.js[] +---- + +This little change solves some of these problems. If there is any sort of connection failure from something like a connection refused (no internet, server is down, etc), or a dropped connection (failed part way through), or certificate errors, this should all be caught with the first exception. + +Whatever happens, it will log something to the user console, and return early. You could imagine this code doing something clever to update the user interface, but for now we're keeping it simple. + +This is a step in the right direction, but once we've eventually got a response there is a lot of other things that can go wrong. What if the response is randomly HTML instead of JSON? Or it's weirdly invalid JSON? + +[,js] +---- +include::code/ch06-connection-problems/03-catch-json-errors.js[] +---- + +Great! Now when the API randomly squirts some unexpected HTML error at you, the function will just return an empty array, and there is an error logged that the developers can go digging into. + +Another step in the right direction, but this still assumes we actually get a response in a reasonable timeframe. + +What if you've been waiting for *thirty seconds*? + +What if you've been waiting for *two minutes*? + +We will deep dive into timeouts later on in the book, but a really helpful quick bit of defensive coding you can do, is to make sure your application isn't spending two minutes doing absolutely nothing for a request that normally takes less than half a second. + +[,js] +---- +include::code/ch06-connection-problems/03-catch-json-errors.js[] +---- + +== Simulating Network Nonsense + +Most of the time developing against an API that works just fine means you cannot test these complicated unhappy paths. -Timeouts are also a concern, but more on those later. +To simulate the sort of nonsense you are coding to defend against, take a look at https://github.com/Shopify/toxiproxy[Toxiproxy] by Shopify. == Rate Limiting -Another common situation to run into is rate limiting: the API telling -you to calm down a bit, and slow down how many requests are being made -in a certain timeframe. The most basic rate limiting strategy is often -"clients can only send X requests per second." +Another common situation to run into is rate limiting, which is basically the +API telling your API client to calm down a bit, and slow down how many requests +are being made. The most basic rate limiting strategy is +often "clients can only send X requests per second." Many APIs implement rate limiting to ensure relative stability when unexpected things happen. If for some reason one client causes a spike @@ -159,7 +184,8 @@ processes handling 5 requests per second each, but you get the idea. If this process was being implemented in NodeJS, you could use https://www.npmjs.com/package/bottleneck[Bottleneck]. -.... +[source,js] +-- const Bottleneck = require("bottleneck"); // Never more than 5 requests running at a time. @@ -176,7 +202,7 @@ const fetchPokemon = id => { limiter.schedule(fetchPokemon, id).then(result => { /* ... */ }) -.... +-- Ruby users who are already using tools like Sidekiq can add plugins like https://github.com/sensortower/sidekiq-throttled[Sidekiq::Throttled], or @@ -195,24 +221,27 @@ the API might lower its limits for some reason. === Am I Being Rate Limited? The appropriate HTTP status code for rate limiting has been argued over -about as much as tabs vs spaces, but there is a clear winner now; -https://tools.ietf.org/html/rfc6585[RFC 6585] defines it as 429, so APIs -should be using 429. +about as much as "tabs" versus "spaces", but there is a clear winner now; +https://tools.ietf.org/html/rfc6585[RFC 6585] defines it as HTTP 429. -image::images/429.jpg[] +.http.cat meme for HTTP 429 +image::images/429.jpg[Lots of cats,500,align="center"] -Twitter's API existed for a few years before this standard, and they -chose "420 - Enhance Your Calm". They've dropped this and moved over to -429, but some others copied them at the time, and might not have updated -since. You cannot rule out bumping into a copycat API, still using that -outdated unofficial status. +Some APIs like Twitter's old API existed for a few years before this standard, +and they chose "420 - Enhance Your Calm". Twitter has dropped 420 and got on +board with the standard 429. Unfortunately some APIs replicated that and have +not yet switched over to using the standard, so you might see either a 429 or +this slow copycat. -image::images/420.jpg[] +.http.cat meme for HTTP 420 +image::images/420.jpg[Cat chewing on a cannabis leaf,500,align="center"] Google also got a little "creative" with their status code utilization. For a -long time were using 403 for their rate limiting, but I have no idea if they are +long time were using 403 for their rate limiting, but I don't know if they are still doing that. Bitbucket are still using 403 in their Server REST API. +// TODO Confirm if google are still doing that. + [quote,REST Resources Provided By: Bitbucket Server,https://docs.atlassian.com/bitbucket-server/rest/5.12.3/bitbucket-rest.html] ____ Actions are usually "forbidden" if they involve breaching the licensed user limit of the server, or degrading the authenticated user's permission level. See the individual resource documentation for more details. @@ -220,7 +249,8 @@ ____ GitHub v3 API has a 403 rate limit too: -.... +[source] +---- HTTP/1.1 403 Forbidden X-RateLimit-Limit: 60 X-RateLimit-Remaining: 0 @@ -229,82 +259,107 @@ X-RateLimit-Reset: 1377013266 "message": "API rate limit exceeded for xxx.xxx.xxx.xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)", "documentation_url": "https://developer.github.com/v3/#rate-limiting" } -.... +---- Getting a 429 (or a 420) is a clear indication that a rate limit has been hit, and a 403 combined with an error code, or maybe some HTTP -headers can also be a thing to check for. Either way, when you're sure -it's a rate limit error, you can move onto the next step: figuring out -how long to wait before trying again. +headers can also be a thing to check for. -=== Proprietary Headers - -Github here are using some proprietary headers, all beginning with -`X-RateLimit-`. These are not at all standard (you can tell by the -`X-`), and could be very different from whatever API you are working -with. - -Successful requests with Github here will show how many requests are -remaining, so maybe keep an eye on those and try to avoid making -requests if the remaining amount on the last response was 0. +Either way, when you're sure it's a rate limit error, you can move onto the next +step: figuring out how long to wait before trying again. -.... -curl -i https://api.github.com/users/octocat -HTTP/1.1 200 OK -X-RateLimit-Limit: 60 -X-RateLimit-Remaining: 56 -X-RateLimit-Reset: 1372700873 -.... +There are three main ways a server might communicate retry logic to you. -You can use a shared key (maybe in Redis or similar) to track that, and -have it expire on the reset provided in -http://en.wikipedia.org/wiki/Unix_time[UTC time] in `X-RateLimit-Reset`. +==== Retry-After Header -=== Retry-After +The Retry-After header is a handy standard way to communicate "this didn't work +now, but it might work if you retry in ". -According to the RFCs for HTTP/1.1 (the obsoleted and irrelevant RFC -2616, and the replacement RFC 9110), the header -https://www.rfc-editor.org/rfc/rfc9110#field.retry-after[Retry-After] is only -for 503 server errors, and maybe redirects. Luckily -https://tools.ietf.org/html/rfc6585[RFC 6584] (the same one which added -HTTP status code 429) says it's totally cool for APIs to use -`Retry-After` there. +``` +Retry-After: +Retry-After: +``` -So, instead of potentially infinite proprietary alternatives, you should -start to see something like this: +The logic for how it works is defined in https://tools.ietf.org/html/rfc6585[RFC 6584] (the same RFC that introduced HTTP 429) but basically it might look a bit like this: -.... +[source] +---- HTTP/1.1 429 Too Many Requests -Retry-After: 3600 +Retry-After: 60 Content-Type: application/json { - "message": "API rate limit exceeded for xxx.xxx.xxx.xxx.", - "documentation_url": "https://developer.example.com/#rate-limiting" + "error": { + "message": "API rate limit exceeded for xxx.xxx.xxx.xxx.", + "link": "https://developer.example.com/#rate-limiting" + } } -.... +---- -An alternative value for Retry-After is an HTTP date: +You might also see a `Retry-After` showing you an HTTP date: -.... -Retry-After: Wed, 21 Oct 2015 07:28:00 GMT -.... +[source] +---- +Retry-After: Sat, 15 April 2023 07:28:00 GMT +---- -Same idea, it just tells the client to wait until then before bothering -the API further. +Same idea, it's just saying "please don't come back before this time". -By checking for these errors, you can catch then retry (or re-queue) -requests that have failed, or if thats not an option try sleeping for a +By checking for these errors, you can catch and retry (or re-queue) +requests that have failed. If that is not an option try sleeping for a bit to calm workers down. + WARNING: Make sure your sleep does not block your background processes from processing other jobs. This can happen in languages where sleep sleeps the whole process, and that process is running multiple types job on the same thread. Don't back up your whole system with an overzealous sleep!_ -Faraday, a ruby gem I work with often, is -https://github.com/lostisland/faraday/pull/773[now aware of -Retry-After]. It uses the value to help calculate the interval between -retry requests. This can be useful for anyone considering implementing -rate limiting detection code, even if you aren't a Ruby fan. +Some HTTP clients like Faraday are +https://github.com/lostisland/faraday/pull/773[aware of Retry-After] and use it +to power their build in retry logic, but other HTTP clients might need some +training. + +// TODO: Code example + + +==== Proprietary Headers + +Some APIs like GitHub v3 use proprietary headers, all beginning with +`X-RateLimit-`. These are not at all standard (you can tell by the +`X-`), and could be very different from whatever API you are working +with. + +Successful requests with Github here will show how many requests are +remaining, so maybe keep an eye on those and try to avoid making +requests if the remaining amount on the last response was 0. + +[source] +---- +$ curl -i https://api.github.com/users/octocat + +HTTP/1.1 200 OK +X-RateLimit-Limit: 60 +X-RateLimit-Remaining: 56 +X-RateLimit-Reset: 1372700873 +---- + +You can use a shared key (maybe in Redis or similar) to track that, and +have it expire on the reset provided in +http://en.wikipedia.org/wiki/Unix_time[UTC time] in `X-RateLimit-Reset`. + + +==== RateLimit Headers (Standard Draft) + +The benefit of the proprietary headers is that you get a lot more information to work with, letting you know you're approaching a limit so you can pre-emptively back off, instead of waiting to stand on that rake then having to respond after being hit round the head. + +There's an IETF RFC draft called https://datatracker.ietf.org/doc/draft-ietf-httpapi-ratelimit-headers/[RateLimit header fields for HTTP] that aims to give you the best of both, and maybe you'll fun into something that resembles this in the distant future of 2024 or 2025. + +---- +RateLimit-Limit: 100 +RateLimit-Remaining: 50 +RateLimit-Reset: 50 +---- + +This says there is a limit of 100 requests in the quota, the client has 50 remaining, and it will reset in 50 seconds. Handy! diff --git a/07-ux-pitfalls.adoc b/07-ux-pitfalls.adoc index 61dbf21..69a0f95 100644 --- a/07-ux-pitfalls.adoc +++ b/07-ux-pitfalls.adoc @@ -65,7 +65,7 @@ https://developers.google.com/web/tools/chrome-devtools/device-mode/#network[net throttling utility]. .Chrome DevTools can simulate different bandwidth options on your site. It's an eye-opening experience. -image::images/ux-chrome-dev-tools.png[] +image::images/ch07-ux-pitfalls/ux-chrome-dev-tools.png[] This, of course, begs the question: what can we do to make the page @@ -76,10 +76,10 @@ when it comes to loading data on a page. You've undoubtedly come across a website whose content loads in unpredictable chunks - with content appearing and unfolding haphazardly on the page right in front of your eyes, pushing content you were trying -to read down without warning.  This is a step better than the "Web 1.0" +to read down without warning. This is a step better than the "Web 1.0" days where you'd need to wait for _everything_ to load before displaying any content, but only because there's a chance you may be able to get to -what you're looking for slightly more quickly.  It's tricky business +what you're looking for slightly more quickly. It's tricky business trying to guess what's going on in an app while content is popping into existence left and right, changing the layout of the page as it arrives back from its journey through the API. @@ -90,16 +90,20 @@ click on it, something on the page loads just above where you're about to click, causing the page to reflow so that you click the _[Sign Out]_ button instead of that one simple thing you wanted to do? -.Yeah, holy shit, right? Me too. Our goal here is to avoid this -image::images/ux-pay-bill-before.png[] -image:images/ux-pay-bill-after.png[] +.User tries to click the Pay Bill button while a component is still loading... +image::images/ch07-ux-pitfalls/ux-pay-bill-before.png[Spin wheel loading above the Pay Bill button,400,200,align="center"] + +.When the component loads that pushes the button down and the user can end up clicking self-destruct. +image::images/ch07-ux-pitfalls/ux-pay-bill-after.png[Cursor is over newly loaded Self Destruct button,400,200,align="center"] + +AHHHH! There's a few great, proven patterns we can use to avoid that kind of rage-inducing experience. While you're waiting on data to come back from an API, render content placeholders on screen. .Facebook does a great job of this - while timeline content is loading, animated data placeholders are displayed in a way that makes the application feel alive. -image::images/ux-facebook-progressive-load.png[] +image::images/ch07-ux-pitfalls/ux-facebook-progressive-load.png[] === Progressive data loading @@ -117,14 +121,14 @@ from "Loading..." to "Still waiting..." after a few moments. .Netflix loads each show individually, maintaining their position in the grid on-screen while others load.  You'll never accidentally tap the wrong show! -image::images/ux-netflix.png[] +image::images/ch07-ux-pitfalls/ux-netflix.png[] -image::images/ux-netflix-2.png[] +image::images/ch07-ux-pitfalls/ux-netflix-2.png[] .The BBC's website loads images last, but holds their position in the layout until images are loaded. -image::images/ux-bbc-loading.png[] +image::images/ch07-ux-pitfalls/ux-bbc-loading.png[] -image::images/ux-bbc.png[] +image::images/ch07-ux-pitfalls/ux-bbc.png[] @@ -151,18 +155,18 @@ activities, this is firmly rooted in science. Spend enough time around seasoned designers, and you'll eventually hear someone prattle on about https://en.wikipedia.org/wiki/Maslow's_hierarchy_of_needs[Maslow's -Hierarchy of Needs].  If up until this moment you've been lucky enough +Hierarchy of Needs]. If up until this moment you've been lucky enough to have never encountered Maslow, I can save you some time: from a UX perspective, the hierarchy tells us that first we should deliver what -the user _needs_ before we give them what they [think they] want.  (In +the user _needs_ before we give them what they [think they] want. (In reality, Maslow tell us quite a bit more than that, but for the sake of -this example, this should be a suitable oversimplification).   +this example, this should be a suitable oversimplification). So let's come back to the example of dealing with a user on a -particularly slow connection.  Looking at any page or interface as a +particularly slow connection. Looking at any page or interface as a hierarchy of data being presented to the user becomes advantageous for us if we have some idea of what actions a user might be looking to -accomplish on a page.   +accomplish on a page. === First: The sublime, a Logical Hierarchy @@ -179,10 +183,10 @@ the biggest/boldest thing, and located somewhere near the very top of the content area of the page. .For example, if you are showing a feed of articles, and next to that feed of data are some suggestions for things to read next, and a list of trending articles, then feed is probably the primary data, the suggestions might be secondary, and the service which shares trending articles might be down, so that could be tertiary. -image::images/progressive-data-loading.png[] +image::images/ch07-ux-pitfalls/progressive-data-loading.png[] .Load the most pertinent content first and progress from there based on how important it is to your reader -image::images/progressive-data-loading-annotated.png[] +image::images/ch07-ux-pitfalls/progressive-data-loading-annotated.png[] === Then: Which data is most expensive? @@ -254,18 +258,18 @@ you detect a loss of connectivity, have a strategy on-hand for presenting that to your users. .Intercom provides a helpful notification when users are offline, without disabling every action on the page. -image::images/ux-intercom-notification.png[] +image::images/ch07-ux-pitfalls/ux-intercom-notification.png[] -You may also be able to cache actions while your users are offline. - You've probably experienced this before with your mail client of -choice.  Gmail, Outlook, Thunderbird, and whatever else you might prefer -will let you draft new emails (and replies to existing emails) while -offline.  You can even send them, which puts the email into your outbox, -to be sent as soon as your connection comes back from the dead. +You may also be able to cache actions while your users are offline. You've +probably experienced this before with your mail client of choice. Gmail, +Outlook, Thunderbird, and whatever else you might prefer will let you draft new +emails (and replies to existing emails) while offline. You can even send them, +which puts the email into your outbox, to be sent as soon as your connection +comes back from the dead. For both web and mobile applications, the strategy for enabling offline actions is fairly similar - first, make sure the user knows they're -offline.  Beyond that, if there are actions that they may reasonably be +offline. Beyond that, if there are actions that they may reasonably be able to perform without loading more information from the web - let them! This generally includes actions where your user is annotating some content (tagging financial records with metadata, marking an action as @@ -274,15 +278,15 @@ a blog post). Behind the scenes, those actions will get cached to local storage on the device using any of a number of techniques, depending on your -implementation.  Once your app detects that connectivity has returned, +implementation. Once your app detects that connectivity has returned, the user's actions are sent off to your API in the order they were executed while offline. Once confirmation comes back from the server that the job is done, data is reloaded on the client-side, and they should be up to speed! In web app parlance, this type of behavior is often called a Progressive -Web App (or PWA).  Depending on your implementation details, there are -loads of different ways to accomplish the PWA dream.  For example, +Web App (or PWA). Depending on your implementation details, there are +loads of different ways to accomplish the PWA dream. For example, Amazon provides a service called https://aws.amazon.com/appsync/[AWS AppSync] for GraphQL, and Google's Firebase has several action caching strategies built into their framework @@ -363,9 +367,9 @@ service went down.  Instead of giving me a frustratingly useless map interface, it let me know the service was down, and hid the view option until the map service came back.  Brilliant! -image::images/ux-spothero.png[] +image::images/ch07-ux-pitfalls/ux-spothero.png[] -image::images/ux-spothero-map-gone.png[] +image::images/ch07-ux-pitfalls/ux-spothero-map-gone.png[] In some cases, you may be able to provide a good fallback: if your video hosting CDN is down, and you have the luxury of a backup CDN, switch to @@ -448,7 +452,7 @@ that even if your request is sent to an API multiple times, it will only ever be executed once. .Stripe's Idempotentcy strategy helps stop API consumers from accidentally creating the same credit card charge more than once. -image::images/ux-idempotency-key.png[Screenshot from the stripe.com API documentation showing an example of the idempotency key.] +image::images/ch07-ux-pitfalls/ux-idempotency-key.png[Screenshot from the stripe.com API documentation showing an example of the idempotency key.] == Dealing with errors @@ -483,7 +487,7 @@ support staff, provide as much detail as possible so that they can find, remedy, and fix any problems they may be tasked with supporting. .Giving your support team as much information as possible can make your helpdesk experience feel like magic. -image::images/ux-errors.png[] +image::images/ch07-ux-pitfalls/ux-errors.png[] You can also use bug tracking services like https://logrocket.com/[LogRocket] or https://sentry.io/[Sentry] to @@ -501,7 +505,7 @@ powers-that-be in your organization, particularly if you have data to back up your story. .An example error report sent to a slack channel via Incoming Webhook. Seeing errors happening in real time will help your team feel empathy for how (infuriatingly) often your users encounter a given bug. -image::images/ux-bug-track.png[] +image::images/ch07-ux-pitfalls/ux-bug-track.png[] == Undo Functionality @@ -509,7 +513,7 @@ If you're doing something semi-permanent, make sure you give users the ability to undo or cancel actions whenever possible. .Toast notification from Gmail serves as an action confirmation and an undo opportunity for a few moments before your email goes out into the world. -image::images/ux-undo.png[] +image::images/ch07-ux-pitfalls/ux-undo.png[] Undo can be accomplished both proactively and reactively.  Proactive undo scenarios are extremely common; most often, this comes in the form @@ -537,4 +541,4 @@ to undo that action here. Finally, this can be presented back to your users as a historical log of their activity on your service. .Stripe provides an Events log to customers using their service, which can be extremely helpful in tracking down failed and missing payments. -image::images/ux-events.png[] +image::images/ch07-ux-pitfalls/ux-events.png[] diff --git a/08-caching.adoc b/08-caching.adoc index 9e7a515..f058c00 100644 --- a/08-caching.adoc +++ b/08-caching.adoc @@ -641,12 +641,12 @@ and maybe not writing the very most performant code possible. The basic idea looks a little like this: -image::images/1HVrgB2mX7EC8tz523pkxaA.png[image] +image::images/ch08-caching/http-caching-miss.png[image] A request being returned early by a cache -server. -- book.varnish-software.com +server. -- book.varnish-software.com -image::images/143stxceFqvz0WDO_2KmOVg.png[image] +image::images/ch08-caching/http-caching-hit.png[image] A request failing to find a match (a.k.a cache miss), and being passed on to the API server to fulfill. -- book.varnish-software.com diff --git a/10-timeouts-retries.adoc b/10-timeouts-retries.adoc index 6dfaa1c..9f47b2f 100644 --- a/10-timeouts-retries.adoc +++ b/10-timeouts-retries.adoc @@ -56,14 +56,14 @@ service B has a bad day, and instead of responding within the usual ~350ms, it starts to take 10 seconds? Do you want to wait 10 seconds for that response?  -image::images/1-new-relic-response-time-spike.png[image] +image::images/ch10-timeouts-retries/1-new-relic-response-time-spike.png[image] What about if service B is fine, but C is taking 20s and D is taking 25s? Are you happy to wait 45 seconds for the response from B? What about two minutes?! 😱 -image::images/2-runscope-two-min-response.png[image] +image::images/ch10-timeouts-retries/2-runscope-two-min-response.png[image] When a server is under load, it can do some pretty wild stuff, and not all servers know to give up. Even fewer client applications know when to @@ -78,7 +78,7 @@ first byte, and if that doesn't happen then the request gets dropped.  Quite often in monitoring systems like NewRelic or CA APM, you will see things like this: -image::images/3-heroku-timeout-chop.png[image] +image::images/ch10-timeouts-retries/3-heroku-timeout-chop.png[image] This controller has an average response time of 2.9s, but the slow ones are floating right around that 30s mark. The Heroku Chop saves the @@ -136,7 +136,7 @@ expected network latency. If you are making an HTTP call to another service in the same data center, the latency is going to be a few milliseconds, but going to another continent takes time.  -image::images/4-cloudping-aws-latency.png[image] +image::images/ch10-timeouts-retries/4-cloudping-aws-latency.png[image] The *read timeout* is how long you want to spend reading data from the server once the connection is open. It's common for this to be set @@ -231,7 +231,7 @@ or whatever monitoring tool is being used. Looking at the response times, you can get an idea of what should be acceptable. Be careful though, *do not look only at the average*. -image::images/5-new-relic-web-transactions-time-damn-lies.png[image] +image::images/ch10-timeouts-retries/5-new-relic-web-transactions-time-damn-lies.png[image] Looking at this graph may lead you to think 300ms is an appropriate timeout. Seems fair right? The biggest spike there is 250ms and so round @@ -241,7 +241,7 @@ Nope! These are averages, and averages are going to be far far lower than the slowest transactions. Click the drop-down and find "Web transaction percentiles." -image::images/6-new-relic-web-transactions-percentile-more-accurate.png[image] +image::images/ch10-timeouts-retries/6-new-relic-web-transactions-percentile-more-accurate.png[image] That is a more honest representation. Most of the responses are 30-50ms, and the average is usually sub 100ms. That said, under *high load* this @@ -361,4 +361,4 @@ controlled more centrally. // the more you want to trust they know what they are trying to do. // also maybe if you can differentiate between server struggling and network latency. -// how do you find that out? \ No newline at end of file +// how do you find that out? diff --git a/book.adoc b/book.adoc index cfb0ffa..6f302ef 100644 --- a/book.adoc +++ b/book.adoc @@ -1,10 +1,10 @@ -= Surviving Other Peoples Web APIs += Surviving Other Peoples APIs Phil Sturgeon ; Mike Bifulco :doctype: book :creator: Phil Sturgeon & Mike Bifulco :front-cover-image: image:cover.jpeg[Front Cover,1050,1600] :keywords: API, architecture, REST, gRPC, RPC, HTTP, GraphQL, microservices, API Design -:source-highlighter: coderay +:source-highlighter: highlight.js :toc: :leveloffset: 1 diff --git a/images/1-new-relic-response-time-spike.png b/images/1-new-relic-response-time-spike.png deleted file mode 100644 index 947c94a..0000000 Binary files a/images/1-new-relic-response-time-spike.png and /dev/null differ diff --git a/images/2-runscope-two-min-response.png b/images/2-runscope-two-min-response.png deleted file mode 100644 index 0cac9f9..0000000 Binary files a/images/2-runscope-two-min-response.png and /dev/null differ diff --git a/images/3-heroku-timeout-chop.png b/images/3-heroku-timeout-chop.png deleted file mode 100644 index 90a9cab..0000000 Binary files a/images/3-heroku-timeout-chop.png and /dev/null differ diff --git a/images/4-cloudping-aws-latency.png b/images/4-cloudping-aws-latency.png deleted file mode 100644 index 58f5f47..0000000 Binary files a/images/4-cloudping-aws-latency.png and /dev/null differ diff --git a/images/5-new-relic-web-transactions-time-damn-lies.png b/images/5-new-relic-web-transactions-time-damn-lies.png deleted file mode 100644 index 995cebd..0000000 Binary files a/images/5-new-relic-web-transactions-time-damn-lies.png and /dev/null differ diff --git a/images/6-new-relic-web-transactions-percentile-more-accurate.png b/images/6-new-relic-web-transactions-percentile-more-accurate.png deleted file mode 100644 index 369467b..0000000 Binary files a/images/6-new-relic-web-transactions-percentile-more-accurate.png and /dev/null differ diff --git a/images/ch04-authentication/github-copilot-crimes.png b/images/ch04-authentication/github-copilot-crimes.png index 0fbd65f..593271b 100644 Binary files a/images/ch04-authentication/github-copilot-crimes.png and b/images/ch04-authentication/github-copilot-crimes.png differ diff --git a/images/ch04-authentication/http-basic.png b/images/ch04-authentication/http-basic.png index 8aa7cdd..6fb01bc 100644 Binary files a/images/ch04-authentication/http-basic.png and b/images/ch04-authentication/http-basic.png differ diff --git a/images/ch04-authentication/stripe-authentication.png b/images/ch04-authentication/stripe-authentication.png index e60cfd8..1b179f5 100644 Binary files a/images/ch04-authentication/stripe-authentication.png and b/images/ch04-authentication/stripe-authentication.png differ diff --git a/images/ch05-error-handling/object-Object.jpg b/images/ch05-error-handling/object-Object.jpg index 8c98230..d2ed39f 100644 Binary files a/images/ch05-error-handling/object-Object.jpg and b/images/ch05-error-handling/object-Object.jpg differ diff --git a/images/ch07-ux-pitfalls/progressive-data-loading-annotated.png b/images/ch07-ux-pitfalls/progressive-data-loading-annotated.png new file mode 100644 index 0000000..2ac8180 Binary files /dev/null and b/images/ch07-ux-pitfalls/progressive-data-loading-annotated.png differ diff --git a/images/ch07-ux-pitfalls/progressive-data-loading.png b/images/ch07-ux-pitfalls/progressive-data-loading.png new file mode 100644 index 0000000..05c83b4 Binary files /dev/null and b/images/ch07-ux-pitfalls/progressive-data-loading.png differ diff --git a/images/ch07-ux-pitfalls/ux-bbc-loading.png b/images/ch07-ux-pitfalls/ux-bbc-loading.png new file mode 100644 index 0000000..0033f1c Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-bbc-loading.png differ diff --git a/images/ch07-ux-pitfalls/ux-bbc.png b/images/ch07-ux-pitfalls/ux-bbc.png new file mode 100644 index 0000000..db606c7 Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-bbc.png differ diff --git a/images/ch07-ux-pitfalls/ux-bug-track.png b/images/ch07-ux-pitfalls/ux-bug-track.png new file mode 100644 index 0000000..8fa06e3 Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-bug-track.png differ diff --git a/images/ch07-ux-pitfalls/ux-chrome-dev-tools.png b/images/ch07-ux-pitfalls/ux-chrome-dev-tools.png new file mode 100644 index 0000000..d3092c4 Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-chrome-dev-tools.png differ diff --git a/images/ch07-ux-pitfalls/ux-errors.png b/images/ch07-ux-pitfalls/ux-errors.png new file mode 100644 index 0000000..21e3e10 Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-errors.png differ diff --git a/images/ch07-ux-pitfalls/ux-events.png b/images/ch07-ux-pitfalls/ux-events.png new file mode 100644 index 0000000..f5b441b Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-events.png differ diff --git a/images/ch07-ux-pitfalls/ux-facebook-progressive-load.png b/images/ch07-ux-pitfalls/ux-facebook-progressive-load.png new file mode 100644 index 0000000..24e32f3 Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-facebook-progressive-load.png differ diff --git a/images/ch07-ux-pitfalls/ux-idempotency-key.png b/images/ch07-ux-pitfalls/ux-idempotency-key.png new file mode 100644 index 0000000..7847b13 Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-idempotency-key.png differ diff --git a/images/ch07-ux-pitfalls/ux-intercom-notification.png b/images/ch07-ux-pitfalls/ux-intercom-notification.png new file mode 100644 index 0000000..680bc63 Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-intercom-notification.png differ diff --git a/images/ch07-ux-pitfalls/ux-netflix-2.png b/images/ch07-ux-pitfalls/ux-netflix-2.png new file mode 100644 index 0000000..dcf4142 Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-netflix-2.png differ diff --git a/images/ch07-ux-pitfalls/ux-netflix.png b/images/ch07-ux-pitfalls/ux-netflix.png new file mode 100644 index 0000000..60f697a Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-netflix.png differ diff --git a/images/ch07-ux-pitfalls/ux-pay-bill-after.png b/images/ch07-ux-pitfalls/ux-pay-bill-after.png new file mode 100644 index 0000000..c40234d Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-pay-bill-after.png differ diff --git a/images/ch07-ux-pitfalls/ux-pay-bill-before.png b/images/ch07-ux-pitfalls/ux-pay-bill-before.png new file mode 100644 index 0000000..a045c37 Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-pay-bill-before.png differ diff --git a/images/ch07-ux-pitfalls/ux-spothero-map-gone.png b/images/ch07-ux-pitfalls/ux-spothero-map-gone.png new file mode 100644 index 0000000..003476d Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-spothero-map-gone.png differ diff --git a/images/ch07-ux-pitfalls/ux-spothero.png b/images/ch07-ux-pitfalls/ux-spothero.png new file mode 100644 index 0000000..e85764a Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-spothero.png differ diff --git a/images/ch07-ux-pitfalls/ux-undo.png b/images/ch07-ux-pitfalls/ux-undo.png new file mode 100644 index 0000000..b8474db Binary files /dev/null and b/images/ch07-ux-pitfalls/ux-undo.png differ diff --git a/images/143stxceFqvz0WDO_2KmOVg.png b/images/ch08-caching/http-caching-hit.png similarity index 100% rename from images/143stxceFqvz0WDO_2KmOVg.png rename to images/ch08-caching/http-caching-hit.png diff --git a/images/1HVrgB2mX7EC8tz523pkxaA.png b/images/ch08-caching/http-caching-miss.png similarity index 100% rename from images/1HVrgB2mX7EC8tz523pkxaA.png rename to images/ch08-caching/http-caching-miss.png diff --git a/images/ch10-timeouts-retries/1-new-relic-response-time-spike.png b/images/ch10-timeouts-retries/1-new-relic-response-time-spike.png new file mode 100644 index 0000000..ffd9656 Binary files /dev/null and b/images/ch10-timeouts-retries/1-new-relic-response-time-spike.png differ diff --git a/images/ch10-timeouts-retries/2-runscope-two-min-response.png b/images/ch10-timeouts-retries/2-runscope-two-min-response.png new file mode 100644 index 0000000..65937a8 Binary files /dev/null and b/images/ch10-timeouts-retries/2-runscope-two-min-response.png differ diff --git a/images/ch10-timeouts-retries/3-heroku-timeout-chop.png b/images/ch10-timeouts-retries/3-heroku-timeout-chop.png new file mode 100644 index 0000000..e48a798 Binary files /dev/null and b/images/ch10-timeouts-retries/3-heroku-timeout-chop.png differ diff --git a/images/ch10-timeouts-retries/4-cloudping-aws-latency.png b/images/ch10-timeouts-retries/4-cloudping-aws-latency.png new file mode 100644 index 0000000..4c513e0 Binary files /dev/null and b/images/ch10-timeouts-retries/4-cloudping-aws-latency.png differ diff --git a/images/ch10-timeouts-retries/5-new-relic-web-transactions-time-damn-lies.png b/images/ch10-timeouts-retries/5-new-relic-web-transactions-time-damn-lies.png new file mode 100644 index 0000000..e8276ba Binary files /dev/null and b/images/ch10-timeouts-retries/5-new-relic-web-transactions-time-damn-lies.png differ diff --git a/images/ch10-timeouts-retries/6-new-relic-web-transactions-percentile-more-accurate.png b/images/ch10-timeouts-retries/6-new-relic-web-transactions-percentile-more-accurate.png new file mode 100644 index 0000000..a06b396 Binary files /dev/null and b/images/ch10-timeouts-retries/6-new-relic-web-transactions-percentile-more-accurate.png differ diff --git a/images/progressive-data-loading-annotated.png b/images/progressive-data-loading-annotated.png deleted file mode 100644 index d285180..0000000 Binary files a/images/progressive-data-loading-annotated.png and /dev/null differ diff --git a/images/progressive-data-loading.png b/images/progressive-data-loading.png deleted file mode 100644 index 1a52e8e..0000000 Binary files a/images/progressive-data-loading.png and /dev/null differ diff --git a/images/ux-bbc-loading.png b/images/ux-bbc-loading.png deleted file mode 100644 index d476570..0000000 Binary files a/images/ux-bbc-loading.png and /dev/null differ diff --git a/images/ux-bbc.png b/images/ux-bbc.png deleted file mode 100644 index 307380f..0000000 Binary files a/images/ux-bbc.png and /dev/null differ diff --git a/images/ux-bug-track.png b/images/ux-bug-track.png deleted file mode 100644 index 2246c52..0000000 Binary files a/images/ux-bug-track.png and /dev/null differ diff --git a/images/ux-chrome-dev-tools.png b/images/ux-chrome-dev-tools.png deleted file mode 100644 index 17fa659..0000000 Binary files a/images/ux-chrome-dev-tools.png and /dev/null differ diff --git a/images/ux-errors.png b/images/ux-errors.png deleted file mode 100644 index 95b037d..0000000 Binary files a/images/ux-errors.png and /dev/null differ diff --git a/images/ux-events.png b/images/ux-events.png deleted file mode 100644 index 2faeeb2..0000000 Binary files a/images/ux-events.png and /dev/null differ diff --git a/images/ux-facebook-progressive-load.png b/images/ux-facebook-progressive-load.png deleted file mode 100644 index accceb3..0000000 Binary files a/images/ux-facebook-progressive-load.png and /dev/null differ diff --git a/images/ux-idempotency-key.png b/images/ux-idempotency-key.png deleted file mode 100644 index a970dc7..0000000 Binary files a/images/ux-idempotency-key.png and /dev/null differ diff --git a/images/ux-intercom-notification.png b/images/ux-intercom-notification.png deleted file mode 100644 index 4458db9..0000000 Binary files a/images/ux-intercom-notification.png and /dev/null differ diff --git a/images/ux-netflix-2.png b/images/ux-netflix-2.png deleted file mode 100644 index 11c34cb..0000000 Binary files a/images/ux-netflix-2.png and /dev/null differ diff --git a/images/ux-netflix.png b/images/ux-netflix.png deleted file mode 100644 index 2d1db94..0000000 Binary files a/images/ux-netflix.png and /dev/null differ diff --git a/images/ux-pay-bill-after.png b/images/ux-pay-bill-after.png deleted file mode 100644 index 3a1e96e..0000000 Binary files a/images/ux-pay-bill-after.png and /dev/null differ diff --git a/images/ux-pay-bill-before.png b/images/ux-pay-bill-before.png deleted file mode 100644 index 927e387..0000000 Binary files a/images/ux-pay-bill-before.png and /dev/null differ diff --git a/images/ux-spothero-map-gone.png b/images/ux-spothero-map-gone.png deleted file mode 100644 index 3ed6d41..0000000 Binary files a/images/ux-spothero-map-gone.png and /dev/null differ diff --git a/images/ux-spothero.png b/images/ux-spothero.png deleted file mode 100644 index c92f221..0000000 Binary files a/images/ux-spothero.png and /dev/null differ diff --git a/images/ux-undo.png b/images/ux-undo.png deleted file mode 100644 index 32443ec..0000000 Binary files a/images/ux-undo.png and /dev/null differ