[WIP - DO NOT MERGE] feat: ASGI support

This patch adds the much-anticipated async support to Falcon by way of a
new ASGI interface, additional testing helpers, and updated internals.

Note that only the HTTP ASGI interface is implemented. WebSocket support
is planned to follow.

I still need to get this to 100% test coverage before we can do a final
review and merge.

Docs will arrive in a follow-up patch once we think the implementation is
ready to merge.

Changelog snippets will be added in a separate patch or a revision of this
one (TBD).

In order to reconcile differences between the WSGI and ASGI interfaces,
several breaking changes were made in this patch, as follows:

BREAKING CHANGE: create_environ no longer sets a default user agent header

BREAKING CHANGE: Renamed `protocol` kwarg for create_environ() to
	`http_version` and also the renamed kwarg only takes the version string
	(no longer prefixed with "HTTP/")

BREAKING CHANGE: Renamed `app` kwarg for create_environ() to `root_path`.
	and deprecated, may be removed in a future release.

BREAKING CHANGE: get_http_status() is deprecated, no longer accepts floats

BREAKING CHANGE: BoundedStream.writeable() changed to writable() per the
	standard file-like I/O interface (the old name was a misspelling).

BREAKING CHANGE: api_helpers.prepare_middleware() no longer accepts a single
	object; the value that is passed must be an iterable.

BREAKING CHANGE: Removed outer "finally" block from API and APP; add an
	exception handler for the base Exception type if you need to deal with
	unhandled exceptions.

BREAKING CHANGE: falcon.request.access_route will now include the value of
  the remote_addr property as the last element in the route, if not already present
  in one of the headers that are checked.

BREAKING CHANGE: When the 'REMOTE_ADDR' field is not present in the WSGI
	environ, Falcon will assume '127.0.0.1' for the value, rather than
	simply returning `None` for Request.remote_addr.

.gitignore

@@ -20,6 +20,7 @@ lib64
 parts
 sdist
 var
+pip-wheel-metadata
 
 # Installer logs
 pip-log.txt

CONTRIBUTING.md

@@ -27,20 +27,18 @@ Please note that all contributors and maintainers of this project are subject to
 
 Before submitting a pull request, please ensure you have added or updated tests as appropriate, and that all existing tests still pass with your changes. Please also ensure that your coding style follows PEP 8.
 
-You can check all this by running the following from within the Falcon project directory (requires Python 3.7 to be installed on your system):
+You can check all this by running the following from within the Falcon project directory (requires Python 3.7 and 3.5 to be installed on your system):
 
 ```bash
 $ tools/mintest.sh
 
 ```
 
-You may also use Python 3.5 or 3.6 if you don't have 3.7 installed on your system. Substitute "py35" or "py36" as appropriate. For example:
-
+If you are useing pyenv, you will need to make sure both 3.7 and 3.5 are available in the current shell, e.g.:
 
 ```bash
-$ pip install -U tox coverage
-$ rm -f .coverage.*
-$ tox -e pep8 && tox -e py35 && tools/testing/combine_coverage.sh
+$ pyenv shell 3.7.3 3.5.7
+```
 
 #### Reviews
 

docs/api/testing.rst

@@ -11,5 +11,5 @@ Reference
         simulate_request, simulate_get, simulate_head, simulate_post,
         simulate_put, simulate_options, simulate_patch, simulate_delete,
         TestClient, TestCase, SimpleTestResource, StartResponseMock,
-        capture_responder_args, rand_string, create_environ, redirected,
-        closed_wsgi_iterable
+        capture_responder_args, rand_string, create_environ, create_req,
+        create_asgi_req, redirected, closed_wsgi_iterable

falcon/__init__.py

@@ -12,9 +12,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-"""Primary package for Falcon, the minimalist WSGI library.
+"""Primary package for Falcon, the minimalist web API framework.
 
-Falcon is a minimalist WSGI library for building speedy web APIs and app
+Falcon is a minimalist web API framework for building speedy web APIs and app
 backends. The `falcon` package can be used to directly access most of
 the framework's classes, functions, and variables::
 
@@ -24,6 +24,8 @@
 
 """
 
+import sys as _sys
+
 # Hoist classes and functions into the falcon namespace
 from falcon.version import __version__  # NOQA
 from falcon.constants import *  # NOQA
@@ -44,3 +46,5 @@
 from falcon.hooks import before, after  # NOQA
 from falcon.request import Request, RequestOptions, Forwarded  # NOQA
 from falcon.response import Response, ResponseOptions  # NOQA
+
+PY35 = _sys.version_info.minor == 5

falcon/api.py

@@ -42,6 +42,8 @@
 ])
 
 
+# TODO(kgriffs): Rename to "App" and alias to "API" for backwards-
+#   compatibility (also grep the docs/samples and update as needed.)
 class API:
     """This class is the main entry point into a Falcon-based app.
 
@@ -55,9 +57,11 @@ class API:
             number of constants for common media types, such as
             ``falcon.MEDIA_MSGPACK``, ``falcon.MEDIA_YAML``,
             ``falcon.MEDIA_XML``, etc.
-        middleware(object or list): Either a single object or a list
-            of objects (instantiated classes) that implement the
-            following middleware component interface::
+        middleware: Either a single middleware component object or an iterable
+            of objects (instantiated classes) that implement the following
+            middleware component interface. Note that it is only necessary
+            to implement the methods for the events you would like to
+            handle; Falcon simply skips over any missing middleware methods::
 
                 class ExampleComponent:
                     def process_request(self, req, resp):
@@ -151,24 +155,25 @@ def process_response(self, req, resp, resource, req_succeeded)
 
     _STREAM_BLOCK_SIZE = 8 * 1024  # 8 KiB
 
+    _STATIC_ROUTE_TYPE = routing.StaticRoute
+
     __slots__ = ('_request_type', '_response_type',
-                 '_error_handlers', '_media_type', '_router', '_sinks',
+                 '_error_handlers', '_router', '_sinks',
                  '_serialize_error', 'req_options', 'resp_options',
                  '_middleware', '_independent_middleware', '_router_search',
-                 '_static_routes')
+                 '_static_routes', '_unprepared_middleware')
 
     def __init__(self, media_type=DEFAULT_MEDIA_TYPE,
                  request_type=Request, response_type=Response,
                  middleware=None, router=None,
                  independent_middleware=True):
         self._sinks = []
-        self._media_type = media_type
         self._static_routes = []
 
         # set middleware
-        self._middleware = helpers.prepare_middleware(
-            middleware, independent_middleware=independent_middleware)
+        self._unprepared_middleware = []
         self._independent_middleware = independent_middleware
+        self.add_middleware(middleware)
 
         self._router = router or routing.DefaultRouter()
         self._router_search = self._router.find
@@ -189,6 +194,32 @@ def __init__(self, media_type=DEFAULT_MEDIA_TYPE,
         self.add_error_handler(falcon.HTTPError, self._http_error_handler)
         self.add_error_handler(falcon.HTTPStatus, self._http_status_handler)
 
+    def add_middleware(self, middleware):
+        """Add one or more additional middleware components.
+
+        Arguments:
+            middleware: Either a single middleware component or an iterable
+                of components to add. The component(s) will be invoked, in
+                order, as if they had been appended to the original middleware
+                list passed to the class initializer.
+        """
+
+        # NOTE(kgriffs): Since this is called by the initializer, there is
+        #   the chance that middleware may be None.
+        if middleware:
+            try:
+                self._unprepared_middleware += middleware
+            except TypeError:  # middleware is not iterable; assume it is just one bare component
+                self._unprepared_middleware.append(middleware)
+
+        # NOTE(kgriffs): Even if middleware is None or an empty list, we still
+        #   need to make sure self._middleware is initialized if this is the
+        #   first call to add_middleware().
+        self._middleware = self._prepare_middleware(
+            self._unprepared_middleware,
+            independent_middleware=self._independent_middleware
+        )
+
     def __call__(self, env, start_response):  # noqa: C901
         """WSGI `app` method.
 
@@ -217,83 +248,77 @@ def __call__(self, env, start_response):  # noqa: C901
         req_succeeded = False
 
         try:
-            try:
-                # NOTE(ealogar): The execution of request middleware
-                # should be before routing. This will allow request mw
-                # to modify the path.
-                # NOTE: if flag set to use independent middleware, execute
-                # request middleware independently. Otherwise, only queue
-                # response middleware after request middleware succeeds.
-                if self._independent_middleware:
-                    for process_request in mw_req_stack:
+            # NOTE(ealogar): The execution of request middleware
+            # should be before routing. This will allow request mw
+            # to modify the path.
+            # NOTE: if flag set to use independent middleware, execute
+            # request middleware independently. Otherwise, only queue
+            # response middleware after request middleware succeeds.
+            if self._independent_middleware:
+                for process_request in mw_req_stack:
+                    process_request(req, resp)
+                    if resp.complete:
+                        break
+            else:
+                for process_request, process_response in mw_req_stack:
+                    if process_request and not resp.complete:
                         process_request(req, resp)
+                    if process_response:
+                        dependent_mw_resp_stack.insert(0, process_response)
+
+            if not resp.complete:
+                # NOTE(warsaw): Moved this to inside the try except
+                # because it is possible when using object-based
+                # traversal for _get_responder() to fail.  An example is
+                # a case where an object does not have the requested
+                # next-hop child resource. In that case, the object
+                # being asked to dispatch to its child will raise an
+                # HTTP exception signalling the problem, e.g. a 404.
+                responder, params, resource, req.uri_template = self._get_responder(req)
+        except Exception as ex:
+            if not self._handle_exception(req, resp, ex, params):
+                raise
+        else:
+            try:
+                # NOTE(kgriffs): If the request did not match any
+                # route, a default responder is returned and the
+                # resource is None. In that case, we skip the
+                # resource middleware methods. Resource will also be
+                # None when a middleware method already set
+                # resp.complete to True.
+                if resource:
+                    # Call process_resource middleware methods.
+                    for process_resource in mw_rsrc_stack:
+                        process_resource(req, resp, resource, params)
                         if resp.complete:
                             break
-                else:
-                    for process_request, process_response in mw_req_stack:
-                        if process_request and not resp.complete:
-                            process_request(req, resp)
-                        if process_response:
-                            dependent_mw_resp_stack.insert(0, process_response)
 
                 if not resp.complete:
-                    # NOTE(warsaw): Moved this to inside the try except
-                    # because it is possible when using object-based
-                    # traversal for _get_responder() to fail.  An example is
-                    # a case where an object does not have the requested
-                    # next-hop child resource. In that case, the object
-                    # being asked to dispatch to its child will raise an
-                    # HTTP exception signalling the problem, e.g. a 404.
-                    responder, params, resource, req.uri_template = self._get_responder(req)
+                    responder(req, resp, **params)
+
+                req_succeeded = True
+            except Exception as ex:
+                if not self._handle_exception(req, resp, ex, params):
+                    raise
+
+        # Call process_response middleware methods.
+        for process_response in mw_resp_stack or dependent_mw_resp_stack:
+            try:
+                process_response(req, resp, resource, req_succeeded)
             except Exception as ex:
                 if not self._handle_exception(req, resp, ex, params):
                     raise
-            else:
-                try:
-                    # NOTE(kgriffs): If the request did not match any
-                    # route, a default responder is returned and the
-                    # resource is None. In that case, we skip the
-                    # resource middleware methods. Resource will also be
-                    # None when a middleware method already set
-                    # resp.complete to True.
-                    if resource:
-                        # Call process_resource middleware methods.
-                        for process_resource in mw_rsrc_stack:
-                            process_resource(req, resp, resource, params)
-                            if resp.complete:
-                                break
-
-                    if not resp.complete:
-                        responder(req, resp, **params)
-
-                    req_succeeded = True
-                except Exception as ex:
-                    if not self._handle_exception(req, resp, ex, params):
-                        raise
-        finally:
-            # NOTE(kgriffs): It may not be useful to still execute
-            # response middleware methods in the case of an unhandled
-            # exception, but this is done for the sake of backwards
-            # compatibility, since it was incidentally the behavior in
-            # the 1.0 release before this section of the code was
-            # reworked.
-
-            # Call process_response middleware methods.
-            for process_response in mw_resp_stack or dependent_mw_resp_stack:
-                try:
-                    process_response(req, resp, resource, req_succeeded)
-                except Exception as ex:
-                    if not self._handle_exception(req, resp, ex, params):
-                        raise
 
-                    req_succeeded = False
+                req_succeeded = False
 
         #
         # Set status and headers
         #
 
         resp_status = resp.status
-        media_type = self._media_type
+        default_media_type = self.resp_options.default_media_type
+
+        body, length = self._get_body(resp, env.get('wsgi.file_wrapper'))
 
         if req.method == 'HEAD' or resp_status in _BODILESS_STATUS_CODES:
             body = []
@@ -307,11 +332,20 @@ def __call__(self, env, start_response):  # noqa: C901
             # presence of the Content-Length header is not similarly
             # enforced.
             if resp_status in _TYPELESS_STATUS_CODES:
-                media_type = None
+                default_media_type = None
+            elif (
+                length is not None and
+                req.method == 'HEAD'and
+                resp_status not in _BODILESS_STATUS_CODES and
+                'content-length' not in resp._headers
+            ):
+                # NOTE(kgriffs): We really should be returning a Content-Length
+                #   in this case according to my reading of the RFCs. By
+                #   optionally using len(data) we let a resource simulate HEAD
+                #   by turning around and calling it's own on_get().
+                resp._headers['content-length'] = str(length)
 
         else:
-            body, length = self._get_body(resp, env.get('wsgi.file_wrapper'))
-
             # PERF(kgriffs): Böse mußt sein. Operate directly on resp._headers
             #   to reduce overhead since this is a hot/critical code path.
             # NOTE(kgriffs): We always set content-length to match the
@@ -322,7 +356,7 @@ def __call__(self, env, start_response):  # noqa: C901
             if length is not None:
                 resp._headers['content-length'] = str(length)
 
-        headers = resp._wsgi_headers(media_type)
+        headers = resp._wsgi_headers(default_media_type)
 
         # Return the response per the WSGI spec.
         start_response(resp_status, headers)
@@ -414,6 +448,10 @@ def add_static_route(self, prefix, directory, downloadable=False, fallback_filen
             For security reasons, the directory and the fallback_filename (if provided)
             should be read only for the account running the application.
 
+        Note:
+            For ASGI apps, file reads are made non-blocking by scheduling
+            them on the default executor.
+
         Static routes are matched in LIFO order. Therefore, if the same
         prefix is used for two routes, the second one will override the
         first. This also means that more specific routes should be added
@@ -448,8 +486,8 @@ def add_static_route(self, prefix, directory, downloadable=False, fallback_filen
 
         self._static_routes.insert(
             0,
-            routing.StaticRoute(prefix, directory, downloadable=downloadable,
-                                fallback_filename=fallback_filename)
+            self._STATIC_ROUTE_TYPE(prefix, directory, downloadable=downloadable,
+                                    fallback_filename=fallback_filename)
         )
 
     def add_sink(self, sink, prefix=r'/'):
@@ -646,11 +684,19 @@ def my_serializer(req, resp, exception):
     # Helpers that require self
     # ------------------------------------------------------------------------
 
-    def _get_responder(self, req):
+    def _prepare_middleware(self, middleware=None, independent_middleware=False):
+        return helpers.prepare_middleware(
+            middleware=middleware,
+            independent_middleware=independent_middleware
+        )
+
+    def _get_responder(self, req, asgi=False):
         """Search routes for a matching responder.
 
         Args:
-            req: The request object.
+            req (Request): The request object.
+            asgi (bool): ``True`` if using an ASGI app, ``False`` otherwise
+                (default ``False``).
 
         Returns:
             tuple: A 4-member tuple consisting of a responder callable,
@@ -694,7 +740,10 @@ def _get_responder(self, req):
             try:
                 responder = method_map[method]
             except KeyError:
-                responder = falcon.responders.bad_request
+                if asgi:
+                    responder = falcon.responders.bad_request_async
+                else:
+                    responder = falcon.responders.bad_request
         else:
             params = {}
 
@@ -712,7 +761,10 @@ def _get_responder(self, req):
                         responder = sr
                         break
                 else:
-                    responder = falcon.responders.path_not_found
+                    if asgi:
+                        responder = falcon.responders.path_not_found_async
+                    else:
+                        responder = falcon.responders.path_not_found
 
         return (responder, params, resource, uri_template)