normalize_path_middleware uses 301 redirect

[dtkav]

Long story short

The normalize_path_middleware in default configuration returns a 301 redirect to append a slash like so:
POST /api/v0/data/ -> {"some": "data"}
POST /api/v0/data -> 301 /api/v0/data/

If a client tries to POST some data to the endpoint without the trailing slash, then a redirect will take place and it will become a GET request. This is due to client implementations of 301, 302, and 303 redirects.

The 308 redirect introduced in https://tools.ietf.org/html/rfc7538 works around this problem, as it forbids clients from changing the method on redirect.

308 redirect in the aiohttp client was added in #2134

Expected behaviour

Following a trailing slash redirect should still work with POST/PUT methods.
Trailing slash redirect should be a 308 redirect.

For comparison, werkzeug uses a 308 redirect for strict_slashes https://github.com/pallets/werkzeug/blob/master/werkzeug/routing.py#L227

relevant werkzeug PR: pallets/werkzeug@fd6eb30

Actual behaviour

301 redirect is returned, causing the aiohttp client to send a GET request instead of the intended POST request with data.
This behavior is also seen with requests and curl (although slightly different).

Steps to reproduce

server

from aiohttp import web
from aiohttp.web_middlewares import normalize_path_middleware

async def handle(request):
    try:
        j = await request.json()
    except Exception:
        return web.json_response({"error": f"no data from {request.method}"})
    return web.json_response(j)

trailing_slash_redirect = normalize_path_middleware(append_slash=True)
app = web.Application(middlewares=[trailing_slash_redirect])
app.add_routes([web.post('/data/', handle),
                web.get('/data/', handle)])

web.run_app(app)

server test with curl -X POST

curl -v -L -H 'Content-Type: application/json' -X POST localhost:8080/data -d '{"echo": "me"}'
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /data HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 14
> 
* upload completely sent off: 14 out of 14 bytes
< HTTP/1.1 301 Moved Permanently
< Content-Type: text/plain; charset=utf-8
< Location: /data/
< Content-Length: 22
< Date: Fri, 25 Jan 2019 00:39:26 GMT
< Server: Python/3.6 aiohttp/3.5.4
< 
* Ignoring the response-body
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:8080/data/'
* Switch from POST to GET
* Found bundle for host localhost: 0x559b853e36a0 [can pipeline]
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /data/ HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Type: application/json
> 
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< Content-Length: 30
< Date: Fri, 25 Jan 2019 00:39:26 GMT
< Server: Python/3.6 aiohttp/3.5.4
< 
* Connection #0 to host localhost left intact
{"error": "no data from POST"}

server test with curl --post301:

curl -v -L -H 'Content-Type: application/json' --post301 localhost:8080/data -d '{"echo": "me"}'
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /data HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 14
> 
* upload completely sent off: 14 out of 14 bytes
< HTTP/1.1 301 Moved Permanently
< Content-Type: text/plain; charset=utf-8
< Location: /data/
< Content-Length: 22
< Date: Thu, 24 Jan 2019 10:52:26 GMT
< Server: Python/3.6 aiohttp/3.5.4
< 
* Ignoring the response-body
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:8080/data/'
* Found bundle for host localhost: 0x56164d97d6a0 [can pipeline]
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /data/ HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 14
> 
* upload completely sent off: 14 out of 14 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< Content-Length: 14
< Date: Thu, 24 Jan 2019 10:52:26 GMT
< Server: Python/3.6 aiohttp/3.5.4
< 
* Connection #0 to host localhost left intact
{"echo": "me"}

aiohttp client test

import aiohttp
import asyncio


async def main():
    async with aiohttp.ClientSession() as session:
        url = 'http://localhost:8080/data'
        response = await session.post(url, data=b'{"echo": "me"}')
        print(url, await response.text())
        url += '/'
        response = await session.post(url, data=b'{"echo": "me"}')
        print(url, await response.text())

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

output

http://localhost:8080/data {"error": "no data from GET"}
http://localhost:8080/data/ {"echo": "me"}

Your environment

Linux, Python 3.6.7, aiohttp 3.5.4 client

pip freeze

aiohttp==3.5.4
async-timeout==3.0.1
attrs==18.2.0
chardet==3.0.4
idna==2.8
idna-ssl==1.1.0
multidict==4.5.2
pkg-resources==0.0.0
typing-extensions==3.7.2
yarl==1.3.0

python --version

Python 3.6.7

[aio-libs-bot]

GitMate.io thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are #3390 (redirects?), #3082 ([client] 301 in POST gets redirected to GET), #3132 (Too many 301 redirects), #3086 (add_static breaks normalize_path_middleware), and #56 (Redirect loop).

[asvetlov]

Fixed by #3579