Add 400 response when `boundary` is missing #1617

Kludex · 2022-05-03T05:53:48Z

Closes Send 400 on missing boundary #1588
Related to multipart/form-data subpart without name causes internal error #1303 (I've not included it on this PR for now...)
Replaces Add 400 response on MultiParser exceptions #1593 (monologue about possible solutions from there is important)

Behavior of this PR:

Custom exception in case "app" not in scope.
HTTPException(status_code=400) in case "app" in scope.

Changes:

Move HTTPException to http_exception.py to avoid circular import.
Add custom exception MultiPartException on missing boundary.
Handle MultiPartException on form() with the behavior mentioned above.

Alternatives/thoughts related to this PR:

Check Add 400 response on MultiParser exceptions #1593 - my thoughts are listed there in a timely manner...

The handling of MultiPartException can also be on an upper level:

starlette/starlette/routing.py

Lines 250 to 261 in 830f348

    
           async def handle(self, scope: Scope, receive: Receive, send: Send) -> None: 
        
               if self.methods and scope["method"] not in self.methods: 
        
                   headers = {"Allow": ", ".join(self.methods)} 
        
                   if "app" in scope: 
        
                       raise HTTPException(status_code=405, headers=headers) 
        
                   else: 
        
                       response = PlainTextResponse( 
        
                           "Method Not Allowed", status_code=405, headers=headers 
        
                       ) 
        
                   await response(scope, receive, send) 
        
               else: 
        
                   await self.app(scope, receive, send)

This alternative would avoid creating the http_exception.py... 🤔

starlette/http_exception.py

starlette/formparsers.py

…dary-2

starlette/requests.py

Kludex · 2022-05-08T06:38:26Z

starlette/exceptions.py


-        async def sender(message: Message) -> None:
-            nonlocal response_started
+def __getattr__(name: str) -> typing.Any:  # pragma: no cover


I need to add a test for this.

Didn't realise you could do this at a module level. 😎

tests/test_formparsers.py

adriangb · 2022-05-12T04:48:33Z

starlette/exceptions.py


-        async def sender(message: Message) -> None:
-            nonlocal response_started
+def __getattr__(name: str) -> typing.Any:  # pragma: no cover


What's the plan to remove all of this deprecation code? I get why we need it, but it'd be nice to know when we're going to remove it so that:

We can put it in the code for our knowledge down the road.

We can put it in the error message.

I suppose this depends on #1623 , but I think we should make an executive decision (maybe @tomchristie ?) on when / if the deprecation shim is ever going to be removed.

Personally, I feel like 3 minor releases or 1 year, whichever comes first, sounds reasonable?

I was more into 6 months and 3 minors... But any number is good for me... I'd just like to have a number 👍 But let's mention those stuff on that discussion.

For here, there's no plan yet. The plan should be defined on that discussion. If this is merged, it's just common sense i.e. in some arbitrary time we just remove it...

adriangb

This is an interesting change.

Overall the change (returning a proper exception to the client instead of a 500) makes absolute sense.

Unfortunately you ended up needing to do a bunch of surgery to the codebase to make it happen because raising an HTTPexception from within a Request requires that Request reference HTTPException, but ExceptionMiddleware was in the same module as HTTPException, which caused circular references. This makes the diff super ugly and huge relative to the small straightforward feature.

But I think that's okay: we are paying the price to fix an issue that already existed in our codebase that might have caused us issues down the road some other time. And personally I think moving the middleware into middlewares/ really cleans things up nicely.

I'm approving this, I do however want 1 more approver and some consensus on the deprecation policy because this is a breaking change for our downstream dependencies.

Kludex · 2022-05-12T17:19:00Z

[...] I do however want 1 more approver [...]

Yep 👍

Thanks for the review! 🙏

tomchristie

Some questions which could be addressed, but they're not blockers so will leave this in your hands at this point @Kludex.

Really nice switch around with moving the middleware. 👍

tomchristie · 2022-05-18T08:47:00Z

starlette/exceptions.py


-        async def sender(message: Message) -> None:
-            nonlocal response_started
+def __getattr__(name: str) -> typing.Any:  # pragma: no cover


Didn't realise you could do this at a module level. 😎

tomchristie · 2022-05-18T08:56:50Z

starlette/middleware/exceptions.py

+from starlette.types import ASGIApp, Message, Receive, Scope, Send
+
+
+class ExceptionMiddleware:


👍 Yup - much better to have this here.

Have confirmed to myself that the class here and the class when it was in exceptions.py are identical.

tomchristie · 2022-05-18T08:59:39Z

starlette/requests.py

+                    multipart_parser = MultiPartParser(self.headers, self.stream())
+                    self._form = await multipart_parser.parse()
+                except MultiPartException as exc:
+                    if "app" in self.scope:


Do we need to have the switch here? Can we just always coerce to an HTTPException here instead? Are we already using this pattern, and if so, where?

https://github.com/encode/starlette/blob/master/starlette/routing.py#L253-L258

Yep, we use the pattern in the code, as @adriangb pointed out.

Always coerce to an HTTPException is an option. I opted to not do it because we only use HTTPExceptions when we are in Starlette (as application) e.g. "app" in self.scope.

Realistically, I think we could remove that pattern everywhere. IMO it's not worth supporting (nice) usage of Starlette's Request object outside of a Starlette app. If someone wants to do that, they can handle the HTTPException.

If someone wants to do that, they can handle the HTTPException.

You have a point, but I think that's another discussion.

I really don't think it's a burden to maintain that, but I wonder if someone really uses that code outside Starlette (app)... 🤔

tomchristie · 2022-05-18T09:04:16Z

tests/test_formparsers.py

    client = test_client_factory(app)
-    with pytest.raises(KeyError, match="boundary"):
-        client.post(
+    with expectation:


I'd personally probably nudge towards the simpler pytest.raises(KeyError, match="boundary") case here, just because it's more obviously readable to me.

The parameterised expectation is neat, but also more complex to understand.

Just my personal perspective tho, happy to go either way.

The idea was to avoid the creation of two very similar tests. Do you think is clear if I create two tests instead?

I think your way or combining an exception and non-exception test into 1 is the cleanest I've seen for that pattern, but if it's
only 2 tests and not 10 I also think making it more explicit would be nice

Kludex · 2022-05-19T18:32:05Z

I'm going to merge this.

Some things left unanswered:

Deprecation policy, how do we proceed?
Should we check if someone is using parts of Starlette without the Starlette application itself? i.e. if "app" in scope:.

Kludex added 4 commits April 18, 2022 08:02

Add 400 response on MultiParser exceptions

efc4342

Merge branch 'master' into 400-on-missing-boundary

b7b4e92

Add 400 response on MultiParser exceptions [2]

d2e190a

Add tests and remove name from here

d52dd9a

Kludex changed the title ~~400 on missing boundary 2~~ Add 400 response when boundary is missing May 3, 2022

Kludex mentioned this pull request May 3, 2022

Add 400 response on MultiParser exceptions #1593

Closed

Kludex requested review from tomchristie and a team May 3, 2022 11:48

adriangb reviewed May 4, 2022

View reviewed changes

starlette/http_exception.py Outdated Show resolved Hide resolved

starlette/formparsers.py Outdated Show resolved Hide resolved

Kludex commented May 4, 2022

View reviewed changes

starlette/formparsers.py Outdated Show resolved Hide resolved

Kludex commented May 4, 2022

View reviewed changes

starlette/formparsers.py Outdated Show resolved Hide resolved

Kludex force-pushed the 400-on-missing-boundary-2 branch from f4a3ab0 to d52dd9a Compare May 5, 2022 05:43

Remove args from exception

f1eb5ba

Kludex force-pushed the 400-on-missing-boundary-2 branch from 9a3d6ad to f1eb5ba Compare May 5, 2022 05:52

Merge remote-tracking branch 'origin/master' into 400-on-missing-boun…

7a7a31e

…dary-2

Kludex commented May 5, 2022

View reviewed changes

starlette/requests.py Show resolved Hide resolved

Kludex added this to the Version 0.21.0 milestone May 5, 2022

Kludex added 3 commits May 6, 2022 06:51

Merge branch 'master' into 400-on-missing-boundary-2

9245408

Merge branch 'master' into 400-on-missing-boundary-2

90aa31b

Move ExceptionMiddleware to starlette.middleware.exceptions

598cfac

Kludex commented May 8, 2022

View reviewed changes

Merge branch 'master' into 400-on-missing-boundary-2

ed1ccb2

adriangb approved these changes May 12, 2022

View reviewed changes

add test for deprecation shim

57e16d6

tomchristie approved these changes May 18, 2022

View reviewed changes

Merge branch 'master' into 400-on-missing-boundary-2

234c908

This comment was marked as off-topic.

Sign in to view

Kludex merged commit 3453fd6 into master May 19, 2022

Kludex deleted the 400-on-missing-boundary-2 branch May 19, 2022 18:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add 400 response when `boundary` is missing #1617

Add 400 response when `boundary` is missing #1617

Kludex commented May 3, 2022 •

edited

Loading

Kludex May 8, 2022

tomchristie May 18, 2022

adriangb May 12, 2022

Kludex May 12, 2022

adriangb left a comment

Kludex commented May 12, 2022

tomchristie left a comment

tomchristie May 18, 2022

tomchristie May 18, 2022

tomchristie May 18, 2022

adriangb May 18, 2022

Kludex May 18, 2022

adriangb May 18, 2022

Kludex May 19, 2022 •

edited

Loading

tomchristie May 18, 2022

Kludex May 18, 2022

adriangb May 18, 2022

This comment was marked as off-topic.

Kludex commented May 19, 2022

	async def handle(self, scope: Scope, receive: Receive, send: Send) -> None:
	if self.methods and scope["method"] not in self.methods:
	headers = {"Allow": ", ".join(self.methods)}
	if "app" in scope:
	raise HTTPException(status_code=405, headers=headers)
	else:
	response = PlainTextResponse(
	"Method Not Allowed", status_code=405, headers=headers
	)
	await response(scope, receive, send)
	else:
	await self.app(scope, receive, send)

		from starlette.types import ASGIApp, Message, Receive, Scope, Send


		class ExceptionMiddleware:

Add 400 response when boundary is missing #1617

Add 400 response when boundary is missing #1617

Conversation

Kludex commented May 3, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adriangb left a comment

Choose a reason for hiding this comment

Kludex commented May 12, 2022

tomchristie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kludex May 19, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as off-topic.

Kludex commented May 19, 2022

Add 400 response when `boundary` is missing #1617

Add 400 response when `boundary` is missing #1617

Kludex commented May 3, 2022 •

edited

Loading

Kludex May 19, 2022 •

edited

Loading