Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multipart Form Data Headers #2537

Closed
ghost opened this issue Jan 3, 2023 · 1 comment
Closed

Multipart Form Data Headers #2537

ghost opened this issue Jan 3, 2023 · 1 comment

Comments

@ghost
Copy link

ghost commented Jan 3, 2023

After testing #2382 it seems that the request headers are not being updated correctly.

filename = 'test.tar'
data = {"file": (Path(filename).name, open(filename, "rb"), "application/x-tar")}
resp = httpx.request("POST", 'URL', files=data)

The request has the following header {'Content-Type': 'multipart/form-data; boundary=b194f2c9bc744ebf40c3fa03d6d53987'}

However if using an initialized client:

client = httpx.Client(headers={"Content-Type": "application/json"})
filename = 'test.tar'
data = {"file": (Path(filename).name, open(filename, "rb"), "application/x-tar")}
resp = client.post('URL', files=data)

The request is still using {'Content-Type': 'application/json'}
Error message from our API: '{"code":"API_MALFORMED_BODY","message":"Malformed JSON"}'

Shouldn't the POST change the headers if using multipart forms?

Specifying the following also does not work because the boundaries are not included in the headers:
resp = client.post('URL', files=data, headers={'Content-Type': 'multipart/form-data'})

@tomchristie
Copy link
Member

Okay, it's a little bit ambiguous what behavior you'd expect in those cases.
But, yes, I can see why you might like the client-default Content-Type to be overridden there.

I don't think this is related to #2382, it looks like the same behaviour that we've had for a while...

import httpx

# Client with Content-Type headers set.
client = httpx.Client(headers={"Content-Type": "application/json"})

# An `application/x-www-form-urlencoded` POST request.
# Essentially the same case as your multipart example, but just a little more simple.
request = client.build_request("POST", "https://www.example.com", data={"a": "b"})

print(request.headers)
print(request.content)

Across a few different version of httpx...

# 0.20.0
Headers({'host': 'www.example.com', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'user-agent': 'python-httpx/0.20.0', 'content-type': 'application/json', 'content-length': '3'})
b'a=b'

# 0.21.0
Headers({'host': 'www.example.com', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'user-agent': 'python-httpx/0.21.0', 'content-type': 'application/json', 'content-length': '3'})
b'a=b'

# 0.22.0
Headers({'host': 'www.example.com', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'user-agent': 'python-httpx/0.22.0', 'content-type': 'application/json', 'content-length': '3'})

# latest
Headers({'host': 'www.example.com', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'user-agent': 'python-httpx/0.23.2', 'content-type': 'application/json', 'content-length': '3'})
b'a=b'

All the cases have Content-Type: application/json set, but have a form-encoded body.

The quickest solution to the issue would be to stop doing this...

# Not needed. You'll get this header if you use `json=...` anyways.
# Specifying a `Content-Type` header on the client instance is almost always going to
# be a bad idea. Eg. you'll also get odd looking GET requests which have a `Content-Type`
# header but no content.
client = httpx.Client(headers={"Content-Type": "application/json"})

But, yeah, there's other options too. I can see that we might be able to change the priority rules so that client headers only get applied if they're not overriding the auto-generated request headers.

Or... we could guard against setting certain headers on the client. In particular we could prevent setting the Content-Type, Content-Length, Host and Transfer-Encoding headers, which are all auto-populated based on the request.

@encode encode locked and limited conversation to collaborators Jan 5, 2023
@tomchristie tomchristie converted this issue into discussion #2545 Jan 5, 2023

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Projects
None yet
Development

No branches or pull requests

1 participant