Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check legacy headers when streaming headers #426

Merged

Conversation

gBillal
Copy link
Contributor

@gBillal gBillal commented Oct 26, 2024

If the option legacy_headers_as_lists is set to true and running the following code will return an error

Req.get!("https://github.com/wojtekmach/req/archive/refs/tags/v0.5.5.zip", into: File.stream!("test.tar.tz"))
** (BadMapError) expected a map, got: []
    (stdlib 5.2.1) :maps.merge([], %{"cache-control" => ["max-age=0, private"], "content-length" => ["0"], "content-security-policy" => ["default-src 'none'; base-uri 'self'; child-src github.com/assets-cdn/worker/ github.com/webpack/ github.com/assets/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com *.rel.tunnels.api.visualstudio.com wss://*.rel.tunnels.api.visualstudio.com objects-origin.githubusercontent.com copilot-proxy.githubusercontent.com proxy.individual.githubcopilot.com proxy.business.githubcopilot.com proxy.enterprise.githubcopilot.com *.actions.githubusercontent.com wss://*.actions.githubusercontent.com productionresultssa0.blob.core.windows.net/ productionresultssa1.blob.core.windows.net/ productionresultssa2.blob.core.windows.net/ productionresultssa3.blob.core.windows.net/ productionresultssa4.blob.core.windows.net/ productionresultssa5.blob.core.windows.net/ productionresultssa6.blob.core.windows.net/ productionresultssa7.blob.core.windows.net/ productionresultssa8.blob.core.windows.net/ productionresultssa9.blob.core.windows.net/ productionresultssa10.blob.core.windows.net/ productionresultssa11.blob.core.windows.net/ productionresultssa12.blob.core.windows.net/ productionresultssa13.blob.core.windows.net/ productionresultssa14.blob.core.windows.net/ productionresultssa15.blob.core.windows.net/ productionresultssa16.blob.core.windows.net/ productionresultssa17.blob.core.windows.net/ productionresultssa18.blob.core.windows.net/ productionresultssa19.blob.core.windows.net/ github-production-repository-image-32fea6.s3.amazonaws.com github-production-release-asset-2e65be.s3.amazonaws.com insights.github.com wss://alive.github.com api.githubcopilot.com api.individual.githubcopilot.com api.business.githubcopilot.com api.enterprise.githubcopilot.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com copilot-workspace.githubnext.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: blob: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com private-avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com secured-user-images.githubusercontent.com/ user-images.githubusercontent.com/ private-user-images.githubusercontent.com opengraph.githubassets.com github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com objects-origin.githubusercontent.com *.githubusercontent.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/ private-user-images.githubusercontent.com github-production-user-asset-6210df.s3.amazonaws.com gist.github.com; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; upgrade-insecure-requests; worker-src github.com/assets-cdn/worker/ github.com/webpack/ github.com/assets/ gist.github.com/assets-cdn/worker/"], "content-type" => ["text/html; charset=utf-8"], "date" => ["Sat, 26 Oct 2024 19:45:23 GMT"], "location" => ["https://codeload.github.com/wojtekmach/req/zip/refs/tags/v0.5.5"], "referrer-policy" => ["no-referrer-when-downgrade"], "server" => ["GitHub.com"], "strict-transport-security" => ["max-age=31536000; includeSubdomains; preload"], "vary" => ["X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, Accept-Encoding, Accept, X-Requested-With"], "x-content-type-options" => ["nosniff"], "x-frame-options" => ["deny"], "x-github-request-id" => ["D9C2:3264D4:92D3045:962FC6F:671D46D2"], "x-xss-protection" => ["0"]})
    (req 0.5.6) lib/req/finch.ex:140: anonymous fn/2 in Req.Finch.finch_stream_into_collectable/5
    (elixir 1.15.4) lib/map.ex:916: Map.update!/3
    (req 0.5.6) lib/req/finch.ex:140: anonymous fn/3 in Req.Finch.finch_stream_into_collectable/5
    (finch 0.19.0) lib/finch.ex:376: anonymous fn/3 in Finch.stream/5
    (finch 0.19.0) lib/finch/http1/conn.ex:324: Finch.HTTP1.Conn.receive_response/8
    (finch 0.19.0) lib/finch/http1/conn.ex:131: Finch.HTTP1.Conn.request/8
    (finch 0.19.0) lib/finch/http1/pool.ex:60: anonymous fn/10 in Finch.HTTP1.Pool.request/6

In this PR, we address the issue by calling Req.Response.put_header/3 to add the headers instead of converting them into a map.

@gBillal gBillal changed the title Check legacy headers when into body Check legacy headers when streaming headers Oct 26, 2024
@wojtekmach wojtekmach merged commit a3c48b8 into wojtekmach:main Nov 12, 2024
2 checks passed
@wojtekmach
Copy link
Owner

Thank you! Note, we plan to drop legacy headers support for Req v1.0 so I'd consider moving off those.

@gBillal gBillal deleted the check-legacy-headers-when-into-body branch November 12, 2024 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants