sharness - t0118 is broken - response for application/vnd.ipld.car SHOULD NOT include Content-Length

[laurentsenta]

Checklist

• This is a bug report, not a question. Ask questions on discuss.ipfs.io.
• I have searched on the issue tracker for my bug.
• I am running the latest kubo version or have an issue updating.

Installation method

ipfs-update or dist.ipfs.tech

Version

Kubo version: 0.18.1
Repo version: 13
System version: arm64/darwin
Golang version: go1.19.1

Config

No response

Description

In sharness t0118 it seems that the test is misusing the invert option in grep:

kubo/test/sharness/t0118-gateway-car.sh

Lines 93 to 96 in ed4d6b7

	# CAR is streamed, gateway may not have the entire thing, unable to calculate total size
	test_expect_success "GET response for application/vnd.ipld.car includes no Content-Length" '
	grep -qv "< Content-Length:" curl_output
	'

The comment says that we should NOT return a Content-Length, but we do:

› cat curl_output | grep Content-Length                                                                                                                        ☹
< Access-Control-Expose-Headers: Content-Length
< Content-Length: 127

(tested with Kubo and a car gateway from go-libipfs)

Changing the test to : ! grep "< Content-Length:" curl_output reveals the error.

I found a few other cases where we use grep -v and that might have a similar issue:

> ack 'grep.*-v' ./ - | pbcopy
t0119-prometheus.sh
t0250-files-api.sh
t0140-swarm.sh
t0060-daemon.sh
t0080-repo.sh
t0230-channel-streaming-http-content-type.sh
t0141-addfilter.sh
t0046-id-hash.sh

[lidel]

Thanks for catching this 👍
As we've discussed during 1:1

• we should not have this header because we don't know how long CAR stream will be
• there is some dance given gateway code for this lives now in https://github.com/ipfs/go-libipfs/blob/main/gateway/handler_car.go, but as long you open a PR in Kubo that passes sharness it will be ok

ps. I wonder if the same thing happens for TAR response?

[BigLep]

This will get handled by ipfs/boxo#177 which is blocked.

[laurentsenta]

Discussion also happening in ipfs/gateway-conformance#64

[lidel]

I believe this got resolved and can be closed:

• test/sharness/t0118-gateway-car.sh was removed by feat(gateway): support for IPIP-402 CAR params #9914
• gateway-conformance no longer makes any assertions about this header (fix: remove the requirement on Content-Length on the gateway gateway-conformance#64 (review))
• gateway specs say nothing about Content-Length being incorrect. It is ok, as long it matches the response body size.
  • Content-Length can be returned when gateway knows the total size, and in golang (boxo/kubo) it may happen when a small DAG is flushed to the response writer