This repository has been archived by the owner on Feb 12, 2024. It is now read-only.
Gateway Improvements: Streaming, Conditional and Range Requests #1989
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ghost
assigned
lidel
added a commit
to ipfs/ipfs-companion
that referenced
this pull request
Apr 12, 2019
This applies cherry-picked patches from: ipfs/js-ipfs#1989 ipfs/js-ipfs#1950 and solves stream issues on page refresh. Content-type sniffing is now done over a meaningful amount of bytes instead of arbitrary number.
4 tasks
lidel
force-pushed
the
fix/streaming-compressed-payload
branch
from
c0838d8
to
373f69e
Compare
|
@alanshaw rebased this PR against latest master and CI no longer fails due to libp2p |
52 tasks
lidel
changed the title
|
As this PR was not merged and I had more improvements built on top of it, I've just pushed those additional improvements here and updated description to reflect what changed. Note: We should review & merge this before we add support for Question: If I wanted to add interop (go-vs-js) tests for relevant HTTP headers, which repo should I use? |
lidel
added a commit
to ipfs/ipfs-companion
that referenced
this pull request
Apr 26, 2019
Switching to js-ipfs from ipfs/js-ipfs#1989 for next beta release
8 tasks
hugomrdias
approved these changes
Apr 29, 2019
This change simplifies code responsible for streaming response and makes the streaming actually work by telling the payload compression stream to flush its content on every read(). (previous version was buffering entire thing in Hapi's compressor memory) We also do content-type detection based on the beginning of the stream by peeking at first `fileType.minimumBytes` bytes. License: MIT Signed-off-by: Marcin Rataj <lidel@lidel.org>
- Switched from deprecated `hapi` and `joi` to `@hapi/hapi` and `@hapi/joi`
- Added support for Conditional Requests (RFC7232)
- Returning `304 Not Modified` if `If-None-Match` is a CID matching `Etag`
- Added `Last-Modified` to `/ipfs/` responses (improves client-side caching)
- Always returning `304 Not Modified`
when `If-Modified-Since` is present for immutable `/ipfs/`
- Added support for Byte Range requests (RFC7233, Section-2.1)
- Added support for `?filename=` parameter (improves downloads of raw cids)
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
lidel
force-pushed
the
fix/streaming-compressed-payload
branch
from
17712a4
to
373eedc
Compare
6 tasks
License: MIT Signed-off-by: Alan Shaw <alan.shaw@protocol.ai>
alanshaw
approved these changes
May 8, 2019
lidel
added a commit
to ipfs/ipfs-companion
that referenced
this pull request
May 8, 2019
tl;dr opening /ipns/tr.wikipedia-on-ipfs.org/wiki/Mars.html works Switched to js-ipfs with cherry-picked changes from: ipfs/js-ipfs#2020 ipfs/js-ipfs#1989 ipfs/js-ipfs#2002 ipfs/js-ipfs-http-response#22 ipfs-inactive/js-ipfs-mfs#48
alanshaw
pushed a commit
that referenced
this pull request
Jul 4, 2019
> Part of an effort to run embedded js-ipfs in Brave 🦁 ipfs/ipfs-companion#716 > Fixes #1918 This PR will add support for `/ipns/` paths at HTTP Gateway. Smoke test: [/ipns/tr.wikipedia-on-ipfs.org](http://127.0.0.1:9090/ipns/tr.wikipedia-on-ipfs.org/wiki/Anasayfa.html) (IPNS+DNSLink+HAMT-sharded website) This PR depends on the following merged PRs: - Gateway Improvements from #1989 (after merging #1989 I will rebase this PR, which will remove first two commits) - PeerID eg. `/ipns/<PeerId-as-multihash-b58>` - requires #2002 to land first - `/ipns/<libp2p-key-in-cidv1>` - requires multiformats/js-multicodec#45 - DNSLink eg. `/ipns/<fqdn>/path/file` like `/ipns/docs.ipfs.io/assets/logo.svg` - requires #2002 to land first - HAMT shard support eg. `/ipns/tr.wikipedia-on-ipfs.org/wiki/Anasayfa.html` (`wiki` is a sharded directory) - requires ipfs/js-ipfs-http-response#22 and ipfs-inactive/js-ipfs-mfs#48 to land first - Tests for `/ipns/` License: MIT Signed-off-by: Marcin Rataj <lidel@lidel.org>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR:
read()Previous version was buffering entire thing in Hapi's compressor memory.
See Provide a way to flush compression stream on demand hapijs/hapi#3599 and hapijs/hapi@c49b9c8 for more details.
fileType.minimumBytesbytes and setscontent-typeheader.Added 2019-04-29:
hapiandjoito@hapi/hapiand@hapi/joi304 Not ModifiedifIf-None-Matchis a CID matchingEtagLast-Modifiedto/ipfs/responses (improves client-side caching)304 Not ModifiedIf-Modified-Sincefor immutable/ipfs/