Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Path address not found error after uploading manifest #2368

Closed
agazso opened this issue Jul 27, 2021 · 7 comments · Fixed by #2380
Closed

Path address not found error after uploading manifest #2368

agazso opened this issue Jul 27, 2021 · 7 comments · Fixed by #2380
Labels
bug Something isn't working issue

Comments

@agazso
Copy link
Member

agazso commented Jul 27, 2021

Summary

I uploaded a React web application as a tar file and when trying to access the files in it some of them are missing. Most of them can be found, only a few one is missing.

The reported error is {"message":"path address not found","code":404} indicating that the files are somehow missing from the manifest itself.

Steps to reproduce

I uploaded both with curl and swarm-cli and it's the same behavior with both, and in both cases the same files are missing.

As a verification I set the loglevel to TRACE to see what happens when uploading:

TRAC[2021-07-27T16:25:48+02:00] uploaded dir file media/ETH.svg with reference 029e659ce4b73ae7fdae79e1a7610c9fa903bde4ae5c10aae130820e7e2daae4 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file media/DAI.svg with reference b711a7d7996d4b803d13bd6d3fba018acb3991dd51be861dec12c9f356e99b77 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file media/BZZ.svg with reference fa64128335f404da941b671d7474d8be3699b37d8bc9334ec90a8e323ffbe0d3 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/Sun.785a5cb6.svg with reference be7f4ccfd47b02ac9b28f838b810e28d018d579302fbbf0ed75afe11062aa975 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/refresh.0fd9bb66.svg with reference 816e8ac30947ea3082e51f3334925af3c53d4d2b3f6fc6b8d88a3e5b57610110 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/question.e7d5b537.svg with reference cbe12b2651aa5bdeee66abc7de4adaa1685dd8ac56747ebddb5de7e7a34a8598 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/chevron.c5b4b58c.svg with reference 6ea32322d683725c6ff29a0a03d09479954ca63eccdcb51e9ddf199dcbdd043c 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/info-icon-white.d5e01bb3.svg with reference f0bf27f49ab755dd43789c9e37815636d073e5f77d5e9b3d5065fa87cbed06a5 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/spinner2.bc3f17b3.svg with reference 1fdf98c90cb54539a908593b48a071678502fe815e55bb60f4804ad0f27dfac9 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/network-dot-blue.ba771482.svg with reference c58e47ba3e9c4e99595cdb1f7a405a0334e6c4769d7a899ddf7cf6f470fe5791 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/close.20bb495b.svg with reference 0277cae6d8de9e1b06942aca8a0d868c637cfdaaff28a528b84e687003db89f8 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/dot.d5063790.svg with reference cae2a0d46dff2890fbf09250d56d2183067bc4267345ae269c5927dd10787164 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/network-dot-red.2736d2aa.svg with reference 158e739570050ddc8bd9ebac62c9ea7ac9b1ac6b1ad071649f4fd7e5e283bdb5 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/check.f40e7d83.svg with reference 1841a060f712a8439308d2fe4fc4d758e41f0d3f66c47c21e5a78dfc88c36ac8 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/network-dot-grey.01631b5f.svg with reference 775824a572aa18791c72d01d1f461ddfbf3f676c0af0f8c6519e17483e046a86 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/switcher.420b6624.svg with reference 9f9cf593c7184a309ada53c4c79ce977c8578c1b58dcc5bb4fd1323d30382fc5 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/network-dot-purple.d2d41de8.svg with reference 2044ef71613acd04cccee63dc615ca4ab2a228169485d1be629e5fc1885b2f98 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/external_link.8d6cd8df.svg with reference 88804e73a45e40a74ac768fc2bf76b1e44c3cfe8456389e6538473ddb928e042 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/exit.17a94979.svg with reference fbe4c80125de03941f77d9deae0d120a46d005f90c2033fb2d769708347bc6d8 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/network-dot-orange.5976f7c1.svg with reference 9c6168a747109ebac008434e8a3a814f08b39d7c1c90459d89c03b6e7a1c49aa 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/info.833d5e9c.svg with reference 8e20cdd8b92c5f9ed7e4556aa3ae965f76e74953125d1be036b3dcab49ea327d 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/network-dot-green.632762b6.svg with reference 510af0d2d2a4414e889d9c453a2b26d5e1974e08f98beff17978d34571f77406 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/info.b534a5d2.svg with reference 3b41285f0db7d3207cf3c834aaaf02c793d534465424010a2b0d24575339c074 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/chevron-down-solid.a4f409dd.svg with reference 114d8ef4287c30b027377e6e53183a37f8ee7a1f3fc56494cc458e6a644adce4 
TRAC[2021-07-27T16:25:48+02:00] uploaded dir file static/media/Moon.9ee68bc0.svg with reference 233860a0314b009c3930444d1aef902a045c0044108fc50132ea2479d744ed7f 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file static/js/main.03546a09.chunk.js with reference b7c356063f06b095b027ea13320cd9c47b540616c48d82d24913abd450efc00d 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file static/js/runtime-main.a038a3a8.js with reference ff6177fc17991578c89aeed9f2ea9d01fee3381968af4753f8e59b6942f2fcb4 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file static/js/2.67d3302a.chunk.js.LICENSE.txt with reference c74347723c06c937ac70d819628c23a479b0c104d30f839f82b19c7dad40a2c2 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file static/js/2.67d3302a.chunk.js with reference 3e6b9ed90c8ef61b87fdcb5b0ee7b3d778013d31ec31e12e57e13bbefb7b1f81 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file static/css/main.a0769d5e.chunk.css with reference ac26631d5f55275ea989f58a346b7aed6cb2b574daae86443f58b39c274310a4 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file static/css/main.a0769d5e.chunk.css.map with reference 25bca13bcda8788ed1628bb6186e720e2447987c2800055831323c1691187f68 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file index.html with reference 9b10c2551bd48fede443da0a9fea90cbeffbf110b3be45f13a7d09e9f21679c4 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file asset-manifest.json with reference 7e76a174900ad1bd41e06a85c9527875d047c64f549240c0b989ed13c9dddfb5 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file BZZ.svg with reference fa64128335f404da941b671d7474d8be3699b37d8bc9334ec90a8e323ffbe0d3 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file style.css with reference b34ca8c22b9e982354f9c7f50b470d66db428d880c8a904d5fe4ec9713171526 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file manifest.json with reference 6575fc654eb66cc8adf095be1c0eb4bf2ab101da42eae4d3d6ae79add1021bff 
TRAC[2021-07-27T16:25:49+02:00] uploaded dir file robots.txt with reference b34ca8c22b9e982354f9c7f50b470d66db428d880c8a904d5fe4ec9713171526 
TRAC[2021-07-27T16:25:49+02:00] finished uploaded dir with reference 495b366049ef1473155f26b9646c00653ad75650b5316251e3c528baceacb77e 
INFO[2021-07-27T16:25:49+02:00] api access                                    duration=0.548374602 ip=127.0.0.1 method=POST proto=HTTP/1.1 size=82 status=201 uri=/bzz user-agent=axios/0.21.1

Then when downloading a missing file:

TRAC[2021-07-27T16:27:49+02:00] name resolve: valid bzz address "495b366049ef1473155f26b9646c00653ad75650b5316251e3c528baceacb77e" 
DEBU[2021-07-27T16:27:49+02:00] bzz download: invalid path 495b366049ef1473155f26b9646c00653ad75650b5316251e3c528baceacb77e/static/css/main.a0769d5e.chunk.css: manifest: not found 
ERRO[2021-07-27T16:27:49+02:00] bzz download: invalid path                   
INFO[2021-07-27T16:27:49+02:00] api access                                    duration=0.000534876 ip=127.0.0.1 method=GET proto=HTTP/1.1 size=49 status=404 uri=/bzz/495b366049ef1473155f26b9646c00653ad75650b5316251e3c528baceacb77e/static/css/main.a0769d5e.chunk.css user-agent=curl/7.78.0

I thought that maybe it has problems when it has several dots (.) in the name, but no:

TRAC[2021-07-27T16:35:11+02:00] name resolve: valid bzz address "495b366049ef1473155f26b9646c00653ad75650b5316251e3c528baceacb77e" 
INFO[2021-07-27T16:35:11+02:00] api access                                    duration=0.000817583 ip=127.0.0.1 method=GET proto=HTTP/1.1 size=6215 status=200 uri=/bzz/495b366049ef1473155f26b9646c00653ad75650b5316251e3c528baceacb77e/static/css/main.a0769d5e.chunk.css.map user-agent=curl/7.78.0

I run the latest master 596768e
in standalone mode.
@agazso agazso added the bug Something isn't working label Jul 27, 2021
@bee-runner bee-runner bot added the issue label Jul 27, 2021
@acud
Copy link
Member

acud commented Jul 28, 2021

@agazso the latest master doesn't have standalone mode anymore. Also, can you please post the exact artifacts you've uploaded (and how), and the exact cli configuration of the bee node you were using?

@agazso
Copy link
Member Author

agazso commented Jul 28, 2021

There is an example.tar file in this zip file (because github does not let uploading tar files).
example.zip

It was uploaded with this command:

curl -X POST http://localhost:1633/bzz -H "Accept: application/json, text/plain, */*" -H "content-type: application/x-tar" -H "swarm-collection: true" -H "swarm-postage-batch-id: 6331e0d72152d99da2d1156a7c227c63f2729d24b753667156344a7860ecff78" -H "swarm-index-document: index.html" --data-binary @example.tar

The config used:

api-addr: :1633
block-hash: ""
block-time: "15"
bootnode: ""
bootnode-mode: false
cache-capacity: "1000000"
cache-retrieval: true
clef-signer-enable: false
clef-signer-endpoint: ""
clef-signer-ethereum-address: ""
config: config.yaml
cors-allowed-origins: []
data-dir: data/1
db-block-cache-capacity: "33554432"
db-disable-seeks-compaction: false
db-open-files-limit: "200"
db-write-buffer-size: "33554432"
debug-api-addr: 127.0.0.1:1635
debug-api-enable: true
full-node: true
gateway-mode: false
global-pinning-enable: false
help: false
mainnet: false
nat-addr: ""
network-id: 4020
p2p-addr: :1634
p2p-quic-enable: false
p2p-ws-enable: false
password: password
password-file: ""
payment-early: "10000000"
payment-threshold: "100000000"
payment-tolerance: "100000000"
postage-stamp-address: 0xCfEB869F69431e42cdB54A4F4f105C19C080A601
price-oracle-address: 0x254dffcd3277C0b1660F6d42EFbB754edaBAbC2B
resolver-options: []
standalone: true
swap-deployment-gas-price: ""
swap-enable: true
swap-endpoint: http://localhost:9545
swap-factory-address: 0x5b1869D9A4C187F2EAa108f3062412ecf0526b24
swap-initial-deposit: 0
swap-legacy-factory-addresses: []
tracing-enable: false
tracing-endpoint: 127.0.0.1:6831
tracing-host: ""
tracing-port: ""
tracing-service-name: bee
transaction: ""
verbosity: 5
warmup-time: 0
welcome-message: ""

@nugaon
Copy link
Member

nugaon commented Jul 29, 2021

I thought that maybe it has problems when it has several dots (.) in the name, but no:

as I tried out it seems this problem occurs when a filename is fully part of an other filename like vmi.js and vmi.js.map. In this case, the former won't be loaded.
I deserialized the constructed manifest of your data and I think the problem is the Mantaray node's type is wrong on upload. For example 2.67d3302a.chunk.js has type 4 instead of 6 (Value[2] + Edge[4]).

@aloknerurkar
Copy link
Contributor

Yes. There is an issue with marshaling and unmarshaling. I am looking into it. Node type should also have PathSeparator.

@nugaon
Copy link
Member

nugaon commented Jul 29, 2021

Yes. There is an issue with marshaling and unmarshaling. I am looking into it. Node type should also have PathSeparator.

Why PathSeparator is needed in case of 2.67d3302a.chunk.js?
I deserailized btw with mantaray-js and you can download object tree of the unmarshaled data from here

@aloknerurkar aloknerurkar linked a pull request Jul 29, 2021 that will close this issue
fix(mantaray): mantaray lookup issue #2380
Merged
@aloknerurkar
Copy link
Contributor

aloknerurkar commented Jul 29, 2021 •
edited
Loading

@agazso

The above PR fixes the lookup issues. All the files should be resolvable now. However when I deploy the website I see:

INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.00060049 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d6
280bb3b9a3c206c/" size=60 status=404 uri=/static/css/main.a0769d5e.chunk.css user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"                   
INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.000594704 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d
6280bb3b9a3c206c/" size=60 status=404 uri=/static/js/main.03546a09.chunk.js user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"                    
INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.000814159 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d
6280bb3b9a3c206c/" size=60 status=404 uri=/static/js/2.67d3302a.chunk.js user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"                       
INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.000537672 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d
6280bb3b9a3c206c/" size=60 status=404 uri=/static/js/2.67d3302a.chunk.js user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"                       
INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.000955943 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d
6280bb3b9a3c206c/" size=60 status=404 uri=/static/js/main.03546a09.chunk.js user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.001265084 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d
6280bb3b9a3c206c/" size=0 status=304 uri=/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d6280bb3b9a3c206c/BZZ.svg user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.
4472.124 Safari/537.36"

The BZZ.svg and index.html are resolved correctly.

In the case of the scripts I see the URI is not prefixed with /bzz/{hash}. I think this could be an HTML issue. If I use the full path of these files I can see them on the page.

INFO[2021-07-30T00:06:50+05:30] api access                                    duration=0.001461834 ip=192.168.0.114 method=GET proto=HTTP/1.1 size=0 status=304 uri=/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d6280bb3b9a3c2
06c/static/css/main.a0769d5e.chunk.css user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"

@agazso
Copy link
Member Author

agazso commented Jul 29, 2021

@agazso

The above PR fixes the lookup issues. All the files should be resolvable now. However when I deploy the website I see:

INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.00060049 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d6
280bb3b9a3c206c/" size=60 status=404 uri=/static/css/main.a0769d5e.chunk.css user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"                   
INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.000594704 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d
6280bb3b9a3c206c/" size=60 status=404 uri=/static/js/main.03546a09.chunk.js user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"                    
INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.000814159 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d
6280bb3b9a3c206c/" size=60 status=404 uri=/static/js/2.67d3302a.chunk.js user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"                       
INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.000537672 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d
6280bb3b9a3c206c/" size=60 status=404 uri=/static/js/2.67d3302a.chunk.js user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"                       
INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.000955943 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d
6280bb3b9a3c206c/" size=60 status=404 uri=/static/js/main.03546a09.chunk.js user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
INFO[2021-07-30T00:10:35+05:30] api access                                    duration=0.001265084 ip=192.168.0.114 method=GET proto=HTTP/1.1 referrer="http://192.168.0.167:1633/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d
6280bb3b9a3c206c/" size=0 status=304 uri=/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d6280bb3b9a3c206c/BZZ.svg user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.
4472.124 Safari/537.36"

The BZZ.svg and index.html are resolved correctly.

In the case of the scripts I see the URI is not prefixed with /bzz/{hash}. I think this could be an HTML issue. If I use the full path of these files I can see them on the page.

INFO[2021-07-30T00:06:50+05:30] api access                                    duration=0.001461834 ip=192.168.0.114 method=GET proto=HTTP/1.1 size=0 status=304 uri=/bzz/74a58d985365f93795595ad99c0077b22eab0014bc71939d6280bb3b9a3c2
06c/static/css/main.a0769d5e.chunk.css user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"

Yeah, that's a different, unrelated problem that the frontend code expects to be deployed at /. That will be taken care of with https://github.com/ethersphere/bzz-proxy-poc ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(mantaray): mantaray lookup issue ethersphere/bee
4 participants
@agazso @aloknerurkar @acud @nugaon