Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default DataGateway is set to localhost, causing file uploads to fail #931

Closed
SamuAlfageme opened this issue Jul 2, 2020 · 0 comments · Fixed by #936
Closed

Default DataGateway is set to localhost, causing file uploads to fail #931

SamuAlfageme opened this issue Jul 2, 2020 · 0 comments · Fixed by #936

Comments

@SamuAlfageme
Copy link
Contributor

In the context of sciencemesh/sciencemesh#105 and while testing the wopiserver deployment together with a ruuning Reva and its simplified config from #826 (standalone.toml), we hit an error when trying to upload a file to Reva from the wopiserver integration tests:

2020-07-01T15:23:53 wopiserver.test[35] ERROR    
  msg="Exception when uploading file to Reva" 
  reason="HTTPConnectionPool(host='localhost', port=19001): Max retries exceeded with url: /data (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd1b0677e80>: Failed to establish a new connection: [Errno 111] Connection refused'))"

By looking at the revad logs:

3:23PM DBG internal/grpc/interceptors/auth/auth.go:85 > skipping auth method=/cs3.gateway.v1beta1.GatewayAPI/InitiateFileUpload pid=1 pkg=rgrpc traceid=98f4c5d19ca44fc77a9292f213a1f3a3
3:23PM INF internal/grpc/interceptors/log/log.go:69 > unary code=OK end="01/Jul/2020:15:23:53 +0000" from=tcp://127.0.0.1:34036 pid=1 pkg=rgrpc start="01/Jul/2020:15:23:53 +0000" time_ns=102801 traceid=98f4c5d19ca44fc77a9292f213a1f3a3 uri=/cs3.storage.registry.v1beta1.RegistryAPI/GetStorageProvider user-agent=grpc-go/1.26.0
3:23PM DBG pkg/storage/utils/localfs/upload.go:124 > localfs: NewUpload info={"ID":"","IsFinal":false,"IsPartial":false,"MetaData":{"dir":"/","filename":"testxattr.txt"},"Offset":0,"PartialUploads":null,"Size":0,"SizeIsDeferred":false,"Storage":null} pid=1 pkg=rgrpc traceid=98f4c5d19ca44fc77a9292f213a1f3a3
3:23PM DBG pkg/storage/utils/localfs/upload.go:140 > localfs: resolved filename info={"ID":"","IsFinal":false,"IsPartial":false,"MetaData":{"dir":"/","filename":"testxattr.txt"},"Offset":0,"PartialUploads":null,"Size":0,"SizeIsDeferred":false,"Storage":null} pid=1 pkg=rgrpc traceid=98f4c5d19ca44fc77a9292f213a1f3a3
3:23PM INF internal/grpc/services/storageprovider/storageprovider.go:302 > file upload data-server=http://localhost:19001/data/654f6dc4-375c-4e94-ae95-6cb5e6bdd9c6 fn=/testxattr.txt pid=1 pkg=rgrpc traceid=98f4c5d19ca44fc77a9292f213a1f3a3 xs="map[md5:100 unset:1000]"
3:23PM INF internal/grpc/interceptors/log/log.go:69 > unary code=OK end="01/Jul/2020:15:23:53 +0000" from=tcp://127.0.0.1:34038 pid=1 pkg=rgrpc start="01/Jul/2020:15:23:53 +0000" time_ns=1554022 traceid=98f4c5d19ca44fc77a9292f213a1f3a3 uri=/cs3.storage.provider.v1beta1.ProviderAPI/InitiateFileUpload user-agent=grpc-go/1.26.0
3:23PM INF internal/grpc/interceptors/log/log.go:69 > unary code=OK end="01/Jul/2020:15:23:53 +0000" from=tcp://10.100.69.158:33074 pid=1 pkg=rgrpc start="01/Jul/2020:15:23:53 +0000" time_ns=3607498 traceid=98f4c5d19ca44fc77a9292f213a1f3a3 uri=/cs3.gateway.v1beta1.GatewayAPI/InitiateFileUpload user-agent="grpc-python/1.30.0 grpc-c/10.0.0 (manylinux; chttp2)"

... we realized the Datagateway is using localhost as fallback when no url is provided in

// this is the default address we use for starting HTTP services
if sharedConf.DataGateway == "" {
sharedConf.DataGateway = "http://localhost:19001/data"
}

For some better and more flexible defaults, we've discussed the option to include a 2nd fallback: the hostname, which e.g. in kubernetes will resolve based on https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-hostname-and-subdomain-fields - it won't be as "correct" as the (kubernetes) service name, but will point at the right instance. The preference could be:

  1. Preferred the provided value from the config file, if any.
  2. Revad's hostname, if set.
  3. Fallback to localhost.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant