Skip to content
This repository has been archived by the owner on Jan 18, 2021. It is now read-only.

[EOS] PROPFIND with <d:prop><d:getetag/></d:prop> does not show " for folder etags #269

Closed
individual-it opened this issue Jun 15, 2020 · 13 comments
Closed

[EOS] PROPFIND with <d:prop><d:getetag/></d:prop> does not show " for folder etags #269

individual-it opened this issue Jun 15, 2020 · 13 comments
Labels
bug Something isn't working eos p2-urgent

Comments

@individual-it
Copy link
Member

  1. setup OCIS with EOS as described in https://github.com/owncloud-docker/compose-playground/tree/master/examples/eos-compose-acceptance-tests
  2. load test users ldapadd -x -D "cn=admin,dc=owncloud,dc=com" -w admin -H ldap://localhost -f ./config/example-ldap-users-groups.ldif
  3. upload a file curl -k -u user1:1234 -X PUT https://localhost:9200/remote.php/dav/files/user1/file.txt -d "123" -v
  4. create a folder curl -k -u user1:1234 -X MKCOL https://localhost:9200/remote.php/dav/files/user1/upload/ -v
  5. get etags with PROPFIND curl -k -u user1:1234 -X PROPFIND https://localhost:9200/remote.php/dav/files/user1 -d '<?xml version="1.0"?><d:propfind xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns" xmlns:ocs="http://open-collaboration-services.org/ns"> <d:prop><d:getetag/></d:prop></d:propfind>' | xmllint --format -

the etags of the folders don't have " around them

<?xml version="1.0" encoding="utf-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns">
  <d:response>
    <d:href>/remote.php/dav/files/user1/</d:href>
    <d:propstat>
      <d:prop>
        <d:getetag>55:1592224385.512</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/files/user1/file.txt</d:href>
    <d:propstat>
      <d:prop>
        <d:getetag>"2415919104:012d0097"</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/files/user1/upload/</d:href>
    <d:propstat>
      <d:prop>
        <d:getetag>59:1592224285.381</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/files/user1/upload1/</d:href>
    <d:propstat>
      <d:prop>
        <d:getetag>5a:1592224385.512</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
</d:multistatus>

But then doing a simple PROPFIND, they do:
curl -k -u user1:1234 -X PROPFIND https://localhost:9200/remote.php/dav/files/user1 | xmllint --format -

<?xml version="1.0" encoding="utf-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns">
  <d:response>
    <d:href>/remote.php/dav/files/user1/</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU4Ojg1</oc:id>
        <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU4Ojg1</oc:fileid>
        <d:getetag>"55:1592224385.512"</d:getetag>
        <oc:permissions>WCKDNVR</oc:permissions>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:getcontenttype>httpd/unix-directory</d:getcontenttype>
        <oc:size>3</oc:size>
        <d:getlastmodified>Mon, 15 Jun 2020 12:33:05 UTC</d:getlastmodified>
        <oc:favorite>0</oc:favorite>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/files/user1/file.txt</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU4OjI0MTU5MTkxMDQ=</oc:id>
        <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU4OjI0MTU5MTkxMDQ=</oc:fileid>
        <d:getetag>"2415919104:012d0097"</d:getetag>
        <oc:permissions>WCKDNVR</oc:permissions>
        <d:getcontentlength>3</d:getcontentlength>
        <d:getcontenttype>application/octet-stream</d:getcontenttype>
        <d:getlastmodified>Mon, 15 Jun 2020 12:31:15 UTC</d:getlastmodified>
        <oc:favorite>0</oc:favorite>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/files/user1/upload/</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU4Ojg5</oc:id>
        <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU4Ojg5</oc:fileid>
        <d:getetag>"59:1592224285.381"</d:getetag>
        <oc:permissions>WCKDNVR</oc:permissions>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:getcontenttype>httpd/unix-directory</d:getcontenttype>
        <oc:size>0</oc:size>
        <d:getlastmodified>Mon, 15 Jun 2020 12:31:25 UTC</d:getlastmodified>
        <oc:favorite>0</oc:favorite>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/files/user1/upload1/</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU4Ojkw</oc:id>
        <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU4Ojkw</oc:fileid>
        <d:getetag>"5a:1592224385.512"</d:getetag>
        <oc:permissions>WCKDNVR</oc:permissions>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:getcontenttype>httpd/unix-directory</d:getcontenttype>
        <oc:size>0</oc:size>
        <d:getlastmodified>Mon, 15 Jun 2020 12:33:05 UTC</d:getlastmodified>
        <oc:favorite>0</oc:favorite>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
</d:multistatus>
@individual-it individual-it added bug Something isn't working eos labels Jun 15, 2020
@PVince81
Copy link
Contributor

So there are two scenarios:

  1. when querying in "allprops" mode, which is when no payload is sent with PROPFIND
  2. when querying the etag property directly with PROPFIND

For 1, the code path seems to always add double quotes: https://github.com/cs3org/reva/blob/master/internal/http/services/owncloud/ocdav/propfind.go#L297
For 2, no double quotes are added: https://github.com/cs3org/reva/blob/master/internal/http/services/owncloud/ocdav/propfind.go#L449

Still, just reading the code and seeing the results it seems that maybe there could be scenarios where the storage already stores the etag with double quotes, so code path 1 would duplicate the double quotes. But with EOS it seems the etag has no double quotes on the storage so needs them to be added in the OCDAV layer.

Needs further research and alignment between the storage implementations.

An alternative could be to have the OCDAV layer detect if a double quote is missing and add it automatically, but then there is no guarantee of consistency on the storage layer. Ideal would be to fix the storage layer.

@PVince81
Copy link
Contributor

also to note: @individual-it mentioned that this is blocking many tests, so would be good to solve this with higher priority to avoid having to create workarounds in the tests

@individual-it individual-it mentioned this issue Jun 18, 2020
Merged
11 tasks
@butonic
Copy link
Member

butonic commented Jun 18, 2020

eos does not enclose the etage in " ... that is something that the eos storage driver should do. cc @labkode

@labkode
Copy link
Member

labkode commented Jun 22, 2020

@butonic that is an issue, cs3org/reva#865

@labkode
Copy link
Member

labkode commented Jun 22, 2020

@ishank011 has fixed the problem. @butonic @PVince81 please check that no double encoding is done on upper layers (ocdav, ...).

@ishank011
Copy link
Contributor

Missed one place where double encoding could happen. Created another PR for that cs3org/reva#867

@PVince81
Copy link
Contributor

thanks. @individual-it can you recheck with latest reva ?

@individual-it individual-it mentioned this issue Jun 24, 2020
Closed
@individual-it
Copy link
Member Author

I tried to retest it with my setup in https://github.com/owncloud-docker/compose-playground/tree/master/examples/eos-compose-acceptance-tests
with additionally ether update the ocis repo to use the newest ocis-reva or using the reva services from the ocis-reva binary to get the newest patches.
I both cases I cannot even upload a file to EOS:

ocis          | {"level":"info","service":"reva","pkg":"rgrpc","traceid":"767afdda49f2f11343b587b573511e43","time":"2020-06-25T05:07:58Z","message":"user id:<idp:\"https://localhost:9200\" opaque_id:\"user0\" > username:\"user0\" mail:\"user0@example.org\"  authenticated"}
ocis          | {"level":"info","service":"reva","pkg":"rgrpc","traceid":"767afdda49f2f11343b587b573511e43","user-agent":"grpc-go/1.30.0","from":"tcp://127.0.0.1:59656","uri":"/cs3.auth.provider.v1beta1.ProviderAPI/Authenticate","start":"25/Jun/2020:05:07:58 +0000","end":"25/Jun/2020:05:07:58 +0000","time_ns":25337587,"code":"OK","time":"2020-06-25T05:07:58Z","message":"unary"}
ocis          | {"level":"info","service":"reva","pkg":"rgrpc","traceid":"767afdda49f2f11343b587b573511e43","user-agent":"grpc-go/1.30.0","from":"tcp://127.0.0.1:45220","uri":"/cs3.storage.registry.v1beta1.RegistryAPI/GetStorageProvider","start":"25/Jun/2020:05:07:58 +0000","end":"25/Jun/2020:05:07:58 +0000","time_ns":171625,"code":"OK","time":"2020-06-25T05:07:58Z","message":"unary"}
ocis          | {"level":"error","service":"reva","pkg":"rgrpc","traceid":"767afdda49f2f11343b587b573511e43","error":"error: not supported: eos: create home not supported","time":"2020-06-25T05:07:58Z","caller":"/root/go/pkg/mod/github.com/cs3org/reva@v0.1.1-0.20200624063447-db5e6635d5f0/internal/grpc/services/storageprovider/storageprovider.go:346","message":"error creating home"}
ocis          | {"level":"error","service":"reva","pkg":"rgrpc","traceid":"767afdda49f2f11343b587b573511e43","error":"error: not supported: eos: create home not supported","time":"2020-06-25T05:07:58Z","message":"storageprovider: error calling CreateHome of storage driver"}
ocis          | {"level":"info","service":"reva","pkg":"rgrpc","traceid":"767afdda49f2f11343b587b573511e43","user-agent":"grpc-go/1.30.0","from":"tcp://127.0.0.1:38104","uri":"/cs3.storage.provider.v1beta1.ProviderAPI/CreateHome","start":"25/Jun/2020:05:07:58 +0000","end":"25/Jun/2020:05:07:58 +0000","time_ns":229708,"code":"OK","time":"2020-06-25T05:07:58Z","message":"unary"}
ocis          | {"level":"error","service":"reva","pkg":"rgrpc","traceid":"767afdda49f2f11343b587b573511e43","error":"gateway: grpc failed with code CODE_INTERNAL","time":"2020-06-25T05:07:58Z","message":"error calling Createhome"}
ocis          | {"level":"error","service":"reva","pkg":"rgrpc","traceid":"767afdda49f2f11343b587b573511e43","error":"gateway: grpc failed with code CODE_INTERNAL","time":"2020-06-25T05:07:58Z","caller":"/root/go/pkg/mod/github.com/cs3org/reva@v0.1.1-0.20200624063447-db5e6635d5f0/internal/grpc/services/gateway/authprovider.go:127","message":"error creating user home"}
ocis          | {"level":"info","service":"reva","pkg":"rgrpc","traceid":"767afdda49f2f11343b587b573511e43","user-agent":"grpc-go/1.30.0","from":"tcp://127.0.0.1:45212","uri":"/cs3.gateway.v1beta1.GatewayAPI/Authenticate","start":"25/Jun/2020:05:07:58 +0000","end":"25/Jun/2020:05:07:58 +0000","time_ns":28379429,"code":"OK","time":"2020-06-25T05:07:58Z","message":"unary"}
ocis          | {"level":"error","service":"reva","pkg":"rhttp","traceid":"767afdda49f2f11343b587b573511e43","error":"auth: grpc failed with code CODE_INTERNAL","time":"2020-06-25T05:07:58Z","message":"error generating access token from credentials"}
ocis          | {"level":"warn","service":"reva","pkg":"rhttp","traceid":"767afdda49f2f11343b587b573511e43","host":"127.0.0.1","method":"PUT","uri":"/remote.php/dav/files/user0/file.txt","url":"/remote.php/dav/files/user0/file.txt","proto":"HTTP/1.1","status":401,"size":0,"start":"25/Jun/2020:05:07:58 +0000","end":"25/Jun/2020:05:07:58 +0000","time_ns":28983013,"time":"2020-06-25T05:07:58Z","message":"http"}

@ishank011 any idea what the issue could be, maybe something went wrong in cs3org/reva#830 ?

@individual-it
Copy link
Member Author

Or are different configurations needed now?

@labkode
Copy link
Member

labkode commented Jun 25, 2020

@individual-it I think so, there are two EOS drivers: eos and eoshome, for your tests it looks like you need to have eoshome.

@ishank011
Copy link
Contributor

@individual-it @labkode yes, that looks like it. You need to change the driver name to eoshome and remove the enable_home specification (although it doesn't matter if it's still there).

@PVince81
Copy link
Contributor

PVince81 commented Jul 6, 2020

@individual-it did you get a chance to recheck this ?

@individual-it
Copy link
Member Author

seems to work with the current master branch of ocis, closing this

@individual-it individual-it mentioned this issue Sep 28, 2020
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug Something isn't working eos p2-urgent
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@PVince81 @butonic @individual-it @micbar @labkode @ishank011