moving a file and providing the fileID of the destination destroys data #6739

individual-it · 2023-07-07T07:14:18Z

Describe the bug

When making a WebDAV MOVE request and giving only the fileID of the destination, all the data of the destination folder or space will be destroyed.

Steps to reproduce

Steps to reproduce the behavior:

folder

upload a file called file-to-be-moved.txt to the personal space
in the personal space create a folder called dest
make a PROPFIND to the personal space to get the full path of file-to-be-moved.txt and the fileId of dest
make a MOVE request to the file-to-be-moved.txt with Destination header pointing to the fileId of dest (make sure to add remote.php to the destination URL, that seems to trigger it)
curl -uadmin:admin -k -XMOVE https://localhost:9200/remote.php/dav/spaces/136ffc9b-d317-4219-a0c3-084d5e070c60/file-to-be-moved.txt -H'Destination: https://localhost:9200/remote.php/dav/spaces/33c9baa8-450f-4873-a049-7beb8227face$136ffc9b-d317-4219-a0c3-084d5e070c60!9c495ede-bd11-439f-8496-85c649ef6121' -v

Expected behavior

the folder dest should be overwritten with the content of the file file-to-be-moved.txt as it happens when using the path of the dest folder
OR
return Error 400, same as it happens when not using remote.php in the Destionation URL

Actual behavior

Error 500 returned

path of the ocis data storage on the server exposed

<?xml version="1.0" encoding="UTF-8"?>
<d:error xmlns:d="DAV" xmlns:s="http://sabredav.org/ns"><s:exception></s:exception><s:message>move:Decomposedfs: could not move child: rename /home/artur/.ocis/storage/users/spaces/13/6ffc9b-d317-4219-a0c3-084d5e070c60/nodes/13/6f/fc/9b/-d317-4219-a0c3-084d5e070c60/file-to-be-moved.txt /home/artur/.ocis/storage/users/spaces/13/6ffc9b-d317-4219-a0c3-084d5e070c60/nodes: file exists</s:message></d:error>

destand all its content are gone
no log output

space

upload a file called file-to-be-moved.txt to the personal space
create an other space
query /graph/v1.0/me/drives to get the id of the new space
make a MOVE request to the file-to-be-moved.txt with Destination header pointing to: <ocis-url>/remote.php/dav/spaces/<spaceId> using remote.php seems to trigger it

Expected behavior

this should not be possible
Error 400 returned, same as it happens when not using remote.php in the Destionation URL

Actual behavior

Error 500 returned
logs {"level":"error","service":"ocdav","name":"com.owncloud.web.ocdav","traceid":"00000000000000000000000000000000","request-id":"deepthought3/0nFVzCuvCk-000764","spaceid":"136ffc9b-d317-4219-a0c3-084d5e070c60","path":"/file-to-be-moved.txt","status":{"code":15,"message":"delete:remove /home/artur/.ocis/storage/users/spaces/9f/429b6f-50a8-431d-9e72-ed59b0c52efc/nodes/Neuer Space: no such file or directory","trace":"00000000000000000000000000000000"},"code":500,"time":"2023-07-07T12:52:20.718783329+05:45","message":"Internal Server Error"}
space is destroyed

Setup

Please describe how you started the server and provide a list of relevant environment variables or configuration files.

PROXY_ENABLE_BASIC_AUTH=true IDM_ADMIN_PASSWORD=admin bin/ocis server

Additional context

Add any other context about the problem here.

The text was updated successfully, but these errors were encountered:

kobergj · 2023-07-07T10:36:54Z

I tried to reproduce. There are slight differences but the issue is the same.

When moving to a folder

The folder dest will be deleted just as you said but I can still find it in my trashbin and can restore it from there.
using remote.php or not has no impact on the behaviour.

When moving to a space

The space and all its contents cannot be accessed by the webUI any more
The data of the space is still there. Only the drive listing doesn't work any more
using remote.php or not has no impact on the behaviour

This ticket will probably need a raise in priority as it can lead to data loss. (not really - but it will look like this) @micbar

2403905 · 2023-07-10T12:54:30Z

@individual-it
It happens in the case of a folder because the destination in the request is the destination parent folder. And the overwrite is true by default.
If we add the file name as a destination it works properly.
The same for the spaces.

curl -uadmin:admin -k -XMOVE https://localhost:9200/remote.php/dav/spaces/136ffc9b-d317-4219-a0c3-084d5e070c60/file-to-be-moved.txt -H'Destination: https://localhost:9200/remote.php/dav/spaces/33c9baa8-450f-4873-a049-7beb8227face$136ffc9b-d317-4219-a0c3-084d5e070c60!9c495ede-bd11-439f-8496-85c649ef6121/file-to-be-moved.txt' -v

Definitely, it is a defect and we have to prevent data loss.

@micbar @kobergj FYI

amrita-shrestha · 2023-08-03T11:52:49Z

COPYmethod faces a similar case

curl -uadmin:admin -k -XCOPY 'https://localhost:9200/remote.php/dav/spaces/b11e098c-f29b-4fb2-816d-bc07bdb7fb18$a86cb5dd-9a94-41d2-87be-58953658f8f0!1da3b1a7-430b-4ee4-981a-26d078650cd6' \
-H'Destination:: https://localhost:9200/remote.php/dav/spaces/b11e098c-f29b-4fb2-816d-bc07bdb7fb18$a86cb5dd-9a94-41d2-87be-58953658f8f0!480d53df-3927-4ad1-9b34-3d3308b3ff18'

Response:
HTTP status code 500 and dest moved to trashbin

saw-jan · 2023-09-07T04:29:57Z

Tested with Infinite Scale 4.0.0+82428024f8 Community

Personal - Personal Space (`Destination uses ID`)

Command:

curl -XMOVE "https://localhost:9200/remote.php/dav/spaces/f1291c6b-0076-4ad2-8a6c-5da9640d50ca\$099e189e-fe3f-439d-a20a-f71192946b2f/afile.txt" \              
-H"Destination: https://localhost:9200/remote.php/dav/spaces/f1291c6b-0076-4ad2-8a6c-5da9640d50ca\$099e189e-fe3f-439d-a20a-f71192946b2f\!1c127019-f497-43bc-93e9-4caa30beccba" \
-uadmin:admin -vk

Response: ❗ issue persists

< HTTP/1.1 500 Internal Server Error
...
<?xml version="1.0" encoding="UTF-8"?>
<d:error
	xmlns:d="DAV"
	xmlns:s="http://sabredav.org/ns">
	<s:exception></s:exception>
	<s:message>move:Decomposedfs: could not move child: rename /home/sawjan/.ocis/storage/users/spaces/09/9e189e-fe3f-439d-a20a-f71192946b2f/nodes/09/9e/18/9e/-fe3f-439d-a20a-f71192946b2f/afile.txt /home/sawjan/.ocis/storage/users/spaces/09/9e189e-fe3f-439d-a20a-f71192946b2f/nodes: file exists</s:message>
</d:error>

Personal - Project Space (`Destination uses ID`)

Command:

curl -XMOVE "https://localhost:9200/remote.php/dav/spaces/f1291c6b-0076-4ad2-8a6c-5da9640d50ca\$099e189e-fe3f-439d-a20a-f71192946b2f/afile.txt" \
-H"Destination: https://localhost:9200/remote.php/dav/spaces/f1291c6b-0076-4ad2-8a6c-5da9640d50ca\$a3d1da3e-281c-4b13-a06d-748f1d255e7a" \
-uadmin:admin -vk

Response: ❗ issue persists

< HTTP/1.1 500 Internal Server Error

Space gets deleted

NOTE: fix PR was https://github.com/cs3org/reva/pull/4056/files

2403905 · 2023-10-04T09:08:20Z

@saw-jan Merget cs3org/reva#4229 to the reva that disallows using the Personal and Project spaces root as a source/destination while move/copy.
Will be better to consider other cases apart.

saw-jan · 2023-10-10T10:37:37Z

We fixed a bug that caused destroying the Personal and Project spaces data when providing as a destination while move/copy file.
Disallow use the Personal and Project spaces root as a source while move/copy file.

@2403905 I see that the PR changelog describes above but I think the issue here is not related to space root but it is about using folder-id as the destination header.

saw-jan · 2023-10-10T10:42:34Z

Steps to reproduce:

create a file afile.txt
create a folder dest
Find:
- personal-space-id: fee02af4-bdc5-4ec6-863d-465f5eb4257d$0126f1a4-01aa-441b-9d9d-e0e808058ff1
- dest folder-id: fee02af4-bdc5-4ec6-863d-465f5eb4257d$0126f1a4-01aa-441b-9d9d-e0e808058ff1!df03d210-6381-4ce2-89a9-5d519ff7c971
try to MOVE afile.txt using id of folder dest as destination

Command:

curl -XMOVE "https://localhost:9200/remote.php/dav/spaces/<personal-space-id>/afile.txt" \
-H"Destination: https://localhost:9200/remote.php/dav/spaces/<dest-folder-id>" \
-uadmin:admin -vk

Example:

curl -XMOVE "https://localhost:9200/remote.php/dav/spaces/fee02af4-bdc5-4ec6-863d-465f5eb4257d\$0126f1a4-01aa-441b-9d9d-e0e808058ff1/afile.txt" \
-H"Destination: https://localhost:9200/remote.php/dav/spaces/fee02af4-bdc5-4ec6-863d-465f5eb4257d\$0126f1a4-01aa-441b-9d9d-e0e808058ff1\!df03d210-6381-4ce2-89a9-5d519ff7c971" \
-uadmin:admin -vk

2403905 · 2023-10-17T07:26:30Z

We fixed a bug that caused destroying the Personal and Project spaces data when providing as a destination while move/copy file.
Disallow use the Personal and Project spaces root as a source while move/copy file.

@2403905 I see that the PR changelog describes above but I think the issue here is not related to space root but it is about using folder-id as the destination header.

@saw-jan Why the issue is not related to space root?
The bug description consists of two cases folder and spase. And the space part's actual behavior said space is destroyed
The fix covers only the case when the Personal or Project spaces root is a destination while move/copy a file.

The folder case that about using folder-id as the destination header is still opened. We need to fix the decomposefs.

saw-jan · 2023-10-17T08:40:20Z

The bug description consists of two cases folder and spase. And the space part's actual behavior said space is destroyed

Yeah, you are right. Sorry for the confusion. I thought the fix was for folder-id realted.

Trying to MOVE to space-id as destination is confirmed to be fixed 👍

curl -XMOVE "https://localhost:9200/remote.php/dav/spaces/storage-users-1%24some-admin-user-id-0000-000000000000/afile.txt" \
-H"Destination: https://localhost:9200/remote.php/dav/spaces/storage-users-1%24a73a43f5-260a-4a1d-92ac-42a98248c3b8" \
-uadmin:admin -vk

< HTTP/1.1 400 Bad Request

saw-jan · 2023-10-17T08:40:43Z

The folder case that about using folder-id as the destination header is still opened. We need to fix the decomposefs.

so now we are waiting for this fix

ScharfViktor · 2023-12-14T22:13:48Z

can we close it?

saw-jan · 2023-12-15T03:54:15Z

can we close it?

Was there any fix for?

The folder case that about using folder-id as the destination header is still opened. We need to fix the decomposefs.

saw-jan · 2023-12-28T04:30:56Z

Steps to reproduce:

create a file afile.txt

create a folder dest

Find:

personal-space-id: fee02af4-bdc5-4ec6-863d-465f5eb4257d$0126f1a4-01aa-441b-9d9d-e0e808058ff1

dest folder-id: fee02af4-bdc5-4ec6-863d-465f5eb4257d$0126f1a4-01aa-441b-9d9d-e0e808058ff1!df03d210-6381-4ce2-89a9-5d519ff7c971

try to MOVE afile.txt using id of folder dest as destination

Command:
curl -XMOVE "https://localhost:9200/remote.php/dav/spaces/<personal-space-id>/afile.txt" \
-H"Destination: https://localhost:9200/remote.php/dav/spaces/<dest-folder-id>" \
-uadmin:admin -vk
Example:
curl -XMOVE "https://localhost:9200/remote.php/dav/spaces/fee02af4-bdc5-4ec6-863d-465f5eb4257d\$0126f1a4-01aa-441b-9d9d-e0e808058ff1/afile.txt" \
-H"Destination: https://localhost:9200/remote.php/dav/spaces/fee02af4-bdc5-4ec6-863d-465f5eb4257d\$0126f1a4-01aa-441b-9d9d-e0e808058ff1\!df03d210-6381-4ce2-89a9-5d519ff7c971" \
-uadmin:admin -vk

There's still an issue with trying to move file to a folder-id as destination

HTTP/1.1 500 Internal Server Error

<?xml version="1.0" encoding="UTF-8"?>
<d:error
	xmlns:d="DAV"
	xmlns:s="http://sabredav.org/ns">
	<s:exception></s:exception>
	<s:message>move:Decomposedfs: could not move child: rename /home/sawjan/.ocis/storage/users/spaces/df/92eaf2-a807-42cf-a8d4-cb9219b8dc4c/nodes/df/92/ea/f2/-a807-42cf-a8d4-cb9219b8dc4c/move-me.txt /home/sawjan/.ocis/storage/users/spaces/df/92eaf2-a807-42cf-a8d4-cb9219b8dc4c/nodes: file exists</s:message>
</d:error>%

CC @micbar

2403905 · 2024-01-23T17:36:38Z

@micbar The issue is partially solved. We disallowed to use the Personal and Project spaces root as a source. It prevents the data loss.
We could also disallow overwrite folder by file or file by folder and sounds to me like this doesn't contradict webDAV spec. I tried but such changes affected a lot of tests.
We need to discuss what we want to achieve.

2403905 · 2024-01-26T13:05:27Z

@dragotin Could we discuss what we can do with the overwriting folder by file or file by folder? It could affect the clients.

butonic · 2024-02-05T14:11:26Z

What is left unclear when reading the rfc? At leas whon tdhe overwrite header is set, the outcome should be clear:

9.9.3. MOVE and the Overwrite Header

If a resource exists at the destination and the Overwrite header is
"T", then prior to performing the move, the server MUST perform a
DELETE with "Depth: infinity" on the destination resource. If the
Overwrite header is set to "F", then the operation will fail.

Edit: ah, if the destination is just a node it would be deleted. The old name should be reused, and the fileid of the destination should be changed to the one of the source.

saw-jan · 2024-02-09T09:47:56Z

Tested and confirmed the behavior.

curl -XMOVE "https://localhost:9200/remote.php/dav/spaces/<space-id>/afile.txt" \
-H"Destination: https://localhost:9200/remote.php/dav/spaces/<folder-id>" \
-uadmin:admin -vk

204 ✔️
```
< HTTP/1.1 204 No Content
```
dest folder is deleted and moved to trashbin ✔️
afile.txt is renamed to dest file ✔️
dest file has the file-id of previous filename afile.txt ✔️

individual-it added the Type:Bug label Jul 7, 2023

individual-it mentioned this issue Jul 7, 2023

create WebDAV API tests for accessing files by fileID #6737

Closed

5 tasks

micbar added the Priority:p2-high Escalation, on top of current planning, release blocker label Jul 7, 2023

micbar added this to Infinite Scale Team Board Jul 7, 2023

github-project-automation bot moved this to Qualification in Infinite Scale Team Board Jul 7, 2023

micbar moved this from Qualification to Prio 2 in Infinite Scale Team Board Jul 7, 2023

2403905 self-assigned this Jul 7, 2023

2403905 moved this from Prio 2 to In progress in Infinite Scale Team Board Jul 7, 2023

2403905 moved this from In progress to Done in Infinite Scale Team Board Jul 17, 2023

amrita-shrestha mentioned this issue Aug 8, 2023

Providing folder as destination while move/copy file destroy folder data #6983

Closed

kobergj mentioned this issue Aug 10, 2023

COPY a folder to received share file/folder seems to work but no change in shares - lost share mountpoint in me/drives #6999

Closed

2403905 removed their assignment Sep 1, 2023

SwikritiT mentioned this issue Sep 5, 2023

Shared mount folder gets deleted when overwritten by a file from personal space #7208

Closed

micbar closed this as completed Dec 27, 2023

individual-it reopened this Dec 28, 2023

github-project-automation bot moved this from Done to In progress in Infinite Scale Team Board Dec 28, 2023

saw-jan mentioned this issue Jan 4, 2024

API Test cases for MOVE between project/personal/shares spaces #8121

Closed

saw-jan mentioned this issue Jan 5, 2024

MOVE a file into same folder with same name returns 404 instead of 403 #1976

Open

micbar moved this from In progress to Prio 2 in Infinite Scale Team Board Jan 22, 2024

micbar added this to the Release 5.0.0 milestone Jan 22, 2024

2403905 self-assigned this Feb 6, 2024

2403905 moved this from Prio 2 to In progress in Infinite Scale Team Board Feb 6, 2024

2403905 mentioned this issue Feb 8, 2024

[bump reva]fix an error when move using destination id #8396

Merged

9 tasks

2403905 closed this as completed in #8396 Feb 8, 2024

github-project-automation bot moved this from In progress to Done in Infinite Scale Team Board Feb 8, 2024

This was referenced Feb 13, 2024

[QA] check and add API test coverage for these bug fixes #8440

Closed

[tests-only] add API test coverage for file MOVE to space-id as destination #8459

Merged

[tests-only] extend tests: MOVE to Shares-space-id as destination #8470

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

moving a file and providing the fileID of the destination destroys data #6739

moving a file and providing the fileID of the destination destroys data #6739

individual-it commented Jul 7, 2023

kobergj commented Jul 7, 2023

2403905 commented Jul 10, 2023 •

edited

Loading

amrita-shrestha commented Aug 3, 2023 •

edited

Loading

saw-jan commented Sep 7, 2023 •

edited

Loading

2403905 commented Oct 4, 2023

saw-jan commented Oct 10, 2023

saw-jan commented Oct 10, 2023 •

edited

Loading

2403905 commented Oct 17, 2023 •

edited

Loading

saw-jan commented Oct 17, 2023

saw-jan commented Oct 17, 2023

ScharfViktor commented Dec 14, 2023

saw-jan commented Dec 15, 2023

saw-jan commented Dec 28, 2023

2403905 commented Jan 23, 2024

2403905 commented Jan 26, 2024

butonic commented Feb 5, 2024 •

edited

Loading

saw-jan commented Feb 9, 2024

moving a file and providing the fileID of the destination destroys data #6739

moving a file and providing the fileID of the destination destroys data #6739

Comments

individual-it commented Jul 7, 2023

Describe the bug

Steps to reproduce

folder

Expected behavior

Actual behavior

space

Expected behavior

Actual behavior

Setup

Additional context

kobergj commented Jul 7, 2023

2403905 commented Jul 10, 2023 • edited Loading

amrita-shrestha commented Aug 3, 2023 • edited Loading

saw-jan commented Sep 7, 2023 • edited Loading

Personal - Personal Space (Destination uses ID)

Personal - Project Space (Destination uses ID)

2403905 commented Oct 4, 2023

saw-jan commented Oct 10, 2023

saw-jan commented Oct 10, 2023 • edited Loading

2403905 commented Oct 17, 2023 • edited Loading

saw-jan commented Oct 17, 2023

saw-jan commented Oct 17, 2023

ScharfViktor commented Dec 14, 2023

saw-jan commented Dec 15, 2023

saw-jan commented Dec 28, 2023

2403905 commented Jan 23, 2024

2403905 commented Jan 26, 2024

butonic commented Feb 5, 2024 • edited Loading

saw-jan commented Feb 9, 2024

2403905 commented Jul 10, 2023 •

edited

Loading

amrita-shrestha commented Aug 3, 2023 •

edited

Loading

saw-jan commented Sep 7, 2023 •

edited

Loading

Personal - Personal Space (`Destination uses ID`)

Personal - Project Space (`Destination uses ID`)

saw-jan commented Oct 10, 2023 •

edited

Loading

2403905 commented Oct 17, 2023 •

edited

Loading

butonic commented Feb 5, 2024 •

edited

Loading