Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

can not copy or move directory between different external file storage, but files work fine #14743

Closed
tuxcrafter opened this issue Mar 18, 2019 · 30 comments · Fixed by #38623
Closed

Comments

@tuxcrafter
Copy link

tuxcrafter commented Mar 18, 2019

How to use GitHub

  • Please use the 👍 reaction to show that you are affected by the same issue.
  • Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
  • Subscribe to receive notifications on status change and new comments.

Steps to reproduce

I can not copy or move directory between different external file storage, but normal files work fine. How can I get this working?

The first argument to copy() function cannot be a directory at

{"reqId":"XJAjnlYH6@rVQbcVclaXfwAAAAg","level":3,"time":"2019-03-18T23:02:54+00:00","remoteAddr":"80.127.158.83","user":"jdejong","app":"PHP","method":"MOVE","url":"\/nextcloud\/remote.php\/dav\/files\/jdejong\/Communications\/communication-jdejong","message":"rename(): The first argument to copy() function cannot be a directory at \/var\/www\/html\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#272","userAgent":"Mozilla\/5.0 (X11; Linux x86_64; rv:60.0) Gecko\/20100101 Firefox\/60.0","version":"15.0.4.0"}
{"reqId":"XJAjnlYH6@rVQbcVclaXfwAAAAg","level":3,"time":"2019-03-18T23:02:54+00:00","remoteAddr":"80.127.158.83","user":"jdejong","app":"PHP","method":"MOVE","url":"\/nextcloud\/remote.php\/dav\/files\/jdejong\/Communications\/communication-jdejong","message":"rename(\/\/mnt\/nfs\/communications\/communication-jdejong,\/\/mnt\/nfs\/programmes\/communication-jdejong): Invalid cross-device link at \/var\/www\/html\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#272","userAgent":"Mozilla\/5.0 (X11; Linux x86_64; rv:60.0) Gecko\/20100101 Firefox\/60.0","version":"15.0.4.0"}

Moving a normal file works but still gives this message:

{"reqId":"XJAmBLAHf2Bocyt--HDF@gAAAAA","level":3,"time":"2019-03-18T23:13:08+00:00","remoteAddr":"80.127.158.83","user":"jdejong","app":"PHP","method":"MOVE","url":"\/nextcloud\/remote.php\/dav\/files\/jdejong\/Communications\/communication-jdejong\/communicaton-jdejong.txt","message":"rename(\/\/mnt\/nfs\/communications\/communication-jdejong\/communicaton-jdejong.txt,\/\/mnt\/nfs\/programmes\/communicaton-jdejong.txt): Operation not permitted at \/var\/www\/html\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#272","userAgent":"Mozilla\/5.0 (X11; Linux x86_64; rv:60.0) Gecko\/20100101 Firefox\/60.0","version":"15.0.4.0"}

The setup:

sudo -u apache php occ group:add communications
sudo -u apache php occ group:add generalinformation
sudo -u apache php occ group:add officemanagement
sudo -u apache php occ group:add programmes
sudo -u apache php occ files_external:create -c datadir=/mnt/nfs/communications "Communications" local null::null
sudo -u apache php occ files_external:create -c datadir=/mnt/nfs/generalinformation "General Information" local null::null
sudo -u apache php occ files_external:create -c datadir=/mnt/nfs/officemanagement "Office Management" local null::null
sudo -u apache php occ files_external:create -c datadir=/mnt/nfs/programmes "Programmes" local null::null
sudo -u apache php occ files_external:applicable --add-group communications 11
sudo -u apache php occ files_external:option 11 enable_sharing true
sudo -u apache php occ files_external:option 11 encrypt false
sudo -u apache php occ files_external:applicable --add-group generalinformation 14
sudo -u apache php occ files_external:option 14 enable_sharing true
sudo -u apache php occ files_external:option 14 encrypt false
sudo -u apache php occ files_external:applicable --add-group officemanagement 15
sudo -u apache php occ files_external:option 15 enable_sharing true
sudo -u apache php occ files_external:option 15 encrypt false
sudo -u apache php occ files_external:applicable --add-group programmes 16
sudo -u apache php occ files_external:option 16 enable_sharing true
sudo -u apache php occ files_external:option 16 encrypt false
# sudo -u apache php occ --version
Nextcloud 15.0.4
# cat /etc/redhat-release 
CentOS Linux release 7.6.1810 (Core) 
@tuxcrafter tuxcrafter added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Mar 18, 2019
@danaivehr
Copy link

Hi! The same issue here.
Nextcloud 16.0.2 in docker
FIles are movable between External storage and data directory, while folders are not, with the same messages in nextcloud.log

I can provide additional info, but don't to, which.

@danaivehr
Copy link

Perhaps the 'Invalid cross-device link' is the main clue in error message. I have external storage pointing to local directory, which is binded from another /dev/

@kesselb
Copy link
Contributor

kesselb commented Jul 7, 2019

@danaivehr would you mind to share the stacktrace (the log message) from 16.0.2? That makes it easier to locate the error.

@kesselb
Copy link
Contributor

kesselb commented Jul 7, 2019

if ($this->is_dir($path1)) {
// we can't move folders across devices, use copy instead
$stat1 = stat(dirname($this->getSourcePath($path1)));
$stat2 = stat(dirname($this->getSourcePath($path2)));
if ($stat1['dev'] !== $stat2['dev']) {
$result = $this->copy($path1, $path2);
if ($result) {
$result &= $this->rmdir($path1);
}
return $result;
}
}

There is code to handle this case. Not sure why you reach line 272. Looks like the is_dir does not return true. Could you add the code below before if ($this->is_dir($path1)) { and check the logs?

\OCP\Util::writeLog('core', 'trying to rename: ' . $path1 . ' is_dir: ' . print_r($this->is_dir($path1), true), ILogger::WARN);

Moving a normal file works but still gives this message:

{"reqId":"XJAmBLAHf2Bocyt--HDF@gAAAAA","level":3,"time":"2019-03-18T23:13:08+00:00","remoteAddr":"80.127.158.83","user":"jdejong","app":"PHP","method":"MOVE","url":"\/nextcloud\/remote.php\/dav\/files\/jdejong\/Communications\/communication-jdejong\/communicaton-jdejong.txt","message":"rename(\/\/mnt\/nfs\/communications\/communication-jdejong\/communicaton-jdejong.txt,\/\/mnt\/nfs\/programmes\/communicaton-jdejong.txt): Operation not permitted at \/var\/www\/html\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#272","userAgent":"Mozilla\/5.0 (X11; Linux x86_64; rv:60.0) Gecko\/20100101 Firefox\/60.0","version":"15.0.4.0"}

https://www.php.net/manual/en/function.rename.php#117590
https://bugs.php.net/bug.php?id=50676

@miguelarios
Copy link

miguelarios commented Oct 1, 2019

I have this similar issue when i delete a folder I get this issue consistently always #289. I have docker version of 16.0.5 for Nextcloud and I am using local as type for external storage plugin and the shares are being mapped through docker volume paths.

{"reqId":"O7mSOgFqennvMzd77f8H","level":3,"time":"2019-09-28T21:52:13+00:00","remoteAddr":"172.18.0.2","user":"miguel","app":"PHP","method":"DELETE","url":"\/remote.php\/dav\/files\/miguel\/Documents\/Shared%20Files","message":"rename(): The first argument to copy() function cannot be a directory at \/config\/www\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#289","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko\/20100101 Firefox\/69.0","version":"16.0.5.1"}
{"reqId":"O7mSOgFqennvMzd77f8H","level":3,"time":"2019-09-28T21:52:13+00:00","remoteAddr":"172.18.0.2","user":"miguel","app":"PHP","method":"DELETE","url":"\/remote.php\/dav\/files\/miguel\/Documents\/Shared%20Files","message":"rename(\/\/miguel\/Documents\/Shared Files,\/\/data\/miguel\/files_trashbin\/files\/Shared Files.d1569707533): Cross-device link at \/config\/www\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#289","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko\/20100101 Firefox\/69.0","version":"16.0.5.1"}

What is very peculiar is that although i get this error and I also get an error pop up on the top if I refresh the site I don't see the file anymore and if I visit the trash bin i don't see the file there at all either.

@pteriss

This comment has been minimized.

@Zegorax

This comment has been minimized.

@dl-lim

This comment has been minimized.

@y0grt

This comment has been minimized.

@scristall

This comment has been minimized.

@madpipeline
Copy link

Detected this issue in 20.0.4 as well, with a local external storage.

@raleonardo
Copy link

Still the same issue in 21.0.0 ( on docker nextcloud:21)

Happens while moving a directory from external storage to my user storage.

When trying to delete the directory from external storage, I get a popup: ' Error deleting file "<folder_name>" '

@Xalaxis
Copy link

Xalaxis commented Apr 17, 2021

I believe this issue causes permanent file deletion in current Nextcloud versions. Attempts to move files to the recycle bin are regarded as a success even though they fail. Without backups you will lose data. I ran into this issue attempting to move a directory from a VirtualFiles backed local folder.

image

@szaimen
Copy link
Contributor

szaimen commented Jul 2, 2021

Seems like is_dir doesn't return true in all cases
e.g. https://www.php.net/manual/de/function.is-dir.php#98338
Does the mentioned workaround help?

@ghost
Copy link

ghost commented Aug 1, 2021

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

@ghost ghost added the stale Ticket or PR with no recent activity label Aug 1, 2021
@ghost ghost closed this as completed Aug 15, 2021
@tuxcrafter
Copy link
Author

Please reopen this ticket!

@raleonardo
Copy link

I agree. The issue still exists in 22.1.0

@szaimen szaimen reopened this Aug 17, 2021
@ghost ghost removed the stale Ticket or PR with no recent activity label Aug 17, 2021
@szaimen szaimen added 1. to develop Accepted and waiting to be taken care of and removed needs info 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels Aug 17, 2021
@NobeKanai
Copy link

Same problem. Any solution here?

@eleminer
Copy link

Same here?

@63OR63
Copy link

63OR63 commented Dec 19, 2021

The ability to move folders to external folders is crucial, especially considering you can't upload to external storage from official Windows Nextcloud app. The only way to do so currently is through web interface, which isn't optimal in many use cases to say the least.

@enchained
Copy link

Can confirm the issue on 23.0.0. Removing a folder from external storage (local type) gives those errors:

Error | PHP | Error: rename(): The first argument to copy() function cannot be a directory at /config/www/nextcloud/lib/private/Files/Storage/Local.php#347
Error | PHP | Error: rename(//ext-data/pathTo/myFolderName,//data/myUser/files_trashbin/files/myFolderName.d1643307398): Cross-device link at /config/www/nextcloud/lib/private/Files/Storage/Local.php#347

And leads to permanent removal instead of moving to the trashbin.

@bendev25689
Copy link

Any progress on this issue? I am still having the same issue on nextcloud 26

@sjash
Copy link

sjash commented May 12, 2023

Can confirm the issue on 23.0.0. Removing a folder from external storage (local type) gives those errors:

Error | PHP | Error: rename(): The first argument to copy() function cannot be a directory at /config/www/nextcloud/lib/private/Files/Storage/Local.php#347
Error | PHP | Error: rename(//ext-data/pathTo/myFolderName,//data/myUser/files_trashbin/files/myFolderName.d1643307398): Cross-device link at /config/www/nextcloud/lib/private/Files/Storage/Local.php#347

And leads to permanent removal instead of moving to the trashbin.

I'm too facing the same issue on Nextcloud 25.0.6. Any leads will be helpfull

@proofrock
Copy link

proofrock commented May 25, 2023

Same issue here whenever I try to delete any directory. It gets deleted even though the error message is shown and it isn't present in the trash bin. Version 26.0.1. I don't have external storages mounted, just a normal setup.

@teije
Copy link

teije commented May 27, 2023

Also experiencing the same issue, this makes Nextcloud unusable on a Raspberry Pi since you're practically only able to use an external drive for file storage.

(Using Portainer/Nextcloud and an 8TB external drive)

"Invalid cross-device link at "<path/of/drive/mount">

@joshtrichards
Copy link
Member

joshtrichards commented Jun 2, 2023

Okay I was looking at #38569 then I realized it was a dup of #37741. Then I realized this was the real original issue so I'm moving my comments here rather than post them there.

Any chance any of you (@mebtte included) have symbolic links involved with some of these mount points/folders? Or maybe the underlying mounts are SMB/CIFS (not in NC, but at the OS level)?

Something is not right here.

That was a good catch on the PHP bug note @rbeier, but we definitely go out of our way to not do a regular PHP rename(), particularly if it's a directory (let alone across devices).

(one has to dig through the classes to see it, but it looks alright to me - note: copy below isn't PHP's built-in one).

I think what is happening is these folders are either not getting detected as such (the is_dir() at line 363) for some reason or the two stat() calls are being called on symbolic links on a parent filesystem (in which case I think adding 'localstorage.allowsymlinks' => true, to your config.php should fix it).

It could also be some other variations on these theme, but those are the two that are most obvious.

if ($this->is_dir($source)) {
// we can't move folders across devices, use copy instead
$stat1 = stat(dirname($this->getSourcePath($source)));
$stat2 = stat(dirname($this->getSourcePath($target)));
if ($stat1['dev'] !== $stat2['dev']) {
$result = $this->copy($source, $target);
if ($result) {
$result &= $this->rmdir($source);
}
return $result;
}
$this->checkTreeForForbiddenItems($this->getSourcePath($source));
}
return rename($this->getSourcePath($source), $this->getSourcePath($target));

So we have one of the following scenarios:

  • have some code that isn't detecting these as directories somewhere/somehow
  • a local misconfiguration
  • weird local underlying filesystem

@kesselb
Copy link
Contributor

kesselb commented Jun 2, 2023

@joshtrichards #19289

@proofrock
Copy link

proofrock commented Jun 2, 2023

weird local underlying filesystem

You bet! 😅 I had this issue with the following setup:

  • docker image of nextcloud (from linuxservers)
  • data dir mounted as a volume
  • PUID/PGID set to 1000 (my local user)
  • in the volume, the _data subdir (the one docker actually uses) has ownership to 1000/1000
  • now the tricky part: the files/ subdir of my nextcloud user subtree is a docker mount to a directory in my home dir

So, I had a docker -v nextcloud_data:/data with the volume, and -v /home/mano/myfiles:/data/mano/files for the files inside it.

It worked... sort of, the only problem is the one described in OP. Keeping everything in the docker volume, i.e. removing the second -v made this issue disappear.

Hope this is useful. I can setup a VPS with a reproduction of the problem, if useful, and give you access.

Thank you,

 G.

@bendev25689
Copy link

bendev25689 commented Jun 3, 2023

The issue in my case is using the official docker image nextcloud:latest, currently 26.0.2.

The external storage directory (Shared) is mounted to the docker container as /mnt/Shared. File system is zfs.
The data directory is also mounted to the docker container, as /mnt/data.

The Shared directory is and its contents is owned by www-data, which is uid 33 both inside and outside the container.

I am receiving an error when moving a directory, however copying a directory works fine, as does moving files to and from the external storage. Moving folders within the external storage also does not work.
The error in the log is:

{"reqId":"clc1937gUFOup8fkbkEb","level":3,"time":"2023-06-03T11:02:53+00:00","remoteAddr":"120.xx.xxx.xx","user":"Benjamin","app":"PHP","method":"MOVE","url":"/cloud/remote.php/dav/files/Benjamin/test_folder_for_move","message":"rename(): The first argument to copy() function cannot be a directory at /var/www/html/lib/private/Files/Storage/Local.php#378","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/112.0","version":"26.0.2.1","data":{"app":"PHP"}}
{"reqId":"clc1937gUFOup8fkbkEb","level":3,"time":"2023-06-03T11:02:53+00:00","remoteAddr":"120.xx.xxx.xx","user":"Benjamin","app":"PHP","method":"MOVE","url":"/cloud/remote.php/dav/files/Benjamin/test_folder_for_move","message":"rename(//mnt/data/Benjamin/files/test_folder_for_move,//mnt/Shared/Documents/test_folder_for_move): Success at /var/www/html/lib/private/Files/Storage/Local.php#378","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/112.0","version":"26.0.2.1","data":{"app":"PHP"}}

@solracsf
Copy link
Member

solracsf commented Jun 5, 2023

Feel free to try #38623

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.