Extemely slow sync with ARM server and Linux/Mac clients (no windows)

[mbelgin]

Hi!

I have a SeaGate GoFlex with an ARM processor, which now runs archlinux and owncloud on it. It has 128MB of ram, but I support it with a swap partition.

I am trying to sync a mac and a linux laptop. I dropped Gigabytes of data (my entire Documents folder) on the Mac and got very excited to see them start popping up on my Linux laptop. However, I noticed that only 28MB was sync'ed in 10 hours, and 280MB when I left it overnight !!

My system is as follows:

Linux goflex 3.1.10-15-ARCH #1 PREEMPT Wed Dec 12 15:25:18 UTC 2012 armv5tel GNU/Linux
Processor : Feroceon 88FR131 rev 1 (v5l)
BogoMIPS : 1191.11
Features : swp half thumb fastmult edsp
Memory : 128MB
Swap : 2GB

I believe the problem is with data comparison and metadata (csync layer), but correct me if I am wrong. I was able to sync a 'single' 25GB truecrypt image at a reasonable speed, so I am thinking the bottleneck is the large numbers of small files, and their comparison, especially when there are many levels of directories needing to be recursively analyzed. Looks like 2 httpd processes are using 85% of cpu and strace shows tons of lstat64 calls on my server.

I would understand it owncloud needs a minimum system requirements, but I also heard people using it with ARM systems. Could the lack of memory be killing the performance?

Thanks a lot!

[mbelgin]

Update: I just noticed that the Mac client stopped uploading with a message: "The folder waits to start syncing" Although there are tons of files waiting to be synced.

[dragotin]

which version of server and client are involved?

[mbelgin]

I am using owncloud server 4.5.4, with clients 1.1.3 on both Linux and Mac.

I wiped out the entire installation to give a shot to lighttpd and sqlite.
Perhaps the standard LAMP was too heavyweight for my ARM+128MB system. In
the mean time, I will appreciate any configuration suggestions that
experienced users may have.

Thank you!

On Mon, Dec 17, 2012 at 5:32 AM, dragotin notifications@github.com wrote:

which version of server and client are involved?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/185#issuecomment-11436780.

Mehmet Belgin, Ph.D. (mehmet.belgin@oit.gatech.edu)
Scientific Computing Consultant | OIT - Academic and Research Technologies
Georgia Institute of Technology
258 Fourth Street, Rich Building, Room 326
Atlanta, GA 30332-0700
Office: (404) 385-0665

[mbelgin]

Update: I re-installed everything using lighttpd and php-fpm. Unfortunately it did not change anything. When I strace the Linux client on my laptop, I am seeing this output in a continuous loop.

write(3, "\1\0\0\0\0\0\0\0", 8) = 8
recvfrom(6, 0x20b52a4, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(8, 0x2153274, 4096, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=3, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}, {fd=13, events=POLLIN}, {fd=6, events=POLLIN}, {fd=17, events=POLLIN}, {fd=18, events=POLLIN}, {fd=12, events=POLLIN}, {fd=20, events=POLLIN}, {fd=11, events=POLLIN}], 10, 0) = 1 ([{fd=3, revents=POLLIN}])

[mbelgin]

This is also interesting:

I was syncing between ubuntu and mac. Just out of curiosity, I installed ubuntu on the mac as well, and tried the linux client. The speed is significantly better!

Can the mac client be causing the problems? Is there any known issues with it?

[danimo]

Are you syncing the exact same data set from all machines? We have leads that this is a server issue. The server issues a lot more SQL requests if the directory (recursively) contains more files (linear dependency). The slow sync is thus a server problerm with your ARM device. We are going to address this on the server side.

[danimo]

owncloud/core#2353 might be related.