Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

csync2 still actively developped or "only" maintained ? #31

Open
ggs67 opened this issue Jan 9, 2021 · 14 comments
Open

csync2 still actively developped or "only" maintained ? #31

ggs67 opened this issue Jan 9, 2021 · 14 comments

Comments

@ggs67
Copy link

ggs67 commented Jan 9, 2021

The title says it all ;-) I am actually asking because I am looking forward for more options in the auto-conflic-resolution part (eg. "auto" statement).

It would be great to have an additional mode specifying the winning node directly (ex.: "auto host host1") specifying which host will win (this may of course require an additional run (possibly automatic) in multi-host configs.

Also every "auto" statement should allow for a file pattern (just like exclude) applying it to only those files. (if not present changes the global setting, thus not breaking any existing config)

@lge
Copy link
Contributor

lge commented Jan 10, 2021 via email

@ggs67
Copy link
Author

ggs67 commented Jan 11, 2021

Ok, I will think about it. Unfortunately my time is very limited as by work and other projects. But I could well consider working on this. I will fork and give it a try, Esspecially since I stumbled over some other features I would like to see, like dynamic auto-conflict resolution.

@stfast
Copy link

stfast commented Jan 18, 2021

Will you consider merging patches from schafdog csync2 fork. Some nice extra features there.

https://github.com/schafdog/csync2

@lge
Copy link
Contributor

lge commented Jan 18, 2021 via email

@erlandl4g
Copy link

I will attempt merging some of the stuff such as code cleanup, less FS scans and less connections to make it work faster , using less resources. I have made a pull request with some fixes to current major problems.

@stfast
Copy link

stfast commented Sep 15, 2021

There is also a "new" issue since there is xinetd package (which triggers csync2) and also csync2 package itself removed from RedHat Fedora 34+ distribution. Obvious reason for this removal is xinetd dependency. Did you consider running csync2 as a daemon (via csync2d)?

@erlandl4g
Copy link

Now, when I know that such a problem will be coming - will see options on how to solve that. I think I saw somewhere Csync2 docs mentioned as it is being able to run as a daemon but cannot find clear instructions on how such installation is made/run. Would you point me in the right direction?

@bes-internal
Copy link

Now, when I know that such a problem will be coming - will see options on how to solve that. I think I saw somewhere Csync2 docs mentioned as it is being able to run as a daemon but cannot find clear instructions on how such installation is made/run. Would you point me in the right direction?

-ii

@erlandl4g
Copy link

No, it does not run for me in that mode, gives:

csync2 -ii
Server error: Address already in use

Will have a look at this later.

@bes-internal
Copy link

Yep... Because you already have a process running on port 30865 and this is most likely another csync2 instance over inetd

# ss -lnp |grep 30865
tcp    LISTEN     0      128       *:30865                 *:*                   users:(("inetd",pid=515,fd=7))

@stfast the only thing the distribution needs to do is run csync2 -ii. My opinion is that creating start scripts should be created by the distributions themselves, but if possible, you can add systemd unit files and init.d to csync2 project

@erlandl4g
Copy link

Yes, there was another process on that port. Not another csync2 instance but reserved by Xinetd for csync2 itself (which is not running at the moment). Would have to disable it to run csync2 -ii however - have not such need now.

@stfast
Copy link

stfast commented Sep 15, 2021

Indeed, you have to stop xinetd:

$ csync2 -ii
Server error: Address already in use

$ systemctl stop xinetd

$ csync2 -vvvvv -ii
Listening on 0.0.0.0:30865 as server.domain.com.
Csync2 daemon running. Waiting for connections.

$ systemctl start xinetd

$ csync2 -ii
Server error: Address already in use

Actualy there is allready systemd csync2.service file

https://github.com/LINBIT/csync2/blob/master/csync2%40.service

but should be improved a little.

@ROBERT-MCDOWELL
Copy link

I use csync2 last git with fedora 34 without xinetd since 6 months now without prob.
here is the units:

csync2.socket

[Socket]
ListenStream=30865
Accept=yes

[Install]
WantedBy=sockets.target

csync2@.service

[Unit]
Description=csync2 connection handler
After=syslog.target

[Service]
ExecStart=-/usr/sbin/csync2 -i -v
StandardInput=socket
StandardOutput=socket

I compiled csync2 with these options
./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc/csync2

@erlandl4g
Copy link

I have moved away from Xinetd use and started using Inotify to provide a list of changes made. Got a shell script which is processing the list of changes and sends only them to peers. This script is included in my forked version.

Since then I got time stamps fully fixed and timestamps being always at nanosecond level. Now working on making parallelism.

This will allow sending changes to peers in parallel and make large number of files being sent to peers faster.

Since MySQL and PostgreSQL allow avoiding table lock on deletions/updates - initials tests were done using them. MySQL was slowest even with InnoDB. But PostgreSQL with parallel updates was not going any faster than SQLite single threaded up to 5 peers .

Thus it was chosen to continue using SQLite but with Deletes delayed while changes being sent. This has been successfully tested on a separate branch and will be implemented in my fork Master branch in the coming days.

It will be sort of Hack, but working stable. A proper implementation would require nearly full rewrite of Csync2 code as it is using global variables in many places it should not and code is written not the best way.

Ideally I would like to get a team of decent developers and get Csync2 a full revamp, making it Csync3 where it would be truly multithreaded, cleaned up code, proper connections with up to date coding standards.

Idealy it would listen to Inotify directly and would send transfers as they happen without queueing them up without any delay and SQLite writes would be done after file transfer is finished.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants