Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintenance #790

Closed
untitaker opened this issue Mar 17, 2019 · 28 comments
Closed

Maintenance #790

untitaker opened this issue Mar 17, 2019 · 28 comments

Comments

@untitaker
Copy link
Member

untitaker commented Mar 17, 2019

I'd like to hand this project over. I have multiple reasons for having stopped investing time in this project:

  • I stopped using vdirsyncer and everything in pimutils, including using mutt since I started using OS X. I still self-host, but use GUI apps for many things.

  • Neverending bugreports about undebuggable servers. Inherent bugginess of everything DAV-related. I wrote about that already, but don't think anything will change there.

  • Horrible packaging story in Python, and worse packaging story for users who are used to installing everything from a distro's packaging manager. Downstream distributors don't provide a way to distribute test builds to end users. This triples the pain of working on a bugreport since most users need extra assistance in installing from sources, or simply wait for a release from me (and the corresponding downstream release) until they provide feedback on a bugfix.

    Larger projects provide their own Debian packages, so I tried that, but: 1.) it's too much work for one person 2.) there is fundamental distrust for things not found in Debian's official repos 3.) what about the other distros?

    Another thing I tried was rewriting the entire thing in Rust so I could distribute static binaries. I hoped that this would make installation on all platforms easier. Particularly on Windows it would make a lot of aspects of packaging easier, should I ever decide to support Windows. But that's where my energy just left me halfway through. Partly because the way I attempted the rewrite introduced a lot of friction, but mostly because I already started to loose interest in the project as a whole.

  • This is the biggest one: The last point about packaging and the bugreport feedback loop is made worse by distros that have multiple stability channels. People report issues about "the latest vdirsyncer version" when running on Debian stretch, which is still on version 0.14. Those people happen to be the ones sending me direct emails because they are too paranoid to open a GitHub account. It's why you won't find my email address on my GitHub profile anymore.

    I had a private conversation with the current Debian package maintainers about this issue when they first looked into packaging vdirsyncer for Debian. I asked them to keep vdirsyncer out of the stable channel, and if that is not possible, to just not package vdirsyncer at all. They did not care and went ahead with packaging the first version anyway. The conversation roughly went the same as the one JWZ had with them about xscreensaver, and I believe a lot has already been said there. This blogpost about RedHat is relevant. To me personally it doesn't matter that one entity is a company and one isn't. Both are enabling a lot of people/companies to use open source work, which is great, but at the cost of maintainers' sanity.

I realize that I made external contributions quite hard by introducing Rust into the codebase. I believe a new maintainer is fully within their rights to just axe master and start from the 0.16 maintenance branch.

@untitaker untitaker pinned this issue Mar 17, 2019
@CounterPillow
Copy link

mpv is dealing with Debian stable users by being extremely hostile to anyone who uses outdated versions. It doesn't really make them any less frequent, but it provides a good source of stress relief. Individual support for getting something built on someone's Debian prehistoricstable Linux 2.6 box seems unwise from a use of developer time point of view; ignoring people like that may be difficult but may also just be the only choice to keep any sort of sanity intact.

@polyzen
Copy link

polyzen commented Mar 17, 2019

To quote @j605 on IRC, "I assume the best way to handle bug reports from debian stable users is to tell them to report it to debian and close it."

On a side note, sounds like mpv needs a Code of Conduct. :p

@Tonus1
Copy link
Contributor

Tonus1 commented Mar 17, 2019

That's really sad news for me.
As a Slackware user I fully understand your point. I really believe a distribution should keep things as close as upstream did.
I hope someone will get in : I do not have sufficient skills to help much...

@untitaker
Copy link
Member Author

untitaker commented Mar 17, 2019

@CounterPillow Releasing stress regularly like that drives off actual potential contributors and doesn't actually make anybody feel better. I know it doesn't make my stress go away. I did this and also observed people doing it with a few projects I've been involved in and I definetly don't want things to end this way for vdirsyncer. I totally get where you're coming from, but I just think it's morally wrong.

@CounterPillow
Copy link

Alright, understandable. Maybe I should do some self-reflection.

@WhyNotHugo
Copy link
Member

I agree with that pushing users away with hostility is unproductive and just hurts FLOSS more than anything else.

It probably makes sense to have a single page in the docs that politely explains that debian chooses to ignore upstream suggestions, and packages outdated versions, how it's a pain for devs to maintain debian packages in general, and how it's out of our hands. I'd just send the over there whenever getting a bug report for outdated version -- if they can build from source, great, but otherwise, there's just no resources to help them (again, politeness being key here).

A rust binary would have been great (but I still suck at rust). I wonder if a snap package would work, for all my dislike of things like snap, I still prefer it to debs, and it works for any distro.

@WhyNotHugo
Copy link
Member

Regarding maintainership, I still use vdirsyncer, but it's worked fine for me for so long, that I really don't need any changes to it. And I [regrettably] don't have much time to contribute on other's bugs any more.

I'm certainly willing to provide high level feedback and guidance though.

@teto
Copy link

teto commented Mar 24, 2019

Thanks for clearing up the situation. I do hope that someone steps up because I don't know of any similar project. Thanks for showing a way and having dealt with all this for as long as you did. I am sad to learn that webdav/caldav is so bad but seems like we have to stick with it. I completely get your frustration with packaging (I have hope for nix/nixos solutions, it lacks polish but the fondamentals are really good, like git at the beginning).

@jelmer
Copy link
Contributor

jelmer commented Mar 31, 2019

I would be interested in contributing the odd bug fix here or there, but I don't think I have enough bandwidth to take over maintainership by myself.

To make external contributions easier, I do think it would be better to revert to the pure python version of vdirsyncer.

@jreut
Copy link

jreut commented Apr 11, 2019

Hello, I'm also interested in contributing. I'm just starting to self-host some PIM servers, and I want to learn more about WebDAV and all its ilk.

I am up to help coordinate a small group of maintainers. Could we perhaps talk about this on the IRC channel?

@Mange
Copy link

Mange commented Jun 8, 2019

I had to stop using vdirsyner since it stopped compiling on Arch a few months ago. Came here to see if there was any update at all. I totally understand part of how you feel; I have a similar situation with software I maintain but haven't used myself anymore for years.

I could maybe help out with the Rust stuff, but not with any Python stuff. Nothing high-bandwidth, but doing trivial Rust fixes shouldn't be a problem. I tried helping out with #787 to get the compilation to pass, but it's not a project that is simple to get into, I'm afraid.

In either case, I hope you are able to put this behind you some day and do something you feel more passionate about. Do FOSS/OSS because it's fun or rewarding, and stop otherwise. :-)

@rEnr3n
Copy link
Contributor

rEnr3n commented Jun 13, 2019

Have you considered this? https://www.codeshelter.co/

@untitaker
Copy link
Member Author

untitaker commented Jun 13, 2019 via email

@bernhardreiter
Copy link
Contributor

Dear Markus (@untitaker)

Thanks!

Thanks for building up vdirsyncer as Free Software. It is important stopgap for me, to connect a few gui based calender applications. As a small contribution I've added your github account to my flattr list.

Handing over maintenanceship

It is very understandable that you hand over this initiative because you are not a user anymore,
lacking other motivations. (Assuming that only minor financial support is generated by it, because it is a small connection piece - though an important one).

It is good that you do so explicitly, while you are still around and somewhat active, so you can make sure that the next people taking care of vdirsyncer have a good chance to keeping users happy.

Bear with Debian (and other distros)

The task of Debian is to make a compliation of software available, in a stable way to many people
(and other distros). Of course with many people, there will be some that are not respectful to upstream products, but most Debian folks are.

In my view it is good and fine if Debian packages an old version of a Free Software product. It maybe very useful already and helps user that want their desktop to stay the same for at least 2-3 years. (This is a very short time span for well run desktops as users need to get accustomed to all the little things that are different in a major version of an desktop operating system). Once a software has a certain stability, it is sufficient to have a version that is a few years old. Even vdirsyncer has been useful in a few places in an old version to me. The good part about the stability is that many users from Debian and Ubuntu distributions can rely on it.

Guide people reporting

Most folks from my perspective try to do a good problem report. If they don't, then it is often because they do not know better. At least that is the best to assume. So providing more guidance often helps, and - yes - it is okay to demand a version number in the report or to politely say people to go to the Debian BTS (or the tracker of their distro or vendor) first or to get a newer version. Some Free Software initiative handle this nicely, even if the development power is small.

Python packaging

While packaging is important to make the installation of software a lot easier and to reach more people, I think that efforts should be limited here. It is more important that vdirsyncer works well. So python packaging is not to most easiest, but it is reasonably. And reasonably described by general documents. Providing something on pypi and being ready for pip from setuptools is a good line where you could stop as a developer. People will figure this out, distributions will pick it up.

Choice of development framework

As you write in your initial message I agree that selecting Rust is limiting the number of potential contributors a lot. Rust is a lot younger and less mature and widespread than Python. Also for vdirsyncer I believe the dynamic nature of Python is really helpful. It is possible to do little improvements to prove a theory about what works or what does not in a life installation. The barrier to me for contributing in python is a magnitude lower than for Rust.

Best wishes

.. for the future to @untitaker and to the new maintainers of vdirsyncer, once found!

Best Regards

@bernhardreiter
Copy link
Contributor

Groupware syncing protocol problems

Reading https://unterwaditzer.net/2015/kill-webdav.html I agree that Groupware syncing protocols and object formats are a real challenge. My experience comes from an involvement with the design, implementation of maintenance of https://en.wikipedia.org/wiki/Kolab in the major revisions 1 and 2 between 2002 and 2012. IMAP was used as a syncing protocol and it had some nice properties. Still up to today groupware object syncing is a very complex problem, often underestimated, with many broken or partial implementations. So interoperability is a mess and not in the interest of many powerful players in the market.

To me the lesson is to keep a product like vdirsyncer very modular, so that a community of people can help to at least fix one and another workaround in their specific server or client "adaptor" and use case. A dynamic language like Python is more suited toward such a neverending goal. (https://github.com/ytdl-org/youtube-dl/graphs/contributors is an example).

@untitaker
Copy link
Member Author

I don't have a lot to say in response to this, but I will respond to one point:

Of course with many people, there will be some that are not respectful to upstream products, but most Debian folks are.

I have had this experience with not one maintainer, but also another maintainer to whom I've tried to escalate this issue. Then there's also the xscreensaver issue which exhibits the same behavior of Debian. I don't think this is a personality trait of maintainers in Debian, but informal policy.

@bernhardreiter
Copy link
Contributor

@untitaker

I have had this experience with not one maintainer, but also another maintainer to whom I've tried to escalate this issue. Then there's also the xscreensaver issue which exhibits the same behavior of Debian. I don't think this is a personality trait of maintainers in Debian, but informal policy.

There are several points that can be discussed separately:

a) Is it okay to ship a Free Software product, if it is useful to users for your distribution? Even a modified product and remove some annoying edges?

I believe it is. Even if the maintainer disagrees. This is coming with the freedom that the holder of the exclusive rights has been giving the software product. Of course the packagers have to consider all arguments, but they may come to a different conclusion than the maintainers. Specific to vdirsyncer: I estimate packaging to Debian was a good thing and the drawback of annoying reports you've mentioned could have been addressed differently. (One possible solution is to add a statement that you are not answering direct emails with problem reports and then stop answering them. Here is an example for some email conditions that goes one step further https://dwheeler.com/contactme.html)

b) Is it okay to direct problem reports to upstream or a person that does not want them?

No. Ideally distributions and upstream cooperate to give sufficient hints towards the users that want to report. Note that Debian by default points people to their own issue tracker, not to the upstream one. In principle distributions care for their upstreams, so does Debian, there are many examples for a successful cooperation. :)

c) Shall someone be respectful and explain their position if there is a difference in opinion?

Yes. In the xscreensaver case, I've seen posts that were not respectful from both sides. I hope something like this hasn't happened to you and Debian's code of conduct - unsurprisingly - favors respectful cooperation. ;)

Again thanks for vdirsyncer!

@untitaker
Copy link
Member Author

I believe it is. Even if the maintainer disagrees

Well then you're agreeing with the course the maintainer of the vdirsyncer Debian package took, while your argument previously was that this was the action of a single individual.

I've seen posts that were not respectful from both sides

This is tone-policing, and especially irrelevant as JWZ never wished to participate in the Debian community. Also unsurprisingly the Debian CoC favors Debian's position because it does not cover the range of behavior that JWZ and I have a problem with.

I'm not interested in this discussion.

@bernhardreiter
Copy link
Contributor

JWZ is sending conflicting messages: When placing something under a Free Software license, he explicitely allows what Debian is doing. On the other hand he says he does not want it. Then he claims Debian would ship xscreensaver with security defects (which probably is not true as there have been backports) and he is unfriendly (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=819703#10) to people who propose to use the freedom he has allowed. Consistent behaviour would be to declare his software to be non-free or respect the decision of the Debian maintainers if he cannot convince them.

then you're agreeing with the course the maintainer of the vdirsyncer Debian package took

Sorry I've probably missunderstood your statement They did not care to be about b) or c). Thanks for clarifying.

@untitaker
Copy link
Member Author

untitaker commented Aug 9, 2019

@bernhardreiter, there is no conflict. JWZ's wish to not distribute outdated versions of his software is just not legally enforceable without making his software non-free.

It's not illegal to show somebody the middle finger, but obviously nobody wishes to be shown the finger. If I were to get aggressive because somebody showed me the middle finger, I wouldn't expect a response like "ah, I got mixed signals".

@untitaker
Copy link
Member Author

I'm actually locking this issue now because there have been multiple discussions in the context of xscreensaver which went exactly like this. There is no point in arguing.

@untitaker untitaker reopened this Aug 9, 2019
@pimutils pimutils locked as off-topic and limited conversation to collaborators Aug 9, 2019
@WhyNotHugo
Copy link
Member

Master is currently broken, and nobody has the time and knowledge to fix it. This is blocking further development and maintenance a lot.

I'm going to rename the current master to v0.17 (for archival purposes), and then switch master to the latest stable release.

In other words, development will continue off the latest v0.16 (which works). The next release will be named v0.18 to avoid confusion.

I'm sorry to have to do this, since it discards a lot of work that went into v0.17. However, as stated; nobody with the knowledge of how to continue work on this is contributing to maintenance any more, and it's very unlikely that someone will step up to maintain a broken master with overly complex changes. Rust is cool, but also set the bar for contributions too high.

If you have any objections, please do voice them, but given that development has essentially halted, I think this is the only way to move forward and will proceed with this today/tomorrow. This will also allow addressing #819 properly. It also allows addressing issue that exist in the latest stable release.

@untitaker
Copy link
Member Author

Thanks for stepping up here and I fully agree.

@WhyNotHugo
Copy link
Member

WhyNotHugo commented Jun 8, 2020

Items that need to be addressed to bring master up to date (I'll keep this updated as I fiddle with things):

  • Update code to work with recent click.
  • Update code to work with recent hypothesis.
  • Update code to work with recent xandikos.
  • Update code to work with recent radicale.
  • Update code to work with recent etesync.
  • Support recent Pythons.
  • Use pre-commit hooks for linting and alike.
  • Use a pre-commit hook to run script/make_travisconf.py.
  • Update flake8 to a recent version (after fixing all the new linting rules).
  • Ping Fastmail to get an update on the test account they're providing us.
  • Re-enable tests on OSX (not gonna deal with that until we're green on Linux).

Lots of code fails because dependencies have changed too much, so I'll be progressively upgrading them after I get master green again (need to pin lots of older versions of things first).

@pimutils pimutils unlocked this conversation Jun 8, 2020
@WhyNotHugo
Copy link
Member

Small update:

Since the last changes were in 2018, our code expects many dependencies from back then, and they're not pinned. Pinning them is a first step into getting master green, and I've just finished that.

Getting master green is very close now, mostly need a response from Fastmail for our test account with them to be renewed (they provide us a test account for free and CI uses that -- since they use Cyrus, it's very representative of what's out there in the wild too).

Next I want to centralise all test-dependency definitions in one place, and then slowly push them up to current versions, making changes to our code as necessary (and keeping master green).

@WhyNotHugo
Copy link
Member

@untitaker Can you give me collaborator permissions on PyPI?

@WhyNotHugo
Copy link
Member

Closing this.

Contributions still welcome, and I'll continue maintaining this since I still use it.

@WhyNotHugo WhyNotHugo unpinned this issue Nov 16, 2020
@bernhardreiter
Copy link
Contributor

@WhyNotHugo Thanks for having stepped up!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests