Enable storing TBSync/Exchange address books in the CardBook add-on for Thunderbird

[ovari]

Your environment

TbSync version: 0.7.10
Thunderbird version: 59.9.1

[ ] Yes, I have installed the latest available (beta) version from
https://github.com/jobisoft/TbSync/releases
and my issue is not yet fixed, I can still reproduce it.

Expected behavior

Enable contacts to be stored in the CardBook add-on.
https://gitlab.com/CardBook/CardBook
https://addons.thunderbird.net/addon/cardbook/
https://gitlab.com/CardBook/CardBook/issues/317

Actual behavior

Contacts are stored in the default Thunderbird Address Book

Could help solve #49

[jobisoft]

I have this on my list and I am in contact with the author of cardbook. Cardbook does not yet have the req. interface to manage an address book externally. This will be integrated eventually. John

[laclaro]

+1

[CardBook]

what do you need ?

[jobisoft]

time :-) This is the next to next big feature scheduled for implementation. I think from your side everything is in place. If not, I will knock. I want to get this done in 2018...

[jobisoft]

@CardBook I have started to think about the actual implementation and I probably will add an intermediate vCard layer. That means all TbSync provider have to map their contact source data into a vCard which has a fixed mapping to the standard Thunderbird address book. (Instead of mapping the properties directly to Thunderbird address book).

If the user selects CardBook as sync destination, I simply pass on that vCard to you. That part should be easy. But the other sync direction has some problems: CardBook allows to add much more data to the vCard which I will not be able to sync. For example, if EAS allows 3 emails, I can map them to WORK HOME and OTHER (or whatever). If the user now adds a 4th email of type WORK, I will not know how to sync that.

I came up with the following solutions:

1: during the creation of the cardbook address book I can define what values are allowed (and how often), cardbook must than obey these definitions

2: vCards in a cardbook adressbook, which have been created by TbSync are not allowed to add/delete options at all, but have all allowed fields present and "empty" fields (which the user might add) are already visible in the edit view as empty fields, but of course not in the standard view. During the creation of the cardbook address book I also define a "template" vCard, which is used by CardBook to create new cards, if the user is adding a new contact.

Your thoughts?

[jobisoft]

@CardBook I think for both suggestions one could use the "template vCard" to define allowed options. Either CardBook has to obey them when the user want to add options, or it has to prefill them and disable addition and deletion of options in the UI ...

And if I do not provide such a template, there should be no restrictions.

Also it would be nice to be able to provide an icon / type so the user can see, what cardbook book is synced by TbSync.

[ovari]

Suggestion:
Add an “EAS” Address book type under the CardBook Preferences → “Types” tab → “Address book” options.

Currently the options are shown as radio buttons being:
Google, Apple, Yahoo!, Other
After TBSync the list could become:
Apple, EAS, Google, Yahoo!, Other
with

• the corresponding images (aka icon/type as requested by @jobisoft)
• a drop-down
• the list being alphabetically sorted

What do you think?

[jobisoft]

From my point of view, that would be the wrong approach. Setup and management of the syncing is domain of TbSync. CardBook is just the "storage" of the cards, the same as it now done with the standard Thunderbird address book.

[jobisoft]

I am looking for a generic solution where CardBook does not need to know anything about the additional providers introduced by TbSync. I also want to put as litte work on Philippe as possible and I do not want him to have to change stuff just because I added a new provider.

So to sum this up:

• I sync cards in a "local" CardBook book and sync changes made there back to the server
• for Cardbook, this is just a local book
• when creating such local books in Cardbook, I would like to specify an image and a description, so the user knows, that it is actually managed by TbSync
• we could also think about a button in CardBook, which opens the connected account in the TbSync AccountManger for quick access to the settings
• we need some sort of restriction to the CardBook UI so the user can only set/add vCard Options that can actually be synced by the TbSync Provider (see comment above)

[CardBook]

I will think a bit more on this, and come back to you very soon...

[CardBook]

Hi John

what I'm sure :

• I don't want to handle|learn|manage the EAS sync
• I'm OK to provide a template (or something) to limit the number of fields in edition

what I'm not sure :

• a LOCAL address book may not be a good idea... I prefer to create a new type "EAS"
• for the address book edition, don't know how to have both the TbSync parameters and CardBook parameters (display name formula for example)
• are there categories with EAS ?

[jobisoft]

Yes exactly, I want to keep the sync code inside TbSync, so I am all with you.

I would not propose an EAS Type book, more of a TbSync type (if you want to create a new one). I will add more providers and I would not like you to have to adapt every time. The interface will be the same for all providers (Icon, description, template and vCards as input/output).

All my providers will support categories and the standard for vCards is to store them in the CATEGORIES field seperated by \u001A. If you need the categories differently, then I will of course send them as you need them in the vCard.

The way TbSync works, it does not allow to change settings of an active book/calendar. One has to disable the account, change settings and re-enable. I implemented this to minimize sync glitches. By disabling the account in TbSync, all connected books in CardBook will be removed and re-created after the account got re-enabled.

I do not know, what settings you want to add by yourself, but from my point of view, a click on the "settings" of a TbSync book in Cardbook should simply open the TbSync Account Manager.

[jobisoft]

Hi Philippe,

I just want to point out, that my main objective is that you have as litte work as possible. We will need the template to limit UI options. Everything else (Icon and description, extra book type) is nice to have but not mandetory. Thanks a lot for your time.

[CardBook]

Hi John

I'm very busy for the next two weeks (I'm trying to add some kind of autoconfig to CardBook)... then I would be back for achieving this :O)

[jobisoft]

Take your time, I am here :-)

[jobisoft]

A side note: If you allow this feature, this would enable ALL my providers in CardBook, even my sabre/dav provider: https://addons.thunderbird.net/de/thunderbird/addon/dav-4-tbsync/

This is part of my "bigger picture" to have a central sync manager for users to set up their sync accounts once (!) and get everything: calendars, todos and contatcs and later maybe even email (if supported by the provider). With dav, currently users have to setup the same account twice (with sogo-connector, or CardBook and lightning).

If we enable CardBook as an alternative storage location for TbSync, the user will get the choice, to use your dav syncing code (and setup everything twice), or my syncing code.

I created the dav provider as a proof of concept for my TbSync API and also as a replacement for the sogo-connector, because one of my clients demanded it.

In the long run, I will also build (or help to build) a Google-provider and an EWS-provider. The idea behind the TbSync API is to reduce redundancy for sync addons and allow other authors to write sync providers for TbSync, so I do not have to maintain them all.

I do not know what your other plans are. I know coding time is expensive and time itself is precious, so we should try to avoid double implementations. With my "bigger picture" in mind, you could concentrate your efforts on the core address book part and not so much on the syncing part of CardBook.

For example, TbSync has already a full implementation of the sabre/dav discovery service, I do not know if this is what you referred to as "I'm trying to add some kind of autoconfig to CardBook".

Best regards,
John

[CardBook]

I have no objection in letting the user choose which sync they may use :O) (CardBook also implements the sabre/dav discovery service)...

but for now it seems to me a big work to integrate EAS, so let's start by this one and the others would be easier...

for the autoconfig in CardBook I meant : if you use john@gmx.de as a mail account, CardBook will ask a repository to get the gmx.de CardDAV URL and then propose the user to add the remote gmx.de address book...

[jobisoft]

ok, i felt it was important to share where I want to go with TbSync.

If we use my dav provider as a first test, than we of course do not yet need the template / UI restriction stuff. Do you want me to go ahead and start coding with your current local address book (which should work), or do you want to add a TbSync type book first, where I can specify icon, description, and the URL to the TbSync settings window (for that book)?

[CardBook]

Hi
I think we should work on adding a TbSync type book first : and for this I'll have to clean my code... I start this week the clean-up...

[CardBook]

another question : do you plan to work on a LDAP sync (this sync is missing for the moment) ?

[jobisoft]

LDAP is somewhat special, it is an address book only protocol, so it does not really fit into TbSync.

I think, I would suggest to add native LDAP support directly in CardBook. Comparable to what the old address book supports.

Does LDAP support autodiscover of some sort, which could lookup the avail LDAP books of a user? If that is the case, we could add a TbSync "wrapper" provider, which is doing the autodiscover and than just set up the found books in CardBook or the TB address book. Both address book implementations would benefit from that. I do something similar with CalDAV: My dav provider is doing the autodiscover for address books and calendars. The address books are synced by the provider but the calendars are simply set up in lightning and lightning is doing the sync.

What do you think?

[CardBook]

I don't know how works LDAP...

[jobisoft]

If the CardBook integration is done, there is of course the option to write a full LDAP provider for TbSync, which also brings LDAP support to CardBook, but due to very little time and lots of projects, I will not be the one coding that. But someone else can, once the API is finalized and CardBook support is integrated.

[ovari]

I want to get this done in 2018.

@jobisoft could you please provide an update? Thank you

[jobisoft]

I am currently underfunded, so I cannot spend all the time I have on this.

[dl5rcw]

I just ran into the difficulties with TBSync in parallel of CardBook. I must say: I love both addons! Looking forward to get these working well with each other! Please keep on it! Unfortunately, both in parallel seem to break contacts right now (current stable TB + TBSync + CardBook, connecting on Tine20).

[jobisoft]

I am currently sorting out all known UI glitches, then there are a few reported bugs which must be fixed and then I will do cardbook.

If you like TbSync, consider supporting me via donations:
#107 (comment)

[jobisoft]

Unfortunately, both in parallel seem to break contacts right now (current stable TB + TBSync + CardBook, connecting on Tine20).

Can you elaborate that more? At current, there is no connection between TbSync and Cardbbook, they act on different storages.

How do they interfere? What is breaking?

[dl5rcw]

When I run TBSync with addressbooks enabled, CardBook does not find my carddav storage. When i disable TBSync, Cardbook resolves TBSync w/o any problems. I can use CardBook then. I added 'remote address book'. It did resolv nicely with TBSync being disabled. I tried to 'trick' them, so after grabbing the CardDav via Cardbook, I enabled the TBSync - but now I got sync errors by TBSync.

URL:
https://lab.lars-engelhard.com/tine/addressbooks (PROPFIND)

Request:
<d:propfind xmlns:d="DAV:"><d:prop><d:current-user-principal /></d:prop></d:propfind>

Response:

Once I disable CardBook again, everything works fine. The above error disappears.

CardDav and CalDav are running on multi-user Tine20 account with multiple devices. All sync in parallel w/o any problems.
Problem occurs on three laptops running latest Windows.
Tine20: latest revision
TBSync: latest revision
CardBook latest revision

All fine on other CalDav/CardDav clients, ranging from Android to iPads, iPhones, ... Reading this threat, I thought this will resolve such issues b/c both modules will create stronger dependencies.

[jobisoft]

Oh. The actual connection to the server does not work, if TbSync and CardBook are connected to the same server?

Could I have a test account on that server, so I can try to reproduce and understand that? Credentials via email.

Are you using a full resource URL in CardBook, or just the server URL as in TbSync?

[dl5rcw]

tine20 is installed in a subdear of the server, i am using something like https://intra.example.com/tine20/, it is grabbing through the .well-known/carddav and .well-known/caldav, both a redirection to the /tine20/addressbooks and /tine20/calendars endpoints.
I am clarifying if/how we could provide a temporary test account, but it might take a while.

[jobisoft]

FYI: I will be able to work on this next Tuesday or next Friday (these are my coding days).

[laclaro]

May I ask about this issue?

[jobisoft]

There will be a huge change in address book code for TB78. I will not invest time before I know how this will turn out.

[pgnd]

@jobisoft

Now that 78's out, any further thoughts on this to share?

I'm particularly interested in what further's planned, if anything, for integration with (my fave) CardBook.

[Blackclaws]

I'd be interested in this as well as the normal address book does not offer things like groups which cardbook handles

[xthursdayx]

Same, wondering about using Thunderbird's native address book vs trying to use it with cardbook.

[thmo]

Thunderbird 91 now supports CardDav - how does that relate here? (maybe not at all?)

[jobisoft]

If built-in CardDAV support is as good as my DAV provider, I will drop it. Otherwise, I currently do not work on direct CardBook support for my EAS provider.

[jobisoft]

I will not have the man power to work on this. It is no longer on my agenda.