fix: Order of messages if Sentbox is synced before Inbox

[iequidoo]

See commit messages.

My devices go online/offline independently and i use both to send messages, so often i see my messages wrongly ordered around incoming messages.

[iequidoo]

The test i added here isn't needed, there's test_old_message_5() already which does the same, but instead records the current (wrong) behaviour. Going to fix it then. EDIT: Done

[iequidoo]

Btw, i think it may be useful to display these outgoing received messages (i.e. from other MUAs/devices) somehow differently e.g. align them to the left as incoming ones. As they still have another brightness (even in Dark mode), it shouldn't be a problem to differ them from incoming messages. May be even useful for shared/community accounts, not sure. CC @adbenitez

[iequidoo]

Recently i started to see my messages weirdly ordered around incoming ones every day when i open my laptop lid (i.e. i see my replies above incoming messages). I don't know whether Delta Chat started scheduling smth differently or i changed its usage pattern, but this PR should help it seems.

[Hocuri]

The PR description is quite short, the PR is easier to understand if the PR description has a short explanation in which way the logic was changed.

Btw, i think it may be useful to display these outgoing received messages (i.e. from other MUAs/devices) somehow differently e.g. align them to the left as incoming ones. As they still have another brightness (even in Dark mode), it shouldn't be a problem to differ them from incoming messages.

I'm pretty sure that non-technical users would find this super confusing.

[Hocuri]

So, this means that this is a breaking change (i.e. will need UI adaptions)? If so, this PR should start with fix! instead of fix, and the PR description should mention this.

And, it means that outgoing messages that were sent by a different device won't have any checkmarks anymore? I personally don't think that's a good idea because that's not how users know it from other messengers, but if others think otherwise, I'd be happy to be outvoted / convinced otherwise.

[iequidoo]

No, it shouldn't be a breaking change, i forgot to map OutRcvd to OutDelivered in all APIs. Fixed this.

EDIT: Only for REPL the display of received outgoing messages changes.

[iequidoo]

The PR description is quite short, the PR is easier to understand if the PR description has a short explanation in which way the logic was changed.

I put everything to commit messages to avoid updating the PR description every time they change. Probably it's better to leave the PR description empty then or just reference the commit messages.

Btw, i think it may be useful to display these outgoing received messages (i.e. from other MUAs/devices) somehow differently e.g. align them to the left as incoming ones. As they still have another brightness (even in Dark mode), it shouldn't be a problem to differ them from incoming messages.

I'm pretty sure that non-technical users would find this super confusing.

Probably yes, though for shared/community accounts this may be useful, but then also we need to check for presence of override_sender_name to avoid confusion for other users. Anyway currently in the message Info the user can check if it's a sent or received outgoing message, so my proposal doesn't add smth new, this is only a question of usability.

[r10s]

closing stale pr, to avoid more resources put in here

from the issue description, it is eg. about since years discouraged "shared account usage" with eg. gmail, which we do not want to care for, let alone introduce a state that is otherwise not needed.

also, we do not want to continue "upload messages via imap" this is about to be removed at some point. same for "watch sent folder" and maybe other dependencies on imap standing in the way of thinking about new ways of transport and resilience (as they cost resources, time and money we do not have)

[iequidoo]

The issue is about Delta Chat -only usage with Gmail, but with "Watch Sent folder" enabled (i quite rarely use other clients, so it's mostly enabled just in case). I.e. while the option is enabled for "shared usage", it makes harm for Delta Chat -only usage, sometimes i just can't understand my previous conversations when i open my laptop lid because all messages are ordered almost randomly. "Save messages to savebox" ("Sent" folder actually) is Gmail's behavior, we can't do anything with it. Gmail's "Sent" folder is actually a label and all messages you send are duplicated there (they even have the same Message-ID).

I'll try to reproduce it w/o "Watch Sent folder" (already tried to disable it, but forgot whether that helped). It's probably difficult to catch the bug w/o "Watch Sent folder":

• Device A must go sleep immediately after you sent a reply, so the incoming message is moved to "Delta Chat", but not the reply.
• Device B should fetch "Inbox" first and "Delta Chat" after that, so the reply will be "higher" in the chat.

But apart from this the PR fixes ordering messages for "shared account usage" because in this case received outgoing messages (which may be sent by another user/device) should be sorted the same way as incoming ones and defenitely not break sorting of them. Currently, if we receive an outgoing message, it works as a barrier for incoming messages (even if the chat isn't open at the moment), so they can't be ordered purely by their "Date".

[r10s]

It's probably difficult to catch the bug w/o "Watch Sent folder"

thanks for confirming.

i clarified the post above bit.

[link2xt]

Related issue seemingly describing the same problem: deltachat/deltachat-android#3900

[r10s]

"Watch Sent folder" was always a questionable option of questionable worth. we were thinking about removing "Watch Sent folder" already years ago, and left it, as it "did not hurt". but that has changed now, we should remove the option alltogether now

that will remove one footgun of ppl going for shared account usage, while at the same time removing complexity and effort for us

[iequidoo]

It's probably difficult to catch the bug w/o "Watch Sent folder":

• Device A must go sleep immediately after you sent a reply, so the incoming message is moved to "Delta Chat", but not the reply.

• Device B should fetch "Inbox" first and "Delta Chat" after that, so the reply will be "higher" in the chat.

It's impossible to reproduce the bug just this way because imap::fetch_new_messages() only fetches messages that have reached their destination folder, i.e. "Delta Chat", so device B must also have MvboxMove disabled. I reproduced it this way. Having MvboxMove set differently on devices is a questionable configuration though, but it's possible.

So the summary is:

• We need to remove SentboxWatch, it's definitely harmful for Gmail (even in Delta Chat -only scenario) and as for other providers, it's better to tell users that they should use To/CC/BCC to themselves to see sent messages in Delta Chat.
• Removing MvboxMove would be also good, this way we also save more battery.
• We should discourage users from sharing the same account accross many people (another kind of "shared account usage") because in this case an outgoing message sent by another person from another device must be sorted like an incoming one, but this can't be done using the current set of message states.