[LIMIT] Too many identical emails

[WhyNotHugo]

General informations

• system/distribution (with version): archlinux
• offlineimap version (offlineimap -V): offlineimap v8.0.0, imaplib2 v3.06, Python v3.10.8, OpenSSL 3.0.7 1 Nov 2022
• Python version: Python 3.10.8
• server name or domain: Fastmail
• CLI options: (None)

Configuration file offlineimaprc

[general]
pythonfile = ~/.local/lib/offlineimap-helper
metadata = ~/.local/share/offlineimap
accounts = Personal
maxsyncaccounts = 12
ui = basic
socktimeout = 30

### Personal Account ###

[Account Personal]
localrepository = LocalPersonal
remoterepository = RemotePersonal
postsynchook = /usr/bin/notmuch new
autorefresh = 0.5
quick = 10

[Repository LocalPersonal]
type = Maildir
localfolders = ~/.local/share/maildir/
restoreatime = no
createfolders = True

[Repository RemotePersonal]
type = IMAP
remotehost = imap.fastmail.com
remoteuser = ...
remotepasseval = ...
ssl = yes
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
maxconnections = 2
createfolders = False

pythonfile (if any)

Logs, error

Thread 'Folder Archive [acc: Personal]' terminated with exception:
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/offlineimap/threadutil.py", line 146, in run
    Thread.run(self)
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/site-packages/offlineimap/accounts.py", line 666, in syncfolder
    localfolder.syncmessagesto(remotefolder, statusfolder)
  File "/usr/lib/python3.10/site-packages/offlineimap/folder/Base.py", line 1164, in syncmessagesto
    action(dstfolder, statusfolder)
  File "/usr/lib/python3.10/site-packages/offlineimap/folder/Base.py", line 991, in __syncmessagesto_copy
    self.copymessageto(uid, dstfolder, statusfolder, register=0)
  File "/usr/lib/python3.10/site-packages/offlineimap/folder/Base.py", line 815, in copymessageto
    new_uid = dstfolder.savemessage(uid, message, flags, rtime)
  File "/usr/lib/python3.10/site-packages/offlineimap/folder/IMAP.py", line 728, in savemessage
    raise OfflineImapError(err_msg, OfflineImapError.ERROR.REPO)
offlineimap.error.OfflineImapError: Saving msg (<20221209165211.26pgrdgsutc7n44b@victory>) in folder 'Archive', repository 'RemotePersonal' failed (abort). Server responded: NO [b'[LIMIT] Too many identical emails']

Steps to reproduce the error

offlineimap

I don't know if something is wrong with the sync state or what.

[WhyNotHugo]

Last time I saw this I deleted my local state and local emails, and synced from scratch again.

Now I'm having the same error again:

Thread 'Folder Archive [acc: Personal]' terminated with exception:
Traceback (most recent call last):
  File "/home/hugo/clones/github.com/OfflineIMAP/offlineimap3/offlineimap/threadutil.py", line 146, in run
    Thread.run(self)
  File "/usr/lib/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "/home/hugo/clones/github.com/OfflineIMAP/offlineimap3/offlineimap/accounts.py", line 669, in syncfolder
    localfolder.syncmessagesto(remotefolder, statusfolder)
  File "/home/hugo/clones/github.com/OfflineIMAP/offlineimap3/offlineimap/folder/Base.py", line 1164, in syncmessagesto
    action(dstfolder, statusfolder)
  File "/home/hugo/clones/github.com/OfflineIMAP/offlineimap3/offlineimap/folder/Base.py", line 991, in __syncmessagesto_copy
    self.copymessageto(uid, dstfolder, statusfolder, register=0)
  File "/home/hugo/clones/github.com/OfflineIMAP/offlineimap3/offlineimap/folder/Base.py", line 815, in copymessageto
    new_uid = dstfolder.savemessage(uid, message, flags, rtime)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hugo/clones/github.com/OfflineIMAP/offlineimap3/offlineimap/folder/IMAP.py", line 728, in savemessage
    raise OfflineImapError(err_msg, OfflineImapError.ERROR.REPO)
offlineimap.error.OfflineImapError: Saving msg (<20221209165211.26pgrdgsutc7n44b@victory>) in folder 'Archive', repository 'RemotePersonal' failed (abort). Server responded: NO [b'[LIMIT] Too many identical emails']



Last 15 debug messages logged for Folder Archive [acc: Personal] prior to exception:
imap: __options2hash returning: {'FLAGS': '(\\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68883', 'INTERNALDATE': '" 6-Mar-2023 08:04:39 -0500"'}
imap: __options2hash returning: {'FLAGS': '(\\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68884', 'INTERNALDATE': '" 7-Mar-2023 08:53:26 -0500"'}
imap: __options2hash returning: {'FLAGS': '(\\Answered \\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68885', 'INTERNALDATE': '" 6-Mar-2023 09:11:52 -0500"'}
imap: __options2hash returning: {'FLAGS': '(\\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68886', 'INTERNALDATE': '" 7-Mar-2023 09:09:52 -0500"'}
imap: __options2hash returning: {'FLAGS': '(\\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68887', 'INTERNALDATE': '" 7-Mar-2023 23:03:29 -0500"'}
imap: __options2hash returning: {'FLAGS': '(\\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68888', 'INTERNALDATE': '" 8-Mar-2023 10:46:51 -0500"'}
imap: __options2hash returning: {'FLAGS': '(\\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68889', 'INTERNALDATE': '" 8-Mar-2023 11:28:17 -0500"'}
imap: __options2hash returning: {'FLAGS': '(\\Answered \\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68890', 'INTERNALDATE': '" 9-Mar-2023 04:33:52 -0500"'}
imap: __options2hash returning: {'FLAGS': '(\\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68891', 'INTERNALDATE': '" 9-Mar-2023 07:33:16 -0500"'}
imap: __options2hash returning: {'FLAGS': '(\\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68892', 'INTERNALDATE': '" 8-Mar-2023 12:06:15 -0500"'}
imap: __options2hash returning: {'FLAGS': '(\\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68893', 'INTERNALDATE': '" 8-Mar-2023 12:01:46 -0500"'}
imap: __options2hash returning: {'FLAGS': '(\\Seen $X-ME-Annot-2 $NotJunk)', 'UID': '68894', 'INTERNALDATE': '" 7-Mar-2023 09:43:18 -0500"'}
imap: Write mail 'Archive:-2' with flags {'S'}
: deletemessageheaders: called to delete []
: getmessageheader: called to get message-id

Looks like offlineimap ends up with its internal state inconsistent every once in a while, but I've no idea how to debug this further.

[chris001]

Hi @WhyNotHugo Thanks for raising this issue, sorry for delay in response.

1. Go to your fastmail web portal and see if you're able to access your email through their site. If you cannot, you'll need to reach out to them directly for more assistance. If you can access all of your fastmail mailboxes, proceed to the next step.
2. In the fastmail web portal, search your Archive folder for all emails with headers containing the text 20221209165211.26pgrdgsutc7n44b@victory. How many duplicates of that message does your Search find? If it's more than one message, then delete the duplicate messages, re-run offlineimap, and report back?
3. Run offlineimap, adding the command line option -1, to enable single-threaded mode, and report back?
4. Add to your config file, under the [general] section, fsync = True, and report back?

[WhyNotHugo]

I deleted the local copy of the mailbox and the state and sync again from scratch (and that worked), so I'm lead to think that aren't any duplicates on the remote side.

However, this has happened three times so far, so I suspect it will happen again. I'll try out your suggestions when it does.