go-imap v2 Backend interface work

.build.yml

@@ -1,19 +1,17 @@
 image: alpine/edge
 packages:
  - go
- # Required by codecov
- - bash
- - findutils
 sources:
  - https://github.com/emersion/go-imap
+artifacts:
+ - coverage.html
 tasks:
  - build: |
  cd go-imap
- go build -v ./...
+ go build -race -v ./...
  - test: |
  cd go-imap
  go test -coverprofile=coverage.txt -covermode=atomic ./...
- - upload-coverage: |
+ - coverage: |
  cd go-imap
- export CODECOV_TOKEN=8c0f7014-fcfa-4ed9-8972-542eb5958fb3
- curl -s https://codecov.io/bash | bash
+ go tool cover -html=coverage.txt -o ~/coverage.html

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,5 @@
+blank_issues_enabled: false
+contact_links:
+ - name: Question
+ url: "https://web.libera.chat/gamja/#emersion"
+ about: "Please ask questions in #emersion on Libera Chat"

.github/ISSUE_TEMPLATE/issue_template.md

@@ -0,0 +1,12 @@
+---
+name: Bug report or feature request
+about: Report a bug or request a new feature
+---
+
+<!--
+
+Please read the following before submitting a new issue:
+
+Do NOT create GitHub issues if you have a question about go-imap or about the IMAP protocol in general. Ask questions on IRC in #emersion on Libera Chat.
+
+-->

README.md

@@ -1,19 +1,14 @@
 # go-imap
 
-[![GoDoc](https://godoc.org/github.com/emersion/go-imap?status.svg)](https://godoc.org/github.com/emersion/go-imap)
+[![godocs.io](https://godocs.io/github.com/emersion/go-imap?status.svg)](https://godocs.io/github.com/emersion/go-imap)
 [![builds.sr.ht status](https://builds.sr.ht/~emersion/go-imap/commits.svg)](https://builds.sr.ht/~emersion/go-imap/commits?)
-[![Codecov](https://codecov.io/gh/emersion/go-imap/branch/master/graph/badge.svg)](https://codecov.io/gh/emersion/go-imap)
 
 An [IMAP4rev1](https://tools.ietf.org/html/rfc3501) library written in Go. It
 can be used to build a client and/or a server.
 
-```shell
-go get github.com/emersion/go-imap/...
-```
-
 ## Usage
 
-### Client [![GoDoc](https://godoc.org/github.com/emersion/go-imap/client?status.svg)](https://godoc.org/github.com/emersion/go-imap/client)
+### Client [![godocs.io](https://godocs.io/github.com/emersion/go-imap/client?status.svg)](https://godocs.io/github.com/emersion/go-imap/client)
 
 ```go
 package main
@@ -71,7 +66,7 @@ func main() {
  from := uint32(1)
  to := mbox.Messages
  if mbox.Messages > 3 {
- // We're using unsigned integers here, only substract if the result is > 0
+ // We're using unsigned integers here, only subtract if the result is > 0
  from = mbox.Messages - 3
  }
  seqset := new(imap.SeqSet)
@@ -96,7 +91,7 @@ func main() {
 }
 ```
 
-### Server [![GoDoc](https://godoc.org/github.com/emersion/go-imap/server?status.svg)](https://godoc.org/github.com/emersion/go-imap/server)
+### Server [![godocs.io](https://godocs.io/github.com/emersion/go-imap/server?status.svg)](https://godocs.io/github.com/emersion/go-imap/server)
 
 ```go
 package main
@@ -128,6 +123,24 @@ func main() {
 
 You can now use `telnet localhost 1143` to manually connect to the server.
 
+## Extensions
+
+Support for several IMAP extensions is included in go-imap itself. This
+includes:
+
+* [APPENDLIMIT](https://tools.ietf.org/html/rfc7889)
+* [CHILDREN](https://tools.ietf.org/html/rfc3348)
+* [ENABLE](https://tools.ietf.org/html/rfc5161)
+* [IDLE](https://tools.ietf.org/html/rfc2177)
+* [IMPORTANT](https://tools.ietf.org/html/rfc8457)
+* [LITERAL+](https://tools.ietf.org/html/rfc7888)
+* [MOVE](https://tools.ietf.org/html/rfc6851)
+* [SASL-IR](https://tools.ietf.org/html/rfc4959)
+* [SPECIAL-USE](https://tools.ietf.org/html/rfc6154)
+* [UNSELECT](https://tools.ietf.org/html/rfc3691)
+
+Support for other extensions is provided via separate packages. See below.
+
 ## Extending go-imap
 
 ### Extensions
@@ -136,18 +149,12 @@ Commands defined in IMAP extensions are available in other packages. See [the
 wiki](https://github.com/emersion/go-imap/wiki/Using-extensions#using-client-extensions)
 to learn how to use them.
 
-* [APPENDLIMIT](https://github.com/emersion/go-imap-appendlimit)
 * [COMPRESS](https://github.com/emersion/go-imap-compress)
-* [ENABLE](https://github.com/emersion/go-imap-enable)
 * [ID](https://github.com/ProtonMail/go-imap-id)
-* [IDLE](https://github.com/emersion/go-imap-idle)
 * [METADATA](https://github.com/emersion/go-imap-metadata)
-* [MOVE](https://github.com/emersion/go-imap-move)
 * [NAMESPACE](https://github.com/foxcpp/go-imap-namespace)
 * [QUOTA](https://github.com/emersion/go-imap-quota)
 * [SORT and THREAD](https://github.com/emersion/go-imap-sortthread)
-* [SPECIAL-USE](https://github.com/emersion/go-imap-specialuse)
-* [UNSELECT](https://github.com/emersion/go-imap-unselect)
 * [UIDPLUS](https://github.com/emersion/go-imap-uidplus)
 
 ### Server backends
@@ -156,6 +163,7 @@ to learn how to use them.
 * [Multi](https://github.com/emersion/go-imap-multi)
 * [PGP](https://github.com/emersion/go-imap-pgp)
 * [Proxy](https://github.com/emersion/go-imap-proxy)
+* [Notmuch](https://github.com/stbenjam/go-imap-notmuch) - Experimental gateway for [Notmuch](https://notmuchmail.org/)
 
 ### Related projects
 

backend/appendlimit.go

@@ -0,0 +1,29 @@
+package backend
+
+import (
+ "errors"
+)
+
+// An error that should be returned by User.CreateMessage when the message size
+// is too big.
+var ErrTooBig = errors.New("Message size exceeding limit")
+
+// A backend that supports retrieving per-user message size limits.
+type AppendLimitBackend interface {
+ Backend
+
+ // Get the fixed maximum message size in octets that the backend will accept
+ // when creating a new message. If there is no limit, return nil.
+ CreateMessageLimit() *uint32
+}
+
+// A user that supports retrieving per-user message size limits.
+type AppendLimitUser interface {
+ User
+
+ // Get the fixed maximum message size in octets that the backend will accept
+ // when creating a new message. If there is no limit, return nil.
+ //
+ // This overrides the global backend limit.
+ CreateMessageLimit() *uint32
+}

backend/backendutil/search.go

@@ -10,6 +10,7 @@ import (
  "github.com/emersion/go-imap"
  "github.com/emersion/go-message"
  "github.com/emersion/go-message/mail"
+ "github.com/emersion/go-message/textproto"
 )
 
 func matchString(s, substr string) bool {
@@ -41,16 +42,28 @@ type lengther interface {
  Len() int
 }
 
+type countWriter struct {
+ N int
+}
+
+func (w *countWriter) Write(b []byte) (int, error) {
+ w.N += len(b)
+ return len(b), nil
+}
+
 func bodyLen(e *message.Entity) (int, error) {
+ headerSize := countWriter{}
+ textproto.WriteHeader(&headerSize, e.Header.Header)
+
  if l, ok := e.Body.(lengther); ok {
- return l.Len(), nil
+ return l.Len() + headerSize.N, nil
  }
 
  b, err := bufferBody(e)
  if err != nil {
  return 0, err
  }
- return b.Len(), nil
+ return b.Len() + headerSize.N, nil
 }
 
 // Match returns true if a message and its metadata matches the provided
@@ -66,12 +79,12 @@ func Match(e *message.Entity, seqNum, uid uint32, date time.Time, flags []string
  if err != nil {
  return false, err
  }
- t = t.Round(24 * time.Hour)
+ t = time.Date(t.Year(), t.Month(), t.Day(), 0, 0, 0, 0, time.UTC)
 
  if !c.SentBefore.IsZero() && !t.Before(c.SentBefore) {
  return false, nil
  }
- if !c.SentSince.IsZero() && !t.After(c.SentSince) {
+ if !c.SentSince.IsZero() && t.Before(c.SentSince) {
  return false, nil
  }
  }
@@ -104,8 +117,17 @@ func Match(e *message.Entity, seqNum, uid uint32, date time.Time, flags []string
  }
  }
  for _, text := range c.Text {
- // TODO: also match header fields
- if ok, err := matchBody(e, text); err != nil || !ok {
+ headerMatch := false
+ for f := e.Header.Fields(); f.Next(); {
+ decoded, err := f.Text()
+ if err != nil {
+ continue
+ }
+ if strings.Contains(f.Key()+": "+decoded, text) {
+ headerMatch = true
+ }
+ }
+ if ok, err := matchBody(e, text); err != nil || !ok && !headerMatch {
  return false, err
  }
  }
@@ -194,7 +216,10 @@ func matchSeqNumAndUid(seqNum uint32, uid uint32, c *imap.SearchCriteria) bool {
 }
 
 func matchDate(date time.Time, c *imap.SearchCriteria) bool {
- date = date.Round(24 * time.Hour)
+ // We discard time zone information by setting it to UTC.
+ // RFC 3501 explicitly requires zone unaware date comparison.
+ date = time.Date(date.Year(), date.Month(), date.Day(), 0, 0, 0, 0, time.UTC)
+
  if !c.Since.IsZero() && !date.After(c.Since) {
  return false
  }

backend/backendutil/search_test.go

@@ -378,9 +378,9 @@ func TestMatchIssue298Regression(t *testing.T) {
  t.Fatal("Expected no error while reading entity, got:", err)
  }
 
- // Search for body size > 1 ("LARGER 1"), which should match messages #2 and #3
+ // Search for body size > 15 ("LARGER 15"), which should match messages #2 and #3
  criteria := &imap.SearchCriteria{
- Larger: 1,
+ Larger: 15,
  }
  ok1, err := Match(e1, 1, 101, time.Now(), nil, criteria)
  if err != nil {
@@ -404,9 +404,9 @@ func TestMatchIssue298Regression(t *testing.T) {
  t.Errorf("Expected message #3 to match search criteria")
  }
 
- // Search for body size < 3 ("SMALLER 3"), which should match messages #1 and #2
+ // Search for body size < 17 ("SMALLER 17"), which should match messages #1 and #2
  criteria = &imap.SearchCriteria{
- Smaller: 3,
+ Smaller: 17,
  }
  ok1, err = Match(e1, 1, 101, time.Now(), nil, criteria)
  if err != nil {

backend/mailbox.go

@@ -1,8 +1,6 @@
 package backend
 
 import (
- "time"
-
  "github.com/emersion/go-imap"
 )
 
@@ -12,26 +10,17 @@ type Mailbox interface {
  // Name returns this mailbox name.
  Name() string
 
+ // Closes the mailbox.
+ Close() error
+
  // Info returns this mailbox info.
  Info() (*imap.MailboxInfo, error)
 
- // Status returns this mailbox status. The fields Name, Flags, PermanentFlags
- // and UnseenSeqNum in the returned MailboxStatus must be always populated.
- // This function does not affect the state of any messages in the mailbox. See
- // RFC 3501 section 6.3.10 for a list of items that can be requested.
- Status(items []imap.StatusItem) (*imap.MailboxStatus, error)
-
- // SetSubscribed adds or removes the mailbox to the server's set of "active"
- // or "subscribed" mailboxes.
- SetSubscribed(subscribed bool) error
-
- // Check requests a checkpoint of the currently selected mailbox. A checkpoint
- // refers to any implementation-dependent housekeeping associated with the
- // mailbox (e.g., resolving the server's in-memory state of the mailbox with
- // the state on its disk). A checkpoint MAY take a non-instantaneous amount of
- // real time to complete. If a server implementation has no such housekeeping
- // considerations, CHECK is equivalent to NOOP.
- Check() error
+ // Poll requests any pending mailbox updates to be sent.
+ //
+ // Argument indicates whether EXPUNGE updates are permitted to be
+ // sent.
+ Poll(expunge bool) error
 
  // ListMessages returns a list of messages. seqset must be interpreted as UIDs
  // if uid is set to true and as message sequence numbers otherwise. See RFC
@@ -44,19 +33,11 @@ type Mailbox interface {
  // uid is set to true, or sequence numbers otherwise.
  SearchMessages(uid bool, criteria *imap.SearchCriteria) ([]uint32, error)
 
- // CreateMessage appends a new message to this mailbox. The \Recent flag will
- // be added no matter flags is empty or not. If date is nil, the current time
- // will be used.
- //
- // If the Backend implements Updater, it must notify the client immediately
- // via a mailbox update.
- CreateMessage(flags []string, date time.Time, body imap.Literal) error
-
  // UpdateMessagesFlags alters flags for the specified message(s).
  //
  // If the Backend implements Updater, it must notify the client immediately
  // via a message update.
- UpdateMessagesFlags(uid bool, seqset *imap.SeqSet, operation imap.FlagsOp, flags []string) error
+ UpdateMessagesFlags(uid bool, seqset *imap.SeqSet, operation imap.FlagsOp, silent bool, flags []string) error
 
  // CopyMessages copies the specified message(s) to the end of the specified
  // destination mailbox. The flags and internal date of the message(s) SHOULD
@@ -75,4 +56,10 @@ type Mailbox interface {
  // If the Backend implements Updater, it must notify the client immediately
  // via an expunge update.
  Expunge() error
+
+ // Idle allows backend to send updates without explicit Poll calls or any other
+ // commands running.
+ // When called - it should block indefinitely and return immediately when
+ // done channel is written to.
+ Idle(done <-chan struct{})
 }