Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reply issue by email #13585

Closed
wants to merge 39 commits into from
Closed
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
404ae49
reply issue by email
a1012112796 Nov 16, 2020
61b9c64
fix vendor
a1012112796 Nov 16, 2020
539421d
Merge branch 'master' into imap
a1012112796 Nov 17, 2020
ca8a716
fix charsets
a1012112796 Nov 17, 2020
30d3479
Merge branch 'master' into imap
a1012112796 Nov 17, 2020
d5dac5d
Merge branch 'master' into imap
a1012112796 Nov 18, 2020
c95f015
simplify logic
a1012112796 Nov 18, 2020
e357517
Merge branch 'master' into imap
a1012112796 Nov 18, 2020
e6ada29
fix test
a1012112796 Nov 18, 2020
4800de2
Merge branch 'master' into imap
a1012112796 Nov 22, 2020
5b5fb6c
Merge branch 'master' into imap
a1012112796 Nov 29, 2020
77cedeb
Merge branch 'master' into imap
a1012112796 Jan 13, 2021
074e6d2
translate
a1012112796 Jan 14, 2021
4baedb5
Merge branch 'master' into imap
a1012112796 Jan 14, 2021
08345f0
add missing config example
a1012112796 Jan 14, 2021
560f48a
Merge branch 'master' into imap
a1012112796 Jan 19, 2021
f30caf9
test code 1 and fix some logic
a1012112796 Jan 19, 2021
f076123
Merge branch 'master' into imap
a1012112796 Jan 19, 2021
b00df2e
lint
a1012112796 Jan 19, 2021
47df1c9
fix bug
a1012112796 Jan 19, 2021
2923155
Merge branch 'main' into imap
a1012112796 May 7, 2021
4176a92
fix bug
a1012112796 May 7, 2021
e881e4c
Merge branch 'main' into imap
a1012112796 Nov 12, 2021
cffb646
apply suggestions from code review
a1012112796 Nov 13, 2021
99921b3
Merge branch 'main' into imap
a1012112796 Nov 13, 2021
65c26ca
fix lint
a1012112796 Nov 13, 2021
68c69bb
Merge branch 'main' into imap
a1012112796 Nov 14, 2021
4069ceb
fix ci and remove test code before finish it
a1012112796 Nov 14, 2021
3cfe21e
Merge branch 'main' into imap
a1012112796 Nov 14, 2021
325392e
upgrade go-imap v1.0.6 -> v1.2.0
a1012112796 Nov 14, 2021
5589c19
fix lint
a1012112796 Nov 14, 2021
e1948eb
Merge branch 'main' into imap
a1012112796 Nov 17, 2021
fe2f5e6
test code 1
a1012112796 Nov 17, 2021
9670bc1
fmt
a1012112796 Nov 18, 2021
0fc7000
Merge branch 'main' into imap
a1012112796 Nov 18, 2021
f0d9b54
fix lint
a1012112796 Nov 18, 2021
0bad64a
Merge branch 'main' into imap
a1012112796 Nov 18, 2021
bbb2513
remove not passd test before fix
a1012112796 Nov 18, 2021
af5a825
fmt
a1012112796 Nov 18, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions custom/conf/app.example.ini
Original file line number Diff line number Diff line change
Expand Up @@ -792,6 +792,24 @@ SENDMAIL_ARGS =
; Timeout for Sendmail
SENDMAIL_TIMEOUT = 5m

[mail_recive]
ENABLED = false
; Buffer length of channel, keep it as it is if you don't know what it is.
READ_BUFFER_LEN = 100
; email address to recive mail
RECIVE_EMAIL =
; recive email box
RECIVE_BOX = INBOX
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
; Mail server
; Gmail: imap.gmail.com:993
; QQ: imap.qq.com:993
HOST =
USER =
PASSWD =
IS_TLS_ENABLED = true
; delete mail after read
DELETE_RODE_MAIL = false
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved

[cache]
; if the cache enabled
ENABLED = true
Expand Down Expand Up @@ -1153,6 +1171,14 @@ NO_SUCCESS_NOTICE = false
SCHEDULE = @every 168h
OLDER_THAN = 8760h

; fetch unread mails
[cron.imap_fetch_mails]
; when [mail_recive] is enable, this will auto enable also
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
ENABLED = false
RUN_AT_START = false
NO_SUCCESS_NOTICE = false
SCHEDULE = @every 5m

[git]
; The path of git executable. If empty, Gitea searches through the PATH environment.
PATH =
Expand Down
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ require (
github.com/dlclark/regexp2 v1.4.0 // indirect
github.com/dustin/go-humanize v1.0.0
github.com/editorconfig/editorconfig-core-go/v2 v2.4.2
github.com/emersion/go-imap v1.0.6
github.com/emersion/go-message v0.13.0
github.com/emirpasic/gods v1.12.0
github.com/ethantkoenig/rupture v1.0.0
github.com/gliderlabs/ssh v0.3.2
Expand Down
11 changes: 11 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,15 @@ github.com/editorconfig/editorconfig-core-go/v2 v2.4.2/go.mod h1:IXeWRVO4LZRoNun
github.com/edsrzf/mmap-go v1.0.0/go.mod h1:YO35OhQPt3KJa3ryjFM5Bs14WD66h8eGKpfaBNrHW5M=
github.com/eknkc/amber v0.0.0-20171010120322-cdade1c07385 h1:clC1lXBpe2kTj2VHdaIu9ajZQe4kcEY9j0NsnDDBZ3o=
github.com/eknkc/amber v0.0.0-20171010120322-cdade1c07385/go.mod h1:0vRUJqYpeSZifjYj7uP3BG/gKcuzL9xWVV/Y+cK33KM=
github.com/emersion/go-imap v1.0.6 h1:N9+o5laOGuntStBo+BOgfEB5evPsPD+K5+M0T2dctIc=
github.com/emersion/go-imap v1.0.6/go.mod h1:yKASt+C3ZiDAiCSssxg9caIckWF/JG7ZQTO7GAmvicU=
github.com/emersion/go-message v0.11.1/go.mod h1:C4jnca5HOTo4bGN9YdqNQM9sITuT3Y0K6bSUw9RklvY=
github.com/emersion/go-message v0.13.0 h1:R4+CZv4Msxfk9tMaERjMkapdvdO2faWLuB5KHFsNLZE=
github.com/emersion/go-message v0.13.0/go.mod h1:kYIioST9GDHte9/BRWgi93rpqbDuFftMjKSMaXS8ABo=
github.com/emersion/go-sasl v0.0.0-20191210011802-430746ea8b9b h1:uhWtEWBHgop1rqEk2klKaxPAkVDCXexai6hSuRQ7Nvs=
github.com/emersion/go-sasl v0.0.0-20191210011802-430746ea8b9b/go.mod h1:G/dpzLu16WtQpBfQ/z3LYiYJn3ZhKSGWn83fyoyQe/k=
github.com/emersion/go-textwrapper v0.0.0-20160606182133-d0e65e56babe h1:40SWqY0zE3qCi6ZrtTf5OUdNm5lDnGnjRSq9GgmeTrg=
github.com/emersion/go-textwrapper v0.0.0-20160606182133-d0e65e56babe/go.mod h1:aqO8z8wPrjkscevZJFVE1wXJrLpC5LtJG7fqLOsPb2U=
github.com/emirpasic/gods v1.12.0 h1:QAUIPSaCu4G+POclxeqb3F+WPpdKqFGlw36+yOzGlrg=
github.com/emirpasic/gods v1.12.0/go.mod h1:YfzfFFoVP/catgzJb4IKIqXjX78Ha8FMSDh3ymbK86o=
github.com/envoyproxy/go-control-plane v0.6.9/go.mod h1:SBwIajubJHhxtWwsL9s8ss4safvEdbitLhGGK48rN6g=
Expand Down Expand Up @@ -801,6 +810,8 @@ github.com/markbates/goth v1.67.1 h1:gU5B0pzHVyhnJPwGynfFnkfvaQ39C1Sy+ewdl+bhAOw
github.com/markbates/goth v1.67.1/go.mod h1:EyLFHGU5ySr2GXRDyJH5nu2dA7parbC8QwIYW/rGcWg=
github.com/markbates/oncer v0.0.0-20181203154359-bf2de49a0be2/go.mod h1:Ld9puTsIW75CHf65OeIOkyKbteujpZVXDpWK6YGZbxE=
github.com/markbates/safe v1.0.1/go.mod h1:nAqgmRi7cY2nqMc92/bSEeQA+R4OheNU2T1kNSCBdG0=
github.com/martinlindhe/base36 v1.0.0 h1:eYsumTah144C0A8P1T/AVSUk5ZoLnhfYFM3OGQxB52A=
github.com/martinlindhe/base36 v1.0.0/go.mod h1:+AtEs8xrBpCeYgSLoY/aJ6Wf37jtBuR0s35750M27+8=
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-colorable v0.1.1/go.mod h1:FuOcm+DKB9mbwrcAfNl7/TZVBZ6rcnceauSikq3lYCQ=
github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
Expand Down
1 change: 0 additions & 1 deletion models/fixtures/issue.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
created_unix: 946684810
updated_unix: 978307190


-
id: 3
repo_id: 1
Expand Down
6 changes: 5 additions & 1 deletion models/issue.go
Original file line number Diff line number Diff line change
Expand Up @@ -409,14 +409,18 @@ func (issue *Issue) HasLabel(labelID int64) bool {
}

// ReplyReference returns tokenized address to use for email reply headers
func (issue *Issue) ReplyReference() string {
func (issue *Issue) ReplyReference(key string) string {
var path string
if issue.IsPull {
path = "pulls"
} else {
path = "issues"
}

if len(key) > 0 {
return fmt.Sprintf("%s/%s/%d?%s@%s", issue.Repo.FullName(), path, issue.Index, key, setting.Domain)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://datatracker.ietf.org/doc/html/rfc2392

     message-id    = url-addr-spec
     url-addr-spec = addr-spec  ; URL encoding of RFC 822 addr-spec

The key might need to be encoded.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this format is too strict to produce a meaningfull messgae, I wonder.

     addr-spec   =  local-part "@" domain        ; global address
     
     local-part  =  word *("." word)             ; uninterpreted
                                                 ; case-preserved

example of this format:

12343@domain
22233.word.123@domain

and github also not use this format:
image

Copy link
Contributor

@wxiaoguang wxiaoguang Nov 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An word is either an atom or a quoted string.

https://jkorpela.fi/rfc/822addr.html

Positively speaking, this means that the valid constituents of an atom are the following:

!"#$%&'*+-/
0123456789
=?@
ABCDEFGHIJKLMNOPQRSTUVWXYZ
^_`
abcdefghijklmnopqrstuvwxyz
{|}~

Not that strict.

I mean maybe we need a sanitizer or encoder for such format. Otherwise there may be potential bugs.

}

return fmt.Sprintf("%s/%s/%d@%s", issue.Repo.FullName(), path, issue.Index, setting.Domain)
}

Expand Down
38 changes: 38 additions & 0 deletions models/issue_comment.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import (
"code.gitea.io/gitea/modules/markup"
"code.gitea.io/gitea/modules/markup/markdown"
"code.gitea.io/gitea/modules/references"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/structs"
"code.gitea.io/gitea/modules/timeutil"
jsoniter "github.com/json-iterator/go"
Expand Down Expand Up @@ -688,6 +689,43 @@ func (c *Comment) LoadPushCommits() (err error) {
return err
}

// ReplyReference returns tokenized address to use for email reply headers
func (c *Comment) ReplyReference(key string) string {
if err := c.LoadIssue(); err != nil {
log.Error("comment.LoadIssue(): %v", err)
return ""
}

if err := c.Issue.LoadRepo(); err != nil {
log.Error("Issue.LoadRepo(): %v", err)
return ""
}

var path string
if c.Issue.IsPull {
path = "pulls"
} else {
path = "issues"
}

if len(key) > 0 {
return fmt.Sprintf("%s/%s/%d#%s?%s@%s",
c.Issue.Repo.FullName(),
path,
c.Issue.Index,
c.HashTag(),
key,
setting.Domain)
}

return fmt.Sprintf("%s/%s/%d#%s@%s",
c.Issue.Repo.FullName(),
path,
c.Issue.Index,
c.HashTag(),
setting.Domain)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've seen this block of code at least twice now. What about refactoring into a new function?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry, but I hasn't found it. can you point it out please? Thanks.

}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, these Message-ID should follow RFC encoding


func createComment(e *xorm.Session, opts *CreateCommentOptions) (_ *Comment, err error) {
var LabelID int64
if opts.Label != nil {
Expand Down
2 changes: 1 addition & 1 deletion modules/base/tool.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ func EncodeSha1(str string) string {
return hex.EncodeToString(h.Sum(nil))
}

// EncodeSha256 string to sha1 hex value.
// EncodeSha256 string to sha256 hex value.
func EncodeSha256(str string) string {
h := sha256.New()
_, _ = h.Write([]byte(str))
Expand Down
12 changes: 12 additions & 0 deletions modules/cron/tasks_extended.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"code.gitea.io/gitea/models"
repo_module "code.gitea.io/gitea/modules/repository"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/services/imap"
)

func registerDeleteInactiveUsers() {
Expand Down Expand Up @@ -131,6 +132,16 @@ func registerDeleteOldActions() {
})
}

func registerImapFetchUnReadMails() {
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
RegisterTaskFatal("imap_fetch_mails", &BaseConfig{
Enabled: setting.MailReciveService != nil,
RunAtStart: false,
Schedule: "@every 5m",
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
}, func(ctx context.Context, _ *models.User, _ Config) error {
return imap.FetchAllUnReadMails()
})
}

func initExtendedTasks() {
registerDeleteInactiveUsers()
registerDeleteRepositoryArchives()
Expand All @@ -142,4 +153,5 @@ func initExtendedTasks() {
registerDeleteMissingRepositories()
registerRemoveRandomAvatars()
registerDeleteOldActions()
registerImapFetchUnReadMails()
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
}
46 changes: 46 additions & 0 deletions modules/setting/mail_recive.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
// Copyright 2020 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package setting

import "code.gitea.io/gitea/modules/log"

// MailReciver represents mail recive service.
type MailReciver struct {
ReciveEmail string
ReciveBox string
QueueLength int

Host string
User, Passwd string
IsTLSEnabled bool

DeleteRodeMail bool
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
}

var (
// MailReciveService mail recive config
MailReciveService *MailReciver
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
)

func newMailReciveService() {
sec := Cfg.Section("mail_recive")
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
// Check mailer setting.
if !sec.Key("ENABLED").MustBool() {
return
}

MailReciveService = &MailReciver{
ReciveEmail: sec.Key("RECIVE_EMAIL").String(),
ReciveBox: sec.Key("RECIVE_BOX").MustString("INBOX"),
QueueLength: sec.Key("READ_BUFFER_LEN").MustInt(100),
Host: sec.Key("HOST").String(),
User: sec.Key("USER").String(),
Passwd: sec.Key("PASSWD").String(),
IsTLSEnabled: sec.Key("IS_TLS_ENABLED").MustBool(true),
DeleteRodeMail: sec.Key("DELETE_RODE_MAIL").MustBool(false),
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
}

log.Info("Mail Recive Service Enabled")
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
}
1 change: 1 addition & 0 deletions modules/setting/setting.go
Original file line number Diff line number Diff line change
Expand Up @@ -1169,6 +1169,7 @@ func NewServices() {
newSessionService()
newCORSService()
newMailService()
newMailReciveService()
newRegisterMailService()
newNotifyMailService()
newWebhookService()
Expand Down
2 changes: 2 additions & 0 deletions options/locale/locale_en-US.ini
Original file line number Diff line number Diff line change
Expand Up @@ -2184,6 +2184,8 @@ dashboard.last_gc_pause = Last GC Pause
dashboard.gc_times = GC Times
dashboard.delete_old_actions = Delete all old actions from database
dashboard.delete_old_actions.started = Delete all old actions from database started.
dashboard.imap_fetch_mails = fetch unread mails
dashboard.imap_fetch_mails.started = fetch unread mails started.
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved

users.user_manage_panel = User Account Management
users.new_account = Create User Account
Expand Down
2 changes: 2 additions & 0 deletions routers/init.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import (
"code.gitea.io/gitea/modules/svg"
"code.gitea.io/gitea/modules/task"
"code.gitea.io/gitea/modules/translation"
"code.gitea.io/gitea/services/imap"
"code.gitea.io/gitea/services/mailer"
mirror_service "code.gitea.io/gitea/services/mirror"
pull_service "code.gitea.io/gitea/services/pull"
Expand Down Expand Up @@ -59,6 +60,7 @@ func NewServices() {
log.Fatal("repository init failed: %v", err)
}
mailer.NewContext()
imap.NewContext()
_ = cache.NewContext()
notification.NewContext()
}
Expand Down
46 changes: 46 additions & 0 deletions services/imap/cron.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
// Copyright 2020 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package imap

import (
"code.gitea.io/gitea/modules/setting"
)

var (
c *Client
)

// FetchAllUnReadMails fetch all unread mails
func FetchAllUnReadMails() (err error) {
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
if c == nil {
c, err = NewImapClient(ClientInitOpt{
Addr: setting.MailReciveService.Host,
UserName: setting.MailReciveService.User,
Passwd: setting.MailReciveService.Passwd,
IsTLS: setting.MailReciveService.IsTLSEnabled,
})
if err != nil {
return
}
}

if mailReadQueue != nil && !mailReadQueue.IsEmpty() {
return
}

mails, err := c.GetUnReadMails(setting.MailReciveService.ReciveBox, 100)
a1012112796 marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
return
}

for _, mail := range mails {
err = mailReadQueue.Push(mail)
if err != nil {
return err
}
}

return nil
}
Loading