From 2b749af50508a28c7088e7d76132c27f96a45789 Mon Sep 17 00:00:00 2001 From: 6543 <6543@obermui.de> Date: Thu, 13 Apr 2023 10:45:54 +0200 Subject: [PATCH 01/28] Changelog v1.19.1 (#24079) (#24092) Frontport #24079 --- CHANGELOG.md | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 90 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 324b0cdfd6023..0bdd4c04f09c8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,96 @@ This changelog goes through all the changes that have been made in each release without substantial changes to our git log; to see the highlights of what has been added to each release, please refer to the [blog](https://blog.gitea.io). -## [1.19.0](https://github.com/go-gitea/gitea/releases/tag/1.19.0) - 2023-03-19 +## [1.19.1](https://github.com/go-gitea/gitea/releases/tag/v1.19.1) - 2023-04-12 + +* BREAKING + * Rename actions unit to `repo.actions` and add docs for it (#23733) (#23881) +* ENHANCEMENTS + * Add card type to org/user level project on creation, edit and view (#24043) (#24066) + * Refactor commit status for Actions jobs (#23786) (#24060) + * Show errors for KaTeX and mermaid on the preview tab (#24009) (#24019) + * Show protected branch rule names again (#23907) (#24018) + * Adjust sticky PR header to cover background (#23956) (#23999) + * Discolor pull request tab labels (#23950) (#23987) + * Treat PRs with agit flow as fork PRs when triggering actions. (#23884) (#23967) + * Left-align review comments (#23937) + * Fix image border-radius (#23886) (#23930) + * Scroll collapsed file into view (#23702) (#23929) + * Fix code view (diff) broken layout (#23096) (#23918) + * Org pages style fixes (#23901) (#23914) + * Fix user profile description rendering (#23882) (#23902) + * Fix review box viewport overflow issue (#23800) (#23898) + * Prefill input values in OAuth settings as intended (#23829) (#23871) + * CSS color tweaks (#23828) (#23842) + * Fix incorrect visibility dropdown list in add/edit user page (#23804) (#23833) + * Add CSS rules for basic colored labels (#23774) (#23777) + * Add creation time in tag list page (#23693) (#23773) + * Fix br display for packages curls (#23737) (#23764) + * Fix issue due date edit toggle bug (#23723) (#23758) + * Improve commit graph page UI alignment (#23751) (#23754) + * Use GitHub Actions compatible globbing for `branches`, `tag`, `path` filter (#22804) (#23740) + * Redirect to project again after editing it (#23326) (#23739) + * Remove row clicking from notification table (#22695) (#23706) + * Remove conflicting CSS rules on notifications, improve notifications table (#23565) (#23621) + * Fix diff tree height and adjust target file style (#23616) +* BUGFIXES + * Improve error logging for LFS (#24072) (#24082) + * Fix custom mailer template on Windows platform (#24081) + * Update the value of `diffEnd` when clicking the `Show More` button in the DiffFileTree (#24069) (#24078) + * Make label templates have consistent behavior and priority (#23749) + * Fix accidental overwriting of LDAP team memberships (#24050) (#24065) + * Fix branch protection priority (#24045) (#24061) + * Use actions job link as commit status URL instead of run link (#24023) (#24032) + * Add actions support to package auth verification (#23729) (#24028) + * Fix protected branch for API (#24013) (#24027) + * Do not escape space between PyPI repository URL and package name… (#23981) (#24008) + * Fix redirect bug when creating issue from a project (#23971) (#23997) + * Set `ref` to fully-formed of the tag when trigger event is `release` (#23944) (#23989) + * Use Get/Set instead of Rename when regenerating session ID (#23975) (#23983) + * Ensure RSS icon is present on all repo tabs (#23904) (#23973) + * Remove `Repository.getFilesChanged` to fix Actions `paths` and `paths-ignore` filter (#23920) (#23969) + * Delete deleted release attachments immediately from storage (#23913) (#23958) + * Use ghost user if package creator does not exist (#23822) (#23915) + * User/Org Feed render description as per web (#23887) (#23906) + * Fix `cases.Title` crash for concurrency (#23885) (#23903) + * Convert .Source.SkipVerify to $cfg.SkipVerify (#23839) (#23899) + * Support "." char as user name for User/Orgs in RSS/ATOM/GPG/KEYS path ... (#23874) (#23878) + * Fix JS error when changing PR's target branch (#23862) (#23864) + * Fix 500 error if there is a name conflict when editing authentication source (#23832) (#23852) + * Fix closed PR also triggers Webhooks and actions (#23782) (#23834) + * Fix checks for `needs` in Actions (#23789) (#23831) + * Fix "Updating branch by merge" bug in "update_branch_by_merge.tmpl" (#23790) (#23825) + * Fix cancel button in the page of project edit not work (#23655) (#23813) + * Don't apply the group filter when listing LDAP group membership if it is empty (#23745) (#23788) + * Fix profile page email display, respect settings (#23747) (#23756) + * Fix project card preview select and template select (#23684) (#23731) + * Check LFS/Packages settings in dump and doctor command (#23631) (#23730) + * Add git dashes separator to some "log" and "diff" commands (#23606) (#23720) + * Create commit status when event is `pull_request_sync` (#23683) (#23691) + * Fix incorrect `HookEventType` of pull request review comments (#23650) (#23678) + * Fix incorrect `show-modal` and `show-panel` class (#23660) (#23663) + * Improve workflow event triggers (#23613) (#23648) + * Introduce path Clean/Join helper functions, partially backport&refactor (#23495) (#23607) + * Fix pagination on `/notifications/watching` (#23564) (#23603) + * Fix submodule is nil panic (#23588) (#23601) + * Polyfill the window.customElements (#23592) (#23595) + * Avoid too long names for actions (#23162) (#23190) +* TRANSLATION + * Backport locales (with manual fixes) (#23808, #23634, #24083) +* BUILD + * Hardcode the path to docker images (#23955) (#23968) +* DOCS + * Update documentation to explain which projects allow Gitea to host static pages (#23993) (#24058) + * Merge `push to create`, `open PR from push`, and `push options` docs articles into one (#23744) (#23959) + * Fix code blocks in the cheat sheet (#23664) (#23669) +* MISC + * Do not crash when parsing an invalid workflow file (#23972) (#23976) + * Remove assertion debug code for show/hide refactoring (#23576) (#23868) + * Add ONLY_SHOW_RELEVANT_REPOS back, fix explore page bug, make code more strict (#23766) (#23791) + * Make minio package support legacy MD5 checksum (#23768) (#23770) + * Improve template error reporting (#23396) (#23600) + +## [1.19.0](https://github.com/go-gitea/gitea/releases/tag/v1.19.0) - 2023-03-19 * BREAKING * Add loading yaml label template files (#22976) (#23232) From 29194a9dd6f5d8a98096a4a01c33d9be89616ed7 Mon Sep 17 00:00:00 2001 From: Gary Moon Date: Thu, 13 Apr 2023 09:14:06 -0400 Subject: [PATCH 02/28] Correct the access log format (#24085) The default access log format has been unnecessarily escaped, leading to spurious backslashes appearing in log lines. Additionally, the `RemoteAddr` field includes the port, which breaks most log parsers attempting to process it. I've added a call to `net.SplitHostPort()` attempting to isolate the address alone, with a fallback to the original address if it errs. Signed-off-by: Gary Moon --- custom/conf/app.example.ini | 2 +- .../doc/administration/config-cheat-sheet.en-us.md | 2 +- .../doc/administration/config-cheat-sheet.zh-cn.md | 2 +- .../doc/administration/logging-documentation.en-us.md | 2 +- modules/context/access_log.go | 9 ++++++++- modules/setting/log.go | 2 +- 6 files changed, 13 insertions(+), 6 deletions(-) diff --git a/custom/conf/app.example.ini b/custom/conf/app.example.ini index 88297fe0d975e..f9f207522c7df 100644 --- a/custom/conf/app.example.ini +++ b/custom/conf/app.example.ini @@ -603,7 +603,7 @@ ROUTER = console ;ACCESS = file ;; ;; Sets the template used to create the access log. -;ACCESS_LOG_TEMPLATE = {{.Ctx.RemoteAddr}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}\" \"{{.Ctx.Req.UserAgent}}" +;ACCESS_LOG_TEMPLATE = {{.Ctx.RemoteHost}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}" "{{.Ctx.Req.UserAgent}}" ;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; diff --git a/docs/content/doc/administration/config-cheat-sheet.en-us.md b/docs/content/doc/administration/config-cheat-sheet.en-us.md index 76952df40e751..f26e7eaa085ab 100644 --- a/docs/content/doc/administration/config-cheat-sheet.en-us.md +++ b/docs/content/doc/administration/config-cheat-sheet.en-us.md @@ -878,7 +878,7 @@ Default templates for project boards: - `ENABLE_ACCESS_LOG`: **false**: Creates an access.log in NCSA common log format, or as per the following template - `ACCESS`: **file**: Logging mode for the access logger, use a comma to separate values. Configure each mode in per mode log subsections `\[log.modename.access\]`. By default the file mode will log to `$ROOT_PATH/access.log`. (If you set this to `,` it will log to the default Gitea logger.) -- `ACCESS_LOG_TEMPLATE`: **`{{.Ctx.RemoteAddr}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}\" \"{{.Ctx.Req.UserAgent}}"`**: Sets the template used to create the access log. +- `ACCESS_LOG_TEMPLATE`: **`{{.Ctx.RemoteHost}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}" "{{.Ctx.Req.UserAgent}}"`**: Sets the template used to create the access log. - The following variables are available: - `Ctx`: the `context.Context` of the request. - `Identity`: the SignedUserName or `"-"` if not logged in. diff --git a/docs/content/doc/administration/config-cheat-sheet.zh-cn.md b/docs/content/doc/administration/config-cheat-sheet.zh-cn.md index 043cc42e53bc7..83e212f32b84a 100644 --- a/docs/content/doc/administration/config-cheat-sheet.zh-cn.md +++ b/docs/content/doc/administration/config-cheat-sheet.zh-cn.md @@ -265,7 +265,7 @@ test01.xls: application/vnd.ms-excel; charset=binary - `LEVEL`: 日志级别，默认为 `Trace`。 - `DISABLE_ROUTER_LOG`: 关闭日志中的路由日志。 - `ENABLE_ACCESS_LOG`: 是否开启 Access Log, 默认为 false。 -- `ACCESS_LOG_TEMPLATE`: `access.log` 输出内容的模板，默认模板：**`{{.Ctx.RemoteAddr}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}\" \"{{.Ctx.Req.UserAgent}}"`** +- `ACCESS_LOG_TEMPLATE`: `access.log` 输出内容的模板，默认模板：**`{{.Ctx.RemoteHost}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}" "{{.Ctx.Req.UserAgent}}"`** 模板支持以下参数: - `Ctx`: 请求上下文。 - `Identity`: 登录用户名，默认: “`-`”。 diff --git a/docs/content/doc/administration/logging-documentation.en-us.md b/docs/content/doc/administration/logging-documentation.en-us.md index 13de8ab882bf0..029eb5de095f9 100644 --- a/docs/content/doc/administration/logging-documentation.en-us.md +++ b/docs/content/doc/administration/logging-documentation.en-us.md @@ -304,7 +304,7 @@ log using the value: `ACCESS = ,` This value represent a go template. It's default value is: -`{{.Ctx.RemoteAddr}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}\" \"{{.Ctx.Req.UserAgent}}"` +`{{.Ctx.RemoteHost}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}" "{{.Ctx.Req.UserAgent}}"` The template is passed following options: diff --git a/modules/context/access_log.go b/modules/context/access_log.go index 515682b64b0e6..64d204733b8a0 100644 --- a/modules/context/access_log.go +++ b/modules/context/access_log.go @@ -7,6 +7,7 @@ import ( "bytes" "context" "fmt" + "net" "net/http" "strings" "text/template" @@ -67,17 +68,23 @@ func AccessLogger() func(http.Handler) http.Handler { requestID = parseRequestIDFromRequestHeader(req) } + reqHost, _, err := net.SplitHostPort(req.RemoteAddr) + if err != nil { + reqHost = req.RemoteAddr + } + next.ServeHTTP(w, r) rw := w.(ResponseWriter) buf := bytes.NewBuffer([]byte{}) - err := logTemplate.Execute(buf, routerLoggerOptions{ + err = logTemplate.Execute(buf, routerLoggerOptions{ req: req, Identity: &identity, Start: &start, ResponseWriter: rw, Ctx: map[string]interface{}{ "RemoteAddr": req.RemoteAddr, + "RemoteHost": reqHost, "Req": req, }, RequestID: &requestID, diff --git a/modules/setting/log.go b/modules/setting/log.go index dabdb543abdf5..1ff710073e44f 100644 --- a/modules/setting/log.go +++ b/modules/setting/log.go @@ -152,7 +152,7 @@ func loadLogFrom(rootCfg ConfigProvider) { Log.EnableSSHLog = sec.Key("ENABLE_SSH_LOG").MustBool(false) Log.EnableAccessLog = sec.Key("ENABLE_ACCESS_LOG").MustBool(false) Log.AccessLogTemplate = sec.Key("ACCESS_LOG_TEMPLATE").MustString( - `{{.Ctx.RemoteAddr}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}\" \"{{.Ctx.Req.UserAgent}}"`, + `{{.Ctx.RemoteHost}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}" "{{.Ctx.Req.UserAgent}}"`, ) Log.RequestIDHeaders = sec.Key("REQUEST_ID_HEADERS").Strings(",") // the `MustString` updates the default value, and `log.ACCESS` is used by `generateNamedLogger("access")` later From 469dc4459bb7f56cf8a6daa9c234164c0889bdda Mon Sep 17 00:00:00 2001 From: silverwind Date: Thu, 13 Apr 2023 21:05:06 +0200 Subject: [PATCH 03/28] Add monospace toggle button to textarea (#24034) - Add new button to textarea to switch font. State is persisted in localStorage. - Change markdown-switch-easymde button from `` to ` +
{{.locale.Tr "loading"}} diff --git a/web_src/css/editor-markdown.css b/web_src/css/editor-markdown.css index 46ced17cdc954..7d6c36635dbff 100644 --- a/web_src/css/editor-markdown.css +++ b/web_src/css/editor-markdown.css @@ -24,6 +24,7 @@ user-select: none; padding: 5px; cursor: pointer; + color: var(--color-text); } .combo-markdown-editor .markdown-toolbar-button:hover { diff --git a/web_src/css/helpers.css b/web_src/css/helpers.css index 834a507b68793..dcec79fcf6f08 100644 --- a/web_src/css/helpers.css +++ b/web_src/css/helpers.css @@ -29,7 +29,7 @@ .gt-mono { font-family: var(--fonts-monospace) !important; - font-size: .9em !important; /* compensate for monospace fonts being usually slightly larger */ + font-size: .95em !important; /* compensate for monospace fonts being usually slightly larger */ } .gt-bold { font-weight: 600 !important; } diff --git a/web_src/js/features/comp/ComboMarkdownEditor.js b/web_src/js/features/comp/ComboMarkdownEditor.js index a7d69af7b4e75..3eb8bf7076b3c 100644 --- a/web_src/js/features/comp/ComboMarkdownEditor.js +++ b/web_src/js/features/comp/ComboMarkdownEditor.js @@ -73,8 +73,25 @@ class ComboMarkdownEditor { // upstream bug: The role code is never executed in base MarkdownButtonElement https://github.com/github/markdown-toolbar-element/issues/70 el.setAttribute('role', 'button'); } - this.switchToEasyMDEButton = this.container.querySelector('.markdown-switch-easymde'); - this.switchToEasyMDEButton?.addEventListener('click', async (e) => { + + const monospaceButton = this.container.querySelector('.markdown-switch-monospace'); + const monospaceEnabled = localStorage?.getItem('markdown-editor-monospace') === 'true'; + const monospaceText = monospaceButton.getAttribute(monospaceEnabled ? 'data-disable-text' : 'data-enable-text'); + monospaceButton.setAttribute('data-tooltip-content', monospaceText); + monospaceButton.setAttribute('aria-checked', String(monospaceEnabled)); + + monospaceButton?.addEventListener('click', (e) => { + e.preventDefault(); + const enabled = localStorage?.getItem('markdown-editor-monospace') !== 'true'; + localStorage.setItem('markdown-editor-monospace', String(enabled)); + this.textarea.classList.toggle('gt-mono', enabled); + const text = monospaceButton.getAttribute(enabled ? 'data-disable-text' : 'data-enable-text'); + monospaceButton.setAttribute('data-tooltip-content', text); + monospaceButton.setAttribute('aria-checked', String(enabled)); + }); + + const easymdeButton = this.container.querySelector('.markdown-switch-easymde'); + easymdeButton?.addEventListener('click', async (e) => { e.preventDefault(); this.userPreferredEditor = 'easymde'; await this.switchToEasyMDE(); From b7221bec34fd49495234a18c26e4f5d81483e102 Mon Sep 17 00:00:00 2001 From: yp05327 <576951401@qq.com> Date: Fri, 14 Apr 2023 04:06:10 +0900 Subject: [PATCH 04/28] Fix admin team access mode value in team_unit table (#24012) Same as https://github.com/go-gitea/gitea/pull/23675 Feedback: https://github.com/go-gitea/gitea/pull/23879#issuecomment-1500923636 --- models/migrations/migrations.go | 2 + models/migrations/v1_20/v252.go | 47 +++++++++++++++ routers/api/v1/org/team.go | 19 ++++++ routers/web/org/teams.go | 93 ++++++++++++++++-------------- templates/org/team/new.tmpl | 4 +- tests/integration/api_team_test.go | 31 ++++++++++ 6 files changed, 151 insertions(+), 45 deletions(-) create mode 100644 models/migrations/v1_20/v252.go diff --git a/models/migrations/migrations.go b/models/migrations/migrations.go index 07240c8e69ed5..35a18fb7f2bbc 100644 --- a/models/migrations/migrations.go +++ b/models/migrations/migrations.go @@ -481,6 +481,8 @@ var migrations = []Migration{ NewMigration("Change Container Metadata", v1_20.ChangeContainerMetadataMultiArch), // v251 -> v252 NewMigration("Fix incorrect owner team unit access mode", v1_20.FixIncorrectOwnerTeamUnitAccessMode), + // v252 -> v253 + NewMigration("Fix incorrect admin team unit access mode", v1_20.FixIncorrectAdminTeamUnitAccessMode), } // GetCurrentDBVersion returns the current db version diff --git a/models/migrations/v1_20/v252.go b/models/migrations/v1_20/v252.go new file mode 100644 index 0000000000000..ab61cd9b8b36e --- /dev/null +++ b/models/migrations/v1_20/v252.go @@ -0,0 +1,47 @@ +// Copyright 2023 The Gitea Authors. All rights reserved. +// SPDX-License-Identifier: MIT + +package v1_20 //nolint + +import ( + "code.gitea.io/gitea/modules/log" + + "xorm.io/xorm" +) + +func FixIncorrectAdminTeamUnitAccessMode(x *xorm.Engine) error { + type UnitType int + type AccessMode int + + type TeamUnit struct { + ID int64 `xorm:"pk autoincr"` + OrgID int64 `xorm:"INDEX"` + TeamID int64 `xorm:"UNIQUE(s)"` + Type UnitType `xorm:"UNIQUE(s)"` + AccessMode AccessMode + } + + const ( + // AccessModeAdmin admin access + AccessModeAdmin = 3 + ) + + sess := x.NewSession() + defer sess.Close() + + if err := sess.Begin(); err != nil { + return err + } + + count, err := sess.Table("team_unit"). + Where("team_id IN (SELECT id FROM team WHERE authorize = ?)", AccessModeAdmin). + Update(&TeamUnit{ + AccessMode: AccessModeAdmin, + }) + if err != nil { + return err + } + log.Debug("Updated %d admin team unit access mode to belong to admin instead of none", count) + + return sess.Commit() +} diff --git a/routers/api/v1/org/team.go b/routers/api/v1/org/team.go index 50439251cc05f..024fee34693c6 100644 --- a/routers/api/v1/org/team.go +++ b/routers/api/v1/org/team.go @@ -166,6 +166,21 @@ func attachTeamUnitsMap(team *organization.Team, unitsMap map[string]string) { } } +func attachAdminTeamUnits(team *organization.Team) { + team.Units = make([]*organization.TeamUnit, 0, len(unit_model.AllRepoUnitTypes)) + for _, ut := range unit_model.AllRepoUnitTypes { + up := perm.AccessModeAdmin + if ut == unit_model.TypeExternalTracker || ut == unit_model.TypeExternalWiki { + up = perm.AccessModeRead + } + team.Units = append(team.Units, &organization.TeamUnit{ + OrgID: team.OrgID, + Type: ut, + AccessMode: up, + }) + } +} + // CreateTeam api for create a team func CreateTeam(ctx *context.APIContext) { // swagger:operation POST /orgs/{org}/teams organization orgCreateTeam @@ -213,6 +228,8 @@ func CreateTeam(ctx *context.APIContext) { ctx.Error(http.StatusInternalServerError, "getTeamUnits", errors.New("units permission should not be empty")) return } + } else { + attachAdminTeamUnits(team) } if err := models.NewTeam(team); err != nil { @@ -300,6 +317,8 @@ func EditTeam(ctx *context.APIContext) { } else if len(form.Units) > 0 { attachTeamUnits(team, form.Units) } + } else { + attachAdminTeamUnits(team) } if err := models.UpdateTeam(team, isAuthChanged, isIncludeAllChanged); err != nil { diff --git a/routers/web/org/teams.go b/routers/web/org/teams.go index 1ed7980145491..e2ec6d87858eb 100644 --- a/routers/web/org/teams.go +++ b/routers/web/org/teams.go @@ -5,6 +5,7 @@ package org import ( + "fmt" "net/http" "net/url" "path" @@ -264,14 +265,26 @@ func NewTeam(ctx *context.Context) { ctx.HTML(http.StatusOK, tplTeamNew) } -func getUnitPerms(forms url.Values) map[unit_model.Type]perm.AccessMode { +func getUnitPerms(forms url.Values, teamPermission perm.AccessMode) map[unit_model.Type]perm.AccessMode { unitPerms := make(map[unit_model.Type]perm.AccessMode) - for k, v := range forms { - if strings.HasPrefix(k, "unit_") { - t, _ := strconv.Atoi(k[5:]) - if t > 0 { - vv, _ := strconv.Atoi(v[0]) - unitPerms[unit_model.Type(t)] = perm.AccessMode(vv) + for _, ut := range unit_model.AllRepoUnitTypes { + // Default accessmode is none + unitPerms[ut] = perm.AccessModeNone + + v, ok := forms[fmt.Sprintf("unit_%d", ut)] + if ok { + vv, _ := strconv.Atoi(v[0]) + if teamPermission >= perm.AccessModeAdmin { + unitPerms[ut] = teamPermission + // Don't allow `TypeExternal{Tracker,Wiki}` to influence this as they can only be set to READ perms. + if ut == unit_model.TypeExternalTracker || ut == unit_model.TypeExternalWiki { + unitPerms[ut] = perm.AccessModeRead + } + } else { + unitPerms[ut] = perm.AccessMode(vv) + if unitPerms[ut] >= perm.AccessModeAdmin { + unitPerms[ut] = perm.AccessModeWrite + } } } } @@ -282,8 +295,8 @@ func getUnitPerms(forms url.Values) map[unit_model.Type]perm.AccessMode { func NewTeamPost(ctx *context.Context) { form := web.GetForm(ctx).(*forms.CreateTeamForm) includesAllRepositories := form.RepoAccess == "all" - unitPerms := getUnitPerms(ctx.Req.Form) p := perm.ParseAccessMode(form.Permission) + unitPerms := getUnitPerms(ctx.Req.Form, p) if p < perm.AccessModeAdmin { // if p is less than admin accessmode, then it should be general accessmode, // so we should calculate the minial accessmode from units accessmodes. @@ -299,17 +312,15 @@ func NewTeamPost(ctx *context.Context) { CanCreateOrgRepo: form.CanCreateOrgRepo, } - if t.AccessMode < perm.AccessModeAdmin { - units := make([]*org_model.TeamUnit, 0, len(unitPerms)) - for tp, perm := range unitPerms { - units = append(units, &org_model.TeamUnit{ - OrgID: ctx.Org.Organization.ID, - Type: tp, - AccessMode: perm, - }) - } - t.Units = units + units := make([]*org_model.TeamUnit, 0, len(unitPerms)) + for tp, perm := range unitPerms { + units = append(units, &org_model.TeamUnit{ + OrgID: ctx.Org.Organization.ID, + Type: tp, + AccessMode: perm, + }) } + t.Units = units ctx.Data["Title"] = ctx.Org.Organization.FullName ctx.Data["PageIsOrgTeams"] = true @@ -422,8 +433,11 @@ func SearchTeam(ctx *context.Context) { func EditTeam(ctx *context.Context) { ctx.Data["Title"] = ctx.Org.Organization.FullName ctx.Data["PageIsOrgTeams"] = true - ctx.Data["team_name"] = ctx.Org.Team.Name - ctx.Data["desc"] = ctx.Org.Team.Description + if err := ctx.Org.Team.LoadUnits(ctx); err != nil { + ctx.ServerError("LoadUnits", err) + return + } + ctx.Data["Team"] = ctx.Org.Team ctx.Data["Units"] = unit_model.Units ctx.HTML(http.StatusOK, tplTeamNew) } @@ -432,7 +446,13 @@ func EditTeam(ctx *context.Context) { func EditTeamPost(ctx *context.Context) { form := web.GetForm(ctx).(*forms.CreateTeamForm) t := ctx.Org.Team - unitPerms := getUnitPerms(ctx.Req.Form) + newAccessMode := perm.ParseAccessMode(form.Permission) + unitPerms := getUnitPerms(ctx.Req.Form, newAccessMode) + if newAccessMode < perm.AccessModeAdmin { + // if newAccessMode is less than admin accessmode, then it should be general accessmode, + // so we should calculate the minial accessmode from units accessmodes. + newAccessMode = unit_model.MinUnitAccessMode(unitPerms) + } isAuthChanged := false isIncludeAllChanged := false includesAllRepositories := form.RepoAccess == "all" @@ -443,14 +463,6 @@ func EditTeamPost(ctx *context.Context) { ctx.Data["Units"] = unit_model.Units if !t.IsOwnerTeam() { - // Validate permission level. - newAccessMode := perm.ParseAccessMode(form.Permission) - if newAccessMode < perm.AccessModeAdmin { - // if p is less than admin accessmode, then it should be general accessmode, - // so we should calculate the minial accessmode from units accessmodes. - newAccessMode = unit_model.MinUnitAccessMode(unitPerms) - } - t.Name = form.TeamName if t.AccessMode != newAccessMode { isAuthChanged = true @@ -467,21 +479,16 @@ func EditTeamPost(ctx *context.Context) { } t.Description = form.Description - if t.AccessMode < perm.AccessModeAdmin { - units := make([]org_model.TeamUnit, 0, len(unitPerms)) - for tp, perm := range unitPerms { - units = append(units, org_model.TeamUnit{ - OrgID: t.OrgID, - TeamID: t.ID, - Type: tp, - AccessMode: perm, - }) - } - if err := org_model.UpdateTeamUnits(t, units); err != nil { - ctx.Error(http.StatusInternalServerError, "UpdateTeamUnits", err.Error()) - return - } + units := make([]*org_model.TeamUnit, 0, len(unitPerms)) + for tp, perm := range unitPerms { + units = append(units, &org_model.TeamUnit{ + OrgID: t.OrgID, + TeamID: t.ID, + Type: tp, + AccessMode: perm, + }) } + t.Units = units if ctx.HasError() { ctx.HTML(http.StatusOK, tplTeamNew) diff --git a/templates/org/team/new.tmpl b/templates/org/team/new.tmpl index 195f8bdcd69d9..2e65d63580319 100644 --- a/templates/org/team/new.tmpl +++ b/templates/org/team/new.tmpl @@ -109,7 +109,7 @@
- +
@@ -137,7 +137,7 @@ {{else}} {{if not (eq .Team.LowerName "owners")}} - + {{end}} {{end}}
diff --git a/tests/integration/api_team_test.go b/tests/integration/api_team_test.go index 2801fdc98934e..934e6bf23046f 100644 --- a/tests/integration/api_team_test.go +++ b/tests/integration/api_team_test.go @@ -12,6 +12,7 @@ import ( auth_model "code.gitea.io/gitea/models/auth" "code.gitea.io/gitea/models/db" "code.gitea.io/gitea/models/organization" + "code.gitea.io/gitea/models/perm" "code.gitea.io/gitea/models/repo" "code.gitea.io/gitea/models/unit" "code.gitea.io/gitea/models/unittest" @@ -189,6 +190,36 @@ func TestAPITeam(t *testing.T) { req = NewRequestf(t, "DELETE", "/api/v1/teams/%d?token="+token, teamID) MakeRequest(t, req, http.StatusNoContent) unittest.AssertNotExistsBean(t, &organization.Team{ID: teamID}) + + // Create admin team + teamToCreate = &api.CreateTeamOption{ + Name: "teamadmin", + Description: "team admin", + IncludesAllRepositories: true, + Permission: "admin", + } + req = NewRequestWithJSON(t, "POST", fmt.Sprintf("/api/v1/orgs/%s/teams?token=%s", org.Name, token), teamToCreate) + resp = MakeRequest(t, req, http.StatusCreated) + apiTeam = api.Team{} + DecodeJSON(t, resp, &apiTeam) + for _, ut := range unit.AllRepoUnitTypes { + up := perm.AccessModeAdmin + if ut == unit.TypeExternalTracker || ut == unit.TypeExternalWiki { + up = perm.AccessModeRead + } + unittest.AssertExistsAndLoadBean(t, &organization.TeamUnit{ + OrgID: org.ID, + TeamID: apiTeam.ID, + Type: ut, + AccessMode: up, + }) + } + teamID = apiTeam.ID + + // Delete team. + req = NewRequestf(t, "DELETE", "/api/v1/teams/%d?token="+token, teamID) + MakeRequest(t, req, http.StatusNoContent) + unittest.AssertNotExistsBean(t, &organization.Team{ID: teamID}) } func checkTeamResponse(t *testing.T, testName string, apiTeam *api.Team, name, description string, includesAllRepositories bool, permission string, units []string, unitsMap map[string]string) { From 5b9557aef59b190c55de9ea218bf51152bc04786 Mon Sep 17 00:00:00 2001 From: wxiaoguang Date: Fri, 14 Apr 2023 03:45:33 +0800 Subject: [PATCH 05/28] Refactor cookie (#24107) Close #24062 At the beginning, I just wanted to fix the warning mentioned by #24062 But, the cookie code really doesn't look good to me, so clean up them. Complete the TODO on `SetCookie`: > TODO: Copied from gitea.com/macaron/macaron and should be improved after macaron removed. --- modules/context/auth.go | 4 +- modules/context/context.go | 100 +++++--------- modules/context/csrf.go | 79 +++++------ modules/setting/session.go | 4 +- modules/web/middleware/cookie.go | 192 ++++---------------------- modules/web/middleware/locale.go | 18 +-- routers/install/install.go | 2 +- routers/web/auth/auth.go | 23 ++- routers/web/auth/oauth.go | 4 +- routers/web/auth/openid.go | 2 +- routers/web/auth/password.go | 2 +- routers/web/home.go | 2 +- services/auth/auth.go | 5 +- services/auth/sspi_windows.go | 16 +-- tests/integration/editor_test.go | 5 +- tests/integration/git_test.go | 3 +- tests/integration/integration_test.go | 3 +- tests/integration/mirror_push_test.go | 5 +- 18 files changed, 141 insertions(+), 328 deletions(-) diff --git a/modules/context/auth.go b/modules/context/auth.go index 7cc29debbd03f..5e071b4fabca3 100644 --- a/modules/context/auth.go +++ b/modules/context/auth.go @@ -67,7 +67,7 @@ func Toggle(options *ToggleOptions) func(ctx *Context) { } if !options.SignOutRequired && !options.DisableCSRF && ctx.Req.Method == "POST" { - ctx.csrf.Validate(ctx) + ctx.Csrf.Validate(ctx) if ctx.Written() { return } @@ -89,7 +89,7 @@ func Toggle(options *ToggleOptions) func(ctx *Context) { // Redirect to log in page if auto-signin info is provided and has not signed in. if !options.SignOutRequired && !ctx.IsSigned && - len(ctx.GetCookie(setting.CookieUserName)) > 0 { + len(ctx.GetSiteCookie(setting.CookieUserName)) > 0 { if ctx.Req.URL.Path != "/user/events" { middleware.SetRedirectToCookie(ctx.Resp, setting.AppSubURL+ctx.Req.URL.RequestURI()) } diff --git a/modules/context/context.go b/modules/context/context.go index e2e120ba384ad..cee533e42ae07 100644 --- a/modules/context/context.go +++ b/modules/context/context.go @@ -45,6 +45,8 @@ import ( "golang.org/x/crypto/pbkdf2" ) +const CookieNameFlash = "gitea_flash" + // Render represents a template render type Render interface { TemplateLookup(tmpl string) (*template.Template, error) @@ -60,7 +62,7 @@ type Context struct { Render Render translation.Locale Cache cache.Cache - csrf CSRFProtector + Csrf CSRFProtector Flash *middleware.Flash Session session.Store @@ -478,38 +480,26 @@ func (ctx *Context) Redirect(location string, status ...int) { http.Redirect(ctx.Resp, ctx.Req, location, code) } -// SetCookie convenience function to set most cookies consistently +// SetSiteCookie convenience function to set most cookies consistently // CSRF and a few others are the exception here -func (ctx *Context) SetCookie(name, value string, expiry int) { - middleware.SetCookie(ctx.Resp, name, value, - expiry, - setting.AppSubURL, - setting.SessionConfig.Domain, - setting.SessionConfig.Secure, - true, - middleware.SameSite(setting.SessionConfig.SameSite)) +func (ctx *Context) SetSiteCookie(name, value string, maxAge int) { + middleware.SetSiteCookie(ctx.Resp, name, value, maxAge) } -// DeleteCookie convenience function to delete most cookies consistently +// DeleteSiteCookie convenience function to delete most cookies consistently // CSRF and a few others are the exception here -func (ctx *Context) DeleteCookie(name string) { - middleware.SetCookie(ctx.Resp, name, "", - -1, - setting.AppSubURL, - setting.SessionConfig.Domain, - setting.SessionConfig.Secure, - true, - middleware.SameSite(setting.SessionConfig.SameSite)) +func (ctx *Context) DeleteSiteCookie(name string) { + middleware.SetSiteCookie(ctx.Resp, name, "", -1) } -// GetCookie returns given cookie value from request header. -func (ctx *Context) GetCookie(name string) string { - return middleware.GetCookie(ctx.Req, name) +// GetSiteCookie returns given cookie value from request header. +func (ctx *Context) GetSiteCookie(name string) string { + return middleware.GetSiteCookie(ctx.Req, name) } // GetSuperSecureCookie returns given cookie value from request header with secret string. func (ctx *Context) GetSuperSecureCookie(secret, name string) (string, bool) { - val := ctx.GetCookie(name) + val := ctx.GetSiteCookie(name) return ctx.CookieDecrypt(secret, val) } @@ -530,10 +520,9 @@ func (ctx *Context) CookieDecrypt(secret, val string) (string, bool) { } // SetSuperSecureCookie sets given cookie value to response header with secret string. -func (ctx *Context) SetSuperSecureCookie(secret, name, value string, expiry int) { +func (ctx *Context) SetSuperSecureCookie(secret, name, value string, maxAge int) { text := ctx.CookieEncrypt(secret, value) - - ctx.SetCookie(name, text, expiry) + ctx.SetSiteCookie(name, text, maxAge) } // CookieEncrypt encrypts a given value using the provided secret @@ -549,19 +538,19 @@ func (ctx *Context) CookieEncrypt(secret, value string) string { // GetCookieInt returns cookie result in int type. func (ctx *Context) GetCookieInt(name string) int { - r, _ := strconv.Atoi(ctx.GetCookie(name)) + r, _ := strconv.Atoi(ctx.GetSiteCookie(name)) return r } // GetCookieInt64 returns cookie result in int64 type. func (ctx *Context) GetCookieInt64(name string) int64 { - r, _ := strconv.ParseInt(ctx.GetCookie(name), 10, 64) + r, _ := strconv.ParseInt(ctx.GetSiteCookie(name), 10, 64) return r } // GetCookieFloat64 returns cookie result in float64 type. func (ctx *Context) GetCookieFloat64(name string) float64 { - v, _ := strconv.ParseFloat(ctx.GetCookie(name), 64) + v, _ := strconv.ParseFloat(ctx.GetSiteCookie(name), 64) return v } @@ -659,7 +648,10 @@ func WithContext(req *http.Request, ctx *Context) *http.Request { // GetContext retrieves install context from request func GetContext(req *http.Request) *Context { - return req.Context().Value(contextKey).(*Context) + if ctx, ok := req.Context().Value(contextKey).(*Context); ok { + return ctx + } + return nil } // GetContextUser returns context user @@ -726,13 +718,13 @@ func Contexter(ctx context.Context) func(next http.Handler) http.Handler { ctx.Data["Context"] = &ctx ctx.Req = WithContext(req, &ctx) - ctx.csrf = PrepareCSRFProtector(csrfOpts, &ctx) + ctx.Csrf = PrepareCSRFProtector(csrfOpts, &ctx) - // Get flash. - flashCookie := ctx.GetCookie("macaron_flash") - vals, _ := url.ParseQuery(flashCookie) - if len(vals) > 0 { - f := &middleware.Flash{ + // Get the last flash message from cookie + lastFlashCookie := middleware.GetSiteCookie(ctx.Req, CookieNameFlash) + if vals, _ := url.ParseQuery(lastFlashCookie); len(vals) > 0 { + // store last Flash message into the template data, to render it + ctx.Data["Flash"] = &middleware.Flash{ DataStore: &ctx, Values: vals, ErrorMsg: vals.Get("error"), @@ -740,40 +732,18 @@ func Contexter(ctx context.Context) func(next http.Handler) http.Handler { InfoMsg: vals.Get("info"), WarningMsg: vals.Get("warning"), } - ctx.Data["Flash"] = f } - f := &middleware.Flash{ - DataStore: &ctx, - Values: url.Values{}, - ErrorMsg: "", - WarningMsg: "", - InfoMsg: "", - SuccessMsg: "", - } + // prepare an empty Flash message for current request + ctx.Flash = &middleware.Flash{DataStore: &ctx, Values: url.Values{}} ctx.Resp.Before(func(resp ResponseWriter) { - if flash := f.Encode(); len(flash) > 0 { - middleware.SetCookie(resp, "macaron_flash", flash, 0, - setting.SessionConfig.CookiePath, - middleware.Domain(setting.SessionConfig.Domain), - middleware.HTTPOnly(true), - middleware.Secure(setting.SessionConfig.Secure), - middleware.SameSite(setting.SessionConfig.SameSite), - ) - return + if val := ctx.Flash.Encode(); val != "" { + middleware.SetSiteCookie(ctx.Resp, CookieNameFlash, val, 0) + } else if lastFlashCookie != "" { + middleware.SetSiteCookie(ctx.Resp, CookieNameFlash, "", -1) } - - middleware.SetCookie(ctx.Resp, "macaron_flash", "", -1, - setting.SessionConfig.CookiePath, - middleware.Domain(setting.SessionConfig.Domain), - middleware.HTTPOnly(true), - middleware.Secure(setting.SessionConfig.Secure), - middleware.SameSite(setting.SessionConfig.SameSite), - ) }) - ctx.Flash = f - // If request sends files, parse them here otherwise the Query() can't be parsed and the CsrfToken will be invalid. if ctx.Req.Method == "POST" && strings.Contains(ctx.Req.Header.Get("Content-Type"), "multipart/form-data") { if err := ctx.Req.ParseMultipartForm(setting.Attachment.MaxSize << 20); err != nil && !strings.Contains(err.Error(), "EOF") { // 32MB max size @@ -785,7 +755,7 @@ func Contexter(ctx context.Context) func(next http.Handler) http.Handler { httpcache.SetCacheControlInHeader(ctx.Resp.Header(), 0, "no-transform") ctx.Resp.Header().Set(`X-Frame-Options`, setting.CORSConfig.XFrameOptions) - ctx.Data["CsrfToken"] = ctx.csrf.GetToken() + ctx.Data["CsrfToken"] = ctx.Csrf.GetToken() ctx.Data["CsrfTokenHtml"] = template.HTML(``) // FIXME: do we really always need these setting? There should be someway to have to avoid having to always set these diff --git a/modules/context/csrf.go b/modules/context/csrf.go index 6639a8b008470..9b0dc2923b532 100644 --- a/modules/context/csrf.go +++ b/modules/context/csrf.go @@ -42,37 +42,26 @@ type CSRFProtector interface { GetToken() string // Validate validates the token in http context. Validate(ctx *Context) + // DeleteCookie deletes the cookie + DeleteCookie(ctx *Context) } type csrfProtector struct { - // Header name value for setting and getting csrf token. - Header string - // Form name value for setting and getting csrf token. - Form string - // Cookie name value for setting and getting csrf token. - Cookie string - // Cookie domain - CookieDomain string - // Cookie path - CookiePath string - // Cookie HttpOnly flag value used for the csrf token. - CookieHTTPOnly bool + opt CsrfOptions // Token generated to pass via header, cookie, or hidden form value. Token string // This value must be unique per user. ID string - // Secret used along with the unique id above to generate the Token. - Secret string } // GetHeaderName returns the name of the HTTP header for csrf token. func (c *csrfProtector) GetHeaderName() string { - return c.Header + return c.opt.Header } // GetFormName returns the name of the form value for csrf token. func (c *csrfProtector) GetFormName() string { - return c.Form + return c.opt.Form } // GetToken returns the current token. This is typically used @@ -138,23 +127,32 @@ func prepareDefaultCsrfOptions(opt CsrfOptions) CsrfOptions { if opt.SessionKey == "" { opt.SessionKey = "uid" } + if opt.CookieLifeTime == 0 { + opt.CookieLifeTime = int(CsrfTokenTimeout.Seconds()) + } + opt.oldSessionKey = "_old_" + opt.SessionKey return opt } +func newCsrfCookie(c *csrfProtector, value string) *http.Cookie { + return &http.Cookie{ + Name: c.opt.Cookie, + Value: value, + Path: c.opt.CookiePath, + Domain: c.opt.CookieDomain, + MaxAge: c.opt.CookieLifeTime, + Secure: c.opt.Secure, + HttpOnly: c.opt.CookieHTTPOnly, + SameSite: c.opt.SameSite, + } +} + // PrepareCSRFProtector returns a CSRFProtector to be used for every request. // Additionally, depending on options set, generated tokens will be sent via Header and/or Cookie. func PrepareCSRFProtector(opt CsrfOptions, ctx *Context) CSRFProtector { opt = prepareDefaultCsrfOptions(opt) - x := &csrfProtector{ - Secret: opt.Secret, - Header: opt.Header, - Form: opt.Form, - Cookie: opt.Cookie, - CookieDomain: opt.CookieDomain, - CookiePath: opt.CookiePath, - CookieHTTPOnly: opt.CookieHTTPOnly, - } + x := &csrfProtector{opt: opt} if opt.Origin && len(ctx.Req.Header.Get("Origin")) > 0 { return x @@ -175,7 +173,7 @@ func PrepareCSRFProtector(opt CsrfOptions, ctx *Context) CSRFProtector { oldUID := ctx.Session.Get(opt.oldSessionKey) uidChanged := oldUID == nil || oldUID.(string) != x.ID - cookieToken := ctx.GetCookie(opt.Cookie) + cookieToken := ctx.GetSiteCookie(opt.Cookie) needsNew := true if uidChanged { @@ -193,21 +191,10 @@ func PrepareCSRFProtector(opt CsrfOptions, ctx *Context) CSRFProtector { if needsNew { // FIXME: actionId. - x.Token = GenerateCsrfToken(x.Secret, x.ID, "POST", time.Now()) + x.Token = GenerateCsrfToken(x.opt.Secret, x.ID, "POST", time.Now()) if opt.SetCookie { - var expires interface{} - if opt.CookieLifeTime == 0 { - expires = time.Now().Add(CsrfTokenTimeout) - } - middleware.SetCookie(ctx.Resp, opt.Cookie, x.Token, - opt.CookieLifeTime, - opt.CookiePath, - opt.CookieDomain, - opt.Secure, - opt.CookieHTTPOnly, - expires, - middleware.SameSite(opt.SameSite), - ) + cookie := newCsrfCookie(x, x.Token) + ctx.Resp.Header().Add("Set-Cookie", cookie.String()) } } @@ -218,8 +205,8 @@ func PrepareCSRFProtector(opt CsrfOptions, ctx *Context) CSRFProtector { } func (c *csrfProtector) validateToken(ctx *Context, token string) { - if !ValidCsrfToken(token, c.Secret, c.ID, "POST", time.Now()) { - middleware.DeleteCSRFCookie(ctx.Resp) + if !ValidCsrfToken(token, c.opt.Secret, c.ID, "POST", time.Now()) { + c.DeleteCookie(ctx) if middleware.IsAPIPath(ctx.Req) { // currently, there should be no access to the APIPath with CSRF token. because templates shouldn't use the `/api/` endpoints. http.Error(ctx.Resp, "Invalid CSRF token.", http.StatusBadRequest) @@ -245,3 +232,11 @@ func (c *csrfProtector) Validate(ctx *Context) { } c.validateToken(ctx, "") // no csrf token, use an empty token to respond error } + +func (c *csrfProtector) DeleteCookie(ctx *Context) { + if c.opt.SetCookie { + cookie := newCsrfCookie(c, "") + cookie.MaxAge = -1 + ctx.Resp.Header().Add("Set-Cookie", cookie.String()) + } +} diff --git a/modules/setting/session.go b/modules/setting/session.go index b8498335d9867..d0bc938973ac1 100644 --- a/modules/setting/session.go +++ b/modules/setting/session.go @@ -21,7 +21,7 @@ var SessionConfig = struct { ProviderConfig string // Cookie name to save session ID. Default is "MacaronSession". CookieName string - // Cookie path to store. Default is "/". + // Cookie path to store. Default is "/". HINT: there was a bug, the old value doesn't have trailing slash, and could be empty "". CookiePath string // GC interval time in seconds. Default is 3600. Gclifetime int64 @@ -49,7 +49,7 @@ func loadSessionFrom(rootCfg ConfigProvider) { SessionConfig.ProviderConfig = path.Join(AppWorkPath, SessionConfig.ProviderConfig) } SessionConfig.CookieName = sec.Key("COOKIE_NAME").MustString("i_like_gitea") - SessionConfig.CookiePath = AppSubURL + SessionConfig.CookiePath = AppSubURL + "/" // there was a bug, old code only set CookePath=AppSubURL, no trailing slash SessionConfig.Secure = sec.Key("COOKIE_SECURE").MustBool(false) SessionConfig.Gclifetime = sec.Key("GC_INTERVAL_TIME").MustInt64(86400) SessionConfig.Maxlifetime = sec.Key("SESSION_LIFE_TIME").MustInt64(86400) diff --git a/modules/web/middleware/cookie.go b/modules/web/middleware/cookie.go index 7c1aaf6daf734..621640895b95f 100644 --- a/modules/web/middleware/cookie.go +++ b/modules/web/middleware/cookie.go @@ -7,184 +7,23 @@ package middleware import ( "net/http" "net/url" - "time" + "strings" "code.gitea.io/gitea/modules/setting" ) -// MaxAge sets the maximum age for a provided cookie -func MaxAge(maxAge int) func(*http.Cookie) { - return func(c *http.Cookie) { - c.MaxAge = maxAge - } -} - -// Path sets the path for a provided cookie -func Path(path string) func(*http.Cookie) { - return func(c *http.Cookie) { - c.Path = path - } -} - -// Domain sets the domain for a provided cookie -func Domain(domain string) func(*http.Cookie) { - return func(c *http.Cookie) { - c.Domain = domain - } -} - -// Secure sets the secure setting for a provided cookie -func Secure(secure bool) func(*http.Cookie) { - return func(c *http.Cookie) { - c.Secure = secure - } -} - -// HTTPOnly sets the HttpOnly setting for a provided cookie -func HTTPOnly(httpOnly bool) func(*http.Cookie) { - return func(c *http.Cookie) { - c.HttpOnly = httpOnly - } -} - -// Expires sets the expires and rawexpires for a provided cookie -func Expires(expires time.Time) func(*http.Cookie) { - return func(c *http.Cookie) { - c.Expires = expires - c.RawExpires = expires.Format(time.UnixDate) - } -} - -// SameSite sets the SameSite for a provided cookie -func SameSite(sameSite http.SameSite) func(*http.Cookie) { - return func(c *http.Cookie) { - c.SameSite = sameSite - } -} - -// NewCookie creates a cookie -func NewCookie(name, value string, maxAge int) *http.Cookie { - return &http.Cookie{ - Name: name, - Value: value, - HttpOnly: true, - Path: setting.SessionConfig.CookiePath, - Domain: setting.SessionConfig.Domain, - MaxAge: maxAge, - Secure: setting.SessionConfig.Secure, - } -} - // SetRedirectToCookie convenience function to set the RedirectTo cookie consistently func SetRedirectToCookie(resp http.ResponseWriter, value string) { - SetCookie(resp, "redirect_to", value, - 0, - setting.AppSubURL, - "", - setting.SessionConfig.Secure, - true, - SameSite(setting.SessionConfig.SameSite)) + SetSiteCookie(resp, "redirect_to", value, 0) } // DeleteRedirectToCookie convenience function to delete most cookies consistently func DeleteRedirectToCookie(resp http.ResponseWriter) { - SetCookie(resp, "redirect_to", "", - -1, - setting.AppSubURL, - "", - setting.SessionConfig.Secure, - true, - SameSite(setting.SessionConfig.SameSite)) + SetSiteCookie(resp, "redirect_to", "", -1) } -// DeleteCSRFCookie convenience function to delete SessionConfigPath cookies consistently -func DeleteCSRFCookie(resp http.ResponseWriter) { - SetCookie(resp, setting.CSRFCookieName, "", - -1, - setting.SessionConfig.CookiePath, - setting.SessionConfig.Domain) // FIXME: Do we need to set the Secure, httpOnly and SameSite values too? -} - -// SetCookie set the cookies. (name, value, lifetime, path, domain, secure, httponly, expires, {sameSite, ...}) -// TODO: Copied from gitea.com/macaron/macaron and should be improved after macaron removed. -func SetCookie(resp http.ResponseWriter, name, value string, others ...interface{}) { - cookie := http.Cookie{} - cookie.Name = name - cookie.Value = url.QueryEscape(value) - - if len(others) > 0 { - switch v := others[0].(type) { - case int: - cookie.MaxAge = v - case int64: - cookie.MaxAge = int(v) - case int32: - cookie.MaxAge = int(v) - case func(*http.Cookie): - v(&cookie) - } - } - - cookie.Path = "/" - if len(others) > 1 { - if v, ok := others[1].(string); ok && len(v) > 0 { - cookie.Path = v - } else if v, ok := others[1].(func(*http.Cookie)); ok { - v(&cookie) - } - } - - if len(others) > 2 { - if v, ok := others[2].(string); ok && len(v) > 0 { - cookie.Domain = v - } else if v, ok := others[2].(func(*http.Cookie)); ok { - v(&cookie) - } - } - - if len(others) > 3 { - switch v := others[3].(type) { - case bool: - cookie.Secure = v - case func(*http.Cookie): - v(&cookie) - default: - if others[3] != nil { - cookie.Secure = true - } - } - } - - if len(others) > 4 { - if v, ok := others[4].(bool); ok && v { - cookie.HttpOnly = true - } else if v, ok := others[4].(func(*http.Cookie)); ok { - v(&cookie) - } - } - - if len(others) > 5 { - if v, ok := others[5].(time.Time); ok { - cookie.Expires = v - cookie.RawExpires = v.Format(time.UnixDate) - } else if v, ok := others[5].(func(*http.Cookie)); ok { - v(&cookie) - } - } - - if len(others) > 6 { - for _, other := range others[6:] { - if v, ok := other.(func(*http.Cookie)); ok { - v(&cookie) - } - } - } - - resp.Header().Add("Set-Cookie", cookie.String()) -} - -// GetCookie returns given cookie value from request header. -func GetCookie(req *http.Request, name string) string { +// GetSiteCookie returns given cookie value from request header. +func GetSiteCookie(req *http.Request, name string) string { cookie, err := req.Cookie(name) if err != nil { return "" @@ -192,3 +31,24 @@ func GetCookie(req *http.Request, name string) string { val, _ := url.QueryUnescape(cookie.Value) return val } + +// SetSiteCookie returns given cookie value from request header. +func SetSiteCookie(resp http.ResponseWriter, name, value string, maxAge int) { + cookie := &http.Cookie{ + Name: name, + Value: url.QueryEscape(value), + MaxAge: maxAge, + Path: setting.SessionConfig.CookiePath, + Domain: setting.SessionConfig.Domain, + Secure: setting.SessionConfig.Secure, + HttpOnly: true, + SameSite: setting.SessionConfig.SameSite, + } + resp.Header().Add("Set-Cookie", cookie.String()) + if maxAge < 0 { + // There was a bug in "setting.SessionConfig.CookiePath" code, the old default value of it was empty "". + // So we have to delete the cookie on path="" again, because some old code leaves cookies on path="". + cookie.Path = strings.TrimSuffix(setting.SessionConfig.CookiePath, "/") + resp.Header().Add("Set-Cookie", cookie.String()) + } +} diff --git a/modules/web/middleware/locale.go b/modules/web/middleware/locale.go index f60be4bbdb476..34a16f04e7fac 100644 --- a/modules/web/middleware/locale.go +++ b/modules/web/middleware/locale.go @@ -6,7 +6,6 @@ package middleware import ( "net/http" - "code.gitea.io/gitea/modules/setting" "code.gitea.io/gitea/modules/translation" "code.gitea.io/gitea/modules/translation/i18n" @@ -49,23 +48,12 @@ func Locale(resp http.ResponseWriter, req *http.Request) translation.Locale { } // SetLocaleCookie convenience function to set the locale cookie consistently -func SetLocaleCookie(resp http.ResponseWriter, lang string, expiry int) { - SetCookie(resp, "lang", lang, expiry, - setting.AppSubURL, - setting.SessionConfig.Domain, - setting.SessionConfig.Secure, - true, - SameSite(setting.SessionConfig.SameSite)) +func SetLocaleCookie(resp http.ResponseWriter, lang string, maxAge int) { + SetSiteCookie(resp, "lang", lang, maxAge) } // DeleteLocaleCookie convenience function to delete the locale cookie consistently // Setting the lang cookie will trigger the middleware to reset the language to previous state. func DeleteLocaleCookie(resp http.ResponseWriter) { - SetCookie(resp, "lang", "", - -1, - setting.AppSubURL, - setting.SessionConfig.Domain, - setting.SessionConfig.Secure, - true, - SameSite(setting.SessionConfig.SameSite)) + SetSiteCookie(resp, "lang", "", -1) } diff --git a/routers/install/install.go b/routers/install/install.go index 8e2d19c73271e..8f8656230a487 100644 --- a/routers/install/install.go +++ b/routers/install/install.go @@ -559,7 +559,7 @@ func SubmitInstall(ctx *context.Context) { } days := 86400 * setting.LogInRememberDays - ctx.SetCookie(setting.CookieUserName, u.Name, days) + ctx.SetSiteCookie(setting.CookieUserName, u.Name, days) ctx.SetSuperSecureCookie(base.EncodeMD5(u.Rands+u.Passwd), setting.CookieRememberName, u.Name, days) diff --git a/routers/web/auth/auth.go b/routers/web/auth/auth.go index 5fba632817e62..d8042afeccb2f 100644 --- a/routers/web/auth/auth.go +++ b/routers/web/auth/auth.go @@ -49,7 +49,7 @@ func AutoSignIn(ctx *context.Context) (bool, error) { return false, nil } - uname := ctx.GetCookie(setting.CookieUserName) + uname := ctx.GetSiteCookie(setting.CookieUserName) if len(uname) == 0 { return false, nil } @@ -58,8 +58,8 @@ func AutoSignIn(ctx *context.Context) (bool, error) { defer func() { if !isSucceed { log.Trace("auto-login cookie cleared: %s", uname) - ctx.DeleteCookie(setting.CookieUserName) - ctx.DeleteCookie(setting.CookieRememberName) + ctx.DeleteSiteCookie(setting.CookieUserName) + ctx.DeleteSiteCookie(setting.CookieRememberName) } }() @@ -90,7 +90,7 @@ func AutoSignIn(ctx *context.Context) (bool, error) { return false, err } - middleware.DeleteCSRFCookie(ctx.Resp) + ctx.Csrf.DeleteCookie(ctx) return true, nil } @@ -125,7 +125,7 @@ func checkAutoLogin(ctx *context.Context) bool { if len(redirectTo) > 0 { middleware.SetRedirectToCookie(ctx.Resp, redirectTo) } else { - redirectTo = ctx.GetCookie("redirect_to") + redirectTo = ctx.GetSiteCookie("redirect_to") } if isSucceed { @@ -291,7 +291,7 @@ func handleSignIn(ctx *context.Context, u *user_model.User, remember bool) { func handleSignInFull(ctx *context.Context, u *user_model.User, remember, obeyRedirect bool) string { if remember { days := 86400 * setting.LogInRememberDays - ctx.SetCookie(setting.CookieUserName, u.Name, days) + ctx.SetSiteCookie(setting.CookieUserName, u.Name, days) ctx.SetSuperSecureCookie(base.EncodeMD5(u.Rands+u.Passwd), setting.CookieRememberName, u.Name, days) } @@ -330,7 +330,7 @@ func handleSignInFull(ctx *context.Context, u *user_model.User, remember, obeyRe } // Clear whatever CSRF cookie has right now, force to generate a new one - middleware.DeleteCSRFCookie(ctx.Resp) + ctx.Csrf.DeleteCookie(ctx) // Register last login u.SetLastLogin() @@ -339,7 +339,7 @@ func handleSignInFull(ctx *context.Context, u *user_model.User, remember, obeyRe return setting.AppSubURL + "/" } - if redirectTo := ctx.GetCookie("redirect_to"); len(redirectTo) > 0 && !utils.IsExternalURL(redirectTo) { + if redirectTo := ctx.GetSiteCookie("redirect_to"); len(redirectTo) > 0 && !utils.IsExternalURL(redirectTo) { middleware.DeleteRedirectToCookie(ctx.Resp) if obeyRedirect { ctx.RedirectToFirst(redirectTo) @@ -368,10 +368,9 @@ func getUserName(gothUser *goth.User) string { func HandleSignOut(ctx *context.Context) { _ = ctx.Session.Flush() _ = ctx.Session.Destroy(ctx.Resp, ctx.Req) - ctx.DeleteCookie(setting.CookieUserName) - ctx.DeleteCookie(setting.CookieRememberName) - middleware.DeleteCSRFCookie(ctx.Resp) - middleware.DeleteLocaleCookie(ctx.Resp) + ctx.DeleteSiteCookie(setting.CookieUserName) + ctx.DeleteSiteCookie(setting.CookieRememberName) + ctx.Csrf.DeleteCookie(ctx) middleware.DeleteRedirectToCookie(ctx.Resp) } diff --git a/routers/web/auth/oauth.go b/routers/web/auth/oauth.go index b3c4a234c1ef6..287136e64c2b9 100644 --- a/routers/web/auth/oauth.go +++ b/routers/web/auth/oauth.go @@ -1112,7 +1112,7 @@ func handleOAuth2SignIn(ctx *context.Context, source *auth.Source, u *user_model } // Clear whatever CSRF cookie has right now, force to generate a new one - middleware.DeleteCSRFCookie(ctx.Resp) + ctx.Csrf.DeleteCookie(ctx) // Register last login u.SetLastLogin() @@ -1148,7 +1148,7 @@ func handleOAuth2SignIn(ctx *context.Context, source *auth.Source, u *user_model return } - if redirectTo := ctx.GetCookie("redirect_to"); len(redirectTo) > 0 { + if redirectTo := ctx.GetSiteCookie("redirect_to"); len(redirectTo) > 0 { middleware.DeleteRedirectToCookie(ctx.Resp) ctx.RedirectToFirst(redirectTo) return diff --git a/routers/web/auth/openid.go b/routers/web/auth/openid.go index aff2e5f780857..5e0e7b258f665 100644 --- a/routers/web/auth/openid.go +++ b/routers/web/auth/openid.go @@ -47,7 +47,7 @@ func SignInOpenID(ctx *context.Context) { if len(redirectTo) > 0 { middleware.SetRedirectToCookie(ctx.Resp, redirectTo) } else { - redirectTo = ctx.GetCookie("redirect_to") + redirectTo = ctx.GetSiteCookie("redirect_to") } if isSucceed { diff --git a/routers/web/auth/password.go b/routers/web/auth/password.go index a5aa9c5344ddb..ed0412d745005 100644 --- a/routers/web/auth/password.go +++ b/routers/web/auth/password.go @@ -336,7 +336,7 @@ func MustChangePasswordPost(ctx *context.Context) { log.Trace("User updated password: %s", u.Name) - if redirectTo := ctx.GetCookie("redirect_to"); len(redirectTo) > 0 && !utils.IsExternalURL(redirectTo) { + if redirectTo := ctx.GetSiteCookie("redirect_to"); len(redirectTo) > 0 && !utils.IsExternalURL(redirectTo) { middleware.DeleteRedirectToCookie(ctx.Resp) ctx.RedirectToFirst(redirectTo) return diff --git a/routers/web/home.go b/routers/web/home.go index ecfecf6e67e4a..b94e3e9eb593d 100644 --- a/routers/web/home.go +++ b/routers/web/home.go @@ -54,7 +54,7 @@ func Home(ctx *context.Context) { } // Check auto-login. - uname := ctx.GetCookie(setting.CookieUserName) + uname := ctx.GetSiteCookie(setting.CookieUserName) if len(uname) != 0 { ctx.Redirect(setting.AppSubURL + "/user/login") return diff --git a/services/auth/auth.go b/services/auth/auth.go index 00e277c41abb2..905c776e5871e 100644 --- a/services/auth/auth.go +++ b/services/auth/auth.go @@ -13,6 +13,7 @@ import ( "code.gitea.io/gitea/models/db" user_model "code.gitea.io/gitea/models/user" "code.gitea.io/gitea/modules/auth/webauthn" + gitea_context "code.gitea.io/gitea/modules/context" "code.gitea.io/gitea/modules/log" "code.gitea.io/gitea/modules/session" "code.gitea.io/gitea/modules/setting" @@ -91,5 +92,7 @@ func handleSignIn(resp http.ResponseWriter, req *http.Request, sess SessionStore middleware.SetLocaleCookie(resp, user.Language, 0) // Clear whatever CSRF has right now, force to generate a new one - middleware.DeleteCSRFCookie(resp) + if ctx := gitea_context.GetContext(req); ctx != nil { + ctx.Csrf.DeleteCookie(ctx) + } } diff --git a/services/auth/sspi_windows.go b/services/auth/sspi_windows.go index b6e8d42980914..176f4f574f56a 100644 --- a/services/auth/sspi_windows.go +++ b/services/auth/sspi_windows.go @@ -13,9 +13,9 @@ import ( "code.gitea.io/gitea/models/avatars" user_model "code.gitea.io/gitea/models/user" "code.gitea.io/gitea/modules/base" + gitea_context "code.gitea.io/gitea/modules/context" "code.gitea.io/gitea/modules/log" "code.gitea.io/gitea/modules/setting" - "code.gitea.io/gitea/modules/templates" "code.gitea.io/gitea/modules/util" "code.gitea.io/gitea/modules/web/middleware" "code.gitea.io/gitea/services/auth/source/sspi" @@ -46,9 +46,7 @@ var ( // via the built-in SSPI module in Windows for SPNEGO authentication. // On successful authentication returns a valid user object. // Returns nil if authentication fails. -type SSPI struct { - rnd *templates.HTMLRender -} +type SSPI struct{} // Init creates a new global websspi.Authenticator object func (s *SSPI) Init(ctx context.Context) error { @@ -58,7 +56,6 @@ func (s *SSPI) Init(ctx context.Context) error { if err != nil { return err } - _, s.rnd = templates.HTMLRenderer(ctx) return nil } @@ -101,12 +98,9 @@ func (s *SSPI) Verify(req *http.Request, w http.ResponseWriter, store DataStore, } store.GetData()["EnableOpenIDSignIn"] = setting.Service.EnableOpenIDSignIn store.GetData()["EnableSSPI"] = true - - err := s.rnd.HTML(w, http.StatusUnauthorized, string(tplSignIn), templates.BaseVars().Merge(store.GetData())) - if err != nil { - log.Error("%v", err) - } - + // in this case, the store is Gitea's web Context + // FIXME: it doesn't look good to render the page here, why not redirect? + store.(*gitea_context.Context).HTML(http.StatusUnauthorized, tplSignIn) return nil, err } if outToken != "" { diff --git a/tests/integration/editor_test.go b/tests/integration/editor_test.go index 791506d8f7d6f..495290ed569cb 100644 --- a/tests/integration/editor_test.go +++ b/tests/integration/editor_test.go @@ -10,6 +10,7 @@ import ( "path" "testing" + gitea_context "code.gitea.io/gitea/modules/context" "code.gitea.io/gitea/modules/json" "github.com/stretchr/testify/assert" @@ -52,7 +53,7 @@ func TestCreateFileOnProtectedBranch(t *testing.T) { }) session.MakeRequest(t, req, http.StatusSeeOther) // Check if master branch has been locked successfully - flashCookie := session.GetCookie("macaron_flash") + flashCookie := session.GetCookie(gitea_context.CookieNameFlash) assert.NotNil(t, flashCookie) assert.EqualValues(t, "success%3DBranch%2Bprotection%2Bfor%2Brule%2B%2527master%2527%2Bhas%2Bbeen%2Bupdated.", flashCookie.Value) @@ -92,7 +93,7 @@ func TestCreateFileOnProtectedBranch(t *testing.T) { assert.EqualValues(t, "/user2/repo1/settings/branches", res["redirect"]) // Check if master branch has been locked successfully - flashCookie = session.GetCookie("macaron_flash") + flashCookie = session.GetCookie(gitea_context.CookieNameFlash) assert.NotNil(t, flashCookie) assert.EqualValues(t, "error%3DRemoving%2Bbranch%2Bprotection%2Brule%2B%25271%2527%2Bfailed.", flashCookie.Value) }) diff --git a/tests/integration/git_test.go b/tests/integration/git_test.go index d21f3994a1d98..95c6d83e54a02 100644 --- a/tests/integration/git_test.go +++ b/tests/integration/git_test.go @@ -23,6 +23,7 @@ import ( repo_model "code.gitea.io/gitea/models/repo" "code.gitea.io/gitea/models/unittest" user_model "code.gitea.io/gitea/models/user" + gitea_context "code.gitea.io/gitea/modules/context" "code.gitea.io/gitea/modules/git" "code.gitea.io/gitea/modules/lfs" "code.gitea.io/gitea/modules/setting" @@ -436,7 +437,7 @@ func doProtectBranch(ctx APITestContext, branch, userToWhitelist, unprotectedFil ctx.Session.MakeRequest(t, req, http.StatusSeeOther) } // Check if master branch has been locked successfully - flashCookie := ctx.Session.GetCookie("macaron_flash") + flashCookie := ctx.Session.GetCookie(gitea_context.CookieNameFlash) assert.NotNil(t, flashCookie) assert.EqualValues(t, "success%3DBranch%2Bprotection%2Bfor%2Brule%2B%2527"+url.QueryEscape(branch)+"%2527%2Bhas%2Bbeen%2Bupdated.", flashCookie.Value) } diff --git a/tests/integration/integration_test.go b/tests/integration/integration_test.go index 716f6d44ab345..965bae576cc88 100644 --- a/tests/integration/integration_test.go +++ b/tests/integration/integration_test.go @@ -23,6 +23,7 @@ import ( "code.gitea.io/gitea/models/auth" "code.gitea.io/gitea/models/unittest" + gitea_context "code.gitea.io/gitea/modules/context" "code.gitea.io/gitea/modules/graceful" "code.gitea.io/gitea/modules/json" "code.gitea.io/gitea/modules/log" @@ -290,7 +291,7 @@ func getTokenForLoggedInUser(t testing.TB, session *TestSession, scopes ...auth. // Log the flash values on failure if !assert.Equal(t, resp.Result().Header["Location"], []string{"/user/settings/applications"}) { for _, cookie := range resp.Result().Cookies() { - if cookie.Name != "macaron_flash" { + if cookie.Name != gitea_context.CookieNameFlash { continue } flash, _ := url.ParseQuery(cookie.Value) diff --git a/tests/integration/mirror_push_test.go b/tests/integration/mirror_push_test.go index b2ec6c09325ae..9abae63b0ad71 100644 --- a/tests/integration/mirror_push_test.go +++ b/tests/integration/mirror_push_test.go @@ -15,6 +15,7 @@ import ( repo_model "code.gitea.io/gitea/models/repo" "code.gitea.io/gitea/models/unittest" user_model "code.gitea.io/gitea/models/user" + gitea_context "code.gitea.io/gitea/modules/context" "code.gitea.io/gitea/modules/git" "code.gitea.io/gitea/modules/repository" "code.gitea.io/gitea/modules/setting" @@ -91,7 +92,7 @@ func doCreatePushMirror(ctx APITestContext, address, username, password string) }) ctx.Session.MakeRequest(t, req, http.StatusSeeOther) - flashCookie := ctx.Session.GetCookie("macaron_flash") + flashCookie := ctx.Session.GetCookie(gitea_context.CookieNameFlash) assert.NotNil(t, flashCookie) assert.Contains(t, flashCookie.Value, "success") } @@ -112,7 +113,7 @@ func doRemovePushMirror(ctx APITestContext, address, username, password string, }) ctx.Session.MakeRequest(t, req, http.StatusSeeOther) - flashCookie := ctx.Session.GetCookie("macaron_flash") + flashCookie := ctx.Session.GetCookie(gitea_context.CookieNameFlash) assert.NotNil(t, flashCookie) assert.Contains(t, flashCookie.Value, "success") } From 985f76dc4b0692c4d6c6f37e82500ef859557c16 Mon Sep 17 00:00:00 2001 From: techknowlogick Date: Thu, 13 Apr 2023 18:41:04 -0400 Subject: [PATCH 06/28] Update redis library to support redis v7 (#24114) --- assets/go-licenses.json | 10 +++++----- go.mod | 4 +++- go.sum | 17 +++++++++++++++-- modules/cache/cache_redis.go | 2 +- modules/nosql/manager.go | 2 +- modules/nosql/manager_redis.go | 6 +----- modules/queue/queue_redis.go | 2 +- modules/queue/unique_queue_redis.go | 2 +- modules/session/redis.go | 2 +- 9 files changed, 29 insertions(+), 18 deletions(-) diff --git a/assets/go-licenses.json b/assets/go-licenses.json index 18b6fc48abc18..3e7db196f721d 100644 --- a/assets/go-licenses.json +++ b/assets/go-licenses.json @@ -419,11 +419,6 @@ "path": "github.com/go-ldap/ldap/v3/LICENSE", "licenseText": "The MIT License (MIT)\n\nCopyright (c) 2011-2015 Michael Mitton (mmitton@gmail.com)\nPortions copyright (c) 2015-2016 go-ldap Authors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, - { - "name": "github.com/go-redis/redis/v8", - "path": "github.com/go-redis/redis/v8/LICENSE", - "licenseText": "Copyright (c) 2013 The github.com/go-redis/redis Authors.\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are\nmet:\n\n * Redistributions of source code must retain the above copyright\nnotice, this list of conditions and the following disclaimer.\n * Redistributions in binary form must reproduce the above\ncopyright notice, this list of conditions and the following disclaimer\nin the documentation and/or other materials provided with the\ndistribution.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n\"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\nLIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\nA PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\nOWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\nSPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\nLIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\nDATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\nTHEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\nOF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n" - }, { "name": "github.com/go-sql-driver/mysql", "path": "github.com/go-sql-driver/mysql/LICENSE", @@ -849,6 +844,11 @@ "path": "github.com/prometheus/procfs/LICENSE", "licenseText": " Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright [yyyy] [name of copyright owner]\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n" }, + { + "name": "github.com/redis/go-redis/v9", + "path": "github.com/redis/go-redis/v9/LICENSE", + "licenseText": "Copyright (c) 2013 The github.com/redis/go-redis Authors.\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are\nmet:\n\n * Redistributions of source code must retain the above copyright\nnotice, this list of conditions and the following disclaimer.\n * Redistributions in binary form must reproduce the above\ncopyright notice, this list of conditions and the following disclaimer\nin the documentation and/or other materials provided with the\ndistribution.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n\"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\nLIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\nA PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\nOWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\nSPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\nLIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\nDATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\nTHEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\nOF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n" + }, { "name": "github.com/rhysd/actionlint", "path": "github.com/rhysd/actionlint/LICENSE.txt", diff --git a/go.mod b/go.mod index 0e73f2fe53cfe..944f6d2c918c8 100644 --- a/go.mod +++ b/go.mod @@ -44,7 +44,6 @@ require ( github.com/go-git/go-billy/v5 v5.4.1 github.com/go-git/go-git/v5 v5.5.2 github.com/go-ldap/ldap/v3 v3.4.4 - github.com/go-redis/redis/v8 v8.11.5 github.com/go-sql-driver/mysql v1.7.0 github.com/go-swagger/go-swagger v0.30.4 github.com/go-testfixtures/testfixtures/v3 v3.8.1 @@ -90,6 +89,7 @@ require ( github.com/pquerna/otp v1.4.0 github.com/prometheus/client_golang v1.14.0 github.com/quasoft/websspi v1.1.2 + github.com/redis/go-redis/v9 v9.0.3 github.com/santhosh-tekuri/jsonschema/v5 v5.2.0 github.com/sergi/go-diff v1.3.1 github.com/shurcooL/vfsgen v0.0.0-20200824052919-0d455de96546 @@ -231,6 +231,8 @@ require ( github.com/nwaples/rardecode v1.1.3 // indirect github.com/oklog/ulid v1.3.1 // indirect github.com/olekukonko/tablewriter v0.0.5 // indirect + github.com/onsi/ginkgo v1.16.5 // indirect + github.com/onsi/gomega v1.18.1 // indirect github.com/pelletier/go-toml v1.9.5 // indirect github.com/pelletier/go-toml/v2 v2.0.5 // indirect github.com/pierrec/lz4/v4 v4.1.17 // indirect diff --git a/go.sum b/go.sum index 6d765b0f2ccdf..df57c65918b05 100644 --- a/go.sum +++ b/go.sum @@ -213,6 +213,8 @@ github.com/boombuler/barcode v1.0.1/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl github.com/bradfitz/gomemcache v0.0.0-20190329173943-551aad21a668/go.mod h1:H0wQNHz2YrLsuXOZozoeDmnHXkNCRmMW0gwFWDfEZDA= github.com/bradfitz/gomemcache v0.0.0-20190913173617-a41fca850d0b h1:L/QXpzIa3pOvUGt1D1lA5KjYhPBAN/3iWdP7xeFS9F0= github.com/bradfitz/gomemcache v0.0.0-20190913173617-a41fca850d0b/go.mod h1:H0wQNHz2YrLsuXOZozoeDmnHXkNCRmMW0gwFWDfEZDA= +github.com/bsm/ginkgo/v2 v2.7.0 h1:ItPMPH90RbmZJt5GtkcNvIRuGEdwlBItdNVoyzaNQao= +github.com/bsm/gomega v1.26.0 h1:LhQm+AFcgV2M0WyKroMASzAzCAJVpAxQXv4SaI9a69Y= github.com/bufbuild/connect-go v1.3.1 h1:doJP6Q8Ypg6haUT2IAZJPWHUN9rAUp+F9MfK7yhu1zs= github.com/bufbuild/connect-go v1.3.1/go.mod h1:9iNvh/NOsfhNBUH5CtvXeVUskQO1xsrEviH7ZArwZ3I= github.com/buildkite/terminal-to-html/v3 v3.7.0 h1:chdLUSpiOj/A4v3dzxyOqixXI6aw7IDA6Dk77FXsvNU= @@ -436,8 +438,6 @@ github.com/go-openapi/validate v0.22.0 h1:b0QecH6VslW/TxtpKgzpO1SNG7GU2FsaqKdP1E github.com/go-openapi/validate v0.22.0/go.mod h1:rjnrwK57VJ7A8xqfpAOEKRH8yQSGUriMu5/zuPSQ1hg= github.com/go-redis/redis v6.15.2+incompatible/go.mod h1:NAIEuMOZ/fxfXJIrKDQDz8wamY7mA7PouImQ2Jvg6kA= github.com/go-redis/redis/v8 v8.4.0/go.mod h1:A1tbYoHSa1fXwN+//ljcCYYJeLmVrwL9hbQN45Jdy0M= -github.com/go-redis/redis/v8 v8.11.5 h1:AcZZR7igkdvfVmQTPnu9WE37LRrO/YrBH5zWyjDC0oI= -github.com/go-redis/redis/v8 v8.11.5/go.mod h1:gREzHqY1hg6oD9ngVRbLStwAWKhA0FEgq8Jd4h5lpwo= github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w= github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w= github.com/go-sql-driver/mysql v1.6.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg= @@ -447,6 +447,7 @@ github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/me github.com/go-swagger/go-swagger v0.30.4 h1:cPrWLSXY6ZdcgfRicOj0lANg72TkTHz6uv/OlUdzO5U= github.com/go-swagger/go-swagger v0.30.4/go.mod h1:YM5D5kR9c1ft3ynMXvDk2uo/7UZHKFEqKXcAL9f4Phc= github.com/go-swagger/scan-repo-boundary v0.0.0-20180623220736-973b3573c013 h1:l9rI6sNaZgNC0LnF3MiE+qTmyBA/tZAg1rtyrGbUMK0= +github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= github.com/go-test/deep v1.0.7 h1:/VSMRlnY/JSyqxQUzQLKVMAskpY/NZKFA5j2P+0pP2M= github.com/go-test/deep v1.0.7/go.mod h1:QV8Hv/iy04NyLBxAdO9njL0iVPN1S4d/A3NVv1V36o8= github.com/go-testfixtures/testfixtures/v3 v3.8.1 h1:uonwvepqRvSgddcrReZQhojTlWlmOlHkYAb9ZaOMWgU= @@ -583,6 +584,7 @@ github.com/google/pprof v0.0.0-20200905233945-acf8798be1f7/go.mod h1:ZgVRPoUq/hf github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20201218002935-b9804c9f04c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20211214055906-6f57359322fd/go.mod h1:KgnwoLYCZ8IQu3XUZ8Nc/bM9CCZFOyjUNOSygVozoDg= github.com/google/pprof v0.0.0-20230222194610-99052d3372e7 h1:pNFnpaSXfibgW7aUbk9pwLmI7LNwh/iR46x/YwN/lNg= github.com/google/pprof v0.0.0-20230222194610-99052d3372e7/go.mod h1:uglQLonpP8qtYCYyzA+8c/9qtqgA3qsXGYqCPKARAFg= @@ -939,6 +941,7 @@ github.com/nwaples/rardecode v1.1.3 h1:cWCaZwfM5H7nAD6PyEdcVnczzV8i/JtotnyW/dD9l github.com/nwaples/rardecode v1.1.3/go.mod h1:5DzqNKiOdpKKBH87u8VlvAnPZMXcGRhxWkRpHbbfGS0= github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A= github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE= +github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU= github.com/oklog/oklog v0.3.2/go.mod h1:FCV+B7mhrz4o+ueLpx+KqkyXRGMWOYEvfiXtdGtbWGs= github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA= github.com/oklog/ulid v1.3.1 h1:EGfNDEx6MqHz8B3uNV6QAib1UR2Lm97sHi3ocA6ESJ4= @@ -955,13 +958,18 @@ github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+W github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= github.com/onsi/ginkgo v1.14.2/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY= +github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0= github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE= +github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU= +github.com/onsi/ginkgo/v2 v2.0.0/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c= github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= github.com/onsi/gomega v1.5.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= github.com/onsi/gomega v1.10.3/go.mod h1:V9xEwhxec5O8UDM77eCW8vLymOMltsqPVYWrpDsH8xc= +github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY= github.com/onsi/gomega v1.18.1 h1:M1GfJqGRrBrrGGsbxzV5dqM2U2ApXefZCQpkukxYRLE= +github.com/onsi/gomega v1.18.1/go.mod h1:0q+aL8jAiMXy9hbwj2mr5GziHiwhAIQpFmmtT5hitRs= github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk= github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= @@ -1050,6 +1058,8 @@ github.com/quasoft/websspi v1.1.2 h1:/mA4w0LxWlE3novvsoEL6BBA1WnjJATbjkh1kFrTidw github.com/quasoft/websspi v1.1.2/go.mod h1:HmVdl939dQ0WIXZhyik+ARdI03M6bQzaSEKcgpFmewk= github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4= github.com/rcrowley/go-metrics v0.0.0-20190826022208-cac0b30c2563/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4= +github.com/redis/go-redis/v9 v9.0.3 h1:+7mmR26M0IvyLxGZUHxu4GiBkJkVDid0Un+j4ScYu4k= +github.com/redis/go-redis/v9 v9.0.3/go.mod h1:WqMKv5vnQbRuZstUwxQI195wHy+t4PuXDOjzMvcuQHk= github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 h1:OdAsTTz6OkFY5QxjkYwrChwuRruF69c169dPK26NUlk= github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= github.com/rhysd/actionlint v1.6.23 h1:041VOXgZddfvSJa9Il+WT3Iwuo/j0Nmu4bhpAScrds4= @@ -1391,6 +1401,7 @@ golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= golang.org/x/net v0.0.0-20210421230115-4e50805a0758/go.mod h1:72T/g9IO56b78aLF+1Kcs5dz7/ng1VjMUvfKvpfy+jM= +golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk= golang.org/x/net v0.0.0-20210501142056-aec3718b3fa0/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk= golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20210610132358-84b48f89b13b/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= @@ -1493,6 +1504,7 @@ golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20201126233918-771906719818/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210225134936-a50acf3fe073/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -1607,6 +1619,7 @@ golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4f golang.org/x/tools v0.0.0-20201124115921-2c860bdd6e78/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= diff --git a/modules/cache/cache_redis.go b/modules/cache/cache_redis.go index 8e056ff2ec5cf..f22482de48a2f 100644 --- a/modules/cache/cache_redis.go +++ b/modules/cache/cache_redis.go @@ -12,7 +12,7 @@ import ( "code.gitea.io/gitea/modules/nosql" "gitea.com/go-chi/cache" - "github.com/go-redis/redis/v8" + "github.com/redis/go-redis/v9" ) // RedisCacher represents a redis cache adapter implementation. diff --git a/modules/nosql/manager.go b/modules/nosql/manager.go index bc530bc148cce..31e43297dcd12 100644 --- a/modules/nosql/manager.go +++ b/modules/nosql/manager.go @@ -11,7 +11,7 @@ import ( "code.gitea.io/gitea/modules/process" - "github.com/go-redis/redis/v8" + "github.com/redis/go-redis/v9" "github.com/syndtr/goleveldb/leveldb" ) diff --git a/modules/nosql/manager_redis.go b/modules/nosql/manager_redis.go index 728bc2f2ef725..7066863b89f8f 100644 --- a/modules/nosql/manager_redis.go +++ b/modules/nosql/manager_redis.go @@ -13,7 +13,7 @@ import ( "code.gitea.io/gitea/modules/log" - "github.com/go-redis/redis/v8" + "github.com/redis/go-redis/v9" ) var replacer = strings.NewReplacer("_", "", "-", "") @@ -193,10 +193,6 @@ func getRedisOptions(uri *url.URL) *redis.UniversalOptions { opts.MinIdleConns, _ = strconv.Atoi(v[0]) case "pooltimeout": opts.PoolTimeout = valToTimeDuration(v) - case "idletimeout": - opts.IdleTimeout = valToTimeDuration(v) - case "idlecheckfrequency": - opts.IdleCheckFrequency = valToTimeDuration(v) case "maxredirects": opts.MaxRedirects, _ = strconv.Atoi(v[0]) case "readonly": diff --git a/modules/queue/queue_redis.go b/modules/queue/queue_redis.go index 039e95241cf5e..f8842fea9fa2a 100644 --- a/modules/queue/queue_redis.go +++ b/modules/queue/queue_redis.go @@ -10,7 +10,7 @@ import ( "code.gitea.io/gitea/modules/log" "code.gitea.io/gitea/modules/nosql" - "github.com/go-redis/redis/v8" + "github.com/redis/go-redis/v9" ) // RedisQueueType is the type for redis queue diff --git a/modules/queue/unique_queue_redis.go b/modules/queue/unique_queue_redis.go index 491ae5d15ea73..ae1df08ebd589 100644 --- a/modules/queue/unique_queue_redis.go +++ b/modules/queue/unique_queue_redis.go @@ -6,7 +6,7 @@ package queue import ( "context" - "github.com/go-redis/redis/v8" + "github.com/redis/go-redis/v9" ) // RedisUniqueQueueType is the type for redis queue diff --git a/modules/session/redis.go b/modules/session/redis.go index b7cdbef6f8bfa..322470743a510 100644 --- a/modules/session/redis.go +++ b/modules/session/redis.go @@ -26,7 +26,7 @@ import ( "code.gitea.io/gitea/modules/nosql" "gitea.com/go-chi/session" - "github.com/go-redis/redis/v8" + "github.com/redis/go-redis/v9" ) // RedisStore represents a redis session store implementation. From 334c899f7b79d639d618c0b2bd4417243d09347f Mon Sep 17 00:00:00 2001 From: wxiaoguang Date: Fri, 14 Apr 2023 07:17:27 +0800 Subject: [PATCH 07/28] Improve git log for debugging (#24095) --- modules/git/command.go | 54 +++++++++++++++++++++---------------- modules/git/command_test.go | 8 ++++++ modules/git/repo.go | 45 +++++++------------------------ 3 files changed, 48 insertions(+), 59 deletions(-) diff --git a/modules/git/command.go b/modules/git/command.go index 9a65279a8cb5f..a42d859f55f7b 100644 --- a/modules/git/command.go +++ b/modules/git/command.go @@ -40,7 +40,7 @@ const DefaultLocale = "C" // Command represents a command with its subcommands or arguments. type Command struct { - name string + prog string args []string parentContext context.Context desc string @@ -49,10 +49,28 @@ type Command struct { } func (c *Command) String() string { - if len(c.args) == 0 { - return c.name + return c.toString(false) +} + +func (c *Command) toString(sanitizing bool) string { + // WARNING: this function is for debugging purposes only. It's much better than old code (which only joins args with space), + // It's impossible to make a simple and 100% correct implementation of argument quoting for different platforms. + debugQuote := func(s string) string { + if strings.ContainsAny(s, " `'\"\t\r\n") { + return fmt.Sprintf("%q", s) + } + return s + } + a := make([]string, 0, len(c.args)+1) + a = append(a, debugQuote(c.prog)) + for _, arg := range c.args { + if sanitizing && (strings.Contains(arg, "://") && strings.Contains(arg, "@")) { + a = append(a, debugQuote(util.SanitizeCredentialURLs(arg))) + } else { + a = append(a, debugQuote(arg)) + } } - return fmt.Sprintf("%s %s", c.name, strings.Join(c.args, " ")) + return strings.Join(a, " ") } // NewCommand creates and returns a new Git Command based on given command and arguments. @@ -67,7 +85,7 @@ func NewCommand(ctx context.Context, args ...internal.CmdArg) *Command { cargs = append(cargs, string(arg)) } return &Command{ - name: GitExecutable, + prog: GitExecutable, args: cargs, parentContext: ctx, globalArgsLength: len(globalCommandArgs), @@ -82,7 +100,7 @@ func NewCommandContextNoGlobals(ctx context.Context, args ...internal.CmdArg) *C cargs = append(cargs, string(arg)) } return &Command{ - name: GitExecutable, + prog: GitExecutable, args: cargs, parentContext: ctx, } @@ -250,28 +268,18 @@ func (c *Command) Run(opts *RunOpts) error { } if len(opts.Dir) == 0 { - log.Debug("%s", c) + log.Debug("git.Command.Run: %s", c) } else { - log.Debug("%s: %v", opts.Dir, c) + log.Debug("git.Command.RunDir(%s): %s", opts.Dir, c) } desc := c.desc if desc == "" { - args := c.args[c.globalArgsLength:] - var argSensitiveURLIndexes []int - for i, arg := range c.args { - if strings.Contains(arg, "://") && strings.Contains(arg, "@") { - argSensitiveURLIndexes = append(argSensitiveURLIndexes, i) - } - } - if len(argSensitiveURLIndexes) > 0 { - args = make([]string, len(c.args)) - copy(args, c.args) - for _, urlArgIndex := range argSensitiveURLIndexes { - args[urlArgIndex] = util.SanitizeCredentialURLs(args[urlArgIndex]) - } + if opts.Dir == "" { + desc = fmt.Sprintf("git: %s", c.toString(true)) + } else { + desc = fmt.Sprintf("git(dir:%s): %s", opts.Dir, c.toString(true)) } - desc = fmt.Sprintf("%s %s [repo_path: %s]", c.name, strings.Join(args, " "), opts.Dir) } var ctx context.Context @@ -285,7 +293,7 @@ func (c *Command) Run(opts *RunOpts) error { } defer finished() - cmd := exec.CommandContext(ctx, c.name, c.args...) + cmd := exec.CommandContext(ctx, c.prog, c.args...) if opts.Env == nil { cmd.Env = os.Environ() } else { diff --git a/modules/git/command_test.go b/modules/git/command_test.go index 4e5f991d31125..9a6228c9ad05f 100644 --- a/modules/git/command_test.go +++ b/modules/git/command_test.go @@ -52,3 +52,11 @@ func TestGitArgument(t *testing.T) { assert.True(t, isSafeArgumentValue("x")) assert.False(t, isSafeArgumentValue("-x")) } + +func TestCommandString(t *testing.T) { + cmd := NewCommandContextNoGlobals(context.Background(), "a", "-m msg", "it's a test", `say "hello"`) + assert.EqualValues(t, cmd.prog+` a "-m msg" "it's a test" "say \"hello\""`, cmd.String()) + + cmd = NewCommandContextNoGlobals(context.Background(), "url: https://a:b@c/") + assert.EqualValues(t, cmd.prog+` "url: https://sanitized-credential@c/"`, cmd.toString(true)) +} diff --git a/modules/git/repo.go b/modules/git/repo.go index 233f7f20cfc2f..d29ec40ae2ab0 100644 --- a/modules/git/repo.go +++ b/modules/git/repo.go @@ -209,49 +209,22 @@ func Push(ctx context.Context, repoPath string, opts PushOptions) error { } else { cmd.SetDescription(fmt.Sprintf("push branch %s to %s (force: %t, mirror: %t)", opts.Branch, opts.Remote, opts.Force, opts.Mirror)) } - var outbuf, errbuf strings.Builder - if opts.Timeout == 0 { - opts.Timeout = -1 - } - - err := cmd.Run(&RunOpts{ - Env: opts.Env, - Timeout: opts.Timeout, - Dir: repoPath, - Stdout: &outbuf, - Stderr: &errbuf, - }) + stdout, stderr, err := cmd.RunStdString(&RunOpts{Env: opts.Env, Timeout: opts.Timeout, Dir: repoPath}) if err != nil { - if strings.Contains(errbuf.String(), "non-fast-forward") { - return &ErrPushOutOfDate{ - StdOut: outbuf.String(), - StdErr: errbuf.String(), - Err: err, - } - } else if strings.Contains(errbuf.String(), "! [remote rejected]") { - err := &ErrPushRejected{ - StdOut: outbuf.String(), - StdErr: errbuf.String(), - Err: err, - } + if strings.Contains(stderr, "non-fast-forward") { + return &ErrPushOutOfDate{StdOut: stdout, StdErr: stderr, Err: err} + } else if strings.Contains(stderr, "! [remote rejected]") { + err := &ErrPushRejected{StdOut: stdout, StdErr: stderr, Err: err} err.GenerateMessage() return err - } else if strings.Contains(errbuf.String(), "matches more than one") { - err := &ErrMoreThanOne{ - StdOut: outbuf.String(), - StdErr: errbuf.String(), - Err: err, - } - return err + } else if strings.Contains(stderr, "matches more than one") { + return &ErrMoreThanOne{StdOut: stdout, StdErr: stderr, Err: err} } + return fmt.Errorf("push failed: %w - %s\n%s", err, stderr, stdout) } - if errbuf.Len() > 0 && err != nil { - return fmt.Errorf("%w - %s", err, errbuf.String()) - } - - return err + return nil } // GetLatestCommitTime returns time for latest commit in repository (across all branches) From 5f4cd715abb50aea4d82416cd9a78dff7d670841 Mon Sep 17 00:00:00 2001 From: GiteaBot Date: Fri, 14 Apr 2023 00:07:25 +0000 Subject: [PATCH 08/28] [skip ci] Updated translations via Crowdin --- options/locale/locale_cs-CZ.ini | 1 - options/locale/locale_de-DE.ini | 1 - options/locale/locale_el-GR.ini | 1 - options/locale/locale_es-ES.ini | 1 - options/locale/locale_fa-IR.ini | 1 - options/locale/locale_fr-FR.ini | 1 - options/locale/locale_is-IS.ini | 1 - options/locale/locale_it-IT.ini | 1 - options/locale/locale_ja-JP.ini | 1 - options/locale/locale_lv-LV.ini | 1 - options/locale/locale_nl-NL.ini | 1 - options/locale/locale_pl-PL.ini | 1 - options/locale/locale_pt-BR.ini | 1 - options/locale/locale_pt-PT.ini | 5 ++++- options/locale/locale_ru-RU.ini | 25 ++++++++++++++++++++++++- options/locale/locale_si-LK.ini | 1 - options/locale/locale_tr-TR.ini | 1 - options/locale/locale_uk-UA.ini | 1 - options/locale/locale_zh-CN.ini | 1 - options/locale/locale_zh-TW.ini | 1 - 20 files changed, 28 insertions(+), 20 deletions(-) diff --git a/options/locale/locale_cs-CZ.ini b/options/locale/locale_cs-CZ.ini index 249aa581bf48b..5f6bfa76bca2d 100644 --- a/options/locale/locale_cs-CZ.ini +++ b/options/locale/locale_cs-CZ.ini @@ -1652,7 +1652,6 @@ pulls.delete.text=Opravdu chcete tento požadavek na natažení smazat? (Tím se milestones.new=Nový milník milestones.closed=Zavřen dne %s -milestones.update_ago=Aktualizováno před %s milestones.no_due_date=Bez lhůty dokončení milestones.open=Otevřít milestones.close=Zavřít diff --git a/options/locale/locale_de-DE.ini b/options/locale/locale_de-DE.ini index 9d49a420e4758..28fc247756724 100644 --- a/options/locale/locale_de-DE.ini +++ b/options/locale/locale_de-DE.ini @@ -1598,7 +1598,6 @@ pulls.delete.text=Willst du diesen Pull-Request wirklich löschen? (Dies wird de milestones.new=Neuer Meilenstein milestones.closed=Geschlossen %s -milestones.update_ago=Vor %s aktualisiert milestones.no_due_date=Kein Fälligkeitsdatum milestones.open=Öffnen milestones.close=Schließen diff --git a/options/locale/locale_el-GR.ini b/options/locale/locale_el-GR.ini index f36f30d9d56d4..8b7d03dac3292 100644 --- a/options/locale/locale_el-GR.ini +++ b/options/locale/locale_el-GR.ini @@ -1670,7 +1670,6 @@ pulls.delete.text=Θέλετε πραγματικά να διαγράψετε α milestones.new=Νέο Ορόσημο milestones.closed=Έκλεισε %s -milestones.update_ago=Ενημερώθηκε πριν από %s milestones.no_due_date=Δεν υπάρχει ημερομηνία παράδοσης milestones.open=Άνοιγμα milestones.close=Κλείσιμο diff --git a/options/locale/locale_es-ES.ini b/options/locale/locale_es-ES.ini index 9d9df7e32acd4..719e4031fe3d8 100644 --- a/options/locale/locale_es-ES.ini +++ b/options/locale/locale_es-ES.ini @@ -1623,7 +1623,6 @@ pulls.delete.text=¿Realmente quieres eliminar esta pull request? (Esto eliminar milestones.new=Nuevo hito milestones.closed=Cerrada %s -milestones.update_ago=Actualizado hace %s milestones.no_due_date=Sin fecha límite milestones.open=Abrir milestones.close=Cerrar diff --git a/options/locale/locale_fa-IR.ini b/options/locale/locale_fa-IR.ini index 439b4cc2d6763..b9be06c23dc61 100644 --- a/options/locale/locale_fa-IR.ini +++ b/options/locale/locale_fa-IR.ini @@ -1476,7 +1476,6 @@ pulls.merge_instruction_step2_desc=تغییرات را ادغام کنید و د milestones.new=نقطه عطف جدید milestones.closed=%s بسته شد -milestones.update_ago=آخرین بروز رسانی %s قبل milestones.no_due_date=بدون موعد مقرر milestones.open=باز milestones.close=بستن diff --git a/options/locale/locale_fr-FR.ini b/options/locale/locale_fr-FR.ini index 3bda14ba73f61..fdda0d28918a8 100644 --- a/options/locale/locale_fr-FR.ini +++ b/options/locale/locale_fr-FR.ini @@ -1338,7 +1338,6 @@ pulls.delete.text=Voulez-vous vraiment supprimer cet demande d'ajout ? (Cela sup milestones.new=Nouveau jalon milestones.closed=%s fermé -milestones.update_ago=Mis à jour il y a %s milestones.no_due_date=Aucune date d'échéance milestones.open=Ouvrir milestones.close=Fermer diff --git a/options/locale/locale_is-IS.ini b/options/locale/locale_is-IS.ini index ff4250761560c..ade8d84efc52d 100644 --- a/options/locale/locale_is-IS.ini +++ b/options/locale/locale_is-IS.ini @@ -944,7 +944,6 @@ pulls.status_checks_details=Nánar milestones.new=Nýtt tímamót milestones.closed=Lokaði %s -milestones.update_ago=Uppfært fyrir %s milestones.no_due_date=Enginn eindagi milestones.open=Opna milestones.close=Loka diff --git a/options/locale/locale_it-IT.ini b/options/locale/locale_it-IT.ini index 21caf3957c248..90477cc95bcbc 100644 --- a/options/locale/locale_it-IT.ini +++ b/options/locale/locale_it-IT.ini @@ -1608,7 +1608,6 @@ pulls.delete.text=Vuoi davvero eliminare questo problema? (Questo rimuoverà per milestones.new=Nuova Milestone milestones.closed=Chiuso %s -milestones.update_ago=Aggiornato %s fa milestones.no_due_date=Nessuna data di scadenza milestones.open=Apri milestones.close=Chiudi diff --git a/options/locale/locale_ja-JP.ini b/options/locale/locale_ja-JP.ini index a3199d72f1d1d..2976b2691f197 100644 --- a/options/locale/locale_ja-JP.ini +++ b/options/locale/locale_ja-JP.ini @@ -1696,7 +1696,6 @@ pulls.delete.text=本当にこのプルリクエストを削除しますか? ( milestones.new=新しいマイルストーン milestones.closed=%s にクローズ -milestones.update_ago=%s 前に更新 milestones.no_due_date=期日なし milestones.open=オープン milestones.close=クローズ diff --git a/options/locale/locale_lv-LV.ini b/options/locale/locale_lv-LV.ini index addaa15394df2..c649a2670e0ad 100644 --- a/options/locale/locale_lv-LV.ini +++ b/options/locale/locale_lv-LV.ini @@ -1668,7 +1668,6 @@ pulls.delete.text=Vai patiešām vēlaties dzēst šo izmaiņu pieprasījumu? (N milestones.new=Jauns atskaites punkts milestones.closed=Aizvērts %s -milestones.update_ago=Atjaunots pirms %s milestones.no_due_date=Bez termiņa milestones.open=Atvērta milestones.close=Aizvērt diff --git a/options/locale/locale_nl-NL.ini b/options/locale/locale_nl-NL.ini index c3aed907b0472..068f67dc51ff6 100644 --- a/options/locale/locale_nl-NL.ini +++ b/options/locale/locale_nl-NL.ini @@ -1606,7 +1606,6 @@ pulls.delete.text=Weet je zeker dat je deze pull-verzoek wilt verwijderen? (Dit milestones.new=Nieuwe mijlpaal milestones.closed=%s werd gesloten -milestones.update_ago=%s dagen geleden bijgewerkt milestones.no_due_date=Geen vervaldatum milestones.open=Open milestones.close=Sluit diff --git a/options/locale/locale_pl-PL.ini b/options/locale/locale_pl-PL.ini index 0045c7c060db5..0bb732b159491 100644 --- a/options/locale/locale_pl-PL.ini +++ b/options/locale/locale_pl-PL.ini @@ -1445,7 +1445,6 @@ pulls.merge_instruction_step2_desc=Połącz zmiany i zaktualizuj na Gitea. milestones.new=Nowy kamień milowy milestones.closed=Zamknięto %s -milestones.update_ago=Zaktualizowano %s temu milestones.no_due_date=Nie ustalono terminu milestones.open=Otwórz milestones.close=Zamknij diff --git a/options/locale/locale_pt-BR.ini b/options/locale/locale_pt-BR.ini index c811390f1fb3b..aea0ad9a20e45 100644 --- a/options/locale/locale_pt-BR.ini +++ b/options/locale/locale_pt-BR.ini @@ -1668,7 +1668,6 @@ pulls.delete.text=Você realmente deseja excluir este pull request? (Isto irá r milestones.new=Novo marco milestones.closed=Fechado %s -milestones.update_ago=Atualizado há %s milestones.no_due_date=Sem data limite milestones.open=Reabrir milestones.close=Fechar diff --git a/options/locale/locale_pt-PT.ini b/options/locale/locale_pt-PT.ini index 4a5cc90e6f360..4339b3d19ac64 100644 --- a/options/locale/locale_pt-PT.ini +++ b/options/locale/locale_pt-PT.ini @@ -611,6 +611,9 @@ cancel=Cancelar language=Idioma ui=Tema hidden_comment_types=Tipos de comentários ocultos +hidden_comment_types_description=Os tipos de comentário marcados aqui não serão mostrados dentro das páginas das questões. Marcar "Rótulo", por exemplo, remove todos os comentários " adicionou/removeu ". +hidden_comment_types.ref_tooltip=Comentários onde esta questão foi referenciada a partir de outra questão/cometimento/… +hidden_comment_types.issue_ref_tooltip=Comentários onde o utilizador altera o ramo/etiqueta associado à questão comment_type_group_reference=Referência comment_type_group_label=Rótulo comment_type_group_milestone=Etapa @@ -1710,7 +1713,7 @@ pulls.delete.text=Tem a certeza que quer eliminar este pedido de integração? I milestones.new=Nova etapa milestones.closed=Encerrada %s -milestones.update_ago=Modificada há %s +milestones.update_ago=Modificou %s milestones.no_due_date=Sem data de vencimento milestones.open=Abrir milestones.close=Fechar diff --git a/options/locale/locale_ru-RU.ini b/options/locale/locale_ru-RU.ini index be6c1f6b33c67..b74ed1609453b 100644 --- a/options/locale/locale_ru-RU.ini +++ b/options/locale/locale_ru-RU.ini @@ -285,6 +285,7 @@ users=Пользователи organizations=Организации search=Поиск code=Код +search.type.tooltip=Тип поиска search.fuzzy=Неточный search.match=Соответствие search.match.tooltip=Включать только результаты, которые точно соответствуют поисковому запросу @@ -508,6 +509,7 @@ cannot_add_org_to_team=Организацию нельзя добавить в invalid_ssh_key=Не удается проверить SSH ключ: %s invalid_gpg_key=Не удается проверить GPG ключ: %s invalid_ssh_principal=Неверный участник: %s +unable_verify_ssh_key=Не удаётся верифицировать ключ SSH, проверьте его на наличие ошибок. auth_failed=Ошибка аутентификации: %v @@ -576,6 +578,9 @@ cancel=Отмена language=Язык ui=Тема hidden_comment_types=Скрытые типы комментариев +hidden_comment_types_description=Отмеченные типы комментариев не будут отображаться на страницах задач. Например, если выбрать «Метки», не станет всех комментариев «<пользователь> добавил/удалил <метку>». +hidden_comment_types.ref_tooltip=Комментарии об упоминании задачи в другой задаче/коммите/… +hidden_comment_types.issue_ref_tooltip=Комментарии об изменении ветки/тега, связанных с этой задачей comment_type_group_reference=Упоминания comment_type_group_label=Операции с метками comment_type_group_milestone=Этап @@ -588,6 +593,7 @@ comment_type_group_dependency=Модификации зависимостей comment_type_group_lock=Смена статуса ограничения на обсуждение comment_type_group_review_request=Запросы на рецензию comment_type_group_project=Проект +comment_type_group_issue_ref=Ссылка на задачу saved_successfully=Ваши настройки успешно сохранены. privacy=Приватность keep_activity_private=Скрыть активность со страницы профиля @@ -736,6 +742,7 @@ delete_token=Удалить access_token_deletion=Удалить токен доступа access_token_deletion_cancel_action=Отменить access_token_deletion_confirm_action=Удалить +access_token_deletion_desc=Удаление токена отзовёт доступ к вашей учетной записи у приложений, использующих его. Это действие не может быть отменено. Продолжить? delete_token_success=Токен удалён. Приложения, использующие его, больше не имеют доступа к вашему аккаунту. manage_oauth2_applications=Управление приложениями OAuth2 @@ -1005,6 +1012,7 @@ download_archive=Скачать репозиторий no_desc=Нет описания quick_guide=Краткое руководство clone_this_repo=Клонировать репозиторий +cite_this_repo=Сослаться на этот репозиторий create_new_repo_command=Создать новый репозиторий из командной строки empty_message=В репозитории нет файлов. @@ -1032,6 +1040,7 @@ release=Релиз releases=Релизы tag=Тег released_this=выпустил(-а) это +tagged_this=добавил(а) тег file.title=%s в %s file_raw=Исходник file_history=История @@ -1196,6 +1205,10 @@ projects.column.deletion_desc=При удалении столбца проек projects.column.color=Цвет projects.open=Открыть projects.close=Закрыть +projects.column.assigned_to=Назначено на +projects.card_type.desc=Предпросмотр карточек +projects.card_type.images_and_text=Изображения и текст +projects.card_type.text_only=Только текст issues.desc=Организация отчетов об ошибках, задач и этапов. issues.filter_assignees=Фильтр назначений @@ -1232,6 +1245,8 @@ issues.choose.get_started=Начать issues.choose.open_external_link=Открыть issues.choose.blank=По умолчанию issues.choose.blank_about=Создать запрос из шаблона по умолчанию. +issues.choose.ignore_invalid_templates=Некорректные шаблоны были проигнорированы +issues.choose.invalid_templates=Найден(ы) %v неверный(х) шаблон(ов) issues.no_ref=Не указана ветка или тэг issues.create=Добавить задачу issues.new_label=Новая метка @@ -1497,6 +1512,7 @@ issues.content_history.created=создано issues.content_history.delete_from_history=Удалить из истории issues.content_history.delete_from_history_confirm=Удалить из истории? issues.content_history.options=Настройки +issues.reference_link=Ссылка: %s compare.compare_base=Основа compare.compare_head=сравнить @@ -1505,6 +1521,9 @@ pulls.desc=Включить запросы на слияние и проверк pulls.new=Новый запрос на слияние pulls.view=Просмотр запроса на слияние pulls.compare_changes=Новый запрос на слияние +pulls.allow_edits_from_maintainers=Разрешить редактирование сопровождающими +pulls.allow_edits_from_maintainers_desc=Пользователи с доступом на запись в основную ветку могут отправлять изменения и в эту ветку +pulls.allow_edits_from_maintainers_err=Не удалось обновить pulls.compare_changes_desc=Сравнить две ветки и создать запрос на слияние для изменений. pulls.expand_files=Показать все файлы pulls.collapse_files=Свернуть все файлы @@ -1608,7 +1627,7 @@ pulls.merge_instruction_step2_desc=Объединить изменения и о milestones.new=Новый этап milestones.closed=Закрыт %s -milestones.update_ago=Обновлено %s назад +milestones.update_ago=Обновлено %s milestones.no_due_date=Срок не указан milestones.open=Открыть milestones.close=Закрыть @@ -1744,6 +1763,7 @@ activity.git_stats_deletion_n=%d удалений search=Поиск search.search_repo=Поиск по репозиторию +search.type.tooltip=Тип поиска search.fuzzy=Неточный search.match=Соответствие search.match.tooltip=Включать только результаты, которые точно соответствуют поисковому запросу @@ -1803,6 +1823,7 @@ settings.pulls_desc=Включить запросы на слияние settings.pulls.ignore_whitespace=Игнорировать незначащие изменения (пробелы, табуляция) при проверке на конфликты слияния settings.pulls.enable_autodetect_manual_merge=Включить автоопределение ручного слияния (Примечание: в некоторых особых случаях могут возникнуть ошибки) settings.pulls.default_delete_branch_after_merge=Удалить ветку запроса после его слияния по умолчанию +settings.pulls.default_allow_edits_from_maintainers=По умолчанию разрешать редактирование сопровождающими settings.projects_desc=Включить проекты репозитория settings.admin_settings=Настройки администратора settings.admin_enable_health_check=Выполнять проверки целостности этого репозитория (git fsck) @@ -2349,6 +2370,7 @@ teams.update_settings=Обновить настройки teams.delete_team=Удалить команду teams.add_team_member=Добавление члена группы разработки teams.invite_team_member=Пригласить в %s +teams.invite_team_member.list=Приглашения в ожидании teams.delete_team_title=Удалить команду teams.delete_team_desc=Удаление команды отменяет доступ к репозиторию для её членов. Продолжить? teams.delete_team_success=Команда удалена. @@ -2372,6 +2394,7 @@ teams.all_repositories_helper=Команда имеет доступ ко все teams.all_repositories_read_permission_desc=Эта команда предоставляет прочтено доступ к всем репозиториям: участники могут просматривать и клонировать репозитории. teams.all_repositories_write_permission_desc=Эта команда предоставляет Написать доступ к всем репозиториям: участники могут читать и выполнять push в репозитории. teams.all_repositories_admin_permission_desc=Эта команда предоставляет администратору доступ к всем репозиториям: участники могут читать, отправлять сообщения и добавлять соавторов в репозитории. +teams.invite.by=Приглашен(а) %s [admin] dashboard=Панель diff --git a/options/locale/locale_si-LK.ini b/options/locale/locale_si-LK.ini index 78db42763c758..99a43974ef1f7 100644 --- a/options/locale/locale_si-LK.ini +++ b/options/locale/locale_si-LK.ini @@ -1427,7 +1427,6 @@ pulls.merge_instruction_step2_desc=Gitea හි වෙනස්කම් සහ milestones.new=නව සන්ධිස්ථානයක් milestones.closed=%s වසා ඇත -milestones.update_ago=යාවත්කාලීන %s පෙර milestones.no_due_date=නියමිත දිනයක් නැත milestones.open=විවෘත milestones.close=වසන්න diff --git a/options/locale/locale_tr-TR.ini b/options/locale/locale_tr-TR.ini index 7893ac8339d1f..a5e7061d49211 100644 --- a/options/locale/locale_tr-TR.ini +++ b/options/locale/locale_tr-TR.ini @@ -1631,7 +1631,6 @@ pulls.delete.text=Bu değişiklik isteğini gerçekten silmek istiyor musunuz? ( milestones.new=Yeni Kilometre Taşı milestones.closed=Kapalı %s -milestones.update_ago=%s önce güncellendi milestones.no_due_date=Bitiş tarihi yok milestones.open=Aç milestones.close=Kapat diff --git a/options/locale/locale_uk-UA.ini b/options/locale/locale_uk-UA.ini index 1fc964172e36f..6b6aa67371d50 100644 --- a/options/locale/locale_uk-UA.ini +++ b/options/locale/locale_uk-UA.ini @@ -1486,7 +1486,6 @@ pulls.merge_instruction_step2_desc=Об'єднати зміни і оновит milestones.new=Новий етап milestones.closed=Закрито %s -milestones.update_ago=Оновлено %s назад milestones.no_due_date=Немає дати завершення milestones.open=Відкрити milestones.close=Закрити diff --git a/options/locale/locale_zh-CN.ini b/options/locale/locale_zh-CN.ini index 49c257660899d..7806a195baf7f 100644 --- a/options/locale/locale_zh-CN.ini +++ b/options/locale/locale_zh-CN.ini @@ -1688,7 +1688,6 @@ pulls.delete.text=你真的要删除这个拉取请求吗？ (这将永久删除 milestones.new=新的里程碑 milestones.closed=于 %s关闭 -milestones.update_ago=更新于 %s 前 milestones.no_due_date=暂无截止日期 milestones.open=开启中 milestones.close=关闭 diff --git a/options/locale/locale_zh-TW.ini b/options/locale/locale_zh-TW.ini index f28e0992a7b38..6dfbe25181a50 100644 --- a/options/locale/locale_zh-TW.ini +++ b/options/locale/locale_zh-TW.ini @@ -1696,7 +1696,6 @@ pulls.delete.text=您真的要刪除此合併請求嗎？(這將會永久移除 milestones.new=新增里程碑 milestones.closed=於 %s關閉 -milestones.update_ago=%s前更新 milestones.no_due_date=暫無截止日期 milestones.open=開啟 milestones.close=關閉 From 5768bafeb28e4e4212ae0e2abc7f22c9c8b7c653 Mon Sep 17 00:00:00 2001 From: yp05327 <576951401@qq.com> Date: Fri, 14 Apr 2023 12:34:10 +0900 Subject: [PATCH 09/28] Fix incorrect server error content in RunnersList (#24118) --- routers/web/shared/actions/runners.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/routers/web/shared/actions/runners.go b/routers/web/shared/actions/runners.go index f63d37f165b8f..2c3614cbbcb2a 100644 --- a/routers/web/shared/actions/runners.go +++ b/routers/web/shared/actions/runners.go @@ -22,13 +22,13 @@ import ( func RunnersList(ctx *context.Context, tplName base.TplName, opts actions_model.FindRunnerOptions) { count, err := actions_model.CountRunners(ctx, opts) if err != nil { - ctx.ServerError("AdminRunners", err) + ctx.ServerError("CountRunners", err) return } runners, err := actions_model.FindRunners(ctx, opts) if err != nil { - ctx.ServerError("AdminRunners", err) + ctx.ServerError("FindRunners", err) return } if err := runners.LoadAttributes(ctx); err != nil { From 1c8bc4081a4f4d0d921ac218cb724ce97924d410 Mon Sep 17 00:00:00 2001 From: wxiaoguang Date: Fri, 14 Apr 2023 13:19:11 +0800 Subject: [PATCH 10/28] Show friendly 500 error page to users and developers (#24110) Close #24104 This also introduces many tests to cover many complex error handling functions. ### Before The details are never shown in production.
![image](https://user-images.githubusercontent.com/2114189/231805004-13214579-4fbe-465a-821c-be75c2749097.png)
### After The details could be shown to site admin users. It is safe. ![image](https://user-images.githubusercontent.com/2114189/231803912-d5660994-416f-4b27-a4f1-a4cc962091d4.png) --- modules/context/context.go | 44 +---- modules/templates/htmlrenderer.go | 251 +++++++++++++------------ modules/templates/htmlrenderer_test.go | 106 +++++++++++ templates/base/head_script.tmpl | 1 + templates/devtest/tmplerr-sub.tmpl | 3 + templates/devtest/tmplerr.tmpl | 12 ++ templates/status/404.tmpl | 2 +- templates/status/500.tmpl | 41 +++- web_src/css/helpers.css | 5 +- web_src/js/bootstrap.js | 12 +- 10 files changed, 305 insertions(+), 172 deletions(-) create mode 100644 modules/templates/htmlrenderer_test.go create mode 100644 templates/devtest/tmplerr-sub.tmpl create mode 100644 templates/devtest/tmplerr.tmpl diff --git a/modules/context/context.go b/modules/context/context.go index cee533e42ae07..2507cc10c0097 100644 --- a/modules/context/context.go +++ b/modules/context/context.go @@ -16,10 +16,8 @@ import ( "net/http" "net/url" "path" - "regexp" "strconv" "strings" - texttemplate "text/template" "time" "code.gitea.io/gitea/models/db" @@ -216,7 +214,7 @@ func (ctx *Context) RedirectToFirst(location ...string) { ctx.Redirect(setting.AppSubURL + "/") } -var templateExecutingErr = regexp.MustCompile(`^template: (.*):([1-9][0-9]*):([1-9][0-9]*): executing (?:"(.*)" at <(.*)>: )?`) +const tplStatus500 base.TplName = "status/500" // HTML calls Context.HTML and renders the template to HTTP response func (ctx *Context) HTML(status int, name base.TplName) { @@ -229,34 +227,11 @@ func (ctx *Context) HTML(status int, name base.TplName) { return strconv.FormatInt(time.Since(tmplStartTime).Nanoseconds()/1e6, 10) + "ms" } if err := ctx.Render.HTML(ctx.Resp, status, string(name), templates.BaseVars().Merge(ctx.Data)); err != nil { - if status == http.StatusInternalServerError && name == base.TplName("status/500") { + if status == http.StatusInternalServerError && name == tplStatus500 { ctx.PlainText(http.StatusInternalServerError, "Unable to find HTML templates, the template system is not initialized, or Gitea can't find your template files.") return } - if execErr, ok := err.(texttemplate.ExecError); ok { - if groups := templateExecutingErr.FindStringSubmatch(err.Error()); len(groups) > 0 { - errorTemplateName, lineStr, posStr := groups[1], groups[2], groups[3] - target := "" - if len(groups) == 6 { - target = groups[5] - } - line, _ := strconv.Atoi(lineStr) // Cannot error out as groups[2] is [1-9][0-9]* - pos, _ := strconv.Atoi(posStr) // Cannot error out as groups[3] is [1-9][0-9]* - assetLayerName := templates.AssetFS().GetFileLayerName(errorTemplateName + ".tmpl") - filename := fmt.Sprintf("(%s) %s", assetLayerName, errorTemplateName) - if errorTemplateName != string(name) { - filename += " (subtemplate of " + string(name) + ")" - } - err = fmt.Errorf("failed to render %s, error: %w:\n%s", filename, err, templates.GetLineFromTemplate(errorTemplateName, line, target, pos)) - } else { - assetLayerName := templates.AssetFS().GetFileLayerName(execErr.Name + ".tmpl") - filename := fmt.Sprintf("(%s) %s", assetLayerName, execErr.Name) - if execErr.Name != string(name) { - filename += " (subtemplate of " + string(name) + ")" - } - err = fmt.Errorf("failed to render %s, error: %w", filename, err) - } - } + err = fmt.Errorf("failed to render template: %s, error: %s", name, templates.HandleTemplateRenderingError(err)) ctx.ServerError("Render failed", err) } } @@ -324,24 +299,25 @@ func (ctx *Context) serverErrorInternal(logMsg string, logErr error) { return } - if !setting.IsProd { + // it's safe to show internal error to admin users, and it helps + if !setting.IsProd || (ctx.Doer != nil && ctx.Doer.IsAdmin) { ctx.Data["ErrorMsg"] = logErr } } ctx.Data["Title"] = "Internal Server Error" - ctx.HTML(http.StatusInternalServerError, base.TplName("status/500")) + ctx.HTML(http.StatusInternalServerError, tplStatus500) } // NotFoundOrServerError use error check function to determine if the error // is about not found. It responds with 404 status code for not found error, // or error context description for logging purpose of 500 server error. -func (ctx *Context) NotFoundOrServerError(logMsg string, errCheck func(error) bool, err error) { - if errCheck(err) { - ctx.notFoundInternal(logMsg, err) +func (ctx *Context) NotFoundOrServerError(logMsg string, errCheck func(error) bool, logErr error) { + if errCheck(logErr) { + ctx.notFoundInternal(logMsg, logErr) return } - ctx.serverErrorInternal(logMsg, err) + ctx.serverErrorInternal(logMsg, logErr) } // PlainTextBytes renders bytes as plain text diff --git a/modules/templates/htmlrenderer.go b/modules/templates/htmlrenderer.go index 26dd365e4ca36..833c2acdca6ae 100644 --- a/modules/templates/htmlrenderer.go +++ b/modules/templates/htmlrenderer.go @@ -4,6 +4,7 @@ package templates import ( + "bufio" "bytes" "context" "errors" @@ -18,19 +19,13 @@ import ( "sync/atomic" texttemplate "text/template" + "code.gitea.io/gitea/modules/assetfs" "code.gitea.io/gitea/modules/log" "code.gitea.io/gitea/modules/setting" "code.gitea.io/gitea/modules/util" ) -var ( - rendererKey interface{} = "templatesHtmlRenderer" - - templateError = regexp.MustCompile(`^template: (.*):([0-9]+): (.*)`) - notDefinedError = regexp.MustCompile(`^template: (.*):([0-9]+): function "(.*)" not defined`) - unexpectedError = regexp.MustCompile(`^template: (.*):([0-9]+): unexpected "(.*)" in operand`) - expectedEndError = regexp.MustCompile(`^template: (.*):([0-9]+): expected end; found (.*)`) -) +var rendererKey interface{} = "templatesHtmlRenderer" type HTMLRender struct { templates atomic.Pointer[template.Template] @@ -107,11 +102,12 @@ func HTMLRenderer(ctx context.Context) (context.Context, *HTMLRender) { renderer := &HTMLRender{} if err := renderer.CompileTemplates(); err != nil { - wrapFatal(handleNotDefinedPanicError(err)) - wrapFatal(handleUnexpected(err)) - wrapFatal(handleExpectedEnd(err)) - wrapFatal(handleGenericTemplateError(err)) - log.Fatal("HTMLRenderer error: %v", err) + p := &templateErrorPrettier{assets: AssetFS()} + wrapFatal(p.handleFuncNotDefinedError(err)) + wrapFatal(p.handleUnexpectedOperandError(err)) + wrapFatal(p.handleExpectedEndError(err)) + wrapFatal(p.handleGenericTemplateError(err)) + log.Fatal("HTMLRenderer CompileTemplates error: %v", err) } if !setting.IsProd { go AssetFS().WatchLocalChanges(ctx, func() { @@ -123,148 +119,153 @@ func HTMLRenderer(ctx context.Context) (context.Context, *HTMLRender) { return context.WithValue(ctx, rendererKey, renderer), renderer } -func wrapFatal(format string, args []interface{}) { - if format == "" { +func wrapFatal(msg string) { + if msg == "" { return } - log.FatalWithSkip(1, format, args...) + log.FatalWithSkip(1, "Unable to compile templates, %s", msg) } -func handleGenericTemplateError(err error) (string, []interface{}) { - groups := templateError.FindStringSubmatch(err.Error()) - if len(groups) != 4 { - return "", nil - } - - templateName, lineNumberStr, message := groups[1], groups[2], groups[3] - filename := fmt.Sprintf("%s (provided by %s)", templateName, AssetFS().GetFileLayerName(templateName+".tmpl")) - lineNumber, _ := strconv.Atoi(lineNumberStr) - line := GetLineFromTemplate(templateName, lineNumber, "", -1) - - return "PANIC: Unable to compile templates!\n%s in template file %s at line %d:\n\n%s\nStacktrace:\n\n%s", []interface{}{message, filename, lineNumber, log.NewColoredValue(line, log.Reset), log.Stack(2)} +type templateErrorPrettier struct { + assets *assetfs.LayeredFS } -func handleNotDefinedPanicError(err error) (string, []interface{}) { - groups := notDefinedError.FindStringSubmatch(err.Error()) - if len(groups) != 4 { - return "", nil - } - - templateName, lineNumberStr, functionName := groups[1], groups[2], groups[3] - functionName, _ = strconv.Unquote(`"` + functionName + `"`) - filename := fmt.Sprintf("%s (provided by %s)", templateName, AssetFS().GetFileLayerName(templateName+".tmpl")) - lineNumber, _ := strconv.Atoi(lineNumberStr) - line := GetLineFromTemplate(templateName, lineNumber, functionName, -1) +var reGenericTemplateError = regexp.MustCompile(`^template: (.*):([0-9]+): (.*)`) - return "PANIC: Unable to compile templates!\nUndefined function %q in template file %s at line %d:\n\n%s", []interface{}{functionName, filename, lineNumber, log.NewColoredValue(line, log.Reset)} -} - -func handleUnexpected(err error) (string, []interface{}) { - groups := unexpectedError.FindStringSubmatch(err.Error()) +func (p *templateErrorPrettier) handleGenericTemplateError(err error) string { + groups := reGenericTemplateError.FindStringSubmatch(err.Error()) if len(groups) != 4 { - return "", nil + return "" } - - templateName, lineNumberStr, unexpected := groups[1], groups[2], groups[3] - unexpected, _ = strconv.Unquote(`"` + unexpected + `"`) - filename := fmt.Sprintf("%s (provided by %s)", templateName, AssetFS().GetFileLayerName(templateName+".tmpl")) - lineNumber, _ := strconv.Atoi(lineNumberStr) - line := GetLineFromTemplate(templateName, lineNumber, unexpected, -1) - - return "PANIC: Unable to compile templates!\nUnexpected %q in template file %s at line %d:\n\n%s", []interface{}{unexpected, filename, lineNumber, log.NewColoredValue(line, log.Reset)} + tmplName, lineStr, message := groups[1], groups[2], groups[3] + return p.makeDetailedError(message, tmplName, lineStr, -1, "") } -func handleExpectedEnd(err error) (string, []interface{}) { - groups := expectedEndError.FindStringSubmatch(err.Error()) - if len(groups) != 4 { - return "", nil - } - - templateName, lineNumberStr, unexpected := groups[1], groups[2], groups[3] - filename := fmt.Sprintf("%s (provided by %s)", templateName, AssetFS().GetFileLayerName(templateName+".tmpl")) - lineNumber, _ := strconv.Atoi(lineNumberStr) - line := GetLineFromTemplate(templateName, lineNumber, unexpected, -1) +var reFuncNotDefinedError = regexp.MustCompile(`^template: (.*):([0-9]+): (function "(.*)" not defined)`) - return "PANIC: Unable to compile templates!\nMissing end with unexpected %q in template file %s at line %d:\n\n%s", []interface{}{unexpected, filename, lineNumber, log.NewColoredValue(line, log.Reset)} +func (p *templateErrorPrettier) handleFuncNotDefinedError(err error) string { + groups := reFuncNotDefinedError.FindStringSubmatch(err.Error()) + if len(groups) != 5 { + return "" + } + tmplName, lineStr, message, funcName := groups[1], groups[2], groups[3], groups[4] + funcName, _ = strconv.Unquote(`"` + funcName + `"`) + return p.makeDetailedError(message, tmplName, lineStr, -1, funcName) } -const dashSeparator = "----------------------------------------------------------------------\n" +var reUnexpectedOperandError = regexp.MustCompile(`^template: (.*):([0-9]+): (unexpected "(.*)" in operand)`) -// GetLineFromTemplate returns a line from a template with some context -func GetLineFromTemplate(templateName string, targetLineNum int, target string, position int) string { - bs, err := AssetFS().ReadFile(templateName + ".tmpl") - if err != nil { - return fmt.Sprintf("(unable to read template file: %v)", err) +func (p *templateErrorPrettier) handleUnexpectedOperandError(err error) string { + groups := reUnexpectedOperandError.FindStringSubmatch(err.Error()) + if len(groups) != 5 { + return "" } + tmplName, lineStr, message, unexpected := groups[1], groups[2], groups[3], groups[4] + unexpected, _ = strconv.Unquote(`"` + unexpected + `"`) + return p.makeDetailedError(message, tmplName, lineStr, -1, unexpected) +} - sb := &strings.Builder{} - - // Write the header - sb.WriteString(dashSeparator) +var reExpectedEndError = regexp.MustCompile(`^template: (.*):([0-9]+): (expected end; found (.*))`) - var lineBs []byte +func (p *templateErrorPrettier) handleExpectedEndError(err error) string { + groups := reExpectedEndError.FindStringSubmatch(err.Error()) + if len(groups) != 5 { + return "" + } + tmplName, lineStr, message, unexpected := groups[1], groups[2], groups[3], groups[4] + return p.makeDetailedError(message, tmplName, lineStr, -1, unexpected) +} - // Iterate through the lines from the asset file to find the target line - for start, currentLineNum := 0, 1; currentLineNum <= targetLineNum && start < len(bs); currentLineNum++ { - // Find the next new line - end := bytes.IndexByte(bs[start:], '\n') +var ( + reTemplateExecutingError = regexp.MustCompile(`^template: (.*):([1-9][0-9]*):([1-9][0-9]*): (executing .*)`) + reTemplateExecutingErrorMsg = regexp.MustCompile(`^executing "(.*)" at <(.*)>: `) +) - // adjust the end to be a direct pointer in to []byte - if end < 0 { - end = len(bs) - } else { - end += start +func (p *templateErrorPrettier) handleTemplateRenderingError(err error) string { + if groups := reTemplateExecutingError.FindStringSubmatch(err.Error()); len(groups) > 0 { + tmplName, lineStr, posStr, msgPart := groups[1], groups[2], groups[3], groups[4] + target := "" + if groups = reTemplateExecutingErrorMsg.FindStringSubmatch(msgPart); len(groups) > 0 { + target = groups[2] } + return p.makeDetailedError(msgPart, tmplName, lineStr, posStr, target) + } else if execErr, ok := err.(texttemplate.ExecError); ok { + layerName := p.assets.GetFileLayerName(execErr.Name + ".tmpl") + return fmt.Sprintf("asset from: %s, %s", layerName, err.Error()) + } else { + return err.Error() + } +} - // set lineBs to the current line []byte - lineBs = bs[start:end] +func HandleTemplateRenderingError(err error) string { + p := &templateErrorPrettier{assets: AssetFS()} + return p.handleTemplateRenderingError(err) +} - // move start to after the current new line position - start = end + 1 +const dashSeparator = "----------------------------------------------------------------------" - // Write 2 preceding lines + the target line - if targetLineNum-currentLineNum < 3 { - _, _ = sb.Write(lineBs) - _ = sb.WriteByte('\n') - } +func (p *templateErrorPrettier) makeDetailedError(errMsg, tmplName string, lineNum, posNum any, target string) string { + code, layer, err := p.assets.ReadLayeredFile(tmplName + ".tmpl") + if err != nil { + return fmt.Sprintf("template error: %s, and unable to find template file %q", errMsg, tmplName) + } + line, err := util.ToInt64(lineNum) + if err != nil { + return fmt.Sprintf("template error: %s, unable to parse template %q line number %q", errMsg, tmplName, lineNum) + } + pos, err := util.ToInt64(posNum) + if err != nil { + return fmt.Sprintf("template error: %s, unable to parse template %q pos number %q", errMsg, tmplName, posNum) } + detail := extractErrorLine(code, int(line), int(pos), target) - // FIXME: this algorithm could provide incorrect results and mislead the developers. - // For example: Undefined function "file" in template ..... - // {{Func .file.Addition file.Deletion .file.Addition}} - // ^^^^ ^(the real error is here) - // The pointer is added to the first one, but the second one is the real incorrect one. - // - // If there is a provided target to look for in the line add a pointer to it - // e.g. ^^^^^^^ - if target != "" { - targetPos := bytes.Index(lineBs, []byte(target)) - if targetPos >= 0 { - position = targetPos - } + var msg string + if pos >= 0 { + msg = fmt.Sprintf("template error: %s:%s:%d:%d : %s", layer, tmplName, line, pos, errMsg) + } else { + msg = fmt.Sprintf("template error: %s:%s:%d : %s", layer, tmplName, line, errMsg) } - if position >= 0 { - // take the current line and replace preceding text with whitespace (except for tab) - for i := range lineBs[:position] { - if lineBs[i] != '\t' { - lineBs[i] = ' ' + return msg + "\n" + dashSeparator + "\n" + detail + "\n" + dashSeparator +} + +func extractErrorLine(code []byte, lineNum, posNum int, target string) string { + b := bufio.NewReader(bytes.NewReader(code)) + var line []byte + var err error + for i := 0; i < lineNum; i++ { + if line, err = b.ReadBytes('\n'); err != nil { + if i == lineNum-1 && errors.Is(err, io.EOF) { + err = nil } + break } + } + if err != nil { + return fmt.Sprintf("unable to find target line %d", lineNum) + } - // write the preceding "space" - _, _ = sb.Write(lineBs[:position]) - - // Now write the ^^ pointer - targetLen := len(target) - if targetLen == 0 { - targetLen = 1 + line = bytes.TrimRight(line, "\r\n") + var indicatorLine []byte + targetBytes := []byte(target) + targetLen := len(targetBytes) + for i := 0; i < len(line); { + if posNum == -1 && target != "" && bytes.HasPrefix(line[i:], targetBytes) { + for j := 0; j < targetLen && i < len(line); j++ { + indicatorLine = append(indicatorLine, '^') + i++ + } + } else if i == posNum { + indicatorLine = append(indicatorLine, '^') + i++ + } else { + if line[i] == '\t' { + indicatorLine = append(indicatorLine, '\t') + } else { + indicatorLine = append(indicatorLine, ' ') + } + i++ } - _, _ = sb.WriteString(strings.Repeat("^", targetLen)) - _ = sb.WriteByte('\n') } - - // Finally write the footer - sb.WriteString(dashSeparator) - - return sb.String() + // if the indicatorLine only contains spaces, trim it together + return strings.TrimRight(string(line)+"\n"+string(indicatorLine), " \t\r\n") } diff --git a/modules/templates/htmlrenderer_test.go b/modules/templates/htmlrenderer_test.go new file mode 100644 index 0000000000000..2a74b74c2333d --- /dev/null +++ b/modules/templates/htmlrenderer_test.go @@ -0,0 +1,106 @@ +// Copyright 2023 The Gitea Authors. All rights reserved. +// SPDX-License-Identifier: MIT + +package templates + +import ( + "errors" + "html/template" + "os" + "strings" + "testing" + + "code.gitea.io/gitea/modules/assetfs" + + "github.com/stretchr/testify/assert" +) + +func TestExtractErrorLine(t *testing.T) { + cases := []struct { + code string + line int + pos int + target string + expect string + }{ + {"hello world\nfoo bar foo bar\ntest", 2, -1, "bar", ` +foo bar foo bar + ^^^ ^^^ +`}, + + {"hello world\nfoo bar foo bar\ntest", 2, 4, "bar", ` +foo bar foo bar + ^ +`}, + + { + "hello world\nfoo bar foo bar\ntest", 2, 4, "", + ` +foo bar foo bar + ^ +`, + }, + + { + "hello world\nfoo bar foo bar\ntest", 5, 0, "", + `unable to find target line 5`, + }, + } + + for _, c := range cases { + actual := extractErrorLine([]byte(c.code), c.line, c.pos, c.target) + assert.Equal(t, strings.TrimSpace(c.expect), strings.TrimSpace(actual)) + } +} + +func TestHandleError(t *testing.T) { + dir := t.TempDir() + + p := &templateErrorPrettier{assets: assetfs.Layered(assetfs.Local("tmp", dir))} + + test := func(s string, h func(error) string, expect string) { + err := os.WriteFile(dir+"/test.tmpl", []byte(s), 0o644) + assert.NoError(t, err) + tmpl := template.New("test") + _, err = tmpl.Parse(s) + assert.Error(t, err) + msg := h(err) + assert.EqualValues(t, strings.TrimSpace(expect), strings.TrimSpace(msg)) + } + + test("{{", p.handleGenericTemplateError, ` +template error: tmp:test:1 : unclosed action +---------------------------------------------------------------------- +{{ +---------------------------------------------------------------------- +`) + + test("{{Func}}", p.handleFuncNotDefinedError, ` +template error: tmp:test:1 : function "Func" not defined +---------------------------------------------------------------------- +{{Func}} + ^^^^ +---------------------------------------------------------------------- +`) + + test("{{'x'3}}", p.handleUnexpectedOperandError, ` +template error: tmp:test:1 : unexpected "3" in operand +---------------------------------------------------------------------- +{{'x'3}} + ^ +---------------------------------------------------------------------- +`) + + // no idea about how to trigger such strange error, so mock an error to test it + err := os.WriteFile(dir+"/test.tmpl", []byte("god knows XXX"), 0o644) + assert.NoError(t, err) + expectedMsg := ` +template error: tmp:test:1 : expected end; found XXX +---------------------------------------------------------------------- +god knows XXX + ^^^ +---------------------------------------------------------------------- +` + actualMsg := p.handleExpectedEndError(errors.New("template: test:1: expected end; found XXX")) + assert.EqualValues(t, strings.TrimSpace(expectedMsg), strings.TrimSpace(actualMsg)) +} diff --git a/templates/base/head_script.tmpl b/templates/base/head_script.tmpl index 670d146b56a04..d19fae629bd97 100644 --- a/templates/base/head_script.tmpl +++ b/templates/base/head_script.tmpl @@ -6,6 +6,7 @@ If you introduce mistakes in it, Gitea JavaScript code wouldn't run correctly. {{template "base/footer" .}} diff --git a/web_src/css/helpers.css b/web_src/css/helpers.css index dcec79fcf6f08..8d64bd751b879 100644 --- a/web_src/css/helpers.css +++ b/web_src/css/helpers.css @@ -46,8 +46,8 @@ text-overflow: ellipsis !important; } -.gt-full-screen-width { width: 100vw !important; } -.gt-full-screen-height { height: 100vh !important; } +.gt-w-screen { width: 100vw !important; } +.gt-h-screen { height: 100vh !important; } .gt-rounded { border-radius: var(--border-radius) !important; } .gt-rounded-top { border-radius: var(--border-radius) var(--border-radius) 0 0 !important; } @@ -202,6 +202,7 @@ .gt-shrink-0 { flex-shrink: 0 !important; } .gt-whitespace-nowrap { white-space: nowrap !important; } +.gt-whitespace-pre-wrap { white-space: pre-wrap !important; } @media (max-width: 767px) { .gt-db-small { display: block !important; } diff --git a/web_src/js/bootstrap.js b/web_src/js/bootstrap.js index 54b7c628873e5..4f88c600f5554 100644 --- a/web_src/js/bootstrap.js +++ b/web_src/js/bootstrap.js @@ -20,6 +20,10 @@ export function showGlobalErrorMessage(msg) { * @param {ErrorEvent} e */ function processWindowErrorEvent(e) { + if (window.config.initCount > 1) { + // the page content has been loaded many times, the HTML/JS are totally broken, don't need to show error message + return; + } if (!e.error && e.lineno === 0 && e.colno === 0 && e.filename === '' && window.navigator.userAgent.includes('FxiOS/')) { // At the moment, Firefox (iOS) (10x) has an engine bug. See https://github.com/go-gitea/gitea/issues/20240 // If a script inserts a newly created (and content changed) element into DOM, there will be a nonsense error event reporting: Script error: line 0, col 0. @@ -33,7 +37,13 @@ function initGlobalErrorHandler() { if (!window.config) { showGlobalErrorMessage(`Gitea JavaScript code couldn't run correctly, please check your custom templates`); } - + if (window.config.initCount > 1) { + // when a sub-templates triggers an 500 error, its parent template has been partially rendered, + // then the 500 page will be rendered after that partially rendered page, which will cause the initCount > 1 + // in this case, the page is totally broken, so do not do any further error handling + console.error('initGlobalErrorHandler: Gitea global config system has already been initialized, there must be something else wrong'); + return; + } // we added an event handler for window error at the very beginning of