Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for docker registry token auth server #16048

Closed
wants to merge 158 commits into from
Closed

add support for docker registry token auth server #16048

wants to merge 158 commits into from

Conversation

dragonflylee
Copy link

Fix #2316

similar to #14919 but only token server

  • do not store registry repo to database
  • fix auth scope type registry:catalog:*
  • fix Unauthorized response message

ref:

@codecov-commenter
Copy link

codecov-commenter commented Jun 2, 2021

Codecov Report

Merging #16048 (75d78bf) into main (36dce0e) will decrease coverage by 0.03%.
The diff coverage is 34.50%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main   #16048      +/-   ##
==========================================
- Coverage   44.10%   44.06%   -0.04%     
==========================================
  Files         682      686       +4     
  Lines       82375    82660     +285     
==========================================
+ Hits        36328    36423      +95     
- Misses      40140    40317     +177     
- Partials     5907     5920      +13     
Impacted Files Coverage Δ
models/repo.go 53.67% <0.00%> (-0.58%) ⬇️
modules/auth/sso/reverseproxy.go 8.51% <0.00%> (ø)
modules/graceful/server_http.go 0.00% <0.00%> (ø)
modules/packages/docker/auth.go 0.00% <0.00%> (ø)
routers/events/events.go 0.00% <0.00%> (ø)
routers/install.go 0.00% <0.00%> (ø)
routers/org/setting.go 0.00% <0.00%> (ø)
routers/private/manager.go 0.00% <0.00%> (ø)
routers/private/restore_repo.go 0.00% <0.00%> (ø)
routers/repo/blame.go 0.00% <0.00%> (ø)
... and 40 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a7bf97...75d78bf. Read the comment docs.

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jun 2, 2021
@KN4CK3R
Copy link
Member

KN4CK3R commented Jun 2, 2021

Is it possible to use the existing JWT functionality? This would prevent another dependency:
https://github.com/go-gitea/gitea/blob/main/models/oauth2_application.go

@@ -1983,6 +1983,10 @@ topic.done=保存
topic.count_prompt=您最多选择25个主题
topic.format_prompt=主题必须以字母或数字开头,可以包含连字符 (-),并且长度不得超过35个字符

container = 容器仓库
Copy link
Member

@lunny lunny Jun 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't change translation files except en-US. We use crowdin for translations.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove in new commit

@lunny lunny added the type/feature Completely new functionality. Can only be merged if feature freeze is not active. label Jun 2, 2021
@dragonflylee
Copy link
Author

dragonflylee commented Jun 2, 2021

Is it possible to use the existing JWT functionality? This would prevent another dependency:
https://github.com/go-gitea/gitea/blob/main/models/oauth2_application.go

Yes it would be

remove dependency in new commit

@lunny
Copy link
Member

lunny commented Jun 3, 2021

@dragonflylee It's better there is a repository level option like issue when global container is enabled.

@KN4CK3R
Copy link
Member

KN4CK3R commented Jun 3, 2021

You may not need a different private key, see #16010

Copy link
Member

@lafriks lafriks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should reuse existing JWT functionality by adding needed scopes etc

@dragonflylee dragonflylee deleted the container branch June 3, 2021 15:29
@dragonflylee dragonflylee reopened this Jun 9, 2021
@dragonflylee
Copy link
Author

You may not need a different private key, see #16010

registry only support rsa/ecdsa sign
not all jwt auth method

zeripath and others added 16 commits July 15, 2021 22:28
Backport #16442

The move to render custom/public as within /assets in #15219 missed updating
several documentation pages.

This PR updates this documentation.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #16447

Unfortunately #16268 contained a terrible error, whereby there was a double
indirection taken when unmarshalling the source data. This fatally breaks
authentication configuration reading.

Fix #16342

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
Frontport the changelog from v1.14.5

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Backport #16461

Unfortunately #16435 asserts the wrong error and should use
os.LinkError not os.PathError.

Fix #16439

Signed-off-by: Andrew Thornton <art27@cantab.net>
When marking notifications read the results may be returned out of order
or be delayed.  This PR sends a sequence number to gitea so that the
browser can ensure that only the results of the latest notification
change are shown.

Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix: support delete non-urlencoded wiki page

* fix: check error
…and to end (#16479) (#16480)

* Make cancel from CatFileBatch and CatFileBatchCheck wait for the command to end (#16479)

Fix #16427 (again!)

* handle sharing violation error code

Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add TestPrepareWikiFileName

* use LsTree as LsFiles is index only

* ajust other tests

Co-authored-by: Andrew Thornton <art27@cantab.net>
Backport #16490

A race has been detected in #1441 relating to getting log levels.

This PR protects the GetLevel and GetStacktraceLevel calls with a RW mutex.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #16496

Unfortunately the chi changes have resulted in the CORS headers for the
git smart http protocol going missing.

This is mostly because the OPTIONS method is not being handled by
httpBase anymore.

This PR adds a GetOptions, PostOptions and Options methods to web
handler to allow OPTIONS method requests to still reach the httpBase
function.

Fix #16350
Close #16491

Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix data race in bleve indexer

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Changelog for 1.15.0-rc2

Results of `~/go/bin/changelog -m 1.15.0 --after 16422 generate`

We need to release RC2 as there are mulitple problems with alpine 3.14 related to
the seccomp issues on Docker <20.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update CHANGELOG.md

Co-authored-by: zeripath <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Backport #16508

Somewhere along the line the creation of git-daemon-export-ok
files disappeared but the updating of these files when
repo visibility changes remained. The problem is that the
current state will create files even when the org or user
is private.

This PR restores creation correctly.

Fix #15521

Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #16465

One of the reasons why #16447 was needed and why #16268 was needed in
the first place was because it appears that editing ldap configuration
doesn't get tested.

This PR therefore adds a basic test that will run the edit pipeline.

In doing so it's now clear that #16447 and #16268 aren't actually
solving #16252. It turns out that what actually happens is that is that
the bytes are actually double encoded.

This PR now changes the json unmarshal wrapper to handle this double
encode.

Fix #16252

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Backport #16532

Co-authored-by: Matti R <matti@mdranta.net>
zeripath and others added 27 commits October 21, 2021 20:38
Backport #17375

Define unique names for image tabs in pull requests, in order to toggle tabs correctly when multiple are displayed on one page.

Fixes position of swipe-bar so it does not overlay other UI components when scrolling.

Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>

Co-authored-by: Mario Lubenka <mario.lubenka@googlemail.com>
* SECURITY
  * Upgrade Bluemonday to v1.0.16 (#17372) (#17374)
  * Ensure correct SSH permissions check for private and restricted users (#17370) (#17373)
* BUGFIXES
  * Prevent NPE in CSV diff rendering when column removed (#17018) (#17377)
  * Offer rsa-sha2-512 and rsa-sha2-256 algorithms in internal SSH (#17281) (#17376)
  * Don't panic if we fail to parse U2FRegistration data (#17304) (#17371)
  * Ensure popup text is aligned left (backport for 1.15) (#17343)
  * Ensure that git daemon export ok is created for mirrors (#17243) (#17306)
  * Disable core.protectNTFS (#17300) (#17302)
  * Use pointer for wrappedConn methods (#17295) (#17296)
  * AutoRegistration is supposed to be working with disabled registration (backport) (#17292)
  * Handle duplicate keys on GPG key ring (#17242) (#17284)
  * Fix SVG side by side comparison link (#17375) (#17391)

Signed-off-by: Andrew Thornton <art27@cantab.net>
* Bug fix: render Markdown `http://AppURL/org/repo/issues/4?a=1&b=2#comment-123 test` to HTML correctly, close #17394
* Bug fix: fix the positions of checkboxes in rendered HTML, close #17395

# Conflicts:
#	modules/markup/html.go
We allow to render empty check list item - [ ], while GitHub doesn't allow.

To make the rendering correct, we need tune the UI (the last PR #17413 uses absolute layout, which makes the empty checkbox item can not be displayed correctly)
Backport of #17409

* Read expected buffer size.

* Changed name.
Backport #17406.

Closes #17378 

Both errors from #17378 were caused by  #15175.

Problem 1 (error with added file):
`ToUTF8WithFallbackReader` creates a `MultiReader` from a `byte[2048]` and the remaining reader. `CreateReaderAndGuessDelimiter` tries to read 10000 bytes from this reader but only gets 2048 because that's the first reader in the `MultiReader`. Then the `if size < 1e4` thinks the input is at EOF and just returns that.

Problem 2 (error with changed file):
The blob reader gets defer closed. That was fine because the old version reads the whole file into memory. Now with the streaming version the close needs to defer after the method.

Co-authored-by: zeripath <art27@cantab.net>
Backport #17434

Unfortunately there was a regression in #17373 which missed that the user is not
for deploy keys. This leads to a panic when pushing with deploy keys.

Fix #17412

Signed-off-by: Andrew Thornton <art27@cantab.net>
…ers (#17460) (#17464)

Backport #17460

There is a small bug in the way that repo access is checked in
repoAssignment: Accessibility is checked by checking if the user has a
marked access to the repository instead of checking if the user has any
team granted access.

This PR changes this permissions check to use HasAccess() which does the
correct test. There is also a fix in the release api ListReleases where
it should return draft releases if the user is a member of a team with
write access to the releases.

The PR also adds a testcase.

Signed-off-by: Andrew Thornton <art27@cantab.net>
…nd (#17456) (#17463)

Backport #17456

modules/private/serv.go has two major functions that are missing testcases to ensure
that Deploy and normal SSH keys work correctly.

This PR adds some basic integration tests for these.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
* Changelog 1.15.6

Unforunately #17435 is a somewhat critical bug and therefore we should
really release 1.15.6 as soon as possible.

 ## [1.15.6](https://github.com/go-gitea/gitea/releases/tag/v1.15.6) - 2021-10-27

* BUGFIXES
  * Prevent panic in serv.go with Deploy Keys (#17434) (#17435)
  * Fix CSV render error (#17406) (#17431)
  * Read expected buffer size (#17409) (#17430)

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add 17456 and its backport

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add 17464

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add final pr

* Update date

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Backport #17475

The underlying problem in #17328 appears to be that users are re-running the install
page during upgrades. The function that tests and creates the db did not intend for
this and thus instead the migration scripts being run - a simple sync tables occurs.

This then causes a weird partially migrated DB which causes, in this release cycle,
the duplicate column in task table error. It is likely the cause of some weird
partial migration errors in other cycles too.

This PR simply ensures that the migration scripts are also run at this point too.

Fix #17328

Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #17487

CountOrphanedObjects needs to quote the table it is joining with as this table may
be `user`.

Fix #17485

Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #17498

The call to html.EscapeString in routers/web/repo/blame.go:renderBlame is extraneous
as the commit message is now rendered by the template. The template will correctly
escape strings - therefore we are currently double escaping.

This PR fixes this.

Fix #17492

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Backport #17482

* Only allow webhook to send requests to allowed hosts (backport #17482)

* use ALLOWED_HOST_LIST=* for default to keep the legacy behavior in 1.15.x
Backport #17572

* Remove appSubUrl from pasted images

Since we fixed the url base for the links in repositories we no longer need to add
the appsuburl to pasted image links.

Fix #17057

Signed-off-by: Andrew Thornton <art27@cantab.net>
- Backport of #17566

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
* support auth scope type registry:catalog:*
* fix Unauthorized response message
@wxiaoguang
Copy link
Contributor

This PR seems broken, many unrelated changes.

Either close this one and start a new clean one, or reset a clean branch from main and force push.

@go-gitea go-gitea locked and limited conversation to collaborators Apr 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. type/feature Completely new functionality. Can only be merged if feature freeze is not active.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate or Interface Docker Registry