Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Allow-/Block-List for Migrate & Mirrors #13610

Merged

Conversation

6543
Copy link
Member

@6543 6543 commented Nov 17, 2020

This PR will add two options blocklist and allowlist for migrating repositories from external URLs.

When you set allowlist then blocklist will be ignored and all users on this gitea instance could only migrate repositories from the domains on allowlist.

When allowlist is empty and you have domains on blocklist, all users on this gitea instance could migrate repositories any domain except the domains on blocklist.

original pull: #8040
close #4462

TODOs:

  • add default blocklist for private networks & localhost
  • test manually to ensure it works

@6543 6543 added this to the 1.14.0 milestone Nov 17, 2020
@6543 6543 added type/feature Completely new functionality. Can only be merged if feature freeze is not active. pr/wip This PR is not ready for review labels Nov 17, 2020
custom/conf/app.example.ini Outdated Show resolved Hide resolved
custom/conf/app.example.ini Outdated Show resolved Hide resolved
custom/conf/app.example.ini Outdated Show resolved Hide resolved
docs/content/doc/advanced/config-cheat-sheet.en-us.md Outdated Show resolved Hide resolved
docs/content/doc/advanced/config-cheat-sheet.en-us.md Outdated Show resolved Hide resolved
modules/matchlist/matchlist.go Outdated Show resolved Hide resolved
modules/matchlist/matchlist.go Outdated Show resolved Hide resolved
@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Nov 17, 2020
modules/setting/migrate.go Outdated Show resolved Hide resolved
@codecov-io
Copy link

codecov-io commented Nov 17, 2020

Codecov Report

Merging #13610 (870ca5b) into master (1e5247d) will increase coverage by 0.02%.
The diff coverage is 63.01%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #13610      +/-   ##
==========================================
+ Coverage   42.21%   42.24%   +0.02%     
==========================================
  Files         697      698       +1     
  Lines       76845    76917      +72     
==========================================
+ Hits        32439    32490      +51     
- Misses      39054    39063       +9     
- Partials     5352     5364      +12     
Impacted Files Coverage Δ
routers/init.go 51.21% <0.00%> (-0.85%) ⬇️
modules/matchlist/matchlist.go 37.50% <37.50%> (ø)
modules/setting/migrations.go 63.63% <42.85%> (-36.37%) ⬇️
models/error.go 38.66% <66.66%> (+1.90%) ⬆️
modules/migrations/migrate.go 31.52% <78.37%> (+9.97%) ⬆️
routers/api/v1/repo/migrate.go 46.71% <100.00%> (+3.75%) ⬆️
services/pull/check.go 48.90% <0.00%> (-3.65%) ⬇️
services/pull/temp_repo.go 26.59% <0.00%> (-3.20%) ⬇️
modules/process/manager.go 72.50% <0.00%> (-2.50%) ⬇️
services/pull/patch.go 53.97% <0.00%> (-1.71%) ⬇️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1e5247d...870ca5b. Read the comment docs.

@6543 6543 force-pushed the allow-block_list_migrate-mirror_8040 branch from da14604 to af32a09 Compare November 21, 2020 17:54
custom/conf/app.example.ini Outdated Show resolved Hide resolved
@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Nov 28, 2020
@silverwind
Copy link
Member

Still need to update app.example.ini with new names.

custom/conf/app.example.ini Outdated Show resolved Hide resolved
custom/conf/app.example.ini Outdated Show resolved Hide resolved
custom/conf/app.example.ini Outdated Show resolved Hide resolved
custom/conf/app.example.ini Outdated Show resolved Hide resolved
@zeripath
Copy link
Contributor

Still need to update app.example.ini with new names.

Is that now fixed?

@silverwind
Copy link
Member

Yes those look fine now.

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Nov 28, 2020
@6543
Copy link
Member Author

6543 commented Nov 29, 2020

🚀

@techknowlogick techknowlogick merged commit b2435af into go-gitea:master Nov 29, 2020
@6543 6543 deleted the allow-block_list_migrate-mirror_8040 branch November 29, 2020 00:45
@6543 6543 added topic/security Something leaks user information or is otherwise vulnerable. Should be fixed! backport/v1.13 skip-changelog This PR is irrelevant for the (next) changelog, for example bug fixes for unreleased features. labels Dec 1, 2020
6543 added a commit to 6543-forks/gitea that referenced this pull request Dec 1, 2020
* add black list and white list support for migrating repositories

* specify log message

* use blocklist/allowlist

* allways use lowercase to match url

* Apply allow/block

* Settings: use existing "migrations" section

* convert domains lower case

* dont store unused value

* Block private addresses for migration by default

* use proposed-upstream func to detect private IP addr

* add own error for blocked migration, add tests, imprufe api

* fix test

* fix-if-localhost-is-ipv4

* rename error & error message

* rename setting options

* Apply suggestions from code review

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
@6543 6543 added the backport/done All backports for this PR have been created label Dec 1, 2020
techknowlogick added a commit that referenced this pull request Dec 2, 2020
* add black list and white list support for migrating repositories

* specify log message

* use blocklist/allowlist

* allways use lowercase to match url

* Apply allow/block

* Settings: use existing "migrations" section

* convert domains lower case

* dont store unused value

* Block private addresses for migration by default

* use proposed-upstream func to detect private IP addr

* add own error for blocked migration, add tests, imprufe api

* fix test

* fix-if-localhost-is-ipv4

* rename error & error message

* rename setting options

* Apply suggestions from code review

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
@go-gitea go-gitea locked and limited conversation to collaborators Jan 18, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backport/done All backports for this PR have been created lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. skip-changelog This PR is irrelevant for the (next) changelog, for example bug fixes for unreleased features. topic/repo-migration Migrate repos from other platforms to Gitea, or from Gitea to them topic/security Something leaks user information or is otherwise vulnerable. Should be fixed! type/feature Completely new functionality. Can only be merged if feature freeze is not active.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Repository migration Domains/IPs whitelist and blacklist
7 participants