Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proxy support broke in Gitea 1.18 #22335

Closed
zfLQ2qx2 opened this issue Jan 4, 2023 · 10 comments
Closed

Proxy support broke in Gitea 1.18 #22335

zfLQ2qx2 opened this issue Jan 4, 2023 · 10 comments
Labels

Comments

@zfLQ2qx2
Copy link

zfLQ2qx2 commented Jan 4, 2023

Description

I run gitea as a tor service so that I do not have to expose ports on my home firewall. Applications on my tor server do not have direct access to the internet, instead the tor daemon acts as a socks5 proxy and is responsible for name resolution and forwarding outgoing traffic through the tor network.

To have gitea work with tor I added the following lines to my app.ini:

[proxy]
ENABLED = true
PROXY_URL = socks://127.0.0.1:9050/
PROXY_HOSTS = *

In versions upto 1.16.1 this worked correctly and I setup mirrors of a number repos from github, gitlab, and .onion domains without issue.

I jumped from 1.16.1 to 1.18.0 and in the latest version all of my mirror connections fail with DNS resolution errors which suggests that gitea is still trying to resolve the names itself instead of entrusting them to the socks5 proxy. This makes it impossible to reach onion addresses also since only the tor client can resolve those.

My test cases to show tor is working are:

ALL_PROXY=socks5h://127.0.0.1:9050/ curl -vvv http://github.com/
ALL_PROXY=socks5h://127.0.0.1:9050/ curl -vvv http://eweiibe6tdjsdprb4px6rqrzzcsi22m4koia44kc5pcjr7nec2rlxyad.onion/

(the later is the gitlab run by the tor project)

I should point out that in curl there is a difference, socks5 and socks5h, where the former uses the libc resolver and the later uses the socks5 resolver. The golang base libraries don't have a distinction.

Gitea Version

1.18

Can you reproduce the bug on the Gitea demo site?

Yes

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

Linux

How are you running Gitea?

gitea-1.18.0-linux-amd64 from github releases

Database

SQLite

@zeripath
Copy link
Contributor

Have you tried adding *.onion to [migrations] ALLOWED_DOMAINS ?

@zfLQ2qx2
Copy link
Author

@zeripath I will try that, but all of the connections are failing. The majority are GitHub (dozens) a couple Gitlab, and a couple .onion.

@zeripath
Copy link
Contributor

Then have you tried setting ALLOWED_DOMAINS=* presumably your proxy is not going to allow connections to local machines?

@zfLQ2qx2
Copy link
Author

@zeripath Apologies, it took me a while to try these. I added a migrations section and tried adding ALLOWED_DOMAINS=*, although the documentation says that by default all domains are allowed so I don't think this is needed, and got the same result. I also tried adding ALLOW_LOCALNETWORKS=true under migrations because the way Tor works it maps resolved onion names to a local address and then requests to that address get forwarded to the onion address. I had no luck with this either. So I think we are back to something having changed in Gitea 1.18 (or 1.17, the last version this worked for sure was 1.16) where Gitea is attempting to resolve the addresses itself rather than allowing the Socks5 proxy to do it.

So to test I need to configure tor to answer DNS queries on 127.0.0.1:53 and we'll see if that is a workaround for this new behavior.

@zfLQ2qx2
Copy link
Author

@zeripath So I did need [migrations] with ALLOWED_DOMAINS=* in order to update the host in the mirroring config. It looks like the Tor project changed the address of their git server which is the test case I used at start of this ticket.

I verified that the gitea user and resolve the onion address and reach it via socks 4, 5, and 5h - onion addresses resolve from command line via 127.0.0.1:53. Test case which works is:

ALL_PROXY=socks://127.0.0.1:9050/ curl -v -v -v http://gzgme7ov25seqjbphab4fkcph3jkobfwwpivt5kzbv3kqx2y2qttl4yd.onion:80/tor.git

Error from gitea is now:

Failed to update mirror repository '/home/gitea/gitea-repositories/tor/tor.git': fatal: unable to access 'http://gzgme7ov25seqjbphab4fkcph3jkobfwwpivt5kzbv3kqx2y2qttl4yd.onion/tor.git/': Failed to connect to gzgme7ov25seqjbphab4fkcph3jkobfwwpivt5kzbv3kqx2y2qttl4yd.onion port 80 after 0 ms: Connection refused
error: Could not fetch origin

Which is not a resolution error so thats ok now but seems like Gitea is not using the Socks proxy for outgoing connections - otherwise would take more then 0ms to fail.

It looks like [proxy] ENABLED is now [proxy] PROXY_ENABLED, made that change but did not help.

I tried changing [proxy] PROXY_HOSTS to , **, ",**", and *.onion (four different settings) but did not help.

I'm not sure what other settings there are or how I can diagnose further. If gitea uses the socks proxy on 127.0.0.1:9050 like curl does, it should have no issue connecting out.

@zeripath
Copy link
Contributor

So the error is now coming from git not being able to resolve the host which implies that it isn't using the tor proxy.

This is strange because the proxy settings should set it.

@zfLQ2qx2
Copy link
Author

@zeripath In the gitea user's .gitconfig I also have an [http] proxy = socks5h://127.0.0.1:9050 setting, if I do '''git clone http://gzgme7ov25seqjbphab4fkcph3jkobfwwpivt5kzbv3kqx2y2qttl4yd.onion/tor.git''' I get a "repository not found" error from the remote which is significant in that it shows the git binary itself can talk to the remove server. The error itself is completely different issue, I need to open an issue with the tor project, neither their new or old onion addresses seem to allow git requests. The "gzgme..." address is the one they list on the bottom of their page https://gitweb.torproject.org/tor.git
...

@zfLQ2qx2
Copy link
Author

Now looking at https://github.com/xjasonlyu/tun2socks as a workaround which does not depend on gitea or git having explicit knowledge of the proxy.

lunny pushed a commit that referenced this issue Jul 24, 2023
…ams (#25855)

This is a simple PR which moves the `GetListener` function to a
`DefaultGetListener` function, and changes `GetListener` to be a
variable which by default points to the `DefaultGetListener` function.
This allows people who may exist quasi-downstream of Gitea to create
alternate "GetListener" functions, with identical signatures, which
return different implementations of the `net.Listener` interface. This
approach is expressly intended to be non-invasive and have the least
possible impact on the gitea codebase. A previous version of this idea
was rejected before: #15544 but
because of issues like: #22335 I
**really** think that recommending people configure proxies by hand is
exactly the wrong way to do things(This is why there is a Tor Browser.).
This tiny change lets me put proper hidden service configuration into
single `i2p.go` file which lives in `modules/graceful/` and which never
has to be checked in to your codebase or affect your dependencies or
bloat your project in any way, it can live on a branch in my fork and
I'll fast-forward every release and never the twain shall meet.

The main use-case for this is to listen on Peer-to-Peer networks and
Hidden Services directly without error-prone and cumbersome
port-forwarding configuration. For instance, I might implement an
"I2PGetListener" as follows:

```Go
// adapted from i2p.go which is unchecked-in in my modules/graceful/ directory
import "github.com/eyedeekay/onramp"

var garlic = &onramp.Garlic{}

func I2PGetListener(network, address string) (net.Listener, error) {
	// Add a deferral to say that we've tried to grab a listener
	defer GetManager().InformCleanup()
	switch network {
	case "tcp", "tcp4", "tcp6", "i2p", "i2pt":
		return garlic.Listen()
	case "unix", "unixpacket":
// I2P isn't really a replacement for the stuff you use Unix sockets for and it's also not an anonymity risk, so treat them normally
		unixAddr, err := net.ResolveUnixAddr(network, address)
		if err != nil {
			return nil, err
		}
		return GetListenerUnix(network, unixAddr)
	default:
		return nil, net.UnknownNetworkError(network)
	}
}
```

I could then substitute that GetListener function and be 50% of the way
to having a fully-functioning gitea-over-hidden-services instance
without any additional configuration(The other 50% doesn't require any
code-changes on gitea's part).

There are 2 advantages here, one being convenience, first this turns
hidden services into a zero-configuration option for self-hosting gitea,
and second safety, these Go libraries are passing around
hidden-service-only versions of the net.Addr struct, they're using
hidden-service-only versions of the sockets, which are both expressly
designed to never require access to any information outside the hidden
service network, manipulating the application so it reveals information
about the host becomes much more difficult, and some attacks become
nearly impossible. It also opens up TLS-over-Hidden Services support
which is niche right now, of course, but in a future where gitea
instances federate if hidden services want to be part of the federation
they're probably going to need TLS certificates. They don't need to be
painful to set up.

This doesn't fix an open issue, but it might affect:
- #22335 - my `i2p.go` file
actually has a mod that fixes this but it requires adding a handful of
new dependencies to gitea and isn't compatible with the normal way you
guys recommend using a proxy so I don't think it's ready to send to you
as a PR, but if I can find a non-invasive way to fix it I will.
 - #18240

I hereby agree to the Code of Conduct published here:
https://github.com/go-gitea/gitea/blob/8b89563bf1031089a218e6d05dc61047281b35ee/CODE_OF_CONDUCT.md
I have read and understood the recommendations published here:
https://github.com/go-gitea/gitea/blob/8b89563bf1031089a218e6d05dc61047281b35ee/CONTRIBUTING.md

Thank you for your consideration.

---------

Co-authored-by: eyedeekay <idk@mulder>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
@lunny
Copy link
Member

lunny commented Jul 27, 2023

Is this still a problem?

@yp05327
Copy link
Contributor

yp05327 commented Sep 26, 2024

Any updates?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants