Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UDP noise #3711

Merged
merged 8 commits into from
Aug 28, 2024
Merged

UDP noise #3711

merged 8 commits into from
Aug 28, 2024

Conversation

dragonbreath2000
Copy link
Contributor

sometimes internet providers block curtain udp based protocols(quic,wireguard) but udp itself is not blocked or throttled
using this strategy we can bypass almost all udp based protocol restrictions
This is how it works:
we want to use wireguard but our provider blockes it,The way they mostly do it is they almost always only inspect the first packet in each socket and if they see it is wireguard they will just drop everything on that socket
As I sayed they will inspect the socket only once and that's it, and with some noise we can fool them
What is noise:it is basicly a random nonesense packet that we send before wireguard or quic handshake,the server will ignore that packet like it was never sent but the gfw will inspect it and once it did not find anything suspicious they will let the connection go smoothly,
I will atach two screen shots of wireshark showing this behavior(this ISP blockes quic)
without-noise
with-noise

this PR is draft and not ready at all,I just wanted to discuss this with maintainer to see if you are interested in this feature
I will attach a sample config for people to test this(it is basicly a splithttp h3 with dialerProxy to freedom)
vless-splithttp-quic-dialer-proxy -clean.json

right now there is no option for enabling or disabling this feature as I did not want to spend a lot of time changing protobuf and json parser before discussing it with maintainers
this strategy I believe has a lot of pottential:
bypassing wireguard and quic protocol blocking
bypassing sni based filtering or whitelisting on quic(this does not happen right now but if or when they start doing it,this can easily bypass them)
right now the type of noise is a random byte generated by crypto/rand and it sends the noise in the first write to socket and if we want we could have other types of noise too(a video call packet,quic handshake packet with whitelisted sni,Dtls or whatever)

@mmmray
Copy link
Collaborator

mmmray commented Aug 20, 2024

This idea was discussed privately with OP for a while. To clarify, the added noise packets work for some protocols and disrupt others, but for QUIC/H3 it allegedly works with CDN (although I have not tested it myself)

I think it would be too much maintenance effort to add specialized support for "fake clienthello", "fake wireguard", "fake voip", etc. Just let the user specify a bytestring directly, and that bytestring will be written as-is. They can write their own clienthello, or copy from some template.

Anyway, I think the idea is good.

@dragonbreath2000
Copy link
Contributor Author

This idea was discussed privately with OP for a while. To clarify, the added noise packets work for some protocols and disrupt others, but for QUIC/H3 it allegedly works with CDN (although I have not tested it myself)

I think it would be too much maintenance effort to add specialized support for "fake clienthello", "fake wireguard", "fake voip", etc. Just let the user specify a bytestring directly, and that bytestring will be written as-is. They can write their own clienthello, or copy from some template.

Anyway, I think the idea is good.

I don't think this would disrupt anything(unless the transport has issues with dialer proxy,if u are talking about the line where I check for port 53 it wasn't because it disrputs dns,without that if condition the udp dns was fine I just added just to be safe),I tried with vless-splithttp quic and wireguard,both worked fine,About the packet type I think it is best to have two modes:1-the one u said with a user input string 2-a random generated byte that user would choose the min and max length for that byte

@Fangliding
Copy link
Member

Fangliding commented Aug 21, 2024

Thanks, but we currently prefer not to merge such 'tricks'

see #3677 (comment)

Looks like rprx would like to merge, see what can we do

@dragonbreath2000
Copy link
Contributor Author

Thanks, but we currently prefer not to merge such 'tricks'

see #3677 (comment)

I undrestand why u say this,u might be right the second u merge it people will use it and providers will just throttle udp for cloudflare,but it is not just about using it with cloudflare ,imagine I want to use a free wireguard service (I am not talking about warp,this one is already throttled heavily but there are others too,don't want to name them here) there is no way I can use them normaly but with this trick it works,basicly more people will have access to free proxy services,this is not a full solution but for a lot of times it could be usefull,and lastly they might throttle cloudflare but what if i want to use this with another cdn?amazon(this one almost never gets QOSed atleast where I live and I think it supports quic too) and I am not saying everyone should go and use amazon I am saying "if" anyone wants to use it this can help , as I mentioned it is not a full solution ,Best of luck to all the developers of this project
other thing is most firewalls have limited resources and usually they do very little inspection when it comes to udp and this trick really takes advantage from that

@bb33bb
Copy link

bb33bb commented Aug 21, 2024

Any how it is useful may be we will merge it privately.

@yuhan6665
Copy link
Member

My take is we can do if change is clean and organized in one place.
This method remind me of https://gfw.report/publications/usenixsecurity23/en/#8-1-customizable-payload-prefixes
and v2fly/v2ray-core#1552

@APT-ZERO
Copy link

It's a good option for SplitHTTP for CDN, but not for QUIC, as they already have Obfuscation options

Just let the user specify a bytestring directly, and that bytestring will be written as-is. They can write their own clienthello, or copy from some template.

Agree, but base64url is better than bytestring
and maybe a list of timing options appended to the payload

@APT-ZERO
Copy link

This method remind me of https://gfw.report/publications/usenixsecurity23/en/#8-1-customizable-payload-prefixes and v2fly/v2ray-core#1552

So it is possible to do the same for non-HTTP/non-TLS TCP connections like VMESS+TCP ?

@dragonbreath2000
Copy link
Contributor Author

dragonbreath2000 commented Aug 23, 2024

seam to be working
usage:

  {
        "protocol": "freedom",
        "settings": {
          "noise":{
            "packet":"rand:5-10", //str:higfw
            "delay":"5-10"
          }
        },
        "tag": "direct"
      },

I wrote two types of packet:
rand:min-max length
str:whatttttever
I added option for delay too,by defualt it is 0 but if anyone wants to add a delay before sending the real data it is here
And the tests should probably pass now

@RPRX
Copy link
Member

RPRX commented Aug 23, 2024

My take is we can do if change is clean and organized in one place. This method remind me of https://gfw.report/publications/usenixsecurity23/en/#8-1-customizable-payload-prefixes and v2fly/v2ray-core#1552

赞同

@APT-ZERO
Copy link

You can make it more advanced, to make firewalls unable to block it easily
you can add multiple payload support (for example: send static fake data and then 2 random data, with or without delay between them, and then the real data)

@Fangliding
Copy link
Member

Fangliding commented Aug 23, 2024

我认为现在的代码应该精简一些 最开始那一版就可以 加上json设置delay和随机字节长度就行了 这东西非常非常容易被滤掉 搞得再高精尖对面patch一下有问题的udp conn track就毙掉了 不用搞得太复杂 还搞个浮动

Comment on lines 445 to 463
if b.UDP != nil {
if w.Writer.UDPOverride.Address != nil {
b.UDP.Address = w.Writer.UDPOverride.Address
}
if w.Writer.UDPOverride.Port != 0 {
b.UDP.Port = w.Writer.UDPOverride.Port
}
if w.Writer.Handler.config.hasStrategy() && b.UDP.Address.Family().IsDomain() {
ip := w.Writer.Handler.resolveIP(w.Writer.Context, b.UDP.Address.Domain(), nil)
if ip != nil {
b.UDP.Address = ip
}
}
destAddr, _ := net.ResolveUDPAddr("udp", b.UDP.NetAddr())
if destAddr == nil {
b.Release()
continue
}
n, err = w.Writer.PacketConnWrapper.WriteTo(b.Bytes(), destAddr)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something is off, why does the noisewriter need to deal with ip resolution? it already wraps PacketWriter which has the same logic

_, _ = w.Writer.PacketConnWrapper.Write(noise)

if w.Writer.config.Noise.DelayMin != 0 {
time.Sleep(time.Duration(randBetween(int64(w.Writer.config.Noise.DelayMin), int64(w.Writer.config.Noise.DelayMax))) * time.Millisecond)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this is not an actual fragment, is the delay actually making a difference in this case? maybe it can be removed entirely to simplify the PR

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not? delay may make the firewall to judge the connection by the first bytes only, and not wait for more data and allow it
as nobody knows exactly how the target firewall works and nobody knows their limits, it could be useful for some users

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm asking if any difference has been observed in practice.

Context: ctx,
UDPOverride: UDPOverride,
if hasNoise {
return &NoisePacketWriter{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i suggest to structure the code similarly to FragmentWriter, and add this wrapper outside of this function

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am almost done , I clean up the code a lot,for this one I basicly had to create another function to return the wrapper,this PacketWriter uses counter,I do not know what it does(when i removed it it worked fine but it might mess something up so I kept it),for fragment they basicly added the wrapper in prooccess function,But in my case if I do that the code will get messy so I added another function(copy NewPacketWriter and return my wrapper instead)

@mmmray
Copy link
Collaborator

mmmray commented Aug 23, 2024

the config parsing for fragment is really extreme levels of copy-pasting, and it seems that in order to be consistent with it, delay should be uint. maybe it should be refactored in a later PR to use something like

// Int32Range deserializes from "1-2" or 1, so can deserialize from both int and number.
// Negative integers can be passed as sentinel values, but do not parse as ranges.
type Int32Range struct {

@Fangliding
Copy link
Member

Yes, the config parsing of fragment is very very confusing

@dragonbreath2000
Copy link
Contributor Author

@mmmray
I will fix the issues u said hopefully in a hour or two
I added that delay so if firewall would try to gather more data from the connection it would make it harder,I think it could be useful,right now it does not make much diffrence so if u say so I could remove it(but I suggest we keep it)

@APT-ZERO
Copy link

APT-ZERO commented Aug 23, 2024

@yuhan6665 Is it possible to do similar thing to full random TCP connections like VMESS+TCP ?
for example send a mssql header and then send the real data and start communication to the proxy server, firewall must consider the connection as a regular mssql connection
or append a fake header to client's first request packet, and make the server ignore it

@mmmray
Copy link
Collaborator

mmmray commented Aug 23, 2024

it is very unlikely to work for any TCP-based protocol. let's try to focus on getting this PR merged and discuss other ideas elsewhere...

@yuhan6665
Copy link
Member

Looks ok to me. Thanks for the good collaboration

@yuhan6665 yuhan6665 changed the title [draft]UDP noise UDP noise Aug 24, 2024
@dyhkwong
Copy link
Contributor

dyhkwong commented Aug 25, 2024

If UseAlternativeSystemDialer is used this will cause issue. e.g. the protected dialer of v2rayNG doesn't return a *PacketConnWrapper(in fact it is *net.UDPConn), so assertion will fail and PacketWriter and PacketReader of freedom will not be used. How come almost nobody found "full cone nat" is broken for freedom in v2rayNG in the past years? Xray and/or AndroidLibXrayLite need fixes for this.

@dragonbreath2000
Copy link
Contributor Author

How come almost nobody found "full cone nat" is broken for freedom in v2rayNG in the past years?

Adding a *net.UDPconn assertion in case of *Packetconnwrapper assertion failure should fix this,right?(not sure if this is the best way to fix)
Is there a specific reason why v2rayng returns net.udpconn instead on packetconnwrapper?

@dyhkwong
Copy link
Contributor

dyhkwong commented Aug 25, 2024

v2rayNG use this for VpnService protection. Maybe Xray need to fix its freedom NewPacketReader and NewPacketWriter not to strongly rely on *PacketConnWrapper first.

As the system dialer is designed to be alterable, and Dial returns a net.Conn, it is not guaranteed that a net.Conn can be casted to *PacketConnWrapper or *net.UDPConn. So you may need a universal net.Conn writer as the last resort.

@developer861
Copy link

i think a developer did it in a xray fork for iranian users

https://github.com/GFW-knocker/Xray-core

this is the project

@dyhkwong
Copy link
Contributor

dyhkwong commented Aug 28, 2024 via email

@mmmray
Copy link
Collaborator

mmmray commented Aug 28, 2024

Thanks, fair enough. I think this commit should fix it, please check it... although this noise generator is mostly for a freedom dialerProxy on the tunnel's connection, and so I think the cone-ness of that outer protocol doesn't matter that much 🤔

@yuhan6665 yuhan6665 merged commit 002d08b into XTLS:main Aug 28, 2024
36 checks passed
@yuhan6665
Copy link
Member

Merged. Thanks all!

@dragonbreath2000 dragonbreath2000 deleted the udp-noise branch August 28, 2024 22:44
@RPRX
Copy link
Member

RPRX commented Aug 30, 2024

又要在 release notes 中改 commit title 了

@RPRX
Copy link
Member

RPRX commented Aug 30, 2024

我简单看了下配置,为什么这个不叫 udpnoise,好吧 fragment 也没叫 tcpfragment,那没事了

zxspirit pushed a commit to zxspirit/Xray-core that referenced this pull request Aug 30, 2024
* added udp noise

* adding protobuf settings

* freedom json parser and clean up

* resolve confict

* fix and clean up

* use net.conn instead of packetconnwrapper

* avoid constructing SequentialWriter directly

---------

Co-authored-by: mmmray <142015632+mmmray@users.noreply.github.com>
@RPRX
Copy link
Member

RPRX commented Aug 30, 2024

感觉这里确实需要 base64 来支持非可打印字符,以及需要数组以支持连发多个 noise 包,“不能出现冒号”的限制应当去掉

@Fangliding 文档把 Freedom 改为“Freedom(fragment、noise)”,还有顶部配置示例中 "noise" 咋被绿了

@RPRX
Copy link
Member

RPRX commented Aug 30, 2024

以及需要数组以支持连发多个 noise 包

大概需要 noise 本身变成数组 noises

Fangliding added a commit to XTLS/Xray-docs-next that referenced this pull request Aug 30, 2024
@Fangliding
Copy link
Member

Fangliding commented Aug 30, 2024

感觉这里确实需要 base64 来支持非可打印字符,以及需要数组以支持连发多个 noise 包,“不能出现冒号”的限制应当去掉

Base64确实可以 不能用冒号是因为代码里用 split : 来分割 用冒号就割成几个了 当然如果换base64或者SplitN就可以解决这个问题了

至于发几个个人觉得就没啥必要了 等有墙开始开始解析第二个数据包但是放过第三个再说

@Fangliding 文档把 Freedom 改为“Freedom(fragment、noise)”,还有顶部配置示例中 "noise" 咋被绿了

Done, 变绿是因为不小心打出了个中文冒号

@RPRX
Copy link
Member

RPRX commented Aug 31, 2024

Base64确实可以 不能用冒号是因为代码里用 split : 来分割 用冒号就割成几个了 当然如果换base64或者SplitN就可以解决这个问题了

有没有一种可能 仅以第一个冒号来分割就可以避免歧义 当然 base64 也是有需求的

至于发几个个人觉得就没啥必要了 等有墙开始开始解析第二个数据包但是放过第三个再说

有没可能 如果直接写成 noises 数组可以一劳永逸 总之我觉得这个 PR 合并得有点快了 我只是说赞同可以加这个功能 还没 review

@Fangliding
Copy link
Member

有没有一种可能 仅以第一个冒号来分割就可以避免歧义 当然 base64 也是有需求的

strings.SplitN就是用第一个冒号啊((

@linehman
Copy link

linehman commented Sep 5, 2024

Thanks, but we currently prefer not to merge such 'tricks'
see #3677 (comment)

I undrestand why u say this,u might be right the second u merge it people will use it and providers will just throttle udp for cloudflare,but it is not just about using it with cloudflare ,imagine I want to use a free wireguard service (I am not talking about warp,this one is already throttled heavily but there are others too,don't want to name them here) there is no way I can use them normaly but with this trick it works,basicly more people will have access to free proxy services,this is not a full solution but for a lot of times it could be usefull,and lastly they might throttle cloudflare but what if i want to use this with another cdn?amazon(this one almost never gets QOSed atleast where I live and I think it supports quic too) and I am not saying everyone should go and use amazon I am saying "if" anyone wants to use it this can help , as I mentioned it is not a full solution ,Best of luck to all the developers of this project other thing is most firewalls have limited resources and usually they do very little inspection when it comes to udp and this trick really takes advantage from that

Hi there, would you mind sharing a sample client json config for wireguard outbound with udp noise. I'm not sure if I'm doing this correctly. Thanks

@dragonbreath2000
Copy link
Contributor Author

my take on having multiple noise packets:
for a trick like this.adding more options and features is not the best idea unless it makes a diffrence,I do not know about China but in I .ran it bypasses quic blockages of mci without issue(which was my initial goal for this pr) so as long as it works adding more stuff is kinda useless,sending multiple noise packets could be nice but why?they hardly do any inspection for udp traffic,adding more noise is only a waste of bandwidth right now

@RPRX
Copy link
Member

RPRX commented Sep 6, 2024

my take on having multiple noise packets: for a trick like this.adding more options and features is not the best idea unless it makes a diffrence,I do not know about China but in I .ran it bypasses quic blockages of mci without issue(which was my initial goal for this pr) so as long as it works adding more stuff is kinda useless,sending multiple noise packets could be nice but why?they hardly do any inspection for udp traffic,adding more noise is only a waste of bandwidth right now

我是觉得如果某件事可以一次性做完一劳永逸,那么从一开始就设计完善比较好,就像 fragment 的 PR 初始版本远没有经我建议修改后的版本完善,同理人们会有发指定 UDP 包(base64)的需求,以及有多个 noise packets 的需求(比如现在 Chrome 的 QUIC hello 应该要占两个 UDP 包),并且当我们加上这些特性后就可以组合出 99% 的需求,以后就不用再改这个功能了。

不过现在的 workaround 是两个 freedom 套娃

@APT-ZERO

This comment was marked as off-topic.

@mmmray
Copy link
Collaborator

mmmray commented Sep 7, 2024

  • I personally did not have a problem with your behavior in this thread until now.
  • Your comments are not being removed, at worst they're being collapsed because they're considered offtopic (RPRX/fang/my opinion may differ case-by-case)
  • The reason I made that remark in telegram was obviously not related to this discussion, so I don't know why we're discussing your person here.
  • Base64 was discussed privately before this PR was opened. At the time I suggested against adding features if they are not needed for Iran, and the main motivation was to get something out of the door fast, and because the code for freedom outbound is already too complex. I don't know about other firewalls. If "full chrome clienthello" works on some other GFW, sure, can do it. But IMO not just for completeness but because somebody managed to prove an effect on some GFW
    • So I am not "clapping for" RPRX in the context of this discussion, although I don't have a strong opinion on this topic at all.

Everybody has ideas all the time. But the fundamental truth about open source is that no matter the idea, it won't happen until somebody puts in the work and implements it. You open new issues with many small things every single week and expect us to respond to each one of them. You don't read existing discussions and re-litigate old decisions. This takes time from core maintainers, and the time that you spend arguing about random things on this forum, you could've learned golang and implemented your own ideas 10 times over.

This has been going on for many months now, and you're not learning anything at all from the feedback we give you. I think it's completely insane to say that any of this happens for "no reason".

@RPRX
Copy link
Member

RPRX commented Sep 9, 2024

@APT-ZERO 这主要是因为你此前的发言经常是 unreasonable,使得大家已经下意识 ignore 你的发言了,当然我还是会看一下

@RPRX
Copy link
Member

RPRX commented Sep 9, 2024

@APT-ZERO 虽然但是,我觉得最近你已经比以前靠谱一些了,然而放隔壁你早就被 block 了,你应该庆幸这里是 Project X

leninalive pushed a commit to amnezia-vpn/amnezia-xray-core that referenced this pull request Oct 29, 2024
* added udp noise

* adding protobuf settings

* freedom json parser and clean up

* resolve confict

* fix and clean up

* use net.conn instead of packetconnwrapper

* avoid constructing SequentialWriter directly

---------

Co-authored-by: mmmray <142015632+mmmray@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants