Project: Support Cronet Go binding

[klzgrad]

Any Go developers who are interested in using the core of naiveproxy in Go natively, please see my proposal and comment below:

The core of naiveproxy is namely Chromium's net stack. It's possible to expose its interface in the style of BSD socket in a library libnaive.so or libnaive.a (or more accurately libchromiumsocket.a), which then could be hooked up with a Go program via FFI. The benefit of doing this: no socks5 round trip overhead for using Chromium's net stack, and more versatile to link directly with Go's net stacks at socket/connection level. The impact of doing this: a better TLS fingerprint parrot than utls with similar Go development friendliness.

To achieve this goal:

• I can provide the code and build system to produce the proposed libnaive.a and corresponding C headers.
• An interested Go developer to discuss the interface definition for the Go FFI.
• Said Go developer develops Go binding and example integration showing Chromium net stack working in Go natively.

• Cronet's official Android build is for Java, but here the build is native and intended for use from Go. Do you @nekohasekai think it's better to use Cronet from Java or Go? I'll take it as a yes.
• The current build produces a shared library. Do you @nekohasekai think it's better to use a static library to link with Go?
• My Cronet build on Android still doesn't work.
• My Cronet build doesn't work with ThinLTO.
• I'm considering finalizing it to -connect-authority-override in the next version. Really hard to name this one. Currently it's decided as -connect-authority, as connect sufficiently defines the scope of this header, the leading - denotes its internal nature, and the suffix -override doesn't seem absolutely necessary and the lack of it doesn't introduce confusion.
• Record optimal static linking ldflags for each arch
• Add CI test case for cornet example and go example
• Document diff from upstream
• Write a specification of protocol used in Naiveproxy
• Add -network-isolation-key header to smuggle it in and configure stream isolation.

[nekohasekai]

Chromium already has cronet for calling from platforms like android, but it's basically https.

I'm not familiar with C/C++, but if you're done, I'll try it.

[klzgrad]

Cronet only provides a request-response API. One example: https://developer.android.com/guide/topics/connectivity/cronet/reference/org/chromium/net/UrlRequest.Builder.html#setHttpMethod(java.lang.String) the CONNECT method is not supported. I don't think there's any other way to create bidirectional sockets. Another proof is Cronet does not support WebSocket.

Cronet does support bidirectional "sockets". Several years ago when I checked, it was being used from gRPC with odd support status (ios only, or whatever, https://source.chromium.org/chromium/chromium/src/+/main:components/grpc_support/README.md: Currently Cronet (//components/cronet/ios) is the only consumer of this API). But it does support it. grpc-java does so via org.chromium.net.ExperimentalBidirectionalStream, grpc does it via third_party/objective_c/Cronet/bidirectional_stream_c.h (note the "objective_c", so it was designed for ios, but the cronet API is in C, which means this path may work but it was never tried). The reason I did not find it is because ExperimentalBidirectionalStream is not documented in https://developer.android.com/guide/topics/connectivity/cronet but it is documented in https://source.chromium.org/chromium/chromium/src/+/main:components/cronet/android/api/src/org/chromium/net/ExperimentalBidirectionalStream.java.

If you use Cronet's ExperimentalBidirectionalStream:

• Its traffic behavior goes through the same TLS code path with NaiveProxy and regular URL requests, so, same TLS behaviors. But on higher levels, internally it goes through BidirectionalStreamSpdyImpl, which is somewhat different from NaiveProxy's SpdyProxyClientSocket and http stream paths of regular URL requests. I'm just admitting imperfect parrot here, because I can't audit them exhaustively now within the large Chromium net stack.
• ExperimentalBidirectionalStream will need some care to avoid 1-RTT latency from waiting for HTTP response.
• ExperimentalBidirectionalStream is not exactly a socket. Some slightly more work is needed to handle its HTTP CONNECT logic. Auth headers.
• This happens in Java or C. It probably doesn't make sense to run heavy I/O between Java and Go at the same time. Assuming I/O heavy work happens in Go, then the work is to link Cronet native API to Go. The C API looks like this: https://source.chromium.org/chromium/chromium/src/+/main:components/grpc_support/include/bidirectional_stream_c.h. I will build a C++ test case to try this out before going with using Cronet's bidirectional stream.

The more ambitious goal of providing a general Go connection API with better TLS parroting seems better achieved with building a Go FFI with Cronet native API (if it works and is without major restrictions from the original Chromium stack). This requires more work, some design review, and more testing.

The less ambitious goal is to provide a specific Go API for running the logic of NaiveProxy inside Go address space without socks5 round trips, as a proxy backend along side other backends. This requires less work and is easier.

Recap of NaiveProxy

The changes made to Chromium

• Build minimization
  c647d3d base: Remove JNI function on Android
  b3a4768 dns: Fix iwyu
  0ac6ca7 libc++: Disable exceptions and RTTI
  543d0bd url: Remove perfetto tracing
  ecd4991 base: Disable trace event
  093898a base: Don't fix Y2038 problem with icu
  be723d4 net, url: Remove icu
  1c639b0 build: Disable Android java templates
  1c30676 build: Disable build_with_chromium
  27de40a base: Add Android stubs
  6750e63 net: Add Android stubs
  1fa442a build: Remove tests and minimize
  79fecf9 cert: Use builtin verifier on Android and Linux
  8708bc1 cert: Add SystemTrustStoreStaticUnix

• OpenWrt build support
  e3ef20e Fix OpenWrt build with use_allocator_shim=false
  1894111 allocator: Improve MIPS coverage of spinlocks
  61512a2 debug: Fix uClibc macro condition
  7e3a06f third_party: Fix missing sgidefs.h for Musl
  7104046 base: Do not forward declare stat64 for Musl
  7105b1e base: Fix narrowing casting for Musl
  eea4cbe base: Disable __close overloading for Musl
  5cfc396 process: Remove use of mallinfo for Musl
  d43ad9e base: Remove use of mallinfo for Musl
  489ba5d udp: Fix mmsghdr struct initializer for Musl
  2c0f6ff dns: Support Musl
  07dedde build: Add OpenWrt toolchains
  de5f764 build: Support MIPS -mtune= flag
  9df671f build: Support ARM build without FPU
  2c44fd6 build: Support -mcpu= on ARM and ARM64
  a8a02e3 libc++: Guard C++20 atomic type aliases
  e543ca2 lss: Avoid naming conflict in fstatat64

• Android build support
  c6606de Fix android build

• CI/CD
  9297696 Add continuous integration and tests
  0d649f1 Add build scripts
  a8c41f6 Add source import tool
  a38b058 build: Handle empty pkgconfig in sysroot
  b62139c build: Add sysroot creator script
  03f0fcf build: Force determinism in official build
  8c4e850 Add .gitignore

• Docs
  d07b4a5 Add example config.json
  a754cb6 Add README
  bc51961 Add LICENSE

• Naive itself: bf42527 Add initial implementation of Naive client, a hopefully complete specification of what it does:

  • http_proxy_socket.cc: an HTTP proxy frontend that accepts HTTP requests from clients (features: CONNECT method only; detects client opt-in of payload padding using padding header in the HTTP CONNECT request; advertise padding capability using padding header in the HTTP CONNECT response)
  • socks5_server_socket.cc: a SOCKS5 proxy frontend that accepts SOCKS5 requests from clients
  • naive_connection.cc: a bidirectional pipe that connects a client socket accepted from a frontend (socks5, http, redir) with Chromium's ClientSocketHandle which is initiated towards the requested proxy origin (features: payload padding encapsulation and decapsulation; )
  • naive_proxy.cc: Configures Chromium proxy settings, TLS params, parallelism with network isolation keys; A socket server that accepts TCP connections and moves them to backend
  • naive_proxy_bin.cc: Configures Chromium ClientSocketPool limits, config parsing, configures logging, builds URLRequestContext
  • naive_proxy_delegate.cc: Detects downstream and upstream padding capability and enables Fast Open logic in H2/H3 code in Chromium net stack.
  • redirect_resolver.cc: Deprecated, don't care

• Changes of Chromium required by Naive
  910fcc5 debug: Fix obsolete max check
  0198ac5 quic: Add support for HTTP/3 CONNECT Fast Open
  5f91e9d h2: Pad RST_STREAM frames
  a9bdea4 h2: Add support for HTTP/2 CONNECT Fast Open
  b95497e h2: Reduce warnings about RST on invalid streams
  6fe5dc1 socket: Force tunneling for all sockets
  3c48a78 socket: Allow higher limits for proxies
  f8e486b socket: Add RawConnect method
  bcd5d09 cert: Handle AIA response in PKCS#7 format

• Chromium itself
  1c761bb Import chromium-100.0.4896.60

Apparently much of Naiveproxy itself can be implemented with less and cleaner Go code.

Key changes and uses of the Chromium net stack, summary

• Adds support in Chromium's H2 and H3 code to support HTTP CONNECT "Fast Open", i.e. does not wait for the response after sending the HTTP CONNECT request and sends the payload immediately.
• Pads H2's RST_STREAM frames to slightly increase packet length entropy.
• "Handles AIA response in PKCS#7 format", otherwise it complains about issues dealing with Let's Encrypt's certificates, but it hasn't been thoroughly investigated whether this is necessary.
• Exposes a "RawConnect" function that creates Chromium ClientSocketHandle directly.
• Configures Chromium's proxy setting, QUIC stack versions, logging, extra headers, DNS resolver static mapping,

[klzgrad]

Cronet's BidirectionalStream:

• With proxy setting via myproxy.com, a BidirectionalStream to https://example.com will send TLS and HTTP headers to example.com, before sending user agent data to example.com. This does not work with the model where the user agent (browser) requires a socket to send TLS and HTTP headers to example.com for E2E encryption. (Cronet is the intermediary, not user agent)
• Without proxy settings, a BidirectionalStream to myproxy.com cannot use the CONNECT method to create HTTP tunnels, because the authority field can only be myproxy.com. Patching it to support this use case is very messy and not realistic, because the entire concept of "stream" in Chromium does not consider tunnels.

So the use case BidirectionalStream can support is something like WebSockets, where the origin is represented in other fields.

In NaiveProxy both the client intermediary and server intermediary assume HTTP/2 CONNECT tunneling. A proxy system using Cronet's BidirectionalStream won't work with Caddy's forwardproxy (without abusing HTTP headers).

TBC

[klzgrad]

Patching it to support this use case is very messy and not realistic
Cronet's BidirectionalStream won't work with Caddy's forwardproxy (without abusing HTTP headers)

I managed to find a not very messy way to smuggle the authority in: Pass a _real_authority HTTP header and use its value to override the authority when constructing the CONNECT header, and delete this fake header so it is not transmitted. This way a CONNECT tunnel can be set up, and then there is Cronet bidirectional stream API for simple read and write. With this, Cronet can be used as a substitute the "core" of Naiveproxy without the "major restrictions" mentioned above.

Architecturally Cronet and Naiveproxy both use the Chromium net core, but Cronet uses BidirectionalStream and Naiveproxy uses SpdyProxyClientSocket/QuicProxyClientSocket. BidirectionalStream and SpdyProxyClientSocket are also very similar. Naiveproxy is a more "lightweight" use of the Chromium net core because it can make more assumptions and use more simplifications. Cronet exposes more complete control interfaces.

In terms of the API as a product, Cronet is a better option because its API is better maintained. So unless more major restrictions are discovered (e.g. parrot detectors), it's better to use Cronet as basis to build proxy apps.

With that said, it's still better to use my fork of Chromium to build Cronet, because the build is minimized and supports OpenWrt and other niceties.

	Official Cronet release	My Cronet fork
Supported platforms	Android, iOS	Linux, Openwrt, Windows, Android, MacOS
Supported API	Java, ObjC	C
Usage	No CONNECT tunnels	Supports CONNECT tunnels

@nekohasekai The following is an example of using Cronet bidirectional stream:

naiveproxy-cronet-test-linux-x64.tar.xz from https://github.com/klzgrad/naiveproxy/releases/tag/cronet-test

# Note this does not use Chromium's build system
g++ bidi_example.cc libcronet.100.0.4896.60.so
# SSLKEYLOGFILE is saved at /tmp/keys so Wireshark can use it.
LD_LIBRARY_PATH=$PWD ./a.out https://my-caddy-forwardproxy.com "Basic $(printf user:pass | base64)"

Remaining questions:

[nekohasekai]

I have just tried

https://gist.github.com/nekohasekai/dda9ba499332433ba7a94f5f3d690a53

$ LD_LIBRARY_PATH=$PWD go run .
[0503/130502.548494:ERROR:cert_verify_proc_builtin.cc(603)] No net_fetcher for performing AIA chasing.
2022/05/03 13:05:02 on_stream_ready_callback
2022/05/03 13:05:02 on_response_headers_received, negotiated_protocol= h2
2022/05/03 13:05:02 :status:302
2022/05/03 13:05:02 cache-control:no-cache
2022/05/03 13:05:02 pragma:no-cache
2022/05/03 13:05:02 content-length:149
2022/05/03 13:05:02 content-type:text/html; charset=utf-8
2022/05/03 13:05:02 expires:-1
2022/05/03 13:05:02 location:https://www4.bing.com/?form=DCDN
2022/05/03 13:05:02 p3p:CP="NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND"
2022/05/03 13:05:02 set-cookie:SUID=M; domain=.bing.com; expires=Wed, 04-May-2022 05:05:02 GMT; path=/; HttpOnly
2022/05/03 13:05:02 set-cookie:MUID=2B8BC37FAEB966EC2616D2E6AFB167E3; domain=.bing.com; expires=Sun, 28-May-2023 05:05:02 GMT; path=/; secure; SameSite=None
2022/05/03 13:05:02 set-cookie:MUIDB=2B8BC37FAEB966EC2616D2E6AFB167E3; expires=Sun, 28-May-2023 05:05:02 GMT; path=/; HttpOnly
2022/05/03 13:05:02 set-cookie:_EDGE_S=F=1&SID=241DA119413A6A0C1EBCB08040326B77; domain=.bing.com; path=/; HttpOnly
2022/05/03 13:05:02 set-cookie:_EDGE_V=1; domain=.bing.com; expires=Sun, 28-May-2023 05:05:02 GMT; path=/; HttpOnly
2022/05/03 13:05:02 set-cookie:SRCHD=AF=NOFORM; domain=.bing.com; expires=Fri, 03-May-2024 05:05:02 GMT; path=/
2022/05/03 13:05:02 set-cookie:SRCHUID=V=2&GUID=FEA003B5EF0546E1B9F9EC8783811781&dmnchg=1; domain=.bing.com; expires=Fri, 03-May-2024 05:05:02 GMT; path=/
2022/05/03 13:05:02 set-cookie:SRCHUSR=DOB=20220503; domain=.bing.com; expires=Fri, 03-May-2024 05:05:02 GMT; path=/
2022/05/03 13:05:02 set-cookie:SRCHHPGUSR=SRCHLANG=zh-Hant; domain=.bing.com; expires=Fri, 03-May-2024 05:05:02 GMT; path=/
2022/05/03 13:05:02 set-cookie:_SS=SID=241DA119413A6A0C1EBCB08040326B77; domain=.bing.com; path=/
2022/05/03 13:05:02 x-snr-routing:1
2022/05/03 13:05:02 strict-transport-security:max-age=31536000; includeSubDomains; preload
2022/05/03 13:05:02 x-cache:CONFIG_NOCACHE
2022/05/03 13:05:02 accept-ch:Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version
2022/05/03 13:05:02 x-msedge-ref:Ref A: 03C48A42AE58420984C42B2E9D1FB30E Ref B: HKBEDGE0513 Ref C: 2022-05-03T05:05:02Z
2022/05/03 13:05:02 date:Tue, 03 May 2022 05:05:01 GMT
2022/05/03 13:05:02 on_read_completed
2022/05/03 13:05:02 <html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="https://www4.bing.com/?form=DCDN">here</a>.</h2>
</body></html>

2022/05/03 13:06:05 on_failed
2022/05/03 13:06:05 net error  -365

Seems to work, but looks like it requires a lot of wrappers to use in go

[klzgrad]

looks like it requires a lot of wrappers to use in go

I don't know how many is a lot. 100-200 lines of code seem reasonable.

There can be something to do on the C side to make things nicer. I can think of this golang/go@918396b _GoString_ which helps a bit. What are the specific complexities in wrapping it? I can check if there are other changes in C that makes things easier in Go.

[nekohasekai]

I don't know how many is a lot. 100-200 lines of code seem reasonable.

I am writing some wrapper like this:

[nekohasekai]

Unless we have a code generator for that idl, I guess

Maybe https://chromium.googlesource.com/chromium/src.git/+/HEAD/mojo/README.md

[klzgrad]

Chromium's Mojo does not support Go bindings.

I kind of get what you're trying to do. cronet.idl exports 100~200 functions. An automatic binding generator wouldn't produce code as nice as that of manually written, which is not a small undertaking but also achievable with a few days of work. And once written, it's also a complete and general Go binding on its own, for the official Cronet library.

I would suggest starting with the important part of the API first so to prove it works, in terms of replacing libnaive.so with native use of Cronet. Even though it can be fun to engage in completionism, as I can imagine, PublicKeyPins is not needed for proving the above concept.

[klzgrad]

https://github.com/klzgrad/naiveproxy/blob/cronet-test/src/net/spdy/spdy_http_utils.cc#L110

I changed it to _real_authority some time ago.

[nekohasekai]

Everything works now

SagerNet/cronet-go@1826de4

[klzgrad]

Only bidirectional_stream.go needs LDFLAGS with libcronet. Others don't need it, or they are just adding duplicate ldflags to linker invocation.

Static linking:

bidirectional_stream.go:
// #cgo LDFLAGS: -fuse-ld=lld -Wl,--as-needed libcronet_static.a -ldl -lpthread -lrt -lresolv -latomic -lm
sudo apt install clang-15 lld-15
CGO_LDFLAGS_ALLOW="-fuse-ld=lld" CC="clang-15" go build example/main.go

This produces working but less optimal binaries than libcronet.so (optimized with LTO and other stuff). The optimal ldflags are complicated and platform-dependent. But this produces a single binary compared to two binaries with libcronet.so (LD_LIBRARY_PATH=$PWD can be saved with -Wl,-rpath,$ORIGIN).

[klzgrad]

Would not recommend binary packing like UPX. It's reinventing something at least, in the linker infra, and may have problems with security measures against self modifying binaries. It could work, but I doubt the aesthetic is worth the effort.

Static linking of libcronet but not the rest of the common libraries. E.g. I don't think you can not link with libandroid.so on Android. But as long as the common libraries have no version and availability issues, they can be depended upon with dynamic linking.

[nekohasekai]

Static linking of libcronet but not the rest of the common libraries

As long as the dynamic linking to cronet is successful, only need to package libcronet.so into apk/lib/arch.

By the way, android doesn't have pthread. Included in libc.

As well gomobile has some problems, it's old and the android part doesn't seem to be maintained anymore.

[nekohasekai]

The version of aarch64-linux-android21-clang in NDK is 12.0.8 and I don't know how to do static linking

[nekohasekai]

/* Returns true if the |stream| was successfully started and is now done
 * (succeeded, canceled, or failed).
 * Returns false if the |stream| stream is not yet started or is in progress.
 */
GRPC_SUPPORT_EXPORT
bool bidirectional_stream_is_done(bidirectional_stream* stream);

ld reports bidirectional_stream_is_done not found

[klzgrad]

bidirectional_stream_is_done was never implemented chromium/chromium@06eab3e#diff-9bb8e6ea79473fee571b5eb74e0959307330d19e9eb9f5a36cf05a6736e5291bR197

It was derived from isDone() in components/cronet/android/java/src/org/chromium/net/impl/CronetBidirectionalStream.java.

CronetBidirectionalStream.isDone() is only used in test cases, so not super useful. I think it can be ignored.

[klzgrad]

I don't know how it's used to build Android apps. If the end product is something like libgojni.so in an apk, then using libcronet.so is quite appropriate. Static linking producing single binaries is for releases like this https://github.com/v2fly/v2ray-core/releases.

[nekohasekai]

Generating a release like v2ray requires disabling CGO, which is not possible with cronet.

Maybe need another get-clang.sh

[klzgrad]

I don't know much about v2ray build, but I don't see an inherent reason or difficulty why CGO is required to be disabled. According to this guy, v2ray/discussion#754 (comment) CGO makes cross compile difficult. I don't see any specifics of where it is difficult, but there is definitely something. Then it has to showcase a Go app where cross compile works with Cronet CGO.

[klzgrad]

Ok, how about this: Use Cronet to recreate naiveproxy in pure Go (except libcronet.so) and see how many platforms Go can build for. As I summarized in "Recap of NaiveProxy" above, I suspect the actual logic of naiveproxy can be expressed in 100-200 lines of code in Go (besides protocol libraries). This should happen in cronet-go repo to see what would happen if a third-party developer tries to use cronet-go for cross compile.

[nekohasekai]

see how many platforms Go can build for

The answer is only host arch can build, we need cross compiler for each target (and sysroot)

SagerNet/cronet-go@d46f601

[nekohasekai]

How to set insecure-concurrency from cronet?

[Riatre]

I don't see any specifics of where it is difficult, but there is definitely something.

Drive by comment on "the specifics":

CGO has to be used together with Go's own build system, which in turn calls $CC and $LD to build C codes. It also does some insane things (parsing gcc stderr to do compile-time introspection on C struct/union/enum). Therefore, a typical CGO_ENABLED=1 go build kinda requires a cross compiler to be presented. And... people are hesitated to pull in a cross compiler for whatever reason.

In this case, since libcronet has to be built with another build system, as long as we ship one prebuilt .a per arch, most of the above does not directly apply (it should be possible to convince cgo to never call $CC in our package, and lld is mostly a hassle free cross-linker), HOWEVER:

1. In Go's build system, it is possible to detect whether cgo is enabled and do different things. Some popular packages include a "faster" variant with C dependency, and use a pure Go fallback in case cgo is disabled, but break if cgo is enabled but broken (e.g. without a working cross compiler). This could happen in transitive dependencies.
2. We need the libc.so/libpthread.so/... or at least their dynsym only stub of the target platform to -lc -lpthread -latomic <...>. This is okay for Android (just download NDK), painful for *-linux-gnu, and may drive people nuts for OpenWRT.

We could try to bundle required dynsym stubs along with prebuilt .a-s, similarly to what Zig does, but it is definitely not trivial.

[klzgrad]

it should be possible to convince cgo to never call $CC

What does this mean? There are some struct definitions in bidirectional_stream_c.h, which cronet-go imports. CC would be needed to get its layout, no?

transitive dependencies

I see, the problem is not creating a perfect cross compile tool chain for building a binary, but instead creating a cross tool chain that can be transitively included with other packages and not conflict with other CGO packages with possibly their own cross tool chain?! It seems cross toolchains don't really coexist with each other, and there has to be one single universal tool chain that handles everything?

painful for *-linux-gnu, and may drive people nuts for OpenWRT

This part is a solved problem in naiveproxy. The solution has three parts: a cross compiler (a single clang binary), sysroots creator, the flags to build it. There is some work to put the sysroots in better packaging so easier to use, and extract build flags from chromium for external use.

[Riatre]

CC would be needed to get its layout, no?

Yes. I mean it is possible if we come up with a struct-free FFI. I agree it would be horrible.

the problem is not creating a perfect cross compile tool chain for building a binary

The problem is Go developers prefer CGO_ENABLED=0 over installing and configuring a cross toolchain. Even if you make a perfect toolchain, you still need to convince those developers that "v2ray has to be built with CGO_ENABLED=0" is a myth and if they do A then B then C it could work.

Remember that Go is its own build system, in this build system, cross toolchains are configured by setting environment variables before invoking go build . It could not be bundled in a Go package, so there is no way to magically make our toolchain available.

And yes, the one toolchain has to be able to build all CGO dependencies of a project.

Actually, in 2022 we do have something close to "the perfect cross toolchain" (with a few rough edges), and people do try to use it with cgo: https://dev.to/kristoff/zig-makes-go-cross-compilation-just-work-29ho. Maybe we could promote that.

[nekohasekai]

Even if you make a perfect toolchain, you still need to convince those developers

I'm just talking about the fact that CGO-enabled projects can't be directly cross-compiled successfully to that many targets, but no myth.

There are already projects like xgo that create cross-compilation environments with gcc to help with compilation, but not perfect.

the perfect cross toolchain

https://ziglang.org/learn/overview/#zig-ships-with-libc

It only comes with musl for linux, glibc for linux and windows, apparently not working out of the box for openwrt/darwin/bsd/...

By the way, currently static linking libcronet depends on llvm-15 which has not been stably released yet

[klzgrad]

Issues related to cross compiling cronet-go should continue in SagerNet/cronet-go#1. This issue is used for tracking the C part of libcronet. Cronet-go is the Go part.

In short it's doable because it's only adapting existing cross tool chain in naiveproxy to cgo. There will be a showcase showing how it's done.

[nekohasekai]

The existing api only provides switches for http2 and quic, while http1 is always enabled. I think we need a mandatory setting, otherwise quic naive client will not work as expected.

[klzgrad]

http1 is always enabled

It's not? Bidirectional stream doesn't allow http endpoint. Didn't understand your issue. Please rephrase.

[nekohasekai]

Sorry, this came up on url request, it doesn't appear to be the problem. (quic seems to almost never be used when using url requests.)

[klzgrad]

https://github.com/klzgrad/naiveproxy/releases/tag/cronet-test6

I admit, CGO on Windows is hard. But it's done.

The hardest part is this:

Chromium uses clang-cl.exe, but CGO officially only supports GCC/MinGW on Windows. See golang/go#17014.

1. CGO hardcodes GCC options incompatible with clang-cl, so an extra clang.exe is required (Chromium only provides clang-cl.exe).
2. We need CGO to link LLVM bitcode from Chromium, so ld cannot work and lld is required.
3. CGO passes GCC options incompatible with lld, so an extra lld wrapper is required to remove those options.
4. I didn't figure out a way to make the whole pipeline use lld-link-wrapper cleanly:
   • -fuse-ld=lld --ld-path=lld-link-wrapper reports --ld-path is an unknown argument.
   • -fuse-ld=.../lld-link-wrapper.exe creates garbled linker path.
     So uses a hack to rename lld-link.exe to lld-link-old.exe which is called from the wrapper "lld-link.exe".
5. lld-13 does not work with bitcode produced by Chromium's lld-15.
   So copies clang-13 from environment to Chromium's LLVM bin directory, and uses clang-13 together with lld-15.
   cgo_ldflags="-fuse-ld=lld $cgo_ldflags"

Tested build mode: dynamic executable linking with dynamic cronet library, dynamic executable linking with static cronet library.

Not tested build mode: dynamic library linking with static cronet library, static executable linking with static cronet library.

Known deficiencies: PIE executables on x86 Ubuntu and OpenWrt (except Android) segfault, so PIE is disabled on those platforms. I still haven't found out why.

[lqhuang]

Any interest to support Rust binding in similar way? I'm glad to help or implement them!

[klzgrad]

It's a C API. If Rust has C FFI, it can use Cronet. I know nothing about Rust's build system, so you'll have to hook it up with the Cronet build artifacts here yourself. I can help from the C side, including the C/C++ build system.

My main concern about a Rust binding is: A Go binding of Cronet allows many Go projects like v2ray-core to use this high fidelity TLS stack with relatively low development cost, so the work has more impact. I don't know if there are many projects that can make use of a Rust binding.

[lqhuang]

Actually, at the very first, I'm curious about could we implement an user space congestion algorithm (eg: BBR) for QUIC protocol to naiveproxy? Then, I saw your discussions on golang binding, good work! It's may easier (or not?) to implement user-space congestion algorithms in rust with existed libs. Perhaps, it wouldn't be solid as an industry-level implementation, but could be an extension. Finally, I just found user space congestion control has already be configurable for chromium (see groups.google.com), we're probably able to export this option to config. Hence, rust binding maybe is a wrapper for fun, or could be implemented as a SIP003 plugin for shadowsocks.

Credit: Inspired by tuic

[klzgrad]

Fun is a enough reason. Feel free to go ahead, and report issues here if any.

Cronet extensions are doable. Post it if you need any.

[klzgrad]

Considered done.