feat: make shorebird easier to use by developers in China

[eseidel]

Update Sept 11, 2024: We know of at least some developers using Shorebird successfully from China today.
Users should be able to download published patches through our CDN (without needing to do anything special):
https://shorebird.dev/blog/improved-patch-delivery/
However the updater client also talks to api.shorebird.dev which is hosted in Google Cloud and thus may not be accessible.

Developers may also still need to find a way to reach Google Cloud for using shorebird tooling such as using a VPN.

=====

The directions for using Shorebird in china include setting FLUTTER_STORAGE_BASE_URL:
https://docs.flutter.dev/community/china

This environment variable is ignored by shorebird, since shorebird itself uses FLUTTER_STORAGE_BASE_URL to pass download.shorebird.dev to flutter.

The larger issue for developers in China is that Shorebird currently uses Google Cloud as our infrastructure provider and Google Cloud is not reliably reachable across China.

We could implement a proxy within china for download.shorebird.dev and either then make it possible to use FLUTTER_STORAGE_BASE_URL to point to such, or some other environment variable.

However we would still need to make access to api.shorebird.dev more reliable for Shorebird to work for Developers without them needing to modify their network.

Regardless, Shorebird works seems to work for some users in china (the api.shorebird.dev request sometimes fails, but if it succeeds patch installs otherwise seem to work fine in China). Developers who cannot reach api.shorebird.dev will still have trouble actually uploading the patches.

[xster]

I suspect it's not as simple as setting an alternate URL. The mirror URLs pointed to in https://docs.flutter.dev/community/china physically host the actual bytes to the Flutter engine etc. You'll need to find a partner (who can register a https://en.wikipedia.org/wiki/ICP_license) who can host the app bits being pushed to point the CLI to.

[AlexV525]

Hi guys, just noticed this issue when I was browsing new stuff. I can answer the questions above since I'm also a maintainer to flutter.cn .

I'm not sure if download.shorebird.dev is accessible from China?

Currently, it's available. If you can provide an exact artifact destination then we can test if the connection is stable enough (download.shorebird.dev returns Unrecognized artifact path).

I'm also not sure how we would tell download.shorebird.dev (possibly in the headers?) what proxy to proxy too?

About how our mirror works: We host our service with a local services provider, mirroring requests with the same path of googleapis, marking artifacts that need to be cached by CDN, and resolving them into buckets.

In order to make our mirror work with shorebird, we'll need another domain or subdomain for it, and create another service that is different from the Flutter one.

So to summarize this, I don't think we'll need a new mirror service for now since everything is working as expected. Let me know if I missed something.

[eseidel]

Thanks Alex. That sounds like an excellent option. I don't think we're quite ready to set this up yet, but if you reach out via email eric@, happy to discuss further when we get there!

[liu731]

another way is to support private deployment.

[eseidel]

We also just got a report that Google OAuth may not work in China (may not connect to oauth2.googleapis.com). So we likely need to support #335 too.

Recommended alternatives included WeChat or QQ OAuth.
Also just having Shorebird support its own accounts (e.g. with email auth).

[956569686]

When will this problem be solved?

[eseidel]

We don't have a specific date. We're quite busy with other customers at the moment and have not yet sat down to try and understand what work remains to be done for Shorebird to be usable within China.

[956569686]

I hope the team can support Chinese developers as soon as possible. A large number of Chinese flutter developers hope to use this tool.

[ZhouC125]

Hoping for Shorebird support in China soon!

[1leontao]

Hoping for Shorebird support in China soon!

[li8607]

Hoping for Shorebird support in China soon!

[majiankang]

the directions for using Shorebird in china include setting FLUTTER_STORAGE_BASE_URL: https://docs.flutter.dev/community/china

Which I believe will not work with our current setup.

I'm not entirely sure how we will make it work as:

1. I'm not sure if download.shorebird.dev is accessible from China?
2. I'm also not sure how we would tell download.shorebird.dev (possibly in the headers?) what proxy to proxy too?

Or maybe we'd work with a local china proxy to have both the shorebird and Flutter artifacts, once #434 is fixed?

This could also go away if we fix: #434. As it's possible the way we hook in does not involve setting FLUTTER_STORAGE_BASE_URL but rather modifying flutter to teach it about our additional artifact type.

If someone in Chinese mainland can't login with 'shorebird login', maybe you need set proxy to your terminal, just input follow command(if you use clash in windows os):
set http_proxy=http://127.0.0.1:7890/
set https_proxy=https://127.0.0.1:7890/
shorbird login

筒子们如果发现开了全局代理之后，用shorebird login命令也登陆不了的话，试下给终端把代理设置上再去登陆！

[956569686]

set http_proxy=http://127.0.0.1:7890/
set https_proxy=https://127.0.0.1:7890/
这种方式可行吗

[okFancy]

Hello, Chinese developers hope that Shorebird can support usage in China.

[franticn]

I think you can solve this problem by mirroring. The most common use in China is TUNA, Tsinghua's own site mirroring a wide range of open source software, These include FLUTTER_STORAGE_BASE_URL and PUB_HOSTED_URL, which can be reached at support@tuna.tsinghua.edu.cn

export FLUTTER_STORAGE_BASE_URL="https://mirrors.tuna.tsinghua.edu.cn/flutter"
export PUB_HOSTED_URL="https://mirrors.tuna.tsinghua.edu.cn/dart-pub"

• https://mirrors.tuna.tsinghua.edu.cn/
• https://mirrors.tuna.tsinghua.edu.cn/help/flutter/

[dongjianxiong]

Hoping for Shorebird support in China soon! Thank you very much!

[eseidel]

We have quite a few customers from China. So this appears to work "well enough" for the moment. I think when we moved our distribution to a CDN that will help customers in china (since their users will get updates more reliably).

[Heilum]

A simple solution: just adding a optional http-proxy-string parameter to ShorebirdCodePush's constructor and let developers from China to use shorebird_code_push package and manually do code-pushing!

[eseidel]

Update: We've updated our infrastructure to use CloudFlare CDN, which as far as we can tell is more reliably reachable across China:
https://shorebird.dev/blog/improved-patch-delivery/

We still make direct connections to Google Cloud as part of using shorebird on a developer machine. So developers in parts of the world that do not have reliable access to Google Cloud will still need to use a VPN or similar access mechanism, but users should now have more reliable access to download patches published with Shorebird.