Add standalone CLI #14270

philipp-spiess · 2024-08-27T11:50:18Z

This PR adds a new standalone client: A single-binary file that you can use to run Tailwind v4 without having a node setup. To make this work we use Bun's single-binary build which can properly package up native modules and the bun runtime for us so we do not have to rely on any expand-into-tmp-folder-at-runtime workarounds.

When running locally, pnpm build will now standalone artifacts inside packages/@tailwindcss-standalone/dist. Note that since we do not build Oxide for other environments in the local setup, you won't be able to use the standalone artifacts for other platforms in local dev mode.

Unfortunately Bun does not have support for Windows ARM builds yet but we found that using the bun-baseline runtime for Windows x64 would make the builds work fine in ARM emulation mode:

Some Bun related issues we faced and worked around:

We found that the regular Windows x64 build of bun does not run on Windows ARM via emulation. Instead, we have to use the bun-baseline builds which emulate correctly.
When we tried to bundle artifacts with embed directories, node binary dependencies were no longer resolved correctly even though they would still be bundled and accessible within the embeddedFiles list. We worked around this by using the import * as from ... with { type: "file" }; and patching the resolver we use in our CLI.
If you have an import to a module that is used as a regular import and a with { type: "file" }, it will either return the module in both cases or the file path when we would expect only the with { type: "file" } import to return the path. We do read the Tailwind CSS version via the file system and require.resolve() in the CLI and via import * from './package.json' in core and had to work around this by patching the version resolution in our CLI.
```
import packageJson from "./package.json"
import packageJsonPath from "./package.json" with {type: "file"}

// We do not expect these to be equal
packageJson === packageJsonPath 
```
We can not customize the app icon used for Windows .exe builds without decompiling the binary. For now we will leave the default but one workaround is to use tools like ResourceHacker to decompile the binary first.

patches/@parcel__watcher@2.4.1.patch

.github/workflows/release.yml

scripts/pack-packages.mjs

Bun added support for referencing embedded files with require.resolve passing in a relative path

…ng version at runtime

This reverts commit e3bdc4c.

…ion mode

packages/tailwindcss/src/index.ts

packages/@tailwindcss-standalone/scripts/build.ts

Jarred-Sumner · 2024-08-30T03:27:53Z

Very exciting - and thanks for the feedback.

We found that the regular Windows x64 build of bun does not run on Windows ARM via emulation. Instead, we have to use the bun-baseline builds which emulate correctly.

This is unfortunately expected, and we should just document it. arm64 emulation (including Rosetta) tends not to support SIMD instructions

If you have an import to a module that is used as a regular import and a with { type: "file" }, it will either return the module in both cases or the file path when we would expect only the with { type: "file" } import to return the path. We do read the Tailwind CSS version via the file system and require.resolve() in the CLI and via import * from './package.json' in core and had to work around this by patching the version resolution in our CLI.

Currently, we treat file paths as the key in some hash tables, but it really should be file path and loader so that this works as expected

Internally, bun build --compile does try to make fs.readFile and Bun.file(path).{text,arrayBuffer,json,bytes} support embedded files without extra code changes, but I don't think we're doing a good enough job of this yet

When we tried to bundle artifacts with embed directories, node binary dependencies were no longer resolved correctly even though they would still be bundled and accessible within the embeddedFiles list. We worked around this by using the import * as from ... with { type: "file" }; and patching the resolver we use in our CLI.

This is a bug. It's probably using the "file" loader instead of our "napi" loader. Would you be able to file an issue?

We can not customize the app icon used for Windows .exe builds without decompiling the binary. For now we will leave the default but one workaround is to use tools like ResourceHacker to decompile the binary first.

We can add a --icon CLI flag or similar.

Jarred-Sumner · 2024-08-30T03:29:06Z

packages/@tailwindcss-cli/src/commands/build/index.ts

+          // with bun, some of the ids might resolve to files inside the bun
+          // embedded files root which can only be read by `node:fs` and not
+          // `node:fs/promises`.
+          return readFileSync(id, 'utf-8')


This seems like a bug in Bun.

Jarred-Sumner · 2024-08-30T03:30:57Z

packages/@tailwindcss-standalone/scripts/build.ts

+
+async function buildForPlatform(triple: string, outfile: string) {
+  // We wrap this in a retry because occasionally the atomic rename fails for some reason
+  for (let i = 0; i < 5; ++i) {


Definitely a bug in Bun

Is this happening on all platforms or on a specific platform?

When I ran into this it was on macOS. Not sure if it applies to other platforms. It happened enough that I added a loop for it but it was very inconsistently reproducible.

Jarred-Sumner · 2024-08-30T03:31:38Z

packages/@tailwindcss-standalone/scripts/build.ts

+
+  await buildForPlatform(triple, outfile)
+
+  await new Promise((resolve) => setTimeout(resolve, 100))


Was this delay a workaround of a bug?

It was because the readFile that follows would occasionally fail unless we waited a little bit. Perhaps it's like a filesystem buffer syncing thing?

I need to give it another test to see if it's still a problem.

Jarred-Sumner · 2024-08-30T03:33:45Z

packages/@tailwindcss-cli/src/utils/resolve.ts

+  return localResolve(id)
+}
+
+const resolver = EnhancedResolve.ResolverFactory.createResolver({


It's probably not feature-ful enough, but you might be able to instead use Bun.resolveSync here. Bun.resolveSync supports a 2nd argument for the path.

You can see some tests for it here:
https://github.com/oven-sh/bun/blob/6603133732892227f0a32e3f19b343fd1e59a604/test/js/bun/resolve/import-meta.test.js#L133-L154

Unfortunately I don't think we can because we need support for the style field and style conditions in exports. Good to know about this though — perhaps it could be usable elsewhere.

thecrypticace · 2024-08-30T10:39:22Z

This is unfortunately expected, and we should just document it. arm64 emulation (including Rosetta) tends not to support SIMD instructions

FWIW, Rosetta 2 translates SSE2 instructions in Sonoma at least. In Sequoia it'll also support translation of AVX2 instructions (I assume that includes normal AVX as well). Translation of AVX512 is not supported tho.

philipp-spiess · 2024-09-02T11:16:18Z

Thanks a bunch @Jarred-Sumner, this is super helpful! I created issues with repros for the bugs and one issue for the --icon feature request :)

Requiering a node binary from a single-file executable with embed directories fails oven-sh/bun#13684
Embedded files can only be read with synchronous fs APIs oven-sh/bun#13685
Add --icon option to customize icon when building single-file executables for Windows oven-sh/bun#13686

RobinMalfait

This seems to work on my machine as expected. Only thing I want to figure out is to extend the integration tests to also test using the standalone CLI.

We currently only test on Linux and Windows, so we don't fully test every single OS that we generate binaries for. But if those two are tested via integration tests already that would be sick.

This reverts commit b9d5666.

Since we are mutating the `family` variable. Nothing to see here.

Should all be fine now 🤞

philipp-spiess commented Aug 27, 2024

View reviewed changes

patches/@parcel__watcher@2.4.1.patch Outdated Show resolved Hide resolved

philipp-spiess changed the title ~~Feat/v4 standalone cli workaround~~ Add standalone CLI Aug 27, 2024

philipp-spiess force-pushed the feat/v4-standalone-cli-workaround branch 2 times, most recently from 5e567f6 to 2ac8a71 Compare August 29, 2024 11:22

philipp-spiess commented Aug 29, 2024

View reviewed changes

.github/workflows/release.yml Show resolved Hide resolved

.github/workflows/release.yml Show resolved Hide resolved

scripts/pack-packages.mjs Show resolved Hide resolved

philipp-spiess marked this pull request as ready for review August 29, 2024 12:05

philipp-spiess force-pushed the feat/v4-standalone-cli-workaround branch from ceed332 to 8db7e61 Compare August 29, 2024 12:42

thecrypticace and others added 15 commits August 29, 2024 14:54

Add preliminary standalone CLI experiment

01378bb

wip

2b48a39

Attempt resolving embedded files with a relative path

b65ff4a

Bun added support for referencing embedded files with require.resolve passing in a relative path

Add commented out test case for embedded files bug

19948a0

Update lockfile

5556bcb

Work around build issues by manually embedding .css files and injecti…

3699c6f

…ng version at runtime

Use postcss style module resolver for CSS imports

c76b552

Fix lint issues

bcd64c8

Use the enhanced-resolve over the resolve npm package

6e68282

Handle Windows bun embedded root dir

cafc5ba

Add resource hacker script to replace .exe icons

698d9c4

Revert "Add resource hacker script to replace .exe icons"

22b30e6

This reverts commit e3bdc4c.

Update release workflow to upload standalone artifacts

3a38513

Use bun-baseline for Windows x64 builds to make it work in ARM emulat…

3427631

…ion mode

Add changelog

29125f0

philipp-spiess force-pushed the feat/v4-standalone-cli-workaround branch from 8db7e61 to 29125f0 Compare August 29, 2024 12:55

philipp-spiess requested a review from adamwathan August 29, 2024 12:57

thecrypticace approved these changes Aug 29, 2024

View reviewed changes

thecrypticace reviewed Aug 29, 2024

View reviewed changes

packages/tailwindcss/src/index.ts Outdated Show resolved Hide resolved

philipp-spiess added 2 commits August 29, 2024 16:27

Remove unnecessary async function

9083f07

Explain why we use baseline

21e335f

thecrypticace reviewed Aug 29, 2024

View reviewed changes

packages/@tailwindcss-standalone/scripts/build.ts Outdated Show resolved Hide resolved

Fix typo

2c28f64

Jarred-Sumner reviewed Aug 30, 2024

View reviewed changes

philipp-spiess requested a review from RobinMalfait September 2, 2024 10:13

RobinMalfait reviewed Sep 2, 2024

View reviewed changes

RobinMalfait added 13 commits September 2, 2024 13:54

add integration test for standalone CLI

4843b5e

tmp: check ldd bun

cf02819

add macos-14 as one of the OS's we test on

51d1f2c

tmp: add debug info to @parcel/watcher patch

b9d5666

remove ldd debug info

0257a7d

Revert "tmp: add debug info to @parcel/watcher patch"

f9c3abf

This reverts commit b9d5666.

tmp: add patch debug info

3a5dfbd

make detect-libc an explicit dependency

722e3e3

fix detect-libc version mismatches due to Bun polyfills

861581b

use let instead of const

03752f7

Since we are mutating the `family` variable. Nothing to see here.

tmp: add debug info

67e132b

make sure we destructure familySync

f32e296

remove debug logs

114dfeb

Should all be fine now 🤞

RobinMalfait approved these changes Sep 2, 2024

View reviewed changes

philipp-spiess merged commit d9e3fd6 into next Sep 2, 2024
3 checks passed

philipp-spiess deleted the feat/v4-standalone-cli-workaround branch September 2, 2024 13:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add standalone CLI #14270

Add standalone CLI #14270

philipp-spiess commented Aug 27, 2024 •

edited

Loading

Jarred-Sumner commented Aug 30, 2024

Jarred-Sumner Aug 30, 2024

Jarred-Sumner Aug 30, 2024

thecrypticace Aug 30, 2024

Jarred-Sumner Aug 30, 2024

thecrypticace Aug 30, 2024 •

edited

Loading

Jarred-Sumner Aug 30, 2024

thecrypticace Aug 30, 2024 •

edited

Loading

thecrypticace commented Aug 30, 2024

philipp-spiess commented Sep 2, 2024

RobinMalfait left a comment


		await buildForPlatform(triple, outfile)

		await new Promise((resolve) => setTimeout(resolve, 100))

Add standalone CLI #14270

Add standalone CLI #14270

Conversation

philipp-spiess commented Aug 27, 2024 • edited Loading

Jarred-Sumner commented Aug 30, 2024

Jarred-Sumner Aug 30, 2024

Choose a reason for hiding this comment

Jarred-Sumner Aug 30, 2024

Choose a reason for hiding this comment

thecrypticace Aug 30, 2024

Choose a reason for hiding this comment

Jarred-Sumner Aug 30, 2024

Choose a reason for hiding this comment

thecrypticace Aug 30, 2024 • edited Loading

Choose a reason for hiding this comment

Jarred-Sumner Aug 30, 2024

Choose a reason for hiding this comment

thecrypticace Aug 30, 2024 • edited Loading

Choose a reason for hiding this comment

thecrypticace commented Aug 30, 2024

philipp-spiess commented Sep 2, 2024

RobinMalfait left a comment

Choose a reason for hiding this comment

philipp-spiess commented Aug 27, 2024 •

edited

Loading

thecrypticace Aug 30, 2024 •

edited

Loading

thecrypticace Aug 30, 2024 •

edited

Loading