Merge pull request #341 from cloudflare/kenton/test-entrypoint

Add `workerd test` for running unit tests under workerd

README.md

@@ -46,7 +46,7 @@ As of this writing, some major features are missing which we intend to fix short
 * **Cron trigger** emulation is not supported yet. We need to figure out how, exactly, this should work in the first place. Typically if you have a cluster of machines, you only want a cron event to run on one of the machines, so some sort of coordination or external driver is needed.
 * **Parameterized workers** are not implemented yet. This is a new feature specified in the config schema, which doesn't have any precedent on Cloudflare.
 * **Devtools inspection** is not supported yet, but this should be straightforward to hook up.
-* **Tests** for most APIs are conspicuously missing. This is because the testing harness we have used for the past five years is deeply tied to the internal version of the codebase. We need to develop a new test harness for `workerd` and revise our API tests to use it. For the time being, we will be counting on the internal tests to catch bugs. We understand this is not ideal for external contributors trying to test their changes.
+* **Tests** for most APIs are conspicuously missing. This is because the testing harness we have used for the past five years is deeply tied to the internal version of the codebase. Ideally, we need to translate those tests into the new `workerd test` format and move them to this repo; this is an ongoing effort. For the time being, we will be counting on the internal tests to catch bugs. We understand this is not ideal for external contributors trying to test their changes.
 * **Documentation** is growing quickly but is definitely still a work in progress.
 
 ### WARNING: `workerd` is not a hardened sandbox

build/wd_test.bzl

@@ -0,0 +1,68 @@
+def wd_test(
+        src,
+        data = [],
+        name = None,
+        args = [],
+        **kwargs):
+    """Rule to define tests that run `workerd test` with a particular config.
+
+    Args:
+     src: A .capnp config file defining the test. (`name` will be derived from this if not
+        specified.) The extension `.wd-test` is also permitted instead of `.capnp`, in order to
+        avoid confusing other build systems that may assume a `.capnp` file should be complied.
+     data: Files which the .capnp config file may embed. Typically JavaScript files.
+     args: Additional arguments to pass to `workerd`. Typically used to pass `--experimental`.
+    """
+
+    # Add workerd binary to "data" dependencies.
+    data = data + [src, "//src/workerd/server:workerd"]
+
+    # Add initial arguments for `workerd test` command.
+    args = [
+        "$(location //src/workerd/server:workerd)",
+        "test",
+        "$(location {})".format(src),
+    ] + args
+
+    # Default name based on src.
+    if name == None:
+        name = src.removesuffix(".capnp").removesuffix(".wd-test")
+
+    _wd_test(
+        name = name,
+        data = data,
+        args = args,
+        **kwargs
+    )
+
+def _wd_test_impl(ctx):
+    # Bazel insists that the rule must actually create the executable that it intends to run; it
+    # can't just specify some other executable with some args. OK, fine, we'll use a script that
+    # just execs its args.
+    ctx.actions.write(
+        output = ctx.outputs.executable,
+        content = "#! /bin/sh\nexec \"$@\"\n",
+        is_executable = True,
+    )
+
+    return [
+        DefaultInfo(
+            executable = ctx.outputs.executable,
+            runfiles = ctx.runfiles(files = ctx.files.data)
+        ),
+    ]
+
+_wd_test = rule(
+    implementation = _wd_test_impl,
+    test = True,
+    attrs = {
+        "workerd": attr.label(
+            allow_single_file = True,
+            executable = True,
+            cfg = "exec",
+            default = "//src/workerd/server:workerd",
+        ),
+        "flags": attr.string_list(),
+        "data": attr.label_list(allow_files = True),
+    },
+)

samples/unit-tests/config.capnp

@@ -0,0 +1,35 @@
+# This is an example configuration for running unit tests.
+#
+# (If you're not already familiar with the basic config format, check out the hello world example
+# first.)
+#
+# Most projects will probably use some sort of script or tooling to auto-generate this
+# configuration (such as Wrangler), and instead focus on the contents of the JavaScript file.
+#
+# To run the tests, do:
+#
+#     workerd test config.capnp
+
+using Workerd = import "/workerd/workerd.capnp";
+
+const unitTests :Workerd.Config = (
+  services = [
+    # Define the service to be tested.
+    (name = "main", worker = .testWorker),
+
+    # Not required, but we can redefine the special "internet" service so that it disallows any
+    # outgoing connections. This prohibits the test from talking to the network.
+    (name = "internet", network = (allow = []))
+  ],
+
+  # For running tests, we do not need to define any sockets, since tests do not accept incoming
+  # connections.
+);
+
+const testWorker :Workerd.Worker = (
+  # Just a regular old worker definition.
+  modules = [
+    (name = "worker", esModule = embed "worker.js")
+  ],
+  compatibilityDate = "2023-01-15",
+);

samples/unit-tests/worker.js

@@ -0,0 +1,21 @@
+// Exporting a test is like exporting a `fetch()` handler.
+//
+// It's common to want to write multiple tests in one file. We can export each
+// test under a different name. (BTW you can also export multiple `fetch()`
+// handlers this way! But that's less commonly-used.)
+
+export let testStrings = {
+  async test(ctrl, env, ctx) {
+    if ("foo" + "bar" != "foobar") {
+      throw new Error("strings are broken!");
+    }
+  }
+};
+
+export let testMath = {
+  async test(ctrl, env, ctx) {
+    if (1 + 1 != 2) {
+      throw new Error("math is broken!");
+    }
+  }
+};

src/workerd/api/BUILD.bazel

@@ -1,5 +1,6 @@
 load("//:build/wd_cc_capnp_library.bzl", "wd_cc_capnp_library")
 load("//:build/kj_test.bzl", "kj_test")
+load("//:build/wd_test.bzl", "wd_test")
 
 filegroup(
     name = "srcs",
@@ -50,3 +51,10 @@ kj_test(
     src = "node/buffer-test.c++",
     deps = ["//src/workerd/tests:test-fixture"],
 )
+
+[wd_test(
+    src = f,
+    data = [f.removesuffix(".wd-test") + ".js"],
+) for f in glob(
+    ["**/*.wd-test"],
+)]

src/workerd/api/blob-test.js

@@ -0,0 +1,80 @@
+function assertEqual(a, b) {
+  if (a !== b) {
+    throw new Error(a + " !== " + b);
+  }
+}
+
+export default {
+  async test(ctrl, env, ctx) {
+    let blob = new Blob(["foo", new TextEncoder().encode("bar"), "baz"]);
+    assertEqual(await blob.text(), "foobarbaz");
+    assertEqual(new TextDecoder().decode(await blob.arrayBuffer()), "foobarbaz");
+    assertEqual(blob.type, "");
+
+    let blob2 = new Blob(["xx", blob, "yy", blob], {type: "application/whatever"});
+    assertEqual(await blob2.text(), "xxfoobarbazyyfoobarbaz");
+    assertEqual(blob2.type, "application/whatever");
+
+    let blob3 = new Blob();
+    assertEqual(await blob3.text(), "");
+
+    let slice = blob2.slice(5, 16);
+    assertEqual(await slice.text(), "barbazyyfoo");
+    assertEqual(slice.type, "");
+
+    let slice2 = slice.slice(-5, 1234, "type/type");
+    assertEqual(await slice2.text(), "yyfoo");
+    assertEqual(slice2.type, "type/type");
+
+    assertEqual(await blob2.slice(5).text(), "barbazyyfoobarbaz");
+    assertEqual(await blob2.slice().text(), "xxfoobarbazyyfoobarbaz");
+    assertEqual(await blob2.slice(3, 1).text(), "");
+
+    {
+      let stream = blob.stream();
+      let reader = stream.getReader();
+      let readResult = await reader.read();
+      assertEqual(readResult.done, false);
+      assertEqual(new TextDecoder().decode(readResult.value), "foobarbaz");
+      readResult = await reader.read();
+      assertEqual(readResult.value, undefined);
+      assertEqual(readResult.done, true);
+      reader.releaseLock();
+    }
+
+    let before = Date.now();
+
+    let file = new File([blob, "qux"], "filename.txt");
+    assertEqual(file instanceof Blob, true);
+    assertEqual(await file.text(), "foobarbazqux");
+    assertEqual(file.name, "filename.txt");
+    assertEqual(file.type, "");
+    if (file.lastModified < before || file.lastModified > Date.now()) {
+      throw new Error("incorrect lastModified");
+    }
+
+    let file2 = new File(["corge", file], "file2", {type: "text/foo", lastModified: 123});
+    assertEqual(await file2.text(), "corgefoobarbazqux");
+    assertEqual(file2.name, "file2");
+    assertEqual(file2.type, "text/foo");
+    assertEqual(file2.lastModified, 123);
+
+    try {
+      new Blob(["foo"], {endings: "native"});
+      throw new Error("use of 'endings' should throw");
+    } catch (err) {
+      if (!err.message.includes("The 'endings' field on 'Options' is not implemented.")) {
+        throw err;
+      }
+    }
+
+    // Test type normalization.
+    assertEqual(new Blob([], {type: "FoO/bAr"}).type, "foo/bar");
+    assertEqual(new Blob([], {type: "FoO\u0019/bAr"}).type, "");
+    assertEqual(new Blob([], {type: "FoO\u0020/bAr"}).type, "foo /bar");
+    assertEqual(new Blob([], {type: "FoO\u007e/bAr"}).type, "foo\u007e/bar");
+    assertEqual(new Blob([], {type: "FoO\u0080/bAr"}).type, "");
+    assertEqual(new File([], "foo.txt", {type: "FoO/bAr"}).type, "foo/bar");
+    assertEqual(blob2.slice(1, 2, "FoO/bAr").type, "foo/bar");
+  }
+}

src/workerd/api/blob-test.wd-test

@@ -0,0 +1,14 @@
+using Workerd = import "/workerd/workerd.capnp";
+
+const unitTests :Workerd.Config = (
+  services = [
+    ( name = "blob-test",
+      worker = (
+        modules = [
+          (name = "worker", esModule = embed "blob-test.js")
+        ],
+        compatibilityDate = "2023-01-15",
+      )
+    ),
+  ],
+);

src/workerd/api/global-scope.c++

@@ -439,6 +439,20 @@ kj::Promise<WorkerInterface::AlarmResult> ServiceWorkerGlobalScope::runAlarm(
   }
 }
 
+jsg::Promise<void> ServiceWorkerGlobalScope::test(
+    Worker::Lock& lock, kj::Maybe<ExportedHandler&> exportedHandler) {
+  // TODO(someday): For Service Workers syntax, do we want addEventListener("test")? Not supporting
+  //   it for now.
+  ExportedHandler& eh = JSG_REQUIRE_NONNULL(exportedHandler, Error,
+      "Tests are not currently supported with Service Workers syntax.");
+
+  auto& testHandler = JSG_REQUIRE_NONNULL(eh.test, Error,
+      "Entrypoint does not export a test() function.");
+
+  return testHandler(lock, jsg::alloc<TestController>(), eh.env.addRef(lock),
+                     eh.getCtx(lock.getIsolate()));
+}
+
 void ServiceWorkerGlobalScope::emitPromiseRejection(
     jsg::Lock& js,
     v8::PromiseRejectEvent event,

src/workerd/api/global-scope.h

@@ -103,6 +103,15 @@ class WorkerGlobalScope: public EventTarget {
   // (EventTarget's constructor confuses the hasConstructorMethod in resource.h)
 };
 
+class TestController: public jsg::Object {
+  // Controller type for test handler.
+  //
+  // At present, this has no methods. It is defined for consistency with other handlers and on the
+  // assumption that we'll probably want to put something here someday.
+public:
+  JSG_RESOURCE_TYPE(TestController) {}
+};
+
 class ExecutionContext: public jsg::Object {
 public:
   void waitUntil(kj::Promise<void> promise);
@@ -137,10 +146,15 @@ struct ExportedHandler {
   // Alarms are only exported on DOs, which receive env bindings from the constructor
   jsg::LenientOptional<jsg::Function<AlarmHandler>> alarm;
 
+  typedef jsg::Promise<void> TestHandler(
+      jsg::Ref<TestController> controller, jsg::Value env,
+      jsg::Optional<jsg::Ref<ExecutionContext>> ctx);
+  jsg::LenientOptional<jsg::Function<TestHandler>> test;
+
   jsg::SelfRef self;
   // Self-ref potentially allows extracting other custom handlers from the object.
 
-  JSG_STRUCT(fetch, trace, scheduled, alarm, self);
+  JSG_STRUCT(fetch, trace, scheduled, alarm, test, self);
 
   JSG_STRUCT_TS_ROOT();
   // ExportedHandler isn't included in the global scope, but we still want to
@@ -151,13 +165,15 @@ struct ExportedHandler {
     type ExportedHandlerTraceHandler<Env = unknown> = (traces: TraceItem[], env: Env, ctx: ExecutionContext) => void | Promise<void>;
     type ExportedHandlerScheduledHandler<Env = unknown> = (controller: ScheduledController, env: Env, ctx: ExecutionContext) => void | Promise<void>;
     type ExportedHandlerQueueHandler<Env = unknown, Message = unknown> = (batch: MessageBatch<Message>, env: Env, ctx: ExecutionContext) => void | Promise<void>;
+    type ExportedHandlerTestHandler<Env = unknown> = (controller: TestController, env: Env, ctx: ExecutionContext) => void | Promise<void>;
   );
   JSG_STRUCT_TS_OVERRIDE(<Env = unknown, QueueMessage = unknown> {
     fetch?: ExportedHandlerFetchHandler<Env>;
     trace?: ExportedHandlerTraceHandler<Env>;
     scheduled?: ExportedHandlerScheduledHandler<Env>;
     alarm: never;
     queue?: ExportedHandlerQueueHandler<Env, QueueMessage>;
+    test?: ExportedHandlerTestHandler<Env>;
   });
   // Make `env` parameter generic
 
@@ -217,6 +233,12 @@ class ServiceWorkerGlobalScope: public WorkerGlobalScope {
       Worker::Lock& lock, kj::Maybe<ExportedHandler&> exportedHandler);
   // Received runAlarm (called from C++, not JS).
 
+  jsg::Promise<void> test(
+      Worker::Lock& lock, kj::Maybe<ExportedHandler&> exportedHandler);
+  // Received test() (called from C++, not JS). See WorkerInterface::test(). This version returns
+  // a jsg::Promise<void>; it fails if an exception is thrown. WorkerEntrypoint will catch these
+  // and report them.
+
   void emitPromiseRejection(
       jsg::Lock& js,
       v8::PromiseRejectEvent event,
@@ -545,6 +567,7 @@ class ServiceWorkerGlobalScope: public WorkerGlobalScope {
 #define EW_GLOBAL_SCOPE_ISOLATE_TYPES                    \
   api::WorkerGlobalScope,                                \
   api::ServiceWorkerGlobalScope,                         \
+  api::TestController,                                   \
   api::ExecutionContext,                                 \
   api::ExportedHandler,                                  \
   api::ServiceWorkerGlobalScope::StructuredCloneOptions, \

src/workerd/io/io-context.h

@@ -234,7 +234,8 @@ class IoContext_IncomingRequest {
   // to be called after the client has received a response or disconnected.
   //
   // This method is also used by some custom event handlers (see WorkerInterface::CustomEvent) that
-  // need similar behavior.
+  // need similar behavior, as well as the test handler. TODO(cleanup): Rename to something more
+  // generic?
 
   RequestObserver& getMetrics() { return *metrics; }