Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removing shim from D1 #628

Merged
merged 1 commit into from
Jul 18, 2023
Merged

Removing shim from D1 #628

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions packages/miniflare/src/plugins/d1/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,27 @@ export const D1_PLUGIN: Plugin<
sharedOptions: D1SharedOptionsSchema,
getBindings(options) {
const databases = namespaceEntries(options.d1Databases);
return databases.map<Worker_Binding>(([name, id]) => ({
name,
service: { name: `${SERVICE_DATABASE_PREFIX}:${id}` },
}));
return databases.map<Worker_Binding>(([name, id]) => {
const binding = name.startsWith("__D1_BETA__")
? // Used before Wrangler 3.3
{
service: { name: `${SERVICE_DATABASE_PREFIX}:${id}` },
}
: // Used after Wrangler 3.3
{
wrapped: {
moduleName: "cloudflare-internal:d1-api",
innerBindings: [
{
name: "fetcher",
service: { name: `${SERVICE_DATABASE_PREFIX}:${id}` },
},
],
},
};

return { name, ...binding };
});
},
getServices({ options, sharedOptions }) {
const persist = sharedOptions.d1Persist;
Expand Down
2 changes: 1 addition & 1 deletion packages/miniflare/src/runtime/config/workerd.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ export type Worker_Binding_CryptoKey = (
export interface Worker_Binding_WrappedBinding {
moduleName?: string;
entrypoint?: string;
innerBinding?: Worker_Binding[];
innerBindings?: Worker_Binding[];
}

export type Worker_Binding_CryptoKey_Algorithm =
Expand Down
8 changes: 4 additions & 4 deletions packages/miniflare/test/fixtures/d1/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# D1 Worker Fixture
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only reason we had this fixture was to get the D1 shim from Wrangler for testing. miniflareTest() (see test/plugins/d1/index.spec.ts) can accept a request handler function like:

miniflare/packages/miniflare/test/plugins/cache/index.spec.ts

Lines 13 to 58 in c0f32f3

const test = miniflareTest({}, async (global, req) => {
// Partition headers
let name: string | undefined;
let cfCacheKey: string | undefined;
let bufferPut = false;
const reqHeaders = new global.Headers();
const resHeaders = new global.Headers();
for (const [key, value] of req.headers) {
const lowerKey = key.toLowerCase();
if (lowerKey === "test-cache-name") {
name = value;
} else if (lowerKey === "test-cf-cache-key") {
cfCacheKey = value;
} else if (lowerKey === "test-buffer") {
bufferPut = true;
} else if (lowerKey.startsWith("test-response-")) {
resHeaders.set(lowerKey.substring("test-response-".length), value);
} else {
reqHeaders.set(lowerKey, value);
}
}
// Get cache and cache key
const cache =
name === undefined ? global.caches.default : await global.caches.open(name);
const key = new global.Request(req.url, {
headers: reqHeaders,
cf: cfCacheKey === undefined ? undefined : { cacheKey: cfCacheKey },
});
// Perform cache operation
if (req.method === "GET") {
const cachedRes = await cache.match(key);
return cachedRes ?? new global.Response("<miss>", { status: 404 });
} else if (req.method === "PUT") {
const body = bufferPut ? await req.arrayBuffer() : req.body;
const res = new global.Response(body, { headers: resHeaders });
await cache.put(key, res);
return new global.Response(null, { status: 204 });
} else if (req.method === "DELETE") {
const deleted = await cache.delete(key);
return new global.Response(null, { status: deleted ? 204 : 404 });
} else {
return new global.Response(null, { status: 405 });
}
});

This handler gets wrapped with all the MF-Experimental-Error-Stack stuff automatically too. Could tidy this up in a future PR though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up just reusing the existing test file, before Wrangler had compiled it. What do you think?


Used by `test/plugins/d1/index.spec.ts`. Rebuild `worker.dist.mjs` by running
the following in the `packages/tre` directory:
the following in the `packages/miniflare` directory:

```shell
$ npx wrangler publish --config test/fixtures/d1/wrangler.toml --dry-run --outdir dist
$ mv test/fixtures/d1/dist/d1-beta-facade.entry.js test/fixtures/d1/worker.dist.mjs
$ npx wrangler@3.2.0 publish --config test/fixtures/d1/wrangler.toml --dry-run --outdir dist
$ mv test/fixtures/d1/dist/worker.js test/fixtures/d1/worker.dist.mjs
```

`wrangler` isn't included as a dependency as that would likely cause issues with
`wrangler@3.2.0` isn't included as a dependency as it predates the moving of D1 out of Wrangler and into the runtime, which would have caused issues with
different versions of Miniflare being installed.
197 changes: 156 additions & 41 deletions packages/miniflare/test/fixtures/d1/worker.dist.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,25 @@
// ../../../../../../../../.npm/_npx/d6768b39ab4bfa9b/node_modules/wrangler/templates/middleware/common.ts
var __facade_middleware__ = [];
function __facade_register__(...args) {
__facade_middleware__.push(...args.flat());
}
function __facade_invokeChain__(request, env, ctx, dispatch, middlewareChain) {
const [head, ...tail] = middlewareChain;
const middlewareCtx = {
dispatch,
next(newRequest, newEnv) {
return __facade_invokeChain__(newRequest, newEnv, ctx, dispatch, tail);
}
};
return head(request, env, ctx, middlewareCtx);
}
function __facade_invoke__(request, env, ctx, dispatch, finalMiddleware) {
return __facade_invokeChain__(request, env, ctx, dispatch, [
...__facade_middleware__,
finalMiddleware
]);
}

// worker.mjs
async function requestJson(request) {
const text = await request.text();
Expand Down Expand Up @@ -69,8 +91,10 @@ var worker = {
};
var worker_default = worker;

// ../../../../../../../../../private/var/folders/jl/n6qj9gxn54b1yh4mqyxj1wz00000gp/T/tmp-20982-osS2JN1Tqwnq/d1-beta-facade.entry.js
var define_D1_IMPORTS_default = ["__D1_BETA__DB"];
// wrangler-config:config:middleware/d1-beta
var D1_IMPORTS = ["__D1_BETA__DB"];

// ../../../../../../../../.npm/_npx/d6768b39ab4bfa9b/node_modules/wrangler/templates/middleware/middleware-d1-beta.ts
var D1Database = class {
constructor(binding) {
this.binding = binding;
Expand All @@ -88,11 +112,11 @@ var D1Database = class {
if (response.status !== 200) {
try {
const err = await response.json();
throw new Error("D1_DUMP_ERROR", {
throw new Error(`D1_DUMP_ERROR: ${err.error}`, {
cause: new Error(err.error)
});
} catch (e) {
throw new Error("D1_DUMP_ERROR", {
throw new Error(`D1_DUMP_ERROR: Status + ${response.status}`, {
cause: new Error("Status " + response.status)
});
}
Expand All @@ -115,11 +139,14 @@ var D1Database = class {
return r.error ? 1 : 0;
}).indexOf(1);
if (error !== -1) {
throw new Error("D1_EXEC_ERROR", {
cause: new Error(
"Error in line " + (error + 1) + ": " + lines[error] + ": " + exec[error].error
)
});
throw new Error(
`D1_EXEC_ERROR: Error in line ${error + 1}: ${lines[error]}: ${exec[error].error}`,
{
cause: new Error(
"Error in line " + (error + 1) + ": " + lines[error] + ": " + exec[error].error
)
}
);
} else {
return {
count: exec.length,
Expand Down Expand Up @@ -149,50 +176,57 @@ var D1Database = class {
const answer = await response.json();
if (answer.error && dothrow) {
const err = answer;
throw new Error("D1_ERROR", { cause: new Error(err.error) });
throw new Error(`D1_ERROR: ${err.error}`, {
cause: new Error(err.error)
});
} else {
return Array.isArray(answer) ? answer.map((r) => mapD1Result(r)) : mapD1Result(answer);
}
} catch (e) {
throw new Error("D1_ERROR", {
cause: new Error(e.cause || "Something went wrong")
const error = e;
throw new Error(`D1_ERROR: ${error.cause || "Something went wrong"}`, {
cause: new Error(`${error.cause}` || "Something went wrong")
});
}
}
};
var D1PreparedStatement = class {
constructor(database, statement, values) {
constructor(database, statement, params = []) {
this.database = database;
this.statement = statement;
this.params = values || [];
this.params = params;
}
bind(...values) {
for (var r in values) {
switch (typeof values[r]) {
const value = values[r];
switch (typeof value) {
case "number":
case "string":
break;
case "object":
if (values[r] == null)
if (value == null)
break;
if (Array.isArray(values[r]) && values[r].map((b) => {
if (Array.isArray(value) && value.map((b) => {
return typeof b == "number" && b >= 0 && b < 256 ? 1 : 0;
}).indexOf(0) == -1)
break;
if (values[r] instanceof ArrayBuffer) {
values[r] = Array.from(new Uint8Array(values[r]));
if (value instanceof ArrayBuffer) {
values[r] = Array.from(new Uint8Array(value));
break;
}
if (ArrayBuffer.isView(values[r])) {
values[r] = Array.from(values[r]);
if (ArrayBuffer.isView(value)) {
values[r] = Array.from(new Uint8Array(value.buffer));
break;
}
default:
throw new Error("D1_TYPE_ERROR", {
cause: new Error(
"Type '" + typeof values[r] + "' not supported for value '" + values[r] + "'"
)
});
throw new Error(
`D1_TYPE_ERROR: Type '${typeof value}' not supported for value '${value}'`,
{
cause: new Error(
`Type '${typeof value}' not supported for value '${value}'`
)
}
);
}
}
return new D1PreparedStatement(this.database, this.statement, values);
Expand All @@ -204,7 +238,7 @@ var D1PreparedStatement = class {
const results = info.results;
if (colName !== void 0) {
if (results.length > 0 && results[0][colName] === void 0) {
throw new Error("D1_COLUMN_NOTFOUND", {
throw new Error(`D1_COLUMN_NOTFOUND: Column not found (${colName})`, {
cause: new Error("Column not found")
});
}
Expand Down Expand Up @@ -249,8 +283,6 @@ function mapD1Result(result) {
result.error && (map.error = result.error);
return map;
}
var D1_IMPORTS = define_D1_IMPORTS_default;
var LOCAL_MODE = false;
var D1_BETA_PREFIX = `__D1_BETA__`;
var envMap = /* @__PURE__ */ new Map();
function getMaskedEnv(env) {
Expand All @@ -262,29 +294,112 @@ function getMaskedEnv(env) {
).forEach((bindingName) => {
newEnv.delete(bindingName);
const newName = bindingName.slice(D1_BETA_PREFIX.length);
const newBinding = !LOCAL_MODE ? new D1Database(env[bindingName]) : env[bindingName];
const newBinding = new D1Database(env[bindingName]);
newEnv.set(newName, newBinding);
});
const newEnvObj = Object.fromEntries(newEnv.entries());
envMap.set(env, newEnvObj);
return newEnvObj;
}
var shim_default = {
function wrap(env) {
return getMaskedEnv(env);
}

// ../../../../../../../../../../private/var/folders/xn/jl0lmfkx5gd06w3_bl12w1l00000gp/T/tmp-33052-rnZHsh2hk8Bc/middleware-insertion-facade.js
var envWrappers = [wrap].filter(Boolean);
var facade = {
...worker_default,
async fetch(request, env, ctx) {
return worker_default.fetch(request, getMaskedEnv(env), ctx);
envWrappers,
middleware: [
void 0,
...worker_default.middleware ? worker_default.middleware : []
].filter(Boolean)
};
var middleware_insertion_facade_default = facade;

// ../../../../../../../../../../private/var/folders/xn/jl0lmfkx5gd06w3_bl12w1l00000gp/T/tmp-33052-rnZHsh2hk8Bc/middleware-loader.entry.ts
var __Facade_ScheduledController__ = class {
constructor(scheduledTime, cron, noRetry) {
this.scheduledTime = scheduledTime;
this.cron = cron;
this.#noRetry = noRetry;
}
#noRetry;
noRetry() {
if (!(this instanceof __Facade_ScheduledController__)) {
throw new TypeError("Illegal invocation");
}
this.#noRetry();
}
};
var __facade_modules_fetch__ = function(request, env, ctx) {
if (middleware_insertion_facade_default.fetch === void 0)
throw new Error("Handler does not export a fetch() function.");
return middleware_insertion_facade_default.fetch(request, env, ctx);
};
function getMaskedEnv2(rawEnv) {
let env = rawEnv;
if (middleware_insertion_facade_default.envWrappers && middleware_insertion_facade_default.envWrappers.length > 0) {
for (const wrapFn of middleware_insertion_facade_default.envWrappers) {
env = wrapFn(env);
}
}
return env;
}
var registeredMiddleware = false;
var facade2 = {
...middleware_insertion_facade_default.tail && {
tail: maskHandlerEnv(middleware_insertion_facade_default.tail)
},
async queue(batch, env, ctx) {
return worker_default.queue(batch, getMaskedEnv(env), ctx);
...middleware_insertion_facade_default.trace && {
trace: maskHandlerEnv(middleware_insertion_facade_default.trace)
},
async scheduled(controller, env, ctx) {
return worker_default.scheduled(controller, getMaskedEnv(env), ctx);
...middleware_insertion_facade_default.scheduled && {
scheduled: maskHandlerEnv(middleware_insertion_facade_default.scheduled)
},
async trace(traces, env, ctx) {
return worker_default.trace(traces, getMaskedEnv(env), ctx);
...middleware_insertion_facade_default.queue && {
queue: maskHandlerEnv(middleware_insertion_facade_default.queue)
},
...middleware_insertion_facade_default.test && {
test: maskHandlerEnv(middleware_insertion_facade_default.test)
},
fetch(request, rawEnv, ctx) {
const env = getMaskedEnv2(rawEnv);
if (middleware_insertion_facade_default.middleware && middleware_insertion_facade_default.middleware.length > 0) {
if (!registeredMiddleware) {
registeredMiddleware = true;
for (const middleware of middleware_insertion_facade_default.middleware) {
__facade_register__(middleware);
}
}
const __facade_modules_dispatch__ = function(type, init) {
if (type === "scheduled" && middleware_insertion_facade_default.scheduled !== void 0) {
const controller = new __Facade_ScheduledController__(
Date.now(),
init.cron ?? "",
() => {
}
);
return middleware_insertion_facade_default.scheduled(controller, env, ctx);
}
};
return __facade_invoke__(
request,
env,
ctx,
__facade_modules_dispatch__,
__facade_modules_fetch__
);
} else {
return __facade_modules_fetch__(request, env, ctx);
}
}
};
function maskHandlerEnv(handler) {
return (data, env, ctx) => handler(data, getMaskedEnv2(env), ctx);
}
var middleware_loader_entry_default = facade2;
export {
shim_default as default
middleware_loader_entry_default as default
};
//# sourceMappingURL=d1-beta-facade.entry.js.map
//# sourceMappingURL=worker.js.map
Loading