HTTP Errors when creating some pins

[boehs]

• https://linear.app/blog/scaling-the-linear-sync-engine
• https://stickerninja.com/blog/some-days-were-sold-out/

Causes HTTP error when creating new pins. Both for the metadata endpoint and the submission endpoint

{
    "success": false,
    "message": "Cannot convert object to primitive value",
    "data": null,
    "status": "Internal Server Error"
}

For submission, we see

  "logs": [
    {
      "message": [
        "\u001b[2m241ms: begin\u001b[22m"
      ],
      "level": "log",
      "timestamp": 1721135008152
    },
    {
      "message": [
        "\u001b[2m268ms: select * from \"entities\" where \"url\" = $1\u001b[22m"
      ],
      "level": "log",
      "timestamp": 1721135008420
    },
    {
      "message": [
        "\u001b[2m132ms: rollback\u001b[22m"
      ],
      "level": "log",
      "timestamp": 1721135008645
    },
    {
      "message": [
        "TypeError: Cannot convert object to primitive value"
      ],
      "level": "log",
      "timestamp": 1721135008645
    }

For metadata, we see

    {
      "message": [
        "TypeError: Cannot convert object to primitive value"
      ],
      "level": "log",
      "timestamp": 1721135192413
    }

[linear]

PFW-15 HTTP Errors when creating some pins

[boehs]

might not be my error

[boehs]

error is inside the psl package

[boehs]

Postmortem

The cause

arises from the deprecated punycode module (lupomontero/psl#323 lupomontero/psl#315) which has caused issues before.

see this thread in cf discord for same error being caused by punycode

experienced issues in aws-sdk related to this asw, see cfdsc and the following issues that are all intrinsically related, which is why we switched to aws4fetch

• Preset cloudflare-pages/cloudflare-module not working with @aws-sdk  nitrojs/nitro#1882 (comment)
• 🐛 BUG: "Cannot convert object to primitive value" at publish time cloudflare/workers-sdk#2081
• Preset cloudflare-pages/cloudflare-module not working with @aws-sdk  nitrojs/nitro#1882 (comment) <-- punycode

The solution

We will follow how others did, see

• punycode deprecation message displayed with Node.js v21.0.0 salesforce/tough-cookie#324
• Replace psl dependency with tldts salesforce/tough-cookie#338
• Replace psl with tldts salesforce/tough-cookie#346

issue will be closed when fix is applied.

[boehs]

Lessons

punycode causes issues on cf runtime. Three different runtimes we use

• production @ cf import.meta.env.MODE == "development"
• node.js based dev import.meta.env.MODE == "production"

the third is the problem child: miniflare with workerd. To use miniflare, you build the application the same way but avoid the deployment step. This means env is set to production, but this isn't production, causing

1. Endpoints to be incorrectly mapped to prod instead of localhost
2. Promise races due to a DB cache which doesn't work in workerd (maybe only with hyperdrive?)

3. I believe there are currently permission issues with the connection string for the local db replica in this mode. I have not diagnosed the causation.

Short term solution was to create a mode constant that we can just edit and then REMEMBER TO REMOVE WHEN COMMITTING (bedba32). long term solution will be harder: nksaraf/vinxi#304, somehow we need full builds to still happen with mode set to development