Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fetching local file returns a scrambled output #8

Closed
lowlighter opened this issue May 1, 2021 · 0 comments · Fixed by #10
Closed

fetching local file returns a scrambled output #8

lowlighter opened this issue May 1, 2021 · 0 comments · Fixed by #10

Comments

@lowlighter
Copy link
Contributor

lowlighter commented May 1, 2021

Hello!

When I try to load a JSON file using file:// protocol content seems to get scrambled 😕

Here's a link to JSON file and below is a snippet which show that parsing with Deno.readTextFile is fine while using fetch output starts at line 4076 out of 4231 (which explains why it can't be parsed)

import { fetch } from "https://deno.land/x/file_fetch/mod.ts"
console.log(Deno.inspect(JSON.parse(await Deno.readTextFile("overworld.gracidea.json")), {depth:1}))
console.log("=======================")
console.log((await fetch(new URL("overworld.gracidea.json", import.meta.url)).then(res => res.text())).substring(0, 200))
console.log("=======================")
console.log(await fetch(new URL("overworld.gracidea.json", import.meta.url)).then(res => res.json()))
PS> deno run --allow-read test.ts
{ pins: [Object], areas: [Array], chunks: [Object] }
=======================
     "id": 215,
      "name": "3-safari-g-se",
      "points": [
        715,
        -1147,
        715,
        -1150,
        719,
        -1152,
        720,
        -1152,
        720,
        -1
=======================
error: Uncaught (in promise) SyntaxError: Unexpected token : in JSON at position 9
console.log(await fetch(new URL("overworld.gracidea.json", import.meta.url)).then(res => res.json()))
            ^
    at JSON.parse (<anonymous>)
    at packageData (deno:op_crates/fetch/22_body.js:247:21)
    at Response.json (deno:op_crates/fetch/22_body.js:192:18)

Any ideas?
Like denoland/deno#2150 (comment) mentioned, this lib is used on deno deploy so it's probably broken on it too


Edit: After more digging I think it's from deno's implementation of Response itself that the problem occurs. It is possible actually possible to reconstruct original string using directly TextDecoder and string. Tested both with Deno.iter and iter from std.

const file = await Deno.open(new URL("overworld.gracidea.json", import.meta.url), { read: true })
const original = await Deno.readTextFile("overworld.gracidea.json")
let copy = ""
const body = new ReadableStream<Uint8Array>({
  start: async (controller) => {
    for await (const chunk of Deno.iter(file)) {
      copy += new TextDecoder().decode(chunk)
    }
    file.close();
    controller.close();
  },
  cancel() {
    file.close();
  },
});
console.log((await new Response(body, {status: 200}).text()).substring(0, 200))
console.log(copy === original) //Output "true"
console.log(Deno.inspect(JSON.parse(copy), {depth:1})) //Output "{ pins: [Object], areas: [Array], chunks: [Object] }"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant