Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve JS Sourcemaps #674

Merged
merged 6 commits into from
Dec 20, 2022
Merged

Improve JS Sourcemaps #674

merged 6 commits into from
Dec 20, 2022

Conversation

natemoo-re
Copy link
Member

Changes

  • Vastly improves our JavaScript sourcemaps
  • Previously, frontmatter was chopped up in a few different places and we were discarding location information
  • Now, we are tracking the location of all hoisted imports, exports, and other content when restructuring frontmatter. This allows us to properly construct a sourcemap while printing.

Testing

Tested manually using https://evanw.github.io/source-map-visualization/

CleanShot.2022-12-19.at.14.08.16.mp4

Docs

N/A

@changeset-bot
Copy link

changeset-bot bot commented Dec 19, 2022

🦋 Changeset detected

Latest commit: 86bd203

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@astrojs/compiler Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@natemoo-re natemoo-re self-assigned this Dec 19, 2022
Comment on lines 15 to +19
type HoistedScripts struct {
Hoisted [][]byte
Body []byte
Hoisted [][]byte
HoistedLocs []loc.Loc
Body [][]byte
BodyLocs []loc.Loc
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously we were throwing away all location info when hoisting imports/exports, which was obviously unhelpful!

Comment on lines +162 to +190
exports := make([][]byte, 0)
exportLocs := make([]loc.Loc, 0)
bodies := make([][]byte, 0)
bodiesLocs := make([]loc.Loc, 0)

if len(render.Body) > 0 {
for i, innerBody := range render.Body {
innerStart := render.BodyLocs[i].Start
if len(bytes.TrimSpace(innerBody)) == 0 {
continue
}

// Extract exports
preprocessed := js_scanner.HoistExports(append(innerBody, '\n'))
if len(preprocessed.Hoisted) > 0 {
for j, exported := range preprocessed.Hoisted {
exportedLoc := preprocessed.HoistedLocs[j]
exportLocs = append(exportLocs, loc.Loc{Start: start + innerStart + exportedLoc.Start})
exports = append(exports, exported)
}
}

if len(preprocessed.Body) > 0 {
for j, body := range preprocessed.Body {
bodyLoc := preprocessed.BodyLocs[j]
bodiesLocs = append(bodiesLocs, loc.Loc{Start: start + innerStart + bodyLoc.Start})
bodies = append(bodies, body)
}
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is interleaving import hoisting with export hoisting, which gets kinda hard to follow, but the main point is that we're preserving the location of each statement.

if len(bytes.TrimSpace(body)) == 0 {
continue
}
p.printTextWithSourcemap(string(body), bodyLoc)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now we can print these with intact location info!

* Add internal/js_scanner/js_scanner_test.go#FuzzHoistImport

Runnable via `go test ./internal/js_scanner -fuzz=FuzzHoistImports`

* Update internal/js_scanner/js_scanner_test.go

Co-authored-by: Nate Moore <natemoo-re@users.noreply.github.com>

* fix(js_scanner): assert that i < len(source) when scanning

Co-authored-by: Nate Moore <natemoo-re@users.noreply.github.com>
Co-authored-by: Nate Moore <nate@astro.build>
@natemoo-re natemoo-re merged commit 20497f4 into main Dec 20, 2022
@natemoo-re natemoo-re deleted the fix/js-sourcemaps branch December 20, 2022 22:28
@jasikpark
Copy link
Contributor

woot 🥳

natemoo-re added a commit that referenced this pull request Dec 20, 2022
@natemoo-re natemoo-re mentioned this pull request Dec 20, 2022
natemoo-re added a commit that referenced this pull request Dec 20, 2022
* Revert "Improve JS Sourcemaps (#674)"

This reverts commit 20497f4.

* chore: add changeset

Co-authored-by: Nate Moore <nate@astro.build>
@github-actions github-actions bot mentioned this pull request Dec 20, 2022
natemoo-re added a commit that referenced this pull request Dec 20, 2022
This reverts commit fd5cb57.
natemoo-re added a commit that referenced this pull request Jan 4, 2023
* Revert "Rollback #674 (#678)"

This reverts commit fd5cb57.

* fix: always emit newline after JS exports

* chore: add changeset

Co-authored-by: Nate Moore <nate@astro.build>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants