Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

expose transform and resolveapi from parcel #9193

Merged
merged 7 commits into from
Sep 8, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 86 additions & 1 deletion packages/core/core/src/Parcel.js
Original file line number Diff line number Diff line change
@@ -1,12 +1,17 @@
// @flow strict-local

import type {
Asset,
AsyncSubscription,
BuildEvent,
BuildSuccessEvent,
InitialParcelOptions,
PackagedBundle as IPackagedBundle,
ParcelTransformOptions,
ParcelResolveOptions,
ParcelResolveResult,
} from '@parcel/types';
import path from 'path';
import type {ParcelOptions} from './types';
// eslint-disable-next-line no-unused-vars
import type {FarmOptions, SharedReference} from '@parcel/workers';
Expand Down Expand Up @@ -37,10 +42,18 @@ import RequestTracker, {
} from './RequestTracker';
import createValidationRequest from './requests/ValidationRequest';
import createParcelBuildRequest from './requests/ParcelBuildRequest';
import createAssetRequest from './requests/AssetRequest';
import createPathRequest from './requests/PathRequest';
import {createEnvironment} from './Environment';
import {createDependency} from './Dependency';
import {Disposable} from '@parcel/events';
import {init as initSourcemaps} from '@parcel/source-map';
import {init as initHash} from '@parcel/hash';
import {toProjectPath} from './projectPath';
import {
fromProjectPath,
toProjectPath,
fromProjectPathRelative,
} from './projectPath';
import {tracer} from '@parcel/profiler';

registerCoreWithSerializer();
Expand Down Expand Up @@ -437,6 +450,78 @@ export default class Parcel {
logger.info({origin: '@parcel/core', message: 'Taking heap snapshot...'});
return this.#farm.takeHeapSnapshot();
}

async transform(options: ParcelTransformOptions): Promise<Array<Asset>> {
if (!this.#initialized) {
await this._init();
}

let projectRoot = nullthrows(this.#resolvedOptions).projectRoot;
let request = createAssetRequest({
...options,
filePath: toProjectPath(projectRoot, options.filePath),
optionsRef: this.#optionsRef,
env: createEnvironment({
...options.env,
loc:
options.env?.loc != null
? {
...options.env.loc,
filePath: toProjectPath(projectRoot, options.env.loc.filePath),
}
: undefined,
}),
});

let res = await this.#requestTracker.runRequest(request, {force: true});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use force here? My understanding is that force means the cache is ignored.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can update that, but it was originally copied from the intiial branch, I will leave this Q to @devongovett on if this is required?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just changed it to a configurable value. Follows the config of shouldCacheDisabled from the main config.
I guess this could be helpful to debug transform problems.

return res.map(asset =>
assetFromValue(asset, nullthrows(this.#resolvedOptions)),
);
}

async resolve(request: ParcelResolveOptions): Promise<?ParcelResolveResult> {
if (!this.#initialized) {
await this._init();
}

let projectRoot = nullthrows(this.#resolvedOptions).projectRoot;
if (request.resolveFrom == null && path.isAbsolute(request.specifier)) {
request.specifier = fromProjectPathRelative(
toProjectPath(projectRoot, request.specifier),
);
}

let dependency = createDependency(projectRoot, {
...request,
env: createEnvironment({
...request.env,
loc:
request.env?.loc != null
? {
...request.env.loc,
filePath: toProjectPath(projectRoot, request.env.loc.filePath),
}
: undefined,
}),
});

let req = createPathRequest({
dependency,
name: 'test',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what the purpose of this name is. Looks like it's normally sourced from the AssetGraphBuilder and it could be Main or Runtimes.

@devongovett Any input on what this should be?

Copy link
Contributor Author

@kiddkai kiddkai Aug 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it to .specifeir

The name seem to be used to construct the req.id only. The first part of id is already hashed, so whatever value here doesn't seem to be matter too much.

Tho, change to specifier should be a lil bit more meaningful.

});

let res = await this.#requestTracker.runRequest(req, {force: true});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question here about force.

if (!res) {
return null;
}

return {
filePath: fromProjectPath(projectRoot, res.filePath),
code: res.code,
query: res.query,
sideEffects: res.sideEffects,
};
}
}

export class BuildError extends ThrowableDiagnostic {
Expand Down
53 changes: 53 additions & 0 deletions packages/core/core/test/Parcel.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,59 @@ describe('Parcel', function () {
});
});

describe('ParcelAPI', function () {
this.timeout(75000);

let workerFarm;
beforeEach(() => {
workerFarm = createWorkerFarm();
});

afterEach(() => workerFarm.end());

describe('parcel.transform()', () => {
it('should transforms simple file', async () => {
let parcel = createParcel({workerFarm});
let res = await parcel.transform({
filePath: path.join(__dirname, 'fixtures/parcel/index.js'),
});
let code = await res[0].getCode();
assert(code.includes('exports.default = "test"'));
});

it('should transform with standalone mode', async () => {
let parcel = createParcel({workerFarm});
let res = await parcel.transform({
filePath: path.join(__dirname, 'fixtures/parcel/other.js'),
query: 'standalone=true',
});
let code = await res[0].getCode();

assert(code.includes('require("./index.js")'));
assert(code.includes('new URL("index.js", "file:" + __filename);'));
assert(code.includes('import("index.js")'));
});
});

describe('parcel.resolve()', () => {
it('should resolve dependencies', async () => {
let parcel = createParcel({workerFarm});
let res = await parcel.resolve({
specifier: './other',
specifierType: 'esm',
resolveFrom: path.join(__dirname, 'fixtures/parcel/index.js'),
});

assert.deepEqual(res, {
filePath: path.join(__dirname, 'fixtures/parcel/other.js'),
code: undefined,
query: undefined,
sideEffects: true,
});
});
});
});

function createParcel(opts?: InitialParcelOptions) {
return new Parcel({
entries: [path.join(__dirname, 'fixtures/parcel/index.js')],
Expand Down
1 change: 1 addition & 0 deletions packages/core/core/test/fixtures/parcel/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export default 'test';
3 changes: 3 additions & 0 deletions packages/core/core/test/fixtures/parcel/other.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import * as idx from './index.js';
new URL('index.js', import.meta.url);
import('index.js')
22 changes: 22 additions & 0 deletions packages/core/types/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -645,6 +645,28 @@ export type ASTGenerator = {|

export type BundleBehavior = 'inline' | 'isolated';

export type ParcelTransformOptions = {|
filePath: FilePath,
code?: string,
env?: EnvironmentOptions,
pipeline?: ?string,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if pipeline makes sense as a public option? Shouldn't this be inferred from the .parcelrc?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am happy to remove this pipline property.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed.

query?: ?string,
|};

export type ParcelResolveOptions = {|
specifier: DependencySpecifier,
specifierType: SpecifierType,
env?: EnvironmentOptions,
resolveFrom?: FilePath,
|};

export type ParcelResolveResult = {|
filePath: FilePath,
code?: string,
query?: ?string,
sideEffects?: boolean,
|};

/**
* An asset represents a file or part of a file. It may represent any data type, including source code,
* binary data, etc. Assets may exist in the file system or may be virtual.
Expand Down
29 changes: 17 additions & 12 deletions packages/transformers/js/core/src/dependency_collector.rs
Original file line number Diff line number Diff line change
Expand Up @@ -125,15 +125,16 @@ impl<'a> DependencyCollector<'a> {
None
}
}
_ => Some(format!(
_ if !self.config.standalone => Some(format!(
"{:x}",
hash!(format!(
"{}:{}:{}",
self.get_project_relative_filename(),
specifier,
kind
))
)),
)),
_ => None,
};

self.items.push(DependencyDescriptor {
Expand All @@ -158,7 +159,7 @@ impl<'a> DependencyCollector<'a> {
source_type: SourceType,
) -> ast::Expr {
// If not a library, replace with a require call pointing to a runtime that will resolve the url dynamically.
if !self.config.is_library {
if !self.config.is_library && !self.config.standalone {
let placeholder =
self.add_dependency(specifier.clone(), span, kind, None, false, source_type);
let specifier = if let Some(placeholder) = placeholder {
Expand All @@ -172,13 +173,17 @@ impl<'a> DependencyCollector<'a> {
// For library builds, we need to create something that can be statically analyzed by another bundler,
// so rather than replacing with a require call that is resolved by a runtime, replace with a `new URL`
// call with a placeholder for the relative path to be replaced during packaging.
let placeholder = format!(
"{:x}",
hash!(format!(
"parcel_url:{}:{}:{}",
self.config.filename, specifier, kind
))
);
let placeholder = if self.config.standalone {
specifier.as_ref().into()
} else {
format!(
"{:x}",
hash!(format!(
"parcel_url:{}:{}:{}",
self.config.filename, specifier, kind
))
)
};
self.items.push(DependencyDescriptor {
kind,
loc: SourceLocation::from(self.source_map, span),
Expand Down Expand Up @@ -666,7 +671,7 @@ impl<'a> Fold for DependencyCollector<'a> {
// Replace import() with require()
if kind == DependencyKind::DynamicImport {
let mut call = node;
if !self.config.scope_hoist {
if !self.config.scope_hoist && !self.config.standalone {
let name = match &self.config.source_type {
SourceType::Module => "require",
SourceType::Script => "__parcel__require__",
Expand Down Expand Up @@ -838,7 +843,7 @@ impl<'a> Fold for DependencyCollector<'a> {

// If this is a library, we will already have a URL object. Otherwise, we need to
// construct one from the string returned by the JSRuntime.
if !self.config.is_library {
if !self.config.is_library && !self.config.standalone {
return Expr::New(NewExpr {
span: DUMMY_SP,
callee: Box::new(Expr::Ident(Ident::new(js_word!("URL"), DUMMY_SP))),
Expand Down
1 change: 1 addition & 0 deletions packages/transformers/js/core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ pub struct Config {
is_esm_output: bool,
trace_bailouts: bool,
is_swc_helpers: bool,
standalone: bool,
}

#[derive(Serialize, Debug, Default)]
Expand Down
1 change: 1 addition & 0 deletions packages/transformers/js/src/JSTransformer.js
Original file line number Diff line number Diff line change
Expand Up @@ -442,6 +442,7 @@ export default (new Transformer({
is_esm_output: asset.env.outputFormat === 'esmodule',
trace_bailouts: options.logLevel === 'verbose',
is_swc_helpers: /@swc[/\\]helpers/.test(asset.filePath),
standalone: asset.query.has('standalone'),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about using a query param to drive this behaviour. Maybe this should be part of the environment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did think about it, for now using query provides some flexibility & less moving part to change the environment. Kinda like a feature flag for jsTransform for now.

});

let convertLoc = (loc): SourceLocation => {
Expand Down