Skip to content

Latest commit

 

History

History
455 lines (317 loc) · 24.4 KB

0012-UI5-Tooling-Extension-API-3.md

File metadata and controls

455 lines (317 loc) · 24.4 KB
Raw

RFC 0012 UI5 Tooling Extension API v3

Table of Contents

Summary

This Request for Comment summarizes potential enhancements to APIs provided to UI5 Tooling extensions. Namely custom tasks and custom middleware.

This RFC is written in the context of the current development- and upcoming release of UI5 Tooling version 3. This major release of UI5 Tooling brings many changes and improvements. And a new Specification Version 3.0.

It is therefore reasonable that most of the new API described below will only be available to extensions defining Specification Version 3.0. However, we might decide to postpone the implementation of some features for a later version, like 3.1 or even a future major version 4.0.

Motivation

There is a great community of UI5 developers, continuously coming up with innovative solutions and improving the development experience with UI5 through the creation of UI5 Tooling extensions like custom tasks and middleware.

As of today, the UI5 community created close to 100 extensions for UI5 Tooling and made them available publicly on npm. This is not only a great investment of many dedicated developers but also a treasure trove for anyone working with UI5.

This RFC shall address two main topics:

  1. A UI5 Tooling extension that has been written once, should continue to work as intended for as long as possible, without requiring major changes to its implementation
  2. Common requirements of extensions should be addressed by providing helpful API, reducing the implementation efforts for some extensions, while further ensuring future compatibility

General Changes in UI5 Tooling

💡 Notice 1: ESM Support

With UI5 Tooling v3, extensions can be written as ES modules. This is a JavaScript language feature and works independently from the defined Specification Version. Current versions of Node.js automatically detect whether a package is ESM or not.

However, due to it's support for older Node.js versions, UI5 Tooling v2 does not support ESM extensions. Therefore we advise all extensions that use ESM to define Specification Version 3, in order to express their incompatibility with UI5 Tooling v2.

Extensions can still be written as CommonJS modules. Note however, that all UI5 Tooling v3 modules are ESM. Thus extensions can't "require" them anymore but have to use (dynamic-) imports. For more information, see also: Sindre Sorhus: Pure ESM package

⚠️ Attention projects currently defining dependencies to UI5 Tooling modules, without specifying a version range (e.g. @ui5/logger: "*"): Unless your project is already ESM, your extension might break. A statement like require("@ui5/logger") will throw an ERR_REQUIRE_ESM exception for @ui5/logger v3.

An example for an ESM extension can be found here: feat(ui5-task-flatten-library): transform to ES Modules with UI5 Tooling version 3.0.0 #648

💡 Notice 2: UI5 Tooling Dependency Resolution Improvements

UI5 Tooling v3 will stop using the "ui5": { "dependencies": [...] } package.json configuration. Instead, for the current root project, all dependencies, devDependencies and optionalDependencies are automatically analyzed.

In the past, devDependencies were not analyzed. Which made the package.json configuration necessary in many cases.

For mode details, see also the Migrate to v3 UI5 Tooling documentation.

Proposed Enhancements

1. Dependencies to UI5 Tooling Modules

✅ This has been realized with UI5 Tooling v3

Description

It is common for extensions to require UI5 Tooling modules like @ui5/fs or @ui5/logger. For example to create Resource entities, or to log messages to the console.

The problem that arises with this, is compatibility across UI5 Tooling versions.

Examples

  1. A Resource created with @ui5/fs v2 is not compatible to @ui5/fs v3 and needs to be migrated by UI5 Tooling v3.
  2. Extensions using @ui5/logger v2 will use a different logger instance than UI5 Tooling v3. This can lead to, for example, log-level configuration not being available to the extension.

Solution

Dependency injection. By providing extensions with callback functions to create resources or log messages, the extension can remove problematic dependencies like @ui5/fs or @ui5/logger from their package.json.

In a future UI5 Tooling version 4, extensions defining Specification Version 3 would then still receive the expected Resource and Logger APIs, thus staying compatible while UI5 Tooling v4 makes internal enhancements and changes to the APIs.

The new APIs should be provided via the existing helper classes taskUtil and middlewareUtil.

New API

  • New parameter log
    • The provided logger instance will automatically have the name of the custom task or middleware.
    • Additional sub-loggers can be crated using log.createSubLogger("name suffix")
  • taskUtil|middlewareUtil.resourceFactory.createResource({...})
    • Creates and returns a Resource with an interface signature according to the Specification Version of the extension.
    • This implies a new resourceFactory object which provides Specification Version-dependent access to functions of the @ui5/fs/resourceFactory

Solution Example

Migrating the custom task example from our current documentation to make use of the new API could look like this:

const path = require("path");
-const {Resource} = require("@ui5/fs");
-const log = require("@ui5/logger").getLogger("builder:customtask:markdown");
const renderMarkdown = require("./renderMarkdown");

-module.exports = async function({workspace, dependencies, taskUtil, options}) {
+module.exports = async function({workspace, dependencies, log, taskUtil, options}) {
+  const {createResource} = taskUtil.resourceFactory;
  const textResources = await workspace.byGlob("**/*.md")
  await Promise.all(textResources.map(async (resource) => {
    const htmlString = await renderMarkdown(await resource.getString());

    const markdownResourcePath = resource.getPath();
    log.info(`Rendering markdown file ${markdownResourcePath}...`) // Provided logger will automatically have a name like "builder:custom-task:generateMarkdownFiles"

    // Note: @ui5/fs virtual paths are always POSIX (on all systems)
    const newResourceName = path.posix.basename(markdownResourcePath, ".md") + ".html";
    const newResourcePath = path.posix.join(path.posix.dirname(markdownResourcePath), newResourceName);

-    const markdownResource = new Resource({
+    const markdownResource = createResource({
      path: newResourcePath,
      string: htmlString
    });
    await workspace.write(markdownResource);
  }));
};

2. Access to Project Information

✅ This has been realized with UI5 Tooling v3

Description

Extensions can not access information about the current project directly. Except for what is provided via the options argument (namely projectNameand projectNamespace) or in the extension's configuration.

Examples

See issue Access package.json in custom task #360.

Solution

UI5 Tooling v3 has switched the internal project handling from a JSON-hierarchy to a graph of project-entities. Extensions should get access to both. With a well-defined, specification version dependent interface.

New API

  • taskUtil|middlewareUtil.getProject(projectName|resource)
    • If no parameter is provided: Retrieves the project currently being built from the graph
    • If a project name is provided: Retrieves a project from the graph by name
    • If a resource instance is provided: Retrieves the project the resource belongs to from the graph
  • taskUtil|middlewareUtil.getDependencies(projectName)
    • Returns the project names of all direct dependencies of the given project

Project interface for Specification Version 3:

  • getType: Returns the type of the project. E.g. "library", "application", etc.
  • getName: Returns the name of the project
  • getNamespace: Returns the detected namespace of the project. E.g. x/y/z
  • getVersion: Typically returns the package.json version of the project
  • getRootReader: Returns an @ui5/fs/AbsatractReader instance for the root directory of the project. Typically the same directory that contains the ui5.yaml
  • getReader: Returns an @ui5/fs/AbsatractReader instance for the resources of the project
  • getCustomConfiguration: Returns the custom configuration of the project
  • isFrameworkProject: Returns true if the project is a UI5 framework (theme-)library

Solution Example

module.exports = async function({workspace, dependencies, taskUtil, options}) {
  const currentProject = taskUtil.getProject();
  const eslintConfig = await currentProject.getRootReader().byPath(".eslintrc");
  // Do something fun with the eslint file of the project that is currently being built

  // [...]

  const dependencyProjects = taskUtil.getDependencies(options.projectName).map((dependencyName) => {
    return taskUtil.getProject(dependencyName);
  });
  // Do something with the direct dependencies. For example collect their versions using dependencyProjects[n].getVersion()
};

3. Tasks Requiring Dependencies

✅ This has been realized with UI5 Tooling v3

Description

With UI5 Tooling v3, tasks can rely on the fact that the resources provided by dependencies are always built. With UI5 Tooling v2, this was only the fact if the build command specified dependencies to be built (e.g. ui5 build --all).

However, in order to optimize the build time, UI5 Tooling v3 now differentiates between tasks that only make use of a project's resources and those that also require dependency resources.

Examples

  • A task that renders all markdown files of a project to HTML, does not need to access the resources of any of the project's dependencies.
  • A task that bundles required CSS files from the project and some dependencies, requires access to the resources of those dependencies.

Solution

Custom tasks defining Specification Version 3.0 shall not receive a dependencies AbstractReader, unless they request dependencies to be available to them. They can do so by exporting an additional callback function determineRequiredDependencies. Before the task is executed, this function will be called with the current build parameters and available dependencies. It can then return a Set of dependencies if requires access to. Or an empty Set if no dependency access is required.

By default, legacy custom tasks defining Specification Versions lower than 3.0 are expected to require dependencies. However, even they can provide the described callback to opt-out.

New API

  • Custom task export: async determineRequiredDependencies({availableDependencies, getProject, getDependencies, options})
    • availableDependencies: Set containing the names of all direct dependencies of the project currently being built. For example by returning it unmodified, all dependencies will be available to the task.
    • getProject, getDependencies: Identical to taskUtil.getProject and taskUtil.getDependencies
    • options: Same as for the main task function. {projectName, projectNamespace, configuration, taskName}
    • Returns: Set containing all dependencies that should be made available to the task. UI5 Tooling will ensure that those dependencies have been built before executing the task.

Solution Example

module.exports = async function({workspace, dependencies, taskUtil, options}) {
  // Task implementation
};

module.exports.determineRequiredDependencies = async function({availableDependencies, getProject, getDependencies, options}) {
  // "availableDependencies" could look like this: Set(3) { "sap.ui.core", "sap.m", "my.lib" }

  // One could for example ignore all framework libraries:
  availableDependencies.forEach((depName) => {
    if (getProject(depName).isFrameworkProject()) {
        availableDependencies.delete(depName)
    }
  });
  // => Only resources of project "my.lib" will be available to the task
  return availableDependencies;
};

4. Middleware Serving Resources

NOTE: The concept described in this chapter has not been implemented with UI5 Tooling v3. It might become part of a future RFC though

Description

Custom middleware that would like to respond with an @ui5/fs/Resource needs to implement a lot of common logic in order to properly do that. I.e. the correct content type and E-Tag headers need to be set.

Custom middleware should be provided with a helper function to serve resources efficiently.

Examples

See the logic for serving a resource in the standard @ui5/server middleware "serveResources":
serveResources.js#L65-L93

Custom (community) middleware "ui5-middleware-stringreplacer" has to implement almost the same logic for serving a resource itself:
stringreplacer.js#L135-L161

Solution

middlewareUtil should provide a convenience method to respond with the content of an @ui5/fs/Resource.

New API

  • async middlewareUtil.sendResource(res, resource)

Solution Example

module.exports = function({resources, middlewareUtil, options}) {
    return function (req, res, next) {
        resources.rootProject.byPath("index.html").then((resource) => {
            await middlewareUtil.sendResource(res, resource);
        }
    }
};

5. Scheduling of Extension Executions

NOTE: The concept described in this chapter has not been implemented with UI5 Tooling v3. It might become part of a future RFC though

Description

Provide a better way for extensions to define a time of execution. Remove the implicit dependency on standard or custom tasks/middleware for the execution order, or for being executed at all.

Examples

specVersion: "2.6"
kind: project
type: application
metadata:
  name: my.application
server:
  customTasks:
    - name: generateMarkdownFiles
      afterTask: buildThemes

Solution

Introduce "Build Phases" for common scheduling of custom tasks:

Build Phase Standard Tasks in that Phase
Pre-Build none
Pre-Prepare-Sources none
Prepare-Sources escapeNonAsciiCharacters, replaceCopyright, replaceVersion, replaceBuildtime, minify
Post-Prepare-Sources none
Pre-Create-Bundles none
Create-Bundles generateFlexChangesBundle, generateManifestBundle,
generateLibraryManifest, generateComponentPreload,
generateLibraryPreload, generateStandaloneAppBundle
Post-Create-Bundles transformBootstrapHtml
Pre-Build-Themes none
Build-Themes buildThemes
Post-Build-Themes generateThemeDesignerResources
Post-Build generateVersionInfo, generateVersionInfo, generateApiIndex, generateResourcesJson

The custom task configuration can reference a build phase and define whether the task should be executed at the start or at the end of a given phase. If multiple custom tasks are defined for the same execution time, the order of their definition in the ui5.yaml configuration is relevant.

TODO/To be decided: Provide similar phases for server middleware?

Solution Example

specVersion: "2.6"
kind: project
type: application
metadata:
  name: my.application
server:
  customTasks:
    - name: generateMarkdownFiles
      phase: post-build
      phase-order: start

6. Restrictions for Project and Extension Names

✅ This has been realized with UI5 Tooling v3

Description

Project- and Extension-names are currently unrestricted. Some boundaries should be applied to align names in general, and to ensure future compatibility.

Additionally, extensions can take the same names as standard tasks or middleware. This can lead to confusion and issues. Especially when new standard elements are introduced in UI5 Tooling, there is a risk of having name clashes with existing extensions.

Note that projects typically do not have this issue, since they are commonly follow a namespace-like structure.

Examples

Currently, projects and extensions can define names with any characters (including special characters), of unrestricted length.

These names are used in various placed within UI5 Tooling. Certain names can have unintended side-effects. Worst-case, they can lead to security issues.

Solution

New boundary conditions for names of projects and extensions (as defined in the metadata.name attribute):

  • Names must be at least 3 characters long
  • Names must be no longer than 50 characters
  • Names must contain lowercase characters only
  • Names must contain alphanumeric characters, dash, underscore, period only
    • Exception: @ and / are allowed at certain positions as explained below
  • Names must start with an alphabetic character or an @-character
  • If a name starts with an @-character, it must contain exactly one forward-slash /
    • This is aligned with the npm concept for package scopes
    • e.g. @org/lib.name

The current proposal of a regular expression for the above is ^(?:@[0-9a-z-_.]+/)?[a-z][0-9a-z-_.]*$.

Examples for allowed names:

  • sky.app
  • @org/heavy.lib
  • ui5-task-fearless-rock
  • ui5-middleware-fluffy-star

Examples for disallowed names:

  • 笑.app (contains a non-alphabetic character)
  • Heavy Lib (contains uppercase characters and a space)
  • ui5~task/fearless-rock (contains special characters ~ and /)

In the future, similar restrictions should be implemented for other configuration properties.

To be Discussed

  • ✅ This would disallow the use of any UTF-8 characters. Including Chinese characters, Emojis, etc. Is this still matching with developer expectations?
    • npm package names already follow similar restrictions. Currently we do not expect to fail anyone's expectations with this change.

Prior-Art

  • npm package.json
    • The name must be less than or equal to 214 characters. This includes the scope for scoped packages.
    • The names of scoped packages can begin with a dot or an underscore. This is not permitted without a scope.
    • New packages must not have uppercase letters in the name.
    • The name ends up being part of a URL, an argument on the command line, and a folder name. Therefore, the name can't contain any non-URL-safe characters.
    • Unofficial JSON schema defines the following pattern:
      (?:@[a-z0-9-*~][a-z0-9-*._~]*/)?[a-z0-9-~][a-z0-9-._~]*$
  • Kubernetes Object Names and IDs:
    • contain at most 63 characters
    • contain only lowercase alphanumeric characters or '-'
    • start with an alphanumeric character
    • end with an alphanumeric character
  • Docker Tags:

    A tag name must be valid ASCII and may contain lowercase and uppercase letters, digits, underscores, periods and dashes. A tag name may not start with a period or a dash and may contain a maximum of 128 characters.

7. Provide Extension Name (minor)

✅ This has been realized with UI5 Tooling v3

Description

If multiple extension definitions share the same implementation, it would be beneficial to have the configured name available at runtime.

Examples

See issue TaskUtil to expose the current tasks name #545

Solution

Enhance options object passed to custom task or middleware with a new attribute taskName or middlewareName containing the name configured in the corresponding ui5.yaml (not the runtime name, which might have a suffix like taskName-1 if multiple executions are scheduled).

Solution Example

module.exports = async function({workspace, dependencies, taskUtil, options}) {
  const taskName = options.taskName;
};

How we teach this

Enhanced UI5 Tooling documentation for custom tasks and custom middleware. Updated API reference.

Drawbacks

Introducing new API for extensions always bears the risk of hindering future development of UI5 Tooling due to compatibility constrains.

Unresolved Questions and Bikeshedding

This section should be removed (i.e. resolved) before merging