[FEATURE] Add JSDoc build functionalities

[codeworrior]

Note: the code is full of console.log just to track down an unexpected exit of the Node.js process.

[RandomByte]

By writing a resource, the FileSystem adapter drains its content stream. Therefore the .clone workaround makes sense. However, I'll look into a way to have the FS adapter to provide a new stream.

There is this undocumented contract that whoever reads from a Resources stream has to provide a new readable stream via setStream

[RandomByte]

Basically, the issue is as follows:

Writing resource A to the file system requires to drain the stream. Currently, after that operation the Resource has irreversibly lost any content, making it impossible to further process this resource (content is somewhat essential).

I see two possible solutions here:

1. Before the write operation, load the content of A into a buffer.
   => After the write operation, content A₂ is still available
   Only downside is the potentially high memory consumption caused by all those buffers.
2. The resource gets a new readable stream of the file that has been written to FS (tmp)
   => After the write operation, content A₂ is still available
   However, any "physical" changes done to that file will be applied to resource A too. This might be unexpected.

[RandomByte]

I think 1. is closer to the expected behavior, I'll try to prepare a ui5-fs PR for that soon.

[RandomByte]

Started working on the discussed ui5-fs fix in SAP/ui5-fs#22

[RandomByte]

Blocker should be resolved with SAP/ui5-fs#22

[CLAassistant]

All committers have signed the CLA.

[RandomByte]

Rebased

[RandomByte]

Implementation Concept

JSDoc generation UI5 Tooling integration.pdf

Additional Implementation Details

• Preferably add a "ui5 build jsdoc” command defining all JSDoc related tasks as enabled by default
  • It is still possible to activate single JSDoc related tasks in any other build using the corresponding CLI parameter
  • Done with [INTERNAL] Build: Add 'jsdoc' command ui5-cli#162
• Add ui5.yaml configuration for JSDoc excludes (like thirdparty- or very big, not relevant resources). Grunt JSDoc build currently does not have any exclude handling. Maven JSDoc build reads this information from .library
  • Basic implementation with 7c98831 - TODO (done) + Pending clarification (clarified)
• Enhance Custom Task implementation for defining "final" tasks (lower part of the chart) and positioning as “first/last” task as there might not be any active "final" task that could be used for relative positioning OBSOLETE - Tasks that where meant to be executed in this "final" stage shall be moved into an "SDK" project that, when being built, will be executed last anyways
• Processor returns array of resources but also writes to fs? (yes, at least for jsdoc config, but maybe different location?) (maybe different dest location?) => Only writes to tmp fs. Currently easiest solution to have easy reuse of UI5 JSDoc libs and proper separation of concerns

CC: @tommyvinhlam

[RandomByte]

Not sure anymore whether the project name should be included in the temporary path directly. At least we should remove all alpha-num characters to prevent directory traversal

Edit: Done with 0d4a77f

[RandomByte]

b93d4bd depends on SAP/ui5-fs#112

[RandomByte]

TODO: Document new JSDoc exclude configuration in UI5 Project - Done: SAP/ui5-project#152

Example from sap.ui.core:

---
specVersion: "0.1"
type: library
metadata:
  name: sap.ui.core
  copyright: |-
   OpenUI5
    * (c) Copyright 2009-${currentYear} SAP SE or an SAP affiliate company.
    * Licensed under the Apache License, Version 2.0 - see LICENSE.txt.
builder:
  jsdoc:
    excludes:
    - "jquery-*"
    - "sap-ui-*"
    - "sap/ui/debug/**"
    - "sap/ui/qunit/**"
    - "sap/ui/thirdparty/**"

[RandomByte]

Still not sure whether JSDoc exclude patterns should include the virtual base path (/resources / /test-resources) to allow differentiation between the two (or more). Currently they are designed to only work within /resources (just like the JSDoc build itself).

This came to my mind when thinking about the definition of project-wide exclude patterns as requested in SAP/ui5-tooling#124. They should probably include the virtual paths to allow excluding resources of /test-resources for example.

So even though the JSDoc build itself is only working on /resources, having the same flavor of include/exclude patterns (that include the virtual base path) in the ui5.yaml might make sense.

[matz3]

@RandomByte I would align this with the bundler, which also doesn't contain resources or test-resources in the pattern/filters but implicitly uses resources.

[RandomByte]

We had another discussion and tend to keep JSDoc exclude patterns aligned with the bundler patterns while use absolute virtual paths (/resources/pony/**) for more general excludes.

[RandomByte]

Rebased

[devtomtom]

Looks good to me!

[codeworrior]

I've tried it out on OpenUI5 (together with the pending change there) and it worked quite well!

[devtomtom]

👍