Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: set deploy URL as runtime property #630

Merged
merged 3 commits into from
Jul 30, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ RUN node schematics/customization/service-worker ${serviceWorker} || true
COPY templates/webpack/* /workspace/templates/webpack/
ARG testing=false
ENV TESTING=${testing}
RUN npm run ng -- build -c ${configuration}
RUN npm run ng -- build -c ${configuration} --deploy-url=DEPLOY_URL_PLACEHOLDER
# synchronize-marker:pwa-docker-build:end

# ^ this part above is copied to Dockerfile_noSSR and should be kept in sync
Expand Down
5 changes: 4 additions & 1 deletion Dockerfile_noSSR
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,14 @@ RUN node schematics/customization/service-worker ${serviceWorker} || true
COPY templates/webpack/* /workspace/templates/webpack/
ARG testing=false
ENV TESTING=${testing}
RUN npm run ng -- build -c ${configuration}
RUN npm run ng -- build -c ${configuration} --deploy-url=DEPLOY_URL_PLACEHOLDER
# synchronize-marker:pwa-docker-build:end

# ^ this part above is copied from the standard Dockerfile and should be kept in sync

ARG deployUrl=/
RUN npx ts-node scripts/set-deploy-url ${deployUrl}

FROM nginx:alpine
COPY templates/nginx.conf /etc/nginx/nginx.conf
COPY --from=buildstep /workspace/dist/browser /usr/share/nginx/html
Expand Down
26 changes: 22 additions & 4 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,32 @@ services:
- TRUST_ICM=true
# - PROMETHEUS=on
# - MULTI_SITE_LOCALE_MAP={"en_US":"/en","de_DE":"/de","fr_FR":"/fr"}

# <CDN-Example>
# add 127.0.0.1 mypwa.net to your hosts file and
# uncomment the following lines
#
# - DEPLOY_URL=http://mypwa.net:4222
# cdn:
# build:
# context: .
# dockerfile: Dockerfile_noSSR
# args:
# configuration: production
# serviceWorker: 'false'
# deployUrl: http://mypwa.net:4222
# ports:
# - '4222:4200'
#
# </CDN-Example>

nginx:
build: nginx
depends_on:
- pwa
ports:
- '4200:80'
# - '9113:9113'
environment:
UPSTREAM_PWA: 'http://pwa:4200'
# DEBUG: 1
Expand Down Expand Up @@ -51,7 +73,3 @@ services:
channel: inSPIRED-inTRONICS_Business-Site
features: quoting,businessCustomerRegistration,advancedVariationHandling,quickorder,orderTemplates,punchout,recently
theme: 'blue|688dc3'

ports:
- '4200:80'
# - '9113:9113'
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ kb_sync_latest_only
- [Concept - Building Blocks of the Intershop PWA](./concepts/pwa-building-blocks.md)
- [Guide - Building and Running Server-Side Rendering](./guides/ssr-startup.md)
- [Guide - Building and Running nginx Docker Image](./guides/nginx-startup.md)
- [Concept - Deploy URL](./concepts/deploy-url.md)
- [Concept - Multi-Site Handling](./concepts/multi-site-handling.md)
- [Guide - Multi-Site Configurations](./guides/multi-site-configurations.md)
- [Concept - Hybrid Approach](./concepts/hybrid-approach.md)
Expand Down
90 changes: 90 additions & 0 deletions docs/concepts/deploy-url.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
<!--
kb_concepts
kb_pwa
kb_everyone
kb_sync_latest_only
-->

# Deploy URL
jometzner marked this conversation as resolved.
Show resolved Hide resolved

This document describes how to provide the Intershop PWA with a dynamic deploy URL to set up client-side retrieval of static assets and JavaScript chunks from a (possibly) different source than the URL from which the pre-rendering is delivered.

## Built-in Angular CLI Support

By default, Angular CLI supports setting the deploy URL of the application with the [`build`](https://angular.io/cli/build) command parameter `--deploy-url`.
As this is only supplied as a build-time setting, the required flexibility for the Intershop PWA cannot be achieved.

For example, if the Intershop PWA Docker image is deployed in multiple environments in an [Intershop CaaS](https://support.intershop.com/kb/index.php/Display/29S118) context, every environment would have to build a new PWA.

Therefore this functionality is not used out of the box.
However, it is possible to dynamically configure the deploy URL as a runtime setting for the [SSR container][guide-ssr-container].

## Setting Deploy URL Dynamically

### Problem

To solve this problem, individual problems had to be solved:

1. The Angular CLI build process inserts references to the initial bundles (`main`, `common`, `polyfills` and `runtime`) into the `index.html` relatively when called without `--deploy-url`, but uses a static deploy URL if requested.

2. The `runtime` bundle loads lazy loaded chunks relatively if no `--deploy-url` is passed, and via deploy URL if built with the `--deploy-url` parameter.

3. The CSS bundles contain absolute and relative references to resources in the `assets` folder.

4. The PWA code itself contains relative and absolute references to the `assets` folder (i.e. images).

### Solution

The solution can be schemed as follows:

- The `ng build` process has to be triggered with a placeholder for the deploy URL.
- The placeholder can then be dynamically replaced in the SSR process as a post-processing for any output:
- Placeholders are replaced with absolute deployment URLs.
- References to the `assets` folder are transformed into absolute URLs pointing to the deployment URLs.

This way, setting the deployment URL becomes a runtime setting and can be changed without rebuilding the Intershop PWA.

### Building with Dynamic Deploy URL

The PWA client-side application has to be built using `ng build ... --deploy-url=DEPLOY_URL_PLACEHOLDER`, which introduces a placeholder for all deployment-dependent functionality.

This placeholder is then dynamically replaced by the SSR process using the environment variable `DEPLOY_URL` (i.e. by setting it at the [SSR container][guide-ssr-container]).
If no `DEPLOY_URL` is supplied, the fallback `/` is applied.

This placeholder can also be replaced statically when no SSR process is used in the deployment by running the script `npx ts-node scripts/set-deploy-url <deploy-url>`.
If no `<deploy-url>` is supplied, the fallback `/` is applied.

> For consistency reasons the build process must always follow this pattern, so that all references to the `assets` folder are made absolute and all JavaScript bundles are loaded from the root of the deployment (see [related issue](https://github.com/intershop/intershop-pwa/issues/624)).
> This build process is also provided in the Docker images as default solution.

## Scenarios

With this new feature, some complex deployment scenarios can be solved with the Intershop PWA out of the box.

### CDN Support

If you want to deploy all static resources of the Intershop PWA to a [Content Delivery Network](https://en.wikipedia.org/wiki/Content_delivery_network), you can use the above build steps to set the deploy URL via script in the build output.
Delivering resources via a CDN significantly reduces the load on the PWA if certain pages can also be stored pre-rendered or if the experimental [Service Worker][concept-pwa-service-worker] support is used.

### Embed PWA with Proxy on Website

Consider the following deployment scenario:

You want to use the Intershop PWA as part of a bigger portal to provide a shopping experience.
However, not all of the product pages are to be handled by the PWA, as you want to use a different integration from the portal that is already available.
Most likely, all product pages have a similar URL pattern and should be available from the same domain for SEO reasons.

This scenario would mean that the portal and the Intershop PWA would share routes similar to the [Hybrid Approach][concept-hybrid-approach].
To set this up manually, a lot of rewriting for static PWA assets would have to be set up in the portal's reverse proxy, so that Intershop PWA client applications can boot up correctly.

By setting a deployment URL, only the incoming routing for server-side rendering would be targeted at the portal's reverse proxy.
After parsing the response, the client-side application pulls all necessary static assets and JavaScript chunks directly from the deployment URL.

# Further References

- [Concept - Hybrid Approach][concept-hybrid-approach]
- [Guide - Building and Running Server-Side Rendering][guide-ssr-container]

[guide-ssr-container]: ../guides/ssr-startup.md
[concept-hybrid-approach]: ./hybrid-approach.md
[concept-pwa-service-worker]: ./progressive-web-app.md#service-worker
3 changes: 3 additions & 0 deletions docs/guides/ssr-startup.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ Make sure to use them as written in the table below.
| | FEATURES | comma-separated list | Overrides active features |
| | THEME | string | Overrides the default theme |
| | MULTI_SITE_LOCALE_MAP | JSON \| false | Used to map locales to [url modification parameters](../guides/multi-site-configurations.md) |
| | DEPLOY_URL | string | Set a [Deploy URL][concept-deploy-url] (default `/`) |
| **Debug** :warning: | TRUST_ICM | any | Use this if ICM is deployed with an insecure certificate |
| | LOGGING | switch | Enables extra log output |
| **Hybrid Approach** | SSR_HYBRID | any | Enables running PWA and ICM in [Hybrid Mode][concept-hybrid] |
Expand All @@ -66,6 +67,7 @@ Extension `crt` is the certificate and `key` represents the private key.
# Further References

- [Concept - Configuration](../concepts/configuration.md)
- [Concept - Deploy URL][concept-deploy-url]
- [Concept - Hybrid Approach][concept-hybrid]
- [Concept - Logging](../concepts/logging.md)
- [Concept - Single Sign-On (SSO) for PWA][concept-sso]
Expand All @@ -77,3 +79,4 @@ Extension `crt` is the certificate and `key` represents the private key.

[concept-sso]: ../concepts/sso.md
[concept-hybrid]: ../concepts/hybrid-approach.md
[concept-deploy-url]: ../concepts/deploy-url.md
2 changes: 1 addition & 1 deletion e2e/test-universal.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ universalTest 8 "${PWA_BASE_URL}/home" "intershop-pwa-state"
universalTest 9 "${PWA_BASE_URL}/home" "&q;baseURL&q;:"
universalTest 10 "${PWA_BASE_URL}/home" "<ish-content-include includeid=.include.homepage.content.pagelet2-Include"
universalTest 11 "${PWA_BASE_URL}/home" "<link rel=.canonical. href=.${PWA_CANONICAL_BASE_URL}/home.>"
universalTest 12 "${PWA_BASE_URL}/home" "<meta property=.og:image. content=.${PWA_CANONICAL_BASE_URL}[^>]*og-image-default"
universalTest 12 "${PWA_BASE_URL}/home" "<meta property=.og:image. content=./assets/img/og-image-default"
universalTest 13 "${PWA_BASE_URL}/home" "<title>inTRONICS Home | Intershop PWA</title>"
universalTest 14 "${PWA_BASE_URL}/sku6997041" "<link rel=.canonical. href=.${PWA_CANONICAL_BASE_URL}/Notebooks/Asus-Eee-PC-1008P-Karim-Rashid-sku6997041-catComputers.1835.151.>"
universalTest 15 "${PWA_BASE_URL}/sku6997041" "<meta property=.og:image. content=[^>]*6997041"
Expand Down
26 changes: 26 additions & 0 deletions scripts/set-deploy-url.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
// tslint:disable: ish-ordered-imports ban-specific-imports no-console
import * as fs from 'fs';
import * as glob from 'glob';
import * as path from 'path';
import { setDeployUrlInFile } from '../src/ssr/deploy-url';

if (process.argv.length < 3) {
console.error('required argument deployUrl missing');
process.exit(1);
}

let deployUrl = process.argv[2];

if (!deployUrl.endsWith('/')) {
deployUrl += '/';
}

glob.sync('dist/browser/*.{js,css,html}').forEach(file => {
console.log(`setting deployUrl "${deployUrl}" in`, file);

const input = fs.readFileSync(file, { encoding: 'utf-8' });

const output = setDeployUrlInFile(deployUrl, path.basename(file), input);

fs.writeFileSync(file, output);
});
41 changes: 32 additions & 9 deletions server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,14 @@ import { join } from 'path';
import * as robots from 'express-robots-txt';
import * as fs from 'fs';
import * as proxy from 'express-http-proxy';
// tslint:disable-next-line: ban-specific-imports
import { AppServerModule, ICM_WEB_URL, HYBRID_MAPPING_TABLE, environment, APP_BASE_HREF } from './src/main.server';
import { ngExpressEngine } from '@nguniversal/express-engine';
import { getDeployURLFromEnv, setDeployUrlInFile } from './src/ssr/deploy-url';

const PORT = process.env.PORT || 4200;

const DEPLOY_URL = getDeployURLFromEnv();

const DIST_FOLDER = join(process.cwd(), 'dist');

// uncomment this block to prevent ssr issues with third-party libraries regarding window, document, HTMLElement and navigator
Expand All @@ -28,8 +30,8 @@ global['HTMLElement'] = win.HTMLElement;
global['navigator'] = win.navigator;
// tslint:enable:no-string-literal
*/

// The Express app is exported so that it can be used by serverless Functions.
// not-dead-code
export function app() {
const logging = /on|1|true|yes/.test(process.env.LOGGING?.toLowerCase());

Expand Down Expand Up @@ -61,7 +63,7 @@ export function app() {
path: '/',
};

const req = https.request(options, res => {
const req = https.request(options, (res: { socket: { authorized: boolean } }) => {
console.log('Certificate for', ICM_BASE_URL, 'authorized:', res.socket.authorized);
});

Expand All @@ -84,7 +86,7 @@ export function app() {
'HOSTNAME_MISMATCH',
];

req.on('error', e => {
req.on('error', (e: { code: string }) => {
if (certErrorCodes.includes(e.code)) {
console.log(
e.code,
Expand Down Expand Up @@ -119,7 +121,7 @@ export function app() {
if (logging) {
server.use(
require('morgan')('tiny', {
skip: req => req.originalUrl.startsWith('/INTERSHOP/static'),
skip: (req: express.Request) => req.originalUrl.startsWith('/INTERSHOP/static'),
})
);
}
Expand Down Expand Up @@ -149,7 +151,18 @@ export function app() {
);
}

// Serve static files from /browser
// Serve static files from browser folder
server.get(/\/.*\.(js|css)$/, (req, res) => {
const path = req.originalUrl.substring(1);
fs.readFile(join(DIST_FOLDER, 'browser', path), { encoding: 'utf-8' }, (err, data) => {
if (err) {
res.sendStatus(404);
} else {
res.set('Content-Type', (path.endsWith('css') ? 'text/css' : 'application/javascript') + '; charset=UTF-8');
res.send(setDeployUrlInFile(DEPLOY_URL, path, data));
}
});
});
server.get(
'*.*',
express.static(join(DIST_FOLDER, 'browser'), {
Expand All @@ -161,14 +174,22 @@ export function app() {
// file should be re-checked more frequently -> 5m
res.set('Cache-Control', 'public, max-age=300');
}
// add cors headers for required resources
if (
DEPLOY_URL.startsWith('http') &&
['manifest.webmanifest', 'woff2', 'woff', 'json'].some(match => path.endsWith(match))
) {
res.set('access-control-allow-origin', '*');
}
},
})
);

const icmProxy = proxy(ICM_BASE_URL, {
// preserve original path
proxyReqPathResolver: req => req.originalUrl,
proxyReqOptDecorator: options => {
proxyReqPathResolver: (req: express.Request) => req.originalUrl,
// tslint:disable-next-line: no-any
proxyReqOptDecorator: (options: any) => {
if (process.env.TRUST_ICM) {
// https://github.com/villadora/express-http-proxy#q-how-to-ignore-self-signed-certificates-
options.rejectUnauthorized = false;
Expand Down Expand Up @@ -227,7 +248,9 @@ export function app() {
}
newHtml = newHtml.replace(/<base href="[^>]*>/, `<base href="${baseHref}" />`);

res.status(res.statusCode).send(newHtml || html);
newHtml = setDeployUrlInFile(DEPLOY_URL, req.originalUrl, newHtml);

res.status(res.statusCode).send(newHtml);
} else {
res.status(500).send(err.message);
}
Expand Down
12 changes: 6 additions & 6 deletions src/app/extensions/seo/store/seo/seo.effects.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,9 @@ export class SeoEffects {
switchMap(() =>
race([
// PRODUCT PAGE
this.productPage$.pipe(map(product => this.baseURL(true) + generateProductUrl(product).substr(1))),
this.productPage$.pipe(map(product => this.baseURL + generateProductUrl(product).substr(1))),
// CATEGORY / FAMILY PAGE
this.categoryPage$.pipe(map(category => this.baseURL(true) + generateCategoryUrl(category).substr(1))),
this.categoryPage$.pipe(map(category => this.baseURL + generateCategoryUrl(category).substr(1))),
// DEFAULT
this.appRef.isStable.pipe(whenTruthy(), mapTo(this.doc.URL.replace(/[;?].*/g, ''))),
])
Expand Down Expand Up @@ -123,7 +123,7 @@ export class SeoEffects {
description: 'seo.defaults.description',
robots: 'index, follow',
'og:type': 'website',
'og:image': `${this.baseURL(false)}assets/img/og-image-default.jpg`.replace('http:', 'https:'),
'og:image': '/assets/img/og-image-default.jpg',
...attributes,
})),
distinctUntilChanged(isEqual),
Expand Down Expand Up @@ -180,12 +180,12 @@ export class SeoEffects {
{ dispatch: false }
);

private baseURL(includeBaseHref: boolean) {
private get baseURL() {
let url: string;
if (this.request) {
url = `${this.request.protocol}://${this.request.get('host')}${includeBaseHref ? this.baseHref : ''}`;
url = `${this.request.protocol}://${this.request.get('host')}${this.baseHref}`;
} else {
url = includeBaseHref ? this.doc.baseURI : this.doc.baseURI.replace(new RegExp(`${this.baseHref}$`), '');
url = this.doc.baseURI;
}
return url.endsWith('/') ? url : url + '/';
}
Expand Down
Loading