feat(multi-pass): support TAA(Temporal Anti-Aliasing)

we can enable TAA in layer's options now, but there're something wrong when picking enabled at same
time.

dev-docs/TAA.md

@@ -0,0 +1,101 @@
+# 在地理场景中应用 TAA
+
+## 问题背景
+
+关于走样产生的原因以及常用的反走样手段，可以参考「知乎 - 反走样技术（一）：几何反走样」[🔗](https://zhuanlan.zhihu.com/p/28800047)。
+我之前也简单总结了下 SSAA、MLAA/SMAA、FXAA 等反走样技术的实现细节。
+
+其中 MSAA 作为浏览器内置实现，开发者使用起来很简单：
+
+> 相对于着色走样，人眼对几何走样更敏感。MSAA 的原理很简单，它仍然把一个像素划分为若干个子采样点，但是相较于 SSAA，每个子采样点的颜色值完全依赖于对应像素的颜色值进行简单的复制（该子采样点位于当前像素光栅化结果的覆盖范围内），不进行单独计算。此外它的做法和 SSAA 相同。由于 MSAA 拥有硬件支持，相对开销比较小，又能很好地解决几何走样问题，在游戏中应用非常广泛（我们在游戏画质选项中常看到的 4x/8x/16x 抗锯齿一般说的就是 MSAA 的子采样点数量分别为4/8/16个）。
+
+下图为 4x MSAA 采样点示意：
+
+![](./screenshots/MSAA.png)
+
+在 Mapbox 中左图未开启 MSAA 而右图选择开启，观察立方体边缘可以发现明显的几何走样：相关 [ISSUE](https://github.com/mapbox/mapbox-gl-js/pull/8474)。
+![](./screenshots/mapbox-MSAA.png)
+
+但是 MSAA 存在一些限制：
+* WebGL1 不支持对 FBO 进行，因此开启 post-processing 后处理时 MSAA 就失效了。当然 WebGL2 支持 🔗。
+* 即使开启，浏览器在某些情况下也不保证应用 🔗。
+
+因此在需要后处理的场景中（例如 L7 的热力图需要 blur pass、PBR 中的 SSAO 环境光遮蔽），只能采用其他反走样手段。
+
+## TAA(Temporal Anti-Aliasing) 原理
+
+来自「知乎 - Experimentalize TAA with no code」🔗：
+
+> 严格来说 TAA 并不能算一个具体的算法，而是更像一个统一的算法框架。和 SSAA 一样，TAA 也能够同时减轻几何走样和着色走样的问题。
+
+关于 TAA 的原理，「GDC - Temporal Reprojection
+Anti-Aliasing in INSIDE」[🔗](http://twvideo01.ubm-us.net/o1/vault/gdc2016/Presentations/Pedersen_LasseJonFuglsang_TemporalReprojectionAntiAliasing.pdf) 讲的十分清晰。如果相机和物体的相对位置在当前帧之前发生过变化，那么当前帧就可以以若干前序帧进行修正。
+
+![](./screenshots/taa-1.png)
+
+但如果在前序帧中相机和物体都没有发生过变化，那对于当前帧就无从修正了。因此可以对视锥进行抖动，在渲染每一帧之前，使用抖动矩阵对投影矩阵进行偏移，最终实现视锥的偏移：
+
+![](./screenshots/taa-step1.png)
+
+然后在 FS 中，最关键的就是 reproject 这一步：
+
+![](./screenshots/taa-step2.png)
+
+对于静止场景，「Three.js - TAA example」[🔗](https://threejs.org/examples/#webgl_postprocessing_taa)、「ECharts.GL - temporalSuperSampling」[🔗](https://echarts.apache.org/zh/option-gl.html#globe.temporalSuperSampling) 都采用了这种方法。
+
+## 实现方法
+
+由于需要对投影矩阵进行抖动，我们需要选取低差异序列。
+来自「知乎 - 低差异序列（一）- 常见序列的定义及性质」🔗，右图明显比左图纯随机生成覆盖面广：
+
+![](./screenshots/halton.png)
+
+参考 Echarts.GL，我们选择 `Halton(2,3)` 低差异序列：
+```typescript
+const offset = this.haltonSequence[this.frame % this.haltonSequence.length];
+this.cameraService.jitterProjectionMatrix(
+  ((offset[0] * 2.0 - 1.0) / width) * jitterScale,
+  ((offset[1] * 2.0 - 1.0) / height) * jitterScale,
+);
+```
+
+在每一帧都会尝试进行累加。如果在连续运动过程中，TAA 的累加过程必然来不及完成，此时只需要输出当前帧原始结果即可，随后尝试继续轮询累加是否完成。因此在累加完成之前，都会输出当前帧未经 TAA 的结果。
+
+最后我们需要进行加权平均，历史帧的权重应当越来越小：
+
+![](./screenshots/taa-step3.png)
+
+这里我们选择当前帧权重为 0.9，历史帧为 0.1，最终的混合结果供后续后处理模块继续处理：
+
+```typescript
+useFramebuffer(this.outputRenderTarget, () => {
+  this.blendModel.draw({
+    uniforms: {
+      u_Opacity: layerStyleOptions.opacity || 1,
+      u_MixRatio: this.frame === 0 ? 1 : 0.9,
+      u_Diffuse1: this.sampleRenderTarget,
+      u_Diffuse2:
+        this.frame === 0
+          ? layer.multiPassRenderer.getPostProcessor().getReadFBO()
+          : this.prevRenderTarget,
+    },
+  });
+});
+```
+
+## 最终效果
+
+为了更直观地看到效果，在 DEMO 中我们可以调节相机抖动范围：
+
+![](./screenshots/taa-result.gif)
+
+## 参考资料
+
+* 「知乎 - 反走样技术（一）：几何反走样」[🔗](https://zhuanlan.zhihu.com/p/28800047)
+* 「知乎 - Experimentalize TAA with no code」[🔗](https://zhuanlan.zhihu.com/p/41642855)
+* 「ECharts.GL - temporalSuperSampling」[🔗](https://echarts.apache.org/zh/option-gl.html#globe.temporalSuperSampling)
+* 「Mapbox - set custom layers and extrusion examples to use antialias: true」[🔗](https://github.com/mapbox/mapbox-gl-js/pull/8474)
+* 「Three.js - TAA example」[🔗](https://threejs.org/examples/#webgl_postprocessing_taa)
+* 「Paper - Amortized Supersampling」[🔗](http://hhoppe.com/supersample.pdf)
+* 「GDC - Temporal Reprojection Anti-Aliasing in INSIDE」[🔗](http://twvideo01.ubm-us.net/o1/vault/gdc2016/Presentations/Pedersen_LasseJonFuglsang_TemporalReprojectionAntiAliasing.pdf)
+* 「知乎 - 低差异序列（一）- 常见序列的定义及性质」[🔗](https://zhuanlan.zhihu.com/p/20197323)

packages/core/src/index.ts

@@ -1,12 +1,12 @@
-import container, { lazyInject } from './inversify.config';
+import container, { lazyInject, lazyMultiInject } from './inversify.config';
 import ClearPass from './services/renderer/passes/ClearPass';
 import MultiPassRenderer from './services/renderer/passes/MultiPassRenderer';
 import PixelPickingPass from './services/renderer/passes/PixelPickingPass';
 import BlurHPass from './services/renderer/passes/post-processing/BlurHPass';
 import BlurVPass from './services/renderer/passes/post-processing/BlurVPass';
 import CopyPass from './services/renderer/passes/post-processing/CopyPass';
 import RenderPass from './services/renderer/passes/RenderPass';
-import SceneService from './services/scene/SceneService';
+import TAAPass from './services/renderer/passes/TAAPass';
 import { TYPES } from './types';
 import { packCircleVertex } from './utils/vertex-compression';
 
@@ -19,14 +19,11 @@ export {
    * lazy inject，供各个 Layer 使用
    */
   lazyInject,
+  lazyMultiInject,
   /**
    * 各个 Service 接口标识符
    */
   TYPES,
-  /**
-   * 各个 Service 接口
-   */
-  SceneService,
   packCircleVertex,
   /** pass */
   MultiPassRenderer,
@@ -36,6 +33,7 @@ export {
   BlurHPass,
   BlurVPass,
   CopyPass,
+  TAAPass,
 };
 
 /** 暴露服务接口供其他 packages 实现 */

packages/core/src/inversify.config.ts

@@ -16,6 +16,7 @@ import { IInteractionService } from './services/interaction/IInteractionService'
 import { ILayerService } from './services/layer/ILayerService';
 import { IStyleAttributeService } from './services/layer/IStyleAttributeService';
 import { ILogService } from './services/log/ILogService';
+import { ISceneService } from './services/scene/ISceneService';
 import { IShaderModuleService } from './services/shader/IShaderModuleService';
 
 /** Service implements */
@@ -29,13 +30,18 @@ import InteractionService from './services/interaction/InteractionService';
 import LayerService from './services/layer/LayerService';
 import StyleAttributeService from './services/layer/StyleAttributeService';
 import LogService from './services/log/LogService';
+import SceneService from './services/scene/SceneService';
 import ShaderModuleService from './services/shader/ShaderModuleService';
 // @see https://github.com/inversify/InversifyJS/blob/master/wiki/container_api.md#defaultscope
 const container = new Container();
 
 /**
  * bind services
  */
+container
+  .bind<ISceneService>(TYPES.ISceneService)
+  .to(SceneService)
+  .inSingletonScope();
 container
   .bind<IGlobalConfigService>(TYPES.IGlobalConfigService)
   .to(GlobalConfigService)
@@ -113,5 +119,25 @@ export const lazyInject = (
     };
   };
 };
+export const lazyMultiInject = (
+  serviceIdentifier: interfaces.ServiceIdentifier<any>,
+) => {
+  const original = DECORATORS.lazyMultiInject(serviceIdentifier);
+  // the 'descriptor' parameter is actually always defined for class fields for Babel, but is considered undefined for TSC
+  // so we just hack it with ?/! combination to avoid "TS1240: Unable to resolve signature of property decorator when called as an expression"
+  return function(
+    this: any,
+    proto: any,
+    key: string,
+    descriptor?: IBabelPropertyDescriptor,
+  ): void {
+    // make it work as usual
+    original.call(this, proto, key);
+    // return link to proto, so own value wont be 'undefined' after component's creation
+    descriptor!.initializer = () => {
+      return proto[key];
+    };
+  };
+};
 
 export default container;

packages/core/src/services/camera/CameraService.ts

@@ -1,3 +1,4 @@
+import { mat4 } from 'gl-matrix';
 import { inject, injectable } from 'inversify';
 import { ICameraService, IViewport } from './ICameraService';
 
@@ -10,6 +11,16 @@ export default class CameraService implements ICameraService {
    */
   private overridedViewProjectionMatrix: number[] | undefined;
 
+  /**
+   * 抖动后的 VP 矩阵
+   */
+  private jitteredViewProjectionMatrix: number[] | undefined;
+
+  /**
+   * 抖动后的 Projection 矩阵
+   */
+  private jitteredProjectionMatrix: number[] | undefined;
+
   public init() {
     //
   }
@@ -22,7 +33,8 @@ export default class CameraService implements ICameraService {
   }
 
   public getProjectionMatrix(): number[] {
-    return this.viewport.getProjectionMatrix();
+    // 优先返回抖动后的 ProjectionMatrix
+    return this.jitteredProjectionMatrix || this.viewport.getProjectionMatrix();
   }
 
   public getViewMatrix(): number[] {
@@ -36,6 +48,7 @@ export default class CameraService implements ICameraService {
   public getViewProjectionMatrix(): number[] {
     return (
       this.overridedViewProjectionMatrix ||
+      this.jitteredViewProjectionMatrix ||
       this.viewport.getViewProjectionMatrix()
     );
   }
@@ -70,4 +83,25 @@ export default class CameraService implements ICameraService {
   public setViewProjectionMatrix(viewProjectionMatrix: number[] | undefined) {
     this.overridedViewProjectionMatrix = viewProjectionMatrix;
   }
+
+  public jitterProjectionMatrix(x: number, y: number) {
+    const translation = mat4.fromTranslation(mat4.create(), [x, y, 0]);
+
+    this.jitteredProjectionMatrix = (mat4.multiply(
+      mat4.create(),
+      translation,
+      (this.viewport.getProjectionMatrix() as unknown) as mat4,
+    ) as unknown) as number[];
+
+    this.jitteredViewProjectionMatrix = (mat4.multiply(
+      mat4.create(),
+      (this.jitteredProjectionMatrix as unknown) as mat4,
+      (this.viewport.getViewMatrix() as unknown) as mat4,
+    ) as unknown) as number[];
+  }
+
+  public clearJitterProjectionMatrix() {
+    this.jitteredProjectionMatrix = undefined;
+    this.jitteredViewProjectionMatrix = undefined;
+  }
 }

packages/core/src/services/camera/ICameraService.ts

@@ -31,4 +31,6 @@ export interface ICameraService extends Omit<IViewport, 'syncWithMapCamera'> {
   init(): void;
   update(viewport: IViewport): void;
   setViewProjectionMatrix(viewProjectionMatrix: number[] | undefined): void;
+  jitterProjectionMatrix(x: number, y: number): void;
+  clearJitterProjectionMatrix(): void;
 }

packages/core/src/services/layer/ILayerService.ts

@@ -107,6 +107,14 @@ export interface ILayerInitializationOptions {
    * 高亮颜色
    */
   highlightColor: string | number[];
+  /**
+   * 开启 TAA
+   */
+  enableTAA: boolean;
+  /**
+   * 相机抖动程度
+   */
+  jitterScale: number;
   onHover(pickedFeature: IPickedFeature): void;
   onClick(pickedFeature: IPickedFeature): void;
 }

packages/core/src/services/renderer/passes/BasePostProcessingPass.ts

@@ -93,19 +93,6 @@ export default class BasePostProcessingPass<InitializationOptions = {}>
         });
       },
     );
-
-    // const useRenderTarget = (this.renderToScreen
-    //   ? postProcessor.useScreenRenderTarget
-    //   : postProcessor.useOffscreenRenderTarget
-    // ).bind(postProcessor);
-
-    // useRenderTarget(async () => {
-    //   this.model.draw({
-    //     uniforms: {
-    //       u_Texture: postProcessor.getReadFBO(),
-    //     },
-    //   });
-    // });
   }
 
   public isEnabled() {

packages/core/src/services/renderer/passes/PixelPickingPass.ts

@@ -104,9 +104,9 @@ export default class PixelPickingPass implements IPass {
       const originRenderFlag = this.layer.multiPassRenderer.getRenderFlag();
       this.layer.multiPassRenderer.setRenderFlag(false);
       // trigger hooks
-      layer.hooks.beforeRender.call();
+      layer.hooks.beforePickingEncode.call();
       layer.render();
-      layer.hooks.afterRender.call();
+      layer.hooks.afterPickingEncode.call();
       this.layer.multiPassRenderer.setRenderFlag(originRenderFlag);
 
       this.alreadyInRendering = false;
@@ -207,15 +207,28 @@ export default class PixelPickingPass implements IPass {
    */
   private highlightPickedFeature(pickedColors: Uint8Array | undefined) {
     const [r, g, b] = pickedColors;
+    const { clear, useFramebuffer } = this.rendererService;
 
-    // TODO: highlight pass 需要 multipass
-    const originRenderFlag = this.layer.multiPassRenderer.getRenderFlag();
-    this.layer.multiPassRenderer.setRenderFlag(false);
+    // 先输出到 PostProcessor
+    const readFBO = this.layer.multiPassRenderer.getPostProcessor().getReadFBO();
     this.layer.hooks.beforeRender.call();
-    this.layer.hooks.beforeHighlight.call([r, g, b]);
-    this.layer.render();
-    this.layer.hooks.afterHighlight.call();
-    this.layer.hooks.afterRender.call();
-    this.layer.multiPassRenderer.setRenderFlag(originRenderFlag);
+    useFramebuffer(readFBO, () => {
+      clear({
+        color: [0, 0, 0, 0],
+        depth: 1,
+        stencil: 0,
+        framebuffer: readFBO,
+      });
+
+      // TODO: highlight pass 需要 multipass
+      const originRenderFlag = this.layer.multiPassRenderer.getRenderFlag();
+      this.layer.multiPassRenderer.setRenderFlag(false);
+      this.layer.hooks.beforeHighlight.call([r, g, b]);
+      this.layer.render();
+      this.layer.hooks.afterHighlight.call();
+      this.layer.hooks.afterRender.call();
+      this.layer.multiPassRenderer.setRenderFlag(originRenderFlag);
+    });
+    this.layer.multiPassRenderer.getPostProcessor().render(this.layer);
   }
 }

packages/core/src/services/renderer/passes/PostProcessor.ts

@@ -52,6 +52,7 @@ export default class PostProcessor implements IPostProcessor {
   public async render(layer: ILayer) {
     for (let i = 0; i < this.passes.length; i++) {
       const pass = this.passes[i];
+
       // last pass should render to screen
       pass.setRenderToScreen(this.isLastEnabledPass(i));
       await pass.render(layer);