Skip to content

Commit

Permalink
sync with upstream 4.2.0
Browse files Browse the repository at this point in the history
  • Loading branch information
quipo committed Apr 12, 2024
2 parents 0edf9b4 + 3c5f743 commit 03f2dd1
Show file tree
Hide file tree
Showing 1,068 changed files with 44,324 additions and 35,632 deletions.
2 changes: 0 additions & 2 deletions .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,3 @@ modules/flatgeobuf/src/flatgeobuf*
modules/potree/test/data/**/cloud.js
modules/tiles/src/bundle.ts
modules/zarr

modules/tile-converter/
13 changes: 9 additions & 4 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,23 +14,28 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18, 20]
node-version: [20]

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4.1.1
- uses: volta-cli/action@v4

- name: Set up Node ${{ matrix.node-version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4.0.0
with:
node-version: ${{ matrix.node-version }}

- name: Bootstrap
run: |
yarn
- name: Bootstrap
run: |
yarn bootstrap
- name: Run tests
run: |
npm run test ci
npm run test-ci
- name: Coveralls
if: matrix.node-version == 18
Expand Down
16 changes: 14 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,23 @@ dist.min.js
# lock files
package-lock.json
*/**/package-lock.json
examples/**/yarn.lock

*/**/yarn.lock
!website/yarn.lock

*/**/.yarn/*
*/**/yarn-error.log

.yarn/*
!.yarn/cache
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions

tsconfig.tsbuildinfo

.reify-cache
.nyc_output/
coverage/
node_modules/
Expand Down
50 changes: 36 additions & 14 deletions .ocularrc.js
Original file line number Diff line number Diff line change
@@ -1,37 +1,59 @@
import {resolve} from 'path';
/** @typedef {import('ocular-dev-tools').OcularConfig} OcularConfig */

export default {
aliases: {
test: resolve('./test')
import {dirname, join} from 'path';
import {fileURLToPath} from 'url';

const packageRoot = dirname(fileURLToPath(import.meta.url));
const devModules = join(packageRoot, 'dev-modules');
const testDir = join(packageRoot, 'test');

/** @type {OcularConfig} */
const config = {
babel: false,

lint: {
paths: ['modules', 'dev-docs', 'docs'], // , 'test', 'examples'],
extensions: ['js', 'ts']
// extensions: ['js', 'jsx', 'mjs', 'ts', 'tsx', 'md']
},

typescript: {
project: 'tsconfig.build.json'
},

aliases: {
// TEST
test: testDir
},

coverage: {
test: 'browser'
},

bundle: {
globalName: 'loader',
globalName: 'loaders',
externals: ['fs', 'path', 'util', 'events', 'stream', 'crypto', 'http', 'https'],
target: ['supports async-functions', 'not dead'],
target: ['chrome110', 'firefox110', 'safari15'],
format: 'umd',
globals: {
'@loaders.gl/*': 'globalThis.loaders'
}
},

lint: {
// TODO - comment out while getting typescript to work
paths: ['dev-docs', 'docs', 'modules'] // 'examples', test', 'website', 'examples'],
// extensions: ['js', 'jsx', 'mjs', 'ts', 'tsx', 'md']
},

webpack: {},

entry: {
test: 'test/node.ts',
'test-browser': 'test/browser.ts',
bench: 'test/bench/node.js',
'bench-browser': 'test/bench/browser.js',
size: 'test/size/import-nothing.js'
}
// entry: {
// test: 'test/index.ts',
// 'test-browser': 'test/browser.ts',
// bench: 'test/bench/index.js',
// 'bench-browser': 'test/bench/browser.js',
// size: 'test/size/import-nothing.js'
// }
};

export default config;
1 change: 1 addition & 0 deletions .yarnrc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
nodeLinker: node-modules
89 changes: 88 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,93 @@
# CHANGELOG for loaders.gl

## v4.2 (alpha)
## v4.2

### v4.2.0

### v4.2.0-beta.2

- fix(parquet): fix stack overflow for large datasets (#2960)
- fix(parquet): incorrect dictionary loading (#2959)
- fix(loader-utils): Do not require dataType and batchType on loaders. (#2961)

### v4.2.0-beta.1

- chore(worker-utils): Use polyfills in worker-utils to avoid build warnings (#2953)
- docs: Polish release notes (#2957)
- fix(core): Case insensitive MIME type comparison (#2956)
- feat(3d-tiles): Test cleanup (#2954)
- chore(draco): Clean up loader (#2955)
- fix(wms): Use correct featureInfo parameters for WMS 1.3 (#2952)
- fix(draco): Ignore unknown attribute types (#2951)
- feat: Add options.<loader>.workerUrl (#2950)
- Throws more useful error when fetch fails (#2921)
- docs: update whats-new (#2946)
- fix(csv): CSVLoader - narrow result types (#2945)
- chore: Bump to 4.2-beta (#2948)
- fix(examples): Fix texture example (+ website) (#2943)
- fix(parquet): zstd decompression (#2937)
- chore(compression/parquet): Improved handling of injected modules (#2940)
- chore: Use `const satisfies on Loaders and Writers (#2938)
- Fix laslaz-decoder.ts: free memory after laz file parsed. (#2933)

### v4.2.0-alpha.6

- docs: migration info loaders v4 (#2908)
- feat(i3s): local slpk rendering example (#2906)
- website: new hero example (#2907)
- chore(lerc): Restore lerc module (#2912)
- fix(parquet): Restore ParquetWASMLoader after esbuild upgrade (#2915)
- fix(*): Fix transitive internal dependencies (#2919)
- fix(parquet): Zstd-codec import turned on (#2922)
- chore(deps): bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /website (#2926)
- chore(deps): bump follow-redirects from 1.15.4 to 1.15.6 in /website (#2920)
- fix (parquet): Increased max dictionary size to the max size of ArrayBuffer (#2924)
- chore(deps): bump katex from 0.16.9 to 0.16.10 in /website (#2927)
- fix(tile-converter): --install-dependencies don't raise error (#2925)
- chore(deps): bump express from 4.18.2 to 4.19.2 in /website (#2931)
- chore: fix eslint issues (#2923)
- chore: Upgrade website and examples to deck.gl v9 (#2932)
- chore(deps): bump express from 4.18.2 to 4.19.2 (#2930)
- chore: Upgrade to yarn 4 (#2917)
- fix(gltf): Support models where some TextureInfo definitions use KHR_texture_transform and others do not. (#2928)
- fix(wkt): Increased amount of memory for polygon indices (#2934)

### v4.2.0-alpha.5

- chore: Upgrade ocular-dev-tools to tsc+esbuild (#2897)
- fix(core): Avoid reassigning loaders._state (#2905)
- feat(loader-utils): Add RequestScheduler 'debounceTime' option (#2892)
- chore: Port I3S examples to v9 (#2903)
- chore: Move examples to deck.gl@9.0.0-beta.5 (#2901)
- chore(examples): loading of slpk in browser experimental(#2904)
- feat(pmtiles): Upgrade to pmtiles v3 (#2902)
- chore(docs): Add I3S Picking Tutorial (#2896)
- chore(examples): delete i3s example (#2899)
- chore(docs): Add live example into I3S BSL tutorial (#2898)
- chore(docs): Update i3s colorization tutorial (#2900)

### v4.2.0-alpha.4

- fix(tile-converter): compose SLPK hash file (#2895)
- chore(i3s examples): Add i3s picking example (#2886)

### v4.2.0-alpha.3

- feat(tile-converter): quiet mode (#2891)
- feat(parquet): restore ParquetWasm loader (#2868)
- feat(tile-converter): 3d-tiles-converter progress (#2890)

### v4.2.0-alpha.2

- feat(tile-converter): convert SLPK (#2888)
- feat(tile-converter): conversion dump json validation (#2883)
- chore(examples): Rewrite I3S BSL example with React FC (#2885)
- fix(flatgeobuf): Add missing index files (#2884)
- feat(tile-converter): add 3d-tiles-converter resume (#2882)

### v4.2.0-alpha.1

- feat(tile-converter): add conversion resume (#2869)

## v4.1

Expand Down
2 changes: 1 addition & 1 deletion docs/README.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<br />
<br />

This documentation describes loaders.gl **v4.0**. See our [**release notes**](/docs/whats-new) to learn what is new.
This documentation describes loaders.gl **v4**. See our [**release notes**](/docs/whats-new) to learn what is new.

Docs for older versions are available on github:
**[v3.3](https://github.com/visgl/loaders.gl/blob/3.3-release/docs/README.md)**,
Expand Down
34 changes: 17 additions & 17 deletions docs/arrowjs/developer-guide/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,30 +25,30 @@ Each data type falls into one of three rough categories: Fixed-width types, vari

Fixed-width data types describe physical primitive values (bytes or bits of some fixed size), or logical values that can be represented as primitive values. In addition to an optional [`Uint8Array`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array) validity bitmask, these data types have a physical data buffer (a [`TypedArray`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray#TypedArray_objects) corresponding to the data type's physical element width).

* [Null](/docs/arrowjs/developer-guide/data-types#null) - A column of NULL values having no physical storage
* [Bool](/docs/arrowjs/developer-guide/data-types#bool) - Booleans as either 0 or 1 (bit-packed, LSB-ordered)
* [Int](/docs/arrowjs/developer-guide/data-types#int) - Signed or unsigned 8, 16, 32, or 64-bit little-endian integers
* [Float](/docs/arrowjs/developer-guide/data-types#float) - 2, 4, or 8-byte floating point values
* [Decimal](/docs/arrowjs/developer-guide/data-types#decimal) - Precision-and-scale-based 128-bit decimal values
* [FixedSizeBinary](/docs/arrowjs/developer-guide/data-types#fixedsizebinary) - A list of fixed-size binary sequences, where each value occupies the same number of bytes
* [Date](/docs/arrowjs/developer-guide/data-types#date) - Date as signed 32-bit integer days or 64-bit integer milliseconds since the UNIX epoch
* [Time](/docs/arrowjs/developer-guide/data-types#time) - Time as signed 32 or 64-bit integers, representing either seconds, millisecond, microseconds, or nanoseconds since midnight (00:00:00)
* [Timestamp](/docs/arrowjs/developer-guide/data-types#timestamp) - Exact timestamp as signed 64-bit integers, representing either seconds, milliseconds, microseconds, or nanoseconds since the UNIX epoch
* [Interval](/docs/arrowjs/developer-guide/data-types#interval) - Time intervals as pairs of either (year, month) or (day, time) in SQL style
* [FixedSizeList](/docs/arrowjs/developer-guide/data-types#fixedsizelist) - Fixed-size sequences of another logical Arrow data type
* [Null](/docs/arrowjs/developer-guide/data-types) - A column of NULL values having no physical storage
* [Bool](/docs/arrowjs/developer-guide/data-types) - Booleans as either 0 or 1 (bit-packed, LSB-ordered)
* [Int](/docs/arrowjs/developer-guide/data-types) - Signed or unsigned 8, 16, 32, or 64-bit little-endian integers
* [Float](/docs/arrowjs/developer-guide/data-types) - 2, 4, or 8-byte floating point values
* [Decimal](/docs/arrowjs/developer-guide/data-types) - Precision-and-scale-based 128-bit decimal values
* [FixedSizeBinary](/docs/arrowjs/developer-guide/data-types) - A list of fixed-size binary sequences, where each value occupies the same number of bytes
* [Date](/docs/arrowjs/developer-guide/data-types) - Date as signed 32-bit integer days or 64-bit integer milliseconds since the UNIX epoch
* [Time](/docs/arrowjs/developer-guide/data-types) - Time as signed 32 or 64-bit integers, representing either seconds, millisecond, microseconds, or nanoseconds since midnight (00:00:00)
* [Timestamp](/docs/arrowjs/developer-guide/data-types) - Exact timestamp as signed 64-bit integers, representing either seconds, milliseconds, microseconds, or nanoseconds since the UNIX epoch
* [Interval](/docs/arrowjs/developer-guide/data-types) - Time intervals as pairs of either (year, month) or (day, time) in SQL style
* [FixedSizeList](/docs/arrowjs/developer-guide/data-types) - Fixed-size sequences of another logical Arrow data type

### Variable-width Data Types

Variable-width types describe lists of values with different widths, including binary blobs, Utf8 code-points, or slices of another underlying Arrow data type. These types store the values contiguously in memory, and have a physical [`Int32Array`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Int32Array) of offsets that describe the start and end indicies of each list element.

* [List](/docs/arrowjs/developer-guide/data-types#list) - Variable-length sequences of another logical Arrow data type
* [Utf8](/docs/arrowjs/developer-guide/data-types#utf8) - Variable-length byte sequences of UTF8 code-points (strings)
* [Binary](/docs/arrowjs/developer-guide/data-types#binary) - Variable-length byte sequences (no guarantee of UTF8-ness)
* [List](/docs/arrowjs/developer-guide/data-types) - Variable-length sequences of another logical Arrow data type
* [Utf8](/docs/arrowjs/developer-guide/data-types) - Variable-length byte sequences of UTF8 code-points (strings)
* [Binary](/docs/arrowjs/developer-guide/data-types) - Variable-length byte sequences (no guarantee of UTF8-ness)

### Composite Data Types

Composite types don't have physical data buffers of their own. They contain other Arrow data types and delegate work to them.

* [Union](/docs/arrowjs/developer-guide/data-types#union) - Union of logical child data types
* [Map](/docs/arrowjs/developer-guide/data-types#map) - Map of named logical child data types
* [Struct](/docs/arrowjs/developer-guide/data-types#struct) - Struct of ordered logical child data types
* [Union](/docs/arrowjs/developer-guide/data-types) - Union of logical child data types
* [Map](/docs/arrowjs/developer-guide/data-types) - Map of named logical child data types
* [Struct](/docs/arrowjs/developer-guide/data-types) - Struct of ordered logical child data types
15 changes: 5 additions & 10 deletions docs/docs-sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,7 @@
{
"type": "category",
"label": "Overview",
"items": [
"README",
"developer-guide/get-started",
"whats-new",
"upgrade-guide"
]
"items": ["README", "developer-guide/get-started", "whats-new", "upgrade-guide"]
},
{
"type": "category",
Expand All @@ -29,9 +24,7 @@
{
"type": "category",
"label": "Sources and Services",
"items": [
"modules/pmtiles/api-reference/pmtiles-source"
]
"items": ["modules/pmtiles/api-reference/pmtiles-source"]
},
"modules/3d-tiles/api-reference/tiles-3d-loader",
"modules/3d-tiles/api-reference/cesium-ion-loader",
Expand Down Expand Up @@ -101,6 +94,7 @@
"modules/arrow/formats/geoarrow",
"modules/bson/formats/bson",
"modules/csv/formats/csv",
"modules/draco/formats/draco",
"modules/pcd/formats/pcd",
"modules/ply/formats/ply",
"modules/flatgeobuf/formats/flatgeobuf",
Expand Down Expand Up @@ -183,7 +177,8 @@
"label": "I3S",
"items": [
"modules/i3s/recipes/building-scene-layer",
"modules/i3s/recipes/attribute-driven-colorization"
"modules/i3s/recipes/attribute-driven-colorization",
"modules/i3s/recipes/picking"
]
}
]
Expand Down
26 changes: 17 additions & 9 deletions docs/modules/draco/api-reference/draco-loader.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,30 @@

The `DracoLoader` decodes a mesh or point cloud (maps of attributes) using [DRACO](https://google.github.io/draco/) compression.

| Loader | Characteristic |
| -------------- | -------------------------------------------- |
| File Extension | `.drc` |
| File Type | Binary |
| File Format | [Draco](https://google.github.io/draco/) |
| Loader | Characteristic |
| -------------- | ------------------------------------------ |
| File Format | [Draco](/docs/modules/draco/formats/draco) |
| Data Format | [Mesh](/docs/specifications/category-mesh) |
| Supported APIs | `parse` |
| File Extension | `.drc` |
| File Type | Binary |
| Supported APIs | `parse` |

Features:
## Support

For detailed information

General:
- Supports meshes and point clouds.
- Loads draco decoders dynamically from CDN (can optionally be bundled).
- Supports meshes and point clouds.

Attributes:
- Supports custom attributes.
- Ability to prevent decompression of specific attributes (returns quantization or octahedron transform parameters, if application wishes to perform decompression on GPU).

Metadata Support:
- Extracts metadata dictionaries, both for the full mesh and for each attribute.
- Supports all Draco metadata field types, including `Int32Array`.
- Loads draco decoders dynamically from CDN (can optionally be bundled).
- Ability to prevent decompression of specific attributes (returns quantization or octahedron transform parameters).

## Usage

Expand Down
10 changes: 7 additions & 3 deletions docs/modules/draco/api-reference/draco-writer.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,20 @@

![logo](../images/draco-small.png)

The `DracoWriter` encodes a mesh or point cloud using [Draco](https://google.github.io/draco/) compression.
The `DracoWriter` encodes a mesh or point cloud using [Draco](/docs/modules/draco/formats/draco) compression.

| Loader | Characteristic |
| -------------- | -------------------------------------------- |
| File Format | [Draco](/docs/modules/draco/formats/draco) |
| Data Format | [Mesh](/docs/specifications/category-mesh) |
| File Extension | `.drc` |
| File Type | Binary |
| File Format | [Draco](https://google.github.io/draco/) |
| Data Format | [Mesh](/docs/specifications/category-mesh) |
| Support API | `encode` |

## Support

See [Draco](/docs/modules/draco/formats/draco) docs.

## Usage

```typescript
Expand Down
Loading

0 comments on commit 03f2dd1

Please sign in to comment.