Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(website): Add GeoArrow example #2802

Merged
merged 4 commits into from
Nov 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/docs-sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@
"modules/3d-tiles/api-reference/cesium-ion-loader",
"modules/arrow/api-reference/arrow-loader",
"modules/arrow/api-reference/arrow-writer",
"modules/arrow/api-reference/geoarrow-loader",
"modules/bson/api-reference/bson-loader",
"modules/bson/api-reference/bson-writer",
"modules/csv/api-reference/csv-loader",
Expand Down
4 changes: 4 additions & 0 deletions docs/modules/arrow/api-reference/geoarrow-loader.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

![arrow-logo](../images/apache-arrow-small.png)

<p class="badges">
<img src="https://img.shields.io/badge/From-v4.1-blue.svg?style=flat-square" alt="From-v4.1" />
</p>

The `GeoArrowLoader` parses Apache Arrow columnar table format files, and looks for `GeoArrow` type extensions to parse geometries from the table.

| Loader | Characteristic |
Expand Down
28 changes: 14 additions & 14 deletions docs/modules/arrow/formats/geoarrow.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,22 +22,22 @@ Geospatial tabular data where one or more columns contains feature geometries an
Note that GeoArrow is not a separate format from Apache Arrow rather, the GeoArrow specification simply describes additional conventions for metadata and layout of geospatial data. This means that a valid GeoArrow file is always a valid Arrow file. This is done through [Arrow extension type](https://arrow.apache.org/docs/format/Columnar.html#extension-types) definitions that ensure type-level metadata (e.g., CRS) is propagated when used in Arrow implementations.


## Geometry Types

| Geometry type | Read | Write | Description |
| -------------------------- | ---- | ----- | -------------------- |
| `geoarrow.point` | ✅ | ❌ | |
| `geoarrow.multipoint` | ✅ | ❌ | |
| `geoarrow.linestring` | ✅ | ❌ | |
| `geoarrow.multilinestring` | ✅ | ❌ | |
| `geoarrow.polygon` | ✅ | ❌ | |
| `geoarrow.multipolygon` | ✅ | ❌ | |
| `geoarrow.wkb` | ✅ | ❌ | `WKB` also supported |
| `geoarrow.wkt` | ✅ | ❌ | `WKT` also supported |

## Relationship with GeoParquet

The [GeoParquet specification](https://github.com/opengeospatial/geoparquet) is closely related to GeoArrow. Notable differences:
The [GeoParquet](/docs/modules/parquet/formats/geoparquet) [specification](https://github.com/opengeospatial/geoparquet) is closely related to GeoArrow. Notable differences:

- GeoParquet is a file-level metadata specification
- GeoArrow is a field-level metadata and memory layout specification

## Geometry Types

| Geometry type | Read | Write | Description |
| -------------------------- | ---- | ----- | ----------- |
| `geoarrow.multipolygon` | ✅ | ❌ | |
| `geoarrow.polygon` | ✅ | ❌ | |
| `geoarrow.multipoint` | ✅ | ❌ | |
| `geoarrow.point` | ✅ | ❌ | |
| `geoarrow.multilinestring` | ✅ | ❌ | |
| `geoarrow.linestring` | ✅ | ❌ | |
| `geoarrow.wkb` | ❌ | ❌ | |
| `geoarrow.wkt` | ❌ | ❌ | |
103 changes: 50 additions & 53 deletions examples/website/geospatial/app.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@ import React, {useState, useEffect} from 'react';
import {createRoot} from 'react-dom/client';

import {Map} from 'react-map-gl';
import maplibregl from 'maplibre-gl';
import maplibregl, {Properties} from 'maplibre-gl';

import {DeckGL} from '@deck.gl/react/typed';
import {MapController} from '@deck.gl/core/typed';
import {GeoJsonLayer} from '@deck.gl/layers/typed';

import {ControlPanel} from './components/control-panel';
import {FileUploader} from './components/file-uploader';
// import {FileUploader} from './components/file-uploader';

import type {Example} from './examples';
import {INITIAL_LOADER_NAME, INITIAL_EXAMPLE_NAME, INITIAL_MAP_STYLE, EXAMPLES} from './examples';
Expand All @@ -22,12 +22,12 @@ import {Loader, load /* registerLoaders */} from '@loaders.gl/core';
import {ParquetLoader, installBufferPolyfill} from '@loaders.gl/parquet';
import {FlatGeobufLoader} from '@loaders.gl/flatgeobuf';
// import {GeoPackageLoader} from '@loaders.gl/geopackage';
import {ArrowLoader} from '@loaders.gl/arrow';
import {GeoArrowLoader} from '@loaders.gl/arrow';

installBufferPolyfill();

const LOADERS: Loader[] = [
ArrowLoader,
GeoArrowLoader,
ParquetLoader,
FlatGeobufLoader
// GeoPackageLoader
Expand Down Expand Up @@ -76,15 +76,14 @@ type AppState = {
};

/**
*
* A Geospatial table map viewer
*/
export default function App(props: AppProps) {

const [state, setState] = useState<AppState>({
// EXAMPLE STATE
examples: EXAMPLES,
selectedExample: INITIAL_EXAMPLE_NAME,
selectedLoader: INITIAL_LOADER_NAME,
selectedExample: null,
selectedLoader: null,

// CURRENT VIEW POINT / CAMERA POSITION
viewState: INITIAL_VIEW_STATE,
Expand All @@ -96,74 +95,75 @@ export default function App(props: AppProps) {
useEffect(() => {
let examples: Record<string, Record<string, Example>> = {...EXAMPLES};
if (props.format) {
// Move the preferred format examples to the "top"
examples = {[props.format]: EXAMPLES[props.format], ...EXAMPLES};
// Remove any keys
for (const key of Object.keys(examples)) {
if (key.endsWith('Test')) {
delete examples[key];
}
}
// Keep only the preferred format examples
examples = {[props.format]: EXAMPLES[props.format]};
}

const selectedLoader = props.format || INITIAL_LOADER_NAME;
let selectedExample = props.format
? Object.keys(examples[selectedLoader])[0]
: INITIAL_EXAMPLE_NAME;

let selectedExample = INITIAL_EXAMPLE_NAME;
if (props.format) {
for (const exampleName of Object.keys(examples[selectedLoader])) {
selectedExample = exampleName;
break;
}
}
setState({...state, examples, selectedExample, selectedLoader});
onExampleChange({selectedLoader, selectedExample, example: examples[selectedLoader][selectedExample], state, setState});
setState(state => ({...state, examples, selectedExample, selectedLoader}));
}, [props.format]);

let schema = state.loadedTable?.schema ? {metadata: state.loadedTable?.schema.metadata, ... state.loadedTable?.schema} : null;

return (
<div style={{position: 'relative', height: '100%'}}>

<ControlPanel
schema={schema && JSON.stringify(schema, null, 2)}
examples={state.examples}
selectedExample={state.selectedExample}
selectedLoader={state.selectedLoader}
onExampleChange={(props) =>onExampleChange({...props, state, setState})}
onExampleChange={(props) => onExampleChange({...props, state, setState})}
>
{state.error ? <div style={{color: 'red'}}>{state.error}</div> : ''}
<div style={{textAlign: 'center'}}>
center long/lat: {state.viewState.longitude.toFixed(3)},
{state.viewState.latitude.toFixed(3)},
zoom: {state.viewState.zoom.toFixed(2)}
{state.viewState.latitude.toFixed(3)}, zoom: {state.viewState.zoom.toFixed(2)}
</div>
<FileUploader
onFileRemoved={() => setState({...state, loadedTable: null})}
onFileSelected={async (uploadedFile: File) => {
{
/* TODO -restore drag and drop
<FileUploader
onFileRemoved={() => setState(state => ({...state, loadedTable: null}))}
onFileSelected={async (uploadedFile: File) => {
// TODO - error handling
const data = (await load(uploadedFile, LOADERS, LOADER_OPTIONS)) as Table;
setState({
setState(state => ({
...state,
selectedExample: uploadedFile.name,
loadedTable: data
});
}}
}));
}}
/>
*/}
</ControlPanel>

<DeckGL
layers={renderLayer(state)}
viewState={state.viewState}
onViewStateChange={({viewState}) => setState({...state, viewState})}
onViewStateChange={({viewState}) => setState(state => ({...state, viewState}))}
controller={{type: MapController, maxPitch: 85}}
getTooltip={({object}) =>
object && {
html: `\
<h2>${object.properties?.name}</h2>
<div>${object.geometry?.coordinates?.[0]}</div>
<div>${object.geometry?.coordinates?.[1]}</div>`,
style: {
backgroundColor: '#ddd',
fontSize: '0.8em'
getTooltip={({object}) => {
const {name, ...properties} = object?.properties || {};
const props = Object.entries(properties)
.map(([key, value]) => `<div>${key}: ${value}</div>`)
.join('\n');
return (
object && {
html: `\
<h2>${name}</h2>
${props}
<div>Coords: ${object.geometry?.coordinates?.[0]};${object.geometry?.coordinates?.[1]}</div>`,
style: {
backgroundColor: '#ddd',
fontSize: '0.8em'
}
}
}
}
);
}}
>
<Map reuseMaps mapLib={maplibregl} mapStyle={INITIAL_MAP_STYLE} preventStyleDiffing />
</DeckGL>
Expand All @@ -181,22 +181,19 @@ async function onExampleChange(args: {
const {selectedLoader, selectedExample, example, state, setState} = args;

const url = example.data;
console.log('Loading', url);
try {
const data = (await load(url, LOADERS, LOADER_OPTIONS)) as Table;
console.log('Loaded data', data);
console.log('Loaded data',url, data);
const viewState = {...state.viewState, ...example.viewState};
setState({...state, selectedLoader, selectedExample, viewState, loadedTable: data});
setState(state => ({...state, selectedLoader, selectedExample, viewState, loadedTable: data}));
} catch (error) {
console.log('Failed to load data', url, error);
setState({...state, error: `Could not load ${selectedExample}: ${error.message}`});
console.error('Failed to load data', url, error);
setState(state => ({...state, error: `Could not load ${selectedExample}: ${error.message}`}));
}
}

function renderLayer({selectedExample, selectedLoader, loadedTable}) {

const geojson = loadedTable as GeoJSON;
console.warn('Rendering layer with', geojson);
return [
new GeoJsonLayer({
id: `geojson-${selectedExample}(${selectedLoader})`,
Expand Down
Loading
Loading