Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add script for extracting style statistics #747

Merged
merged 17 commits into from
Feb 9, 2023
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,16 @@ boilerplate in `scripts/taginfo_template.json`.
[90]: https://prettier.io/
[svgo]: https://github.com/svg/svgo/

### Style complexity checks

When adding or changing style layer code, it can be helpful to assess the change in size and complexity. In general, higher layer counts and higher layer size have a negative impact in performance. Contributors should attempt to consolidate layers when possible.

There is a "stats" script that will generate various statistics about layer composition and complexity:

- `npm run stats -- -a -s` - overall size and breakdown of layers
- `npm run stats -- -c` - total layer count
- `npm run stats -- -h` - list all options

## Highway Shield Contributor's Guide

Highway shields are a key feature of the OpenStreetMap Americana style. This guide describes some of the style principles that contributors of highway shield artwork should consider when submitting new shields. The required elements are as follows:
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
"build:code": "node scripts/build.js",
"build": "run-s clean sprites build:code taginfo status_map",
"test": "mocha",
"stats": "node scripts/stats.js",
"style": "node scripts/generate_style.js -o dist/style.json"
},
"dependencies": {
Expand Down
90 changes: 90 additions & 0 deletions scripts/stats.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
import * as Style from "../src/js/style.js";
import config from "../src/config.js";
import { Command } from "commander";

const program = new Command();
program
.option("-a, --all-layers", "summary layer stats")
.option("-c, --layer-count", "count number of layers")
.option("-s, --layer-size", "size of all layers")
.option("-l, --layer <layer id>", "stats about one layer")
.option("-loc, --locales <locale1 locale2...>", "language codes", ["mul"])
.option("-pp, --pretty", "pretty-print JSON output")
.option(
"-pg, --print-group <group prefix>",
"print a list of the layers in a group"
)
.option("-pl, --print-layer <layer id>", "print the JSON of a layer");
program.parse(process.argv);

let opts = program.opts();

ZeLonewolf marked this conversation as resolved.
Show resolved Hide resolved
let style = Style.build(
config.OPENMAPTILES_URL,
"https://zelonewolf.github.io/openstreetmap-americana/sprites/sprite",
opts.locales
);

const layers = style.layers;
const layerCount = layers.length;
const layerSize = JSON.stringify(layers).length;

const layerMap = new Map();
const layerGroupMap = new Map();
const layerSizeStats = new Map();
const layerGroupSizeStats = new Map();
const layerGroupCountStats = new Map();

for (let i = 0; i < layers.length; i++) {
let layer = layers[i];
layerMap.set(layer.id, layers[i]);
let layerSize = JSON.stringify(layer).length;
let layerGroup = layer.id.split("_", 1)[0];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this PR introduces a new naming convention for layer IDs that the statistics code depends on. We should document this naming convention.

Better yet, we should avail ourselves of the metadata property, which is an object that you can stuff arbitrary properties into. A style authored in Mapbox Studio inserts metadata in abundance, including for grouping layers.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uh oh 😅 that sounds like several KB worth of style JSON. Maybe let's document it for now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - contributor's guide updated!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still seems fragile, but let’s ticket out something less fragile as tail work. At a glance, we could group the layers by source layer to get pretty similar statistics, modulo some layer organization weirdness in OpenMapTiles.

layerSizeStats.set(layer.id, JSON.stringify(layer).length);
if (!layerGroupSizeStats.has(layerGroup)) {
layerGroupSizeStats.set(layerGroup, layerSize);
layerGroupCountStats.set(layerGroup, 1);
layerGroupMap.set(layerGroup, [layer.id]);
} else {
layerGroupSizeStats.set(
layerGroup,
layerGroupSizeStats.get(layerGroup) + layerSize
);
layerGroupCountStats.set(
layerGroup,
layerGroupCountStats.get(layerGroup) + 1
);
layerGroupMap.get(layerGroup).push(layer.id);
}
}

if (opts.layerCount) {
console.log(`${layerCount} layers`);
}

if (opts.layerSize) {
console.log(`Total layer size ${layerSize.toLocaleString("en-US")} bytes`);
}

if (opts.allLayers) {
console.log(`${layerCount} layers, grouped as follows:`);
layerGroupSizeStats.forEach((v, k) => {
let layerCount = layerGroupCountStats.get(k);
let layerString = `${k}(${layerCount})`.padEnd(30, ".");
console.log(
`${layerString}${v.toLocaleString("en-US").padStart(10, ".")} bytes`
);
});
}

if (opts.printGroup) {
layerGroupMap.get(opts.printGroup).forEach((lyr) => console.log(lyr));
}

if (opts.printLayer) {
if (opts.pretty) {
console.log(JSON.stringify(layerMap.get(opts.printLayer), null, 2));
} else {
console.log(JSON.stringify(layerMap.get(opts.printLayer)));
}
}
4 changes: 2 additions & 2 deletions src/layer/aeroway.js
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ export const airportRefLabel = {
};

export const minorAirportRefLabel = {
id: "minor_airport_ref_label",
id: "airport_ref_label_minor",
type: "symbol",
minzoom: 13,
maxzoom: 15,
Expand Down Expand Up @@ -225,7 +225,7 @@ export const airportLabel = {
};

export const minorAirportLabel = {
id: "minor_airport_label",
id: "airport_label_minor",
type: "symbol",
minzoom: 13,
maxzoom: 15,
Expand Down
4 changes: 2 additions & 2 deletions src/layer/background.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ export const base = {
};

export const pierArea = {
id: "pierArea",
id: "pier_area",
type: "fill",
source: "openmaptiles",
"source-layer": "transportation",
Expand All @@ -36,7 +36,7 @@ export const pierArea = {
};

export const pierLine = {
id: "pierLine",
id: "pier_line",
type: "line",
source: "openmaptiles",
"source-layer": "transportation",
Expand Down
2 changes: 1 addition & 1 deletion src/layer/highway_exit.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import * as Label from "../constants/label.js";

export const exits = {
id: "highway_exit",
id: "highway-exit",
type: "symbol",
filter: [
"all",
Expand Down
2 changes: 1 addition & 1 deletion src/layer/highway_shield.js
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ export const shield = {
type: "symbol",
source: "openmaptiles",
"source-layer": "transportation_name",
id: "highway_shield",
id: "highway-shield",
layout: shieldLayout,
paint: {
"text-opacity": [
Expand Down
12 changes: 6 additions & 6 deletions src/layer/park.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import * as Label from "../constants/label.js";
import * as Color from "../constants/color.js";

export const fill = {
id: "protected-area-fill",
id: "protected-area_fill",
type: "fill",
paint: {
"fill-color": Color.parkFill,
Expand All @@ -14,7 +14,7 @@ export const fill = {
};

export const outline = {
id: "protected-area-outline",
id: "protected-area_outline",
type: "line",
paint: {
"line-color": Color.parkOutline,
Expand All @@ -25,7 +25,7 @@ export const outline = {
};

export const label = {
id: "protected-area-label",
id: "protected-area_label",
type: "symbol",
filter: ["has", "rank"],
paint: {
Expand All @@ -46,21 +46,21 @@ export const label = {

export const parkFill = {
...fill,
id: "park-fill",
id: "park_fill",
filter: ["==", ["get", "subclass"], "park"],
"source-layer": "landcover",
};

export const parkOutline = {
...outline,
id: "park-outline",
id: "park_outline",
filter: ["==", ["get", "subclass"], "park"],
"source-layer": "landcover",
};

export const parkLabel = {
...label,
id: "park-label",
id: "park_label",
filter: ["==", ["get", "class"], "park"],
"source-layer": "poi",
};
Expand Down
10 changes: 5 additions & 5 deletions src/layer/place.js
Original file line number Diff line number Diff line change
Expand Up @@ -267,7 +267,7 @@ export const state = {
"source-layer": "place",
};
export const countryOther = {
id: "country_other",
id: "place_country-other",
type: "symbol",
paint: {
"text-color": "#334",
Expand Down Expand Up @@ -296,7 +296,7 @@ export const countryOther = {
"source-layer": "place",
};
export const country3 = {
id: "country_3",
id: "place_country-3",
type: "symbol",
paint: {
"text-color": "#334",
Expand Down Expand Up @@ -326,7 +326,7 @@ export const country3 = {
"source-layer": "place",
};
export const country2 = {
id: "country_2",
id: "place_country-2",
type: "symbol",
paint: {
"text-color": "#334",
Expand Down Expand Up @@ -356,7 +356,7 @@ export const country2 = {
"source-layer": "place",
};
export const country1 = {
id: "country_1",
id: "place_country-1",
type: "symbol",
paint: {
"text-color": "#334",
Expand Down Expand Up @@ -404,7 +404,7 @@ export const country1 = {
"source-layer": "place",
};
export const continent = {
id: "continent",
id: "place_continent",
type: "symbol",
paint: {
"text-color": "#633",
Expand Down
2 changes: 1 addition & 1 deletion src/layer/rail.js
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ var opacity = [
// Bridge casing layers
export const bridgeCasing = {
...defRail,
id: "railway-bridge-casing",
id: "rail_bridge-casing",
filter: [
"all",
["==", ["get", "brunnel"], "bridge"],
Expand Down