New module `@turf/clean-coords` #875

stebogit · 2017-07-30T08:43:40Z

New module `@turf/clean-coords`

Implements #874

/**
 * Removes redundant coordinates from any GeoJSON Geometry.
 *
 * @name cleanCoords
 * @param {Geometry|Feature} geojson Feature or Geometry
 * @param {boolean} [mutate=false] allows GeoJSON input to be mutated
 * @returns {Geometry|Feature} the cleaned input Feature/Geometry
 * @example
 * var line = turf.lineString([[0, 0], [0, 2], [0, 5], [0, 8], [0, 8], [0, 10]]);
 * var multiPoint = turf.multiPoint([[0, 0], [0, 0], [2, 2]]);
 *
 * turf.cleanCoords(line).geometry.coordinates;
 * //= [[0, 0], [0, 10]]
 *
 * turf.cleanCoords(multiPoint).geometry.coordinates;
 * //= [[0, 0], [2, 2]]
 */
module.exports = function (geojson, mutate) {

DenisCarriere · 2017-07-31T04:17:31Z

@stebogit Looking at this right now, I'll spend more time on it tomorrow, but would it be worth cleaning MultiPoint as well? points which have multiple points on top of each other should be cleaned?

DenisCarriere · 2017-07-31T04:40:45Z

packages/turf-clean-coords/index.js

+ * Removes redundant coordinates from a (Multi)LineString or (Multi)Polygon; ignores (Multi)Point.
+ *
+ * @name cleanCoords
+ * @param {Geometry|Feature<any>} geojson Feature or Geometry


I've been dropping the <any> in the recent JSDocs.
{Geometry|Feature<any>} => {Geometry|Feature}

DenisCarriere · 2017-07-31T04:42:06Z

packages/turf-clean-coords/index.js

+        throw new Error(type + ' geometry not supported');
+    }
+
+    var output = (mutate === true) ? geojson : clone(geojson, true);


It's a bit of "black magic" how clone works, but you don't need to define a new variable if you apply clone (aka. JSON.parse + stringify).

This below is valid:

if (mutate !== true) geojson = clone(geojson, false);

2nd - Since we aren't mutating properties we don't need to define clone(geojson, true) as TRUE, if we use the default or FALSE it will only clone the coordinates (which is a lot faster process).

@DenisCarriere that's right! Now that I looked better at it, @turf/clone does already clone (if defined) all the "standard" GeoJSON attributes (i.e. properties, id and bbox); that's what I was aiming for. Very well 👍

if (mutate !== true) geojson = clone(geojson, false);

I really like this, it's much nicer to look at, even if for some reason it's always really hard for me to decode... 🤔
I am a big fan of immutable data though, so I was trying to avoid changing the value of geojson. I guess however in this small modules mutate the data is not really a big deal, and the code gains in readability 👍👍

stebogit · 2017-07-31T04:46:15Z

@DenisCarriere we can definitely clean MultiPoint; at this point we can allow Points for completeness and just ignoring them.
I think a MultiPoint with overlapping points it's not necessarily wrong (or "dirt" 😄 ), but if the user is using this module on MultiPoint I guess it means he wants to clean it! 👍

DenisCarriere · 2017-07-31T04:54:33Z

packages/turf-clean-coords/index.js

+var getGeomType = invariant.getGeomType;
+
+/**
+ * Removes redundant coordinates from a (Multi)LineString or (Multi)Polygon; ignores (Multi)Point.


I've added MultiPoint support, we can simply say this now:

* Removes redundant coordinates from any GeoJSON Geometry.

DenisCarriere · 2017-07-31T05:10:20Z

packages/turf-clean-coords/index.d.ts

+/**
+ * http://turfjs.org/docs/#cleancoords
+ */
+declare function cleanCoords(feature: Feature, mutate?: boolean): Feature;


@stebogit This one gets a bit tricky, one thing to keep in mind is trying to preserve the Input/Output type as much as possible.

At the moment, this definition is saying the input is any Feature or Geometry and it outputs either Geometry or a Feature.

Using the extends you validate the input, however the output will be the same as what the user enters, that way it preserves their Types.

type GeometryObject = GeoJSON.GeometryObject type Feature = GeoJSON.Feature<any> /** * http://turfjs.org/docs/#cleancoords */ declare function cleanCoords<T extends GeometryObject|Feature>(feature: T, mutate?: boolean): T; declare namespace cleanCoords {} export = cleanCoords;

DenisCarriere · 2017-07-31T05:14:43Z

packages/turf-clean-coords/bench.js

+    console.time(name);
+    cleanCoords(geojson);
+    console.timeEnd(name);
+    suite.add(name, () => cleanCoords(geojson));


For internal benchmark tests, we can add mutate=true, that way we get the "real" benchmarks results.

* geometry x 3,053,905 ops/sec ±1.84% (87 runs sampled) * multiline x 3,943,840 ops/sec ±2.60% (85 runs sampled) * multipoint x 465,553 ops/sec ±1.04% (89 runs sampled) * multipolygon x 2,046,659 ops/sec ±0.77% (91 runs sampled) * point x 22,318,913 ops/sec ±1.20% (86 runs sampled) * polygon-with-hole x 3,109,098 ops/sec ±2.10% (91 runs sampled) * polygon x 4,603,124 ops/sec ±1.99% (84 runs sampled) * simple-line x 7,732,876 ops/sec ±1.18% (86 runs sampled) * triangle x 4,950,167 ops/sec ±1.36% (89 runs sampled)

DenisCarriere · 2017-07-31T05:16:47Z

but if the user is using this module on MultiPoint I guess it means he wants to clean it!

Exactly.. :)

DenisCarriere

👍 Looks good to go! @stebogit

stebogit · 2017-07-31T07:23:59Z

packages/turf-clean-coords/index.js

+        var existing = {};
+        for (var i = 0; i < coords.length; i++) {
+            var point = coords[i];
+            var key = point.join('-');


@DenisCarriere this is really smart! 🤓 👍

Thanks 🤓 !

stebogit · 2017-07-31T07:36:01Z

@DenisCarriere now that I cleaned the horrible for-loops I used initially I feel better. I can't stand them! 😄
Good for me to merge this.

@rowanwins do you have any suggestion for an alternative name, if you feel it should be different?
I am personally fine with this (of course 😄 ).

DenisCarriere · 2017-07-31T12:35:49Z

@stebogit 👍 I'm on the same page with you for those for loops, there's some slight performance decrease (~8-10%) when using forEach, but it's a lot easier to read!

rowanwins · 2017-07-31T13:04:03Z

Happy to go with group consensus on the name - thoughts @DenisCarriere ? See history

DenisCarriere · 2017-07-31T13:25:50Z

@stebogit Let me know what you think about my last commit d684814, I dropped clone entirely so that the cleanCoords module doesn't mutate the input by adding geometry & feature helper functions.

will need to update @turf/helpers to support bbox & id properties, also a geometry method would be useful.

DenisCarriere · 2017-07-31T13:30:12Z

@rowanwins Happy to go with group consensus on the name

I'm good with this name, struggling to find a better name for this.

stebogit · 2017-07-31T15:14:13Z

@DenisCarriere

I dropped clone entirely

Why? is't this the perfect situation for using this module? It is basically exactly using geometry & feature helpers.

so that the cleanCoords module doesn't mutate the input

Why? I would leave this option to the user. We have this useful option in quite a few other modules, like truncate, transform, kmeans and other

DenisCarriere · 2017-07-31T16:08:20Z

@stebogit
Well unlike truncate or those other modules, we aren't using coordEach which is impossible to prevent input mutation without using clone. Since we've already deconstructed the entire GeoJSON geometry we can easily put it back together with little performance loss (1.3x slower).

Reason why I changed it to geometry/feature is purely for performance since mutate will most likely always be defined as false (which will be the slowest option).

I ran some benchmarks using the different techniques.

Considerations

Foreign members are dropped using the geometry/feature approach, however if we are enforcing a "strict" GeoJSON implementation that might be a good thing, also this might be considered a positive +1 for this "clean" module.

👍 It's a good idea to keep the mutate options that way it allows the user to preserve foreign members and/or wants the input to be mutated.

Summary

mutate=true (0x - speed factors are compared to these benchmark results)
mutate=false
- using geometry/feature (~1.3x slower)
- using @turf/clone (~10x slower)
- using JSON.parse + JSON.stringify (~20x slower)

 * // mutate=false (using geometry/feature)
 * geometry x 1,524,640 ops/sec ±4.60% (74 runs sampled)
 * multiline x 1,511,608 ops/sec ±8.79% (72 runs sampled)
 * multipoint x 382,429 ops/sec ±3.56% (84 runs sampled)
 * multipolygon x 808,277 ops/sec ±2.84% (82 runs sampled)
 * point x 14,675,464 ops/sec ±4.42% (80 runs sampled)
 * polygon-with-hole x 1,493,507 ops/sec ±5.53% (72 runs sampled)
 * polygon x 2,386,278 ops/sec ±1.27% (86 runs sampled)
 * simple-line x 4,195,499 ops/sec ±2.88% (86 runs sampled)
 * triangle x 2,254,753 ops/sec ±1.10% (88 runs sampled)
 *
 * // mutate=false (using @turf/clone)
 * geometry x 202,410 ops/sec ±1.43% (88 runs sampled)
 * multiline x 235,421 ops/sec ±3.48% (86 runs sampled)
 * multipoint x 280,757 ops/sec ±1.59% (87 runs sampled)
 * multipolygon x 127,353 ops/sec ±1.35% (88 runs sampled)
 * point x 18,233,987 ops/sec ±1.34% (86 runs sampled)
 * polygon-with-hole x 199,203 ops/sec ±2.61% (84 runs sampled)
 * polygon x 355,616 ops/sec ±1.58% (86 runs sampled)
 * simple-line x 515,430 ops/sec ±2.40% (83 runs sampled)
 * triangle x 286,315 ops/sec ±1.64% (86 runs sampled)
 *
 * // mutate=false (using JSON.parse + JSON.stringify)
 * geometry x 93,681 ops/sec ±7.66% (75 runs sampled)
 * multiline x 112,836 ops/sec ±4.60% (82 runs sampled)
 * multipoint x 113,937 ops/sec ±1.09% (90 runs sampled)
 * multipolygon x 71,131 ops/sec ±1.32% (90 runs sampled)
 * point x 18,181,612 ops/sec ±1.36% (91 runs sampled)
 * polygon-with-hole x 100,011 ops/sec ±1.14% (85 runs sampled)
 * polygon x 154,331 ops/sec ±1.31% (89 runs sampled)
 * simple-line x 193,304 ops/sec ±1.33% (90 runs sampled)
 * triangle x 130,921 ops/sec ±3.37% (87 runs sampled)
 *
 * // mutate=true
 * geometry x 2,016,365 ops/sec ±1.83% (85 runs sampled)
 * multiline x 2,266,787 ops/sec ±3.69% (79 runs sampled)
 * multipoint x 411,246 ops/sec ±0.81% (89 runs sampled)
 * multipolygon x 1,011,846 ops/sec ±1.34% (85 runs sampled)
 * point x 17,635,961 ops/sec ±1.47% (89 runs sampled)
 * polygon-with-hole x 2,110,166 ops/sec ±1.59% (89 runs sampled)
 * polygon x 2,887,298 ops/sec ±1.75% (86 runs sampled)
 * simple-line x 7,109,682 ops/sec ±1.52% (87 runs sampled)
 * triangle x 3,116,940 ops/sec ±0.71% (87 runs sampled)

stebogit · 2017-08-01T05:28:15Z

Great! I can't wait to use this in @turf/boolean-equal @DenisCarriere 😃

DenisCarriere · 2017-08-01T05:37:52Z

Now all I need to do is find some time to push out a new minor release.

stebogit added 4 commits July 29, 2017 10:19

created base module structure

195cadd

first implementation

fcdda7b

added tests, updated other files

8cdd1a3

added yarn, updated README

79a9b57

stebogit added the new-module label Jul 30, 2017

stebogit self-assigned this Jul 30, 2017

stebogit requested a review from DenisCarriere July 30, 2017 08:43

Merge branch 'master' into clean-coords

93b2dd3

Add cleaning of MultiPoint

27103e2

DenisCarriere reviewed Jul 31, 2017

View reviewed changes

Update JSDocs

69dd6b4

DenisCarriere reviewed Jul 31, 2017

View reviewed changes

DenisCarriere added 2 commits July 31, 2017 00:54

Update JSDocs

7b34a56

Update JSDocs

8c94d5f

DenisCarriere reviewed Jul 31, 2017

View reviewed changes

Update Typescript definition

cc1f7a0

DenisCarriere reviewed Jul 31, 2017

View reviewed changes

Update bench results

cf079ef

Fix linting issues

ce1e3a7

DenisCarriere approved these changes Jul 31, 2017

View reviewed changes

DenisCarriere added this to the 4.6.0 milestone Jul 31, 2017

stebogit commented Jul 31, 2017

View reviewed changes

replaced for loops with forEach

45e8ab1

Added extra tests

36ca0d1

DenisCarriere added 2 commits July 31, 2017 09:19

Remove clone

d684814

Add helpers

d4a296d

Add mutate param (include benchmark results)

11f20ee

DenisCarriere added 2 commits July 31, 2017 12:17

Merge branch 'master' into clean-coords

54018e6

Merge branch 'master' into clean-coords

b718b5c

stebogit assigned DenisCarriere Aug 1, 2017

DenisCarriere merged commit adb56ea into master Aug 1, 2017

DenisCarriere deleted the clean-coords branch August 1, 2017 05:36

stebogit mentioned this pull request Aug 1, 2017

New module proposal @turf/clean-coords #874

Closed

DenisCarriere mentioned this pull request Aug 2, 2017

New minor release! Turf 4.6.0 🎉 #884

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New module `@turf/clean-coords` #875

New module `@turf/clean-coords` #875

stebogit commented Jul 30, 2017 •

edited by DenisCarriere

Loading

DenisCarriere commented Jul 31, 2017 •

edited

Loading

DenisCarriere Jul 31, 2017

DenisCarriere Jul 31, 2017

DenisCarriere Jul 31, 2017

stebogit Jul 31, 2017 •

edited

Loading

stebogit commented Jul 31, 2017

DenisCarriere Jul 31, 2017

DenisCarriere Jul 31, 2017

DenisCarriere Jul 31, 2017

DenisCarriere commented Jul 31, 2017

DenisCarriere left a comment

stebogit Jul 31, 2017

DenisCarriere Jul 31, 2017

stebogit commented Jul 31, 2017 •

edited

Loading

DenisCarriere commented Jul 31, 2017

rowanwins commented Jul 31, 2017

DenisCarriere commented Jul 31, 2017 •

edited

Loading

DenisCarriere commented Jul 31, 2017

stebogit commented Jul 31, 2017

DenisCarriere commented Jul 31, 2017 •

edited by stebogit

Loading

stebogit commented Aug 1, 2017

DenisCarriere commented Aug 1, 2017

New module @turf/clean-coords #875

New module @turf/clean-coords #875

Conversation

stebogit commented Jul 30, 2017 • edited by DenisCarriere Loading

New module @turf/clean-coords

DenisCarriere commented Jul 31, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stebogit Jul 31, 2017 • edited Loading

Choose a reason for hiding this comment

stebogit commented Jul 31, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DenisCarriere commented Jul 31, 2017

DenisCarriere left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stebogit commented Jul 31, 2017 • edited Loading

DenisCarriere commented Jul 31, 2017

rowanwins commented Jul 31, 2017

DenisCarriere commented Jul 31, 2017 • edited Loading

DenisCarriere commented Jul 31, 2017

stebogit commented Jul 31, 2017

DenisCarriere commented Jul 31, 2017 • edited by stebogit Loading

Considerations

Summary

stebogit commented Aug 1, 2017

DenisCarriere commented Aug 1, 2017

New module `@turf/clean-coords` #875

New module `@turf/clean-coords` #875

stebogit commented Jul 30, 2017 •

edited by DenisCarriere

Loading

New module `@turf/clean-coords`

DenisCarriere commented Jul 31, 2017 •

edited

Loading

stebogit Jul 31, 2017 •

edited

Loading

stebogit commented Jul 31, 2017 •

edited

Loading

DenisCarriere commented Jul 31, 2017 •

edited

Loading

DenisCarriere commented Jul 31, 2017 •

edited by stebogit

Loading