Performance optimizations in forEach, powerMap, map and filter functi…

…ons.

Exported the itr!FromImpureFunction() interface function.
Added some simple benchmarks.
Some updates to ROADMAP.md

ROADMAP.md

@@ -2,8 +2,46 @@
 
 This is a bunch of ideas of things to add or change.
 
+## Improve performance
+
+**IN PROGRESS**
+
+Stumbling upon vitality-t's iter-ops library, I got curious about performance, and startedworking on some performance improvements for the (very limited) test.
+
+These are the current results. With itr8 being about 3 times slower than iter-ops for the synchronous tests, but being slightly faster for the asynchronous tests.
+
+Synchronous test for 1e+7 items:
+
+| library    | duration | length  |
+| ---------- | -------- | ------- |
+| array      | 362      | 5000000 |
+| iter-ops   | 224      | 5000000 |
+| rxjs       | 531      | 5000000 |
+| rxjs + sub | 1183     | 5000000 |
+| itr8       | 862      | 5000000 |
+
+Asynchronous test for 1e+7 items:
+
+| library    | duration | length  |
+| ---------- | -------- | ------- |
+| iter-ops   | 5383     | 5000000 |
+| rxjs       | 2281     | 5000000 |
+| rxjs + sub | 5884     | 5000000 |
+| itr8       | 3741     | 5000000 |
+
+I must point out though: itr8's map and filter functions are more powerful than the ones in iter-ops, because the mapping and filter functions can also be async, which is not the case in most other libraries!
+
+In order to simplify writing code that supports both sync and async inputs, I started out with a utility function called thenable(variable). This makes writing the code easier, but it is not a good fit for iterators, because once we've established (during the first next() call) whether the resulting iterator will be synchronous or asynchronous, we can stick to that conclusion for all the other items of that iterator, allowing for some optimization.
+
+Currently I kind of hand-crafted the optimizations in the filter and map functions (both of which use the also optimized powerMap operator internally). I'm still thinking about away to do the same without the manual labour behind it.
+
+Anyway, after these optimizations, the results are pretty decent, although not on par with iter-ops yet.
+
 ## Support the full iterator protocol
 
+** IN PROGRESS **
+2023-08-22: Implemented in some places, but not everywhere and largely untested
+
 ### Cleanup methods (return and throw)
 
 The return(value) method indicates that the caller does not intend to make any more next calls.
@@ -36,6 +74,8 @@ promises to resolve. This makes a generic retry mechanism in the form of a trans
 
 ## Make it usable both in NodeJS and in the browser
 
+** DONE **
+
 Currently we use module: "CommonJS" in tsconfig.json, but ideally it should be ES2015
 or something (an Ecmascript module instead of a CommonJS module),
 so that the compiled typescript code can be used unmodified both in NodeJS and in the
@@ -53,6 +93,8 @@ I could also drop support for NodeJS < 18 and older browsers and use the more mo
 
 ## Entirely remove all OO-style stuff (especially .pipe)
 
+** DONE ** in version 0.4.5
+
 I first implemented a .pipe function on every iterator that the library would return,
 but that makes the iterators returned by the lib 'special' instead of being simple and plain
 (sync or async) iterators.

benchmarks/LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2021 Vitaly Tomilov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

benchmarks/README.md

@@ -0,0 +1,48 @@
+## Benchmarks
+
+Shamelessly copied from https://github.com/vitaly-t/iter-ops/tree/main/benchmarks (see LICENSE)
+
+This README is largely copied as well, so everything below is written by vitaly-t. I just use this to compare itr8 against the competition, in order to have an idea whether I'm on the right track. (I think there are other benefits to using this library besides performance, but of course we also want it to perform well).
+
+---
+
+Testing `iter-ops` against the latest `rxjs` and against `itr8`:
+
+- We use an array of numbers as the input (`1e7` items)
+- We first `filter` out all even numbers
+- Then we `map` each value into an object
+- Collecting all values into an array
+
+### Running Tests
+
+Running tests separately is recommended, or else results may become skewed.
+
+- `npm run sync` - runs tests on synchronous iterables
+- `npm run async` - runs tests on asynchronous iterables
+- `npm test` - runs all tests (not recommended)
+
+### Test Results
+
+Testing against an asynchronous source produces the result that for the most part depends on how fast the source
+iterable is. This makes it difficult to test objectively. On one hand, `iter-ops`
+has embedded optimization for wrapping an asynchronous iterable, so if we test that against a standard async iterable
+for `rxjs`, we get the result as above:
+
+- Testing against `rxjs` asynchronous pipeline, we get ~7x times better performance
+- Testing against `rxjs` with a single empty subscription - we get ~15x better performance
+
+However, if we optimize the source iterable similar to how `iter-ops` does it, then figures become very comparative.
+
+### Conclusions
+
+This library performs about 2.5x faster than `rxjs` synchronous pipeline. However, just as you add a single subscription
+in `rxjs` (
+which is inevitable with `rxjs`), then `iter-ops` performance is about 5x times better. So ultimately, this library can
+process synchronous iterables about 5x times faster than synchronous `rxjs`.
+
+For the asynchronous test, even though we do have very impressive results versus `rxjs`, those for the most part depend
+on how fast the source iterable is. Library `iter-ops` does come with some good performance optimization for wrapping
+asynchronous iterables, which we achieve by using [toAsync]. However, it is possible to optimize an iterable for `rxjs`
+separately. Therefore, it is nearly impossible to draw the line, in how to define an objective test for this.
+
+[toasync]: https://vitaly-t.github.io/iter-ops/functions/toAsync

benchmarks/package.json

@@ -0,0 +1,14 @@
+{
+  "name": "iter-ops-benchmarks",
+  "version": "0.0.4",
+  "description": "Benchmarks for iter-ops library",
+  "scripts": {
+    "async": "tsc -p src && node ./src/async.js",
+    "sync": "tsc -p src && node ./src/sync.js",
+    "test": "tsc -p src && node ./src/sync.js && node ./src/async.js"
+  },
+  "devDependencies": {
+    "iter-ops": "^3.1.1",
+    "rxjs": "^7.8.1"
+  }
+}

benchmarks/src/async.d.ts

@@ -0,0 +1 @@
+export {};

benchmarks/src/async.js

@@ -0,0 +1,33 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+const iter_ops_1 = require("iter-ops");
+const iter_ops_2 = require("./tests/iter-ops");
+const rxjs_1 = require("./tests/rxjs");
+const itr8_1 = require("./tests/itr8");
+// tslint:disable:no-console
+const maxItems = 1e7;
+const data = [];
+for (let i = 0; i < maxItems; i++) {
+  data.push(i);
+}
+// regular/popular way of wrapping into asynchronous iterable
+const input = {
+  [Symbol.asyncIterator]() {
+    const i = data.values();
+    return {
+      async next() {
+        return i.next();
+      },
+    };
+  },
+};
+(async function testAsync() {
+  const result = {
+    ...(await (0, iter_ops_2.testIterOps)((0, iter_ops_1.toAsync)(data))),
+    ...(await (0, rxjs_1.testRXJS)(input)),
+    ...(await (0, rxjs_1.testRXJS)(input, true)),
+    ...(await (0, itr8_1.testItr8)((0, iter_ops_1.toAsync)(data))),
+  };
+  console.log(`Asynchronous test for ${maxItems.toExponential()} items:`);
+  console.table(result);
+})();

benchmarks/src/async.ts

@@ -0,0 +1,36 @@
+import { toAsync } from "iter-ops";
+import { testIterOps } from "./tests/iter-ops";
+import { testRXJS } from "./tests/rxjs";
+import { testItr8 } from "./tests/itr8";
+
+// tslint:disable:no-console
+
+const maxItems = 1e7;
+
+const data: number[] = [];
+for (let i = 0; i < maxItems; i++) {
+  data.push(i);
+}
+
+// regular/popular way of wrapping into asynchronous iterable
+const input: AsyncIterable<number> = {
+  [Symbol.asyncIterator](): AsyncIterator<number> {
+    const i = data.values();
+    return {
+      async next(): Promise<IteratorResult<number>> {
+        return i.next();
+      },
+    };
+  },
+};
+
+(async function testAsync() {
+  const result = {
+    ...(await testIterOps(toAsync(data))),
+    ...(await testRXJS(input)),
+    ...(await testRXJS(input, true)),
+    ...(await testItr8(toAsync(data))),
+  };
+  console.log(`Asynchronous test for ${maxItems.toExponential()} items:`);
+  console.table(result);
+})();

benchmarks/src/sync.d.ts

@@ -0,0 +1 @@
+export {};

benchmarks/src/sync.js

@@ -0,0 +1,23 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+const array_1 = require("./tests/array");
+const iter_ops_1 = require("./tests/iter-ops");
+const rxjs_1 = require("./tests/rxjs");
+const itr8_1 = require("./tests/itr8");
+// tslint:disable:no-console
+const maxItems = 1e7;
+const input = [];
+for (let i = 0; i < maxItems; i++) {
+  input.push(i);
+}
+(async function testSync() {
+  const result = {
+    ...(await (0, array_1.testArray)(input)),
+    ...(await (0, iter_ops_1.testIterOps)(input)),
+    ...(await (0, rxjs_1.testRXJS)(input)),
+    ...(await (0, rxjs_1.testRXJS)(input, true)),
+    ...(await (0, itr8_1.testItr8)(input)),
+  };
+  console.log(`Synchronous test for ${maxItems.toExponential()} items:`);
+  console.table(result);
+})();

benchmarks/src/sync.ts

@@ -0,0 +1,24 @@
+import { testArray } from "./tests/array";
+import { testIterOps } from "./tests/iter-ops";
+import { testRXJS } from "./tests/rxjs";
+import { testItr8 } from "./tests/itr8";
+// tslint:disable:no-console
+
+const maxItems = 1e7;
+
+const input: number[] = [];
+for (let i = 0; i < maxItems; i++) {
+  input.push(i);
+}
+
+(async function testSync() {
+  const result = {
+    ...(await testArray(input)),
+    ...(await testIterOps(input)),
+    ...(await testRXJS(input)),
+    ...(await testRXJS(input, true)),
+    ...(await testItr8(input)),
+  };
+  console.log(`Synchronous test for ${maxItems.toExponential()} items:`);
+  console.table(result);
+})();

benchmarks/src/tests/array.d.ts

@@ -0,0 +1,6 @@
+export declare function testArray(input: Array<number>): Promise<{
+  array: {
+    duration: number;
+    length: number;
+  };
+}>;

benchmarks/src/tests/array.js

@@ -0,0 +1,11 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.testArray = void 0;
+async function testArray(input) {
+  const start = Date.now();
+  const i = (await input).filter((a) => a % 2 === 0).map((b) => ({ value: b }));
+  const length = await i.length;
+  const duration = Date.now() - start;
+  return { array: { duration, length: length } };
+}
+exports.testArray = testArray;