Enumerable and its methods

An Enumerable is an async sequence with the methods you know from arrays. run(), read(), and range() return one, and enumerate() wraps anything else you can iterate: an array, a Set, a generator, a ReadableStream.

import { read } from "@j50n/proc";

// How many lines at each level, leaving out INFO.
const levels = await read("app.log")
  .lines
  .map((line) => line.split(/\s+/)[2]) // "INFO", "ERROR", "WARN"
  .filterNot((level) => level === "INFO")
  .reduce(
    (counts, level) => counts.set(level, (counts.get(level) ?? 0) + 1),
    new Map<string, number>(),
  );

console.log(levels);

Map(2) { "ERROR" => 2, "WARN" => 1 }

Each step returns a new Enumerable, and the consumer at the end (reduce here) pulls the items through and returns a promise. Callbacks may be async; each call is awaited before the next one starts. To run several at once, use concurrentMap. Key ideas covers how sources, steps, and consumers fit together.

enumerate() is not .enum()

import { enumerate } from "@j50n/proc";

// enumerate() wraps an iterable. The items stay as they are.
console.log(await enumerate(["apple", "pear"]).collect());

// .enum() numbers the items. Each becomes [item, index].
console.log(await enumerate(["apple", "pear"]).enum().collect());

[ "apple", "pear" ]
[ [ "apple", 0 ], [ "pear", 1 ] ]

enumerate(iterable) is a function: it wraps an iterable so you can call the methods on it, and leaves the items alone. .enum() is a method on an Enumerable: it numbers the items, turning each into [item, index] from 0. Python’s enumerate is proc’s .enum().

The methods

Change items:

Filter:

Slice:

Combine:

Split:

Consume (each returns a promise):

A for await loop is a consumer too.

Text, processes, and output:

The output of run() also has pid and status; see Running a command. Files and stdout are in Files and standard streams.

Steps to pass to transform()

The data formats (fromCsvToRows(), toTsv(), …) are steps too, from @j50n/proc/transforms; see Data formats. To write your own, see Writing your own steps.

More examples

import { enumerate, range } from "@j50n/proc";

const words = ["one", "two", "three", "four", "five"];

console.log(await enumerate(words).drop(1).take(2).collect());
console.log(await range({ from: 1, to: 4 }).zip(enumerate(words)).collect());
console.log(
  await enumerate(["a b", "c"]).flatMap((s) => s.split(" ")).collect(),
);
console.log(
  await enumerate(["x", "y"]).concat(enumerate(["z"])).collect(),
);

[ "two", "three" ]
[ [ 1, "one" ], [ 2, "two" ], [ 3, "three" ] ]
[ "a", "b", "c" ]
[ "x", "y", "z" ]

concat and zip take an async iterable, so wrap an array in enumerate() first; the types refuse a plain array.

import { enumerate, range } from "@j50n/proc";

const numbers = () => range({ from: 1, until: 6 }); // 1 to 6, a new one each time

console.log(await numbers().count((n) => n % 2 === 0)); // 3
console.log(await numbers().find((n) => n > 4)); // 5
console.log(await numbers().find((n) => n > 10)); // undefined
console.log(await numbers().some((n) => n > 5)); // true
console.log(await numbers().every((n) => n > 5)); // false
console.log(await numbers().first); // 1, a getter

try {
  await enumerate<number>([]).first;
} catch (error) {
  if (error instanceof RangeError) console.log("empty: no first item");
}

3
5
undefined
true
false
1
empty: no first item

numbers() makes a new Enumerable for each line because each one can be read only once. find, some, every, and first stop reading as soon as they have their answer, and close the source.

import { read } from "@j50n/proc";

// Read the file once, and go over the lines two ways side by side.
const [a, b] = read("app.log").lines.tee();

const [total, errors] = await Promise.all([
  a.count(),
  b.count((line) => line.includes("ERROR")),
]);

console.log(`${errors} of ${total} lines are errors`);

2 of 7 lines are errors

Traps

• An Enumerable is read once. A second collect() gets nothing, without an error. Keep the array from the first, or tee() the sequence.
• tee() and unzip() keep an item until every branch has read it. Branches read side by side hold little; a branch that runs far ahead of another makes everything in between pile up in memory. If the source throws, every branch throws, after the items before the error.
• Read every branch of a tee(), at least to a break. The source is closed once every branch has stopped; a branch that is never read keeps it open, and keeps every item from the start.
• first throws on an empty sequence (RangeError), and reduce with no starting value throws TypeError. find returns undefined instead.
• There is no sort. Sorting needs every item at once, so it isn’t a step. Filter first to keep only what you need, collect(), and sort the array (or pipe lines through the sort command, which handles more than fits in memory).
• flatten() on strings gives characters, because a string is iterable.
• lines and chunkedLines are getters: .lines, not .lines(). Only an Enumerable of bytes has them; on anything else the type is never.