CSV

fromCsvToRows() parses CSV into batches of rows, and toCsv() writes rows as CSV, quoting the fields that need it.

import { read } from "@j50n/proc";
import { fromCsvToRows } from "@j50n/proc/transforms";

const rows = await read("data-orders.csv")
  .transform(fromCsvToRows())
  .flatten()
  .drop(1) // the header
  .collect();

console.log(rows);

[
  [ "1", "Ada", "Widget, large", "2" ],
  [ "2", "Grace", 'Gear "XL"', "1" ],
  [ "3", "Linus", "Bolt", "10" ]
]

The quotes are gone: "Gear ""XL""" in the file is Gear "XL" in the row. The first row is the header, read like any other row; see Header rows. The parser runs in WebAssembly and yields a batch of rows for about every 128 KiB of input, so .flatten() comes before the per-row steps.

fromCsvToLazyRows() parses the same way but yields LazyRows, which decode a field only when you read it.

Writing

import { enumerate } from "@j50n/proc";
import { toCsv } from "@j50n/proc/transforms";

const rows = [
  ["item", "note"],
  ["Widget, large", 'says "hi"'],
  ["Bolt", "two\nlines"],
  ["Nut", " spaces kept "],
];

await enumerate(rows).transform(toCsv()).toStdout();
console.log("--");
await enumerate(rows).transform(toCsv({ separator: ";" })).toStdout();

item,note
"Widget, large","says ""hi"""
Bolt,"two
lines"
Nut, spaces kept 
--
item;note
Widget, large;"says ""hi"""
Bolt;"two
lines"
Nut; spaces kept 

toCsv() quotes a field that holds the separator, a quote, CR, or LF, and doubles the quotes inside it. Other fields are written as they are, spaces included, except that a first field starting with U+FEFF is quoted, since readers drop a byte order mark at the start. It refuses only what CSV can’t hold: a row with no fields ([] inside a batch), which would be a blank line and read back as no row, and a field holding a lone surrogate, which UTF-8 can’t hold. Each item can be a row or a batch, of Rows or LazyRows.

Options

The separator must be one ASCII character other than ", CR, or LF. ";", "|", and "\t" all work. Anything else throws a RangeError from the call that takes the options, such as fromCsvToRows() itself, before any data moves. There are no other options: no header handling, no quote character, no comment lines (a line starting with # is data), no trimming.

What the parser accepts

It reads RFC 4180 and is lenient about the rest. It refuses two things: a CR outside quotes that isn’t part of a CRLF, and a quote still open at the end of the input:

import { enumerate } from "@j50n/proc";
import { fromCsvToRows, toCsv } from "@j50n/proc/transforms";

async function parse(text: string): Promise<string> {
  try {
    const rows = await enumerate([new TextEncoder().encode(text)])
      .transform(fromCsvToRows())
      .flatten()
      .collect();
    return JSON.stringify(rows);
  } catch (error) {
    return error instanceof Error ? error.message : String(error);
  }
}

const inputs = [
  "a,b\r\nc,d", // CRLF; no line end on the last row
  "a\n\n\nb\n", // blank lines are skipped
  ",a,,\n", // empty fields
  '""\n', // one empty field: a row, not a blank line
  'a,"x\ny, ""z"""\n', // quoted line break, separator, quotes
  "a\nb,c,d\n", // rows may differ in length
  ' a , "b"\n', // spaces are kept, so this quote is text
  'a"b,c\n', // a quote inside an unquoted field is text
  '"ab" ,c\n', // text after a closing quote is kept
  'a,"bc\nd,e\n', // a quote still open at the end is an error
  '"x\ry",z\n', // a CR inside quotes is text
  "a,b\rc,d\r", // a lone CR outside quotes is an error
];
for (const text of inputs) {
  console.log(JSON.stringify(text).padEnd(26), await parse(text));
}

// toCsv writes a row holding one empty field as "", so it survives.
const text = await enumerate([["a"], [""], ["b"]])
  .transform(toCsv())
  .map((bytes) => new TextDecoder().decode(bytes))
  .reduce((all, chunk) => all + chunk, "");
console.log(JSON.stringify(text), await parse(text));

"a,b\r\nc,d"               [["a","b"],["c","d"]]
"a\n\n\nb\n"               [["a"],["b"]]
",a,,\n"                   [["","a","",""]]
"\"\"\n"                   [[""]]
"a,\"x\ny, \"\"z\"\"\"\n"  [["a","x\ny, \"z\""]]
"a\nb,c,d\n"               [["a"],["b","c","d"]]
" a , \"b\"\n"             [[" a "," \"b\""]]
"a\"b,c\n"                 [["a\"b","c"]]
"\"ab\" ,c\n"              [["ab ","c"]]
"a,\"bc\nd,e\n"            Unclosed quote in CSV data at row 1, field 2
"\"x\ry\",z\n"             [["x\ry","z"]]
"a,b\rc,d\r"               Invalid character (CR) in CSV data at row 1, field 2
"a\n\"\"\nb\n" [["a"],[""],["b"]]

• LF and CRLF end a row, and the last row needs no line end.
• Blank lines are skipped, but "" on a line is a row holding one empty field; toCsv() writes such a row that way, so it survives a round trip, as the last line shows.
• A quoted field can hold separators, line breaks (CR included), and doubled quotes.
• Rows may have different numbers of fields. Check row.length (a LazyRow’s columnCount) if yours must match: getField past the end throws a RangeError that can’t say which row, so count rows yourself to report one.
• Spaces around a field are kept. A quote opens a quoted field only at the very start of a field, so a quote after a space, or inside an unquoted field, is text. Text after a closing quote is kept: "ab" ,c reads as ["ab ", "c"].
• A UTF-8 byte order mark at the start, as spreadsheet programs write, is dropped.
• Any other CR outside quotes throws an Error naming the row and field, so a file with CR-only line ends fails at its first line rather than reading as one long row.
• A quote still open at the end of the input throws an Error naming the row and field where it opened, as in Unclosed quote in CSV data at row 1, field 2, rather than make the rest of the file one field.

Either error comes after the batches before the one holding it have been yielded.

Converting to and from TSV

csvToTsv() and tsvToCsv() convert bytes to bytes in WebAssembly, without making rows or strings, several times faster than a parser and a writer:

import { read } from "@j50n/proc";
import { csvToTsv, tsvToCsv } from "@j50n/proc/transforms";

// Bytes to bytes, with no rows in between.
await read("data-orders.csv").transform(csvToTsv()).writeTo("orders.tsv");
await read("orders.tsv").transform(tsvToCsv({ separator: ";" })).toStdout();

id;customer;item;qty
1;Ada;Widget, large;2
2;Grace;"Gear ""XL""";1
3;Linus;Bolt;10

They read and write as the parsers and writers do, errors included. TSV can’t hold a tab, CR, or LF in a field, so csvToTsv() throws on a CSV field holding one, as toTsv() does. Nor can it hold a row of one empty field ("" on a line), which would be a blank line, so csvToTsv() throws on that too: Invalid row (one empty field) in TSV data at row 4. The first error in the input is the one thrown: an unclosed quote whose field takes in a line break reports the LF. The output passed on before it can end partway through a row.

They copy field bytes without decoding them, so invalid UTF-8 doesn’t throw here: it passes through to the output unchanged, as text in any other ASCII-compatible encoding does. The parsers would throw a TypeError on the same input.

Other traps

• Invalid UTF-8 throws a TypeError naming the row and field, from fromCsvToRows() as its batch is converted, and from fromCsvToLazyRows() only when the bad field is decoded.
• Every field is a string. Number(row[3]) for numbers; an empty field is "", and Number("") is 0.
• A row is held whole until it ends, in the WebAssembly module’s memory, which can’t pass 4 GiB. A row takes about its own size plus 8 bytes for each of its fields, and up to twice that while its buffers grow by doubling: about 2 times its size for a row of long fields, and up to 18 times for one of empty fields (,,,,). So a row of about a gigabyte of long fields, or about 150 MB of empty ones, is too large, and throws Row too large for the WebAssembly module's memory in CSV data at row 7. tsvToCsv() holds the longest field whole instead, with about the same limit. The memory a row took isn’t given back until the stream ends. A quote opened near the start of untrusted input and never closed holds all the rest before the error, at about twice its size. Each open stream also holds a floor of about 1 MiB.

See fromCsvToRows, toCsv, and csvToTsv for the reference.