Jennifer - Milestones
Development is split into milestones. Each milestone produces a working interpreter that runs a strictly larger subset of the language.
Note on compaction. Older milestone entries are periodically compacted - condensed to a short summary (and sometimes grouped) - to keep this file readable and its length in check. The compacted text records what shipped, not the full implementation design. For the detailed design, rationale, and step-by-step evolution of any milestone, dig into the git history (the commits for that milestone) - that is the authoritative record of how it was built.
M1 - End-to-end MVP
Smallest vertical slice that proves the pipeline (source → tokens → preprocessed tokens → AST → result):
- Types:
int,string def x as int init 5;,$varreferences, method defs (zero-arg, top level),import "file.j";,use io;, single-argprintf- Arithmetic
+ - * / %on ints; comments#and/* */ - Source-context caret in error messages
- Golden-file integration test and TinyGo build verified
Exit criterion: ./jennifer run examples/hello.j prints 42.
M2 - Types, constants, scoping, control flow
Rounds out the "ordinary" feature set:
- New types
float,null,boolwith literals3.14,null,true,false - Uninitialized
def x as T;givesT's zero value def const NAME as TYPE init VALUE;(reassignment is an error)- Nested block scoping; inner scopes cannot redeclare visible names
- Assignment statement
$x = EXPR; - Comparison
< > <= >= ==,+for string concat,int↔floatpromotion - Escape parsing in
'...'strings (previously only"...") - Control flow:
if/elseif/else,while,for, all requiringboolconditions (no implicit truthiness)
M3 - Methods with parameters and return values
func name(a as int, b as string) { ... }with typed parameters, by-value argument passing, call-site arity + type checksreturn;andreturn EXPR;; recursion workssprintfand format verbs%d %f %s %t %v %%for bothprintfandsprintf- The omnibus
stdlibretired in favor of topic-based libraries;iois the first.
M4 - Polish & ergonomics
- Logical operators
and,or,not(word-based, short-circuit) - Unary minus
- Python-3 division:
/always returnsfloat; newdivkeyword for floor division (//is taken by line comments) - Floats always display with a decimal (
5.0, not5) so the type stays visible - New libraries (all
use-gated):convert,math,strings - Interpreter gained
RegisterConstso libraries can expose constants (PI,E).
M5 - Interpreter improvements
- Cross-file error sources - errors raised inside an imported
.jdisplay the line from the imported file. See technical/interpreter.md > Errors and positions. - REPL -
jennifer repl, persistent globals/methods/imports across inputs, multi-line input via brace balancing, expression results printed. See technical/cli_repl.md > REPL. - REPL line editor - cursor keys, Home/End, word motions, Ctrl+W / Ctrl+U / Ctrl+K, in-memory history (Up/Down), Ctrl+C cancel. Non-TTY stdin falls back to plain line reading. See technical/cli_repl.md > Line editor.
- Auto-loaded
corelibrary - new library kind, pre-imported at startup; writinguse core;is a runtime error. Contents:JENNIFER_VERSION(agit describe-derived build-version constant) andlen(polymorphic over strings now; lists/maps in M6).lenmoved here fromstrings. (M15.4 later promotedlento a language built-in and deletedcore; see M15.4 for the migration.) Version injection details at technical/cli.md. - Formatter -
jennifer fmtre-emits canonical source per user-guide/style-guide.md. Token-level walker so file imports and user-written parentheses survive. See technical/cli_fmt.md > Formatter. - Inspection subcommands -
jennifer tokens <file>dumps the lexer output;jennifer ast <file>dumps the preprocessed AST as JSON. See technical/cli_inspect.md > Inspection. - Underscore-in-constants - constant names became
[A-Z]+(_[A-Z]+)*, enablingMAX_RETRIESand theJENNIFER_VERSIONrename. See technical/lexer.md > Identifier rule. - Documentation overhaul -
docs/technical.mdsplit intodocs/technical/<topic>.md;docs/lib_*.mdmoved todocs/libraries/; newdocs/user-guide/style-guide.md.
M6 - Lists and maps
Two new compound types - list and map - plus the strings library functions deferred until compound types existed.
- Syntax:
def xs as list of int init [1, 2, 3];,def m as map of string to int init {"a": 1};. Index read/write$xs[i],$m["k"], chains$g[i][j]. Iteration viafor (def x in $coll) { ... }(new keywordin). New tokens[ ] :and keywordslist,map,of,to,in. - Semantics: value-typed (copy on assignment and on function-parameter binding; no aliasing);
constis deep ($NUMS[0] = ...is a runtime error ifNUMSisconst); out-of-bounds list reads/writes and missing map keys are positioned runtime errors; map iteration is insertion-order deterministic. - Type system:
parser.Typebecame a recursive struct (Element,KeyType,ValType*Typeslots), so nesting likelist of list of intfalls out without depth cap. 3+ levels is a documented code smell. - Stdlib:
core.lenextended to lists and maps;core.has(m, key)for membership tests;strings.split,strings.chars,strings.joinfinished. - Tooling: formatter handles
[...]/{...}per user-guide/style-guide.md (no inner padding, space after,/:, block-vs-map disambiguation via a small brace stack); AST JSON emitter handlesListLit,MapLit,IndexExpr,IndexAssignStmt,ForEachStmt.
See user-guide/types-and-values.md > Lists and maps for the user-facing tour, and technical/grammar.md / technical/interpreter.md for the implementation contract.
M7 - printf modifiers, stdin input, comment/division swap
A breaking syntax change to free up // for integer division and to allow shebangs, the long-promised format-verb modifier system, and the first stdin-reading builtins.
- Comments and integer division (BREAKING). Line comments moved from
//to#, freeing//for floor division (Python 3 shape).divkeyword removed. A Jennifer file can now begin with#!/usr/bin/env -S jennifer run. (s)printfformat-verb modifiers. Each format verb except%vaccepts a pipe-separated, order-independent flag list:%verb[|key=value]*. Modifiers shape presentation only - data transformations (case=upperon strings, markdown rendering, etc.) are explicitly out of scope; libraries do that work. Verbs gained:pad/max/align/mode(%s);pad/fill/align/base/sign/group/sep(%d);prec/trim/sci/pad/align/sign(%f);case(%t); sharednull=empty|null|literal(...)across all four typed verbs.%vdeliberately takes none.- Format-string breaking change.
|immediately after a verb now starts a modifier list. Pre-M7 strings with|as a literal separator ("%d|%d") need either a different separator or the||escape (parallels%%). iostdin input. New builtinsreadLine(),readLine(prompt),eof()- one-line-at-a-time reads with an explicit EOF predicate (while (not eof()) { ... }). Refuses inside the REPL since the line editor owns stdin.- Internals. Builtin signature changed from
func(out io.Writer, args)tofunc(ctx BuiltinCtx, args)so stdin and the REPL flag are plumbed symmetrically with stdout. Mechanical refactor across the ~30 existing builtins.
See:
- libraries/io.md - full modifier and input reference.
- technical/lexer.md and technical/grammar.md - the comment / division syntax change.
- technical/rejected.md - what the modifier system deliberately doesn't do (data transformations,
%aaggregate,null=sql/null=skip) and why the literal-pipe lookahead alternative was turned down. - technical/interpreter.md > Builtins and libraries - the
BuiltinCtxsignature.
M8 - System library namespacing
A hybrid namespace model so domain libraries can ship without polluting the bare-name pool, plus the first real namespaced library (os) so the machinery has a non-synthetic exercise.
- Hybrid model. Essential libraries (
io,convert,math,strings, auto-loadedcore) stay flat - their builtins are bare names. Domain libraries register through a new namespaced API (RegisterNamespaced/RegisterNamespacedConst) and are addressed byprefix.name(...)/prefix.NAME. The library's name doubles as the namespace prefix. - Qualified calls and constants. New AST nodes
QualifiedCallExprandQualifiedConstRefExpr; parsed asIDENT "." IDENT(then(decides). Lookup is keyed by(namespace, name)and gated byuse lib;. use NAME as ALIAS;aliasing. Optionalasclause onuse. Rename-not-addition: afteruse bio as b;onlyb.resolves,bio.foo()errors with a "did you meanb?" hint; the canonical namebiois freed for ordinary identifier use. Matches Python'simport foo as bar. Aliasing is rejected for flat libraries (use math as m;errors as meaningless).- Namespace prefix is a reserved identifier. After bare
use bio;,func bio() {}errors withshadows imported namespace 'bio'. Afteruse bio as b;, onlybis reserved. - No migration. The change is purely additive; all five flat essentials continue to work unchanged.
- Demo library
os(minimal slice). First namespaced library:os.platform() -> string,os.getEnv(name) -> string,os.JENNIFER_LF,os.JENNIFER_OS. Two functions plus two constants - enough to exercise namespaced zero-arg calls, namespaced calls with arguments, namespaced constants, and aliasing end-to-end. Expands in M15.1.
See:
- libraries/os.md - the shipping demo library.
- libraries/index.md - flat vs namespaced policy and the rule for library authors.
- user-guide/imports.md > Namespaced libraries and aliasing - user-facing reference for
use NAME [as ALIAS];and qualified calls. - user-guide/style-guide.md > Namespaced calls - spacing convention around
.. - technical/grammar.md - EBNF for
qualifiedCall/qualifiedConstRefand theuse ... as ...shape; AST table entries for the new nodes. - technical/interpreter.md > Namespaced libraries (M8) - registration API,
nsPrefixes/nsAliasedAwayresolution tables, no-shadowing rule for namespace prefixes.
M9 - Collection operations
Two new namespaced libraries cover the M6-deferred list/map manipulation helpers, a small append sugar shortens the common write pattern, and two follow-on breaking changes tidy up the flat-vs-namespaced split.
listslibrary (use lists;, namespaced).lists.push,lists.pop,lists.first,lists.last,lists.head,lists.tail,lists.reverse,lists.sort,lists.contains,lists.concat,lists.slice. Non-mutating - every function returns a new list.sortaccepts numeric, string, or bool elements (mixed int/float promotes; other mixes error); comparator-based sort is deferred until methods are first-class.mapslibrary (use maps;, namespaced).maps.keys,maps.values,maps.has,maps.delete,maps.merge. Same shape.maps.deleteof a missing key errors (strict at boundaries, matching$m[missing]);maps.mergelayers the second arg over the first.- Sugar:
$xs[] = item;- write-only target meaning "just past the end of the list". Equivalent to$xs = lists.push($xs, item);. Reads of$xs[]and chained forms ($xs[0][]) are parse errors; non-list targets error at runtime. New AST nodeAppendStmt. - BREAKING:
has()moved fromcoretomapsasmaps.has(m, key). Barehas(...)callers now needuse maps;and the qualified form.haswas the only non-polymorphic name in core;lenstays because it genuinely spans string / list / map. - BREAKING:
stringslibrary moved from flat to namespaced.upper(s)→strings.upper(s),contains(s, sub)→strings.contains(s, sub), etc. across all 15 functions.use strings;itself is unchanged. The M8 library-author rule named exactly these collision-prone verbs (contains,split,replace,join); acting on it now keeps callers off the wrong shape before more libraries arrive. After M9 the remaining flat libraries areio,convert,math, and auto-loadedcore.
See:
- libraries/lists.md / libraries/maps.md - function reference for each new library.
- libraries/strings.md - now namespaced (M9 migration note at top).
- libraries/index.md - updated flat-vs-namespaced catalog and the library-author rule.
- user-guide/imports.md and user-guide/types-and-values.md > The
$xs[]append sugar - user-facing reference. - technical/grammar.md - EBNF and AST entry for
AppendStmt.
M10 - Namespace-first library architecture
A pre-language-completion API-shape correction: every library is now namespaced, with bare-name globals reserved as a narrow core-only exception. Small implementation surface, large API shape; pre-1.0 is the window for this kind of change.
- BREAKING:
io,math,convertmigrate to namespaced-only.printf(x)→io.printf(x),sqrt(x)→math.sqrt(x), etc. The "io is special, keep it flat" alternative was considered and rejected at kickoff to keep a uniform "every call carries its library name" rule.strings,lists,maps,oswere already namespaced (M9/M8). - BREAKING:
convert's four conversion callees are renamed toconvert.toInt,convert.toFloat,convert.toString,convert.toBoolso they don't collide with the type keywords (int,float,string,bool);convert.typeOfkeeps its name. Theto-prefix also reads as English ("convert to int") at the call site. - BREAKING: file-splice keyword
import→include.include "x.j";is the textual splice; theimportkeyword is reserved for the M17 module system and produces a migration-hint error today. Mixing-mistake diagnostics updated. - BREAKING for embedders: registration API renamed.
Register/RegisterConst→RegisterGlobal/RegisterGlobalConst, making their role explicit ("expose this name globally"). The namespaced API (RegisterNamespaced/RegisterNamespacedConst) keeps its name and is the recommended default. Per-library storage (globalFnsByLib,globalConstsByLib) so two libraries with the same global name can no longer silently overwrite each other at Install time; the resolution map is populated byprocessImportswhen a library activates. mathabsorbs the planned non-crypto random helpers:math.rand(),math.randInt(lo, hi),math.randSeed(n). Three functions don't justify their own library under the new threshold (next bullet); pseudo-random fitsmath's pure-numeric charter. The crypto-grade variant still ships in M20.1crypto. The originally planned M14.2randomlibrary is removed.coreis the only library publishing bare-name globals.lenandJENNIFER_VERSIONonly - nocore.len/core.JENNIFER_VERSIONqualified form, because shipping the same name two ways violates stance #1.coreis the auto-loaded escape hatch, and its asymmetric exposure is the whole point.- Three globals-publishing rules in
processImports, all forward-looking (inert today sincecoreis the only globals-publishing library and can't beused):- Duplicate
useof a globals-publishing library is rejected (library 'X' already in scope); REPL no-ops a repeat. use X as Y;whereXhas globals but no namespaced names is rejected as meaningless.- Two active libraries publishing the same global name are rejected at the second
use(library "B" collides with already-active library "A" on global "VER"). The pre-M10 flat-only-alias-meaningless check is removed for the general case but kept as rule 2.
- Duplicate
- Library-author guidance updated. The
docs/libraries/index.md"flat vs namespaced" framing is retired; the new policy is "every library is namespaced; onlycoreships globals viaRegisterGlobal." The "deserves its own library" threshold is raised from M8's "3+" to "5+ functions or constants": anything smaller folds into the most-related existing library. The non-crypto random helpers (3 functions) are the first case the new rule caught.
See:
- libraries/io.md, libraries/math.md, libraries/convert.md - migrated library references.
- libraries/index.md - retired flat-vs-namespaced framing; new library-author policy and 5+ threshold.
- user-guide/imports.md -
useandincludekeyword reference; namespaced-call and aliasing rules. - user-guide/types-and-values.md -
convert.toInt/convert.toFloatexample placement in the "explicit conversions" section. - technical/rejected.md - "Methods on structs" (M14.3 trigger; recorded here in M10's wake because M10's review touched the same call-shape question) and other related rejected alternatives.
No new language features land here - that's M11.
M11 - Control-flow completion
Closes the biggest daily-use gap in the language and rounds out the printf modifier table at the same time. Five new keywords (break, continue, repeat, until, exit) and two new printf features.
break;/continue;in every loop kind (while/for/for-each/repeat). Innermost loop only; misuse outside a loop or across a method-call boundary is a positioned runtime error.continuein C-styleforstill runs the step before re-checking the condition (matches C/Go).repeat { } until (cond);post-test loop. New keywordsrepeatanduntil;do { } while ...considered and rejected because the inverted condition is the whole point of switching tountil.exit;/exit EXPR;terminate the whole program (exit code 0 / EXPR-as-int). Distinct fromreturn(method-scoped): skips every caller frame and remaining top-level statement. Implemented as anExitSignalsentinel error the CLI translates into the OS exit status.- Bundled: printf
%s|align=centerrounds out the align set. Rejected on every other typed verb (centred numbers break columnar output). - Bundled: printf
%aaggregate verb for lists and maps (deferred from M7; unblocked by M6 + M9). Modifiers:sep,kv,open,close,depth=N,null=skip. The modifier-list parser was extended with a"..."quoted-value form (%a|sep=", ") so values can contain spaces / reserved characters; standard\n \r \t \\ \"escapes. - Post-dot name relaxation. Reserved words read as identifiers in the name slot of a qualified call (
strings.repeat,lists.breakif anyone wrote one), preserving thestrings.repeatlibrary function afterrepeatwas reserved as a loop keyword.
See:
- user-guide/control-flow.md -
repeat/until,break/continuescope rules,exitvsreturn. - libraries/io.md -
%amodifier table,%s|align=centerexample, quoted modifier values. - technical/rejected.md -
%a|json=*/%a|xml=*/%a|yaml=*(serialisation modifiers stayed rejected even after%aitself shipped) and thedo { } whileshape for the post-test loop.
M12 - Bytes and bit operators
Adds the buffer-shaped primitive and the bit-twiddling vocabulary the standard library needs for hashing, encoding, crypto, and network code in later milestones.
- New primitive type
bytes- mutable byte sequence; value semantics on assignment / parameter binding; deep-const. Reads yieldintin[0, 255]; writes accept the same range and reject anything else. Append via the existing M9$b[] = byte;sugar.len($b)returns the byte count. - New
convert.bytesFromString(s, codec)andconvert.stringFromBytes(b, codec)- bytes ↔ string codecs. Only"utf-8"today (further codecs ship in M15.7encoding). Invalid UTF-8 input is an error - no silent replacement characters. - Bit operators on
int:& | ^ ~ << >>. Python-style precedence (comparison <|<^<&< shifts <+ -), so$x & 0xff == 0parses as($x & 0xff) == 0.~is bitwise NOT. Shifts are arithmetic; negative count rejected; count >= 64 saturates to 0 / -1.^ships as a primitive operator (CPU primitive with unique algebraic properties - same justification-has against being composable from+and unary-). - Non-decimal integer literals: hex
0xff, octal0o755, binary0b1010_0110._accepted between digits in any base (including decimal1_000_000and float mantissas). Never adjacent to the prefix or another_. Lexer-only change. - Resolves M7-deferred stdin builtins:
io.readBytes(n) -> bytes(exact n; partial at EOF thenio.eof()becomes true) andio.readChars(n) -> string(n runes, UTF-8 decoded). Both compose with M7'sio.eof()unchanged.
See:
- user-guide/types-and-values.md -
bytestype, value semantics, index-write rules. - libraries/convert.md - codec functions, UTF-8 strictness.
- libraries/io.md -
io.readBytes,io.readChars. - user-guide/control-flow.md - bit-operator precedence table.
- user-guide/syntax.md - non-decimal literals + digit separator.
M13-M13.2 - structs and catchable errors
The composite-data milestone, batched in dependency order: M13.1 ships the struct mechanism, M13.2 the recoverable-error story built on it (the canonical error value is a struct), together unblocking composite library returns. Design detail: git history, the user-guide (types-and-values, control-flow) and technical/interpreter.md docs, and examples/structs.j / examples/trycatch.j (plus the matching showcase.j sections).
| M# | Topic | Summary |
|---|---|---|
| M13.1 | structs / records | def struct Name { field as type, ... }; at top level (hoisted before the first statement; duplicate names error in Run, silently redefine in the REPL). Literals Name{ field: expr, ... } with every field required; def x as Name; zero-fills, recursing nested struct fields. Field read $p.field / write $p.field = ...;; lvalue chains mix [index] and .field freely ($L.from.x = 5;, $bag.items[0] = 99;) through one shared index/field walker. Value semantics like lists / maps; const deep at any depth. Strict at boundaries - unknown struct type, missing / unknown field at a literal, field-type mismatch on write, and field access on a non-struct are all positioned errors. Runtime: a KindStruct tagged-union value. |
| M13.2 | try / catch / throw | Catchable errors (keywords try / catch / throw). try { body } catch (NAME) { handler } binds the thrown value to $NAME in a fresh per-handler scope; throw EXPR; raises any value, convention the auto-hoisted Error{kind, message, file, line, col} struct. The runtime errors today's builtins / ops raise (out-of-bounds, missing key, type mismatch) are wrapped into Error on entry to the catch (kind defaults to "runtime" until a site opts into a tag); throw $err; in a catch re-raises to the enclosing try. Not catchable: exit (propagates through try), and return / break / continue (control flow, flow through try unchanged). No finally, no typed catch in v1. Internals: an ErrorSignal sentinel parallels ExitSignal, runtimeError.Kind threads the tag, and the Error struct is reserved (user code may not redefine it). |
M14 - Lexer comment + blank-line preservation
Closes the two M5-deferred items (fmt drops comments, fmt drops blank lines). No language change - the runtime still never sees comments.
- Lexer emits trivia tokens (
TOKEN_COMMENT_LINE,TOKEN_COMMENT_BLOCK,TOKEN_COMMENT_SHEBANG,TOKEN_BLANK_LINE). Shebang on line 1 col 1 is its own kind; runs of blank lines collapse to one. - Preprocessor and parser strip trivia at entry;
jennifer fmtwalks the raw lexer stream and re-emits trivia via a dedicatedemitTriviapath that doesn't disturb the surrounding state machine. - Block comments nest via depth counter; unterminated comments error at the outermost
/*. - Token-level over AST-level: the original spec proposed AST-attached
LeadingComments/TrailingCommentslots and ajennifer ast --with-commentsflag - dropped in favour of the simpler token-level path. Add them back if a future doc generator needs structured per-statement attachment.
See:
- user-guide/style-guide.md - Comments section (block comments nest; inline-comment spacing exception).
- technical/lexer.md - trivia emission, shebang detection, nesting depth counter.
- technical/cli_fmt.md -
fmt's trivia re-emission.
M15 - foundational libraries + first public release
Nine sub-milestones - two language (M15.2, M15.4), the rest library / tooling / release - that built out the foundational stdlib and shipped the first public release. Design detail: git history and the linked per-library docs. Two API patterns established here recur across later libraries:
- Codec-table shape - the algorithm / format / codec is a string argument (
hash.compute(b, algo),crc.compute(b, algo),encoding.encode(s, codec),encoding.toText(b, format)), collapsing parallel verbs into one (stance #1) and sidestepping the letters-only identifier rule that rejectshash.md5(...). - Integer-handle struct for opaque resources - a namespaced struct with a single
id as intindexing a Go-side map (os.Process,hash.Stream,crc.Stream).
| M# | Topic | Summary |
|---|---|---|
| M15.0 | existing-library extensions | lists.shuffle(xs) (Fisher-Yates, respects math.randSeed) and lists.range(start, end[, step]) (half-open; the single-arg form is deliberately omitted, stance #2). |
| M15.1 | os reshape + meta | Immutable per-run host facts became uppercase constants (PLATFORM / ARCH / EOL / DIRSEP / PATHSEP / ARGS), operations stay functions (getEnv / hasFlag / flag); dropped the JENNIFER_ prefix. New meta library for interpreter identity (VERSION / BUILD); the CLI forwards trailing args to os.ARGS (script path at index 0). Breaking renames (JENNIFER_VERSION -> meta.VERSION, os.platform() -> os.PLATFORM, os.JENNIFER_LF -> os.EOL), old names now plain "undefined" errors. |
| M15.2 | library-provided namespaced structs (language) | def x as lib.Name; type syntax + lib.Name{field: ...} literals + the Go Interpreter.RegisterNamespacedStruct API, reusing M13.1's value-semantics / deep-const / strict-boundary machinery (only resolution differs). User code can't register structs (Go-side only); no methods-on-structs / inheritance. Unblocks os.Result/Process, time.*, the hash/crc streams, later fs/net. |
| M15.3 | os external-program execution | First consumer of M15.2. os.Result{exitCode, stdout, stderr} + os.Process{pid}; os.run(argv) -> Result (blocking), os.spawn(argv) -> Process (non-blocking) + wait / poll / kill. argv is always list of string (no shell parsing - use ["sh", "-c", $cmd] explicitly). Non-zero exit codes are values, not errors. First user-visible two-binary split: jennifer-tiny has no os/exec, so it returns a friendly "use the default jennifer binary" error. |
| M15.4 | len built-in, core removed (language) | Promoted len(EXPR) from the auto-loaded core library to a reserved keyword + primary expression (polymorphic over string / list / map / bytes). Deleted internal/lib/core/; use core; returns a friendly migration error pointing at the built-in and meta.VERSION/BUILD. Every library now lives behind a use NAME; (stance #2, no exceptions). |
| M15.5 | time | Instants, durations, fixed-offset zones, strftime format / parse, ISO 8601 round-trip. Structs time.Time{nanos, offset} (private fields), time.Duration{nanos}, time.Zone{offset, name} (public, so an IANA/DST companion can build them). Granularity is a formatting property, not a type; Unix timestamps are constructor/accessor pairs, not a type. IANA/DST stay out of the fixed-offset core (a Go-backed extension, not a .j data map). Three parts: core + Unix + calendar + 1-based-ISO-weekday + arithmetic; strftime (%Y %m %d %H %M %S %z %a %A %b %B %j %u %%) + time.zone/inZone + time.UTC const alongside time.utc() + time.iso/fromIso; and examples/benchmark.j (the TinyGo-vs-Go suite; the sieve became trial-division since value-semantic list mutation makes a sieve O(N^2)). |
| M15.6 | hash + crc | Two parallel codec-table libraries: hash (crypto-style digests md5 / sha1 / sha256), crc (crc32 IEEE / crc64 ECMA) - the split keeps "transport integrity" vs "content addressing" visible at the import line (mirrors Go's crypto/* vs hash/crc*). Output is raw bytes. compute(b, algo) one-shot + stream(algo) / update / finalize (the integer-handle pattern). No convenience wrappers like hash.md5String - compose convert + encoding (stance #1). Struct hashing deferred. |
| M15.7 | encoding | Introspection (isAscii / lenBytes / lenRunes), binary-to-text toText / fromText (hex / base64 / base64-url), character encode / decode / codecs (ascii / iso-8859-1 / windows-1252 / ebcdic). The cross-kind UTF-8 pair stays in convert (M12); encoding owns the codec proliferation. Exact-match codec / format names (the original alias / case-normalisation layer was later dropped, stance #2); Windows-1252's five undefined positions reject symmetrically. The long-tail single-byte codecs shipped later in M16.15. |
| M15.8 | distribution + first public release | Packaging / CI / release only, no language change. CI: PR gate (go vet + gofmt + go test ./... + make build + per-binary smoke + em-dash scan); release on bare-semver tags cross-compiling linux/{amd64,arm64}, QEMU-smoke-testing the non-native arch, running the benchmark, publishing a draft Release. Packaging under packaging/{debian,arch,mime,man}/ (.deb via build-deb.sh with man pages + text/x-jennifer MIME; AUR PKGBUILD-bin / -git, auto-filled by the pipeline). Docs site via pinned mdBook 0.5.3 -> GitHub Pages. Conventions kept: bare semver tags (no v prefix), no top-level LICENSE (LGPL text in packaging/debian/copyright + a README link). Deferred (not gated): the macOS / Windows cross-build and a real apt repo stay in Requirements for 1.0.0 stable. |
M16 - I/O libraries and developer tooling
System libraries that touch the OS or do significant compute, opened by the spawn concurrency primitive (M16.0) the I/O libraries build on, then a lint / profile / test developer-tooling trio and a run of self-contained data libraries. Design detail: git history and the linked per-library / per-tool docs.
| M# | Topic | Summary |
|---|---|---|
| M16.0 | lightweight concurrency | spawn { ... } (block primary expression), task of T (new compound kind), the task library (wait / poll / discard / waitAll / waitAny). Goroutine-backed but race-free by construction: snapshotForSpawn deep-copies a globals+locals snapshot at launch; tasks share only the TaskState pointer (the one value-semantics carve-out). A per-run registry loud-fails unobserved task errors at exit (an undiscarded non-terminating spawn hangs at exit - the documented trade-off); task.wait re-raises a body error at the wait site; waitAny is the runtime's only reflect.Select. TinyGo builds with -stack-size=4mb -scheduler=tasks. |
| M16.1 | fs | Blocking filesystem I/O composed with spawn (no *Async): whole-file read/write/append (String/Bytes), metadata (exists/isFile/isDir/stat -> fs.Stat), dir ops with the two-verb recursion split (mkdir/mkdirAll, remove/removeAll, rename/list/walk), buffered fs.File handles (open/readLine/.../close, eof peeks a byte). Path- vs handle-form verbs dispatch on the first-arg kind; fs.File shares state across copies (handle carve-out). |
| M16.2 | net | TCP (connect/listen/accept/readBytes/writeBytes), UDP (listenUDP/sendTo/recvFrom), DNS (lookup/reverseLookup); polymorphic close/address over three handle registries; blocking calls compose with spawn (accept-loop-per-connection). Build-tag split: jennifer-tiny returns friendly-error stubs (no netdev in TinyGo). |
| M16.3 | regex | RE2 (Go regexp, linear-time) over strings: matches/find/findAll/replace/split/escape; regex.Match{text,start,end,groups,groupsNamed} (rune indices, start=-1 = no match); implicit 128-entry LRU pattern cache. Both binaries. |
| M16.4 | testing (primitives) | The irreducible system side a .j test framework needs: testing.run(name) invokes a zero-arg user method via Interpreter.CallByName, times it, classifies failures into a testing.Result; results/reset (mutex-guarded); report (text / TAP / JUnit). The one place exit is caught (Go-level, so language try/catch still can't). |
| M16.5 | interpreter performance pass | Five sub-parts, behaviour unchanged: .1 shared-marker COW on compound Values (append-in-a-loop O(N^2) -> amortised O(N)); .2 parse-time lexical slot resolution ((Depth,Slot) coordinates + a slots slice; undefined/shadowing promoted to parse-time errors); .3 pooled + pre-resolved + slot-bound method-call frames; .4 namespaced-call / comparison / arg-bind / root-cache fast paths; .5 compile-time constant folding + a Share() scalar fast path. Numbers in tinygo.md. |
| M16.6 | lint | jennifer lint flags compile-legal-but-suspect patterns: grouped IDs (L0nn source errors / L1nn correctness / L2nn style / L3nn lifecycle), # lint-disable[-file]: IDS suppression, --checks / .jennifer-lint config, human / JSON / GitHub output (a JSON pipeline stays valid even on a source error); exit 0/1/2. !tinygo. |
| M16.7 | profile | jennifer profile attributes work to .j source positions (what go tool pprof can't): a statement profile (hit count + self/cumulative wall-clock) and an --allocs value-copy profile; table / pprof (hand-encoded gzipped protobuf) / Chrome-trace output; program output to stderr so the profile owns stdout. !tinygo. |
| M16.8 | testing framework consolidation | An assertion vocabulary on M16.4 (assertEqual ... assertThrows, throwing Error{kind:"assertion"} at the call site), CallByNameWith/runWith arg dispatch, and the jennifer test subcommand (test* discovery or --filter, setUp/tearDown, --isolated per-test subprocess, text/TAP/JUnit, exit 0/1/2). Builtins can now raise a catchable error via interpreter.RaiseError. |
| M16.9 | json | Hand-rolled RFC 8259 encode/decode onto the tagged-union Value (no encoding/json, no reflect). encode/encodePretty/decode; structs and map of string to V -> objects, bytes -> base64, integral numbers -> int else float. Also closed a type hole: a generic collection (a fresh literal or decode result) is validated entry-by-entry against the declared element type at every binding boundary. Decode's return shape was later superseded by M16.16 (json.Value). |
| M16.10 | uuid | RFC 9562: generate("v4") (random) / generate("v7") (time-ordered) - the version a string arg since identifiers were letters-only - plus parse/isValid/version and constant NIL. Randomness was later repointed from math to crypto's crypto-grade source (M20.1), making v4/v7 unguessable; M22.3 renamed generate("v4") -> v4(). |
| M16.11 | compress | Byte-stream size reduction (distinct from encoding's representation codecs): pack/unpack for gzip/zlib/deflate with an optional fast/default/best level, plus a streaming compress.Stream. Go compress/*, TinyGo-clean. |
| M16.12 | archive | tar / zip containers over bytes (no fs, value-semantic): pack/unpack (verbs shared with compress) for tar/zip/tar.gz; a bundle is a list of archive.Entry{name,data,mode,mtime}. Go archive/tar+archive/zip. |
| M16.13 | os.isTerminal | os.isTerminal("stdout"/"stderr"/"stdin") -> bool (the ANSI-colour gate) via the char-device mode bit (os.ModeCharDevice) - pure stdlib (keeps x/term CLI-scoped), TinyGo-clean; an unstattable stream reports false. |
| M16.14 | net TLS | net.connectTLS(address) (implicit) and net.startTLS(conn) (in-place STARTTLS upgrade), both yielding the transport-agnostic net.Conn. Cert verification on by default, net.TLSOptions{skipVerify, caCert} opt-out. Go crypto/tls on the !tinygo build (stubbed on tiny). |
| M16.15 | encoding completion | toText/fromText gained quoted-printable, base32/base32-hex, ascii85, z85; the full ISO-8859-{1..16} / Windows-{1250..1258} single-byte codecs, generated from the Unicode mapping files (gen_codecs.go -> codecs_gen.go) so only ascii/ebcdic stay hand-written. Exact-match codec/format names (the normalisation layer was dropped, stance #2). |
| M16.16 | json.Value | The strict home for heterogeneous JSON without a language top type: json.decode returns an opaque json.Value - the first KindObject (the opaque sibling of KindStruct: discriminated by (namespace, name), minted only by a library, rejecting operators / [i] / .field). convert.typeOf -> "object", convert.objectType -> "json.Value". Reads + non-mutating writes share JSON Pointer (RFC 6901): typeOf/get/has/keys/length/as*/isNull and map/list/set/insert/append/remove/move (strict / no-vivify, - end-marker), node types in list/map vocabulary; a displayer renders a handle as its JSON. json.decode's return type changed (a pre-1.0 break); the decoder's number grammar tightened to json.org. No any keyword (rationale in rejected.md). |
M17 - module system for Jennifer-coded libraries
Jennifer-coded libraries get their own namespace, scope, and explicit exports via a real module boundary (import "x.j" as x;, a parser
- interpreter feature) beside the textual
include "x.j";splice (a preprocessor
operation for composing one module from several files). Settled cross-cutting decisions (turned-down alternatives in rejected.md): each module is its own resolution context (own use set, namespace + export tables); the module top level is declarations-only (no mutable state, so spawn capture is unaffected); the one global Error stands (modules add distinctly-named error structs, never redefine it); private by default, a leading export publishes (no public / private keyword); multi-file modules assemble via include behind one entry file (no directory-as-module); a module needs a filesystem (an FS-less jennifer-tiny host fails with the ordinary search-path error). Design detail: git history, imports.md, interpreter.md.
| M# | Topic | Summary |
|---|---|---|
| M17.1 | source tree + resolution | internal/module Classify + Resolve map an import path (local ./ / ../, absolute /, or a bare name walked on the search path) to a canonical absolute path, rejecting an ambiguous name (found in two search dirs) and a not-found. The system module dir resolves --sysmoddir > JENNIFER_SYSMODDIR > compile default (surfaced as meta.SYSMODDIR; a named-but-missing dir refuses to start, the compile default is best-effort); -I DIR (repeatable) appends after it. jennifer version -v reports the layers. |
| M17.2 | import statement + loader | import "path.j" [as NAME]; is a real statement (ModuleImportStmt - the preprocessor passes it through, the parser builds the node). The loader runs each module in a fresh sub-interpreter sharing one moduleReg, so run-once (cached by canonical path), depth-first post-order init, and cycle detection (erroring with every edge named) all fall out of the recursion. Load-time errors (a parse error or a throwing def const init) aren't catchable (import is a declaration, not an expression). fmt / ast / tokens round-trip an import line. |
| M17.3 | module scope + namespacing | Declarations-only top level (checkModuleDeclarationsOnly: only def const / def struct / func / use / import; scripts keep both). loadModuleImports binds each alias (the as NAME, or the file stem) into moduleAliases, collision-checked against library prefixes. Consumer resolution rides the qualified-reference eval layer: evalQualifiedCall / evalQualifiedConst dispatch alias.fn(args) into the module's own interpreter via CallByNameWith (args evaluated in the consumer, body run against the module's globals + methods) and read alias.CONST. use non-transitivity, run-once sharing, and -race safety all follow from the fresh-sub-interpreter-per-module model. |
| M17.4 | exports + visibility | export publishes a top-level func / def struct / def const; unmarked names stay private (reaching mod.helper is a positioned "not exported" error), and export in a run script is rejected. checkReferentialClosure rejects an exported field / parameter typed as a private module struct (library / namespaced types cross freely). Cross-module struct identity = boundary translation (retagStructs): a module's structs are bare inside it and re-tagged to (module-stem, name) as a value crosses out to an importer and back, so def p as mod.Point, mod.Point{...}, field reads, and pass-back all type-check while a.Point / b.Point stay distinct; the retag also tags the element-type metadata a list / map carries (retagType). A co-located MODULE_test.j overlay (a token splice in jennifer test) runs white-box tests against the module's private names. |
| M17.5 | ansi module | First module built on the system (pure .j, one use os; across the boundary; a real dogfood of import / export / resolution). Exports color / bgColor / style (bold / dim / italic / underline / reverse) / rgb truecolor / strip, plus per-colour and per-style shortcuts (ansi.red(s), ansi.bold(s), ...). The ESC byte is built from a one-byte bytes; strip uses regex; unknown names throw. Stateless + TTY-aware: enabled() re-reads NO_COLOR / FORCE_COLOR / os.isTerminal("stdout") per call (no toggle state; degrades to always-on when os.isTerminal is absent). Colour is a string wrapper, so a `%s |
| M17.6 | semver module | Strict SemVer 2.0.0 as a second pure-.j reference module (use strings / convert / regex, so both binaries), and the base a future jvc package manager needs. Exports Version{major, minor, patch, prerelease, build} + parse (throws on invalid) / isValid / toString; compare / lt / eq / gt (full SemVer precedence: numeric core, prerelease ranks below release, build ignored); isStable (0.y.z unstable by convention) / isPrerelease; incMajor / incMinor / incPatch; and sort (own pass over compare, since lists.sort is scalar-only). Strict - a loose 1.2.3.4 is rejected. parse uses the anchored RE2 pattern with named groups; precedence + sort are hand-written (the algorithmic dogfood). Building it surfaced + fixed the M17.4 retagType gap (a consumer list of semver.Version handed back into a module list of Version). Range / constraint matching (^1.2.0, >=1.0.0, ~1.2.3) deferred to jvc. |
M18 - Jennifer-coded modules
Built atop the existing system libraries. Each one ships as a Jennifer module under modules/ (the directory introduced in M17); none of them are compiled into the interpreter binary. Sub-milestones in priority order.
Forty sub-milestones (with their nested parts) shipped as pure-Jennifer modules/ (except where noted as a Go system library), each with the standard discipline: a 100%-passing *_test.j overlay, a cmd/jennifer/*_test.go integration test, a docs/modules/*.md reference, an examples/modules/*_demo.j, and catalog / README / JENNIFER.md entries. Per-function detail lives in docs/modules/; this table is the milestone-number index (numbers were assigned in rough priority order).
| M# | Module(s) | Surface |
|---|---|---|
| M18.1 | csv | RFC 4180 parse / format (+ *With for any delimiter), header-keyed toRecords / fromRecords. |
| M18.2 | htmlwriter | build an HTML element tree and render escaped HTML5 (element / text / raw / render). |
| M18.3 | markdown | Markdown -> HTML. |
| M18.4.1/.7 | mime | RFC 5322 / 2045 message build + parse, incl. RFC 2047 encoded-words. |
| M18.4.2/.4 | smtp / pop / imap | mail send + POP3 / IMAP receive over net (plaintext / STARTTLS / implicit TLS). |
| M18.4.5/.6 | sasl / idna | SASL auth encoders (incl. XOAUTH2); Punycode / IDNA domains. |
| M18.5 | redis | RESP2 client over net. |
| M18.5.1 | resque | Resque-wire-compatible background jobs on redis. |
| M18.6 | memcache | memcached text-protocol client over net. |
| M18.6.1/.2 | session / ratelimit | server-side sessions + fixed-window rate limiting on memcache. |
| M18.7 | http | HTTP/1.1 client over net (https:// via TLS). |
| M18.7.1/.3 | gotify / rest / oauth | push notifications; ergonomic REST layer; OAuth2 get-a-token - all on http. |
| M18.8 | toml (library) | RFC TOML 1.0 encode / decode; opaque toml.Value, JSON-Pointer walk. TinyGo-clean. |
| M18.9.1 | httpd (library) | HTTP/1.1 server engine over net/http; pull-loop accept / respond. |
| M18.9.2 | web + jennifer serve | .j routing framework over httpd (routes by handler name, :param, middleware, web.Context), dispatched by meta.callMain; serve runs / --watch-reloads a program. |
| M18.10 | flatdb | file-backed JSON document store over json + fs; JSON-Pointer query / edit; crash-atomic save. |
| M18.11 | gpio | Linux GPIO over the sysfs / character-device interface. |
| M18.12 | docblock | the Jennifer doc-comment format + parser (FileDoc tree, drift diagnostics). |
| M18.13 | mqtt | MQTT 3.1.1 pub / sub client over net. |
| M18.14 | prometheus | metrics exposition (produce) + retrieval (query the HTTP API). |
| M18.15 | label | industrial label printing: build / render (ZPL + cab JScript) / emit pipeline. |
| M18.16 | web cookies + sessions | cookie helpers + cookie-keyed sessions on the web framework. |
| M18.17 | totp | RFC 6238 TOTP: generate / verify / uri. Over hash.hmac + encoding + time. |
| M18.18 | webhook | GitHub X-Hub-Signature-256 HMAC sign / verify (pure) + send (over http). |
| M18.19 | s3 | S3-compatible object storage over http (AWS SigV4): connect / get / put / delete / listObjects. One module for AWS S3 + MinIO / R2 / B2. Renamed bucket -> s3 once the M22.2 digit-identifier rule made s3 a legal namespace (cf. M22.3's iic -> i2c); a pre-1.0 break, one batch across the module / overlay / Go test / demo / docs. |
| M18.20 | dotenv | .env config: parse / read / load (into env via os.setEnv). Over fs + strings + os. |
| M18.21 | cron | parse cron expressions; next(schedule, after) / matches. A calculator over time. |
| M18.22 | log | leveled structured logging (debug..error; text / logfmt / json) to stdout / stderr / file / RFC 5424 syslog. |
| M18.23 | ical | iCalendar (RFC 5545) build + parse: a Calendar of VEVENTs, escaped + line-folded, dates through time. |
| M18.24 | vcard | vCard (RFC 6350) contacts build + parse; shares the content-line codec (ical_vcard_shared.inc.j) with ical. |
| M18.25 | jsonl | JSON Lines (NDJSON): encode / decode + whole-file + streaming Reader, over json + fs. |
| M18.26 | ipnet | IPv4 / IPv6 addresses + CIDR math: parseAddress / toString (RFC 5952) / parse / contains / netmask / broadcast. |
| M18.27 | ntp | SNTP network-time client over UDP: query / queryWith -> Result (server time + clock offset + round-trip delay). |
| M18.28 | statsd | fire-and-forget StatsD metrics over UDP (count / gauge / timing / set); the push counterpart to prometheus. |
| M18.29 | influxdb | InfluxDB 1.x client on http: line-protocol Point builders + write; query -> parsed Series. |
| M18.30 | slack / discord | incoming-webhook chat notifiers on http: plain send + Block Kit / embed builders (sendMessage). |
| M18.31 | telegram | Telegram Bot API on http: sendMessage / sendPhoto / getMe, getUpdates long-poll (stateful receive loop). |
| M18.32 | websocket | RFC 6455 client over net (ws:// / wss://): handshake + masked send / receive (auto-pong, fragment reassembly). |
| M18.33 | amqp | AMQP 0-9-1 client for RabbitMQ over net: handshake, declareQueue, publish, get (Basic.Get pull), ack. |
| M18.34 | multipart | multipart/form-data (RFC 7578) build + parse (binary-safe); web.multipartForm pairs it with web. |
| M18.35 | pdfwriter | generate PDF documents (text / lines / rects, Standard-14 fonts, FlateDecode via compress); byte-identical output. |
| M18.36 | bloom / ringbuffer | data structures: a Bloom filter (probabilistic set) + a fixed-capacity ring buffer (bounded FIFO). |
| M18.37 | tengine | a text/template-subset engine over a json.Value tree (if / range / with / pipes / layout inheritance). |
| M18.38 | barcode | QR (Reed-Solomon over GF(256), masking, versions 1-10) + 1D (code128 / code39 / ean13 / ean8 / itf); SVG / PNG / terminal. |
| M18.39 | mikrotik | MikroTik RouterOS API client over net: sentence-based binary framing, talk / print / run, plaintext + MD5 login. |
| M18.40 | password | password generate / validate / score against a policy Schema; entropy-based complexity (non-crypto RNG). |
Enabling changes these modules pulled into the system side (each documented under its library):
net.setReadDeadline/net.setWriteDeadline- separate read and write deadlines for socket timeouts (M18.13, first shipped as a combinedsetDeadline; extended to UDP sockets forntp, M18.27; split into independent read/write calls so a read timeout cannot block a later write).io.eprintf- the stdout-printftwin that writes to stderr (a newInterpreter.Err/BuiltinCtx.Errwriter), the stderr sinklogbuilds on (M18.22).tomlandhttpd- two new Go system libraries (a char-by-char TOML parser and anet/httpserver engine both belong in Go, not a.jmodule); M18.8 / M18.9.1.meta.callMain/meta.definedMain- resolve a method against the entry program (retagging module-own struct args across the boundary), the capability thewebframework dispatches handlers through (M18.9.2).hash.hmac(RFC 2104) and thesha512digest - the HMAC primitivetotp/webhookbuild on (and thatjwt/ SigV4 will reuse).
M19 - cross-cutting tooling
The catch-all bucket for interpreter / tooling work belonging to neither the M18 .j modules nor the M20 Go system libraries: M19.1-M19.5 a correctness / performance hardening pass over the core + libraries; M19.6 the coverage tool; M19.7 the @scope/package vendored-module resolver; M19.8 the one-time org / vanity-path relocation; M19.9 the audit-driven hardening pass. No reflect, no TinyGo-cleanliness break. Design detail: git history and the linked docs.
| M# | Topic | Summary |
|---|---|---|
| M19.1 | interpreter concurrency-safety | Both interpreter data races fixed, each pinned by a -race stress test. snapshotForSpawn snapshots the launching goroutine's own root frame (effectiveGlobal(env)), not the live i.global, so a nested spawn no longer races the main goroutine's global writes. Declared struct types are stamped once, single-threaded, before any statement (resolveDeclaredTypesOnce, after loadModuleImports) with a parser.Type.Resolved marker, so the per-execution re-resolve is a read-only no-op (also fixes a latent aliased-library-struct "canonical is aliased" rejection). Error timing unchanged. |
| M19.2 | value representation cleanup | Removed the inert copy-on-write machinery (Value.shared, Share(), Ensure(), ensureCOW, the per-VarExpr-read Share()); the mutation sites grow the binding's own backing in place, reads return the binding directly. Value semantics rest (as before) on eager deep copies at every store site; the write-through alternative is in rejected.md. Dead COW reporting stripped from the --allocs profiler. A fresh list / map / struct literal RHS is already private, so execDefine / execAssign skip the redundant copy (rhsFreshLiteral), proven by a profiler-backed test. |
| M19.3 | runtime perf: maps + call/loop hot path | Maps gained an advisory hash index (Value.mapIdx, encoded scalar key -> position) guarded by a len(mapIdx) == len(Map) stamp, so $m[$k] = $v over N keys is O(N) not O(N^2) while insertion order + value semantics are untouched (any stale / duplicate-key / non-hashable case fails the stamp and falls to the linear scan; a 100k-key build + for-each runs sub-second where the quadratic path took minutes). Plus: execForEach / execFor borrow frames from envPool; DefineAt skips the shadow walk on the slot path; Run pre-sizes i.global's slots; the three mutation sites write the root binding through (Depth, Slot); lists.reverse/head/tail/slice/concat shallow-copy instead of deep-copying an argument they overwrite. |
| M19.4 | resource lifecycle + numeric strictness | os.spawn handles keyed by a monotonic id, not the OS pid (a recycled pid can't alias a handle or misroute wait/poll/kill); the reaper drains captured buffers to strings and drops the live *bytes.Buffers (idempotent wait / poll-after-wait preserved). Numeric strictness: convert.toInt and math.floor/ceil/round reject NaN / +/-Inf / out-of-int64 floats; math.abs(MinInt64) errors; the toml decoder errors on an int past int64 (json keeps its deliberate fallback); the most-negative int literal -9223372036854775808 parses to MinInt64 (folded at unary-minus with a 2^63 range check). |
| M19.5 | module struct identity: canonical path | Module structs were tagged only by the file stem, so two modules sharing a basename (a/util.j, b/util.j, or two @scope/package decks) produced identical (namespace, name) identity and a foreign struct passed the other's type check. Identity is now keyed by the module's canonical (resolved) path: Value and parser.Type gained a ModPath field that Equal / MatchesDeclared compare alongside StructNS (which stays the stem, for display, so %v still reads benchmark.Point); the boundary retag + method-parameter stamping thread the path. Two imports of the same file stay one type; different files differ - no import error. |
| M19.6 | .j code coverage | `jennifer test --coverage[=text |
| M19.7 | @scope/package resolution (vendored decks) | A leading @ is a vendored-deck reference expanded by one function (resolveVendor): the @ swaps in the vendor root and a reference not ending in .j gets the package-named entry appended, so @jennifer/bitcoin, @jennifer/bitcoin/, and @jennifer/bitcoin/utils.j all reduce to a plain absolute path (after which run-once cache + M19.5 path identity are untouched). The entry is <package>.j, so moduleStem gives the package name and the display namespace / default alias fall out (import "@jennifer/bitcoin/" binds bitcoin.); two same-package decks across scopes are distinct types, colliding only on the default alias (resolve with as). Vendor root via FindVendorRoot: --vendor > JENNIFER_VENDOR > nearest vendor/ above the program. Path safety: @ front-only, no ./.., the file must stay inside the deck; a missing root is a guided error. The jvc manager over this is M26. |
| M19.8 | relocation: jennifer-language org + vanity path | A one-time mechanical relocation, no language / interpreter behavior change: the repo moves to a jennifer-language GitHub org, and (separately) the Go module path moves off GitHub to a vanity path jennifer-lang.dev/jennifer served by a go-import meta page, so module identity is host-independent. Two distinct targets: the Go module path -> the vanity domain (go.mod + 112 .go imports rewritten; the meta page maps it to the org repo, plus a go-source tag for pkg.go.dev); human-facing URLs -> github.com/jennifer-language/jennifer (the -lang.dev vs -language org spelling is deliberate; the redirect keeps old links working, but canonical in-tree URLs are updated). Metadata / CI / packaging swept until grep -rn 'mplx/jennifer-lang' is empty; the first-party deck scope placeholder flips @mplx/ -> @jennifer/ (doc-only until jvc). |
| M19.9 | audit-driven correctness + hardening | A systematic severity-ordered sweep of ~190 findings from a full bug / performance audit of internal/ + modules/, each fix with a regression test (Go test or _test.j overlay). Crash / safety: OS-entropy RNG seeding (predictable UUIDs / session ids / passwords fixed; math.randSeed stays the deterministic opt-in), json / toml decode nesting caps, a try-body scope fix (a throw-skipped def reads undefined, not null), tengine recursion guards, archive zip-slip + aggregate-decompression caps, a JSON-pointer overflow guard, the two interpreter -race races + REPL-vs-spawn table mutation. Correctness: lvalue writes re-fetch their root after the RHS, index / append stamp the element type, path-keyed module structs, barcode (Code 128 stop / EAN check digit / Code 39 * / QR mask 3), pdfwriter WinAnsi / Info encoding, a full toml conformance pass, http / web (CSRF / cookie / CORS / ETag) hardening, quote-aware vcard / ical. Performance: the O(N^2) accumulation patterns retired (map hash index, json object decode, wire-framing reads in redis / amqp / mqtt / mikrotik / imap / websocket, list-join builders in csv / barcode / influxdb / jsonl / statsd, GF(256) inline). Lifecycle: os.release + capped child output, non-blocking net.eof + mutex-guarded net.Conn, httpd admission bounds / must-respond timeout / TLS-1.2 floor / safe unix-socket unlink, per-stream mutex + a discard verb on hash / crc / compress streams, lint descending into spawn / repeat. Plus six coordinated pre-1.0 strictness breaks (each with tests + operator / scoping docs): % is floored (Python; -7 % 3 == 2, 7 % -3 == -2); integer arithmetic overflow is a positioned error, not a silent wrap; a duplicate map-literal key is an error; mixed int / float comparison is exact (no lossy promotion, so 9007199254740993 == 9007199254740992.0 is false); a method may not share a top-level var / const name (no-shadowing both directions); reading a constant with the $ sigil ($MAX) is a parse error. |
M20 - system libraries
Go system libraries (cryptographic primitives, plus formats too heavy or too reflect-bound for a Jennifer-coded .j module - the json pattern, M16.9), plus two cleanup keywords (defer / errdefer) and Unix-signal support that the libraries needed. Ten sub-milestones, each shipped with the standard discipline: a cmd/jennifer / *_test.go suite, a docs/libraries/*.md reference, cheatsheet + JENNIFER.md entries, and a runnable example. Per-function detail lives in docs/libraries/; this table is the milestone-number index.
| M# | Library / feature | Surface |
|---|---|---|
| M20.1 | crypto | Security primitives above hash, Go stdlib only: crypto-grade randBytes / randInt (rejection-sampled, unseedable), constant-time hmacEqual, key derivation hkdf / pbkdf (algo sha1/256/512). Repointed uuid's random source here, so v4 / v7 are unguessable. |
| M20.2 | xml | Hand-rolled XML encode / decode over an opaque xml.Value (KindObject) element tree; read (tag / text / attr / children) + XPath-style get / findAll / has (name / name[k] / *) + build (element / setAttr / setText / append). No encoding/xml. |
| M20.3 | yaml | YAML 1.2 over yaml.Value, the same read / walk / write surface as json / toml + asDatetime; decode / decodeAll, anchors / aliases by value, << merge keys. Backed by gopkg.in/yaml.v3 (the one config parser that earns a dep); pre-parse depth + node-budget guards. |
| M20.4 | intl | Message catalogs + locale-aware tr(key[, params]) ({name} interpolation, locale -> base -> default -> key fallback); load / setLocale / locale. A system library (global mutable state + O(1) Go-map lookup); single-pass, output-capped interpolation. |
| M20.5 | term | Terminal host control for TUIs: raw mode (makeRaw / restore, single-use State), size -> Size{rows, cols}, readByte. Over golang.org/x/term, build-tag split (stub on jennifer-tiny); refused in the REPL. |
| M20.6 | signals | Cooperative Unix signals: SIGUSR1 -> live interpreter diagnostics (dump-and-continue); os.catchSignal / os.gotSignal (opt-in int / term / hup / usr2) for graceful shutdown; CLI terminal-restore-on-abort. Establishes the checkpoint hook the loop-cancellation follow-on will build on. |
| M20.7 | defer / errdefer | defer CALL(args); - single call, args snapshotted, block-scoped LIFO, runs on every exit path, never crosses the method / spawn boundary (no finally - rejected). errdefer (Zig-style) fires only on a propagating error; adopted by the seven connect-then-handshake modules. Paired with the new fs.sync. |
| M20.8 | serial / spi / iic / gpio | Device I/O over Linux /dev + ioctl (x/sys/unix), build-tag split linux && !tinygo: three buses + character-device GPIO (reusing the sysfs M18.11 module's pin-keyed shape). I2C is iic (letters-only). Shared plumbing in internal/lib/devio; ioctl struct layouts pinned to the kernel ABI by size assertions. |
| M20.9 | sql | Relational client over database/sql: MySQL / MariaDB (go-sql-driver/mysql) + PostgreSQL (jackc/pgx), both pure-Go. open -> Connection, query / exec, pull cursor + typed as* accessors, begin / commit / rollback, prepared statements. Values bind only through placeholders. The first heavyweight library-layer dependency (design-decisions.md); build-tag split, default binary only. |
| M20.10 | crypto AE + signatures | AES-256-GCM encrypt / decrypt (32-byte key, nonce prepended, AEAD-only) + Ed25519 signKeypair / sign / verify (crypto.Keypair); added sha384 to hash. All Go stdlib, TinyGo-clean. Safe-by-construction: AEAD-only, one algorithm per verb, length-validated keys, internal nonce. Out: password hashing, raw block modes, RSA / ECDSA, x509, AAD. |
Cross-cutting additions these pulled into the language / system side (each documented under its library):
defer/errdefer(M20.7) - the deterministic-cleanup keywords; thefinallyrejection and theprintf-for-translation rejection (M20.4) are recorded in rejected.md.fs.sync(M20.7) - fsync a write / append handle to the device, distinct fromclose's reach-the-OS: durability you check, versus cleanup you defer.sha384/sha512(M20.10) - the SHA-2 digests filled out acrosshash.compute/hmac/ streaming; MD5 / SHA-1 stay checksum-only.- The nesting cap (
internal/limits.MaxNestingDepth) shared by the language parser and thejson/toml/xmldecoders - build-tag split so it stays belowjennifer-tiny's fixed-stack crash point (see technical/tinygo.md). - The
SIGUSR1diagnostics checkpoint hook (M20.6) - the loop-iteration / call checkpoint the deferred cooperative loop-cancellation follow-on (interruptible loops, REPL Ctrl-C, terminating-signal terminal restore) builds on.
M21 - general backlog
The catch-all bucket: milestones that fit no other track - not a Jennifer-coded module (M18), not interpreter / tooling work (M19), not a Go system library (M20), and not a beyond-1.0.0 idea (those live in the horizon collection). Anything worth recording with no natural home landed here, and graduated out once a cluster grew big enough to earn its own bucket. Thirteen sub-milestones across Jennifer-coded modules, security hardening, interpreter / language work, byte throughput, and Windows packaging. Each shipped module carries the standard discipline (a 100%-passing *_test.j overlay, a cmd/jennifer/*_test.go integration test, docs/modules/*.md + catalog + JENNIFER.md entries, a runnable demo); per-item detail lives in those docs - this table is the milestone-number index.
| M# | Topic | Summary |
|---|---|---|
| M21.1 | screen module | Terminal UI. Output-only layer (both binaries): a value-semantic cell Buffer (text / textColor / box / fill / hline / vline), ANSI control builders, and a flicker-free render / diff paint loop (repaints only changed row-runs). Interactive layer (term, default binary): a pure decodeKey(seq) -> Key (printable / arrows / nav / F1-F12 / ctrl-* / alt-*) plus nextKey / begin / end / size over raw mode. 0-based coords; drawing past an edge is clipped. The curses / bubbletea subset. |
| M21.2 | feed module | RSS 2.0 + Atom 1.0 build and parse in one module (format chosen on build, sniffed on parse); a value-semantic Feed of Entry plus fetch(url). Rides xml (parse / escaped build) and time (RFC 822 / 3339 dates). Hardened for untrusted feeds: xml's nesting cap + no-custom-entity decode, lenient dates, and a new 64 MiB http body cap closing an unbounded-body OOM. |
| M21.3 | jwt module | JWT (RFC 7519) sign / verify / decode / header, claims a json.Value; ten algorithms (HS* / RS* / ES* / EdDSA). verify pins the expected alg (blocks algorithm-confusion), enforces exp / nbf, HMAC-compares constant-time. Brought M20.10's deferred asymmetric crypto due: extends crypto with rsaSign / rsaVerify / ecdsaSign / ecdsaVerify over PEM keys (build-tag split - RS* / ES* need the default binary; HS* / EdDSA on both). |
| M21.4 | acme module | ACME (RFC 8555) client over http + json: directory / account / order / authorization / challenge (HTTP-01 keyAuthorization + DNS-01 dnsRecord math) / accept / finalize(CSR) / downloadCertificate. Per-request JWS + a fresh anti-replay nonce; CA problem docs surface as a catchable Error. Second crypto expansion: rsaGenerateKey / ecGenerateKey, jwkPublic (RFC 7638 thumbprint), csr (PKCS#10). Default binary only. |
| M21.5 | orm module | Data Mapper over sql (value-semantic, method-less structs rule out Active Record): repository CRUD keyed off an explicit orm.Schema (no reflection), a non-mutating functional query builder (from / where / orderBy / limit / join -> toSql) with per-dialect placeholders (mysql / postgres backend selector, injection-safe), records as map of string to string, plus a createTable DDL emitter. Enabling sql change: toDriverArgs spreads a single list argument (Jennifer has no spread). |
| M21.6 | font module | Pure-.j TrueType / SFNT parser (no Go - bytes + bitwise + fs, both binaries): parse / open -> Font, then unitsPerEm / name / advance / glyphPath (SVG d) / glyph (contours). Parses head / cmap (fmt 4 + 12) / maxp / hhea / hmtx / loca / glyf (simple and composite) / name. Closed the dogfood gap: scripts/genwordmark.j reproduces the wordmark byte-for-byte without fontTools. |
| M21.7 | injection / DoS hardening | Real wire-boundary fixes, each with a regression test: CRLF / control rejection in smtp (envelope + EHLO), http (method), websocket (handshake URL), web (cookie Path / Domain); jwt.verify rejects an unsupported crit; acme control-char escaping + validated nonce; websocket 64 MiB message cap (refused before allocation, 1009 close). Shipped the security model (SECURITY.md + security-model.md): full host access is by design, untrusted data on a wire is the bug class, the untrusted-code sandbox is DRAFT#11. |
| M21.8 | call-depth limit + profiler metric | Deep recursion that overflowed the Go stack (fatal crash / jennifer-tiny segfault) now raises a catchable "call stack too deep" - Python's RecursionError analogue. The counter lives on the per-goroutine root env (spawn-safe); the cap is build-tag split in internal/limits (10000 default / 48 tiny, whose stack rose 2 -> 4 MiB). Feeds a jennifer profile max-call-depth metric. |
| M21.9 | network + credential hardening | Sensible-default / optional checks, one regression test each: an optional timeoutMs on net.connect / connectTLS / startTLS, fs.chmod / chown, web Secure cookies, smtp no-cleartext-auth + anti-downgrade STARTTLS check + envelope validation, oauth 0600 token file, jwt.verifyWith (iss / aud), 64 MiB received-data caps across the network clients, amqp AMQPS. |
| M21.10 | byte-oriented throughput | The binary library (concat / slice / indexOf / contains / split / startsWith / endsWith - the byte counterpart to strings, value-semantic, TinyGo-clean) plus net.readAll / readN (bulk reads with catchable size / close-mid-frame caps). Reworked http / mqtt / imap onto bulk reads; a binary.indexOf benchmark fixture. |
| M21.11 | range syntax (..) | Half-open lo..hi (int bounds), three materializing / value-semantic uses: list construction (0..n), lazy for-each (for i in 0..n, no list built), and slicing ($xs[a..b] + open forms, over list / bytes / string). lo > hi a positioned error; materialisation bounded by a catchable limits.MaxRangeElements (int64-span-overflow-safe, not the uncatchable makeslice panic). New RangeExpr / SliceExpr AST; fmt emits .. tight. |
| M21.12 | per-frame allocation elimination | A call / block frame now does no per-binding or per-call heap allocation: a slot-backed binding (Slot >= 0) writes only the pooled slots slice (the identifier travels in Binding.Name; the rare name-based readers scan the small slot slice via lookupLocal), and evalCall binds args interleaved with no intermediate []Value. Recursive fib ~300k -> ~59 allocs/op; Go's minor page faults fell ~7x. Value semantics and the vars fallback (REPL) intact; guarded by TestFrameAllocationsStayLow. |
| M21.13 | Windows installer | An Inno Setup script (packaging/windows/jennifer.iss) built by a windows-latest CI job into jennifer-<ver>-setup.exe: per-user (no admin), adds to PATH, bundles the system modules + sets JENNIFER_SYSMODDIR, opt-in .j association, Apps & Features uninstaller. Unsigned, best-effort unsupported build. scripts/build-windows-installer.sh recreates it locally via Wine. Promoting Windows to supported is the follow-on, M29.1. |
Cross-cutting threads:
- The
cryptoasymmetric surface grew here, not in M20: M21.3 addedrsaSign/rsaVerify/ecdsaSign/ecdsaVerify(PEM; PKCS#1 v1.5 / JOSE R||S) for JWT interop, and M21.4 addedrsaGenerateKey/ecGenerateKey/jwkPublic/csrfor ACME - all build-tag split off the TinyGo build (crypto/x509is absent there), so the asymmetric verbs are default-binary only while the symmetric primitives and Ed25519 stay on both. - A 64 MiB received-data cap became the standard DoS guard across the network clients (
http/redis/pop/imap/mqtt/websocket, in M21.2 / M21.7 / M21.9), so an attacker-declared length or an unbounded stream fails catchably instead of OOMing. - Not all of M21 is modules. M21.8 / M21.11 / M21.12 are interpreter and language work (the call-depth guard, range syntax, the allocation model), M21.10 is the throughput library, and M21.13 is packaging - which is exactly why this bucket is the general catch-all rather than an M18 module run.
- The
sqllist-spread (M21.5) -toDriverArgsspreading a singlelistargument into the placeholder sequence - is the one small language-adjacent enabler this track pulled in, for runtime-shaped parameterized queries.
M22 - additional libraries and language refinements
Post-backlog work belonging to neither the M20 system-library set nor the M21 catch-all: focused standard-library and .j-module additions or enhancements, plus small language cleanups, each landing when the need was concrete. Eighteen sub-milestones. Each shipped with the standard discipline (a library adds a Go package + internal/stdlib.InstallAll line + docs/libraries/ reference + cheatsheet rows; a .j module ships a 100%-passing *_test.j overlay + a cmd/jennifer/*_test.go integration test + docs/modules/ doc + catalog + JENNIFER.md bullet + a demo; a language feature updates the spec + grammar EBNF/PEG + the editor highlighters). Per-item surface detail lives in those docs; this table is the milestone-number index.
| M# | Topic | Summary |
|---|---|---|
| M22.1 | path library | OS-aware path manipulation over Go path/filepath, the pure-string counterpart to fs's I/O: base / dir / ext / stem / join (variadic) / clean / isAbs / split. Manipulation subset only (no disk-touching Abs / Glob / Walk), so no build-tag split, TinyGo-clean, both binaries. Host separator (portable). Explicitly not a filename sanitizer. |
| M22.2 | digits in identifiers | Relaxed letters-only to letter-initial [A-Za-z][A-Za-z0-9]* (still no _, <= 64); constants [A-Z][A-Z0-9]*(_[A-Z][A-Z0-9]*)* (digits within a chunk). Additive / non-breaking - so sha256 / SHA256 / HTTP2 / SCRAM_SHA256 legal, AES_256 still illegal (write AES256). One IDENT token, so it applies uniformly (vars / params / methods / struct type and field names / use as / import as aliases); constants the one separately-lexed class. Graduated DRAFT#20; unlocks the M22.3 renames. |
| M22.3 | library renames | The breaking renames the digit rule unlocked, one pre-1.0 batch (no deprecation window): use iic; -> use i2c;; uuid.generate("v4"/"v7") -> uuid.v4() / v7() (generate removed); crypto.pbkdf -> pbkdf2. Not renamed: binary (the bytes keyword, not a digit), intl (JS Intl). Each batch updated the library, its overlay + Go test, docs, cheatsheet, and every caller. Rejected (stance #1): per-algorithm digest shortcuts (hash.sha256 / crc.crc32) - the hash / crypto family is irreducibly algorithm-as-value (SCRAM / JWT / TLS negotiate the hash), so compute(b, algo) stays canonical (rejected.md). |
| M22.4 | match statement | Multi-way value dispatch match (EXPR) { when V [, V ...] { } ... else { } }: subject evaluated once, strict == (Value.Equal), first match wins, values short-circuit left-to-right, no fall-through, not a break target (break / continue act on the enclosing loop), optional else last, no-match-no-else is a no-op; a statement, each arm its own scope. Keywords match / when (pre-1.0 break). fmt lays arms out flat (each when / else on its own line, not cuddled). New MatchStmt; the header { ambiguity (when Name { vs a Name{...} literal) is resolved by the parser's noStructLit flag. Designed to grow into M22.5 patterns. |
| M22.5 | sum types (enums) + pattern match | def enum Name { Variant [ { field as type, ... } ], ... }; (top-level, hoisted like def struct); Name.Variant{...} / Name.Variant construction; match gains variant patterns when Variant(bind) { } binding the payload into a fresh per-arm scope, with exhaustiveness checked at resolve time for a local / same-module enum subject. New Value KindEnum mirroring KindStruct (value semantics, deep-const, cross-module identity by canonical path, retag), tagged-union / reflect-free. Case-agnostic naming - Prefix.Member resolved from the tables at eval, not capitalisation, so no PascalCase rule is forced on a teaching language (only the pre-existing "ALL-CAPS is a constant" rule still applies). Graduated DRAFT#19. |
| M22.6 | TLS options for http / rest | Reach an HTTPS host with a self-signed / private-CA cert. http.TlsOptions{skipVerify, caCert} + send variants http.requestTls / requestWithTls; rest.Client.tls field + a rest.client(baseUrl) constructor (the added required field breaks the bare literal) + rest.withCA(c, pem) (preferred) / rest.insecure(c). Secure by default (verification stays on unless explicitly relaxed). Pure .j plumbing to net.connectTLS (M16.14) - no interpreter or system-library change. Verified end-to-end against a self-signed loopback (http_tls_test.go). Dogfoods M22.9 (http.TlsOptions as a struct field across main -> rest -> http). |
| M22.7 | graphql client module | Thin GraphQL client over http / rest: client(endpoint) + bearer / basic / header / withCA / insecure builders, query(c, query, variables) -> json.Value (POST {query, variables}; result under /data). Gets the GraphQL convention right - a non-empty top-level errors array is an HTTP 200, not a non-2xx - raising a graphql error with the joined messages; a non-2xx also raises. queryNamed / tryQueryNamed add an operationName; tryQuery / tryQueryNamed return the raw envelope (no raise on GraphQL errors) for structured-error handling via exported hasErrors / errorMessages + json accessors. POSTs the endpoint verbatim via http.requestTls (rest's joinUrl would append a trailing slash). The GraphQL dependency the M27 Unraid deck consumes. |
| M22.8 | self-referential struct guard | A struct containing itself by value (direct or mutual) has no finite zero value and used to fatally stack-overflow when its zero / a literal was built; Interpreter.checkStructCycles (a gray/black DFS over direct struct-typed fields, run after hoisting at both Run and EvalInteractive) now rejects it at hoist time with a positioned error pointing at list of Self. Recursion through a list / map / task field and ordinary nesting stay legal. |
| M22.9 | module structs as struct fields | A module struct used as a struct field type now type-checks (was rejected "expects geo.Point, got struct" though it worked as a variable type). resolveDeclaredTypesOnce now also stamps struct field types with the module's (stem, path) identity (recursing into list / map elements), and a module struct's own sibling-struct field types retag to the module identity at the boundary check (construction + field assignment, via retagType). Value semantics + chained lvalues into a nested module-struct field work. |
| M22.10 | byte-capable http download | http could not fetch a binary body (the response was always convert.stringFromBytes(_, "utf-8"), which throws on non-UTF-8). Added a byte path reusing the already byte-exact framing: http.BytesResponse (body as bytes) + requestBytes / requestWithBytes / getBytes (parseResponse split into a parseRaw byte core + a text decoder). Text verbs unchanged (still throw on non-UTF-8, by design); rest stays text / JSON. Pinned by http_bytes_test.go (a gzip round-trips with matching sha256). |
| M22.11 | hardening: injection & output-encoding | From two security / robustness audits (internal/ + modules/; per-finding detail - severity, reproducer, sites - in the two report files). orm identifier / operator allowlists; imap / pop CRLF rejection; statsd metric validation; htmlwriter tag / attr checks + exported safeUrl; json.encode HTML-escaping (< > & U+2028 U+2029); a new constant-time hash.equal; http.parseUrl authority split; dotenv env-name validation (OM-021); ipnet v4-mapped ::ffff:0:0/96 fold (unmap). Reasoned non-literal: a tengine no-auto-escape SECURITY warning, not an auto-escape mode (OM-012, keeps text/template semantics); OM-010 folds only the well-defined ::ffff:0:0/96 mapped form. |
| M22.12 | hardening: network / resource / path robustness | 64 MiB caps on server-declared lengths; connect / read timeouts + cleartext warnings; a net.readAll default cap; net.startTLS locking; bounded handle registries + a sql query deadline + DSN-password redaction; archive per-entry budget; httpd.serveDir traversal rejection + unix:-socket perms; the json / toml / yaml write-API depth guard. Reasoned non-literal: OF-007 applied full bounds + deadline + teardown to sql (the worst instance) and registry bounds to fs / os / compress / net. |
| M22.13 | hardening: web framework | web.sessionId trusts only a minted-UUID-shaped cookie + web.renewSession; the web.onError hook (else stderr); lenient form / percent decoding + csrf content-type gate; a HEAD served by the matching GET route; httpd per-request timer cleanup. Reasoned non-literal: OM-003 shipped serial for v1 with loud docs because concurrent dispatch raced shared interpreter state - later removed properly in M22.17; OF-006 (unrestricted meta.call) shipped a docs allowlist pattern + meta.md example, not a new primitive (a two-line .j allowlist suffices; web only dispatches author-registered handler names). |
| M22.14 | imap criteria-based search | imap.search(session) -> imap.search(session, criteria) (breaking; an empty imap.criteria() = the old SEARCH ALL). imap.Criteria filters hybrid: server-side fields map to one IMAP SEARCH (substring on subject/from/to/text, a since / before day-range as time.Time, flags, size - all ANDed, one round-trip), client-side fields refine the candidates by fetching only headers / BODYSTRUCTURE (subjectRegex / fromRegex, hasAttachments heuristic). Injection-safe (quoteArg + control-checked line); a time-of-day bound is transparently refined client-side against each candidate's INTERNALDATE. Pure .j. |
| M22.15 | imap browse / APPEND / rename | Rounded imap into a full read / browse / manage / save client: folders (LIST) + status (STATUS, no select), append / appendWith (APPEND via the synchronizing-literal continuation flow). Terminology rename (pre-1.0 breaking): mailbox -> folder (selectFolder / createFolder / the Folder struct); the LIST verb is folders (list is a reserved type keyword). |
| M22.16 | core hardening sweep | Small internal/ correctness / resource / performance residuals, one reviewed pass: sql cursor deadline split from the acquire / statement deadline (caller-settable sql.setQueryTimeout); httpd registry maxServers bound + catchable "too many open"; md5 / sha1 labelled non-cryptographic; the json / toml / yaml write-depth guard checks only the touched node (not a full re-scan) and folds three exceedsDepth copies into a shared helper; an include total-token cap (a diamond that re-includes the previous file expands 2^n); the printf field cap lowered from 1<<20 + closed-form padding; incremental json / xml decoder error positions. |
| M22.17 | web hardening (concurrent dispatch) | Turned web from strictly-serial into safely-concurrent, interpreter-first (four ordered steps, each with a gating test; reasoning in design-decisions.md). (1) Error crosses meta.callMain intact - track a module's declared structs separately from the auto-injected Error, retag only the former. (2) Race-safe dispatch - the call-depth counter became a per-chain *int threaded down the frames (fresh at each goroutine root, incremented at evalCall and every cross-boundary dispatch), fixing both the -race data race and a fatal re-entry Go-stack overflow; the rest of the reachable shared state (map hash-index reads, resolver caches, profiler, diag) audited clean. (3) spawn-per-request in web (errors caught inside; task.discard prunes; concurrency bounded by httpd). (4) web.onError fail-safe - always stderr and the hook, which now binds as Error. The 1700 ms -> 3 ms latency probe is the regression gate. Supersedes the M22.13 OM-003 "web stays serial" note. |
| M22.18 | dotenv layering, profiles, interpolation | Grew dotenv into a layered loader: readCascade / resolve / loadCascade / autoload merge .env -> .env.local -> .env.<profile> -> .env.<profile>.local from one fixed dir (no walk-up, closing the file-hijack class), with a real OS env var always winning over a file value; profile from JENNIFER_ENV (empty = base files only). Enhanced parse: backward-reference ${VAR} (unquoted + double-quoted, resolving earlier keys -> real OS env -> "", so cycles are impossible; no $(...) command substitution), multi-line double-quoted values (positioned unterminated-quote error). Strict profile validation ^[A-Za-z0-9_-]{1,64}$ (no traversal). Single-file load keeps unconditional-override (the primitive); the cascade loaders are real-env-wins. os.getEnv(k) != "" is the "already set" test (no os.hasEnv; "" counts as unset). Consolidated the ${VAR} / multi-line items parked in M23.8. |
M23 - module improvements
Done. M22 lifted a handful of modules (imap, http); M23 generalized that across the module ecosystem. A survey of all 63 .j modules found the gaps cluster into cross-cutting themes rather than scattering per module, so the work was organized by theme - a shared pattern (a receive loop, a persistent connection, a backend selector) built once and applied to every module that needs it. Fifteen sub-milestones. Each shipped the standard per-module discipline (a 100%-passing *_test.j overlay, a cmd/jennifer/*_test.go integration test where a live server applies, docs/modules/ + catalog + JENNIFER.md entries, a demo, both binaries build); per-item surface detail lives in those docs, and the biggest sub-milestones keep their own reference docs. This table is the milestone-number index. Deliberate non-goals: password hashing (needs x/crypto) and fully-typed orm rows (awaits struct reflection).
| M# | Topic | Summary |
|---|---|---|
| M23.1 | streaming / server-push read loops | One cooperative receive-loop-over-net shape - a blocking receive* (+ timeout-bounded poll), no callbacks, the app opting into concurrency via spawn - across redis (pub/sub + one-round-trip pipeline + multi/exec + scan), amqp (Basic.Consume + exchanges + publisher confirms), mqtt (QoS-1 + retained + Last-Will + reconnect), mikrotik (.tag-correlated + /listen), imap (RFC 2177 IDLE). Wire framing factored into pure encode/parse (100% overlay); live loops on mock-server Go tests. Fixed redis coalesced-frame buffering and imap stale-deadline. Residual: a value-semantic Session can't retain a cross-call buffer (a buffered net reader is the general fix). |
| M23.2 | connection reuse / persistent sessions | Reusable connections so a loop stops re-handshaking: http.Session (reused net.Conn + cookie jar, framed one-response reader) + a policy http.send over http.Options (3xx redirects, cookie jar, 429/5xx retry+backoff); rest.Client routes through it (tls folded into Options, pre-1.0 break; paginate / paginateCursor walk every page); smtp split into open / sendOn / close so N messages pay one TLS+auth handshake. Pinned by keep-alive / cookie / pagination / session-reuse Go tests. |
| M23.3 | stable-identity verbs | Volatile sequence numbers broke "fetch only what's new": imap went UID-only (every verb sends its UID form + search returns UIDs + atomic move + ranged fetchPartial; pre-1.0 break, a seq+UID twin set rejected on stance #1), pop added uidl -> MessageId + top / reset / noop (additive). Pinned by exact-wire-command fake-server tests. |
| M23.4 | byte-exact binary values | redis / memcache threw on a non-UTF-8 bulk value; added bytes-valued setBytes / getBytes (text verbs unchanged, still strict-throw) plus typed redis hash/list/set helpers and memcache getMulti / gets / cas. NUL/CR/LF/0xFF byte-count-framed round-trip tests. |
| M23.5 | selectable backends | session / ratelimit were memcache-only; added the kv system library (in-process per-key-TTL store, integer handle shared across spawn, openFile persisted) and the kvstore module selector (Store a sum-type enum Memcache/Redis/Local, exhaustiveness-matched), and moved session (values now a json.Value) and ratelimit (fixedWindow / slidingWindow -> Result) onto it (both pre-1.0 break). Interpreter fix: a module enum as a struct-field type across a boundary. |
| M23.6 | format & coverage completeness | The broadest track (13 pieces) - the deepest per-module gaps. ipnet subnet math + scope classifier; orm ordinary-query surface (select / aggregate / join / groupBy+having, render-time allowlist re-check); markdown images / blockquotes / nested lists; vcard full N / TYPE; ical recurrence + TZID + VTODO/VALARM; mime charset-on-decode + RFC 2231; feed enclosures + author/categories; s3 presign + byte bodies + multipart; barcode UPC / code93 / DataMatrix + QR 11-40; font a CFF/OTTO backend + an O(1)-per-query fix; pdfwriter embedded TrueType fonts + image XObjects + text layout. Several pre-1.0 struct-shape breaks; each validated against an independent reference. |
| M23.7 | observability completeness | prometheus histogram + summary types (cumulative buckets, nearest-rank quantiles, exact text format, observeAt / pushgatewayPath); statsd *Rate / *Tagged / *Float verbs + a Batch datagram packer + a control-char-validated prefix; influxdb 2.x / 3.x via a Version enum (Flux queryFlux, token redaction). Each validated against an independent reference parser. |
| M23.8 | ergonomic papercuts + notifier richness | Cheap high-value wins across 15 modules: log child logger + fatal; cron named months/weekdays + @-macros; jwt verifyLeeway / verifyWithKeys / verifyJwks; totp generateSecret / hotp / verifyWindow; bloom optimal / serialize; csv formatSafe + Dialect + streaming; richer discord / telegram / slack / gotify messages; webhook replay-protected signing; and more. Core fall-out: http.requestRawBody / requestRawBodyTls (a byte request body, for telegram uploads). |
| M23.9 | fmt: shape-aware wrapping + raw-literal fidelity | Rebuilt jennifer fmt (token-stream, no AST): width-aware wrapping (one element/arg per line past 100 cols, struct/map over 6 members, calls hug ), inline single-statement when arms, operator-chain fill-break), tight Go-style literal spacing, and raw-literal fidelity via a lexer Token.Raw (digit separators / base prefix / quote style / embedded newlines survive verbatim). -w is atomic + self-verifying (re-lexes its output, refuses to corrupt). Corpus reflowed (~700 -> ~215 over-limit lines). Pre-1.0 break (canonical output changed); TestFmtPreservesTokenStream proves a format changes only whitespace. |
| M23.10 | interactive stdin for os.run / os.spawn | os.run(argv, stdin) feeds an optional trailing string / bytes to the child's stdin then closes it (variadic - one-arg calls unchanged), output drained into the 16 MiB-capped buffers - deadlock-free by construction. Unblocks a stateless filter / subprocess exchange (M23.13's mcp.connectStdio built on it). Deferred: interactive streaming pipes on a spawned Process. |
| M23.11 | jsonrpc module | JSON-RPC 2.0 client + server, pure .j over json + http. Client call / notify (json.Value params/results, every failure a unified catchable Error); a transport-agnostic handle(requestBody) runs the whole protocol (single / notification / batch / reserved codes) dispatching each method to a top-level func by name via meta.callMain; a thrown handler yields a generic -32603 (detail stays server-side). Chosen over gRPC (protobuf + HTTP/2 + codegen would be a heavy Go library, not a .j module). |
| M23.12 | match / enum adoption | Applied match / enum to the genuine closed-variant-set cases (open sets stayed strings): non-breaking match in 14 modules; a non-breaking private enum (markdown); breaking API enums (htmlwriter.NodeKind, prometheus.MetricType, barcode.SymbolKind, orm.Dialect / ColumnKind, totp.Algorithm); and a shared transport.Security{None,Tls,Starttls} replacing the stringly-typed security field on six socket clients (the mail three accept all modes, the other three reject Starttls). Exhaustiveness-checked. |
| M23.13 | mcp module (Model Context Protocol, stateless) | A server exposing tools / resources / prompts (server / addTool / addResource / addPrompt, handle / serveStdio) plus a client (HTTP connect / stdio connectStdio over os.run) sharing one call surface; MCP is JSON-RPC 2.0 so the client reuses jsonrpc.call. tools/call is allow-listed (only a registered handler; a thrown handler yields a generic message). Validated end-to-end against the official MCP Python SDK. Not planned: the stateful Streamable-HTTP transport (needs an SSE push primitive httpd lacks). |
| M23.14 | raw single-quoted string literals | Breaking split so each delimiter does one job: "..." stays cooked (escapes processed), '...' becomes raw (verbatim to the next ', spanning newlines - a free heredoc); embed a ' via the cooked form ("it's"). No r"..." prefix (rejected - the delimiter is the mode). Lexer-confined (raw := quote == '\''); src is []rune so multibyte content is exact; Token.Raw round-trips (incl. multi-line). Migration a no-op; docs / grammar / four editor highlighters updated. |
| M23.15 | orm: a batteries-included data-mapper ORM | Lifted orm from a thin query builder to a full ORM (still Data Mapper): a Session unit-of-work + column-attribute DDL builders (.1); migrations split into the sibling sqlmigrate module (.1b); associations + joinRelation (.2); eager loading in a fixed 1+R queries (.3); a write path - upsert / insertMany / insertReturning / updateWhere / deleteWhere / save (.4); finders + whereNull / whereBetween / distinct / page (.5). Rows stay map of string to string (no reflection), relations attach via a side Result (no any); whereRaw / typed-struct mapping / Active Record rejected. Keeps its own docs/modules/orm.md + sqlmigrate.md. |
Cross-cutting threads:
- Not all of M23 is modules. M23.9 (
fmt), M23.10 (os.runstdin), and M23.14 (raw single-quoted strings) are tooling / interpreter / language work; the rest is the module ecosystem. - Build once, apply everywhere was the payoff of organizing by theme: a shared receive-loop shape (M23.1) and a shared backend selector (the
kvlibrary +kvstoreenum, M23.5), each built once and threaded through every module that needs it. match/enum(M22.4 / M22.5) landed across the ecosystem in M23.12, and the largest sub-milestone - M23.15'sorm+ the newsqlmigrate- is a full batteries-included ORM in its own right.
M24 - language, concurrency, and libraries
Done. The first batch of horizon drafts graduated into a scheduled track, then grew into the broadest single milestone: a CLI ergonomic, the language's biggest expressiveness feature (first-class functions), a concurrency-coordination layer, a numeric / ML library stack, two ASN.1 protocol clients, and a run of library / language refinements. Twenty-three sub-milestones. Each shipped with the standard discipline - spec + grammar EBNF/PEG + editor highlighters where a language feature lands; a Go package + internal/stdlib.InstallAll line + docs/libraries/ reference + cheatsheet where a library lands; a 100%-passing *_test.j overlay + cmd/jennifer/*_test.go + docs/modules/ + catalog + JENNIFER.md + demo where a .j module lands; both binaries build; the full test close-out. Per-item surface detail lives in those docs - this table is the milestone-number index.
| M# | Topic | Summary |
|---|---|---|
| M24.1 | run profiles / --env flag | jennifer run --env=prod script.j sets JENNIFER_ENV=prod before Run - identical to the env var, no interpreter change (dotenv's .env.<profile> selection is the first consumer). cmd/jennifer-only: parseRunArgs + a validRunProfile allowlist ([A-Za-z0-9_-], 1-64, blocks a .env.<profile> traversal); an explicit flag overrides an inherited env. Both binaries. Graduated DRAFT#26. |
| M24.2 | first-class functions | Closed the largest expressiveness gap - a function held in a value (type func), immutable, not a pointer (so the value-semantics stance is untouched); a bare method name in expression position is the value, a name + ( is a call (no &NAME sigil). New KindFunc / TypeFunc; a CallValueExpr postfix-( node calls any function-valued expression ($f(x), $fns[0](x), makeAdder(1)(2)). Enables the higher-order lists layer (map / filter / reduce / find / any / all / sortBy, via BuiltinCtx.Invoke). Deferred: Tier 2 closures + migrating string-name dispatch. Graduated DRAFT#18. |
| M24.3 | concurrency coordination | Gave spawn / task cancellation, bounded waits, and channels. task.cancel / cancelled - cooperative, observed at a shared loopCheckpoint (a catchable "task cancelled" at loop safe points; one atomic-nil check per iteration, hot path untouched); waitTimeout / waitAnyTimeout throw on timeout. channel of T + the channel library (make / send / recv / close / select / len / capacity): a KindChannel shares a *ChannelState while send deep-copies the value in (conduit shared, data copied), and channel is a contextual keyword. Buffer capacity capped (OOM guard); send-on-closed / double-close catchable. Graduated DRAFT#21. Deferred: cancellable channel ops, index-returning select. |
| M24.4 | stats library | 26 descriptive statistics over list of int / list of float - central tendency (geo / harmonic / weighted means, modes), spread (population + sample* Bessel, iqr, mad), shape (skewness / excess kurtosis), order (percentile / quartiles), sum / zscore, bivariate (correlation / covariance), describe -> stats.Summary. Strict like math (an undefined result is a catchable error, never a NaN); real reductions -> float, selections preserve the input kind. Pure Go, TinyGo-clean. Graduated DRAFT#5. |
| M24.5 | ml library | A classical / predictive ML core (scikit-learn-lite) over stats / linalg, not deep learning. Fit/predict: a fit fn -> an opaque ml.Model handle applied with predict / transform / predictProba / free. Regression (linear / ridge / lasso / kNN / tree / forest), classifiers (kNN / naiveBayes / logistic / CART tree / forest), kMeans, pca, scalers; metrics (accuracy / precision / recall / f1 / rocAuc / rmse / r2 / ...); trainTestSplit / kFold / polynomialFeatures. Random draws from math's seedable source; hyper-parameters bounded (a runaway value -> catchable). Pure Go, both binaries. |
| M24.6 | linalg library | Linear algebra over Jennifer's value types, the stats companion. Vectors (list of float): dot / distance / cross / normalize; matrices (list of list of float): transpose / trace / determinant / inverse / solve / identity / zeros / shape; polymorphic norm / scale / add / sub; matmul dispatches on operand shape. Direct algorithms (Gaussian / Gauss-Jordan), no gonum. Strict (dimension mismatch / singular / non-finite -> catchable); every value bounded by MaxMatrixElements. Pure Go, both binaries. |
| M24.7 | asn1 library | ASN.1 BER decode / DER encode, the byte enabler for LDAP / SNMP. Hand-rolled (Go's encoding/asn1 is DER-only + reflect-bound). An opaque asn1.Value element tree (like json / xml) walked by (node, pointer) accessors whose tokens are child indices (tagClass / tagNumber / asInt / asOid / ...); typed constructors (integer / oid / sequence / tagged EXPLICIT / retag IMPLICIT). A decode-node budget + nesting cap turn a bomb into a catchable error; output byte-verified against Go's encoding/asn1. Pure Go, both binaries. |
| M24.8 | snmp client + agent | SNMP v1 / v2c client and agent (modules/snmp.j) over asn1 + net UDP. Client get / getNext / set / walk -> list of Varbind typed by SNMP type (counter32 / gauge32 / timeTicks via asn1.retag); request-id check, deadline + retries. Agent agent / serve answers GET / GETNEXT / SET for a MIB (GETNEXT in numeric OID order so a walk traverses it). Codec factored pure (socketless overlay); live loopback + real-hardware verified. No v3 / traps / GETBULK. Default binary only. |
| M24.9 | ldap client + directory server | LDAP v3 (RFC 4511) client and lightweight directory server over asn1 BER + net, LDAPS / StartTLS via transport.Security. Client bind / SASL bindSasl, search (RFC 4515 parseFilter or constructors) / searchPaged, writes add / modify / delete / modifyDn / passwordModify. Server answers simple bind (password schemes) + filtered search - read-only over the wire, mutable from code via a shared kv store (in-memory or file-backed), enough to back Authelia. Codec factored pure. Default binary only. |
| M24.10 | math foundations + special fns | Filled the math gaps for 1.0, folded in. Everyday: trig (+ inverses, atan2), hyperbolic, exp / log (exp / ln / log2 / arbitrary-base log), cbrt / hypot / sign, combinatorics (factorial / comb / perm / gcd / lcm, exact-int overflow-checked), TAU. Special: erf / gamma / lgamma / beta + the regularized incomplete gamma (regGammaP / regGammaQ) / beta (regBetaI) the CDF engine needs, hand-rolled to machine precision. Strict (domain / overflow / non-finite -> catchable). Go stdlib base; both binaries. |
| M24.11 | distributions + inferential stats | The classical numerical-inference layer (the scipy.stats / Excel surface) folded into stats on M24.10's specials - no separate prob library. Distributions (normal / t / chi-square / F / binomial / Poisson): pdf/pmf, cdf, quantile (Acklam + bisection), Box-Muller normalSample. Inference: linearRegression / multipleRegression, confidenceInterval, proportionCi (Wald / Wilson / Clopper-Pearson), tTest / chiSquareTest / fTest / anova, histogram -> Regression / Interval / Test structs. Every p-value finite-guarded, clamped [0, 1]; a degenerate input -> catchable. Pinned to scipy; both binaries. |
| M24.12 | scientific-notation float literals | Float literals accept an [eE][+-]?digits exponent (6.022e23, 1e10 - the exponent makes a literal a float even with no fraction), no _ in the exponent, decimal-only (0xe5 keeps e a hex digit). Lexer-only, additive / non-breaking. Strict at the edge: overflow (1e400) a positioned parse error (never Inf); underflow (1e-400) rounds to a finite 0.0. Reasoned in design-decisions.md. |
| M24.13 | uri module + encoding codecs | URL handling factored - byte encoding in encoding, URL semantics in a .j module. encoding gains "uri-percent" (RFC 3986) + "uri-form" (form-urlencoded, space +) toText / fromText codecs. The uri module (pure .j, both binaries): parse -> Uri / build (round-trips), encode / decode / encodeForm / decodeForm, buildQuery / parseQuery, RFC 3986 resolve(base, ref). Seven modules de-duplicated onto them. Pre-1.0 break: rest query strings now form-encode a space as +. |
| M24.14 | args CLI parser module | A declarative argparse-style parser (the "write your tools in Jennifer" gap os.flag / os.ARGS left). A value-semantic Parser: typed optional flags (flag / intFlag / boolFlag / countFlag -vvv / listFlag), nargs positionals, subcommands, version; args.parse($p, os.ARGS) -> a Result with typed accessors. Normalises --flag=value, bundled shorts, -- end-of-flags; an error is a catchable Error{kind:"args"} (not exit(2)), -h / --version set done. Pure .j, both binaries. 32-test overlay. |
| M24.15 | validate module | Declarative validation of a map of string to string -> a structured failure list. Value-semantic rules (required / isInt / min / maxLen / pattern / email / url / datetime / oneOf / password / custom func predicate + withMessage), grouped per field; validate.check -> list of Failure, plus ok / messages / byField / localize (i18n %param% markers). An absent / blank field passes every rule but required. Pure .j over regex / uri / time / password; both binaries. 16-test overlay. |
| M24.16 | module suite hardening | A security / robustness pass whose audit findings collapsed to systemic root causes, each fixed once across its cluster (~35 fixes / ~24 modules, both binaries clean), remotely-exploitable first. Received-data caps (shared transport.checkReceiveSize); injection (label digits-only command class, CRLF / control stripping in log / ical / vcard, barcode colour); a cross-origin redirect credential leak (http.send drops Authorization / Cookie behind allowCrossOriginRedirect); typed error propagation; numeric clamps; correctness (semver.satisfies(v, garbage) -> false). |
| M24.17 | html module (rebrand + parser) | Fold HTML building and parsing into one bare-named html. Rebrand htmlwriter -> html (pre-1.0 break; a format module is named for the format, like xml / json). A tolerant hand-rolled parse (pure .j, no x/net/html) producing the same transparent Node the writers do (build / parse round-trip): void / self-closing / unquoted-boolean attrs, mismatched-nesting auto-close, comments, DOCTYPE, raw script, entities; depth + node budget. Added parse / attrOf / hasAttr + XPath-ish get / findAll / has. Both binaries; 23-test overlay. |
| M24.18 | markdown reader | Gave markdown reading (the consistency move M24.17 set up). markdown.parse(md) -> a public Node (a string kind, recursive over children) walked by the xml / html vocabulary (typeOf / children / text / level / attr / get / findAll / has); the private Block / Span model is converted, inline spans eagerly expanded (a link's href walkable), the fence language captured. One model: toHtml / toAnsi became render(parse(md), ...) wrappers (old renderers deleted; byte-identical output). Nesting cap + node budget. Pure .j, both binaries; +16 overlay tests. |
| M24.19 | string interpolation | Cooked-string interpolation "total: {$sum}, up {strings.upper($x)}" - sugar over concat + convert.toString. Only a cooked "..." interpolates, each {expr} one expression (a statement / empty {} is a parse error); a raw '...' never (the off form); a literal brace is \{ / \}; f"..." rejected. Part 1 moved the two {name}-template consumers (intl.tr, validate) to a %name% marker. Part 2 added the lexer TOKEN_STRING_INTERP (Parts) + the parser sub-lex / sub-parse of each slot -> InterpStringExpr. The whole toolchain treats a slot as real code (resolver / lint L204 / profile); every literal-brace cooked string was migrated (prefer raw). Graduated the horizon entry; both binaries. |
| M24.20 | module version + capability header | A per-file # pragma-jennifer-<key>: <value> header stating the minimum interpreter version + required host capabilities, checked at read time (the lightweight sibling of the M26 jvc / deck.toml constraints, no package manager). version (>=0.25.0, a min-compare via version.AtLeast; any dev build bypasses) + capability (net / exec / sql, from a build-tag-split set, queryable as meta.CAPABILITIES / hasCapability). Enforced per file at three first-read seams (CLI / include / loadModule), with bounded diagnostics; a malformed or duplicate-version directive is a hard error. Ships an L303 lint + fmt canonicalisation. Every module carries a >=0.24.0 floor; capability: only where mandatory (an optional backend stays ungated). No interpreter-core change. |
| M24.21 | Markdown -> PDF (markdown.toPdf) | The markup-driven document story for pdf, folded into markdown as a third target beside toHtml / toAnsi: toPdf(md) / toPdfWith(md, opts) / renderPdf(doc, opts) lay a document out to a paginated PDF over pdf's layout primitives. A flow engine threads a value-semantic Layout through per-block renderers (heading, word-wrapped paragraph with per-run fonts, nested lists, ruled GFM table, code, blockquote), paginating at the bottom margin; PdfOptions sets size / margins / standard-14 fonts. Folded in (reasoned in design-decisions.md) at ~2x markdown import time / +~6 MB RSS; needed M24.18's parse tree. Graduated DRAFT#13. Pure .j, both binaries; +15 overlay tests. Dogfooded by gen-module-docs.j -> jennifer-module-api.pdf. |
| M24.22 | dot module | Graphviz DOT graph description: build a graph of nodes and edges with attributes (digraph / graph, node / nodeWith, edge / edgeWith, graphAttr / nodeAttr / edgeAttr) and render it to .dot text for an external Graphviz tool to lay out (dot -Tsvg). Value-semantic builders; DOT-escaped strings; emits the description only (graph layout is Graphviz's job, deliberately not reimplemented). Pure .j over strings / lists, both binaries. |
| M24.23 | plot module | Data plotting to SVG: a unified chart(series, opts) renders one or more Series (line / points / both / area, dashed, error bars, marker shapes) on shared axes with a positioned legend; line / scatter / bar / bars (grouped / stacked / diverging) / histogram wrap it. Automatic "nice" ticks, log scales, a time-labelled date axis, reference lines (hline / vline), fonts / margins, data labels, <title> hover tooltips, and save. The visual companion to stats / ml; pure .j over math / time / fs / strings / lists / convert, both binaries. |
Cross-cutting threads:
- Graduated horizon drafts. M24 drained the first scheduled batch from the horizon collection:
DRAFT#26(M24.1--env),DRAFT#18(M24.2 first-class functions),DRAFT#21(M24.3 concurrency coordination),DRAFT#5(M24.4stats), andDRAFT#13(M24.21 Markdown -> PDF), plus the string-interpolation entry (M24.19). - A numeric / statistical stack landed together:
stats(M24.4) +linalg(M24.6) +ml(M24.5) + themathspecial functions (M24.10) + the distributions / inference layer folded back intostats(M24.11) - each strict (an undefined result is a catchable error, never a NaN) and bounded by a materialised-value cap. - An ASN.1 protocol stack: the hand-rolled
asn1byte layer (M24.7) enabled thesnmp(M24.8) andldap(M24.9) clients-and-servers, each factoring its codec pure so the overlay round-trips without a socket. - Consistency across the format modules. The
htmlrebrand + parser (M24.17) and themarkdownreader (M24.18) gave both modules the same build-and-parseNodemodel asxml/json, which M24.21'stoPdfthen consumed. - Not all of M24 is libraries. M24.2 (first-class functions), M24.3 (cancellation / channels), M24.12 (scientific-notation literals), and M24.19 (string interpolation) are language / interpreter work; M24.1 is a CLI flag; M24.16 / M24.20 are a cross-suite hardening pass and a read-time guard.
M25 - read-only-parameter borrow (compound copy elision)
Parameter binding deep-copies compound arguments (list, map, struct) to uphold value semantics. When the callee never writes the parameter - no $p = ..., $p[i] = ..., $p[] = ..., $p.f = ... anywhere in its body - that copy is pure waste: a read-only alias to the caller's backing is observationally identical to a copy, so the parameter can be bound by alias. This is the standard value-semantic-language optimization (the "borrow" half of copy-on-write), narrower and more clearly sound than the shared-marker COW that was tried and reverted as inert (see technical/rejected.md): it never detaches-on-write, because a borrowed parameter is by construction never written. No .j changes and no new syntax - a pure interpreter optimization. Split into the borrow itself (M25.1) and the escape analysis that widens where it is sound (M25.2). Independent of the horizon bytecode-VM direction but composes with it.
M25.1 - borrow in modules and globals-free scripts
Done. The borrow, enabled in the two contexts where it is sound without a whole-program analysis.
- The soundness subtlety. "Never writes the parameter" alone is not sufficient: if the argument aliases a mutable global and the body mutates that global, a borrowed read observes the mutation where a copy would not (
def g ...; func f(p) { $g[0] = 9; return $p[0]; } f($g)). The synchronous call model rules out the caller changing the backing mid-call, but not the callee changing it through a global alias. Borrow is therefore enabled only where no mutable global can exist to alias: (a) a module, whose top level is declarations-only (and whose cross-boundary arguments are copies anyway); and (b) a single-file script that declares no mutable top-leveldef- checked once byhasMutableTopLevelGlobal, so a script's own read-only helpers benefit without being moved into a module. A script with a mutable global keeps the copying bind (M25.2 is what lifts that). - Mechanism. Extends the existing scalar copy-elision in
bindParamValueto compound Kinds - the "mutation-safety proof" that elision path was documented as requiring.ResolverunsmarkBorrowableParamsper method: a statement-level write-scan of the body (through nested control flow; no-shadowing makes a written name unambiguously the parameter;spawnbodies mutate their own snapshot so are correctly skipped) setsParam.Borrowwhen the parameter is never written and its type is borrow-safe - a compound whosestampDeclaredTypedoes not recurse into element backing (a flatlist/map, or a struct/bytes; alist of listfalls back to copy so the stamp can't mutate the shared backing).Interpreter.bindArgaliases whenParam.Borrow && (i.isModule || i.entryGlobalsImmutable), else copies as before. - What it buys. It removes a class of accidental quadratics transparently. The
markdownblock collectors (collectFence/collectQuote/collectList/tableFrom, alllines as list of string) deep-copied the whole line list once per block (lines x blocks); parsing a 160-chapter synthetic book as one document drops from 5.1 s to 0.85 s and turns from super-linear (2.9x per input doubling) to linear (2.0x). ThecountNodesbudget walk borrows itsNodeargument, and a globals-free script's own read-only helpers go linear the same way. The.j-level workaround does not pay off: a count-during-construction rewrite of thecountNodesguard measured a wash (9.5 s vs 9.6 s on a 100k-node document), because eliminating the read-only walk forces threading the count back through the builders, re-introducing an equal volume of copies. No reference-semantic handle a pure.jmodule can hang a large list on exists, so the copy is only removable below the language. - Discipline. TinyGo-clean, reflect-free, strict behaviour parity, race-clean.
internal/parser/borrow_test.gopins the write-scan and type gate (a false positive would alias a mutated parameter);internal/interpreter/borrow_test.goandborrow_internal_test.gopin the soundness gate (a script with a mutable global keeps copy semantics; a globals-free script and a module borrow) and the value-semantics parity (borrowed reads correct, caller values intact, returned borrows copied at the receiver).
M25.2 - per-function escape analysis for scripts with mutable globals
Done. M25.1 gates a whole script all-or-nothing on "does it declare any mutable top-level global". M25.2 adds a per-function decision on top: borrow a never-written parameter in any method proven not to mutate a global transitively, even in a script that has mutable globals elsewhere.
- The analysis (
computeEntryGlobalSafe, run after module load so the module-alias and namespace tables exist). Each entry-program method is scanned for a local hazard, then a fixpoint propagates it along named-call edges (CallExpr.Method), so a method that reaches a hazardous method is itself hazardous. A local hazard is: a write whose root is not a method-local name (a mutable global write - aconsttarget is already rejected, so any non-local write hits a mutable global); aCallValueExpr(dynamic func-value dispatch); a module call (a module can re-enter the host viameta.callMainand reach a host global write); a callback builtin (the higher-orderlistslayer, which invokes a func-value argument, and the by-name dispatchersmeta.call/meta.callMain/testing.run/runWith/assertThrows- keyed by canonical namespace, so an aliaseduse lists as lis caught); or an unresolved bare call. The walker defaults an unrecognised node to hazardous, so a new AST node never silently escapes.spawnbodies are skipped (they mutate a deep-copied snapshot, not the live globals). A method with no hazard is stampedMethodDef.GlobalSafe, andmethodBorrowCtxwidens the bind gate toi.isModule || i.entryGlobalsImmutable || m.GlobalSafe. - Why callback builtins are a denylist. A func value is a
ConstRefExprat the AST level (indistinguishable from a constant), and func values also arrive throughfunc-typed variables and untyped curried returns (makeAdder(1)- methods declare no return type), so a func-value argument cannot be spotted statically. The sound alternative is to name the builtins that can invoke.jcode (viaBuiltinCtx.InvokeorCallByName*/CallHostWith*). The invariant Conservative by construction: a script function that calls a module or uses a func value does not borrow (M25.1 still covers it when the script is globals-free or the call is into a module, whose own helpers borrow via isModule).- any such builtin must be listed - lives beside the set in
globalsafe.go.
- any such builtin must be listed - lives beside the set in
- What it buys. A read-only helper in a script that holds mutable globals goes linear: a per-element
peek(xs, i)scanned over a growing list drops from quadratic to linear oncepeek/scanare provenGlobalSafe. - Discipline.
internal/interpreter/globalsafe_internal_test.gopins the flag (pure / direct-write / index+append-write / nested-block / transitive / mutual recursion / cycle-reaching-a-writer / function-value-call);borrow_test.gopins the value-semantics soundness end to end (direct, transitive-via-callee,meta.call, andlists.mapglobal mutations all keep copy semantics; a GlobalSafe helper in a mutable-globals script borrows correctly). Race-clean.
M26 - jvc package manager (decks)
Planned. The package manager for Jennifer and its distribution: the tool itself (M26.1), distributing it so a Jennifer install can reach a working deck manager - jvc as its own OS package that depends on the interpreter, plus a copy baked into the OCI image (M26.2), gating each interpreter release on jvc's own test suite (M26.3), and keeping the language core package-manager-agnostic - the core reads no manager's lockfile, and engine enforcement stays the per-file pragma (M26.4).
M26.1 - the package manager itself
Planned. A package manager for Jennifer, in the shape of PHP's Composer (or Rust's Cargo): declare dependencies in a manifest, jvc install resolves and fetches them, and the app imports what it pulled. Installing an app becomes git clone + jvc install, and jvc update advances within the declared constraints.
- Packages are "decks". A deck is a distributable, versioned bundle of
.jmodules, published to a public deck repository / registry (provided later) thatjvcresolves and fetches from - packagist-style; a deck can also come straight from a git URL. - Naming conventions. A deck has two separate identities. Its canonical name is the
@vendor/decknamescope indeck.toml(what imports and jvc key on) - kept clean, with no "deck" word: an official deck is@jennifer/routeros, importedimport "@jennifer/routeros/";->routeros.*. Its GitHub repo name is cosmetic (jvc readsdeck.toml, not the repo name), so the "deck" marker lives there and only there:- Official decks (in the
jennifer-languageorg) use adeck-prefix -jennifer-language/deck-routeros. The prefix is the "not core, but official deck" statement: it clusters the decks and keeps the org's top level readable (corejennifer/homebrew-tapvsdeck-*). The org already supplies "jennifer", sodeck-is the useful signal (deck vs core). - Community decks (any account) can be named anything - jvc only needs it configured - but the suggested form is a
jennifer-prefix,alice/jennifer-routeros: on a personal account there is no Jennifer namespace, so "which ecosystem" is the useful signal instead. - All deck repos carry the GitHub topic
jennifer-deckfor discovery, which works regardless of the repo name.
- Official decks (in the
- Installed into the
vendor/tree M19.7 resolves.jvcwrites decks into the project-localvendor/tree that the interpreter already addresses through the@scope/packageimport form and vendor-root discovery shipped in M19.7 - so a hand-populatedvendor/imports beforejvcexists, andjvcis just the manager layered over that resolver. Nothing is global; each app owns its decks beside it. - The one remaining language-surface question: inline version selectors. M19.7 resolves
import "@jennifer/supercms/" as cms;(the trailing/expands to the package-named entrysupercms/supercms.j) against whatever is installed.jvcsupplies the default - plainimport @jennifer/supercms;takes the versionjvcresolved (declared indeck.toml, pinned in the lockfile), version-transparent, which is what almost every script wants. The opt-in for side-by-side versions is a per-import selector matched against the installed set (never triggering a fetch):@jennifer/supercms=1.2.3(exact) or asemverrange (>=1.2.3/~/^) over what is installed; one script can pin=1.xwhile another pins=2.x, and two versions in one file take distinctasaliases. An unsatisfiable selector errors pointing atjvc install, not a silent download. The range operators are not redefined here: they reference the registry'sspecs-server.mdsection 2.4 (normative for^/~, including^1/^1.2/^0/^0.0/~1), so a string means the same thing in adeck.tomland in an import - the drift between deck-spec and the registry specs is exactly what a parallel definition would reintroduce. A#commitselector is out of v1: git decks are already pinned by commit through[sources]+ the lockfile, and#commitis the one form with no home in the shared grammar. Cost: the selector is still new grammar (the lexer reads@vendor/deck+ asemverop as one token up to;); the plain M19.7 string-path form is the no-new-grammar fallback that loses only the inline selector. deck.tomlmanifest + lockfile.deck.toml(TOML, so it needs thetomllibrary) declares required decks and constraints (bitcoin = ">=1.2.0"), andjvcproduces acamcorder.locksogit clone+jvc installis reproducible: each lock entry pins either asha256:checksum (atar.gzdeck) or a commit (agitdeck), exactly one of the two - a git deck has no content hash. Dependency sets split by section ([decks]/[dev-decks]), with a taxative (the section is the exact set) vs additive (base plus the section's extras) mode still to design.jvcowns the lifecycle: dependency resolution (semver constraint solving across the graph), downloading,jvc update(advance to the newest constraint-satisfying versions, rewritecamcorder.lock), integrity pinning, and the publish flow to the registry. What jvc does not own is the authoritative engine check: jvc's install-time[engines]check runs against the installing interpreter, which may not be the one that later runs the app, so it is fail-fast advice; the binding check - run against the running engine - is the interpreter's per-file# pragma-jennifer-*header (M24.20), which stays package-manager-agnostic (M26.4).
Migrating bundled modules out to decks. Once decks exist, niche or product-specific modules that ship bundled today should graduate out into decks (the archetype is gotify - a single-product push integration every install need not carry); language-fundamental modules stay bundled. Moving one changes its import, so it is a breaking change under semver: within 1.x ship it both ways (bundled + @-deck) with the bundled copy marked @deprecated so imports re-point at their own pace, let the two drift without breaking, and remove the bundled copy in 2.0.0. Conversely, new third-party service integrations ship as decks from the start rather than as core modules (core stays general primitives; specific-vendor clients live in the ecosystem) - the M27 candidate-deck ecosystem collects the list (GitLab, GitHub, Steam, TheMovieDB, Jellyfin, Frigate, RouterOS, ...).
Requires: the public deck registry (separate infrastructure, provided later); its language-side prerequisites - the @scope/package resolver + vendor root (M19.7), toml, the module system, and semver's range surface - have shipped.
M26.2 - distributing jvc with Jennifer
Planned. Make a Jennifer install able to reach a working deck manager without a manual source fetch, while keeping jvc a separate project on its own release cadence. jvc is Jennifer source (src/*.j with co-located *_test.j, a bin/jvc launcher, completions/, a deck.toml) - no build step, no per-platform artifact - but it uses http / os.run / archive, so it runs on the default jennifer binary only: jennifer-tiny (no net / exec) never carries it, and every launcher must exec jennifer, never the tiny build.
The delivery path is chosen by whether the channel has a dependency resolver:
- OS packages: jvc is its own package that depends on the interpreter. The
.deb/ Arch / Homebrew jvc packages declarejennifer >= <floor>(the floor is jvc'sdeck.tomlengines.jennifer); installing jvc pulls the interpreter, and the interpreter package advertises jvc (optdepends/Recommends) for discovery. Compatibility needs no bundling: theM24.20# pragma-jennifer-version:header already fails a too-old interpreter loud at read time. On Debian and Homebrew the packageDependsalso carries a real floor, so there the dependency and the pragma are belt and suspenders. On Arch a versioned floor is not expressible until the interpreter package declares a versionedprovides-provides=("jennifer=$pkgver")onjennifer-bin/jennifer-git, replacing the bareprovides=(jennifer)that makesdepends=('jennifer>=0.25.0')unsatisfiable (pacman refuses it outright); until that lands, jvc's Arch package depends on an unversionedjenniferand the pragma is the sole floor there. jvc owns/usr/bin/jvcand its own packaging (its repo already carriespackaging/archlinux/; apackaging/debian/and a Homebrew formula join it), so there is noprovides/conflictsdance with the interpreter. This drops the whole vendoring surface an earlier design carried: no copy of jvc in the jennifer tree, no sync script, no pinned-SHA drift check, and nojennifer version->(jvc X)codegen stamp.jvc versionstays the sole authority on which copy is running - a self-installed~/.local/bin/jvcshadows the packaged one by PATH order, and its provenance report is the mitigation. - The OCI image bakes jvc in. A container image has no dependency resolver -
FROM jenniferthenRUN jvc installin an app build is the core workflow, so the image itself must carry jvc, or vendored-package installs are impossible in it. The Dockerfile fetches jvc at a pinned ref (a commit, or a tag once jvc cuts one; never a mutable branch) in its build stage - the image build is already online (it pulls the base image andwgets TinyGo), so a fetch-at-build here does not touch the interpreter's own offline release build - and stages jvc under the same layout its OS package uses (/usr/share/jvc/withsrc/minus*_test.j+bin/jvc, and/usr/bin/jvca symlink to the launcher), so that if the slim image later installs the jvc.debinstead of fetching source, the two layouts already agree andjvckeeps resolving. (jvc's package ships exactly this symlink layout and relies on the launcher's relativeimport "../src/cli.j"resolving through it - confirm that resolution rather than re-inventing a wrapper.) Only the slim stage carries it (the full host build). Registry-deck installs are git-free (jvc'sgitcalls sit only in the publish /[sources]paths), so the base image does not strictly needgit; add it for the two cases that do - a project with a[sources]git deck, andjvc app install- and document those.ca-certificatesfor the https registry are already present. The static / distroless stage does not carry jvc (no/bin/sh, nogit) - that is the shape, not a gap: the recommended deploy is multi-stage,jvc installin ajennifer:slimbuild stage to populate./vendor, thenCOPYthe app plusvendor/intojennifer:staticfor a small runtime image. TheJVC_REFpin lives as a build-arg indocker.yml; once jvc ships a.deb, the slim stage may instead install that package, so the image jvc and the OS-package jvc are the same artifact. - The raw tarball /
curlinstall has no resolver and is not a container, so it points at jvc's own plain tarball (which jvc will ship) for the first install -jvc app install/jvc app updatecover the channel afterward but are chicken-and-egg for the first - with no bespoke installer script. Bundling jvc into the interpreter tarball is the one place bundling could still earn its keep, deferred until there is demand.
Rejected: vendoring jvc's source into the jennifer tree (a third_party/ copy or a git submodule, with a sync script and a CI drift check) - it couples two repositories that want independent cadences and adds a maintenance surface the pragma + package dependency make unnecessary; and stamping jvc's version into jennifer version - redundant once jvc is its own package carrying its own version.
Sequencing: jvc's deck.toml sets engines.jennifer = ">=0.25.0", so the first release wiring any of this ships at or after 0.25.0; jvc has no tags yet, so the OCI pin is a commit SHA until it does.
Requires: M26.1 (a jvc to distribute). Do not wire the OCI pin ahead of a jvc that runs against the target interpreter.
M26.3 - jvc suite as a release gate
Planned. Run jvc's test overlays against the release-candidate interpreter in release.yml - jennifer's CI checks jvc out at a pinned commit and runs its suite against the RC binary; a red suite blocks the tag. This is independent of how jvc is distributed (M26.2): the gate protects the language's own package manager whether jvc ships as its own OS package or baked into an image, so no vendored copy is needed for it. jvc is the language's most complete integration test - its assertions exercise toml / json / semver / archive / http / fs / os / path / hash / encoding / convert / strings / lists / maps / docblock / flatdb, git through exec, and the module system itself (cross-module struct identity, the @vendor resolver, the capability pragmas), so a change that breaks the language's own package manager cannot ship.
- jvc is a first-class caller. The gate fits the existing breaking-change discipline (one batch updates a library, its overlays, docs, and every caller): jvc becomes another caller updated in the same batch. An unintended break is a hard red caught in the language CI instead of jvc's; an intended break (a pre-1.0 milestone is free to make one) is handled by advancing the pinned jvc commit to a version already green against the RC, in that same batch - the gate passes because the pin moved, not because anything reverted.
- The pin policy is written, not implicit. A release pins a jvc commit known-green against that RC. An intended interpreter break requires a compatible jvc commit to exist before the tag; if jvc cannot keep up, the release waits or holds the pin back and documents the incompatibility. The language cut thereby depends on jvc being green - an acceptable reverse dependency for one maintainer who owns both repositories, named here so it is a deliberate choice.
- The gate needs
gitand a writable temp. 65 of jvc's 538 overlay tests shell out to the realgitbinary (its[sources]/ publish paths); the rest are hermetic - no network, no credentials, no fixtures outside the temp dir. The CI job must havegitonPATHand a writable temp, or those 65 fail for reasons unrelated to the RC (the worst kind of gate failure); GitHub'subuntu-latestsupplies both. The suite is green on0.24.0-dev(538 passed, lint clean), and jvc's HEAD is pinnable today - move the pin to the0.1.0tag's commit once jvc tags.
Requires: M26.1 and jvc's overlay suite (the pin is a jvc commit in jennifer's CI config, not vendored source). The two recent breaks it would have caught - cooked-string {expr} interpolation turning embedded JSON literals into lex errors, and the fmt / lint line-width disagreement on a func signature (fmt joining one past 100 without counting the trailing {, and never wrapping a single-parameter signature at all) - motivate it; both halves are now fixed in the tree (fmt wraps an overflowing signature of any arity), which the gate would confirm ships.
M26.4 - the core stays package-manager-agnostic
Planned. The language core reads no package-manager lockfile - not camcorder.lock, not any alternative manager's. jvc is the endorsed package manager, not a required one: a lockfile is the manager's private artifact (jvc reads and writes camcorder.lock; a different manager would carry its own), and teaching the core to parse one would both couple the language to jvc and foreclose the alternative managers an ecosystem should stay free to grow. So no jvc-specific file or requirement enters the core.
- Enforcement is the per-file pragma, not a lockfile - with the flavour split spelled out. The M24.20
# pragma-jennifer-version:/# pragma-jennifer-capability:headers are enforced at every import seam (CLI /include/loadModule), against the running engine (authoritative in the way an installing manager's check cannot be) and manager-agnostic (they fire the same for a jvc-installed deck, an alternative manager's, a hand-populatedvendor/, or agit clone). But the pragma checks version and capability, not a general flavour, sojennifervsjennifer-tinyseparates in three parts:- Version - the pragma refuses a too-old engine at import.
net/exec/sql- the capability axis (meta.CAPABILITIES,[]on tiny) is the flavour gate for the common case: a deck that needs one declares it and is refused on tiny at import. Core, authoritative, neutral.- The other default-only surfaces (
term/serial/spi/i2c/gpio, andcryptoRSA/ECDSA) are not capabilities, so the pragma does not reject such a deck at import; each library'sjennifer-tinybuild-tag stub catches it instead, with a friendly "not available on this build" error at the first call - tiny never silently mis-runs a default-only deck, it just fails at call time rather than import time. This gap is jvc's to cover, as fail-fast install-time advice (its[engines]namingjennifer, refusing the install under tiny before the app runs) - the flavour separation that falls to jvc, mirroring how its version check is advice while the pragma is binding. An authoritative import-time gate for these surfaces would instead be a neutral core follow-on (widen the capability set to name them, or a coarse engine key), kept in the core because flavour is the language's own distinction and only the running engine can check it - never a jvc-specific read. The lean is to defer that: the tiny stubs already stop a silent mis-run, so the capability pragma (cases 1-2) plus the stubs plus jvc's install-time advice cover case 3 well enough; add the core axis only if an import-time miss actually bites, not preemptively.
- What the core does at a
@scope/deck/import stays the M19.7 job: resolve the path in thevendor/tree and load it, enforcing each file's pragma. No manifest, no lockfile, no package-manager knowledge - which is what lets a second manager write into the samevendor/tree and Just Work. - Coordination with jvc. jvc's
deck-spec.mdcurrently makes the resolver readingcamcorder.locknormative (section 5 "the authoritative check", section 11.2 "SHOULD consult the nearestcamcorder.lock"). That is the one point where the two documents disagree, and it resolves on jvc's side: deck-spec is revised (and minor-bumped) so the authoritative run-time check is the interpreter's (the per-file pragma, backed by the tiny build-tag stubs) andcamcorder.lockstays jvc's reproducibility / integrity artifact, not a file the core is required to read. jvc's install-time[engines]check remains, as fail-fast advice (version and flavour), as an input to resolution.
Requires: nothing new in the core (the M24.20 pragma already ships); a deck-spec revision on jvc's side dropping the core-reads-lockfile requirement.
M27 - candidate decks (deck ecosystem)
Planned. A running parking lot of deck ideas - third-party service and integration clients that, once jvc / decks (M26) land, ship as decks rather than core modules/. The rule: core stays general primitives (protocols, formats, infrastructure - http, graphql, csv); a client for one specific vendor or service lives in the ecosystem as a deck (independently versioned, community-maintainable, so vendor API churn never touches the core). This list is demand-driven and open-ended, a collection not a commitment.
Most are thin clients over http / rest + json (plus xml where a vendor returns XML, and the graphql module where the API is GraphQL). Each is a login / token step, a generic call(path, params) -> json.Value, and a handful of conveniences; a fat typed wrapper is explicitly not the plan - these APIs are enormous and firmware-versioned, so a thin client ages far better.
M27.1 - self-hosted infrastructure
A LAN appliance, usually a self-signed cert. Per-vendor maturity differs and should set the order, not the vendor:
routeros- a full RouterOS abstraction layer (MikroTik), well beyond the small bundledmikrotik.jAPI client. Already in progress; the likely first published deck.proxmox- Proxmox VE: documented REST / JSON, API-token header auth (PVEAPIToken=...). Clean.vmware- vCenter / vSphere (vCSA): the vSphere Automation REST API (JSON, session auth), clean and tractable like Proxmox. A standalone ESXi host is the messier case - historically only the SOAP / VMOMI Web Services API (XML / SOAP, heavier), with just partial REST on 8.x - so vCenter is the target, a bare ESXi host best-effort.synology- DSM: the cleanest NAS API - a documented Web API (SYNO.API.Authlogin -> sessionsid, JSON responses).unraid- the newer official API is GraphQL over HTTP with an API key.qnap- QTS: the messiest - much of the useful surface is undocumented, reverse-engineered CGI with XML responses and a legacy hashed-password auth; firmware-fragile and hard to keep green. Lowest priority, on real need only.ugreen- NASync / UGOS Pro: no official API - only the internal token-based API the web GUI uses (log in for a token, then GET), reverse- engineered by the community (a working Home Assistant integration exists). Same keep-green risk asqnap; UGOS is new and evolving, so revisit if UGREEN ever ships official docs.jellyfin- the self-hosted media server's REST API (API-key / token auth).frigate- the Frigate NVR REST API (events / config / recordings), often paired with its MQTT feed (themqttmodule).
M27.2 - public / SaaS APIs
gitlab/github- dev-platform clients; both expose REST and GraphQL, token auth.steam- the Steam Web API (JSON,?key=auth); parts of the surface are community-reverse-engineered, which is exactly why it belongs in a deck, not core.themoviedb- TMDB's clean, well-documented REST JSON API (bearer / key auth).
M27.3 - daily helpers
A grab-bag of small, everyday helper decks - the little utilities a script reaches for often, each a tiny pure-.j deck.
- cli spinners - a small deck of terminal progress spinners for a long-running task: a catalogue of spinner styles (dots / line / bar / braille / arc / ...), each individually colourable (through
ansi) and driven at its own speed (frame interval). Self-suppresses when stdout is not a TTY (os.isTerminal), so piped output stays clean.
M27.4 - bioinformatics
A sequence-manipulation deck for DNA / RNA / protein, modelled on the Sequence Manipulation Suite (SMS) tool catalogue - the classic, comprehensive reference for this surface. Almost all of it is pure string / list / map work, so it is a natural pure-.j deck (leaning on strings, regex, lists, maps, math, and stats, no Go), dogfooding the language and community-maintainable as tables and algorithms are added. Grouped SMS-style:
- Transforms -
complement,reverseComplement,reverse,transcribe(DNA <-> RNA),translate(codon table), six-frame translation,splitCodons, amino-acidoneToThree/threeToOne. - Composition & properties -
gcContent, base / amino-acid composition,dnaStats/proteinStatssummaries,molecularWeight(DNA / RNA / protein),meltingTemp(Wallace + nearest-neighbour),isoelectricPoint(pI),gravy(hydropathy),codonUsage. The thermodynamic / pI / nearest-neighbour formulas wantexp/ln/log, so they pull inM24.10(mathfoundations). - Search - exact and IUPAC-ambiguity pattern find (via
regexcharacter classes), fuzzy search (n mismatches), ORF finder (six frames), CpG islands, restriction-site search andrestrictionDigest(fragments) over a bundled enzyme table. - Formats - FASTA parse / write (a
Record{id, description, sequence}list), FASTQ reads (sequence + quality), format conversion,filterDna/filterProtein(strip non-sequence characters). - Manipulation -
randomDna/randomProtein(length + optional composition), composition-preservingshuffle, pointmutate(rate), range / sliding-window extraction. - Comparison - pairwise
alignGlobal(Needleman-Wunsch) /alignLocal(Smith-Waterman) with percentidentity/similarity, plushammingDistance/editDistance. Alignment is the oneO(nm)hot loop - fine in.jfor the small sequences a deck user handles; a candidate Go primitive only if whole-genome throughput is ever needed. - Reference data - the standard genetic code + alternative codon tables, IUPAC ambiguity codes, per-residue property tables (MW / pKa / hydropathy), and a common-enzyme table - all plain
.jmaps a community can extend.
FASTA / FASTQ I/O and alignment cover the "molecule structures" the original note gestured at; PDB / 3-D structure parsing stays out of v1 (a much larger, separate effort).
M27.5 - forensic / statistical genetics
The statistical-genetics sibling of the sequence deck (M27.4): it works on profiles (an unordered allele pair per autosomal STR locus, plus uniparental haplotypes for mtDNA / Y-STR) and reference frequency / count data, not on sequences, so it shares no code and no audience with SMS. Pure .j (probability arithmetic over allele-frequency maps, no Go, TinyGo-clean), leaning on math and on xml / text parsing to ingest a published frequency table.
- Match probabilities - Hardy-Weinberg single-locus genotype frequencies (homozygote
p^2, heterozygote2 pa pb), with the NRC II theta / Fst sub-population correction (Balding-Nichols), multiplied across loci:randomMatchProbability(RMP), the single-source matchLR = 1 / RMP, andcpi/cpe(combined probability of inclusion / exclusion) for mixtures. A minimum-allele-frequency floor (5 / 2N, so the table keeps each locus'sN) handles rare or unobserved alleles. - Lineage markers (mtDNA / Y-STR) - mitochondrial and Y-chromosome markers are haploid, non-recombining, and uniparentally inherited, so match probability is not Hardy-Weinberg but a direct haplotype count: frequency
k / Nin a reference database, with a Clopper-Pearson exact-binomial upper bound as the conservative estimate (which pulls in thebetadistributionM24.11adds tostats). Deliberately database-independent, so the deck supplies the estimator (haplotypeFrequency(k, N)/lineageMatchProbability) and the caller feeds the count from whatever database they queried. mtDNA adds a haplotype coded as differences from the rCRS (e.g.263G 315.1C) plus the substantivealignstep that renders a raw sequence into that standard nomenclature; Y-STR is a per-locus repeat-count vector with exact / single-step comparison. - Kinship likelihood ratios - relationships encoded as IBD coefficients (kappa0 / kappa1 / kappa2: parent-child
(0, 1, 0), full sibs(1/4, 1/2, 1/4), half-sib / avuncular / grandparent(1/2, 1/2, 0), first cousins(3/4, 1/4, 0), ...).kinshipLR(a, b, relationship, db)tests one relationship against another;paternityIndex(mother, child, allegedFather, db)gives a combined paternity index (CPI) andprobabilityOfPaternity(W = CPI / (CPI + 1)), with a stepwise STR mutation model to survive a lone inconsistency. General pedigrees (beyond pairs / trios) need a peeling engine - Elston-Stewart - the one substantial algorithm, the Familias /forrel-style capability.
Mixture deconvolution (multi-contributor, drop-in / drop-out - the EuroForMix space) is a larger, later effort layered on this base.
A concrete frequency source to wire in first, as a sample: the ENFSI STR reference database STRidER - a loadFrequencies(xml, "strider") over the xml library, with its formulae page and online calculator as the exact-conventions spec and a validation target.
M27.6 - NGS / high-throughput sequencing
Unlike the pure-.j decks above, NGS breaks the model on two axes - scale (FASTQ.gz files run 10s of GB, BAM 100s, so everything streams, never load-into-memory) and compute (alignment / assembly / variant calling are heavily SIMD-optimised C a tree-walker is ~100-1000x too slow to replace). So the deck is deliberately the glue and light-I/O layer, not the heavy engine, with a different posture from the pure-.j decks: Go-backed streaming parsers (the decompress-and-parse hot loop in Go, the net.readAll / binary pattern) with the per-record logic in .j. In scope:
- Streaming format I/O - FASTQ (gzipped, over
compress+fshandles) and the tab-delimited SAM / VCF / BED / GFF / GTF text formats: parse / filter / convert as a stream, so a 50 GB file never lands in memory. - QC + preprocessing (FastQC / fastp-lite) - per-base quality and read-length / GC distributions, adapter detection, quality / adapter trimming, length filtering, subsampling: one streaming pass over FASTQ.
- Interval ops (bedtools-lite) - overlap / intersect / merge over BED / GFF features.
- Pipeline orchestration - the real sweet spot: NGS is fundamentally
bwa | samtools | gatkglued together, and Jennifer already hasos.run/os.spawn+spawnconcurrency, so it can drive the standard tools, parse their output, manage intermediate files, fan out over samples in parallel, and handle errors - a Snakemake-lite in a real language.
Out of scope (wrap and pipe the native tool, do not reimplement): read alignment (BWA / Bowtie2 / minimap2), de-novo assembly (SPAdes), variant calling (GATK), and heavy BAM / CRAM handling - which additionally needs BGZF (blocked gzip for random access), a compress gap and a Go addition whose standalone value is limited without the downstream compute the deck omits.
M28
M28.1 - runtime .j module loader (plugin library)
Planned. Load a .j module from a path computed at run time and dispatch into it, so a plugin needs no os.run + stdio bridge - the three current loaders resolve before execution from a string literal (use compiles a library in; include splices tokens in preproc; import resolves at Run() start), so no runtime-chosen module can be loaded today. This is a system library over the existing M17 loadModule, not a new keyword: include / import are compile / load-time (token splice, parse-time type stamping, pre-execution resolution), whereas a runtime load is side-effecting and takes the handle-returning shape of fs.open / kv.open.
- Surface.
plugin.load(path) -> plugin.Module(resolve + run once, cached);plugin.func(m, name) -> func;plugin.call(m, name, args...);plugin.const(m, name);plugin.has(m, name);plugin.exports(m) -> list of string. - Dispatch.
plugin.funcreturns afuncvalue whoseFnHomeruns the body in the module's context with argument / return struct retagging;plugin.callis the by-name wrapper (meta.callpath). - Reuse.
loadModulesupplies run-once caching, the@vendor/ search-path sandbox (module.Resolve), and the M24.20 version / capability pragma gates unchanged (anetplugin fails to load onjennifer-tiny). Run-once, no unload or reload. - Interpreter deltas. Expose
loadModulethroughBuiltinCtx; skip the static-import compile-time steps (bindModuleAlias,resolveDeclaredTypesOnce). Add a mutex to the module registry (cache/stack/resolved) - lock-free today, correct only because imports load single-threaded before anyspawn, which a runtime load breaks. - Type boundary. A plugin's struct types are unknown at parse time, so the host cannot write
def x as plugin.T. Contract: a shared interface module, imported normally, declares the struct types and the plugin implements funcs over them (values retag at the boundary); primitives, collections, and opaque handles cross without a shared declaration. - Trust. Loading
.jruns code; no new sandbox (untrusted code staysDRAFT#11), and path resolution is theimportsandbox.
Requires: M17 (loadModule), M19.7 (@vendor resolver), M24.20 (version / capability pragma), FnHome func values. Open: handle as an integer registry (kv.Store pattern, survives spawn / boundaries) vs KindObject; runtime load inside a spawn vs main-goroutine-only; a stricter plugin.load path allowlist than import.
M28.2 - non-blocking / timed terminal read
Planned. A bounded terminal-read primitive that fixes two coupled defects: a free-running TUI loop stalls in the blocking term.readByte, and screen.nextKey cannot deliver a lone Escape (it reads a second byte unconditionally to tell a bare Escape from a CSI / SS3 opener, and that read blocks until the next keypress - which is then swallowed into the sequence, so Escape then x arrives as one alt-x). The "escape" name the screen.Key docblock advertises is therefore reachable only through decodeKey directly, never from nextKey.
- Primitive.
term.readByteTimeout(ms) -> int: the byte0..255,-1at EOF,-2on timeout (a distinct sentinel, not conflated with EOF).ms == 0polls (return immediately). The blockingterm.readBytestays as the simple path. - Implementation. It cannot use the
ctx.Inio.Readerabstraction (what the injected-input tests feed; it has no deadline), so it reaches the raw fd and eitherpoll(2)s it with a timeout or sets termiosVMIN/VTIME. That conflicts withterm.makeRaw's blockingVMIN=1, so entry into raw mode and the timed read must be reconciled (the timed call setsVMIN=0/VTIMEfor its duration, orpolls without touching termios). Unix-only,x/sys/unix-backed, build-tag split like the rest ofterm(real onlinux && !tinygo, friendly stub elsewhere). screenfix.nextKeyreads the escape-disambiguation byte with a short timeout (25-50 ms) and returnsKey{name: "escape"}when it expires, so a loneEscapeis delivered and the following keypress is no longer consumed. This makes the documented"escape"name reachable fromnextKeyand removes the docblock caveat.- Bonus. The primitive also lets an app stop raw input and return to line mode mid-program (the
screenbackground-reader task blocks in the syscall and cannot be interrupted bytask.cancelor a channel close today), and lets a caller bound a single read without a dedicated reader goroutine.
Requires: M20.5 (term library), M21's screen module. Graduated from DRAFT#29 (retired). Open: whether to also expose term.pollByte() as sugar for readByteTimeout(0).
M28.3 - append sugar for any place expression
Planned. Extend the $xs[] = v append target from a bare variable to any assignable place expression - $obj.field[], $xs[i][], $obj.a.b[] - the same lvalues indexed assignment already accepts ($grid[i][j] = v is legal today). The sugar currently rejects them at parse time (unexpected token RBRACKET), so a list that lives in a struct field has no in-place append and the only direct fill is $obj.field = lists.push($obj.field, v), which copies the whole list each pass - O(N^2), measured at roughly 1000x slower over 20,000 appends. That collides with JENNIFER.md's active "prefer $xs[] over lists.push in loops" recommendation, which is unfollowable for the struct field where most real data lives.
- Lowering. An append into a place expression resolves the target the same way indexed assignment does (walk to the innermost backing, mutate in place), then appends - reusing
execIndexAssign/execFieldAssign's lvalue resolution rather than adding a parallel path. Element-type checked and const-rejected like the bare-variable form. - Surface. Additive and non-breaking (it only accepts input that errors today). Removes the "Chained forms (
$xs[0][]) are not supported" limitation. - Touches. Parser append-target grammar, the resolver, the interpreter append path, the grammar EBNF in
docs/technical/, and the JENNIFER.md notes that currently document the limitation and the workaround.
Requires: M13.1 (structs), the existing indexed / field-assignment lvalue machinery.
M28.4 - streaming xml decode
Planned. xml.decode builds a full DOM up front - every element a KindObject wrapping a {tag, attrs, children} map, every node / attr / text a 280-byte interpreter Value - so the tree runs ~150x the source and is all live at once: a 10 MB GPX measures ~1.5 GB RSS, and 3 copies do not fit in 4 GB. Reported twice (the geo deck wrote a ~900-line hand scanner to avoid it); the whole XML family is a long flat list of small records, the exact shape a pull parser handles in constant memory. Keep xml.decode for small documents; add a streaming reader for large ones.
- Primitive (pull loop, no callbacks - the M23.1 shape).
xml.reader(bytes) -> xml.Reader(operates onbytes, so a caller that scanned a buffer hands it over without a string copy, leaning on the no-copybytes-arg guarantee), thenxml.next(reader) -> xml.Eventyielding start / text / end / eof. Memory is O(depth) + the source, not O(document). xml.Eventis small (kind,tag,text,depth); attributes read through the existingxml.attr/xml.attrsaccessors (no new vocabulary).- Subtree iterator (the ergonomic layer):
xml.nextElement(reader, name) -> xml.Valueyields one small per-element DOM at a time (the "iterparse" shape), sofor each trkptholds O(one trackpoint), reusing every existing accessor. - Local-name matching. The reader matches
trkpt/laton agpx:-prefixed real-world file (todayxml.attrraises ongpx:lat), closing the namespace pain the geo report also hit. - Handle is an integer-registry entry (the
net.Conn/fs.Filepattern), closed with the polymorphicclose; hand-rolled over the existing position-based scanner, so TinyGo-clean. Malformed markup is a positioned catchable error.
Requires: M20.2 (xml library). Follow-on: the same DOM blowup is shared by json / toml / yaml (all 280-byte-Value trees); a streaming decode there is a later generalization, not part of this.
M28.5 - strings cutset / affix trim
Planned. trim / trimLeft / trimRight strip Unicode whitespace only, so there is no way to strip a chosen trailing character - writing a fixed-precision number without trailing zeros, stripping a \r from a CRLF line where leading / trailing spaces are significant, peeling a known prefix. Requested in three deck reports; each hand-rolled a loop.
- Add
strings.trimPrefix(s, prefix)/strings.trimSuffix(s, suffix)(exact affixes, the most-used Go pair) and the cutset familytrimSet(s, cutset)/trimLeftSet/trimRightSet(strip any rune incutset). - One way per thing: the existing
trim*stay whitespace-only (their canonical job); the new names are distinct operations, not an overload - no arity change to the current trio (consistent with howtoInttook a parameter rather than spawningparseHex-style siblings).
Requires: none (additive strings surface).
M28.6 - value-semantics copy elision (return / const read)
Planned. Value semantics deep-copy a compound on every store, which is correct but compounds: reading 50,000 points into a nested Document copies the point list ~7 times (out of the reader, into each literal, into each enclosing list), and a def const ... as bytes probe profiled 11,486 eager copies of an immutable value. GOGC=20 barely moves peak RSS (1.23 -> 0.93 GB), confirming retention from copying rather than collectable garbage. The read-only-parameter borrow (note 24) already removes the argument-binding copy; this extends that reasoning to two more hand-off shapes, keeping value semantics observably intact (a borrow of an immutable source is identical to a copy).
constcompound read. Adef x as T init SOME_CONSTcopies a deeply-immutable value for no semantic reason. Elide it where the target cannot mutate the shared backing (the borrow analysis already in place), or mark const-sourced bindings copy-on-write.- Return-then-store hand-off. A value returned and immediately stored (
def xs init readSegment(...)then$segs[] = Segment{points: $xs}) has one live reference throughout; an escape analysis narrow enough to catchreturn $localremoves most of the seven copies without weakening any guarantee. - Document the existing guarantee the geo report verified as a positive: a large
bytesargument is not deep-copied (note-24 borrow) - state it as a contract, not an observation.
Requires: the note-24 borrow/escape machinery. Pin value-semantics parity with the value_alias stress tests (note 10) for every elision.
M28.7 - unify resource-exhaustion errors under kind: "limit"
Planned. A caught cap breach should tell a caller "your input was too big / too deep" apart from "something inside me is broken" - so a library walking a user-supplied tree can re-raise it in its own vocabulary and a deck can keep its kind contract without first imposing its own limit. The "limit" kind and the boundary mechanism already exist (the call-depth cap and archive.unpack / unpackWith raise it); this sweep moves the remaining size / depth / count caps - which today raise a generic kind: "runtime" - onto it, each with a regression test asserting the kind.
Scope, highest-value first:
- Tier A - input-driven decode / expansion bombs (the clearest win, untrusted input):
json/toml/yaml/xmlnesting caps, theyamlalias-bomb node budget,asn1depth + node-count,compressdecompressed-size,intltranslation amplification, and thenet.readAll/readBytes/readN/recvFrombyte caps. - Tier B - explicit size / allocation caps (caller asked for too much):
binary.make,channel.make,crypto.randBytes/hkdf/pbkdf2(size + work),devioread,fs.readBytes, theioprintf field cap,linalgmatrix / vector element cap. - Tier C - handle-leak guards (
fs/net/httpd/compress"too many open ...", thehttpdbuffer budget): deliberately out of scope - these signal a lifecycle leak, not oversized data, so they stay"runtime"(revisit if a distinct lifecycle kind is ever wanted). - Mechanism. A shared helper so each site is one call, not per-builtin boilerplate: a library returns a sentinel/typed cap error (the
archivecapErrorpattern) and a common boundary translator raisesinterpreter.RaiseError("limit", ...)with the call-site position - or a directraiseLimit(ctx, msg)where the builtin hasctx. Interpreter-core caps setruntimeError.Kind = "limit"directly (as the call-depth sites do). Pure error-kind reclassification: no message or control-flow change, each paired with akind == "limit"test.
Requires: none hard (the "limit" kind + RaiseError boundary shipped with the archive / call-depth work). Breaking only for code that catches these and branches on kind == "runtime" (pre-1.0, documented per site).
M28.8 - scope-rule and binder diagnostics
Planned. Three small, independent refinements at the edges of the scope rules.
- A parameter may shadow a global (decide: forbid, or formally allow). The no-shadowing rule rejects a
deflocal, a for-each iterator, acatchbinding, and amatchbinder that collides with a visible global, but a method parameter is not checked -def x ...; func f(x as int) { ... }is accepted and$xinsidefsilently means the parameter. The behaviour is well-defined (innermost binding wins, and the borrow scan relies on that), so this is an inconsistency, not a correctness bug. The decision is which way to make it consistent: forbid it (as the sibling binders are) or formally allow it (a parameter may shadow a global, innermost-wins, narrowing the written rule to local-over-local). Evidence leans toward allowing - several shipped examples name handler parameters (token/user/params/srv/driver/data) that coincidentally match a global, so a ban is a pre-1.0 break rejecting a natural, unambiguous pattern. Either way the fix is a few lines inresolveMethodplus a one-direction spec wording change, so the rule and the implementation finally agree. - A read-only
matchbinder reports the wrong rule on mutation. An enum payload binder (when Circle(c) { ... }) is read-only - it binds a copy - but is registered as an ordinary non-const untyped variable, so an attempt to reassign or mutate it is caught late with a message that does not describe why:$c.r = 99reports "cannot mutate contents of constant" and$c = ...reports "cannot assign enum to<invalid>variable" (the<invalid>is the binder's absent declared type leaking into the text). Registering it as a constant is wrong - a binder is referenced with the$sigil like a variable, and the constant machinery would demand the bare form - so it needs a distinct read-only variable flag on the binding (aslotInfobit plus an assign-time check), yielding one accurate message ("the match bindercis read-only; its payload is a copy"). This adds a third binding mode (mutable /const/ read-only-binder) the resolver and interpreter thread consistently. - A hoisted method can read a global before its
defhas run. Top-level statements execute in source order and methods are hoisted, so a method called above thedefof a global it references reads the pre-sized zero slot (anull) rather than the eventual value - and for adef constthe empty slot is misclassified, emitting the wrong hint ("MAXis a variable; use$MAX"). The behaviour is order-dependent by design, but a cleaner outcome is an explicit "used before its definition" diagnostic naming the laterdef's line, which needs a per-global "initialised yet?" flag checked on read. The largest of the three and the smallest payoff (a rare, order-dependent footgun); recorded mainly so the misleading constant hint is not forgotten.
Requires: none. The first two are self-contained resolver / interpreter changes (the first also needs a one-line spec decision); the third is additive init-order tracking.
M28.9 - static call-site checking in jennifer lint
Planned. jennifer lint resolves a call's name (L107) but not its arity or argument types, so compile-legal code that cannot run - a bytes passed to a string parameter, a call left one argument short after a signature changed - clears a CI lint gate and surfaces only as a runtime error. The runtime diagnostic is already precise (argument 3 to "token" must be string, got bytes); this surfaces the decidable subset at lint time, emitting only where the type is statically known so the checks stay false-positive-free (the property that earns a place in a CI gate). New IDs in the L1nn correctness band: L109 (arity), L110 (argument type), L111 (initializer type). Same-file L109 has shipped; the remainder:
- Same-file argument / initializer types (
L110/L111). Compare an argument's, or adef ... initexpression's, static type to the declared parameter / binding type, but only when that type is knowable without inference: a literal, or a variable / parameter / constant / struct-field reference the lint scope pass (scope.go) already resolves. Catches arity-drift and same-file type mismatches with zero false positives; additive over the existing AST and scope machinery. - Cross-module tier. Extend the same three checks across an
importboundary, socallee.wantsString(42)is caught the way a same-file call is. Lint sees the import declarations today but never loads the target file; parsing the imported module's AST to read callee parameter types (reusing the interpreter's module resolver) closes that gap, making lint a trustworthy gate for a module-heavy repo whose checked logic sits behind module boundaries. - Library-builtin signatures. The tier that catches a library return type flowing into a typed slot (
web.body(ctx), abytes, passed to astringparameter). Builtins carry no static signature (each validates its own arguments at runtime), so this adds a static signature table (arity + argument kinds + return kind), attached at registration or as a declarative side table, across the ~40 libraries - TinyGo-clean and without bloatingRegisterNamespaced. Independently valuable for editor tooling and doc / cheatsheet generation. How to attach signatures without that bloat earns adesign-decisions.mdrecord.
A user method declares no return type by design, so f(g()) with g a user method stays undecidable in one pass and is deliberately skipped (inferring a return type, or an optional annotation, is a separate decision).
Requires: none hard; builds on the shipped L107 / L109 lint and scope machinery.
M28.10 - compile-time validation of literal printf format strings
Planned. io.printf / sprintf / eprintf parse their format string at runtime, so a malformed verb or modifier in a string literal fails on the line that runs rather than when the file is checked: an unknown verb (%x), a modifier the verb does not take (%d|prec=2), a bad value (%d|base=3), a duplicate key, or a verb count that does not match the argument count all surface only when that line executes - possibly on a rarely-taken error path.
When the format argument is a string literal the whole spec is statically known, so the same io/format.go parser that runs at call time can run at check time. Two shippable shapes: a jennifer lint rule that re-parses each literal format and reports a bad verb, modifier, or argument count at its source position (additive, false-positive-free - a non-literal format is simply not checked); or, stronger, a parse-time error for a statically-known format whose spec cannot parse (the "undefined results error" stance applied to a format string). The lint form is the lower-risk first step and turns a whole class of runtime surprise into a pre-commit finding.
Requires: none hard; builds on the existing io/format.go spec parser and the jennifer lint framework.
M29 - multiplatform: promote macOS / Windows to supported
Linux is the only supported platform, but best-effort unsupported macOS / Windows binaries (the standard-Go jennifer, via cross-compile) already ship each release - so the work is not "add the ports" but promoting them to supported, which graduates the "cross-build for macOS / Windows" 1.0.0 distribution requirement. A portability audit found the surface small: separators / EOL / $HOME / temp are already runtime.GOOS-derived (internal/lib/os/oslib.go), os/exec keys on runtime.Compiler != "tinygo" (not GOOS, so it is enabled on Windows), and signals + the four Linux-only hardware libs (serial / spi / i2c / gpio) stub cleanly on non-Linux (*_other.go). Extra distribution packaging (a Homebrew tap, Snap, Nix flake, Flatpak / AppImage) stays a per-format nice-to-have, shipped only when a user asks and a maintainer keeps it green - none blocks a release.
M29.1 - Windows: promote to supported
Planned. The Windows track is a handful of concrete gaps, not a rewrite:
- Exe-relative module default.
compileDefaultSysmoddirbakes one hardcoded POSIX path (/usr/share/jennifer/modules,internal/module/sysmoddir.go) into a Windows binary; give it a Windows-native, exe-relative default (<dir(os.Executable())>\share\jennifer\modules) via a build-tag split (sysmoddir_windows.go/_unix.go), so a portable-zip user'simport "name.j";resolves with no env var (theM21.13installer'sJENNIFER_SYSMODDIRstays the explicit override; precedence unchanged). - Per-OS golden strategy.
examples/expected/osinfo.txtis the sole platform-pinned golden (pinslinux/amd64///:, compared byte-exact incmd/jennifer/examples_test.go); add per-OS expected-file selection or aruntime.GOOS-gated skip for the osinfo canary (already flagged atexamples/osinfo.j). fs.chmod/fs.chownon Windows. Define the Windows behaviour (a friendly catchable error is acceptable; thechowntest is already Linux-gated), and document that signal-based graceful shutdown is limited on Windows (signal_other.gostubsos.catchSignal).- A
windows-latestCI test job runninggo test ./...so Windows correctness is actually verified (the exec suite self-skips off Linux; the osinfo golden is the known failure the item above resolves); once green, move windows/amd64 out of thebuild-unsupportedmatrix into the supported set and drop the "unsupported" labelling for that arch.
Requires: none.
M29.2 - macOS: promote to supported
Planned. The parallel case: the same "already ships unsupported, promote it" shape as M29.1, and simpler - macOS lacks even the module-path blocker Windows has (the POSIX exe-relative default resolves cleanly), and its separators / EOL / $HOME match Linux. A macos-latest CI test job running go test ./... verifies correctness (reusing the per-OS osinfo golden strategy from M29.1); once green, move darwin/amd64 + darwin/arm64 out of the build-unsupported matrix into the supported set and drop the "unsupported" labelling. Requires: none.
M30 - project governance, licensing, and contribution policy
Planned. A hard requirement for 1.0.0 stable (also listed under Requirements for 1.0.0 stable). The rules for how the project is run and how outside contributions are taken - organizational, not code. Untouched while the project is solo (one author, Copyright (C) 2026 mplx <jennifer@mplx.dev>, LGPL-3.0-only, no outside PRs), but it must be settled before the first external contribution is merged: several of the choices are hard to reverse once other people's copyrightable work is in the tree. The open questions, roughly by urgency:
- Copyright-holder model. Under distributed copyright (the default, no paperwork) every non-trivial contributor automatically holds copyright in their patch, so the tree becomes a mosaic of holders and any future relicensing needs each one's agreement. The alternatives are a CLA (contributor grants the project a broad license, keeps their own copyright) or an assignment / CAA (contributor transfers copyright to a single holder) - both consolidate the rights but add contributor friction, and assignment needs an entity to hold them. This is the decision that is expensive to undo.
- The copyright notice. Whether headers stay per-author (
(C) <name>) or move to a collective label ((C) The Jennifer Authors, defined by git history). The trap to avoid: a two-fileAUTHORS(holders) /CONTRIBUTORS(credit) split only carries information when a work-for-hire contributor exists (employer holds copyright, individual is merely credited); for an all-volunteer project the two lists are identical, so the split is pointless. Either keep no enumerated holder file (the collective label refers to git history) or consolidate ownership via CLA / assignment. - Relicensing headroom. LGPL already lets anyone embed / link Jennifer without permission, so ordinary use never needs a contributor's sign-off. The only thing distributed copyright forecloses is issuing a different license - e.g. a commercial embedding exception for a deep-embedded
jennifer-tinytarget that cannot meet LGPL's static-relink terms. If keeping that option open matters (embedding is a first-class goal), a CLA is the tool; if "LGPL-only forever" is acceptable, distributed copyright is fine and the constraint never bites. - Contribution mechanics.
CONTRIBUTING.md, the sign-off mechanism (a lightweight DCOSigned-off-byline, which asserts "I have the right to submit this" without a license grant, vs a full CLA-bot, which also grants one - the choice follows from the relicensing decision above), a code of conduct, and the PR / review workflow. - Project governance. Who decides (BDFL vs a maintainer group), how commit rights are granted (judgment and sustained involvement, never an LOC or commit-count threshold - metrics are a bad proxy and get gamed), and a
MAINTAINERSfile once more than one decision-maker exists. Being listed as a contributor confers no authority; credit and governance are separate. - Name / mark. Whether the "Jennifer" /
jennifer-langidentity needs any trademark-style usage policy (forks, the deck registry) or stays informal.
The license itself stays LGPL-3.0-only unless a deliberate relicensing decision above changes it; this milestone is about the process and ownership around it, not a license change. Not legal advice - the chosen model should get a real legal review before it is published.
Requires: none (organizational, independent of the codebase). Socially paired with the M19.8 org move and triggered by the first external contribution, but no code prerequisite - which is why it is the one 1.0.0 requirement that can be settled at any time before the first outside PR.
Requirements for 1.0.0 stable
- Project governance, licensing, and contribution policy - the
M30work: the copyright-holder model, copyright notice, relicensing headroom,CONTRIBUTING.md/ sign-off, governance, and name / mark. A hard requirement for 1.0.0, and it must be settled before the first external contribution is merged (whichever comes first). - Cross-build for macOS / Windows. The
M29multiplatform track (M29.1Windows,M29.2macOS) does this; ships as soon as it lands. - Real apt repository (replacing the "GitHub Release artifact" install of the M15.8
.deb) if user demand warrants the maintenance. - Container image (OCI). Done -
.github/workflows/docker.ymlbuilds and pushes multi-arch (linux/amd64+linux/arm64)ghcr.io/<owner>/jenniferimages on each release tag: a Debian-slim default (:latest/:<ver>, full host features) and a distroless:staticvariant. The image bundles both binaries and the system modules under the compile-default module dir, so a bareimport "name.j";resolves with no env var. Seepackaging/docker/. - Homebrew tap (macOS, best-effort unsupported). Available -
packaging/homebrew/jennifer.rbbuilds the standardjenniferfrom source (so Intel + Apple Silicon work with no Gatekeeper friction) and bakes the version and module path via-ldflags -X, so a bareimport "name.j";resolves with no env var. Published to thejennifer-language/taptap by a manualpublish.sh(the AUR convention; no CI secret). The-UNSUPPORTEDmacOS tarball stays alongside it.
The extra Linux / macOS distribution formats (Homebrew, Snap, Nix, Flatpak, AppImage, ...) are not requirements; they live in the horizon idea collection and ship when there's user demand and a maintainer willing to keep one green.
Long horizon
Ideas for development beyond 1.0.0 - embedding, a WASM runtime, specialised-domain libraries, and a grab-bag of smaller possibilities - live in their own collection, kept out of the near-term plan so this file stays focused on the road to 1.0.0. See the beyond-1.0.0 idea collection.