# Polars facts Each fact records what the engine does at a pinned version, what the reference semantics does, and the checks behind it, from [`characterization/`](https://github.com/AlexanderNenninger/substrait-lean/tree/main/characterization) ([the format](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/fact.schema.json)). Evidence runs from `docs` to [source](../reference/io.Sink.md#sverify.io.Sink.source), `engine_test`, `probe`, `fuzz` and `proof`. # integers ## polars-int-abs: `abs` of an integer: `abs(MIN)` is `MIN` Engine: polars py-2.1.0; area: integers; status: open; evidence: probe. Verdict: wraps at `MIN`; the model's `abs` takes `overflow: ERROR` only `abs` is `wrapping_abs`: `abs(MIN)` is `MIN` at every width. Every other value is exact. The docs are silent on `abs(MIN)`: a docs gap. The reference: `abs` with `overflow: ERROR`, the only value the model accepts for it (`SILENT` and `SATURATE` are refused), so `abs(MIN)` is an overflow issue. Options: `overflow`. Rows of the choice tables: `int.abs` `overflow` (i8, i16, i32, i64). Constructs: `abs`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Int8.abs()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int16.abs()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int32.abs()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int64.abs()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `Int8.abs()` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `Int16.abs()` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `Int32.abs()` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `Int64.abs()` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, docs, by characterization pipeline reverse pass - silent on abs(MIN) - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_abs_overflow_wrapping`: Int8 abs(-128) = -128 (eager, in memory) ## polars-int-arith: Integer `+ - *`, unary `-` and `**`: wrap at the column's width Engine: polars py-2.1.0; area: integers; status: open; evidence: probe. Verdict: matches `overflow: SILENT` at every width Integer arithmetic wraps at the operands' width (two's complement, Rust's `wrapping_add`, `wrapping_sub`, `wrapping_mul`, `wrapping_neg`): `Int8` 127 + 1 is -128, `-MIN` is `MIN`. No setting makes it raise or saturate. Polars keeps the width of the operands; a literal takes the column's type. `**` on integers wraps too (`2 ** 63` is `-2⁶³` in `Int64`); a negative integer exponent raises (`InvalidOperationError`, the exponent is cast to `u32`). The docs are silent on integer overflow: a docs gap. The reference: `add`, `subtract`, `multiply`, `negate` with `overflow: ERROR` when the plan omits it (#202); `SILENT` when the plan asks. Polars gives `SILENT` natively; `ERROR` needs a guard, at each width. Options: `overflow`. Rows of the choice tables: `int.arith` `overflow` (i8, i16, i32, i64); `int.pow` `overflow` (i64); `int.pow` `overflow` (i8, i16, i32; evidence source). Constructs: `+`, `-`, `*`, `unary -`, `**`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-11](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-11); issues [\#202](https://github.com/AlexanderNenninger/substrait-lean/issues/202). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Int8 + Int8` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int8 - Int8` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int8 * Int8` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int16 + Int16` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int16 - Int16` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int16 * Int16` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int32 + Int32` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int32 - Int32` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int32 * Int32` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int64 + Int64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int64 - Int64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int64 * Int64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `-Int8` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `-Int16` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `-Int32` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `-Int64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `int-pow` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `int-pow-negative` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, source-reading, by characterization pipeline round 2 (integers): Integer `**` with an integer exponent is one generic function, `pow_to_uint_dtype`, instantiated for each width by `with_match_physical_integer_type!`. An exponent literal from 2 to 10 multiplies the column by itself with the `*` kernel, which wraps (`wrapping_mul`, the same macro for i8 to i64); any other exponent calls num-traits's `Pow`, the primitive `i8::pow` to `i64::pow`. The wheels build with the `dist-release` profile, which inherits `release` and leaves `overflow-checks` off, so the primitive `pow` wraps too. So `Int8`, `Int16` and `Int32` wrap as the `Int64` probe does, and the result keeps the base's type (`pow_dtype`). Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - `pow_to_uint_dtype`: an exponent literal of 2 to 10 multiplies the column by itself, any other calls `Pow::pow`. - `pow_on_series`: an integer base with an integer exponent, for every width by `with_match_physical_integer_type!`. - num-traits 0.2.19 (Polars's Cargo.lock): `Pow` for i8, i16, i32, i64 is the primitive `pow`. - Release profiles: none turns `overflow-checks` on. - Wheels build with `--profile dist-release`. - 2026-11-04, engine-test, by characterization pipeline round 3 (integers): The tests pin wrapping `+ - *` and negation at `Int8`. For `**` they pin that the result keeps the base's width at every signed width, but no input overflows, so the `int.pow` overflow rows for `i8` to `i32` stay at the source reading. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_list_arithmetic_values` (Int8): -5 - 127 is 124 and -5 \* 127 is -123 (wrapped), on columns, broadcasts and lists, eager and lazy - `test_neg_overflow_wrapping`: `-(-128)` in `Int8` is -128 (eager `DataFrame.select`, in memory) - `test_power_series`: `Int8 ** UInt8`, `Int16 ** UInt8`, `Int32 ** UInt8` and `Int64 ** UInt8` keep the base's type (1, 4); `Int64 ** UInt64(2**33)` raises `conversion from u64 to u32 failed` (eager) - 2026-11-04, docs, by characterization pipeline reverse pass - add/sub/mul/neg docstrings silent on integer overflow - Expr.pow: an integer to a positive integer is an integer; cast to float for a negative exponent ## polars-int-div-domain: Integer `//` has no domain error Engine: polars py-2.1.0; area: integers; status: no-finding; evidence: engine_test. Verdict: every input is in the domain; zero divisors and `MIN // -1` are other options Integer floor division gives a quotient for every pair of operands but two kinds: a zero divisor is NULL (`on_division_by_zero`) and `MIN // -1` wraps to `MIN` (`overflow`), both in polars-int-divide. No input raises or gives a value of its own. The kernels are one macro for `Int8` to `Int64`. `/` on integers divides doubles (polars-int-true-divide). The reference: `divide` on integers: `on_domain_error` never applies (`int.div` in docs/core-semantics.md, `Choice.any`). Options: `on_domain_error`. Rows of the choice tables: `int.div` `on_domain_error` (i8); `int.div` `on_domain_error` (i16, i32, i64; evidence source). Constructs: `//`, `floordiv`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, source-reading, by characterization pipeline round 2 (integers): One macro instantiates the integer kernels for i8 to i64, so the reading covers every width. Each path handles a zero divisor and `MIN // -1` and returns a quotient for every other input. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - Column by column: a zero divisor is masked to NULL, the rest is `wrapping_floor_div_mod`. - A literal divisor: 0 gives NULL, -1 negates (wrapping), others divide the absolute values. - - - 2026-11-04, engine-test, by characterization pipeline round 3 (integers): The Rust test shows the floor-division kernel gives a value for every `Int8` pair (the zero divisor's (0, 0) is masked to NULL by the caller), so no `Int8` input raises. `test_floordiv_mod` pins NULL for a zero divisor at `Int8` and `Int32`. `i16` and `i64` rest on the source reading of the shared macro. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - Rust `test_signed_wrapping_div_mod`: for every `i8` pair, `wrapping_floor_div_mod` equals the floor quotient and remainder of the doubles, wrapped to `i8` (`(x as i32) as i8`), and (0, 0) for a zero divisor. So `-128 // -1` is -128, `-128 % -1` is 0, and no pair panics. - `test_floordiv_mod`: `lit(1, Int8) // lit(0, Int8)` and `%` are NULL, as for the default `Int32` literals; 10 // 3 is 3. Eager `pl.select`, in memory - `test_truncdiv_int` (Int8, lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect)): `truncdiv` gives -128 for -128 / -1 and NULL for 5 / 0; its comment says `floordiv` does the same ## polars-int-divide: Integer `//` floors, `truncdiv` truncates; NULL for a zero divisor Engine: polars py-2.1.0; area: integers; status: open; evidence: probe. Verdict: `//` floors, `truncdiv` truncates; `x // 0` is NULL; `MIN // -1` wraps `//` on integers is floor division (`wrapping_floor_div_mod`): `-7 // 2` is -4, `1 // -2` is -1. A zero divisor gives NULL (the result's validity masks it), and `MIN // -1` wraps to `MIN`. `Expr.truncdiv` divides rounding toward zero (`Operator::TruncDivide`, the kernel `prim_wrapping_trunc_div`), on both engines: `-7 truncdiv 2` is -3. It shares the rest with `//`: a zero divisor gives NULL and `MIN truncdiv -1` wraps to `MIN`. The result type is that of `//`. So sverify could translate Substrait's integer `divide` to `truncdiv` in place of its floor-division shim (`int_div_trunc` in `_polars_shims.py`), keeping the guards for a zero divisor and `MIN / -1`. `/` on integers is a division of doubles (polars-int-true-divide). The docs are silent on a zero divisor giving NULL and on `MIN // -1` wrapping: a docs gap. The reference: `divide` on integers truncates toward zero, with `on_division_by_zero: ERROR` when the plan omits it (`NULL` when asked) and `overflow: ERROR` at `MIN / -1`. Polars's `//` differs on every quotient of operands of opposite signs that is not exact (26 of the 121 rows of the `NULL` candidate). Options: `division_type`, `on_division_by_zero`, `overflow`. Rows of the choice tables: `int.div` `overflow` (i8, i16, i32, i64); `int.div` `on_division_by_zero` (i8, i16, i32, i64); `int.div` `division_type` (i8, i16, i32, i64). Constructs: `//`, `floordiv`, `truncdiv`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Int8 // Int8` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int16 // Int16` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int32 // Int32` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int64 // Int64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, engine-test, by characterization pipeline round 3 (integers): Contradicts the description's second paragraph: Polars 2.1.0 has a truncating integer division in Python, `Expr.truncdiv` (`Operator::TruncDivide`), on both engines. Its tests pin truncation toward zero, `MIN truncdiv -1` wrapping to `MIN` and NULL for a zero divisor, which matches Substrait's `divide` on integers except for the overflow and division-by-zero options. The description is left as it is for review. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_truncdiv_int`: `Int8` `a.truncdiv(b)` of 7, -7, 7, -7, -128, -128, 5, NULL by 2, 2, -2, -2, 2, -1, 0, 1 is 3, -3, -3, 3, -64, -128, NULL, NULL (lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect)) - `test_truncdiv_signed_min`: `Int64` MIN truncdiv 2 and 3 truncate toward zero - `Expr.truncdiv`: divides, rounding toward zero; engine support in-memory, streaming, distributed; same result type as `floordiv` - `test_floordiv_mod`: `//` and `%` of `Int8` and `Int32` literals, a zero divisor gives NULL (eager, in memory) - 2026-11-04, manual-repro, by characterization pipeline round 4 (streaming probes): Corrected the description, title, verdict, constructs and settings. Round 1 said the Python API has no truncating integer division; Polars 2.1.0 has `Expr.truncdiv` (`Operator::TruncDivide`), documented for in-memory, streaming and distributed, and its tests (`test_truncdiv_int`, `test_truncdiv_signed_min`, round 3) pin truncation toward zero, `MIN truncdiv -1` wrapping to `MIN` and NULL for a zero divisor. The repro above confirms both engines, with a column and a literal divisor. The rows are unchanged: `//` and `truncdiv` share the overflow and division-by-zero behaviour. Polars 2.1.0 (Python), macOS arm64. - `pl.LazyFrame({'a': [7, -7, 7, -7, -128, 5, None], 'b': [2, 2, -2, -2, -1, 0, 1]}, schema={'a': pl.Int8, 'b': pl.Int8}).select(pl.col('a').truncdiv(pl.col('b')), pl.col('a').truncdiv(2).alias('lit')).collect(engine=e).rows() for e in ('in-memory', 'streaming')` gives `both engines: [(3, 3), (-3, -3), (-3, 3), (3, -3), (-128, -64), (None, 2), (None, None)]` - `test_truncdiv_int` and `test_truncdiv_signed_min` - `Expr.truncdiv` - 2026-11-04, source-reading, by characterization pipeline reverse pass - FloorDivide/TruncDivide dispatch to floor_div_series/trunc_div_series - - wrapping_div then -1 on a nonzero remainder of the other sign - 2026-11-04, docs, by characterization pipeline reverse pass - states floordiv rounds down by contrast; silent on a zero divisor (NULL) and MIN // -1 - 2026-11-04, engine-test, by characterization pipeline reverse pass - \#\[test\] test_signed_wrapping_div_mod: all i8 pairs, floored and wrapped (-128 // -1 = -128) - Int8 -5 // 3 = -2 (eager and lazy, both engines; slow) - Int32 // 5 matches numpy's floor (eager) ## polars-int-mod-overflow: Integer `%` never overflows Engine: polars py-2.1.0; area: integers; status: no-finding; evidence: engine_test. Verdict: no overflow at any width; `MIN % -1` is 0 `%` on integers has no overflowing input. The kernel takes `wrapping_rem` and, when the remainder is nonzero and its sign differs from the divisor's, adds the divisor: the sum lies strictly between 0 and the divisor, so it fits. `MIN % -1` is 0 (`wrapping_rem`), and a literal divisor of ±1 fills 0. One macro instantiates the kernels for `Int8` to `Int64`, so every width behaves the same. The floor remainder and NULL for a zero divisor are in polars-int-modulus. The reference: `modulus` on integers is the truncated remainder (`Int.tmod`), which never overflows either; `MIN % -1` is 0 (`min_mod_neg_one`). Options: `overflow`. Rows of the choice tables: `int.mod` `overflow` (i8); `int.mod` `overflow` (i16, i32, i64; evidence source). Constructs: `%`, `mod`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, source-reading, by characterization pipeline round 2 (integers): The kernels are one generic macro, so the reading covers i8 to i64. The adjustment `mod_ += other` only runs when the remainder and the divisor have opposite signs, where the sum cannot overflow. The wheels build without `overflow-checks` (polars-int-arith), so the wrapping operations cannot panic either. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - - - - A literal divisor: the remainder of the absolute values, below \|divisor\|, with the divisor's sign. - `impl_signed_arith_kernel!` for i8, i16, i32, i64, i128. - 2026-11-04, engine-test, by characterization pipeline round 3 (integers): The Rust unit test runs `wrapping_floor_div_mod`, the kernel of `%` and `//` between two integer columns, on all 65 536 `Int8` pairs: `MIN % -1` is 0 and every remainder fits. It is a test of the kernel, below either engine; both engines call it. It covers `i8` only (its comment says the result should transfer to other widths), so `i16` to `i64` stay at the source reading of the macro. The path of a literal divisor (`prim_wrapping_mod_scalar`, strength reduction) is not covered by it. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - Rust `test_signed_wrapping_div_mod`: for every `i8` pair, `wrapping_floor_div_mod` equals the floor quotient and remainder of the doubles, wrapped to `i8` (`(x as i32) as i8`), and (0, 0) for a zero divisor. So `-128 // -1` is -128, `-128 % -1` is 0, and no pair panics. - `test_list_arithmetic_values` (Int8): -5 // 3 is -2, -5 % 3 is 1, through the column, broadcast and list paths ## polars-int-modulus: Integer `%`: the floor remainder, NULL for a zero divisor Engine: polars py-2.1.0; area: integers; status: open; evidence: probe. Verdict: `division_type: FLOOR`; `x % 0` is NULL; `MIN % -1` is 0 `%` on integers is the remainder of floor division, with the sign of the divisor (`-1 % 2` is 1, `1 % -7` is -6). A zero divisor gives NULL, and `MIN % -1` is 0. The docs are silent on the remainder's sign, a zero divisor and `MIN % -1`: a docs gap. The reference: `modulus` with `division_type: TRUNCATE` (the remainder has the dividend's sign); `FLOOR` is refused by the model. `on_division_by_zero: ERROR` when omitted. So the reference matches Polars only where the operands have the same sign or the remainder is 0. Options: `division_type`, `on_division_by_zero`, `min_mod_neg_one`. Rows of the choice tables: `int.mod` `on_division_by_zero` (i8, i16, i32, i64); `int.mod` `division_type` (i8, i16, i32, i64); `int.mod` `min_mod_neg_one` (i8, i16, i32, i64). Constructs: `%`, `mod`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Int8 % Int8` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int16 % Int16` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int32 % Int32` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int64 % Int64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int8.mod(Int8)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int16.mod(Int16)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int32.mod(Int32)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Int64.mod(Int64)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, source-reading, by characterization pipeline round 2 (integers): Confirms the probes for every width: one macro (`impl_signed_arith_kernel!`) instantiates the kernels for i8, i16, i32, i64 and i128. The remainder is `wrapping_rem` moved into the divisor's sign class, so it is a floor remainder; `MIN % -1` is 0 because `wrapping_rem` gives 0 there (and a literal `-1` divisor fills 0). Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - `wrapping_floor_div_mod`: `wrapping_div` and `wrapping_rem`, then the divisor is added to a nonzero remainder whose sign differs from it. - - Column by column: a zero divisor is masked to NULL. - A literal divisor: 0 gives an all-NULL column, ±1 fills 0, any other a strength-reduced unsigned remainder with the divisor's sign. - - 2026-11-04, docs, by characterization pipeline reverse pass - silent on the remainder's sign, x % 0 (NULL) and MIN % -1 - 2026-11-04, engine-test, by characterization pipeline reverse pass - \#\[test\] test_signed_wrapping_div_mod: all i8 pairs, remainder with the divisor's sign, -128 % -1 = 0 - Int8 -5 % 3 = 1 (eager and lazy, both engines; slow) - test_floordiv_mod: 1 % 0 is NULL (eager, in memory) ## polars-int-pow-exponent: Integer `**`: a negative exponent fails the query Engine: polars py-2.1.0; area: integers; status: no-finding; evidence: probe. Verdict: raises `InvalidOperationError` for a negative exponent With an integer base and an integer exponent, `**` first casts the whole exponent column to `UInt32` with `strict_cast`. A negative exponent anywhere in it, or one above 2³² − 1, fails the query with `InvalidOperationError` and a hint to cast to float first. The cast does not depend on the base's width. Every other input gives a value (`0 ** 0` is 1); overflow wraps (polars-int-arith). A float exponent turns the base into the exponent's float type, and the result is that type. The reference: `power` on integers: a negative exponent raises (`negExponent` in `Safety.lean`), and `on_domain_error` never applies (`int.pow` in docs/core-semantics.md). An exponent above 64 with `|base| > 1` is an overflow, which raises; with `|base| ≤ 1` the reference computes it, where Polars raises from 2³² on. Options: `on_domain_error`. Rows of the choice tables: `int.pow` `on_domain_error` (i64); `int.pow` `on_domain_error` (i8, i16, i32; evidence source). Constructs: `**`, `pow`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `Int64` base, `** -1`: `InvalidOperationError: conversion from i32 to u32 failed`. - probe `int-pow-negative` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by characterization pipeline round 2 (integers): The exponent's cast runs before the base's type is looked at (`with_match_physical_integer_type!` dispatches the base, for every width), so the `Int64` probe holds for `Int8`, `Int16` and `Int32`. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - An integer exponent is `strict_cast` to `UInt32`, with the hint in the error. - - `pow_dtype`: an integer base keeps its type unless the exponent is a float. - 2026-11-04, engine-test, by characterization pipeline round 3 (integers): The test pins the strict cast of the exponent to `UInt32` for an `Int64` base only, and has no negative exponent. The `i8` to `i32` rows stay at the source reading. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_power_series`: `Int64 ** UInt64` with 2\*\*33 raises `InvalidOperationError` `conversion from u64 to u32 failed`; an exponent that fits succeeds; `a ** lit(None)` raises (null exponent dtype). Eager `Series` ## polars-int-shift: No bit-shift expressions Engine: polars py-2.1.0; area: integers; status: open; evidence: probe. Verdict: no `<<` or `>>` on expressions Polars's Python expressions have no bit shift: `pl.Expr` has no `__lshift__` or `__rshift__`, and its `bitwise_*` methods are `and`, `or`, `xor` and bit counts. `shift` moves rows. The reference: `shift_left` and `shift_right` (`functions_arithmetic.yaml`), with the range open (ST-1). A plan with them is refused on Polars or written as a multiplication or floor division by a power of two. Options: `shift_range`. Rows of the choice tables: `int.shift` `shift_range` (i8, i16, i32, i64; evidence docs). Constructs: `shift_left`, `shift_right`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-1](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-1). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `int-shift` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, docs, by characterization pipeline round 1 (integers): The API reference's expression operators list `&`, `|`, `^` and `~` as the bitwise operators, with no shift; the `bitwise_*` methods are `and`, `or`, `xor` and bit counts, and `Expr.shift` moves rows. The same holds for every integer type, so the shift rows are settled: Polars has no shift to characterize. - Expression operators (API reference, version 2): bitwise `&`, `|`, `^`, `~` and no shift. - `Expr`'s operator dunders at tag py-2.1.0: `__and__`, `__invert__`, `__or__`, `__xor__`, no `__lshift__` or `__rshift__`. - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observation reads the Python API (`dir(pl.Expr)`), which both engines share. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `int-shift` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass - Operator: no shift; polars-sql has none either ## polars-int-true-divide: Integer `/`: a division of doubles Engine: polars py-2.1.0; area: integers; status: open; evidence: probe. Verdict: `Int64 / Int64` is `Float64`; `/ 0` is ±inf or NaN; a literal divisor multiplies by its reciprocal `/` on integers casts both operands to `Float64` and divides: `7 / 2` is 3.5, `x / 0` is ±inf, `0 / 0` is NaN. Integers above 2⁵³ round on the way. The cast happens before the division (binary.rs#L151-L158), so the float kernels run, and a literal or one-row divisor multiplies by its reciprocal (polars-double-scalar-divisor, PL-3): on a frame of two rows or more `49 / 49` is 0.9999999999999999 and `5 / 3` is 1.6666666666666665, and so is `x / x.max()` for `x = 49`. The docs are silent on the rounding above 2⁵³: a docs gap. The `truediv` docs call the division IEEE 754, which the reciprocal contradicts: PL-8. No upstream test pins quotients above 2⁵³ or an inexact quotient by a literal; they may change between versions. The reference: Substrait's `divide` on integers returns the integer type and truncates. A plan that divides integers to a double casts first, and the double division's options apply (polars-double-divide). Options: `implicit_casts`, `on_division_by_zero`. Constructs: `/`, `truediv`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [PL-8](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-8). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `int-true-divide` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `int-true-divide` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass: Corrected the route: an earlier check cited `prim_true_div` of signed.rs, which `Expr /` does not reach. binary.rs#L151-L158 casts both sides to Float64 first, so a literal divisor takes the float reciprocal kernel; added to the description and the verdict. - integer / integer: both cast to Float64, then the float kernel; signed.rs's prim_true_div is not on this route - a length-1 divisor multiplies by its reciprocal - 2026-11-04, manual-repro, by characterization pipeline reverse pass - `pl.DataFrame({'a': [49, 5, 0]}).select(pl.col('a') / 49, pl.col('a') / 3, pl.col('a') / 0)` gives `49/49 = 0.9999999999999999, 5/3 = 1.6666666666666665, 49/0 = inf, 0/0 = nan (Polars 2.1.0, macOS arm64)` - 2026-11-04, engine-test, by characterization pipeline reverse pass - UInt8 1 / 0 = inf, 0 / 0 = NaN (eager and lazy, both engines; slow) - \[1, 2\] / 2 = \[0.5, 1.0\] (eager) # decimals ## polars-decimal-arith: Decimals: 38 digits, the larger scale, half to even Engine: polars py-2.1.0; area: decimals; status: shimmed; evidence: probe. Verdict: `Decimal(38, max(s₁, s₂))`; rounds half to even; raises on overflow and on `/ 0` Arithmetic on decimals returns `Decimal(38, max(s₁, s₂))` for `+`, `-`, `*` and `/` (an integer operand counts as scale 0), and `sum` returns `Decimal(38, s)`: the result's precision is always 38, so `Decimal(6, 2) * 1000` gives 9999990.00 and overflows only past 38 digits, where it raises (`ComputeError`). A result with more digits than the scale rounds half to even: `0.05 * 0.5` is 0.02, `1.00 / 8` is 0.12, a cast from scale 3 to scale 1 makes 1.050 into 1.0 and 1.150 into 1.2, and a double cast to a decimal rounds half to even on its exact value (0.625 is 0.62). Division by zero raises. `mean` of decimals is a `Float64`. The docs are silent on the result's precision and scale and on the half-to-even rounding: a docs gap. The reference: `functions_arithmetic_decimal.yaml` gives the result types (multiply: scale s₁ + s₂, divide: max(6, s₁ + p₂ + 1), precision capped at 38 by borrowing scale) and no rounding; the reference rounds half away from zero (`decimal_rescale`). Decimal division is refused until \#27. ST-8 records the open rounding. The translation computes a product on the unscaled values at scale s₁ + s₂ and checks each result against the plan's precision (`fit_decimal`); `AVG` and `sverify_decimal_div` divide half away from zero (`decimal_div_half_away`, `Polars.decimal_div_half_away`); an empty `SUM` is NULL (`sum_empty_null`). Options: `overflow`, `decimal_rescale`, `decimal_result_type`, `raise_set`. Rows of the choice tables: `dec.arith` `overflow` (decimal); `dec.arith` `decimal_rescale` (decimal). Constructs: `+`, `-`, `*`, `/`, `sum`, `mean`, `cast`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-8](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-8); issues [\#27](https://github.com/AlexanderNenninger/substrait-lean/issues/27). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `decimal-result-types` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `decimal-rounding` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `decimal-overflow` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `decimal-precision-38` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `decimal-divide-by-zero` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `decimal-mean` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - - - - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `decimal-result-types` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `decimal-rounding` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `decimal-overflow` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `decimal-precision-38` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `decimal-divide-by-zero` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `decimal-mean` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass - streaming decimal sum: i128 checked_add with a sticky overflow sentinel; finish() raises 'overflow in decimal addition in sum' past 38 digits. - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_decimal_arithmetic`: Decimal(20, 10) + - \* / all give Decimal(38, 10); 0.1 / 20.1 == 0.0049751244, 10.1 \* 10.19 == 102.919; eager, in memory. - `test_decimal_mul_rounds_half_even`: \[0.25, 0.35, -0.25, -0.35\] \* 0.5 == \[0.12, 0.18, -0.12, -0.18\] as Decimal(38, 2), eager. - `test_decimal_arithmetic_schema_int`: an integer operand counts as scale 0, result Decimal(38, 6); s / 3 == 0.333333. - sum to Decimal(38, s), overflow past 38 digits raises ComputeError, partial sums past 38 digits allowed; parametrized engine in \['streaming', 'in-memory'\]. - `test_decimal_div_by_zero_mixed_scale`: 1.5 / 0.000 raises ComputeError 'division by zero'. - 2026-11-04, docs, by characterization pipeline reverse pass - Precision 38 is the default and the maximum (SRC py-polars/src/polars/datatypes/classes.py L447-L459); silent on result precision and scale of arithmetic and on its rounding. - 2026-11-05, source-reading, by session polars-decimals: Read at tag py-2.1.0 (commit 7f902bd3). `tests/test_polars_decimal.py` runs each decimal operation's written plan on Polars and DuckDB against `Rel.eval`. - dec128_mul_scaled: the product of the unscaled values on 256 bits (i64 × i64 on 128), then rounded to the result's scale half to even: no intermediate overflow. - dec128_fits: the precision check against 10^p. ## polars-decimal-casts: Decimal casts: half to even, correctly rounded to double, truncated to integer Engine: polars py-2.1.0; area: decimals; status: shimmed; evidence: probe. Verdict: a smaller scale and a double round half to even; to a double correctly rounded; to an integer truncates; past the precision raises A decimal cast to a smaller scale rounds half to even (1.050 at scale 1 is 1.0, 1.150 is 1.2). A double cast to `Decimal(p, s)` is `x · 10ˢ` in doubles, rounded half to even (0.625 is 0.62, 0.125 is 0.12, -0.375 is -0.38), with the double `10ˢ` exact only up to `s = 22`; NaN and ±∞ raise. A decimal cast to `Float64` is correctly rounded: one division where the unscaled value and `10ˢ` are exact doubles, else the decimal's text through Rust's float parser. A decimal cast to an integer truncates toward zero, and raises outside the integer's range (strict) or is NULL (`strict=False`). A cast to a smaller precision raises where the value does not fit (strict) or is NULL. `to_physical` gives the unscaled value as an `Int128` (1.25 at scale 2 is 125), and `Int128` arithmetic wraps silently (2¹²⁶ · 2 is -2¹²⁷). The reference: `decimal_rescale: TIE_AWAY_FROM_ZERO` (`roundDiv10`, `UnFn.dround`); a double to a decimal is `x · 10ˢ` rounded once to a double, then half away from zero (`FloatCode.toDecimal?`); a decimal to a double is correctly rounded (`FloatCode.ofDecimal`); to an integer truncates (`UnFn.dtrunc`). The translation shims the halves (`decimal_round_half_away`, `round_half_away` on the double product) and refuses a double cast to more than 22 decimal places. Options: `decimal_rescale`, [float_to_int](../reference/sql.Dialect.md#sverify.sql.Dialect.float_to_int), `cast_failure`. Rows of the choice tables: `dec.arith` `decimal_rescale` (decimal). Constructs: `cast`, `to_physical`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: issues [\#27](https://github.com/AlexanderNenninger/substrait-lean/issues/27). Provenance: - 2026-11-05, probe, by sverify_dev.probe_polars (session polars-decimals): Polars 2.1.0 (Python), macOS arm64. `decimal-to-double` takes values whose unscaled value, as a double, divided by 10²⁰ rounds twice; Polars gives the nearest double for each. - probe `decimal-rounding` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `decimal-from-double` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `decimal-to-double` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-decimal-to-int` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `decimal-narrow-precision` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `decimal-unscaled` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `int128-wrap` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-05, probe, by sverify_dev.probe_polars -diff (session polars-decimals): Streaming gives what in memory gives on each observation. - probe `decimal-rounding` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `decimal-from-double` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `decimal-to-double` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `decimal-narrow-precision` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `decimal-unscaled` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `int128-wrap` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-05, source-reading, by session polars-decimals: Read at tag py-2.1.0 (commit 7f902bd3). - dec128_to_f64: x as f64 / 10^s where both are exact (\|x\| ≤ 2^53, s ≤ 22), else parse_dec128, the decimal's text through Rust's float parser. - f64_to_dec128: (x \* POW10_F64\[s\]).round_ties_even(); 'TODO: correctly rounded result. This rounds multiple times.' - dec128_rescale: div_128_pow10_round to a smaller scale (half to even), checked multiplication to a larger one, then the precision check. - dec128_to_i128: div_128_pow10_trunc, truncation toward zero. ## polars-decimal-compare: Decimal comparisons: exact across scales and with integers Engine: polars py-2.1.0; area: decimals; status: no-finding; evidence: probe. Verdict: exact `==` and `<` compare decimals of different scales, and a decimal with an integer, by value: 12.35 at scale 2 equals 12.350 at scale 3, and -0.05 \< -0.051 is false. A rescale that would overflow decides by the sign instead. `sum` of no decimals, or only NULLs, is 0 at the column's scale (as for integers, polars-sum-int); `min` of only NULLs is NULL. The reference: Comparisons at the larger scale, exactly (`docs/reference-semantics.md`, Decimals); `SUM` of no values is NULL (`empty_group_aggregates`), which `sum_empty_null` gives. Constructs: `==`, `<`, `>`, `min`, `max`, `sum`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: issues [\#27](https://github.com/AlexanderNenninger/substrait-lean/issues/27). Provenance: - 2026-11-05, probe, by sverify_dev.probe_polars (session polars-decimals): Polars 2.1.0 (Python), macOS arm64. - probe `decimal-compare` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `decimal-sum-empty` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-05, probe, by sverify_dev.probe_polars -diff (session polars-decimals): Streaming gives what in memory gives. - probe `decimal-compare` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `decimal-sum-empty` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-05, source-reading, by session polars-decimals: Read at tag py-2.1.0 (commit 7f902bd3). - dec128_eq and dec128_cmp rescale to the larger scale; an overflowing rescale is unequal, or ordered by its sign. # doubles ## polars-atan2: `arctan2(0, 0)` is 0 Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: 0, where the reference raises `pl.arctan2(y, x)` is Rust's `atan2`: `atan2(0, 0)` is 0 and the infinities give the IEEE angles. It is the only edge case that differs. The docs are silent on `atan2(0, 0)` and the infinities: a docs gap. No upstream test pins this; it may change between versions. The reference: `atan2` with `on_domain_error: ERROR`, which raises at `(0, 0)`; Substrait leaves its domain open (ST-2). Options: `on_domain_error`. Rows of the choice tables: `fp.elem` `on_domain_error` (fp64); `fp.elem` `on_domain_error` (fp32; evidence source). Constructs: `arctan2`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-2](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-2). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `pl.arctan2(Float64, Float64)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): `atan2` is generic over the float type: a `Float32` `y` computes `f32::atan2` and returns `Float32`, so `atan2(0, 0)` is 0 there too. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - `arctan2_on_floats`: `x` is cast to `y`'s float type, then `yv.atan2(xv)`. - - 2026-11-04, docs, by characterization pipeline reverse pass - silent on atan2(0, 0) and the infinities - the docstring at the tag - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_arctan2`: only (±√2/2, ±√2/2) → ±45°, ±135° (eager, in memory); no upstream test pins (0, 0) or ±inf ## polars-double-arith: Double `+ - *`: IEEE, NaN for `inf - inf` and `0 * inf` Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: IEEE: NaN where the reference raises Double arithmetic is IEEE 754: `inf - inf`, `inf + (-inf)` and `0 * inf` are NaN. Every other edge case agrees with the reference. An operation on finite values whose result overflows gives `±inf` without raising, in `Float64` and in `Float32`: `1e308 + 1e308` and `1e308 * 10` are inf, `-1e308 - 1e308` is -inf; `Float32` overflows at its own range (`3e38 + 3e38` is inf). Same on both engines. A literal `±0.0` addend or subtrahend returns the column unchanged, and `0.0 - x` is `-x` (the literal kernels' shortcuts, float.rs#L45-L65): `-0.0 + 0.0` is -0.0 and `0.0 - 0.0` is -0.0, where IEEE gives +0.0. Two columns give +0.0, and so does a one-row morsel of the streaming engine (polars-zero-sign-streaming). The NaN cases above are unaffected. No upstream test pins `inf - inf`, `inf + (-inf)`, `0 * inf` or the `±0.0` shortcuts; they may change between versions. The reference: `add`, `subtract`, `multiply` raise a domain issue for `inf - inf` and `0 · inf` (`nan_creation`: raise, \#257). Options: `nan_creation`, `float_overflow`. Rows of the choice tables: `fp.arith` `nan_creation` (fp64); `fp.arith` `nan_creation` (fp32; evidence source); `fp.arith` `float_overflow` (fp32, fp64). Constructs: `+`, `-`, `*`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-5](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-5), [ST-9](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-9); issues [\#257](https://github.com/AlexanderNenninger/substrait-lean/issues/257). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Float64 + Float64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64 - Float64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64 * Float64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, docs, by characterization pipeline round 1 (doubles): Polars's user guide says it generally follows IEEE 754 for `Float32` and `Float64`, with two exceptions: any NaN equals any other NaN and is greater than every non-NaN value, and operations guarantee nothing about the sign of zero or NaN or a NaN's payload (a sort or group by may make every zero +0 and every NaN a positive NaN). It promises no error bound unless a function says otherwise. The user guide is not versioned; the section reads the same at tag py-2.1.0. So `inf - inf` and `0 * inf` are NaN in `Float32` as in `Float64` (IEEE 754's invalid operation). Settles the `Float32` row at docs level; the probes cover `Float64`. - User guide, Floating point numbers (stable). - The same section at tag py-2.1.0. - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): `impl_float_arith_kernel!` is one macro instantiated for `pf16`, `f32` and `f64`, so `Float32` and `Float64` run the same operations. The streaming engine evaluates expressions through polars-expr on each morsel (`StreamExpr` in crates/polars-stream/src/expression.rs), so it shares these kernels with the in-memory engine. `+ - *` are the IEEE operations, so `inf - inf` and `0 * inf` are NaN in `Float32` as in `Float64`. The literal shortcuts (`x + 0`, `x * 1`) return `x`, which changes no NaN. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - - - - 2026-11-04, probe, by characterization pipeline session (float_overflow rows), polars 2.1.0 - lf = pl.LazyFrame({'a': \[big, -big\], 'b': \[big, big\], 'c': \[10.0, small\]}) with all three Float64 (big 1e308, small 1e-10) and Float32 (big 3e38, small 1e-3); select a + b, a - b, a \* c, a / c; pl.LazyFrame(…).collect(engine=e) for e in ('in-memory', 'streaming'), polars 2.1.0 → Float64: \[(inf, 0.0, inf, 1e+307), (0.0, -inf, -1.0000000000000001e+298, -inf)\]; Float32: \[(inf, 0.0, inf, 3.000000106909804e+37), (0.0, -inf, -3.0000001227554363e+35, -inf)\]; streaming the same - 2026-11-04, source-reading, by characterization pipeline reverse pass: Added to the description: the ±0 literal shortcuts (`-0.0 + 0.0` is -0.0), which polars-zero-sign-streaming recorded and this fact lacked. - a ±0 literal addend or subtrahend returns lhs unchanged, and 0 - x is neg(x): -0.0 + 0.0 and 0.0 - 0.0 give -0.0 (IEEE: +0.0) - 2026-11-04, manual-repro, by characterization pipeline reverse pass - `pl.DataFrame({'z': [-0.0, 0.0, -0.0]}).select(a=pl.col('z') + 0.0, b=pl.col('z') - (-0.0), c=0.0 - pl.col('z'), d=pl.col('z') + pl.lit(pl.Series([0.0, 0.0, 0.0])))` gives `signs a: -,+,-; b: -,+,-; c: +,-,+; d: +,+,+ (Polars 2.1.0, macOS arm64)` - 2026-11-04, engine-test, by characterization pipeline reverse pass - no upstream test pins inf - inf, inf + (-inf) or 0 \* inf (searched py-polars/tests/unit/operations/arithmetic, expr, series, dataframe at py-2.1.0) ## polars-double-divide: Double `/` between columns: IEEE Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: `on_division_by_zero: IEEE`, but `±inf / ±inf` is NaN `/` between two double columns is IEEE 754: `x / 0` is ±inf, `0 / 0` is NaN, `±inf / ±inf` is NaN. No setting changes it. With the `IEEE` candidate, only the four `±inf / ±inf` rows differ, as for DuckDB (duckdb-R15-double-arith). A length-1 divisor against a longer column (a literal on a frame of two rows or more) is a multiplication by its reciprocal. Zero and infinite divisors give the same results there, but a subnormal divisor's reciprocal is inf, so `0.0 / 1e-310` is NaN and `1e-300 / 1e-310` is inf (polars-double-scalar-divisor, PL-3). A quotient of finite values too large for the type is `±inf`: `-1e308 / 1e-10` is -inf in `Float64`, `-3e38 / 1e-3` is -inf in `Float32`, on both engines. On the streaming engine the kernel is chosen per morsel, so a morsel of one row divides exactly; which rows land in one depends on the thread count and `POLARS_IDEAL_MORSEL_SIZE` (polars-double-scalar-divisor). A division by an aggregate (`x / x.max()`) has a length-1 divisor too. The `truediv` docs and the user guide call float division IEEE 754, which the reciprocal kernel contradicts: PL-8. The reference: `divide` with `on_division_by_zero: ERROR` when omitted; `IEEE`, `NULL`, `LIMIT` when asked. `±inf / ±inf` is a domain issue (#257). Options: `on_division_by_zero`, `nan_creation`, `on_domain_error`, `float_overflow`. Rows of the choice tables: `fp.div` `on_division_by_zero` (fp64); `fp.div` `on_division_by_zero` (fp32; evidence engine_test); `fp.div` `nan_creation` (fp64); `fp.div` `nan_creation` (fp32; evidence engine_test); `fp.div` `on_domain_error` (fp64); `fp.div` `on_domain_error` (fp32; evidence source); `fp.div` `float_overflow` (fp32, fp64). Constructs: `/`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [PL-3](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-3), [PL-8](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-8); issues [\#204](https://github.com/AlexanderNenninger/substrait-lean/issues/204), [\#257](https://github.com/AlexanderNenninger/substrait-lean/issues/257). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Float64 / Float64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, docs, by characterization pipeline round 1 (doubles): `Expr.truediv`'s notes say zero division follows IEEE-754: `0/0` is an invalid operation and returns NaN, `n/0` on finite operands is ±infinity. The note is not tied to a float type, and the user guide says `Float32` and `Float64` both follow IEEE 754, so `Float32` division behaves the same (and `±inf / ±inf` is NaN, IEEE's invalid operation). Settles the `Float32` rows and `on_domain_error` (NaN for `0/0` and `±inf / ±inf`) at docs level; the probes cover `Float64`. - Expr.truediv, Notes (API reference, version 2). - - User guide, Floating point numbers (stable). - The same section at tag py-2.1.0. - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): `impl_float_arith_kernel!` is one macro instantiated for `pf16`, `f32` and `f64`, so `Float32` and `Float64` run the same operations. The streaming engine evaluates expressions through polars-expr on each morsel (`StreamExpr` in crates/polars-stream/src/expression.rs), so it shares these kernels with the in-memory engine. Column by column `/` is `l / r`, so `x / 0` is ±inf and `0 / 0` and `inf / inf` are NaN in `Float32`; with a literal divisor `x * (1 / c)` gives the same at `c = ±0` and `c = ±inf`. The literal path creates NaN where IEEE does not: `0.0 / c` for a subnormal `c` is `0 * inf`, NaN (polars-double-scalar-divisor). So the `nan_creation` row's "IEEE" holds column by column only, and its `Float32` entry stays at the docs. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - - - `+ - * // %` on two ChunkedArrays: equal lengths take the array kernel, a length-1 side the scalar kernel. - - A literal is a scalar column, passed to the operation as a one-value Series. - The streaming engine evaluates each expression with polars-expr's `PhysicalExpr` on every morsel, so it runs the same kernels. - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): Corrects the description: the IEEE claim now covers `/` between two columns, and a second paragraph gives the literal path's NaN and inf for a subnormal divisor, with links to polars-double-scalar-divisor and PL-3. The `nan_creation` rows hold column by column; the `Float32` entry stays at the docs. Read at tag py-2.1.0 (commit 7f902bd3). - - - - - 2026-11-04, engine-test, by characterization pipeline round 3 (doubles): Pins the `Float32` results of a zero divisor: `x / 0` is inf and `0 / 0` NaN. The test runs each case on one-row operands and, through `BROADCAST_SERIES_COMBINATIONS`, with the left side broadcast to three rows against a one-row right side, which takes the reciprocal-divisor kernel (PL-3); `EXEC_OP_COMBINATIONS` runs it on `Series` (in memory) and as a lazy [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select) with [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect). These inputs give the same results on the IEEE and the reciprocal path. Marked `slow`, which CI still runs. `±inf / ±inf` is not tested, so the `on_domain_error` row for `fp32` stays at the source. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_list_arithmetic_values`, `Float32`: 1 // 0 is inf, 1 % 0 NaN, 1 / 0 inf, 0 // 0 NaN, 0 % 0 NaN, 0 / 0 NaN - `test_list_arithmetic_values`, `Float32` (picked so nothing upcasts to `Float64`): 7 // 3 is 2, -5 % 3 is 1, 2 / 128 is 0.015625 - 2026-11-04, probe, by characterization pipeline session (float_overflow rows), polars 2.1.0 - pl.LazyFrame({'a': \[big, -big\], 'c': \[10.0, small\]}) with Float64 (big 1e308, small 1e-10) and Float32 (big 3e38, small 1e-3); select a / c; pl.LazyFrame(…).collect(engine=e) for e in ('in-memory', 'streaming'), polars 2.1.0 → Float64: 1e+307, -inf; Float32: 3.000000106909804e+37, -inf; streaming the same - 2026-11-04, source-reading, by characterization pipeline reverse pass: Added to the description: the streaming engine picks the kernel by morsel length, and `x / x.max()` takes the reciprocal; the docs contradiction is PL-8. - streaming: morsels of len.div_ceil(k \* num_pipelines) rows; a frame shorter than the thread count streams as one-row morsels, which take the equal-length IEEE kernel - equal lengths: array kernel; a length-1 side: scalar (reciprocal) kernel - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_floordiv_truediv`: DataFrame / scalar equals Python exactly (eager, in memory); DataFrame.\_div materializes the scalar to full height, so this pins the column kernel, not pl.col / literal - ## polars-double-modulus: Double `%`: `x - y·floor(x / y)` Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: floor remainder; NaN for `x % 0` and for finite `x % ±inf` `%` on doubles computes `x - y * floor(x / y)`: the remainder has the divisor's sign (`5.5 % -2` is -0.5), `x % 0` is NaN, and a finite `x % ±inf` is NaN too (`x / inf` is 0, and `inf * 0` is NaN), where C's `fmod` and Python's `%` give `x`. The formula also loses the exactness of `fmod` for large quotients. `//` on doubles is `floor(x / y)`. A zero remainder is always 0.0: `x - y * q` with equal terms is +0.0 under round to nearest, and for `x = -0.0` the product `y * q` is -0.0, so `-0.0 % y` is 0.0 too (`fmod` gives -0.0 for `-4 % 2` and `-0.0 % y`). The docs are silent on `%` of doubles (the floored remainder, `x % 0`, `x % ±inf`): a docs gap. No upstream test pins `x % ±inf`; it may change between versions. The reference: `modulus` on doubles (sverify's `sverify_fmod`): the truncated remainder (`fmod`), with a zero divisor an error as on integers, and `fmod(x, ±inf) = x`. 60 of the 144 rows differ. Options: `division_type`, `on_division_by_zero`, `nan_creation`, `zero_sign`. Rows of the choice tables: `fp.mod` `division_type` (fp64); `fp.mod` `on_division_by_zero` (fp64); `fp.mod` `nan_creation` (fp64); `fp.mod` `division_type` (fp32; evidence engine_test); `fp.mod` `on_division_by_zero` (fp32; evidence engine_test); `fp.mod` `nan_creation` (fp32; evidence engine_test); `fp.mod` `zero_sign` (fp32, fp64; evidence source). Constructs: `%`, `//`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [PL-1](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-1), [PL-3](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-3). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Float64 % Float64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `double-modulus-sign` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `double-floordiv` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): `impl_float_arith_kernel!` is one macro instantiated for `pf16`, `f32` and `f64`, so `Float32` and `Float64` run the same operations. The streaming engine evaluates expressions through polars-expr on each morsel (`StreamExpr` in crates/polars-stream/src/expression.rs), so it shares these kernels with the in-memory engine. So the floor remainder, NaN for a zero divisor and NaN for a finite `x % ±inf` hold for `Float32` as the `Float64` probes show. With a literal divisor on a frame of two rows or more, the kernel multiplies by the reciprocal: where `x * (1 / y)` rounds below an integer that `x / y` reaches, the floor is one less and the remainder is `y` itself (`49.0 % 49.0` is 49.0), and a subnormal divisor makes the reciprocal infinite. That path is polars-double-scalar-divisor. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - Column by column: `l - r * (l / r).floor()`. - A literal divisor: `x - rhs * (x * (1 / rhs)).floor()`; a literal dividend: `lhs - x * (lhs / x).floor()`. - - - `+ - * // %` on two ChunkedArrays: equal lengths take the array kernel, a length-1 side the scalar kernel. - - A literal is a scalar column, passed to the operation as a one-value Series. - The streaming engine evaluates each expression with polars-expr's `PhysicalExpr` on every morsel, so it runs the same kernels. - 2026-11-04, engine-test, by characterization pipeline round 3 (doubles): Pins the `Float32` floor remainder (-5 % 3 is 1, the divisor's sign) and NaN for a zero divisor (`1 % 0`, `0 % 0`). The test runs each case on one-row operands and, through `BROADCAST_SERIES_COMBINATIONS`, with the left side broadcast to three rows against a one-row right side, which takes the reciprocal-divisor kernel (PL-3); `EXEC_OP_COMBINATIONS` runs it on `Series` (in memory) and as a lazy [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select) with [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect). These inputs give the same results on the IEEE and the reciprocal path. Marked `slow`, which CI still runs. No test checks the sign of a zero remainder, so `zero_sign` stays at the source. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_list_arithmetic_values`, `Float32`: 1 // 0 is inf, 1 % 0 NaN, 1 / 0 inf, 0 // 0 NaN, 0 % 0 NaN, 0 / 0 NaN - `test_list_arithmetic_values`, `Float32` (picked so nothing upcasts to `Float64`): 7 // 3 is 2, -5 % 3 is 1, 2 / 128 is 0.015625 - 2026-11-04, docs, by characterization pipeline reverse pass - silent on float semantics (floored remainder, x % 0, x % inf) - - 2026-11-04, engine-test, by characterization pipeline reverse pass - SQL % truncates (-7.5 % 2 = -1.5); the comment says the expression API floors instead (eager, in memory). No upstream test of x % ±inf - 2026-11-04, source-reading, by characterization pipeline reverse pass: From the reverse pass's check of PL-1, which this fact records. - source comment: the same division / modulo combination as Python - 2026-11-04, engine-test, by characterization pipeline reverse pass: From the reverse pass's check of PL-1, which this fact records. - no upstream test of x % ±inf at py-2.1.0; test_list.py#L169-L170 and \#L210-L217 pin only finite and zero divisors ## polars-double-scalar-divisor: Dividing by a literal multiplies by its reciprocal Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: `x / c` is `x * (1 / c)`: not correctly rounded, NaN or inf for a subnormal `c`; `49.0 % 49.0` is 49.0 When one operand of `/`, `//` or `%` on floats has length 1 and the other more (a literal, `pl.lit`, or a one-row column against a longer one), Polars computes the reciprocal of the divisor once and multiplies: `x / c` is `x * (1 / c)`, `x // c` is `floor(x * (1 / c))`, `x % c` is `x - c * floor(x * (1 / c))`. Two roundings replace one, so `x / c` can miss the IEEE quotient by an ulp: `49.0 / 49.0` is 0.9999999999999999, and then `49.0 // 49.0` is 0.0 and `49.0 % 49.0` is 49.0. Recorded as PL-3. For a subnormal `c` the reciprocal overflows to inf: `0.0 / 1e-310` is NaN where IEEE gives 0.0, and `1e-300 / 1e-310` is inf where IEEE gives 1e10 (`%` gives -inf). The same holds for `Float32` below about 2.9e-39. On a one-row frame both sides have length 1 and the IEEE kernel runs, so the one-row probes do not show it. Two columns of equal length use `l / r`. Both engines share the kernels, but the streaming engine picks one per morsel: its in-memory source cuts a frame into morsels of `len.div_ceil(k · threads)` rows, so a frame shorter than the thread count streams as one-row morsels and divides exactly, and a frame a little longer mixes both results in one column. With `POLARS_MAX_THREADS=10`, `49.0 / 49.0` on 4 or 10 rows gave 1.0, on 11 rows both 1.0 and 0.9999999999999999, on 40 rows 0.9999999999999999. With one thread a 4-row frame gives 0.9999999999999999, and `% 49.0` gives 49.0. So the result depends on `POLARS_MAX_THREADS` and `POLARS_IDEAL_MORSEL_SIZE`. polars-scalar-divisor records the same finding from \#590's probe and is the id the engine tables cite. Integer `/` casts both sides to `Float64` and runs the same kernel, so a literal integer divisor takes the reciprocal too: `Int64` `49 / 49` is 0.9999999999999999 and `5 / 3` is 1.6666666666666665 (polars-int-true-divide). A division by an aggregate (`x / x.max()`) has a length-1 divisor as well. `DataFrame / 3` repeats the scalar to the frame's height and divides exactly. The `truediv` docs and the user guide call float division IEEE 754, which this contradicts: PL-8. No upstream test pins this; it may change between versions. The reference: `divide` on doubles is the IEEE quotient, rounded once (`rounding: TIE_TO_EVEN`), and creates no NaN from non-NaN inputs other than `0/0` and `inf/inf`, which raise. `sverify_fmod` is the truncated remainder, smaller than the divisor in magnitude. Options: `rounding`, `nan_creation`, `division_type`. Rows of the choice tables: `fp.div` `rounding` (fp64); `fp.div` `rounding` (fp32; evidence source); `fp.div` `nan_creation` (fp64); `fp.div` `nan_creation` (fp32; evidence source); `fp.mod` `division_type` (fp64); `fp.mod` `division_type` (fp32; evidence source). Constructs: `/`, `//`, `%`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [PL-3](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-3), [PL-8](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-8). Provenance: - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): `impl_float_arith_kernel!` is one macro instantiated for `pf16`, `f32` and `f64`, so `Float32` and `Float64` run the same operations. The streaming engine evaluates expressions through polars-expr on each morsel (`StreamExpr` in crates/polars-stream/src/expression.rs), so it shares these kernels with the in-memory engine. The broadcast in `apply_binary_kernel_broadcast_owned` sends a length-1 right side to the scalar kernel, and a literal reaches it as a one-value Series. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - `floor_div`, [trunc_div](../reference/sql.Dialect.md#sverify.sql.Dialect.trunc_div) and `mod` with a literal divisor use `inv = 1 / rhs`. - `prim_true_div_scalar` is `prim_wrapping_mul_scalar(lhs, 1 / rhs)`. - - `legacy_div_scalar`, which `ChunkedArray / ChunkedArray` reaches for floats, is `true_div_scalar`. - `/` on floats is `left / right` on the columns. - `+ - * // %` on two ChunkedArrays: equal lengths take the array kernel, a length-1 side the scalar kernel. - - A literal is a scalar column, passed to the operation as a one-value Series. - The streaming engine evaluates each expression with polars-expr's `PhysicalExpr` on every morsel, so it runs the same kernels. - 2026-11-04, manual-repro, by characterization pipeline round 2 (doubles): Polars 2.1.0, macOS arm64. Streaming splits a small frame into one-row morsels, where both sides have length 1 and the IEEE kernel runs; morsels of two rows or more take the literal path, as the in-memory engine does. - pl.DataFrame({'x': \[49.0, 3.0, -0.0, 0.0, 0.5\]}).select(pl.col('x') / 49.0, pl.col('x') % 49.0, pl.col('x') // 49.0): 0.9999999999999999, 49.0 and 0.0 for x = 49.0. The same on pl.DataFrame({'x': \[49.0\]}): 1.0 and 0.0. - pl.DataFrame({'x': \[0.0, 1e-300, 2.0\]}).select(pl.col('x') / 1e-310, pl.col('x') % 1e-310, pl.col('x') // 1e-310): \[nan, inf, inf\], \[nan, -inf, -inf\], \[nan, inf, inf\]. On pl.DataFrame({'x': \[0.0\]}): 0.0. - pl.DataFrame({'x': \[0.0, 1.0, 0.5\]}, schema={'x': pl.Float32}).select(pl.col('x') / pl.lit(1e-40, dtype=pl.Float32)): \[nan, inf, inf\]. - pl.LazyFrame({'x': \[49.0\] \* 5000 + \[0.0\]}).select(pl.col('x') / 49.0, pl.col('x') % 49.0).collect(engine='streaming'): 0.9999999999999999 and 49.0 for x = 49.0. The same query on a 4-row LazyFrame gave the IEEE results (1.0, 0.0, and 1e-300 / 1e-310 = 10000000000.00003) under streaming and the deviations under engine='in-memory'. - 2026-11-04, engine-test, by characterization pipeline round 3 (doubles): The broadcast cases of the test run `x / c`, `x // c` and `x % c` with a one-row `c` against three rows, the reciprocal path, in `Float32`: 1 / 0 is inf, 0 / 0 NaN, 7 // 3 is 2, -5 % 3 is 1. Each of these agrees with the IEEE quotient too, so the test neither confirms nor contradicts PL-3; no Polars test at py-2.1.0 asserts an inexact quotient of a broadcast divisor (searched `py-polars/tests/unit` for float divisions by literals and scalar aggregates). The rows stay at the source. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_list_arithmetic_values`, `Float32`: 1 // 0 is inf, 1 % 0 NaN, 1 / 0 inf, 0 // 0 NaN, 0 % 0 NaN, 0 / 0 NaN - `test_list_arithmetic_values`, `Float32` (picked so nothing upcasts to `Float64`): 7 // 3 is 2, -5 % 3 is 1, 2 / 128 is 0.015625 - 2026-11-04, probe, by characterization pipeline round 4 (streaming probes): Streaming agrees with memory on 9 000 rows: `49.0 % 49.0` is 49.0, `49.0 // 49.0` 0.0, `0.0 / 1e-310` NaN, `1e-300 / 1e-310` inf. So PL-3 holds on the streaming engine when the morsels hold more than one row. `python -m sverify_dev.probe_polars --diff --write` runs each observation (`STREAMING_OBSERVATIONS`, python/tools/sverify_dev/probe_polars.py) with `collect(engine='in-memory')` and `collect(engine='streaming')` and records both results. Polars 2.1.0 (Python), macOS arm64. - probe `s-scalar-divisor` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, probe, by characterization pipeline (merging duplicates after \#636): polars-scalar-divisor records the same finding from \#590's probe and is the id the engine tables cite. - probe `s-scalar-divisor` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass: Corrected the description: it said a small frame under streaming gives the IEEE results; that holds only while the frame is shorter than the thread count (in_memory_source.rs#L118-L122). Added integer `/` and division by an aggregate, which take the reciprocal too. - morsel length = len.div_ceil(k \* num_pipelines): which kernel runs depends on threads and POLARS_IDEAL_MORSEL_SIZE - - integer / integer casts both sides to Float64 first, so a literal integer divisor takes the float reciprocal kernel too - 2026-11-04, manual-repro, by characterization pipeline reverse pass - `POLARS_MAX_THREADS=10 POLARS_IDEAL_MORSEL_SIZE=64: pl.LazyFrame({'x': [49.0] * n}).select(pl.col('x') / 49.0).collect(engine='streaming') for n in (4, 10, 11, 40)` gives `4: [1.0]; 10: [1.0]; 11: [0.9999999999999999, 1.0]; 40: [0.9999999999999999] (Polars 2.1.0, macOS arm64)` - `POLARS_MAX_THREADS=1 vs 10: pl.LazyFrame({'x': [49.0, 0.0, 1e-300, 49.0]}).select(pl.col('x') / 49.0, pl.col('x') % 49.0).collect(engine='streaming')` gives `1 thread: (0.9999999999999999, 49.0) for x = 49.0; 10 threads: (1.0, 0.0); in-memory: (0.9999999999999999, 49.0) either way` - `pl.DataFrame({'a': [49, 5, 0]}).select(pl.col('a') / 49, pl.col('a') / 3, pl.col('a') / pl.col('a').max())` gives `49/49 = 0.9999999999999999, 5/3 = 1.6666666666666665 (IEEE 1.6666666666666667), a / a.max() = 0.9999999999999999 for a = 49` - `pl.DataFrame({'y': [5.0, -7.0]}) / 3 vs .select(pl.col('y') / 3)` gives `DataFrame / 3: 1.6666666666666667, -2.3333333333333335 (IEEE); pl.col('y') / 3: 1.6666666666666665, -2.333333333333333` - 2026-11-04, engine-test, by characterization pipeline reverse pass - no upstream test pins pl.col(x) / literal on two rows or more; test_df.py#L2982-L3016 checks DataFrame / scalar, which materializes the scalar and runs the column kernel - - 2026-11-04, manual-repro, by characterization pipeline reverse pass: From the reverse pass's check of PL-3, which this fact records. - `POLARS_MAX_THREADS=10 POLARS_IDEAL_MORSEL_SIZE=64: pl.LazyFrame({'x': [49.0] * n}).select(pl.col('x') / 49.0).collect(engine='streaming')` gives `n=4: [1.0]; n=11: [0.9999999999999999, 1.0]; n=40: [0.9999999999999999]` ## polars-elem-rounding: Elementary functions: Rust's libm calls, in the column's float type Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: `sqrt` correctly rounded; the rest is the platform's libm; `x ** n` for a literal n from 2 to 9 multiplies `sqrt`, `exp`, `sin`, `cos`, `tan`, `arcsin`, `arccos`, `arctan`, `arctan2` and `**` are generic over the float type and call Rust's `f32` or `f64` method: `Float32` computes in single precision and returns `Float32`, `Float64` in double. `sqrt` is IEEE's correctly rounded square root; the others go to the platform's libm (`sinf`, `exp`, `pow`, …), which promises no correct rounding, so their last bits can differ between macOS and Linux. Integer inputs are cast to `Float64` first. `x ** y` with a literal exponent takes shortcuts: 1 returns `x`, 0.5 is `sqrt(x)`, and an integer from 2 to 9 multiplies `x` by itself that many times, rounding after each product, where `powf` rounds once. With a column exponent it is `powf`, except that a one-row exponent of 2 is `x * x`. The docs are silent on the rounding of these functions and on `Float32` staying single precision: a docs gap. No upstream test pins their last bits or `Float32`'s precision; they may change between versions. The reference: The reference computes these with Lean's `Float` (the platform's libm); only NULL, the sign and the domain are exact, the last bits are each engine's (#54). Options: `rounding`. Rows of the choice tables: `fp.elem` `rounding` (fp32, fp64). Constructs: `sqrt`, `exp`, `sin`, `cos`, `tan`, `arcsin`, `arccos`, `arctan`, `arctan2`, `**`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): Each function matches `Float32` and `Float64` and calls one generic body with `num_traits::Float`, so `Float32` never goes through `f64`. The output type is the input's float type (`map_to_float_dtype`, `pow_dtype`). Rust's `f32::sqrt` and `f64::sqrt` compile to the hardware instruction; the others call the platform's libm. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - - - - `pow_on_floats`: the literal shortcuts, then `Pow::pow` (`powf`). - `pow_on_chunked_arrays`: a one-row exponent of 2 is `base * base`. - - `exp` on `Float32` is `f32::exp`. - - - - - 2026-11-04, manual-repro, by characterization pipeline round 2 (doubles): `Float32` stays in single precision. - pl.DataFrame({'x': \[49.0, 1.0, 0.5, 2.0, -0.0\]}, schema={'x': pl.Float32}).select(sqrt, sin, exp, \*\* 0.3): every column Float32; sqrt(0.5) = 0.70710677, sin(1) = 0.84147096, exp(1) = 2.7182817, the correctly rounded single-precision values (Polars 2.1.0, macOS arm64). - 2026-11-04, docs, by characterization pipeline reverse pass - no error bound promised; silent on libm and Float32 precision - Expr.pow: the result dtype rule only - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_exp_log1p`: Float32 → Float32, integers → Float64 (lazy, both engines); values compared with tolerance - `test_trigonometric` against numpy with tolerance (eager) ## polars-fp-ieee: `Float32` and `Float64` arithmetic: IEEE 754 between columns Engine: polars py-2.1.0; area: doubles; status: no-finding; evidence: source. Verdict: IEEE 754 rounding for `+ - *`, and for `/` between two columns Polars's user guide says `Float32` and `Float64` generally follow IEEE 754, with exceptions for NaN equality and order and for the sign of zero and NaN. The kernels for `+ - *` are the IEEE operations in both float types, so they round as IEEE 754 binary32 and binary64 do, to nearest with ties to even. `/` between two columns of equal length is the IEEE quotient too. A length-1 divisor against a longer column (a literal on a frame of two rows or more) is divided by multiplying with its reciprocal, two roundings: polars-double-scalar-divisor and PL-3. The guide promises no error bound for other computations unless a function says so, so it does not settle the rounding of the elementary functions or logarithms (polars-elem-rounding, polars-log-base). The reference: `rounding: TIE_TO_EVEN` for `+ - × /` on doubles (native `Float`). `fp32` arithmetic is refused in every profile until \#417's stage 3. Options: `rounding`. Rows of the choice tables: `fp.arith` `rounding` (fp32, fp64); `fp.div` `rounding` (fp32, fp64). Constructs: `+`, `-`, `*`, `/`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [PL-3](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-3). Provenance: - 2026-11-04, docs, by characterization pipeline round 1 (doubles): Polars's user guide says it generally follows IEEE 754 for `Float32` and `Float64`, with two exceptions: any NaN equals any other NaN and is greater than every non-NaN value, and operations guarantee nothing about the sign of zero or NaN or a NaN's payload (a sort or group by may make every zero +0 and every NaN a positive NaN). It promises no error bound unless a function says otherwise. The user guide is not versioned; the section reads the same at tag py-2.1.0. - User guide, Floating point numbers (stable). - The same section at tag py-2.1.0. - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): Raises `fp.arith` rounding to source: `+ - *` are the IEEE operations for `f32` and `f64`, column by column and with a literal (whose shortcuts return `x` exactly). The streaming engine runs the same polars-expr kernels. Corrects the `/` claim: IEEE only when both sides are columns of equal length (or both have length 1). With a length-1 divisor against a longer side, `x / c` is `x * (1 / c)`, which can miss by an ulp (`49.0 / 49.0` is 0.9999999999999999). Title, verdict and description now say so and link polars-double-scalar-divisor (PL-3). Read at tag py-2.1.0 (commit 7f902bd3). - Column kernels: `l + r`, `l - r`, `l * r`. - Literal kernels: `x + c`, `x * c`, with shortcuts that return `x` for `c = 0` (add, subtract) and `c = 1`, exact. - `prim_true_div` is `l / r`; `prim_true_div_scalar` multiplies by `1 / rhs`. One macro for pf16, f32, f64. - - - ## polars-fp32: `Float32` arithmetic stays in `Float32`, and pulls in a double literal Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: native single precision; `Float32 + 0.1` is `Float32` Arithmetic on `Float32` is single precision (`3e38 + 3e38` is inf, `16777216 + 1` is 16777216), and `sum` and `mean` of `Float32` are `Float32`. `sum` adds in `Float32` on both engines; the streaming `mean` sums in `f64` and rounds the quotient to `Float32` once (polars-mean-float-streaming). A Python float or int literal takes the column's type, so `Float32 + 0.1` adds 0.1 rounded to a float. Casts between the formats round to nearest (polars-cast-to-double). The docs are silent on a literal taking a `Float32` column's type: a docs gap. No upstream test pins single-precision overflow or rounding (`3e38 + 3e38`, `16777216 + 1`); they may change between versions. The reference: `fp32` arithmetic is refused in every profile until \#417's stage 3. Options: `fp32`, `implicit_casts`. Rows of the choice tables: `fp32` (fp32). Constructs: `+`, `sum`, `mean`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: issues [\#417](https://github.com/AlexanderNenninger/substrait-lean/issues/417). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `fp32-arith` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `fp32-arith` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass: Added to the description: the streaming `mean` of `Float32` sums in `f64` (reduce/mean.rs#L30-L52); the result type stays `Float32`. - the f32 kernels - a dynamic int or float literal takes a float column's type - SumCast: f32 sums as f32 - streaming mean of Float32: f64 sum, quotient cast to f32 - 2026-11-04, engine-test, by characterization pipeline reverse pass - literals with a Float32 column stay Float32 (lazy, both engines) - Float32 / 2 stays Float32 ## polars-int-float-compare: An integer compared with a double: both as doubles Engine: polars py-2.1.0; area: doubles; status: no-finding; evidence: probe. Verdict: `Int64 == Float64` compares in `Float64` Comparing an `Int64` with a `Float64` casts the integer to `Float64` first: `2⁵³ + 1 == 2.0⁵³` is true. The reference: Substrait's comparisons take two arguments of one type; a plan casts first, and the cast's rounding applies. Options: `implicit_casts`. Constructs: `==`, `<`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `int-float-compare` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `int-float-compare` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-log-base: `log10` and `log(base)` divide natural logarithms Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: `ln(x) / ln(base)`: last-bit differences from `log10`, `log2` `log10()` is `log(10.0)`, and `log(base)` is Rust's `f64::log`, `ln(x) / ln(base)`: two roundings, so it can miss the correctly rounded result by an ulp. `log10(0.5)` is -0.30102999566398114 where `log10` gives -0.3010299956639812, and `log2(1e308)` is 1023.1538532253077 where `log2` gives 1023.1538532253076. On `Float32` the same division runs in single precision, and the default base `math.e` becomes the float nearest e, whose `ln` rounds to 0.99999994. So the natural `log()` of a `Float32` misses by an ulp: `log(0.5)` is -0.69314724 where the correctly rounded value is -0.6931472, and `log10(0.5)` is -0.30102998 where it is -0.30103001. Recorded as PL-4. The docs are silent on `log10` and `log` dividing two logarithms: a docs gap. The user guide promises no error bound. No upstream test pins this; it may change between versions. The reference: `log10` and `log2` are the platform's `log10` and `log2`; only NULL, the sign and the domain are exact in the reference, the last bits are each engine's (#54). Options: `rounding`. Rows of the choice tables: `fp.log` `rounding` (fp64); `fp.log` `rounding` (fp32). Constructs: `log10`, `log`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [PL-4](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-4); issues [\#54](https://github.com/AlexanderNenninger/substrait-lean/issues/54). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Float64.log10()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.log(2)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): `log` keeps a `Float32` input in `Float32` (`log_dtype`) and computes `f32::log`, two single-precision roundings. The base literal is cast to `Float32`. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - Equal float types: `x.log(base)` in that type; a float input casts the base to its type. - `Expr.log(base=math.e)`. - `f32::log` may not be correctly rounded; it is `ln(self) / ln(base)`. - 2026-11-04, manual-repro, by characterization pipeline round 2 (doubles): The correctly rounded `Float32` values of `ln(0.5)` and `log10(0.5)` are -0.693147182464599609375 and -0.30103000998497009277. - pl.DataFrame({'x': \[0.5, 2.0, 49.0\]}, schema={'x': pl.Float32}).select(pl.col('x').log(), pl.col('x').log10()): log gave -0.6931472420692444, 0.6931472420692444, 3.8918204307556152, log10 gave -0.3010299801826477, … (Polars 2.1.0, macOS arm64). - 2026-11-04, docs, by characterization pipeline reverse pass - 'Compute the base 10 logarithm of the input array, element-wise'; no accuracy note. The body is `self.log(10.0)` (expr.py L914-L935). - Polars 'does not provide guarantees on the error unless mentioned otherwise' (SRC docs/source/user-guide/concepts/data-types-and-structures.md L186-L189). - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_log`: \[1, 3, 9, 27, 81\].log(base \[3, 3, 9, 3, 9\]) == \[0, 1, 1, 3, 2\] for Float16/32/64 and mixed types, lazy collect() (both engines in CI), rel_tol 1e-5: pins result types and values, not the last bits. - `test_log_exp`: log10 of \[1, 100, 1000\] == \[0, 2, 3\], log vs np.log, rel_tol 1e-5. No upstream test pins the one-ulp misses of ln(x)/ln(base). - 2026-11-04, source-reading, by characterization pipeline reverse pass: From the reverse pass's check of PL-4, which this fact records. - A float column with a base of another type casts the base to the column's type (math.e to 2.7182817 for Float32), then x.log(base) in f32. ## polars-math-domain: Elementary functions: NaN outside the domain, -inf for `log(0)` Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: `on_domain_error: NAN`; `on_log_zero: MINUS_INFINITY` `sqrt`, the logarithms, `arcsin`, `arccos` and `**` give NaN outside their domain (`sqrt(-1)`, `log(-1)`, `arcsin(2.5)`, `(-1) ** 0.5`), `log(0)` is -inf, and `sin`, `cos`, `tan` of ±inf are NaN. Inside the domain they are Rust's `f64` functions (the platform's libm). The docs are silent on results outside a function's domain: a docs gap. No upstream test pins `log(0)`, `log(-1)`, `(-1) ** 0.5` or `sin(±inf)`; they may change between versions. The reference: `on_domain_error: ERROR` and `on_log_zero: ERROR` (#215); `sin(±inf)` is a domain issue. Options: `on_domain_error`, `on_log_zero`. Rows of the choice tables: `fp.elem` `on_domain_error` (fp64); `fp.log` `on_domain_error` (fp64); `fp.log` `on_log_zero` (fp64); `fp.elem` `on_domain_error` (fp32; evidence source); `fp.log` `on_domain_error` (fp32; evidence source); `fp.log` `on_log_zero` (fp32). Constructs: `sqrt`, `log`, `log10`, `sin`, `cos`, `tan`, `arcsin`, `arccos`, `**`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-16](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-16); issues [\#215](https://github.com/AlexanderNenninger/substrait-lean/issues/215). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Float64.sqrt()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.log()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.log10()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.log(2)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.sin()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.cos()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.tan()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.arcsin()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.arccos()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64 ** Float64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.exp()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.arctan()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): The elementary functions are generic over the float type and call Rust's `f32` methods for `Float32` (single-precision libm), so `sqrt(-1)`, `asin(2)`, `(-1) ** 0.5` and `log(-1)` are NaN and `sin(±inf)` is NaN in `Float32` as in `Float64`. `log(0)` is `ln(0) / ln(base)`, -inf for any base above 1. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - `Float32` stays `Float32`; integers are cast to `Float64`. - Generic over `T: PolarsFloatType` with `num_traits::Float`: `sin`, `cos`, `tan`, `asin`, `acos`, `atan`. - `pow_on_floats`, generic; a literal 0.5 is `sqrt`. - `sqrt` on `Float32` is `f32::sqrt`. - `log(base)`: `x.log(base)` in the column's float type. - `map_to_float_dtype`: `Float32` maps to `Float32`. - `log_dtype`: a float input keeps its type. - 2026-11-04, manual-repro, by characterization pipeline round 2 (doubles): `on_log_zero` on `Float32`. - pl.DataFrame({'x': \[-0.0\]}, schema={'x': pl.Float32}).select(pl.col('x').log(), pl.col('x').log10(), pl.col('x').log(2)): each -inf, dtype Float32 (Polars 2.1.0, macOS arm64). - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_sqrt_neg_inf`: sqrt(\[-inf, -9, 0, 9, inf\]) == \[nan, nan, 0.0, 3.0, inf\] (compared as strings), eager, in memory. - `test_trigonometric`: sin, cos, tan, arcsin, arccos, arctan and the hyperbolics of \[0.0, pi, None, nan\] equal numpy's (arcsin(pi), arccos(pi) NaN), eager, in memory. No upstream test of log(0) = -inf, log(-1), (-1) \*\* 0.5 or sin(inf). - 2026-11-04, docs, by characterization pipeline reverse pass - 'Polars generally follows the IEEE 754 floating point standard' (data-types-and-structures.md L178-L179); the function docstrings say nothing on domain errors. ## polars-nan-equality: NaN equals NaN: in `==`, grouping, [unique](../reference/unique.md#sverify.unique) and joins Engine: polars py-2.1.0; area: doubles; status: no-finding; evidence: probe. Verdict: `nan_equality: NAN_IS_NAN` natively `NaN == NaN` is true and `NaN != NaN` false; `is_in([nan])` finds NaN; `group_by`, [unique](../reference/unique.md#sverify.unique), `n_unique` put every NaN in one group, and a join matches NaN keys. NULL compares as NULL (`eq_missing` makes NULL equal NULL). The reference: `NAN_IS_NAN` (#89, an override of IEEE equality) for `=`, grouping, `DISTINCT` and joins. Options: `nan_equality`, `nan_zero_grouping`. Rows of the choice tables: `cmp` `nan_equality` (fp64); `cmp` `nan_equality` (fp32; evidence engine_test). Constructs: `==`, `is_in`, `group_by`, [unique](../reference/unique.md#sverify.unique), `join`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-4](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-4); issues [\#89](https://github.com/AlexanderNenninger/substrait-lean/issues/89). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Float64 == Float64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `nan-compare` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `nan-is-in` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `group-nan-zero` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `unique-nan-zero` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `join-nan-zero-null` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, docs, by characterization pipeline round 1 (doubles): Polars's user guide says it generally follows IEEE 754 for `Float32` and `Float64`, with two exceptions: any NaN equals any other NaN and is greater than every non-NaN value, and operations guarantee nothing about the sign of zero or NaN or a NaN's payload (a sort or group by may make every zero +0 and every NaN a positive NaN). It promises no error bound unless a function says otherwise. The user guide is not versioned; the section reads the same at tag py-2.1.0. The NaN equality is stated for both float types, so it settles the `Float32` row; the probes cover `Float64`. - User guide, Floating point numbers (stable). - The same section at tag py-2.1.0. - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): Equality, order, sorting and hashing of floats are one generic implementation for `f32` and `f64` (and `pf16`), so `Float32` behaves as the `Float64` probes show. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - `impl_float_eq_ord!` for pf16, f32, f64: NaN equals NaN and is the largest value. - `impl_float_total_ord_kernel!` for f32 and f64: the column comparison kernels. - `canonical_f32` and `canonical_f64`: `x + 0.0`, every NaN one quiet NaN, for hashing (group by, unique, joins). - - Sorting uses `TotalOrd::tot_cmp`. - 2026-11-04, engine-test, by characterization pipeline round 3 (doubles): Pins NaN equality for `Float32`: `NaN == NaN` and `-NaN == NaN` are true in comparisons, `is_in` and `index_of`. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_total_ordering_float_series` (slow): every pair of 0.0, -0.0, ±1.0, ±NaN, ±inf, NULL as `Float32` against `Float32` and `Float64` against `Float32`, column against column and against a broadcast first value or literal; `==`, `!=`, `<`, `<=`, `>`, `>=`, `eq_missing`, `ne_missing` must follow `reference_ordering_propagating`: NaN equals NaN (also -NaN), NaN is above every number including inf, 0.0 equals -0.0, NULL propagates. Eager `DataFrame.select`, in memory - `test_is_in_float`, `Float32` and `Float64`: NaN `is_in([-0.0, -NaN])` and 0.0 `is_in([-0.0, -NaN])` are both true (eager) - `test_float`, `Float32` and `Float64`: `index_of(-NaN)` finds NaN and `index_of(-0.0)` finds 0.0, on the series, its ascending and descending sorts and a chunked copy (eager and lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect)) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. join-nan-zero-null's recorded differ 1 is a tie order: `sort('y')` ties the rows 0.0 and -0.0 (both y = 2), and the engines list them in different orders. The join results are the same set of rows on both engines; it is no engine difference. - probe `Float64 == Float64` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `nan-compare` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `nan-is-in` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `group-nan-zero` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `unique-nan-zero` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `join-nan-zero-null` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-nan-order: NaN sorts above +inf, in comparisons and in sorts Engine: polars py-2.1.0; area: doubles; status: no-finding; evidence: probe. Verdict: `nan_order: GREATEST` natively, ascending and descending `NaN > x` is true for every number x, `inf < NaN` is true, and `sort` puts NaN after +inf ascending and first (after the NULLs) descending. `rank`, `median` and [quantile](../reference/sql.Dialect.md#sverify.sql.Dialect.quantile) use the same order, so `median(1, 2, NaN)` is 2. The reference: `GREATEST`: NaN above +inf in sorts and in `<`. Options: `nan_order`. Rows of the choice tables: `cmp` `nan_order` (fp64); `cmp` `nan_order` (fp32; evidence engine_test); `agg.quant` `nan_order` (fp64). Constructs: `<`, `>`, `sort`, `rank`, `median`, [quantile](../reference/sql.Dialect.md#sverify.sql.Dialect.quantile). Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-4](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-4). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Float64 < Float64` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `sort-doubles` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `rank-descending-nan` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `median-nan` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, docs, by characterization pipeline round 1 (doubles): Polars's user guide says it generally follows IEEE 754 for `Float32` and `Float64`, with two exceptions: any NaN equals any other NaN and is greater than every non-NaN value, and operations guarantee nothing about the sign of zero or NaN or a NaN's payload (a sort or group by may make every zero +0 and every NaN a positive NaN). It promises no error bound unless a function says otherwise. The user guide is not versioned; the section reads the same at tag py-2.1.0. NaN above every non-NaN value is stated for both float types, so it settles `cmp`'s `Float32` row; the probes cover `Float64`. - User guide, Floating point numbers (stable). - The same section at tag py-2.1.0. - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): Equality, order, sorting and hashing of floats are one generic implementation for `f32` and `f64` (and `pf16`), so `Float32` behaves as the `Float64` probes show. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - `impl_float_eq_ord!` for pf16, f32, f64: NaN equals NaN and is the largest value. - `impl_float_total_ord_kernel!` for f32 and f64: the column comparison kernels. - `canonical_f32` and `canonical_f64`: `x + 0.0`, every NaN one quiet NaN, for hashing (group by, unique, joins). - - Sorting uses `TotalOrd::tot_cmp`. - 2026-11-04, engine-test, by characterization pipeline round 3 (doubles): Pins the NaN order of `Float32` comparisons: NaN is greater than +inf and every number, and equal to NaN. The streaming sort test pins that the streaming engine places NaN, the infinities and the zeros of a `Float32` key as the in-memory sort does; it compares the engines and states no order of its own. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_total_ordering_float_series` (slow): every pair of 0.0, -0.0, ±1.0, ±NaN, ±inf, NULL as `Float32` against `Float32` and `Float64` against `Float32`, column against column and against a broadcast first value or literal; `==`, `!=`, `<`, `<=`, `>`, `>=`, `eq_missing`, `ne_missing` must follow `reference_ordering_propagating`: NaN equals NaN (also -NaN), NaN is above every number including inf, 0.0 equals -0.0, NULL propagates. Eager `DataFrame.select`, in memory - `test_sort_direct_key_dtypes` with the `f32` key (517 rows with NaN, ±inf, -0.0, NULL), ascending and descending, `maintain_order=True`: the streaming sort must equal the in-memory sort exactly, with the in-memory and the partitioned (`POLARS_SORT_PARTITION_THRESHOLD_BYTES=1`, morsels of 7) sort modes - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `Float64 < Float64` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `sort-doubles` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `rank-descending-nan` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `median-nan` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-nan-sign: A NaN's sign survives Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: `-NaN` keeps its sign bit Negation flips a NaN's sign bit (`-nan` is `0xfff8000000000000`), `abs` clears it and `sqrt` keeps it. The sign of a NaN that arithmetic creates (`inf - inf`) is the platform's: positive on arm64, negative on x86. No upstream test pins this; it may change between versions. The reference: One NaN (`nan_kind: ONE_NAN`). Options: `nan_kind`. Rows of the choice tables: `nan_kind` (fp64); `nan_kind` (fp32; evidence source). Constructs: `unary -`, `abs`, `sqrt`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-4](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-4). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `nan-sign-negate` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, docs, by characterization pipeline round 1 (doubles): Polars's user guide says it generally follows IEEE 754 for `Float32` and `Float64`, with two exceptions: any NaN equals any other NaN and is greater than every non-NaN value, and operations guarantee nothing about the sign of zero or NaN or a NaN's payload (a sort or group by may make every zero +0 and every NaN a positive NaN). It promises no error bound unless a function says otherwise. The user guide is not versioned; the section reads the same at tag py-2.1.0. So the sign the probes show surviving is what Polars 2.1.0 happens to do: the docs do not promise it, and a sort or group by may make every NaN positive. The `Float32` row stays open. - User guide, Floating point numbers (stable). - The same section at tag py-2.1.0. - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): `impl_float_arith_kernel!` is one macro instantiated for `pf16`, `f32` and `f64`, so `Float32` and `Float64` run the same operations. The streaming engine evaluates expressions through polars-expr on each morsel (`StreamExpr` in crates/polars-stream/src/expression.rs), so it shares these kernels with the in-memory engine. `-x` flips the sign bit and `abs` clears it for `f32` as for `f64`, and `sqrt` is the hardware square root, which keeps a NaN's sign. A NaN that arithmetic creates takes the platform's default NaN. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - `abs` is `x.abs()`, negation `-x`. - - - 2026-11-04, engine-test, by characterization pipeline reverse pass - Closest test, `test_hash`: hash(-nan) == hash(nan) (eager). No upstream test reads the sign bit of a NaN from negation, abs, sqrt or inf - inf. ## polars-round: `round()`: half to even by default Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: `rounding: TIE_TO_EVEN`; `TIE_AWAY_FROM_ZERO` with `mode='half_away_from_zero'` `round()` rounds half to even (`round_ties_even`): `0.5` is 0, `2.5` is 2, `-2.5` is -2. `round(mode='half_away_from_zero')` is Rust's `f64::round`, and `mode='to_zero'` truncates. `floor` and `ceil` agree with the reference on every edge case. The reference: `round` with `rounding: TIE_AWAY_FROM_ZERO` (#335), the only value the model accepts. So the writer needs `mode='half_away_from_zero'`, which matches on every row. Options: `rounding`. Rows of the choice tables: `fp.round` `rounding` (fp64); `fp.round` `rounding` (fp32; evidence engine_test). Constructs: `round`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: issues [\#335](https://github.com/AlexanderNenninger/substrait-lean/issues/335). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Float64.round()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.round(mode='half_away_from_zero')` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `round-default-mode` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.floor()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.ceil()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, docs, by characterization pipeline round 1 (doubles): `Expr.round` rounds "floating point data" with [mode](../reference/sql.Dialect.md#sverify.sql.Dialect.mode) `half_to_even` (the default), `half_away_from_zero` or `to_zero`, described by value (`2.5` rounds to 2 under the default). The docstring names no float type, so it holds for `Float32` too. Settles the `Float32` row at docs level; the probes cover `Float64`. - Expr.round (API reference, version 2). - - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): `round()` on `Float32` calls `f32::round_ties_even`, half to even as for `Float64`. With decimals it scales in `f64` and casts back. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - The default mode is `f32::round_ties_even` or `f64::round_ties_even`. - `Float32` with 0 decimals applies the `f32` operation. - 2026-11-04, engine-test, by characterization pipeline round 3 (doubles): The ties at zero digits are exact in `Float32`, so the test pins round half to even (and half away from zero by option) for `fp32` as for `fp64`. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_round_mode`, for `Float16`, `Float32` and `Float64` (lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect)): `half_to_even` gives 0.5 → 0.0, 1.5 → 2.0, 2.5 → 2.0, -0.5 → -0.0, -1.5 → -2.0, 2.25 → 2.2 and 2.75 → 2.8 at one digit; `half_away_from_zero` gives 1.0, 2.0, 3.0, -1.0, -2.0, 2.3, 2.8 - `round(decimals=0, mode='half_to_even')`: half to even is the default - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_decimal_rounding_widens_precision`: Decimal(2, 1) \[9.5, -9.5, 0.5\].round(0) == \[10.0, -10.0, 0.0\] (half to even), mode='half_away_from_zero' == \[10.0, -10.0, 1.0\], eager, in memory. - \#\[test\] `test_round_series`: \[1.003, 2.23222, 3.4352\].round(2, default) gives 1.0 first (assert_eq!, exact). ## polars-round-digits: `round(d)`: `op(x · 10ᵈ) / 10ᵈ`, no negative `d` Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: matches the reference on the edge cases; `d < 0` refused `round(d)` multiplies by `10ᵈ`, rounds (half to even unless [mode](../reference/sql.Dialect.md#sverify.sql.Dialect.mode) says otherwise) and divides back; from `d = 300` it splits the multiplier into powers of 2 and 5, from `d = 326` it returns the input, and a non-finite intermediate returns the input. The two roundings make ties of the decimal text round either way: `round(2.675, 2)` is 2.68 though the double is below 2.675, and `round(0.285, 2)` is 0.28. A negative `d` is refused by the Python API (`OverflowError`, [decimals](../reference/sql.Dialect.md#sverify.sql.Dialect.decimals) is a `u32`). The `round` docs say the `half_` modes round to the nearest rounded value, which `round(2.675, 2) = 2.68` contradicts: PL-9. The docs are silent on a negative [decimals](../reference/sql.Dialect.md#sverify.sql.Dialect.decimals) and on `d ≥ 326` returning the input: a docs gap. No upstream test pins this; it may change between versions. The reference: `round` to `d` digits (#335), `TIE_AWAY_FROM_ZERO`, with negative digits. On the probe's edge cases at `d` = 1, 2, 16, 308, 309, 400 Polars and the reference agree. Options: `rounding`. Rows of the choice tables: `fp.round` `rounding` (fp64). Constructs: `round`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [PL-9](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-9); issues [\#335](https://github.com/AlexanderNenninger/substrait-lean/issues/335). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `Float64.round(1)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.round(2)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.round(16)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.round(308)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.round(309)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `Float64.round(400)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `round-digits-ties` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `round-digits-negative` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `Float64.round(1)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `Float64.round(2)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `Float64.round(16)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `Float64.round(308)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `Float64.round(309)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `Float64.round(400)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `round-digits-ties` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `round-digits-negative` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, docs, by characterization pipeline reverse pass - 'half\_' strategies 'round all values to the nearest rounded value, only using the strategy to break ties when a value falls exactly between two rounded values' (SRC py-polars/src/polars/expr/expr.py L2001-L2025). Contradicted: round(2.675, 2) is 2.68 though the double 2.67499999999999982… is nearer 2.67. Drafted as a docs entry (PL-D2). - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_round`: lazy collect(), Float32/Float64, (1835.665, 2) expected 1835.67, (1.005, 2) 1.0, (1e-20, 20) 1e-20, rel_tol 1e-5. Py-2.1.0 returns 1835.66 for the default mode and the test passes on tolerance: it does not pin the x \* 10^d rounding. No test for d \>= 300. - 2026-11-04, probe, by characterization pipeline reverse pass - pl.LazyFrame({'x': \[2.675, 1835.665, 0.285, 1.005\]}).select(e=pl.col('x').round(2), a=pl.col('x').round(2, mode='half_away_from_zero')).collect(engine=…) with POLARS_IDEAL_MORSEL_SIZE=64: in-memory and streaming both e \[2.68, 1835.66, 0.28, 1.0\], a \[2.68, 1835.67, 0.28, 1.0\]; Python round gives \[2.67, 1835.66, 0.28, 1.0\] (Polars 2.1.0, macOS arm64). ## polars-scalar-divisor: Double `/`, `//`, `%` by a scalar: a multiplication by the reciprocal Engine: polars py-2.1.0; area: doubles; status: decided; evidence: probe. Verdict: `divide` of doubles by a scalar rounds twice (2u, `approx`) and overflows below the smallest normal divisor; `//` and `%` give wrong results With a length-1 divisor (a literal) against a column of two or more rows, Polars's in-memory engine computes `x / c` as `x * (1 / c)`: `49.0 / 49.0` is `0.9999999999999999`, and `0.0 / 1e-310` is NaN (`1 / 1e-310` is `inf`, and `0 * inf` NaN). `//` and `%` divide the same way. A column divisor of the same length divides as IEEE does. Here the streaming engine gave the IEEE quotient on four rows, because a frame shorter than the thread count streams as one-row morsels; on longer frames, or on one thread, streaming takes the reciprocal too (polars-double-scalar-divisor, PL-3). The same finding as polars-double-scalar-divisor (PL-3), which carries the source reading, the streaming probes (the reciprocal path also runs on streaming once a morsel holds two or more rows: `s-scalar-divisor`, 9,000 rows) and the ulp bound (polars-ulp-arith). This fact keeps its id for the engine tables (`SubstraitLean/EngineTables.lean`) that cite it. The reference: `divide` of doubles rounds the exact quotient once (`TIE_TO_EVEN`), and `0.0 / c` is `0.0` for a finite nonzero `c`. Decided on \#444: Polars's `/` is `approx` with 2u per division, under the precondition that a divisor is 0 or not subnormal; `//` and `%` floor the quotient (`49.0 % 49.0` is `49.0`), so their translation broadcasts the divisor to a column (`column_divisor`). Options: `rounding`, `nan_creation`. Constructs: `/`, `//`, `%`. Provenance: - 2026-11-04, probe, by Claude session frosty-bassi-b2cf7d (#444 step 3a, PR \#590): Polars 2.1.0 (Python) on macOS arm64. - `pl.DataFrame({'x': [49.0, 0.0, 7.0, -0.0]}).select(pl.col('x') / pl.lit(49.0))` gives `[0.9999999999999999, 0.0, 0.14285714285714285, -0.0]` - `pl.DataFrame({'x': [49.0, 0.0, 7.0, -0.0]}).select(pl.col('x') / pl.lit(1e-310))` gives `[inf, nan, inf, nan]` - `pl.DataFrame({'x': [49.0, 0.0, 7.0, -0.0]}).lazy().select(pl.col('x') / pl.lit(49.0)).collect(engine='streaming')` gives `[1.0, 0.0, 0.14285714285714285, -0.0]` - `pl.DataFrame({'x': [1e-300, 49.0, 0.0]}).select(pl.col('x') / pl.lit(1e-310))` gives `[inf, inf, nan]` - `pl.DataFrame({'x': [1e-300, 49.0, 0.0]}).select(pl.col('x') % pl.lit(49.0))` gives `[1e-300, 49.0, 0.0]` - 2026-11-04, probe, by characterization pipeline (merging duplicates after \#636): The same finding as polars-double-scalar-divisor (PL-3), which carries the source reading, the streaming probes (the reciprocal path also runs on streaming once a morsel holds two or more rows: `s-scalar-divisor`, 9,000 rows) and the ulp bound (polars-ulp-arith). This fact keeps its id for the engine tables (`SubstraitLean/EngineTables.lean`) that cite it. - probe `s-scalar-divisor` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass: Corrected the description: the four-row streaming result is a thread-count effect; with POLARS_MAX_THREADS=1 the same frame gives 0.9999999999999999 (polars-double-scalar-divisor's reverse-pass repro). - morsel length = len.div_ceil(k \* num_pipelines): a frame shorter than the thread count streams as one-row morsels ## polars-ulp-arith: `+ - * /` round once; `/` by a literal and `x ** n` round more Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: column ops ≤ 0.5 ulp; `x / c` \< 1.5 ulp (1.2 seen); `x ** n` for literal n in 3..9 up to n − 1 ulp (3.8 seen) `+`, `-`, `*` and `/` of two columns are one IEEE operation in the column's type (`Float32` stays binary32): correctly rounded, ≤ 0.5 ulp, on every platform. `x / c` with a literal (or any length-1) divisor is `x * (1 / c)` (PL-3, polars-double-scalar-divisor): two roundings. Bound: `1 / c` is off by at most 2^-53 relative (2^-24 for `Float32`), which is at most 1 ulp of the quotient, plus the product's 0.5, so \< 1.5 ulp. Observed, fp64 / fp32: c = 49: 1.20 / 0.84 ulp; c = 7.3: 1.20 / 0.81; c = 0.1: 1.00 / 0.75; c = 10: 0.80 / 0.60; c = 3: 0.67 / 0.67. A third to a half of the quotients miss the correctly rounded value. `x ** n` with a literal integer `n` from 2 to 9 multiplies `x` by itself `n − 1` times (polars-elem-rounding), rounding after each product: `n − 1` roundings, so up to about `n − 1` ulp. `x ** 2` is one product, correctly rounded. Observed max, fp64 / fp32: n = 3: 1.24 / 1.24; 4: 1.55 / 1.63; 5: 2.16 / 2.27; 6: 2.51 / 2.44; 7: 3.43 / 2.96; 8: 3.17 / 3.50; 9: 3.59 / 3.78 ulp, with 25 to 55 % of results off. DuckDB's `pow(x, n)` on the same inputs stays within 0.51 ulp (libm `pow`). These are hardware operations: the bounds hold on every platform. No upstream test pins this; it may change between versions. The reference: `add`, `subtract`, `multiply` and `divide` round once (`rounding: TIE_TO_EVEN`); `power` is one libm `pow` call in Lean's `Float`. sverify's ε (#85) for Polars has to take 1.5 ulp for a division by a literal and n − 1 ulp for `x ** n`, or shim them (divide by a column, `pow` with a non-literal exponent). Options: `rounding`, `fp32`. Rows of the choice tables: `fp.arith` `rounding` (fp32, fp64); `fp.div` `rounding` (fp32, fp64); `fp.elem` `rounding` (fp32, fp64). Constructs: `+`, `-`, `*`, `/`, `**`. Settings: `{"engine": "streaming", "platform": "macOS arm64"}`. Provenance: - 2026-11-04, source-reading, by characterization pipeline round 5 (ulp bounds) - prim_true_div_scalar: lhs \* (1 / rhs) - pow_on_floats: a literal integer exponent in 2..9 is `out = out * base` n − 1 times - 2026-11-04, probe, by characterization pipeline round 5 (ulp bounds): Probe script ulp_probe.py (session scratchpad, not in the repository), run from python/ as `uv run --with mpmath python ulp_probe.py --n 3000 --seed 1 --out ulp.json`: about 3 000 inputs per operation and type, random and hard cases, against mpmath at 256 bits. 'bad' counts results that differ from the correctly rounded value; the max is in ulps of the result type at the exact value's binade. DuckDB runs on a table loaded from Parquet, Polars on a LazyFrame with collect(engine='streaming'), so the column kernels run. DuckDB 2.0.0 and Polars 2.1.0 (Python wheels), macOS 26.6 arm64, Python 3.12.11, mpmath 1.3.0. Elementwise kernels are shared by the streaming and in-memory engines (polars-elem-rounding, polars-double-scalar-divisor); the probe collects with engine='streaming' on 3 000-row frames, so morsels hold more than one row and the literal paths run. - pl.LazyFrame({'a': X, 'b': Y}).select(a + b, a - b, a \* b, a / b).collect(engine='streaming'), Float64 and Float32, 3 000 rows: max 0.5000, 0.5000, 0.4994, 0.4999 ulp (fp64), 0.5 each (fp32); 0 off the correctly rounded value. - select(pl.col('a') / pl.lit(c, dtype=…)) for c in 49, 3, 10, 0.1, 7.3: fp64 max 1.2041 (1 490 of 3 000 off), 0.6667 (1 040), 0.8000 (1 018), 0.9999 (975), 1.2006 (1 550); fp32 0.8367 (755), 0.6667 (1 044), 0.6000 (606), 0.7500 (450), 0.8073 (670). Worst fp64 case: -0.8922618801772502 / 7.3. - select(pl.col('x') \*\* n) for n = 2 … 9, x in ±\[0.5, 2\]: fp64 max 0.49999, 1.2436, 1.5533, 2.1604, 2.5084, 3.4332, 3.1737, 3.5886 ulp (0, 772, 1 084, 1 264, 1 383, 1 520, 1 589, 1 656 of 3 000 off); fp32 0.4998, 1.2412, 1.6263, 2.2720, 2.4416, 2.9587, 3.5025, 3.7812. Worst fp64: 1.7975193975534105 \*\* 7. - 2026-11-04, source-reading, by characterization pipeline reverse pass - column kernels: add, sub, mul are one IEEE operation per pair (prim_binary_values). - prim_true_div: l / r, one rounding. - 2026-11-04, docs, by characterization pipeline reverse pass - No error guarantee 'unless mentioned otherwise'; no operator documents its rounding. No upstream test pins the ulp error of / by a literal or of x \*\* n. ## polars-ulp-libm: Elementary functions and logarithms: the platform libm per type; logs divide Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: fp64 as the libm (tan 2.3 ulp on macOS); `log10`, `log(b)` 1.5 to 2.1 ulp; `Float32` in single precision, `log()` 1.5 ulp Each function is Rust's `f64` or `f32` method, so `Float64` calls the libm's double function and `Float32` its float one (`sinf`, `expf`, …) and returns `Float32` (polars-elem-rounding). `sqrt` is the hardware instruction, correctly rounded. `log(base)` is Rust's `x.log(base)`, `ln(x) / ln(base)` in the column's type (polars-log-base, PL-4); `log10()` is `log(10)`, and there is no `log2`, so `log(2)` serves. For `Float64` the default base `e` rounds to a double whose `ln` is 1.0, so `log()` is `ln`. Max error in ulps, measured on macOS arm64 (3 000 inputs each), fp64 / fp32: sqrt 0.5 / 0.5; exp 0.502 / 0.511; log() 0.500 / 1.500 (528 of 746 fp32 results off, PL-4); log10() 2.09 / 1.53; log(2) 1.46 / 1.15; log(3) 2.05 / 1.53; log(b) with a column base 2.13 / 1.63; `**` with a column exponent (powf) 0.504 / 0.510; sin 0.77 / 0.77; cos 0.83 / 0.72; tan 2.30 / 0.63; arcsin 0.61 / 0.63; arccos 0.89 / 0.63; arctan 0.72 / 0.65; arctan2 0.56 / 0.68. The fp64 functions return the same values as DuckDB's on every probed input (same libm). Unlike DuckDB, `log10` and `log(2)` divide two logarithms where DuckDB calls `log10` and `log2` (≤ 0.502 ulp), and `Float32` computes in single precision where DuckDB widens to double: the macOS float functions stay below 0.8 ulp but miss the correctly rounded value up to 30 % of the time (arctan 893 of 3 000). Linux x86_64 (glibc, version unknown; the glibc 2.39 manual's bounds): double exp, log, sin, cos, asin, acos, atan, pow 1 ulp, tan and atan2 no known error; float expf, logf, sinf, cosf, tanf, asinf, acosf, atanf, powf 1 ulp, atan2f 2 ulp. Polars's `log10` and `log(b)` add about 1.5 ulp to the libm's `log` error by their formula. To be measured on homelab. Linux x86_64 (glibc 2.39), same 3000 inputs per function: most functions are within about 0.5 ulp where macOS is not (sin 0.50 against 0.77, acos 0.50 against 0.89, tan at moderate arguments 0.51 against 2.4), but `cos` and `tan` of DOUBLE reach 7.95 and 14.36 ulp at x = 5.319372648326541e+255, near a zero of `cos` (glibc gives cos = -4.68716592425462e-19, the exact value is -4.6871659242546276e-19; macOS is within 0.05 ulp there, checked with mpmath at 3000 bits). `log10` reaches 1.35 ulp on Linux (0.50 on macOS) and `log(b, x)` 1.47 to 1.56. So no libm-backed function has a platform-independent bound below glibc's: take the larger of the two columns, and more for huge arguments of `cos` and `tan`. No upstream test pins this; it may change between versions. The reference: The reference computes these with Lean's `Float` (the platform's libm, in double) and fixes `rounding: TIE_TO_EVEN`; the last bits are each engine's (#54). sverify's ε (#85) per function, type and platform comes from this table; `Float32` would round to binary32 in the reference. Options: `rounding`, `fp32`. Rows of the choice tables: `fp.elem` `rounding` (fp32, fp64); `fp.log` `rounding` (fp32, fp64). Constructs: `sqrt`, `exp`, `log`, `log10`, `sin`, `cos`, `tan`, `arcsin`, `arccos`, `arctan`, `arctan2`, `**`. Settings: `{"engine": "streaming", "platform": ["macOS arm64", "Linux x86_64"]}`. Provenance: - 2026-11-04, source-reading, by characterization pipeline round 5 (ulp bounds) - Float32 and Float64 each call the generic body: f32 or f64 methods - log(base): x.log(base) in the column's float type - a non-integer or column exponent: Pow::pow (powf) - f64::log: 'may not be correctly rounded', ln(self) / ln(base) - 2026-11-04, docs, by characterization pipeline round 5 (ulp bounds): For the Linux x86_64 column. The glibc version on a given Linux machine is unknown here (`ldd --version` gives it); the 2.39 table is the last that lists x86_64. macOS's libm documents no error bounds (the SDK's man pages for sin, tan, log, pow state special values only), so the macOS column rests on the probe. - glibc 2.39 manual, table 'Known Maximum Errors in Math Functions', x86_64 column, in ulps: double acos 1, asin 1, atan 1, atan2 -, cos 1, exp 1, log 1, log10 2, log2 2, pow 1, sin 1, sqrt -, tan -; float acosf 1, asinf 1, atanf 1, atan2f 2, cosf 1, expf 1, logf 1, log10f 2, log2f 1, powf 1, sinf 1, sqrtf -, tanf 1. '-' is no known error beyond correct rounding. - The current manual drops the table: glibc aims at results 'within a few ulp' and correct rounding only for sqrt, fma, rint and string conversions; its test suite flags errors above 9 ulp. The numbers are the largest errors glibc's tests have found, not proved bounds. - 2026-11-04, probe, by characterization pipeline round 5 (ulp bounds): Probe script ulp_probe.py (session scratchpad, not in the repository), run from python/ as `uv run --with mpmath python ulp_probe.py --n 3000 --seed 1 --out ulp.json`: about 3 000 inputs per operation and type, random and hard cases, against mpmath at 256 bits. 'bad' counts results that differ from the correctly rounded value; the max is in ulps of the result type at the exact value's binade. DuckDB runs on a table loaded from Parquet, Polars on a LazyFrame with collect(engine='streaming'), so the column kernels run. DuckDB 2.0.0 and Polars 2.1.0 (Python wheels), macOS 26.6 arm64, Python 3.12.11, mpmath 1.3.0. Elementwise kernels are shared by the streaming and in-memory engines (polars-elem-rounding, polars-double-scalar-divisor); the probe collects with engine='streaming' on 3 000-row frames, so morsels hold more than one row and the literal paths run. - pl.LazyFrame({'x': xs}, schema={'x': pl.Float64}).select(f(pl.col('x'))).collect(engine='streaming'): max ulp (off of 3 000): sqrt 0.49998 (0), exp 0.5015 (5), log() 0.5000 (0), log10() 2.0931 (1 735), log(2) 1.4574 (983), sin 0.7687 (108), cos 0.8319 (118), tan 2.3017 (1 252), arcsin 0.6068 (121), arccos 0.8857 (274), arctan 0.7217 (13), arctan2 0.5561 (10), a \*\* b 0.5044 (4), x.log(b) with a column b 2.1326 (1 200), log(10) literal 2.1326 (1 787), log(3) literal 2.0527 (1 625). Worst log10: x = 2.208308239418851e-256. - The same with schema Float32 (results Float32, ulps of binary32): sqrt 0.49996 (0), exp 0.5113 (5 of 1 697), log() 1.4999 (528 of 746), log10() 1.5296 (166), log(2) 1.1541 (167), sin 0.7691 (45 of 2 323), cos 0.7175 (35), tan 0.6323 (126), arcsin 0.6333 (571 of 3 000), arccos 0.6333 (294), arctan 0.6526 (893), arctan2 0.6828 (312), a \*\* b 0.5105 (5 of 1 149), x.log(b) 1.6333 (631 of 2 115), log(10) literal 1.5169, log(3) literal 1.5264. - 2026-11-04, probe, by characterization pipeline platform run (homelab, Linux x86_64, glibc 2.39-0ubuntu8.9; scratchpad ulp_probe.py -n 3000 -seed 1, the same inputs as the macOS run): Same seed and inputs as the macOS measurement. - ulp_probe.py output on homelab: cos fp64 max 7.954598 ulp, tan fp64 max 14.360645 ulp at 5.319372648326541e+255; sin 0.501170, asin 0.501869, acos 0.501590, atan 0.499671, atan2 0.499917, log10 1.347590 (DuckDB), log(b, x) 1.470205 (DuckDB), 2.132601 (Polars) - `python3 -c 'import math; x=5.319372648326541e+255; print(repr(math.cos(x)), repr(math.tan(x)))'` gives `-4.68716592425462e-19 -2.1334853857537075e+18` - 2026-11-04, source-reading, by characterization pipeline reverse pass - Polars py-2.1.0 builds with nightly-2026-09-01, so the std docs to read are the nightly ones, not 1.90.0. - 2026-11-04, docs, by characterization pipeline reverse pass - Rust std: 'Unspecified precision': varies by platform, Rust version and invocation; corresponds to the libc function. The same note is on exp, ln, log, powf, tan, asin, acos, atan, atan2. - Polars gives no error guarantee 'unless mentioned otherwise'. - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_trigonometric` compares with numpy at rel_tol 1e-5; no upstream test pins any function's ulp error. ## polars-ulp-round: `round(d)`: one rounding after the decimal pick Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: result ≤ 0.5 ulp from the decimal it picks; the pick follows x·10^d, off the exact decimal on near-ties `round(d)` for `Float64` is `round_half_even(x · 10^d) / 10^d` (`10^d` by `powi`, exact up to 10^22), for `Float32` the same in `f64` and cast to `f32` (polars-round-digits). The division rounds `n / 10^d` once. Bounds: the fp64 result is within 0.5 ulp of `n / 10^d` (observed max 0.4999 for d up to 12); the `Float32` result within 0.5 ulp of binary32 in the probes, after two roundings. The pick `n` differs from the exact decimal rounding of `x` where `x · 10^d` rounds onto or across a tie: 0.6 to 3.9 % of random decimals with d to d + 4 digits (6 to 39 of 1 000), about half of decimal-text ties (round(39.55, 1) is 39.6 though the double is below 39.55). `Float32` inputs gave no wrong pick. Hardware operations only (and `powi` for d \> 22): platform-independent. The `round` docs' "nearest rounded value" is contradicted by the picks: PL-9. No upstream test pins this; it may change between versions. The reference: The reference rounds with DuckDB's formula, `TIE_AWAY_FROM_ZERO` (#335); Polars's default mode is half to even (polars-round, polars-round-digits). Only the final division's 0.5 ulp is rounding error in the \#85 sense. Options: `rounding`, `fp32`. Rows of the choice tables: `fp.round` `rounding` (fp32, fp64). Constructs: `round`. Settings: `{"engine": "streaming", "platform": "macOS arm64"}`. See: upstream ledger [PL-9](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-9). Provenance: - 2026-11-04, source-reading, by characterization pipeline round 5 (ulp bounds) - Float32: computed in f64, cast to f32; Float64: f64_op(val \* multiplier) / multiplier - 2026-11-04, probe, by characterization pipeline round 5 (ulp bounds): Probe script ulp_probe.py (session scratchpad, not in the repository), run from python/ as `uv run --with mpmath python ulp_probe.py --n 3000 --seed 1 --out ulp.json`: about 3 000 inputs per operation and type, random and hard cases, against mpmath at 256 bits. 'bad' counts results that differ from the correctly rounded value; the max is in ulps of the result type at the exact value's binade. DuckDB runs on a table loaded from Parquet, Polars on a LazyFrame with collect(engine='streaming'), so the column kernels run. DuckDB 2.0.0 and Polars 2.1.0 (Python wheels), macOS 26.6 arm64, Python 3.12.11, mpmath 1.3.0. Elementwise kernels are shared by the streaming and in-memory engines (polars-elem-rounding, polars-double-scalar-divisor); the probe collects with engine='streaming' on 3 000-row frames, so morsels hold more than one row and the literal paths run. - select(pl.col('x').round(d)) for d in 0, 1, 2, 3, 5, 8, 12 on the inputs of duckdb-ulp-round (Float64): final rounding ≤ 0.4000, 0.4800, 0.4960, 0.4998, 0.4998, 0.4999 ulp (d = 1 … 12); wrong pick 0, 697, 827, 864, 831, 771, 940 of 3 000 (0, 7, 12, 16, 10, 6, 39 of the 1 000 random decimals). - The same on Float32: ≤ 0.5 ulp of binary32 from the exact decimal rounding, 0 wrong picks for every d. - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_round` uses rel_tol 1e-5: (1835.665, 2) expects 1835.67 and py-2.1.0 returns 1835.66 in the default mode; the wrong-pick rate is not pinned by any upstream test. - 2026-11-04, docs, by characterization pipeline reverse pass - 'nearest rounded value' wording contradicted by the x \* 10^d pick (PL-D2). ## polars-zero-add: `x + 0.0` and `0.0 - x` keep or flip a zero's sign Engine: polars py-2.1.0; area: doubles; status: shimmed; evidence: probe. Verdict: `zero_sign`: not IEEE either, so up to one zero ([canon](../reference/sql.Dialect.md#sverify.sql.Dialect.canon)) `-0.0 + 0.0` is `-0.0` and `0.0 - 0.0` (as `0.0 - x`) is `-0.0`, where IEEE 754 gives `+0.0` for both under round-to-nearest; `-0.0 - 0.0` is `-0.0`, as IEEE gives. The sign of a zero result of `+` and `-` is therefore Polars's own (polars-zero-sign). The same finding as polars-zero-sign-streaming (which supersedes polars-zero-sign): the literal shortcuts `x + 0.0 → x` and `0.0 - x → -x` run on both engines once a morsel holds more than one row; on streaming frames under about 100 rows (one-row morsels) the IEEE zero comes back (`s-zero-sign-literal`). This fact keeps its id for the engine tables that cite it. The reference: One zero (`zero_sign: ONE_ZERO`, \#89): `-0.0` is `0.0` everywhere, so the translation canonicalises (`one_zero`) where a result's zeros are observed. Options: `zero_sign`. Constructs: `+`, `-`. Provenance: - 2026-11-04, probe, by Claude session frosty-bassi-b2cf7d (#444 step 3a, PR \#590): Polars 2.1.0 (Python) on macOS arm64. - `pl.DataFrame({'x': [-0.0, -0.0]}).select(pl.col('x') + 0.0)` gives `[-0.0, -0.0]` - `pl.DataFrame({'x': [0.0, 0.0]}).select(pl.lit(0.0) - pl.col('x'))` gives `[-0.0, -0.0]` - `pl.DataFrame({'x': [-0.0, -0.0]}).select(pl.col('x') - 0.0)` gives `[-0.0, -0.0]` - 2026-11-04, probe, by characterization pipeline (merging duplicates after \#636): The same finding as polars-zero-sign-streaming (which supersedes polars-zero-sign): the literal shortcuts `x + 0.0 → x` and `0.0 - x → -x` run on both engines once a morsel holds more than one row; on streaming frames under about 100 rows (one-row morsels) the IEEE zero comes back (`s-zero-sign-literal`). This fact keeps its id for the engine tables that cite it. - probe `s-zero-sign-literal` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-zero-order: `-0.0` equals `0.0` in comparisons, sorts, grouping and joins Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: one zero for equality and order; which zero a group or [unique](../reference/unique.md#sverify.unique) keeps depends on `maintain_order` `0.0 == -0.0` is true, `sort` treats them as equal (with `maintain_order` the input order stays), and `group_by`, [unique](../reference/unique.md#sverify.unique) and joins put them together. Without `maintain_order`, the group's key and the surviving row of [unique](../reference/unique.md#sverify.unique) carry the zero seen first, which can be `-0.0`; with `group_by(maintain_order=True)` the key was `0.0` in a probe where `-0.0` came first. Polars promises neither. Arithmetic still tells the zeros apart (polars-zero-sign). No upstream test pins which zero a group key or [unique](../reference/unique.md#sverify.unique) keeps, or zero grouping on the streaming engine; it may change between versions. The reference: `ONE_ZERO` (#89): `-0.0` is `0.0` everywhere, so a group key is never `-0.0`. Options: `zero_sign`, `nan_zero_grouping`. Rows of the choice tables: `cmp` `zero_sign` (fp64); `cmp` `zero_sign` (fp32; evidence engine_test). Constructs: `==`, `sort`, `group_by`, [unique](../reference/unique.md#sverify.unique), `join`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-4](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-4); issues [\#89](https://github.com/AlexanderNenninger/substrait-lean/issues/89). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `nan-compare` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `sort-zero-sign` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `group-nan-zero` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `unique-nan-zero` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `join-nan-zero-null` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, docs, by characterization pipeline round 1 (doubles): Polars's user guide says it generally follows IEEE 754 for `Float32` and `Float64`, with two exceptions: any NaN equals any other NaN and is greater than every non-NaN value, and operations guarantee nothing about the sign of zero or NaN or a NaN's payload (a sort or group by may make every zero +0 and every NaN a positive NaN). It promises no error bound unless a function says otherwise. The user guide is not versioned; the section reads the same at tag py-2.1.0. Comparisons are not among the exceptions, so `-0.0 == 0.0` holds in `Float32` as IEEE 754 compares them, and a sort or group by may make every zero +0, which puts them together. Settles `cmp`'s `Float32` row at docs level. The docs do not promise which zero a group key carries. - User guide, Floating point numbers (stable). - The same section at tag py-2.1.0. - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): Equality, order, sorting and hashing of floats are one generic implementation for `f32` and `f64` (and `pf16`), so `Float32` behaves as the `Float64` probes show. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - `impl_float_eq_ord!` for pf16, f32, f64: NaN equals NaN and is the largest value. - `impl_float_total_ord_kernel!` for f32 and f64: the column comparison kernels. - `canonical_f32` and `canonical_f64`: `x + 0.0`, every NaN one quiet NaN, for hashing (group by, unique, joins). - - Sorting uses `TotalOrd::tot_cmp`. - 2026-11-04, engine-test, by characterization pipeline round 3 (doubles): Pins `0.0 == -0.0` (and neither `<` nor `>`) for `Float32`, in comparisons, `is_in` and `index_of`; the streaming sort of a key with both zeros equals the in-memory one with `maintain_order=True`. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_total_ordering_float_series` (slow): every pair of 0.0, -0.0, ±1.0, ±NaN, ±inf, NULL as `Float32` against `Float32` and `Float64` against `Float32`, column against column and against a broadcast first value or literal; `==`, `!=`, `<`, `<=`, `>`, `>=`, `eq_missing`, `ne_missing` must follow `reference_ordering_propagating`: NaN equals NaN (also -NaN), NaN is above every number including inf, 0.0 equals -0.0, NULL propagates. Eager `DataFrame.select`, in memory - `test_is_in_float`, `Float32` and `Float64`: NaN `is_in([-0.0, -NaN])` and 0.0 `is_in([-0.0, -NaN])` are both true (eager) - `test_float`, `Float32` and `Float64`: `index_of(-NaN)` finds NaN and `index_of(-0.0)` finds 0.0, on the series, its ascending and descending sorts and a chunked copy (eager and lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect)) - `test_sort_direct_key_dtypes` with the `f32` key (517 rows with NaN, ±inf, -0.0, NULL), ascending and descending, `maintain_order=True`: the streaming sort must equal the in-memory sort exactly, with the in-memory and the partitioned (`POLARS_SORT_PARTITION_THRESHOLD_BYTES=1`, morsels of 7) sort modes - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. s-zero-key-first groups and dedups 100 000 zeros, `-0.0` first, lazily on each engine: `group_by` without `maintain_order`, [unique()](../reference/unique.md#sverify.unique) and `unique(maintain_order=True)` keep `-0.0`, the zero seen first; `group_by(maintain_order=True)` gives the key `0.0`, on both engines. That is not the zero seen first, which the description says; reported for review, description unchanged. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. join-nan-zero-null's recorded differ 1 is a tie order: `sort('y')` ties the rows 0.0 and -0.0 (both y = 2), and the engines list them in different orders. The join results are the same set of rows on both engines; it is no engine difference. - probe `nan-compare` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `sort-zero-sign` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `group-nan-zero` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `unique-nan-zero` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `join-nan-zero-null` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `s-zero-key-first` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Corrects the verdict and description after s-zero-key-first, on both engines: `group_by` without `maintain_order` and [unique](../reference/unique.md#sverify.unique) (with and without `maintain_order`) keep `-0.0`, the zero seen first, but `group_by(maintain_order=True)` gives the key `0.0`. The description had said the key is always the zero seen first. - probe `s-zero-key-first` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_group_by_float`: -0.0 and 0.0 land in one group (and -nan with nan), keys x and (x, a), maintain_order True and False, eager, in memory. - `test_unique`, `test_unique_counts`: \[-0.0, -nan, 0.0, None, 1.0, nan\] has 4 unique values, counts \[2, 2, 1, 1\]; the expected 0.0 does not pin the sign (assert_series_equal holds -0.0 == 0.0). - `test_joins`: -0.0 joins 0.0, NaN joins NaN, eager. No upstream test pins which zero a group key or unique row keeps, nor zero grouping on the streaming engine. ## polars-zero-sign: `-0.0` in arithmetic: two zeros Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: IEEE: `1 / -0.0` is -inf Arithmetic keeps the sign of zero: `1.0 / -0.0` is -inf, `-(0.0)` and `0.0 * -1` are -0.0, `sqrt(-0.0)` is -0.0. Comparison, grouping and sorting are in polars-zero-order. With a literal operand on a frame of two rows or more, the kernels take shortcuts that move the sign of a zero away from IEEE: `x + 0.0` and `x - 0.0` return `x`, so `-0.0 + 0.0` and `-0.0 - (-0.0)` are -0.0 (IEEE gives 0.0), and `0.0 - x` is `-x`, so `0.0 - 0.0` is -0.0. Two columns use the IEEE operations. The reference: One zero (`zero_sign: ONE_ZERO`, \#89): `-0.0` is `0.0` everywhere. Options: `zero_sign`. Rows of the choice tables: `fp.arith` `zero_sign` (fp64); `fp.neg` `zero_sign` (fp64); `fp.div` `zero_sign` (fp64); `fp.arith` `zero_sign` (fp32; evidence source); `fp.neg` `zero_sign` (fp32; evidence source); `fp.div` `zero_sign` (fp32; evidence source). Constructs: `/`, `unary -`, `*`, `sqrt`. Settings: `{"engine": "in-memory"}`. See: upstream ledger [ST-4](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-4); issues [\#89](https://github.com/AlexanderNenninger/substrait-lean/issues/89); superseded by [polars-zero-sign-streaming](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-facts.md#polars-zero-sign-streaming). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `zero-sign-arith` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, docs, by characterization pipeline round 1 (doubles): Polars's user guide says it generally follows IEEE 754 for `Float32` and `Float64`, with two exceptions: any NaN equals any other NaN and is greater than every non-NaN value, and operations guarantee nothing about the sign of zero or NaN or a NaN's payload (a sort or group by may make every zero +0 and every NaN a positive NaN). It promises no error bound unless a function says otherwise. The user guide is not versioned; the section reads the same at tag py-2.1.0. So the two zeros the probes show are what Polars 2.1.0 happens to do: the docs do not promise them, and a sort or group by may turn `-0.0` into `0.0`. The `Float32` rows stay open. - User guide, Floating point numbers (stable). - The same section at tag py-2.1.0. - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): `impl_float_arith_kernel!` is one macro instantiated for `pf16`, `f32` and `f64`, so `Float32` and `Float64` run the same operations. The streaming engine evaluates expressions through polars-expr on each morsel (`StreamExpr` in crates/polars-stream/src/expression.rs), so it shares these kernels with the in-memory engine. The column kernels are the IEEE operations, so `Float32` keeps two zeros as `Float64` does. The kernels for a literal operand take shortcuts that move the sign of a zero away from IEEE, at both widths: `x + 0.0` and `x - 0.0` return `x` unchanged, so `-0.0 + 0.0` and `-0.0 - (-0.0)` are -0.0 (IEEE gives 0.0), and `0.0 - x` is `-x`, so `0.0 - 0.0` is -0.0. A literal is a length-1 column, so the shortcut runs on frames of two rows or more; on one row both sides have length 1 and the column kernel runs. Read at tag py-2.1.0 (commit 7f902bd3). The streaming engine evaluates expressions through polars-expr on each morsel (crates/polars-stream/src/expression.rs), so these kernels serve both engines. - The column kernels: `-x`, `l + r`, `l - r`, `l * r`, `l / r`. - The literal kernels: `x + 0` and `x - 0` return `x`, `0 - x` is `-x`, `x * ±1` is `x` or `-x`. - - `+ - * // %` on two ChunkedArrays: equal lengths take the array kernel, a length-1 side the scalar kernel. - - A literal is a scalar column, passed to the operation as a one-value Series. - The streaming engine evaluates each expression with polars-expr's `PhysicalExpr` on every morsel, so it runs the same kernels. - 2026-11-04, manual-repro, by characterization pipeline round 2 (doubles): Confirms the literal shortcuts on `Float64`. - pl.DataFrame({'x': \[49.0, 3.0, -0.0, 0.0, 0.5\]}).select(a=pl.col('x') + 0.0, s=pl.col('x') - (-0.0), l=pl.lit(0.0) - pl.col('x')) gave a = -0.0 and s = -0.0 for x = -0.0, and l = -0.0 for x = 0.0 (Polars 2.1.0, macOS arm64). - 2026-11-04, source-reading, by characterization pipeline round 2 (doubles): Adds the second paragraph of the description: the literal shortcuts for `x + 0`, `x - 0` and `0 - x`, read in the source and seen in the manual repro of this round. Read at tag py-2.1.0 (commit 7f902bd3). - - - - - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): In memory, s-zero-sign-literal shows the shortcuts on 4 rows and on 6000. Streaming takes them only in morsels of two rows or more: polars-zero-sign-streaming, which supersedes this fact for the streaming engine. - probe `s-zero-sign-literal` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-zero-sign-streaming: `-0.0` in arithmetic on the streaming engine: two zeros, literal shortcuts per morsel Engine: polars py-2.1.0; area: doubles; status: open; evidence: probe. Verdict: IEEE: `1 / -0.0` is -inf; with a literal operand the sign depends on the morsel's length Arithmetic keeps the sign of zero: `1.0 / -0.0` is -inf, `-(0.0)` and `0.0 * -1` are -0.0, `sqrt(-0.0)` is -0.0. Comparison, grouping and sorting are in polars-zero-order. The streaming engine runs the in-memory kernels on each morsel. With a literal operand, a morsel of two rows or more takes the literal kernels' shortcuts: `x + 0.0` and `x - 0.0` return `x`, so `-0.0 + 0.0` and `-0.0 - (-0.0)` are -0.0, and `0.0 - x` is `-x`, so `0.0 - 0.0` is -0.0. A morsel of one row takes the IEEE operations (0.0 in those three cases). Which rows land in a one-row morsel depends on the frame's size and the machine's thread count: on 10 cores every row of a frame of up to 10 rows did, so `-0.0 + 0.0` was 0.0 there and -0.0 in memory; from about 100 rows none did. The in-memory engine's behaviour is in polars-zero-sign. polars-zero-add records the literal-shortcut part of this finding from \#590's probe and is the id the engine tables cite. No upstream test pins this; it may change between versions. The reference: One zero (`zero_sign: ONE_ZERO`, \#89): `-0.0` is `0.0` everywhere. Options: `zero_sign`. Rows of the choice tables: `fp.arith` `zero_sign` (fp64); `fp.neg` `zero_sign` (fp64); `fp.div` `zero_sign` (fp64); `fp.arith` `zero_sign` (fp32; evidence source); `fp.neg` `zero_sign` (fp32; evidence source); `fp.div` `zero_sign` (fp32; evidence source). Constructs: `/`, `unary -`, `*`, `sqrt`, `+`, `-`. Settings: `{"engine": "streaming"}`. See: issues [\#89](https://github.com/AlexanderNenninger/substrait-lean/issues/89); supersedes [polars-zero-sign](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-facts.md#polars-zero-sign). Provenance: - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): zero-sign-arith (`1 / z`, `-z`, `z * -1`, `sqrt` on -0.0 and 0.0) gives the same on both engines; for these the shortcuts and IEEE agree. s-zero-sign-literal counts the negative zeros of `x + 0.0`, `x - (-0.0)`, `x - 0.0` and `0.0 - x` over x = -0.0, 0.0, … : on 6000 rows 3000 in each column on both engines (the shortcuts); on 4 rows in memory 2 in each (the shortcuts), streaming 0, 0, 2, 0 (IEEE). A manual repro of `x + 0.0` on n copies of -0.0 under streaming gave IEEE's 0.0 in every row for n up to 10 and the shortcut's -0.0 in all rows from n = 100; between them one row of some sizes was 0.0 (n = 11: 10 of 11 rows -0.0). In memory every size gave -0.0. Streaming splits a small frame into several morsels on this 10-core machine, and a one-row morsel takes the column kernel, as for the scalar divisor (polars-double-scalar-divisor, PL-3). `python -m sverify_dev.probe_polars --diff --write` collects each `s-` observation's lazy query with `engine='in-memory'` and `engine='streaming'`. Polars 2.1.0 (Python), macOS arm64. - probe `zero-sign-arith` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `s-zero-sign-literal` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline round 5 (streaming engine): Carried over from polars-zero-sign's round-2 reading; the morsel's length decides between the kernels, which the probe confirms. `impl_float_arith_kernel!` serves `f32` and `f64` alike, so the `Float32` rows rest on the source. - The literal kernels: `x + 0` and `x - 0` return `x`, `0 - x` is `-x`, `x * ±1` is `x` or `-x`. - Equal lengths take the array kernel, a length-1 side the scalar kernel. - The streaming engine evaluates each expression with polars-expr's `PhysicalExpr` on every morsel. - 2026-11-04, probe, by characterization pipeline (merging duplicates after \#636): polars-zero-add records the literal-shortcut part of this finding from \#590's probe and is the id the engine tables cite. - probe `s-zero-sign-literal` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, docs, by characterization pipeline reverse pass - 'Operations do not guarantee any particular behavior on the sign of zero' (SRC docs/source/user-guide/concepts/data-types-and-structures.md L180-L184). No upstream test of the literal kernels' zero shortcuts or of the sign of a zero result. # strings ## polars-case: `to_uppercase`, `to_lowercase`: Rust's Unicode mappings Engine: polars py-2.1.0; area: strings; status: no-finding; evidence: probe. Verdict: `char_set: UTF8`, Rust's `str::to_uppercase` Case mapping is Rust's `str::to_uppercase` and `to_lowercase`: full Unicode mappings, which can change the length (`ß` is `SS`, `İ` lowercases to `i̇`), with a final sigma (`ΣΑΣ` lowercases to `σας`). On ASCII they agree with the reference on every probe row. The reference: `upper`, `lower` with `char_set: UTF8`, a black box defined as Rust's `str::to_uppercase` (`Text.upper`); Lean evaluates them on ASCII only. Options: `char_set`. Rows of the choice tables: `str.case` `char_set` (string). Constructs: `str.to_uppercase`, `str.to_lowercase`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-10](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-10). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `String.str.to_uppercase()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.to_lowercase()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `case-unicode` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `String.str.to_uppercase()` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.to_lowercase()` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `case-unicode` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-concat: `concat_str`: NULL in, NULL out; `ignore_nulls=True` skips them Engine: polars py-2.1.0; area: strings; status: no-finding; evidence: probe. Verdict: `null_handling: ACCEPT_NULLS` by default; `IGNORE_NULLS` with the flag `pl.concat_str(a, b)` and `a + b` are NULL when an input is NULL; `concat_str(..., ignore_nulls=True)` skips NULLs. The reference: `str_concat` with `null_handling: ACCEPT_NULLS` (NULL in, NULL out); `IGNORE_NULLS` refused. Options: `null_handling`. Rows of the choice tables: `str.concat` `null_handling` (string). Constructs: `concat_str`, `+`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `pl.concat_str(String, String)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `concat-nulls` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `pl.concat_str(String, String)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `concat-nulls` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-length: `len_chars` counts code points; `len_bytes` bytes Engine: polars py-2.1.0; area: strings; status: no-finding; evidence: probe. Verdict: `character_length` natively `len_chars` counts code points (`e` with a combining accent is 2), as `character_length` does; `len_bytes` counts UTF-8 bytes. Both return `UInt32`. The reference: `character_length` counts code points. Options: `char_set`. Constructs: `str.len_chars`, `str.len_bytes`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `String.str.len_chars()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `len-chars-bytes` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `String.str.len_chars()` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `len-chars-bytes` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-match-functions: `starts_with`, `ends_with`, literal `replace_all` Engine: polars py-2.1.0; area: strings; status: no-finding; evidence: probe. Verdict: native, case-sensitive `starts_with`, `ends_with`, `contains(literal=True)` and `replace_all(literal=True)` are case-sensitive and agree with the reference on every probe row (the empty prefix matches everything). Case-insensitive matching is a regex flag, `(?i)`. The reference: `starts_with`, `ends_with`, [replace](../reference/sql.Dialect.md#sverify.sql.Dialect.replace) with `case_sensitivity: CASE_SENSITIVE`. Options: `case_sensitivity`. Rows of the choice tables: `str.match` `case_sensitivity` (string). Constructs: `str.starts_with`, `str.ends_with`, `str.contains`, `str.replace_all`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `String.str.starts_with('a')` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.starts_with('')` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.ends_with('a')` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.ends_with('')` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.replace_all('a', 'xy', literal=True)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `String.str.starts_with('a')` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.starts_with('')` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.ends_with('a')` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.ends_with('')` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.replace_all('a', 'xy', literal=True)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-regex: Regexes are Rust's `regex` crate: no backreferences or lookaround, Unicode classes Engine: polars py-2.1.0; area: strings; status: open; evidence: probe. Verdict: `regex` crate syntax; `extract` is NULL without a match; `$1` in replacements Patterns are the Rust `regex` crate's: no lookaround and no backreferences (both raise `ComputeError` when the query runs), `\d` and `\w` are Unicode (`\d` matches `٣`), POSIX classes are ASCII. An invalid pattern raises. `str.extract(p, g)` is NULL where the pattern does not match or the group did not take part; DuckDB's `regexp_extract` gives `''` there. `replace_all` replaces every match, including empty ones (`b*` in `aaa` gives `-a-a-a-`), and its replacement names groups as `$1` or `${1}`; `\1` is literal. [replace](../reference/sql.Dialect.md#sverify.sql.Dialect.replace) replaces the first match. On ASCII, `replace_all` agrees with `regexp_replace(…, 'g')` on the probe's patterns, and `extract` with `regexp_extract` wherever the pattern matches. No upstream test pins the refusal of lookaround and backreferences, Unicode `\d` or empty-match replacement; they may change between versions. The reference: `regexp_extract`, `regexp_replace` with a constant pattern (#50), RE2's syntax, which has no backreferences or lookaround either; a pattern that does not match extracts `''`. Lean evaluates the classes on ASCII only (`Text.classesDiffer`). Options: `regex_dialect`, `case_sensitivity`. Constructs: `str.contains`, `str.extract`, `str.replace`, `str.replace_all`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: issues [\#50](https://github.com/AlexanderNenninger/substrait-lean/issues/50). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `String.str.extract('^(a+)(b)?', 2)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.extract('(a|ab)(c|bcd)?', 0)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.extract('(a*)*', 1)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.extract('x(?:a)(y)', 1)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.extract('.$', 0)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.extract('(b)', 1)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.replace_all('a+', '-')` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.replace_all('b*', '-')` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.replace_all('$', '!')` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `regex-syntax` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `regex-invalid` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `regex-replace-refs` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `regex-extract-nomatch` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `String.str.extract('^(a+)(b)?', 2)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.extract('(a|ab)(c|bcd)?', 0)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.extract('(a*)*', 1)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.extract('x(?:a)(y)', 1)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.extract('.$', 0)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.extract('(b)', 1)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.replace_all('a+', '-')` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.replace_all('b*', '-')` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.replace_all('$', '!')` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `regex-syntax` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `regex-invalid` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `regex-replace-refs` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `regex-extract-nomatch` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass - patterns compile with regex::RegexBuilder (regex 1.13.1) - extract: NULL on no match or a group that did not take part - replace_all: regex_replacen with n=0 (\$1 / \${1} groups, empty matches replaced) - 2026-11-04, docs, by characterization pipeline reverse pass - regex crate syntax; null if the regex captures nothing - \$ names a capture group; \$\$ for a literal \$ - 2026-11-04, engine-test, by characterization pipeline reverse pass - extract: no match → None (in-memory) - replace_all: \$1 substitution; invalid regex raises ComputeError (in-memory) ## polars-reverse: `str.reverse` reverses grapheme clusters Engine: polars py-2.1.0; area: strings; status: open; evidence: probe. Verdict: graphemes, where the reference reverses code points `str.reverse` reverses extended grapheme clusters (`unicode_reverse`): a letter keeps its combining accent (`e\u0301x` becomes `xe\u0301`) and a flag's two regional indicators stay together. It agrees with the reference on ASCII and on precomposed text. The reference: `reverse` reverses the code points (`String.toList.reverse`), so a combining accent moves to the other side of its letter. Options: `char_set`. Constructs: `str.reverse`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `String.str.reverse()` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `reverse-graphemes` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `String.str.reverse()` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `reverse-graphemes` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, docs, by characterization pipeline reverse pass - example: 'man303ana' reverses to 'anañam', the combining tilde stays on its n - 2026-11-04, engine-test, by characterization pipeline reverse pass - test_string_reverse: 'man303ana' → 'anan303am' (grapheme clusters, in-memory) ## polars-string-functions-missing: No `levenshtein`, digests, [repeat](../reference/sql.Dialect.md#sverify.sql.Dialect.repeat), `ascii` or `LIKE` on expressions Engine: polars py-2.1.0; area: strings; status: open; evidence: probe. Verdict: missing from `pl.Expr.str` `pl.Expr.str` has no `levenshtein`, `md5`, `sha256`, [repeat](../reference/sql.Dialect.md#sverify.sql.Dialect.repeat), `ascii`, [like](../reference/sql.Dialect.md#sverify.sql.Dialect.like) or [ilike](../reference/sql.Dialect.md#sverify.sql.Dialect.ilike). `LIKE` exists in Polars's SQL interface; through expressions it is a regex (`str.contains`) with the pattern translated. `hash` is Polars's own hash, unrelated to the digests. The docs are silent on SQL `LIKE` running as a regex: a docs gap. The reference: Each is a Substrait function (or sverify's black box) the reference evaluates. Options: [digests](../reference/io.Contract.md#sverify.io.Contract.digests). Constructs: `levenshtein`, `md5`, `sha256`, [repeat](../reference/sql.Dialect.md#sverify.sql.Dialect.repeat), `ascii`, [like](../reference/sql.Dialect.md#sverify.sql.Dialect.like), [ilike](../reference/sql.Dialect.md#sverify.sql.Dialect.ilike). Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `string-functions-missing` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observation reads the Python API (`hasattr(pl.Expr.str, …)`), which both engines share. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `string-functions-missing` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass - the str namespace defines no levenshtein, md5, sha256, repeat, ascii, like or ilike - SQL LIKE/ILIKE: literal fast path, else regex (?s)/(?is) with % → .*, \_ → ., via str.contains - 2026-11-04, engine-test, by characterization pipeline reverse pass - test_string_like: LIKE/ILIKE/~~/~~* patterns, regex metacharacters literal ## polars-string-order: Strings sort by UTF-8 bytes; `False` before `True` Engine: polars py-2.1.0; area: strings; status: no-finding; evidence: probe. Verdict: code point order, no collation Strings compare and sort by their UTF-8 bytes, which is code point order: `''` \< `B` \< `Z` \< `a` \< `b` \< `é`. Booleans sort `False` before `True`. NULLs first by default (polars-null-order). The reference: Strings in code point order (Lean's `String` order); booleans `false < true`. Constructs: `sort`, `<`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `sort-strings-bools` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `sort-strings-bools` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-strpos: `str.find`: a 0-based byte offset, NULL when absent Engine: polars py-2.1.0; area: strings; status: open; evidence: probe. Verdict: byte offset from 0; NULL where `strpos` is 0 `str.find(t)` returns the byte offset of the first match, from 0, or NULL when there is none: `find('héllo', 'l')` is 3 (the character position is 2), and `find('', 'a')` is NULL. The empty pattern is found at 0. No upstream test pins byte offsets on non-ASCII text; it may change between versions. The reference: `strpos(s, t)` is the 1-based character position of the first match, 0 when there is none (#209). A writer adds 1 and maps NULL to 0, and on text that is not ASCII it needs the character position (the bytes before the match are not the characters). Options: `negative_start`, `string_positions`. Rows of the choice tables: `str.match` `string_positions` (string). Constructs: `str.find`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `String.str.find('a', literal=True)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.find('b', literal=True)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.find('', literal=True)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `find-bytes` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `String.str.find('a', literal=True)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.find('b', literal=True)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.find('', literal=True)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `find-bytes` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, docs, by characterization pipeline reverse pass - 'Return the bytes offset of the first substring matching a pattern'; None when not found - 2026-11-04, engine-test, by characterization pipeline reverse pass - test_str_find: 0-based offsets and None on no match, ASCII only (in-memory); bytes vs characters not pinned - 2026-11-04, probe, by characterization pipeline reverse pass - LazyFrame({'s':\['héllo'\]}).select(str.find('l', literal=True)) on in-memory and streaming: 3 ## polars-substring: `str.slice`: 0-based, negative offsets from the end Engine: polars py-2.1.0; area: strings; status: no-finding; evidence: probe. Verdict: [substring](../reference/sql.Dialect.md#sverify.sql.Dialect.substring) with `negative_start: WRAP_FROM_END`, offset - 1 `str.slice(offset, length)` counts characters (code points) from 0; a negative offset counts from the end, and the part before the start is cut off (`slice(-10, 7)` of `hello` is `he`). An offset past the end gives `''`. A length of NULL takes the rest; a negative length is refused (`InvalidOperationError`, cast to `u64`). With the offset shifted by one for positive starts, it matches Substrait's [substring](../reference/sql.Dialect.md#sverify.sql.Dialect.substring) with `WRAP_FROM_END` on every probe row. The reference: `substring(s, start, length)`, 1-based, `negative_start: WRAP_FROM_END` (#209); offsets outside `[-2³², 2³²)` raise (#336). Options: `negative_start`, `string_positions`. Rows of the choice tables: `str.sub` `negative_start` (string); `str.sub` `string_positions` (string). Constructs: `str.slice`, `str.head`, `str.tail`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: issues [\#209](https://github.com/AlexanderNenninger/substrait-lean/issues/209), [\#336](https://github.com/AlexanderNenninger/substrait-lean/issues/336). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `String.str.slice(0, 2)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.slice(1, 2)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.slice(-2, 5)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.slice(-10, 3)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `String.str.slice(5, 1)` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `slice` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `slice-negative-length` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `String.str.slice(0, 2)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.slice(1, 2)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.slice(-2, 5)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.slice(-10, 3)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `String.str.slice(5, 1)` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `slice` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `slice-negative-length` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-substring-range: `str.slice` takes any 64-bit offset Engine: polars py-2.1.0; area: strings; status: open; evidence: probe. Verdict: computes for offsets past 32 bits; a negative length raises No offset makes `str.slice` raise: an offset of 2⁴⁰ gives `''`, and `slice(-2⁴⁰, 2⁴¹)` the whole string. A negative length raises (`InvalidOperationError`): the length is cast to `UInt64` strictly. The docs are silent on out-of-range offsets and a negative length raising: a docs gap. No upstream test pins offsets as large as 2⁴⁰; it may change between versions. The reference: `substring_range`: offsets outside `[-2³², 2³²)` raise (#336), as DuckDB does (duckdb-substring-range). Options: `substring_range`. Rows of the choice tables: `str.sub` `substring_range` (string). Constructs: `str.slice`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: issues [\#336](https://github.com/AlexanderNenninger/substrait-lean/issues/336). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `substring-huge` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `substring-huge` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass: Corrected the description: it said no offset or length makes `str.slice` raise. A negative length raises (namespace.rs#L915-L922, pinned by test_string.py#L59-L61); the claim holds for offsets. - offset cast to Int64; length strict-cast to UInt64, so a negative length raises - offset \>= len gives ''; a negative offset before the start is clamped with saturating arithmetic; never raises - 2026-11-04, engine-test, by characterization pipeline reverse pass - test_str_slice_expr: negative offsets; str.slice(0, -1) raises InvalidOperationError (in-memory) - test_str_slice_multibyte: offsets outside the string clip (in-memory) - 2026-11-04, probe, by characterization pipeline reverse pass - LazyFrame({'s':\['abc'\]}).select(str.slice(0, -1)) raises InvalidOperationError (conversion from i32 to u64 failed) on in-memory and streaming ## polars-trim: `strip_chars()` trims Unicode whitespace; with characters, those Engine: polars py-2.1.0; area: strings; status: open; evidence: probe. Verdict: `btrim(s, ' ')` natively; the default trims all whitespace `strip_chars()` with no argument is Rust's `str::trim`, every Unicode whitespace character (tab, newline, no-break space, em space). With characters it trims those: `strip_chars(' ')` is `btrim(s, ' ')` on every probe row. The docs are silent on whitespace meaning Unicode's `White_Space`: a docs gap. No upstream test pins non-ASCII whitespace; it may change between versions. The reference: `btrim(s)` trims spaces only; `btrim(s, chars)` the given characters. Constructs: `str.strip_chars`, `str.strip_chars_start`, `str.strip_chars_end`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `String.str.strip_chars(' ')` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `strip-whitespace` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `String.str.strip_chars(' ')` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `strip-whitespace` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, docs, by characterization pipeline reverse pass - None strips 'all leading and trailing whitespace'; silent on Unicode whitespace (docs gap) - str::trim: Unicode White_Space - 2026-11-04, engine-test, by characterization pipeline reverse pass - test_str_strip_chars: tab and space stripped; character sets (ASCII only, in-memory) # temporal ## polars-date-parts: Date parts: ISO weeks, weekday 1-7, floor before 1970 Engine: polars py-2.1.0; area: temporal; status: no-finding; evidence: probe. Verdict: ISO week and year; Monday is 1; parts and epoch floor `week` is the ISO week and `iso_year` its year (2024-12-30 is week 1 of 2025; 2021-01-03 is week 53 of 2020); `weekday` runs from Monday 1 to Sunday 7; `millisecond` and `microsecond` are the fraction of the second in those units. Before 1970 parts floor: 1969-12-31 23:59:59.5 has `second` 59, `microsecond` 500000, `epoch('s')` -1, and `truncate('1s')` goes down to 23:59:59. `truncate('1w')` goes to Monday, `'1mo'`, `'1q'`, `'1y'` to the first day. The reference: [date_part](../reference/sql.Dialect.md#sverify.sql.Dialect.date_part) and [date_trunc](../reference/sql.Dialect.md#sverify.sql.Dialect.date_trunc) (`Time.lean`): ISO weeks, Monday-based `isodow`, flooring before the epoch; their range is open in the reference. Options: `week_numbering`. Constructs: `dt.year`, `dt.iso_year`, `dt.week`, `dt.weekday`, `dt.ordinal_day`, `dt.quarter`, `dt.epoch`, `dt.second`, `dt.truncate`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `date-parts` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `before-epoch` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `truncate-units` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `date-diff` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `date-parts` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `before-epoch` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `truncate-units` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `date-diff` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-date-range: Dates past 9999 and before year 1; out-of-range days panic Engine: polars py-2.1.0; area: temporal; status: open; evidence: probe. Verdict: no `date_range` limit; `Int32` days beyond chrono's range panic Date arithmetic is not limited to 0001-01-01 … 9999-12-31: one day before 0001-01-01 is 0000-12-31 and one after 9999-12-31 is +10000-01-01; timestamps (microseconds) reach year 12736 and beyond. A date is an `Int32` of days, wider than chrono's `NaiveDate` (about ±262 000 years). Casting such a day count to `Date` succeeds, `dt.year()` of it is NULL, and turning it into text or a Python value panics (`PanicException: out-of-range date`, PL-2). A `Datetime` with a time zone adds a `Duration` on its physical `Int64` (UTC) and keeps the zone, the same addition as a naive timestamp, so it has no `date_range` limit either: Europe/Berlin 9999-12-31 12:00 plus a day is +10000-01-01 12:00 CET, and 0001-01-01 12:00 minus a day is 0000-12-31 12:00 LMT. `dt.offset_by` with months works in local time and crosses both bounds the same way. The docs are silent on chrono's range and the panic: a docs gap. No upstream test pins the panic (PL-2) or 0000-12-31; it may change between versions. The reference: `date_range`: date arithmetic raises outside 0001-01-01 … 9999-12-31 (#251). Options: `date_range`, `raise_set`. Rows of the choice tables: `time.arith` `date_range` (date, timestamp, timestamp_tz). Constructs: `+`, `-`, `dt.offset_by`, `cast`. Settings: `{"engine": ["streaming", "in-memory"]}`. See: upstream ledger [PL-2](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-2); issues [\#251](https://github.com/AlexanderNenninger/substrait-lean/issues/251). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `date-range` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `date-physical-range` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `timestamp-range` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by session brave-borg-41f7cc: Read at tag py-2.1.0 (commit 7f902bd3). - - 2026-11-04, source-reading, by characterization pipeline round 2 (temporal): Read at tag py-2.1.0 (commit 7f902bd3). - `Datetime(tu, tz)` ± `Duration`: `Int64` arithmetic, then `into_datetime(tu, tz)` - `Duration` with a zoned or naive `Datetime`: the `Datetime` - `offset_by` on a zoned `Datetime` adds through `Duration::add` with the zone - 2026-11-04, manual-repro, by characterization pipeline round 2 (temporal): Polars 2.1.0 (Python) from the project's environment, macOS arm64, `POLARS_MAX_THREADS=2`; each `LazyFrame` collected with `engine="in-memory"` and `engine="streaming"`. - Datetime('us', 'Europe/Berlin') column \[9999-12-31 12:00, 0001-01-01 12:00\]: + pl.duration(days=1) gives years 10000 and 1, - pl.duration(days=1) years 9999 and 0, dt.offset_by('1mo') +10000-01-31 and 0001-02-01, dt.offset_by('-1mo') 9999-11-30 and 0000-12-01, on both engines - 2026-11-04, source-reading, by characterization pipeline reverse pass - Date ± Duration: cast to Datetime(us), Int64 arithmetic, cast back; no range check - to_temporal_unit: date32_to_datetime_opt, None becomes NULL (date_to_year at L106-L112) - Date to text calls the panicking date32_to_date - 2026-11-04, docs, by characterization pipeline reverse pass - a 32-bit signed integer of days since 1970-01-01; silent on chrono's range and the panic (docs gap) - 2026-11-04, engine-test, by characterization pipeline reverse pass - test_out_of_range_date_year_11991: cast(-96465659 → Date).dt.year() is None (in-memory) - date(9999,12,31).offset_by('1d') builds a date past 9999; item() raises Python's ValueError (Python \>= 3.14 only) ## polars-month-arith: `offset_by('1mo')` clamps to the month's end Engine: polars py-2.1.0; area: temporal; status: no-finding; evidence: probe. Verdict: end-of-month clamping Adding months keeps the day where it exists and otherwise takes the month's last day: 2024-01-31 plus a month is 2024-02-29, 2024-02-29 plus a year is 2025-02-28, 2023-03-31 minus a month is 2023-02-28. `pl.duration` has no months; calendar steps go through `offset_by`. The reference: Adding months keeps the day of month, or takes the month's last day where it is shorter (`Time.addMonths`: 2024-01-31 + 1 month is 2024-02-29). Options: `date_range`. Constructs: `dt.offset_by`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `date-month-arith` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `date-month-arith` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-strftime: `strftime` is chrono's format Engine: polars py-2.1.0; area: temporal; status: open; evidence: probe. Verdict: chrono's specifiers: `%.f` milliseconds here, `%U` Sunday weeks `dt.strftime` uses chrono's specifiers: `%j` the day of the year, `%a` the short weekday, `%U` the week with weeks starting Sunday, `%V` the ISO week; `%.f` printed `.600` for 600 000 µs (the shortest of 3, 6 or 9 digits). No upstream test pins this; it may change between versions. The reference: `strftime` is opaque in the reference (`time.part`); DuckDB's specifiers are its own. Options: `cast_formatting`. Constructs: `dt.strftime`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `strftime` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `strftime` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass - strftime: chrono StrftimeItems + format_with_items (chrono 0.4.45) - Date: the same, via date32_to_date - 2026-11-04, docs, by characterization pipeline reverse pass - refers to chrono's strftime table - %j, %a, %U, %V; %.f chooses 3, 6 or 9 digits ## polars-temporal-errors: `TIME + duration` unsupported; DST gaps and invalid dates raise Engine: polars py-2.1.0; area: temporal; status: open; evidence: probe. Verdict: raises where the reference refuses or computes `time + duration` is refused ([SchemaError](../reference/errors.SchemaError.md#sverify.errors.SchemaError), no supertype). `replace_time_zone` raises on a local time in a DST gap (`non_existent='null'` gives NULL). `str.to_date` raises on an invalid date such as 2024-02-30 and, with `strict=False`, gives NULL; it accepts `2024-2-3` for `%Y-%m-%d`. Months only exist in `dt.offset_by` (`pl.duration` has none), and `offset_by` accepts `Date` and `Datetime`: on a `Time` column it raises `InvalidOperationError` (`expected Date or Datetime, got time`). `Time ± Duration` finds no supertype, so `TIME ± INTERVAL` with or without months is refused when the plan is resolved. The docs are silent on `time + duration` being refused and `2024-2-3` parsing: a docs gap. No upstream test pins `time + duration`, `offset_by` on `Time`, `2024-02-30` or `2024-2-3`; they may change between versions. The reference: `TIME ± INTERVAL` is refused (`time_interval_months`); time zones are outside the model. Options: `time_interval_months`, `time_zones`, `cast_parsing`. Rows of the choice tables: `time.arith` `time_interval_months` (time). Constructs: `+`, `dt.replace_time_zone`, `str.to_date`. Settings: `{"engine": ["streaming", "in-memory"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `time-plus-duration` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `tz-nonexistent` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `str-to-date` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by characterization pipeline round 2 (temporal): Read at tag py-2.1.0 (commit 7f902bd3). - `+` of `Time` and `Duration` asks for their supertype - `-` the same - the supertypes of `Time` and of `Duration`: none for the pair - `offset_by`: `Date` or `Datetime`, else `InvalidOperation` - 2026-11-04, manual-repro, by characterization pipeline round 2 (temporal): Polars 2.1.0 (Python) from the project's environment, macOS arm64, `POLARS_MAX_THREADS=2`; each `LazyFrame` collected with `engine="in-memory"` and `engine="streaming"`. - Time column \[23:00\]: pl.col('t') + pl.duration(hours=2) raises SchemaError (failed to determine supertype of time and duration\[μs\]); pl.col('t').dt.offset_by('1mo') raises InvalidOperationError (expected Date or Datetime, got time) - 2026-11-04, source-reading, by characterization pipeline reverse pass - str.to_date: fast parser, then chrono NaiveDate::parse_from_str (accepts 2024-2-3, rejects 2024-02-30); failure NULL, raised under strict - 2026-11-04, docs, by characterization pipeline reverse pass - non_existent: 'raise' (default) or 'null' - weeks … nanoseconds; no months - 2026-11-04, engine-test, by characterization pipeline reverse pass - replace_time_zone non_existent='null' gives None in the DST gap (in-memory) - to_date strict=False gives None, strict=True raises (in-memory) ## polars-timestamp-unit: Timestamps are microseconds by default; nanoseconds reach 2262 Engine: polars py-2.1.0; area: temporal; status: no-finding; evidence: probe. Verdict: `Datetime('us')`; `ns` casts fail past 2262-04-11 A Python `datetime` becomes `Datetime('us')`. `Datetime('ns')` holds 1677-09-21 … 2262-04-11; a cast to it outside that range raises (`InvalidOperationError`) unless `strict=False`. The reference: `precision_timestamp` at 0, 3, 6, 9 (#323); the reference's timestamps are microseconds. Options: `timestamp_precision`. Constructs: `Datetime`, `cast`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: issues [\#323](https://github.com/AlexanderNenninger/substrait-lean/issues/323). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `timestamp-unit` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The default unit is the frame's, set when it is built, before any engine runs; the cast to `Datetime('ns')` and its overflow ran on both. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `timestamp-unit` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) # casts ## polars-cast-double-to-int: Double to integer: truncates; out of range raises or is NULL Engine: polars py-2.1.0; area: casts; status: no-finding; evidence: probe. Verdict: `float_to_int: TRUNCATE`; `strict` raises, `strict=False` NULL; `wrap_numerical` saturates A cast from `Float64` to an integer truncates toward zero (2.7 is 2, -2.7 is -2). A value out of the target's range, ±inf or NaN raises with `strict=True` (the default, `InvalidOperationError`) and is NULL with `strict=False`. `wrap_numerical=True` saturates for doubles (1e19 is 127 in `Int8`, -1e19 is -128, NaN is 0, Rust's `as`), where it wraps for integers (polars-cast-int). The reference: `float_to_int: TRUNCATE` (#209); `cast_failure` as the cast states, `THROW_EXCEPTION` for `UNSPECIFIED`. Options: [float_to_int](../reference/sql.Dialect.md#sverify.sql.Dialect.float_to_int), `cast_failure`. Rows of the choice tables: [float_to_int](../reference/sql.Dialect.md#sverify.sql.Dialect.float_to_int) (any); `cast_failure` (any). Constructs: `cast`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [ST-6](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-6); issues [\#209](https://github.com/AlexanderNenninger/substrait-lean/issues/209). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `cast-double-to-int` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-double-to-int-strict` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-double-to-int-wrap` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-decimal-to-int` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `cast-double-to-int` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-double-to-int-strict` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-double-to-int-wrap` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-decimal-to-int` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-cast-int: Integer to a narrower integer or decimal: raises, NULL, or wraps Engine: polars py-2.1.0; area: casts; status: no-finding; evidence: probe. Verdict: `strict` raises, `strict=False` NULL, `wrap_numerical` wraps A cast to a narrower integer that does not fit raises with `strict=True`, is NULL with `strict=False`, and wraps with `wrap_numerical=True` (300 is 44 in `Int8`). A cast to a decimal too narrow for it raises or is NULL the same way. A decimal to an integer truncates. The reference: `cast_failure`: `THROW_EXCEPTION` or `RETURN_NULL`, as the cast states. Options: `cast_failure`. Rows of the choice tables: `cast_failure` (any). Constructs: `cast`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `cast-int-narrow` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-int-narrow-strict` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-int-to-decimal-overflow` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-decimal-to-int` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `cast-int-narrow` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-int-narrow-strict` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-int-to-decimal-overflow` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-decimal-to-int` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-cast-string: Casts from and to strings Engine: polars py-2.1.0; area: casts; status: open; evidence: probe. Verdict: strict parsing; shortest round-trip for doubles; no string → bool or date Text to an integer accepts an optional sign and digits only (`2`, `3.0`, `1e2`, `0x10` fail); text to a double accepts `nan`, `inf`, `-Infinity`, `.5`, `5.` and rounds `1e400` to inf. Failures raise with `strict=True` and are NULL with `strict=False`. A double becomes its shortest round-trip text, with an exponent at 1e16 (`1e+16`) and at 1e-7 (`1e-7`), without one at 1e15 and 1e-5, `-0.0`, `inf` and `NaN`. Booleans become `true` and `false`; integers and doubles become booleans by `!= 0` (NaN is true). Text to `Boolean` or to `Date` is refused (`InvalidOperationError`); `str.to_date` parses dates. The docs are silent on the text forms that parse, how a double prints and text to `Boolean` being refused: a docs gap. No upstream test pins the rejected forms, `nan`/`inf` parsing or the double's text format; they may change between versions. The reference: Casts to and from strings follow the reference's `cast` rules; `cast_failure` as stated. Options: `cast_parsing`, `cast_formatting`, `cast_failure`. Rows of the choice tables: `cast_failure` (any). Constructs: `cast`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `cast-string-to-int` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-string-to-double` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-double-to-string` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-bool` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-string-to-bool` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-string-to-date` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `cast-string-to-int` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-string-to-double` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-double-to-string` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-bool` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-string-to-bool` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-string-to-date` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass - integers: atoi_simd::parse::\<\_, true, true\> (0.18.1), sign and digits; doubles: fast_float2::parse (0.2.4) - unparsable text becomes NULL; strict mode raises on the new NULLs - double to text: zmij 1.0.23 shortest round-trip - Boolean to text: true/false (the doc comments at L38 and mod.rs L379 say 1/0) - text to Date bails (removed in 2.0); number to Boolean is != 0; no text arm for Boolean, so it bails - 2026-11-04, docs, by characterization pipeline reverse pass - 'Remove casts from string to temporal types': cast(pl.Date) of text raises, use str.to_date (docs/source/releases/upgrade/2.md L1357-L1390) - states strict/non-strict only; silent on accepted text formats, float text format and text to Boolean - 2026-11-04, engine-test, by characterization pipeline reverse pass - test_leading_plus_zero_int/uint/float: sign and leading zeros accepted, '-.5', '0.', '+.5' parse; eager Series.cast (in-memory) - test_string_to_temporal_cast_unsupported: text to Date/Datetime/Time raises InvalidOperationError (in-memory) ## polars-cast-to-double: Integer to double and double to float round to nearest Engine: polars py-2.1.0; area: casts; status: no-finding; evidence: probe. Verdict: round to nearest even; overflow to ±inf `Int64` to `Float64` rounds to the nearest double (2⁵³ + 1 is 2⁵³). `Float64` to `Float32` rounds to the nearest float, overflows to inf and underflows to 0. The reference: The conversions between integers and doubles are opaque constants with proved properties (`SubstraitLean/Conversions.lean`), the same in every profile; `fp32` arithmetic is refused until \#417. Options: `fp32`. Rows of the choice tables: `fp32` (fp32). Constructs: `cast`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: issues [\#417](https://github.com/AlexanderNenninger/substrait-lean/issues/417). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `cast-int-to-double` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `cast-double-to-float` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `cast-int-to-double` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `cast-double-to-float` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-ulp-casts: Integer and decimal to Float64 and Float32: correctly rounded Engine: polars py-2.1.0; area: casts; status: no-finding; evidence: probe. Verdict: ≤ 0.5 ulp: Int64, Int128 and Decimal to Float64 and Float32 Int64 and Int128 to Float64 and Float32 are Rust `as` casts (through `num_traits`), which round to nearest, ties to even: ≤ 0.5 ulp, also above 2^53 and for 128-bit values. Decimal to Float64 divides once when the unscaled value is at most 2^53 and the scale at most 22, and otherwise formats `e-` and parses it with Rust's correctly rounded parser; Decimal to Float32 rounds the Float64 again unless it sits exactly on a binary32 midpoint, then parses. Both correctly rounded. Observed, 3 000 values each: 0 off the correctly rounded value for Int64, Int128 (to ±2^126) and Decimal(18, 6), (18, 17), (38, 10), (38, 30), (38, 2), to Float64 and Float32. Platform-independent. DuckDB's HUGEINT cast misses by up to 0.625 ulp where Polars's Int128 cast is exact on the same values (duckdb-ulp-casts). The reference: Casts of integers and decimals to doubles round once to nearest, ties to even (Lean's `Float.ofInt`, `Float.ofScientific`). Options: `fp32`. Constructs: `cast(pl.Float64)`, `cast(pl.Float32)`. Settings: `{"engine": "streaming", "platform": "macOS arm64"}`. Provenance: - 2026-11-04, source-reading, by characterization pipeline round 5 (ulp bounds) - Int64 to Float32, Float64: primitive_to_primitive_dyn - Int128 to Float64 - primitive_as_primitive: `as` - Decimal to Float32, Float64: dec128_to_f32, dec128_to_f64 - dec128_to_f64: one exact division or parse_dec128; dec128_to_f32 checks the midpoint - int to float `as`: rounds to nearest, ties to even - 2026-11-04, probe, by characterization pipeline round 5 (ulp bounds): Probe script ulp_probe.py (session scratchpad, not in the repository), run from python/ as `uv run --with mpmath python ulp_probe.py --n 3000 --seed 1 --out ulp.json`: about 3 000 inputs per operation and type, random and hard cases, against mpmath at 256 bits. 'bad' counts results that differ from the correctly rounded value; the max is in ulps of the result type at the exact value's binade. DuckDB runs on a table loaded from Parquet, Polars on a LazyFrame with collect(engine='streaming'), so the column kernels run. DuckDB 2.0.0 and Polars 2.1.0 (Python wheels), macOS 26.6 arm64, Python 3.12.11, mpmath 1.3.0. Elementwise kernels are shared by the streaming and in-memory engines (polars-elem-rounding, polars-double-scalar-divisor); the probe collects with engine='streaming' on 3 000-row frames, so morsels hold more than one row and the literal paths run. - pl.LazyFrame({'x': ints}, schema={'x': pl.Int64}).select(pl.col('x').cast(pl.Float64), …cast(pl.Float32)).collect(engine='streaming'): max 0.5000 / 0.49999 ulp, 0 of 3 000 off. The same for Int128 (±2^126): 0.5000 / 0.49998, 0 off. - pl.Series(decimals, dtype=pl.Decimal(p, s)).cast(…) for (18,6), (18,17), (38,10), (38,30), (38,2): max ≤ 0.49997 (Float64), ≤ 0.5000 (Float32) ulp, 0 off for each. # nulls ## polars-kleene: Boolean logic is Kleene's; `all` and `any` skip NULLs by default Engine: polars py-2.1.0; area: nulls; status: no-finding; evidence: probe. Verdict: three-valued `&`, `|`, `~`; `all(ignore_nulls=False)` is Kleene's `&`, `|` and `~` follow three-valued logic (`False & NULL` is False, `True | NULL` is True, `True & NULL` is NULL). `all()` and `any()` skip NULLs (`all(True, NULL)` is True); with `ignore_nulls=False` they are Kleene's (`all(True, NULL)` is NULL). The reference: `and`, `or`, `not` are Kleene's; `bool_and` and `bool_or` skip NULLs as aggregates do. Options: `three_valued_logic`. Constructs: `&`, `|`, `~`, `all`, `any`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `kleene` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `any-all-nulls` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `kleene` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `any-all-nulls` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-null-keys: NULL keys: one group; joins skip them by default Engine: polars py-2.1.0; area: nulls; status: no-finding; evidence: probe. Verdict: NULLs group together; `nulls_equal=True` joins them `group_by` and [unique](../reference/unique.md#sverify.unique) put the NULLs in one group. A join does not match NULL keys unless `nulls_equal=True`. The reference: Grouping and `DISTINCT` put NULLs together; an equi-join's `=` does not match NULL (and `IS NOT DISTINCT FROM` keys do). Options: `null_join_keys`. Rows of the choice tables: `null_join_keys` (any). Constructs: `group_by`, [unique](../reference/unique.md#sverify.unique), `join`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `group-nan-zero` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `unique-nan-zero` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `join-nan-zero-null` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. join-nan-zero-null's recorded differ 1 is a tie order: `sort('y')` ties the rows 0.0 and -0.0 (both y = 2), and the engines list them in different orders. The join results are the same set of rows on both engines; it is no engine difference. - probe `group-nan-zero` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `unique-nan-zero` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `join-nan-zero-null` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) # nested ## polars-explode: `explode` keeps a row for a NULL list and drops an empty list Engine: polars py-2.1.0; area: nested; status: open; evidence: probe. Verdict: a NULL list gives a NULL row, `[]` no row; `keep_nulls=False` drops the NULL row too `explode` gives one row per element, one NULL row for a NULL list and no row for an empty list (the defaults `empty_as_null=False, keep_nulls=True`). With `keep_nulls=False` a NULL list gives no row either, which is SQL's `UNNEST`; `empty_as_null=True` gives an empty list a NULL row. The reference: `UNNEST`: one row per element, none for an empty list or NULL. Options: `unnest_null_empty`. Constructs: `explode`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `explode` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. explode calls the eager `DataFrame.explode`, in memory; s-explode explodes 6000 rows lazily on each engine, with and without `empty_as_null=False, keep_nulls=False`: the same rows in the same order (6000 and 4500). `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `explode` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `s-explode` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass - empty list: NULL row only if empty_as_null; NULL list: NULL row only if keep_nulls; else dropped - defaults empty_as_null=False, keep_nulls=True - streaming: Explode is streamable and runs as a Map node calling the in-memory evaluate_function_ir per morsel - 2026-11-04, docs, by characterization pipeline reverse pass - empty_as_null and keep_nulls documented; defaults False and True - 2026-11-04, engine-test, by characterization pipeline reverse pass - test_explode_params: \[\[1,2,3\], None, \[4,5,6\], \[\]\] under all four flag pairs (in-memory) - test_explode_parametric: Hypothesis over the flags - 2026-11-04, probe, by characterization pipeline reverse pass: Corrected the description: it said the default gives a NULL row for an empty list. At 2.1.0 the default `empty_as_null=False` gives no row for an empty list (the probe explode shows it too) and one NULL row for a NULL list. - LazyFrame({'i':\[1,2,3,4\],'l':\[\[1,2\],\[\],None,\[None\]\]}).explode('l') on in-memory and streaming (morsel 64): \[(1,1),(1,2),(3,None),(4,None)\]; the empty list gives no row by default ## polars-list-index: `list.get`: 0-based, negative from the end, out of range raises Engine: polars py-2.1.0; area: nested; status: open; evidence: probe. Verdict: 0-based; raises out of range unless `null_on_oob=True` `list.get(i)` counts from 0, a negative index from the end (`-1` is the last element). An index out of range raises (`ComputeError: get index is out of bounds`) by default and is NULL with `null_on_oob=True`; a NULL list gives NULL. The `list.get` docs say an out-of-range index returns None, and the default raises: PL-12. No upstream test pins the default `null_on_oob=False`; it may change between versions. The reference: `list_extract(l, i)` (reference-semantics.md, Nested) counts from 1, as DuckDB's does. Options: `list_indexing`, `raise_set`. Constructs: `list.get`, `list.slice`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: upstream ledger [PL-12](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-12). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `list-get` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `list-get` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass - lst_get: bails 'get index is out of bounds' unless null_on_oob; negative index from the end - per-row index: start+idx or end+idx, bail or NULL; NULL list or index gives NULL - 2026-11-04, docs, by characterization pipeline reverse pass - docs bug (draft PL-D5): says an out-of-bounds index returns None, but the default null_on_oob=False raises - 2026-11-04, engine-test, by characterization pipeline reverse pass - test_list_arr_get: get(0)→\[1,4,6\], get(-1)→\[3,5,9\], get(3)/get(-3) with null_on_oob=False raise (in-memory) - test_list_arr_get_null_on_oob: out of range → None ## polars-list-order: Lists sort with NULL elements lowest; no `<` or `min` on lists Engine: polars py-2.1.0; area: nested; status: open; evidence: probe. Verdict: `list_null_order: LEAST` in sorts; list comparisons refused `sort` orders lists lexicographically, a prefix first, and a NULL element below every value: `[] < [NULL] < [1] < [1, NULL] < [1, 2]` ascending, reversed descending, NULL lists first by default. `<` between lists and `min`, `max` of lists raise (`InvalidOperationError`). The docs are silent on how lists sort: a docs gap. No upstream test pins the prefix order; it may change between versions. The reference: `list_null_order`: NULL where a NULL element decides `<`, and `GREATEST` in `ORDER BY` (#207). DuckDB puts the NULL element above every value (duckdb-R11). Options: `list_null_order`. Rows of the choice tables: `list.order` `list_null_order` (list); `agg.order` `list_null_order` (list). Constructs: `sort`, `<`, `min`. Settings: `{"engine": ["in-memory", "streaming"]}`. See: issues [\#207](https://github.com/AlexanderNenninger/substrait-lean/issues/207). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `list-sort` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `list-compare` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `list-min` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, docs, by characterization pipeline round 1 (nested): The docs are silent on the order of lists and on `min`/`max` of lists; the rows rest on the 2026-11-04 probes (list-sort, list-compare, list-min). The docs at `stable` are those of Polars 2.1.0: their \[source\] links point at tag py-2.1.0. - `list.sort(descending=False, nulls_last=False)` sorts the elements inside each list; it says nothing about how lists compare - `sort(descending, nulls_last)`; no word on list keys - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `list-sort` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `list-compare` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `list-min` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, source-reading, by characterization pipeline reverse pass - list sort: arg_sort over row-encoded keys - list encoding: continuation token per element, termination token after; prefix sorts first - into_nested: element NULLS_LAST = DESCENDING, so a NULL element is lowest ascending (#22557) - streaming sort row-encodes a list key the same way - \< between lists bails InvalidOperation - 2026-11-04, engine-test, by characterization pipeline reverse pass - test_nested_sorting_22557: \[\['A'\], None, \[None\], \['B'\]\] sorts to \[None, \[None\], \['A'\], \['B'\]\] (in-memory) ## polars-nested-equality: Lists and structs: NULL elements and fields compare equal Engine: polars py-2.1.0; area: nested; status: no-finding; evidence: probe. Verdict: structural equality, NULL inside equals NULL `==` on lists and on structs treats a NULL element or field as equal to NULL: `[1, NULL] == [1, NULL]` and `{a: 1, b: NULL} == {a: 1, b: NULL}` are true; a NULL list or struct gives NULL (`eq_missing` makes it equal). Grouping puts equal structs together. Structs sort field by field, a NULL field first. The reference: Equality is structural, and list equality treats NULLs as equal, as grouping does (reference-semantics.md, "Equality for grouping" and Nested). Options: `struct_equality`. Constructs: `==`, `group_by`, `sort`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `list-equality` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `struct-equality` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `struct-group-sort` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `list-equality` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `struct-equality` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `struct-group-sort` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) # aggregates ## polars-arg-min-max: `arg_min`, `arg_max`: the first index of a tie Engine: polars py-2.1.0; area: aggregates; status: no-finding; evidence: probe. Verdict: the first of tied rows `arg_max` over 1, 3, 3, 1 is 1 and `arg_min` is 0: the first of the tied rows. NaN is skipped (polars-min-max-nan). The reference: `tie_pick`: any row of the tie. Options: `tie_pick`. Rows of the choice tables: `agg.pick` `tie_pick` (any). Constructs: `arg_min`, `arg_max`. Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `arg-min-max-ties` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `min-max-nan` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `arg-min-max-ties` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `min-max-nan` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-avg-int: `mean` and `median` of integers: `Float64` Engine: polars py-2.1.0; area: aggregates; status: open; evidence: probe. Verdict: a double: the `Float64` sum over the count `mean` of integers is a `Float64` (the values cast to doubles and summed, polars-sum-float), so it rounds above 2⁵³ (`mean(2⁶², 2⁶², 1)` is 3.0744573456182584e18). `median` of integers is a `Float64` too. Both are NULL for no values. Every integer width takes one generic path (`with_match_physical_numeric_polars_type`, `T: PolarsIntegerType`): each value becomes an `f64`, exactly for 8 to 32 bits, and the sum is an `f64`: pairwise ungrouped, Kahan per group in memory, value by value per group when streaming. A sum below 2⁵³ is exact in any order, so for `Int8` to `Int32` the mean is the exact quotient rounded once on either engine. Above 2⁵³ (`Int64`) the engines can round the sum differently. The docs are silent on the rounding above 2⁵³: a docs gap. No upstream test pins the rounding above 2⁵³; it may change between versions. The reference: `avg` of `i64` returns `i64` in Substrait; the reference follows the declared output type (`avg_int_result`, \#209). Options: `avg_int_result`, `empty_group_aggregates`. Rows of the choice tables: `agg.isum` `avg_int_result` (i64); `agg.isum` `avg_int_result` (i8, i16, i32; evidence engine_test); `agg.isum` `empty_group_aggregates` (i64); `agg.isum` `empty_group_aggregates` (i8, i16, i32; evidence source). Constructs: `mean`, `median`. Settings: `{"engine": ["streaming", "in-memory"]}`. See: issues [\#209](https://github.com/AlexanderNenninger/substrait-lean/issues/209). Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `int-mean` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by characterization pipeline round 2 (aggregates): Read at tag py-2.1.0 (commit 7f902bd3). - `mean` of any numeric `ChunkedArray`: `_sum_as_f64` / count - `sum_arr_as_f64` converts each integer to `f64` (`as_`) and sums pairwise - in-memory grouped mean of integers: `KahanSum` of `to_f64` values, `Float64` result - streaming: one `NumMeanReducer` per physical numeric type, `Float64` output for integers - `reduce_one` adds `b.as_()` to an `f64`; `reduce_ca` adds `_sum_as_f64` - 2026-11-04, engine-test, by characterization pipeline round 3 (aggregates): Pins that the mean of `Int8` to `Int32` is a `Float64` with the exact quotient on these inputs, on both engines, and that the sum does not overflow the input width. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_group_by_mean_by_dtype`: grouped `mean` of 1, 2, 3 \| 4 is `Float64` 2.0, 4.0 for `Int8`, `Int16`, `Int32` (and the unsigned widths); `Float32` stays `Float32`. Lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect), schema and values checked - `test_overflow_mean_partitioned_group_by_5194` (slow): grouped `mean` of 100 000 `Int32` values of 10⁷ is 10000000.0, past the `Int32` range of the sum (eager, in memory) - `test_window_avg_of_large_integers`: SQL `AVG` of `Int32` 2·10⁹ over three window frames matches DuckDB, on `engines=['in-memory', 'streaming']` - 2026-11-04, docs, by characterization pipeline reverse pass - the example shows mean of an integer column as f64; silent on rounding above 2^53 - ## polars-corr: `corr`: shared state; NaN for one pair or zero variance Engine: polars py-2.1.0; area: aggregates; status: open; evidence: probe. Verdict: dp_xy / sqrt(dp_xx · dp_yy); NaN where the reference is NULL; no clamp `PearsonState` returns `dp_xy / sqrt(dp_xx * dp_yy)` if `dp_xx * dp_yy > 0`, else NaN: one pair or a zero variance is NaN. Not clamped to \[-1, 1\]. `dp_xx * dp_yy` can overflow or underflow before the square root. No pairs: null in the streaming reducer; the in-memory path (`polars-ops` `cov.rs`) returns NaN. Integers become doubles (`as f64`) and decimals are cast to `Float64` in memory, so the degenerate cases are those of doubles: one pair or a constant side is NaN for `Int64` and `Decimal` too. Above 2⁵³ distinct integers can round to one double: `corr` of 2⁶², 2⁶² + 1, 2⁶² + 2 with 1, 2, 3 is NaN. The streaming reduction accepts primitive numerics only: `corr` of a `Decimal` column raises `InvalidOperationError` on the default lazy engine and succeeds in memory. No pairs is null when streaming and NaN in memory (above). The docs are silent on one pair, a constant side and no pairs, the types `corr` takes and the engines: a docs gap. No upstream test pins no pairs or a `Decimal` input; they may change between versions. The reference: `AggFn.corr` (`corrOf`): NULL for fewer than two pairs, or for a side whose values are all equal; else the co-deviation sum by the square root of the product of the two squared-deviation sums. It is not clamped to \[-1, 1\]. Engines may differ in the last bits (#54) until \#85 bounds the error. Where the reference is NULL (one pair, a constant side), Polars gives NaN. Options: `stat_algo`, `stat_degenerate`. Rows of the choice tables: `agg.stat` `stat_algo` (i64, fp64, decimal); `agg.stat` `stat_degenerate` (i64, fp64, decimal). Constructs: `corr`, `pearson_corr`. Settings: `{"engine": ["streaming", "in-memory"]}`. See: upstream ledger [PL-5](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-5), [ST-3](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/spec-gaps.md#ST-3); issues [\#54](https://github.com/AlexanderNenninger/substrait-lean/issues/54), [\#85](https://github.com/AlexanderNenninger/substrait-lean/issues/85). Provenance: - 2026-11-03, source-reading, by coordinator subagent of session local_974f7865 (aggregate-algorithms research): Read at tag py-2.1.0 (commit 7f902bd3). Unverified: what the SQL layer (polars-sql) returns for no pairs; only the Rust paths were read. - - - - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): In a [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select) (in memory): a constant side, one pair and no pairs are all NaN. Polars 2.1.0. - probe `corr-degenerate` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, source-reading, by characterization pipeline round 2 (aggregates): Read at tag py-2.1.0 (commit 7f902bd3). - in memory: a length-1 side is a broadcast constant (cov 0.0 or null, corr NaN); else `CovState`/`PearsonState` per chunk pair, merged by `combine` - in memory: `Int32`, `Int64`, `UInt32`, `UInt64` (and `UInt64` for cov) read `as f64`; other types, decimals among them, cast to `Float64` - result type: the inputs' float supertype, else `Float64` - `CovState`, `PearsonState`: two-pass blocks, Welford's `insert_one`, Chan's `combine` - `cov`, `pearson_corr`: pairs with both values valid, `as_()` `f64`, blocks of 128 - streaming `new_cov_reduction`: `polars_ensure!(dtype.is_primitive_numeric())`, which a decimal is not - streaming: `update_group` casts to `Float64` and merges `moment::cov` per chunk (ungrouped); `update_groups_while_evicting` uses `insert_one` per row (grouped) - streaming `new_pearson_corr_reduction`: the same primitive-numeric check - streaming corr finish: null at weight 0, else `PearsonState::finalize` - `is_primitive_numeric`: float or integer - ungrouped `cov`/`corr` lower to a streaming reduction - grouped `cov`/`corr` lower to a streaming reduction - 2026-11-04, manual-repro, by characterization pipeline round 2 (aggregates): Polars 2.1.0 (Python) from the project's environment, macOS arm64, `POLARS_MAX_THREADS=2`; each `LazyFrame` collected with `engine="in-memory"` and `engine="streaming"`. - Decimal(10, 2) columns a = \[1.10, 2.20, 3.35\], b = \[1.00, 2.00, 4.00\]: pl.cov('a', 'b') is 1.6916666666666667 and pl.corr('a', 'b') 0.9843241382880894 in memory; streaming raises InvalidOperationError: `cov` (`corr`) operation not supported for dtype `decimal[10,2]` - Int64 a = \[5\], b = \[7\]: pl.corr NaN on both engines; a = \[5, 5, 5\], b = \[1, 2, 3\]: NaN on both; a = \[None\], b = \[1\]: NaN in memory, null streaming; Decimal(10, 2) a = \[5\], b = \[7\]: NaN in memory, InvalidOperationError streaming - Int64 a = \[2**62, 2**62 + 1, 2\*\*62 + 2\], b = \[1, 2, 3\]: pl.corr NaN on both engines - 2026-11-04, engine-test, by characterization pipeline round 3 (aggregates): Pins NaN for a zero variance in memory, past the 128-value block boundary (n = 245). Not tested on the streaming engine. Polars's CI runs the unit tests by default, where a lazy [collect()](../reference/sql.Dialect.md#sverify.sql.Dialect.collect) without `engine` runs the streaming engine (also once with `POLARS_IDEAL_MORSEL_SIZE=4`), and again with `POLARS_ENGINE_AFFINITY=in-memory` (.github/workflows/test-python.yml#L55-L78, \#L168-L178); eager `Series` and `DataFrame` calls run in memory. Read at tag py-2.1.0 (commit 7f902bd3), not run. - `test_pearson_corr_zero_column_29315`: `corr` of a constant 0.0 column with 0..n-1 is NaN for n = 117 and 245 (eager, in memory) - `test_corr_cov_lit_produces_zero_nan_26633`: `corr(lit(1), a)` is NaN and `cov` 0.0 (eager) - 2026-11-04, probe, by characterization pipeline round 4 (streaming probes): Confirms PL-5 for `corr`: the same query (with `cov` and `corr` of `Decimal` columns) raises on the streaming engine and gives 0.9707253433941508 in memory. `python -m sverify_dev.probe_polars --diff --write` runs each observation (`STREAMING_OBSERVATIONS`, python/tools/sverify_dev/probe_polars.py) with `collect(engine='in-memory')` and `collect(engine='streaming')` and records both results. Polars 2.1.0 (Python), macOS arm64. - probe `s-cov-decimal` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - 2026-11-04, docs, by characterization pipeline reverse pass - silent on degenerate inputs (one pair, a constant side, no pairs), on accepted dtypes and on the engines; no engine-support directive - 2026-11-04, engine-test, by characterization pipeline reverse pass - `test_corr_nan`: a = \[1.0, 1.0\], b = \[1.0, 2.0\] gives NaN (eager, in memory). No upstream test pins no-pairs (null streaming, NaN in memory) or Decimal input - 2026-11-04, engine-test, by characterization pipeline reverse pass: From the reverse pass's check of PL-5, which this fact records. - no test of cov/corr on Decimal; skew's streaming-vs-in-memory hypothesis test includes pl.Decimal in its dtypes, so the suite expects decimal parity for moment reductions - 2026-11-04, docs, by characterization pipeline reverse pass: From the reverse pass's check of PL-5, which this fact records. - no dtypes and no engine-support directive; pl.corr likewise ## polars-count-empty: `count` skips NULLs; aggregates of nothing Engine: polars py-2.1.0; area: aggregates; status: no-finding; evidence: probe. Verdict: `count` skips NULLs, `len` counts rows; `min`, `first` of NULLs are NULL `count` counts the non-NULL values and `len` (`pl.len()`) the rows. Over NULLs only, `min`, `max` and `first` are NULL, `count` is 0 and [mode](../reference/sql.Dialect.md#sverify.sql.Dialect.mode) is `[NULL]`. `sum` gives 0 there (polars-sum-int, polars-sum-float). The reference: `count(x)` skips NULLs and `count(*)` counts rows; other aggregates of no values are NULL. Options: `empty_group_aggregates`. Constructs: `count`, `len`, `min`, `first`, [mode](../reference/sql.Dialect.md#sverify.sql.Dialect.mode). Settings: `{"engine": ["in-memory", "streaming"]}`. Provenance: - 2026-11-04, probe, by sverify_dev.probe_polars (session brave-borg-41f7cc): `sverify_dev.probe_polars` runs each expression on edge cases (0, ±1, ±2, 3, ±7, the bounds of the integer width; 0.5, ±2.5, 7, 1e308, ±inf, NaN; NULL), one row at a time, and compares each row with Lean's outcome of the Substrait call under each candidate option (`substrait-check --outcomes`). Observations are Polars programs whose result is recorded as it is. Polars 2.1.0 (Python), Rust crates at tag py-2.1.0 (commit 7f902bd3), on macOS arm64. - probe `count-nulls` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - probe `aggregates-empty` in [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json) - 2026-11-04, probe, by characterization pipeline round 5 (streaming engine): Streaming gives what in memory gives on every probe and observation cited here. The observations' frames hold a few rows, which streaming runs as one-row morsels; the probes' 6000-row columns cover morsels of many rows. `python -m sverify_dev.probe_polars --diff --write` runs each probe one row at a time and on a 6000-row column with `collect(engine='in-memory')` and `collect(engine='streaming')`, and each observation twice: its `df(...)` frame's [select](../reference/cli.Discovery.md#sverify.cli.Discovery.select), `with_columns`, `filter`, `sort`, [unique](../reference/unique.md#sverify.unique), `join` and `group_by` go through the streaming engine the second time (`StreamingFrame`); the `s-` observations collect a lazy query on each engine themselves. Polars 2.1.0 (Python), macOS arm64. - probe `count-nulls` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) - probe `aggregates-empty` in [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json) ## polars-cov: `cov`: two-pass blocks with Chan merge; Welford when grouped on streaming Engine: polars py-2.1.0; area: aggregates; status: open; evidence: probe. Verdict: two-pass per block + Chan (in memory; streaming ungrouped); Welford (streaming grouped); no clamp; Decimal raises on streaming (PL-5) `CovState` follows `var` (polars-var-std): two-pass per block of 128 merged by Chan's combine in memory, Welford's update in the streaming reducer. A weight at most `ddof` is null. `CovState` does not clamp. Integers become doubles (`as f64`), so values above 2⁵³ round; decimals are cast to `Float64` in memory. The streaming engine checks for a primitive numeric type before it builds the reduction, and a decimal is not one: `cov` of a `Decimal` column raises `InvalidOperationError` on the default lazy engine and succeeds in memory. Streaming, ungrouped, merges `moment::cov` per chunk, the block algorithm of the in-memory path; the per-value Welford update is the grouped streaming path. In memory a column of length 1 is taken as a broadcast constant, so the covariance of one pair is 0.0 with `ddof=0` and null with `ddof=1` without reaching `CovState`. The docs are silent on a weight at most `ddof` giving NULL, the algorithm, the types `cov` takes and the engines: a docs gap. No upstream test pins a `Decimal` input; it may change between versions. The reference: `AggFn.cov` (`covOf`) is two-pass: the two means, then the products of the deviations added in list order from +0.0, by n or n - 1. No pairs is NULL, and so is a sample of one pair. Engines may differ in the last bits (#54) until \#85 bounds the error. Options: `stat_algo`. Rows of the choice tables: `agg.stat` `stat_algo` (i64, fp64, decimal). Constructs: `cov`. Settings: `{"engine": ["streaming", "in-memory"]}`. See: upstream ledger [PL-5](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/upstream-bugs.md#PL-5); issues [\#54](https://github.com/AlexanderNenninger/substrait-lean/issues/54), [\#85](https://github.com/AlexanderNenninger/substrait-lean/issues/85). Provenance: - 2026-11-03, source-reading, by coordinator subagent of session local_974f7865 (aggregate-algorithms research): Read at tag py-2.1.0 (commit 7f902bd3). - - - 2026-11-04, source-reading, by characterization pipeline round 2 (aggregates): Read at tag py-2.1.0 (commit 7f902bd3). - in memory: a length-1 side is a broadcast constant (cov 0.0 or null, corr NaN); else `CovState`/`PearsonState` per chunk pair, merged by `combine` - in memory: `Int32`, `Int64`, `UInt32`, `UInt64` (and `UInt64` for cov) read `as f64`; other types, decimals among them, cast to `Float64` - result type: the inputs' float supertype, else `Float64` -