Upstream bugs
57 bugs found upstream, with how sverify handles each
Bugs and surprising behaviours sverify found in the projects it builds on, from upstream/bugs/ (the format, the prefixes).
DataFusion
DF-1: EXCEPT ALL is planned as a left anti join, which loses multiplicities
EXCEPT ALL is planned as a left anti join, which loses multiplicities.
Reproduction:
[1,1,2,3] EXCEPT ALL [1,3]gives[2], not[1,2](DuckDBsetops/test_pg_union.test:339,350)
How sverify handles it:
- Frontend no longer used (#264)
Status: filed #12956, open; reproduces on 55.0.0 and main still plans a left anti join; fix PR #25742 closed unmerged
See: facts datafusion-S6; upstream https://github.com/apache/datafusion/issues/12956, https://github.com/apache/datafusion/pull/25742; pull requests #264.
Found by replay, FACTS S6.
- Checked 2026-11-03 against datafusion 55.0.0: against the latest releases
- Checked 2026-11-04 against datafusion 55.0.0, datafusion core 55.1.0: repro on 51.0.0 and 55.0.0 (latest PyPI); main read at ee55effccc
DF-2: FETCH FIRST n ROWS ONLY is dropped: the plan has no limit
FETCH FIRST n ROWS ONLY is dropped: the plan has no limit.
Reproduction:
- DuckDB
parser/fetch_first.test
How sverify handles it:
- Frontend no longer used (#264)
Status: fixed upstream in 52.0.0 (#17716, PR #18691): FETCH is now refused
See: facts datafusion-S6; upstream https://github.com/apache/datafusion/issues/17716, https://github.com/apache/datafusion/pull/18691; pull requests #264.
Found by FACTS S6.
- Checked 2026-11-03 against datafusion 55.0.0: against the latest releases
- Checked 2026-11-04 against datafusion 55.0.0, datafusion core 55.1.0: 51.0.0 drops FETCH, 55.0.0 refuses it; 52.0.0 tag contains c89cb70
DF-3: The Substrait producer drops DISTINCT and FILTER (WHERE …) of a window aggregate: `COUNT(DISTINCT s) …
The Substrait producer drops DISTINCT and FILTER (WHERE …) of a window aggregate: COUNT(DISTINCT s) OVER (…) counts every row.
Reproduction:
- DuckDB
window/test_window_distinct.test:76
How sverify handles it:
- Frontend no longer used (#264)
Status: filed #22661; fixed on main by #22664, on branch-56 for 56.0.0 (unreleased, #24461); reproduces on 55.0.0
See: facts datafusion-S6; upstream https://github.com/apache/datafusion/issues/22661, https://github.com/apache/datafusion/pull/22664, https://github.com/apache/datafusion/issues/24461; pull requests #264.
Found by FACTS S6.
- Checked 2026-11-03 against datafusion 55.0.0: against the latest releases
- Checked 2026-11-04 against datafusion 55.0.0, substrait 0.37.1, datafusion core 55.1.0: reproduces on 51.0.0 and 55.0.0; fix on main and branch-56, no release yet
DF-4: GROUP BY k, k writes two grouping expressions, and the projection above reads the aggregate’s columns …
GROUP BY k, k writes two grouping expressions, and the projection above reads the aggregate’s columns shifted by one. On main (since #23468) the key is written once but referenced twice in the grouping, and the round trip is still wrong.
Reproduction:
- DuckDB
aggregate/group/test_group_by.test:149
Status: filed #27058, open; fix PR #27073 open. Reproduces on 55.0.0; on main #23468 shares the repeated key in grouping_expressions but still references it twice, so the round trip stays wrong
See: facts datafusion-S6; upstream https://github.com/apache/datafusion/issues/27058, https://github.com/apache/datafusion/pull/27073.
Found by FACTS S6.
- Checked 2026-11-03 against datafusion 55.0.0: against the latest releases
- Checked 2026-11-04 against datafusion 55.0.0, substrait 0.37.1, datafusion core 55.1.0: reproduces on 51.0.0 and 55.0.0; main read at ee55effccc, not run
DF-5: SELECT * of a NATURAL JOIN keeps both copies of the join columns
SELECT * of a NATURAL JOIN keeps both copies of the join columns.
Reproduction:
- DuckDB
catalog/case_insensitive_using.test:123-138
How sverify handles it:
- Frontend no longer used (#264)
Status: not a bug: DataFusion keeps quoted identifiers case-sensitive (enable_ident_normalization), so "hello" and "HELLO" share no column
See: facts datafusion-S6; pull requests #264.
Found by FACTS S6.
- Checked 2026-11-03 against datafusion 55.0.0: against the latest releases
- Checked 2026-11-04 against datafusion 55.0.0, substrait 0.37.1, datafusion core 55.1.0: same on 51.0.0 and 55.0.0
DF-6: The Substrait producer writes FROM (VALUES …) as an empty virtual table: the rows are dropped
The Substrait producer writes FROM (VALUES …) as an empty virtual table: the rows are dropped.
Reproduction:
- any
SELECT * FROM (VALUES (1), (2))
How sverify handles it:
- Frontend no longer used (#264)
Status: fixed upstream in 55.0.0 (#23608, PR #23672): the producer writes the rows to expressions
See: facts datafusion-S3; upstream https://github.com/apache/datafusion/issues/23608, https://github.com/apache/datafusion/pull/23672; pull requests #264.
Found by FACTS S3.
- Checked 2026-11-03 against datafusion 55.0.0: against the latest releases
- Checked 2026-11-04 against datafusion 55.0.0, substrait 0.37.1, datafusion core 55.1.0: 51.0.0 writes no
expressions, 55.0.0 writes 2 rows
DF-7: SQL decimal literals beyond double precision are read as doubles: `100000000000000000000 > …
SQL decimal literals beyond double precision are read as doubles: 100000000000000000000 > 99999999999999999999.99999 is false. Possibly intended.
Reproduction:
- DuckDB
cast/decimal_float_cast.test:10
Status: not a bug: datafusion.sql_parser.parse_float_as_decimal defaults to false; #14612 (open) proposes flipping it
See: facts datafusion-S4; upstream https://github.com/apache/datafusion/issues/14612.
Found by FACTS S4.
- Checked 2026-11-03 against datafusion 55.0.0: against the latest releases
- Checked 2026-11-04 against datafusion 55.0.0, datafusion core 55.1.0: same on 51.0.0 and 55.0.0
DF-8: The Substrait producer encodes grouping sets against the spec: the extra column is a bitmask where the …
The Substrait producer encodes grouping sets against the spec: the extra column is a bitmask where the spec says the grouping set’s index, and the grouping expressions are laid out differently.
Reproduction:
- TPC-DS q27/q36 (#229)
How sverify handles it:
- Grouping sets come through the SQLGlot lift instead (M2)
Status: filed #25208 (bitmask), open, fix PR #25527 closed unmerged; the layout (#17910) is fixed on main by #23468, for 56.0.0 (unreleased)
See: issues #229; upstream https://github.com/apache/datafusion/issues/25208, https://github.com/apache/datafusion/issues/17910, https://github.com/apache/datafusion/pull/25527, https://github.com/apache/datafusion/pull/23468.
Found by pi3, #229.
- Checked 2026-11-03 against datafusion 55.0.0: against the latest releases
- Checked 2026-11-04 against datafusion 55.0.0, substrait 0.37.1, datafusion core 55.1.0: reproduces on 51.0.0 and 55.0.0 (bitmask and layout)
DF-9: The Substrait producer registers no proper extension URNs (placeholder references)
The Substrait producer registers no proper extension URNs (placeholder references).
Reproduction:
- any plan with a function
How sverify handles it:
- Not needed anymore
Status: filed #11545, open; reproduces on 55.0.0 and main still writes u32::MAX references
See: issues #231; upstream https://github.com/apache/datafusion/issues/11545.
Found by #231 findings.
- Checked 2026-11-03 against datafusion 55.0.0: against the latest releases
- Checked 2026-11-04 against datafusion 55.0.0, substrait 0.37.1, datafusion core 55.1.0: reproduces on 51.0.0 and 55.0.0
DF-10: The Substrait consumer doesn’t know Substrait’s shift_left/shift_right
The Substrait consumer doesn’t know Substrait’s shift_left/shift_right.
Reproduction:
- a plan calling them, run through
from_substrait_plan
How sverify handles it:
- DataFusion refuses those plans
Status: filed #27059, open; consumer fix PR #27072 open. Reproduces on 55.0.0 and main
See: upstream https://github.com/apache/datafusion/issues/27059, https://github.com/apache/datafusion/pull/27072; pull requests #215.
Found by #215.
- Checked 2026-11-03 against datafusion 55.0.0: against the latest releases
- Checked 2026-11-04 against datafusion 55.0.0, substrait 0.37.1, datafusion core 55.1.0: reproduces on 51.0.0 and 55.0.0; main read at ee55effccc
DF-11: NTILE(k) computes ⌊i · min(k, n) / n⌋ + 1, which spreads the larger buckets out; the SQL standard (and …
NTILE(k) computes ⌊i · min(k, n) / n⌋ + 1, which spreads the larger buckets out; the SQL standard (and DuckDB, Spark, Snowflake, PostgreSQL) puts the n % k larger buckets first.
Reproduction:
select ntile(4) over (order by i) from (values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) t(i)gives1 1 1 2 2 3 3 3 4 4; the standard gives1 1 1 2 2 2 3 3 4 4(DataFusion 51.0.0)
How sverify handles it:
- Lean’s
ntileOfis the standard’s;datafusion_planrefuses NTILE
Status: fixed upstream in 54.0.0 (#22049, PR #22051)
See: upstream https://github.com/apache/datafusion/issues/22049, https://github.com/apache/datafusion/pull/22051; files python/src/sverify/engine.py, docs/reference-semantics.md.
Found by M2 (ntile).
- Checked 2026-11-03 against datafusion 55.0.0: against the latest releases
- Checked 2026-11-04 against datafusion 55.0.0, datafusion core 55.1.0: 51.0.0 wrong, 55.0.0 right; 54.0.0 tag contains 2bf1db5
DuckDB
DD-1: levenshtein counts bytes, not code points, on non-ASCII text, and reverse works on grapheme clusters
levenshtein counts bytes, not code points, on non-ASCII text, and reverse works on grapheme clusters. Possibly intended: check the docs before filing.
Reproduction:
levenshtein('é', 'e')
How sverify handles it:
- sverify raises “non-ASCII text differs between engines” for these on DuckDB (#195)
Status: levenshtein: filed #25113, closed 2026-10-28 as fixed on main by #27619 (backported to v2.0-cyanoptera); 2.0.0 gives 2, nightlies 2.0.1.dev2610291258 and 2.1.0.dev2610310630 give 1; not in a release yet. reverse: not a bug, it reverses grapheme clusters on purpose.
See: upstream https://github.com/duckdb/duckdb/issues/25113, https://github.com/duckdb/duckdb/pull/27619; pull requests #195.
Found by #195.
- Checked 2026-11-03 against duckdb 2.0.0: against the latest releases
- Checked 2026-11-04 against duckdb 2.0.0: also nightlies 2.0.1.dev2610291258 and 2.1.0.dev2610310630, which give 1
- Checked 2026-11-04 against duckdb 2.0.0: reverse pass: levenshtein’s DP runs over bytes (extension/core_functions/scalar/string/levenshtein.cpp#L15-L60), reverse over grapheme clusters (extension/core_functions/scalar/string/reverse.cpp#L24-L29); the text docs say ‘single-character edits’; no upstream test covers either on non-ASCII text
DD-2: error() with a message built by concat shows a NULL argument as an empty string
error() with a message built by concat shows a NULL argument as an empty string. That’s concat’s documented NULL handling.
Reproduction:
error(concat('x: ', NULL))
How sverify handles it:
- Guards keep NULL out of the error branch (#255)
Status: not a bug: the docs say concat skips NULL inputs; same on 2.0.0 and the 2.0.1/2.1.0 nightlies.
See: pull requests #255.
Found by #255.
- Checked 2026-11-03 against duckdb 2.0.0: against the latest releases
- Checked 2026-11-04 against duckdb 2.0.0: also nightlies 2.0.1.dev2610291258 and 2.1.0.dev2610310630
- Checked 2026-11-04 against duckdb 2.0.0: reverse pass: concat skips NULLs (src/function/scalar/string/concat.cpp#L40-L70), as the docs state (sql/functions/text#concatvalue-); pinned by test/sql/function/string/test_concat.test#L68-L72
DD-3: VAR_POP and STDDEV_POP of a single NaN are 0.0, not NaN; with a second value they are NaN
VAR_POP and STDDEV_POP of a single NaN are 0.0, not NaN; with a second value they are NaN. IEEE and every other engine propagate the NaN.
Reproduction:
select var_pop(x) from (values ('nan'::double)) t(x)gives0.0
How sverify handles it:
- The SQL writer returns NaN when an input is NaN (#54)
Status: filed #28329, open (reproduced); fix #28344 open. Reproduces on 2.0.0 and the 2.0.1.dev2610291258 and 2.1.0.dev2610310630 nightlies.
See: issues #54; upstream https://github.com/duckdb/duckdb/issues/28329, https://github.com/duckdb/duckdb/pull/28344; files python/src/sverify/sql/core/_base.py.
Found by #54.
- Checked 2026-11-03 against duckdb 2.0.0: against the latest releases
- Checked 2026-11-04 against duckdb 2.0.0: also nightlies 2.0.1.dev2610291258 and 2.1.0.dev2610310630
- Checked 2026-11-04 against duckdb 2.0.0: reverse pass: VarPop/STDDevPop Finalize returns count > 1 ? dsquared / count : 0, so a single NaN gives 0 (extension/core_functions/include/core_functions/aggregate/algebraic/stddev.hpp#L93-L123); probe select var_pop(x), stddev_pop(x) from (values (‘nan’::double)) t(x) gives (0.0, 0.0); the var_pop docs are silent on one value and NaN; no upstream test
DD-4: read_parquet trusts a file’s min/max statistics for doubles, which Parquet writers compute without NaN
read_parquet trusts a file’s min/max statistics for doubles, which Parquet writers compute without NaN. Comparisons are folded from them and row groups skipped, though DuckDB orders NaN above every double: NaN rows go missing or compare wrongly.
Reproduction:
- A Parquet file of
x = [0.5, NaN]written by pyarrow 26 (min = max = 0.5):select x <= 100.0 from read_parquet(f)is TRUE for the NaN,... where x > 100.0counts 0 rows,select max(x)gives 0.5.
How sverify handles it:
- A
Scanreads each double column throughCASE WHEN isnan(x) THEN 'NaN' ELSE x + 0.0 END, which DuckDB keeps no statistics for (#173). Settingcan_have_nan := trueonread_parquetmight replace theisnanrewrite; that is for the sverify owners to decide.
Status: filed #25521, open (reproduced); reproduces on 2.0.0 and the 2.0.1/2.1.0 nightlies. Main adds an opt-in parquet_can_have_nan setting (#27969); the default still trusts the statistics. The wrong max(x) is new since 1.5 (1.4.1 gives NaN). The Parquet docs document the can_have_nan option (default false), with which the reader accounts for NaN in a column’s min/max statistics (https://duckdb.org/docs/2.0/data/parquet/overview#parameters): the wrong results under the default are documented behaviour, and the report should cite the option.
See: issues #173; upstream https://github.com/duckdb/duckdb/issues/25521, https://github.com/duckdb/duckdb/pull/27969; files python/tests/test_io_native.py, docs/how-it-works.md.
Found by #173.
- Checked 2026-11-03 against duckdb 2.0.0: against the latest releases
- Checked 2026-11-04 against duckdb 2.0.0, pyarrow 26.0.1: also nightlies 2.0.1.dev2610291258 and 2.1.0.dev2610310630
- Checked 2026-11-04 against duckdb 2.0.0: reverse pass: float stats are trusted unless nan_count != 0 or can_have_nan (extension/parquet/parquet_statistics.cpp#L90-L117, #L518-L527; default false at extension/parquet/include/parquet_reader.hpp#L302). The docs state can_have_nan’s default and effect (data/parquet/overview#parameters), so the default path is documented; test/sql/copy/parquet/parquet_nan.test#L14-L37 pins only can_have_nan := true
DD-5: DuckDB exports HUGEINT and UHUGEINT to Arrow as decimal128(38, 0) without a range check
DuckDB exports HUGEINT and UHUGEINT to Arrow as decimal128(38, 0) without a range check. A UHUGEINT of 2^127 or more arrives as a negative decimal (the maximum as -1), and positive values of 10^38 or more are decimals Arrow’s validation rejects (pyarrow 26.0.1 accepts -2^127). Nothing raises.
Reproduction:
- ```python import duckdb duckdb.sql(“SELECT ‘340282366920938463463374607431768211455’::UHUGEINT AS u, ‘170141183460469231731687303715884105727’::HUGEINT AS h”).to_arrow_table().to_pylist()
- observed: `[{'u': Decimal('-1'), 'h': Decimal('170141183460469231731687303715884105727')}]`, and `validate(full=True)` of `h` raises "Decimal value … does not fit in precision of decimal128(38, 0)".
How sverify handles it:
- sverify has no 128-bit integers. The replay harness (`sverify_dev.slt`) mirrors a `HUGEINT` or `UHUGEINT` column through Arrow only when all its values are below 10^38 in magnitude, which export exactly, and refuses the table otherwise, or when the column is nested (#618)
Status: known upstream: [duckdb-python#568](https://github.com/duckdb/duckdb-python/issues/568) (UHUGEINT >= 2^127 negative), open, untriaged; the lossy decimal mapping is deliberate ([duckdb-python#104](https://github.com/duckdb/duckdb-python/issues/104)). The HUGEINT >= 10^38 half has no issue. Reproduces on 2.0.0 and the 2.0.1/2.1.0 nightlies; `arrow_converter.cpp` unchanged on main (02aa9fa).
See: issues [#618](https://github.com/AlexanderNenninger/substrait-lean/issues/618); upstream <https://github.com/duckdb/duckdb-python/issues/568>; files [`python/tools/sverify_dev/slt.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/tools/sverify_dev/slt.py), [`python/tests/test_slt.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/tests/test_slt.py).
Found by the characterization pipeline's round 3 (DuckDB's `uhugeint_try_cast.test` L115 replayed through `sverify_dev.slt`).
- Checked 2026-11-04 against duckdb 2.0.0: Python client 2.0.0, pyarrow from the repository's lock
- Checked 2026-11-04 against duckdb 2.0.0, pyarrow 26.0.1: also nightlies 2.0.1.dev2610291258 and 2.1.0.dev2610310630
### DD-6: `least` and `greatest` on `FLOAT` return a value that is none of their arguments when the winner is `-0.0` … {#DD-6}
::: {.ledger-meta}
Project: DuckDB 2.0.0; status: confirmed.
:::
`least` and `greatest` on `FLOAT` return a value that is none of their arguments when the winner is `-0.0` or a negative NaN: they compare sort keys and decode the winner from its key, which encodes both zeros alike and every NaN as the positive quiet NaN. So `least(-0.0::FLOAT, -0.0::FLOAT)` is `+0.0`. On `DOUBLE` they keep the argument (`least(-0.0, -0.0)` is `-0.0`, `least(-0.0, 0.0)` is `-0.0`), so the two widths disagree.
Reproduction:
- `SELECT signbit(least(-0.0::FLOAT, -0.0::FLOAT)), signbit(least(-0.0::DOUBLE, -0.0::DOUBLE))` gives `false, true`.
- `SELECT signbit(-'nan'::FLOAT), signbit(least(-'nan'::FLOAT, -'nan'::FLOAT)), signbit(least(-'nan'::DOUBLE, -'nan'::DOUBLE))` gives `true, false, true`.
- The same from table columns: `CREATE TABLE t AS SELECT -0.0::FLOAT f, -0.0::FLOAT g, -0.0::DOUBLE d, -0.0::DOUBLE e; SELECT signbit(least(f, g)), signbit(least(d, e)) FROM t` gives `false, true`. (`least(f, f)` is simplified to `f` and keeps the sign.)
How sverify handles it:
- No effect on sverify's results: the reference has one zero and one NaN, and sverify's SQL normalises `-0.0` to `0.0` (duckdb-R14, duckdb-float-signs).
Status: confirmed on 2.0.0 (Python, macOS arm64) by the probe observations least-greatest-zero, least-greatest-zero-columns and least-greatest-nan-sign. Reproduces on 2.0.0 and on both nightly builds (checked 2026-11-04 by the upstream-verify session).
See: facts [duckdb-float-signs](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/duckdb-facts.md#duckdb-float-signs); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444); files [`python/tools/sverify_dev/probe.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/tools/sverify_dev/probe.py), [`characterization/duckdb/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/duckdb/probes.json).
Found by characterization pipeline round 4 probe (#444), from round 2's source reading.
- Checked 2026-11-04 against duckdb 2.0.0: probe on the pinned release; main not checked
- Checked 2026-11-04 against duckdb nightly: upstream-verify session: reproduces on both nightly builds
- Checked 2026-11-04 against duckdb 2.0.0: reverse pass: FLOAT least/greatest go through SortKeyLeastGreatest (extension/core_functions/scalar/generic/least.cpp#L81-L110, #L267-L296); the numeric docs are silent on signed zero and NaN; test_least_greatest.test has no -0.0 or NaN case
### DD-7: A `CASE` whose later `WHEN` is a constant expression that raises fails while planning, even when no row … {#DD-7}
::: {.ledger-meta}
Project: DuckDB 2.0.0; status: confirmed.
:::
A `CASE` whose later `WHEN` is a constant expression that raises fails while planning, even when no row reaches that `WHEN`: on a table where every row takes the first branch, and on an empty table. The optimizer's `CASE` simplification evaluates every foldable `WHEN` with `ExpressionExecutor::EvaluateScalar`, which throws, where constant folding uses `TryEvaluateScalar` and leaves a failing expression to run time.
At run time `CASE` is lazy (a raising `ELSE` on rows that never reach it does not raise), so the plan-time fold is the one place where a `CASE` branch is evaluated eagerly.
Reproduction:
- `CREATE TABLE t AS SELECT 1 x; SELECT CASE WHEN x > 0 THEN 1 WHEN 1 // 0 = 1 THEN 2 END FROM t;` raises `Invalid Input Error: Division by zero in expression (1 // 0)`. The same over `CREATE TABLE t (x INTEGER)` (no rows), and `EXPLAIN` of it, raise too; `WHEN 'a'::INTEGER = 1` raises `Conversion Error: Could not convert string 'a' to INT32`.
- For contrast, `SELECT CASE WHEN x > 0 THEN 1 ELSE 1 // 0 END FROM t` returns 1.
How sverify handles it:
- No workaround yet: duckdb-exec-short-circuit records it for the evaluation-order analysis (#604). The fix upstream would be `TryEvaluateScalar` in `CaseSimplificationRule::Apply`, keeping the `WHEN` when folding fails (https://github.com/duckdb/duckdb/blob/v2.0.0/src/optimizer/rule/case_simplification.cpp#L15-L46).
- Since D21 (docs/core-semantics.md, #609) the reference itself raises on a constant subexpression that raises, in any CASE branch and on an empty table too (`Rel.constIssues`), so DuckDB's plan-time fold (DD-7) no longer breaks a verified plan's no-error guarantee: such a plan is not verified error-free.
Status: confirmed on 2.0.0 (Python, macOS arm64) by the probe observations exec-case-when-folded, exec-case-when-folded-empty and exec-case-when-folded-cast. Fixed in DuckDB's nightly builds (checked 2026-11-04 by the upstream-verify session), not yet in a release: it becomes "fixed upstream" once a release has the fix.
See: facts [duckdb-exec-short-circuit](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/duckdb-facts.md#duckdb-exec-short-circuit); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444), [#604](https://github.com/AlexanderNenninger/substrait-lean/issues/604); files [`python/tools/sverify_dev/probe.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/tools/sverify_dev/probe.py), [`characterization/duckdb/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/duckdb/probes.json).
Found by characterization pipeline round 4 probe (#444), from round 2's source reading.
- Checked 2026-11-04 against duckdb 2.0.0: probe on the pinned release; main not checked
- Checked 2026-11-04 against duckdb nightly: upstream-verify session: fixed in the nightly builds; no release has the fix yet
- Checked 2026-11-04 against duckdb 2.0.0: reverse pass: case_simplification.cpp#L15-L46 evaluates a foldable WHEN with EvaluateScalar; the CASE docs (https://duckdb.org/docs/2.0/sql/expressions/case) are silent; no upstream test has a raising constant WHEN (test/optimizer/case_simplification.test#L1-L34)
### DD-8: The documentation, not the engine: the numeric functions page … {#DD-8}
::: {.ledger-meta}
Project: DuckDB 2.0.0; status: confirmed; kind: bug.
:::
The documentation, not the engine: the numeric functions page (https://duckdb.org/docs/2.0/sql/functions/numeric.html, "Division and Modulo Operators") says "Integer division (`//`), modulo (`%`), and `INTERVAL / 0` throw an `Invalid Input Error` on division by zero instead of returning `NULL`." For FLOAT and DOUBLE, `%` by zero is NaN under the default `ieee_floating_point_ops`: `%` of floats binds `BindBinaryFloatingPoint<ModuloOperator>` (https://github.com/duckdb/duckdb/blob/v2.0.0/src/function/scalar/operator/arithmetic.cpp#L1452-L1469), which runs `std::fmod` without a zero check.
On the same page, the function `fmod` is documented as floored (true: `fmod(-7.5, 2)` is 0.5), while `%` on doubles is C's truncating `fmod` (`-7.5::DOUBLE % 2` is -1.5), and the page never says so: the C name and the operator behave oppositely without a word.
Suggested fix: "modulo (`%`) of integers and decimals", and a sentence that `%` on FLOAT and DOUBLE truncates (C's `fmod`) while the function `fmod` floors.
Reproduction:
- `SELECT 1.0::DOUBLE % 0, 0.0::DOUBLE % 0, 1.0::FLOAT % 0` gives `nan, nan, nan`; `SELECT 1::INTEGER % 0` raises `Invalid Input Error`.
- `SELECT -7.5::DOUBLE % 2, fmod(-7.5, 2)` gives `-1.5, 0.5`.
How sverify handles it:
- No effect on sverify's results: its DuckDB SQL guards a zero divisor of `%` on doubles and writes NaN and ±inf out (duckdb-R15-double-modulus).
Status: A bug in the documentation, confirmed on 2.0.0 (Python, macOS arm64) against https://duckdb.org/docs/2.0/sql/functions/numeric.html; not filed. The engine's NaN is pinned by its own test (test/sql/types/float/ieee_floating_points.test#L23-L31), so the page is what is wrong.
See: facts [duckdb-R15-double-modulus](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/duckdb-facts.md#duckdb-R15-double-modulus); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444).
Found by characterization pipeline reverse pass (#444): the docs read against duckdb-R15-double-modulus.
- Checked 2026-11-04 against duckdb 2.0.0: probe on the pinned release and the 2.0 docs; main and the nightly docs not checked
### DD-9: The documentation, not the engine: the numeric functions page … {#DD-9}
::: {.ledger-meta}
Project: DuckDB 2.0.0; status: confirmed; kind: bug.
:::
The documentation, not the engine: the numeric functions page (https://duckdb.org/docs/2.0/sql/functions/numeric.html, the operator table and the `pow`/`power` entries) shows `3 ** 4` → `81`, `3 ^ 4` → `81`, `pow(2, 3)` → `8` and `power(2, 3)` → `8`, and never states the result type. The result is always DOUBLE: the function has one signature, `(DOUBLE, DOUBLE) -> DOUBLE` (https://github.com/duckdb/duckdb/blob/v2.0.0/extension/core_functions/scalar/math/numeric.cpp#L1402-L1408). The same page prints DOUBLE results as `1.0` (`fmod(5, 2)`), so the integer-looking examples suggest an integer power.
Suggested fix: show `81.0` and `8.0`, and say the result is DOUBLE: integer arguments convert, so a BIGINT above 2^53 loses digits and nothing checks integer overflow.
Reproduction:
- `SELECT 3 ** 4, typeof(3 ** 4), pow(2, 3), typeof(pow(2, 3))` gives `81.0, DOUBLE, 8.0, DOUBLE`.
How sverify handles it:
- No effect on sverify's results: its DuckDB SQL never calls `POWER` for an integer power and writes it out as checked BIGINT multiplication (duckdb-int-pow).
Status: A minor bug in the documentation, confirmed on 2.0.0 (Python, macOS arm64) against https://duckdb.org/docs/2.0/sql/functions/numeric.html; not filed.
See: facts [duckdb-int-pow](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/duckdb-facts.md#duckdb-int-pow); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444); files [`python/src/sverify/sql/duckdb.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/src/sverify/sql/duckdb.py).
Found by characterization pipeline reverse pass (#444): the docs read against duckdb-int-pow.
- Checked 2026-11-04 against duckdb 2.0.0: probe on the pinned release and the 2.0 docs; main and the nightly docs not checked
## Ibis
### IB-1: Ibis reads the deprecated `pd.options.future.infer_string` in `PandasType.from_ibis` … {#IB-1}
::: {.ledger-meta}
Project: Ibis 12.0.0; status: confirmed.
:::
Ibis reads the deprecated `pd.options.future.infer_string` in `PandasType.from_ibis` (`ibis/formats/pandas.py:60`), so every `execute()` warns `Pandas4Warning: The 'future.infer_string' option is deprecated` under pandas 3.1, also for a scalar with no string in it.
Reproduction:
- `python -W error::DeprecationWarning -c "import ibis; ibis.literal(1).execute()"` raises `Pandas4Warning`.
How sverify handles it:
- The checks on data fetch their counts through Arrow (`_count` in `contracts.py`: `to_pyarrow().as_py()`), so users see no warning ([#479](https://github.com/AlexanderNenninger/substrait-lean/issues/479)). The tests still ignore the message (`filterwarnings` in `python/pyproject.toml`).
Status: Reproduces on 12.0.0 with pandas 3.1.0 and 3.1.1, and on Ibis main (f73d329, 2026-11-02).
See: issues [#479](https://github.com/AlexanderNenninger/substrait-lean/issues/479); files [`python/src/sverify/contracts.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/src/sverify/contracts.py), [`python/pyproject.toml`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/pyproject.toml).
Found by #479.
- Checked 2026-11-04 against ibis-framework 12.0.0, pandas 3.1.0: confirmed in 429157a (#479)
- Checked 2026-11-04 against ibis-framework 12.0.0, ibis main main f73d329, pandas 3.1.1, duckdb 1.5.6: also pandas 3.1.0
### IB-2: Ibis's DuckDB SQL for `TimeTruncate` is `DATE_TRUNC('HOUR', t)` of a TIME, which DuckDB has no overload … {#IB-2}
::: {.ledger-meta}
Project: Ibis 12.0.0; status: confirmed.
:::
Ibis's DuckDB SQL for `TimeTruncate` is `DATE_TRUNC('HOUR', t)` of a TIME, which DuckDB has no overload for, so `t.time().truncate("h")` fails to bind on DuckDB.
Reproduction:
- `ibis.to_sql(t.select(x=t.ts.time().truncate("h")), dialect="duckdb")` gives `DATE_TRUNC('HOUR', CAST("t0"."ts" AS TIME))`; DuckDB 2.0.0 raises `Binder Error: No function matches the given name and argument types 'date_trunc(STRING_LITERAL, TIME)'`.
How sverify handles it:
- sverify's DuckDB SQL truncates a time of day on 1970-01-01 and casts back to TIME (`DuckDBDialect.time_trunc`). The coverage case "time truncate" is compared with Lean and the other engines rather than with Ibis's DuckDB SQL (`NO_DUCKDB_SQL` in `coverage_cases/temporal.py`).
Status: Reproduces on 12.0.0 with DuckDB 2.0.0 and 1.5.6, and on Ibis main (f73d329, 2026-11-02).
See: facts [duckdb-time-plus-interval](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/duckdb-facts.md#duckdb-time-plus-interval); issues [#507](https://github.com/AlexanderNenninger/substrait-lean/issues/507); files [`python/src/sverify/sql/duckdb.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/src/sverify/sql/duckdb.py), [`python/tools/sverify_dev/coverage_cases/temporal.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/tools/sverify_dev/coverage_cases/temporal.py).
Found by #507.
- Checked 2026-11-04 against ibis-framework 12.0.0, duckdb 2.0.0: confirmed while adding the coverage case (#507)
- Checked 2026-11-04 against ibis-framework 12.0.0, ibis main main f73d329, duckdb 2.0.0: also duckdb 1.5.6
### IB-3: Ibis's DuckDB SQL for `argmin`/`argmax` is `FIRST(x ORDER BY key)` {#IB-3}
::: {.ledger-meta}
Project: Ibis 12.0.0; status: confirmed.
:::
Ibis's DuckDB SQL for `argmin`/`argmax` is `FIRST(x ORDER BY key)`. A NULL `x` at the extreme key is intended (since ibis#10227). In a group whose keys are all NULL, it returns the `x` of a NULL-key row, where Ibis's Postgres SQL filters `key IS NOT NULL` and returns NULL.
Reproduction:
- For `t = ibis.memtable({"x": [1, None], "k": [1, 2]})`, `t.x.argmax(t.k)` on the DuckDB backend gives NULL (`FIRST("x" ORDER BY "k" DESC)`), while DuckDB's `max_by(x, k)` and `arg_max(x, k)` give 1. For `x = [5]` and `k = [NULL]`, `argmin` gives 5 and `arg_min` NULL.
How sverify handles it:
- sverify's producer gives `argmin`/`argmax` DuckDB's `arg_min`/`arg_max` meaning, the rows where neither side is NULL (`aggregates.arg_min_max`, `duckdb-arg-min-nulls`). The coverage case "argmin, argmax" is compared with Lean and the other engines rather than with Ibis's DuckDB SQL (`NO_DUCKDB_SQL` in `coverage_cases/aggregation.py`), and the catalogue's `iot-downsample` differs from Ibis's DuckDB SQL on synthetic data with NULL readings.
Status: Reproduces on 12.0.0 and Ibis main (f73d329, 2026-11-02). A NULL `x` at the extreme key is Ibis's chosen meaning ([#10227](https://github.com/ibis-project/ibis/pull/10227), [#10193](https://github.com/ibis-project/ibis/issues/10193)); the difference left is a group whose keys are all NULL: DuckDB gives an `x` of a NULL-key row, Ibis's Postgres SQL filters `key IS NOT NULL` and gives NULL.
See: facts [duckdb-arg-min-nulls](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/duckdb-facts.md#duckdb-arg-min-nulls); issues [#53](https://github.com/AlexanderNenninger/substrait-lean/issues/53); files [`python/src/sverify/ibis_substrait/aggregates.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/src/sverify/ibis_substrait/aggregates.py), [`python/tools/sverify_dev/coverage_cases/aggregation.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/tools/sverify_dev/coverage_cases/aggregation.py).
Found by #53.
- Checked 2026-11-04 against ibis-framework 12.0.0, duckdb 2.0.0: found by the catalogue probe of iot-downsample (#53)
- Checked 2026-11-04 against ibis-framework 12.0.0, ibis main main f73d329, duckdb 2.0.0: also duckdb 1.5.6
### IB-4: `ibis.window(preceding=10, following=-1)` compiles to `ROWS BETWEEN 10 preceding AND -1 following` {#IB-4}
::: {.ledger-meta}
Project: Ibis 12.0.0; status: confirmed.
:::
`ibis.window(preceding=10, following=-1)` compiles to `ROWS BETWEEN 10 preceding AND -1 following`. The SQL standard and DuckDB take only non-negative offsets, so DuckDB refuses the query; the signed offset means `1 PRECEDING`.
Reproduction:
- For `t = ibis.table({'s': 'string', 'm': 'timestamp', 'v': 'int64'}, name='t')`, `ibis.to_sql(t.mutate(a=t.v.mean().over(ibis.window(group_by='s', order_by='m', preceding=10, following=-1))), dialect='duckdb')` writes `ROWS BETWEEN 10 preceding AND -1 following`, and DuckDB raises `Invalid Input Error: Window ROWS FOLLOWING expression cannot be negative`. `preceding=-2` writes `-2 preceding` likewise.
How sverify handles it:
- sverify's Ibis producer reads a negative constant offset as the same number of rows on the other side (`following=-1` is `1 PRECEDING`, `windows._bound`), so the catalogue's `fin-volume-spike` verifies and runs; its `native` status (Ibis's own DuckDB SQL) stays an error.
Status: Reproduces on 12.0.0 with DuckDB 2.0.0. Not yet checked against Ibis main.
See: files [`python/src/sverify/ibis_substrait/windows.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/src/sverify/ibis_substrait/windows.py), [`python/tests/test_windows.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/tests/test_windows.py).
Found by catalogue probe of fin-volume-spike.
- Checked 2026-11-05 against ibis-framework 12.0.0, duckdb 2.0.0: ibis.to_sql and DuckDB's error, from the catalogue probe
## Polars
### PL-1: `%` on floats computes `x - y * floor(x / y)`, so a finite `x % inf` is NaN (`inf * 0`), where Python's … {#PL-1}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
`%` on floats computes `x - y * floor(x / y)`, so a finite `x % inf` is NaN (`inf * 0`), where Python's `%`, which Polars's `%` follows on integers, gives `x` for positive `x` and `inf` for negative `x`. `x % -inf` and `0.0 % inf` are NaN as well.
Reproduction:
- import polars as pl
pl.DataFrame({"a": [5.0, -5.0, 0.0], "b": [float("inf")] * 3}).select(pl.col("a") % pl.col("b")).to_series().to_list()
# [nan, nan, nan]; Python: [5.0 % inf, -5.0 % inf, 0.0 % inf] == [5.0, inf, 0.0]
How sverify handles it:
- No Polars profile yet; the characterization records it (polars-double-modulus). A writer for the reference's `fmod` would not use `%` for doubles.
Status: Reproduces on Polars 2.1.0, the latest release (in-memory and streaming); `prim_wrapping_mod` is unchanged on main (35a2cb73, 2026-11-04), read in the source. No issue on `x % inf`; [pola-rs/polars#30398](https://github.com/pola-rs/polars/issues/30398) (open) reports the same kernel's lost remainder for `1e20 % 3.0`, and the exact-`fmod` fix proposed there would cover this case too.
See: facts [polars-double-modulus](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-double-modulus); upstream <https://github.com/pola-rs/polars/issues/30398>; files [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json).
Found by the Polars characterization (sverify_dev.probe_polars, `Float64 % Float64`).
- Checked 2026-11-04 against polars 2.1.0: crates/polars-compute/src/arithmetic/float.rs at py-2.1.0, prim_wrapping_mod
- Checked 2026-11-04 against polars 2.1.0, polars main 35a2cb73fdf81d89b2117bec152c3530eb3e4146: repros/PL-1.py on 2.1.0 (macOS arm64), in-memory and streaming; main source only: crates/polars-compute/src/arithmetic/float.rs. Duplicate search via gh issue list --state all.
### PL-2: A `Date` is an `Int32` of days, but chrono's `NaiveDate` covers only about ±262 000 years {#PL-2}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
A `Date` is an `Int32` of days, but chrono's `NaiveDate` covers only about ±262 000 years. A cast of a larger day count to `Date` succeeds, and then casting it to `String`, printing the frame or reading the value panics (`PanicException: out-of-range date`, from `date32_to_date`'s `expect`) where an error or NULL is expected; `dt.year()` of it is NULL.
Reproduction:
- import polars as pl
df = pl.DataFrame({"x": [2**31 - 1]}, schema={"x": pl.Int32}).select(pl.col("x").cast(pl.Date))
df.select(pl.col("x").dt.year()).item() # None
df.select(pl.col("x").cast(pl.String)) # PanicException: out-of-range date
How sverify handles it:
- No Polars profile yet; the characterization records it (polars-date-range). The reference's dates stay within 0001-01-01 … 9999-12-31 (#251), so a guarded plan never builds such a date.
Status: Reproduces on Polars 2.1.0, the latest release: the cast to `String` panics in-memory and streaming, as do printing and `item()`; `date32_to_date` is unchanged on main (35a2cb73, 2026-11-04), read in the source. No issue on `Date`; related: [pola-rs/polars#25742](https://github.com/pola-rs/polars/issues/25742) (open, the same panic for out-of-range `Datetime`) and [#11991](https://github.com/pola-rs/polars/issues/11991) (closed by #25471, which meant `dt.year()` of such a date to raise; 2.1.0 returns NULL).
See: facts [polars-date-range](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-date-range); issues [#251](https://github.com/AlexanderNenninger/substrait-lean/issues/251); upstream <https://github.com/pola-rs/polars/issues/25742>, <https://github.com/pola-rs/polars/issues/11991>; files [`characterization/polars/probes.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes.json).
Found by the Polars characterization (sverify_dev.probe_polars, `date-physical-range`).
- Checked 2026-11-04 against polars 2.1.0: crates/polars-arrow/src/temporal_conversions.rs at py-2.1.0, date32_to_date
- Checked 2026-11-04 against polars 2.1.0, polars main 35a2cb73fdf81d89b2117bec152c3530eb3e4146: repros/PL-2.py on 2.1.0 (macOS arm64), each step in a subprocess, in-memory and streaming; main source only: crates/polars-arrow/src/temporal_conversions.rs date32_to_date.
### PL-3: When one side of a float `/`, `//` or `%` has length 1 and the other more (a literal divisor on a frame of … {#PL-3}
::: {.ledger-meta}
Project: Polars 2.1.0; status: filed; kind: bug.
:::
When one side of a float `/`, `//` or `%` has length 1 and the other more (a literal divisor on a frame of two rows or more), Polars computes the reciprocal of the divisor once and multiplies: `x / c` is `x * (1 / c)`, `x // c` is `floor(x * (1 / c))`, `x % c` is `x - c * floor(x * (1 / c))` (`prim_true_div_scalar`, `prim_wrapping_floor_div_scalar`, `prim_wrapping_mod_scalar` in crates/polars-compute/src/arithmetic/float.rs).
So `49.0 / 49.0` is 0.9999999999999999, `49.0 // 49.0` is 0.0 and `49.0 % 49.0` is 49.0, a remainder equal to the divisor. For a subnormal divisor the reciprocal is inf: `0.0 / 1e-310` is NaN and `1e-300 / 1e-310` is inf (IEEE: 0.0 and 10000000000.00003); `%` gives NaN and -inf. `Float32` does the same below about 2.9e-39. On one row, or with two columns, the IEEE kernel runs and gives 1.0, 1.0, 0.0.
Reproduction:
- import polars as pl
df = pl.DataFrame({"x": [49.0, 0.0, 1e-300]})
df.select(div=pl.col("x") / 49.0, floordiv=pl.col("x") // 49.0, mod=pl.col("x") % 49.0, div_subnormal=pl.col("x") / 1e-310).to_dicts()
# [{'div': 0.9999999999999999, 'floordiv': 0.0, 'mod': 49.0, 'div_subnormal': inf},
# {'div': 0.0, 'floordiv': 0.0, 'mod': 0.0, 'div_subnormal': nan},
# {'div': 2.0408163265306122e-302, 'floordiv': 0.0, 'mod': 1e-300, 'div_subnormal': inf}]
df.head(1).select(div=pl.col("x") / 49.0, floordiv=pl.col("x") // 49.0, mod=pl.col("x") % 49.0).to_dicts()
# [{'div': 1.0, 'floordiv': 1.0, 'mod': 0.0}]
- `pl.LazyFrame({"x": [49.0, 0.0, 1e-300] * 2000})` with the same select and `collect(engine="streaming")` gives the same rows as the 3-row frame. The full script is `repros/PL-3.py` in the reports folder.
How sverify handles it:
- sverify does not work around it yet. The Polars profile must widen `fp.div` (`rounding`, `nan_creation`) and `fp.mod` (`division_type`) for a literal divisor, or shim them, for example by dividing two columns.
Status: filed [pola-rs/polars#30594](https://github.com/pola-rs/polars/issues/30594), open. Reproduces on Polars 2.1.0, the latest release, in-memory and streaming (two rows suffice); the scalar kernels are unchanged on main (35a2cb73, 2026-11-04), read in the source. Known in part: the reciprocal's ulp errors in `/` and `//` were closed as intended ([pola-rs/polars#23741](https://github.com/pola-rs/polars/issues/23741), [#14596](https://github.com/pola-rs/polars/issues/14596)); [#30398](https://github.com/pola-rs/polars/issues/30398) (open) covers the inexact float `%`. Not reported: a subnormal divisor's reciprocal overflowing to inf, and `49.0 % 49.0` = 49.0.
See: facts [polars-double-scalar-divisor](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-double-scalar-divisor), [polars-double-divide](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-double-divide), [polars-double-modulus](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-double-modulus), [polars-fp-ieee](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-fp-ieee); upstream <https://github.com/pola-rs/polars/issues/30594>, <https://github.com/pola-rs/polars/issues/23741>, <https://github.com/pola-rs/polars/issues/14596>, <https://github.com/pola-rs/polars/issues/30398>; files [`characterization/polars/facts/polars-double-scalar-divisor.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-double-scalar-divisor.json), [`characterization/polars/facts/polars-double-divide.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-double-divide.json), [`characterization/polars/facts/polars-double-modulus.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-double-modulus.json), [`characterization/polars/facts/polars-fp-ieee.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-fp-ieee.json).
Found by characterizing Polars against the formalization (characterization/polars/facts, round 2 source reading of the arithmetic kernels, then a manual repro).
- Checked 2026-11-04 against polars 2.1.0: repro in-memory and streaming, macOS arm64; crates/polars-compute/src/arithmetic/float.rs at py-2.1.0
- Checked 2026-11-04 against polars main 35a2cb73fdf81d89b2117bec152c3530eb3e4146: source only: float.rs's scalar kernels unchanged (last change #29356, a rename); no build. Duplicate search (gh issue list --search, gh search issues: modulo, reciprocal, subnormal, denormal, floordiv, division nan) found #23741, #14596, #30398, none on subnormal divisors or a remainder equal to the divisor.
- Checked 2026-11-04 against polars 2.1.0, polars main 35a2cb73fdf81d89b2117bec152c3530eb3e4146: repros/PL-3.py on 2.1.0 (macOS arm64), in-memory and streaming with POLARS_MAX_THREADS=1; main source only: float.rs scalar kernels.
- Checked 2026-11-04 against polars 2.1.0: the report's example with POLARS_VERBOSE=1 and pl.show_versions() on macOS arm64, Python 3.13.14, before filing
### PL-4: `Expr.log()` on `Float32` computes `f32::log(x, base)` with the default base `math.e` cast to `Float32`, … {#PL-4}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
`Expr.log()` on `Float32` computes `f32::log(x, base)` with the default base `math.e` cast to `Float32`, that is `ln(x) / ln(2.7182817)`, and `ln(2.7182817)` is 0.99999994 in single precision. So the natural logarithm of a `Float32` misses the correctly rounded value by an ulp for many inputs (`log(0.5)` is -0.69314724, numpy's `log` gives -0.6931472). `log10()` is `log(10.0)`, two roundings, also an ulp off (`log10(0.5)`). `Float64` is not affected for the natural log, since `ln(math.e)` rounds to 1.0.
Reproduction:
- import polars as pl
s = pl.Series("x", [0.5, 2.0, 10.0], dtype=pl.Float32)
s.log().to_list() # [-0.6931472420692444, 0.6931472420692444, 2.3025853633880615]
s.log10().to_list() # [-0.3010299801826477, 0.3010299801826477, 1.0]
# numpy float32: log [-0.6931471824645996, 0.6931471824645996, 2.3025851249694824], log10 [-0.3010300099849701, 0.3010300099849701, 1.0]
How sverify handles it:
- sverify does not work around it. `fp32` arithmetic is refused in every profile until #417's stage 3; a Polars profile that takes `fp32` must widen `fp.log` rounding or shim the natural log.
Status: Reproduces on Polars 2.1.0, the latest release; `LogSeries::log` and `Expr.log` are unchanged on main (35a2cb73, 2026-11-04), read in the source. No matching issue found; [pola-rs/polars#23741](https://github.com/pola-rs/polars/issues/23741) (closed as not planned) is the related ulp discussion for division. Not a bug by Polars's own terms (2026-11-04): Polars's user guide says it "does not provide guarantees on the error unless mentioned otherwise" for floating point ([data types, floating point](https://docs.pola.rs/user-guide/concepts/data-types/overview/#floating-point)), and a maintainer quoted that passage to close [pola-rs/polars#14596](https://github.com/pola-rs/polars/issues/14596), so a one-ulp error is within what Polars documents. To be filed as a feature request: `log` with base e, 2 or 10 could call `ln`, `log2` or `log10`, which is also faster.
See: facts [polars-log-base](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-log-base); issues [#417](https://github.com/AlexanderNenninger/substrait-lean/issues/417); upstream <https://github.com/pola-rs/polars/issues/23741>; files [`characterization/polars/facts/polars-log-base.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-log-base.json).
Found by characterizing Polars against the formalization (characterization/polars/facts, round 2 source reading of `LogSeries::log`, then a manual repro).
- Checked 2026-11-04 against polars 2.1.0, numpy 2.5.4: repro on macOS arm64; crates/polars-ops/src/series/ops/log.rs at py-2.1.0
- Checked 2026-11-04 against polars main 35a2cb73fdf81d89b2117bec152c3530eb3e4146: source only: log.rs and Expr.log unchanged. Duplicate search (float32 log, log precision, natural log float32, log ulp, log base e) found nothing matching.
- Checked 2026-11-04 against polars 2.1.0, numpy 2.5.4, polars main 35a2cb73fdf81d89b2117bec152c3530eb3e4146: repros/PL-4.py on 2.1.0 (macOS arm64); main source only: crates/polars-ops/src/series/ops/log.rs, py-polars/src/polars/expr/expr.py.
### PL-5: `pl.cov` and `pl.corr` of `Decimal` columns raise `InvalidOperationError` on the streaming engine and … {#PL-5}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
`pl.cov` and `pl.corr` of `Decimal` columns raise `InvalidOperationError` on the streaming engine and succeed on the in-memory engine. The streaming reductions (`new_cov_reduction`, `new_pearson_corr_reduction`) require a primitive numeric type, which a decimal is not, while the in-memory path casts decimals to `Float64`. The planner types the result as `Float64` for any numeric input, so the plan resolves and the query fails only at execution.
Reproduction:
- import polars as pl
from decimal import Decimal as D
lf = pl.LazyFrame(
{"a": [D("1.10"), D("2.20"), D("3.35")], "b": [D("1.00"), D("2.00"), D("4.00")]},
schema={"a": pl.Decimal(10, 2), "b": pl.Decimal(10, 2)},
)
lf.select(pl.cov("a", "b")).collect(engine="in-memory").item() # 1.6916666666666667
lf.select(pl.corr("a", "b")).collect(engine="in-memory").item() # 0.9843241382880894
lf.select(pl.cov("a", "b")).collect(engine="streaming")
# InvalidOperationError: `cov` operation not supported for dtype `decimal[10,2]`
lf.select(pl.corr("a", "b")).collect(engine="streaming")
# InvalidOperationError: `corr` operation not supported for dtype `decimal[10,2]`
How sverify handles it:
- sverify doesn't work around it yet. The workaround is to cast both inputs to `Float64` before `cov` or `corr`, which is what the in-memory engine does and gives the same result.
Status: Reproduces on Polars 2.1.0, the latest release, with `engine="streaming"`; the in-memory engine gives a result. The streaming reductions' `is_primitive_numeric` checks are unchanged on main (35a2cb73, 2026-11-04), read in the source; the one later change to `cov.rs`, [pola-rs/polars#30370](https://github.com/pola-rs/polars/pull/30370), does not touch them. No matching issue found.
See: facts [polars-cov](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-cov), [polars-corr](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-corr); files [`characterization/polars/facts/polars-cov.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-cov.json), [`characterization/polars/facts/polars-corr.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-corr.json).
Found by the Polars characterization round 2 (source reading of crates/polars-expr/src/reduce/cov.rs, then a manual repro).
- Checked 2026-11-04 against polars 2.1.0: crates/polars-expr/src/reduce/cov.rs at py-2.1.0 (L23-L30, L171-L178: `is_primitive_numeric` checks); crates/polars-expr/src/dispatch/misc.rs L967-L1040 (in memory: other types cast to Float64). Duplicate search: `gh search issues --repo pola-rs/polars` for "cov decimal", "corr decimal", "cov decimal streaming", "cov streaming", "corr streaming", "pearson_corr decimal", "cov operation not supported for dtype decimal" and "operation not supported for dtype decimal" found no matching issue (the last returned other closed Decimal issues: #26721 product, #24715 sign, #29435 floor division).
- Checked 2026-11-04 against polars 2.1.0, polars main 35a2cb73fdf81d89b2117bec152c3530eb3e4146: repros/PL-5.py on 2.1.0 (macOS arm64), both engines; main source only: crates/polars-expr/src/reduce/cov.rs, reduce/convert.rs.
### PL-6: Common-subexpression elimination hoists a fallible expression that two `when/then` branches share (a … {#PL-6}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
Common-subexpression elimination hoists a fallible expression that two `when/then` branches share (a strict cast) into a `WITH_COLUMNS` evaluated on every row, so the cast raises on the rows the branches' conditions exclude. Without CSE, `when/then` masks the branch and the query succeeds; an eager `DataFrame.select` (no CSE) succeeds too. The same query raises or not depending on an optimization.
Reproduction:
- import polars as pl
lf = pl.LazyFrame({"x": ["1", "a"]})
q = lf.select(
pl.when(pl.col("x") == "1").then(pl.col("x").cast(pl.Int64) + 1).alias("a"),
pl.when(pl.col("x") == "1").then(pl.col("x").cast(pl.Int64) + 2).alias("b"),
)
q.collect() # InvalidOperationError: conversion from `str` to `i64` failed in column 'x' for 1 out of 2 values: ["a"]
q.collect(engine="streaming") # the same error
q.collect(optimizations=pl.QueryOptFlags(comm_subexpr_elim=False)).rows() # [(2, 3), (None, None)]
print(q.explain())
# SELECT [when(col("__POLARS_CSER_…3")).then(col("__POLARS_CSER_…4") + 1)…]
# WITH_COLUMNS: [(col("x") == "1").alias("__POLARS_CSER_…3"), col("x").strict_cast(Int64).alias("__POLARS_CSER_…4")]
How sverify handles it:
- sverify doesn't work around it yet. Its Polars route collects lazily with CSE on, so a plan Lean proves error-free (a CASE guarding a cast, used in two columns) can raise. The workaround is to collect with `optimizations=pl.QueryOptFlags(comm_subexpr_elim=False)`, or to keep fallible expressions out of CSE (Polars could treat strict casts as it does in predicate pushdown's fallible list).
Status: Reproduces on Polars 2.1.0, the latest release, with a lazy `collect()` on both engines with CSE on; without CSE, and eagerly, the query succeeds. `csee.rs` is unchanged on main (35a2cb73, 2026-11-04), read in the source. No matching issue found (related: [pola-rs/polars#30252](https://github.com/pola-rs/polars/issues/30252), closed; [#20305](https://github.com/pola-rs/polars/issues/20305), open).
See: facts [polars-exec-cse-hoist](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-exec-cse-hoist), [polars-exec-when-eager](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-exec-when-eager); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444), [#604](https://github.com/AlexanderNenninger/substrait-lean/issues/604); files [`characterization/polars/facts/polars-exec-cse-hoist.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-exec-cse-hoist.json), [`characterization/polars/probes-streaming.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/probes-streaming.json), [`python/tools/sverify_dev/probe_polars.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/tools/sverify_dev/probe_polars.py).
Found by the Polars characterization round 4 (streaming probes, `sverify_dev.probe_polars --diff`, construct s-when-cse-lazy), after round 2's source reading of crates/polars-plan/src/plans/optimizer/cse/csee.rs.
- Checked 2026-11-04 against polars 2.1.0: Probe s-when-cse-lazy in characterization/polars/probes-streaming.json: raises with CSE on both engines, [(2, 3), (None, None)] without. Duplicate search (`gh issue list --repo pola-rs/polars --state all --search`) for "CSE when then", "subexpression when then", "when then otherwise cse", "strict cast when then raise" and "cse error branch": no matching issue (related: #30252 when/then PanicException, closed; #20305 CSE in group_by, open).
- Checked 2026-11-04 against polars 2.1.0, polars main 35a2cb73fdf81d89b2117bec152c3530eb3e4146: repros/PL-6.py on 2.1.0 (macOS arm64), both engines, CSE on and off, plus eager; main source only: crates/polars-plan/src/plans/optimizer/cse/csee.rs.
### PL-7: Predicate pushdown merges a later filter into an earlier one unless the later predicate is fallible, and … {#PL-7}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
Predicate pushdown merges a later filter into an earlier one unless the later predicate is fallible, and the fallible list (`is_fallible_top_level`, crates/polars-plan/src/plans/aexpr/mod.rs) holds strict casts, `list.get`/`arr.get` without `null_on_oob`, `replace_strict`, decimal and truncating arithmetic and strict `strptime`. `str.to_integer(strict=True)` raises on text that is not an integer, and it is missing from the list.
So `filter(s != 'x').filter(s.str.to_integer() == 1)` becomes one `FILTER (s != 'x') & (s.str.to_integer() == 1)`. `&` evaluates both sides on every row, so the parse runs on the row the first filter removes and raises. With predicate pushdown off, the query returns the row. Other raising parsers outside the list (`str.to_decimal`, `str.json_decode`, strict temporal parsers other than `strptime`) are the same class; not probed.
Reproduction:
- import polars as pl
lf = pl.LazyFrame({"s": ["1", "x"]})
q = lf.filter(pl.col("s") != "x").filter(pl.col("s").str.to_integer() == 1)
q.collect() # ComputeError: strict integer parsing failed for 1 value(s): ["x"]
q.collect(engine="streaming") # the same error
q.collect(optimizations=pl.QueryOptFlags(predicate_pushdown=False)).rows() # [('1',)]
print(q.explain())
# FILTER (col("s") != "x") & (col("s").str.to_integer([dyn int: 10]) == 1)
How sverify handles it:
- sverify collects every translated plan with `predicate_pushdown=False` (`sverify.polars_engine`), so its plans keep their filter order. The fix upstream is to add `str.to_integer(strict=True)` and the other raising string parsers to `is_fallible_top_level`.
Status: Reproduces on Polars 2.1.0 (Python, macOS arm64) with a lazy `collect()` on both engines. On main (35a2cb73, 2026-11-04) `is_fallible_top_level` still lists no string parser other than strict `strptime`, read in the source; not run there. No matching issue found on pola-rs/polars (2026-11-04); pola-rs/polars#22284 fixed the same class for strict casts.
See: facts [polars-exec-predicate-pushdown](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-exec-predicate-pushdown); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444); upstream <https://github.com/pola-rs/polars/issues/22284>; files [`python/src/sverify/polars_engine.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/src/sverify/polars_engine.py), [`characterization/polars/facts/polars-exec-predicate-pushdown.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-exec-predicate-pushdown.json).
Found by the Polars characterization reverse pass (2026-11-04): each open fact checked against the source, docstrings and user guide at py-2.1.0 and Polars's own tests, with light probes on both engines; the source reading of predicate pushdown for polars-exec-predicate-pushdown, then a manual repro.
- Checked 2026-11-04 against polars 2.1.0: Repro on both engines, with and without predicate pushdown (Polars 2.1.0, macOS arm64, POLARS_IDEAL_MORSEL_SIZE=64). Source at py-2.1.0: crates/polars-plan/src/plans/aexpr/mod.rs#L294-L355 (the fallible list), crates/polars-plan/src/plans/aexpr/function_expr/strings.rs#L49-L50 and #L262-L263 (`ToInteger`, a plain elementwise function), crates/polars-plan/src/plans/optimizer/predicate_pushdown/mod.rs#L266-L300 (a filter's predicate is pushed into the filter below). Main at 35a2cb73 (2026-11-04, `gh api repos/pola-rs/polars/contents/crates/polars-plan/src/plans/aexpr/mod.rs?ref=main`): `is_fallible_top_level` at L298-L360 has the same arms, with no `ToInteger`. Duplicate search (`gh issue list --repo pola-rs/polars --state all --search`) for "to_integer predicate pushdown", "str.to_integer filter", "predicate pushdown strict parsing failed", "strict integer parsing failed filter", "fallible predicate pushdown", "is_fallible", "predicate pushdown raises filtered rows", "to_integer strict filter error", "filter pushdown cast error rows removed", "str.to_decimal filter" and "predicate pushdown error string parse": no matching issue. Related: #22284 (closed: strict casts pushed down, the fix that added the list), #28521 (open: a fallible `strptime` pushed into `scan_parquet`).
### PL-8: The documentation says float division follows IEEE 754, and division by a literal or one-row divisor does not {#PL-8}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
The documentation says float division follows IEEE 754, and division by a literal or one-row divisor does not. Pages: `Expr.truediv` (https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.Expr.truediv.html, py-polars/src/polars/expr/expr.py#L6928-L6933 at py-2.1.0), Notes: "Zero-division behaviour follows IEEE-754", with 0/0 NaN and n/0 ±infinity; and the user guide (https://docs.pola.rs/user-guide/concepts/data-types-and-structures/#floating-point-numbers, docs/source/user-guide/concepts/data-types-and-structures.md#L176-L190): Polars "generally follows the IEEE 754 floating point standard", with NaN ordering and the sign of zero or NaN as the listed exceptions.
With a length-1 divisor, `/`, `//` and `%` multiply by the reciprocal (crates/polars-compute/src/arithmetic/float.rs#L78-L119, PL-3). A nonzero finite divisor then creates NaN and infinity: `0.0 / 1e-310` is NaN (IEEE 0.0), `1e-300 / 1e-310` is inf (IEEE 1e10), and `49.0 % 49.0` is 49.0, outside `[0, |y|)`. The user guide's accuracy paragraph ("does not provide guarantees on the error") covers the ulp in `49.0 / 49.0 = 0.9999999999999999`; it does not cover a NaN from two finite nonzero operands. Integer `/` casts to `Float64` first, so it takes the reciprocal too.
Secondary: the `floordiv` docstring (expr.py#L6702-L6745) says Polars "first does the floating-point division ... and then performs the flooring". With a literal divisor that is `floor(x * (1 / c))`: `pl.col("x") // 49.0` on `x = 49.0` is 0.0 on a frame of two rows or more.
Reproduction:
- import polars as pl
pl.DataFrame({"x": [49.0, 0.0, 1e-300]}).select(
pl.col("x") / 1e-310, (pl.col("x") // 49.0).alias("fd"), (pl.col("x") % 49.0).alias("m")
).to_dict(as_series=False)
# {'x': [inf, nan, inf], 'fd': [0.0, 0.0, 0.0], 'm': [49.0, 0.0, 1e-300]}
pl.DataFrame({"a": [49, 5]}).select(pl.col("a") / 49, pl.col("a") / 3)
# 49 / 49 = 0.9999999999999999, 5 / 3 = 1.6666666666666665
- On the streaming engine, a frame shorter than the thread count streams as one-row morsels and gives the IEEE values, so which result a row gets depends on the morsel length (polars-double-scalar-divisor).
How sverify handles it:
- Nothing to work around: sverify follows the engine's behaviour, which the facts record. The fix is in Polars's documentation. Suggested fix: list the reciprocal for a scalar divisor among the user guide's exceptions, with the subnormal case, or drop the reciprocal for divisors whose reciprocal overflows.
Status: A bug in the documentation. The engine's behaviour is PL-3. Reproduces on Polars 2.1.0 (Python, macOS arm64); the `truediv` Notes and the user guide's paragraph are unchanged on main (35a2cb73, 2026-11-04). No matching docs issue found on pola-rs/polars (2026-11-04); #23741 and #14596 (closed) discuss the reciprocal's ulp errors, not the docs.
See: facts [polars-double-scalar-divisor](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-double-scalar-divisor), [polars-double-divide](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-double-divide), [polars-int-true-divide](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-int-true-divide); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444); upstream <https://github.com/pola-rs/polars/issues/23741>, <https://github.com/pola-rs/polars/issues/14596>, <https://github.com/pola-rs/polars/issues/30398>; files [`characterization/polars/facts/polars-double-scalar-divisor.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-double-scalar-divisor.json), [`characterization/polars/facts/polars-double-divide.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-double-divide.json), [`characterization/polars/facts/polars-int-true-divide.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-int-true-divide.json).
Found by the Polars characterization reverse pass (2026-11-04): the docstrings and user guide at py-2.1.0 checked against each open fact.
- Checked 2026-11-04 against polars 2.1.0: Repro on Polars 2.1.0 (macOS arm64). Main at 35a2cb73: expr.py#L6931 still reads "Zero-division behaviour follows IEEE-754", the user guide's IEEE paragraph is at data-types-and-structures.md#L178, and `floordiv` still says Polars "first does the floating-point division" (expr.py#L6715). Duplicate search for "truediv reciprocal", "division scalar reciprocal precision", "division by literal not IEEE" and "x / 49.0 0.9999999999999999": no docs issue.
### PL-9: The documentation of `Expr.round` … {#PL-9}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
The documentation of `Expr.round` (https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.Expr.round.html, py-polars/src/polars/expr/expr.py#L2001-L2025 at py-2.1.0) says strategies starting with `half_` "round all values to the *nearest* rounded value, only using the strategy to break ties when a value falls exactly between two rounded values".
`round(d)` rounds `x * 10^d`, itself rounded, and divides back (crates/polars-ops/src/series/ops/round.rs#L100-L136), so a double just below a decimal tie can round up. The double 2.675 is 2.67499999999999982236…, nearer 2.67, and Polars gives 2.68 in both `half_` modes; 1835.665 (1835.66499999999996362…) gives 1835.67 with `half_away_from_zero`. Polars's own `test_round` (py-polars/tests/unit/lazyframe/test_lazyframe.py#L564) expects 1835.67 for the default mode, gets 1835.66, and passes only on its `rel_tol` of 1e-5.
Reproduction:
- import polars as pl
pl.Series([2.675, 1835.665]).round(2).to_list() # [2.68, 1835.66]
pl.Series([2.675, 1835.665]).round(2, mode="half_away_from_zero").to_list() # [2.68, 1835.67]
# 2.675 is 2.67499999999999982236…, 1835.665 is 1835.66499999999996362…:
# neither is a tie, the nearest values are 2.67 and 1835.66 (Python's round gives both).
How sverify handles it:
- Nothing to work around: sverify follows the engine's behaviour, which the facts record. The fix is in Polars's documentation. Suggested fix: say that a decimal text's ties may round either way (as Python's `round` docs warn), or round the exact value in the kernel.
Status: A bug in the documentation, or in `round`: the docs and the kernel disagree. Reproduces on Polars 2.1.0 (Python, macOS arm64), both engines; the docstring is unchanged on main (35a2cb73, 2026-11-04). No matching issue found on pola-rs/polars (2026-11-04); #12812 (closed) was about `round` in `f32`.
See: facts [polars-round-digits](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-round-digits), [polars-ulp-round](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-ulp-round); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444); upstream <https://github.com/pola-rs/polars/issues/12812>; files [`characterization/polars/facts/polars-round-digits.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-round-digits.json), [`characterization/polars/facts/polars-ulp-round.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-ulp-round.json).
Found by the Polars characterization reverse pass (2026-11-04): the docstrings and user guide at py-2.1.0 checked against each open fact.
- Checked 2026-11-04 against polars 2.1.0: Probe on both engines (POLARS_IDEAL_MORSEL_SIZE=64): `round(2)` of [2.675, 1835.665, 0.285, 1.005] is [2.68, 1835.66, 0.28, 1.0], with `half_away_from_zero` [2.68, 1835.67, 0.28, 1.0]. Main at 35a2cb73: the wording is at expr.py#L2016-L2017. Duplicate search for "round nearest docs 2.675", "round half_to_even 2.675" and "round decimals incorrect": no matching issue.
### PL-10: The documentation of `Expr.nan_min` and `Series.nan_min` … {#PL-10}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
The documentation of `Expr.nan_min` and `Series.nan_min` (https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.Expr.nan_min.html, py-polars/src/polars/expr/expr.py#L3751 and py-polars/src/polars/series/series.py#L2316 at py-2.1.0) says "This differs from numpy's `nanmax` as numpy defaults to propagating NaN values, whereas polars defaults to ignoring them." It should name `nanmin`. In both `nan_max` and `nan_min` the reason describes numpy's `max`/`min`, which propagate NaN, while numpy's `nanmax`/`nanmin` ignore NaN, the opposite of what the sentence attributes to them.
Reproduction:
- Text only. py-polars/src/polars/expr/expr.py at py-2.1.0, `nan_min`: "This differs from numpy's `nanmax` as numpy defaults to propagating NaN values, whereas polars defaults to ignoring them." numpy: `np.nanmin([1.0, np.nan])` is 1.0 and `np.min([1.0, np.nan])` is nan.
How sverify handles it:
- Nothing to work around: sverify follows the engine's behaviour, which the facts record. The fix is in Polars's documentation. Suggested wording: "This matches numpy's `min`, which propagates NaN; Polars's `min` ignores NaN, as numpy's `nanmin` does."
Status: A bug in the documentation's text, minor. On main (35a2cb73, 2026-11-04) `nan_min` names numpy's `nanmin` (Expr and Series), so the wrong name is fixed there; the reason in both `nan_max` and `nan_min` is unchanged. No matching issue found on pola-rs/polars (2026-11-04); #17513 (open) is about `max`/`min` and NaN on length-1 groups.
See: facts [polars-min-max-nan](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-min-max-nan); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444); upstream <https://github.com/pola-rs/polars/issues/17513>; files [`characterization/polars/facts/polars-min-max-nan.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-min-max-nan.json).
Found by the Polars characterization reverse pass (2026-11-04): the docstrings and user guide at py-2.1.0 checked against each open fact.
- Checked 2026-11-04 against polars 2.1.0: Docstrings at py-2.1.0 (expr.py `nan_max` and `nan_min` both say `nanmax`). Main at 35a2cb73: expr.py and series.py say `nanmax` in `nan_max` and `nanmin` in `nan_min`, with the same reason. Duplicate search for "nan_min docstring nanmax": only #17513 (related).
### PL-11: The documentation of `Expr.quantile` … {#PL-11}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
The documentation of `Expr.quantile` (https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.Expr.quantile.html, py-polars/src/polars/expr/expr.py#L4813-L4830 at py-2.1.0) says "If a single float, returns a single f64 value per row." and "If a list of floats, returns a list of f64 values per row".
The result type is `moment_dtype` (crates/polars-plan/src/plans/aexpr/function_expr/schema.rs#L689-L704): `Float32` and `Float16` keep their type, `Date` gives `Datetime(us)`, and other numeric types give `Float64`.
Reproduction:
- import polars as pl
from datetime import date
lf = pl.LazyFrame({"a": pl.Series([1.0, 2.0, 3.0, 4.0], dtype=pl.Float32)})
lf.select(pl.col("a").quantile(0.4, "linear")).collect().schema # Schema({'a': Float32})
pl.LazyFrame({"d": [date(2020, 1, 1), date(2020, 1, 2)]}).select(pl.col("d").quantile(0.5)).collect().schema
# Schema({'d': Datetime(time_unit='us', time_zone=None)})
How sverify handles it:
- Nothing to work around: sverify follows the engine's behaviour, which the facts record. The fix is in Polars's documentation. Suggested wording: "returns one value per row: Float32 for Float32 input, Float64 for other numeric input, and the temporal type for temporal input (Date gives Datetime)".
Status: A bug in the documentation. Reproduces on Polars 2.1.0 (Python, macOS arm64), both engines; the docstring is unchanged on main (35a2cb73, 2026-11-04). No matching issue found on pola-rs/polars (2026-11-04).
See: facts [polars-quantile](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-quantile); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444); files [`characterization/polars/facts/polars-quantile.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-quantile.json).
Found by the Polars characterization reverse pass (2026-11-04): the docstrings and user guide at py-2.1.0 checked against each open fact.
- Checked 2026-11-04 against polars 2.1.0: Probe: `Float32` [1, 2, 3, 4] `quantile(0.4, 'linear')` and `median()` are `Float32` on both engines; `Date` `quantile(0.5)` is `Datetime(us)`. Main at 35a2cb73: expr.py#L4829 still says "returns a single f64 value per row". Duplicate search for "quantile Float32 dtype docs" and "quantile returns f64": no matching issue.
### PL-12: The documentation of `list.get` … {#PL-12}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
The documentation of `list.get` (https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.Expr.list.get.html, py-polars/src/polars/expr/list.py#L603-L624 at py-2.1.0) says "So index `0` would return the first item of every sublist and index `-1` would return the last item of every sublist if an index is out of bounds, it will return a `None`." The same sentence is in `Series.list.get` (series/list.py#L434), `Expr.arr.get` (expr/array.py#L822) and `Series.arr.get` (series/array.py#L613).
The same docstring's `null_on_oob` says "False -> raise an error", and its default is `False`: an out-of-range index raises `ComputeError: get index is out of bounds` (crates/polars-ops/src/chunked_array/list/namespace.rs#L320-L338, whose Rust doc comment at L320-L323 has the same stale sentence). The `arr` variants were not probed; their signature has the same default (expr/array.py#L816).
Reproduction:
- import polars as pl
pl.LazyFrame({"l": [[1], []]}).select(pl.col("l").list.get(0)).collect()
# ComputeError: get index is out of bounds
pl.LazyFrame({"l": [[1], []]}).select(pl.col("l").list.get(0, null_on_oob=True)).collect().rows()
# [(1,), (None,)]
How sverify handles it:
- Nothing to work around: sverify follows the engine's behaviour, which the facts record. The fix is in Polars's documentation. Suggested fix: drop the sentence, or say "if an index is out of bounds, it raises, or returns `None` with `null_on_oob=True`".
Status: A bug in the documentation. Reproduces on Polars 2.1.0 (Python, macOS arm64), both engines; the sentence is unchanged on main (35a2cb73, 2026-11-04). No matching issue found on pola-rs/polars (2026-11-04).
See: facts [polars-list-index](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-list-index); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444); files [`characterization/polars/facts/polars-list-index.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-list-index.json).
Found by the Polars characterization reverse pass (2026-11-04): the docstrings and user guide at py-2.1.0 checked against each open fact.
- Checked 2026-11-04 against polars 2.1.0: Repro on both engines (POLARS_IDEAL_MORSEL_SIZE=64). Main at 35a2cb73: expr/list.py#L616 still says "it will return a `None`". Duplicate search for "list.get out of bounds docs None" and "list.get null_on_oob docstring": no matching issue.
### PL-13: The documentation of `pl.when` … {#PL-13}
::: {.ledger-meta}
Project: Polars 2.1.0; status: confirmed; kind: bug.
:::
The documentation of `pl.when` (https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.when.html, py-polars/src/polars/functions/whenthen.py#L48-L51 and #L80 at py-2.1.0) says "Polars computes all expressions passed to `when-then-otherwise` in parallel and filters afterwards. This means each expression must be valid on its own" and "Note that `when-then` always executes all expressions."
An elementwise branch is evaluated with its input columns masked to NULL on the rows its condition excludes (crates/polars-expr/src/expressions/ternary.rs#L110-L203), on both engines, so a strict cast in such a branch does not raise there. Polars's own tests pin the masking (py-polars/tests/unit/functions/test_when_then.py#L916-L960). The masking has limits the docs could state: non-elementwise branches, branches under `group_by().agg` or `over`, and a subexpression that common-subexpression elimination hoists out of two branches (PL-6).
Reproduction:
- import polars as pl
pl.LazyFrame({"x": ["1", "a"]}).select(
pl.when(pl.col("x") == "1").then(pl.col("x").cast(pl.Int64))
).collect().rows()
# [(1,), (None,)] on both engines; the docs predict an InvalidOperationError
How sverify handles it:
- sverify relies on the masking for elementwise branches (polars-exec-when-eager) and on the facts' limits for the rest. The fix is in Polars's documentation: document the masking and its limits, or state that it is not guaranteed.
Status: A bug in the documentation, low priority: the docs are stricter than the code, so a reader who follows them is safe. Reproduces on Polars 2.1.0 (Python, macOS arm64), both engines; the warning is unchanged on main (35a2cb73, 2026-11-04). Related: #25457 (open) asks to clarify that `when-then-otherwise` filters at the end.
See: facts [polars-exec-when-eager](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-exec-when-eager), [polars-exec-cse-hoist](https://github.com/AlexanderNenninger/substrait-lean/blob/main/upstream/bugs/polars-facts.md#polars-exec-cse-hoist); issues [#444](https://github.com/AlexanderNenninger/substrait-lean/issues/444); upstream <https://github.com/pola-rs/polars/issues/25457>; files [`characterization/polars/facts/polars-exec-when-eager.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-exec-when-eager.json), [`characterization/polars/facts/polars-exec-cse-hoist.json`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/characterization/polars/facts/polars-exec-cse-hoist.json).
Found by the Polars characterization reverse pass (2026-11-04): the docstrings and user guide at py-2.1.0 checked against each open fact.
- Checked 2026-11-04 against polars 2.1.0: Repro on both engines (POLARS_IDEAL_MORSEL_SIZE=64). Main at 35a2cb73: whenthen.py#L49 and #L80 keep the warning. Duplicate search for "when then docs computes all expressions": #25457 (open, related: an aggregation in a branch computed over all rows).
## SQLGlot
### SG-1: `qualify` resolves a name in `WHERE`/`GROUP BY`/`HAVING` that is both an input column and an output alias … {#SG-1}
::: {.ledger-meta}
Project: SQLGlot 30.27.0; status: filed.
:::
`qualify` resolves a name in `WHERE`/`GROUP BY`/`HAVING` that is both an input column and an output alias as the **alias**; DuckDB resolves it as the **column**.
Reproduction:
- `select a + 1 as a from t group by a having a = 42` with `t(a) = [42]`: DuckDB returns 43, the qualified tree returns nothing.
How sverify handles it:
- The lift refuses such names (#252, b387dcd).
Status: filed [#8887](https://github.com/tobymao/sqlglot/issues/8887); fixed on main by [#8898](https://github.com/tobymao/sqlglot/pull/8898) (bdaf7d1, 2026-11-04), not released. Reproduces on 30.29.0.
See: upstream <https://github.com/tobymao/sqlglot/issues/8887>, <https://github.com/tobymao/sqlglot/pull/8898>; pull requests [#252](https://github.com/AlexanderNenninger/substrait-lean/pull/252).
Found by sqllogictest replay through the lift (#252).
- Checked 2026-11-03 against sqlglot 30.29.0: against the latest releases
- Checked 2026-11-04 against sqlglot 30.29.0, sqlglot main main bdaf7d1, duckdb 2.0.0: repro on 30.27.0, 30.29.0 and main
### SG-2: Transpiling between dialects keeps the text of an operator whose meaning differs: `transpile("select 7/2", … {#SG-2}
::: {.ledger-meta}
Project: SQLGlot 30.27.0; status: not a bug.
:::
Transpiling between dialects keeps the text of an operator whose meaning differs: `transpile("select 7/2", read="postgres", write="duckdb")` gives `SELECT 7 / 2` (3 in Postgres, 3.5 in DuckDB). With a schema plus `annotate_types`, it becomes `CAST(TRUNC(a / b) AS BIGINT)` (right value, BIGINT where Postgres gives INTEGER).
Reproduction:
- as stated in the summary
How sverify handles it:
- sverify never uses SQLGlot's transpiler for semantics (#231); the lift's profiles decide. May be a known limitation upstream, not a bug: check their docs and issues first.
Status: not a bug: out of scope upstream ([#8531](https://github.com/tobymao/sqlglot/issues/8531), [comment](https://github.com/tobymao/sqlglot/pull/8532#issuecomment-6035457336)). Unchanged on 30.29.0 and main (bdaf7d1, 2026-11-04).
See: issues [#231](https://github.com/AlexanderNenninger/substrait-lean/issues/231); upstream <https://github.com/tobymao/sqlglot/issues/8531>, <https://github.com/tobymao/sqlglot/pull/8532#issuecomment-6030704299>, <https://github.com/tobymao/sqlglot/pull/8532#issuecomment-6035457336>.
Found by #231 findings.
- Checked 2026-11-03 against sqlglot 30.29.0: against the latest releases
- Checked 2026-11-04 against sqlglot 30.29.0, sqlglot main main bdaf7d1, duckdb 2.0.0: repro on 30.27.0, 30.29.0 and main
### SG-3: DuckDB → DataFusion transpiling rewrites some functions with another meaning: `strftime`, top-n `min(x, … {#SG-3}
::: {.ledger-meta}
Project: SQLGlot 30.27.0; status: filed.
:::
DuckDB → DataFusion transpiling rewrites some functions with another meaning: `strftime`, top-n `min(x, n)`, `POSITIONAL`/`ASOF` joins, `TRY`. Only the top-n `min`/`max` part is filed (#8888) and re-checked; the other cases were not re-checked on 2026-11-04.
Reproduction:
- see `tools/sverify_dev/slt.py`'s refusal list
How sverify handles it:
- Obsolete for sverify: the replay no longer transpiles (#264). File only if still present upstream.
Status: top-n `min(x, n)` filed as [#8888](https://github.com/tobymao/sqlglot/issues/8888), fixed on main by [#8896](https://github.com/tobymao/sqlglot/pull/8896) (bdaf7d1, 2026-11-04; other dialects now warn), not released. Reproduces on 30.29.0. `strftime`, `POSITIONAL`/`ASOF` and `TRY` not rechecked: SQLGlot has no DataFusion dialect.
See: upstream <https://github.com/tobymao/sqlglot/issues/8541>, <https://github.com/tobymao/sqlglot/issues/8888>, <https://github.com/tobymao/sqlglot/pull/8896>; pull requests [#201](https://github.com/AlexanderNenninger/substrait-lean/pull/201), [#264](https://github.com/AlexanderNenninger/substrait-lean/pull/264); files [`python/tools/sverify_dev/slt.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/tools/sverify_dev/slt.py).
Found by sqllogictest replay (#201).
- Checked 2026-11-03 against sqlglot 30.29.0: against the latest releases
- Checked 2026-11-04 against sqlglot 30.29.0, sqlglot main main bdaf7d1, duckdb 2.0.0: top-n case on 30.27.0, 30.29.0 and main
### SG-4: The tokenizer reads every underscore in a number as a digit separator {#SG-4}
::: {.ledger-meta}
Project: SQLGlot 30.22.0; status: filed.
:::
The tokenizer reads every underscore in a number as a digit separator. DuckDB reads only an underscore between two digits so; anywhere else it starts an alias.
Reproduction:
- `SELECT 1__2` is 1 (named `__2`) in DuckDB, 12 in SQLGlot; `SELECT 1._5` is 1 (named `_5`), 1.5 in SQLGlot. `1_000` agrees.
How sverify handles it:
- The lift refuses a number with an underscore not between two digits (`_check_numbers`).
Status: filed [#8889](https://github.com/tobymao/sqlglot/issues/8889); fixed on main by [#8892](https://github.com/tobymao/sqlglot/pull/8892) (bdaf7d1, 2026-11-04), not released. Reproduces on 30.29.0, also with the compiled tokenizer (`sqlglotc`).
See: upstream <https://github.com/tobymao/sqlglot/issues/8889>, <https://github.com/tobymao/sqlglot/pull/8892>; files [`python/src/sverify/sql_lift/_base.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/src/sverify/sql_lift/_base.py).
Found by sqllogictest replay through the lift, `parser/digit_separators.test`, `parser/unquoted_underscore_digit_identifiers.test`.
- Checked 2026-11-03 against sqlglot 30.29.0: against the latest releases
- Checked 2026-11-04 against sqlglot 30.29.0, sqlglotc 30.29.0, sqlglot main main bdaf7d1, duckdb 2.0.0: repro on 30.22.0, 30.29.0 (pure Python and sqlglotc) and main (pure Python)
### SG-5: `qualify` expands `t.*` of the right table of a `JOIN … USING` without the `USING` columns; DuckDB keeps … {#SG-5}
::: {.ledger-meta}
Project: SQLGlot 30.22.0; status: filed.
:::
`qualify` expands `t.*` of the right table of a `JOIN … USING` without the `USING` columns; DuckDB keeps them in `t.*` (only `*` shows them once).
Reproduction:
- `SELECT a.*, b.* FROM a JOIN b USING (i)`: DuckDB gives 4 columns, the qualified tree 3.
How sverify handles it:
- The lift refuses `t.*` with `USING` (`_check_using_stars`).
Status: filed [#8890](https://github.com/tobymao/sqlglot/issues/8890); fixed on main by [#8893](https://github.com/tobymao/sqlglot/pull/8893) (bdaf7d1, 2026-11-04), not released. Reproduces on 30.29.0.
See: upstream <https://github.com/tobymao/sqlglot/issues/8890>, <https://github.com/tobymao/sqlglot/pull/8893>; files [`python/src/sverify/sql_lift/_base.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/src/sverify/sql_lift/_base.py).
Found by replay, `join/empty_joins.test`.
- Checked 2026-11-03 against sqlglot 30.29.0: against the latest releases
- Checked 2026-11-04 against sqlglot 30.29.0, sqlglot main main bdaf7d1, duckdb 2.0.0: repro on 30.22.0, 30.29.0 and main
### SG-6: DuckDB's `TABLE t` (`SELECT * FROM t`) does not parse: as a set operation's operand it fails, and on its … {#SG-6}
::: {.ledger-meta}
Project: SQLGlot 30.22.0; status: filed.
:::
DuckDB's `TABLE t` (`SELECT * FROM t`) does not parse: as a set operation's operand it fails, and on its own it parses as an alias.
Reproduction:
- `SELECT 1 EXCEPT ALL TABLE t`: "Required keyword: 'expression' missing for Except"; `(TABLE a EXCEPT ALL TABLE b)`: "Expecting )"; `TABLE t` gives `Alias(Column)`.
How sverify handles it:
- The lift and the replay write it out as `(SELECT * FROM t)` on SQLGlot's own tokens before parsing (`sql_lift.expand_table_shorthand`, #344).
Status: filed [#8891](https://github.com/tobymao/sqlglot/issues/8891); fixed on main by [#8897](https://github.com/tobymao/sqlglot/pull/8897) (bdaf7d1, 2026-11-04), not released. Reproduces on 30.29.0 in the `duckdb`, `postgres` and `mysql` dialects.
See: issues [#344](https://github.com/AlexanderNenninger/substrait-lean/issues/344); upstream <https://github.com/tobymao/sqlglot/issues/8891>, <https://github.com/tobymao/sqlglot/pull/8897>; files [`python/src/sverify/sql_lift/_base.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/src/sverify/sql_lift/_base.py), [`python/tests/test_sql_lift_relations.py`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/tests/test_sql_lift_relations.py).
Found by sqllogictest replay through the lift (#344), `setops/`.
- Checked 2026-11-03 against sqlglot 30.29.0: against the latest releases
- Checked 2026-11-04 against sqlglot 30.29.0, sqlglot main main bdaf7d1, duckdb 2.0.0: repro on 30.22.0, 30.29.0 and main; postgres/mysql on 30.29.0 and main
### SG-7: From 30.18, `Drop` reads its names from `expressions`, and one given as `this` is not written {#SG-7}
::: {.ledger-meta}
Project: SQLGlot 30.18.0–30.29.0; status: filed.
:::
From 30.18, `Drop` reads its names from `expressions`, and one given as `this` is not written. Ibis 12.0.0 passes `this`, so its DuckDB `create_table`, which drops a temporary view, fails. Ibis 12 allows any `sqlglot>=23.4`.
Reproduction:
- `sge.Drop(kind="VIEW", this="t", exists=True).sql("duckdb")` is `DROP VIEW IF EXISTS` (30.17: `DROP VIEW IF EXISTS t`); `ibis.duckdb.connect().create_table("x", pa.table({"a": [1]}))` fails with DuckDB's "syntax error at or near EXISTS".
How sverify handles it:
- sverify requires `sqlglot~=30.17.0` (#442). Development stays on 30.22 for fakesnow (`[tool.uv] override-dependencies`), the wheel's tests run 30.17, and the lift reads both versions' GROUP BY. File with Ibis.
Status: filed [ibis#12124](https://github.com/ibis-project/ibis/issues/12124) (open); fixed on Ibis main by [#12104](https://github.com/ibis-project/ibis/pull/12104) (f73d329, 2026-11-02), not released. Reproduces with sqlglot 30.29.0 and Ibis 12.0.0.
See: issues [#442](https://github.com/AlexanderNenninger/substrait-lean/issues/442); upstream <https://github.com/ibis-project/ibis/issues/12124>, <https://github.com/ibis-project/ibis/pull/12104>, <https://github.com/ibis-project/ibis/issues/12101>; files [`python/pyproject.toml`](https://github.com/AlexanderNenninger/substrait-lean/blob/main/python/pyproject.toml).
Found by homelab cycling demo (#442).
- Checked 2026-11-03 against sqlglot 30.29.0, ibis-framework 12.0.0: against the latest releases
- Checked 2026-11-04 against sqlglot 30.29.0, ibis-framework 12.0.0, ibis main main f73d329, sqlglot main main bdaf7d1, duckdb 1.5.6: Ibis 12.0.0 with sqlglot 30.17.0/30.18.0/30.29.0/main; Ibis main with sqlglot 30.29.0
### SG-8: In the DuckDB dialect, an interval string of one unit followed by `ago` loses the `ago`: `INTERVAL '1 day … {#SG-8}
::: {.ledger-meta}
Project: SQLGlot 30.17.0–30.29.0; status: confirmed.
:::
In the DuckDB dialect, an interval string of one unit followed by `ago` loses the `ago`: `INTERVAL '1 day ago'` reads as `INTERVAL '1' DAY`, a positive interval, where DuckDB negates it. A string of several units keeps its text (`INTERVAL '2 days 3 hours ago'`).
Reproduction:
- ```python
import sqlglot
sqlglot.parse_one("SELECT INTERVAL '1 day ago'", read="duckdb").sql("duckdb")- observed:
SELECT INTERVAL '1' DAY; DuckDB 2.0.0 givesSELECT INTERVAL '1 day ago'as-1 day.
How sverify handles it:
- The SQL lift refuses an INTERVAL string with
ago, from the tokens, before parsing (_check_intervals, #626)
Status: reproduces on 30.17.0, 30.22.0 and 30.29.0; not filed yet
See: issues #626; files python/src/sverify/sql_lift/_base.py, python/tests/test_sql_lift_temporal.py.
Found by the characterization pipeline’s round 3 (DuckDB’s test_interval.test L93 replayed through sverify_dev.slt).
- Checked 2026-11-04 against sqlglot 30.29.0, duckdb 2.0.0: also 30.17.0 and 30.22.0
Spark
SP-1: -x of a DECIMAL with more than 34 significant digits rounds it to 34 digits, half to even, silently: …
-x of a DECIMAL with more than 34 significant digits rounds it to 34 digits, half to even, silently: -CAST('12345678901234567890123456789012345678' AS DECIMAL(38,0)) is -12345678901234567890123456789012350000. abs(x) negates a negative x the same way and returns a positive one unchanged. 0 - x is exact up to a scale of 6.
At 10³⁸ - 1 the rounding reaches 10³⁸, which does not fit DECIMAL(38,0), so -x raises NUMERIC_VALUE_OUT_OF_RANGE even with ANSI off. Decimal.unary_- (sql/api/…/types/Decimal.scala#L567-L575) builds -decimalVal on a Scala BigDecimal, which rounds to its default MathContext.DECIMAL128 (34 digits, HALF_EVEN); abs calls it for a negative value.
Reproduction:
- SELECT CAST(-a AS STRING), CAST(abs(-a) AS STRING), CAST(0 - a AS STRING) FROM VALUES (CAST(‘12345678901234567890123456789012345678’ AS DECIMAL(38,0))) AS s(a); – -12345678901234567890123456789012350000, 12345678901234567890123456789012350000, -12345678901234567890123456789012345678 SELECT -CAST(‘99999999999999999999999999999999999999’ AS DECIMAL(38,0)); – NUMERIC_VALUE_OUT_OF_RANGE, with spark.sql.ansi.enabled true or false
How sverify handles it:
- The Spark profile (#739) should spell
negateof a decimal as0 - xandabsasCASE WHEN x < 0 THEN 0 - x ELSE x END. With precision loss on (the default)0 - xof aDECIMAL(38,s)drops one digit of scale pasts = 6(it needs 39 digits), so for those scales the profile needs another spelling orspark.sql.decimalOperations.allowPrecisionLossoff. The fix upstream is to negate the JavaBigDecimal(decimalVal.underlying.negate()), which is exact.
Status: Reproduces on Spark 4.3.0 in both ANSI modes, interpreted and codegen. SPARK-45438 (open since 2023, Scala 2.13 builds) reports the raise for -x and abs(x) at the largest 38-digit decimal; the silent rounding of any decimal past 34 significant digits is the same cause and is not in it. Source at v4.3.0 (98bc0212): Decimal.unary_- still negates the Scala BigDecimal, whose MathContext is DECIMAL128.
See: facts spark-decimal-negate; issues #738, #739; upstream https://issues.apache.org/jira/browse/SPARK-45438; files characterization/spark/probes.json, characterization/spark/facts/spark-decimal-negate.json, characterization/spark/fuzz.json.
Found by the Spark characterization (#738): sverify_dev.probe_spark, then a manual repro of decimal-overflow-subtract.
- Checked 2026-11-05 against spark 4.3.0 (pyspark 4.3.0, Java 21.0.12, Linux x86_64): probes decimal-negate-34-digits and decimal-negate-max of characterization/spark/probes.json, both ANSI modes, folded and codegen. Source at v4.3.0: Decimal.scala#L567-L575, arithmetic.scala#L70. JIRA search (project SPARK) for “unary minus decimal precision”, “negate decimal 38”, “Decimal unary_- MathContext”, “abs decimal precision loss”: SPARK-45438 only.
- Checked 2026-11-05 against spark 4.3.0 (pyspark 4.3.0, Java 21.0.12, Linux x86_64): sverify_dev.spark_fuzz decimal_negate (characterization/spark/fuzz.json): random DECIMAL(38,0) and DECIMAL(38,10) values, ties at 34 digits among them, both ANSI modes, folded and codegen. Ties round half to even; abs of a positive value is exact.
SP-2: date_add(d, n) adds the days as Ints with no overflow check (DateAdd.nullSafeEval: start + d), so …
date_add(d, n) adds the days as Ints with no overflow check (DateAdd.nullSafeEval: start + d), so a result past 2³¹ - 1 days wraps to a valid but wrong date: date_add(DATE'2024-01-01', 2147483647) is -5877587-06-21, with ANSI on or off. Adding months, intervals to timestamps and timestampadd raise DATETIME_OVERFLOW instead.
Reproduction:
- SELECT CAST(date_add(DATE’2024-01-01’, 2147483647) AS STRING); – -5877587-06-21 (spark.sql.ansi.enabled true or false)
How sverify handles it:
- The reference raises outside 0001-01-01 … 9999-12-31 (
date_range, #251), so the Spark profile needs a range guard on the day count beforedate_add, or computes the date fromBIGINTdays. The fix upstream isMath.addExactwithDATETIME_OVERFLOW, as SPARK-59859 proposes.
Status: Reproduces on Spark 4.3.0 in both ANSI modes, interpreted and codegen. Filed upstream as SPARK-59859 (open, 2026-09-29), which describes the same wrap for date_add, date_sub, date ± int and try_add.
See: facts spark-date-range; issues #738, #251; upstream https://issues.apache.org/jira/browse/SPARK-59859; files characterization/spark/probes.json, characterization/spark/facts/spark-date-range.json.
Found by the Spark characterization (#738): sverify_dev.probe_spark, probe date-range-far.
- Checked 2026-11-05 against spark 4.3.0 (pyspark 4.3.0, Java 21.0.12, Linux x86_64): probe date-range-far, both ANSI modes, folded and codegen. Source at v4.3.0: datetimeExpressions.scala#L524-L545. JIRA search for “date_add overflow”, “date_add integer overflow”: SPARK-59859 (open), and SPARK-60910 for
datediff.
SP-3: Adding an interval to a timestamp near the end of the microsecond range raises a bare …
Adding an interval to a timestamp near the end of the microsecond range raises a bare java.lang.ArithmeticException: long overflow with no error class or SQLSTATE, where timestampadd and date arithmetic raise DATETIME_OVERFLOW. Clients that sort errors by class (sverify’s runner maps classes to issues) cannot tell it from an internal failure.
Reproduction:
- SELECT TIMESTAMP_NTZ’+294247-01-10 04:00:54.775807’ + INTERVAL 1 DAY; – java.lang.ArithmeticException: long overflow (no error class), ANSI on or off SELECT timestampadd(YEAR, 300000, TIMESTAMP’9999-12-31 23:59:59’); – [DATETIME_OVERFLOW] Datetime operation overflow
How sverify handles it:
- No workaround needed for results: the reference raises far earlier (
date_range, 9999-12-31), so a guarded plan never reaches it. The fix upstream is to wrap the overflow inDATETIME_OVERFLOWinDateTimeUtils.timestampAddInterval.
Status: Reproduces on Spark 4.3.0 in both ANSI modes, interpreted and codegen. SPARK-54836 (open, 4.1.0) asks for the timestamp’s value in the same ArithmeticException; it does not ask for an error class. No issue found for the missing class.
See: facts spark-date-range; issues #738; files characterization/spark/probes.json, characterization/spark/facts/spark-date-range.json.
Found by the Spark characterization (#738): sverify_dev.probe_spark, probe timestamp-overflow.
- Checked 2026-11-05 against spark 4.3.0 (pyspark 4.3.0, Java 21.0.12, Linux x86_64): probe timestamp-overflow, both ANSI modes, folded and codegen. JIRA search for “timestamp interval long overflow ArithmeticException”: SPARK-54836 only.
SP-4: An invalid constant regex ('a' RLIKE '(') raises INVALID_PARAMETER_VALUE.PATTERN on the interpreted …
An invalid constant regex ('a' RLIKE '(') raises INVALID_PARAMETER_VALUE.PATTERN on the interpreted path and a bare java.util.regex.PatternSyntaxException: Unclosed group without an error class on the codegen path, so the error a user sees depends on how the optimizer ran the query.
Reproduction:
- SET spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation,org.apache.spark.sql.catalyst.optimizer.ConstantFolding; SELECT s RLIKE ‘(’ FROM VALUES (‘a’) AS t(s); – java.util.regex.PatternSyntaxException: Unclosed group near index 1 RESET spark.sql.optimizer.excludedRules; SELECT s RLIKE ‘(’ FROM VALUES (‘a’) AS t(s); – [INVALID_PARAMETER_VALUE.PATTERN]
How sverify handles it:
- sverify writes constant patterns it has checked (#50), so no plan reaches it; the probe records it under spark-regex. The fix upstream is to compile the constant pattern in codegen through the same helper that wraps the error.
Status: Reproduces on Spark 4.3.0 in both ANSI modes. The interpreted path (the optimizer folding a constant query) raises INVALID_PARAMETER_VALUE.PATTERN; with ConvertToLocalRelation and ConstantFolding excluded, so the query runs through whole-stage codegen, the same pattern raises a bare java.util.regex.PatternSyntaxException. No matching issue found.
See: facts spark-regex; issues #738, #50; files characterization/spark/probes.json, characterization/spark/facts/spark-regex.json.
Found by the Spark characterization (#738): sverify_dev.probe_spark, probe string-regex-invalid.
- Checked 2026-11-05 against spark 4.3.0 (pyspark 4.3.0, Java 21.0.12, Linux x86_64): probe string-regex-invalid, both ANSI modes, folded and codegen. JIRA search for “PatternSyntaxException codegen”, “rlike PatternSyntaxException”, “regexp invalid pattern codegen error class”: no match.
SP-5: floor and ceil of a double return BIGINT
floor and ceil of a double return BIGINT. Under ANSI they raise ARITHMETIC_OVERFLOW for ±inf and values past 64 bits, but return 0 for NaN, where CAST(NaN AS BIGINT) raises CAST_OVERFLOW. So ANSI mode turns a NaN into a plausible integer silently.
Reproduction:
- SET spark.sql.ansi.enabled=true; SELECT floor(CAST(‘NaN’ AS DOUBLE)), ceil(CAST(‘NaN’ AS DOUBLE)); – 0, 0 SELECT floor(CAST(‘Infinity’ AS DOUBLE)); – [ARITHMETIC_OVERFLOW] SELECT CAST(CAST(‘NaN’ AS DOUBLE) AS BIGINT); – [CAST_OVERFLOW]
How sverify handles it:
- Substrait’s
floorandceilreturn the double, so the Spark profile writes them asCAST(floor(x) AS DOUBLE)only for finitexbelow 2⁶³ in magnitude, and passes NaN and ±inf through (CASE WHEN isnan(x) OR abs(x) >= 9.2e18 THEN x ELSE …). The fix upstream is to raise for NaN under ANSI as the cast does.
Status: Reproduces on Spark 4.3.0 with ANSI on, interpreted and codegen. No matching issue found; SPARK-28135 (in progress since 2019) is about large values of floor, ceil and power, not NaN.
See: facts spark-round; issues #738, #739; files characterization/spark/probes.json, characterization/spark/facts/spark-round.json.
Found by the Spark characterization (#738): sverify_dev.probe_spark, probes double-floor-nan and double-floor-inf.
- Checked 2026-11-05 against spark 4.3.0 (pyspark 4.3.0, Java 21.0.12, Linux x86_64): probes double-floor-nan, double-floor-inf, double-floor-ceil, both ANSI modes, folded and codegen. Source at v4.3.0: mathExpressions.scala#L599-L660 (
Floor,LongType). JIRA search for “floor NaN”, “floor NaN ansi”, “ceil NaN returns 0”: no match.
SP-6: to_timestamp of a year past the microsecond range raises a bare `java.lang.ArithmeticException: long …
to_timestamp of a year past the microsecond range raises a bare java.lang.ArithmeticException: long overflow with no error class or SQLSTATE, in both ANSI modes. The parser’s extractMicros multiplies the epoch seconds with Math.multiplyExact; under ANSI parseToTimestampExact wraps only DateTimeException and ParseException in an error class, and without ANSI isParseError turns only those into NULL, so the overflow escapes both.
Reproduction:
- SELECT to_timestamp(‘294248’, ‘y’); – java.lang.ArithmeticException: long overflow (no error class), ANSI on or off
How sverify handles it:
- The profile must treat the timestamp range as a precondition on parsed values or refuse the construct. The reference raises far earlier (
date_range, 9999-12-31), so a guarded plan never reaches it. The fix upstream is to catch theArithmeticExceptioninextractMicrosand raiseDATETIME_OVERFLOW, or NULL undertry_to_timestampand without ANSI.
Status: Spark’s own golden files at v4.3.0 expect it, in both ANSI modes. No issue found yet; JIRA not searched.
See: facts spark-raise-set, spark-date-range; issues #738; files characterization/spark/facts/spark-raise-set.json, characterization/spark/facts/spark-date-range.json.
Found by Spark’s SQL query tests at v4.3.0 (sverify_dev.spark_tests, Spark evidence session): datetime-parsing-invalid.sql.
- Checked 2026-11-05 against spark 4.3.0 golden files at v4.3.0: https://github.com/apache/spark/blob/v4.3.0/sql/core/src/test/resources/sql-tests/results/datetime-parsing-invalid.sql.out#L2-L8 and https://github.com/apache/spark/blob/v4.3.0/sql/core/src/test/resources/sql-tests/results/nonansi/datetime-parsing-invalid.sql.out#L2-L8. Source: https://github.com/apache/spark/blob/v4.3.0/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/TimestampFormatter.scala#L315-L322 (
Math.multiplyExact), https://github.com/apache/spark/blob/v4.3.0/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/DateTimeExpressionUtils.java#L153-L168 and https://github.com/apache/spark/blob/v4.3.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala#L2145-L2150 (the exceptions caught).
SP-7: TIMESTAMP_SECONDS of a decimal with more than six fractional digits raises a bare …
TIMESTAMP_SECONDS of a decimal with more than six fractional digits raises a bare java.lang.ArithmeticException: Rounding necessary with no error class or SQLSTATE, in both ANSI modes, where the same value as a double truncates to microseconds. decimalToMicros multiplies by 10⁶ and calls longValueExact(); it wraps the exception in DATETIME_OVERFLOW only when the value is outside the long range and rethrows it otherwise, in the interpreted and the codegen path.
Reproduction:
- SELECT TIMESTAMP_SECONDS(0.1234567); – java.lang.ArithmeticException: Rounding necessary (no error class), ANSI on or off SELECT TIMESTAMP_SECONDS(0.1234567d); – 1969-12-31 16:00:00.123456
How sverify handles it:
- The profile must treat a decimal argument’s scale (at most six fractional digits, or a value that is a whole number of microseconds) as a precondition or refuse the construct. The fix upstream is to truncate or round the decimal as the double path does, or to raise a classed error.
Status: Spark’s own golden files at v4.3.0 expect it, in both ANSI modes. No issue found yet; JIRA not searched.
See: facts spark-raise-set, spark-timestamp-precision; issues #738; files characterization/spark/facts/spark-raise-set.json, characterization/spark/facts/spark-timestamp-precision.json.
Found by Spark’s SQL query tests at v4.3.0 (sverify_dev.spark_tests, Spark evidence session): timestamp.sql.
- Checked 2026-11-05 against spark 4.3.0 golden files at v4.3.0: https://github.com/apache/spark/blob/v4.3.0/sql/core/src/test/resources/sql-tests/results/timestamp.sql.out#L675-L681, https://github.com/apache/spark/blob/v4.3.0/sql/core/src/test/resources/sql-tests/results/nonansi/timestamp.sql.out#L647-L653 and the double at https://github.com/apache/spark/blob/v4.3.0/sql/core/src/test/resources/sql-tests/results/timestamp.sql.out#L684-L689. Source: https://github.com/apache/spark/blob/v4.3.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala#L944-L956 (
longValueExact,throw e) and #L983-L988 (codegen).
Substrait
ST-13: Thirteen function test cases expect values their own inputs do not give
Thirteen function test cases expect values their own inputs do not give. variance and std_dev (eleven cases): the sample variance of -5, -3, -1, 1, 3, 5 is 70 / 5 = 14, the cases say 15.166666666666666; the sample variance of 0, 25, 50, 75, 100 is 1562.5 and the population variance 1250, the cases say 1681.25 and 1345.0; the sample variance of 1.1 … 5.5 is 3.025, the cases say 3.03; the std_dev cases are the square roots of the same wrong values. multiply(-13::i8, -10::i8) [overflow:SATURATE] is 130, which saturates to 127, and the case says -128. shift_left(301989888::i64, 8::i32) is 0x12000000 << 8 = 77309411328, and the case says 19791209299968, a shift by 16.
Reproduction:
- variance.test L19-L20, L44, L52-L53, std_dev.test L19-L20, L44-L45, L52-L53, multiply.test L16, shift_left.test L8. DuckDB 2.0.0:
SELECT var_samp(x), var_pop(x) FROM (VALUES (0.0), (100.0), (50.0), (25.0), (75.0)) t(x)gives(1562.5, 1250.0);SELECT var_samp(x) FROM (VALUES (-5.0), (-3.0), (-1.0), (1.0), (3.0), (5.0)) t(x)gives14.0. sverify’s model gives the same.
How sverify handles it:
- The cases are triaged
caseinupstream/substrait-cases.json: they count as disagreements, and the replay fails only on a disagreement the ledger does not list.
Status: Still on main (a4d8e00342d0): variance.test, std_dev.test, multiply.test L16, shift_left.test L8. No existing issue found.
See: issues #444; files python/tools/sverify_dev/substrait_cases.py, upstream/substrait-cases.json.
Found by sverify_dev.substrait_cases, the replay of Substrait’s function test cases against Profile.substrait (#444 step 4).
- Checked 2026-11-04 against substrait main@a4d8e00342d0, substrait-extensions 0.106.0: the cases are the same on main
ST-14: logb’s YAML defines logb(x, b) => log_{b} (x): the value first, the base second
logb’s YAML defines logb(x, b) => log_{b} (x): the value first, the base second. Four of its test cases read the base first: logb(10::i64, 100000::i64) = 5.0, logb(2::fp64, 7::fp64) = 2.8073549220576041 (log₂ 7), logb(7::fp64, 1.0::fp64) = 0 (log₇ 1) and logb(2.34::fp64, inf::fp64) = inf. By the YAML they are log₁₀₀₀₀₀ 10 = 0.2, log₇ 2 ≈ 0.356, log₁ 7 (a division by zero) and log_∞ 2.34 = 0. The cases further down the file (logb(10::fp64, -inf::fp64), logb(2.0::fp64, 0.0::fp64)) also put the value second.
Reproduction:
- logb.test L5-L10; functions_logarithmic.yaml L152-L164 names the arguments
xandbase, in that order.
How sverify handles it:
- sverify reads
logbas the YAML defines it. The four cases are triagedcaseinupstream/substrait-cases.json.
Status: Still on main (a4d8e00342d0): logb.test L5-L10 against functions_logarithmic.yaml. No existing issue found.
See: issues #444; files python/tools/sverify_dev/substrait_cases.py, upstream/substrait-cases.json.
Found by sverify_dev.substrait_cases, the replay of Substrait’s function test cases against Profile.substrait (#444 step 4).
- Checked 2026-11-04 against substrait main@a4d8e00342d0, substrait-extensions 0.106.0: the cases are the same on main
ST-15: Forty-five test cases state an option their function’s YAML does not have, or a value the option does not list
Forty-five test cases state an option their function’s YAML does not have, or a value the option does not list. lookaround:TRUE (regexp_count_substring, regexp_match_substring, regexp_replace, regexp_string_split, 28 cases), spaces_only:FALSE (ltrim, rtrim, trim, 9 cases, three written False) and full_unicode:TRUE (upper, lower, 2 cases) are options of no YAML. on_domain_error:NONE (ln, log10, log2, logb, acosh, 5 cases) is not a value of on_domain_error (NAN, NULL, ERROR; acosh has NAN, ERROR), and on_division_by_zero:NAN on divide of i8 is not one of NULL, ERROR. The README says a case runs only on dialects that support its options, so a consumer can run none of these. Close by: ln(0.0::fp64) [on_log_zero:NAN] and its three siblings expect null, where NAN names a NaN.
Reproduction:
- For example regexp_replace.test, trim.test, upper.test, ln.test L12, L17, acosh.test L12, divide.test L11.
python -m sverify_dev.substrait_caseslists every case.
How sverify handles it:
- The replay checks each case’s options against the YAML impl its arguments select before it runs the case; a case with an option or a value the YAML does not list is a disagreement triaged
caseinupstream/substrait-cases.json.
Status: Still on main (a4d8e00342d0): the options lookaround, spaces_only and full_unicode appear in none of the extension YAMLs, and the values below are not in their options’ lists. No existing issue found.
See: issues #444; files python/tools/sverify_dev/substrait_cases.py, upstream/substrait-cases.json.
Found by sverify_dev.substrait_cases, the replay of Substrait’s function test cases against Profile.substrait (#444 step 4).
- Checked 2026-11-04 against substrait main@a4d8e00342d0, substrait-extensions 0.106.0: the cases are the same on main
delta-rs
DL-1: DeltaTable.history() and DeltaTable.alter.set_table_properties() keep the GIL during object store requests
DeltaTable.history() and DeltaTable.alter.set_table_properties() keep the GIL during object store requests. A server in the same Python process, such as moto’s ThreadedMotoServer, never answers, and the call hangs. optimize.compact, vacuum, create_checkpoint, merge and write_deltalake release it and finish.
Reproduction:
- Start
moto.server.ThreadedMotoServer(port=5066), write a table tos3://lake/twithAWS_ENDPOINT_URL=http://127.0.0.1:5066, then callDeltaTable(uri, storage_options=...).history(): it never returns.
How sverify handles it:
- Test against moto in another process (
moto_server). DeltaSink reads the history for its contract, so it hangs the same way in-process. This was the compaction “hang” of the homelab example: its maintenance set table properties first (#377).
Status: Reproduces on 1.6.6. history(): fixed on main by #4797 (merged 2026-10-06), not released (latest python-v1.6.6, 2026-09-24). set_table_properties(): #4978 open, fix #5005 open, review required.
See: issues #377; upstream https://github.com/delta-io/delta-rs/pull/4797, https://github.com/delta-io/delta-rs/issues/4978, https://github.com/delta-io/delta-rs/pull/5005; files python/pyproject.toml, python/tests/test_io_delta_s3.py.
Found by #377.
- Checked 2026-11-03 against deltalake 1.6.6: against the latest releases
- Checked 2026-11-04 against deltalake 1.6.6, moto 5.2.5, pyarrow 26.0.1: latest release; main read at dc415dc0a0, not built
DL-2: DeltaTable.history() on a table loaded at an older version (load_as_version) lists the latest commits, …
DeltaTable.history() on a table loaded at an older version (load_as_version) lists the latest commits, numbered as if the loaded version were the latest: after two commits, a table loaded at version 0 gives [(0, 'WRITE'), (-1, 'WRITE')], where the commits are versions 1 and 0.
Reproduction:
write_deltalake(d, pa.table({'a': [1]})),write_deltalake(d, pa.table({'a': [2]}), mode='append'),dt = DeltaTable(d); dt.load_as_version(0), then[(c['version'], c['operation']) for c in dt.history(8)]gives[(0, 'WRITE'), (-1, 'WRITE')].
How sverify handles it:
DeltaSource.contract(version)reads the latest table’s history and keeps the commits up toversion(_last_contract), where it loaded the table atversionbefore and so returned the latest contract.
Status: Not yet reported upstream.
See: files python/src/sverify/io/delta.py, python/tests/test_api_1_0_io.py.
Found by the API review’s sweep (E2), tests/test_api_1_0_io.py.
- Checked 2026-11-05 against deltalake 1.6.6: latest release