io.sink_from()

A sink that hands the node’s whole table to fn whenever it

Usage

Source

io.sink_from(
    fn,
    *,
    name,
)

changes: node.write(sink_from(save, name="paid")).

Each state is written once per process: a refresh that changes nothing does not call fn. A restarted job calls it again with the table it computes.

Parameters

fn: Callable[[pa.Table], Any]

Called with the table (an Arrow table) to store; what it returns is ignored.

name: str

The stored table’s name in the plans that read it.

Returns

Sink

The sink.

Examples

import pyarrow as pa
import sverify as sv
from sverify.io import StaticSource, sink_from

orders = StaticSource(pa.table({"id": [1, 2], "amount": [30, 5]}), name="orders")


@sv.transform()
def big_orders(orders):
    return orders.filter(orders.amount > 10)


big = big_orders(orders).write(
    sink_from(lambda t: t.to_pandas().to_csv("big.csv"), name="big")
)