Skip to main content

Node.pipe()

Apply step to this node, left to right: orders.pipe(paid) is

Usage

Source

Node.pipe(
    step,
    /,
    **inputs,
)

paid(orders), and a job reads in the order its data flows.

The chained node fills the step’s first table parameter, and the step’s other inputs come by keyword. The contract is checked as when the transform is called: orders.pipe(paid) builds the same node, and the same plan, as paid(orders).

Parameters

step: Step

What to apply: a contracted transform (sverify.transform); a SQL step, sverify.sql(query), whose inputs are the tables the query reads; or a plain function from Ibis tables to an Ibis table, which is a transform with no contract.

**inputs: Any
The step’s other inputs, by parameter name: sources, nodes, or plain values. For a SQL step, the keywords are table names in the query, and the chained node is the one table left over.

Returns

Node
The node step builds.

Raises

TypeError

step is not a step, or an input is missing, given twice (as the chained node and by keyword) or unknown; the message names the step’s parameters. For a SQL step, a table of the query has no input, the chained node is left unused, or as_ is not the table left over.

sverify.errors.CompositionError

A precondition on another node’s result does not follow from what that node guarantees.

sverify.errors.VerificationError
A postcondition could not be proved.

Examples

With orders, paid and doubled from the Node example, and a lookup table of customers:

customers = StaticSource(
    pa.table({"customer": ["ada", "cy"], "region": ["eu", "us"]}),
    name="customers",
)

r = orders.pipe(paid).pipe(doubled)  # the same node as doubled(paid(orders))
regional = orders.pipe(paid).pipe(  # SQL, with the chained node as orders
    sv.sql(
        "SELECT o.id, c.region FROM orders o "
        "JOIN customers c ON o.customer = c.customer"
    ),
    customers=customers,
)
big = orders.pipe(lambda t: t.filter(t.amount > 20))  # no contract
sv.group(r, regional, big).refresh()
regional.to_arrow().to_pylist()  # [{'id': 3, 'region': 'us'}]