Patterns is now Atoma 🚀
Sign up here to automate analytics!
Skip to main content

Storing data

Table

Table nodes are storage nodes that are similar to standard database tables, but provide a layer of abstraction that enables data versioning, retention, and cross-platform support. They are attached to a specific database and create actual database tables based on the operations of Python and SQL nodes.

Schema

All tables have an associated Schema that is represented in a database engine agnostic format with a Common Model schema. Schemas define both the structure and data types of a table as well as semantic information that can be used by Patterns nodes and components to provide smart default behavior. An example schema:

OrderSchema.yml
name: Order
description: E-commerce order
unique_on:
- id
field_roles:
created_ordering: created_at
updated_ordering: updated_at
strictly_monotonic_ordering: id
immutable: false
fields:
id:
type: Integer
customer_id:
type: Text
amount:
type: Decimal(12, 2)
created_at:
type: DateTime
updated_at:
type: DateTime
info

N.B. field_roles are used by components and the Patterns protocol to provided semantic information for automatic behavior. A primary use case for field roles is to provide the default stream ordering of a table (i.e. used when calling table.as_stream() without an order_by argument), which uses strictly_montonic_ordering by default and falls back to created_ordering otherwise. If neither are defined, calling as_stream() with no order_by argument will result in an error.