Components in Patterns are reusable graphs. They consist of one or more nodes, are given a unique slug name, and are uploaded to the publishing organization's component library. They can be simple utilities, like a SQL deduplication node, or complex flows, like an entire underwriting model and scoring system.
For now, component development is best done locally with help from the Patterns Devkit. Install it if you haven’t already (
pip install patterns-devkit ) and then login (
patterns login ) to connect to the Patterns platform. Sign up at studio.patterns.app if you don’t have an account.
For our example, we’ll make a simple one node component that takes in a table, streams it and augments the records with a timestamp:
from datetime import datetime, timezone
from patterns import *
input_table = Table("input_table")
output_table = Table("output_table", "w")
timestamp_field = Parameter("timestamp_field", type=str, default="timestamp")
for record in input_table.as_stream():
record[timestamp_field] = datetime.now(timezone.utc)
To make this node into a component we will make a create a graph.yml and fill in the special fields
exposes. In addition, it is recommended to add a
tags, and an
icon image (which will display in graph UI and component marketplace).
title: Augment w/ Timestamp
- node_file: augment_with_timestamp.py
We can then upload and publish our component to our organization with the devkit command
patterns upload path/to/graph.yml --publish-component. (You may see “graph errors” when you upload a component that has an unconnected input or unfilled parameter, these are ok to ignore, since they will be connected by users of your component) Now if we run
patterns list components we should see it included in the output.
To use the component in a different graph, we need to include it as a node with the
uses field of our
title: Example using component
- node_file: generate_stream.py
- uses: my-org/augment-with-timestamp@v0
- table: my_table
- table: output_table
Schemas are recommended for any component where the structure of the data is known ahead of time. Schemas allow users to get clean and documented data, and allow other components to safely interoperate. Read more about how to use schemas here.