Skip to main content

Quick Start

Build your first Patterns App in minutes​

Patterns lets you quickly build apps that ingest data from any sort of API or database, store data in tables, process it with Python and SQL, build interactive charts and dashboards, and serve data to external APIs or databases. This written guide mirrors the in-app guided tutorial where you learn to build an app that monitors Hacker News comments for keywords, visualizes data, and sends alerts to Slack. In this tutorial we will demonstrate how to:

  1. Import comments from the Hacker News API with a Python node
  2. Write a SQL node to aggregate comments by minute
  3. Visualize comment timeseries data in a chart
  4. Write a Python node to search comment text for keywords
  5. Use a Slack Component from the Marketplace that posts messages to a channel

Foo Clicking on the image above will link you to a completed version of the tutorial app that you can clone

1. Create an account and start a new app​

Log in to Patterns, then click + Create App. If you don't have an account yet, sign up here for free.

You can either follow the in-app guided tutorial (recommended), or follow along from scratch using the written tutorial that follows.

Lastly, you can also clone a completed copy of this tutorial by clicking the image below and following the button to clone.

2. Import comments from the Hacker News API with a Python node​

Every app begins by getting data into Patterns. You can do this by linking a database, using a component, querying an API, or loading a CSV. In this example, we will use a Python node to extract data from the Hacker News API.

Try it out​

  1. Click on the +Add button and select a Python node to add to your graph
  2. Rename your node Ingest Hacker News Comments
  3. Add the below code to your node
  4. On line 10, where we create the comments table object, click the red table icon and select + create Table Store to add a new table object to your app.
  5. Afterwards, press run to ingesting data from the HN API
ingest-hacker-news-comments.py
from patterns import (
    Parameter,
    State,
    Table,
)

import requests

# define a table object that we will use to write and store comments
comments = Table('comments', mode='w')

# set urls
hn_url_stub = "https://hacker-news.firebaseio.com/v0/item/"
hn_url_latest_item = "https://hacker-news.firebaseio.com/v0/maxitem.json"

# get latest item from hacker news
latest_item = requests.get(hn_url_latest_item).json()

# get last 100 comments from hacker news
count_items = 100
starting_item = latest_item - count_items

for i in range(starting_item, latest_item):
    url = hn_url_stub + str(i) + '.json'
    record = requests.get(url).json()
    if record["type"] == "comment":
        comments.append(record)

3. Write a SQL node to aggregate comments by minute​

Now that we have imported data to our comments table, let's write a SQL aggregation query to count comments by minute.

  1. Hover over the comments table with your mouse, click the + button, and select Add SQL.
  2. Rename this SQL node count comments and it's output table comments_by_minute.
  3. Copy/paste the below code into the editor and click the Run.
comments-by-minute.sql
select count(1) as count_comments
, date_trunc('minute', to_timestamp(time)) as minute_posted

from {{ Table("comments") }}

group by 2

4. Visualize comment timeseries data in a chart​

Patterns uses a powerful charting library called Vegalite that uses a JSON syntax like the one below to specify how charts are rendered.

  1. Hover over the table comments_by_minute, press +, select Add Chart. This will create a new chart node with the table comments_by_minute as the input data.
  2. Select Simple Bar Chart
  3. Copy/paste the below code
comments-by-minute.json
{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "mark": { "type": "bar", "tooltip": true },
  "encoding": {
    "x": { "field": "minute_posted", "type": "temporal", "axis": { "labelAngle": 35 } },
    "y": { "field": "count_comments", "type": "quantitative" }
  }
}
  • Click to open and rename output table to daily_avg_review
  • Run your node!

5. Write a Python node to search comment text for keywords​

  1. Add a new Python node to the canvas, rename it search keywords
  2. Copy/paste the below code into the IDE. You will notice there is now an edge with the comments table, this represents a reactive dependency between the nodes.
  3. Click on the red icon next to the alerts table and + Create Table Store
search-keywords.py
from patterns import Table
from datetime import datetime

# create table objects
comments = Table("comments", "r")
alerts = Table("alerts", "w")
alerts.init(add_monotonic_id='id')

# create a list of keywords to watch for in comments
keywords = ['data', 'python', ' airflow ', ' javascript ',
' ai ', 'gpt', 'generative', 'elt', 'sql', ' pipeline ']

# loop through comments and check for matching keywords
for i in comments.as_stream(order_by="id"):
    text = i.get('text')
    id = i.get('id')
    time = datetime.fromtimestamp(i.get('time'))
    if not text:
        continue
    if any(s in text.lower() for s in keywords):
        alert = {"time": time,
                    "id": id,
                    "url": f"https://news.ycombinator.com/item?id={id}",
                    "comment": text}
        alerts.append(alert)

6. Use a Slack Component from the Marketplace that posts messages to a channel​

  1. Press / on your keyboard or navigate to the search bar at the top of your page to access the marketplace
  2. Search for the Post message to Slack component and add it to your app
  3. Click on the Settings tab to configure the component
  4. Click on Add store and select the alerts table.
  5. Follow the instructions in the component readme to create a Slack app and add your slack_webhook_url (found at https://api.slack.com/apps/)
  6. Copy/paste the below text into the message_template field. Data fields from the alerts table are used as variable inputs into the message text.
A keyword was detected at {time}
URL: {url}

Comment: {comment}