Stateful Dataflows Examples

This repository offers a comprehensive range of dataflow examples from basic to advanced levels, designed to familiarize you with the concepts and techniques essential for deploying Stateful Dataflows.

Prerequisites

• Install and Updated Rust

  curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh rustup update

• Install Fluvio and SDF

  curl -fsS https://hub.infinyon.cloud/install/install.sh | bash fvm install sdf-beta10

• Run SDF setup to install dependencies

sdf setup

You might be prompted to install wasm dependency. Copy paste the command and run.

• Start Cluster

  fluvio cluster start

DataFlows

Dataflow examples:

• bank-processing
• car-processing
• hackernews-notify
• helsinki-transit
• ny-transit
• http-callout
• mask-user-pii
• openai-callout
• split-sentence
• word-counter
• word-probe

Primitives

Examples centered around primitive concepts and operations.

• map
• filter
• filter-map
• flat-map
• split (filter)
• split (filter-map)
• merge
• regex
• sql (cli)
• sql (join)
• update-state
• key-value (input)
• key-value (output)
• key-value (chained)

Packages

Step-by-step instructions on how to build your own package:

• hello-world

Pre-built packages used in dataflows:

• mask-ssn
• parse-sentence

References

• connectors