Proton is a unified streaming and historical data processing engine in a single binary. It helps data engineers and platform engineers solve complex real-time analytics use cases, and powers the Timeplus streaming analytics platform.
Proton extends the historical data, storage, and computing functionality of the popular ClickHouse project with streaming and OLAP data processing.
Why use Proton?
- A unified, lightweight engine to connect streaming and historical data processing tasks with efficiency and robust performance.
- A smooth developer experience with powerful streaming and analytical functionality.
- Flexible deployments with Proton's single binary and no external service dependencies.
- Low total cost of ownership compared to other analytical frameworks.
Plus built-in support for powerful streaming and analytical functionality:
|Data transformation||Scrub sensitive fields, derive new columns from raw data, or convert identifiers to human-readable information.|
|Joining streams||Combine data from different sources to add freshness to the resulting stream.|
|Aggregating streams||Developer-friendly functions to derive insights from streaming and historical data.|
|Windowed stream processing (tumble / hop / session)||Collect insightful snapshots of streaming data.|
|Substreams||Maintain separate watermarks and streaming windows.|
|Data revision processing (changelog)||Create and manage non-append streams with primary keys and change data capture (CDC) semantics.|
|Federated streaming queries||Query streaming data in external systems (e.g. Kafka) without duplicating them.|
|Materialized views||Create long-running and internally-stored queries.|
With Docker engine installed on your local machine, pull and run the latest version of the Proton Docker image.
docker run -d --pull always --name proton ghcr.io/timeplus-io/proton:latest
Connect to your
proton container and run the
proton-client tool to connect to the local Proton server:
docker exec -it proton proton-client -n
If you stop the container and want to start it again, run
docker start proton.
Query a test stream
proton-client, run the following SQL to create a stream of random data:
-- Create a stream with random data.
CREATE RANDOM STREAM devices(device string default 'device'||to_string(rand()%4), temperature float default rand()%1000/10);
-- Run the long-running stream query.
SELECT device, count(*), min(temperature), max(temperature) FROM devices GROUP BY device;
You should see data like the following:
│ device0 │ 2256 │ 0 │ 99.6 │
│ device1 │ 2260 │ 0.1 │ 99.7 │
│ device3 │ 2259 │ 0.3 │ 99.9 │
│ device2 │ 2225 │ 0.2 │ 99.8 │
Now that you're running Proton and have created your first stream, query, and view, you can explore reading and writing data from Apache Kafka with External Streams, or view the the subpages to explore additional capabilities.
To see how such a deployment of Proton works as a demo, using
owl-shop sample live data, check out our tutorial with Docker Compose.
To see more examples of using Proton, check out the examples folder.
The following drivers are available:
- https://github.com/timeplus-io/proton-java-driver JDBC and other Java clients
Integrations with other systems:
- Grafana https://github.com/timeplus-io/proton-grafana-source
- Metabase https://github.com/timeplus-io/metabase-proton-driver
- Pulse UI https://github.com/timeplus-io/pulseui/tree/proton
- Homebrew https://github.com/timeplus-io/homebrew-timeplus
- dbt https://github.com/timeplus-io/dbt-proton
Get more with Timeplus
We welcome your contributions! If you are looking for issues to work on, try looking at the issue list.
We also encourage you to join the Timeplus Community Slack to ask questions and meet other active contributors from Timeplus and beyond.
Join the Timeplus Community Slack to connect with Timeplus engineers and other Proton users.
For filing bugs, suggesting improvements, or requesting new features, see the open issues here on GitHub.
Proton uses Apache License 2.0. See details in the LICENSE.