Analytics library
Project description
Python Data Streams
Installation
Install from pip:
pip install tributary
or from source
python setup.py install
Stream Types
Tributary offers several kinds of streams:
Streaming
These are synchronous, reactive data streams, built using asynchronous python generators. They are designed to mimic complex event processors in terms of event ordering.
Functional
These are functional streams, built by currying python functions (callbacks).
Lazy
These are lazily-evaluated python streams, where outputs are propogated only as inputs change.
Examples
Sources and Sinks
Sources
- Python Function/Generator/Async Function/Async Generator
- Random
- File
- Kafka
- Websocket
- Http
- SocketIO
Sinks
- File
- Kafka
- Http
- Websocket
- SocketIO
Transforms
Modulate
- Delay - Streaming wrapper to delay a stream
- Apply - Streaming wrapper to apply a function to an input stream
- Window - Streaming wrapper to collect a window of values
- Unroll - Streaming wrapper to unroll an iterable stream
- UnrollDataFrame - Streaming wrapper to unroll a dataframe into a stream
- Merge - Streaming wrapper to merge 2 inputs into a single output
- ListMerge - Streaming wrapper to merge 2 input lists into a single output list
- DictMerge - Streaming wrapper to merge 2 input dicts into a single output dict. Preference is given to the second input (e.g. if keys overlap)
- Reduce - Streaming wrapper to merge any number of inputs
Calculations
Arithmetic Operators
- Noop
- Negate
- Invert
- Add
- Sub
- Mult
- Div
- RDiv
- Mod
- Pow
- Sum
- Average
Boolean Operators
- Not
- And
- Or
Comparators
- Equal
- NotEqual
- Less
- LessOrEqual
- Greater
- GreaterOrEqual
Math
- Log
- Sin
- Cos
- Tan
- Arcsin
- Arccos
- Arctan
- Sqrt
- Abs
- Exp
- Erf
Converters
- Int
- Float
- Bool
- Str
Python Builtins
- Len
Rolling
- RollingCount - Node to count inputs
- RollingMin - Node to take rolling min of inputs
- RollingMax - Node to take rolling max of inputs
- RollingSum - Node to take rolling sum inputs
- RollingAverage - Node to take the running average
Node Type Converters
- Lazy->Streaming
Graph Visualization
You can visualize the graph with Graphviz. All streaming and lazy nodes support a graphviz
method.
Streaming and lazy nodes also support ipydagred3 for live update monitoring.
Streaming
Here green indicates executing, yellow indicates stalled for backpressure, and red indicates that StreamEnd
has been propogated (e.g. stream has ended).
Lazy
Here green indicates executing, and red indicates that the node is dirty. Note the the determination if a node is dirty is also done lazily (we can check with isDirty
whcih will update the node's graph state.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tributary-0.1.0rc2-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2d5b2ec8035f0cf2d147c2bf22d6e30070cdf8432ef879a80e2552b611f233e7 |
|
MD5 | c986e38ee8a4d52f9f99f5141caeea11 |
|
BLAKE2b-256 | ce1a4d243d5ce4a61c0530c90ca3adf4cf55fe3be9d8bf5c90df280fe83e7673 |