Skip to content
This repository has been archived by the owner on Mar 9, 2022. It is now read-only.

Commit

Permalink
Merge pull request #40 from jumptrading/improve_readme
Browse files Browse the repository at this point in the history
Improve readme
  • Loading branch information
oplehto authored Feb 28, 2018
2 parents 5249beb + 1b35933 commit 02605c0
Showing 1 changed file with 20 additions and 8 deletions.
28 changes: 20 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,30 @@

## Overview

influx-spout consists of a number of components which receive [InfluxDB] [line
protocol] measurements sent by agents such as [Telegraf], efficiently filter
them and then forward them on to one or more InfluxDB backends. Much effort has
been put in to ensuring that high volumes of measurements can be
supported. [NATS] is used for messaging between the various influx-spout
components.
influx-spout is an open source messaging system that routes and processes
[InfluxDB line protocol] metrics from agents (for example [Telegraf]) to
processing and storage backends ([InfluxDB], [Kapacitor] etc.).

Key features:
- Proven ability to handle high volumes of data (>500k points per second)
in a production environment
- Horizontal scalability
- Multithreaded data processing within components
- Components can be distributed on multiple servers or a fleet of containers
- Ability to add and remove endpoints without disrupting existing data flows
- Fine-grained, regexp-based control over routing of metrics data to specific
backends
- Sanity checking of the data stream that prevents corrupt metrics data from
reaching backends
- Batching of outgoing data to larger chunks, making it easier for backends
to handle high-volume dataflows
- Leverages the high-performance [NATS] messaging system

[InfluxDB]: https://www.influxdata.com/time-series-platform/influxdb/
[line protocol]: https://docs.influxdata.com/influxdb/v1.4/write_protocols/line_protocol_tutorial/
[InfluxDB line protocol]: https://docs.influxdata.com/influxdb/v1.4/write_protocols/line_protocol_tutorial/
[Telegraf]: https://www.influxdata.com/time-series-platform/telegraf/
[NATS]: https://nats.io/
[Kapacitor]: https://www.influxdata.com/time-series-platform/kapacitor/

The following diagram shows a typical influx-spout deployment, and the
flow of data between the various components:
Expand Down Expand Up @@ -53,7 +66,6 @@ All the influx-spout components may be run on a single host or may be
spread across multiple hosts depending on scaling and operational
requirements.


## Building

Ensure the `GOPATH` environment is properly exported and simply run
Expand Down

0 comments on commit 02605c0

Please sign in to comment.