The config file may contain sensitive data such as credentials
for an elasticsearch cluster. To provide the possibility
to store that sensitive data in an other way, support for
environment variables is introduced. Sensitive data can then
be replaced by variables such as ${PASSWORD} and provided by
setting the appropriate environment variables. In kubernetes
the configuration and the sensitive data can be split up into
a configmap and a secret.
Date formatted index names are a common pattern in Elastic setups. This
commit as a config field to specify a string pattern for index names.
The pattern is searched for sequenences inside {curly braces} and passes
any text found plus the current time through the Go time formatting function.
Before the change, the engine didn't call the `Close()` method of the
sinks. This is needed in some cases, i.e when a sink implementation is
buffered.
This change adds a `Close()` method to the registry that
will signal sinks to exit and wait for all sinks to exit before
returning. This is then used in the engine stop logic.
In the channel-based registry, the closing of all sinks is done in
parallel (using a `sync.WaitGroup`). In the sync registry, sinks are
closed sequentially.
Fixes issue #10
This new sink adds basic support for Kafka, using `sarama.SyncProducer`.
It allows minimal configuration to make it useful:
- Topic
- Broker List
- TLS settings
The layout transformations are supported in the same way as for kinesis.
Implements issue #6.
Mustafa Akın
Mustafa Akın
Dmitriy Kononov
Martin Heck
shikloshi
k.bigwheel
Aaron Brodersen
aseker
Marc Demierre