I known File Sink is currently mostly used in local development, but we also tend to use it in our production environment with fluentd, because this combination is very stable and reliable.
Currently File Sink lacks the file rolling function, which is very important for the production environment. So I want to add file rolling function to File Sink through this pr without breaking changes.
file rolling is disabled by default, and can be enabled by adding the following arguments:
```yaml
- receivers:
file:
path: ./test.log
maxsize: 20 // if ./test.log is larger than 20MB, it will be renamed to ./test-<datetime>.log as a backup file,
and create a new file named test.log to receive events. It defaults to 100 megabytes.
maxage: 20 // if backup files ./test-<datetime>.log are created 20 days ago, they will be deleted. The default
is not to remove old log files based on age.
maxbackups: 3 // if backup files are more than 3, older backup files will be removed.
```
the rolling function is powered by [lumberjack](https://github.com/natefinch/lumberjack). Becasue `lumberjack.Logger` implmented `io.WriteCloser` interface, so it is easy to replace `file *os.file` with `writer &lumberjack.Logger`.
run kubernetes-event-exporter with following config:
```yaml
logLevel: debug
route:
match:
- receiver: "file"
receivers:
- name: "file"
file:
path: "./event.log"
maxsize: 1
maxbackups: 4
```
run following script to generate events and output `./event*.log` status:
```shell
while true
do
kubectl run nginx --image nginx || kubectl delete deployment nginx
ls -lh event*.log
sleep 2
done
```
the result is:
```
deployment.apps "nginx" deleted
-rw-r--r-- 1 ysyou staff 1.0M Mar 4 18:03 event-2020-03-04T10-03-30.271.log
-rw-r--r-- 1 ysyou staff 1.0M Mar 4 18:15 event-2020-03-04T10-15-52.496.log
-rw-r--r-- 1 ysyou staff 1.0M Mar 4 18:28 event-2020-03-04T10-28-27.334.log
-rw-r--r-- 1 ysyou staff 1.0M Mar 4 18:41 event-2020-03-04T10-41-11.316.log
-rw-r--r-- 1 ysyou staff 1023K Mar 4 18:54 event.log
deployment.apps/nginx created
-rw-r--r-- 1 ysyou staff 1.0M Mar 4 18:15 event-2020-03-04T10-15-52.496.log
-rw-r--r-- 1 ysyou staff 1.0M Mar 4 18:28 event-2020-03-04T10-28-27.334.log
-rw-r--r-- 1 ysyou staff 1.0M Mar 4 18:41 event-2020-03-04T10-41-11.316.log
-rw-r--r-- 1 ysyou staff 1.0M Mar 4 18:54 event-2020-03-04T10-54-11.451.log
-rw-r--r-- 1 ysyou staff 2.0K Mar 4 18:54 event.log
```
when `event.log` reaches 1M, it is renamed to `event-2020-03-04T10-54-11.451.log`, because backup files are more than 4, the oldest one `event-2020-03-04T10-03-30.271.log` is deleted.
leader election function is disabled by default, and could be enabled by adding the
following section in `config.yaml`:
```yaml
leaderElection:
enabled: true
leaderElectionID: kubernetes-event-exporter // this field can be omited, default value is kubernetes-event-exporter
```
Date formatted index names are a common pattern in Elastic setups. This
commit as a config field to specify a string pattern for index names.
The pattern is searched for sequenences inside {curly braces} and passes
any text found plus the current time through the Go time formatting function.
Before the change, the engine didn't call the `Close()` method of the
sinks. This is needed in some cases, i.e when a sink implementation is
buffered.
This change adds a `Close()` method to the registry that
will signal sinks to exit and wait for all sinks to exit before
returning. This is then used in the engine stop logic.
In the channel-based registry, the closing of all sinks is done in
parallel (using a `sync.WaitGroup`). In the sync registry, sinks are
closed sequentially.
Fixes issue #10
This new sink adds basic support for Kafka, using `sarama.SyncProducer`.
It allows minimal configuration to make it useful:
- Topic
- Broker List
- TLS settings
The layout transformations are supported in the same way as for kinesis.
Implements issue #6.
Mustafa Akin
Mustafa AkınandGitHub
Jaydeep Solanki
Njegos Railic
lesh366andGitHub
elads
youyongsong
Brian Terry
Ameya Joshi
Martin Heck
Dmitriy Kononov
shikloshi
Aaron Brodersen
aseker
Marc Demierre