From 658fc4872934bfc49746364fd7c35bf36a37bbf6 Mon Sep 17 00:00:00 2001 From: Frank Reno Date: Wed, 9 May 2018 17:22:54 -0600 Subject: [PATCH] [stable/sumologic-fluentd] Misc updates (#5338) * Adding a new maintainer for my eyes on project. * update to latest release * add missing timeKey documentation * upgrade to latest and expose new addTimeStamp setting --- stable/sumologic-fluentd/Chart.yaml | 6 +- stable/sumologic-fluentd/README.md | 85 ++++++++++++++++++- .../templates/daemonset.yaml | 4 + stable/sumologic-fluentd/values.yaml | 5 +- 4 files changed, 96 insertions(+), 4 deletions(-) diff --git a/stable/sumologic-fluentd/Chart.yaml b/stable/sumologic-fluentd/Chart.yaml index 03da2fd9b4..744c4a24a9 100755 --- a/stable/sumologic-fluentd/Chart.yaml +++ b/stable/sumologic-fluentd/Chart.yaml @@ -1,6 +1,6 @@ name: sumologic-fluentd -version: 0.4.0 -appVersion: 1.11 +version: 0.4.1 +appVersion: 1.14 description: Sumologic Log Collector keywords: - monitoring @@ -14,3 +14,5 @@ maintainers: email: techops@adaptly.com - name: frankreno email: freno@sumologic.com + - name: darend + email: darend@gmail.com diff --git a/stable/sumologic-fluentd/README.md b/stable/sumologic-fluentd/README.md index 10037e6c00..4e7a67a9c6 100644 --- a/stable/sumologic-fluentd/README.md +++ b/stable/sumologic-fluentd/README.md @@ -88,8 +88,10 @@ The following tables lists the configurable parameters of the sumologic-fluentd | `sumologic.readFromHead` | Start to read the logs from the head of file, not bottom. Only applies to containers log files. See in_tail doc for more information | `true` | | `sumologic.concatSeparator` | The character to use to delimit lines within the final concatenated message. Most multi-line messages contain a newline at the end of each line | `Nil` | | `sumologic.auditLogPath` | Define the path to the [Kubernetes Audit Log](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/) | `/mnt/log/kube-apiserver-audit.log` | +| `sumologic.timeKey` | The field name for json formatted sources that should be used as the time. See [time_key](https://docs.fluentd.org/v0.12/articles/formatter_json#time_key-(string,-optional,-defaults-to-%E2%80%9Ctime%E2%80%9D)). | `time` +| `sumologic.addTimeStamp` | Option to control adding timestamp to logs. | `true` | `image.name` | The image repository and name to pull from | `sumologic/fluentd-kubernetes-sumologic` | -| `image.tag` | The image tag to pull | `v1.11` | +| `image.tag` | The image tag to pull | `v1.14` | | `image.pullPolicy` | Image pull policy | `IfNotPresent` | | `persistence.enabled` | Boolean value, used to turn on or off fluentd position file persistence, on nodes (requires Kubernetes >= 1.8) | `false` | | `persistence.hostPath` | The path, on each node, to a directory for fluentd pos files. You must create the directory on each node first or set `persistence.createPath` (requires Kubernetes >= 1.8) | `/var/run/fluentd-pos` | @@ -161,3 +163,84 @@ To enable the creation of RBAC resources, do the following ```console $ helm install --name my-release stable/sumologic-fluentd --set rbac.create=true ``` + +### Excluding and Including data + +You have several options controlling the filtering of data that gets sent to Sumo Logic. + +#### Excluding data using environment variables + +There are several environment variables that can exclude data. The following table show which environment variables affect which Fluentd sources. + +| Environment Variable | Containers | Docker | Kubernetes | Systemd | +|----------------------|------------|--------|------------|---------| +| `EXCLUDE_CONTAINER_REGEX` | ✔ | ✘ | ✘ | ✘ | +| `EXCLUDE_FACILITY_REGEX` | ✘ | ✘ | ✘ | ✔ | +| `EXCLUDE_HOST_REGEX `| ✔ | ✘ | ✘ | ✔ | +| `EXCLUDE_NAMESPACE_REGEX` | ✔ | ✘ | ✔ | ✘ | +| `EXCLUDE_PATH` | ✔ | ✔ | ✔ | ✘ | +| `EXCLUDE_PRIORITY_REGEX` | ✘ | ✘ | ✘ | ✔ | +| `EXCLUDE_POD_REGEX` | ✔ | ✘ | ✘ | ✘ | +| `EXCLUDE_UNIT_REGEX` | ✘ | ✘ | ✘ | ✔ | + +#### Excluding data using annotations + +You can also use the sumologic.com/exclude annotation to exclude data from Sumo. This data is sent to FluentD, but not to Sumo Logic. + +```yaml +apiVersion: v1 +kind: ReplicationController +metadata: + name: nginx +spec: + replicas: 1 + selector: + app: mywebsite + template: + metadata: + name: nginx + labels: + app: mywebsite + annotations: + sumologic.com/format: "text" + sumologic.com/sourceCategory: "mywebsite/nginx" + sumologic.com/sourceName: "mywebsite_nginx" + sumologic.com/exclude: "true" + spec: + containers: + - name: nginx + image: nginx + ports: + - containerPort: 80 +``` + +#### Include excluded using annotations + +If you excluded a whole namespace, but still need one or few pods to be still included for shipping to Sumo Logic, you can use the sumologic.com/include annotation to include data to Sumo. It takes precedence over the exclusion described above. + +```yaml +apiVersion: v1 +kind: ReplicationController +metadata: + name: nginx +spec: + replicas: 1 + selector: + app: mywebsite + template: + metadata: + name: nginx + labels: + app: mywebsite + annotations: + sumologic.com/format: "text" + sumologic.com/sourceCategory: "mywebsite/nginx" + sumologic.com/sourceName: "mywebsite_nginx" + sumologic.com/include: "true" + spec: + containers: + - name: nginx + image: nginx + ports: + - containerPort: 80 +``` diff --git a/stable/sumologic-fluentd/templates/daemonset.yaml b/stable/sumologic-fluentd/templates/daemonset.yaml index fa08ee42a8..204f85b769 100644 --- a/stable/sumologic-fluentd/templates/daemonset.yaml +++ b/stable/sumologic-fluentd/templates/daemonset.yaml @@ -143,6 +143,10 @@ spec: - name: TIME_KEY value: {{ quote .Values.sumologic.timeKey }} {{- end }} + {{- if quote .Values.sumologic.addTimeStamp }} + - name: ADD_TIMESTAMP + value: {{ quote .Values.sumologic.addTimeStamp }} + {{- end }} {{- if quote .Values.sumologic.verifySsl }} - name: VERIFY_SSL value: {{ quote .Values.sumologic.verifySsl }} diff --git a/stable/sumologic-fluentd/values.yaml b/stable/sumologic-fluentd/values.yaml index 6afa5970d0..2988df09f2 100644 --- a/stable/sumologic-fluentd/values.yaml +++ b/stable/sumologic-fluentd/values.yaml @@ -1,7 +1,7 @@ # Default values for sumologic-fluentd. image: name: sumologic/fluentd-kubernetes-sumologic - tag: v1.11 + tag: v1.14 pullPolicy: IfNotPresent ## Annotations to add to the DaemonSet's Pods @@ -119,6 +119,9 @@ sumologic: ## ref: https://docs.fluentd.org/v0.12/articles/formatter_json#time_key-(string,-optional,-defaults-to-%E2%80%9Ctime%E2%80%9D) # timeKey: + ## Option to control adding timestamp to logs. (Default true) + addTimeStamp: true + ## Fluentd command line options ## ref: http://docs.fluentd.org/v0.12/articles/command-line-option fluentdOpt: ""