diff --git a/stable/sumologic-fluentd/Chart.yaml b/stable/sumologic-fluentd/Chart.yaml index 6fd6bc0acc..3c8691e105 100755 --- a/stable/sumologic-fluentd/Chart.yaml +++ b/stable/sumologic-fluentd/Chart.yaml @@ -1,5 +1,5 @@ name: sumologic-fluentd -version: 0.1.1 +version: 0.2.0 description: Sumologic Log Collector keywords: - monitoring diff --git a/stable/sumologic-fluentd/README.md b/stable/sumologic-fluentd/README.md index 7a29d92e8e..8ed243d328 100644 --- a/stable/sumologic-fluentd/README.md +++ b/stable/sumologic-fluentd/README.md @@ -6,27 +6,26 @@ ## Introduction -This chart adds the Sumo Logic Collector to all nodes in your cluster via a -DaemonSet. After you have installed the chart, each pod, deployment, etc. can be -optionally +This chart adds the Sumo Logic Collector to all nodes in your cluster as a +DaemonSet. The image supports fluentd `file` and `systemd` log sources. + +After you have installed the chart, each pod, deployment, etc. can be optionally [configured](https://github.com/SumoLogic/fluentd-kubernetes-sumologic#options) to specify its log format, source category, source name, or exclude itself from SumoLogic. +### Configure an individual pod + annotations: sumologic.com/format: "text" sumologic.com/sourceCategory: "mywebsite/nginx" sumologic.com/sourceName: "mywebsite_nginx" + +### Prevent an individual pod from logging + + annotations: sumologic.com/exclude: "true" -### Systemd - -The current docker image, for [fluentd-kubernetes-sumologic](https://github.com/SumoLogic/fluentd-kubernetes-sumologic) -does not support systemd. The only logs available to SumoLogic are those in -`/var/log/containers`. Logs generated in the `kube-system` namespace can only be -excluded using `EXCLUDE_NAMESPACE` and `EXCLUDE_CONTAINER`. The `EXPORT_PATH` -option is not relevant on a systemd OS. - ## Prerequisites - Kubernetes 1.2+ with Beta APIs enabled @@ -64,6 +63,7 @@ The following tables lists the configurable parameters of the sumologic-fluentd | `tolerations` | List of node taints to tolerate (requires Kubernetes >= 1.6) | `[]` | | `updateStrategy` | `OnDelete` or `RollingUpdate` (requires Kubernetes >= 1.6) | `OnDelete` | | `sumologic.collectorUrl` | An HTTP collector in SumoLogic that the container can send logs to via HTTP | `Nil` You must provide your own | +| `sumologic.fluentdSource` | The fluentd input source, `file` or `systemd` | `file` | | `sumologic.flushInterval` | How frequently to push logs to sumo, in seconds | `5` | | `sumologic.numThreads` | The number of http threads sending data to sumo | `1` | | `sumologic.sourceName` | Set the sumo `_sourceName` | `%{namespace}.%{pod}.%{container}` | @@ -72,11 +72,14 @@ The following tables lists the configurable parameters of the sumologic-fluentd | `sumologic.sourceCategoryReplaceDash` | Used to replace `-` with another character | `/` | | `sumologic.logFormat` | Format to post logs, into sumo (`json`, `json_merge`, or `text`) | `json` | | `sumologic.kubernetesMeta` | Include or exclude kubernetes metadata, with `json` format | `true` | -| `sumologic.excludePath` | Files in this pattern will not be sent to sumo, ie `"[\"/var/log/containers/*.log\", \"/var/log/*.log\"]` | `Nil` | -| `sumologic.excludeNamespaceRegex` | All matching namespaces will not be sent to sumo | `Nil` | -| `sumologic.excludePodRegex` | All matching pods will not be sent to sumo | `Nil` | | `sumologic.excludeContainerRegex` | All matching containers will not be sent to sumo | `Nil` | +| `sumologic.excludeFacilityRegex` | All matching facilities will not be sent to sumo | `Nil` | | `sumologic.excludeHostRegex` | All matching hosts will not be sent to sumo | `Nil` | +| `sumologic.excludeNamespaceRegex` | All matching namespaces will not be sent to sumo | `Nil` | +| `sumologic.excludePath` | Files in this pattern will not be sent to sumo, ie `"[\"/var/log/containers/*.log\", \"/var/log/*.log\"]` | `Nil` | +| `sumologic.excludePodRegex` | All matching pods will not be sent to sumo | `Nil` | +| `sumologic.excludePriorityRegex` | All matching priorities will not be sent to sumo | `Nil` | +| `sumologic.excludeUnitRegex` | All matching systemd units will not be sent to sumo | `Nil` | | `sumologic.fluentdOpt` | Additional command line options, sent to fluentd | `Nil` | | `sumologic.verifySsl` | Verify SumoLogic HTTPS certificates | `true` | | `image.name` | The image repository and name to pull from | `sumologic/fluentd-kubernetes-sumologic` | @@ -88,6 +91,7 @@ The following tables lists the configurable parameters of the sumologic-fluentd | `resources.limits.cpu` | CPU resource limits | 256m | | `resources.requests.memory` | Memory resource requests | 128Mi | | `resources.limits.memory` | Memory resource limits | 256Mi | +| `rbac.create` | Is Role Based Authentication enabled in the cluster | `false` | | `rbac.serviceAccountName` | RBAC service account name | {{ fullname }} | @@ -121,3 +125,32 @@ $ helm install --name my-release \ --set sumologic.collectorUrl=URL,persistence.hostPath=/var/run/fluentd \ stable/sumologic-fluentd ``` + +### RBAC + +By default the chart will not install the associated RBAC rolebinding, +using beta annotations. + +To determine if your cluster supports this running the following: + +```console +$ kubectl api-versions | grep rbac +``` + +You also need to have the following parameter on the api server. See the +following document for how to enable +[RBAC](https://kubernetes.io/docs/admin/authorization/rbac/) + +``` +--authorization-mode=RBAC +``` + +If the output contains "beta" or both "alpha" and "beta" you can enable rbac. + +### Enable RBAC role/rolebinding creation + +To enable the creation of RBAC resources, do the following + +```console +$ helm install --name my-release stable/sumologic-fluentd --set rbac.create=true +``` diff --git a/stable/sumologic-fluentd/templates/clusterrole.yaml b/stable/sumologic-fluentd/templates/clusterrole.yaml new file mode 100644 index 0000000000..69efdabd97 --- /dev/null +++ b/stable/sumologic-fluentd/templates/clusterrole.yaml @@ -0,0 +1,15 @@ +{{- if .Values.rbac.create }} +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRole +metadata: + name: {{ template "fullname" . }} + labels: + app: {{ template "name" . }} + chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" + release: "{{ .Release.Name }}" + heritage: "{{ .Release.Service }}" +rules: +- apiGroups: [""] + resources: ["namespaces", "pods"] + verbs: ["get", "list", "watch"] +{{- end }} diff --git a/stable/sumologic-fluentd/templates/clusterrolebinding.yaml b/stable/sumologic-fluentd/templates/clusterrolebinding.yaml new file mode 100644 index 0000000000..45028ef154 --- /dev/null +++ b/stable/sumologic-fluentd/templates/clusterrolebinding.yaml @@ -0,0 +1,19 @@ +{{ if .Values.rbac.create }} +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: {{ template "fullname" . }} + labels: + app: {{ template "name" . }} + chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" + release: "{{ .Release.Name }}" + heritage: "{{ .Release.Service }}" +subjects: +- kind: ServiceAccount + name: {{ template "fullname" . }} + namespace: {{ .Release.Namespace }} +roleRef: + kind: ClusterRole + name: {{ template "fullname" . }} + apiGroup: rbac.authorization.k8s.io +{{ end }} diff --git a/stable/sumologic-fluentd/templates/daemonset.yaml b/stable/sumologic-fluentd/templates/daemonset.yaml index 2521b0e9a5..86d50fdbe2 100644 --- a/stable/sumologic-fluentd/templates/daemonset.yaml +++ b/stable/sumologic-fluentd/templates/daemonset.yaml @@ -3,29 +3,25 @@ kind: DaemonSet metadata: name: {{ template "fullname" . }} labels: - app: {{ template "fullname" . }} - chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" - release: "{{ .Release.Name }}" - heritage: "{{ .Release.Service }}" + app: {{ template "name" . }} + chart: {{ .Chart.Name }}-{{ .Chart.Version }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} spec: template: metadata: name: {{ template "fullname" . }} - labels: - app: {{ template "fullname" . }} {{- if .Values.podAnnotations }} annotations: {{ toYaml .Values.podAnnotations | indent 8 }} {{- end }} + labels: + app: {{ template "name" . }} + release: {{ .Release.Name }} spec: - {{- if .Values.rbac.serviceAccountName }} - serviceAccountName: {{ .Values.rbac.serviceAccountName }} - {{- else }} - serviceAccountName: {{ template "fullname" . }} - {{- end }} containers: - name: {{ template "fullname" . }} - image: "{{.Values.image.name}}:{{.Values.image.tag}}" + image: "{{ .Values.image.name }}:{{ .Values.image.tag }}" imagePullPolicy: {{.Values.image.pullPolicy}} resources: {{ toYaml .Values.resources | indent 12 }} @@ -41,12 +37,19 @@ spec: readOnly: true - name: pos-files mountPath: /mnt/pos/ + {{- if eq .Values.sumologic.fluentdSource "systemd" }} + - name: rkt-logs + mountPath: /var/lib/rkt/ + readOnly: true + {{- end }} env: - name: COLLECTOR_URL valueFrom: secretKeyRef: name: "{{ template "fullname-secrets" . }}" key: collector-url + - name: FLUENTD_SOURCE + value: {{ quote .Values.sumologic.fluentdSource }} {{- if .Values.sumologic.flushInterval }} - name: FLUSH_INTERVAL value: {{ quote .Values.sumologic.flushInterval }} @@ -79,25 +82,37 @@ spec: - name: KUBERNETES_META value: {{ quote .Values.sumologic.kubernetesMeta }} {{- end }} - {{- if .Values.sumologic.excludePath }} - - name: EXCLUDE_PATH - value: {{ quote .Values.sumologic.excludePath }} + {{- if .Values.sumologic.excludeContainerRegex }} + - name: EXCLUDE_CONTAINER_REGEX + value: {{ quote .Values.sumologic.excludeContainerRegex }} + {{- end }} + {{- if .Values.sumologic.excludeFacilityRegex }} + - name: EXCLUDE_FACILITY_REGEX + value: {{ quote .Values.sumologic.excludeFacilityRegex }} + {{- end }} + {{- if .Values.sumologic.excludeHostRegex }} + - name: EXCLUDE_HOST_REGEX + value: {{ quote .Values.sumologic.excludeHostRegex }} {{- end }} {{- if .Values.sumologic.excludeNamespaceRegex }} - name: EXCLUDE_NAMESPACE_REGEX value: {{ quote .Values.sumologic.excludeNamespaceRegex }} {{- end }} + {{- if .Values.sumologic.excludePath }} + - name: EXCLUDE_PATH + value: {{ quote .Values.sumologic.excludePath }} + {{- end }} {{- if .Values.sumologic.excludePodRegex }} - name: EXCLUDE_POD_REGEX value: {{ quote .Values.sumologic.excludePodRegex }} {{- end }} - {{- if .Values.sumologic.excludeContainerRegex }} - - name: EXCLUDE_CONTAINER_REGEX - value: {{ quote .Values.sumologic.excludeContainerRegex }} + {{- if .Values.sumologic.excludePriorityRegex }} + - name: EXCLUDE_PRIORITY_REGEX + value: {{ quote .Values.sumologic.excludePriorityRegex }} {{- end }} - {{- if .Values.sumologic.excludeHostRegex }} - - name: EXCLUDE_HOST_REGEX - value: {{ quote .Values.sumologic.excludeHostRegex }} + {{- if .Values.sumologic.excludeUnitRegex }} + - name: EXCLUDE_UNIT_REGEX + value: {{ quote .Values.sumologic.excludeUnitRegex }} {{- end }} {{- if .Values.sumologic.fluentdOpt }} - name: FLUENTD_OPT @@ -107,6 +122,7 @@ spec: - name: VERIFY_SSL value: {{ quote .Values.sumologic.verifySsl }} {{- end }} + serviceAccountName: {{ if .Values.rbac.create }}{{ template "fullname" . }}{{ else }}"{{ .Values.rbac.serviceAccountName }}"{{ end }} volumes: - name: pos-files {{- if .Values.persistence.enabled }} @@ -118,6 +134,11 @@ spec: - name: host-logs hostPath: path: /var/log/ + {{- if eq .Values.sumologic.fluentdSource "systemd" }} + - name: rkt-logs + hostPath: + path: /var/lib/rkt + {{- end }} - name: docker-logs hostPath: path: /var/lib/docker diff --git a/stable/sumologic-fluentd/templates/secrets.yaml b/stable/sumologic-fluentd/templates/secrets.yaml index a86403df04..8b5dcf5121 100644 --- a/stable/sumologic-fluentd/templates/secrets.yaml +++ b/stable/sumologic-fluentd/templates/secrets.yaml @@ -3,7 +3,7 @@ kind: Secret metadata: name: "{{ template "fullname-secrets" . }}" labels: - app: {{ template "fullname" . }} + app: {{ template "name" . }} chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" release: "{{ .Release.Name }}" heritage: "{{ .Release.Service }}" diff --git a/stable/sumologic-fluentd/templates/serviceaccount.yaml b/stable/sumologic-fluentd/templates/serviceaccount.yaml index 3969620f0b..04f6dacced 100644 --- a/stable/sumologic-fluentd/templates/serviceaccount.yaml +++ b/stable/sumologic-fluentd/templates/serviceaccount.yaml @@ -1,10 +1,10 @@ -{{- if not .Values.rbac.serviceAccountName }} +{{- if .Values.rbac.create }} apiVersion: v1 kind: ServiceAccount metadata: name: {{ template "fullname" . }} labels: - app: {{ template "fullname" . }} + app: {{ template "name" . }} chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" release: "{{ .Release.Name }}" heritage: "{{ .Release.Service }}" diff --git a/stable/sumologic-fluentd/values.yaml b/stable/sumologic-fluentd/values.yaml index 2dfd41d7d2..73bfcff523 100644 --- a/stable/sumologic-fluentd/values.yaml +++ b/stable/sumologic-fluentd/values.yaml @@ -1,8 +1,8 @@ # Default values for sumologic-fluentd. image: name: sumologic/fluentd-kubernetes-sumologic - tag: latest - pullPolicy: Always + tag: v1.4 + pullPolicy: IfNotPresent ## Annotations to add to the DaemonSet's Pods podAnnotations: {} @@ -21,7 +21,10 @@ updateStrategy: OnDelete sumologic: ## You'll need to set this to sumo collector, before the agent will run. ## ref: https://github.com/SumoLogic/fluentd-kubernetes-sumologic#sumologic - collectorUrl: + collectorUrl: "" + + ## The source of fluentd logs, either file or systemd + fluentdSource: file ## How frequently to push logs to SumoLogic (default 5s) ## ref: https://github.com/SumoLogic/fluentd-kubernetes-sumologic#options @@ -31,52 +34,74 @@ sumologic: # numThreads: 1 ## Set the _sourceName metadata field in SumoLogic. (Default "%{namespace}.%{pod}.%{container}") - # sourceName: + sourceName: "" ## Set the _sourceCategory metadata field in SumoLogic. (Default "%{namespace}/%{pod_name}") - # sourceCategory: + sourceCategory: "" ## Set the prefix, for _sourceCategory metadata. (Default nil) - # sourceCategoryPrefix: + sourceCategoryPrefix: "" ## Used to replace - with another character. (default /) - # sourceCategoryReplaceDash: + sourceCategoryReplaceDash: "" ## Format to post logs into Sumo. json, json_merge, or text (default json) - # logFormat: json + logFormat: json ## Include or exclude Kubernetes metadata such as namespace and pod_name if ## using json log format. (default true) - # kubernetesMeta: true + kubernetesMeta: true + + ## A ruby regex for containers. All matching containers will be excluded + ## from Sumo Logic. The logs will still be sent to FluentD + ## ref: http://rubular.com/ + ## ref: https://github.com/SumoLogic/fluentd-kubernetes-sumologic#options + excludeContainerRegex: "" ## Files matching this pattern will be ignored by the in_tail plugin, and will ## not be sent to Kubernetes or Sumo Logic. This can be a comma separated list. ## ref: http://docs.fluentd.org/v0.12/articles/in_tail#excludepath - # excludePath: - - ## A ruby regex for namespaces. All matching namespaces will be excluded - ## from Sumo Logic. The logs will still be sent to FluentD - ## ref: http://rubular.com/ - # excludeNamespaceRegex: - - ## A ruby regex for pods. All matching pods will be excluded from Sumo - ## Logic. The logs will still be sent to FluentD - ## ref: http://rubular.com/ - # excludePodRegex: - - ## A ruby regex for containers. All matching containers will be excluded - ## from Sumo Logic. The logs will still be sent to FluentD - ## ref: http://rubular.com/ - # excludeContainerRegex: + excludePath: "" ## A ruby regex for hosts. All matching hosts will be excluded from Sumo ## Logic. The logs will still be sent to FluentD ## ref: http://rubular.com/ - # excludeHostRegex: + ## ref: https://github.com/SumoLogic/fluentd-kubernetes-sumologic#options + excludeHostRegex: "" + + ## A ruby regex for namespaces. All matching namespaces will be excluded + ## from Sumo Logic. The logs will still be sent to FluentD + ## ref: http://rubular.com/ + ## ref: https://github.com/SumoLogic/fluentd-kubernetes-sumologic#options + excludeNamespaceRegex: "" + + ## A ruby regex for pods. All matching pods will be excluded from Sumo + ## Logic. The logs will still be sent to FluentD + ## ref: http://rubular.com/ + excludePodRegex: "" + + ## A ruby regex for systemd units. All matching facilities will be excluded from + ## Sumo Logic. The logs will still be sent to FluentD + ## ref: http://rubular.com/ + ## ref: https://github.com/SumoLogic/fluentd-kubernetes-sumologic#options + excludeFacilityRegex: "" + + ## A ruby regex for syslog priorities, which are integers represented as + ## strings. All matching priorities will be excluded from + ## Sumo Logic. The logs will still be sent to FluentD + ## ref: http://rubular.com/ + ## ref: https://github.com/SumoLogic/fluentd-kubernetes-sumologic#options + excludePriorityRegex: "" + + ## A ruby regex for systemd units. All matching hosts will be excluded from + ## Sumo Logic. The logs will still be sent to FluentD + ## ref: http://rubular.com/ + ## ref: https://github.com/SumoLogic/fluentd-kubernetes-sumologic#options + excludeUnitRegex: "" ## Fluentd command line options ## ref: http://docs.fluentd.org/v0.12/articles/command-line-option - # fluentdOpt: + fluentdOpt: "" ## By default, the daemonset will store position files, for logs tailed, in an ## emptyDir. If you have a directory, on the host, to store pos files, specify @@ -93,4 +118,10 @@ resources: cpu: 256m memory: 256Mi -rbac: {} +rbac: + ## If true, create and use RBAC resources + create: false + + ## Ignored if rbac.create is true + serviceAccountName: default +