Not exiting when executable not found (#702)

Regrading https://github.com/aquasecurity/kube-bench/issues/701 where kube bench is crushing when not finding components

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,18 @@
+---
+name: Bug report
+about: Tell us about a problem you are experiencing
+---
+
+**What steps did you take and what happened:**
+
+[A clear and concise description of what the bug is, and what commands you ran. If possible please supply logs generated with the `-v 3` parameter.)
+
+**What did you expect to happen:**
+
+**Environment** 
+
+[Please specify the version of kube-bench and Kubernetes]
+
+**Anything else you would like to add:**
+
+[Miscellaneous information that will assist in solving the issue.]

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,9 @@
+---
+blank_issues_enabled: false
+contact_links:
+  - name: Feature request
+    url: https://github.com/aquasecurity/kube-bench/discussions/new?category_id=19113743
+    about: Share ideas for new features
+  - name: Ask a question
+    url: https://github.com/aquasecurity/kube-bench/discussions/new?category_id=19113742
+    about: Ask questions and discuss with other community members

.gitignore

@@ -2,3 +2,12 @@ kube-bench
 *.swp
 vendor
 dist
+.vscode/
+hack/kind.test.yaml
+coverage.txt
+
+.idea/
+
+# Directory junk file
+.DS_Store
+thumbs.db

.goreleaser.yml

@@ -1,18 +1,37 @@
+---
+env:
+  - GO111MODULE=on
+  - KUBEBENCH_CFG=/etc/kube-bench/cfg
 builds:
   - main: main.go
     binary: kube-bench
     goos:
-      - darwin
       - linux
     goarch:
       - amd64
+      - arm
+      - arm64
+    goarm:
+      - 6
+      - 7
+    ldflags:
+      - "-X github.com/aquasecurity/kube-bench/cmd.KubeBenchVersion={{.Version}}"
+      - "-X github.com/aquasecurity/kube-bench/cmd.cfgDir={{.Env.KUBEBENCH_CFG}}"
 # Archive customization
-archive:
-  format: tar.gz
-fpm:
-  vendor: Aqua Security
-  description: "The Kubernetes Bench for Security is a Go application that checks whether Kubernetes is deployed according to security best practices"
-  license: Apache-2.0
-  formats:
-    - deb
-    - rpm
+archives:
+  - id: default
+    format: tar.gz
+    name_template: '{{ .Binary }}_{{.Version}}_{{ .Os }}_{{ .Arch }}{{ if .Arm }}v{{.Arm }}{{ end }}'
+    files:
+      - "cfg/**/*"
+nfpms:
+  -
+    vendor: Aqua Security
+    description: "The Kubernetes Bench for Security is a Go application that checks whether Kubernetes is deployed according to security best practices"
+    license: Apache-2.0
+    homepage: https://github.com/aquasecurity/kube-bench
+    files:
+      "cfg/**/*": "/etc/kube-bench/cfg"
+    formats:
+      - deb
+      - rpm

.travis.yml

@@ -1,25 +1,53 @@
 ---
 language: go
 
+services:
+  - docker
+
 notifications:
   email: false
 
 before_install:
   - sudo apt-get -qq update
   - sudo apt-get install -y rpm
-  - gem install --no-ri --no-rdoc fpm
-
-install:
-  - go get -v github.com/Masterminds/glide
-  - cd $GOPATH/src/github.com/Masterminds/glide && git checkout tags/v0.12.3 && go install && cd - # use a known good glide version
-  - glide install
+  - pip install --user yamllint==1.18.0
+  - gem install --no-document fpm
+  - go get -t -v ./...
+  - |
+    if [ "$TRAVIS_BRANCH" = "master" -a "$TRAVIS_PULL_REQUEST" = "false" ]; then
+      echo "$DOCKER_PASSWORD" | docker login --username "$DOCKER_USERNAME" --password-stdin
+    fi
 
 script:
-  - go test ./...
+  - yamllint -c ./.yamllint.yaml .
+
+  # Run unit and integration tests
+  - make tests
+  - make integration-tests
+
+  # Build a local container image to test that the install sub-command works
+  - IMAGE_NAME=kube-bench make build-docker
+  - docker run -v `pwd`:/host kube-bench install
+  - test -d cfg
+  - test -f kube-bench
+  # Build and push the multi-arch Docker image
+  - |
+    if [ "$TRAVIS_BRANCH" = "master" -a "$TRAVIS_PULL_REQUEST" = "false" ]; then
+      make docker
+    fi
 
 after_success:
-  - test -n "$TRAVIS_TAG" && curl -sL https://git.io/goreleaser | bash
+  - bash <(curl -s https://codecov.io/bash)
+deploy:
+  - provider: script
+    skip_cleanup: true
+    script: curl -sL https://git.io/goreleaser | bash
+    on:
+      tags: true
+      condition: "$TRAVIS_OS_NAME = linux"
 
 env:
   global:
     secure: mb8AYZKDo6hkKN+2F9ldXcw27Yn2AfxpXvKlD8GD7NdGOI+TaiSFbE0I+qqTa/1DqcRekCQwqN7OG/17s9JDkgzUXYuYUGlVUOM4WbeJoSlzJFIOh9r9R/JddluYJohypgkE20IBHIrEHq5sY0Nn1Pl9WgSQFaVcQjxkX009AOuVjN0o5HcoXsb5hAzvHrpoSPkcSSqq7VWab60TgUttVaRlZSGwGdSYQEqk5TdO0hWHuXyxaaEPybgFIyZLLbxPS4JmMz8n3Sngetpw9Jgc+V9Fc7wKXpjvZZ33SpArG5p5ZFFu2YQOXFLZth9qtQOjduQ2gU1kHN6WjWnJ8QX2s8vmU38Tk19kd5i+mz9dvc87IdBvmTIqVYSpM6AAYa2osBGP3f97Rj2S68lTad4ecSVyHdsjz56vdE3ZH4wskswmogbKkVdvO4biPHxT6odszBxYLEJuRJyZ7ckXd52MCzqAUPrw7YUuH8N1mLIlf7V5bW5R+q4DlKw774zxnHiWrymXGvlINSrB0qxBn8Fii6ib+Pacl3PuqSumCcgIHlVjqrzIXaqcTMn2/ABZYC99mralGvwA/EgNa8CBKB5evMCEwWa5Ntvcs2I2DFcO5Q2WzN4H0YScyAzzCzK7/3hWJE/rUIJntwiSXkV3MSa1yxWSGGH8F1lcz+lzgTBm/MU=
+  matrix:
+    - GO111MODULE=on

.yamllint.yaml

@@ -0,0 +1,6 @@
+---
+extends: default
+
+rules:
+  line-length: disable
+  truthy: disable

CONTRIBUTING.md

@@ -0,0 +1,24 @@
+Thank you for taking an interest in contributing to kube-bench !
+
+## Issues
+
+- Feel free to open an issue for any reason as long as you make it clear if the issue is about a bug/feature/question/comment.
+- Please spend some time giving due diligence to the issue tracker. Your issue might be a duplicate. If it is, please add your comment to the existing issue.
+- Remember, users might be searching for your issue in the future. So please give it a meaningful title to help others.
+- The issue should clearly explain the reason for opening the proposal if you have any, along with any relevant technical information.
+- For questions and bug reports, please include the following information:
+  - version of kube-bench you are running (from kube-bench version) along with the command line options you are using.
+  - version of Kubernetes you are running (from kubectl version or oc version for Openshift).
+  - Verbose log output, by setting the `-v 10` command line option.
+
+## Pull Requests
+
+1. Every Pull Request should have an associated Issue, unless you are fixing a trivial documentation issue.
+1. We will not accept changes to LICENSE, NOTICE or CONTRIBUTING from outside the Aqua Security team. Please raise an Issue if you believe there is a problem with any of these files. 
+1. Your PR is more likely to be accepted if it focuses on just one change.
+1. Describe what the PR does. There's no convention enforced, but please try to be concise and descriptive. Treat the PR description as a commit message. Titles that start with "fix"/"add"/"improve"/"remove" are good examples.
+1. Please add the associated Issue in the PR description.
+1. There's no need to add or tag reviewers.
+1. If a reviewer commented on your code or asked for changes, please remember to mark the discussion as resolved after you address it. PRs with unresolved issues should not be merged (even if the comment is unclear or requires no action from your side).
+1. Please include a comment with the results before and after your change.
+1. Your PR is more likely to be accepted if it includes tests (We have not historically been very strict about tests, but we would like to improve this!).

Dockerfile

@@ -1,13 +1,31 @@
-FROM golang:1.9
-WORKDIR /kube-bench
-RUN go get github.com/aquasecurity/kube-bench
+FROM golang:1.15 AS build
+WORKDIR /go/src/github.com/aquasecurity/kube-bench/
+COPY go.mod go.sum ./
+COPY main.go .
+COPY check/ check/
+COPY cmd/ cmd/
+ARG KUBEBENCH_VERSION
+ARG GOOS=linux
+ARG GOARCH=amd64
+RUN GO111MODULE=on CGO_ENABLED=0 GOOS=$GOOS GOARCH=$GOARCH go build -a -ldflags "-X github.com/aquasecurity/kube-bench/cmd.KubeBenchVersion=${KUBEBENCH_VERSION} -w" -o /go/bin/kube-bench
 
-FROM alpine:latest
-WORKDIR /
-COPY --from=0 /go/bin/kube-bench /kube-bench 
-COPY --from=0 /go/src/github.com/aquasecurity/kube-bench/cfg /cfg
-COPY --from=0 /go/src/github.com/aquasecurity/kube-bench/entrypoint.sh /entrypoint.sh
-ENTRYPOINT /entrypoint.sh
+FROM alpine:3.12 AS run
+WORKDIR /opt/kube-bench/
+# add GNU ps for -C, -o cmd, and --no-headers support
+# https://github.com/aquasecurity/kube-bench/issues/109
+RUN apk --no-cache add procps
+
+# Openssl is used by OpenShift tests
+# https://github.com/aquasecurity/kube-bench/issues/535
+RUN apk --no-cache add openssl
+
+ENV PATH=$PATH:/usr/local/mount-from-host/bin
+
+COPY --from=build /go/bin/kube-bench /usr/local/bin/kube-bench
+COPY entrypoint.sh .
+COPY cfg/ cfg/
+ENTRYPOINT ["./entrypoint.sh"]
+CMD ["install"]
 
 # Build-time metadata as defined at http://label-schema.org
 ARG BUILD_DATE

NOTICE

@@ -0,0 +1,5 @@
+kube-bench
+Copyright 2017-2019 Aqua Security Software Ltd. 
+
+This product includes software developed by Aqua Security (https://aquasec.com).
+

README.md

@@ -1,125 +1,383 @@
 [![Build Status](https://travis-ci.org/aquasecurity/kube-bench.svg?branch=master)](https://travis-ci.org/aquasecurity/kube-bench)
-[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/aquasecurity/kube-bench/blob/master/LICENSE)
 [![Docker image](https://images.microbadger.com/badges/image/aquasec/kube-bench.svg)](https://microbadger.com/images/aquasec/kube-bench "Get your own image badge on microbadger.com")
 [![Source commit](https://images.microbadger.com/badges/commit/aquasec/kube-bench.svg)](https://microbadger.com/images/aquasec/kube-bench)
+[![Coverage Status][cov-img]][cov]
 
-# kube-bench
+[cov-img]: https://codecov.io/github/aquasecurity/kube-bench/branch/master/graph/badge.svg
+[cov]: https://codecov.io/github/aquasecurity/kube-bench
+<img src="images/kube-bench.png" width="200" alt="kube-bench logo">
 
-The Kubernetes Bench for Security is a Go application that checks whether Kubernetes is deployed securely by running the checks documented in the CIS Kubernetes Benchmark.
+kube-bench is a Go application that checks whether Kubernetes is deployed securely by running the checks documented in the [CIS Kubernetes Benchmark](https://www.cisecurity.org/benchmark/kubernetes/).
+
+Tests are configured with YAML files, making this tool easy to update as test specifications evolve.
+
+### Please Note
+
+1. kube-bench implements the [CIS Kubernetes Benchmark](https://www.cisecurity.org/benchmark/kubernetes/) as closely as possible. Please raise issues here if kube-bench is not correctly implementing the test as described in the Benchmark. To report issues in the Benchmark itself (for example, tests that you believe are inappropriate), please join the [CIS community](https://cisecurity.org).
+
+1. There is not a one-to-one mapping between releases of Kubernetes and releases of the CIS benchmark. See [CIS Kubernetes Benchmark support](#cis-kubernetes-benchmark-support) to see which releases of Kubernetes are covered by different releases of the benchmark.
+
+1. It is impossible to inspect the master nodes of managed clusters, e.g. GKE, EKS and AKS, using kube-bench as one does not have access to such nodes, although it is still possible to use kube-bench to check worker node configuration in these environments.
 
-Tests are configured with YAML files, making this tool easy to update as test specifications evolve. 
 
 ![Kubernetes Bench for Security](https://raw.githubusercontent.com/aquasecurity/kube-bench/master/images/output.png "Kubernetes Bench for Security")
 
+Table of Contents
+=================
+
+  - [CIS Kubernetes Benchmark support](#cis-kubernetes-benchmark-support)
+  - [Installation](#installation)
+  - [Running kube-bench](#running-kube-bench)
+    - [Running inside a container](#running-inside-a-container)
+    - [Running in a Kubernetes cluster](#running-in-a-kubernetes-cluster)
+    - [Running in an AKS cluster](#running-in-an-aks-cluster)
+    - [Running in an EKS cluster](#running-in-an-eks-cluster)
+    - [Running on OpenShift](#running-on-openshift)
+    - [Running in an GKE cluster](#running-in-a-gke-cluster)
+    - [Installing from a container](#installing-from-a-container)
+    - [Installing from sources](#installing-from-sources)
+  - [Output](#output)
+  - [Configuration](#configuration)
+  - [Troubleshooting](#troubleshooting)
+  - [Test config YAML representation](#test-config-yaml-representation)
+    - [Omitting checks](#omitting-checks)
+  - [Roadmap](#roadmap)
+  - [Testing locally with kind](#testing-locally-with-kind)
+  - [Contributing](#contributing)
+    - [Bugs](#bugs)
+    - [Features](#features)
+    - [Pull Requests](#pull-requests)
+
+
 ## CIS Kubernetes Benchmark support
 
-kube-bench supports the tests for multiple versions of Kubernetes (1.6, 1.7 and 1.8) as defined in the CIS Benchmarks 1.0.0, 1.1.0 and 1.2.0 respectively. It will determine the test set to run based on the Kubernetes version running on the machine. 
+kube-bench supports the tests for Kubernetes as defined in the [CIS Kubernetes Benchmarks](https://www.cisecurity.org/benchmark/kubernetes/).
+
+| CIS Kubernetes Benchmark | kube-bench config | Kubernetes versions |
+|---|---|---|
+| [1.5.1](https://workbench.cisecurity.org/benchmarks/4892) | cis-1.5 | 1.15- |
+| [1.6.0](https://workbench.cisecurity.org/benchmarks/4834) | cis-1.6 | 1.16- |
+| [GKE 1.0.0](https://workbench.cisecurity.org/benchmarks/4536) | gke-1.0 | GKE |
+| [EKS 1.0.0](https://workbench.cisecurity.org/benchmarks/5190) | eks-1.0 | EKS |
+| Red Hat OpenShift hardening guide | rh-0.7 | OCP 3.10-3.11 |
+
+By default, kube-bench will determine the test set to run based on the Kubernetes version running on the machine, but please note that kube-bench does not automatically detect OpenShift and GKE - see the section below on [Running kube-bench](https://github.com/aquasecurity/kube-bench#running-kube-bench).
 
 ## Installation
 
-You can either install kube-bench through a dedicated container, or compile it from source:
+You can choose to
+* run kube-bench from inside a container (sharing PID namespace with the host)
+* run a container that installs kube-bench on the host, and then run kube-bench directly on the host
+* install the latest binaries from the [Releases page](https://github.com/aquasecurity/kube-bench/releases), though please note that you also need to download the config and test files from the `cfg` directory
+* compile it from source.
 
-1. Container installation:
-Run ```docker run --rm -v `pwd`:/host aquasec/kube-bench:latest```. This will copy the kube-bench binary and configuration to you host. You can then run ```./kube-bench <master|node>```.
+## Running kube-bench
 
-2. Install from sources:
-If Go is installed on the target machines, you can simply clone this repository and run as follows (assuming your [$GOPATH is set](https://github.com/golang/go/wiki/GOPATH)):
+If you run kube-bench directly from the command line you may need to be root / sudo to have access to all the config files.
 
-```go get github.com/aquasecurity/kube-bench
-go get github.com/Masterminds/glide
-cd $GOPATH/src/github.com/aquasecurity/kube-bench
-$GOPATH/bin/glide install
-go build -o kube-bench . 
-./kube-bench <master|node>
-```
+kube-bench automatically selects which `controls` to use based on the detected
+node type and the version of Kubernetes a cluster is running. This behavior
+can be overridden by specifying the `master` or `node` subcommand and the
+`--version` flag on the command line.
 
-## Usage
-```./kube-bench [command]```
+
+The Kubernetes version can also be set with the `KUBE_BENCH_VERSION` environment variable.
+The value of `--version` takes precedence over the value of `KUBE_BENCH_VERSION`.
+
+For example, run kube-bench against a master with version auto-detection:
 
 ```
-Available Commands:
-  federated   Run benchmark checks for a Kubernetes federated deployment.
-  help        Help about any command
-  master      Run benchmark checks for a Kubernetes master node.
-  node        Run benchmark checks for a Kubernetes node.
-
-Flags:
-  -c, --check string          A comma-delimited list of checks to run as specified in CIS document. Example --check="1.1.1,1.1.2"
-      --config string         config file (default is ./cfg/config.yaml)
-  -g, --group string          Run all the checks under this comma-delimited list of groups. Example --group="1.1"
-      --installation string   Specify how kubernetes cluster was installed. Possible values are default,hyperkube,kops,kubeadm (default "default")
-      --json                  Prints the results as JSON
-  -v, --verbose               verbose output (default false)
+kube-bench master
 ```
 
-## Configuration
-Kubernetes config and binary file locations and names can vary from installation to installation, so these are configurable in the `cfg/config.yaml` file. 
+Or run kube-bench against a worker node using the tests for Kubernetes version 1.13:
 
-For each type of node (*master*, *node* or *federated*) there is a list of components, and for each component there is a set of binaries (*bins*) and config files (*confs*) that kube-bench will look for (in the order they are listed). If your installation uses a different binary name or config file location for a Kubernetes component, you can add it to `cfg/config.yaml`.  
-
-* **bins** - If there is a *bins* list for a component, at least one of these binaries must be running. The tests will consider the parameters for the first binary in the list found to be running. 
-* **podspecs** - From version 1.2.0 of the benchmark (tests for Kubernetes 1.8), the remediation instructions were updated to assume that the configuration for several kubernetes components is defined in a pod YAML file, and podspec settings define where to look for that configuration. 
-* **confs** - If one of the listed config files is found, this will be considered for the test. Tests can continue even if no config file is found. If no file is found at any of the listed locations, and a *defaultconf* location is given for the component, the test will give remediation advice using the *defaultconf* location. 
-* **unitfiles** - From version 1.2.0 of the benchmark  (tests for Kubernetes 1.8), the remediation instructions were updated to assume that kubelet configuration is defined in a service file, and this setting defines where to look for that configuration.
-
-## Test config YAML representation
-The tests are represented as YAML documents (installed by default into ./cfg).
-
-An example is as listed below:
 ```
----
-controls:
-id: 1
-text: "Master Checks"
-type: "master"
-groups:
-- id: 1.1
-  text: "Kube-apiserver"
-  checks:
-    - id: 1.1.1
-      text: "Ensure that the --allow-privileged argument is set (Scored)"
-      audit: "ps -ef | grep kube-apiserver | grep -v grep"
-      tests:
-      bin_op: or
-      test_items:
-      - flag: "--allow-privileged"
-        set: true
-      - flag: "--some-other-flag"
-        set: false
-      remediation: "Edit the /etc/kubernetes/config file on the master node and set the KUBE_ALLOW_PRIV parameter to '--allow-privileged=false'"
-      scored: true
+kube-bench node --version 1.13
 ```
 
-Recommendations (called `checks` in this document) can run on Kubernetes Master, Node or Federated API Servers.
-Checks are organized into `groups` which share similar controls (things to check for) and are grouped together in the section of the CIS Kubernetes document.
-These groups are further organized under `controls` which can be of the type `master`, `node` or `federated apiserver` to reflect the various Kubernetes node types.
+`kube-bench` will map the `--version` to the corresponding CIS Benchmark version as indicated by the mapping table above. For example, if you specify `--version 1.15`, this is mapped to CIS Benchmark version `cis-1.15`.
 
-## Tests
-Tests are the items we actually look for to determine if a check is successful or not. Checks can have multiple tests, which must all be successful for the check to pass.
+Alternatively, you can specify `--benchmark` to run a specific CIS Benchmark version:
 
-The syntax for tests:
 ```
-tests:
-- flag:
-  set:
-  compare:
-    op:
-    value:
+kube-bench node --benchmark cis-1.5
+```
+
+If you want to target specific CIS Benchmark `target` (i.e master, node, etcd, etc...)
+you can use the `run --targets` subcommand.
+```
+kube-bench --benchmark cis-1.5 run --targets master,node
+```
+or
+```
+kube-bench --benchmark cis-1.5 run --targets master,node,etcd,policies
+```
+
+The following table shows the valid targets based on the CIS Benchmark version.
+| CIS Benchmark | Targets |
+|---|---|
+| cis-1.5| master, controlplane, node, etcd, policies |
+| cis-1.6| master, controlplane, node, etcd, policies |
+| gke-1.0| master, controlplane, node, etcd, policies, managedservices |
+| eks-1.0| controlplane, node, policies, managedservices |
+
+If no targets are specified, `kube-bench` will determine the appropriate targets based on the CIS Benchmark version.
+
+`controls` for the various versions of CIS Benchmark can be found in directories
+with same name as the CIS Benchmark versions under `cfg/`, for example `cfg/cis-1.5`.
+
+**Note:**  **`It is an error to specify both --version and --benchmark flags together`**
+
+### Running inside a container
+
+You can avoid installing kube-bench on the host by running it inside a container using the host PID namespace and mounting the `/etc` and `/var` directories where the configuration and other files are located on the host so that kube-bench can check their existence and permissions.
+
+```
+docker run --pid=host -v /etc:/etc:ro -v /var:/var:ro -t aquasec/kube-bench:latest [master|node] --version 1.13
+```
+
+> Note: the tests require either the kubelet or kubectl binary in the path in order to auto-detect the Kubernetes version. You can pass `-v $(which kubectl):/usr/local/mount-from-host/bin/kubectl` to resolve this. You will also need to pass in kubeconfig credentials. For example:
+
+```
+docker run --pid=host -v /etc:/etc:ro -v /var:/var:ro -v $(which kubectl):/usr/local/mount-from-host/bin/kubectl -v ~/.kube:/.kube -e KUBECONFIG=/.kube/config -t aquasec/kube-bench:latest [master|node]
+```
+
+You can use your own configs by mounting them over the default ones in `/opt/kube-bench/cfg/`
+
+```
+docker run --pid=host -v /etc:/etc:ro -v /var:/var:ro -t -v path/to/my-config.yaml:/opt/kube-bench/cfg/config.yam -v $(which kubectl):/usr/local/mount-from-host/bin/kubectl -v ~/.kube:/.kube -e KUBECONFIG=/.kube/config aquasec/kube-bench:latest [master|node]
+```
+
+### Running in a Kubernetes cluster
+
+You can run kube-bench inside a pod, but it will need access to the host's PID namespace in order to check the running processes, as well as access to some directories on the host where config files and other files are stored.
+
+Master nodes are automatically detected by kube-bench and will run master checks when possible.
+The detection is done by verifying that mandatory components for master, as defined in the config files, are running (see [Configuration](#configuration)).
+
+The supplied `job.yaml` file can be applied to run the tests as a job. For example:
+
+```bash
+$ kubectl apply -f job.yaml
+job.batch/kube-bench created
+
+$ kubectl get pods
+NAME                      READY   STATUS              RESTARTS   AGE
+kube-bench-j76s9   0/1     ContainerCreating   0          3s
+
+# Wait for a few seconds for the job to complete
+$ kubectl get pods
+NAME                      READY   STATUS      RESTARTS   AGE
+kube-bench-j76s9   0/1     Completed   0          11s
+
+# The results are held in the pod's logs
+kubectl logs kube-bench-j76s9
+[INFO] 1 Master Node Security Configuration
+[INFO] 1.1 API Server
 ...
 ```
-Tests have various `operations` which are used to compare the output of audit commands for success.
-These operations are:
 
-- `eq`: tests if the flag value is equal to the compared value.
-- `noteq`: tests if the flag value is unequal to the compared value.
-- `gt`: tests if the flag value is greater than the compared value.
-- `gte`: tests if the flag value is greater than or equal to the compared value.
-- `lt`: tests if the flag value is less than the compared value.
-- `lte`: tests if the flag value is less than or equal to the compared value.
-- `has`: tests if the flag value contains the compared value.
-- `nothave`: tests if the flag value does not contain the compared value.
+You can still force to run specific master or node checks using respectively `job-master.yaml` and `job-node.yaml`.
 
-# Roadmap 
-Going forward we plan to release updates to kube-bench to add support for new releases of the Benchmark, which in turn we can anticipate being made for each new Kubernetes release. 
+To run the tests on the master node, the pod needs to be scheduled on that node. This involves setting a nodeSelector and tolerations in the pod spec.
 
-We welcome PRs and issue reports. 
+The default labels applied to master nodes has changed since Kubernetes 1.11, so if you are using an older version you may need to modify the nodeSelector and tolerations to run the job on the master node.
+
+
+### Running in an AKS cluster
+
+1. Create an AKS cluster(e.g. 1.13.7) with RBAC enabled, otherwise there would be 4 failures
+
+1. Use the [kubectl-enter plugin] (https://github.com/kvaps/kubectl-enter) to shell into a node
+`
+kubectl-enter {node-name}
+`
+or ssh to one agent node
+could open nsg 22 port and assign a public ip for one agent node (only for testing purpose)
+
+1. Run CIS benchmark to view results:
+```
+docker run --rm -v `pwd`:/host aquasec/kube-bench:latest install
+./kube-bench node
+```
+kube-bench cannot be run on AKS master nodes
+
+### Running in an EKS cluster
+
+There is a `job-eks.yaml` file for running the kube-bench node checks on an EKS cluster. The significant difference on EKS is that it's not possible to schedule jobs onto the master node, so master checks can't be performed
+
+1. To create an EKS Cluster refer to [Getting Started with Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html) in the *Amazon EKS User Guide*
+  - Information on configuring `eksctl`, `kubectl` and the AWS CLI is within
+2. Create an [Amazon Elastic Container Registry (ECR)](https://docs.aws.amazon.com/AmazonECR/latest/userguide/what-is-ecr.html) repository to host the kube-bench container image
+```
+aws ecr create-repository --repository-name k8s/kube-bench --image-tag-mutability MUTABLE
+```
+3. Download, build and push the kube-bench container image to your ECR repo
+```
+git clone https://github.com/aquasecurity/kube-bench.git
+cd kube-bench
+aws ecr get-login-password --region <AWS_REGION> | docker login --username <AWS_USERNAME> --password-stdin <AWS_ACCT_NUMBER>.dkr.ecr.<AWS_REGION>.amazonaws.com
+docker build -t k8s/kube-bench .
+docker tag k8s/kube-bench:latest <AWS_ACCT_NUMBER>.dkr.ecr.<AWS_REGION>.amazonaws.com/k8s/kube-bench:latest
+docker push <AWS_ACCT_NUMBER>.dkr.ecr.<AWS_REGION>.amazonaws.com/k8s/kube-bench:latest
+```
+4. Copy the URI of your pushed image, the URI format is like this: `<AWS_ACCT_NUMBER>.dkr.ecr.<AWS_REGION>.amazonaws.com/k8s/kube-bench:latest`
+5. Replace the `image` value in `job-eks.yaml` with the URI from Step 4
+6. Run the kube-bench job on a Pod in your Cluster: `kubectl apply -f job-eks.yaml`
+7. Find the Pod that was created, it *should* be in the `default` namespace: `kubectl get pods --all-namespaces`
+8. Retrieve the value of this Pod and output the report, note the Pod name will vary: `kubectl logs kube-bench-<value>`
+  - You can save the report for later reference: `kubectl logs kube-bench-<value> > kube-bench-report.txt`
+
+### Running on OpenShift
+
+| OpenShift Hardening Guide | kube-bench config |
+|---|---|
+| ocp-3.10| rh-0.7 |
+| ocp-3.11| rh-0.7 |
+| ocp-4.* | Not supported |
+
+kube-bench includes a set of test files for Red Hat's OpenShift hardening guide for OCP 3.10 and 3.11. To run this you will need to specify `--benchmark rh-07`, or `--version ocp-3.10` or `--version ocp-3.11`
+
+when you run the `kube-bench` command (either directly or through YAML).
+
+There is work in progress on a [CIS Red Hat OpenShift Container Platform Benchmark](https://workbench.cisecurity.org/benchmarks/5248) which we believe should cover OCP 4.* and we intend to add support in kube-bench when it's published. 
+
+### Running in a GKE cluster
+
+| CIS Benchmark | Targets |
+|---|---|
+| gke-1.0| master, controlplane, node, etcd, policies, managedservices |
+
+kube-bench includes benchmarks for GKE. To run this you will need to specify `--benchmark gke-1.0` when you run the `kube-bench` command.
+
+To run the benchmark as a job in your GKE cluster apply the included `job-gke.yaml`.
+
+```
+kubectl apply -f job-gke.yaml
+```
+
+### Installing from a container
+
+This command copies the kube-bench binary and configuration files to your host from the Docker container:
+** binaries compiled for linux-x86-64 only (so they won't run on macOS or Windows) **
+```
+docker run --rm -v `pwd`:/host aquasec/kube-bench:latest install
+```
+
+You can then run `./kube-bench [master|node]`.
+
+### Installing from sources
+
+If Go is installed on the target machines, you can simply clone this repository and run as follows (assuming your [`GOPATH` is set](https://github.com/golang/go/wiki/GOPATH)):
+
+```shell
+go get github.com/aquasecurity/kube-bench
+cd $GOPATH/src/github.com/aquasecurity/kube-bench
+go build -o kube-bench .
+
+# See all supported options
+./kube-bench --help
+
+# Run all checks
+./kube-bench
+```
+
+## Output
+
+There are four output states:
+- [PASS] indicates that the test was run successfully, and passed.
+- [FAIL] indicates that the test was run successfully, and failed. The remediation output describes how to correct the configuration, or includes an error message describing why the test could not be run. 
+- [WARN] means this test needs further attention, for example it is a test that needs to be run manually. Check the remediation output for further information. 
+- [INFO] is informational output that needs no further action.
+
+Note:
+- If the test is Manual, this always generates WARN (because the user has to run it manually)
+- If the test is Scored, and kube-bench was unable to run the test, this generates FAIL (because the test has not been passed, and as a Scored test, if it doesn't pass then it must be considered a failure).
+- If the test is Not Scored, and kube-bench was unable to run the test, this generates WARN.
+- If the test is Scored, type is empty, and there are no `test_items` present, it generates a WARN. This is to highlight tests that appear to be incompletely defined. 
+
+## Configuration
+
+Kubernetes configuration and binary file locations and names can vary from installation to installation, so these are configurable in the `cfg/config.yaml` file.
+
+Any settings in the version-specific config file `cfg/<version>/config.yaml` take precedence over settings in the main `cfg/config.yaml` file.
+
+You can read more about `kube-bench` configuration in our [documentation](docs/README.md#configuration-and-variables).
+
+## Troubleshooting
+
+Running `kube-bench` with the `-v 3` parameter will generate debug logs that can be very helpful for debugging problems.
+
+If you are using one of the example `job*.yaml` files, you will need to edit the `command` field, for example `["kube-bench", "-v", "3"]`. Once the job has run, the logs can be retrieved using `kubectl logs` on the job's pod.
+
+## Test config YAML representation
+
+The tests (or "controls") are represented as YAML documents (installed by default into `./cfg`). There are different versions of these test YAML files reflecting different versions of the CIS Kubernetes Benchmark. You will find more information about the test file YAML definitions in our [documentation](docs/README.md).
+
+### Omitting checks
+
+If you decide that a recommendation is not appropriate for your environment, you can choose to omit it by editing the test YAML file to give it the check type `skip` as in this example:
+
+```yaml
+  checks:
+  - id: 2.1.1
+    text: "Ensure that the --allow-privileged argument is set to false (Scored)"
+    type: "skip"
+    scored: true
+```
+
+No tests will be run for this check and the output will be marked [INFO].
+
+## Roadmap
+
+Going forward we plan to release updates to kube-bench to add support for new releases of the CIS Benchmark. Note that these are not released as frequently as Kubernetes releases.
+
+We welcome PRs and issue reports.
+
+## Testing locally with kind
+
+Our makefile contains targets to test your current version of kube-bench inside a [Kind](https://kind.sigs.k8s.io/) cluster. This can be very handy if you don't want to run a real Kubernetes cluster for development purposes.
+
+First, you'll need to create the cluster using `make kind-test-cluster` this will create a new cluster if it cannot be found on your machine. By default, the cluster is named `kube-bench` but you can change the name by using the environment variable `KIND_PROFILE`.
+
+*If kind cannot be found on your system the target will try to install it using `go get`*
+
+Next, you'll have to build the kube-bench docker image using `make build-docker`, then we will be able to push the docker image to the cluster using `make kind-push`.
+
+Finally, we can use the `make kind-run` target to run the current version of kube-bench in the cluster and follow the logs of pods created. (Ctrl+C to exit)
+
+Every time you want to test a change, you'll need to rebuild the docker image and push it to cluster before running it again. ( `make build-docker kind-push kind-run` )
+
+## Contributing
+
+### Bugs
+
+If you think you have found a bug please follow the instructions below.
+
+- Please spend a small amount of time giving due diligence to the issue tracker. Your issue might be a duplicate.
+- Open a [new issue](https://github.com/aquasecurity/kube-bench/issues/new) if a duplicate doesn't already exist.
+- Note the version of kube-bench you are running (from `kube-bench version`) and the command line options you are using.
+- Note the version of Kubernetes you are running (from `kubectl version` or `oc version` for OpenShift).
+- Set `-v 10` command line option and save the log output. Please paste this into your issue.
+- Remember users might be searching for your issue in the future, so please give it a meaningful title to help others.
+
+### Features
+
+We also use the GitHub issue tracker to track feature requests. If you have an idea to make kube-bench even more awesome follow the steps below.
+
+- Open a [new issue](https://github.com/aquasecurity/kube-bench/issues/new).
+- Remember users might be searching for your issue in the future, so please give it a meaningful title to helps others.
+- Clearly define the use case, using concrete examples. For example, I type `this` and kube-bench does `that`.
+- If you would like to include a technical design for your feature please feel free to do so.
+
+### Pull Requests
+
+We welcome pull requests!
+
+- Your PR is more likely to be accepted if it focuses on just one change.
+- Please include a comment with the results before and after your change.
+- Your PR is more likely to be accepted if it includes tests. (We have not historically been very strict about tests, but we would like to improve this!).
+- You're welcome to submit a draft PR if you would like early feedback on an idea or an approach.
+- Happy coding!

cfg/1.6/federated.yaml

@@ -1,285 +0,0 @@
----
-controls:
-version: 1.6
-id: 3
-text: "Federated Deployments"
-type: "federated"
-groups:
-- id: 3.1
-  text: "Federation API Server"
-  checks:
-     - id: 3.1.1
-       text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-           - flag: "--anonymous-auth"
-             compare:
-               op: eq
-               value: false
-             set: true
-       remediation: "Edit the deployment specs and set --anonymous-auth=false.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.2
-       text: "Ensure that the --basic-auth-file argument is not set (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-           - flag: "--basic-auth-file"
-             set: false
-       remediation: "Follow the documentation and configure alternate mechanisms for authentication.
-               Then, edit the deployment specs and remove \"--basic-auth-file=<filename>\".\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-  
-     - id: 3.1.3
-       text: "Ensure that the --insecure-allow-any-token argument is not set (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-           - flag: "--insecure-allow-any-token"
-             set: false
-       remediation: "Edit the deployment specs and remove --insecure-allow-any-token.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system" 
-       scored: true
- 
-     - id: 3.1.4
-       text: "Ensure that the --insecure-bind-address argument is not set (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-           - flag: "--insecure-bind-address"
-             set: false
-       remediation: "Edit the deployment specs and remove --insecure-bind-address.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.5
-       text: "Ensure that the --insecure-port argument is set to 0 (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--insecure-port"
-           compare:
-             op: eq
-             value: 0
-           set: true
-       remediation: "Edit the deployment specs and set --insecure-port=0.\n
-              kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.6
-       text: "Ensure that the --secure-port argument is not set to 0 (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests: 
-         bin_op: or
-         test_items:
-           - flag:  "--secure-port"
-             compare:
-               op: gt
-               value: 0
-             set: true
-           - flag: "--secure-port"
-             set: false
-       remediation: "Edit the deployment specs and set the --secure-port argument to the desired port.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.7
-       text: "Ensure that the --profiling argument is set to false (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--profiling"
-           compare:
-             op: eq
-             value: false
-           set: true
-       remediation: "Edit the deployment specs and set \"--profiling=false\".\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       score: true
-
-     - id: 3.1.8
-       text: "Ensure that the admission control policy is not set to AlwaysAdmit (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--admission-control"
-           compare:
-             op: nothave
-             value: AlwaysAdmit 
-           set: true
-       remediation: "Edit the deployment specs and set --admission-control argument to a value that does 
-               not include AlwaysAdmit.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.9
-       text: "Ensure that the admission control policy is set to NamespaceLifecycle (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "admission-control"
-           compare:
-             op: has
-             value: "NamespaceLifecycle"
-           set: true
-       remediation: "Edit the deployment specs and set --admission-control argument to a value that includes NamespaceLifecycle.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.10
-       text: "Ensure that the --audit-log-path argument is set as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--audit-log-path"
-           set: true
-       remediation: "Edit the deployment specs and set --audit-log-path argument as appropriate.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-  
-     - id: 3.1.11
-       text: "Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--audit-log-maxage"
-           compare:
-             op: gte
-             value: 30
-           set: true
-       remediation: "Edit the deployment specs and set --audit-log-maxage to 30 or as appropriate.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-               
-     - id: 3.1.12
-       text: "Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--audit-log-maxbackup"
-           compare:
-             op: gte
-             value: 10
-           set: true
-       remediation: "Edit the deployment specs and set --audit-log-maxbackup to 10 or as appropriate.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.13
-       text: "Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--audit-log-maxsize"
-           compare:
-             op: gte
-             value: 100
-           set: true
-       remediation: "Edit the deployment specs and set --audit-log-maxsize=100 to 100 or as appropriate.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.14
-       text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--authorization-mode"
-           compare:
-             op: nothave
-             value: "AlwaysAllow"
-           set: true
-       remediation: "Edit the deployment specs and set --authorization-mode argument to a value other than AlwaysAllow.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
- 
-     - id: 3.1.15
-       text: "Ensure that the --token-auth-file parameter is not set (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--token-auth-file"
-           set: false
-       remediation: "Follow the documentation and configure alternate mechanisms for authentication.
-               Then, edit the deployment specs and remove the --token-auth-file=<filename> argument.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-  
-     - id: 3.1.16
-       text: "Ensure that the --service-account-lookup argument is set to true (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--service-account-lookup"
-           compare:
-             op: eq
-             value: true
-           set: true
-       remediation: "Edit the deployment specs and set \"--service-account-lookup=true\".\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.17
-       text: "Ensure that the --service-account-key-file argument is set as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--service-account-key-file"
-           set: true
-       remediation: "Edit the deployment specs and set --service-account-key-file argument as appropriate.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.18
-       text: "Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Scored"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         bin_op: and
-         test_items:
-         - flag: "--etcd-certfile"
-           set: true
-         - flag: "--etcd-keyfile"
-           set: true
-       remediation: "Follow the Kubernetes documentation and set up the TLS connection between the
-               federation apiserver and etcd. Then, edit the deployment specs and set \"--etcd-
-               certfile=<path/to/client-certificate-file>\" and \"--etcd-
-               keyfile=<path/to/client-key-file>\" arguments.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.19
-       text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         bin_op: and
-         test_items:
-         - flag: "--tls-cert-file"
-           set: true
-         - flag: "--tls-private-key-file"
-           set: true
-       remediation: "Follow the Kubernetes documentation and set up the TLS connection on the federation
-               apiserver. Then, edit the deployment specs and set \"--tls-cert-file=<path/to/tls-
-               certificate-file>\" and \"--tls-private-key-file=<path/to/tls-key-file>\" :
-                       kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-- id: 3.2
-  text: "Federation Controller Manager"
-  checks:
-     - id: 3.2.1
-       text: "Ensure that the --profiling argument is set to false (Scored)"
-       audit: "ps -ef | grep $fedcontrollermanagerbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--profiling"
-           compare:
-             op: eq
-             value: false
-           set: true
-       remediation: "Edit the deployment specs and set \"--profiling=false\".\n
-               kubectl edit deployments federation-controller-manager-deployment --namespace=federation-system"
-       scored: true

cfg/1.6/master.yaml

@@ -1,966 +0,0 @@
----
-controls:
-version: 1.6
-id: 1
-text: "Master Node Security Configuration"
-type: "master"
-groups:
-- id: 1.1
-  text: "API Server"
-  checks:
-    - id: 1.1.1
-      text: "Ensure that the --allow-privileged argument is set to false (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "allow-privileged"
-          compare:
-            op: eq
-            value: false
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set 
-              the KUBE_ALLOW_PRIV parameter to \"--allow-privileged=false\""
-      scored: true
-
-    - id: 1.1.2
-      text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--anonymous-auth"
-          compare:
-            op: eq
-            value: false
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set 
-              the KUBE_API_ARGS parameter to \"--anonymous-auth=false\""
-      scored: true
-
-    - id: 1.1.3
-      text: "Ensure that the --basic-auth-file argument is not set (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--basic-auth-file"
-          set: false
-      remediation: "Follow the documentation and configure alternate mechanisms for 
-              authentication. Then, edit the $apiserverconf file on the master 
-              node and remove the \"--basic-auth-file=<filename>\" argument from the 
-              KUBE_API_ARGS parameter."
-      scored: true
-
-    - id: 1.1.4
-      text: "Ensure that the --insecure-allow-any-token argument is not set (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag:  "--insecure-allow-any-token"
-          set: false
-      remediation: "Edit the $apiserverconf file on the master node and remove 
-              the --insecure-allow-any-token argument from the KUBE_API_ARGS parameter."
-      scored: true
-
-    - id: 1.1.5
-      text: "Ensure that the --kubelet-https argument is set to true (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests: 
-        bin_op: or
-        test_items:
-        - flag: "--kubelet-https"
-          compare:
-            op: eq
-            value: true
-          set: true
-        - flag: "--kubelet-https"
-          set: false
-      remediation: "Edit the $apiserverconf file on the master node and remove 
-              the --kubelet-https argument from the KUBE_API_ARGS parameter."
-      scored: true
-
-    - id: 1.1.6
-      text: "Ensure that the --insecure-bind-address argument is not set (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--insecure-bind-address"
-          set: false
-      remediation: "Edit the $apiserverconf file on the master node and remove 
-              the --insecure-bind-address argument from the KUBE_API_ADDRESS parameter."
-      scored: true
-
-    - id: 1.1.7
-      text: "Ensure that the --insecure-port argument is set to 0 (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--insecure-port"
-          compare:
-            op: eq
-            value: 0
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set
-              --insecure-port=0 in the KUBE_API_PORT parameter."
-      scored: true
-
-    - id: 1.1.8
-      text: "Ensure that the --secure-port argument is not set to 0 (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests: 
-        bin_op: or
-        test_items:
-          - flag:  "--secure-port"
-            compare:
-              op: gt
-              value: 0
-            set: true
-          - flag: "--secure-port"
-            set: false
-      remediation: "Edit the $apiserverconf file on the master node and either 
-              remove the --secure-port argument from the KUBE_API_ARGS parameter or set 
-              it to a different desired port."
-      scored: true
-
-    - id: 1.1.9
-      text: "Ensure that the --profiling argument is set to false (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--profiling"
-          compare:
-            op: eq
-            value: false
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_API_ARGS parameter to \"--profiling=false\""
-      scored: true
-
-    - id: 1.1.10
-      text: "Ensure that the --repair-malformed-updates argument is set to false (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--repair-malformed-updates"
-          compare:
-            op: eq
-            value: false
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_API_ARGS parameter to \"--repair-malformed-updates=false\""
-      scored: true
-
-    - id: 1.1.11
-      text: "Ensure that the admission control policy is not set to AlwaysAdmit (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--admission-control"
-          compare:
-            op: nothave
-            value: AlwaysAdmit 
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_ADMISSION_CONTROL parameter to a value that does not include AlwaysAdmit"
-      scored: true
- 
-    - id: 1.1.12
-      text: "Ensure that the admission control policy is set to AlwaysPullImages (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--admission-control"
-          compare:
-            op: has
-            value: "AlwaysPullImages"
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_ADMISSION_CONTROL parameter to \"--admission-control=...,AlwaysPullImages,...\""
-      scored: true
- 
-    - id: 1.1.13
-      text: "Ensure that the admission control policy is set to DenyEscalatingExec (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--admission-control"
-          compare:
-            op: has
-            value: "DenyEscalatingExec"
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_ADMISSION_CONTROL parameter to \"--admission-control=...,DenyEscalatingExec,...\""
-      scored: true
-
-    - id: 1.1.14
-      text: "Ensure that the admission control policy is set to SecurityContextDeny (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--admission-control"
-          compare:
-            op: has
-            value: "SecurityContextDeny"
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_ADMISSION_CONTROL parameter to \"--admission-control=...,SecurityContextDeny,...\""
-      scored: true
-
-    - id: 1.1.15
-      text: "Ensure that the admission control policy is set to NamespaceLifecycle (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "admission-control"
-          compare:
-            op: has
-            value: "NamespaceLifecycle"
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_ADMISSION_CONTROL parameter to \"--admission-control=NamespaceLifecycle,...\""
-      scored: true
-
-    - id: 1.1.16
-      text: "Ensure that the --audit-log-path argument is set as appropriate (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--audit-log-path"
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_API_ARGS parameter to \"--audit-log-path=<filename>\""
-      scored: true
-
-    - id: 1.1.17
-      text: "Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--audit-log-maxage"
-          compare:
-            op: gte
-            value: 30
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_API_ARGS parameter to \"--audit-log-maxage=30\""
-      scored: true
-
-    - id: 1.1.18
-      text: "Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--audit-log-maxbackup"
-          compare:
-            op: gte
-            value: 10
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_API_ARGS parameter to \"--audit-log-maxbackup=10\""
-      scored: true
-
-    - id: 1.1.19
-      text: "Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--audit-log-maxsize"
-          compare:
-            op: gte
-            value: 100
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_API_ARGS parameter to \"--audit-log-maxsize=100\""
-      scored: true
-
-    - id: 1.1.20
-      text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--authorization-mode"
-          compare:
-            op: nothave
-            value: "AlwaysAllow"
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the 
-              KUBE_API_ARGS parameter to values other than \"--authorization-mode=AlwaysAllow\""
-      scored: true
-
-    - id: 1.1.21
-      text: "Ensure that the --token-auth-file parameter is not set (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--token-auth-file"
-          set: false
-      remediation: "Follow the documentation and configure alternate mechanisms for authentication. 
-              Then, edit the $apiserverconf file on the master node and remove the 
-              \"--tokenauth-file=<filename>\" argument from the KUBE_API_ARGS parameter."
-      scored: true
- 
-    - id: 1.1.22
-      text: "Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--kubelet-certificate-authority"
-          set: true
-      remediation: "Follow the Kubernetes documentation and setup the TLS connection between 
-              the apiserver and kubelets. Then, edit the $apiserverconf file on the 
-              master node and set the KUBE_API_ARGS parameter to 
-              \"--kubelet-certificate-authority=<ca-string>\""
-      scored: true
-
-    - id: 1.1.23
-      text: "Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        bin_op: and
-        test_items:
-        - flag: "--kubelet-client-certificate"
-          set: true
-        - flag: "--kubelet-client-key"
-          set: true
-      remediation: "Follow the Kubernetes documentation and set up the TLS connection between the apiserver 
-              and kubelets. Then, edit the $apiserverconf file on the master node and set the 
-              KUBE_API_ARGS parameter to \"--kubelet-clientcertificate=<path/to/client-certificate-file>\" 
-              and \"--kubelet-clientkey=<path/to/client-key-file>\""
-      scored: true
-
-    - id: 1.1.24
-      text: "Ensure that the --service-account-lookup argument is set to true (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--service-account-lookup"
-          compare:
-            op: eq
-            value: true
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the KUBE_API_ARGS parameter 
-              to \"--service-account-lookup=true\""
-      scored: true
- 
-    - id: 1.1.25
-      text: "Ensure that the admission control policy is set to PodSecurityPolicy (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--admission-control"
-          compare:
-            op: has
-            value: "PodSecurityPolicy"
-          set: true
-      remediation: "Follow the documentation and create Pod Security Policy objects as per your environment. 
-              Then, edit the $apiserverconf file on the master node and set the KUBE_ADMISSION_CONTROL 
-              parameter to \"--admission-control=...,PodSecurityPolicy,...\""
-      scored: true
-
-    - id: 1.1.26
-      text: "Ensure that the --service-account-key-file argument is set as appropriate (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--service-account-key-file"
-          set: true
-      remediation: "Edit the $apiserverconf file on the master node and set the KUBE_API_ARGS 
-              parameter to \"--service-account-key-file=<filename>\""
-      scored: true
-
-    - id: 1.1.27
-      text: "Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Scored"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        bin_op: and
-        test_items:
-        - flag: "--etcd-certfile"
-          set: true
-        - flag: "--etcd-keyfile"
-          set: true
-      remediation: "Follow the Kubernetes documentation and set up the TLS connection between the apiserver 
-              and etcd. Then, edit the $apiserverconf file on the master node and set the 
-              KUBE_API_ARGS parameter to include \"--etcd-certfile=<path/to/clientcertificate-file>\" 
-              and \"--etcd-keyfile=<path/to/client-key-file>\""
-      scored: true
- 
-    - id: 1.1.28
-      text: "Ensure that the admission control policy is set to ServiceAccount (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--admission-control"
-          compare:
-            op: has
-            value: "ServiceAccount"
-          set: true
-      remediation: "Follow the documentation and create ServiceAccount objects as per your environment. 
-              Then, edit the $apiserverconf file on the master node and set the 
-              KUBE_ADMISSION_CONTROL parameter to \"--admissioncontrol=...,ServiceAccount,...\""
-      scored: true
-
-    - id: 1.1.29
-      text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        bin_op: and
-        test_items:
-        - flag: "--tls-cert-file"
-          set: true
-        - flag: "--tls-private-key-file"
-          set: true
-      remediation: "Follow the Kubernetes documentation and set up the TLS connection on the apiserver. 
-              Then, edit the $apiserverconf file on the master node and set the KUBE_API_ARGS parameter to 
-              include \"--tls-cert-file=<path/to/tls-certificatefile>\" and 
-              \"--tls-private-key-file=<path/to/tls-key-file>\""
-      scored: true
-
-    - id: 1.1.30
-      text: "Ensure that the --client-ca-file argument is set as appropriate (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--client-ca-file"
-          set: true
-      remediation: "Follow the Kubernetes documentation and set up the TLS connection on the apiserver. 
-              Then, edit the $apiserverconf file on the master node and set the 
-              KUBE_API_ARGS parameter to include \"--client-ca-file=<path/to/client-ca-file>\""
-      scored: true
-
-    - id: 1.1.31
-      text: "Ensure that the --etcd-cafile argument is set as appropriate (Scored)"
-      audit: "ps -ef | grep $apiserverbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--etcd-cafile"
-          set: true
-      remediation: "Follow the Kubernetes documentation and set up the TLS connection between the apiserver 
-              and etcd. Then, edit the $apiserverconf file on the master node and set the 
-              KUBE_API_ARGS parameter to include \"--etcd-cafile=<path/to/ca-file>\""
-      scored: true
-
-- id: 1.2
-  text: "Scheduler"
-  checks:
-    - id: 1.2.1
-      text: "Ensure that the --profiling argument is set to false (Scored)"
-      audit: "ps -ef | grep $schedulerbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--profiling"
-          compare:
-            op: eq
-            value: false
-          set: true
-      remediation: "Edit the $schedulerconf file on the master node and set the KUBE_SCHEDULER_ARGS 
-              parameter to \"--profiling=false\""
-      scored: true
-
-- id: 1.3
-  text: "Controller Manager"
-  checks:
-    - id: 1.3.1
-      text: "Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Scored)"
-      audit: "ps -ef | grep $controllermanagerbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--terminated-pod-gc-threshold"
-            set: true
-      remediation: "Edit the $controllermanagerconf file on the master node and set the 
-              KUBE_CONTROLLER_MANAGER_ARGS parameter to \"--terminated-pod-gcthreshold=<appropriate-number>\""
-      scored: true
- 
-    - id: 1.3.2
-      text: "Ensure that the --profiling argument is set to false (Scored)"
-      audit: "ps -ef | grep $controllermanagerbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--profiling"
-          compare:
-            op: eq
-            value: false
-          set: true
-      remediation: "Edit the $controllermanagerconf file on the master node and set the 
-              KUBE_CONTROLLER_MANAGER_ARGS parameter to \"--profiling=false\""
-      scored: true
-
-    - id: 1.3.3
-      text: "Ensure that the --insecure-experimental-approve-all-kubelet-csrs-for-group argument is not set (Scored)"
-      audit: "ps -ef | grep $controllermanagerbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--insecure-experimental-approve-all-kubelet-csrs-for-group"
-          set: false
-      remediation: "Edit the /etc/kubernetes/controller-manager file on the master node and remove the 
-              --insecure-experimental-approve-all-kubelet-csrs-for-group argument from the
-              KUBE_CONTROLLER_MANAGER_ARGS parameter."
-      scored: true
- 
-    - id: 1.3.4
-      text: "Ensure that the --use-service-account-credentials argument is set"
-      audit: "ps -ef | grep $controllermanagerbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--use-service-account-credentials"
-          compare:
-            op: eq
-            value: true
-          set: true
-      remediation: "Edit the $controllermanagerconf file on the master node and set the 
-              KUBE_CONTROLLER_MANAGER_ARGS parameter to --use-service-account-credentials=true"
-      scored: true
-
-    - id: 1.3.5
-      text: "Ensure that the --service-account-private-key-file argument is set as appropriate (Scored)"
-      audit: "ps -ef | grep $controllermanagerbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--service-account-private-key-file"
-          set: true
-      remediation: "Edit the $controllermanagerconf file on the master node and set the 
-              KUBE_CONTROLLER_MANAGER_ARGS parameter to --service-account-private-keyfile=<filename>"
-      scored: true
-
-    - id: 1.3.6
-      text: "Ensure that the --root-ca-file argument is set as appropriate (Scored)"
-      audit: "ps -ef | grep $controllermanagerbin | grep -v grep"
-      tests:
-        test_items:
-        - flag: "--root-ca-file"
-          set: true
-      remediation: "Edit the $controllermanagerconf file on the master node and set the 
-              KUBE_CONTROLLER_MANAGER_ARGS parameter to include --root-ca-file=<file>"
-      scored: true
- 
-- id: 1.4
-  text: "Configure Files"
-  checks:
-    - id: 1.4.1
-      text: "Ensure that the apiserver file permissions are set to 644 or more restrictive (Scored)"
-      # audit: "/bin/bash -c 'if test -e $apiserverconf; then stat -c %a $apiserverconf; fi'"
-      audit: "/bin/sh -c 'if test -e $apiserverconf; then stat -c %a $apiserverconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-        - flag: "644"
-          compare:
-            op: eq
-            value: "644"
-          set: true
-        - flag: "640"
-          compare:
-            op: eq
-            value: "640"
-          set: true
-        - flag: "600"
-          compare:
-            op: eq
-            value: "600"
-          set: true
-      remediation: "Run the below command (based on the file location on your system) on the master node. 
-              \nFor example, chmod 644 $apiserverconf"
-      scored: true
-
-    - id: 1.4.2
-      text: "Ensure that the apiserver file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $apiserverconf; then stat -c %U:%G $apiserverconf; fi'"
-      tests:
-        test_items:
-        - flag: "root:root"
-          compare:
-            op: eq
-            value: "root:root"
-          set: true
-      remediation: "Run the below command (based on the file location on your system) on the master node. 
-              \nFor example, chown root:root $apiserverconf"
-      scored: true
-
-    - id: 1.4.3
-      text: "Ensure that the config file permissions are set to 644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $config; then stat -c %a $config; fi'"
-      tests:
-        bin_op: or
-        test_items:
-        - flag: "644"
-          compare:
-            op: eq
-            value: "644"
-          set: true
-        - flag: "640"
-          compare:
-            op: eq
-            value: "640"
-          set: true
-        - flag: "600"
-          compare:
-            op: eq
-            value: "600"
-          set: true
-      remediation: "Run the below command (based on the file location on your system) on the master node. 
-              \nFor example, chmod 644 $config"
-      scored: true
-
-    - id: 1.4.4
-      text: "Ensure that the config file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $config; then stat -c %U:%G $config; fi'"
-      tests:
-        test_items:
-        - flag: "root:root"
-          compare:
-            op: eq
-            value: "root:root"
-          set: true
-      remediation: "Run the below command (based on the file location on your system) on the master node. 
-              \nFor example, chown root:root $config"
-      scored: true
-
-    - id: 1.4.5
-      text: "Ensure that the scheduler file permissions are set to 644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $schedulerconf; then stat -c %a $schedulerconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "644"
-            compare:
-              op: eq
-              value: "644"
-            set: true
-          - flag: "640"
-            compare:
-              op: eq
-              value: "640"
-            set: true
-          - flag: "600"
-            compare:
-              op: eq
-              value: "600"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the master node. 
-              \nFor example, chmod 644 $schedulerconf"
-      scored: true
-
-    - id: 1.4.6
-      text: "Ensure that the scheduler file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $schedulerconf; then stat -c %U:%G $schedulerconf; fi'"
-      tests:
-        test_items:
-          - flag: "root:root"
-            compare:
-              op: eq
-              value: "root:root"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the master node. 
-              \nFor example, chown root:root $schedulerconf"
-      scored: true
-
-    - id: 1.4.7
-      text: "Ensure that the etcd.conf file permissions are set to 644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $etcdconf; then stat -c %a $etcdconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "644"
-            compare:
-              op: eq
-              value: "644"
-            set: true
-          - flag: "640"
-            compare:
-              op: eq
-              value: "640"
-            set: true
-          - flag: "600"
-            compare:
-              op: eq
-              value: "600"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the master node. 
-              \nFor example, chmod 644 $etcdconf"
-      scored: true
-
-    - id: 1.4.8
-      text: "Ensure that the etcd.conf file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $etcdconf; then stat -c %U:%G $etcdconf; fi'"
-      tests:
-        test_items:
-        - flag: "root:root"
-          compare:
-            op: eq
-            value: "root:root"
-          set: true
-      remediation: "Run the below command (based on the file location on your system) on the master node. 
-              \nFor example, chown root:root $etcdconf"
-      scored: true
-
-    - id: 1.4.9
-      text: "Ensure that the flanneld file permissions are set to 644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $flanneldconf; then stat -c %a $flanneldconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-        - flag: "644"
-          compare:
-            op: eq
-            value: "644"
-          set: true
-        - flag: "640"
-          compare:
-            op: eq
-            value: "640"
-          set: true
-        - flag: "600"
-          compare:
-            op: eq
-            value: "600"
-          set: true
-      remediation: "Run the below command (based on the file location on your system) on the master node. 
-              \nFor example, chmod 644 $flanneldconf"
-      scored: true
-
-    - id: 1.4.10
-      text: "Ensure that the flanneld file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $flanneldconf; then stat -c %U:%G $flanneldconf; fi'"
-      tests:
-        test_items:
-        - flag: "root:root"
-          compare:
-            op: eq
-            value: "root:root"
-          set: true
-      remediation: "Run the below command (based on the file location on your system) on the master node. 
-              \nFor example, chown root:root $flanneldconf"
-      scored: true
-      
-    - id: 1.4.11
-      text: "Ensure that the etcd data directory permissions are set to 700 or more restrictive (Scored)"
-      audit: ps -ef | grep $etcdbin | grep -v grep | sed 's%.*data-dir[= ]\(\S*\)%\1%' | xargs stat -c %a
-      tests:
-        test_items:
-        - flag: "700"
-          compare:
-            op: eq
-            value: "700"
-          set: true
-      remediation: "On the etcd server node, get the etcd data directory, passed as an argument --data-dir ,
-              from the below command:\n
-              ps -ef | grep $etcdbin\n
-              Run the below command (based on the etcd data directory found above). For example,\n
-              chmod 700 /var/lib/etcd/default.etcd"
-      scored: true
-
-    - id: 1.4.12
-      text: "Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)"
-      audit: ps -ef | grep $etcdbin | grep -v grep | sed 's%.*data-dir[= ]\(\S*\)%\1%' | xargs stat -c %U:%G
-      tests:
-        test_items:
-        - flag: "etcd:etcd"
-          set: true
-      remediation: "On the etcd server node, get the etcd data directory, passed as an argument --data-dir ,
-              from the below command:\n
-                      ps -ef | grep etcd\n
-              Run the below command (based on the etcd data directory found above). For example,\n
-                      chown etcd:etcd /var/lib/etcd/default.etcd"
-      scored: true
-
-- id: 1.5
-  text: "etcd"
-  checks:
-    - id: 1.5.1
-      text: "Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)"
-      audit: "ps -ef | grep $etcdbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--cert-file"
-            set: true
-          - flag:  "--key-file"
-            set: true
-      remediation: "Follow the etcd service documentation and configure TLS encryption."
-      scored: true
-
-    - id: 1.5.2
-      text: "Ensure that the --client-cert-auth argument is set to true (Scored)"
-      audit: "ps -ef | grep $etcdbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--client-cert-auth"
-            compare:
-              op: eq
-              value: true
-            set: true
-      remediation: "Edit the etcd envrironment file (for example, $etcdconf) on the 
-              etcd server node and set the ETCD_CLIENT_CERT_AUTH parameter to \"true\". 
-              Edit the etcd startup file (for example, /etc/systemd/system/multiuser.target.wants/etcd.service) 
-              and configure the startup parameter for --clientcert-auth and set it to \"${ETCD_CLIENT_CERT_AUTH}\""
-      scored: true
-
-    - id: 1.5.3
-      text: "Ensure that the --auto-tls argument is not set to true (Scored)"
-      audit: "ps -ef | grep $etcdbin | grep -v grep"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "--auto-tls"
-            set: false
-          - flag: "--auto-tls"
-            compare:
-              op: neq
-              value: true
-      remediation: "Edit the etcd environment file (for example, $etcdconf) on the etcd server 
-              node and comment out the ETCD_AUTO_TLS parameter. Edit the etcd startup file (for example, 
-              /etc/systemd/system/multiuser.target.wants/etcd.service) and remove the startup parameter 
-              for --auto-tls."
-      scored: true
-
-    - id: 1.5.4
-      text: "Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Scored)"
-      audit: "ps -ef | grep $etcdbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--peer-cert-file"
-            set: true
-          - flag: "--peer-key-file"
-            set: true
-      remediation: "Note: This recommendation is applicable only for etcd clusters. If you are using only 
-              one etcd server in your environment then this recommendation is not applicable. 
-              Follow the etcd service documentation and configure peer TLS encryption as appropriate for 
-              your etcd cluster."
-      scored: true
-
-    - id: 1.5.5
-      text: "Ensure that the --peer-client-cert-auth argument is set to true (Scored)"
-      audit: "ps -ef | grep $etcdbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--peer-client-cert-auth"
-            compare:
-              op: eq
-              value: true
-            set: true
-      remediation: "Note: This recommendation is applicable only for etcd clusters. If you are using only 
-              one etcd server in your environment then this recommendation is not applicable.
-              Edit the etcd environment file (for example, $etcdconf) on the etcd server node 
-              and set the ETCD_PEER_CLIENT_CERT_AUTH parameter to \"true\". Edit the etcd startup file 
-              (for example, /etc/systemd/system/multiuser.target.wants/etcd.service) and configure the 
-              startup parameter for --peer-client-cert-auth and set it to \"${ETCD_PEER_CLIENT_CERT_AUTH}\""
-      scored: true
-
-    - id: 1.5.6
-      text: "Ensure that the --peer-auto-tls argument is not set to true (Scored)"
-      audit: "ps -ef | grep $etcdbin | grep -v grep"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "--peer-auto-tls"
-            set: false
-          - flag: "--peer-auto-tls"
-            compare:
-              op: eq
-              value: false
-            set: true
-      remediation: "Note: This recommendation is applicable only for etcd clusters. 
-              If you are using only one etcd server in your environment then this recommendation is 
-              not applicable. Edit the etcd environment file (for example, $etcdconf) 
-              on the etcd server node and comment out the ETCD_PEER_AUTO_TLS parameter. 
-              Edit the etcd startup file (for example, /etc/systemd/system/multiuser.target.wants/etcd.service) 
-              and remove the startup parameter for --peer-auto-tls."
-      scored: true
-
-    - id: 1.5.7
-      text: "Ensure that the --wal-dir argument is set as appropriate (Scored)"
-      audit: "ps -ef | grep $etcdbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--wal-dir"
-            set: true
-      remediation: "Edit the etcd environment file (for example, $etcdconf) on the etcd server node 
-              and set the ETCD_WAL_DIR parameter as appropriate. Edit the etcd startup file (for example, 
-              /etc/systemd/system/multiuser.target.wants/etcd.service) and configure the startup parameter for 
-              --wal-dir and set it to \"${ETCD_WAL_DIR}\""
-      scored: true
-
-    - id: 1.5.8
-      text: "Ensure that the --max-wals argument is set to 0 (Scored)"
-      audit: "ps -ef | grep $etcdbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--max-wals"
-            compare:
-              op: eq
-              value: 0
-            set: true
-      remediation: "Edit the etcd environment file (for example, $etcdconf) on the etcd server node 
-              and set the ETCD_MAX_WALS parameter to 0. Edit the etcd startup file (for example, 
-              /etc/systemd/system/multiuser.target.wants/etcd.service) and configure the startup parameter 
-              for --max-wals and set it to \"${ETCD_MAX_WALS}\"."
-      scored: true
-
-    - id: 1.5.9
-      text: "Ensure that a unique Certificate Authority is used for etcd (Not Scored)"
-      audit: "ps -ef | grep $etcdbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--trusted-ca-file"
-            set: true
-      remediation: "Follow the etcd documentation and create a dedicated certificate authority setup for the 
-              etcd service."
-      scored: false 
-
-- id: 1.6
-  text: "General Security Primitives"
-  checks:
-    - id: 1.6.1
-      text: "Ensure that the cluster-admin role is only used where required (Not Scored)"
-      type: "manual"
-      remediation: "Remove any unneeded clusterrolebindings: kubectl delete clusterrolebinding [name]"
-      scored: false
-
-    - id: 1.6.2
-      text: "Create Pod Security Policies for your cluster (Not Scored)"
-      type: "manual"
-      remediation: "Follow the documentation and create and enforce Pod Security Policies for your cluster.
-              Additionally, you could refer the \"CIS Security Benchmark for Docker\" and follow the
-              suggested Pod Security Policies for your environment."
-      scored: false
-
-    - id: 1.6.3
-      text: "Create administrative boundaries between resources using namespaces (Not Scored)"
-      type: "manual"
-      remediation: "Follow the documentation and create namespaces for objects in your deployment as you
-              need them."
-      scored: false
-
-    - id: 1.6.4
-      text: "Create network segmentation using Network Policies (Not Scored)"
-      type: "manual"
-      remediation: "Follow the documentation and create NetworkPolicy objects as you need them."
-      scored: false
-
-    - id: 1.6.5
-      text: "Avoid using Kubernetes Secrets (Not Scored)"
-      type: "manual"
-      remediation: "Use other mechanisms such as vaults to manage your cluster secrets."
-      scored: false
-
-
-    - id: 1.6.6
-      text: "Ensure that the seccomp profile is set to docker/default in your pod definitions (Not Scored)"
-      type: "manual"
-      remediation: "Seccomp is an alpha feature currently. By default, all alpha features are disabled. So, you
-              would need to enable alpha features in the apiserver by passing \"--feature-
-              gates=AllAlpha=true\" argument.\n
-              Edit the $apiserverconf file on the master node and set the KUBE_API_ARGS
-              parameter to \"--feature-gates=AllAlpha=true\"
-              KUBE_API_ARGS=\"--feature-gates=AllAlpha=true\""
-      scored: false
-
-    - id: 1.6.7
-      text: "Apply Security Context to Your Pods and Containers (Not Scored)"
-      type: "manual"
-      remediation: "Follow the Kubernetes documentation and apply security contexts to your pods. For a
-              suggested list of security contexts, you may refer to the CIS Security Benchmark for Docker
-              Containers."
-      scored: false
-
-    - id: 1.6.8
-      text: "Configure Image Provenance using ImagePolicyWebhook admission controller (Not Scored)"
-      type: "manual"
-      remediation: "Follow the Kubernetes documentation and setup image provenance."
-      scored: false

cfg/1.6/node.yaml

@@ -1,304 +0,0 @@
----
-controls:
-version: 1.6
-id: 2
-text: "Worker Node Security Configuration"
-type: "node"
-groups:
-- id: 2.1
-  text: "Kubelet"
-  checks:
-    - id: 2.1.1
-      text: "Ensure that the --allow-privileged argument is set to false (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--allow-privileged"
-            compare:
-              op: eq
-              value: false
-            set: true
-      remediation: "Edit the $config file on each node and set the KUBE_ALLOW_PRIV 
-              parameter to \"--allow-privileged=false\""
-      scored: true
-
-    - id: 2.1.2
-      text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--anonymous-auth"
-            compare:
-              op: eq
-              value: false
-            set: true
-      remediation: "Edit the $kubeletconf file on the master node and set the 
-              KUBELET_ARGS parameter to \"--anonymous-auth=false\""
-      scored: true
- 
-    - id: 2.1.3
-      text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--authorization-mode"
-            compare:
-              op: nothave
-              value: "AlwaysAllow"
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the 
-              KUBELET_ARGS parameter to \"--authorization-mode=Webhook\""
-      scored: true
-   
-    - id: 2.1.4
-      text: "Ensure that the --client-ca-file argument is set as appropriate (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--client-ca-file"
-            set: true 
-      remediation: "Follow the Kubernetes documentation and setup the TLS connection between 
-              the apiserver and kubelets. Then, edit the $kubeletconf file on each node 
-              and set the KUBELET_ARGS parameter to \"--client-ca-file=<path/to/client-ca-file>\""
-      scored: true
-
-    - id: 2.1.5
-      text: "Ensure that the --read-only-port argument is set to 0 (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--read-only-port"
-            compare:
-              op: eq
-              value: 0
-            set: true 
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS 
-              parameter to \"--read-only-port=0\""
-      scored: true
- 
-    - id: 2.1.6
-      text: "Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--streaming-connection-idle-timeout"
-            compare:
-              op: gt
-              value: 0
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS 
-              parameter to \"--streaming-connection-idle-timeout=<appropriate-timeout-value>\""
-      scored: true
-
-    - id: 2.1.7
-      text: "Ensure that the --protect-kernel-defaults argument is set to true (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--protect-kernel-defaults"
-            compare:
-              op: eq
-              value: true
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS 
-              parameter to \"--protect-kernel-defaults=true\""
-      scored: true
-
-    - id: 2.1.8
-      text: "Ensure that the --make-iptables-util-chains argument is set to true (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "--make-iptables-util-chains"
-            compare:
-              op: eq
-              value: true
-            set: true
-          - flag: "--make-iptables-util-chains"
-            set: false
-      remediation: "Edit the $kubeletconf file on each node and remove the 
-              --make-iptables-util-chains argument from the KUBELET_ARGS parameter."
-      scored: true
-
-    - id: 2.1.9
-      text: "Ensure that the --keep-terminated-pod-volumes argument is set to false (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--keep-terminated-pod-volumes"
-            compare:
-              op: eq
-              value: false
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS 
-              parameter to \"--keep-terminated-pod-volumes=false\""
-      scored: true
-
-    - id: 2.1.10
-      text: "Ensure that the --hostname-override argument is not set (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--hostname-override"
-            set: false
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_HOSTNAME 
-              parameter to \"\""
-      scored: true
-
-    - id: 2.1.11
-      text: "Ensure that the --event-qps argument is set to 0 (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--event-qps"
-            compare:
-              op: eq
-              value: 0
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS 
-              parameter to \"--event-qps=0\""
-      scored: true
-
-    - id: 2.1.12
-      text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--tls-cert-file"
-            set: true
-          - flag: "--tls-private-key-file"
-            set: true
-      remediation: "Follow the Kubernetes documentation and set up the TLS connection on the Kubelet. 
-              Then, edit the $kubeletconf file on the master node and set the KUBELET_ARGS 
-              parameter to include \"--tls-cert-file=<path/to/tls-certificate-file>\" and 
-              \"--tls-private-key-file=<path/to/tls-key-file>\""
-      scored: true
-
-    - id: 2.1.13
-      text: "Ensure that the --cadvisor-port argument is set to 0 (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--cadvisor-port"
-            compare:
-              op: eq
-              value: 0
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS parameter 
-              to \"--cadvisor-port=0\""
-      scored: true
-
-- id: 2.2
-  text: "Configuration Files"
-  checks:
-    - id: 2.2.1
-      text: "Ensure that the config file permissions are set to 644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $config; then stat -c %a $config; fi'"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "644"
-            compare:
-              op: eq
-              value: "644"
-            set: true
-          - flag: "640"
-            compare:
-              op: eq
-              value: "640"
-            set: true
-          - flag: "600"
-            compare:
-              op: eq
-              value: "600"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chmod 644 $config"
-      scored: true
-
-    - id: 2.2.2
-      text: "Ensure that the config file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $config; then stat -c %U:%G $config; fi'"
-      tests:
-        test_items:
-          - flag: "root:root"
-            compare:
-              op: eq
-              value: root:root
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chown root:root $config"
-      scored: true
-
-    - id: 2.2.3
-      text: "Ensure that the kubelet file permissions are set to 644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %a $kubeletconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "644"
-            compare:
-              op: eq
-              value: 644
-            set: true
-          - flag: "640"
-            compare:
-              op: eq
-              value: "640"
-            set: true
-          - flag: "600"
-            compare:
-              op: eq
-              value: "600"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chmod 644 $kubeletconf"
-      scored: true
-
-    - id: 2.2.4
-      text: "Ensure that the kubelet file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %U:%G $kubeletconf; fi'"
-      tests:
-        test_items:
-          - flag: "root:root"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chown root:root $kubeletconf"
-      scored: true
-
-    - id: 2.2.5
-      text: "Ensure that the proxy file permissions are set to 644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $proxyconf; then stat -c %a $proxyconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "644"
-            compare:
-              op: eq
-              value: "644"
-            set: true
-          - flag: "640"
-            compare:
-              op: eq
-              value: "640"
-            set: true
-          - flag: "600"
-            compare:
-              op: eq
-              value: "600"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chmod 644 $proxyconf"
-      scored: true
-
-    - id: 2.2.6
-      text: "Ensure that the proxy file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $proxyconf; then stat -c %U:%G $proxyconf; fi'"
-      tests:
-        test_items:
-          - flag: "root:root"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chown root:root $proxyconf"
-      scored: true

cfg/1.7/federated.yaml

@@ -1,285 +0,0 @@
----
-controls:
-version: 1.7
-id: 3
-text: "Federated Deployments"
-type: "federated"
-groups:
-- id: 3.1
-  text: "Federation API Server"
-  checks:
-     - id: 3.1.1
-       text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-           - flag: "--anonymous-auth"
-             compare:
-               op: eq
-               value: false
-             set: true
-       remediation: "Edit the deployment specs and set --anonymous-auth=false.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.2
-       text: "Ensure that the --basic-auth-file argument is not set (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-           - flag: "--basic-auth-file"
-             set: false
-       remediation: "Follow the documentation and configure alternate mechanisms for authentication.
-               Then, edit the deployment specs and remove \"--basic-auth-file=<filename>\".\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-  
-     - id: 3.1.3
-       text: "Ensure that the --insecure-allow-any-token argument is not set (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-           - flag: "--insecure-allow-any-token"
-             set: false
-       remediation: "Edit the deployment specs and remove --insecure-allow-any-token.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system" 
-       scored: true
- 
-     - id: 3.1.4
-       text: "Ensure that the --insecure-bind-address argument is not set (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-           - flag: "--insecure-bind-address"
-             set: false
-       remediation: "Edit the deployment specs and remove --insecure-bind-address.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.5
-       text: "Ensure that the --insecure-port argument is set to 0 (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--insecure-port"
-           compare:
-             op: eq
-             value: 0
-           set: true
-       remediation: "Edit the deployment specs and set --insecure-port=0.\n
-              kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.6
-       text: "Ensure that the --secure-port argument is not set to 0 (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests: 
-         bin_op: or
-         test_items:
-           - flag:  "--secure-port"
-             compare:
-               op: gt
-               value: 0
-             set: true
-           - flag: "--secure-port"
-             set: false
-       remediation: "Edit the deployment specs and set the --secure-port argument to the desired port.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.7
-       text: "Ensure that the --profiling argument is set to false (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--profiling"
-           compare:
-             op: eq
-             value: false
-           set: true
-       remediation: "Edit the deployment specs and set \"--profiling=false\".\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       score: true
-
-     - id: 3.1.8
-       text: "Ensure that the admission control policy is not set to AlwaysAdmit (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--admission-control"
-           compare:
-             op: nothave
-             value: AlwaysAdmit 
-           set: true
-       remediation: "Edit the deployment specs and set --admission-control argument to a value that does 
-               not include AlwaysAdmit.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.9
-       text: "Ensure that the admission control policy is set to NamespaceLifecycle (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "admission-control"
-           compare:
-             op: has
-             value: "NamespaceLifecycle"
-           set: true
-       remediation: "Edit the deployment specs and set --admission-control argument to a value that includes NamespaceLifecycle.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.10
-       text: "Ensure that the --audit-log-path argument is set as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--audit-log-path"
-           set: true
-       remediation: "Edit the deployment specs and set --audit-log-path argument as appropriate.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-  
-     - id: 3.1.11
-       text: "Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--audit-log-maxage"
-           compare:
-             op: gte
-             value: 30
-           set: true
-       remediation: "Edit the deployment specs and set --audit-log-maxage to 30 or as appropriate.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-               
-     - id: 3.1.12
-       text: "Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--audit-log-maxbackup"
-           compare:
-             op: gte
-             value: 10
-           set: true
-       remediation: "Edit the deployment specs and set --audit-log-maxbackup to 10 or as appropriate.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.13
-       text: "Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--audit-log-maxsize"
-           compare:
-             op: gte
-             value: 100
-           set: true
-       remediation: "Edit the deployment specs and set --audit-log-maxsize=100 to 100 or as appropriate.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.14
-       text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--authorization-mode"
-           compare:
-             op: nothave
-             value: "AlwaysAllow"
-           set: true
-       remediation: "Edit the deployment specs and set --authorization-mode argument to a value other than AlwaysAllow.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
- 
-     - id: 3.1.15
-       text: "Ensure that the --token-auth-file parameter is not set (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--token-auth-file"
-           set: false
-       remediation: "Follow the documentation and configure alternate mechanisms for authentication.
-               Then, edit the deployment specs and remove the --token-auth-file=<filename> argument.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-  
-     - id: 3.1.16
-       text: "Ensure that the --service-account-lookup argument is set to true (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--service-account-lookup"
-           compare:
-             op: eq
-             value: true
-           set: true
-       remediation: "Edit the deployment specs and set \"--service-account-lookup=true\".\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.17
-       text: "Ensure that the --service-account-key-file argument is set as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--service-account-key-file"
-           set: true
-       remediation: "Edit the deployment specs and set --service-account-key-file argument as appropriate.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.18
-       text: "Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Scored"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         bin_op: and
-         test_items:
-         - flag: "--etcd-certfile"
-           set: true
-         - flag: "--etcd-keyfile"
-           set: true
-       remediation: "Follow the Kubernetes documentation and set up the TLS connection between the
-               federation apiserver and etcd. Then, edit the deployment specs and set \"--etcd-
-               certfile=<path/to/client-certificate-file>\" and \"--etcd-
-               keyfile=<path/to/client-key-file>\" arguments.\n
-               kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-     - id: 3.1.19
-       text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)"
-       audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-       tests:
-         bin_op: and
-         test_items:
-         - flag: "--tls-cert-file"
-           set: true
-         - flag: "--tls-private-key-file"
-           set: true
-       remediation: "Follow the Kubernetes documentation and set up the TLS connection on the federation
-               apiserver. Then, edit the deployment specs and set \"--tls-cert-file=<path/to/tls-
-               certificate-file>\" and \"--tls-private-key-file=<path/to/tls-key-file>\" :
-                       kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-       scored: true
-
-- id: 3.2
-  text: "Federation Controller Manager"
-  checks:
-     - id: 3.2.1
-       text: "Ensure that the --profiling argument is set to false (Scored)"
-       audit: "ps -ef | grep $fedcontrollermanagerbin | grep -v grep"
-       tests:
-         test_items:
-         - flag: "--profiling"
-           compare:
-             op: eq
-             value: false
-           set: true
-       remediation: "Edit the deployment specs and set \"--profiling=false\".\n
-               kubectl edit deployments federation-controller-manager-deployment --namespace=federation-system"
-       scored: true

cfg/1.7/node.yaml

@@ -1,370 +0,0 @@
----
-controls:
-version: 1.7
-id: 2
-text: "Worker Node Security Configuration"
-type: "node"
-groups:
-- id: 2.1
-  text: "Kubelet"
-  checks:
-    - id: 2.1.1
-      text: "Ensure that the --allow-privileged argument is set to false (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--allow-privileged"
-            compare:
-              op: eq
-              value: false
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBE_ALLOW_PRIV 
-              parameter to \"--allow-privileged=false\""
-      scored: true
-
-    - id: 2.1.2
-      text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--anonymous-auth"
-            compare:
-              op: eq
-              value: false
-            set: true
-      remediation: "Edit the $kubeletconf file on the master node and set the 
-              KUBELET_ARGS parameter to \"--anonymous-auth=false\""
-      scored: true
- 
-    - id: 2.1.3
-      text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--authorization-mode"
-            compare:
-              op: nothave
-              value: "AlwaysAllow"
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the 
-              KUBELET_ARGS parameter to \"--authorization-mode=Webhook\""
-      scored: true
-   
-    - id: 2.1.4
-      text: "Ensure that the --client-ca-file argument is set as appropriate (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--client-ca-file"
-            set: true 
-      remediation: "Follow the Kubernetes documentation and setup the TLS connection between 
-              the apiserver and kubelets. Then, edit the $kubeletconf file on each node 
-              and set the KUBELET_ARGS parameter to \"--client-ca-file=<path/to/client-ca-file>\""
-      scored: true
-
-    - id: 2.1.5
-      text: "Ensure that the --read-only-port argument is set to 0 (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--read-only-port"
-            compare:
-              op: eq
-              value: 0
-            set: true 
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS 
-              parameter to \"--read-only-port=0\""
-      scored: true
- 
-    - id: 2.1.6
-      text: "Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "--streaming-connection-idle-timeout"
-            compare:
-              op: noteq
-              value: 0
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS 
-              parameter to \"--streaming-connection-idle-timeout=<appropriate-timeout-value>\""
-      scored: true
-
-    - id: 2.1.7
-      text: "Ensure that the --protect-kernel-defaults argument is set to true (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--protect-kernel-defaults"
-            compare:
-              op: eq
-              value: true
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS 
-              parameter to \"--protect-kernel-defaults=true\""
-      scored: true
-
-    - id: 2.1.8
-      text: "Ensure that the --make-iptables-util-chains argument is set to true (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "--make-iptables-util-chains"
-            compare:
-              op: eq
-              value: true
-            set: true
-          - flag: "--make-iptables-util-chains"
-            set: false
-      remediation: "Edit the $kubeletconf file on each node and remove the 
-              --make-iptables-util-chains argument from the KUBELET_ARGS parameter."
-      scored: true
-
-    - id: 2.1.9
-      text: "Ensure that the --keep-terminated-pod-volumes argument is set to false (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--keep-terminated-pod-volumes"
-            compare:
-              op: eq
-              value: false
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS 
-              parameter to \"--keep-terminated-pod-volumes=false\""
-      scored: true
-
-    - id: 2.1.10
-      text: "Ensure that the --hostname-override argument is not set (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--hostname-override"
-            set: false
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_HOSTNAME 
-              parameter to \"\""
-      scored: true
-
-    - id: 2.1.11
-      text: "Ensure that the --event-qps argument is set to 0 (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--event-qps"
-            compare:
-              op: eq
-              value: 0
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS 
-              parameter to \"--event-qps=0\""
-      scored: true
-
-    - id: 2.1.12
-      text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--tls-cert-file"
-            set: true
-          - flag: "--tls-private-key-file"
-            set: true
-      remediation: "Follow the Kubernetes documentation and set up the TLS connection on the Kubelet. 
-              Then, edit the $kubeletconf file on the master node and set the KUBELET_ARGS 
-              parameter to include \"--tls-cert-file=<path/to/tls-certificate-file>\" and 
-              \"--tls-private-key-file=<path/to/tls-key-file>\""
-      scored: true
-
-    - id: 2.1.13
-      text: "Ensure that the --cadvisor-port argument is set to 0 (Scored)"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "--cadvisor-port"
-            compare:
-              op: eq
-              value: 0
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS parameter 
-              to \"--cadvisor-port=0\""
-      scored: true
-
-    - id: 2.1.14
-      text: "Ensure that the RotateKubeletClientCertificate argument is set to true"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "RotateKubeletClientCertificate"
-            compare:
-              op: eq
-              value: true
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS parameter
-              to a value to include \"--feature-gates=RotateKubeletClientCertificate=true\"."
-      scored: true
-
-    - id: 2.1.15
-      text: "Ensure that the RotateKubeletServerCertificate argument is set to true"
-      audit: "ps -ef | grep $kubeletbin | grep -v grep"
-      tests:
-        test_items:
-          - flag: "RotateKubeletServerCertificate"
-            compare:
-              op: eq
-              value: true
-            set: true
-      remediation: "Edit the $kubeletconf file on each node and set the KUBELET_ARGS parameter
-              to a value to include \"--feature-gates=RotateKubeletServerCertificate=true\"."
-      scored: true
-
-- id: 2.2
-  text: "Configuration Files"
-  checks:
-    - id: 2.2.1
-      text: "Ensure that the config file permissions are set to 644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubernetesconf; then stat -c %a $kubernetesconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "644"
-            compare:
-              op: eq
-              value: "644"
-            set: true
-          - flag: "640"
-            compare:
-              op: eq
-              value: "640"
-            set: true
-          - flag: "600"
-            compare:
-              op: eq
-              value: "600"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chmod 644 $kubernetesconf"
-      scored: true
-
-    - id: 2.2.2
-      text: "Ensure that the config file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubernetesconf; then stat -c %U:%G $kubernetesconf; fi'"
-      tests:
-        test_items:
-          - flag: "root:root"
-            compare:
-              op: eq
-              value: root:root
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chown root:root $kubernetesconf"
-      scored: true
-
-    - id: 2.2.3
-      text: "Ensure that the kubelet file permissions are set to 644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %a $kubeletconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "644"
-            compare:
-              op: eq
-              value: 644
-            set: true
-          - flag: "640"
-            compare:
-              op: eq
-              value: "640"
-            set: true
-          - flag: "600"
-            compare:
-              op: eq
-              value: "600"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chmod 644 $kubeletconf"
-      scored: true
-
-    - id: 2.2.4
-      text: "Ensure that the kubelet file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %U:%G $kubeletconf; fi'"
-      tests:
-        test_items:
-          - flag: "root:root"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chown root:root $kubeletconf"
-      scored: true
-
-    - id: 2.2.5
-      text: "Ensure that the proxy file permissions are set to 644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $proxyconf; then stat -c %a $proxyconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "644"
-            compare:
-              op: eq
-              value: "644"
-            set: true
-          - flag: "640"
-            compare:
-              op: eq
-              value: "640"
-            set: true
-          - flag: "600"
-            compare:
-              op: eq
-              value: "600"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chmod 644 $proxyconf"
-      scored: true
-
-    - id: 2.2.6
-      text: "Ensure that the proxy file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $proxyconf; then stat -c %U:%G $proxyconf; fi'"
-      tests:
-        test_items:
-          - flag: "root:root"
-            set: true
-      remediation: "Run the below command (based on the file location on your system) on the each worker node. 
-              \nFor example, chown root:root $proxyconf"
-      scored: true
-
-    - id: 2.2.7
-      text: "Ensure that the certificate authorities file permissions are set to
-              644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $ca-file; then stat -c %a $ca-file; fi'"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "644"
-            compare:
-              op: eq
-              value: "644"
-            set: true
-          - flag: "640"
-            compare:
-              op: eq
-              value: "640"
-            set: true
-          - flag: "600"
-            compare:
-              op: eq
-              value: "600"
-            set: true
-      remediation: "Run the following command to modify the file permissions of the --client-ca-file
-              \nchmod 644 <filename>"
-      scored: true
-
-    - id: 2.2.8
-      text: "Ensure that the client certificate authorities file ownership is set to root:root"
-      audit: "/bin/sh -c 'if test -e $ca-file; then stat -c %U:%G $ca-file; fi'"
-      tests:
-        test_items:
-          - flag: "notexist:notexist"
-            set: true
-      remediation: "Run the following command to modify the ownership of the --client-ca-file.
-              \nchown root:root <filename>"
-      scored: true

cfg/1.8/config.yaml

@@ -1,38 +0,0 @@
----
-## Controls Files. 
-# These are YAML files that hold all the details for running checks.
-#
-## Uncomment to use different control file paths.
-# masterControls: ./cfg/master.yaml
-# nodeControls: ./cfg/node.yaml
-# federatedControls: ./cfg/federated.yaml
-
-master:
-  apiserver:
-    defaultconf: /etc/kubernetes/manifests/kube-apiserver.yaml
-
-  scheduler:
-    confs: 
-      - /etc/kubernetes/manifests/kube-scheduler.yaml
-    defaultconf: /etc/kubernetes/manifests/kube-scheduler.yaml
-
-  controllermanager:
-    confs:
-      - /etc/kubernetes/manifests/kube-controller-manager.yaml
-    defaultconf: /etc/kubernetes/manifests/kube-controller-manager.yaml
-
-  etcd:
-    confs:
-      - /etc/kubernetes/manifests/etcd.yaml
-    defaultconf: /etc/kubernetes/manifests/etcd.yaml
-
-node:
-  kubelet:
-    confs:
-     - /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
-    defaultconf: /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
-  
-  proxy:
-    confs:
-      - /etc/kubernetes/addons/kube-proxy-daemonset.yaml
-    defaultconf: /etc/kubernetes/addons/kube-proxy-daemonset.yaml

cfg/1.8/federated.yaml

@@ -1,309 +0,0 @@
----
-controls:
-version: 1.8
-id: 3
-text: "Federated Deployments"
-type: "federated"
-groups:
-- id: 3.1
-  text: "Federation API Server"
-  checks:
-  - id: 3.1.1
-    text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--anonymous-auth"
-        compare:
-          op: eq
-          value: false
-        set: true
-    remediation: |
-      Edit the deployment specs and set --anonymous-auth=false.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.2
-    text: "Ensure that the --basic-auth-file argument is not set (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--basic-auth-file"
-        set: false
-    remediation: |
-      Follow the documentation and configure alternate mechanisms for authentication. Then,
-      edit the deployment specs and remove "--basic-auth-file=<filename>".
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.3
-    text: "Ensure that the --insecure-allow-any-token argument is not set (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--insecure-allow-any-token"
-        set: false
-    remediation: |
-      Edit the deployment specs and remove --insecure-allow-any-token.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.4
-    text: "Ensure that the --insecure-bind-address argument is not set (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--insecure-bind-address"
-        set: false
-    remediation: |
-      Edit the deployment specs and remove --insecure-bind-address.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.5
-    text: "Ensure that the --insecure-port argument is set to 0 (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--insecure-port"
-        compare:
-          op: eq
-          value: 0
-        set: true
-    remediation: |
-      Edit the deployment specs and set --insecure-port=0.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.6
-    text: "Ensure that the --secure-port argument is not set to 0 (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests: 
-      bin_op: or
-      test_items:
-      - flag:  "--secure-port"
-        compare:
-          op: gt
-          value: 0
-        set: true
-      - flag: "--secure-port"
-        set: false
-    remediation: |
-      Edit the deployment specs and set the --secure-port argument to the desired port.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.7
-    text: "Ensure that the --profiling argument is set to false (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--profiling"
-        compare:
-          op: eq
-          value: false
-        set: true
-    remediation: |
-      Edit the deployment specs and set "--profiling=false":
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    score: true
-
-  - id: 3.1.8
-    text: "Ensure that the admission control policy is not set to AlwaysAdmit (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--admission-control"
-        compare:
-          op: nothave
-          value: AlwaysAdmit 
-        set: true
-    remediation: |
-      Edit the deployment specs and set --admission-control argument to a value that does not
-      include AlwaysAdmit .
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.9
-    text: "Ensure that the admission control policy is set to NamespaceLifecycle (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "admission-control"
-        compare:
-          op: has
-          value: "NamespaceLifecycle"
-        set: true
-    remediation: |
-      Edit the deployment specs and set --admission-control argument to a value that includes
-      NamespaceLifecycle.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.10
-    text: "Ensure that the --audit-log-path argument is set as appropriate (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--audit-log-path"
-        set: true
-    remediation: "Edit the deployment specs and set --audit-log-path argument as appropriate.\n
-            kubectl edit deployments federation-apiserver-deployment --namespace=federation-system"
-    scored: true
-
-  - id: 3.1.11
-    text: "Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--audit-log-maxage"
-        compare:
-          op: gte
-          value: 30
-        set: true
-    remediation: |
-      Edit the deployment specs and set --audit-log-maxage to 30 or as appropriate.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-            
-  - id: 3.1.12
-    text: "Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--audit-log-maxbackup"
-        compare:
-          op: gte
-          value: 10
-        set: true
-    remediation: |
-      Edit the deployment specs and set --audit-log-maxbackup to 10 or as appropriate.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.13
-    text: "Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--audit-log-maxsize"
-        compare:
-          op: gte
-          value: 100
-        set: true
-    remediation: |
-      Edit the deployment specs and set --audit-log-maxsize=100 to 100 or as appropriate.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.14
-    text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--authorization-mode"
-        compare:
-          op: nothave
-          value: "AlwaysAllow"
-        set: true
-    remediation: |
-      Edit the deployment specs and set --authorization-mode argument to a value other than
-      AlwaysAllow
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.15
-    text: "Ensure that the --token-auth-file parameter is not set (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--token-auth-file"
-        set: false
-    remediation: |
-      Follow the documentation and configure alternate mechanisms for authentication. Then,
-      edit the deployment specs and remove the --token-auth-file=<filename> argument.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.16
-    text: "Ensure that the --service-account-lookup argument is set to true (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--service-account-lookup"
-        compare:
-          op: eq
-          value: true
-        set: true
-    remediation: |
-      Edit the deployment specs and set "--service-account-lookup=true" .
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.17
-    text: "Ensure that the --service-account-key-file argument is set as appropriate (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--service-account-key-file"
-        set: true
-    remediation: |
-      Edit the deployment specs and set --service-account-key-file argument as appropriate.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.18
-    text: "Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as 
-    appropriate (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      bin_op: and
-      test_items:
-      - flag: "--etcd-certfile"
-        set: true
-      - flag: "--etcd-keyfile"
-        set: true
-    remediation: |
-      Follow the Kubernetes documentation and set up the TLS connection between the
-      federation apiserver and etcd. Then, edit the deployment specs and set 
-      "--etcd-certfile=<path/to/client-certificate-file>" and 
-      "--etcd-keyfile=<path/to/client-key-file>" arguments.
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-  - id: 3.1.19
-    text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as 
-    appropriate (Scored)"
-    audit: "ps -ef | grep $fedapiserverbin | grep -v grep"
-    tests:
-      bin_op: and
-      test_items:
-      - flag: "--tls-cert-file"
-        set: true
-      - flag: "--tls-private-key-file"
-        set: true
-    remediation: |
-      Follow the Kubernetes documentation and set up the TLS connection on the federation
-      apiserver. Then, edit the deployment specs and set 
-      "--tls-cert-file=<path/to/tls-certificate-file>" and 
-      "--tls-private-key-file=<path/to/tls-key-file>":
-      kubectl edit deployments federation-apiserver-deployment --namespace=federation-system
-    scored: true
-
-- id: 3.2
-  text: "Federation Controller Manager"
-  checks:
-  - id: 3.2.1
-    text: "Ensure that the --profiling argument is set to false (Scored)"
-    audit: "ps -ef | grep $fedcontrollermanagerbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--profiling"
-        compare:
-          op: eq
-          value: false
-        set: true
-    remediation: |
-      Edit the deployment specs and set "--profiling=false":
-      kubectl edit deployments federation-controller-manager-deployment --namespace=federation-system
-    scored: true

cfg/1.8/node.yaml

@@ -1,440 +0,0 @@
----
-controls:
-version: 1.8
-id: 2
-text: "Worker Node Security Configuration"
-type: "node"
-groups:
-- id: 2.1
-  text: "Kubelet"
-  checks:
-  - id: 2.1.1
-    text: "Ensure that the --allow-privileged argument is set to false (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--allow-privileged"
-        compare:
-          op: eq
-          value: false
-        set: true
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
-      --allow-privileged=false
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.2
-    text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--anonymous-auth"
-        compare:
-          op: eq
-          value: false
-        set: true
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
-      --anonymous-auth=false
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.3
-    text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--authorization-mode"
-        compare:
-          op: nothave
-          value: "AlwaysAllow"
-        set: true
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and set the below parameter in KUBELET_AUTHZ_ARGS variable.
-      --authorization-mode=Webhook
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.4
-    text: "Ensure that the --client-ca-file argument is set as appropriate (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--client-ca-file"
-        set: true 
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and set the below parameter in KUBELET_AUTHZ_ARGS variable.
-      --client-ca-file=<path/to/client-ca-file>
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.5
-    text: "Ensure that the --read-only-port argument is set to 0 (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--read-only-port"
-        compare:
-          op: eq
-          value: 0
-        set: true 
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
-      --read-only-port=0
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.6
-    text: "Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--streaming-connection-idle-timeout"
-        compare:
-          op: noteq
-          value: 0
-        set: true
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
-      --streaming-connection-idle-timeout=5m
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.7
-    text: "Ensure that the --protect-kernel-defaults argument is set to true (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--protect-kernel-defaults"
-        compare:
-          op: eq
-          value: true
-        set: true
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
-      --protect-kernel-defaults=true
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.8
-    text: "Ensure that the --make-iptables-util-chains argument is set to true (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      bin_op: or
-      test_items:
-      - flag: "--make-iptables-util-chains"
-        compare:
-          op: eq
-          value: true
-        set: true
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and remove the --make-iptables-util-chains argument from the
-      KUBELET_SYSTEM_PODS_ARGS variable.
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.9
-    text: "Ensure that the --keep-terminated-pod-volumes argument is set to false (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--keep-terminated-pod-volumes"
-        compare:
-          op: eq
-          value: false
-        set: true
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
-      --keep-terminated-pod-volumes=false
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.10
-    text: "Ensure that the --hostname-override argument is not set (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--hostname-override"
-        set: false
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and remove the --hostname-override argument from the
-      KUBELET_SYSTEM_PODS_ARGS variable.
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.11
-    text: "Ensure that the --event-qps argument is set to 0 (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--event-qps"
-        compare:
-          op: eq
-          value: 0
-        set: true
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
-      --event-qps=0
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.12
-    text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--tls-cert-file"
-        set: true
-      - flag: "--tls-private-key-file"
-        set: true
-    remediation: |
-      Follow the Kubernetes documentation and set up the TLS connection on the Kubelet.
-      Then edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-
-      kubeadm.conf on each worker node and set the below parameters in
-      KUBELET_CERTIFICATE_ARGS variable.
-      --tls-cert-file=<path/to/tls-certificate-file>
-      file=<path/to/tls-key-file>
-      --tls-private-key-
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.13
-    text: "Ensure that the --cadvisor-port argument is set to 0 (Scored)"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "--cadvisor-port"
-        compare:
-          op: eq
-          value: 0
-        set: true
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and set the below parameter in KUBELET_CADVISOR_ARGS variable.
-      --cadvisor-port=0
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.14
-    text: "Ensure that the RotateKubeletClientCertificate argument is set to true"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "RotateKubeletClientCertificate"
-        compare:
-          op: eq
-          value: true
-        set: true
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and remove the --feature-
-      gates=RotateKubeletClientCertificate=false argument from the
-      KUBELET_CERTIFICATE_ARGS variable.
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-  - id: 2.1.15
-    text: "Ensure that the RotateKubeletServerCertificate argument is set to true"
-    audit: "ps -ef | grep $kubeletbin | grep -v grep"
-    tests:
-      test_items:
-      - flag: "RotateKubeletServerCertificate"
-        compare:
-          op: eq
-          value: true
-        set: true
-    remediation: |
-      Edit the kubelet service file $kubeletconf
-      on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
-      --feature-gates=RotateKubeletServerCertificate=true
-      Based on your system, restart the kubelet service. For example:
-      systemctl daemon-reload
-      systemctl restart kubelet.service
-    scored: true
-
-- id: 2.2
-  text: "Configuration Files"
-  checks:
-    - id: 2.2.1
-      text: "Ensure that the kubelet.conf file permissions are set to 644 or
-      more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %a $kubeletconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-          - flag: "644"
-            compare:
-              op: eq
-              value: "644"
-            set: true
-          - flag: "640"
-            compare:
-              op: eq
-              value: "640"
-            set: true
-          - flag: "600"
-            compare:
-              op: eq
-              value: "600"
-            set: true
-      remediation: |
-        Run the below command (based on the file location on your system) on the each worker
-        node. For example,
-        chmod 644 $kubeletconf
-      scored: true
-
-    - id: 2.2.2
-      text: "Ensure that the kubelet.conf file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %U:%G $kubeletconf; fi'"
-      tests:
-        test_items:
-          - flag: "root:root"
-            compare:
-              op: eq
-              value: root:root
-            set: true
-      remediation: |
-        Run the below command (based on the file location on your system) on the each worker
-        node. For example,
-        chown root:root /etc/kubernetes/kubelet.conf
-      scored: true
-
-    - id: 2.2.3
-      text: "Ensure that the kubelet service file permissions are set to 644 or
-      more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %a $kubeletconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-        - flag: "644"
-          compare:
-            op: eq
-            value: 644
-          set: true
-        - flag: "640"
-          compare:
-            op: eq
-            value: "640"
-          set: true
-        - flag: "600"
-          compare:
-            op: eq
-            value: "600"
-          set: true
-      remediation: |
-        Run the below command (based on the file location on your system) on the each worker
-        node. For example,
-        chmod 755 $kubeletconf
-      scored: true
-
-    - id: 2.2.4
-      text: "Ensure that the kubelet service file permissions are set to 644 or
-      more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %U:%G $kubeletconf; fi'"
-      tests:
-        test_items:
-        - flag: "root:root"
-          set: true
-      remediation: |
-        Run the below command (based on the file location on your system) on the each worker
-        node. For example,
-        chown root:root $kubeletconf
-      scored: true
-
-    - id: 2.2.5
-      text: "Ensure that the proxy kubeconfig file permissions are set to 644 or more
-      restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $proxyconf; then stat -c %a $proxyconf; fi'"
-      tests:
-        bin_op: or
-        test_items:
-        - flag: "644"
-          compare:
-            op: eq
-            value: "644"
-          set: true
-        - flag: "640"
-          compare:
-            op: eq
-            value: "640"
-          set: true
-        - flag: "600"
-          compare:
-            op: eq
-            value: "600"
-          set: true
-      remediation: |
-        Run the below command (based on the file location on your system) on the each worker
-        node. For example,
-        chmod 644 $proxyconf
-      scored: true
-
-    - id: 2.2.6
-      text: "Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)"
-      tests:
-        test_items:
-        - flag: "root:root"
-          set: true
-      remediation: |
-          Run the below command (based on the file location on your system) on the each worker
-          node. For example,
-          chown root:root $proxyconf
-      scored: true
-
-    - id: 2.2.7
-      text: "Ensure that the certificate authorities file permissions are set to
-      644 or more restrictive (Scored)"
-      type: manual
-      remediation: |
-        Run the following command to modify the file permissions of the --client-ca-file
-        chmod 644 <filename>
-      scored: true
-
-    - id: 2.2.8
-      text: "Ensure that the client certificate authorities file ownership is set to root:root"
-      audit: "/bin/sh -c 'if test -e $ca-file; then stat -c %U:%G $ca-file; fi'"
-      type: manual
-      remediation: |
-        Run the following command to modify the ownership of the --client-ca-file .
-        chown root:root <filename>
-      scored: true

cfg/cis-1.5/config.yaml

@@ -0,0 +1,2 @@
+---
+## Version-specific settings that override the values in cfg/config.yaml

cfg/cis-1.5/controlplane.yaml

@@ -0,0 +1,35 @@
+---
+controls:
+version: 1.5
+id: 3
+text: "Control Plane Configuration"
+type: "controlplane"
+groups:
+  - id: 3.1
+    text: "Authentication and Authorization"
+    checks:
+      - id: 3.1.1
+        text: "Client certificate authentication should not be used for users (Not Scored)"
+        type: "manual"
+        remediation: |
+          Alternative mechanisms provided by Kubernetes such as the use of OIDC should be
+          implemented in place of client certificates.
+        scored: false
+
+  - id: 3.2
+    text: "Logging"
+    checks:
+      - id: 3.2.1
+        text: "Ensure that a minimal audit policy is created (Scored)"
+        type: "manual"
+        remediation: |
+          Create an audit policy file for your cluster.
+        scored: true
+
+      - id: 3.2.2
+        text: "Ensure that the audit policy covers key security concerns (Not Scored)"
+        type: "manual"
+        remediation: |
+          Consider modification of the audit policy in use on the cluster to include these items, at a
+          minimum.
+        scored: false

cfg/cis-1.5/etcd.yaml

@@ -0,0 +1,131 @@
+---
+controls:
+version: 1.15
+id: 2
+text: "Etcd Node Configuration"
+type: "etcd"
+groups:
+  - id: 2
+    text: "Etcd Node Configuration Files"
+    checks:
+      - id: 2.1
+        text: "Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          bin_op: and
+          test_items:
+            - flag: "--cert-file"
+              set: true
+            - flag: "--key-file"
+              set: true
+        remediation: |
+          Follow the etcd service documentation and configure TLS encryption.
+          Then, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml
+          on the master node and set the below parameters.
+          --cert-file=</path/to/ca-file>
+          --key-file=</path/to/key-file>
+        scored: true
+
+      - id: 2.2
+        text: "Ensure that the --client-cert-auth argument is set to true (Scored)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          test_items:
+            - flag: "--client-cert-auth"
+              compare:
+                op: eq
+                value: true
+              set: true
+        remediation: |
+          Edit the etcd pod specification file $etcdconf on the master
+          node and set the below parameter.
+          --client-cert-auth="true"
+        scored: true
+
+      - id: 2.3
+        text: "Ensure that the --auto-tls argument is not set to true (Scored)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--auto-tls"
+              set: false
+            - flag: "--auto-tls"
+              compare:
+                op: eq
+                value: false
+        remediation: |
+          Edit the etcd pod specification file $etcdconf on the master
+          node and either remove the --auto-tls parameter or set it to false.
+            --auto-tls=false
+        scored: true
+
+      - id: 2.4
+        text: "Ensure that the --peer-cert-file and --peer-key-file arguments are
+        set as appropriate (Scored)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          bin_op: and
+          test_items:
+            - flag: "--peer-cert-file"
+              set: true
+            - flag: "--peer-key-file"
+              set: true
+        remediation: |
+          Follow the etcd service documentation and configure peer TLS encryption as appropriate
+          for your etcd cluster. Then, edit the etcd pod specification file $etcdconf on the
+          master node and set the below parameters.
+          --peer-client-file=</path/to/peer-cert-file>
+          --peer-key-file=</path/to/peer-key-file>
+        scored: true
+
+      - id: 2.5
+        text: "Ensure that the --peer-client-cert-auth argument is set to true (Scored)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          test_items:
+            - flag: "--peer-client-cert-auth"
+              compare:
+                op: eq
+                value: true
+              set: true
+        remediation: |
+          Edit the etcd pod specification file $etcdconf on the master
+          node and set the below parameter.
+          --peer-client-cert-auth=true
+        scored: true
+
+      - id: 2.6
+        text: "Ensure that the --peer-auto-tls argument is not set to true (Scored)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--peer-auto-tls"
+              set: false
+            - flag: "--peer-auto-tls"
+              compare:
+                op: eq
+                value: false
+              set: true
+        remediation: |
+          Edit the etcd pod specification file $etcdconf on the master
+          node and either remove the --peer-auto-tls parameter or set it to false.
+          --peer-auto-tls=false
+        scored: true
+
+      - id: 2.7
+        text: "Ensure that a unique Certificate Authority is used for etcd (Not Scored)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          test_items:
+            - flag: "--trusted-ca-file"
+              set: true
+        remediation: |
+          [Manual test]
+          Follow the etcd documentation and create a dedicated certificate authority setup for the
+          etcd service.
+          Then, edit the etcd pod specification file $etcdconf on the
+          master node and set the below parameter.
+          --trusted-ca-file=</path/to/ca-file>
+        scored: false

cfg/cis-1.5/node.yaml

@@ -0,0 +1,478 @@
+---
+controls:
+version: 1.5
+id: 4
+text: "Worker Node Security Configuration"
+type: "node"
+groups:
+  - id: 4.1
+    text: "Worker Node Configuration Files"
+    checks:
+      - id: 4.1.1
+        text: "Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)"
+        audit: '/bin/sh -c ''if test -e $kubeletsvc; then stat -c permissions=%a $kubeletsvc; fi'' '
+        tests:
+          test_items:
+            - flag: "permissions"
+              set: true
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example,
+          chmod 644 $kubeletsvc
+        scored: true
+
+      - id: 4.1.2
+        text: "Ensure that the kubelet service file ownership is set to root:root (Scored)"
+        audit: '/bin/sh -c ''if test -e $kubeletsvc; then stat -c %U:%G $kubeletsvc; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+              set: true
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example,
+          chown root:root $kubeletsvc
+        scored: true
+
+      - id: 4.1.3
+        text: "Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)"
+        audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c permissions=%a $proxykubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: "permissions"
+              set: true
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example,
+          chmod 644 $proxykubeconfig
+        scored: true
+
+      - id: 4.1.4
+        text: "Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)"
+        audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c %U:%G $proxykubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+              set: true
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example, chown root:root $proxykubeconfig
+        scored: true
+
+      - id: 4.1.5
+        text: "Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored)"
+        audit: '/bin/sh -c ''if test -e $kubeletkubeconfig; then stat -c permissions=%a $kubeletkubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: "permissions"
+              set: true
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example,
+          chmod 644 $kubeletkubeconfig
+        scored: true
+
+      - id: 4.1.6
+        text: "Ensure that the kubelet.conf file ownership is set to root:root (Scored)"
+        audit: '/bin/sh -c ''if test -e $kubeletkubeconfig; then stat -c %U:%G $kubeletkubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+              set: true
+              compare:
+                op: eq
+                value: root:root
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example,
+          chown root:root $kubeletkubeconfig
+        scored: true
+
+      - id: 4.1.7
+        text: "Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)"
+        audit: |
+          CAFILE=$(ps -ef | grep kubelet | grep -v apiserver | grep -- --client-ca-file= | awk -F '--client-ca-file=' '{print $2}' | awk '{print $1}')
+          if test -z $CAFILE; then CAFILE=$kubeletcafile; fi
+          if test -e $CAFILE; then stat -c permissions=%a $CAFILE; fi
+        tests:
+          test_items:
+            - flag: "permissions"
+              set: true
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the following command to modify the file permissions of the
+          --client-ca-file chmod 644 <filename>
+        scored: true
+
+      - id: 4.1.8
+        text: "Ensure that the client certificate authorities file ownership is set to root:root (Scored)"
+        audit: |
+          CAFILE=$(ps -ef | grep kubelet | grep -v apiserver | grep -- --client-ca-file= | awk -F '--client-ca-file=' '{print $2}' | awk '{print $1}')
+          if test -z $CAFILE; then CAFILE=$kubeletcafile; fi
+          if test -e $CAFILE; then stat -c %U:%G $CAFILE; fi
+        tests:
+          test_items:
+            - flag: root:root
+              set: true
+              compare:
+                op: eq
+                value: root:root
+        remediation: |
+          Run the following command to modify the ownership of the --client-ca-file.
+          chown root:root <filename>
+        scored: true
+
+      - id: 4.1.9
+        text: "Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)"
+        audit: '/bin/sh -c ''if test -e $kubeletconf; then stat -c permissions=%a $kubeletconf; fi'' '
+        tests:
+          test_items:
+            - flag: "permissions"
+              set: true
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the following command (using the config file location identified in the Audit step)
+          chmod 644 $kubeletconf
+        scored: true
+
+      - id: 4.1.10
+        text: "Ensure that the kubelet configuration file ownership is set to root:root (Scored)"
+        audit: '/bin/sh -c ''if test -e $kubeletconf; then stat -c %U:%G $kubeletconf; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+              set: true
+        remediation: |
+          Run the following command (using the config file location identified in the Audit step)
+          chown root:root $kubeletconf
+        scored: true
+
+  - id: 4.2
+    text: "Kubelet"
+    checks:
+      - id: 4.2.1
+        text: "Ensure that the anonymous-auth argument is set to false (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: "--anonymous-auth"
+              path: '{.authentication.anonymous.enabled}'
+              set: true
+              compare:
+                op: eq
+                value: false
+        remediation: |
+          If using a Kubelet config file, edit the file to set authentication: anonymous: enabled to
+          false.
+          If using executable arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --anonymous-auth=false
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.2
+        text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --authorization-mode
+              path: '{.authorization.mode}'
+              set: true
+              compare:
+                op: nothave
+                value: AlwaysAllow
+        remediation: |
+          If using a Kubelet config file, edit the file to set authorization: mode to Webhook. If
+          using executable arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_AUTHZ_ARGS variable.
+          --authorization-mode=Webhook
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.3
+        text: "Ensure that the --client-ca-file argument is set as appropriate (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --client-ca-file
+              path: '{.authentication.x509.clientCAFile}'
+              set: true
+        remediation: |
+          If using a Kubelet config file, edit the file to set authentication: x509: clientCAFile to
+          the location of the client CA file.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_AUTHZ_ARGS variable.
+          --client-ca-file=<path/to/client-ca-file>
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.4
+        text: "Ensure that the --read-only-port argument is set to 0 (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--read-only-port"
+              path: '{.readOnlyPort}'
+              set: true
+              compare:
+                op: eq
+                value: 0
+            - flag: "--read-only-port"
+              path: '{.readOnlyPort}'
+              set: false
+        remediation: |
+          If using a Kubelet config file, edit the file to set readOnlyPort to 0.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --read-only-port=0
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.5
+        text: "Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --streaming-connection-idle-timeout
+              path: '{.streamingConnectionIdleTimeout}'
+              set: true
+              compare:
+                op: noteq
+                value: 0
+            - flag: --streaming-connection-idle-timeout
+              path: '{.streamingConnectionIdleTimeout}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to set streamingConnectionIdleTimeout to a
+          value other than 0.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --streaming-connection-idle-timeout=5m
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.6
+        text: "Ensure that the --protect-kernel-defaults argument is set to true (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --protect-kernel-defaults
+              path: '{.protectKernelDefaults}'
+              set: true
+              compare:
+                op: eq
+                value: true
+        remediation: |
+          If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --protect-kernel-defaults=true
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.7
+        text: "Ensure that the --make-iptables-util-chains argument is set to true (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --make-iptables-util-chains
+              path: '{.makeIPTablesUtilChains}'
+              set: true
+              compare:
+                op: eq
+                value: true
+            - flag: --make-iptables-util-chains
+              path: '{.makeIPTablesUtilChains}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to set makeIPTablesUtilChains: true.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          remove the --make-iptables-util-chains argument from the
+          KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.8
+        text: "Ensure that the --hostname-override argument is not set (Not Scored)"
+        # This is one of those properties that can only be set as a command line argument.
+        # To check if the property is set as expected, we need to parse the kubelet command
+        # instead reading the Kubelet Configuration file.
+        audit: "/bin/ps -fC $kubeletbin "
+        tests:
+          test_items:
+            - flag: --hostname-override
+              set: false
+        remediation: |
+          Edit the kubelet service file $kubeletsvc
+          on each worker node and remove the --hostname-override argument from the
+          KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false
+
+      - id: 4.2.9
+        text: "Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture (Not Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --event-qps
+              path: '{.eventRecordQPS}'
+              set: true
+              compare:
+                op: eq
+                value: 0
+        remediation: |
+          If using a Kubelet config file, edit the file to set eventRecordQPS: to an appropriate level.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false
+
+      - id: 4.2.10
+        text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --tls-cert-file
+              path: '{.tlsCertFile}'
+              set: true
+            - flag: --tls-private-key-file
+              path: '{.tlsPrivateKeyFile}'
+              set: true
+        remediation: |
+          If using a Kubelet config file, edit the file to set tlsCertFile to the location
+          of the certificate file to use to identify this Kubelet, and tlsPrivateKeyFile
+          to the location of the corresponding private key file.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameters in KUBELET_CERTIFICATE_ARGS variable.
+          --tls-cert-file=<path/to/tls-certificate-file>
+          --tls-private-key-file=<path/to/tls-key-file>
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.11
+        text: "Ensure that the --rotate-certificates argument is not set to false (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --rotate-certificates
+              path: '{.rotateCertificates}'
+              set: true
+              compare:
+                op: eq
+                value: true
+            - flag: --rotate-certificates
+              path: '{.rotateCertificates}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to add the line rotateCertificates: true or
+          remove it altogether to use the default value.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          remove --rotate-certificates=false argument from the KUBELET_CERTIFICATE_ARGS
+          variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.12
+        text: "Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: RotateKubeletServerCertificate
+              path: '{.featureGates.RotateKubeletServerCertificate}'
+              set: true
+              compare:
+                op: eq
+                value: true
+        remediation: |
+          Edit the kubelet service file $kubeletsvc
+          on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
+          --feature-gates=RotateKubeletServerCertificate=true
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.13
+        text: "Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --tls-cipher-suites
+              path: '{range .tlsCipherSuites[:]}{}{'',''}{end}'
+              set: true
+              compare:
+                op: valid_elements
+                value: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+        remediation: |
+          If using a Kubelet config file, edit the file to set TLSCipherSuites: to
+          TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+          or to a subset of these values.
+          If using executable arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the --tls-cipher-suites parameter as follows, or to a subset of these values.
+          --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false

cfg/cis-1.5/policies.yaml

@@ -0,0 +1,239 @@
+---
+controls:
+version: 1.5
+id: 5
+text: "Kubernetes Policies"
+type: "policies"
+groups:
+  - id: 5.1
+    text: "RBAC and Service Accounts"
+    checks:
+      - id: 5.1.1
+        text: "Ensure that the cluster-admin role is only used where required (Not Scored)"
+        type: "manual"
+        remediation: |
+          Identify all clusterrolebindings to the cluster-admin role. Check if they are used and
+          if they need this role or if they could use a role with fewer privileges.
+          Where possible, first bind users to a lower privileged role and then remove the
+          clusterrolebinding to the cluster-admin role :
+          kubectl delete clusterrolebinding [name]
+        scored: false
+
+      - id: 5.1.2
+        text: "Minimize access to secrets (Not Scored)"
+        type: "manual"
+        remediation: |
+          Where possible, remove get, list and watch access to secret objects in the cluster.
+        scored: false
+
+      - id: 5.1.3
+        text: "Minimize wildcard use in Roles and ClusterRoles (Not Scored)"
+        type: "manual"
+        remediation: |
+          Where possible replace any use of wildcards in clusterroles and roles with specific
+          objects or actions.
+        scored: false
+
+      - id: 5.1.4
+        text: "Minimize access to create pods (Not Scored)"
+        type: "manual"
+        remediation: |
+          Where possible, remove create access to pod objects in the cluster.
+        scored: false
+
+      - id: 5.1.5
+        text: "Ensure that default service accounts are not actively used. (Scored)"
+        type: "manual"
+        remediation: |
+          Create explicit service accounts wherever a Kubernetes workload requires specific access
+          to the Kubernetes API server.
+          Modify the configuration of each default service account to include this value
+          automountServiceAccountToken: false
+        scored: true
+
+      - id: 5.1.6
+        text: "Ensure that Service Account Tokens are only mounted where necessary (Not Scored)"
+        type: "manual"
+        remediation: |
+          Modify the definition of pods and service accounts which do not need to mount service
+          account tokens to disable it.
+        scored: false
+
+  - id: 5.2
+    text: "Pod Security Policies"
+    checks:
+      - id: 5.2.1
+        text: "Minimize the admission of privileged containers (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that
+          the .spec.privileged field is omitted or set to false.
+        scored: false
+
+      - id: 5.2.2
+        text: "Minimize the admission of containers wishing to share the host process ID namespace (Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostPID field is omitted or set to false.
+        scored: true
+
+      - id: 5.2.3
+        text: "Minimize the admission of containers wishing to share the host IPC namespace (Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostIPC field is omitted or set to false.
+        scored: true
+
+      - id: 5.2.4
+        text: "Minimize the admission of containers wishing to share the host network namespace (Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostNetwork field is omitted or set to false.
+        scored: true
+
+      - id: 5.2.5
+        text: "Minimize the admission of containers with allowPrivilegeEscalation (Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.allowPrivilegeEscalation field is omitted or set to false.
+        scored: true
+
+      - id: 5.2.6
+        text: "Minimize the admission of root containers (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.runAsUser.rule is set to either MustRunAsNonRoot or MustRunAs with the range of
+          UIDs not including 0.
+        scored: false
+
+      - id: 5.2.7
+        text: "Minimize the admission of containers with the NET_RAW capability (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.requiredDropCapabilities is set to include either NET_RAW or ALL.
+        scored: false
+
+      - id: 5.2.8
+        text: "Minimize the admission of containers with added capabilities (Not Scored)"
+        type: "manual"
+        remediation: |
+          Ensure that allowedCapabilities is not present in PSPs for the cluster unless
+          it is set to an empty array.
+        scored: false
+
+      - id: 5.2.9
+        text: "Minimize the admission of containers with capabilities assigned (Not Scored)"
+        type: "manual"
+        remediation: |
+          Review the use of capabilites in applications runnning on your cluster. Where a namespace
+          contains applicaions which do not require any Linux capabities to operate consider adding
+          a PSP which forbids the admission of containers which do not drop all capabilities.
+        scored: false
+
+  - id: 5.3
+    text: "Network Policies and CNI"
+    checks:
+      - id: 5.3.1
+        text: "Ensure that the CNI in use supports Network Policies (Not Scored)"
+        type: "manual"
+        remediation: |
+          If the CNI plugin in use does not support network policies, consideration should be given to
+          making use of a different plugin, or finding an alternate mechanism for restricting traffic
+          in the Kubernetes cluster.
+        scored: false
+
+      - id: 5.3.2
+        text: "Ensure that all Namespaces have Network Policies defined (Scored)"
+        type: "manual"
+        remediation: |
+          Follow the documentation and create NetworkPolicy objects as you need them.
+        scored: true
+
+  - id: 5.4
+    text: "Secrets Management"
+    checks:
+      - id: 5.4.1
+        text: "Prefer using secrets as files over secrets as environment variables (Not Scored)"
+        type: "manual"
+        remediation: |
+          if possible, rewrite application code to read secrets from mounted secret files, rather than
+          from environment variables.
+        scored: false
+
+      - id: 5.4.2
+        text: "Consider external secret storage (Not Scored)"
+        type: "manual"
+        remediation: |
+          Refer to the secrets management options offered by your cloud provider or a third-party
+          secrets management solution.
+        scored: false
+
+  - id: 5.5
+    text: "Extensible Admission Control"
+    checks:
+      - id: 5.5.1
+        text: "Configure Image Provenance using ImagePolicyWebhook admission controller (Not Scored)"
+        type: "manual"
+        remediation: |
+          Follow the Kubernetes documentation and setup image provenance.
+        scored: false
+
+  - id: 5.7
+    text: "General Policies"
+    checks:
+      - id: 5.7.1
+        text: "Create administrative boundaries between resources using namespaces (Not Scored)"
+        type: "manual"
+        remediation: |
+          Follow the documentation and create namespaces for objects in your deployment as you need
+          them.
+        scored: false
+
+      - id: 5.7.2
+        text: "Ensure that the seccomp profile is set to docker/default in your pod definitions (Not Scored)"
+        type: "manual"
+        remediation: |
+          Seccomp is an alpha feature currently. By default, all alpha features are disabled. So, you
+          would need to enable alpha features in the apiserver by passing "--feature-
+          gates=AllAlpha=true" argument.
+          Edit the /etc/kubernetes/apiserver file on the master node and set the KUBE_API_ARGS
+          parameter to "--feature-gates=AllAlpha=true"
+          KUBE_API_ARGS="--feature-gates=AllAlpha=true"
+          Based on your system, restart the kube-apiserver service. For example:
+          systemctl restart kube-apiserver.service
+          Use annotations to enable the docker/default seccomp profile in your pod definitions. An
+          example is as below:
+          apiVersion: v1
+          kind: Pod
+          metadata:
+            name: trustworthy-pod
+            annotations:
+              seccomp.security.alpha.kubernetes.io/pod: docker/default
+          spec:
+            containers:
+              - name: trustworthy-container
+                image: sotrustworthy:latest
+        scored: false
+
+      - id: 5.7.3
+        text: "Apply Security Context to Your Pods and Containers (Not Scored)"
+        type: "manual"
+        remediation: |
+          Follow the Kubernetes documentation and apply security contexts to your pods. For a
+          suggested list of security contexts, you may refer to the CIS Security Benchmark for Docker
+          Containers.
+        scored: false
+
+      - id: 5.7.4
+        text: "The default namespace should not be used (Scored)"
+        type: "manual"
+        remediation: |
+          Ensure that namespaces are created to allow for appropriate segregation of Kubernetes
+          resources and that all new resources are created in a specific namespace.
+        scored: true

cfg/cis-1.6/config.yaml

@@ -0,0 +1,2 @@
+---
+## Version-specific settings that override the values in cfg/config.yaml

cfg/cis-1.6/controlplane.yaml

@@ -0,0 +1,35 @@
+---
+controls:
+version: 1.6
+id: 3
+text: "Control Plane Configuration"
+type: "controlplane"
+groups:
+  - id: 3.1
+    text: "Authentication and Authorization"
+    checks:
+      - id: 3.1.1
+        text: "Client certificate authentication should not be used for users (Manual)"
+        type: "manual"
+        remediation: |
+          Alternative mechanisms provided by Kubernetes such as the use of OIDC should be
+          implemented in place of client certificates.
+        scored: false
+
+  - id: 3.2
+    text: "Logging"
+    checks:
+      - id: 3.2.1
+        text: "Ensure that a minimal audit policy is created (Manual)"
+        type: "manual"
+        remediation: |
+          Create an audit policy file for your cluster.
+        scored: false
+
+      - id: 3.2.2
+        text: "Ensure that the audit policy covers key security concerns (Manual)"
+        type: "manual"
+        remediation: |
+          Consider modification of the audit policy in use on the cluster to include these items, at a
+          minimum.
+        scored: false

cfg/cis-1.6/etcd.yaml

@@ -0,0 +1,124 @@
+---
+controls:
+version: 1.6
+id: 2
+text: "Etcd Node Configuration"
+type: "etcd"
+groups:
+  - id: 2
+    text: "Etcd Node Configuration Files"
+    checks:
+      - id: 2.1
+        text: "Ensure that the --cert-file and --key-file arguments are set as appropriate (Automated)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          bin_op: and
+          test_items:
+            - flag: "--cert-file"
+            - flag: "--key-file"
+        remediation: |
+          Follow the etcd service documentation and configure TLS encryption.
+          Then, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml
+          on the master node and set the below parameters.
+          --cert-file=</path/to/ca-file>
+          --key-file=</path/to/key-file>
+        scored: true
+
+      - id: 2.2
+        text: "Ensure that the --client-cert-auth argument is set to true (Automated)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          test_items:
+            - flag: "--client-cert-auth"
+              compare:
+                op: eq
+                value: true
+        remediation: |
+          Edit the etcd pod specification file $etcdconf on the master
+          node and set the below parameter.
+          --client-cert-auth="true"
+        scored: true
+
+      - id: 2.3
+        text: "Ensure that the --auto-tls argument is not set to true (Automated)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--auto-tls"
+              set: false
+            - flag: "--auto-tls"
+              compare:
+                op: eq
+                value: false
+        remediation: |
+          Edit the etcd pod specification file $etcdconf on the master
+          node and either remove the --auto-tls parameter or set it to false.
+            --auto-tls=false
+        scored: true
+
+      - id: 2.4
+        text: "Ensure that the --peer-cert-file and --peer-key-file arguments are
+        set as appropriate (Automated)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          bin_op: and
+          test_items:
+            - flag: "--peer-cert-file"
+            - flag: "--peer-key-file"
+        remediation: |
+          Follow the etcd service documentation and configure peer TLS encryption as appropriate
+          for your etcd cluster.
+          Then, edit the etcd pod specification file $etcdconf on the
+          master node and set the below parameters.
+          --peer-client-file=</path/to/peer-cert-file>
+          --peer-key-file=</path/to/peer-key-file>
+        scored: true
+
+      - id: 2.5
+        text: "Ensure that the --peer-client-cert-auth argument is set to true (Automated)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          test_items:
+            - flag: "--peer-client-cert-auth"
+              compare:
+                op: eq
+                value: true
+        remediation: |
+          Edit the etcd pod specification file $etcdconf on the master
+          node and set the below parameter.
+          --peer-client-cert-auth=true
+        scored: true
+
+      - id: 2.6
+        text: "Ensure that the --peer-auto-tls argument is not set to true (Automated)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--peer-auto-tls"
+              set: false
+            - flag: "--peer-auto-tls"
+              compare:
+                op: eq
+                value: false
+        remediation: |
+          Edit the etcd pod specification file $etcdconf on the master
+          node and either remove the --peer-auto-tls parameter or set it to false.
+          --peer-auto-tls=false
+        scored: true
+
+      - id: 2.7
+        text: "Ensure that a unique Certificate Authority is used for etcd (Manual)"
+        audit: "/bin/ps -ef | /bin/grep $etcdbin | /bin/grep -v grep"
+        tests:
+          test_items:
+            - flag: "--trusted-ca-file"
+        remediation: |
+          [Manual test]
+          Follow the etcd documentation and create a dedicated certificate authority setup for the
+          etcd service.
+          Then, edit the etcd pod specification file $etcdconf on the
+          master node and set the below parameter.
+          --trusted-ca-file=</path/to/ca-file>
+        scored: false

cfg/cis-1.6/master.yaml

@@ -0,0 +1,982 @@
+---
+controls:
+version: 1.6
+id: 1
+text: "Master Node Security Configuration"
+type: "master"
+groups:
+  - id: 1.1
+    text: "Master Node Configuration Files"
+    checks:
+      - id: 1.1.1
+        text: "Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Automated)"
+        audit: "/bin/sh -c 'if test -e $apiserverconf; then stat -c permissions=%a $apiserverconf; fi'"
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the
+          master node.
+          For example, chmod 644 $apiserverconf
+        scored: true
+
+      - id: 1.1.2
+        text: "Ensure that the API server pod specification file ownership is set to root:root (Automated)"
+        audit: "/bin/sh -c 'if test -e $apiserverconf; then stat -c %U:%G $apiserverconf; fi'"
+        tests:
+          test_items:
+            - flag: "root:root"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chown root:root $apiserverconf
+        scored: true
+
+      - id: 1.1.3
+        text: "Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Automated)"
+        audit: "/bin/sh -c 'if test -e $controllermanagerconf; then stat -c permissions=%a $controllermanagerconf; fi'"
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chmod 644 $controllermanagerconf
+        scored: true
+
+      - id: 1.1.4
+        text: "Ensure that the controller manager pod specification file ownership is set to root:root (Automated)"
+        audit: "/bin/sh -c 'if test -e $controllermanagerconf; then stat -c %U:%G $controllermanagerconf; fi'"
+        tests:
+          test_items:
+            - flag: "root:root"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chown root:root $controllermanagerconf
+        scored: true
+
+      - id: 1.1.5
+        text: "Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Automated)"
+        audit: "/bin/sh -c 'if test -e $schedulerconf; then stat -c permissions=%a $schedulerconf; fi'"
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chmod 644 $schedulerconf
+        scored: true
+
+      - id: 1.1.6
+        text: "Ensure that the scheduler pod specification file ownership is set to root:root (Automated)"
+        audit: "/bin/sh -c 'if test -e $schedulerconf; then stat -c %U:%G $schedulerconf; fi'"
+        tests:
+          test_items:
+            - flag: "root:root"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chown root:root $schedulerconf
+        scored: true
+
+      - id: 1.1.7
+        text: "Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Automated)"
+        audit: "/bin/sh -c 'if test -e $etcdconf; then stat -c permissions=%a $etcdconf; fi'"
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chmod 644 $etcdconf
+        scored: true
+
+      - id: 1.1.8
+        text: "Ensure that the etcd pod specification file ownership is set to root:root (Automated)"
+        audit: "/bin/sh -c 'if test -e $etcdconf; then stat -c %U:%G $etcdconf; fi'"
+        tests:
+          test_items:
+            - flag: "root:root"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chown root:root $etcdconf
+        scored: true
+
+      - id: 1.1.9
+        text: "Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Manual)"
+        audit: "stat -c permissions=%a <path/to/cni/files>"
+        type: "manual"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chmod 644 <path/to/cni/files>
+        scored: false
+
+      - id: 1.1.10
+        text: "Ensure that the Container Network Interface file ownership is set to root:root (Manual)"
+        audit: "stat -c %U:%G <path/to/cni/files>"
+        type: "manual"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chown root:root <path/to/cni/files>
+        scored: false
+
+      - id: 1.1.11
+        text: "Ensure that the etcd data directory permissions are set to 700 or more restrictive (Automated)"
+        audit: ps -ef | grep $etcdbin | grep -- --data-dir | sed 's%.*data-dir[= ]\([^ ]*\).*%\1%' | xargs stat -c permissions=%a
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "700"
+        remediation: |
+          On the etcd server node, get the etcd data directory, passed as an argument --data-dir,
+          from the below command:
+          ps -ef | grep etcd
+          Run the below command (based on the etcd data directory found above). For example,
+          chmod 700 /var/lib/etcd
+        scored: true
+
+      - id: 1.1.12
+        text: "Ensure that the etcd data directory ownership is set to etcd:etcd (Automated)"
+        audit: ps -ef | grep $etcdbin | grep -- --data-dir | sed 's%.*data-dir[= ]\([^ ]*\).*%\1%' | xargs stat -c %U:%G
+        tests:
+          test_items:
+            - flag: "etcd:etcd"
+        remediation: |
+          On the etcd server node, get the etcd data directory, passed as an argument --data-dir,
+          from the below command:
+          ps -ef | grep etcd
+          Run the below command (based on the etcd data directory found above).
+          For example, chown etcd:etcd /var/lib/etcd
+        scored: true
+
+      - id: 1.1.13
+        text: "Ensure that the admin.conf file permissions are set to 644 or more restrictive (Automated)"
+        audit: "/bin/sh -c 'if test -e /etc/kubernetes/admin.conf; then stat -c permissions=%a /etc/kubernetes/admin.conf; fi'"
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chmod 644 /etc/kubernetes/admin.conf
+        scored: true
+
+      - id: 1.1.14
+        text: "Ensure that the admin.conf file ownership is set to root:root (Automated)"
+        audit: "/bin/sh -c 'if test -e /etc/kubernetes/admin.conf; then stat -c %U:%G /etc/kubernetes/admin.conf; fi'"
+        tests:
+          test_items:
+            - flag: "root:root"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chown root:root /etc/kubernetes/admin.conf
+        scored: true
+
+      - id: 1.1.15
+        text: "Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Automated)"
+        audit: "/bin/sh -c 'if test -e /etc/kubernetes/scheduler.conf; then stat -c permissions=%a /etc/kubernetes/scheduler.conf; fi'"
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chmod 644 /etc/kubernetes/scheduler.conf
+        scored: true
+
+      - id: 1.1.16
+        text: "Ensure that the scheduler.conf file ownership is set to root:root (Automated)"
+        audit: "/bin/sh -c 'if test -e /etc/kubernetes/scheduler.conf; then stat -c %U:%G /etc/kubernetes/scheduler.conf; fi'"
+        tests:
+          test_items:
+            - flag: "root:root"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chown root:root /etc/kubernetes/scheduler.conf
+        scored: true
+
+      - id: 1.1.17
+        text: "Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive (Automated)"
+        audit: "/bin/sh -c 'if test -e /etc/kubernetes/controller-manager.conf; then stat -c permissions=%a /etc/kubernetes/controller-manager.conf; fi'"
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chmod 644 /etc/kubernetes/controller-manager.conf
+        scored: true
+
+      - id: 1.1.18
+        text: "Ensure that the controller-manager.conf file ownership is set to root:root (Automated)"
+        audit: "/bin/sh -c 'if test -e /etc/kubernetes/controller-manager.conf; then stat -c %U:%G /etc/kubernetes/controller-manager.conf; fi'"
+        tests:
+          test_items:
+            - flag: "root:root"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chown root:root /etc/kubernetes/controller-manager.conf
+        scored: true
+
+      - id: 1.1.19
+        text: "Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Automated)"
+        audit: "find /etc/kubernetes/pki/ | xargs stat -c %U:%G"
+        use_multiple_values: true
+        tests:
+          test_items:
+            - flag: "root root"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chown -R root:root /etc/kubernetes/pki/
+        scored: true
+
+      - id: 1.1.20
+        text: "Ensure that the Kubernetes PKI certificate file permissions are set to 644 or more restrictive (Manual)"
+        audit: "find /etc/kubernetes/pki -name '*.crt' | xargs stat -c permissions=%a"
+        use_multiple_values: true
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chmod -R 644 /etc/kubernetes/pki/*.crt
+        scored: false
+
+      - id: 1.1.21
+        text: "Ensure that the Kubernetes PKI key file permissions are set to 600 (Manual)"
+        audit: "find /etc/kubernetes/pki -name '*.key' | xargs stat -c permissions=%a"
+        use_multiple_values: true
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "600"
+        remediation: |
+          Run the below command (based on the file location on your system) on the master node.
+          For example,
+          chmod -R 600 /etc/kubernetes/pki/*.key
+        scored: false
+
+  - id: 1.2
+    text: "API Server"
+    checks:
+      - id: 1.2.1
+        text: "Ensure that the --anonymous-auth argument is set to false (Manual)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        type: manual
+        tests:
+          test_items:
+            - flag: "--anonymous-auth"
+              compare:
+                op: eq
+                value: false
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the below parameter.
+          --anonymous-auth=false
+        scored: false
+
+      - id: 1.2.2
+        text: "Ensure that the --basic-auth-file argument is not set (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--basic-auth-file"
+              set: false
+        remediation: |
+          Follow the documentation and configure alternate mechanisms for authentication. Then,
+          edit the API server pod specification file $apiserverconf
+          on the master node and remove the --basic-auth-file=<filename> parameter.
+        scored: true
+
+      - id: 1.2.3
+        text: "Ensure that the --token-auth-file parameter is not set (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--token-auth-file"
+              set: false
+        remediation: |
+          Follow the documentation and configure alternate mechanisms for authentication. Then,
+          edit the API server pod specification file $apiserverconf
+          on the master node and remove the --token-auth-file=<filename> parameter.
+        scored: true
+
+      - id: 1.2.4
+        text: "Ensure that the --kubelet-https argument is set to true (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--kubelet-https"
+              compare:
+                op: eq
+                value: true
+            - flag: "--kubelet-https"
+              set: false
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and remove the --kubelet-https parameter.
+        scored: true
+
+      - id: 1.2.5
+        text: "Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          bin_op: and
+          test_items:
+            - flag: "--kubelet-client-certificate"
+            - flag: "--kubelet-client-key"
+        remediation: |
+          Follow the Kubernetes documentation and set up the TLS connection between the
+          apiserver and kubelets. Then, edit API server pod specification file
+          $apiserverconf on the master node and set the
+          kubelet client certificate and key parameters as below.
+          --kubelet-client-certificate=<path/to/client-certificate-file>
+          --kubelet-client-key=<path/to/client-key-file>
+        scored: true
+
+      - id: 1.2.6
+        text: "Ensure that the --kubelet-certificate-authority argument is set as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--kubelet-certificate-authority"
+        remediation: |
+          Follow the Kubernetes documentation and setup the TLS connection between
+          the apiserver and kubelets. Then, edit the API server pod specification file
+          $apiserverconf on the master node and set the
+          --kubelet-certificate-authority parameter to the path to the cert file for the certificate authority.
+          --kubelet-certificate-authority=<ca-string>
+        scored: true
+
+      - id: 1.2.7
+        text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--authorization-mode"
+              compare:
+                op: nothave
+                value: "AlwaysAllow"
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the --authorization-mode parameter to values other than AlwaysAllow.
+          One such example could be as below.
+          --authorization-mode=RBAC
+        scored: true
+
+      - id: 1.2.8
+        text: "Ensure that the --authorization-mode argument includes Node (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--authorization-mode"
+              compare:
+                op: has
+                value: "Node"
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the --authorization-mode parameter to a value that includes Node.
+          --authorization-mode=Node,RBAC
+        scored: true
+
+      - id: 1.2.9
+        text: "Ensure that the --authorization-mode argument includes RBAC (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--authorization-mode"
+              compare:
+                op: has
+                value: "RBAC"
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the --authorization-mode parameter to a value that includes RBAC,
+          for example:
+          --authorization-mode=Node,RBAC
+        scored: true
+
+      - id: 1.2.10
+        text: "Ensure that the admission control plugin EventRateLimit is set (Manual)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--enable-admission-plugins"
+              compare:
+                op: has
+                value: "EventRateLimit"
+        remediation: |
+          Follow the Kubernetes documentation and set the desired limits in a configuration file.
+          Then, edit the API server pod specification file $apiserverconf
+          and set the below parameters.
+          --enable-admission-plugins=...,EventRateLimit,...
+          --admission-control-config-file=<path/to/configuration/file>
+        scored: false
+
+      - id: 1.2.11
+        text: "Ensure that the admission control plugin AlwaysAdmit is not set (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--enable-admission-plugins"
+              compare:
+                op: nothave
+                value: AlwaysAdmit
+            - flag: "--enable-admission-plugins"
+              set: false
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and either remove the --enable-admission-plugins parameter, or set it to a
+          value that does not include AlwaysAdmit.
+        scored: true
+
+      - id: 1.2.12
+        text: "Ensure that the admission control plugin AlwaysPullImages is set (Manual)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--enable-admission-plugins"
+              compare:
+                op: has
+                value: "AlwaysPullImages"
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the --enable-admission-plugins parameter to include
+          AlwaysPullImages.
+          --enable-admission-plugins=...,AlwaysPullImages,...
+        scored: false
+
+      - id: 1.2.13
+        text: "Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--enable-admission-plugins"
+              compare:
+                op: has
+                value: "SecurityContextDeny"
+            - flag: "--enable-admission-plugins"
+              compare:
+                op: has
+                value: "PodSecurityPolicy"
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the --enable-admission-plugins parameter to include
+          SecurityContextDeny, unless PodSecurityPolicy is already in place.
+          --enable-admission-plugins=...,SecurityContextDeny,...
+        scored: false
+
+      - id: 1.2.14
+        text: "Ensure that the admission control plugin ServiceAccount is set (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--disable-admission-plugins"
+              compare:
+                op: nothave
+                value: "ServiceAccount"
+            - flag: "--disable-admission-plugins"
+              set: false
+        remediation: |
+          Follow the documentation and create ServiceAccount objects as per your environment.
+          Then, edit the API server pod specification file $apiserverconf
+          on the master node and ensure that the --disable-admission-plugins parameter is set to a
+          value that does not include ServiceAccount.
+        scored: true
+
+      - id: 1.2.15
+        text: "Ensure that the admission control plugin NamespaceLifecycle is set (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--disable-admission-plugins"
+              compare:
+                op: nothave
+                value: "NamespaceLifecycle"
+            - flag: "--disable-admission-plugins"
+              set: false
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the --disable-admission-plugins parameter to
+          ensure it does not include NamespaceLifecycle.
+        scored: true
+
+      - id: 1.2.16
+        text: "Ensure that the admission control plugin PodSecurityPolicy is set (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--enable-admission-plugins"
+              compare:
+                op: has
+                value: "PodSecurityPolicy"
+        remediation: |
+          Follow the documentation and create Pod Security Policy objects as per your environment.
+          Then, edit the API server pod specification file $apiserverconf
+          on the master node and set the --enable-admission-plugins parameter to a
+          value that includes PodSecurityPolicy:
+          --enable-admission-plugins=...,PodSecurityPolicy,...
+          Then restart the API Server.
+        scored: true
+
+      - id: 1.2.17
+        text: "Ensure that the admission control plugin NodeRestriction is set (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--enable-admission-plugins"
+              compare:
+                op: has
+                value: "NodeRestriction"
+        remediation: |
+          Follow the Kubernetes documentation and configure NodeRestriction plug-in on kubelets.
+          Then, edit the API server pod specification file $apiserverconf
+          on the master node and set the --enable-admission-plugins parameter to a
+          value that includes NodeRestriction.
+          --enable-admission-plugins=...,NodeRestriction,...
+        scored: true
+
+      - id: 1.2.18
+        text: "Ensure that the --insecure-bind-address argument is not set (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--insecure-bind-address"
+              set: false
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and remove the --insecure-bind-address parameter.
+        scored: true
+
+      - id: 1.2.19
+        text: "Ensure that the --insecure-port argument is set to 0 (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--insecure-port"
+              compare:
+                op: eq
+                value: 0
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the below parameter.
+          --insecure-port=0
+        scored: true
+
+      - id: 1.2.20
+        text: "Ensure that the --secure-port argument is not set to 0 (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--secure-port"
+              compare:
+                op: gt
+                value: 0
+            - flag: "--secure-port"
+              set: false
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and either remove the --secure-port parameter or
+          set it to a different (non-zero) desired port.
+        scored: true
+
+      - id: 1.2.21
+        text: "Ensure that the --profiling argument is set to false (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--profiling"
+              compare:
+                op: eq
+                value: false
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the below parameter.
+          --profiling=false
+        scored: true
+
+      - id: 1.2.22
+        text: "Ensure that the --audit-log-path argument is set (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--audit-log-path"
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the --audit-log-path parameter to a suitable path and
+          file where you would like audit logs to be written, for example:
+          --audit-log-path=/var/log/apiserver/audit.log
+        scored: true
+
+      - id: 1.2.23
+        text: "Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--audit-log-maxage"
+              compare:
+                op: gte
+                value: 30
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the --audit-log-maxage parameter to 30 or as an appropriate number of days:
+          --audit-log-maxage=30
+        scored: true
+
+      - id: 1.2.24
+        text: "Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--audit-log-maxbackup"
+              compare:
+                op: gte
+                value: 10
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the --audit-log-maxbackup parameter to 10 or to an appropriate
+          value.
+          --audit-log-maxbackup=10
+        scored: true
+
+      - id: 1.2.25
+        text: "Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--audit-log-maxsize"
+              compare:
+                op: gte
+                value: 100
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the --audit-log-maxsize parameter to an appropriate size in MB.
+          For example, to set it as 100 MB:
+          --audit-log-maxsize=100
+        scored: true
+
+      - id: 1.2.26
+        text: "Ensure that the --request-timeout argument is set as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--request-timeout"
+              set: false
+            - flag: "--request-timeout"
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          and set the below parameter as appropriate and if needed.
+          For example,
+          --request-timeout=300s
+        scored: true
+
+      - id: 1.2.27
+        text: "Ensure that the --service-account-lookup argument is set to true (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--service-account-lookup"
+              set: false
+            - flag: "--service-account-lookup"
+              compare:
+                op: eq
+                value: true
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the below parameter.
+          --service-account-lookup=true
+          Alternatively, you can delete the --service-account-lookup parameter from this file so
+          that the default takes effect.
+        scored: true
+
+      - id: 1.2.28
+        text: "Ensure that the --service-account-key-file argument is set as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--service-account-key-file"
+        remediation: |
+          Edit the API server pod specification file $apiserverconf
+          on the master node and set the --service-account-key-file parameter
+          to the public key file for service accounts:
+          --service-account-key-file=<filename>
+        scored: true
+
+      - id: 1.2.29
+        text: "Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          bin_op: and
+          test_items:
+            - flag: "--etcd-certfile"
+            - flag: "--etcd-keyfile"
+        remediation: |
+          Follow the Kubernetes documentation and set up the TLS connection between the apiserver and etcd.
+          Then, edit the API server pod specification file $apiserverconf
+          on the master node and set the etcd certificate and key file parameters.
+          --etcd-certfile=<path/to/client-certificate-file>
+          --etcd-keyfile=<path/to/client-key-file>
+        scored: true
+
+      - id: 1.2.30
+        text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          bin_op: and
+          test_items:
+            - flag: "--tls-cert-file"
+            - flag: "--tls-private-key-file"
+        remediation: |
+          Follow the Kubernetes documentation and set up the TLS connection on the apiserver.
+          Then, edit the API server pod specification file $apiserverconf
+          on the master node and set the TLS certificate and private key file parameters.
+          --tls-cert-file=<path/to/tls-certificate-file>
+          --tls-private-key-file=<path/to/tls-key-file>
+        scored: true
+
+      - id: 1.2.31
+        text: "Ensure that the --client-ca-file argument is set as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--client-ca-file"
+        remediation: |
+          Follow the Kubernetes documentation and set up the TLS connection on the apiserver.
+          Then, edit the API server pod specification file $apiserverconf
+          on the master node and set the client certificate authority file.
+          --client-ca-file=<path/to/client-ca-file>
+        scored: true
+
+      - id: 1.2.32
+        text: "Ensure that the --etcd-cafile argument is set as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--etcd-cafile"
+        remediation: |
+          Follow the Kubernetes documentation and set up the TLS connection between the apiserver and etcd.
+          Then, edit the API server pod specification file $apiserverconf
+          on the master node and set the etcd certificate authority file parameter.
+          --etcd-cafile=<path/to/ca-file>
+        scored: true
+
+      - id: 1.2.33
+        text: "Ensure that the --encryption-provider-config argument is set as appropriate (Manual)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--encryption-provider-config"
+        remediation: |
+          Follow the Kubernetes documentation and configure a EncryptionConfig file.
+          Then, edit the API server pod specification file $apiserverconf
+          on the master node and set the --encryption-provider-config parameter to the path of that file: --encryption-provider-config=</path/to/EncryptionConfig/File>
+        scored: false
+
+      - id: 1.2.34
+        text: "Ensure that encryption providers are appropriately configured (Manual)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        type: "manual"
+        remediation: |
+          Follow the Kubernetes documentation and configure a EncryptionConfig file.
+          In this file, choose aescbc, kms or secretbox as the encryption provider.
+        scored: false
+
+      - id: 1.2.35
+        text: "Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Manual)"
+        audit: "/bin/ps -ef | grep $apiserverbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--tls-cipher-suites"
+              compare:
+                op: has
+                value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256"
+        remediation: |
+          Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+          on the master node and set the below parameter.
+          --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM
+          _SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM
+          _SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM
+          _SHA384
+        scored: false
+
+  - id: 1.3
+    text: "Controller Manager"
+    checks:
+      - id: 1.3.1
+        text: "Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Manual)"
+        audit: "/bin/ps -ef | grep $controllermanagerbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--terminated-pod-gc-threshold"
+        remediation: |
+          Edit the Controller Manager pod specification file $controllermanagerconf
+          on the master node and set the --terminated-pod-gc-threshold to an appropriate threshold,
+          for example:
+          --terminated-pod-gc-threshold=10
+        scored: false
+
+      - id: 1.3.2
+        text: "Ensure that the --profiling argument is set to false (Automated)"
+        audit: "/bin/ps -ef | grep $controllermanagerbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--profiling"
+              compare:
+                op: eq
+                value: false
+        remediation: |
+          Edit the Controller Manager pod specification file $controllermanagerconf
+          on the master node and set the below parameter.
+          --profiling=false
+        scored: true
+
+      - id: 1.3.3
+        text: "Ensure that the --use-service-account-credentials argument is set to true (Automated)"
+        audit: "/bin/ps -ef | grep $controllermanagerbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--use-service-account-credentials"
+              compare:
+                op: noteq
+                value: false
+        remediation: |
+          Edit the Controller Manager pod specification file $controllermanagerconf
+          on the master node to set the below parameter.
+          --use-service-account-credentials=true
+        scored: true
+
+      - id: 1.3.4
+        text: "Ensure that the --service-account-private-key-file argument is set as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $controllermanagerbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--service-account-private-key-file"
+        remediation: |
+          Edit the Controller Manager pod specification file $controllermanagerconf
+          on the master node and set the --service-account-private-key-file parameter
+          to the private key file for service accounts.
+          --service-account-private-key-file=<filename>
+        scored: true
+
+      - id: 1.3.5
+        text: "Ensure that the --root-ca-file argument is set as appropriate (Automated)"
+        audit: "/bin/ps -ef | grep $controllermanagerbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--root-ca-file"
+        remediation: |
+          Edit the Controller Manager pod specification file $controllermanagerconf
+          on the master node and set the --root-ca-file parameter to the certificate bundle file`.
+          --root-ca-file=<path/to/file>
+        scored: true
+
+      - id: 1.3.6
+        text: "Ensure that the RotateKubeletServerCertificate argument is set to true (Automated)"
+        audit: "/bin/ps -ef | grep $controllermanagerbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--feature-gates"
+              compare:
+                op: eq
+                value: "RotateKubeletServerCertificate=true"
+        remediation: |
+          Edit the Controller Manager pod specification file $controllermanagerconf
+          on the master node and set the --feature-gates parameter to include RotateKubeletServerCertificate=true.
+          --feature-gates=RotateKubeletServerCertificate=true
+        scored: true
+
+      - id: 1.3.7
+        text: "Ensure that the --bind-address argument is set to 127.0.0.1 (Automated)"
+        audit: "/bin/ps -ef | grep $controllermanagerbin | grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--bind-address"
+              compare:
+                op: eq
+                value: "127.0.0.1"
+            - flag: "--bind-address"
+              set: false
+        remediation: |
+          Edit the Controller Manager pod specification file $controllermanagerconf
+          on the master node and ensure the correct value for the --bind-address parameter
+        scored: true
+
+  - id: 1.4
+    text: "Scheduler"
+    checks:
+      - id: 1.4.1
+        text: "Ensure that the --profiling argument is set to false (Automated)"
+        audit: "/bin/ps -ef | grep $schedulerbin | grep -v grep"
+        tests:
+          test_items:
+            - flag: "--profiling"
+              compare:
+                op: eq
+                value: false
+        remediation: |
+          Edit the Scheduler pod specification file $schedulerconf file
+          on the master node and set the below parameter.
+          --profiling=false
+        scored: true
+
+      - id: 1.4.2
+        text: "Ensure that the --bind-address argument is set to 127.0.0.1 (Automated)"
+        audit: "/bin/ps -ef | grep $schedulerbin | grep -v grep"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--bind-address"
+              compare:
+                op: eq
+                value: "127.0.0.1"
+            - flag: "--bind-address"
+              set: false
+        remediation: |
+          Edit the Scheduler pod specification file $schedulerconf
+          on the master node and ensure the correct value for the --bind-address parameter
+        scored: true

cfg/cis-1.6/node.yaml

@@ -0,0 +1,452 @@
+---
+controls:
+version: 1.6
+id: 4
+text: "Worker Node Security Configuration"
+type: "node"
+groups:
+  - id: 4.1
+    text: "Worker Node Configuration Files"
+    checks:
+      - id: 4.1.1
+        text: "Ensure that the kubelet service file permissions are set to 644 or more restrictive (Automated)"
+        audit: '/bin/sh -c ''if test -e $kubeletsvc; then stat -c permissions=%a $kubeletsvc; fi'' '
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example,
+          chmod 644 $kubeletsvc
+        scored: true
+
+      - id: 4.1.2
+        text: "Ensure that the kubelet service file ownership is set to root:root (Automated)"
+        audit: '/bin/sh -c ''if test -e $kubeletsvc; then stat -c %U:%G $kubeletsvc; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example,
+          chown root:root $kubeletsvc
+        scored: true
+
+      - id: 4.1.3
+        text: "If proxy kubeconfig file exists ensure permissions are set to 644 or more restrictive (Manual)"
+        audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c permissions=%a $proxykubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example,
+          chmod 644 $proxykubeconfig
+        scored: false
+
+      - id: 4.1.4
+        text: "Ensure that the proxy kubeconfig file ownership is set to root:root (Manual)"
+        audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c %U:%G $proxykubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example, chown root:root $proxykubeconfig
+        scored: false
+
+      - id: 4.1.5
+        text: "Ensure that the --kubeconfig kubelet.conf file permissions are set to 644 or more restrictive (Automated)"
+        audit: '/bin/sh -c ''if test -e $kubeletkubeconfig; then stat -c permissions=%a $kubeletkubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example,
+          chmod 644 $kubeletkubeconfig
+        scored: true
+
+      - id: 4.1.6
+        text: "Ensure that the --kubeconfig kubelet.conf file ownership is set to root:root (Manual)"
+        audit: '/bin/sh -c ''if test -e $kubeletkubeconfig; then stat -c %U:%G $kubeletkubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+        remediation: |
+          Run the below command (based on the file location on your system) on the each worker node.
+          For example,
+          chown root:root $kubeletkubeconfig
+        scored: false
+
+      - id: 4.1.7
+        text: "Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Manual)"
+        audit: |
+          CAFILE=$(ps -ef | grep kubelet | grep -v apiserver | grep -- --client-ca-file= | awk -F '--client-ca-file=' '{print $2}' | awk '{print $1}')
+          if test -z $CAFILE; then CAFILE=$kubeletcafile; fi
+          if test -e $CAFILE; then stat -c permissions=%a $CAFILE; fi
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the following command to modify the file permissions of the
+          --client-ca-file chmod 644 <filename>
+        scored: false
+
+      - id: 4.1.8
+        text: "Ensure that the client certificate authorities file ownership is set to root:root (Manual)"
+        audit: |
+          CAFILE=$(ps -ef | grep kubelet | grep -v apiserver | grep -- --client-ca-file= | awk -F '--client-ca-file=' '{print $2}' | awk '{print $1}')
+          if test -z $CAFILE; then CAFILE=$kubeletcafile; fi
+          if test -e $CAFILE; then stat -c %U:%G $CAFILE; fi
+        tests:
+          test_items:
+            - flag: root:root
+              compare:
+                op: eq
+                value: root:root
+        remediation: |
+          Run the following command to modify the ownership of the --client-ca-file.
+          chown root:root <filename>
+        scored: false
+
+      - id: 4.1.9
+        text: "Ensure that the kubelet --config configuration file has permissions set to 644 or more restrictive (Automated)"
+        audit: '/bin/sh -c ''if test -e $kubeletconf; then stat -c permissions=%a $kubeletconf; fi'' '
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+        remediation: |
+          Run the following command (using the config file location identified in the Audit step)
+          chmod 644 $kubeletconf
+        scored: true
+
+      - id: 4.1.10
+        text: "Ensure that the kubelet --config configuration file ownership is set to root:root (Automated)"
+        audit: '/bin/sh -c ''if test -e $kubeletconf; then stat -c %U:%G $kubeletconf; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+        remediation: |
+          Run the following command (using the config file location identified in the Audit step)
+          chown root:root $kubeletconf
+        scored: true
+
+  - id: 4.2
+    text: "Kubelet"
+    checks:
+      - id: 4.2.1
+        text: "Ensure that the anonymous-auth argument is set to false (Automated)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: "--anonymous-auth"
+              path: '{.authentication.anonymous.enabled}'
+              compare:
+                op: eq
+                value: false
+        remediation: |
+          If using a Kubelet config file, edit the file to set authentication: anonymous: enabled to
+          false.
+          If using executable arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --anonymous-auth=false
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.2
+        text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --authorization-mode
+              path: '{.authorization.mode}'
+              compare:
+                op: nothave
+                value: AlwaysAllow
+        remediation: |
+          If using a Kubelet config file, edit the file to set authorization: mode to Webhook. If
+          using executable arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_AUTHZ_ARGS variable.
+          --authorization-mode=Webhook
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.3
+        text: "Ensure that the --client-ca-file argument is set as appropriate (Automated)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --client-ca-file
+              path: '{.authentication.x509.clientCAFile}'
+        remediation: |
+          If using a Kubelet config file, edit the file to set authentication: x509: clientCAFile to
+          the location of the client CA file.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_AUTHZ_ARGS variable.
+          --client-ca-file=<path/to/client-ca-file>
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.4
+        text: "Ensure that the --read-only-port argument is set to 0 (Manual)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "--read-only-port"
+              path: '{.readOnlyPort}'
+              compare:
+                op: eq
+                value: 0
+            - flag: "--read-only-port"
+              path: '{.readOnlyPort}'
+              set: false
+        remediation: |
+          If using a Kubelet config file, edit the file to set readOnlyPort to 0.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --read-only-port=0
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false
+
+      - id: 4.2.5
+        text: "Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Manual)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --streaming-connection-idle-timeout
+              path: '{.streamingConnectionIdleTimeout}'
+              compare:
+                op: noteq
+                value: 0
+            - flag: --streaming-connection-idle-timeout
+              path: '{.streamingConnectionIdleTimeout}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to set streamingConnectionIdleTimeout to a
+          value other than 0.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --streaming-connection-idle-timeout=5m
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false
+
+      - id: 4.2.6
+        text: "Ensure that the --protect-kernel-defaults argument is set to true (Automated)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --protect-kernel-defaults
+              path: '{.protectKernelDefaults}'
+              compare:
+                op: eq
+                value: true
+        remediation: |
+          If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --protect-kernel-defaults=true
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.7
+        text: "Ensure that the --make-iptables-util-chains argument is set to true (Automated)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --make-iptables-util-chains
+              path: '{.makeIPTablesUtilChains}'
+              compare:
+                op: eq
+                value: true
+            - flag: --make-iptables-util-chains
+              path: '{.makeIPTablesUtilChains}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to set makeIPTablesUtilChains: true.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          remove the --make-iptables-util-chains argument from the
+          KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.8
+        text: "Ensure that the --hostname-override argument is not set (Manual)"
+        # This is one of those properties that can only be set as a command line argument.
+        # To check if the property is set as expected, we need to parse the kubelet command
+        # instead reading the Kubelet Configuration file.
+        audit: "/bin/ps -fC $kubeletbin "
+        tests:
+          test_items:
+            - flag: --hostname-override
+              set: false
+        remediation: |
+          Edit the kubelet service file $kubeletsvc
+          on each worker node and remove the --hostname-override argument from the
+          KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false
+
+      - id: 4.2.9
+        text: "Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture (Manual)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --event-qps
+              path: '{.eventRecordQPS}'
+              compare:
+                op: eq
+                value: 0
+        remediation: |
+          If using a Kubelet config file, edit the file to set eventRecordQPS: to an appropriate level.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false
+
+      - id: 4.2.10
+        text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --tls-cert-file
+              path: '{.tlsCertFile}'
+            - flag: --tls-private-key-file
+              path: '{.tlsPrivateKeyFile}'
+        remediation: |
+          If using a Kubelet config file, edit the file to set tlsCertFile to the location
+          of the certificate file to use to identify this Kubelet, and tlsPrivateKeyFile
+          to the location of the corresponding private key file.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameters in KUBELET_CERTIFICATE_ARGS variable.
+          --tls-cert-file=<path/to/tls-certificate-file>
+          --tls-private-key-file=<path/to/tls-key-file>
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false
+
+      - id: 4.2.11
+        text: "Ensure that the --rotate-certificates argument is not set to false (Manual)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --rotate-certificates
+              path: '{.rotateCertificates}'
+              compare:
+                op: eq
+                value: true
+            - flag: --rotate-certificates
+              path: '{.rotateCertificates}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to add the line rotateCertificates: true or
+          remove it altogether to use the default value.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          remove --rotate-certificates=false argument from the KUBELET_CERTIFICATE_ARGS
+          variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false
+
+      - id: 4.2.12
+        text: "Verify that the RotateKubeletServerCertificate argument is set to true (Manual)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: RotateKubeletServerCertificate
+              path: '{.featureGates.RotateKubeletServerCertificate}'
+              compare:
+                op: eq
+                value: true
+        remediation: |
+          Edit the kubelet service file $kubeletsvc
+          on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
+          --feature-gates=RotateKubeletServerCertificate=true
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false
+
+      - id: 4.2.13
+        text: "Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Manual)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --tls-cipher-suites
+              path: '{range .tlsCipherSuites[:]}{}{'',''}{end}'
+              compare:
+                op: valid_elements
+                value: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+        remediation: |
+          If using a Kubelet config file, edit the file to set TLSCipherSuites: to
+          TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+          or to a subset of these values.
+          If using executable arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the --tls-cipher-suites parameter as follows, or to a subset of these values.
+          --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false

cfg/cis-1.6/policies.yaml

@@ -0,0 +1,239 @@
+---
+controls:
+version: 1.6
+id: 5
+text: "Kubernetes Policies"
+type: "policies"
+groups:
+  - id: 5.1
+    text: "RBAC and Service Accounts"
+    checks:
+      - id: 5.1.1
+        text: "Ensure that the cluster-admin role is only used where required (Manual)"
+        type: "manual"
+        remediation: |
+          Identify all clusterrolebindings to the cluster-admin role. Check if they are used and
+          if they need this role or if they could use a role with fewer privileges.
+          Where possible, first bind users to a lower privileged role and then remove the
+          clusterrolebinding to the cluster-admin role :
+          kubectl delete clusterrolebinding [name]
+        scored: false
+
+      - id: 5.1.2
+        text: "Minimize access to secrets (Manual)"
+        type: "manual"
+        remediation: |
+          Where possible, remove get, list and watch access to secret objects in the cluster.
+        scored: false
+
+      - id: 5.1.3
+        text: "Minimize wildcard use in Roles and ClusterRoles (Manual)"
+        type: "manual"
+        remediation: |
+          Where possible replace any use of wildcards in clusterroles and roles with specific
+          objects or actions.
+        scored: false
+
+      - id: 5.1.4
+        text: "Minimize access to create pods (Manual)"
+        type: "manual"
+        remediation: |
+          Where possible, remove create access to pod objects in the cluster.
+        scored: false
+
+      - id: 5.1.5
+        text: "Ensure that default service accounts are not actively used. (Manual)"
+        type: "manual"
+        remediation: |
+          Create explicit service accounts wherever a Kubernetes workload requires specific access
+          to the Kubernetes API server.
+          Modify the configuration of each default service account to include this value
+          automountServiceAccountToken: false
+        scored: false
+
+      - id: 5.1.6
+        text: "Ensure that Service Account Tokens are only mounted where necessary (Manual)"
+        type: "manual"
+        remediation: |
+          Modify the definition of pods and service accounts which do not need to mount service
+          account tokens to disable it.
+        scored: false
+
+  - id: 5.2
+    text: "Pod Security Policies"
+    checks:
+      - id: 5.2.1
+        text: "Minimize the admission of privileged containers (Manual)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that
+          the .spec.privileged field is omitted or set to false.
+        scored: false
+
+      - id: 5.2.2
+        text: "Minimize the admission of containers wishing to share the host process ID namespace (Manual)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostPID field is omitted or set to false.
+        scored: false
+
+      - id: 5.2.3
+        text: "Minimize the admission of containers wishing to share the host IPC namespace (Manual)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostIPC field is omitted or set to false.
+        scored: false
+
+      - id: 5.2.4
+        text: "Minimize the admission of containers wishing to share the host network namespace (Manual)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostNetwork field is omitted or set to false.
+        scored: false
+
+      - id: 5.2.5
+        text: "Minimize the admission of containers with allowPrivilegeEscalation (Manual)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.allowPrivilegeEscalation field is omitted or set to false.
+        scored: false
+
+      - id: 5.2.6
+        text: "Minimize the admission of root containers (Manual)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.runAsUser.rule is set to either MustRunAsNonRoot or MustRunAs with the range of
+          UIDs not including 0.
+        scored: false
+
+      - id: 5.2.7
+        text: "Minimize the admission of containers with the NET_RAW capability (Manual)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.requiredDropCapabilities is set to include either NET_RAW or ALL.
+        scored: false
+
+      - id: 5.2.8
+        text: "Minimize the admission of containers with added capabilities (Manual)"
+        type: "manual"
+        remediation: |
+          Ensure that allowedCapabilities is not present in PSPs for the cluster unless
+          it is set to an empty array.
+        scored: false
+
+      - id: 5.2.9
+        text: "Minimize the admission of containers with capabilities assigned (Manual)"
+        type: "manual"
+        remediation: |
+          Review the use of capabilites in applications runnning on your cluster. Where a namespace
+          contains applicaions which do not require any Linux capabities to operate consider adding
+          a PSP which forbids the admission of containers which do not drop all capabilities.
+        scored: false
+
+  - id: 5.3
+    text: "Network Policies and CNI"
+    checks:
+      - id: 5.3.1
+        text: "Ensure that the CNI in use supports Network Policies (Manual)"
+        type: "manual"
+        remediation: |
+          If the CNI plugin in use does not support network policies, consideration should be given to
+          making use of a different plugin, or finding an alternate mechanism for restricting traffic
+          in the Kubernetes cluster.
+        scored: false
+
+      - id: 5.3.2
+        text: "Ensure that all Namespaces have Network Policies defined (Manual)"
+        type: "manual"
+        remediation: |
+          Follow the documentation and create NetworkPolicy objects as you need them.
+        scored: false
+
+  - id: 5.4
+    text: "Secrets Management"
+    checks:
+      - id: 5.4.1
+        text: "Prefer using secrets as files over secrets as environment variables (Manual)"
+        type: "manual"
+        remediation: |
+          if possible, rewrite application code to read secrets from mounted secret files, rather than
+          from environment variables.
+        scored: false
+
+      - id: 5.4.2
+        text: "Consider external secret storage (Manual)"
+        type: "manual"
+        remediation: |
+          Refer to the secrets management options offered by your cloud provider or a third-party
+          secrets management solution.
+        scored: false
+
+  - id: 5.5
+    text: "Extensible Admission Control"
+    checks:
+      - id: 5.5.1
+        text: "Configure Image Provenance using ImagePolicyWebhook admission controller (Manual)"
+        type: "manual"
+        remediation: |
+          Follow the Kubernetes documentation and setup image provenance.
+        scored: false
+
+  - id: 5.7
+    text: "General Policies"
+    checks:
+      - id: 5.7.1
+        text: "Create administrative boundaries between resources using namespaces (Manual)"
+        type: "manual"
+        remediation: |
+          Follow the documentation and create namespaces for objects in your deployment as you need
+          them.
+        scored: false
+
+      - id: 5.7.2
+        text: "Ensure that the seccomp profile is set to docker/default in your pod definitions (Manual)"
+        type: "manual"
+        remediation: |
+          Seccomp is an alpha feature currently. By default, all alpha features are disabled. So, you
+          would need to enable alpha features in the apiserver by passing "--feature-
+          gates=AllAlpha=true" argument.
+          Edit the /etc/kubernetes/apiserver file on the master node and set the KUBE_API_ARGS
+          parameter to "--feature-gates=AllAlpha=true"
+          KUBE_API_ARGS="--feature-gates=AllAlpha=true"
+          Based on your system, restart the kube-apiserver service. For example:
+          systemctl restart kube-apiserver.service
+          Use annotations to enable the docker/default seccomp profile in your pod definitions. An
+          example is as below:
+          apiVersion: v1
+          kind: Pod
+          metadata:
+            name: trustworthy-pod
+            annotations:
+              seccomp.security.alpha.kubernetes.io/pod: docker/default
+          spec:
+            containers:
+              - name: trustworthy-container
+                image: sotrustworthy:latest
+        scored: false
+
+      - id: 5.7.3
+        text: "Apply Security Context to Your Pods and Containers (Manual)"
+        type: "manual"
+        remediation: |
+          Follow the Kubernetes documentation and apply security contexts to your pods. For a
+          suggested list of security contexts, you may refer to the CIS Security Benchmark for Docker
+          Containers.
+        scored: false
+
+      - id: 5.7.4
+        text: "The default namespace should not be used (Manual)"
+        type: "manual"
+        remediation: |
+          Ensure that namespaces are created to allow for appropriate segregation of Kubernetes
+          resources and that all new resources are created in a specific namespace.
+        scored: false

cfg/config.yaml

@@ -1,18 +1,17 @@
 ---
-## Controls Files. 
+## Controls Files.
 # These are YAML files that hold all the details for running checks.
 #
 ## Uncomment to use different control file paths.
 # masterControls: ./cfg/master.yaml
 # nodeControls: ./cfg/node.yaml
-# federatedControls: ./cfg/federated.yaml
 
 master:
   components:
     - apiserver
     - scheduler
     - controllermanager
-    - etcd 
+    - etcd
     - flanneld
     # kubernetes is a component to cover the config file /etc/kubernetes/config that is referred to in the benchmark
     - kubernetes
@@ -24,39 +23,58 @@ master:
     bins:
       - "kube-apiserver"
       - "hyperkube apiserver"
+      - "hyperkube kube-apiserver"
       - "apiserver"
     confs:
-      - /etc/kubernetes/apiserver.conf
-      - /etc/kubernetes/apiserver
-    defaultconf: /etc/kubernetes/apiserver
+      - /etc/kubernetes/manifests/kube-apiserver.yaml
+      - /etc/kubernetes/manifests/kube-apiserver.yml
+      - /etc/kubernetes/manifests/kube-apiserver.manifest
+      - /var/snap/kube-apiserver/current/args
+      - /var/snap/microk8s/current/args/kube-apiserver
+    defaultconf: /etc/kubernetes/manifests/kube-apiserver.yaml
 
   scheduler:
     bins:
       - "kube-scheduler"
       - "hyperkube scheduler"
+      - "hyperkube kube-scheduler"
       - "scheduler"
-    confs: 
-      - /etc/kubernetes/scheduler.conf
-      - /etc/kubernetes/scheduler
-    defaultconf: /etc/kubernetes/scheduler
+    confs:
+      - /etc/kubernetes/manifests/kube-scheduler.yaml
+      - /etc/kubernetes/manifests/kube-scheduler.yml
+      - /etc/kubernetes/manifests/kube-scheduler.manifest
+      - /var/snap/kube-scheduler/current/args
+      - /var/snap/microk8s/current/args/kube-scheduler
+    defaultconf: /etc/kubernetes/manifests/kube-scheduler.yaml
 
   controllermanager:
     bins:
       - "kube-controller-manager"
+      - "kube-controller"
       - "hyperkube controller-manager"
+      - "hyperkube kube-controller-manager"
       - "controller-manager"
     confs:
-      - /etc/kubernetes/controller-manager.conf
-      - /etc/kubernetes/controller-manager
-    defaultconf: /etc/kubernetes/controller-manager
+      - /etc/kubernetes/manifests/kube-controller-manager.yaml
+      - /etc/kubernetes/manifests/kube-controller-manager.yml
+      - /etc/kubernetes/manifests/kube-controller-manager.manifest
+      - /var/snap/kube-controller-manager/current/args
+      - /var/snap/microk8s/current/args/kube-controller-manager
+    defaultconf: /etc/kubernetes/manifests/kube-controller-manager.yaml
 
   etcd:
     optional: true
     bins:
       - "etcd"
     confs:
+      - /etc/kubernetes/manifests/etcd.yaml
+      - /etc/kubernetes/manifests/etcd.yml
+      - /etc/kubernetes/manifests/etcd.manifest
       - /etc/etcd/etcd.conf
-    defaultconf: /etc/etcd/etcd.conf
+      - /var/snap/etcd/common/etcd.conf.yml
+      - /var/snap/etcd/common/etcd.conf.yaml
+      - /var/snap/microk8s/current/args/etcd
+    defaultconf: /etc/kubernetes/manifests/etcd.yaml
 
   flanneld:
     optional: true
@@ -72,40 +90,141 @@ node:
     - kubernetes
 
   kubernetes:
-    defaultconf: /etc/kubernetes/config    
+    defaultconf: "/etc/kubernetes/config"
 
   kubelet:
+    cafile:
+      - "/etc/kubernetes/pki/ca.crt"
+      - "/etc/kubernetes/certs/ca.crt"
+      - "/etc/kubernetes/cert/ca.pem"
+      - "/var/snap/microk8s/current/certs/ca.crt"
+    svc:
+      # These paths must also be included
+      #  in the 'confs' property below
+      - "/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"
+      - "/etc/systemd/system/kubelet.service"
+      - "/lib/systemd/system/kubelet.service"
+      - "/etc/systemd/system/snap.kubelet.daemon.service"
+      - "/etc/systemd/system/snap.microk8s.daemon-kubelet.service"
     bins:
       - "hyperkube kubelet"
       - "kubelet"
+    kubeconfig:
+      - "/etc/kubernetes/kubelet.conf"
+      - "/var/lib/kubelet/kubeconfig"
+      - "/etc/kubernetes/kubelet-kubeconfig"
+      - "/var/snap/microk8s/current/credentials/kubelet.config"
     confs:
-      - /etc/kubernetes/kubelet.conf
-      - /etc/kubernetes/kubelet 
-    defaultconf: "/etc/kubernetes/kubelet.conf"
+      - "/var/lib/kubelet/config.yaml"
+      - "/var/lib/kubelet/config.yml"
+      - "/etc/kubernetes/kubelet/kubelet-config.json"
+      - "/home/kubernetes/kubelet-config.yaml"
+      - "/home/kubernetes/kubelet-config.yml"
+      - "/etc/default/kubelet"
+      - "/var/lib/kubelet/kubeconfig"
+      - "/var/snap/kubelet/current/args"
+      - "/var/snap/microk8s/current/args/kubelet"
+      ## Due to the fact that the kubelet might be configured
+      ## without a kubelet-config file, we use a work-around
+      ## of pointing to the systemd service file (which can also
+      ## hold kubelet configuration).
+      ## Note: The following paths must match the one under 'svc'
+      - "/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"
+      - "/etc/systemd/system/kubelet.service"
+      - "/lib/systemd/system/kubelet.service"
+      - "/etc/systemd/system/snap.kubelet.daemon.service"
+      - "/etc/systemd/system/snap.microk8s.daemon-kubelet.service"
+    defaultconf: "/var/lib/kubelet/config.yaml"
+    defaultsvc: "/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"
+    defaultkubeconfig: "/etc/kubernetes/kubelet.conf"
+    defaultcafile: "/etc/kubernetes/pki/ca.crt"
 
   proxy:
+    optional: true
     bins:
       - "kube-proxy"
       - "hyperkube proxy"
+      - "hyperkube kube-proxy"
       - "proxy"
     confs:
-      - /etc/kubernetes/proxy.conf
       - /etc/kubernetes/proxy
       - /etc/kubernetes/addons/kube-proxy-daemonset.yaml
+      - /etc/kubernetes/addons/kube-proxy-daemonset.yml
+      - /var/snap/kube-proxy/current/args
+      - /var/snap/microk8s/current/args/kube-proxy
+    kubeconfig:
+      - "/etc/kubernetes/kubelet-kubeconfig"
+      - "/var/lib/kubelet/kubeconfig"
+      - "/var/snap/microk8s/current/credentials/proxy.config"
+    svc:
+      - "/lib/systemd/system/kube-proxy.service"
+      - "/etc/systemd/system/snap.microk8s.daemon-proxy.service"
+    defaultconf: /etc/kubernetes/addons/kube-proxy-daemonset.yaml
+    defaultkubeconfig: "/etc/kubernetes/proxy.conf"
 
-federated:
+etcd:
   components:
-    - fedapiserver
-    - fedcontrollermanager
+    - etcd
 
-  fedapiserver:
+  etcd:
     bins:
-      - "hyperkube federation-apiserver"
-      - "kube-federation-apiserver"
-      - "federation-apiserver"
+      - "etcd"
+    confs:
+      - /etc/kubernetes/manifests/etcd.yaml
+      - /etc/kubernetes/manifests/etcd.yml
+      - /etc/kubernetes/manifests/etcd.manifest
+      - /etc/etcd/etcd.conf
+      - /var/snap/etcd/common/etcd.conf.yml
+      - /var/snap/etcd/common/etcd.conf.yaml
+      - /var/snap/microk8s/current/args/etcd
+    defaultconf: /etc/kubernetes/manifests/etcd.yaml
 
-  fedcontrollermanager:
-    bins:
-      - "hyperkube federation-controller-manager"
-      - "kube-federation-controller-manager"
-      - "federation-controller-manager"
+controlplane:
+  components: []
+
+policies:
+  components: []
+
+managedservices:
+  components: []
+
+version_mapping:
+  "1.15": "cis-1.5"
+  "1.16": "cis-1.6"
+  "1.17": "cis-1.6"
+  "1.18": "cis-1.6"
+  "1.19": "cis-1.6"
+  "eks-1.0": "eks-1.0"
+  "gke-1.0": "gke-1.0"
+  "ocp-3.10": "rh-0.7"
+  "ocp-3.11": "rh-0.7"
+
+target_mapping:
+  "cis-1.5":
+    - "master"
+    - "node"
+    - "controlplane"
+    - "etcd"
+    - "policies"
+  "cis-1.6":
+    - "master"
+    - "node"
+    - "controlplane"
+    - "etcd"
+    - "policies"
+  "gke-1.0":
+    - "master"
+    - "node"
+    - "controlplane"
+    - "etcd"
+    - "policies"
+    - "managedservices"
+  "eks-1.0":
+    - "master"
+    - "node"
+    - "controlplane"
+    - "policies"
+    - "managedservices"
+  "rh-0.7":
+    - "master"
+    - "node"

cfg/eks-1.0/config.yaml

@@ -0,0 +1,2 @@
+---
+## Version-specific settings that override the values in cfg/config.yaml

cfg/eks-1.0/controlplane.yaml

@@ -0,0 +1,14 @@
+---
+controls:
+version: "eks-1.0"
+id: 2
+text: "Control Plane Configuration"
+type: "controlplane"
+groups:
+  - id: 2.1
+    text: "Logging"
+    checks:
+      - id: 2.1.1
+        text: "Enable audit logs"
+        remediation: "Enable control plane logging for API Server, Audit, Authenticator, Controller Manager, and Scheduler."
+        scored: false

cfg/eks-1.0/managedservices.yaml

@@ -0,0 +1,104 @@
+---
+controls:
+version: "eks-1.0"
+id: 5
+text: "Managed Services"
+type: "managedservices"
+groups:
+  - id: 5.1
+    text: "Image Registry and Image Scanning"
+    checks:
+      - id: 5.1.1
+        text: "Ensure Image Vulnerability Scanning using Amazon ECR image scanning or a third-party provider (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+      - id: 5.1.2
+        text: "Minimize user access to Amazon ECR (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+      - id: 5.1.3
+        text: "Minimize cluster access to read-only for Amazon ECR (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+      - id: 5.1.4
+        text: "Minimize Container Registries to only those approved (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+  - id: 5.2
+    text: "Identity and Access Management (IAM)"
+    checks:
+      - id: 5.2.1
+        text: "Prefer using dedicated Amazon EKS Service Accounts (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+  - id: 5.3
+    text: "AWS Key Management Service (AWS KMS)"
+    checks:
+      - id: 5.3.1
+        text: "Ensure Kubernetes Secrets are encrypted using Customer Master Keys (CMKs) managed in AWS KMS (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+  - id: 5.4
+    text: "Cluster Networking"
+    checks:
+      - id: 5.4.1
+        text: "Restrict Access to the Control Plane Endpoint (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+      - id: 5.4.2
+        text: "Ensure clusters are created with Private Endpoint Enabled and Public Access Disabled (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+      - id: 5.4.3
+        text: "Ensure clusters are created with Private Nodes (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+      - id: 5.4.4
+        text: "Ensure Network Policy is Enabled and set as appropriate (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+      - id: 5.4.5
+        text: "Encrypt traffic to HTTPS load balancers with TLS certificates (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+
+  - id: 5.5
+    text: "Authentication and Authorization"
+    checks:
+      - id: 5.5.1
+        text: "Manage Kubernetes RBAC users with AWS IAM Authenticator for Kubernetes (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false
+
+
+  - id: 5.6
+    text: "Other Cluster Configurations"
+    checks:
+      - id: 5.6.1
+        text: "Consider Fargate for running untrusted workloads (Not Scored)"
+        type: "manual"
+        remediation:
+        scored: false

cfg/eks-1.0/master.yaml

@@ -0,0 +1,6 @@
+---
+controls:
+version: "eks-1.0"
+id: 1
+text: "Control Plane Components"
+type: "master"

cfg/eks-1.0/node.yaml

@@ -0,0 +1,388 @@
+---
+controls:
+version: "eks-1.0"
+id: 3
+text: "Worker Node Security Configuration"
+type: "node"
+groups:
+  - id: 3.1
+    text: "Worker Node Configuration Files"
+    checks:
+      - id: 3.1.1
+        text: "Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)"
+        audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c %a $proxykubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: "644"
+              compare:
+                op: eq
+                value: "644"
+              set: true
+            - flag: "640"
+              compare:
+                op: eq
+                value: "640"
+              set: true
+            - flag: "600"
+              compare:
+                op: eq
+                value: "600"
+              set: true
+            - flag: "444"
+              compare:
+                op: eq
+                value: "444"
+              set: true
+            - flag: "440"
+              compare:
+                op: eq
+                value: "440"
+              set: true
+            - flag: "400"
+              compare:
+                op: eq
+                value: "400"
+              set: true
+            - flag: "000"
+              compare:
+                op: eq
+                value: "000"
+              set: true
+          bin_op: or
+        remediation: |
+          Run the below command (based on the file location on your system) on each worker node.
+          For example,
+          chmod 644 $proxykubeconfig
+        scored: true
+
+      - id: 3.1.2
+        text: "Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)"
+        audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c %U:%G $proxykubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+              set: true
+        remediation: |
+          Run the below command (based on the file location on your system) on each worker node.
+          For example, chown root:root $proxykubeconfig
+        scored: true
+
+      - id: 3.1.3
+        text: "Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)"
+        audit: '/bin/sh -c ''if test -e $kubeletconf; then stat -c %a $kubeletconf; fi'' '
+        tests:
+          test_items:
+            - flag: "644"
+              set: true
+              compare:
+                op: eq
+                value: "644"
+            - flag: "640"
+              set: true
+              compare:
+                op: eq
+                value: "640"
+            - flag: "600"
+              set: true
+              compare:
+                op: eq
+                value: "600"
+            - flag: "444"
+              compare:
+                op: eq
+                value: "444"
+              set: true
+            - flag: "440"
+              compare:
+                op: eq
+                value: "440"
+              set: true
+            - flag: "400"
+              compare:
+                op: eq
+                value: "400"
+              set: true
+            - flag: "000"
+              compare:
+                op: eq
+                value: "000"
+              set: true
+          bin_op: or
+        remediation: |
+          Run the following command (using the config file location identied in the Audit step)
+          chmod 644 $kubeletconf
+        scored: true
+
+      - id: 3.1.4
+        text: "Ensure that the kubelet configuration file ownership is set to root:root (Scored)"
+        audit: '/bin/sh -c ''if test -e $kubeletconf; then stat -c %U:%G $kubeletconf; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+              set: true
+        remediation: |
+          Run the following command (using the config file location identied in the Audit step)
+          chown root:root $kubeletconf
+        scored: true
+
+  - id: 3.2
+    text: "Kubelet"
+    checks:
+      - id: 3.2.1
+        text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: "--anonymous-auth"
+              path: '{.authentication.anonymous.enabled}'
+              set: true
+              compare:
+                op: eq
+                value: false
+        remediation: |
+          If using a Kubelet config file, edit the file to set authentication: anonymous: enabled to
+          false.
+          If using executable arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --anonymous-auth=false
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 3.2.2
+        text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --authorization-mode
+              path: '{.authorization.mode}'
+              set: true
+              compare:
+                op: nothave
+                value: AlwaysAllow
+        remediation: |
+          If using a Kubelet config file, edit the file to set authorization: mode to Webhook. If
+          using executable arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_AUTHZ_ARGS variable.
+          --authorization-mode=Webhook
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 3.2.3
+        text: "Ensure that the --client-ca-file argument is set as appropriate (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --client-ca-file
+              path: '{.authentication.x509.clientCAFile}'
+              set: true
+        remediation: |
+          If using a Kubelet config file, edit the file to set authentication: x509: clientCAFile to
+          the location of the client CA file.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_AUTHZ_ARGS variable.
+          --client-ca-file=<path/to/client-ca-file>
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 3.2.4
+        text: "Ensure that the --read-only-port argument is set to 0 (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: "--read-only-port"
+              path: '{.readOnlyPort}'
+              set: true
+              compare:
+                op: eq
+                value: 0
+        remediation: |
+          If using a Kubelet config file, edit the file to set readOnlyPort to 0.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --read-only-port=0
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 3.2.5
+        text: "Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --streaming-connection-idle-timeout
+              path: '{.streamingConnectionIdleTimeout}'
+              set: true
+              compare:
+                op: noteq
+                value: 0
+            - flag: --streaming-connection-idle-timeout
+              path: '{.streamingConnectionIdleTimeout}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to set streamingConnectionIdleTimeout to a
+          value other than 0.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --streaming-connection-idle-timeout=5m
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 3.2.6
+        text: "Ensure that the --protect-kernel-defaults argument is set to true (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --protect-kernel-defaults
+              path: '{.protectKernelDefaults}'
+              set: true
+              compare:
+                op: eq
+                value: true
+        remediation: |
+          If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --protect-kernel-defaults=true
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 3.2.7
+        text: "Ensure that the --make-iptables-util-chains argument is set to true (Scored) "
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --make-iptables-util-chains
+              path: '{.makeIPTablesUtilChains}'
+              set: true
+              compare:
+                op: eq
+                value: true
+            - flag: --make-iptables-util-chains
+              path: '{.makeIPTablesUtilChains}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to set makeIPTablesUtilChains: true.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          remove the --make-iptables-util-chains argument from the
+          KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 3.2.8
+        text: "Ensure that the --hostname-override argument is not set (Scored)"
+        # This is one of those properties that can only be set as a command line argument.
+        # To check if the property is set as expected, we need to parse the kubelet command
+        # instead reading the Kubelet Configuration file.
+        audit: "/bin/ps -fC $kubeletbin "
+        tests:
+          test_items:
+            - flag: --hostname-override
+              set: false
+        remediation: |
+          Edit the kubelet service file $kubeletsvc
+          on each worker node and remove the --hostname-override argument from the
+          KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 3.2.9
+        text: "Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --event-qps
+              path: '{.eventRecordQPS}'
+              set: true
+              compare:
+                op: eq
+                value: 0
+        remediation: |
+          If using a Kubelet config file, edit the file to set eventRecordQPS: to an appropriate level.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: false
+
+      - id: 3.2.10
+        text: "Ensure that the --rotate-certificates argument is not set to false (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --rotate-certificates
+              path: '{.rotateCertificates}'
+              set: true
+              compare:
+                op: eq
+                value: true
+            - flag: --rotate-certificates
+              path: '{.rotateCertificates}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to add the line rotateCertificates: true or
+          remove it altogether to use the default value.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          remove --rotate-certificates=false argument from the KUBELET_CERTIFICATE_ARGS
+          variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 3.2.11
+        text: "Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: RotateKubeletServerCertificate
+              path: '{.featureGates.RotateKubeletServerCertificate}'
+              set: true
+              compare:
+                op: eq
+                value: true
+        remediation: |
+          Edit the kubelet service file $kubeletsvc
+          on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
+          --feature-gates=RotateKubeletServerCertificate=true
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true

cfg/eks-1.0/policies.yaml

@@ -0,0 +1,237 @@
+---
+controls:
+version: "eks-1.0"
+id: 4
+text: "Policies"
+type: "policies"
+groups:
+  - id: 4.1
+    text: "RBAC and Service Accounts"
+    checks:
+      - id: 4.1.1
+        text: "Ensure that the cluster-admin role is only used where required (Not Scored)"
+        type: "manual"
+        remediation: |
+          Identify all clusterrolebindings to the cluster-admin role. Check if they are used and
+          if they need this role or if they could use a role with fewer privileges.
+          Where possible, first bind users to a lower privileged role and then remove the
+          clusterrolebinding to the cluster-admin role :
+          kubectl delete clusterrolebinding [name]
+        scored: false
+
+      - id: 4.1.2
+        text: "Minimize access to secrets (Not Scored)"
+        type: "manual"
+        remediation: |
+          Where possible, remove get, list and watch access to secret objects in the cluster.
+        scored: false
+
+      - id: 4.1.3
+        text: "Minimize wildcard use in Roles and ClusterRoles (Not Scored)"
+        type: "manual"
+        remediation: |
+          Where possible replace any use of wildcards in clusterroles and roles with specific
+          objects or actions.
+        scored: false
+
+      - id: 4.1.4
+        text: "Minimize access to create pods (Not Scored)"
+        type: "manual"
+        Remediation: |
+          Where possible, remove create access to pod objects in the cluster.
+        scored: false
+
+      - id: 4.1.5
+        text: "Ensure that default service accounts are not actively used. (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create explicit service accounts wherever a Kubernetes workload requires specific access
+          to the Kubernetes API server.
+          Modify the configuration of each default service account to include this value
+          automountServiceAccountToken: false
+        scored: false
+
+      - id: 4.1.6
+        text: "Ensure that Service Account Tokens are only mounted where necessary (Not Scored)"
+        type: "manual"
+        remediation: |
+          Modify the definition of pods and service accounts which do not need to mount service
+          account tokens to disable it.
+        scored: false
+
+  - id: 4.2
+    text: "Pod Security Policies"
+    checks:
+      - id: 4.2.1
+        text: "Minimize the admission of privileged containers (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that
+          the .spec.privileged field is omitted or set to false.
+        scored: false
+
+      - id: 4.2.2
+        text: "Minimize the admission of containers wishing to share the host process ID namespace (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostPID field is omitted or set to false.
+        scored: false
+
+      - id: 4.2.3
+        text: "Minimize the admission of containers wishing to share the host IPC namespace (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostIPC field is omitted or set to false.
+        scored: false
+
+      - id: 4.2.4
+        text: "Minimize the admission of containers wishing to share the host network namespace (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostNetwork field is omitted or set to false.
+        scored: false
+
+      - id: 4.2.5
+        text: "Minimize the admission of containers with allowPrivilegeEscalation (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.allowPrivilegeEscalation field is omitted or set to false.
+        scored: false
+
+      - id: 4.2.6
+        text: "Minimize the admission of root containers (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.runAsUser.rule is set to either MustRunAsNonRoot or MustRunAs with the range of
+          UIDs not including 0.
+        scored: false
+
+      - id: 4.2.7
+        text: "Minimize the admission of containers with the NET_RAW capability (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.requiredDropCapabilities is set to include either NET_RAW or ALL.
+        scored: false
+
+      - id: 4.2.8
+        text: "Minimize the admission of containers with added capabilities (Not Scored)"
+        type: "manual"
+        remediation: |
+          Ensure that allowedCapabilities is not present in PSPs for the cluster unless
+          it is set to an empty array.
+        scored: false
+
+      - id: 4.2.9
+        text: "Minimize the admission of containers with capabilities assigned (Not Scored)"
+        type: "manual"
+        remediation: |
+          Review the use of capabilites in applications runnning on your cluster. Where a namespace
+          contains applications which do not require any Linux capabities to operate consider adding
+          a PSP which forbids the admission of containers which do not drop all capabilities.
+        scored: false
+
+  - id: 4.3
+    text: "CNI Plugin"
+    checks:
+      - id: 4.3.1
+        text: "Ensure that the latest CNI version is used (Not Scored)"
+        type: "manual"
+        remediation: |
+          Review the documentation of AWS CNI plugin, and ensure latest CNI version is used.
+        scored: false
+
+      - id: 4.3.2
+        text: "Ensure that all Namespaces have Network Policies defined (Not Scored)"
+        type: "manual"
+        remediation: |
+          Follow the documentation and create NetworkPolicy objects as you need them.
+        scored: false
+
+  - id: 4.4
+    text: "Secrets Management"
+    checks:
+      - id: 4.4.1
+        text: "Prefer using secrets as files over secrets as environment variables (Not Scored)"
+        type: "manual"
+        remediation: |
+          If possible, rewrite application code to read secrets from mounted secret files, rather than
+          from environment variables.
+        scored: false
+
+      - id: 4.4.2
+        text: "Consider external secret storage (Not Scored)"
+        type: "manual"
+        remediation: |
+          Refer to the secrets management options offered by your cloud provider or a third-party
+          secrets management solution.
+        scored: false
+
+  - id: 4.5
+    text: "Extensible Admission Control"
+    checks:
+      - id: 4.5.1
+        text: "Configure Image Provenance using ImagePolicyWebhook admission controller (Not Scored)"
+        type: "manual"
+        remediation: |
+          Follow the Kubernetes documentation and setup image provenance.
+        scored: false
+
+  - id: 4.6
+    text: "General Policies"
+    checks:
+      - id: 4.6.1
+        text: "Create administrative boundaries between resources using namespaces (Not Scored)"
+        type: "manual"
+        remediation: |
+          Follow the documentation and create namespaces for objects in your deployment as you need
+          them.
+        scored: false
+
+      - id: 4.6.2
+        text: "Ensure that the seccomp profile is set to docker/default in your pod definitions (Not Scored)"
+        type: "manual"
+        remediation: |
+          Seccomp is an alpha feature currently. By default, all alpha features are disabled. So, you
+          would need to enable alpha features in the apiserver by passing "--feature-
+          gates=AllAlpha=true" argument.
+          Edit the /etc/kubernetes/apiserver file on the master node and set the KUBE_API_ARGS
+          parameter to "--feature-gates=AllAlpha=true"
+          KUBE_API_ARGS="--feature-gates=AllAlpha=true"
+          Based on your system, restart the kube-apiserver service. For example:
+          systemctl restart kube-apiserver.service
+          Use annotations to enable the docker/default seccomp profile in your pod definitions. An
+          example is as below:
+          apiVersion: v1
+          kind: Pod
+          metadata:
+            name: trustworthy-pod
+            annotations:
+              seccomp.security.alpha.kubernetes.io/pod: docker/default
+          spec:
+            containers:
+              - name: trustworthy-container
+                image: sotrustworthy:latest
+        scored: false
+
+      - id: 4.6.3
+        text: "Apply Security Context to Your Pods and Containers (Not Scored)"
+        type: "manual"
+        remediation: |
+          Follow the Kubernetes documentation and apply security contexts to your pods. For a
+          suggested list of security contexts, you may refer to the CIS Security Benchmark for Docker
+          Containers.
+        scored: false
+
+      - id: 4.6.4
+        text: "The default namespace should not be used (Not Scored)"
+        type: "manual"
+        remediation: |
+          Ensure that namespaces are created to allow for appropriate segregation of Kubernetes
+          resources and that all new resources are created in a specific namespace.
+        scored: false

cfg/gke-1.0/config.yaml

@@ -0,0 +1,2 @@
+---
+## Version-specific settings that override the values in cfg/config.yaml

cfg/gke-1.0/controlplane.yaml

@@ -0,0 +1,32 @@
+---
+controls:
+version: "gke-1.0"
+id: 3
+text: "Control Plane Configuration"
+type: "controlplane"
+groups:
+  - id: 3.1
+    text: "Authentication and Authorization"
+    checks:
+      - id: 3.1.1
+        text: "Client certificate authentication should not be used for users (Not Scored)"
+        type: "manual"
+        remediation: |
+          Alternative mechanisms provided by Kubernetes such as the use of OIDC should be
+          implemented in place of client certificates.
+          You can remediate the availability of client certificates in your GKE cluster. See
+          Recommendation 6.8.2.
+        scored: false
+
+  - id: 3.2
+    text: "Logging"
+    checks:
+      - id: 3.2.1
+        text: "Ensure that a minimal audit policy is created (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 3.2.2
+        text: "Ensure that the audit policy covers key security concerns (Not Scored) "
+        remediation: "This control cannot be modified in GKE."
+        scored: false

cfg/gke-1.0/etcd.yaml

@@ -0,0 +1,45 @@
+---
+controls:
+version: "gke-1.0"
+id: 2
+text: "Etcd Node Configuration"
+type: "etcd"
+groups:
+  - id: 2
+    text: "Etcd Node Configuration Files"
+    checks:
+      - id: 2.1
+        text: "Ensure that the --cert-file and --key-file arguments are set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 2.2
+        text: "Ensure that the --client-cert-auth argument is set to true (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 2.3
+        text: "Ensure that the --auto-tls argument is not set to true (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 2.4
+        text: "Ensure that the --peer-cert-file and --peer-key-file arguments are
+        set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 2.5
+        text: "Ensure that the --peer-client-cert-auth argument is set to true (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 2.6
+        text: "Ensure that the --peer-auto-tls argument is not set to true (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 2.7
+        text: "Ensure that a unique Certificate Authority is used for etcd (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false

cfg/gke-1.0/managedservices.yaml

@@ -0,0 +1,703 @@
+---
+controls:
+version: "gke-1.0"
+id: 6
+text: "Managed Services"
+type: "managedservices"
+groups:
+  - id: 6.1
+    text: "Image Registry and Image Scanning"
+    checks:
+      - id: 6.1.1
+        text: "Ensure Image Vulnerability Scanning using GCR Container Analysis
+        or a third-party provider (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+
+            gcloud services enable containerscanning.googleapis.com
+        scored: true
+
+      - id: 6.1.2
+        text: "Minimize user access to GCR (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To change roles at the GCR bucket level:
+            Firstly, run the following if read permissions are required:
+
+              gsutil iam ch [TYPE]:[EMAIL-ADDRESS]:objectViewer
+              gs://artifacts.[PROJECT_ID].appspot.com
+
+            Then remove the excessively privileged role (Storage Admin / Storage Object Admin /
+            Storage Object Creator) using:
+
+              gsutil iam ch -d [TYPE]:[EMAIL-ADDRESS]:[ROLE]
+              gs://artifacts.[PROJECT_ID].appspot.com
+
+            where:
+              [TYPE] can be one of the following:
+                    o user, if the [EMAIL-ADDRESS] is a Google account
+                    o serviceAccount, if [EMAIL-ADDRESS] specifies a Service account
+              [EMAIL-ADDRESS] can be one of the following:
+                    o a Google account (for example, someone@example.com)
+                    o a Cloud IAM service account
+                    To modify roles defined at the project level and subsequently inherited within the GCR
+                    bucket, or the Service Account User role, extract the IAM policy file, modify it accordingly
+            and apply it using:
+
+              gcloud projects set-iam-policy [PROJECT_ID] [POLICY_FILE]
+        scored: true
+
+      - id: 6.1.3
+        text: "Minimize cluster access to read-only for GCR (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            For an account explicitly granted to the bucket. First, add read access to the Kubernetes
+            Service Account
+
+              gsutil iam ch [TYPE]:[EMAIL-ADDRESS]:objectViewer
+              gs://artifacts.[PROJECT_ID].appspot.com
+
+              where:
+              [TYPE] can be one of the following:
+                      o user, if the [EMAIL-ADDRESS] is a Google account
+                      o serviceAccount, if [EMAIL-ADDRESS] specifies a Service account
+              [EMAIL-ADDRESS] can be one of the following:
+                      o a Google account (for example, someone@example.com)
+                      o a Cloud IAM service account
+
+              Then remove the excessively privileged role (Storage Admin / Storage Object Admin /
+              Storage Object Creator) using:
+
+                gsutil iam ch -d [TYPE]:[EMAIL-ADDRESS]:[ROLE]
+                gs://artifacts.[PROJECT_ID].appspot.com
+
+              For an account that inherits access to the GCR Bucket through Project level permissions,
+              modify the Projects IAM policy file accordingly, then upload it using:
+
+                gcloud projects set-iam-policy [PROJECT_ID] [POLICY_FILE]
+        scored: true
+
+      - id: 6.1.4
+        text: "Minimize Container Registries to only those approved (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            First, update the cluster to enable Binary Authorization:
+
+              gcloud container cluster update [CLUSTER_NAME] \
+                --enable-binauthz
+
+            Create a Binary Authorization Policy using the Binary Authorization Policy Reference
+            (https://cloud.google.com/binary-authorization/docs/policy-yaml-reference) for guidance.
+            Import the policy file into Binary Authorization:
+
+              gcloud container binauthz policy import [YAML_POLICY]
+        scored: false
+
+  - id: 6.2
+    text: "Identity and Access Management (IAM)"
+    checks:
+      - id: 6.2.1
+        text: "Ensure GKE clusters are not running using the Compute Engine
+        default service account (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            Firstly, create a minimally privileged service account:
+
+              gcloud iam service-accounts create [SA_NAME] \
+                --display-name "GKE Node Service Account"
+              export NODE_SA_EMAIL=`gcloud iam service-accounts list \
+                --format='value(email)' \
+                --filter='displayName:GKE Node Service Account'`
+
+            Grant the following roles to the service account:
+
+              export PROJECT_ID=`gcloud config get-value project`
+              gcloud projects add-iam-policy-binding $PROJECT_ID \
+                --member serviceAccount:$NODE_SA_EMAIL \
+                --role roles/monitoring.metricWriter
+              gcloud projects add-iam-policy-binding $PROJECT_ID \
+                --member serviceAccount:$NODE_SA_EMAIL \
+                --role roles/monitoring.viewer
+              gcloud projects add-iam-policy-binding $PROJECT_ID \
+                --member serviceAccount:$NODE_SA_EMAIL \
+                --role roles/logging.logWriter
+
+            To create a new Node pool using the Service account, run the following command:
+
+              gcloud container node-pools create [NODE_POOL] \
+                --service-account=[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com \
+                --cluster=[CLUSTER_NAME] --zone [COMPUTE_ZONE]
+
+            You will need to migrate your workloads to the new Node pool, and delete Node pools that
+            use the default service account to complete the remediation.
+        scored: true
+
+      - id: 6.2.2
+        text: "Prefer using dedicated GCP Service Accounts and Workload Identity (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+
+              gcloud beta container clusters update [CLUSTER_NAME] --zone [CLUSTER_ZONE] \
+                --identity-namespace=[PROJECT_ID].svc.id.goog
+
+            Note that existing Node pools are unaffected. New Node pools default to --workload-
+            metadata-from-node=GKE_METADATA_SERVER .
+
+            Then, modify existing Node pools to enable GKE_METADATA_SERVER:
+
+              gcloud beta container node-pools update [NODEPOOL_NAME] \
+                --cluster=[CLUSTER_NAME] --zone [CLUSTER_ZONE] \
+                --workload-metadata-from-node=GKE_METADATA_SERVER
+
+            You may also need to modify workloads in order for them to use Workload Identity as
+            described within https://cloud.google.com/kubernetes-engine/docs/how-to/workload-
+            identity. Also consider the effects on the availability of your hosted workloads as Node
+            pools are updated, it may be more appropriate to create new Node Pools.
+        scored: false
+
+  - id: 6.3
+    text: "Cloud Key Management Service (Cloud KMS)"
+    checks:
+      - id: 6.3.1
+        text: "Ensure Kubernetes Secrets are encrypted using keys managed in Cloud KMS (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To create a key
+
+            Create a key ring:
+
+              gcloud kms keyrings create [RING_NAME] \
+                --location [LOCATION] \
+                --project [KEY_PROJECT_ID]
+
+            Create a key:
+
+              gcloud kms keys create [KEY_NAME] \
+                --location [LOCATION] \
+                --keyring [RING_NAME] \
+                --purpose encryption \
+                --project [KEY_PROJECT_ID]
+
+            Grant the Kubernetes Engine Service Agent service account the Cloud KMS CryptoKey
+            Encrypter/Decrypter role:
+
+              gcloud kms keys add-iam-policy-binding [KEY_NAME] \
+                --location [LOCATION] \
+                --keyring [RING_NAME] \
+                --member serviceAccount:[SERVICE_ACCOUNT_NAME] \
+                --role roles/cloudkms.cryptoKeyEncrypterDecrypter \
+                --project [KEY_PROJECT_ID]
+
+            To create a new cluster with Application-layer Secrets Encryption:
+
+              gcloud container clusters create [CLUSTER_NAME] \
+                --cluster-version=latest \
+                --zone [ZONE] \
+                --database-encryption-key projects/[KEY_PROJECT_ID]/locations/[LOCATION]/keyRings/[RING_NAME]/cryptoKey s/[KEY_NAME] \
+                --project [CLUSTER_PROJECT_ID]
+
+            To enable on an existing cluster:
+
+              gcloud container clusters update [CLUSTER_NAME] \
+                --zone [ZONE] \
+                --database-encryption-key projects/[KEY_PROJECT_ID]/locations/[LOCATION]/keyRings/[RING_NAME]/cryptoKey s/[KEY_NAME] \
+                --project [CLUSTER_PROJECT_ID]
+        scored: true
+
+  - id: 6.4
+    text: "Node Metadata"
+    checks:
+      - id: 6.4.1
+        text: "Ensure legacy Compute Engine instance metadata APIs are Disabled (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To update an existing cluster, create a new Node pool with the legacy GCE metadata
+            endpoint disabled:
+
+              gcloud container node-pools create [POOL_NAME] \
+                --metadata disable-legacy-endpoints=true \
+                --cluster [CLUSTER_NAME] \
+                --zone [COMPUTE_ZONE]
+
+            You will need to migrate workloads from any existing non-conforming Node pools, to the
+            new Node pool, then delete non-conforming Node pools to complete the remediation.
+        scored: true
+
+      - id: 6.4.2
+        text: "Ensure the GKE Metadata Server is Enabled (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+              gcloud beta container clusters update [CLUSTER_NAME] \
+                --identity-namespace=[PROJECT_ID].svc.id.goog
+            Note that existing Node pools are unaffected. New Node pools default to --workload-
+            metadata-from-node=GKE_METADATA_SERVER .
+
+            To modify an existing Node pool to enable GKE Metadata Server:
+
+              gcloud beta container node-pools update [NODEPOOL_NAME] \
+                --cluster=[CLUSTER_NAME] \
+                --workload-metadata-from-node=GKE_METADATA_SERVER
+
+            You may also need to modify workloads in order for them to use Workload Identity as
+            described within https://cloud.google.com/kubernetes-engine/docs/how-to/workload-
+            identity.
+        scored: false
+
+  - id: 6.5
+    text: "Node Configuration and Maintenance"
+    checks:
+      - id: 6.5.1
+        text: "Ensure Container-Optimized OS (COS) is used for GKE node images (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To set the node image to cos for an existing cluster's Node pool:
+
+              gcloud container clusters upgrade [CLUSTER_NAME]\
+                --image-type cos \
+                --zone [COMPUTE_ZONE] --node-pool [POOL_NAME]
+        scored: true
+
+      - id: 6.5.2
+        text: "Ensure Node Auto-Repair is enabled for GKE nodes (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To enable node auto-repair for an existing cluster with Node pool, run the following
+            command:
+
+              gcloud container node-pools update [POOL_NAME] \
+                --cluster [CLUSTER_NAME] --zone [COMPUTE_ZONE] \
+                --enable-autorepair
+        scored: true
+
+      - id: 6.5.3
+        text: "Ensure Node Auto-Upgrade is enabled for GKE nodes (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To enable node auto-upgrade for an existing cluster's Node pool, run the following
+            command:
+
+              gcloud container node-pools update [NODE_POOL] \
+                --cluster [CLUSTER_NAME] --zone [COMPUTE_ZONE] \
+                --enable-autoupgrade
+        scored: true
+
+      - id: 6.5.4
+        text: "Automate GKE version management using Release Channels (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            Create a new cluster by running the following command:
+
+              gcloud beta container clusters create [CLUSTER_NAME] \
+                --zone [COMPUTE_ZONE] \
+                --release-channel [RELEASE_CHANNEL]
+
+            where [RELEASE_CHANNEL] is stable or regular according to your needs.
+        scored: false
+
+      - id: 6.5.5
+        text: "Ensure Shielded GKE Nodes are Enabled (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To create a Node pool within the cluster with Integrity Monitoring enabled, run the
+            following command:
+
+              gcloud beta container node-pools create [NODEPOOL_NAME] \
+                --cluster [CLUSTER_NAME] --zone [COMPUTE_ZONE] \
+                --shielded-integrity-monitoring
+
+            You will also need to migrate workloads from existing non-conforming Node pools to the
+            newly created Node pool, then delete the non-conforming pools.
+        scored: false
+
+      - id: 6.5.6
+        text: "Ensure Shielded GKE Nodes are Enabled (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To migrate an existing cluster, you will need to specify the --enable-shielded-nodes flag
+            on a cluster update command:
+
+              gcloud beta container clusters update [CLUSTER_NAME] \
+                --zone [CLUSTER_ZONE] \
+                --enable-shielded-nodes
+        scored: false
+
+      - id: 6.5.7
+        text: "Ensure Secure Boot for Shielded GKE Nodes is Enabled (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To create a Node pool within the cluster with Secure Boot enabled, run the following
+            command:
+
+              gcloud beta container node-pools create [NODEPOOL_NAME] \
+                --cluster [CLUSTER_NAME] --zone [COMPUTE_ZONE] \
+                --shielded-secure-boot
+
+            You will also need to migrate workloads from existing non-conforming Node pools to the
+            newly created Node pool, then delete the non-conforming pools.
+        scored: false
+
+  - id: 6.6
+    text: "Cluster Networking"
+    checks:
+      - id: 6.6.1
+        text: "Enable VPC Flow Logs and Intranode Visibility (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To enable intranode visibility on an existing cluster, run the following command:
+
+              gcloud beta container clusters update [CLUSTER_NAME] \
+                --enable-intra-node-visibility
+        scored: false
+
+      - id: 6.6.2
+        text: "Ensure use of VPC-native clusters (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To enable Alias IP on a new cluster, run the following command:
+
+              gcloud container clusters create [CLUSTER_NAME] \
+                --zone [COMPUTE_ZONE] \
+                --enable-ip-alias
+        scored: true
+
+      - id: 6.6.3
+        text: "Ensure Master Authorized Networks is Enabled (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To check Master Authorized Networks status for an existing cluster, run the following
+            command;
+
+              gcloud container clusters describe [CLUSTER_NAME] \
+                --zone [COMPUTE_ZONE] \
+                --format json | jq '.masterAuthorizedNetworksConfig'
+
+            The output should return
+
+              {
+                "enabled": true
+              }
+
+            if Master Authorized Networks is enabled.
+
+            If Master Authorized Networks is disabled, the
+            above command will return null ( { } ).
+        scored: true
+
+      - id: 6.6.4
+        text: "Ensure clusters are created with Private Endpoint Enabled and Public Access Disabled (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            Create a cluster with a Private Endpoint enabled and Public Access disabled by including
+            the --enable-private-endpoint flag within the cluster create command:
+
+              gcloud container clusters create [CLUSTER_NAME] \
+                --enable-private-endpoint
+
+            Setting this flag also requires the setting of --enable-private-nodes , --enable-ip-alias
+            and --master-ipv4-cidr=[MASTER_CIDR_RANGE] .
+        scored: true
+
+      - id: 6.6.5
+        text: "Ensure clusters are created with Private Nodes (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To create a cluster with Private Nodes enabled, include the --enable-private-nodes flag
+            within the cluster create command:
+
+              gcloud container clusters create [CLUSTER_NAME] \
+                --enable-private-nodes
+
+            Setting this flag also requires the setting of --enable-ip-alias and --master-ipv4-
+            cidr=[MASTER_CIDR_RANGE] .
+        scored: true
+
+      - id: 6.6.6
+        text: "Consider firewalling GKE worker nodes (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            Use the following command to generate firewall rules, setting the variables as appropriate.
+            You may want to use the target [TAG] and [SERVICE_ACCOUNT] previously identified.
+
+              gcloud compute firewall-rules create FIREWALL_RULE_NAME \
+                --network [NETWORK] \
+                --priority [PRIORITY] \
+                --direction [DIRECTION] \
+                --action [ACTION] \
+                --target-tags [TAG] \
+                --target-service-accounts [SERVICE_ACCOUNT] \
+                --source-ranges [SOURCE_CIDR-RANGE] \
+                --source-tags [SOURCE_TAGS] \
+                --source-service-accounts=[SOURCE_SERVICE_ACCOUNT] \
+                --destination-ranges [DESTINATION_CIDR_RANGE] \
+                --rules [RULES]
+        scored: false
+
+      - id: 6.6.7
+        text: "Ensure Network Policy is Enabled and set as appropriate (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To enable Network Policy for an existing cluster, firstly enable the Network Policy add-on:
+
+              gcloud container clusters update [CLUSTER_NAME] \
+                --zone [COMPUTE_ZONE] \
+                --update-addons NetworkPolicy=ENABLED
+
+            Then, enable Network Policy:
+
+              gcloud container clusters update [CLUSTER_NAME] \
+                --zone [COMPUTE_ZONE] \
+                --enable-network-policy
+        scored: false
+
+      - id: 6.6.8
+        text: "Ensure use of Google-managed SSL Certificates (Not Scored)"
+        type: "manual"
+        remediation: |
+          If services of type:LoadBalancer are discovered, consider replacing the Service with an
+          Ingress.
+
+          To configure the Ingress and use Google-managed SSL certificates, follow the instructions
+          as listed at https://cloud.google.com/kubernetes-engine/docs/how-to/managed-certs.
+        scored: false
+
+  - id: 6.7
+    text: "Logging"
+    checks:
+      - id: 6.7.1
+        text: "Ensure Stackdriver Kubernetes Logging and Monitoring is Enabled (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+
+            STACKDRIVER KUBERNETES ENGINE MONITORING SUPPORT (PREFERRED):
+            To enable Stackdriver Kubernetes Engine Monitoring for an existing cluster, run the
+            following command:
+
+              gcloud container clusters update [CLUSTER_NAME] \
+                --zone [COMPUTE_ZONE] \
+                --enable-stackdriver-kubernetes
+
+            LEGACY STACKDRIVER SUPPORT:
+            Both Logging and Monitoring support must be enabled.
+            To enable Legacy Stackdriver Logging for an existing cluster, run the following command:
+
+              gcloud container clusters update [CLUSTER_NAME] --zone [COMPUTE_ZONE] \
+                --logging-service logging.googleapis.com
+
+            To enable Legacy Stackdriver Monitoring for an existing cluster, run the following
+            command:
+
+              gcloud container clusters update [CLUSTER_NAME] --zone [COMPUTE_ZONE] \
+                --monitoring-service monitoring.googleapis.com
+        scored: true
+
+      - id: 6.7.2
+        text: "Enable Linux auditd logging (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            Download the example manifests:
+
+              curl https://raw.githubusercontent.com/GoogleCloudPlatform/k8s-node-tools/master/os-audit/cos-auditd-logging.yaml \
+                > cos-auditd-logging.yaml
+
+            Edit the example manifests if needed. Then, deploy them:
+
+              kubectl apply -f cos-auditd-logging.yaml
+
+            Verify that the logging Pods have started. If you defined a different Namespace in your
+            manifests, replace cos-auditd with the name of the namespace you're using:
+
+              kubectl get pods --namespace=cos-auditd
+        scored: false
+
+  - id: 6.8
+    text: "Authentication and Authorization"
+    checks:
+      - id: 6.8.1
+        text: "Ensure Basic Authentication using static passwords is Disabled (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To update an existing cluster and disable Basic Authentication by removing the static
+            password:
+
+              gcloud container clusters update [CLUSTER_NAME] \
+                --no-enable-basic-auth
+        scored: true
+
+      - id: 6.8.2
+        text: "Ensure authentication using Client Certificates is Disabled (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            Create a new cluster without a Client Certificate:
+
+              gcloud container clusters create [CLUSTER_NAME] \
+                --no-issue-client-certificate
+        scored: true
+
+      - id: 6.8.3
+        text: "Manage Kubernetes RBAC users with Google Groups for GKE (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            Follow the G Suite Groups instructions at https://cloud.google.com/kubernetes-
+            engine/docs/how-to/role-based-access-control#google-groups-for-gke.
+
+            Then, create a cluster with
+
+              gcloud beta container clusters create my-cluster \
+                --security-group="gke-security-groups@[yourdomain.com]"
+
+            Finally create Roles, ClusterRoles, RoleBindings, and ClusterRoleBindings that
+            reference your G Suite Groups.
+        scored: false
+
+      - id: 6.8.4
+        text: "Ensure Legacy Authorization (ABAC) is Disabled (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To disable Legacy Authorization for an existing cluster, run the following command:
+
+              gcloud container clusters update [CLUSTER_NAME] \
+                --zone [COMPUTE_ZONE] \
+                --no-enable-legacy-authorization
+        scored: true
+
+  - id: 6.9
+    text: "Storage"
+    checks:
+      - id: 6.9.1
+        text: "Enable Customer-Managed Encryption Keys (CMEK) for GKE Persistent Disks (PD) (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            FOR NODE BOOT DISKS:
+            Create a new node pool using customer-managed encryption keys for the node boot disk, of
+            [DISK_TYPE] either pd-standard or pd-ssd :
+
+              gcloud beta container node-pools create [CLUSTER_NAME] \
+                --disk-type [DISK_TYPE] \
+                --boot-disk-kms-key \
+                projects/[KEY_PROJECT_ID]/locations/[LOCATION]/keyRings/[RING_NAME]/cryptoKeys/[KEY_NAME]
+
+            Create a cluster using customer-managed encryption keys for the node boot disk, of
+            [DISK_TYPE] either pd-standard or pd-ssd :
+
+              gcloud beta container clusters create [CLUSTER_NAME] \
+                --disk-type [DISK_TYPE] \
+                --boot-disk-kms-key \
+                projects/[KEY_PROJECT_ID]/locations/[LOCATION]/keyRings/[RING_NAME]/cryptoKeys/[KEY_NAME]
+
+            FOR ATTACHED DISKS:
+            Follow the instructions detailed at https://cloud.google.com/kubernetes-
+            engine/docs/how-to/using-cmek.
+        scored: false
+
+  - id: 6.10
+    text: "Other Cluster Configurations"
+    checks:
+      - id: 6.10.1
+        text: "Ensure Kubernetes Web UI is Disabled (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To disable the Kubernetes Dashboard on an existing cluster, run the following command:
+
+              gcloud container clusters update [CLUSTER_NAME] \
+                --zone [ZONE] \
+                --update-addons=KubernetesDashboard=DISABLED
+        scored: true
+
+      - id: 6.10.2
+        text: "Ensure that Alpha clusters are not used for production workloads (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            Upon creating a new cluster
+
+              gcloud container clusters create [CLUSTER_NAME] \
+                --zone [COMPUTE_ZONE]
+
+            Do not use the --enable-kubernetes-alpha argument.
+        scored: true
+
+      - id: 6.10.3
+        text: "Ensure Pod Security Policy is Enabled and set as appropriate (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To enable Pod Security Policy for an existing cluster, run the following command:
+
+              gcloud beta container clusters update [CLUSTER_NAME] \
+                --zone [COMPUTE_ZONE] \
+                --enable-pod-security-policy
+        scored: false
+
+      - id: 6.10.4
+        text: "Consider GKE Sandbox for running untrusted workloads (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            To enable GKE Sandbox on an existing cluster, a new Node pool must be created.
+
+              gcloud container node-pools create [NODE_POOL_NAME] \
+                --zone=[COMPUTE-ZONE] \
+                --cluster=[CLUSTER_NAME] \
+                --image-type=cos_containerd \
+                --sandbox type=gvisor
+        scored: false
+
+      - id: 6.10.5
+        text: "Ensure use of Binary Authorization (Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            Firstly, update the cluster to enable Binary Authorization:
+
+              gcloud container cluster update [CLUSTER_NAME] \
+                --zone [COMPUTE-ZONE] \
+                --enable-binauthz
+
+            Create a Binary Authorization Policy using the Binary Authorization Policy Reference
+            (https://cloud.google.com/binary-authorization/docs/policy-yaml-reference) for
+            guidance.
+
+            Import the policy file into Binary Authorization:
+
+              gcloud container binauthz policy import [YAML_POLICY]
+        scored: true
+
+      - id: 6.10.6
+        text: "Enable Cloud Security Command Center (Cloud SCC) (Not Scored)"
+        type: "manual"
+        remediation: |
+          Using Command Line:
+            Follow the instructions at https://cloud.google.com/security-command-
+            center/docs/quickstart-scc-setup.
+        scored: false

cfg/gke-1.0/master.yaml

@@ -0,0 +1,348 @@
+---
+controls:
+version: "gke-1.0"
+id: 1
+text: "Control Plane Components"
+type: "master"
+groups:
+  - id: 1.1
+    text: "Master Node Configuration Files "
+    checks:
+      - id: 1.1.1
+        text: "Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.2
+        text: "Ensure that the API server pod specification file ownership is set to root:root (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.3
+        text: "Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.4
+        text: "Ensure that the controller manager pod specification file ownership is set to root:root (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.5
+        text: "Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.6
+        text: "Ensure that the scheduler pod specification file ownership is set to root:root (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.7
+        text: "Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.8
+        text: "Ensure that the etcd pod specification file ownership is set to root:root (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.9
+        text: "Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.10
+        text: "Ensure that the Container Network Interface file ownership is set to root:root (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.11
+        text: "Ensure that the etcd data directory permissions are set to 700 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.12
+        text: "Ensure that the etcd data directory ownership is set to etcd:etcd (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.13
+        text: "Ensure that the admin.conf file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.14
+        text: "Ensure that the admin.conf file ownership is set to root:root (Not Scored) "
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.15
+        text: "Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: true
+
+      - id: 1.1.16
+        text: "Ensure that the scheduler.conf file ownership is set to root:root (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.17
+        text: "Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.18
+        text: "Ensure that the controller-manager.conf file ownership is set to root:root (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.19
+        text: "Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.20
+        text: "Ensure that the Kubernetes PKI certificate file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.1.21
+        text: "Ensure that the Kubernetes PKI key file permissions are set to 600 (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+  - id: 1.2
+    text: "API Server"
+    checks:
+      - id: 1.2.1
+        text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.2
+        text: "Ensure that the --basic-auth-file argument is not set (Not Scored)"
+        remediation: |
+          Although the use of the --basic-auth-file argument cannot be audited on GKE, you can
+          remediate the use of basic authentication. See Recommendation 6.8.1.
+        scored: false
+
+      - id: 1.2.3
+        text: "Ensure that the --token-auth-file parameter is not set (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.4
+        text: "Ensure that the --kubelet-https argument is set to true (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.5
+        text: "Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.6
+        text: "Ensure that the --kubelet-certificate-authority argument is set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.7
+        text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.8
+        text: "Ensure that the --authorization-mode argument includes Node (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.9
+        text: "Ensure that the --authorization-mode argument includes RBAC (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.10
+        text: "Ensure that the admission control plugin EventRateLimit is set (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.11
+        text: "Ensure that the admission control plugin AlwaysAdmit is not set (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.12
+        text: "Ensure that the admission control plugin AlwaysPullImages is set (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.13
+        text: "Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.14
+        text: "Ensure that the admission control plugin ServiceAccount is set (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.15
+        text: "Ensure that the admission control plugin NamespaceLifecycle is set (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.16
+        text: "Ensure that the admission control plugin PodSecurityPolicy is set (Not Scored)"
+        remediation: |
+          To verify and remediate the use of Pod Security Policy on GKE, see Recommendation 6.10.3.
+        scored: false
+
+      - id: 1.2.17
+        text: "Ensure that the admission control plugin NodeRestriction is set (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.18
+        text: "Ensure that the --insecure-bind-address argument is not set (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.19
+        text: "Ensure that the --insecure-port argument is set to 0 (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.20
+        text: "Ensure that the --secure-port argument is not set to 0 (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.21
+        text: "Ensure that the --profiling argument is set to false (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.22
+        text: "Ensure that the --audit-log-path argument is set (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.23
+        text: "Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.24
+        text: "Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.25
+        text: "Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.26
+        text: "Ensure that the --request-timeout argument is set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.27
+        text: "Ensure that the --service-account-lookup argument is set to true (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.28
+        text: "Ensure that the --service-account-key-file argument is set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.29
+        text: "Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.30
+        text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.31
+        text: "Ensure that the --client-ca-file argument is set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.32
+        text: "Ensure that the --etcd-cafile argument is set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.2.33
+        text: "Ensure that the --encryption-provider-config argument is set as appropriate (Not Scored)"
+        remediation: |
+          To verify and remediate the use of secret encryption on GKE, see Recommendation 6.3.1.
+        scored: false
+
+      - id: 1.2.34
+        text: "Ensure that encryption providers are appropriately configured (Not Scored)"
+        remediation: |
+          To verify and remediate the use of secret encryption on GKE, see Recommendation 6.3.1.
+        scored: false
+
+      - id: 1.2.35
+        text: "Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+  - id: 1.3
+    text: "Controller Manager"
+    checks:
+      - id: 1.3.1
+        text: "Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.3.2
+        text: "Ensure that the --profiling argument is set to false (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.3.3
+        text: "Ensure that the --use-service-account-credentials argument is set to true (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.3.4
+        text: "Ensure that the --service-account-private-key-file argument is set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.3.5
+        text: "Ensure that the --root-ca-file argument is set as appropriate (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.3.6
+        text: "Ensure that the RotateKubeletServerCertificate argument is set to true (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.3.7
+        text: "Ensure that the --bind-address argument is set to 127.0.0.1 (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+  - id: 1.4
+    text: "Scheduler"
+    checks:
+      - id: 1.4.1
+        text: "Ensure that the --profiling argument is set to false (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 1.4.2
+        text: "Ensure that the --bind-address argument is set to 127.0.0.1 (Not Scored) "
+        remediation: "This control cannot be modified in GKE."
+        scored: false

cfg/gke-1.0/node.yaml

@@ -0,0 +1,449 @@
+---
+controls:
+version: "gke-1.0"
+id: 4
+text: "Worker Node Security Configuration"
+type: "node"
+groups:
+  - id: 4.1
+    text: "Worker Node Configuration Files"
+    checks:
+      - id: 4.1.1
+        text: "Ensure that the kubelet service file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 4.1.2
+        text: "Ensure that the kubelet service file ownership is set to root:root (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 4.1.3
+        text: "Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)"
+        audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c %a $proxykubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: "644"
+              compare:
+                op: eq
+                value: "644"
+              set: true
+            - flag: "640"
+              compare:
+                op: eq
+                value: "640"
+              set: true
+            - flag: "600"
+              compare:
+                op: eq
+                value: "600"
+              set: true
+            - flag: "444"
+              compare:
+                op: eq
+                value: "444"
+              set: true
+            - flag: "440"
+              compare:
+                op: eq
+                value: "440"
+              set: true
+            - flag: "400"
+              compare:
+                op: eq
+                value: "400"
+              set: true
+            - flag: "000"
+              compare:
+                op: eq
+                value: "000"
+              set: true
+          bin_op: or
+        remediation: |
+          Run the below command (based on the file location on your system) on each worker node.
+          For example,
+          chmod 644 $proxykubeconfig
+        scored: true
+
+      - id: 4.1.4
+        text: "Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)"
+        audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c %U:%G $proxykubeconfig; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+              set: true
+        remediation: |
+          Run the below command (based on the file location on your system) on each worker node.
+          For example, chown root:root $proxykubeconfig
+        scored: true
+
+      - id: 4.1.5
+        text: "Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 4.1.6
+        text: "Ensure that the kubelet.conf file ownership is set to root:root (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 4.1.7
+        text: "Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 4.1.8
+        text: "Ensure that the client certificate authorities file ownership is set to root:root (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false
+
+      - id: 4.1.9
+        text: "Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)"
+        audit: '/bin/sh -c ''if test -e $kubeletconf; then stat -c %a $kubeletconf; fi'' '
+        tests:
+          test_items:
+            - flag: "644"
+              set: true
+              compare:
+                op: eq
+                value: "644"
+            - flag: "640"
+              set: true
+              compare:
+                op: eq
+                value: "640"
+            - flag: "600"
+              set: true
+              compare:
+                op: eq
+                value: "600"
+            - flag: "444"
+              compare:
+                op: eq
+                value: "444"
+              set: true
+            - flag: "440"
+              compare:
+                op: eq
+                value: "440"
+              set: true
+            - flag: "400"
+              compare:
+                op: eq
+                value: "400"
+              set: true
+            - flag: "000"
+              compare:
+                op: eq
+                value: "000"
+              set: true
+          bin_op: or
+        remediation: |
+          Run the following command (using the config file location identified in the Audit step)
+          chmod 644 $kubeletconf
+        scored: true
+
+      - id: 4.1.10
+        text: "Ensure that the kubelet configuration file ownership is set to root:root (Scored)"
+        audit: '/bin/sh -c ''if test -e $kubeletconf; then stat -c %U:%G $kubeletconf; fi'' '
+        tests:
+          test_items:
+            - flag: root:root
+              set: true
+        remediation: |
+          Run the following command (using the config file location identified in the Audit step)
+          chown root:root $kubeletconf
+        scored: true
+
+  - id: 4.2
+    text: "Kubelet"
+    checks:
+      - id: 4.2.1
+        text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: "--anonymous-auth"
+              path: '{.authentication.anonymous.enabled}'
+              set: true
+              compare:
+                op: eq
+                value: false
+        remediation: |
+          If using a Kubelet config file, edit the file to set authentication: anonymous: enabled to
+          false.
+          If using executable arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --anonymous-auth=false
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.2
+        text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --authorization-mode
+              path: '{.authorization.mode}'
+              set: true
+              compare:
+                op: nothave
+                value: AlwaysAllow
+        remediation: |
+          If using a Kubelet config file, edit the file to set authorization: mode to Webhook. If
+          using executable arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_AUTHZ_ARGS variable.
+          --authorization-mode=Webhook
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.3
+        text: "Ensure that the --client-ca-file argument is set as appropriate (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --client-ca-file
+              path: '{.authentication.x509.clientCAFile}'
+              set: true
+        remediation: |
+          If using a Kubelet config file, edit the file to set authentication: x509: clientCAFile to
+          the location of the client CA file.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_AUTHZ_ARGS variable.
+          --client-ca-file=<path/to/client-ca-file>
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.4
+        text: "Ensure that the --read-only-port argument is set to 0 (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: "--read-only-port"
+              path: '{.readOnlyPort}'
+              set: true
+              compare:
+                op: eq
+                value: 0
+        remediation: |
+          If using a Kubelet config file, edit the file to set readOnlyPort to 0.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --read-only-port=0
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.5
+        text: "Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --streaming-connection-idle-timeout
+              path: '{.streamingConnectionIdleTimeout}'
+              set: true
+              compare:
+                op: noteq
+                value: 0
+            - flag: --streaming-connection-idle-timeout
+              path: '{.streamingConnectionIdleTimeout}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to set streamingConnectionIdleTimeout to a
+          value other than 0.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --streaming-connection-idle-timeout=5m
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.6
+        text: "Ensure that the --protect-kernel-defaults argument is set to true (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --protect-kernel-defaults
+              path: '{.protectKernelDefaults}'
+              set: true
+              compare:
+                op: eq
+                value: true
+        remediation: |
+          If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          --protect-kernel-defaults=true
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.7
+        text: "Ensure that the --make-iptables-util-chains argument is set to true (Scored) "
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --make-iptables-util-chains
+              path: '{.makeIPTablesUtilChains}'
+              set: true
+              compare:
+                op: eq
+                value: true
+            - flag: --make-iptables-util-chains
+              path: '{.makeIPTablesUtilChains}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to set makeIPTablesUtilChains: true.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          remove the --make-iptables-util-chains argument from the
+          KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.8
+        text: "Ensure that the --hostname-override argument is not set (Scored)"
+        # This is one of those properties that can only be set as a command line argument.
+        # To check if the property is set as expected, we need to parse the kubelet command
+        # instead reading the Kubelet Configuration file.
+        audit: "/bin/ps -fC $kubeletbin "
+        tests:
+          test_items:
+            - flag: --hostname-override
+              set: false
+        remediation: |
+          Edit the kubelet service file $kubeletsvc
+          on each worker node and remove the --hostname-override argument from the
+          KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.9
+        text: "Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --event-qps
+              path: '{.eventRecordQPS}'
+              set: true
+              compare:
+                op: eq
+                value: 0
+        remediation: |
+          If using a Kubelet config file, edit the file to set eventRecordQPS: to an appropriate level.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.10
+        text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --tls-cert-file
+              path: '{.tlsCertFile}'
+              set: true
+            - flag: --tls-private-key-file
+              path: '{.tlsPrivateKeyFile}'
+              set: true
+        remediation: |
+          If using a Kubelet config file, edit the file to set tlsCertFile to the location
+          of the certificate file to use to identify this Kubelet, and tlsPrivateKeyFile
+          to the location of the corresponding private key file.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          set the below parameters in KUBELET_CERTIFICATE_ARGS variable.
+          --tls-cert-file=<path/to/tls-certificate-file>
+          --tls-private-key-file=<path/to/tls-key-file>
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.11
+        text: "Ensure that the --rotate-certificates argument is not set to false (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: --rotate-certificates
+              path: '{.rotateCertificates}'
+              set: true
+              compare:
+                op: eq
+                value: true
+            - flag: --rotate-certificates
+              path: '{.rotateCertificates}'
+              set: false
+          bin_op: or
+        remediation: |
+          If using a Kubelet config file, edit the file to add the line rotateCertificates: true or
+          remove it altogether to use the default value.
+          If using command line arguments, edit the kubelet service file
+          $kubeletsvc on each worker node and
+          remove --rotate-certificates=false argument from the KUBELET_CERTIFICATE_ARGS
+          variable.
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.12
+        text: "Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)"
+        audit: "/bin/ps -fC $kubeletbin"
+        audit_config: "/bin/cat $kubeletconf"
+        tests:
+          test_items:
+            - flag: RotateKubeletServerCertificate
+              path: '{.featureGates.RotateKubeletServerCertificate}'
+              set: true
+              compare:
+                op: eq
+                value: true
+        remediation: |
+          Edit the kubelet service file $kubeletsvc
+          on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
+          --feature-gates=RotateKubeletServerCertificate=true
+          Based on your system, restart the kubelet service. For example:
+          systemctl daemon-reload
+          systemctl restart kubelet.service
+        scored: true
+
+      - id: 4.2.13
+        text: "Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)"
+        remediation: "This control cannot be modified in GKE."
+        scored: false

cfg/gke-1.0/policies.yaml

@@ -0,0 +1,239 @@
+---
+controls:
+version: "gke-1.0"
+id: 5
+text: "Kubernetes Policies"
+type: "policies"
+groups:
+  - id: 5.1
+    text: "RBAC and Service Accounts"
+    checks:
+      - id: 5.1.1
+        text: "Ensure that the cluster-admin role is only used where required (Not Scored)"
+        type: "manual"
+        remediation: |
+          Identify all clusterrolebindings to the cluster-admin role. Check if they are used and
+          if they need this role or if they could use a role with fewer privileges.
+          Where possible, first bind users to a lower privileged role and then remove the
+          clusterrolebinding to the cluster-admin role :
+          kubectl delete clusterrolebinding [name]
+        scored: false
+
+      - id: 5.1.2
+        text: "Minimize access to secrets (Not Scored)"
+        type: "manual"
+        remediation: |
+          Where possible, remove get, list and watch access to secret objects in the cluster.
+        scored: false
+
+      - id: 5.1.3
+        text: "Minimize wildcard use in Roles and ClusterRoles (Not Scored)"
+        type: "manual"
+        remediation: |
+          Where possible replace any use of wildcards in clusterroles and roles with specific
+          objects or actions.
+        scored: false
+
+      - id: 5.1.4
+        text: "Minimize access to create pods (Not Scored)"
+        type: "manual"
+        Remediation: |
+          Where possible, remove create access to pod objects in the cluster.
+        scored: false
+
+      - id: 5.1.5
+        text: "Ensure that default service accounts are not actively used. (Scored)"
+        type: "manual"
+        remediation: |
+          Create explicit service accounts wherever a Kubernetes workload requires specific access
+          to the Kubernetes API server.
+          Modify the configuration of each default service account to include this value
+          automountServiceAccountToken: false
+        scored: true
+
+      - id: 5.1.6
+        text: "Ensure that Service Account Tokens are only mounted where necessary (Not Scored)"
+        type: "manual"
+        remediation: |
+          Modify the definition of pods and service accounts which do not need to mount service
+          account tokens to disable it.
+        scored: false
+
+  - id: 5.2
+    text: "Pod Security Policies"
+    checks:
+      - id: 5.2.1
+        text: "Minimize the admission of privileged containers (Not Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that
+          the .spec.privileged field is omitted or set to false.
+        scored: false
+
+      - id: 5.2.2
+        text: "Minimize the admission of containers wishing to share the host process ID namespace (Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostPID field is omitted or set to false.
+        scored: true
+
+      - id: 5.2.3
+        text: "Minimize the admission of containers wishing to share the host IPC namespace (Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostIPC field is omitted or set to false.
+        scored: true
+
+      - id: 5.2.4
+        text: "Minimize the admission of containers wishing to share the host network namespace (Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.hostNetwork field is omitted or set to false.
+        scored: true
+
+      - id: 5.2.5
+        text: "Minimize the admission of containers with allowPrivilegeEscalation (Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.allowPrivilegeEscalation field is omitted or set to false.
+        scored: true
+
+      - id: 5.2.6
+        text: "Minimize the admission of root containers (Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.runAsUser.rule is set to either MustRunAsNonRoot or MustRunAs with the range of
+          UIDs not including 0.
+        scored: true
+
+      - id: 5.2.7
+        text: "Minimize the admission of containers with the NET_RAW capability (Scored)"
+        type: "manual"
+        remediation: |
+          Create a PSP as described in the Kubernetes documentation, ensuring that the
+          .spec.requiredDropCapabilities is set to include either NET_RAW or ALL.
+        scored: true
+
+      - id: 5.2.8
+        text: "Minimize the admission of containers with added capabilities (Scored)"
+        type: "manual"
+        remediation: |
+          Ensure that allowedCapabilities is not present in PSPs for the cluster unless
+          it is set to an empty array.
+        scored: true
+
+      - id: 5.2.9
+        text: "Minimize the admission of containers with capabilities assigned (Scored) "
+        type: "manual"
+        remediation: |
+          Review the use of capabilites in applications runnning on your cluster. Where a namespace
+          contains applications which do not require any Linux capabities to operate consider adding
+          a PSP which forbids the admission of containers which do not drop all capabilities.
+        scored: true
+
+  - id: 5.3
+    text: "Network Policies and CNI"
+    checks:
+      - id: 5.3.1
+        text: "Ensure that the CNI in use supports Network Policies (Not Scored)"
+        type: "manual"
+        remediation: |
+          To use a CNI plugin with Network Policy, enable Network Policy in GKE, and the CNI plugin
+          will be updated. See Recommendation 6.6.7.
+        scored: false
+
+      - id: 5.3.2
+        text: "Ensure that all Namespaces have Network Policies defined (Scored)"
+        type: "manual"
+        remediation: |
+          Follow the documentation and create NetworkPolicy objects as you need them.
+        scored: true
+
+  - id: 5.4
+    text: "Secrets Management"
+    checks:
+      - id: 5.4.1
+        text: "Prefer using secrets as files over secrets as environment variables (Not Scored)"
+        type: "manual"
+        remediation: |
+          if possible, rewrite application code to read secrets from mounted secret files, rather than
+          from environment variables.
+        scored: false
+
+      - id: 5.4.2
+        text: "Consider external secret storage (Not Scored)"
+        type: "manual"
+        remediation: |
+          Refer to the secrets management options offered by your cloud provider or a third-party
+          secrets management solution.
+        scored: false
+
+  - id: 5.5
+    text: "Extensible Admission Control"
+    checks:
+      - id: 5.5.1
+        text: "Configure Image Provenance using ImagePolicyWebhook admission controller (Not Scored)"
+        type: "manual"
+        remediation: |
+          Follow the Kubernetes documentation and setup image provenance.
+          See also Recommendation 6.10.5 for GKE specifically.
+        scored: false
+
+  - id: 5.6
+    text: "General Policies"
+    checks:
+      - id: 5.6.1
+        text: "Create administrative boundaries between resources using namespaces (Not Scored)"
+        type: "manual"
+        remediation: |
+          Follow the documentation and create namespaces for objects in your deployment as you need
+          them.
+        scored: false
+
+      - id: 5.6.2
+        text: "Ensure that the seccomp profile is set to docker/default in your pod definitions (Not Scored)"
+        type: "manual"
+        remediation: |
+          Seccomp is an alpha feature currently. By default, all alpha features are disabled. So, you
+          would need to enable alpha features in the apiserver by passing "--feature-
+          gates=AllAlpha=true" argument.
+          Edit the /etc/kubernetes/apiserver file on the master node and set the KUBE_API_ARGS
+          parameter to "--feature-gates=AllAlpha=true"
+          KUBE_API_ARGS="--feature-gates=AllAlpha=true"
+          Based on your system, restart the kube-apiserver service. For example:
+          systemctl restart kube-apiserver.service
+          Use annotations to enable the docker/default seccomp profile in your pod definitions. An
+          example is as below:
+          apiVersion: v1
+          kind: Pod
+          metadata:
+            name: trustworthy-pod
+            annotations:
+              seccomp.security.alpha.kubernetes.io/pod: docker/default
+          spec:
+            containers:
+              - name: trustworthy-container
+                image: sotrustworthy:latest
+        scored: false
+
+      - id: 5.6.3
+        text: "Apply Security Context to Your Pods and Containers (Not Scored)"
+        type: "manual"
+        remediation: |
+          Follow the Kubernetes documentation and apply security contexts to your pods. For a
+          suggested list of security contexts, you may refer to the CIS Security Benchmark for Docker
+          Containers.
+        scored: false
+
+      - id: 5.6.4
+        text: "The default namespace should not be used (Scored)"
+        type: "manual"
+        remediation: |
+          Ensure that namespaces are created to allow for appropriate segregation of Kubernetes
+          resources and that all new resources are created in a specific namespace.
+        scored: true

cfg/rh-0.7/config.yaml

@@ -0,0 +1,32 @@
+---
+## Version-specific settings that override the values in cfg/config.yaml
+
+master:
+  apiserver:
+    bins:
+      - openshift start master api
+      - hypershift openshift-kube-apiserver
+
+  scheduler:
+    bins:
+      - "openshift start master controllers"
+      - "hyperkube kube-scheduler"
+    confs:
+      - /etc/origin/master/scheduler.json
+
+  controllermanager:
+    bins:
+      - "openshift start master controllers"
+      - "hypershift openshift-controller-manager"
+
+  etcd:
+    bins:
+      - openshift start etcd
+
+node:
+  svcs:
+    - /etc/systemd/system/atomic-openshift-node.service
+    - /etc/systemd/system/origin-node.service
+  proxy:
+    bins:
+      - openshift start network

cfg/rh-0.7/node.yaml

@@ -0,0 +1,332 @@
+---
+controls:
+id: 2
+text: "Worker Node Security Configuration"
+type: "node"
+groups:
+  - id: 7
+    text: "Kubelet"
+    checks:
+      - id: 7.1
+        text: "Use Security Context Constraints to manage privileged containers as needed"
+        type: "skip"
+        scored: true
+
+      - id: 7.2
+        text: "Ensure anonymous-auth is not disabled"
+        type: "skip"
+        scored: true
+
+      - id: 7.3
+        text: "Verify that the --authorization-mode argument is set to WebHook"
+        audit: "grep -A1 authorization-mode /etc/origin/node/node-config.yaml"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "authorization-mode"
+              set: false
+            - flag: "authorization-mode"
+              compare:
+                op: has
+                value: "Webhook"
+              set: true
+        remediation: |
+          Edit the Openshift node config file /etc/origin/node/node-config.yaml and remove authorization-mode under
+          kubeletArguments in /etc/origin/node/node-config.yaml or set it to "Webhook".
+        scored: true
+
+      - id: 7.4
+        text: "Verify the OpenShift default for the client-ca-file argument"
+        audit: "grep -A1 client-ca-file /etc/origin/node/node-config.yaml"
+        tests:
+          test_items:
+            - flag: "client-ca-file"
+              set: false
+        remediation: |
+          Edit the Openshift node config file /etc/origin/node/node-config.yaml and remove any configuration returned by the following:
+          grep -A1 client-ca-file /etc/origin/node/node-config.yaml
+
+          Reset to the OpenShift default.
+          See https://github.com/openshift/openshift-ansible/blob/release-3.10/roles/openshift_node_group/templates/node-config.yaml.j2#L65
+          The config file does not have this defined in kubeletArgument, but in PodManifestConfig.
+        scored: true
+
+      - id: 7.5
+        text: "Verify the OpenShift default setting for the read-only-port argument"
+        audit: "grep -A1 read-only-port /etc/origin/node/node-config.yaml"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "read-only-port"
+              set: false
+            - flag: "read-only-port"
+              compare:
+                op: has
+                value: "0"
+              set: true
+        remediation: |
+          Edit the Openshift node config file /etc/origin/node/node-config.yaml and removed so that the OpenShift default is applied.
+        scored: true
+
+      - id: 7.6
+        text: "Adjust the streaming-connection-idle-timeout argument"
+        audit: "grep -A1 streaming-connection-idle-timeout /etc/origin/node/node-config.yaml"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "streaming-connection-idle-timeout"
+              set: false
+            - flag: "5m"
+              set: false
+        remediation: |
+          Edit the Openshift node config file /etc/origin/node/node-config.yaml and set the streaming-connection-timeout
+          value like the following in node-config.yaml.
+
+          kubeletArguments:
+            streaming-connection-idle-timeout:
+               - "5m"
+        scored: true
+
+      - id: 7.7
+        text: "Verify the OpenShift defaults for the protect-kernel-defaults argument"
+        type: "skip"
+        scored: true
+
+      - id: 7.8
+        text: "Verify the OpenShift default value of true for the make-iptables-util-chains argument"
+        audit: "grep -A1 make-iptables-util-chains /etc/origin/node/node-config.yaml"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "make-iptables-util-chains"
+              set: false
+            - flag: "make-iptables-util-chains"
+              compare:
+                op: has
+                value: "true"
+              set: true
+        remediation: |
+          Edit the Openshift node config file /etc/origin/node/node-config.yaml and reset make-iptables-util-chains to the OpenShift
+          default value of true.
+        scored: true
+
+      - id: 7.9
+        text: "Verify that the --keep-terminated-pod-volumes argument is set to false"
+        audit: "grep -A1 keep-terminated-pod-volumes /etc/origin/node/node-config.yaml"
+        tests:
+          test_items:
+            - flag: "keep-terminated-pod-volumes"
+              compare:
+                op: has
+                value: "false"
+              set: true
+        remediation: |
+          Reset to the OpenShift defaults
+        scored: true
+
+      - id: 7.10
+        text: "Verify the OpenShift defaults for the hostname-override argument"
+        type: "skip"
+        scored: true
+
+      - id: 7.11
+        text: "Set the --event-qps argument to 0"
+        audit: "grep -A1 event-qps /etc/origin/node/node-config.yaml"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "event-qps"
+              set: false
+            - flag: "event-qps"
+              compare:
+                op: has
+                value: "0"
+              set: true
+        remediation: |
+          Edit the Openshift node config file /etc/origin/node/node-config.yaml set the event-qps argument to 0 in
+          the kubeletArguments section of.
+        scored: true
+
+      - id: 7.12
+        text: "Verify the OpenShift cert-dir flag for HTTPS traffic"
+        audit: "grep -A1 cert-dir /etc/origin/node/node-config.yaml"
+        tests:
+          test_items:
+            - flag: "/etc/origin/node/certificates"
+              compare:
+                op: has
+                value: "/etc/origin/node/certificates"
+              set: true
+        remediation: |
+          Reset to the OpenShift default values.
+        scored: true
+
+      - id: 7.13
+        text: "Verify the OpenShift default of 0 for the cadvisor-port argument"
+        audit: "grep -A1 cadvisor-port /etc/origin/node/node-config.yaml"
+        tests:
+          bin_op: or
+          test_items:
+            - flag: "cadvisor-port"
+              set: false
+            - flag: "cadvisor-port"
+              compare:
+                op: has
+                value: "0"
+              set: true
+        remediation: |
+          Edit the Openshift node config file /etc/origin/node/node-config.yaml and remove the cadvisor-port flag
+          if it is set in the  kubeletArguments section.
+        scored: true
+
+      - id: 7.14
+        text: "Verify that the RotateKubeletClientCertificate argument is set to true"
+        audit: "grep -B1 RotateKubeletClientCertificate=true /etc/origin/node/node-config.yaml"
+        tests:
+          test_items:
+            - flag: "RotateKubeletClientCertificate=true"
+              compare:
+                op: has
+                value: "true"
+              set: true
+        remediation: |
+          Edit the Openshift node config file /etc/origin/node/node-config.yaml and set RotateKubeletClientCertificate to true.
+        scored: true
+
+      - id: 7.15
+        text: "Verify that the RotateKubeletServerCertificate argument is set to true"
+        audit: "grep -B1 RotateKubeletServerCertificate=true /etc/origin/node/node-config.yaml"
+        tests:
+          test_items:
+            - flag: "RotateKubeletServerCertificate=true"
+              compare:
+                op: has
+                value: "true"
+              set: true
+        remediation: |
+          Edit the Openshift node config file /etc/origin/node/node-config.yaml and set RotateKubeletServerCertificate to true.
+        scored: true
+
+
+  - id: 8
+    text: "Configuration Files"
+    checks:
+      - id: 8.1
+        text: "Verify the OpenShift default permissions for the kubelet.conf file"
+        audit: "stat -c permissions=%a  /etc/origin/node/node.kubeconfig"
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+              set: true
+        remediation: |
+          Run the below command on each worker node.
+          chmod 644 /etc/origin/node/node.kubeconfig
+        scored: true
+
+      - id: 8.2
+        text: "Verify the kubeconfig file ownership of root:root"
+        audit: "stat -c %U:%G /etc/origin/node/node.kubeconfig"
+        tests:
+          test_items:
+            - flag: "root:root"
+              compare:
+                op: eq
+                value: root:root
+              set: true
+          remediation: |
+            Run the below command on each worker node.
+            chown root:root /etc/origin/node/node.kubeconfig
+          scored: true
+
+      - id: 8.3
+        text: "Verify the kubelet service file permissions of 644"
+        audit: "stat -c permissions=%a $nodesvc"
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+              set: true
+        remediation: |
+          Run the below command on each worker node.
+          chmod 644 $nodesvc
+        scored: true
+
+      - id: 8.4
+        text: "Verify the kubelet service file ownership of root:root"
+        audit: "stat -c %U:%G $nodesvc"
+        tests:
+          test_items:
+            - flag: "root:root"
+              compare:
+                op: eq
+                value: root:root
+              set: true
+          remediation: |
+            Run the below command on each worker node.
+            chown root:root $nodesvc
+          scored: true
+
+      - id: 8.5
+        text: "Verify the OpenShift default permissions for the proxy kubeconfig file"
+        audit: "stat -c permissions=%a /etc/origin/node/node.kubeconfig"
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+              set: true
+        remediation: |
+          Run the below command on each worker node.
+          chmod 644 /etc/origin/node/node.kubeconfig
+        scored: true
+
+      - id: 8.6
+        text: "Verify the proxy kubeconfig file ownership of root:root"
+        audit: "stat -c %U:%G /etc/origin/node/node.kubeconfig"
+        tests:
+          test_items:
+            - flag: "root:root"
+              compare:
+                op: eq
+                value: root:root
+              set: true
+          remediation: |
+            Run the below command on each worker node.
+            chown root:root /etc/origin/node/node.kubeconfig
+          scored: true
+
+      - id: 8.7
+        text: "Verify the OpenShift default permissions for the certificate authorities file."
+        audit: "stat -c permissions=%a /etc/origin/node/client-ca.crt"
+        tests:
+          test_items:
+            - flag: "permissions"
+              compare:
+                op: bitmask
+                value: "644"
+              set: true
+        remediation: |
+          Run the below command on each worker node.
+          chmod 644 /etc/origin/node/client-ca.crt
+        scored: true
+
+      - id: 8.8
+        text: "Verify the client certificate authorities file ownership of root:root"
+        audit: "stat -c %U:%G /etc/origin/node/client-ca.crt"
+        tests:
+          test_items:
+            - flag: "root:root"
+              compare:
+                op: eq
+                value: root:root
+              set: true
+          remediation: |
+            Run the below command on each worker node.
+            chown root:root /etc/origin/node/client-ca.crt
+          scored: true

check/check.go

@@ -17,16 +17,13 @@ package check
 import (
 	"bytes"
 	"fmt"
-	"io"
-	"os"
 	"os/exec"
-	"regexp"
 	"strings"
 
 	"github.com/golang/glog"
 )
 
-// NodeType indicates the type of node (master, node, federated).
+// NodeType indicates the type of node (master, node).
 type NodeType string
 
 // State is the state of a control check.
@@ -36,11 +33,11 @@ const (
 	// PASS check passed.
 	PASS State = "PASS"
 	// FAIL check failed.
-	FAIL = "FAIL"
+	FAIL State = "FAIL"
 	// WARN could not carry out check.
-	WARN = "WARN"
+	WARN State = "WARN"
 	// INFO informational message
-	INFO = "INFO"
+	INFO State = "INFO"
 
 	// MASTER a master node
 	MASTER NodeType = "master"
@@ -48,184 +45,227 @@ const (
 	NODE NodeType = "node"
 	// FEDERATED a federated deployment.
 	FEDERATED NodeType = "federated"
+
+	// ETCD an etcd node
+	ETCD NodeType = "etcd"
+	// CONTROLPLANE a control plane node
+	CONTROLPLANE NodeType = "controlplane"
+	// POLICIES a node to run policies from
+	POLICIES NodeType = "policies"
+	// MANAGEDSERVICES a node to run managedservices from
+	MANAGEDSERVICES = "managedservices"
+
+	// MANUAL Check Type
+	MANUAL string = "manual"
 )
 
-func handleError(err error, context string) (errmsg string) {
-	if err != nil {
-		errmsg = fmt.Sprintf("%s, error: %s\n", context, err)
-	}
-	return
+// Check contains information about a recommendation in the
+// CIS Kubernetes document.
+type Check struct {
+	ID                string   `yaml:"id" json:"test_number"`
+	Text              string   `json:"test_desc"`
+	Audit             string   `json:"audit"`
+	AuditConfig       string   `yaml:"audit_config"`
+	Type              string   `json:"type"`
+	Tests             *tests   `json:"omit"`
+	Set               bool     `json:"omit"`
+	Remediation       string   `json:"remediation"`
+	TestInfo          []string `json:"test_info"`
+	State             `json:"status"`
+	ActualValue       string `json:"actual_value"`
+	Scored            bool   `json:"scored"`
+	IsMultiple        bool   `yaml:"use_multiple_values"`
+	ExpectedResult    string `json:"expected_result"`
+	Reason            string `json:"reason,omitempty"`
+	AuditOutput       string `json:"omit"`
+	AuditConfigOutput string `json:"omit"`
 }
 
-// Check contains information about a recommendation in the
-// CIS Kubernetes 1.6+ document.
-type Check struct {
-	ID          string 		`yaml:"id" json:"test_number"`
-	Text        string		`json:"test_desc"`
-	Audit       string      `json:"omit"`
-	Type        string      `json:"type"`
-	Commands    []*exec.Cmd `json:"omit"`
-	Tests       *tests      `json:"omit"`
-	Set         bool        `json:"omit"`
-	Remediation string 		`json:"-"`
-	TestInfo    []string 	`json:"test_info"`
-	State 					`json:"status"`
+// Runner wraps the basic Run method.
+type Runner interface {
+	// Run runs a given check and returns the execution state.
+	Run(c *Check) State
+}
+
+// NewRunner constructs a default Runner.
+func NewRunner() Runner {
+	return &defaultRunner{}
+}
+
+type defaultRunner struct{}
+
+func (r *defaultRunner) Run(c *Check) State {
+	return c.run()
 }
 
 // Run executes the audit commands specified in a check and outputs
 // the results.
-func (c *Check) Run() {
-	// If check type is manual, force result to WARN.
-	if c.Type == "manual" {
+func (c *Check) run() State {
+
+	// Since this is an Scored check
+	// without tests return a 'WARN' to alert
+	// the user that this check needs attention
+	if c.Scored && strings.TrimSpace(c.Type) == "" && c.Tests == nil {
+		c.Reason = "There are no tests"
 		c.State = WARN
-		return
+		return c.State
 	}
 
-	var out bytes.Buffer
-	var errmsgs string
-
-	// Check if command exists or exit with WARN.
-	for _, cmd := range c.Commands {
-		if !isShellCommand(cmd.Path) {
-			c.State = WARN
-			return
-		}
+	// If check type is skip, force result to INFO
+	if c.Type == "skip" {
+		c.Reason = "Test marked as skip"
+		c.State = INFO
+		return c.State
 	}
 
-	// Run commands.
-	n := len(c.Commands)
-	if n == 0 {
-		// Likely a warning message.
+	// If check type is manual force result to WARN
+	if c.Type == MANUAL {
+		c.Reason = "Test marked as a manual test"
 		c.State = WARN
-		return
+		return c.State
 	}
 
-	// Each command runs,
-	//   cmd0 out -> cmd1 in, cmd1 out -> cmd2 in ... cmdn out -> os.stdout
-	//   cmd0 err should terminate chain
-	cs := c.Commands
-
-	// Initialize command pipeline
-	cs[n-1].Stdout = &out
-	i := 1
-
-	var err error
-	errmsgs = ""
-
-	for i < n {
-		cs[i-1].Stdout, err = cs[i].StdinPipe()
-		errmsgs += handleError(
-			err,
-			fmt.Sprintf("failed to run: %s\nfailed command: %s",
-				c.Audit,
-				cs[i].Args,
-			),
-		)
-		i++
-	}
-
-	// Start command pipeline
-	i = 0
-	for i < n {
-		err := cs[i].Start()
-		errmsgs += handleError(
-			err,
-			fmt.Sprintf("failed to run: %s\nfailed command: %s",
-				c.Audit,
-				cs[i].Args,
-			),
-		)
-		i++
-	}
-
-	// Complete command pipeline
-	i = 0
-	for i < n {
-		err := cs[i].Wait()
-		errmsgs += handleError(
-			err,
-			fmt.Sprintf("failed to run: %s\nfailed command:%s",
-				c.Audit,
-				cs[i].Args,
-			),
-		)
-
-		if i < n-1 {
-			cs[i].Stdout.(io.Closer).Close()
-		}
-
-		i++
-	}
-
-	if errmsgs != "" {
-		glog.V(2).Info(errmsgs)
-	}
-
-	res := c.Tests.execute(out.String())
-	if res {
-		c.State = PASS
-	} else {
-		c.State = FAIL
-	}
-}
-
-// textToCommand transforms an input text representation of commands to be
-// run into a slice of commands.
-// TODO: Make this more robust.
-func textToCommand(s string) []*exec.Cmd {
-	cmds := []*exec.Cmd{}
-
-	cp := strings.Split(s, "|")
-
-	for _, v := range cp {
-		v = strings.Trim(v, " ")
-
-		// TODO:
-		// GOAL: To split input text into arguments for exec.Cmd.
-		//
-		// CHALLENGE: The input text may contain quoted strings that
-		// must be passed as a unit to exec.Cmd.
-		// eg. bash -c 'foo bar'
-		// 'foo bar' must be passed as unit to exec.Cmd if not the command
-		// will fail when it is executed.
-		// eg. exec.Cmd("bash", "-c", "foo bar")
-		//
-		// PROBLEM: Current solution assumes the grouped string will always
-		// be at the end of the input text.
-		re := regexp.MustCompile(`^(.*)(['"].*['"])$`)
-		grps := re.FindStringSubmatch(v)
-
-		var cs []string
-		if len(grps) > 0 {
-			s := strings.Trim(grps[1], " ")
-			cs = strings.Split(s, " ")
-
-			s1 := grps[len(grps)-1]
-			s1 = strings.Trim(s1, "'\"")
-
-			cs = append(cs, s1)
+	// If there aren't any tests defined this is a FAIL or WARN
+	if c.Tests == nil || len(c.Tests.TestItems) == 0 {
+		c.Reason = "No tests defined"
+		if c.Scored {
+			c.State = FAIL
 		} else {
-			cs = strings.Split(v, " ")
+			c.State = WARN
+		}
+		return c.State
+	}
+
+	// Command line parameters override the setting in the config file, so if we get a good result from the Audit command that's all we need to run
+	var finalOutput *testOutput
+	var lastCommand string
+
+	lastCommand, err := c.runAuditCommands()
+	if err == nil {
+		finalOutput, err = c.execute()
+	}
+
+	if finalOutput != nil {
+		if finalOutput.testResult {
+			c.State = PASS
+		} else {
+			if c.Scored {
+				c.State = FAIL
+			} else {
+				c.State = WARN
+			}
 		}
 
-		cmd := exec.Command(cs[0], cs[1:]...)
-		cmds = append(cmds, cmd)
+		c.ActualValue = finalOutput.actualResult
+		c.ExpectedResult = finalOutput.ExpectedResult
 	}
 
-	return cmds
-}
-
-func isShellCommand(s string) bool {
-	cmd := exec.Command("/bin/sh", "-c", "command -v "+s)
-
-	out, err := cmd.Output()
 	if err != nil {
-		fmt.Fprintf(os.Stderr, "%s\n", err)
-		os.Exit(1)
+		c.Reason = err.Error()
+		if c.Scored {
+			c.State = FAIL
+		} else {
+			c.State = WARN
+		}
 	}
 
-	if strings.Contains(string(out), s) {
-		return true
+	if finalOutput != nil {
+		glog.V(3).Infof("Check.ID: %s Command: %q TestResult: %t State: %q \n", c.ID, lastCommand, finalOutput.testResult, c.State)
+	} else {
+		glog.V(3).Infof("Check.ID: %s Command: %q TestResult: <<EMPTY>> \n", c.ID, lastCommand)
 	}
-	return false
+
+	if c.Reason != "" {
+		glog.V(2).Info(c.Reason)
+	}
+	return c.State
+}
+
+func (c *Check) runAuditCommands() (lastCommand string, err error) {
+	// Run the audit command and auditConfig commands, if present
+	c.AuditOutput, err = runAudit(c.Audit)
+	if err != nil {
+		return c.Audit, err
+	}
+
+	c.AuditConfigOutput, err = runAudit(c.AuditConfig)
+	return c.AuditConfig, err
+}
+
+func (c *Check) execute() (finalOutput *testOutput, err error) {
+	finalOutput = &testOutput{}
+
+	ts := c.Tests
+	res := make([]testOutput, len(ts.TestItems))
+	expectedResultArr := make([]string, len(res))
+
+	glog.V(3).Infof("%d tests", len(ts.TestItems))
+	for i, t := range ts.TestItems {
+
+		t.isMultipleOutput = c.IsMultiple
+
+		// Try with the auditOutput first, and if that's not found, try the auditConfigOutput
+		t.isConfigSetting = false
+		result := *(t.execute(c.AuditOutput))
+		if !result.flagFound {
+			t.isConfigSetting = true
+			result = *(t.execute(c.AuditConfigOutput))
+		}
+		res[i] = result
+		expectedResultArr[i] = res[i].ExpectedResult
+	}
+
+	var result bool
+	// If no binary operation is specified, default to AND
+	switch ts.BinOp {
+	default:
+		glog.V(2).Info(fmt.Sprintf("unknown binary operator for tests %s\n", ts.BinOp))
+		finalOutput.actualResult = fmt.Sprintf("unknown binary operator for tests %s\n", ts.BinOp)
+		return finalOutput, fmt.Errorf("unknown binary operator for tests %s", ts.BinOp)
+	case and, "":
+		result = true
+		for i := range res {
+			result = result && res[i].testResult
+		}
+		// Generate an AND expected result
+		finalOutput.ExpectedResult = strings.Join(expectedResultArr, " AND ")
+
+	case or:
+		result = false
+		for i := range res {
+			result = result || res[i].testResult
+		}
+		// Generate an OR expected result
+		finalOutput.ExpectedResult = strings.Join(expectedResultArr, " OR ")
+	}
+
+	finalOutput.testResult = result
+	finalOutput.actualResult = res[0].actualResult
+
+	glog.V(3).Infof("Returning from execute on tests: finalOutput %#v", finalOutput)
+	return finalOutput, nil
+}
+
+func runAudit(audit string) (output string, err error) {
+	var out bytes.Buffer
+
+	audit = strings.TrimSpace(audit)
+	if len(audit) == 0 {
+		return output, err
+	}
+
+	cmd := exec.Command("/bin/sh")
+	cmd.Stdin = strings.NewReader(audit)
+	cmd.Stdout = &out
+	cmd.Stderr = &out
+	err = cmd.Run()
+	output = out.String()
+
+	if err != nil {
+		err = fmt.Errorf("failed to run: %q, output: %q, error: %s", audit, output, err)
+	} else {
+		glog.V(3).Infof("Command %q\n - Output:\n %q", audit, output)
+
+	}
+	return output, err
 }

check/check_test.go

@@ -0,0 +1,188 @@
+// Copyright © 2017-2020 Aqua Security Software Ltd. <info@aquasec.com>
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package check
+
+import (
+	"strings"
+	"testing"
+)
+
+func TestCheck_Run(t *testing.T) {
+	type TestCase struct {
+		name     string
+		check    Check
+		Expected State
+	}
+
+	testCases := []TestCase{
+		{name: "Manual check should WARN", check: Check{Type: MANUAL}, Expected: WARN},
+		{name: "Skip check should INFO", check: Check{Type: "skip"}, Expected: INFO},
+		{name: "Unscored check (with no type) should WARN on failure", check: Check{Scored: false}, Expected: WARN},
+		{
+			name: "Unscored check that pass should PASS",
+			check: Check{
+				Scored: false,
+				Audit:  "echo hello",
+				Tests: &tests{TestItems: []*testItem{{
+					Flag: "hello",
+					Set:  true,
+				}}},
+			},
+			Expected: PASS,
+		},
+
+		{name: "Check with no tests should WARN", check: Check{Scored: true}, Expected: WARN},
+		{name: "Scored check with empty tests should FAIL", check: Check{Scored: true, Tests: &tests{}}, Expected: FAIL},
+		{
+			name: "Scored check that doesn't pass should FAIL",
+			check: Check{
+				Scored: true,
+				Audit:  "echo hello",
+				Tests: &tests{TestItems: []*testItem{{
+					Flag: "hello",
+					Set:  false,
+				}}},
+			},
+			Expected: FAIL,
+		},
+		{
+			name: "Scored checks that pass should PASS",
+			check: Check{
+				Scored: true,
+				Audit:  "echo hello",
+				Tests: &tests{TestItems: []*testItem{{
+					Flag: "hello",
+					Set:  true,
+				}}},
+			},
+			Expected: PASS,
+		},
+	}
+
+	for _, testCase := range testCases {
+		t.Run(testCase.name, func(t *testing.T) {
+			testCase.check.run()
+			if testCase.check.State != testCase.Expected {
+				t.Errorf("expected %s, actual %s", testCase.Expected, testCase.check.State)
+			}
+		})
+	}
+}
+
+func TestCheckAuditConfig(t *testing.T) {
+
+	passingCases := []*Check{
+		controls.Groups[1].Checks[0],
+		controls.Groups[1].Checks[3],
+		controls.Groups[1].Checks[5],
+		controls.Groups[1].Checks[7],
+		controls.Groups[1].Checks[9],
+		controls.Groups[1].Checks[15],
+	}
+
+	failingCases := []*Check{
+		controls.Groups[1].Checks[1],
+		controls.Groups[1].Checks[2],
+		controls.Groups[1].Checks[4],
+		controls.Groups[1].Checks[6],
+		controls.Groups[1].Checks[8],
+		controls.Groups[1].Checks[10],
+		controls.Groups[1].Checks[11],
+		controls.Groups[1].Checks[12],
+		controls.Groups[1].Checks[13],
+		controls.Groups[1].Checks[14],
+		controls.Groups[1].Checks[16],
+	}
+
+	for _, c := range passingCases {
+		t.Run(c.Text, func(t *testing.T) {
+			c.run()
+			if c.State != "PASS" {
+				t.Errorf("Should PASS, got: %v", c.State)
+			}
+		})
+	}
+
+	for _, c := range failingCases {
+		t.Run(c.Text, func(t *testing.T) {
+			c.run()
+			if c.State != "FAIL" {
+				t.Errorf("Should FAIL, got: %v", c.State)
+			}
+		})
+	}
+}
+
+func Test_runAudit(t *testing.T) {
+	type args struct {
+		audit  string
+		output string
+	}
+	tests := []struct {
+		name   string
+		args   args
+		errMsg string
+		output string
+	}{
+		{
+			name: "run success",
+			args: args{
+				audit: "echo 'hello world'",
+			},
+			errMsg: "",
+			output: "hello world\n",
+		},
+		{
+			name: "run multiple lines script",
+			args: args{
+				audit: `
+hello() {
+  echo "hello world"
+}
+
+hello
+`,
+			},
+			errMsg: "",
+			output: "hello world\n",
+		},
+		{
+			name: "run failed",
+			args: args{
+				audit: "unknown_command",
+			},
+			errMsg: "failed to run: \"unknown_command\", output: \"/bin/sh: ",
+			output: "not found\n",
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			var errMsg string
+			output, err := runAudit(tt.args.audit)
+			if err != nil {
+				errMsg = err.Error()
+			}
+			if errMsg != "" && !strings.Contains(errMsg, tt.errMsg) {
+				t.Errorf("name %s errMsg = %q, want %q", tt.name, errMsg, tt.errMsg)
+			}
+			if errMsg == "" && output != tt.output {
+				t.Errorf("name %s output = %q, want %q", tt.name, output, tt.output)
+			}
+			if errMsg != "" && !strings.Contains(output, tt.output) {
+				t.Errorf("name %s output = %q, want %q", tt.name, output, tt.output)
+			}
+		})
+	}
+}

check/controls.go

@@ -15,17 +15,21 @@
 package check
 
 import (
+	"bytes"
 	"encoding/json"
+	"encoding/xml"
 	"fmt"
 
-	yaml "gopkg.in/yaml.v2"
+	"github.com/golang/glog"
+	"github.com/onsi/ginkgo/reporters"
+	"gopkg.in/yaml.v2"
 )
 
 // Controls holds all controls to check for master nodes.
 type Controls struct {
-	ID      string `yaml:"id" json:"id"`
-	Version string	`json:"version"`
-	Text    string 	`json:"text"`
+	ID      string   `yaml:"id" json:"id"`
+	Version string   `json:"version"`
+	Text    string   `json:"text"`
 	Type    NodeType `json:"node_type"`
 	Groups  []*Group `json:"tests"`
 	Summary
@@ -37,17 +41,22 @@ type Group struct {
 	Pass   int      `json:"pass"`
 	Fail   int      `json:"fail"`
 	Warn   int      `json:"warn"`
+	Info   int      `json:"info"`
 	Text   string   `json:"desc"`
 	Checks []*Check `json:"results"`
 }
 
 // Summary is a summary of the results of control checks run.
 type Summary struct {
-	Pass int   `json:"total_pass"`
-	Fail int   `json:"total_fail"`
-	Warn int   `json:"total_warn"`
+	Pass int `json:"total_pass"`
+	Fail int `json:"total_fail"`
+	Warn int `json:"total_warn"`
+	Info int `json:"total_info"`
 }
 
+// Predicate a predicate on the given Group and Check arguments.
+type Predicate func(group *Group, check *Check) bool
+
 // NewControls instantiates a new master Controls object.
 func NewControls(t NodeType, in []byte) (*Controls, error) {
 	c := new(Controls)
@@ -61,86 +70,47 @@ func NewControls(t NodeType, in []byte) (*Controls, error) {
 		return nil, fmt.Errorf("non-%s controls file specified", t)
 	}
 
-	// Prepare audit commands
-	for _, group := range c.Groups {
-		for _, check := range group.Checks {
-			check.Commands = textToCommand(check.Audit)
-		}
-	}
-
 	return c, nil
 }
 
-// RunGroup runs all checks in a group.
-func (controls *Controls) RunGroup(gids ...string) Summary {
-	g := []*Group{}
-	controls.Summary.Pass, controls.Summary.Fail, controls.Summary.Warn = 0, 0, 0
-
-	// If no groupid is passed run all group checks.
-	if len(gids) == 0 {
-		gids = controls.getAllGroupIDs()
-	}
-
-	for _, group := range controls.Groups {
-
-		for _, gid := range gids {
-			if gid == group.ID {
-				for _, check := range group.Checks {
-					check.Run()
-					check.TestInfo = append(check.TestInfo, check.Remediation)
-					summarize(controls, check)
-					summarizeGroup(group, check)
-				}
-
-				g = append(g, group)
-			}
-		}
-	}
-
-	controls.Groups = g
-	return controls.Summary
-}
-
-// RunChecks runs the checks with the supplied IDs.
-func (controls *Controls) RunChecks(ids ...string) Summary {
-	g := []*Group{}
+// RunChecks runs the checks with the given Runner. Only checks for which the filter Predicate returns `true` will run.
+func (controls *Controls) RunChecks(runner Runner, filter Predicate) Summary {
+	var g []*Group
 	m := make(map[string]*Group)
-	controls.Summary.Pass, controls.Summary.Fail, controls.Summary.Warn = 0, 0, 0
-
-	// If no groupid is passed run all group checks.
-	if len(ids) == 0 {
-		ids = controls.getAllCheckIDs()
-	}
+	controls.Summary.Pass, controls.Summary.Fail, controls.Summary.Warn, controls.Info = 0, 0, 0, 0
 
 	for _, group := range controls.Groups {
 		for _, check := range group.Checks {
-			for _, id := range ids {
-				if id == check.ID {
-					check.Run()
-					check.TestInfo = append(check.TestInfo, check.Remediation)
-					summarize(controls, check)
 
-					// Check if we have already added this checks group.
-					if v, ok := m[group.ID]; !ok {
-						// Create a group with same info
-						w := &Group{
-							ID:     group.ID,
-							Text:   group.Text,
-							Checks: []*Check{},
-						}
-
-						// Add this check to the new group
-						w.Checks = append(w.Checks, check)
-
-						// Add to groups we have visited.
-						m[w.ID] = w
-						g = append(g, w)
-					} else {
-						v.Checks = append(v.Checks, check)
-					}
-
-				}
+			if !filter(group, check) {
+				continue
 			}
+
+			state := runner.Run(check)
+			check.TestInfo = append(check.TestInfo, check.Remediation)
+
+			// Check if we have already added this checks group.
+			if v, ok := m[group.ID]; !ok {
+				// Create a group with same info
+				w := &Group{
+					ID:     group.ID,
+					Text:   group.Text,
+					Checks: []*Check{},
+				}
+
+				// Add this check to the new group
+				w.Checks = append(w.Checks, check)
+				summarizeGroup(w, state)
+
+				// Add to groups we have visited.
+				m[w.ID] = w
+				g = append(g, w)
+			} else {
+				v.Checks = append(v.Checks, check)
+				summarizeGroup(v, state)
+			}
+
+			summarize(controls, state)
 		}
 	}
 
@@ -153,45 +123,84 @@ func (controls *Controls) JSON() ([]byte, error) {
 	return json.Marshal(controls)
 }
 
-func (controls *Controls) getAllGroupIDs() []string {
-	var ids []string
-
-	for _, group := range controls.Groups {
-		ids = append(ids, group.ID)
+// JUnit encodes the results of last run to JUnit.
+func (controls *Controls) JUnit() ([]byte, error) {
+	suite := reporters.JUnitTestSuite{
+		Name:      controls.Text,
+		TestCases: []reporters.JUnitTestCase{},
+		Tests:     controls.Summary.Pass + controls.Summary.Fail + controls.Summary.Info + controls.Summary.Warn,
+		Failures:  controls.Summary.Fail,
 	}
-	return ids
-}
+	for _, g := range controls.Groups {
+		for _, check := range g.Checks {
+			jsonCheck := ""
+			jsonBytes, err := json.Marshal(check)
+			if err != nil {
+				jsonCheck = fmt.Sprintf("Failed to marshal test into JSON: %v. Test as text: %#v", err, check)
+			} else {
+				jsonCheck = string(jsonBytes)
+			}
+			tc := reporters.JUnitTestCase{
+				Name:      fmt.Sprintf("%v %v", check.ID, check.Text),
+				ClassName: g.Text,
 
-func (controls *Controls) getAllCheckIDs() []string {
-	var ids []string
+				// Store the entire json serialization as system out so we don't lose data in cases where deeper debugging is necessary.
+				SystemOut: jsonCheck,
+			}
 
-	for _, group := range controls.Groups {
-		for _, check := range group.Checks {
-			ids = append(ids, check.ID)
+			switch check.State {
+			case FAIL:
+				tc.FailureMessage = &reporters.JUnitFailureMessage{Message: check.Remediation}
+			case WARN, INFO:
+				// WARN and INFO are two different versions of skipped tests. Either way it would be a false positive/negative to report
+				// it any other way.
+				tc.Skipped = &reporters.JUnitSkipped{}
+			case PASS:
+			default:
+				glog.Warningf("Unrecognized state %s", check.State)
+			}
+
+			suite.TestCases = append(suite.TestCases, tc)
 		}
 	}
-	return ids
 
+	var b bytes.Buffer
+	encoder := xml.NewEncoder(&b)
+	encoder.Indent("", "    ")
+	err := encoder.Encode(suite)
+	if err != nil {
+		return nil, fmt.Errorf("Failed to generate JUnit report: %s", err.Error())
+	}
+
+	return b.Bytes(), nil
 }
 
-func summarize(controls *Controls, check *Check) {
-	switch check.State {
+func summarize(controls *Controls, state State) {
+	switch state {
 	case PASS:
 		controls.Summary.Pass++
 	case FAIL:
 		controls.Summary.Fail++
 	case WARN:
 		controls.Summary.Warn++
+	case INFO:
+		controls.Summary.Info++
+	default:
+		glog.Warningf("Unrecognized state %s", state)
 	}
 }
 
-func summarizeGroup(group *Group, check *Check) {
-	switch check.State {
+func summarizeGroup(group *Group, state State) {
+	switch state {
 	case PASS:
 		group.Pass++
 	case FAIL:
 		group.Fail++
 	case WARN:
 		group.Warn++
+	case INFO:
+		group.Info++
+	default:
+		glog.Warningf("Unrecognized state %s", state)
 	}
 }

check/controls_test.go

@@ -1,41 +1,292 @@
+// Copyright © 2017-2019 Aqua Security Software Ltd. <info@aquasec.com>
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package check
 
 import (
+	"bytes"
+	"encoding/json"
+	"encoding/xml"
 	"io/ioutil"
+	"os"
+	"path/filepath"
 	"testing"
 
-	yaml "gopkg.in/yaml.v2"
+	"github.com/onsi/ginkgo/reporters"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/mock"
+	"gopkg.in/yaml.v2"
 )
 
 const cfgDir = "../cfg/"
 
+type mockRunner struct {
+	mock.Mock
+}
+
+func (m *mockRunner) Run(c *Check) State {
+	args := m.Called(c)
+	return args.Get(0).(State)
+}
+
 // validate that the files we're shipping are valid YAML
 func TestYamlFiles(t *testing.T) {
-	// TODO: make this list dynamic
-	dirs := []string{"1.6/", "1.7/"}
-
-	for _, dir := range dirs {
-		dir = cfgDir + dir
-
-		files, err := ioutil.ReadDir(dir)
+	err := filepath.Walk(cfgDir, func(path string, info os.FileInfo, err error) error {
 		if err != nil {
-			t.Fatalf("error reading %s directory: %v", dir, err)
+			t.Fatalf("failure accessing path %q: %v\n", path, err)
 		}
-
-		for _, file := range files {
-
-			fileName := file.Name()
-			in, err := ioutil.ReadFile(dir + fileName)
+		if !info.IsDir() {
+			t.Logf("reading file: %s", path)
+			in, err := ioutil.ReadFile(path)
 			if err != nil {
-				t.Fatalf("error opening file %s: %v", fileName, err)
+				t.Fatalf("error opening file %s: %v", path, err)
 			}
 
 			c := new(Controls)
-
 			err = yaml.Unmarshal(in, c)
-			if err != nil {
-				t.Fatalf("failed to load YAML from %s: %v", fileName, err)
+			if err == nil {
+				t.Logf("YAML file successfully unmarshalled: %s", path)
+			} else {
+				t.Fatalf("failed to load YAML from %s: %v", path, err)
 			}
 		}
+		return nil
+	})
+	if err != nil {
+		t.Fatalf("failure walking cfg dir: %v\n", err)
 	}
 }
+
+func TestNewControls(t *testing.T) {
+
+	t.Run("Should return error when node type is not specified", func(t *testing.T) {
+		// given
+		in := []byte(`
+---
+controls:
+type: # not specified
+groups:
+`)
+		// when
+		_, err := NewControls(MASTER, in)
+		// then
+		assert.EqualError(t, err, "non-master controls file specified")
+	})
+
+	t.Run("Should return error when input YAML is invalid", func(t *testing.T) {
+		// given
+		in := []byte("BOOM")
+		// when
+		_, err := NewControls(MASTER, in)
+		// then
+		assert.EqualError(t, err, "failed to unmarshal YAML: yaml: unmarshal errors:\n  line 1: cannot unmarshal !!str `BOOM` into check.Controls")
+	})
+
+}
+
+func TestControls_RunChecks(t *testing.T) {
+
+	t.Run("Should run checks matching the filter and update summaries", func(t *testing.T) {
+		// given
+		runner := new(mockRunner)
+		// and
+		in := []byte(`
+---
+type: "master"
+groups:
+- id: G1
+  checks:
+  - id: G1/C1
+- id: G2
+  checks:
+  - id: G2/C1
+    text: "Verify that the SomeSampleFlag argument is set to true"
+    audit: "grep -B1 SomeSampleFlag=true /this/is/a/file/path"
+    tests:
+      test_items:
+      - flag: "SomeSampleFlag=true"
+        compare:
+          op: has
+          value: "true"
+        set: true
+    remediation: |
+      Edit the config file /this/is/a/file/path and set SomeSampleFlag to true.
+    scored: true
+`)
+		// and
+		controls, err := NewControls(MASTER, in)
+		assert.NoError(t, err)
+		// and
+		runner.On("Run", controls.Groups[0].Checks[0]).Return(PASS)
+		runner.On("Run", controls.Groups[1].Checks[0]).Return(FAIL)
+		// and
+		var runAll Predicate = func(group *Group, c *Check) bool {
+			return true
+		}
+		// when
+		controls.RunChecks(runner, runAll)
+		// then
+		assert.Equal(t, 2, len(controls.Groups))
+		// and
+		G1 := controls.Groups[0]
+		assert.Equal(t, "G1", G1.ID)
+		assert.Equal(t, "G1/C1", G1.Checks[0].ID)
+		assertEqualGroupSummary(t, 1, 0, 0, 0, G1)
+		// and
+		G2 := controls.Groups[1]
+		assert.Equal(t, "G2", G2.ID)
+		assert.Equal(t, "G2/C1", G2.Checks[0].ID)
+		assert.Equal(t, "has", G2.Checks[0].Tests.TestItems[0].Compare.Op)
+		assert.Equal(t, "true", G2.Checks[0].Tests.TestItems[0].Compare.Value)
+		assert.Equal(t, true, G2.Checks[0].Tests.TestItems[0].Set)
+		assert.Equal(t, "SomeSampleFlag=true", G2.Checks[0].Tests.TestItems[0].Flag)
+		assert.Equal(t, "Edit the config file /this/is/a/file/path and set SomeSampleFlag to true.\n", G2.Checks[0].Remediation)
+		assert.Equal(t, true, G2.Checks[0].Scored)
+		assertEqualGroupSummary(t, 0, 1, 0, 0, G2)
+		// and
+		assert.Equal(t, 1, controls.Summary.Pass)
+		assert.Equal(t, 1, controls.Summary.Fail)
+		assert.Equal(t, 0, controls.Summary.Info)
+		assert.Equal(t, 0, controls.Summary.Warn)
+		// and
+		runner.AssertExpectations(t)
+	})
+}
+
+func TestControls_JUnitIncludesJSON(t *testing.T) {
+	testCases := []struct {
+		desc   string
+		input  *Controls
+		expect []byte
+	}{
+		{
+			desc: "Serializes to junit",
+			input: &Controls{
+				Groups: []*Group{
+					{
+						ID: "g1",
+						Checks: []*Check{
+							{ID: "check1id", Text: "check1text", State: PASS},
+						},
+					},
+				},
+			},
+			expect: []byte(`<testsuite name="" tests="0" failures="0" errors="0" time="0">
+    <testcase name="check1id check1text" classname="" time="0">
+        <system-out>{&#34;test_number&#34;:&#34;check1id&#34;,&#34;test_desc&#34;:&#34;check1text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;PASS&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+    </testcase>
+</testsuite>`),
+		}, {
+			desc: "Summary values come from summary not checks",
+			input: &Controls{
+				Summary: Summary{
+					Fail: 99,
+					Pass: 100,
+					Warn: 101,
+					Info: 102,
+				},
+				Groups: []*Group{
+					{
+						ID: "g1",
+						Checks: []*Check{
+							{ID: "check1id", Text: "check1text", State: PASS},
+						},
+					},
+				},
+			},
+			expect: []byte(`<testsuite name="" tests="402" failures="99" errors="0" time="0">
+    <testcase name="check1id check1text" classname="" time="0">
+        <system-out>{&#34;test_number&#34;:&#34;check1id&#34;,&#34;test_desc&#34;:&#34;check1text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;PASS&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+    </testcase>
+</testsuite>`),
+		}, {
+			desc: "Warn and Info are considered skips and failed tests properly reported",
+			input: &Controls{
+				Groups: []*Group{
+					{
+						ID: "g1",
+						Checks: []*Check{
+							{ID: "check1id", Text: "check1text", State: PASS},
+							{ID: "check2id", Text: "check2text", State: INFO},
+							{ID: "check3id", Text: "check3text", State: WARN},
+							{ID: "check4id", Text: "check4text", State: FAIL},
+						},
+					},
+				},
+			},
+			expect: []byte(`<testsuite name="" tests="0" failures="0" errors="0" time="0">
+    <testcase name="check1id check1text" classname="" time="0">
+        <system-out>{&#34;test_number&#34;:&#34;check1id&#34;,&#34;test_desc&#34;:&#34;check1text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;PASS&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+    </testcase>
+    <testcase name="check2id check2text" classname="" time="0">
+        <skipped></skipped>
+        <system-out>{&#34;test_number&#34;:&#34;check2id&#34;,&#34;test_desc&#34;:&#34;check2text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;INFO&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+    </testcase>
+    <testcase name="check3id check3text" classname="" time="0">
+        <skipped></skipped>
+        <system-out>{&#34;test_number&#34;:&#34;check3id&#34;,&#34;test_desc&#34;:&#34;check3text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;WARN&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+    </testcase>
+    <testcase name="check4id check4text" classname="" time="0">
+        <failure type=""></failure>
+        <system-out>{&#34;test_number&#34;:&#34;check4id&#34;,&#34;test_desc&#34;:&#34;check4text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;FAIL&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+    </testcase>
+</testsuite>`),
+		},
+	}
+	for _, tc := range testCases {
+		t.Run(tc.desc, func(t *testing.T) {
+			junitBytes, err := tc.input.JUnit()
+			if err != nil {
+				t.Fatalf("Failed to serialize to JUnit: %v", err)
+			}
+
+			var out reporters.JUnitTestSuite
+			if err := xml.Unmarshal(junitBytes, &out); err != nil {
+				t.Fatalf("Unable to deserialize from resulting JUnit: %v", err)
+			}
+
+			// Check that each check was serialized as json and stored as systemOut.
+			for iGroup, group := range tc.input.Groups {
+				for iCheck, check := range group.Checks {
+					jsonBytes, err := json.Marshal(check)
+					if err != nil {
+						t.Fatalf("Failed to serialize to JUnit: %v", err)
+					}
+
+					if out.TestCases[iGroup*iCheck+iCheck].SystemOut != string(jsonBytes) {
+						t.Errorf("Expected\n\t%v\n\tbut got\n\t%v",
+							out.TestCases[iGroup*iCheck+iCheck].SystemOut,
+							string(jsonBytes),
+						)
+					}
+				}
+			}
+
+			if !bytes.Equal(junitBytes, tc.expect) {
+				t.Errorf("Expected\n\t%v\n\tbut got\n\t%v",
+					string(tc.expect),
+					string(junitBytes),
+				)
+			}
+		})
+	}
+}
+
+func assertEqualGroupSummary(t *testing.T, pass, fail, info, warn int, actual *Group) {
+	t.Helper()
+	assert.Equal(t, pass, actual.Pass)
+	assert.Equal(t, fail, actual.Fail)
+	assert.Equal(t, info, actual.Info)
+	assert.Equal(t, warn, actual.Warn)
+}

check/data

@@ -17,7 +17,7 @@ groups:
     - id: 1
       text: "flag is not set"
       tests:
-        test_item:
+        test_items:
           - flag: "--basic-auth"
             set: false
 
@@ -157,4 +157,403 @@ groups:
               value: Something
             set: true
 
+    - id: 14
+      text: "check that flag some-arg is set to some-val with ':' separator"
+      tests:
+        test_items:
+          - flag: "some-arg"
+            compare:
+              op: eq
+              value: some-val
+            set: true
 
+    - id: 15
+      text: "jsonpath correct value on field"
+      tests:
+        test_items:
+          - path: "{.readOnlyPort}"
+            compare:
+              op: eq
+              value: 15000
+            set: true
+          - path: "{.readOnlyPort}"
+            compare:
+              op: gte
+              value: 15000
+            set: true
+          - path: "{.readOnlyPort}"
+            compare:
+              op: lte
+              value: 15000
+            set: true
+
+    - id: 16
+      text: "jsonpath correct case-sensitive value on string field"
+      tests:
+        test_items:
+          - path: "{.stringValue}"
+            compare:
+              op: noteq
+              value: "None"
+            set: true
+          - path: "{.stringValue}"
+            compare:
+              op: noteq
+              value: "webhook,Something,RBAC"
+            set: true
+          - path: "{.stringValue}"
+            compare:
+              op: eq
+              value: "WebHook,Something,RBAC"
+            set: true
+
+    - id: 17
+      text: "jsonpath correct value on boolean field"
+      tests:
+        test_items:
+          - path: "{.trueValue}"
+            compare:
+              op: noteq
+              value: somethingElse
+            set: true
+          - path: "{.trueValue}"
+            compare:
+              op: noteq
+              value: false
+            set: true
+          - path: "{.trueValue}"
+            compare:
+              op: eq
+              value: true
+            set: true
+
+    - id: 18
+      text: "jsonpath field absent"
+      tests:
+        test_items:
+          - path: "{.notARealField}"
+            set: false
+
+    - id: 19
+      text: "jsonpath correct value on nested field"
+      tests:
+        test_items:
+          - path: "{.authentication.anonymous.enabled}"
+            compare:
+              op: eq
+              value: "false"
+            set: true
+
+    - id: 20
+      text: "yamlpath correct value on field"
+      tests:
+        test_items:
+          - path: "{.readOnlyPort}"
+            compare:
+              op: gt
+              value: 14999
+            set: true
+
+    - id: 21
+      text: "yamlpath field absent"
+      tests:
+        test_items:
+          - path: "{.fieldThatIsUnset}"
+            set: false
+
+    - id: 22
+      text: "yamlpath correct value on nested field"
+      tests:
+        test_items:
+          - path: "{.authentication.anonymous.enabled}"
+            compare:
+              op: eq
+              value: "false"
+            set: true
+
+    - id: 23
+      text: "path on invalid json"
+      tests:
+        test_items:
+          - path: "{.authentication.anonymous.enabled}"
+            compare:
+              op: eq
+              value: "false"
+            set: true
+
+    - id: 24
+      text: "path with broken expression"
+      tests:
+        test_items:
+          - path: "{.missingClosingBrace"
+            set: true
+
+    - id: 25
+      text: "yamlpath on invalid yaml"
+      tests:
+        test_items:
+          - path: "{.authentication.anonymous.enabled}"
+            compare:
+              op: eq
+              value: "false"
+            set: true
+
+    - id: 26
+      text: "check regex op matches"
+      tests:
+        test_items:
+          - path: "{.currentMasterVersion}"
+            compare:
+              op: regex
+              value: '^1\.12.*$'
+            set: true
+
+    - id: 27
+      text: "check boolean flag with no value"
+      tests:
+        test_items:
+          - flag: "--peer-client-cert-auth"
+            compare:
+              op: eq
+              value: true
+            set: true
+
+    - id: 28
+      text: "check boolean flag with false value"
+      tests:
+        test_items:
+          - flag: "--peer-client-cert-auth"
+            compare:
+              op: eq
+              value: false
+            set: true
+
+- id: 2.1
+  text: "audit and audit_config commands"
+  checks:
+    - id: 0
+      text: "audit finds flag and passes, audit_config doesn't exist -> pass"
+      audit: "echo flag=correct"
+      tests:
+        test_items:
+          - flag: "flag"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 1
+      text: "audit finds flag and fails, audit_config doesn't exist -> fail"
+      audit: "echo flag=wrong"
+      tests:
+        test_items:
+          - flag: "flag"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 2
+      text: "audit doesn't find flag, audit_config doesn't exist -> fail"
+      audit: "echo somethingElse=correct"
+      tests:
+        test_items:
+          - flag: "flag"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 3
+      text: "audit doesn't find flag, audit_config has correct setting -> pass"
+      audit: "echo somethingElse=correct"
+      audit_config: "echo 'flag: correct'"
+      tests:
+        test_items:
+          - flag: "flag"
+            path: "{.flag}"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 4
+      text: "audit doesn't find flag, audit_config has wrong setting -> fail"
+      audit: "echo somethingElse=correct"
+      audit_config: "echo 'flag: wrong'"
+      tests:
+        test_items:
+          - flag: "flag"
+            path: "{.flag}"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 5
+      text: "audit finds correct flag, audit_config has wrong setting -> pass"
+      audit: "echo flag=correct"
+      audit_config: "echo 'flag: wrong'"
+      tests:
+        test_items:
+          - flag: "flag"
+            path: "{.flag}"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 6
+      text: "neither audit nor audit_config has correct setting -> fail"
+      audit: "echo flag=wrong"
+      audit_config: "echo 'flag: wrong'"
+      tests:
+        test_items:
+          - flag: "flag"
+            path: "{.flag}"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 7
+      text: "audit isn't present, superfluous flag field,audit_config is correct -> pass"
+      audit_config: "echo 'flag: correct'"
+      tests:
+        test_items:
+          - flag: "flag"
+            path: "{.flag}"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 8
+      text: "audit isn't present, superfluous flag field,audit_config is wrong -> fail"
+      audit_config: "echo 'flag: wrong'"
+      tests:
+        test_items:
+          - flag: "flag"
+            path: "{.flag}"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 9
+      text: "test use_multiple_values is correct -> pass"
+      audit: "printf 'permissions=600\npermissions=600\npermissions=600'"
+      use_multiple_values: true
+      tests:
+        test_items:
+          - flag: "permissions"
+            compare:
+              op: bitmask
+              value: "600"
+            set: true
+      scored: true
+    - id: 10
+      text: "test use_multiple_values is wrong -> fail"
+      audit: "printf 'permissions=600\npermissions=600\npermissions=644'"
+      use_multiple_values: true
+      tests:
+        test_items:
+          - flag: "permissions"
+            compare:
+              op: bitmask
+              value: "600"
+            set: true
+      scored: true
+    - id: 11
+      text: "test use_multiple_values include empty value -> fail"
+      audit: "printf 'permissions=600\n\npermissions=600'"
+      use_multiple_values: true
+      tests:
+        test_items:
+          - flag: "permissions"
+            compare:
+              op: bitmask
+              value: "600"
+            set: true
+      scored: true
+    - id: 12
+      text: "audit is present and wrong, audit_config is right -> fail (command line parameters override config file)"
+      audit: "echo flag=wrong"
+      audit_config: "echo 'flag: correct'"
+      tests:
+        test_items:
+          - flag: "flag"
+            path: "{.flag}"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 13
+      text: "parameter and config file don't have same default - parameter has failing value"
+      audit: "echo '--read-only-port=1'"
+      audit_config: "echo 'readOnlyPort: 0'"
+      tests:
+        bin_op: and
+        test_items:
+          - flag: "--read-only-port"
+            path: "{.readOnlyPort}"
+            set: true
+            compare:
+              op: eq
+              value: 0
+          - flag: "--read-only-port"
+            path: '{.readOnlyPort}'
+            set: false
+      scored: true
+    - id: 14
+      text: "parameter and config file don't have same default - config file has failing value"
+      audit: "echo ''"
+      audit_config: "echo 'readOnlyPort: 1'"
+      tests:
+        bin_op: or
+        test_items:
+          - flag: "--read-only-port"
+            path: '{.readOnlyPort}'
+            set: true
+            compare:
+              op: eq
+              value: 0
+          - flag: "--read-only-port"
+            path: '{.readOnlyPort}'
+            set: false
+      scored: true
+    - id: 15
+      text: "parameter and config file don't have same default - passing"
+      audit: "echo ''"
+      audit_config: "echo ''"
+      tests:
+        bin_op: or
+        test_items:
+          - flag: "--read-only-port"
+            path: '{.readOnlyPort}'
+            set: true
+            compare:
+              op: eq
+              value: 0
+          - flag: "--read-only-port"
+            path: '{.readOnlyPort}'
+            set: false
+      scored: true
+    - id: 15
+      text: "parameter and config file don't have same default - parameter has bad value and config is not present - failing"
+      audit: "echo '--read-only-port=1'"
+      audit_config: "echo ''"
+      tests:
+        bin_op: or
+        test_items:
+          - flag: "--read-only-port"
+            path: '{.readOnlyPort}'
+            set: true
+            compare:
+              op: eq
+              value: 0
+          - flag: "--read-only-port"
+            path: '{.readOnlyPort}'
+            set: false
+      scored: true

check/test.go

@@ -15,11 +15,17 @@
 package check
 
 import (
+	"bytes"
+	"encoding/json"
 	"fmt"
 	"os"
 	"regexp"
 	"strconv"
 	"strings"
+
+	"github.com/golang/glog"
+	yaml "gopkg.in/yaml.v2"
+	"k8s.io/client-go/util/jsonpath"
 )
 
 // test:
@@ -32,137 +38,353 @@ import (
 type binOp string
 
 const (
-	and binOp = "and"
-	or        = "or"
+	and                   binOp = "and"
+	or                          = "or"
+	defaultArraySeparator       = ","
 )
 
-type testItem struct {
-	Flag    string
-	Output  string
-	Value   string
-	Set     bool
-	Compare compare
-}
-
-type compare struct {
-	Op    string
-	Value string
-}
-
-func (t *testItem) execute(s string) (result bool) {
-	result = false
-	match := strings.Contains(s, t.Flag)
-
-	if t.Set {
-		var flagVal string
-		isset := match
-
-		if isset && t.Compare.Op != "" {
-			// Expects flags in the form;
-			// --flag=somevalue
-			// --flag
-			// somevalue
-			//pttn := `(` + t.Flag + `)(=)*([^\s,]*) *`
-			pttn := `(` + t.Flag + `)(=)*([^\s]*) *`
-			flagRe := regexp.MustCompile(pttn)
-			vals := flagRe.FindStringSubmatch(s)
-
-			if len(vals) > 0 {
-				if vals[3] != "" {
-					flagVal = vals[3]
-				} else {
-					flagVal = vals[1]
-				}
-			} else {
-				fmt.Fprintf(os.Stderr, "invalid flag in testitem definition")
-				os.Exit(1)
-			}
-
-			switch t.Compare.Op {
-			case "eq":
-				result = flagVal == t.Compare.Value
-
-			case "noteq":
-				result = !(flagVal == t.Compare.Value)
-
-			case "gt":
-				a, b := toNumeric(flagVal, t.Compare.Value)
-				result = a > b
-
-			case "gte":
-				a, b := toNumeric(flagVal, t.Compare.Value)
-				result = a >= b
-
-			case "lt":
-				a, b := toNumeric(flagVal, t.Compare.Value)
-				result = a < b
-
-			case "lte":
-				a, b := toNumeric(flagVal, t.Compare.Value)
-				result = a <= b
-
-			case "has":
-				result = strings.Contains(flagVal, t.Compare.Value)
-
-			case "nothave":
-				result = !strings.Contains(flagVal, t.Compare.Value)
-			}
-		} else {
-			result = isset
-		}
-
-	} else {
-		notset := !match
-		result = notset
-	}
-
-	return
-}
-
 type tests struct {
 	TestItems []*testItem `yaml:"test_items"`
 	BinOp     binOp       `yaml:"bin_op"`
 }
 
-func (ts *tests) execute(s string) (result bool) {
-	res := make([]bool, len(ts.TestItems))
+type testItem struct {
+	Flag             string
+	Path             string
+	Output           string
+	Value            string
+	Set              bool
+	Compare          compare
+	isMultipleOutput bool
+	isConfigSetting  bool
+}
 
-	for i, t := range ts.TestItems {
-		res[i] = t.execute(s)
+type pathTestItem testItem
+type flagTestItem testItem
+
+type compare struct {
+	Op    string
+	Value string
+}
+
+type testOutput struct {
+	testResult     bool
+	flagFound      bool
+	actualResult   string
+	ExpectedResult string
+}
+
+func failTestItem(s string) *testOutput {
+	return &testOutput{testResult: false, actualResult: s}
+}
+
+func (t testItem) flagValue() string {
+	if t.isConfigSetting {
+		return t.Path
 	}
 
-	// If no binary operation is specified, default to AND
-	switch ts.BinOp {
-	default:
-		fmt.Fprintf(os.Stderr, "unknown binary operator for tests %s\n", ts.BinOp)
-		os.Exit(1)
-	case and, "":
-		result = true
-		for i := range res {
-			result = result && res[i]
+	return t.Flag
+}
+
+func (t testItem) findValue(s string) (match bool, value string, err error) {
+	if t.isConfigSetting {
+		pt := pathTestItem(t)
+		return pt.findValue(s)
+	}
+
+	ft := flagTestItem(t)
+	return ft.findValue(s)
+}
+
+func (t flagTestItem) findValue(s string) (match bool, value string, err error) {
+	if s == "" || t.Flag == "" {
+		return
+	}
+	match = strings.Contains(s, t.Flag)
+	if match {
+		// Expects flags in the form;
+		// --flag=somevalue
+		// flag: somevalue
+		// --flag
+		// somevalue
+		pttn := `(` + t.Flag + `)(=|: *)*([^\s]*) *`
+		flagRe := regexp.MustCompile(pttn)
+		vals := flagRe.FindStringSubmatch(s)
+
+		if len(vals) > 0 {
+			if vals[3] != "" {
+				value = vals[3]
+			} else {
+				// --bool-flag
+				if strings.HasPrefix(t.Flag, "--") {
+					value = "true"
+				} else {
+					value = vals[1]
+				}
+			}
+		} else {
+			err = fmt.Errorf("invalid flag in testItem definition: %s", s)
 		}
-	case or:
-		result = false
-		for i := range res {
-			result = result || res[i]
+	}
+	glog.V(3).Infof("In flagTestItem.findValue %s, match %v, s %s, t.Flag %s", value, match, s, t.Flag)
+
+	return match, value, err
+}
+
+func (t pathTestItem) findValue(s string) (match bool, value string, err error) {
+	var jsonInterface interface{}
+
+	err = unmarshal(s, &jsonInterface)
+	if err != nil {
+		return false, "", fmt.Errorf("failed to load YAML or JSON from input \"%s\": %v", s, err)
+	}
+
+	value, err = executeJSONPath(t.Path, &jsonInterface)
+	if err != nil {
+		return false, "", fmt.Errorf("unable to parse path expression \"%s\": %v", t.Path, err)
+	}
+
+	glog.V(3).Infof("In pathTestItem.findValue %s", value)
+	match = (value != "")
+	return match, value, err
+}
+
+func (t testItem) execute(s string) *testOutput {
+	result := &testOutput{}
+	s = strings.TrimRight(s, " \n")
+
+	// If the test has output that should be evaluated for each row
+	var output []string
+	if t.isMultipleOutput {
+		output = strings.Split(s, "\n")
+	} else {
+		output = []string{s}
+	}
+
+	for _, op := range output {
+		result = t.evaluate(op)
+		// If the test failed for the current row, no need to keep testing for this output
+		if !result.testResult {
+			break
 		}
 	}
 
-	return
+	result.actualResult = s
+	return result
 }
 
-func toNumeric(a, b string) (c, d int) {
-	var err error
-	c, err = strconv.Atoi(a)
+func (t testItem) evaluate(s string) *testOutput {
+	result := &testOutput{}
+
+	match, value, err := t.findValue(s)
 	if err != nil {
-		fmt.Fprintf(os.Stderr, "error converting %s: %s\n", a, err)
-		os.Exit(1)
-	}
-	d, err = strconv.Atoi(b)
-	if err != nil {
-		fmt.Fprintf(os.Stderr, "error converting %s: %s\n", b, err)
-		os.Exit(1)
+		fmt.Fprintf(os.Stderr, err.Error())
+		return failTestItem(err.Error())
 	}
 
-	return c, d
+	if t.Set {
+		if match && t.Compare.Op != "" {
+			result.ExpectedResult, result.testResult = compareOp(t.Compare.Op, value, t.Compare.Value)
+		} else {
+			result.ExpectedResult = fmt.Sprintf("'%s' is present", t.flagValue())
+			result.testResult = match
+		}
+	} else {
+		result.ExpectedResult = fmt.Sprintf("'%s' is not present", t.flagValue())
+		result.testResult = !match
+	}
+
+	result.flagFound = match
+	glog.V(3).Info(fmt.Sprintf("flagFound %v", result.flagFound))
+
+	return result
+}
+
+func compareOp(tCompareOp string, flagVal string, tCompareValue string) (string, bool) {
+
+	expectedResultPattern := ""
+	testResult := false
+
+	switch tCompareOp {
+	case "eq":
+		expectedResultPattern = "'%s' is equal to '%s'"
+		value := strings.ToLower(flagVal)
+		// Do case insensitive comparaison for booleans ...
+		if value == "false" || value == "true" {
+			testResult = value == tCompareValue
+		} else {
+			testResult = flagVal == tCompareValue
+		}
+
+	case "noteq":
+		expectedResultPattern = "'%s' is not equal to '%s'"
+		value := strings.ToLower(flagVal)
+		// Do case insensitive comparaison for booleans ...
+		if value == "false" || value == "true" {
+			testResult = !(value == tCompareValue)
+		} else {
+			testResult = !(flagVal == tCompareValue)
+		}
+
+	case "gt", "gte", "lt", "lte":
+		a, b, err := toNumeric(flagVal, tCompareValue)
+		if err != nil {
+			glog.V(1).Infof(fmt.Sprintf("Not numeric value - flag: %q - compareValue: %q %v\n", flagVal, tCompareValue, err))
+			return "Invalid Number(s) used for comparison", false
+		}
+		switch tCompareOp {
+		case "gt":
+			expectedResultPattern = "%s is greater than %s"
+			testResult = a > b
+
+		case "gte":
+			expectedResultPattern = "%s is greater or equal to %s"
+			testResult = a >= b
+
+		case "lt":
+			expectedResultPattern = "%s is lower than %s"
+			testResult = a < b
+
+		case "lte":
+			expectedResultPattern = "%s is lower or equal to %s"
+			testResult = a <= b
+		}
+
+	case "has":
+		expectedResultPattern = "'%s' has '%s'"
+		testResult = strings.Contains(flagVal, tCompareValue)
+
+	case "nothave":
+		expectedResultPattern = " '%s' not have '%s'"
+		testResult = !strings.Contains(flagVal, tCompareValue)
+
+	case "regex":
+		expectedResultPattern = " '%s' matched by '%s'"
+		opRe := regexp.MustCompile(tCompareValue)
+		testResult = opRe.MatchString(flagVal)
+
+	case "valid_elements":
+		expectedResultPattern = "'%s' contains valid elements from '%s'"
+		s := splitAndRemoveLastSeparator(flagVal, defaultArraySeparator)
+		target := splitAndRemoveLastSeparator(tCompareValue, defaultArraySeparator)
+		testResult = allElementsValid(s, target)
+
+	case "bitmask":
+		expectedResultPattern = "bitmask '%s' AND '%s'"
+		requested, err := strconv.ParseInt(flagVal, 8, 64)
+		if err != nil {
+			glog.V(1).Infof(fmt.Sprintf("Not numeric value - flag: %q - compareValue: %q %v\n", flagVal, tCompareValue, err))
+			return fmt.Sprintf("Not numeric value - flag: %s", flagVal), false
+		}
+		max, err := strconv.ParseInt(tCompareValue, 8, 64)
+		if err != nil {
+			glog.V(1).Infof(fmt.Sprintf("Not numeric value - flag: %q - compareValue: %q %v\n", flagVal, tCompareValue, err))
+			return fmt.Sprintf("Not numeric value - flag: %s", tCompareValue), false
+		}
+		testResult = (max & requested) == requested
+	}
+	if expectedResultPattern == "" {
+		return expectedResultPattern, testResult
+	}
+
+	return fmt.Sprintf(expectedResultPattern, flagVal, tCompareValue), testResult
+}
+
+func unmarshal(s string, jsonInterface *interface{}) error {
+	data := []byte(s)
+	err := json.Unmarshal(data, jsonInterface)
+	if err != nil {
+		err := yaml.Unmarshal(data, jsonInterface)
+		if err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func executeJSONPath(path string, jsonInterface interface{}) (string, error) {
+	j := jsonpath.New("jsonpath")
+	j.AllowMissingKeys(true)
+	err := j.Parse(path)
+	if err != nil {
+		return "", err
+	}
+
+	buf := new(bytes.Buffer)
+	err = j.Execute(buf, jsonInterface)
+	if err != nil {
+		return "", err
+	}
+	jsonpathResult := buf.String()
+	return jsonpathResult, nil
+}
+
+func allElementsValid(s, t []string) bool {
+	sourceEmpty := len(s) == 0
+	targetEmpty := len(t) == 0
+
+	if sourceEmpty && targetEmpty {
+		return true
+	}
+
+	// XOR comparison -
+	//     if either value is empty and the other is not empty,
+	//     not all elements are valid
+	if (sourceEmpty || targetEmpty) && !(sourceEmpty && targetEmpty) {
+		return false
+	}
+
+	for _, sv := range s {
+		found := false
+		for _, tv := range t {
+			if sv == tv {
+				found = true
+				break
+			}
+		}
+		if !found {
+			return false
+		}
+	}
+	return true
+}
+
+func splitAndRemoveLastSeparator(s, sep string) []string {
+	cleanS := strings.TrimRight(strings.TrimSpace(s), sep)
+	if len(cleanS) == 0 {
+		return []string{}
+	}
+
+	ts := strings.Split(cleanS, sep)
+	for i := range ts {
+		ts[i] = strings.TrimSpace(ts[i])
+	}
+
+	return ts
+}
+
+func toNumeric(a, b string) (c, d int, err error) {
+	c, err = strconv.Atoi(strings.TrimSpace(a))
+	if err != nil {
+		return -1, -1, fmt.Errorf("toNumeric - error converting %s: %s", a, err)
+	}
+	d, err = strconv.Atoi(strings.TrimSpace(b))
+	if err != nil {
+		return -1, -1, fmt.Errorf("toNumeric - error converting %s: %s", b, err)
+	}
+
+	return c, d, nil
+}
+
+func (t *testItem) UnmarshalYAML(unmarshal func(interface{}) error) error {
+	type buildTest testItem
+
+	// Make Set parameter to be true by default.
+	newTestItem := buildTest{Set: true}
+	err := unmarshal(&newTestItem)
+	if err != nil {
+		return err
+	}
+	*t = testItem(newTestItem)
+	return nil
 }

check/test_test.go

@@ -1,4 +1,4 @@
-// Copyright © 2017 Aqua Security Software Ltd. <info@aquasec.com>
+// Copyright © 2017-2020 Aqua Security Software Ltd. <info@aquasec.com>
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
@@ -15,6 +15,7 @@
 package check
 
 import (
+	"fmt"
 	"io/ioutil"
 	"os"
 	"strings"
@@ -48,74 +49,759 @@ func TestTestExecute(t *testing.T) {
 
 	cases := []struct {
 		*Check
-		str string
+		str       string
+		strConfig string
 	}{
 		{
 			controls.Groups[0].Checks[0],
 			"2:45 ../kubernetes/kube-apiserver --allow-privileged=false --option1=20,30,40",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[1],
 			"2:45 ../kubernetes/kube-apiserver --allow-privileged=false",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[2],
 			"niinai   13617  2635 99 19:26 pts/20   00:03:08 ./kube-apiserver --insecure-port=0 --anonymous-auth",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[3],
 			"2:45 ../kubernetes/kube-apiserver --secure-port=0 --audit-log-maxage=40 --option",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[4],
 			"2:45 ../kubernetes/kube-apiserver --max-backlog=20 --secure-port=0 --audit-log-maxage=40 --option",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[5],
 			"2:45 ../kubernetes/kube-apiserver --option --admission-control=WebHook,RBAC ---audit-log-maxage=40",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[6],
 			"2:45 .. --kubelet-clientkey=foo --kubelet-client-certificate=bar --admission-control=Webhook,RBAC",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[7],
 			"2:45 ..  --secure-port=0 --kubelet-client-certificate=bar --admission-control=Webhook,RBAC",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[8],
 			"644",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[9],
 			"640",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[9],
 			"600",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[10],
 			"2:45 ../kubernetes/kube-apiserver --option --admission-control=WebHook,RBAC ---audit-log-maxage=40",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[11],
 			"2:45 ../kubernetes/kube-apiserver --option --admission-control=WebHook,RBAC ---audit-log-maxage=40",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[12],
 			"2:45 ../kubernetes/kube-apiserver --option --admission-control=WebHook,Something,RBAC ---audit-log-maxage=40",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[13],
 			"2:45 ../kubernetes/kube-apiserver --option --admission-control=Something ---audit-log-maxage=40",
+			"",
+		},
+		{
+			// check for ':' as argument-value separator, with space between arg and val
+			controls.Groups[0].Checks[14],
+			"2:45 kube-apiserver some-arg: some-val --admission-control=Something ---audit-log-maxage=40",
+			"",
+		},
+		{
+			// check for ':' as argument-value separator, with no space between arg and val
+			controls.Groups[0].Checks[14],
+			"2:45 kube-apiserver some-arg:some-val --admission-control=Something ---audit-log-maxage=40",
+			"",
+		},
+		{
+			controls.Groups[0].Checks[15],
+			"",
+			"{\"readOnlyPort\": 15000}",
+		},
+		{
+			controls.Groups[0].Checks[16],
+			"",
+			"{\"stringValue\": \"WebHook,Something,RBAC\"}",
+		},
+		{
+			controls.Groups[0].Checks[17],
+			"",
+			"{\"trueValue\": true}",
+		},
+		{
+			controls.Groups[0].Checks[18],
+			"",
+			"{\"readOnlyPort\": 15000}",
+		},
+		{
+			controls.Groups[0].Checks[19],
+			"",
+			"{\"authentication\": { \"anonymous\": {\"enabled\": false}}}",
+		},
+		{
+			controls.Groups[0].Checks[20],
+			"",
+			"readOnlyPort: 15000",
+		},
+		{
+			controls.Groups[0].Checks[21],
+			"",
+			"readOnlyPort: 15000",
+		},
+		{
+			controls.Groups[0].Checks[22],
+			"",
+			"authentication:\n  anonymous:\n    enabled: false",
+		},
+		{
+			controls.Groups[0].Checks[26],
+			"",
+			"currentMasterVersion: 1.12.7",
+		},
+		{
+			controls.Groups[0].Checks[27],
+			"--peer-client-cert-auth",
+			"",
+		},
+		{
+			controls.Groups[0].Checks[27],
+			"--abc=true --peer-client-cert-auth --efg=false",
+			"",
+		},
+		{
+			controls.Groups[0].Checks[27],
+			"--abc --peer-client-cert-auth --efg",
+			"",
+		},
+		{
+			controls.Groups[0].Checks[27],
+			"--peer-client-cert-auth=true",
+			"",
+		},
+		{
+			controls.Groups[0].Checks[27],
+			"--abc --peer-client-cert-auth=true --efg",
+			"",
+		},
+		{
+			controls.Groups[0].Checks[28],
+			"--abc --peer-client-cert-auth=false --efg",
+			"",
 		},
 	}
 
 	for _, c := range cases {
-		res := c.Tests.execute(c.str)
-		if !res {
-			t.Errorf("%s, expected:%v, got:%v\n", c.Text, true, res)
-		}
+		t.Run(c.Text, func(t *testing.T) {
+			c.Check.AuditOutput = c.str
+			c.Check.AuditConfigOutput = c.strConfig
+			res, err := c.Check.execute()
+			if err != nil {
+				t.Errorf(err.Error())
+			}
+			if !res.testResult {
+				t.Errorf("expected:%v, got:%v", true, res)
+			}
+		})
+	}
+}
+
+func TestTestExecuteExceptions(t *testing.T) {
+
+	cases := []struct {
+		*Check
+		str string
+	}{
+		{
+			controls.Groups[0].Checks[23],
+			"this is not valid json {} at all",
+		},
+		{
+			controls.Groups[0].Checks[24],
+			"{\"key\": \"value\"}",
+		},
+		{
+			controls.Groups[0].Checks[25],
+			"broken } yaml\nenabled: true",
+		},
+		{
+			controls.Groups[0].Checks[26],
+			"currentMasterVersion: 1.11",
+		},
+		{
+			controls.Groups[0].Checks[26],
+			"currentMasterVersion: ",
+		},
+	}
+
+	for _, c := range cases {
+		t.Run(c.Text, func(t *testing.T) {
+			c.Check.AuditConfigOutput = c.str
+			res, err := c.Check.execute()
+			if err != nil {
+				t.Errorf(err.Error())
+			}
+			if res.testResult {
+				t.Errorf("expected:%v, got:%v", false, res)
+			}
+		})
+	}
+}
+
+func TestTestUnmarshal(t *testing.T) {
+	type kubeletConfig struct {
+		Kind       string
+		ApiVersion string
+		Address    string
+	}
+	cases := []struct {
+		content        string
+		jsonInterface  interface{}
+		expectedToFail bool
+	}{
+		{
+			`{
+			"kind": "KubeletConfiguration",
+			"apiVersion": "kubelet.config.k8s.io/v1beta1",
+			"address": "0.0.0.0"
+			}
+			`,
+			kubeletConfig{},
+			false,
+		}, {
+			`
+kind: KubeletConfiguration
+address: 0.0.0.0
+apiVersion: kubelet.config.k8s.io/v1beta1
+authentication:
+  anonymous:
+    enabled: false
+  webhook:
+    cacheTTL: 2m0s
+  enabled: true
+  x509:
+    clientCAFile: /etc/kubernetes/pki/ca.crt
+tlsCipherSuites:
+  - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
+  - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
+`,
+			kubeletConfig{},
+			false,
+		},
+		{
+			`
+kind: ddress: 0.0.0.0
+apiVersion: kubelet.config.k8s.io/v1beta
+`,
+			kubeletConfig{},
+			true,
+		},
+	}
+
+	for id, c := range cases {
+		t.Run(fmt.Sprintf("%d", id), func(t *testing.T) {
+			err := unmarshal(c.content, &c.jsonInterface)
+			if err != nil {
+				if !c.expectedToFail {
+					t.Errorf("should pass, got error:%v", err)
+				}
+			} else {
+				if c.expectedToFail {
+					t.Errorf("should fail, but passed")
+				}
+			}
+		})
+	}
+}
+
+func TestExecuteJSONPath(t *testing.T) {
+	type kubeletConfig struct {
+		Kind       string
+		ApiVersion string
+		Address    string
+	}
+	cases := []struct {
+		name           string
+		jsonPath       string
+		jsonInterface  kubeletConfig
+		expectedResult string
+		expectedToFail bool
+	}{
+		{
+			"JSONPath parse works, results don't match",
+			"{.Kind}",
+			kubeletConfig{
+				Kind:       "KubeletConfiguration",
+				ApiVersion: "kubelet.config.k8s.io/v1beta1",
+				Address:    "127.0.0.0",
+			},
+			"blah",
+			true,
+		},
+		{
+			"JSONPath parse works, results match",
+			"{.Kind}",
+			kubeletConfig{
+				Kind:       "KubeletConfiguration",
+				ApiVersion: "kubelet.config.k8s.io/v1beta1",
+				Address:    "127.0.0.0",
+			},
+			"KubeletConfiguration",
+			false,
+		},
+		{
+			"JSONPath parse fails",
+			"{.ApiVersion",
+			kubeletConfig{
+				Kind:       "KubeletConfiguration",
+				ApiVersion: "kubelet.config.k8s.io/v1beta1",
+				Address:    "127.0.0.0",
+			},
+			"",
+			true,
+		},
+	}
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			result, err := executeJSONPath(c.jsonPath, c.jsonInterface)
+			if err != nil && !c.expectedToFail {
+				t.Fatalf("jsonPath:%q, expectedResult:%q got:%v", c.jsonPath, c.expectedResult, err)
+			}
+			if c.expectedResult != result && !c.expectedToFail {
+				t.Errorf("jsonPath:%q, expectedResult:%q got:%q", c.jsonPath, c.expectedResult, result)
+			}
+		})
+	}
+}
+
+func TestAllElementsValid(t *testing.T) {
+	cases := []struct {
+		source []string
+		target []string
+		valid  bool
+	}{
+		{
+			source: []string{},
+			target: []string{},
+			valid:  true,
+		},
+		{
+			source: []string{"blah"},
+			target: []string{},
+			valid:  false,
+		},
+		{
+			source: []string{},
+			target: []string{"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
+				"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
+				"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
+				"TLS_RSA_WITH_AES_256_GCM_SHA384", "TLS_RSA_WITH_AES_128_GCM_SHA256"},
+			valid: false,
+		},
+		{
+			source: []string{"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"},
+			target: []string{"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
+				"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
+				"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
+				"TLS_RSA_WITH_AES_256_GCM_SHA384", "TLS_RSA_WITH_AES_128_GCM_SHA256"},
+			valid: true,
+		},
+		{
+			source: []string{"blah"},
+			target: []string{"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
+				"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
+				"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
+				"TLS_RSA_WITH_AES_256_GCM_SHA384", "TLS_RSA_WITH_AES_128_GCM_SHA256"},
+			valid: false,
+		},
+		{
+			source: []string{"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "blah"},
+			target: []string{"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
+				"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
+				"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
+				"TLS_RSA_WITH_AES_256_GCM_SHA384", "TLS_RSA_WITH_AES_128_GCM_SHA256"},
+			valid: false,
+		},
+	}
+	for id, c := range cases {
+		t.Run(fmt.Sprintf("%d", id), func(t *testing.T) {
+			if !allElementsValid(c.source, c.target) && c.valid {
+				t.Errorf("Not All Elements in %q are found in %q", c.source, c.target)
+			}
+		})
+	}
+}
+
+func TestSplitAndRemoveLastSeparator(t *testing.T) {
+	cases := []struct {
+		source     string
+		valid      bool
+		elementCnt int
+	}{
+		{
+			source:     "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256",
+			valid:      true,
+			elementCnt: 8,
+		},
+		{
+			source:     "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,",
+			valid:      true,
+			elementCnt: 2,
+		},
+		{
+			source:     "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,",
+			valid:      true,
+			elementCnt: 2,
+		},
+		{
+			source:     "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, ",
+			valid:      true,
+			elementCnt: 2,
+		},
+		{
+			source:     " TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,",
+			valid:      true,
+			elementCnt: 2,
+		},
+	}
+
+	for id, c := range cases {
+		t.Run(fmt.Sprintf("%d", id), func(t *testing.T) {
+			as := splitAndRemoveLastSeparator(c.source, defaultArraySeparator)
+			if len(as) == 0 && c.valid {
+				t.Errorf("Split did not work with %q", c.source)
+			}
+
+			if c.elementCnt != len(as) {
+				t.Errorf("Split did not work with %q expected: %d got: %d", c.source, c.elementCnt, len(as))
+			}
+		})
+	}
+}
+
+func TestCompareOp(t *testing.T) {
+	cases := []struct {
+		label                 string
+		op                    string
+		flagVal               string
+		compareValue          string
+		expectedResultPattern string
+		testResult            bool
+	}{
+		// Test Op not matching
+		{label: "empty - op", op: "", flagVal: "", compareValue: "", expectedResultPattern: "", testResult: false},
+		{label: "op=blah", op: "blah", flagVal: "foo", compareValue: "bar", expectedResultPattern: "", testResult: false},
+
+		// Test Op "eq"
+		{label: "op=eq, both empty", op: "eq", flagVal: "", compareValue: "", expectedResultPattern: "'' is equal to ''", testResult: true},
+
+		{label: "op=eq, true==true", op: "eq", flagVal: "true",
+			compareValue:          "true",
+			expectedResultPattern: "'true' is equal to 'true'",
+			testResult:            true},
+
+		{label: "op=eq, false==false", op: "eq", flagVal: "false",
+			compareValue:          "false",
+			expectedResultPattern: "'false' is equal to 'false'",
+			testResult:            true},
+
+		{label: "op=eq, false==true", op: "eq", flagVal: "false",
+			compareValue:          "true",
+			expectedResultPattern: "'false' is equal to 'true'",
+			testResult:            false},
+
+		{label: "op=eq, strings match", op: "eq", flagVal: "KubeletConfiguration",
+			compareValue:          "KubeletConfiguration",
+			expectedResultPattern: "'KubeletConfiguration' is equal to 'KubeletConfiguration'",
+			testResult:            true},
+
+		{label: "op=eq, flagVal=empty", op: "eq", flagVal: "",
+			compareValue:          "KubeletConfiguration",
+			expectedResultPattern: "'' is equal to 'KubeletConfiguration'",
+			testResult:            false},
+
+		{label: "op=eq, compareValue=empty", op: "eq", flagVal: "KubeletConfiguration",
+			compareValue:          "",
+			expectedResultPattern: "'KubeletConfiguration' is equal to ''",
+			testResult:            false},
+
+		// Test Op "noteq"
+		{label: "op=noteq, both empty", op: "noteq", flagVal: "",
+			compareValue: "", expectedResultPattern: "'' is not equal to ''",
+			testResult: false},
+
+		{label: "op=noteq, true!=true", op: "noteq", flagVal: "true",
+			compareValue:          "true",
+			expectedResultPattern: "'true' is not equal to 'true'",
+			testResult:            false},
+
+		{label: "op=noteq, false!=false", op: "noteq", flagVal: "false",
+			compareValue:          "false",
+			expectedResultPattern: "'false' is not equal to 'false'",
+			testResult:            false},
+
+		{label: "op=noteq, false!=true", op: "noteq", flagVal: "false",
+			compareValue:          "true",
+			expectedResultPattern: "'false' is not equal to 'true'",
+			testResult:            true},
+
+		{label: "op=noteq, strings match", op: "noteq", flagVal: "KubeletConfiguration",
+			compareValue:          "KubeletConfiguration",
+			expectedResultPattern: "'KubeletConfiguration' is not equal to 'KubeletConfiguration'",
+			testResult:            false},
+
+		{label: "op=noteq, flagVal=empty", op: "noteq", flagVal: "",
+			compareValue:          "KubeletConfiguration",
+			expectedResultPattern: "'' is not equal to 'KubeletConfiguration'",
+			testResult:            true},
+
+		{label: "op=noteq, compareValue=empty", op: "noteq", flagVal: "KubeletConfiguration",
+			compareValue:          "",
+			expectedResultPattern: "'KubeletConfiguration' is not equal to ''",
+			testResult:            true},
+
+		// Test Op "gt"
+		{label: "op=gt, both empty", op: "gt", flagVal: "",
+			compareValue: "", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+		{label: "op=gt, 0 > 0", op: "gt", flagVal: "0",
+			compareValue: "0", expectedResultPattern: "0 is greater than 0",
+			testResult: false},
+		{label: "op=gt, 4 > 5", op: "gt", flagVal: "4",
+			compareValue: "5", expectedResultPattern: "4 is greater than 5",
+			testResult: false},
+		{label: "op=gt, 5 > 4", op: "gt", flagVal: "5",
+			compareValue: "4", expectedResultPattern: "5 is greater than 4",
+			testResult: true},
+		{label: "op=gt, 5 > 5", op: "gt", flagVal: "5",
+			compareValue: "5", expectedResultPattern: "5 is greater than 5",
+			testResult: false},
+		{label: "op=gt, Pikachu > 5", op: "gt", flagVal: "Pikachu",
+			compareValue: "5", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+		{label: "op=gt, 5 > Bulbasaur", op: "gt", flagVal: "5",
+			compareValue: "Bulbasaur", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+		// Test Op "lt"
+		{label: "op=lt, both empty", op: "lt", flagVal: "",
+			compareValue: "", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+		{label: "op=lt, 0 < 0", op: "lt", flagVal: "0",
+			compareValue: "0", expectedResultPattern: "0 is lower than 0",
+			testResult: false},
+		{label: "op=lt, 4 < 5", op: "lt", flagVal: "4",
+			compareValue: "5", expectedResultPattern: "4 is lower than 5",
+			testResult: true},
+		{label: "op=lt, 5 < 4", op: "lt", flagVal: "5",
+			compareValue: "4", expectedResultPattern: "5 is lower than 4",
+			testResult: false},
+		{label: "op=lt, 5 < 5", op: "lt", flagVal: "5",
+			compareValue: "5", expectedResultPattern: "5 is lower than 5",
+			testResult: false},
+		{label: "op=lt, Charmander < 5", op: "lt", flagVal: "Charmander",
+			compareValue: "5", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+		{label: "op=lt, 5 < Charmeleon", op: "lt", flagVal: "5",
+			compareValue: "Charmeleon", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+		// Test Op "gte"
+		{label: "op=gte, both empty", op: "gte", flagVal: "",
+			compareValue: "", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+		{label: "op=gte, 0 >= 0", op: "gte", flagVal: "0",
+			compareValue: "0", expectedResultPattern: "0 is greater or equal to 0",
+			testResult: true},
+		{label: "op=gte, 4 >= 5", op: "gte", flagVal: "4",
+			compareValue: "5", expectedResultPattern: "4 is greater or equal to 5",
+			testResult: false},
+		{label: "op=gte, 5 >= 4", op: "gte", flagVal: "5",
+			compareValue: "4", expectedResultPattern: "5 is greater or equal to 4",
+			testResult: true},
+		{label: "op=gte, 5 >= 5", op: "gte", flagVal: "5",
+			compareValue: "5", expectedResultPattern: "5 is greater or equal to 5",
+			testResult: true},
+		{label: "op=gte, Ekans >= 5", op: "gte", flagVal: "Ekans",
+			compareValue: "5", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+		{label: "op=gte, 4 >= Zubat", op: "gte", flagVal: "4",
+			compareValue: "Zubat", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+		// Test Op "lte"
+		{label: "op=lte, both empty", op: "lte", flagVal: "",
+			compareValue: "", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+		{label: "op=lte, 0 <= 0", op: "lte", flagVal: "0",
+			compareValue: "0", expectedResultPattern: "0 is lower or equal to 0",
+			testResult: true},
+		{label: "op=lte, 4 <= 5", op: "lte", flagVal: "4",
+			compareValue: "5", expectedResultPattern: "4 is lower or equal to 5",
+			testResult: true},
+		{label: "op=lte, 5 <= 4", op: "lte", flagVal: "5",
+			compareValue: "4", expectedResultPattern: "5 is lower or equal to 4",
+			testResult: false},
+		{label: "op=lte, 5 <= 5", op: "lte", flagVal: "5",
+			compareValue: "5", expectedResultPattern: "5 is lower or equal to 5",
+			testResult: true},
+		{label: "op=lte, Venomoth <= 4", op: "lte", flagVal: "Venomoth",
+			compareValue: "4", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+		{label: "op=lte, 5 <= Meowth", op: "lte", flagVal: "5",
+			compareValue: "Meowth", expectedResultPattern: "Invalid Number(s) used for comparison",
+			testResult: false},
+
+		// Test Op "has"
+		{label: "op=has, both empty", op: "has", flagVal: "",
+			compareValue: "", expectedResultPattern: "'' has ''",
+			testResult: true},
+		{label: "op=has, flagVal=empty", op: "has", flagVal: "",
+			compareValue: "blah", expectedResultPattern: "'' has 'blah'",
+			testResult: false},
+		{label: "op=has, compareValue=empty", op: "has", flagVal: "blah",
+			compareValue: "", expectedResultPattern: "'blah' has ''",
+			testResult: true},
+		{label: "op=has, 'blah' has 'la'", op: "has", flagVal: "blah",
+			compareValue: "la", expectedResultPattern: "'blah' has 'la'",
+			testResult: true},
+		{label: "op=has, 'blah' has 'LA'", op: "has", flagVal: "blah",
+			compareValue: "LA", expectedResultPattern: "'blah' has 'LA'",
+			testResult: false},
+		{label: "op=has, 'blah' has 'lo'", op: "has", flagVal: "blah",
+			compareValue: "lo", expectedResultPattern: "'blah' has 'lo'",
+			testResult: false},
+
+		// Test Op "nothave"
+		{label: "op=nothave, both empty", op: "nothave", flagVal: "",
+			compareValue: "", expectedResultPattern: " '' not have ''",
+			testResult: false},
+		{label: "op=nothave, flagVal=empty", op: "nothave", flagVal: "",
+			compareValue: "blah", expectedResultPattern: " '' not have 'blah'",
+			testResult: true},
+		{label: "op=nothave, compareValue=empty", op: "nothave", flagVal: "blah",
+			compareValue: "", expectedResultPattern: " 'blah' not have ''",
+			testResult: false},
+		{label: "op=nothave, 'blah' not have 'la'", op: "nothave", flagVal: "blah",
+			compareValue: "la", expectedResultPattern: " 'blah' not have 'la'",
+			testResult: false},
+		{label: "op=nothave, 'blah' not have 'LA'", op: "nothave", flagVal: "blah",
+			compareValue: "LA", expectedResultPattern: " 'blah' not have 'LA'",
+			testResult: true},
+		{label: "op=nothave, 'blah' not have 'lo'", op: "nothave", flagVal: "blah",
+			compareValue: "lo", expectedResultPattern: " 'blah' not have 'lo'",
+			testResult: true},
+
+		// Test Op "regex"
+		{label: "op=regex, both empty", op: "regex", flagVal: "",
+			compareValue: "", expectedResultPattern: " '' matched by ''",
+			testResult: true},
+		{label: "op=regex, flagVal=empty", op: "regex", flagVal: "",
+			compareValue: "blah", expectedResultPattern: " '' matched by 'blah'",
+			testResult: false},
+
+		// Test Op "valid_elements"
+		{label: "op=valid_elements, valid_elements both empty", op: "valid_elements", flagVal: "",
+			compareValue: "", expectedResultPattern: "'' contains valid elements from ''",
+			testResult: true},
+
+		{label: "op=valid_elements, valid_elements flagVal empty", op: "valid_elements", flagVal: "",
+			compareValue: "a,b", expectedResultPattern: "'' contains valid elements from 'a,b'",
+			testResult: false},
+
+		{label: "op=valid_elements, valid_elements expectedResultPattern empty", op: "valid_elements", flagVal: "a,b",
+			compareValue: "", expectedResultPattern: "'a,b' contains valid elements from ''",
+			testResult: false},
+		// Test Op "bitmask"
+		{label: "op=bitmask, 644 AND 640", op: "bitmask", flagVal: "640",
+			compareValue: "644", expectedResultPattern: "bitmask '640' AND '644'",
+			testResult: true},
+		{label: "op=bitmask, 644 AND 777", op: "bitmask", flagVal: "777",
+			compareValue: "644", expectedResultPattern: "bitmask '777' AND '644'",
+			testResult: false},
+		{label: "op=bitmask, 644 AND 444", op: "bitmask", flagVal: "444",
+			compareValue: "644", expectedResultPattern: "bitmask '444' AND '644'",
+			testResult: true},
+		{label: "op=bitmask, 644 AND 211", op: "bitmask", flagVal: "211",
+			compareValue: "644", expectedResultPattern: "bitmask '211' AND '644'",
+			testResult: false},
+		{label: "op=bitmask, Harry AND 211", op: "bitmask", flagVal: "Harry",
+			compareValue: "644", expectedResultPattern: "Not numeric value - flag: Harry",
+			testResult: false},
+		{label: "op=bitmask, 644 AND Potter", op: "bitmask", flagVal: "211",
+			compareValue: "Potter", expectedResultPattern: "Not numeric value - flag: Potter",
+			testResult: false},
+	}
+
+	for _, c := range cases {
+		t.Run(c.label, func(t *testing.T) {
+			expectedResultPattern, testResult := compareOp(c.op, c.flagVal, c.compareValue)
+			if expectedResultPattern != c.expectedResultPattern {
+				t.Errorf("'expectedResultPattern' did not match - op: %q expected:%q  got:%q", c.op, c.expectedResultPattern, expectedResultPattern)
+			}
+
+			if testResult != c.testResult {
+				t.Errorf("'testResult' did not match - lop: %q expected:%t  got:%t", c.op, c.testResult, testResult)
+			}
+		})
+	}
+}
+
+func TestToNumeric(t *testing.T) {
+	cases := []struct {
+		firstValue     string
+		secondValue    string
+		expectedToFail bool
+	}{
+		{
+			firstValue:     "a",
+			secondValue:    "b",
+			expectedToFail: true,
+		},
+		{
+			firstValue:     "5",
+			secondValue:    "b",
+			expectedToFail: true,
+		},
+		{
+			firstValue:     "5",
+			secondValue:    "6",
+			expectedToFail: false,
+		},
+	}
+
+	for id, c := range cases {
+		t.Run(fmt.Sprintf("%d", id), func(t *testing.T) {
+			f, s, err := toNumeric(c.firstValue, c.secondValue)
+			if c.expectedToFail && err == nil {
+				t.Errorf("Expected error while converting %s and %s", c.firstValue, c.secondValue)
+			}
+
+			if !c.expectedToFail && (f != 5 || s != 6) {
+				t.Errorf("Expected to return %d,%d - got %d,%d", 5, 6, f, s)
+			}
+		})
 	}
 }

cmd/common.go

@@ -15,105 +15,111 @@
 package cmd
 
 import (
+	"bufio"
+	"encoding/json"
 	"fmt"
 	"io/ioutil"
+	"os"
+	"path/filepath"
+	"sort"
+	"strconv"
+	"strings"
 
 	"github.com/aquasecurity/kube-bench/check"
 	"github.com/golang/glog"
 	"github.com/spf13/viper"
 )
 
-var (
-	errmsgs string
-)
+// NewRunFilter constructs a Predicate based on FilterOpts which determines whether tested Checks should be run or not.
+func NewRunFilter(opts FilterOpts) (check.Predicate, error) {
 
-func runChecks(t check.NodeType) {
-	var summary check.Summary
-	var nodetype string
-	var file string
-	var err error
-	var typeConf *viper.Viper
-
-	switch t {
-	case check.MASTER:
-		file = masterFile
-		nodetype = "master"
-	case check.NODE:
-		file = nodeFile
-		nodetype = "node"
-	case check.FEDERATED:
-		file = federatedFile
-		nodetype = "federated"
+	if opts.CheckList != "" && opts.GroupList != "" {
+		return nil, fmt.Errorf("group option and check option can't be used together")
 	}
 
-	ver := getKubeVersion()
-	path := fmt.Sprintf("%s/%s", cfgDir, ver)
+	var groupIDs map[string]bool
+	if opts.GroupList != "" {
+		groupIDs = cleanIDs(opts.GroupList)
+	}
 
-	def := fmt.Sprintf("%s/%s", path, file)
-	in, err := ioutil.ReadFile(def)
+	var checkIDs map[string]bool
+	if opts.CheckList != "" {
+		checkIDs = cleanIDs(opts.CheckList)
+	}
+
+	return func(g *check.Group, c *check.Check) bool {
+		var test = true
+		if len(groupIDs) > 0 {
+			_, ok := groupIDs[g.ID]
+			test = test && ok
+		}
+
+		if len(checkIDs) > 0 {
+			_, ok := checkIDs[c.ID]
+			test = test && ok
+		}
+
+		test = test && (opts.Scored && c.Scored || opts.Unscored && !c.Scored)
+
+		return test
+	}, nil
+}
+
+func runChecks(nodetype check.NodeType, testYamlFile string) {
+	// Verify config file was loaded into Viper during Cobra sub-command initialization.
+	if configFileError != nil {
+		colorPrint(check.FAIL, fmt.Sprintf("Failed to read config file: %v\n", configFileError))
+		os.Exit(1)
+	}
+
+	in, err := ioutil.ReadFile(testYamlFile)
 	if err != nil {
-		exitWithError(fmt.Errorf("error opening %s controls file: %v", t, err))
+		exitWithError(fmt.Errorf("error opening %s test file: %v", testYamlFile, err))
 	}
 
-	// Merge kubernetes version specific config if any.
-	viper.SetConfigFile(path + "/config.yaml")
-	err = viper.MergeInConfig()
+	glog.V(1).Info(fmt.Sprintf("Using test file: %s\n", testYamlFile))
+
+	// Get the viper config for this section of tests
+	typeConf := viper.Sub(string(nodetype))
+	if typeConf == nil {
+		colorPrint(check.FAIL, fmt.Sprintf("No config settings for %s\n", string(nodetype)))
+		os.Exit(1)
+	}
+
+	// Get the set of executables we need for this section of the tests
+	binmap, err := getBinaries(typeConf, nodetype)
+
+	// Checks that the executables we need for the section are running.
 	if err != nil {
-		continueWithError(err, fmt.Sprintf("Reading %s specific configuration file", ver))
+		glog.V(1).Info(fmt.Sprintf("failed to get a set of executables needed for tests: %v", err))
 	}
-	typeConf = viper.Sub(nodetype)
 
-	// Get the set of exectuables and config files we care about on this type of node. This also
-	// checks that the executables we need for the node type are running.
-	binmap := getBinaries(typeConf)
-	confmap := getConfigFiles(typeConf)
+	confmap := getFiles(typeConf, "config")
+	svcmap := getFiles(typeConf, "service")
+	kubeconfmap := getFiles(typeConf, "kubeconfig")
+	cafilemap := getFiles(typeConf, "ca")
 
 	// Variable substitutions. Replace all occurrences of variables in controls files.
 	s := string(in)
 	s = makeSubstitutions(s, "bin", binmap)
 	s = makeSubstitutions(s, "conf", confmap)
+	s = makeSubstitutions(s, "svc", svcmap)
+	s = makeSubstitutions(s, "kubeconfig", kubeconfmap)
+	s = makeSubstitutions(s, "cafile", cafilemap)
 
-	glog.V(1).Info(fmt.Sprintf("Using config file: %s\n", viper.ConfigFileUsed()))
-	glog.V(1).Info(fmt.Sprintf("Using benchmark file: %s\n", def))
-
-	controls, err := check.NewControls(t, []byte(s))
+	controls, err := check.NewControls(nodetype, []byte(s))
 	if err != nil {
-		exitWithError(fmt.Errorf("error setting up %s controls: %v", t, err))
+		exitWithError(fmt.Errorf("error setting up %s controls: %v", nodetype, err))
 	}
 
-	if groupList != "" && checkList == "" {
-		ids := cleanIDs(groupList)
-		summary = controls.RunGroup(ids...)
-	} else if checkList != "" && groupList == "" {
-		ids := cleanIDs(checkList)
-		summary = controls.RunChecks(ids...)
-	} else if checkList != "" && groupList != "" {
-		exitWithError(fmt.Errorf("group option and check option can't be used together"))
-	} else {
-		summary = controls.RunGroup()
+	runner := check.NewRunner()
+	filter, err := NewRunFilter(filterOpts)
+	if err != nil {
+		exitWithError(fmt.Errorf("error setting up run filter: %v", err))
 	}
 
-	// if we successfully ran some tests and it's json format, ignore the warnings
-	if (summary.Fail > 0 || summary.Warn > 0 || summary.Pass > 0) && jsonFmt {
-		out, err := controls.JSON()
-		if err != nil {
-			exitWithError(fmt.Errorf("failed to output in JSON format: %v", err))
-		}
-
-		fmt.Println(string(out))
-	} else {
-		// if we want to store in PostgreSQL, convert to JSON and save it
-		if (summary.Fail > 0 || summary.Warn > 0 || summary.Pass > 0) && pgSql {
-			out, err := controls.JSON()
-			if err != nil {
-				exitWithError(fmt.Errorf("failed to output in JSON format: %v", err))
-			}
-
-			savePgsql(string(out))
-		} else {
-			prettyPrint(controls, summary)
-		}
-	}
+	controls.RunChecks(runner, filter)
+	controlsCollection = append(controlsCollection, controls)
 }
 
 // colorPrint outputs the state in a specific colour, along with a message string
@@ -124,41 +130,322 @@ func colorPrint(state check.State, s string) {
 
 // prettyPrint outputs the results to stdout in human-readable format
 func prettyPrint(r *check.Controls, summary check.Summary) {
-	colorPrint(check.INFO, fmt.Sprintf("%s %s\n", r.ID, r.Text))
-	for _, g := range r.Groups {
-		colorPrint(check.INFO, fmt.Sprintf("%s %s\n", g.ID, g.Text))
-		for _, c := range g.Checks {
-			colorPrint(c.State, fmt.Sprintf("%s %s\n", c.ID, c.Text))
-		}
-	}
-
-	fmt.Println()
-
-	// Print remediations.
-	if summary.Fail > 0 || summary.Warn > 0 {
-		colors[check.WARN].Printf("== Remediations ==\n")
+	// Print check results.
+	if !noResults {
+		colorPrint(check.INFO, fmt.Sprintf("%s %s\n", r.ID, r.Text))
 		for _, g := range r.Groups {
+			colorPrint(check.INFO, fmt.Sprintf("%s %s\n", g.ID, g.Text))
 			for _, c := range g.Checks {
-				if c.State != check.PASS {
-					fmt.Printf("%s %s\n", c.ID, c.Remediation)
+				colorPrint(c.State, fmt.Sprintf("%s %s\n", c.ID, c.Text))
+
+				if includeTestOutput && c.State == check.FAIL && len(c.ActualValue) > 0 {
+					printRawOutput(c.ActualValue)
 				}
 			}
 		}
+
 		fmt.Println()
 	}
 
-	// Print summary setting output color to highest severity.
-	var res check.State
-	if summary.Fail > 0 {
-		res = check.FAIL
-	} else if summary.Warn > 0 {
-		res = check.WARN
-	} else {
-		res = check.PASS
+	// Print remediations.
+	if !noRemediations {
+		if summary.Fail > 0 || summary.Warn > 0 {
+			colors[check.WARN].Printf("== Remediations ==\n")
+			for _, g := range r.Groups {
+				for _, c := range g.Checks {
+					if c.State == check.FAIL {
+						fmt.Printf("%s %s\n", c.ID, c.Remediation)
+					}
+					if c.State == check.WARN {
+						// Print the error if test failed due to problem with the audit command
+						if c.Reason != "" && c.Type != "manual" {
+							fmt.Printf("%s audit test did not run: %s\n", c.ID, c.Reason)
+						} else {
+							fmt.Printf("%s %s\n", c.ID, c.Remediation)
+						}
+					}
+				}
+			}
+			fmt.Println()
+		}
 	}
 
-	colors[res].Printf("== Summary ==\n")
-	fmt.Printf("%d checks PASS\n%d checks FAIL\n%d checks WARN\n",
-		summary.Pass, summary.Fail, summary.Warn,
-	)
+	// Print summary setting output color to highest severity.
+	if !noSummary {
+		var res check.State
+		if summary.Fail > 0 {
+			res = check.FAIL
+		} else if summary.Warn > 0 {
+			res = check.WARN
+		} else {
+			res = check.PASS
+		}
+
+		colors[res].Printf("== Summary ==\n")
+		fmt.Printf("%d checks PASS\n%d checks FAIL\n%d checks WARN\n%d checks INFO\n",
+			summary.Pass, summary.Fail, summary.Warn, summary.Info,
+		)
+	}
+}
+
+// loadConfig finds the correct config dir based on the kubernetes version,
+// merges any specific config.yaml file found with the main config
+// and returns the benchmark file to use.
+func loadConfig(nodetype check.NodeType, benchmarkVersion string) string {
+	var file string
+	var err error
+
+	switch nodetype {
+	case check.MASTER:
+		file = masterFile
+	case check.NODE:
+		file = nodeFile
+	case check.CONTROLPLANE:
+		file = controlplaneFile
+	case check.ETCD:
+		file = etcdFile
+	case check.POLICIES:
+		file = policiesFile
+	case check.MANAGEDSERVICES:
+		file = managedservicesFile
+	}
+
+	path, err := getConfigFilePath(benchmarkVersion, file)
+	if err != nil {
+		exitWithError(fmt.Errorf("can't find %s controls file in %s: %v", nodetype, cfgDir, err))
+	}
+
+	// Merge version-specific config if any.
+	mergeConfig(path)
+
+	return filepath.Join(path, file)
+}
+
+func mergeConfig(path string) error {
+	viper.SetConfigFile(path + "/config.yaml")
+	err := viper.MergeInConfig()
+	if err != nil {
+		if os.IsNotExist(err) {
+			glog.V(2).Info(fmt.Sprintf("No version-specific config.yaml file in %s", path))
+		} else {
+			return fmt.Errorf("couldn't read config file %s: %v", path+"/config.yaml", err)
+		}
+	}
+
+	glog.V(1).Info(fmt.Sprintf("Using config file: %s\n", viper.ConfigFileUsed()))
+
+	return nil
+}
+
+func mapToBenchmarkVersion(kubeToBenchmarkMap map[string]string, kv string) (string, error) {
+	kvOriginal := kv
+	cisVersion, found := kubeToBenchmarkMap[kv]
+	glog.V(2).Info(fmt.Sprintf("mapToBenchmarkVersion for k8sVersion: %q cisVersion: %q found: %t\n", kv, cisVersion, found))
+	for !found && (kv != defaultKubeVersion && !isEmpty(kv)) {
+		kv = decrementVersion(kv)
+		cisVersion, found = kubeToBenchmarkMap[kv]
+		glog.V(2).Info(fmt.Sprintf("mapToBenchmarkVersion for k8sVersion: %q cisVersion: %q found: %t\n", kv, cisVersion, found))
+	}
+
+	if !found {
+		glog.V(1).Info(fmt.Sprintf("mapToBenchmarkVersion unable to find a match for: %q", kvOriginal))
+		glog.V(3).Info(fmt.Sprintf("mapToBenchmarkVersion kubeToBenchmarkMap: %#v", kubeToBenchmarkMap))
+		return "", fmt.Errorf("unable to find a matching Benchmark Version match for kubernetes version: %s", kvOriginal)
+	}
+
+	return cisVersion, nil
+}
+
+func loadVersionMapping(v *viper.Viper) (map[string]string, error) {
+	kubeToBenchmarkMap := v.GetStringMapString("version_mapping")
+	if kubeToBenchmarkMap == nil || (len(kubeToBenchmarkMap) == 0) {
+		return nil, fmt.Errorf("config file is missing 'version_mapping' section")
+	}
+
+	return kubeToBenchmarkMap, nil
+}
+
+func loadTargetMapping(v *viper.Viper) (map[string][]string, error) {
+	benchmarkVersionToTargetsMap := v.GetStringMapStringSlice("target_mapping")
+	if len(benchmarkVersionToTargetsMap) == 0 {
+		return nil, fmt.Errorf("config file is missing 'target_mapping' section")
+	}
+
+	return benchmarkVersionToTargetsMap, nil
+}
+
+func getBenchmarkVersion(kubeVersion, benchmarkVersion string, v *viper.Viper) (bv string, err error) {
+	if !isEmpty(kubeVersion) && !isEmpty(benchmarkVersion) {
+		return "", fmt.Errorf("It is an error to specify both --version and --benchmark flags")
+	}
+
+	if isEmpty(benchmarkVersion) {
+		if isEmpty(kubeVersion) {
+			kubeVersion, err = getKubeVersion()
+			if err != nil {
+				return "", fmt.Errorf("Version check failed: %s\nAlternatively, you can specify the version with --version", err)
+			}
+		}
+
+		kubeToBenchmarkMap, err := loadVersionMapping(v)
+		if err != nil {
+			return "", err
+		}
+
+		benchmarkVersion, err = mapToBenchmarkVersion(kubeToBenchmarkMap, kubeVersion)
+		if err != nil {
+			return "", err
+		}
+
+		glog.V(2).Info(fmt.Sprintf("Mapped Kubernetes version: %s to Benchmark version: %s", kubeVersion, benchmarkVersion))
+	}
+
+	glog.V(1).Info(fmt.Sprintf("Kubernetes version: %q to Benchmark version: %q", kubeVersion, benchmarkVersion))
+	return benchmarkVersion, nil
+}
+
+// isMaster verify if master components are running on the node.
+func isMaster() bool {
+	return isThisNodeRunning(check.MASTER)
+}
+
+// isEtcd verify if etcd components are running on the node.
+func isEtcd() bool {
+	return isThisNodeRunning(check.ETCD)
+}
+
+func isThisNodeRunning(nodeType check.NodeType) bool {
+	glog.V(3).Infof("Checking if the current node is running %s components", nodeType)
+	nodeTypeConf := viper.Sub(string(nodeType))
+	if nodeTypeConf == nil {
+		glog.V(2).Infof("No config for %s components found", nodeType)
+		return false
+	}
+
+	components, err := getBinariesFunc(nodeTypeConf, nodeType)
+	if err != nil {
+		glog.V(2).Infof("Failed to find %s binaries: %v", nodeType, err)
+		return false
+	}
+	if len(components) == 0 {
+		glog.V(2).Infof("No %s binaries specified", nodeType)
+		return false
+	}
+
+	glog.V(2).Infof("Node is running %s components", nodeType)
+	return true
+}
+
+func writeOutput(controlsCollection []*check.Controls) {
+	sort.Slice(controlsCollection, func(i, j int) bool {
+		iid, _ := strconv.Atoi(controlsCollection[i].ID)
+		jid, _ := strconv.Atoi(controlsCollection[j].ID)
+		return iid < jid
+	})
+	if junitFmt {
+		writeJunitOutput(controlsCollection)
+		return
+	}
+	if jsonFmt {
+		writeJSONOutput(controlsCollection)
+		return
+	}
+	if pgSQL {
+		writePgsqlOutput(controlsCollection)
+		return
+	}
+	writeStdoutOutput(controlsCollection)
+}
+
+func writeJSONOutput(controlsCollection []*check.Controls) {
+	out, err := json.Marshal(controlsCollection)
+	if err != nil {
+		exitWithError(fmt.Errorf("failed to output in JSON format: %v", err))
+	}
+	printOutput(string(out), outputFile)
+}
+
+func writeJunitOutput(controlsCollection []*check.Controls) {
+	for _, controls := range controlsCollection {
+		out, err := controls.JUnit()
+		if err != nil {
+			exitWithError(fmt.Errorf("failed to output in JUnit format: %v", err))
+		}
+		printOutput(string(out), outputFile)
+	}
+}
+
+func writePgsqlOutput(controlsCollection []*check.Controls) {
+	for _, controls := range controlsCollection {
+		out, err := controls.JSON()
+		if err != nil {
+			exitWithError(fmt.Errorf("failed to output in Postgresql format: %v", err))
+		}
+		savePgsql(string(out))
+	}
+}
+
+func writeStdoutOutput(controlsCollection []*check.Controls) {
+	for _, controls := range controlsCollection {
+		summary := controls.Summary
+		prettyPrint(controls, summary)
+	}
+}
+
+func printRawOutput(output string) {
+	for _, row := range strings.Split(output, "\n") {
+		fmt.Println(fmt.Sprintf("\t %s", row))
+	}
+}
+
+func writeOutputToFile(output string, outputFile string) error {
+	file, err := os.Create(outputFile)
+	if err != nil {
+		return err
+	}
+	defer file.Close()
+
+	w := bufio.NewWriter(file)
+	fmt.Fprintln(w, output)
+	return w.Flush()
+}
+
+func printOutput(output string, outputFile string) {
+	if outputFile == "" {
+		fmt.Println(output)
+	} else {
+		err := writeOutputToFile(output, outputFile)
+		if err != nil {
+			exitWithError(fmt.Errorf("Failed to write to output file %s: %v", outputFile, err))
+		}
+	}
+}
+
+// validTargets helps determine if the targets
+// are legitimate for the benchmarkVersion.
+func validTargets(benchmarkVersion string, targets []string, v *viper.Viper) (bool, error) {
+	benchmarkVersionToTargetsMap, err := loadTargetMapping(v)
+	if err != nil {
+		return false, err
+	}
+	providedTargets, found := benchmarkVersionToTargetsMap[benchmarkVersion]
+	if !found {
+		return false, fmt.Errorf("No targets configured for %s", benchmarkVersion)
+	}
+
+	for _, pt := range targets {
+		f := false
+		for _, t := range providedTargets {
+			if pt == strings.ToLower(t) {
+				f = true
+				break
+			}
+		}
+
+		if !f {
+			return false, nil
+		}
+	}
+
+	return true, nil
 }

cmd/common_test.go

@@ -0,0 +1,601 @@
+// Copyright © 2017-2019 Aqua Security Software Ltd. <info@aquasec.com>
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package cmd
+
+import (
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path"
+	"path/filepath"
+	"testing"
+	"time"
+
+	"github.com/aquasecurity/kube-bench/check"
+	"github.com/spf13/viper"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestNewRunFilter(t *testing.T) {
+
+	type TestCase struct {
+		Name       string
+		FilterOpts FilterOpts
+		Group      *check.Group
+		Check      *check.Check
+
+		Expected bool
+	}
+
+	testCases := []TestCase{
+		{
+			Name:       "Should return true when scored flag is enabled and check is scored",
+			FilterOpts: FilterOpts{Scored: true, Unscored: false},
+			Group:      &check.Group{},
+			Check:      &check.Check{Scored: true},
+			Expected:   true,
+		},
+		{
+			Name:       "Should return false when scored flag is enabled and check is not scored",
+			FilterOpts: FilterOpts{Scored: true, Unscored: false},
+			Group:      &check.Group{},
+			Check:      &check.Check{Scored: false},
+			Expected:   false,
+		},
+
+		{
+			Name:       "Should return true when unscored flag is enabled and check is not scored",
+			FilterOpts: FilterOpts{Scored: false, Unscored: true},
+			Group:      &check.Group{},
+			Check:      &check.Check{Scored: false},
+			Expected:   true,
+		},
+		{
+			Name:       "Should return false when unscored flag is enabled and check is scored",
+			FilterOpts: FilterOpts{Scored: false, Unscored: true},
+			Group:      &check.Group{},
+			Check:      &check.Check{Scored: true},
+			Expected:   false,
+		},
+
+		{
+			Name:       "Should return true when group flag contains group's ID",
+			FilterOpts: FilterOpts{Scored: true, Unscored: true, GroupList: "G1,G2,G3"},
+			Group:      &check.Group{ID: "G2"},
+			Check:      &check.Check{},
+			Expected:   true,
+		},
+		{
+			Name:       "Should return false when group flag doesn't contain group's ID",
+			FilterOpts: FilterOpts{GroupList: "G1,G3"},
+			Group:      &check.Group{ID: "G2"},
+			Check:      &check.Check{},
+			Expected:   false,
+		},
+
+		{
+			Name:       "Should return true when check flag contains check's ID",
+			FilterOpts: FilterOpts{Scored: true, Unscored: true, CheckList: "C1,C2,C3"},
+			Group:      &check.Group{},
+			Check:      &check.Check{ID: "C2"},
+			Expected:   true,
+		},
+		{
+			Name:       "Should return false when check flag doesn't contain check's ID",
+			FilterOpts: FilterOpts{CheckList: "C1,C3"},
+			Group:      &check.Group{},
+			Check:      &check.Check{ID: "C2"},
+			Expected:   false,
+		},
+	}
+
+	for _, testCase := range testCases {
+		t.Run(testCase.Name, func(t *testing.T) {
+			filter, _ := NewRunFilter(testCase.FilterOpts)
+			assert.Equal(t, testCase.Expected, filter(testCase.Group, testCase.Check))
+		})
+	}
+
+	t.Run("Should return error when both group and check flags are used", func(t *testing.T) {
+		// given
+		opts := FilterOpts{GroupList: "G1", CheckList: "C1"}
+		// when
+		_, err := NewRunFilter(opts)
+		// then
+		assert.EqualError(t, err, "group option and check option can't be used together")
+	})
+
+}
+
+func TestIsMaster(t *testing.T) {
+	testCases := []struct {
+		name            string
+		cfgFile         string
+		getBinariesFunc func(*viper.Viper, check.NodeType) (map[string]string, error)
+		isMaster        bool
+	}{
+		{
+			name:    "valid config, is master and all components are running",
+			cfgFile: "../cfg/config.yaml",
+			getBinariesFunc: func(viper *viper.Viper, nt check.NodeType) (strings map[string]string, i error) {
+				return map[string]string{"apiserver": "kube-apiserver"}, nil
+			},
+			isMaster: true,
+		},
+		{
+			name:    "valid config, is master and but not all components are running",
+			cfgFile: "../cfg/config.yaml",
+			getBinariesFunc: func(viper *viper.Viper, nt check.NodeType) (strings map[string]string, i error) {
+				return map[string]string{}, nil
+			},
+			isMaster: false,
+		},
+		{
+			name:    "valid config, is master, not all components are running and fails to find all binaries",
+			cfgFile: "../cfg/config.yaml",
+			getBinariesFunc: func(viper *viper.Viper, nt check.NodeType) (strings map[string]string, i error) {
+				return map[string]string{}, errors.New("failed to find binaries")
+			},
+			isMaster: false,
+		},
+		{
+			name:     "valid config, does not include master",
+			cfgFile:  "../hack/node_only.yaml",
+			isMaster: false,
+		},
+	}
+	cfgDirOld := cfgDir
+	cfgDir = "../cfg"
+	defer func() {
+		cfgDir = cfgDirOld
+	}()
+
+	execCode := `#!/bin/sh
+	echo "Server Version: v1.13.10"
+	`
+	restore, err := fakeExecutableInPath("kubectl", execCode)
+	if err != nil {
+		t.Fatal("Failed when calling fakeExecutableInPath ", err)
+	}
+	defer restore()
+
+	for _, tc := range testCases {
+		cfgFile = tc.cfgFile
+		initConfig()
+
+		oldGetBinariesFunc := getBinariesFunc
+		getBinariesFunc = tc.getBinariesFunc
+		defer func() {
+			getBinariesFunc = oldGetBinariesFunc
+			cfgFile = ""
+		}()
+
+		assert.Equal(t, tc.isMaster, isMaster(), tc.name)
+	}
+}
+
+func TestMapToCISVersion(t *testing.T) {
+
+	viperWithData, err := loadConfigForTest()
+	if err != nil {
+		t.Fatalf("Unable to load config file %v", err)
+	}
+	kubeToBenchmarkMap, err := loadVersionMapping(viperWithData)
+	if err != nil {
+		t.Fatalf("Unable to load config file %v", err)
+	}
+
+	cases := []struct {
+		kubeVersion string
+		succeed     bool
+		exp         string
+		expErr      string
+	}{
+		{kubeVersion: "1.9", succeed: false, exp: "", expErr: "unable to find a matching Benchmark Version match for kubernetes version: 1.9"},
+		{kubeVersion: "1.11", succeed: false, exp: "", expErr: "unable to find a matching Benchmark Version match for kubernetes version: 1.11"},
+		{kubeVersion: "1.12", succeed: false, exp: "", expErr: "unable to find a matching Benchmark Version match for kubernetes version: 1.12"},
+		{kubeVersion: "1.13", succeed: false, exp: "", expErr: "unable to find a matching Benchmark Version match for kubernetes version: 1.13"},
+		{kubeVersion: "1.14", succeed: false, exp: "", expErr: "unable to find a matching Benchmark Version match for kubernetes version: 1.14"},
+		{kubeVersion: "1.15", succeed: true, exp: "cis-1.5"},
+		{kubeVersion: "1.16", succeed: true, exp: "cis-1.6"},
+		{kubeVersion: "1.17", succeed: true, exp: "cis-1.6"},
+		{kubeVersion: "1.18", succeed: true, exp: "cis-1.6"},
+		{kubeVersion: "1.19", succeed: true, exp: "cis-1.6"},
+		{kubeVersion: "gke-1.0", succeed: true, exp: "gke-1.0"},
+		{kubeVersion: "ocp-3.10", succeed: true, exp: "rh-0.7"},
+		{kubeVersion: "ocp-3.11", succeed: true, exp: "rh-0.7"},
+		{kubeVersion: "unknown", succeed: false, exp: "", expErr: "unable to find a matching Benchmark Version match for kubernetes version: unknown"},
+	}
+	for _, c := range cases {
+		rv, err := mapToBenchmarkVersion(kubeToBenchmarkMap, c.kubeVersion)
+		if c.succeed {
+			if err != nil {
+				t.Errorf("[%q]-Unexpected error: %v", c.kubeVersion, err)
+			}
+
+			if len(rv) == 0 {
+				t.Errorf("[%q]-missing return value", c.kubeVersion)
+			}
+
+			if c.exp != rv {
+				t.Errorf("[%q]- expected %q but Got %q", c.kubeVersion, c.exp, rv)
+			}
+
+		} else {
+			if c.exp != rv {
+				t.Errorf("[%q]-mapToBenchmarkVersion kubeversion: %q Got %q expected %s", c.kubeVersion, c.kubeVersion, rv, c.exp)
+			}
+
+			if c.expErr != err.Error() {
+				t.Errorf("[%q]-mapToBenchmarkVersion expected Error: %q instead Got %q", c.kubeVersion, c.expErr, err.Error())
+			}
+		}
+	}
+}
+
+func TestLoadVersionMapping(t *testing.T) {
+	setDefault := func(v *viper.Viper, key string, value interface{}) *viper.Viper {
+		v.SetDefault(key, value)
+		return v
+	}
+
+	viperWithData, err := loadConfigForTest()
+	if err != nil {
+		t.Fatalf("Unable to load config file %v", err)
+	}
+
+	cases := []struct {
+		n       string
+		v       *viper.Viper
+		succeed bool
+	}{
+		{n: "empty", v: viper.New(), succeed: false},
+		{
+			n:       "novals",
+			v:       setDefault(viper.New(), "version_mapping", "novals"),
+			succeed: false,
+		},
+		{
+			n:       "good",
+			v:       viperWithData,
+			succeed: true,
+		},
+	}
+	for _, c := range cases {
+		rv, err := loadVersionMapping(c.v)
+		if c.succeed {
+			if err != nil {
+				t.Errorf("[%q]-Unexpected error: %v", c.n, err)
+			}
+
+			if len(rv) == 0 {
+				t.Errorf("[%q]-missing mapping value", c.n)
+			}
+		} else {
+			if err == nil {
+				t.Errorf("[%q]-Expected error but got none", c.n)
+			}
+		}
+	}
+}
+
+func TestGetBenchmarkVersion(t *testing.T) {
+	viperWithData, err := loadConfigForTest()
+	if err != nil {
+		t.Fatalf("Unable to load config file %v", err)
+	}
+
+	type getBenchmarkVersionFnToTest func(kubeVersion, benchmarkVersion string, v *viper.Viper) (string, error)
+
+	withFakeKubectl := func(kubeVersion, benchmarkVersion string, v *viper.Viper, fn getBenchmarkVersionFnToTest) (string, error) {
+		execCode := `#!/bin/sh
+		echo "Server Version: v1.15.10"
+		`
+		restore, err := fakeExecutableInPath("kubectl", execCode)
+		if err != nil {
+			t.Fatal("Failed when calling fakeExecutableInPath ", err)
+		}
+		defer restore()
+
+		return fn(kubeVersion, benchmarkVersion, v)
+	}
+
+	withNoPath := func(kubeVersion, benchmarkVersion string, v *viper.Viper, fn getBenchmarkVersionFnToTest) (string, error) {
+		restore, err := prunePath()
+		if err != nil {
+			t.Fatal("Failed when calling prunePath ", err)
+		}
+		defer restore()
+
+		return fn(kubeVersion, benchmarkVersion, v)
+	}
+
+	type getBenchmarkVersionFn func(string, string, *viper.Viper, getBenchmarkVersionFnToTest) (string, error)
+	cases := []struct {
+		n                string
+		kubeVersion      string
+		benchmarkVersion string
+		v                *viper.Viper
+		callFn           getBenchmarkVersionFn
+		exp              string
+		succeed          bool
+	}{
+		{n: "both versions", kubeVersion: "1.11", benchmarkVersion: "cis-1.3", exp: "cis-1.3", callFn: withNoPath, v: viper.New(), succeed: false},
+		{n: "no version-missing-kubectl", kubeVersion: "", benchmarkVersion: "", v: viperWithData, exp: "", callFn: withNoPath, succeed: false},
+		{n: "no version-fakeKubectl", kubeVersion: "", benchmarkVersion: "", v: viperWithData, exp: "cis-1.5", callFn: withFakeKubectl, succeed: true},
+		{n: "kubeVersion", kubeVersion: "1.15", benchmarkVersion: "", v: viperWithData, exp: "cis-1.5", callFn: withNoPath, succeed: true},
+		{n: "ocpVersion310", kubeVersion: "ocp-3.10", benchmarkVersion: "", v: viperWithData, exp: "rh-0.7", callFn: withNoPath, succeed: true},
+		{n: "ocpVersion311", kubeVersion: "ocp-3.11", benchmarkVersion: "", v: viperWithData, exp: "rh-0.7", callFn: withNoPath, succeed: true},
+		{n: "gke10", kubeVersion: "gke-1.0", benchmarkVersion: "", v: viperWithData, exp: "gke-1.0", callFn: withNoPath, succeed: true},
+	}
+	for _, c := range cases {
+		rv, err := c.callFn(c.kubeVersion, c.benchmarkVersion, c.v, getBenchmarkVersion)
+		if c.succeed {
+			if err != nil {
+				t.Errorf("[%q]-Unexpected error: %v", c.n, err)
+			}
+
+			if len(rv) == 0 {
+				t.Errorf("[%q]-missing return value", c.n)
+			}
+
+			if c.exp != rv {
+				t.Errorf("[%q]- expected %q but Got %q", c.n, c.exp, rv)
+			}
+		} else {
+			if err == nil {
+				t.Errorf("[%q]-Expected error but got none", c.n)
+			}
+		}
+	}
+}
+
+func TestValidTargets(t *testing.T) {
+	viperWithData, err := loadConfigForTest()
+	if err != nil {
+		t.Fatalf("Unable to load config file %v", err)
+	}
+	cases := []struct {
+		name      string
+		benchmark string
+		targets   []string
+		expected  bool
+	}{
+		{
+			name:      "cis-1.5 no dummy",
+			benchmark: "cis-1.5",
+			targets:   []string{"master", "node", "controlplane", "etcd", "dummy"},
+			expected:  false,
+		},
+		{
+			name:      "cis-1.5 valid",
+			benchmark: "cis-1.5",
+			targets:   []string{"master", "node", "controlplane", "etcd", "policies"},
+			expected:  true,
+		},
+		{
+			name:      "cis-1.6 no Pikachu",
+			benchmark: "cis-1.6",
+			targets:   []string{"master", "node", "controlplane", "etcd", "Pikachu"},
+			expected:  false,
+		},
+		{
+			name:      "cis-1.6 valid",
+			benchmark: "cis-1.6",
+			targets:   []string{"master", "node", "controlplane", "etcd", "policies"},
+			expected:  true,
+		},
+		{
+			name:      "gke-1.0 valid",
+			benchmark: "gke-1.0",
+			targets:   []string{"master", "node", "controlplane", "etcd", "policies", "managedservices"},
+			expected:  true,
+		},
+		{
+			name:      "eks-1.0 valid",
+			benchmark: "eks-1.0",
+			targets:   []string{"node", "policies", "controlplane", "managedservices"},
+			expected:  true,
+		},
+	}
+
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			ret, err := validTargets(c.benchmark, c.targets, viperWithData)
+			if err != nil {
+				t.Fatalf("Expected nil error, got: %v", err)
+			}
+			if ret != c.expected {
+				t.Fatalf("Expected %t, got %t", c.expected, ret)
+			}
+		})
+	}
+}
+
+func TestIsEtcd(t *testing.T) {
+	testCases := []struct {
+		name            string
+		cfgFile         string
+		getBinariesFunc func(*viper.Viper, check.NodeType) (map[string]string, error)
+		isEtcd          bool
+	}{
+		{
+			name:    "valid config, is etcd and all components are running",
+			cfgFile: "../cfg/config.yaml",
+			getBinariesFunc: func(viper *viper.Viper, nt check.NodeType) (strings map[string]string, i error) {
+				return map[string]string{"etcd": "etcd"}, nil
+			},
+			isEtcd: true,
+		},
+		{
+			name:    "valid config, is etcd and but not all components are running",
+			cfgFile: "../cfg/config.yaml",
+			getBinariesFunc: func(viper *viper.Viper, nt check.NodeType) (strings map[string]string, i error) {
+				return map[string]string{}, nil
+			},
+			isEtcd: false,
+		},
+		{
+			name:    "valid config, is etcd, not all components are running and fails to find all binaries",
+			cfgFile: "../cfg/config.yaml",
+			getBinariesFunc: func(viper *viper.Viper, nt check.NodeType) (strings map[string]string, i error) {
+				return map[string]string{}, errors.New("failed to find binaries")
+			},
+			isEtcd: false,
+		},
+		{
+			name:    "valid config, does not include etcd",
+			cfgFile: "../hack/node_only.yaml",
+			isEtcd:  false,
+		},
+	}
+	cfgDirOld := cfgDir
+	cfgDir = "../cfg"
+	defer func() {
+		cfgDir = cfgDirOld
+	}()
+
+	execCode := `#!/bin/sh
+	echo "Server Version: v1.15.03"
+	`
+	restore, err := fakeExecutableInPath("kubectl", execCode)
+	if err != nil {
+		t.Fatal("Failed when calling fakeExecutableInPath ", err)
+	}
+	defer restore()
+
+	for _, tc := range testCases {
+		cfgFile = tc.cfgFile
+		initConfig()
+
+		oldGetBinariesFunc := getBinariesFunc
+		getBinariesFunc = tc.getBinariesFunc
+		defer func() {
+			getBinariesFunc = oldGetBinariesFunc
+			cfgFile = ""
+		}()
+
+		assert.Equal(t, tc.isEtcd, isEtcd(), tc.name)
+	}
+}
+
+func TestWriteResultToJsonFile(t *testing.T) {
+	defer func() {
+		controlsCollection = []*check.Controls{}
+		jsonFmt = false
+		outputFile = ""
+	}()
+	var err error
+	jsonFmt = true
+	outputFile = path.Join(os.TempDir(), fmt.Sprintf("%d", time.Now().UnixNano()))
+
+	controlsCollection, err = parseControlsJsonFile("./testdata/controlsCollection.json")
+	if err != nil {
+		t.Error(err)
+	}
+	writeOutput(controlsCollection)
+
+	var expect []*check.Controls
+	var result []*check.Controls
+	result, err = parseControlsJsonFile(outputFile)
+	if err != nil {
+		t.Error(err)
+	}
+	expect, err = parseControlsJsonFile("./testdata/result.json")
+	if err != nil {
+		t.Error(err)
+	}
+
+	assert.Equal(t, expect, result)
+}
+
+func parseControlsJsonFile(filepath string) ([]*check.Controls, error) {
+	var result []*check.Controls
+
+	d, err := ioutil.ReadFile(filepath)
+	if err != nil {
+		return nil, err
+	}
+	err = json.Unmarshal(d, &result)
+	if err != nil {
+		return nil, err
+	}
+
+	return result, nil
+}
+
+func loadConfigForTest() (*viper.Viper, error) {
+	viperWithData := viper.New()
+	viperWithData.SetConfigFile("../cfg/config.yaml")
+	if err := viperWithData.ReadInConfig(); err != nil {
+		return nil, err
+	}
+	return viperWithData, nil
+}
+
+type restoreFn func()
+
+func fakeExecutableInPath(execFile, execCode string) (restoreFn, error) {
+	pathenv := os.Getenv("PATH")
+	tmp, err := ioutil.TempDir("", "TestfakeExecutableInPath")
+	if err != nil {
+		return nil, err
+	}
+
+	wd, err := os.Getwd()
+	if err != nil {
+		return nil, err
+	}
+
+	if len(execCode) > 0 {
+		ioutil.WriteFile(filepath.Join(tmp, execFile), []byte(execCode), 0700)
+	} else {
+		f, err := os.OpenFile(execFile, os.O_CREATE|os.O_EXCL, 0700)
+		if err != nil {
+			return nil, err
+		}
+		err = f.Close()
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	err = os.Setenv("PATH", fmt.Sprintf("%s:%s", tmp, pathenv))
+	if err != nil {
+		return nil, err
+	}
+
+	restorePath := func() {
+		os.RemoveAll(tmp)
+		os.Chdir(wd)
+		os.Setenv("PATH", pathenv)
+	}
+
+	return restorePath, nil
+}
+
+func prunePath() (restoreFn, error) {
+	pathenv := os.Getenv("PATH")
+	err := os.Setenv("PATH", "")
+	if err != nil {
+		return nil, err
+	}
+	restorePath := func() {
+		os.Setenv("PATH", pathenv)
+	}
+	return restorePath, nil
+}

cmd/database.go

@@ -7,7 +7,7 @@ import (
 
 	"github.com/golang/glog"
 	"github.com/jinzhu/gorm"
-	_ "github.com/jinzhu/gorm/dialects/postgres"
+	_ "github.com/jinzhu/gorm/dialects/postgres" // database packages get blank imports
 	"github.com/spf13/viper"
 )
 
@@ -49,10 +49,10 @@ func savePgsql(jsonInfo string) {
 	}
 
 	db, err := gorm.Open("postgres", connInfo)
-	defer db.Close()
 	if err != nil {
 		exitWithError(fmt.Errorf("received error connecting to database: %s", err))
 	}
+	defer db.Close()
 
 	db.Debug().AutoMigrate(&ScanResult{})
 	db.Save(&ScanResult{ScanHost: hostname, ScanTime: timestamp, ScanInfo: jsonInfo})

cmd/federated.go

@@ -1,41 +0,0 @@
-// Copyright © 2017 Aqua Security Software Ltd. <info@aquasec.com>
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package cmd
-
-import (
-	"github.com/aquasecurity/kube-bench/check"
-	"github.com/spf13/cobra"
-)
-
-// nodeCmd represents the node command
-var federatedCmd = &cobra.Command{
-	Use:   "federated",
-	Short: "Run benchmark checks for a Kubernetes federated deployment.",
-	Long:  `Run benchmark checks for a Kubernetes federated deployment.`,
-	Run: func(cmd *cobra.Command, args []string) {
-		runChecks(check.FEDERATED)
-	},
-}
-
-func init() {
-	federatedCmd.PersistentFlags().StringVarP(&federatedFile,
-		"file",
-		"f",
-		"/federated.yaml",
-		"Alternative YAML file for federated checks",
-	)
-
-	RootCmd.AddCommand(federatedCmd)
-}

cmd/kubernetes_version.go

@@ -0,0 +1,161 @@
+package cmd
+
+import (
+	"crypto/tls"
+	"encoding/json"
+	"encoding/pem"
+	"fmt"
+	"io/ioutil"
+	"net/http"
+	"os"
+	"strings"
+	"time"
+
+	"github.com/golang/glog"
+)
+
+func getKubeVersionFromRESTAPI() (string, error) {
+	k8sVersionURL := getKubernetesURL()
+	serviceaccount := "/var/run/secrets/kubernetes.io/serviceaccount"
+	cacertfile := fmt.Sprintf("%s/ca.crt", serviceaccount)
+	tokenfile := fmt.Sprintf("%s/token", serviceaccount)
+
+	tlsCert, err := loadCertficate(cacertfile)
+	if err != nil {
+		return "", err
+	}
+
+	tb, err := ioutil.ReadFile(tokenfile)
+	if err != nil {
+		return "", err
+	}
+	token := strings.TrimSpace(string(tb))
+
+	data, err := getWebDataWithRetry(k8sVersionURL, token, tlsCert)
+	if err != nil {
+		return "", err
+	}
+
+	k8sVersion, err := extractVersion(data)
+	if err != nil {
+		return "", err
+	}
+	return k8sVersion, nil
+}
+
+// The idea of this function is so if Kubernetes DNS is not completely seetup and the
+// Container where kube-bench is running needs time for DNS configure.
+// Basically try 10 times, waiting 1 second until either it is successful or it fails.
+func getWebDataWithRetry(k8sVersionURL, token string, cacert *tls.Certificate) (data []byte, err error) {
+	tries := 0
+	// We retry a few times in case the DNS service has not had time to come up
+	for tries < 10 {
+		data, err = getWebData(k8sVersionURL, token, cacert)
+		if err == nil {
+			return
+		}
+		tries++
+		time.Sleep(1 * time.Second)
+	}
+
+	return
+}
+
+func extractVersion(data []byte) (string, error) {
+	type versionResponse struct {
+		Major        string
+		Minor        string
+		GitVersion   string
+		GitCommit    string
+		GitTreeState string
+		BuildDate    string
+		GoVersion    string
+		Compiler     string
+		Platform     string
+	}
+
+	vrObj := &versionResponse{}
+	glog.V(2).Info(fmt.Sprintf("vd: %s\n", string(data)))
+	err := json.Unmarshal(data, vrObj)
+	if err != nil {
+		return "", err
+	}
+	glog.V(2).Info(fmt.Sprintf("vrObj: %#v\n", vrObj))
+
+	// Some provides return the minor version like "15+"
+	minor := strings.Replace(vrObj.Minor, "+", "", -1)
+	ver := fmt.Sprintf("%s.%s", vrObj.Major, minor)
+	return ver, nil
+}
+
+func getWebData(srvURL, token string, cacert *tls.Certificate) ([]byte, error) {
+	glog.V(2).Info(fmt.Sprintf("getWebData srvURL: %s\n", srvURL))
+
+	tlsConf := &tls.Config{
+		Certificates:       []tls.Certificate{*cacert},
+		InsecureSkipVerify: true,
+	}
+	tr := &http.Transport{
+		TLSClientConfig: tlsConf,
+	}
+	client := &http.Client{Transport: tr}
+	req, err := http.NewRequest(http.MethodGet, srvURL, nil)
+	if err != nil {
+		return nil, err
+	}
+
+	authToken := fmt.Sprintf("Bearer %s", token)
+	glog.V(2).Info(fmt.Sprintf("getWebData AUTH TOKEN --[%q]--\n", authToken))
+	req.Header.Set("Authorization", authToken)
+
+	resp, err := client.Do(req)
+	if err != nil {
+		glog.V(2).Info(fmt.Sprintf("HTTP ERROR: %v\n", err))
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusOK {
+		glog.V(2).Info(fmt.Sprintf("URL:[%s], StatusCode:[%d] \n Headers: %#v\n", srvURL, resp.StatusCode, resp.Header))
+		err = fmt.Errorf("URL:[%s], StatusCode:[%d]", srvURL, resp.StatusCode)
+		return nil, err
+	}
+
+	return ioutil.ReadAll(resp.Body)
+}
+
+func loadCertficate(certFile string) (*tls.Certificate, error) {
+	cacert, err := ioutil.ReadFile(certFile)
+	if err != nil {
+		return nil, err
+	}
+
+	var tlsCert tls.Certificate
+	block, _ := pem.Decode(cacert)
+	if block == nil {
+		return nil, fmt.Errorf("unable to Decode certificate")
+	}
+
+	glog.V(2).Info("Loading CA certificate")
+	tlsCert.Certificate = append(tlsCert.Certificate, block.Bytes)
+	return &tlsCert, nil
+}
+
+func getKubernetesURL() string {
+	k8sVersionURL := "https://kubernetes.default.svc/version"
+
+	// The following provides flexibility to use
+	// K8S provided variables is situations where
+	// hostNetwork: true
+	if !isEmpty(os.Getenv("KUBE_BENCH_K8S_ENV")) {
+		k8sHost := os.Getenv("KUBERNETES_SERVICE_HOST")
+		k8sPort := os.Getenv("KUBERNETES_SERVICE_PORT_HTTPS")
+		if !isEmpty(k8sHost) && !isEmpty(k8sPort) {
+			return fmt.Sprintf("https://%s:%s/version", k8sHost, k8sPort)
+		}
+
+		glog.V(2).Info("KUBE_BENCH_K8S_ENV is set, but environment variables KUBERNETES_SERVICE_HOST or KUBERNETES_SERVICE_PORT_HTTPS are not set")
+	}
+
+	return k8sVersionURL
+}

cmd/kubernetes_version_test.go

@@ -0,0 +1,281 @@
+package cmd
+
+import (
+	"crypto/tls"
+	"fmt"
+	"io/ioutil"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"strconv"
+	"testing"
+)
+
+func TestLoadCertficate(t *testing.T) {
+	tmp, err := ioutil.TempDir("", "TestFakeLoadCertficate")
+	if err != nil {
+		t.Fatalf("unable to create temp directory: %v", err)
+	}
+	defer os.RemoveAll(tmp)
+
+	goodCertFile, _ := ioutil.TempFile(tmp, "good-cert-*")
+	_, _ = goodCertFile.Write([]byte(`-----BEGIN CERTIFICATE-----
+MIICyDCCAbCgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
+cm5ldGVzMB4XDTE5MTEwODAxNDAwMFoXDTI5MTEwNTAxNDAwMFowFTETMBEGA1UE
+AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMn6
+wjvhMc9e0MDwpQNhp8SPxmv1DsYJ4Btp1GeScIgKKDwppuoOmVizLiMNdV5+70yI
+MgNfm/gwFRNDOtN3R7msfZDD5Dd1vI6qRTP21DFOGVdysFdwqJTs0nGcmfvZEOtw
+9cjcsXrBi2Mg54v+X/pq2w51xajCGBt2+bpxJJ3WBiWqKYv0RQdNL0WZGm+V9BuP
+pHRWPBeLxuCzt5K3Gx+1QDy8o6Y4sSRPssWC4RhD9Hs5/9eeGRyZslLs+AuqdDLQ
+aziiSjHVtgCfRXE9nYVxaDIwTFuh+Q1IvtB36NRLyX47oya+BbX3PoCtSjA36RBb
+tcJfulr3oNHnb2ZlfcUCAwEAAaMjMCEwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB
+/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAAeQDkbM6DilLkIVQDyxauETgJDV
+2AaVzYaAgDApQGAoYV6WIY7Exk4TlmLeKQjWt2s/GtthQWuzUDKTcEvWcG6gNdXk
+gzuCRRDMGu25NtG3m67w4e2RzW8Z/lzvbfyJZGoV2c6dN+yP9/Pw2MXlrnMWugd1
+jLv3UYZRHMpuNS8BJU74BuVzVPHd55RAl+bV8yemdZJ7pPzMvGbZ7zRXWODTDlge
+CQb9lY+jYErisH8Sq7uABFPvi7RaTh8SS7V7OxqHZvmttNTdZs4TIkk45JK7Y+Xq
+FAjB57z2NcIgJuVpQnGRYtr/JcH2Qdsq8bLtXaojUIWOOqoTDRLYozdMOOQ=
+-----END CERTIFICATE-----`))
+	badCertFile, _ := ioutil.TempFile(tmp, "bad-cert-*")
+
+	cases := []struct {
+		file string
+		fail bool
+	}{
+		{
+			file: "missing cert file",
+			fail: true,
+		},
+		{
+			file: badCertFile.Name(),
+			fail: true,
+		},
+		{
+			file: goodCertFile.Name(),
+			fail: false,
+		},
+	}
+
+	for id, c := range cases {
+		t.Run(strconv.Itoa(id), func(t *testing.T) {
+			tlsCert, err := loadCertficate(c.file)
+			if !c.fail {
+				if err != nil {
+					t.Errorf("unexpected error: %v", err)
+				}
+
+				if tlsCert == nil {
+					t.Errorf("missing returned TLS Certificate")
+				}
+			} else {
+				if err == nil {
+					t.Errorf("Expected error")
+				}
+			}
+
+		})
+	}
+}
+
+func TestGetWebData(t *testing.T) {
+	okfn := func(w http.ResponseWriter, r *http.Request) {
+		_, _ = fmt.Fprintln(w, `{
+			"major": "1",
+			"minor": "15"}`)
+	}
+	errfn := func(w http.ResponseWriter, r *http.Request) {
+		http.Error(w, http.StatusText(http.StatusInternalServerError),
+			http.StatusInternalServerError)
+	}
+	token := "dummyToken"
+	var tlsCert tls.Certificate
+
+	cases := []struct {
+		fn   http.HandlerFunc
+		fail bool
+	}{
+		{
+			fn:   okfn,
+			fail: false,
+		},
+		{
+			fn:   errfn,
+			fail: true,
+		},
+	}
+
+	for id, c := range cases {
+		t.Run(strconv.Itoa(id), func(t *testing.T) {
+			ts := httptest.NewServer(c.fn)
+			defer ts.Close()
+			data, err := getWebData(ts.URL, token, &tlsCert)
+			if !c.fail {
+				if err != nil {
+					t.Errorf("unexpected error: %v", err)
+				}
+
+				if len(data) == 0 {
+					t.Errorf("missing data")
+				}
+			} else {
+				if err == nil {
+					t.Errorf("Expected error")
+				}
+			}
+		})
+	}
+
+}
+func TestGetWebDataWithRetry(t *testing.T) {
+	okfn := func(w http.ResponseWriter, r *http.Request) {
+		_, _ = fmt.Fprintln(w, `{
+			"major": "1",
+			"minor": "15"}`)
+	}
+	errfn := func(w http.ResponseWriter, r *http.Request) {
+		http.Error(w, http.StatusText(http.StatusInternalServerError),
+			http.StatusInternalServerError)
+	}
+	token := "dummyToken"
+	var tlsCert tls.Certificate
+
+	cases := []struct {
+		fn   http.HandlerFunc
+		fail bool
+	}{
+		{
+			fn:   okfn,
+			fail: false,
+		},
+		{
+			fn:   errfn,
+			fail: true,
+		},
+	}
+
+	for id, c := range cases {
+		t.Run(strconv.Itoa(id), func(t *testing.T) {
+			ts := httptest.NewServer(c.fn)
+			defer ts.Close()
+			data, err := getWebDataWithRetry(ts.URL, token, &tlsCert)
+			if !c.fail {
+				if err != nil {
+					t.Errorf("unexpected error: %v", err)
+				}
+
+				if len(data) == 0 {
+					t.Errorf("missing data")
+				}
+			} else {
+				if err == nil {
+					t.Errorf("Expected error")
+				}
+			}
+		})
+	}
+
+}
+func TestExtractVersion(t *testing.T) {
+	okJSON := []byte(`{
+	"major": "1",
+	"minor": "15",
+	"gitVersion": "v1.15.3",
+	"gitCommit": "2d3c76f9091b6bec110a5e63777c332469e0cba2",
+	"gitTreeState": "clean",
+	"buildDate": "2019-08-20T18:57:36Z",
+	"goVersion": "go1.12.9",
+	"compiler": "gc",
+	"platform": "linux/amd64"
+    }`)
+
+	invalidJSON := []byte(`{
+	"major": "1",
+	"minor": "15",
+	"gitVersion": "v1.15.3",
+	"gitCommit": "2d3c76f9091b6bec110a5e63777c332469e0cba2",
+	"gitTreeState": "clean",`)
+
+	cases := []struct {
+		data        []byte
+		fail        bool
+		expectedVer string
+	}{
+		{
+			data:        okJSON,
+			fail:        false,
+			expectedVer: "1.15",
+		},
+		{
+			data: invalidJSON,
+			fail: true,
+		},
+	}
+
+	for id, c := range cases {
+		t.Run(strconv.Itoa(id), func(t *testing.T) {
+			ver, err := extractVersion(c.data)
+			if !c.fail {
+				if err != nil {
+					t.Errorf("unexpected error: %v", err)
+				}
+				if c.expectedVer != ver {
+					t.Errorf("Expected %q but Got %q", c.expectedVer, ver)
+				}
+			} else {
+				if err == nil {
+					t.Errorf("Expected error")
+				}
+			}
+		})
+	}
+}
+
+func TestGetKubernetesURL(t *testing.T) {
+
+	resetEnvs := func() {
+		os.Unsetenv("KUBE_BENCH_K8S_ENV")
+		os.Unsetenv("KUBERNETES_SERVICE_HOST")
+		os.Unsetenv("KUBERNETES_SERVICE_PORT_HTTPS")
+	}
+
+	setEnvs := func() {
+		os.Setenv("KUBE_BENCH_K8S_ENV", "1")
+		os.Setenv("KUBERNETES_SERVICE_HOST", "testHostServer")
+		os.Setenv("KUBERNETES_SERVICE_PORT_HTTPS", "443")
+	}
+
+	cases := []struct {
+		useDefault bool
+		expected   string
+	}{
+		{
+			useDefault: true,
+			expected:   "https://kubernetes.default.svc/version",
+		},
+		{
+			useDefault: false,
+			expected:   "https://testHostServer:443/version",
+		},
+	}
+	for id, c := range cases {
+		t.Run(strconv.Itoa(id), func(t *testing.T) {
+			resetEnvs()
+			defer resetEnvs()
+			if !c.useDefault {
+				setEnvs()
+			}
+			k8sURL := getKubernetesURL()
+
+			if !c.useDefault {
+				if k8sURL != c.expected {
+					t.Errorf("Expected %q but Got %q", k8sURL, c.expected)
+				}
+			} else {
+				if k8sURL != c.expected {
+					t.Errorf("Expected %q but Got %q", k8sURL, c.expected)
+				}
+			}
+		})
+	}
+
+}

cmd/master.go

@@ -1,4 +1,4 @@
-// Copyright © 2017 Aqua Security Software Ltd. <info@aquasec.com>
+// Copyright © 2017-2019 Aqua Security Software Ltd. <info@aquasec.com>
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
@@ -15,17 +15,27 @@
 package cmd
 
 import (
+	"fmt"
+
 	"github.com/aquasecurity/kube-bench/check"
 	"github.com/spf13/cobra"
+	"github.com/spf13/viper"
 )
 
 // masterCmd represents the master command
 var masterCmd = &cobra.Command{
 	Use:   "master",
-	Short: "Run benchmark checks for a Kubernetes master node.",
-	Long:  `Run benchmark checks for a Kubernetes master node.`,
+	Short: "Run Kubernetes benchmark checks from the master.yaml file.",
+	Long:  `Run Kubernetes benchmark checks from the master.yaml file in cfg/<version>.`,
 	Run: func(cmd *cobra.Command, args []string) {
-		runChecks(check.MASTER)
+		bv, err := getBenchmarkVersion(kubeVersion, benchmarkVersion, viper.GetViper())
+		if err != nil {
+			exitWithError(fmt.Errorf("unable to determine benchmark version: %v", err))
+		}
+
+		filename := loadConfig(check.MASTER, bv)
+		runChecks(check.MASTER, filename)
+		writeOutput(controlsCollection)
 	},
 }
 

cmd/node.go

@@ -1,4 +1,4 @@
-// Copyright © 2017 Aqua Security Software Ltd. <info@aquasec.com>
+// Copyright © 2017-2019 Aqua Security Software Ltd. <info@aquasec.com>
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
@@ -15,17 +15,27 @@
 package cmd
 
 import (
+	"fmt"
+
 	"github.com/aquasecurity/kube-bench/check"
 	"github.com/spf13/cobra"
+	"github.com/spf13/viper"
 )
 
 // nodeCmd represents the node command
 var nodeCmd = &cobra.Command{
 	Use:   "node",
-	Short: "Run benchmark checks for a Kubernetes node.",
-	Long:  `Run benchmark checks for a Kubernetes node.`,
+	Short: "Run Kubernetes benchmark checks from the node.yaml file.",
+	Long:  `Run Kubernetes benchmark checks from the node.yaml file in cfg/<version>.`,
 	Run: func(cmd *cobra.Command, args []string) {
-		runChecks(check.NODE)
+		bv, err := getBenchmarkVersion(kubeVersion, benchmarkVersion, viper.GetViper())
+		if err != nil {
+			exitWithError(fmt.Errorf("unable to determine benchmark version: %v", err))
+		}
+
+		filename := loadConfig(check.NODE, bv)
+		runChecks(check.NODE, filename)
+		writeOutput(controlsCollection)
 	},
 }
 

cmd/root.go

@@ -20,64 +20,165 @@ import (
 	"os"
 
 	"github.com/aquasecurity/kube-bench/check"
+	"github.com/golang/glog"
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
 )
 
+type FilterOpts struct {
+	CheckList string
+	GroupList string
+	Scored    bool
+	Unscored  bool
+}
+
 var (
-	envVarsPrefix      = "KUBE_BENCH"
-	cfgDir             = "./cfg"
-	defaultKubeVersion = "1.6"
-	cfgFile            string
-	jsonFmt            bool
-	pgSql              bool
-	checkList          string
-	groupList          string
-	masterFile         string
-	nodeFile           string
-	federatedFile      string
+	envVarsPrefix       = "KUBE_BENCH"
+	defaultKubeVersion  = "1.18"
+	kubeVersion         string
+	benchmarkVersion    string
+	cfgFile             string
+	cfgDir              = "./cfg/"
+	jsonFmt             bool
+	junitFmt            bool
+	pgSQL               bool
+	masterFile          = "master.yaml"
+	nodeFile            = "node.yaml"
+	etcdFile            = "etcd.yaml"
+	controlplaneFile    = "controlplane.yaml"
+	policiesFile        = "policies.yaml"
+	managedservicesFile = "managedservices.yaml"
+	noResults           bool
+	noSummary           bool
+	noRemediations      bool
+	filterOpts          FilterOpts
+	includeTestOutput   bool
+	outputFile          string
+	configFileError     error
+	controlsCollection  []*check.Controls
 )
 
 // RootCmd represents the base command when called without any subcommands
 var RootCmd = &cobra.Command{
 	Use:   os.Args[0],
 	Short: "Run CIS Benchmarks checks against a Kubernetes deployment",
-	Long:  `This tool runs the CIS Kubernetes 1.6 Benchmark v1.0.0 checks.`,
+	Long:  `This tool runs the CIS Kubernetes Benchmark (https://www.cisecurity.org/benchmark/kubernetes/)`,
+	Run: func(cmd *cobra.Command, args []string) {
+		bv, err := getBenchmarkVersion(kubeVersion, benchmarkVersion, viper.GetViper())
+		if err != nil {
+			exitWithError(fmt.Errorf("unable to determine benchmark version: %v", err))
+		}
+		glog.V(1).Infof("Running checks for benchmark %v", bv)
+
+		if isMaster() {
+			glog.V(1).Info("== Running master checks ==")
+			runChecks(check.MASTER, loadConfig(check.MASTER, bv))
+
+			// Control Plane is only valid for CIS 1.5 and later,
+			// this a gatekeeper for previous versions
+			valid, err := validTargets(bv, []string{string(check.CONTROLPLANE)}, viper.GetViper())
+			if err != nil {
+				exitWithError(fmt.Errorf("error validating targets: %v", err))
+			}
+			if valid {
+				glog.V(1).Info("== Running control plane checks ==")
+				runChecks(check.CONTROLPLANE, loadConfig(check.CONTROLPLANE, bv))
+			}
+		}
+
+		// Etcd is only valid for CIS 1.5 and later,
+		// this a gatekeeper for previous versions.
+		valid, err := validTargets(bv, []string{string(check.ETCD)}, viper.GetViper())
+		if err != nil {
+			exitWithError(fmt.Errorf("error validating targets: %v", err))
+		}
+		if valid && isEtcd() {
+			glog.V(1).Info("== Running etcd checks ==")
+			runChecks(check.ETCD, loadConfig(check.ETCD, bv))
+		}
+
+		glog.V(1).Info("== Running node checks ==")
+		runChecks(check.NODE, loadConfig(check.NODE, bv))
+
+		// Policies is only valid for CIS 1.5 and later,
+		// this a gatekeeper for previous versions.
+		valid, err = validTargets(bv, []string{string(check.POLICIES)}, viper.GetViper())
+		if err != nil {
+			exitWithError(fmt.Errorf("error validating targets: %v", err))
+		}
+		if valid {
+			glog.V(1).Info("== Running policies checks ==")
+			runChecks(check.POLICIES, loadConfig(check.POLICIES, bv))
+		}
+
+		// Managedservices is only valid for GKE 1.0 and later,
+		// this a gatekeeper for previous versions.
+		valid, err = validTargets(bv, []string{string(check.MANAGEDSERVICES)}, viper.GetViper())
+		if err != nil {
+			exitWithError(fmt.Errorf("error validating targets: %v", err))
+		}
+		if valid {
+			glog.V(1).Info("== Running managed services checks ==")
+			runChecks(check.MANAGEDSERVICES, loadConfig(check.MANAGEDSERVICES, bv))
+		}
+
+		writeOutput(controlsCollection)
+	},
 }
 
 // Execute adds all child commands to the root command sets flags appropriately.
 // This is called by main.main(). It only needs to happen once to the rootCmd.
 func Execute() {
-	goflag.Set("logtostderr", "true")
 	goflag.CommandLine.Parse([]string{})
 
 	if err := RootCmd.Execute(); err != nil {
 		fmt.Println(err)
+		// flush before exit non-zero
+		glog.Flush()
 		os.Exit(-1)
 	}
+	// flush before exit
+	glog.Flush()
 }
 
 func init() {
 	cobra.OnInitialize(initConfig)
 
+	// Output control
+	RootCmd.PersistentFlags().BoolVar(&noResults, "noresults", false, "Disable printing of results section")
+	RootCmd.PersistentFlags().BoolVar(&noSummary, "nosummary", false, "Disable printing of summary section")
+	RootCmd.PersistentFlags().BoolVar(&noRemediations, "noremediations", false, "Disable printing of remediations section")
 	RootCmd.PersistentFlags().BoolVar(&jsonFmt, "json", false, "Prints the results as JSON")
-	RootCmd.PersistentFlags().BoolVar(&pgSql, "pgsql", false, "Save the results to PostgreSQL")
+	RootCmd.PersistentFlags().BoolVar(&junitFmt, "junit", false, "Prints the results as JUnit")
+	RootCmd.PersistentFlags().BoolVar(&pgSQL, "pgsql", false, "Save the results to PostgreSQL")
+	RootCmd.PersistentFlags().BoolVar(&filterOpts.Scored, "scored", true, "Run the scored CIS checks")
+	RootCmd.PersistentFlags().BoolVar(&filterOpts.Unscored, "unscored", true, "Run the unscored CIS checks")
+	RootCmd.PersistentFlags().BoolVar(&includeTestOutput, "include-test-output", false, "Prints the actual result when test fails")
+	RootCmd.PersistentFlags().StringVar(&outputFile, "outputfile", "", "Writes the JSON results to output file")
+
 	RootCmd.PersistentFlags().StringVarP(
-		&checkList,
+		&filterOpts.CheckList,
 		"check",
 		"c",
 		"",
 		`A comma-delimited list of checks to run as specified in CIS document. Example --check="1.1.1,1.1.2"`,
 	)
 	RootCmd.PersistentFlags().StringVarP(
-		&groupList,
+		&filterOpts.GroupList,
 		"group",
 		"g",
 		"",
 		`Run all the checks under this comma-delimited list of groups. Example --group="1.1"`,
 	)
 	RootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default is ./cfg/config.yaml)")
+	RootCmd.PersistentFlags().StringVarP(&cfgDir, "config-dir", "D", cfgDir, "config directory")
+	RootCmd.PersistentFlags().StringVar(&kubeVersion, "version", "", "Manually specify Kubernetes version, automatically detected if unset")
+	RootCmd.PersistentFlags().StringVar(&benchmarkVersion, "benchmark", "", "Manually specify CIS benchmark version. It would be an error to specify both --version and --benchmark flags")
 
+	if err := goflag.Set("logtostderr", "true"); err != nil {
+		fmt.Printf("unable to set logtostderr: %+v\n", err)
+		os.Exit(-1)
+	}
 	goflag.CommandLine.VisitAll(func(goflag *goflag.Flag) {
 		RootCmd.PersistentFlags().AddGoFlag(goflag)
 	})
@@ -93,12 +194,27 @@ func initConfig() {
 		viper.AddConfigPath(cfgDir)   // adding ./cfg as first search path
 	}
 
+	// Read flag values from environment variables.
+	// Precedence: Command line flags take precedence over environment variables.
 	viper.SetEnvPrefix(envVarsPrefix)
-	viper.AutomaticEnv() // read in environment variables that match
+	viper.AutomaticEnv()
+
+	if kubeVersion == "" {
+		if env := viper.Get("version"); env != nil {
+			kubeVersion = env.(string)
+		}
+	}
 
 	// If a config file is found, read it in.
 	if err := viper.ReadInConfig(); err != nil {
-		colorPrint(check.FAIL, fmt.Sprintf("Failed to read config file: %v\n", err))
-		os.Exit(1)
+		if _, ok := err.(viper.ConfigFileNotFoundError); ok {
+			// Config file not found; ignore error for now to prevent commands
+			// which don't need the config file exiting.
+			configFileError = err
+		} else {
+			// Config file was found but another error was produced
+			colorPrint(check.FAIL, fmt.Sprintf("Failed to read config file: %v\n", err))
+			os.Exit(1)
+		}
 	}
 }

cmd/run.go

@@ -0,0 +1,107 @@
+package cmd
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+
+	"github.com/aquasecurity/kube-bench/check"
+	"github.com/golang/glog"
+	"github.com/spf13/cobra"
+	"github.com/spf13/viper"
+)
+
+func init() {
+	RootCmd.AddCommand(runCmd)
+	runCmd.Flags().StringSliceP("targets", "s", []string{},
+		`Specify targets of the benchmark to run. These names need to match the filenames in the cfg/<version> directory.
+	For example, to run the tests specified in master.yaml and etcd.yaml, specify --targets=master,etcd
+	If no targets are specified, run tests from all files in the cfg/<version> directory.
+	`)
+}
+
+// runCmd represents the run command
+var runCmd = &cobra.Command{
+	Use:   "run",
+	Short: "Run tests",
+	Long:  `Run tests. If no arguments are specified, runs tests from all files`,
+	Run: func(cmd *cobra.Command, args []string) {
+		targets, err := cmd.Flags().GetStringSlice("targets")
+		if err != nil {
+			exitWithError(fmt.Errorf("unable to get `targets` from command line :%v", err))
+		}
+
+		bv, err := getBenchmarkVersion(kubeVersion, benchmarkVersion, viper.GetViper())
+		if err != nil {
+			exitWithError(fmt.Errorf("unable to get benchmark version. error: %v", err))
+		}
+
+		glog.V(2).Infof("Checking targets %v for %v", targets, bv)
+		benchmarkVersionToTargetsMap, err := loadTargetMapping(viper.GetViper())
+		if err != nil {
+			exitWithError(fmt.Errorf("error loading targets: %v", err))
+		}
+		valid, err := validTargets(bv, targets, viper.GetViper())
+		if err != nil {
+			exitWithError(fmt.Errorf("error validating targets: %v", err))
+		}
+		if len(targets) > 0 && !valid {
+			exitWithError(fmt.Errorf(fmt.Sprintf(`The specified --targets "%s" are not configured for the CIS Benchmark %s\n Valid targets %v`, strings.Join(targets, ","), bv, benchmarkVersionToTargetsMap[bv])))
+		}
+
+		// Merge version-specific config if any.
+		path := filepath.Join(cfgDir, bv)
+		mergeConfig(path)
+
+		err = run(targets, bv)
+		if err != nil {
+			fmt.Printf("Error in run: %v\n", err)
+		}
+	},
+}
+
+func run(targets []string, benchmarkVersion string) (err error) {
+	yamlFiles, err := getTestYamlFiles(targets, benchmarkVersion)
+	if err != nil {
+		return err
+	}
+
+	glog.V(3).Infof("Running tests from files %v\n", yamlFiles)
+
+	for _, yamlFile := range yamlFiles {
+		_, name := filepath.Split(yamlFile)
+		testType := check.NodeType(strings.Split(name, ".")[0])
+		runChecks(testType, yamlFile)
+	}
+
+	writeOutput(controlsCollection)
+	return nil
+}
+
+func getTestYamlFiles(targets []string, benchmarkVersion string) (yamlFiles []string, err error) {
+	// Check that the specified targets have corresponding YAML files in the config directory
+	configFileDirectory := filepath.Join(cfgDir, benchmarkVersion)
+	for _, target := range targets {
+		filename := translate(target) + ".yaml"
+		file := filepath.Join(configFileDirectory, filename)
+		if _, err := os.Stat(file); err != nil {
+			return nil, fmt.Errorf("file %s not found for version %s", filename, benchmarkVersion)
+		}
+		yamlFiles = append(yamlFiles, file)
+	}
+
+	// If no targets were specified, we will run tests from all the files in the directory
+	if len(yamlFiles) == 0 {
+		yamlFiles, err = getYamlFilesFromDir(configFileDirectory)
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	return yamlFiles, err
+}
+
+func translate(target string) string {
+	return strings.Replace(strings.ToLower(target), "worker", "node", -1)
+}

cmd/run_test.go

@@ -0,0 +1,122 @@
+package cmd
+
+import (
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+func TestGetTestYamlFiles(t *testing.T) {
+	cases := []struct {
+		name      string
+		targets   []string
+		benchmark string
+		succeed   bool
+		expCount  int
+	}{
+		{
+			name:      "Specify two targets",
+			targets:   []string{"one", "two"},
+			benchmark: "benchmark",
+			succeed:   true,
+			expCount:  2,
+		},
+		{
+			name:      "Specify a target that doesn't exist",
+			targets:   []string{"one", "missing"},
+			benchmark: "benchmark",
+			succeed:   false,
+		},
+		{
+			name:      "No targets specified - should return everything except config.yaml",
+			targets:   []string{},
+			benchmark: "benchmark",
+			succeed:   true,
+			expCount:  3,
+		},
+		{
+			name:      "Specify benchmark that doesn't exist",
+			targets:   []string{"one"},
+			benchmark: "missing",
+			succeed:   false,
+		},
+	}
+
+	// Set up temp config directory
+	var err error
+	cfgDir, err = ioutil.TempDir("", "kube-bench-test")
+	if err != nil {
+		t.Fatalf("Failed to create temp directory")
+	}
+	defer os.RemoveAll(cfgDir)
+
+	d := filepath.Join(cfgDir, "benchmark")
+	err = os.Mkdir(d, 0766)
+	if err != nil {
+		t.Fatalf("Failed to create temp dir")
+	}
+
+	// We never expect config.yaml to be returned
+	for _, filename := range []string{"one.yaml", "two.yaml", "three.yaml", "config.yaml"} {
+		err = ioutil.WriteFile(filepath.Join(d, filename), []byte("hello world"), 0666)
+		if err != nil {
+			t.Fatalf("error writing temp file %s: %v", filename, err)
+		}
+	}
+
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			yamlFiles, err := getTestYamlFiles(c.targets, c.benchmark)
+			if err != nil && c.succeed {
+				t.Fatalf("Error %v", err)
+			}
+
+			if err == nil && !c.succeed {
+				t.Fatalf("Expected failure")
+			}
+
+			if len(yamlFiles) != c.expCount {
+				t.Fatalf("Expected %d, got %d", c.expCount, len(yamlFiles))
+			}
+		})
+	}
+}
+
+func TestTranslate(t *testing.T) {
+	cases := []struct {
+		name     string
+		original string
+		expected string
+	}{
+		{
+			name:     "keep",
+			original: "controlplane",
+			expected: "controlplane",
+		},
+		{
+			name:     "translate",
+			original: "worker",
+			expected: "node",
+		},
+		{
+			name:     "translateLower",
+			original: "Worker",
+			expected: "node",
+		},
+		{
+			name:     "Lower",
+			original: "ETCD",
+			expected: "etcd",
+		},
+	}
+
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			ret := translate(c.original)
+			if ret != c.expected {
+				t.Fatalf("Expected %q, got %q", c.expected, ret)
+			}
+		})
+	}
+}

cmd/testdata/controlsCollection.json

@@ -0,0 +1,114 @@
+[
+  {
+    "id": "2",
+    "version": "1.15",
+    "text": "Etcd Node Configuration",
+    "node_type": "etcd",
+    "tests": [
+      {
+        "section": "2",
+        "pass": 7,
+        "fail": 0,
+        "warn": 0,
+        "info": 0,
+        "desc": "Etcd Node Configuration Files",
+        "results": [
+          {
+            "test_number": "2.1",
+            "test_desc": "Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)",
+            "audit": "/bin/ps -ef | /bin/grep etcd | /bin/grep -v grep",
+            "AuditConfig": "",
+            "type": "",
+            "remediation": "Follow the etcd service documentation and configure TLS encryption.\nThen, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml\non the master node and set the below parameters.\n--cert-file=</path/to/ca-file>\n--key-file=</path/to/key-file>\n",
+            "test_info": [
+              "Follow the etcd service documentation and configure TLS encryption.\nThen, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml\non the master node and set the below parameters.\n--cert-file=</path/to/ca-file>\n--key-file=</path/to/key-file>\n"
+            ],
+            "status": "PASS",
+            "actual_value": "root      3277  3218  3 Apr19 ?        03:57:52 etcd --advertise-client-urls=https://192.168.64.4:2379 --cert-file=/var/lib/minikube/certs/etcd/server.crt --client-cert-auth=true --data-dir=/var/lib/minikube/etcd --initial-advertise-peer-urls=https://192.168.64.4:2380 --initial-cluster=minikube=https://192.168.64.4:2380 --key-file=/var/lib/minikube/certs/etcd/server.key --listen-client-urls=https://127.0.0.1:2379,https://192.168.64.4:2379 --listen-metrics-urls=http://127.0.0.1:2381 --listen-peer-urls=https://192.168.64.4:2380 --name=minikube --peer-cert-file=/var/lib/minikube/certs/etcd/peer.crt --peer-client-cert-auth=true --peer-key-file=/var/lib/minikube/certs/etcd/peer.key --peer-trusted-ca-file=/var/lib/minikube/certs/etcd/ca.crt --snapshot-count=10000 --trusted-ca-file=/var/lib/minikube/certs/etcd/ca.crt\nroot      4624  4605  8 Apr21 ?        04:55:10 kube-apiserver --advertise-address=192.168.64.4 --allow-privileged=true --authorization-mode=Node,RBAC --client-ca-file=/var/lib/minikube/certs/ca.crt --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodSecurityPolicy --enable-bootstrap-token-auth=true --etcd-cafile=/var/lib/minikube/certs/etcd/ca.crt --etcd-certfile=/var/lib/minikube/certs/apiserver-etcd-client.crt --etcd-keyfile=/var/lib/minikube/certs/apiserver-etcd-client.key --etcd-servers=https://127.0.0.1:2379 --insecure-port=0 --kubelet-client-certificate=/var/lib/minikube/certs/apiserver-kubelet-client.crt --kubelet-client-key=/var/lib/minikube/certs/apiserver-kubelet-client.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --proxy-client-cert-file=/var/lib/minikube/certs/front-proxy-client.crt --proxy-client-key-file=/var/lib/minikube/certs/front-proxy-client.key --requestheader-allowed-names=front-proxy-client --requestheader-client-ca-file=/var/lib/minikube/certs/front-proxy-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=8443 --service-account-key-file=/var/lib/minikube/certs/sa.pub --service-cluster-ip-range=10.96.0.0/12 --tls-cert-file=/var/lib/minikube/certs/apiserver.crt --tls-private-key-file=/var/lib/minikube/certs/apiserver.key\n",
+            "scored": true,
+            "expected_result": "'--cert-file' is present AND '--key-file' is present"
+          }
+        ]
+      }
+    ],
+    "total_pass": 7,
+    "total_fail": 0,
+    "total_warn": 0,
+    "total_info": 0
+  },
+  {
+    "id": "3",
+    "version": "1.5",
+    "text": "Control Plane Configuration",
+    "node_type": "controlplane",
+    "tests": [
+      {
+        "section": "3.1",
+        "pass": 0,
+        "fail": 0,
+        "warn": 1,
+        "info": 0,
+        "desc": "Authentication and Authorization",
+        "results": [
+          {
+            "test_number": "3.1.1",
+            "test_desc": "Client certificate authentication should not be used for users (Not Scored)",
+            "audit": "",
+            "AuditConfig": "",
+            "type": "manual",
+            "remediation": "Alternative mechanisms provided by Kubernetes such as the use of OIDC should be\nimplemented in place of client certificates.\n",
+            "test_info": [
+              "Alternative mechanisms provided by Kubernetes such as the use of OIDC should be\nimplemented in place of client certificates.\n"
+            ],
+            "status": "WARN",
+            "actual_value": "",
+            "scored": false,
+            "expected_result": "",
+            "reason": "Test marked as a manual test"
+          }
+        ]
+      }
+    ],
+    "total_pass": 0,
+    "total_fail": 0,
+    "total_warn": 3,
+    "total_info": 0
+  },
+  {
+    "id": "1",
+    "version": "1.5",
+    "text": "Master Node Security Configuration",
+    "node_type": "master",
+    "tests": [
+      {
+        "section": "1.1",
+        "pass": 15,
+        "fail": 1,
+        "warn": 5,
+        "info": 0,
+        "desc": "Master Node Configuration Files",
+        "results": [
+          {
+            "test_number": "1.1.1",
+            "test_desc": "Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)",
+            "audit": "/bin/sh -c 'if test -e /etc/kubernetes/manifests/kube-apiserver.yaml; then stat -c permissions=%a /etc/kubernetes/manifests/kube-apiserver.yaml; fi'",
+            "AuditConfig": "",
+            "type": "",
+            "remediation": "Run the below command (based on the file location on your system) on the\nmaster node.\nFor example, chmod 644 /etc/kubernetes/manifests/kube-apiserver.yaml\n",
+            "test_info": [
+              "Run the below command (based on the file location on your system) on the\nmaster node.\nFor example, chmod 644 /etc/kubernetes/manifests/kube-apiserver.yaml\n"
+            ],
+            "status": "PASS",
+            "actual_value": "permissions=600\n",
+            "scored": true,
+            "expected_result": "bitmask '600' AND '644'"
+          }
+        ]
+      }
+    ],
+    "total_pass": 42,
+    "total_fail": 12,
+    "total_warn": 11,
+    "total_info": 0
+  }
+]

cmd/testdata/result.json

@@ -0,0 +1,114 @@
+[
+  {
+    "id": "1",
+    "version": "1.5",
+    "text": "Master Node Security Configuration",
+    "node_type": "master",
+    "tests": [
+      {
+        "section": "1.1",
+        "pass": 15,
+        "fail": 1,
+        "warn": 5,
+        "info": 0,
+        "desc": "Master Node Configuration Files",
+        "results": [
+          {
+            "test_number": "1.1.1",
+            "test_desc": "Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)",
+            "audit": "/bin/sh -c 'if test -e /etc/kubernetes/manifests/kube-apiserver.yaml; then stat -c permissions=%a /etc/kubernetes/manifests/kube-apiserver.yaml; fi'",
+            "AuditConfig": "",
+            "type": "",
+            "remediation": "Run the below command (based on the file location on your system) on the\nmaster node.\nFor example, chmod 644 /etc/kubernetes/manifests/kube-apiserver.yaml\n",
+            "test_info": [
+              "Run the below command (based on the file location on your system) on the\nmaster node.\nFor example, chmod 644 /etc/kubernetes/manifests/kube-apiserver.yaml\n"
+            ],
+            "status": "PASS",
+            "actual_value": "permissions=600\n",
+            "scored": true,
+            "expected_result": "bitmask '600' AND '644'"
+          }
+        ]
+      }
+    ],
+    "total_pass": 42,
+    "total_fail": 12,
+    "total_warn": 11,
+    "total_info": 0
+  },
+  {
+    "id": "2",
+    "version": "1.15",
+    "text": "Etcd Node Configuration",
+    "node_type": "etcd",
+    "tests": [
+      {
+        "section": "2",
+        "pass": 7,
+        "fail": 0,
+        "warn": 0,
+        "info": 0,
+        "desc": "Etcd Node Configuration Files",
+        "results": [
+          {
+            "test_number": "2.1",
+            "test_desc": "Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)",
+            "audit": "/bin/ps -ef | /bin/grep etcd | /bin/grep -v grep",
+            "AuditConfig": "",
+            "type": "",
+            "remediation": "Follow the etcd service documentation and configure TLS encryption.\nThen, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml\non the master node and set the below parameters.\n--cert-file=</path/to/ca-file>\n--key-file=</path/to/key-file>\n",
+            "test_info": [
+              "Follow the etcd service documentation and configure TLS encryption.\nThen, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml\non the master node and set the below parameters.\n--cert-file=</path/to/ca-file>\n--key-file=</path/to/key-file>\n"
+            ],
+            "status": "PASS",
+            "actual_value": "root      3277  3218  3 Apr19 ?        03:57:52 etcd --advertise-client-urls=https://192.168.64.4:2379 --cert-file=/var/lib/minikube/certs/etcd/server.crt --client-cert-auth=true --data-dir=/var/lib/minikube/etcd --initial-advertise-peer-urls=https://192.168.64.4:2380 --initial-cluster=minikube=https://192.168.64.4:2380 --key-file=/var/lib/minikube/certs/etcd/server.key --listen-client-urls=https://127.0.0.1:2379,https://192.168.64.4:2379 --listen-metrics-urls=http://127.0.0.1:2381 --listen-peer-urls=https://192.168.64.4:2380 --name=minikube --peer-cert-file=/var/lib/minikube/certs/etcd/peer.crt --peer-client-cert-auth=true --peer-key-file=/var/lib/minikube/certs/etcd/peer.key --peer-trusted-ca-file=/var/lib/minikube/certs/etcd/ca.crt --snapshot-count=10000 --trusted-ca-file=/var/lib/minikube/certs/etcd/ca.crt\nroot      4624  4605  8 Apr21 ?        04:55:10 kube-apiserver --advertise-address=192.168.64.4 --allow-privileged=true --authorization-mode=Node,RBAC --client-ca-file=/var/lib/minikube/certs/ca.crt --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodSecurityPolicy --enable-bootstrap-token-auth=true --etcd-cafile=/var/lib/minikube/certs/etcd/ca.crt --etcd-certfile=/var/lib/minikube/certs/apiserver-etcd-client.crt --etcd-keyfile=/var/lib/minikube/certs/apiserver-etcd-client.key --etcd-servers=https://127.0.0.1:2379 --insecure-port=0 --kubelet-client-certificate=/var/lib/minikube/certs/apiserver-kubelet-client.crt --kubelet-client-key=/var/lib/minikube/certs/apiserver-kubelet-client.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --proxy-client-cert-file=/var/lib/minikube/certs/front-proxy-client.crt --proxy-client-key-file=/var/lib/minikube/certs/front-proxy-client.key --requestheader-allowed-names=front-proxy-client --requestheader-client-ca-file=/var/lib/minikube/certs/front-proxy-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=8443 --service-account-key-file=/var/lib/minikube/certs/sa.pub --service-cluster-ip-range=10.96.0.0/12 --tls-cert-file=/var/lib/minikube/certs/apiserver.crt --tls-private-key-file=/var/lib/minikube/certs/apiserver.key\n",
+            "scored": true,
+            "expected_result": "'--cert-file' is present AND '--key-file' is present"
+          }
+        ]
+      }
+    ],
+    "total_pass": 7,
+    "total_fail": 0,
+    "total_warn": 0,
+    "total_info": 0
+  },
+  {
+    "id": "3",
+    "version": "1.5",
+    "text": "Control Plane Configuration",
+    "node_type": "controlplane",
+    "tests": [
+      {
+        "section": "3.1",
+        "pass": 0,
+        "fail": 0,
+        "warn": 1,
+        "info": 0,
+        "desc": "Authentication and Authorization",
+        "results": [
+          {
+            "test_number": "3.1.1",
+            "test_desc": "Client certificate authentication should not be used for users (Not Scored)",
+            "audit": "",
+            "AuditConfig": "",
+            "type": "manual",
+            "remediation": "Alternative mechanisms provided by Kubernetes such as the use of OIDC should be\nimplemented in place of client certificates.\n",
+            "test_info": [
+              "Alternative mechanisms provided by Kubernetes such as the use of OIDC should be\nimplemented in place of client certificates.\n"
+            ],
+            "status": "WARN",
+            "actual_value": "",
+            "scored": false,
+            "expected_result": "",
+            "reason": "Test marked as a manual test"
+          }
+        ]
+      }
+    ],
+    "total_pass": 0,
+    "total_fail": 0,
+    "total_warn": 3,
+    "total_info": 0
+  }
+]

cmd/util.go

@@ -4,7 +4,9 @@ import (
 	"fmt"
 	"os"
 	"os/exec"
+	"path/filepath"
 	"regexp"
+	"strconv"
 	"strings"
 
 	"github.com/aquasecurity/kube-bench/check"
@@ -25,67 +27,59 @@ var (
 
 var psFunc func(string) string
 var statFunc func(string) (os.FileInfo, error)
+var getBinariesFunc func(*viper.Viper, check.NodeType) (map[string]string, error)
+var TypeMap = map[string][]string{
+	"ca":         []string{"cafile", "defaultcafile"},
+	"kubeconfig": []string{"kubeconfig", "defaultkubeconfig"},
+	"service":    []string{"svc", "defaultsvc"},
+	"config":     []string{"confs", "defaultconf"},
+}
 
 func init() {
 	psFunc = ps
 	statFunc = os.Stat
-}
-
-func printlnWarn(msg string) {
-	fmt.Fprintf(os.Stderr, "[%s] %s\n",
-		colors[check.WARN].Sprintf("%s", check.WARN),
-		msg,
-	)
-}
-
-func sprintlnWarn(msg string) string {
-	return fmt.Sprintf("[%s] %s",
-		colors[check.WARN].Sprintf("%s", check.WARN),
-		msg,
-	)
+	getBinariesFunc = getBinaries
 }
 
 func exitWithError(err error) {
 	fmt.Fprintf(os.Stderr, "\n%v\n", err)
+	// flush before exit non-zero
+	glog.Flush()
 	os.Exit(1)
 }
 
-func continueWithError(err error, msg string) string {
-	if err != nil {
-		glog.V(2).Info(err)
-	}
-
-	if msg != "" {
-		fmt.Fprintf(os.Stderr, "%s\n", msg)
-	}
-
-	return ""
-}
-
-func cleanIDs(list string) []string {
+func cleanIDs(list string) map[string]bool {
 	list = strings.Trim(list, ",")
 	ids := strings.Split(list, ",")
 
+	set := make(map[string]bool)
+
 	for _, id := range ids {
 		id = strings.Trim(id, " ")
+		set[id] = true
 	}
 
-	return ids
+	return set
 }
 
 // ps execs out to the ps command; it's separated into a function so we can write tests
 func ps(proc string) string {
-	cmd := exec.Command("ps", "-C", proc, "-o", "cmd", "--no-headers")
+	// TODO: truncate proc to 15 chars
+	// See https://github.com/aquasecurity/kube-bench/issues/328#issuecomment-506813344
+	glog.V(2).Info(fmt.Sprintf("ps - proc: %q", proc))
+	cmd := exec.Command("/bin/ps", "-C", proc, "-o", "cmd", "--no-headers")
 	out, err := cmd.Output()
 	if err != nil {
-		continueWithError(fmt.Errorf("%s: %s", cmd.Args, err), "")
+		glog.V(2).Info(fmt.Errorf("%s: %s", cmd.Args, err))
 	}
 
+	glog.V(2).Info(fmt.Sprintf("ps - returning: %q", string(out)))
 	return string(out)
 }
 
-// getBinaries finds which of the set of candidate executables are running
-func getBinaries(v *viper.Viper) map[string]string {
+// getBinaries finds which of the set of candidate executables are running.
+// It returns an error if one mandatory executable is not running.
+func getBinaries(v *viper.Viper, nodetype check.NodeType) (map[string]string, error) {
 	binmap := make(map[string]string)
 
 	for _, component := range v.GetStringSlice("components") {
@@ -99,7 +93,8 @@ func getBinaries(v *viper.Viper) map[string]string {
 		if len(bins) > 0 {
 			bin, err := findExecutable(bins)
 			if err != nil && !optional {
-				exitWithError(fmt.Errorf("need %s executable but none of the candidates are running", component))
+				glog.Warning(buildComponentMissingErrorMessage(nodetype, component, bins))
+				return nil, fmt.Errorf("unable to detect running programs for component %q", component)
 			}
 
 			// Default the executable name that we'll substitute to the name of the component
@@ -113,14 +108,66 @@ func getBinaries(v *viper.Viper) map[string]string {
 		}
 	}
 
-	return binmap
+	return binmap, nil
 }
 
-// getConfigFiles finds which of the set of candidate config files exist
-// accepts a string 't' which indicates the type of config file, conf,
-// podspec or untifile.
-func getConfigFiles(v *viper.Viper) map[string]string {
-	confmap := make(map[string]string)
+// getConfigFilePath locates the config files we should be using for CIS version
+func getConfigFilePath(benchmarkVersion string, filename string) (path string, err error) {
+	glog.V(2).Info(fmt.Sprintf("Looking for config specific CIS version %q", benchmarkVersion))
+
+	path = filepath.Join(cfgDir, benchmarkVersion)
+	file := filepath.Join(path, string(filename))
+	glog.V(2).Info(fmt.Sprintf("Looking for file: %s", file))
+
+	if _, err := os.Stat(file); err != nil {
+		glog.V(2).Infof("error accessing config file: %q error: %v\n", file, err)
+		return "", fmt.Errorf("no test files found <= benchmark version: %s", benchmarkVersion)
+	}
+
+	return path, nil
+}
+
+// getYamlFilesFromDir returns a list of yaml files in the specified directory, ignoring config.yaml
+func getYamlFilesFromDir(path string) (names []string, err error) {
+	err = filepath.Walk(path, func(path string, info os.FileInfo, err error) error {
+		if err != nil {
+			return err
+		}
+
+		_, name := filepath.Split(path)
+		if name != "" && name != "config.yaml" && filepath.Ext(name) == ".yaml" {
+			names = append(names, path)
+		}
+
+		return nil
+	})
+	return names, err
+}
+
+// decrementVersion decrements the version number
+// We want to decrement individually even through versions where we don't supply test files
+// just in case someone wants to specify their own test files for that version
+func decrementVersion(version string) string {
+	split := strings.Split(version, ".")
+	if len(split) < 2 {
+		return ""
+	}
+	minor, err := strconv.Atoi(split[1])
+	if err != nil {
+		return ""
+	}
+	if minor <= 1 {
+		return ""
+	}
+	split[1] = strconv.Itoa(minor - 1)
+	return strings.Join(split, ".")
+}
+
+// getFiles finds which of the set of candidate files exist
+func getFiles(v *viper.Viper, fileType string) map[string]string {
+	filemap := make(map[string]string)
+	mainOpt := TypeMap[fileType][0]
+	defaultOpt := TypeMap[fileType][1]
 
 	for _, component := range v.GetStringSlice("components") {
 		s := v.Sub(component)
@@ -128,25 +175,25 @@ func getConfigFiles(v *viper.Viper) map[string]string {
 			continue
 		}
 
-		// See if any of the candidate config files exist
-		conf := findConfigFile(s.GetStringSlice("confs"))
-		if conf == "" {
-			if s.IsSet("defaultconf") {
-				conf = s.GetString("defaultconf")
-				glog.V(2).Info(fmt.Sprintf("Using default config file name '%s' for component %s", conf, component))
+		// See if any of the candidate files exist
+		file := findConfigFile(s.GetStringSlice(mainOpt))
+		if file == "" {
+			if s.IsSet(defaultOpt) {
+				file = s.GetString(defaultOpt)
+				glog.V(2).Info(fmt.Sprintf("Using default %s file name '%s' for component %s", fileType, file, component))
 			} else {
-				// Default the config file name that we'll substitute to the name of the component
-				glog.V(2).Info(fmt.Sprintf("Missing config file for %s", component))
-				conf = component
+				// Default the file name that we'll substitute to the name of the component
+				glog.V(2).Info(fmt.Sprintf("Missing %s file for %s", fileType, component))
+				file = component
 			}
 		} else {
-			glog.V(2).Info(fmt.Sprintf("Component %s uses config file '%s'", component, conf))
+			glog.V(2).Info(fmt.Sprintf("Component %s uses %s file '%s'", component, fileType, file))
 		}
 
-		confmap[component] = conf
+		filemap[component] = file
 	}
 
-	return confmap
+	return filemap
 }
 
 // verifyBin checks that the binary specified is running
@@ -168,6 +215,7 @@ func verifyBin(bin string) bool {
 	reFirstWord := regexp.MustCompile(`^(\S*\/)*` + bin)
 	lines := strings.Split(out, "\n")
 	for _, l := range lines {
+		glog.V(3).Info(fmt.Sprintf("reFirstWord.Match(%s)", l))
 		if reFirstWord.Match([]byte(l)) {
 			return true
 		}
@@ -196,9 +244,8 @@ func findExecutable(candidates []string) (string, error) {
 	for _, c := range candidates {
 		if verifyBin(c) {
 			return c, nil
-		} else {
-			glog.V(1).Info(fmt.Sprintf("executable '%s' not running", c))
 		}
+		glog.V(1).Info(fmt.Sprintf("executable '%s' not running", c))
 	}
 
 	return "", fmt.Errorf("no candidates running")
@@ -213,27 +260,94 @@ func multiWordReplace(s string, subname string, sub string) string {
 	return strings.Replace(s, subname, sub, -1)
 }
 
-func getKubeVersion() string {
-	// These executables might not be on the user's path.
-	_, err := exec.LookPath("kubectl")
-	if err != nil {
-		exitWithError(fmt.Errorf("kubernetes version check failed: %v", err))
+const missingKubectlKubeletMessage = `
+Unable to find the programs kubectl or kubelet in the PATH.
+These programs are used to determine which version of Kubernetes is running.
+Make sure the /usr/local/mount-from-host/bin directory is mapped to the container,
+either in the job.yaml file, or Docker command.
+
+For job.yaml:
+...
+- name: usr-bin
+  mountPath: /usr/local/mount-from-host/bin
+...
+
+For docker command:
+   docker -v $(which kubectl):/usr/local/mount-from-host/bin/kubectl ....
+
+Alternatively, you can specify the version with --version
+   kube-bench --version <VERSION> ...
+`
+
+func getKubeVersion() (string, error) {
+
+	if k8sVer, err := getKubeVersionFromRESTAPI(); err == nil {
+		glog.V(2).Info(fmt.Sprintf("Kubernetes REST API Reported version: %s", k8sVer))
+		return k8sVer, nil
 	}
 
+	// These executables might not be on the user's path.
+	_, err := exec.LookPath("kubectl")
+
+	if err != nil {
+		_, err = exec.LookPath("kubelet")
+		if err != nil {
+			// Search for the kubelet binary all over the filesystem and run the first match to get the kubernetes version
+			cmd := exec.Command("/bin/sh", "-c", "`find / -type f -executable -name kubelet 2>/dev/null | grep -m1 .` --version")
+			out, err := cmd.CombinedOutput()
+			if err == nil {
+				return getVersionFromKubeletOutput(string(out)), nil
+			}
+
+			glog.Warning(missingKubectlKubeletMessage)
+			return "", fmt.Errorf("unable to find the programs kubectl or kubelet in the PATH")
+		}
+		return getKubeVersionFromKubelet(), nil
+	}
+
+	return getKubeVersionFromKubectl(), nil
+}
+
+func getKubeVersionFromKubectl() string {
 	cmd := exec.Command("kubectl", "version", "--short")
 	out, err := cmd.CombinedOutput()
 	if err != nil {
-		continueWithError(fmt.Errorf("%s", out), "")
+		glog.V(2).Info(err)
 	}
 
 	return getVersionFromKubectlOutput(string(out))
 }
 
+func getKubeVersionFromKubelet() string {
+	cmd := exec.Command("kubelet", "--version")
+	out, err := cmd.CombinedOutput()
+
+	if err != nil {
+		glog.V(2).Info(err)
+	}
+
+	return getVersionFromKubeletOutput(string(out))
+}
+
 func getVersionFromKubectlOutput(s string) string {
 	serverVersionRe := regexp.MustCompile(`Server Version: v(\d+.\d+)`)
 	subs := serverVersionRe.FindStringSubmatch(s)
 	if len(subs) < 2 {
-		printlnWarn(fmt.Sprintf("Unable to get kubectl version, using default version: %s", defaultKubeVersion))
+		if strings.Contains(s, "The connection to the server") {
+			msg := fmt.Sprintf(`Warning: Kubernetes version was not auto-detected because kubectl could not connect to the Kubernetes server. This may be because the kubeconfig information is missing or has credentials that do not match the server. Assuming default version %s`, defaultKubeVersion)
+			fmt.Fprintln(os.Stderr, msg)
+		}
+		glog.V(1).Info(fmt.Sprintf("Unable to get Kubernetes version from kubectl, using default version: %s", defaultKubeVersion))
+		return defaultKubeVersion
+	}
+	return subs[1]
+}
+
+func getVersionFromKubeletOutput(s string) string {
+	serverVersionRe := regexp.MustCompile(`Kubernetes v(\d+.\d+)`)
+	subs := serverVersionRe.FindStringSubmatch(s)
+	if len(subs) < 2 {
+		glog.V(1).Info(fmt.Sprintf("Unable to get Kubernetes version from kubelet, using default version: %s", defaultKubeVersion))
 		return defaultKubeVersion
 	}
 	return subs[1]
@@ -243,12 +357,49 @@ func makeSubstitutions(s string, ext string, m map[string]string) string {
 	for k, v := range m {
 		subst := "$" + k + ext
 		if v == "" {
-			glog.V(2).Info(fmt.Sprintf("No subsitution for '%s'\n", subst))
+			glog.V(2).Info(fmt.Sprintf("No substitution for '%s'\n", subst))
 			continue
 		}
-		glog.V(1).Info(fmt.Sprintf("Substituting %s with '%s'\n", subst, v))
+		glog.V(2).Info(fmt.Sprintf("Substituting %s with '%s'\n", subst, v))
 		s = multiWordReplace(s, subst, v)
 	}
 
 	return s
 }
+
+func isEmpty(str string) bool {
+	return strings.TrimSpace(str) == ""
+
+}
+
+func buildComponentMissingErrorMessage(nodetype check.NodeType, component string, bins []string) string {
+
+	errMessageTemplate := `
+Unable to detect running programs for component %q
+The following %q programs have been searched, but none of them have been found:
+%s
+
+These program names are provided in the config.yaml, section '%s.%s.bins'
+`
+
+	var componentRoleName, componentType string
+	switch nodetype {
+
+	case check.NODE:
+		componentRoleName = "worker node"
+		componentType = "node"
+	case check.ETCD:
+		componentRoleName = "etcd node"
+		componentType = "etcd"
+	default:
+		componentRoleName = "master node"
+		componentType = "master"
+	}
+
+	binList := ""
+	for _, bin := range bins {
+		binList = fmt.Sprintf("%s\t- %s\n", binList, bin)
+	}
+
+	return fmt.Sprintf(errMessageTemplate, component, componentRoleName, binList, componentType, component)
+}

cmd/util_test.go

@@ -15,11 +15,14 @@
 package cmd
 
 import (
+	"io/ioutil"
 	"os"
+	"path/filepath"
 	"reflect"
 	"strconv"
 	"testing"
 
+	"github.com/aquasecurity/kube-bench/check"
 	"github.com/spf13/viper"
 )
 
@@ -107,38 +110,51 @@ func TestFindExecutable(t *testing.T) {
 
 func TestGetBinaries(t *testing.T) {
 	cases := []struct {
-		config map[string]interface{}
-		psOut  string
-		exp    map[string]string
+		config    map[string]interface{}
+		psOut     string
+		exp       map[string]string
+		expectErr bool
 	}{
 		{
-			config: map[string]interface{}{"components": []string{"apiserver"}, "apiserver": map[string]interface{}{"bins": []string{"apiserver", "kube-apiserver"}}},
-			psOut:  "kube-apiserver",
-			exp:    map[string]string{"apiserver": "kube-apiserver"},
+			config:    map[string]interface{}{"components": []string{"apiserver"}, "apiserver": map[string]interface{}{"bins": []string{"apiserver", "kube-apiserver"}}},
+			psOut:     "kube-apiserver",
+			exp:       map[string]string{"apiserver": "kube-apiserver"},
+			expectErr: false,
 		},
 		{
 			// "thing" is not in the list of components
-			config: map[string]interface{}{"components": []string{"apiserver"}, "apiserver": map[string]interface{}{"bins": []string{"apiserver", "kube-apiserver"}}, "thing": map[string]interface{}{"bins": []string{"something else", "thing"}}},
-			psOut:  "kube-apiserver thing",
-			exp:    map[string]string{"apiserver": "kube-apiserver"},
+			config:    map[string]interface{}{"components": []string{"apiserver"}, "apiserver": map[string]interface{}{"bins": []string{"apiserver", "kube-apiserver"}}, "thing": map[string]interface{}{"bins": []string{"something else", "thing"}}},
+			psOut:     "kube-apiserver thing",
+			exp:       map[string]string{"apiserver": "kube-apiserver"},
+			expectErr: false,
 		},
 		{
 			// "anotherthing" in list of components but doesn't have a defintion
-			config: map[string]interface{}{"components": []string{"apiserver", "anotherthing"}, "apiserver": map[string]interface{}{"bins": []string{"apiserver", "kube-apiserver"}}, "thing": map[string]interface{}{"bins": []string{"something else", "thing"}}},
-			psOut:  "kube-apiserver thing",
-			exp:    map[string]string{"apiserver": "kube-apiserver"},
+			config:    map[string]interface{}{"components": []string{"apiserver", "anotherthing"}, "apiserver": map[string]interface{}{"bins": []string{"apiserver", "kube-apiserver"}}, "thing": map[string]interface{}{"bins": []string{"something else", "thing"}}},
+			psOut:     "kube-apiserver thing",
+			exp:       map[string]string{"apiserver": "kube-apiserver"},
+			expectErr: false,
 		},
 		{
 			// more than one component
-			config: map[string]interface{}{"components": []string{"apiserver", "thing"}, "apiserver": map[string]interface{}{"bins": []string{"apiserver", "kube-apiserver"}}, "thing": map[string]interface{}{"bins": []string{"something else", "thing"}}},
-			psOut:  "kube-apiserver \nthing",
-			exp:    map[string]string{"apiserver": "kube-apiserver", "thing": "thing"},
+			config:    map[string]interface{}{"components": []string{"apiserver", "thing"}, "apiserver": map[string]interface{}{"bins": []string{"apiserver", "kube-apiserver"}}, "thing": map[string]interface{}{"bins": []string{"something else", "thing"}}},
+			psOut:     "kube-apiserver \nthing",
+			exp:       map[string]string{"apiserver": "kube-apiserver", "thing": "thing"},
+			expectErr: false,
 		},
 		{
 			// default binary to component name
-			config: map[string]interface{}{"components": []string{"apiserver", "thing"}, "apiserver": map[string]interface{}{"bins": []string{"apiserver", "kube-apiserver"}}, "thing": map[string]interface{}{"bins": []string{"something else", "thing"}, "optional": true}},
-			psOut:  "kube-apiserver \notherthing some params",
-			exp:    map[string]string{"apiserver": "kube-apiserver", "thing": "thing"},
+			config:    map[string]interface{}{"components": []string{"apiserver", "thing"}, "apiserver": map[string]interface{}{"bins": []string{"apiserver", "kube-apiserver"}}, "thing": map[string]interface{}{"bins": []string{"something else", "thing"}, "optional": true}},
+			psOut:     "kube-apiserver \notherthing some params",
+			exp:       map[string]string{"apiserver": "kube-apiserver", "thing": "thing"},
+			expectErr: false,
+		},
+		{
+			// missing mandatory component
+			config:    map[string]interface{}{"components": []string{"apiserver", "thing"}, "apiserver": map[string]interface{}{"bins": []string{"apiserver", "kube-apiserver"}}, "thing": map[string]interface{}{"bins": []string{"something else", "thing"}, "optional": true}},
+			psOut:     "otherthing some params",
+			exp:       map[string]string{"apiserver": "kube-apiserver", "thing": "thing"},
+			expectErr: true,
 		},
 	}
 
@@ -151,8 +167,12 @@ func TestGetBinaries(t *testing.T) {
 			for k, val := range c.config {
 				v.Set(k, val)
 			}
-			m := getBinaries(v)
-			if !reflect.DeepEqual(m, c.exp) {
+			m, err := getBinaries(v, check.MASTER)
+			if c.expectErr {
+				if err == nil {
+					t.Fatal("Got nil Expected error")
+				}
+			} else if !reflect.DeepEqual(m, c.exp) {
 				t.Fatalf("Got %v\nExpected %v", m, c.exp)
 			}
 		})
@@ -190,8 +210,8 @@ func TestKubeVersionRegex(t *testing.T) {
 	}
 
 	ver = getVersionFromKubectlOutput("Something completely different")
-	if ver != "1.6" {
-		t.Fatalf("Expected 1.6 got %s", ver)
+	if ver != defaultKubeVersion {
+		t.Fatalf("Expected %s got %s", defaultKubeVersion, ver)
 	}
 }
 
@@ -279,7 +299,82 @@ func TestGetConfigFiles(t *testing.T) {
 			e = c.statResults
 			eIndex = 0
 
-			m := getConfigFiles(v)
+			m := getFiles(v, "config")
+			if !reflect.DeepEqual(m, c.exp) {
+				t.Fatalf("Got %v\nExpected %v", m, c.exp)
+			}
+		})
+	}
+}
+
+func TestGetServiceFiles(t *testing.T) {
+	cases := []struct {
+		config      map[string]interface{}
+		exp         map[string]string
+		statResults []error
+	}{
+		{
+			config: map[string]interface{}{
+				"components": []string{"kubelet"},
+				"kubelet":    map[string]interface{}{"svc": []string{"kubelet", "10-kubeadm.conf"}},
+			},
+			statResults: []error{os.ErrNotExist, nil},
+			exp:         map[string]string{"kubelet": "10-kubeadm.conf"},
+		},
+		{
+			// Component "thing" isn't included in the list of components
+			config: map[string]interface{}{
+				"components": []string{"kubelet"},
+				"kubelet":    map[string]interface{}{"svc": []string{"kubelet", "10-kubeadm.conf"}},
+				"thing":      map[string]interface{}{"svc": []string{"/my/file/thing"}},
+			},
+			statResults: []error{os.ErrNotExist, nil},
+			exp:         map[string]string{"kubelet": "10-kubeadm.conf"},
+		},
+		{
+			// More than one component
+			config: map[string]interface{}{
+				"components": []string{"kubelet", "thing"},
+				"kubelet":    map[string]interface{}{"svc": []string{"kubelet", "10-kubeadm.conf"}},
+				"thing":      map[string]interface{}{"svc": []string{"/my/file/thing"}},
+			},
+			statResults: []error{os.ErrNotExist, nil, nil},
+			exp:         map[string]string{"kubelet": "10-kubeadm.conf", "thing": "/my/file/thing"},
+		},
+		{
+			// Default thing to specified default service
+			config: map[string]interface{}{
+				"components": []string{"kubelet", "thing"},
+				"kubelet":    map[string]interface{}{"svc": []string{"kubelet", "10-kubeadm.conf"}},
+				"thing":      map[string]interface{}{"svc": []string{"/my/file/thing"}, "defaultsvc": "another/thing"},
+			},
+			statResults: []error{os.ErrNotExist, nil, os.ErrNotExist},
+			exp:         map[string]string{"kubelet": "10-kubeadm.conf", "thing": "another/thing"},
+		},
+		{
+			// Default thing to component name
+			config: map[string]interface{}{
+				"components": []string{"kubelet", "thing"},
+				"kubelet":    map[string]interface{}{"svc": []string{"kubelet", "10-kubeadm.conf"}},
+				"thing":      map[string]interface{}{"svc": []string{"/my/file/thing"}},
+			},
+			statResults: []error{os.ErrNotExist, nil, os.ErrNotExist},
+			exp:         map[string]string{"kubelet": "10-kubeadm.conf", "thing": "thing"},
+		},
+	}
+
+	v := viper.New()
+	statFunc = fakestat
+
+	for id, c := range cases {
+		t.Run(strconv.Itoa(id), func(t *testing.T) {
+			for k, val := range c.config {
+				v.Set(k, val)
+			}
+			e = c.statResults
+			eIndex = 0
+
+			m := getFiles(v, "service")
 			if !reflect.DeepEqual(m, c.exp) {
 				t.Fatalf("Got %v\nExpected %v", m, c.exp)
 			}
@@ -306,3 +401,111 @@ func TestMakeSubsitutions(t *testing.T) {
 		})
 	}
 }
+
+func TestGetConfigFilePath(t *testing.T) {
+	var err error
+	cfgDir, err = ioutil.TempDir("", "kube-bench-test")
+	if err != nil {
+		t.Fatalf("Failed to create temp directory")
+	}
+	defer os.RemoveAll(cfgDir)
+	d := filepath.Join(cfgDir, "cis-1.4")
+	err = os.Mkdir(d, 0766)
+	if err != nil {
+		t.Fatalf("Failed to create temp dir")
+	}
+	err = ioutil.WriteFile(filepath.Join(d, "master.yaml"), []byte("hello world"), 0666)
+	if err != nil {
+		t.Logf("Failed to create temp file")
+	}
+
+	cases := []struct {
+		benchmarkVersion string
+		succeed          bool
+		exp              string
+	}{
+		{benchmarkVersion: "cis-1.4", succeed: true, exp: d},
+		{benchmarkVersion: "cis-1.5", succeed: false, exp: ""},
+		{benchmarkVersion: "1.1", succeed: false, exp: ""},
+	}
+
+	for _, c := range cases {
+		t.Run(c.benchmarkVersion, func(t *testing.T) {
+			path, err := getConfigFilePath(c.benchmarkVersion, "/master.yaml")
+			if c.succeed {
+				if err != nil {
+					t.Fatalf("Error %v", err)
+				}
+				if path != c.exp {
+					t.Fatalf("Got %s expected %s", path, c.exp)
+				}
+			} else {
+				if err == nil {
+					t.Fatalf("Expected Error, but none")
+				}
+			}
+		})
+	}
+}
+
+func TestDecrementVersion(t *testing.T) {
+
+	cases := []struct {
+		kubeVersion string
+		succeed     bool
+		exp         string
+	}{
+		{kubeVersion: "1.13", succeed: true, exp: "1.12"},
+		{kubeVersion: "1.15", succeed: true, exp: "1.14"},
+		{kubeVersion: "1.11", succeed: true, exp: "1.10"},
+		{kubeVersion: "1.1", succeed: true, exp: ""},
+		{kubeVersion: "invalid", succeed: false, exp: ""},
+	}
+	for _, c := range cases {
+		rv := decrementVersion(c.kubeVersion)
+		if c.succeed {
+			if c.exp != rv {
+				t.Fatalf("decrementVersion(%q) - Got %q expected %s", c.kubeVersion, rv, c.exp)
+			}
+		} else {
+			if len(rv) > 0 {
+				t.Fatalf("decrementVersion(%q) - Expected empty string but Got %s", c.kubeVersion, rv)
+			}
+		}
+	}
+}
+
+func TestGetYamlFilesFromDir(t *testing.T) {
+	cfgDir, err := ioutil.TempDir("", "kube-bench-test")
+	if err != nil {
+		t.Fatalf("Failed to create temp directory")
+	}
+	defer os.RemoveAll(cfgDir)
+
+	d := filepath.Join(cfgDir, "cis-1.4")
+	err = os.Mkdir(d, 0766)
+	if err != nil {
+		t.Fatalf("Failed to create temp dir")
+	}
+
+	err = ioutil.WriteFile(filepath.Join(d, "something.yaml"), []byte("hello world"), 0666)
+	if err != nil {
+		t.Fatalf("error writing file %v", err)
+	}
+	err = ioutil.WriteFile(filepath.Join(d, "config.yaml"), []byte("hello world"), 0666)
+	if err != nil {
+		t.Fatalf("error writing file %v", err)
+	}
+
+	files, err := getYamlFilesFromDir(d)
+	if err != nil {
+		t.Fatalf("Unexpected error: %v", err)
+	}
+	if len(files) != 1 {
+		t.Fatalf("Expected to find one file, found %d", len(files))
+	}
+
+	if files[0] != filepath.Join(d, "something.yaml") {
+		t.Fatalf("Expected to find something.yaml, found %s", files[0])
+	}
+}

cmd/version.go

@@ -0,0 +1,22 @@
+package cmd
+
+import (
+	"fmt"
+	"github.com/spf13/cobra"
+)
+
+var KubeBenchVersion string
+
+// versionCmd represents the version command
+var versionCmd = &cobra.Command{
+	Use:   "version",
+	Short: "Shows the version of kube-bench.",
+	Long:  `Shows the version of kube-bench.`,
+	Run: func(cmd *cobra.Command, args []string) {
+		fmt.Println(KubeBenchVersion)
+	},
+}
+
+func init() {
+	RootCmd.AddCommand(versionCmd)
+}

codecov.yml

@@ -0,0 +1,6 @@
+coverage:
+  status:
+    project:
+      default:
+        target: auto  # auto compares coverage to the previous base commit
+        threshold: 1%

docs/README.md

@@ -0,0 +1,341 @@
+# Test and config files
+
+`kube-bench` runs checks specified in `controls` files that are a YAML 
+representation of the CIS Kubernetes Benchmark checks (or other distribution-specific hardening guides). 
+
+## Controls
+
+`controls` is a YAML document that contains checks that must be run against a 
+specific Kubernetes node type, master or node and version.
+
+`controls` is the fundamental input to `kube-bench`. The following is an example 
+of a basic `controls`:
+
+```yml
+---
+controls:
+id: 1
+text: "Master Node Security Configuration"
+type: "master"
+groups:
+- id: 1.1
+  text: API Server
+  checks:
+    - id: 1.1.1
+      text: "Ensure that the --allow-privileged argument is set (Scored)"
+      audit: "ps -ef | grep kube-apiserver | grep -v grep"
+      tests:
+      bin_op: or
+      test_items:
+      - flag: "--allow-privileged"
+        set: true
+      - flag: "--some-other-flag"
+        set: false
+      remediation: "Edit the /etc/kubernetes/config file on the master node and
+        set the KUBE_ALLOW_PRIV parameter to '--allow-privileged=false'"
+      scored: true
+- id: 1.2
+  text: Scheduler
+  checks:
+    - id: 1.2.1
+      text: "Ensure that the --profiling argument is set to false (Scored)"
+      audit: "ps -ef | grep kube-scheduler | grep -v grep"
+      tests:
+        bin_op: or
+        test_items:
+          - flag: "--profiling"
+            set: true
+          - flag: "--some-other-flag"
+            set: false
+      remediation: "Edit the /etc/kubernetes/config file on the master node and
+        set the KUBE_ALLOW_PRIV parameter to '--allow-privileged=false'"
+      scored: true
+```
+
+`controls` is composed of a hierarchy of groups, sub-groups and checks. Each of
+the `controls` components have an id and a text description which are displayed 
+in the `kube-bench` output.
+
+`type` specifies what kubernetes node type a `controls` is for. Possible values
+for `type` are `master` and `node`.
+
+## Groups
+
+`groups` is a list of subgroups that test the various Kubernetes components
+that run on the node type specified in the `controls`. 
+
+For example, one subgroup checks parameters passed to the API server binary, while 
+another subgroup checks parameters passed to the controller-manager binary.
+
+```yml
+groups:
+- id: 1.1
+  text: API Server
+  # ...
+- id: 1.2
+  text: Scheduler
+  # ...
+```
+
+These subgroups have `id`, `text` fields which serve the same purposes described
+in the previous paragraphs. The most important part of the subgroup is the
+`checks` field which is the collection of actual `check`s that form the subgroup.
+
+This is an example of a subgroup and checks in the subgroup.
+
+```yml
+id: 1.1
+text: API Server
+checks:
+  - id: 1.1.1
+    text: "Ensure that the --allow-privileged argument is set (Scored)"
+    audit: "ps -ef | grep kube-apiserver | grep -v grep"
+    tests:
+    # ...
+  - id: 1.1.2
+    text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
+    audit: "ps -ef | grep kube-apiserver | grep -v grep"
+    tests:
+    # ...
+``` 
+
+`kube-bench` supports running a subgroup by specifying the subgroup `id` on the
+command line, with the flag `--group` or `-g`.
+
+## Check
+
+The CIS Kubernetes Benchmark recommends configurations to harden Kubernetes components. These recommendations are usually configuration options and can be 
+specified by flags to Kubernetes binaries, or in configuration files.
+
+The Benchmark also provides commands to audit a Kubernetes installation, identify
+places where the cluster security can be improved, and steps to remediate these
+identified problems.
+
+In `kube-bench`, `check` objects embody these recommendations.  This an example
+`check` object:
+
+```yml
+id: 1.1.1
+text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
+audit: "ps -ef | grep kube-apiserver | grep -v grep"
+tests:
+  test_items:
+  - flag: "--anonymous-auth"
+    compare:
+      op: eq
+      value: false
+    set: true
+remediation: |
+  Edit the API server pod specification file kube-apiserver
+  on the master node and set the below parameter.
+  --anonymous-auth=false
+scored: false
+```
+
+A `check` object has an `id`, a `text`, an `audit`, a `tests`, `remediation`
+and `scored` fields.
+
+`kube-bench` supports running individual checks by specifying the check's `id`
+as a comma-delimited list on the command line with the `--check` flag.
+
+The `audit` field specifies the command to run for a check. The output of this
+command is then evaluated for conformance with the CIS Kubernetes Benchmark
+recommendation.
+
+The audit is evaluated against criteria specified by the `tests`
+object. `tests` contain `bin_op` and `test_items`.
+
+`test_items` specify the criteria(s) the `audit` command's output should meet to
+pass a check. This criteria is made up of keywords extracted from the output of
+the `audit` command and operations that compare these keywords against
+values expected by the CIS Kubernetes Benchmark. 
+
+There are two ways to extract keywords from the output of the `audit` command,
+`flag` and `path`.
+
+`flag` is used when the keyword is a command-line flag. The associated `audit`
+command is usually a `ps` command and a `grep` for the binary whose flag we are
+checking:
+
+```sh
+ps -ef | grep somebinary | grep -v grep
+``` 
+
+Here is an example usage of the `flag` option:
+
+```yml
+# ...
+audit: "ps -ef | grep kube-apiserver | grep -v grep"
+tests:
+  test_items:
+  - flag: "--anonymous-auth"
+  # ...
+```
+
+`path` is used when the keyword is an option set in a JSON or YAML config file.
+The associated `audit` command is usually `cat /path/to/config-yaml-or-json`.
+For example:
+
+```yml
+# ...
+text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
+audit: "cat /path/to/some/config"
+tests:
+  test_items:
+  - path: "{.someoption.value}"
+    # ...
+```
+
+`test_item` compares the output of the audit command and keywords using the
+`set` and `compare` fields.
+
+```yml
+  test_items:
+  - flag: "--anonymous-auth"
+    compare:
+      op: eq
+      value: false
+    set: true
+```
+
+`set` checks if a keyword is present in the output of the audit command or a config file. The possible values for `set` are true and false.
+
+If `set` is true, the check passes only if the keyword is present in the output
+of the audit command, or config file. If `set` is false, the check passes only
+if the keyword is not present in the output of the audit command, or config file.
+
+`compare` has two fields `op` and `value` to compare keywords with expected
+value. `op` specifies which operation is used for the comparison, and `value`
+specifies the value to compare against.
+
+> To use `compare`, `set` must true. The comparison will be ignored if `set` is
+> false
+
+The `op` (operations) currently supported in `kube-bench` are:
+- `eq`: tests if the keyword is equal to the compared value.
+- `noteq`: tests if the keyword is unequal to the compared value.
+- `gt`: tests if the keyword is greater than the compared value.
+- `gte`: tests if the keyword is greater than or equal to the compared value.
+- `lt`: tests if the keyword is less than the compared value.
+- `lte`: tests if the keyword is less than or equal to the compared value.
+- `has`: tests if the keyword contains the compared value.
+- `nothave`: tests if the keyword does not contain the compared value.
+- `regex`: tests if the flag value matches the compared value regular expression.
+   When defining regular expressions in YAML it is generally easier to wrap them in
+   single quotes, for example `'^[abc]$'`, to avoid issues with string escaping.
+
+## Configuration and Variables
+
+Kubernetes component configuration and binary file locations and names 
+vary based on cluster deployment methods and Kubernetes distribution used.
+For this reason, the locations of these binaries and config files are configurable
+by editing the `cfg/config.yaml` file and these binaries and files can be
+referenced in a `controls` file via variables.
+
+The `cfg/config.yaml` file is a global configuration file. Configuration files
+can be created for specific Kubernetes versions (distributions). Values in the
+version-specific config overwrite similar values in `cfg/config.yaml`.
+
+For example, the kube-apiserver in Red Hat OCP distribution is run as 
+`hypershift openshift-kube-apiserver` instead of the default `kube-apiserver`.
+This difference can be specified by editing the `master.apiserver.defaultbin`
+entry `cfg/ocp-3.10/config.yaml`.
+
+Below is the structure of `cfg/config.yaml`:
+
+```
+nodetype
+  |-- components
+    |-- component1
+  |-- component1
+    |-- bins
+    |-- defaultbin (optional)
+    |-- confs
+    |-- defaultconf (optional)
+    |-- svcs
+    |-- defaultsvc (optional)
+    |-- kubeconfig
+    |-- defaultkubeconfig (optional)
+```
+
+Every node type has a subsection that specifies the main configuration items.
+
+- `components`: A list of components for the node type. For example master 
+  will have an entry for **apiserver**, **scheduler** and **controllermanager**.
+  
+  Each component has the following entries:
+
+- `bins`: A list of candidate binaries for a component. `kube-bench` checks this
+   list and selects the first binary that is running on the node.
+
+   If none of the binaries in `bins` list is running, `kube-bench` checks if the
+   binary specified by `defaultbin` is running and terminates if none of the 
+   binaries in both `bins` and `defaultbin` is running.
+   
+   The selected binary for a component can be referenced in `controls` using a 
+   variable in the form `$<component>bin`. In the example below, we reference 
+   the selected API server binary with the variable `$apiserverbin` in an `audit`
+   command.
+   
+   ```yml
+   id: 1.1.1
+    text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
+    audit: "ps -ef | grep $apiserverbin | grep -v grep"
+    # ...
+   ```
+   
+- `confs`: A list of candidate configuration files for a component. `kube-bench`
+  checks this list and selects the first config file that is found on the node.
+  If none of the config files exists, `kube-bench` defaults conf to the value
+  of `defaultconf`.
+  
+  The selected config for a component can be referenced in `controls` using a
+  variable in the form `$<component>conf`. In the example below, we reference the 
+  selected API server config file with the variable `$apiserverconf` in an `audit`
+  command.
+  
+  ```yml
+  id: 1.4.1
+    text: "Ensure that the API server pod specification file permissions are
+    set to 644 or more restrictive (Scored)"
+    audit: "/bin/sh -c 'if test -e $apiserverconf; then stat -c %a $apiserverconf; fi'"
+  ```
+  
+- `svcs`:  A list of candidate unitfiles for a component. `kube-bench` checks this 
+  list and selects the first unitfile that is found on the node. If none of the
+  unitfiles exists, `kube-bench` defaults unitfile to the value of `defaultsvc`.
+  
+  The selected unitfile for a component can be referenced in `controls` via a
+  variable in the form `$<component>svc`. In the example below, the selected 
+  kubelet unitfile is referenced with `$kubeletsvc` in the `remediation` of the 
+  `check`.
+  
+  ```yml
+  id: 2.1.1
+    # ...
+    remediation: |
+      Edit the kubelet service file $kubeletsvc
+      on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+      --allow-privileged=false
+      Based on your system, restart the kubelet service. For example:
+      systemctl daemon-reload
+      systemctl restart kubelet.service
+    # ...
+  ```
+  
+  - `kubeconfig`: A list of candidate kubeconfig files for a component. `kube-bench`
+    checks this list and selects the first file that is found on the node. If none
+    of the files exists, `kube-bench` defaults kubeconfig to the value of 
+    `defaultkubeconfig`.
+    
+    The selected kubeconfig for a component can be referenced in `controls` with a variable in the form `$<component>kubeconfig`. In the example below, the
+    selected kubelet kubeconfig is referenced with `$kubeletkubeconfig` in the
+    `audit` command.
+    
+    ```yml
+    id: 2.2.1
+      text: "Ensure that the kubelet.conf file permissions are set to 644 or
+      more restrictive (Scored)"
+      audit: "/bin/sh -c 'if test -e $kubeletkubeconfig; then stat -c %a $kubeletkubeconfig; fi'"
+      # ...
+    ```

entrypoint.sh

@@ -1,14 +1,19 @@
-#!/bin/sh
-if [ -d /host ]; then
-  mkdir -p /host/cfg/
-  yes | cp -rf /cfg/* /host/cfg/
-  yes | cp -rf /kube-bench /host/
-  echo "==============================================="
-  echo "kube-bench is now installed on your host       "
-  echo "Run ./kube-bench to perform a security check   "
-  echo "==============================================="
+#!/bin/sh -e
+if [ "$1" == "install" ]; then
+  if [ -d /host ]; then
+    mkdir -p /host/cfg/
+    yes | cp -rf cfg/* /host/cfg/
+    yes | cp -rf /usr/local/bin/kube-bench /host/
+    echo "==============================================="
+    echo "kube-bench is now installed on your host       "
+    echo "Run ./kube-bench to perform a security check   "
+    echo "==============================================="
+  else
+    echo "Usage:"
+    echo "  install: docker run --rm -v \`pwd\`:/host aquasec/kube-bench install"
+    echo "  run:     docker run --rm --pid=host aquasec/kube-bench [command]"
+    exit
+  fi
 else
-  echo "Usage:"
-  echo "  docker run --rm -v \`pwd\`:/host aquasec/kube-bench"
-  exit 
+  exec kube-bench "$@"
 fi

glide.lock

@@ -1,72 +0,0 @@
-hash: f3cf12cf95d66d315c4aef2f3d0940770bd26267f84703e53c4928b786a91c14
-updated: 2018-01-09T12:49:41.3014329-08:00
-imports:
-- name: github.com/fatih/color
-  version: 570b54cabe6b8eb0bc2dfce68d964677d63b5260
-- name: github.com/fsnotify/fsnotify
-  version: 4da3e2cfbabc9f751898f250b49f2439785783a1
-- name: github.com/golang/glog
-  version: 23def4e6c14b4da8ac2ed8007337bc5eb5007998
-- name: github.com/hashicorp/hcl
-  version: 23c074d0eceb2b8a5bfdbb271ab780cde70f05a8
-  subpackages:
-  - hcl/ast
-  - hcl/parser
-  - hcl/scanner
-  - hcl/strconv
-  - hcl/token
-  - json/parser
-  - json/scanner
-  - json/token
-- name: github.com/inconshreveable/mousetrap
-  version: 76626ae9c91c4f2a10f34cad8ce83ea42c93bb75
-- name: github.com/jinzhu/gorm
-  version: 5174cc5c242a728b435ea2be8a2f7f998e15429b
-  subpackages:
-  - dialects/postgres
-- name: github.com/jinzhu/inflection
-  version: 1c35d901db3da928c72a72d8458480cc9ade058f
-- name: github.com/lib/pq
-  version: 83612a56d3dd153a94a629cd64925371c9adad78
-  subpackages:
-  - hstore
-  - oid
-- name: github.com/magiconair/properties
-  version: 49d762b9817ba1c2e9d0c69183c2b4a8b8f1d934
-- name: github.com/mattn/go-colorable
-  version: 5411d3eea5978e6cdc258b30de592b60df6aba96
-  repo: https://github.com/mattn/go-colorable
-- name: github.com/mattn/go-isatty
-  version: 57fdcb988a5c543893cc61bce354a6e24ab70022
-  repo: https://github.com/mattn/go-isatty
-- name: github.com/mitchellh/mapstructure
-  version: 06020f85339e21b2478f756a78e295255ffa4d6a
-- name: github.com/pelletier/go-toml
-  version: 0131db6d737cfbbfb678f8b7d92e55e27ce46224
-- name: github.com/spf13/afero
-  version: 57afd63c68602b63ed976de00dd066ccb3c319db
-  subpackages:
-  - mem
-- name: github.com/spf13/cast
-  version: acbeb36b902d72a7a4c18e8f3241075e7ab763e4
-- name: github.com/spf13/cobra
-  version: 7b2c5ac9fc04fc5efafb60700713d4fa609b777b
-- name: github.com/spf13/jwalterweatherman
-  version: 12bd96e66386c1960ab0f74ced1362f66f552f7b
-- name: github.com/spf13/pflag
-  version: 4c012f6dcd9546820e378d0bdda4d8fc772cdfea
-- name: github.com/spf13/viper
-  version: 25b30aa063fc18e48662b86996252eabdcf2f0c7
-- name: golang.org/x/sys
-  version: e24f485414aeafb646f6fca458b0bf869c0880a1
-  repo: https://go.googlesource.com/sys
-  subpackages:
-  - unix
-- name: golang.org/x/text
-  version: e19ae1496984b1c655b8044a65c0300a3c878dd3
-  subpackages:
-  - transform
-  - unicode/norm
-- name: gopkg.in/yaml.v2
-  version: c95af922eae69f190717a0b7148960af8c55a072
-testImports: []

glide.yaml

@@ -1,14 +0,0 @@
-package: github.com/aquasecurity/kube-bench
-import:
-- package: github.com/fatih/color
-  version: ^1.5.0
-- package: github.com/golang/glog
-- package: github.com/jinzhu/gorm
-  version: ^1.0.0
-  subpackages:
-  - dialects/postgres
-- package: github.com/spf13/cobra
-  version: ^0.0.1
-- package: github.com/spf13/viper
-  version: ^1.0.0
-- package: gopkg.in/yaml.v2

go.mod

@@ -0,0 +1,32 @@
+module github.com/aquasecurity/kube-bench
+
+go 1.13
+
+require (
+	github.com/denisenkom/go-mssqldb v0.0.0-20190515213511-eb9f6a1743f3 // indirect
+	github.com/erikstmartin/go-testdb v0.0.0-20160219214506-8d10e4a1bae5 // indirect
+	github.com/fatih/color v1.5.0
+	github.com/go-sql-driver/mysql v1.4.1 // indirect
+	github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
+	github.com/imdario/mergo v0.3.5 // indirect
+	github.com/jinzhu/gorm v0.0.0-20160404144928-5174cc5c242a
+	github.com/jinzhu/inflection v0.0.0-20170102125226-1c35d901db3d // indirect
+	github.com/jinzhu/now v1.0.1 // indirect
+	github.com/lib/pq v0.0.0-20171126050459-83612a56d3dd // indirect
+	github.com/mattn/go-colorable v0.0.0-20170210172801-5411d3eea597 // indirect
+	github.com/mattn/go-isatty v0.0.0-20170307163044-57fdcb988a5c // indirect
+	github.com/mattn/go-sqlite3 v1.10.0 // indirect
+	github.com/onsi/ginkgo v1.10.1
+	github.com/pkg/errors v0.8.1
+	github.com/spf13/cobra v0.0.3
+	github.com/spf13/viper v1.4.0
+	github.com/stretchr/testify v1.3.0
+	golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a // indirect
+	google.golang.org/appengine v1.5.0 // indirect
+	gopkg.in/yaml.v2 v2.2.4
+	k8s.io/api v0.0.0-20190409021203-6e4e0e4f393b
+	k8s.io/apimachinery v0.0.0-20190404173353-6a84e37a896d
+	k8s.io/client-go v11.0.0+incompatible
+	k8s.io/utils v0.0.0-20191114200735-6ca3b61696b6 // indirect
+	sigs.k8s.io/kind v0.5.1
+)

go.sum

@@ -0,0 +1,351 @@
+cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+cloud.google.com/go v0.37.4 h1:glPeL3BQJsbF6aIIYfZizMwc5LTYz250bDMjttbBGAU=
+cloud.google.com/go v0.37.4/go.mod h1:NHPJ89PdicEuT9hdPXMROBD91xc5uRDxsMtSB16k7hw=
+github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
+github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
+github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ=
+github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
+github.com/PuerkitoBio/purell v1.0.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
+github.com/PuerkitoBio/purell v1.1.1 h1:WEQqlqaGbrPkxLJWfBwQmfEAE1Z7ONdDLqrN38tNFfI=
+github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
+github.com/PuerkitoBio/urlesc v0.0.0-20160726150825-5bd2802263f2/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE=
+github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 h1:d+Bc7a5rLufV/sSk/8dngufqelfh6jnri85riMAaF/M=
+github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE=
+github.com/Shopify/sarama v1.19.0/go.mod h1:FVkBWblsNy7DGZRfXLU0O9RCGt5g3g3yEuWXgklEdEo=
+github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI=
+github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
+github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
+github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
+github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
+github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
+github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
+github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
+github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
+github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
+github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
+github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
+github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
+github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
+github.com/davecgh/go-spew v0.0.0-20151105211317-5215b55f46b2/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/denisenkom/go-mssqldb v0.0.0-20190515213511-eb9f6a1743f3 h1:tkum0XDgfR0jcVVXuTsYv/erY2NnEDqwRojbxR1rBYA=
+github.com/denisenkom/go-mssqldb v0.0.0-20190515213511-eb9f6a1743f3/go.mod h1:zAg7JM8CkOJ43xKXIj7eRO9kmWm/TW578qo+oDO6tuM=
+github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
+github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
+github.com/eapache/go-resiliency v1.1.0/go.mod h1:kFI+JgMyC7bLPUVY133qvEBtVayf5mFgVsvEsIPBvNs=
+github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21/go.mod h1:+020luEh2TKB4/GOp8oxxtq0Daoen/Cii55CzbTV6DU=
+github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I=
+github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs=
+github.com/emicklei/go-restful v2.9.6+incompatible h1:tfrHha8zJ01ywiOEC1miGY8st1/igzWB8OmvPgoYX7w=
+github.com/emicklei/go-restful v2.9.6+incompatible/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs=
+github.com/erikstmartin/go-testdb v0.0.0-20160219214506-8d10e4a1bae5 h1:Yzb9+7DPaBjB8zlTR87/ElzFsnQfuHnVUVqpZZIcV5Y=
+github.com/erikstmartin/go-testdb v0.0.0-20160219214506-8d10e4a1bae5/go.mod h1:a2zkGnVExMxdzMo3M0Hi/3sEU+cWnZpSni0O6/Yb/P0=
+github.com/evanphx/json-patch v4.5.0+incompatible h1:ouOWdg56aJriqS0huScTkVXPC5IcNrDCXZ6OoTAWu7M=
+github.com/evanphx/json-patch v4.5.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
+github.com/fatih/color v1.5.0 h1:vBh+kQp8lg9XPr56u1CPrWjFXtdphMoGWVHr9/1c+A0=
+github.com/fatih/color v1.5.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
+github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
+github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
+github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
+github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
+github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
+github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
+github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
+github.com/go-openapi/jsonpointer v0.0.0-20160704185906-46af16f9f7b1/go.mod h1:+35s3my2LFTysnkMfxsJBAMHj/DoqoB9knIWoYG/Vk0=
+github.com/go-openapi/jsonpointer v0.19.2 h1:A9+F4Dc/MCNB5jibxf6rRvOvR/iFgQdyNx9eIhnGqq0=
+github.com/go-openapi/jsonpointer v0.19.2/go.mod h1:3akKfEdA7DF1sugOqz1dVQHBcuDBPKZGEoHC/NkiQRg=
+github.com/go-openapi/jsonreference v0.0.0-20160704190145-13c6e3589ad9/go.mod h1:W3Z9FmVs9qj+KR4zFKmDPGiLdk1D9Rlm7cyMvf57TTg=
+github.com/go-openapi/jsonreference v0.19.2 h1:o20suLFB4Ri0tuzpWtyHlh7E7HnkqTNLq6aR6WVNS1w=
+github.com/go-openapi/jsonreference v0.19.2/go.mod h1:jMjeRr2HHw6nAVajTXJ4eiUwohSTlpa0o73RUL1owJc=
+github.com/go-openapi/spec v0.0.0-20160808142527-6aced65f8501/go.mod h1:J8+jY1nAiCcj+friV/PDoE1/3eeccG9LYBs0tYvLOWc=
+github.com/go-openapi/spec v0.19.2 h1:SStNd1jRcYtfKCN7R0laGNs80WYYvn5CbBjM2sOmCrE=
+github.com/go-openapi/spec v0.19.2/go.mod h1:sCxk3jxKgioEJikev4fgkNmwS+3kuYdJtcsZsD5zxMY=
+github.com/go-openapi/swag v0.0.0-20160704191624-1d0bd113de87/go.mod h1:DXUve3Dpr1UfpPtxFw+EFuQ41HhCWZfha5jSVRG7C7I=
+github.com/go-openapi/swag v0.19.2 h1:jvO6bCMBEilGwMfHhrd61zIID4oIFdwb76V17SM88dE=
+github.com/go-openapi/swag v0.19.2/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
+github.com/go-sql-driver/mysql v1.4.1 h1:g24URVg0OFbNUTx9qqY1IRZ9D9z3iPyi5zKhQZpNwpA=
+github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
+github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
+github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
+github.com/gogo/protobuf v1.2.0/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
+github.com/gogo/protobuf v1.2.1 h1:/s5zKNz0uPFCZ5hddgPdo2TK2TVrUNMn0OOX8/aZMTE=
+github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
+github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=
+github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
+github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
+github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
+github.com/golang/protobuf v0.0.0-20161109072736-4bd1920723d7/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/protobuf v1.3.1 h1:YF8+flBXS5eO826T4nzqPrxfhQThhXl0YzfuUPu4SBg=
+github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
+github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
+github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
+github.com/google/go-cmp v0.2.0 h1:+dTQ8DZQJz0Mb/HjFlkptS1FeQ4cWSnN941F8aEG4SQ=
+github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
+github.com/google/gofuzz v0.0.0-20161122191042-44d81051d367/go.mod h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI=
+github.com/google/gofuzz v1.0.0 h1:A8PeW59pxE9IoFRqBp37U+mSNaQoZ46F1f0f863XSXw=
+github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
+github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
+github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
+github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
+github.com/googleapis/gnostic v0.0.0-20170426233943-68f4ded48ba9/go.mod h1:sJBsCZ4ayReDTBIg8b9dl28c5xFWyhBTVRp3pOg5EKY=
+github.com/googleapis/gnostic v0.3.0 h1:CcQijm0XKekKjP/YCz28LXVSpgguuB+nCxaSjCe09y0=
+github.com/googleapis/gnostic v0.3.0/go.mod h1:sJBsCZ4ayReDTBIg8b9dl28c5xFWyhBTVRp3pOg5EKY=
+github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
+github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
+github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
+github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
+github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=
+github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
+github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
+github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
+github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
+github.com/hpcloud/tail v1.0.0 h1:nfCOvKYfkgYP8hkirhJocXT2+zOD8yUNjXaWfTlyFKI=
+github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
+github.com/imdario/mergo v0.3.5 h1:JboBksRwiiAJWvIYJVo46AfV+IAIKZpfrSzVKj42R4Q=
+github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
+github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
+github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
+github.com/jinzhu/gorm v0.0.0-20160404144928-5174cc5c242a h1:pfPxlCVlKqBRqHpyCxOIKhhB4ERpz02iadDpRVevLm4=
+github.com/jinzhu/gorm v0.0.0-20160404144928-5174cc5c242a/go.mod h1:Vla75njaFJ8clLU1W44h34PjIkijhjHIYnZxMqCdxqo=
+github.com/jinzhu/inflection v0.0.0-20170102125226-1c35d901db3d h1:jRQLvyVGL+iVtDElaEIDdKwpPqUIZJfzkNLV34htpEc=
+github.com/jinzhu/inflection v0.0.0-20170102125226-1c35d901db3d/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc=
+github.com/jinzhu/now v1.0.1 h1:HjfetcXq097iXP0uoPCdnM4Efp5/9MsM0/M+XOTeR3M=
+github.com/jinzhu/now v1.0.1/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
+github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
+github.com/json-iterator/go v0.0.0-20180612202835-f2b4162afba3/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
+github.com/json-iterator/go v1.1.6 h1:MrUvLMLTMxbqFJ9kzlvat/rYZqZnW3u4wkLzWTaFwKs=
+github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
+github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
+github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
+github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
+github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
+github.com/konsorten/go-windows-terminal-sequences v1.0.1 h1:mweAR1A6xJ3oS2pRaGiHgQ4OO8tzTaLawm8vnODuwDk=
+github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
+github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
+github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
+github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
+github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
+github.com/kr/pty v1.1.5/go.mod h1:9r2w37qlBe7rQ6e1fg1S/9xpWHSnaqNdHD3WcMdbPDA=
+github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
+github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
+github.com/lib/pq v0.0.0-20171126050459-83612a56d3dd h1:2RDaVc4/izhWyAvYxNm8c9saSyCDIxefNwOcqaH7pcU=
+github.com/lib/pq v0.0.0-20171126050459-83612a56d3dd/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
+github.com/magiconair/properties v1.8.0 h1:LLgXmsheXeRoUOBOjtwPQCWIYqM/LU1ayDtDePerRcY=
+github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
+github.com/mailru/easyjson v0.0.0-20160728113105-d5b7844b561a/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
+github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
+github.com/mailru/easyjson v0.0.0-20190620125010-da37f6c1e481 h1:IaSjLMT6WvkoZZjspGxy3rdaTEmWLoRm49WbtVUi9sA=
+github.com/mailru/easyjson v0.0.0-20190620125010-da37f6c1e481/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
+github.com/mattn/go-colorable v0.0.0-20170210172801-5411d3eea597 h1:hGizH4aMDFFt1iOA4HNKC13lqIBoCyxIjWcAnWIy7aU=
+github.com/mattn/go-colorable v0.0.0-20170210172801-5411d3eea597/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
+github.com/mattn/go-isatty v0.0.0-20170307163044-57fdcb988a5c h1:AHfQR/s6GNi92TOh+kfGworqDvTxj2rMsS+Hca87nck=
+github.com/mattn/go-isatty v0.0.0-20170307163044-57fdcb988a5c/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
+github.com/mattn/go-sqlite3 v1.10.0 h1:jbhqpg7tQe4SupckyijYiy0mJJ/pRyHvXf7JdWK860o=
+github.com/mattn/go-sqlite3 v1.10.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
+github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
+github.com/mitchellh/mapstructure v1.1.2 h1:fmNYVwqnSfB9mZU6OS2O6GsXM+wcskZDuKQzvN1EDeE=
+github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
+github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
+github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
+github.com/modern-go/reflect2 v0.0.0-20180320133207-05fbef0ca5da/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
+github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9AWI=
+github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
+github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
+github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
+github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
+github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
+github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
+github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
+github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
+github.com/onsi/ginkgo v1.10.1 h1:q/mM8GF/n0shIN8SaAZ0V+jnLPzen6WIVZdiwrRlMlo=
+github.com/onsi/ginkgo v1.10.1/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
+github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
+github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
+github.com/onsi/gomega v1.5.0 h1:izbySO9zDPmjJ8rDjLvkA2zJHIo+HkYXHnf7eN7SSyo=
+github.com/onsi/gomega v1.5.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
+github.com/openzipkin/zipkin-go v0.1.6/go.mod h1:QgAqvLzwWbR/WpD4A3cGpPtJrZXNIiJc5AZX7/PBEpw=
+github.com/pelletier/go-toml v1.2.0 h1:T5zMGML61Wp+FlcbWjRDT7yAxhJNAiPPLOFECq181zc=
+github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
+github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
+github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
+github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pmezard/go-difflib v0.0.0-20151028094244-d8ed2627bdf0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
+github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs=
+github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso=
+github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
+github.com/prometheus/client_model v0.0.0-20190115171406-56726106282f/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
+github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
+github.com/prometheus/common v0.2.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
+github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
+github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
+github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
+github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
+github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
+github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
+github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
+github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
+github.com/sirupsen/logrus v1.4.1 h1:GL2rEmy6nsikmW0r8opw9JIRScdMF5hA8cOYLH7In1k=
+github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
+github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
+github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
+github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
+github.com/spf13/afero v1.2.2 h1:5jhuqJyZCZf2JRofRvN/nIFgIWNzPa3/Vz8mYylgbWc=
+github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk=
+github.com/spf13/cast v1.3.0 h1:oget//CVOEoFewqQxwr0Ej5yjygnqGkvggSE/gB35Q8=
+github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
+github.com/spf13/cobra v0.0.2/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
+github.com/spf13/cobra v0.0.3 h1:ZlrZ4XsMRm04Fr5pSFxBgfND2EBVa1nLpiy1stUsX/8=
+github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
+github.com/spf13/jwalterweatherman v1.0.0 h1:XHEdyB+EcvlqZamSM4ZOMGlc93t6AcsBEu9Gc1vn7yk=
+github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
+github.com/spf13/pflag v0.0.0-20170130214245-9ff6c6923cff/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
+github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg=
+github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
+github.com/spf13/viper v1.4.0 h1:yXHLWeravcrgGyFSyCgdYpXQ9dR9c/WED3pg1RhxqEU=
+github.com/spf13/viper v1.4.0/go.mod h1:PTJ7Z/lr49W6bUbkmS1V3by4uWynFiR9p7+dSq/yZzE=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.2.0 h1:Hbg2NidpLE8veEBkEZTL3CvlkUIVzuU9jDplZO54c48=
+github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
+github.com/stretchr/testify v0.0.0-20151208002404-e3a8ff8ce365/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
+github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
+github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
+github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
+github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
+github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
+github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
+github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
+go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
+go.opencensus.io v0.20.1/go.mod h1:6WKK9ahsWS3RSO+PY9ZHZUfv2irvY6gN279GOPZjmmk=
+go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
+go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
+go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
+golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20190325154230-a5d413f7728c/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8 h1:1wopBVtVdWnn03fZelqdXTqk7U7zPQCb+T4rbU9ZEoU=
+golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
+golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
+golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
+golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
+golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
+golang.org/x/net v0.0.0-20170114055629-f2499483f923/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190125091013-d26f9f9a57f3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
+golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859 h1:R/3boaszxrf1GEUWTVDzSKVwLmSJpwZ1yqXm8j0v2QI=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
+golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
+golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a h1:tImsplftrFpALCYumobsd0K86vlAs/eXGFms2txfJfA=
+golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
+golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20170830134202-bb24a47a89ea/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181122145206-62eef0e2fa9b/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190621203818-d432491b9138 h1:t8BZD9RDjkm9/h7yYN6kE8oaeov5r9aztkB7zKA5Tkg=
+golang.org/x/sys v0.0.0-20190621203818-d432491b9138/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/text v0.0.0-20160726164857-2910a502d2bf/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
+golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
+golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/time v0.0.0-20190308202827-9d24e82272b4 h1:SvFZT6jyqRaOeXpc5h/JSfZenJ2O330aBsf7JfSUXmQ=
+golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20181011042414-1f849cf54d09/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
+golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/tools v0.0.0-20190614205625-5aca471b1d59/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
+google.golang.org/api v0.3.1/go.mod h1:6wY9I6uQWHQ8EM57III9mq/AjF+i8G65rmVagqKMtkk=
+google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
+google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
+google.golang.org/appengine v1.5.0 h1:KxkO13IPW4Lslp2bz+KHP2E3gtFlrIGNThxkZQ3g+4c=
+google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
+google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
+google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
+google.golang.org/genproto v0.0.0-20190404172233-64821d5d2107/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
+google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
+google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
+google.golang.org/grpc v1.21.0/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
+gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
+gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/fsnotify.v1 v1.4.7 h1:xOHLXZwVvI9hhs+cLKq5+I5onOuwQLhQwiu63xxlHs4=
+gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
+gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
+gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
+gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
+gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
+gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
+gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
+gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.2.4 h1:/eiJrUcujPVeJ3xlSWaiNi3uSVmDGBK1pDHUHAnao1I=
+gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+honnef.co/go/tools v0.0.0-20180728063816-88497007e858/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+k8s.io/api v0.0.0-20190313235455-40a48860b5ab/go.mod h1:iuAfoD4hCxJ8Onx9kaTIt30j7jUFS00AXQi6QMi99vA=
+k8s.io/api v0.0.0-20190409021203-6e4e0e4f393b h1:aBGgKJUM9Hk/3AE8WaZIApnTxG35kbuQba2w+SXqezo=
+k8s.io/api v0.0.0-20190409021203-6e4e0e4f393b/go.mod h1:iuAfoD4hCxJ8Onx9kaTIt30j7jUFS00AXQi6QMi99vA=
+k8s.io/apimachinery v0.0.0-20190313205120-d7deff9243b1/go.mod h1:ccL7Eh7zubPUSh9A3USN90/OzHNSVN6zxzde07TDCL0=
+k8s.io/apimachinery v0.0.0-20190404173353-6a84e37a896d h1:Jmdtdt1ZnoGfWWIIik61Z7nKYgO3J+swQJtPYsP9wHA=
+k8s.io/apimachinery v0.0.0-20190404173353-6a84e37a896d/go.mod h1:ccL7Eh7zubPUSh9A3USN90/OzHNSVN6zxzde07TDCL0=
+k8s.io/client-go v11.0.0+incompatible h1:LBbX2+lOwY9flffWlJM7f1Ct8V2SRNiMRDFeiwnJo9o=
+k8s.io/client-go v11.0.0+incompatible/go.mod h1:7vJpHMYJwNQCWgzmNV+VYUl1zCObLyodBc8nIyt8L5s=
+k8s.io/gengo v0.0.0-20190128074634-0689ccc1d7d6/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
+k8s.io/klog v0.0.0-20181102134211-b9b56d5dfc92/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
+k8s.io/klog v0.3.0/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
+k8s.io/klog v0.3.3 h1:niceAagH1tzskmaie/icWd7ci1wbG7Bf2c6YGcQv+3c=
+k8s.io/klog v0.3.3/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
+k8s.io/kube-openapi v0.0.0-20190603182131-db7b694dc208 h1:5sW+fEHvlJI3Ngolx30CmubFulwH28DhKjGf70Xmtco=
+k8s.io/kube-openapi v0.0.0-20190603182131-db7b694dc208/go.mod h1:nfDlWeOsu3pUf4yWGL+ERqohP4YsZcBJXWMK+gkzOA4=
+k8s.io/utils v0.0.0-20191114200735-6ca3b61696b6 h1:p0Ai3qVtkbCG/Af26dBmU0E1W58NID3hSSh7cMyylpM=
+k8s.io/utils v0.0.0-20191114200735-6ca3b61696b6/go.mod h1:sZAwmy6armz5eXlNoLmJcl4F1QuKu7sr+mFQ0byX7Ew=
+sigs.k8s.io/kind v0.5.1 h1:BYnHEJ9DC+0Yjlyyehqd3xnKtEmFdLKU8QxqOqvQzdw=
+sigs.k8s.io/kind v0.5.1/go.mod h1:L+Kcoo83/D1+ryU5P2VFbvYm0oqbkJn9zTZq0KNxW68=
+sigs.k8s.io/kustomize/v3 v3.1.1-0.20190821175718-4b67a6de1296 h1:iQaIG5Dq+3qSiaFrJ/l/0MjjxKmdwyVNpKRYJwUe/+0=
+sigs.k8s.io/kustomize/v3 v3.1.1-0.20190821175718-4b67a6de1296/go.mod h1:ztX4zYc/QIww3gSripwF7TBOarBTm5BvyAMem0kCzOE=
+sigs.k8s.io/structured-merge-diff v0.0.0-20190525122527-15d366b2352e/go.mod h1:wWxsB5ozmmv/SG7nM11ayaAW51xMvak/t1r0CSlcokI=
+sigs.k8s.io/yaml v1.1.0 h1:4A07+ZFc2wgJwo8YNlQpr1rVlgUDlxXHhPJciaPY5gs=
+sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o=

hack/debug.yaml

@@ -0,0 +1,50 @@
+---
+# use this pod with: kubectl run ubuntu -it --pid=host  -- /bin/bash
+# this allows you to debug what is running on the host.
+apiVersion: v1
+kind: Pod
+metadata:
+  name: ubuntu
+spec:
+  hostPID: true
+  containers:
+    - name: ubuntu
+      image: ubuntu
+      command: ["/bin/bash", "-c", "--"]
+      args: ["while true; do sleep 30; done;"]
+      env:
+        - name: PATH
+          value: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/mount-from-host/bin"
+      volumeMounts:
+        - name: var-lib-kubelet
+          mountPath: /var/lib/kubelet
+        - name: etc-systemd
+          mountPath: /etc/systemd
+        - name: etc-kubernetes
+          mountPath: /etc/kubernetes
+          # /usr/local/mount-from-host/bin is mounted to access kubectl / kubelet, for auto-detecting the Kubernetes version.
+          # You can omit this mount if you specify --version as part of the command.
+        - name: usr-bin
+          mountPath: /usr/local/mount-from-host/bin
+        - name: kind-bin
+          mountPath: /kind/bin
+      resources:
+        limits:
+          memory: "128Mi"
+          cpu: "500m"
+  volumes:
+    - name: var-lib-kubelet
+      hostPath:
+        path: "/var/lib/kubelet"
+    - name: etc-systemd
+      hostPath:
+        path: "/etc/systemd"
+    - name: etc-kubernetes
+      hostPath:
+        path: "/etc/kubernetes"
+    - name: usr-bin
+      hostPath:
+        path: "/usr/bin"
+    - name: kind-bin
+      hostPath:
+        path: "/kind/bin"

hack/kind.yaml

@@ -0,0 +1,51 @@
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: kube-bench
+spec:
+  template:
+    metadata:
+      labels:
+        app: kube-bench
+    spec:
+      hostPID: true
+      containers:
+        - name: kube-bench
+          image: aquasec/kube-bench:${VERSION}
+          command: ["kube-bench"]
+          volumeMounts:
+            - name: var-lib-etcd
+              mountPath: /var/lib/etcd
+            - name: var-lib-kubelet
+              mountPath: /var/lib/kubelet
+            - name: etc-systemd
+              mountPath: /etc/systemd
+            - name: etc-kubernetes
+              mountPath: /etc/kubernetes
+              # /usr/local/mount-from-host/bin is mounted to access kubectl / kubelet, for auto-detecting the Kubernetes version.
+              # You can omit this mount if you specify --version as part of the command.
+            - name: usr-bin
+              mountPath: /usr/local/mount-from-host/bin
+            - name: kind-bin
+              mountPath: /kind/bin
+      restartPolicy: Never
+      volumes:
+        - name: var-lib-etcd
+          hostPath:
+            path: "/var/lib/etcd"
+        - name: var-lib-kubelet
+          hostPath:
+            path: "/var/lib/kubelet"
+        - name: etc-systemd
+          hostPath:
+            path: "/etc/systemd"
+        - name: etc-kubernetes
+          hostPath:
+            path: "/etc/kubernetes"
+        - name: usr-bin
+          hostPath:
+            path: "/usr/bin"
+        - name: kind-bin
+          hostPath:
+            path: "/kind/bin"

hack/node_only.yaml

@@ -0,0 +1,73 @@
+---
+node:
+  components:
+    - kubelet
+    - proxy
+    # kubernetes is a component to cover the config file /etc/kubernetes/config that is referred to in the benchmark
+    - kubernetes
+
+  kubernetes:
+    defaultconf: "/etc/kubernetes/config"
+
+  kubelet:
+    cafile:
+      - "/etc/kubernetes/pki/ca.crt"
+      - "/etc/kubernetes/certs/ca.crt"
+      - "/etc/kubernetes/cert/ca.pem"
+    svc:
+      # These paths must also be included
+      #  in the 'confs' property below
+      - "/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"
+      - "/etc/systemd/system/kubelet.service"
+      - "/lib/systemd/system/kubelet.service"
+    bins:
+      - "hyperkube kubelet"
+      - "kubelet"
+    kubeconfig:
+      - "/etc/kubernetes/kubelet.conf"
+      - "/var/lib/kubelet/kubeconfig"
+      - "/etc/kubernetes/kubelet-kubeconfig"
+    confs:
+      - "/var/lib/kubelet/config.yaml"
+      - "/var/lib/kubelet/config.yml"
+      - "/etc/kubernetes/kubelet/kubelet-config.json"
+      - "/home/kubernetes/kubelet-config.yaml"
+      - "/home/kubernetes/kubelet-config.yml"
+      - "/etc/default/kubelet"
+      ## Due to the fact that the kubelet might be configured
+      ## without a kubelet-config file, we use a work-around
+      ## of pointing to the systemd service file (which can also
+      ## hold kubelet configuration).
+      ## Note: The following paths must match the one under 'svc'
+      - "/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"
+      - "/etc/systemd/system/kubelet.service"
+      - "/lib/systemd/system/kubelet.service"
+    defaultconf: "/var/lib/kubelet/config.yaml"
+    defaultsvc: "/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"
+    defaultkubeconfig: "/etc/kubernetes/kubelet.conf"
+    defaultcafile: "/etc/kubernetes/pki/ca.crt"
+
+  proxy:
+    bins:
+      - "kube-proxy"
+      - "hyperkube proxy"
+      - "hyperkube kube-proxy"
+      - "proxy"
+    confs:
+      - /etc/kubernetes/proxy
+      - /etc/kubernetes/addons/kube-proxy-daemonset.yaml
+    kubeconfig:
+      - /etc/kubernetes/kubelet-kubeconfig
+    svc:
+      - "/lib/systemd/system/kube-proxy.service"
+    defaultconf: /etc/kubernetes/addons/kube-proxy-daemonset.yaml
+    defaultkubeconfig: "/etc/kubernetes/proxy.conf"
+
+version_mapping:
+  "1.15": "cis-1.5"
+  "1.16": "cis-1.6"
+  "1.17": "cis-1.6"
+  "1.18": "cis-1.6"
+  "1.19": "cis-1.6"
+  "ocp-3.10": "rh-0.7"
+  "ocp-3.11": "rh-0.7"

hooks/build

@@ -3,4 +3,5 @@
 # $IMAGE_NAME var is injected into the build so the tag is correct. 
 docker build --build-arg BUILD_DATE=`date -u +"%Y-%m-%dT%H:%M:%SZ"` \
              --build-arg VCS_REF=`git rev-parse --short HEAD` \
+             --build-arg KUBEBENCH_VERSION=`git describe --tags --abbrev=0` \
              -t $IMAGE_NAME .

images/kube-bench.svg

@@ -0,0 +1,121 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<svg
+   xmlns:dc="http://purl.org/dc/elements/1.1/"
+   xmlns:cc="http://creativecommons.org/ns#"
+   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+   xmlns:svg="http://www.w3.org/2000/svg"
+   xmlns="http://www.w3.org/2000/svg"
+   viewBox="0 0 831.49597 755.90533"
+   height="755.90533"
+   width="831.49597"
+   xml:space="preserve"
+   id="svg2"
+   version="1.1"><metadata
+     id="metadata8"><rdf:RDF><cc:Work
+         rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type
+           rdf:resource="http://purl.org/dc/dcmitype/StillImage" /></cc:Work></rdf:RDF></metadata><defs
+     id="defs6"><clipPath
+       id="clipPath22"
+       clipPathUnits="userSpaceOnUse"><path
+         id="path20"
+         d="M 0,566.929 H 623.622 V 0 H 0 Z" /></clipPath></defs><g
+     transform="matrix(1.3333333,0,0,-1.3333333,0,755.90533)"
+     id="g10"><g
+       transform="translate(314.8111,521.959)"
+       id="g12"><path
+         id="path14"
+         style="fill:#0ab1d5;fill-opacity:1;fill-rule:nonzero;stroke:none"
+         d="M 0,0 -106.784,-145.31 0,-280.384 105.477,-147.025 Z" /></g><g
+       id="g16"><g
+         clip-path="url(#clipPath22)"
+         id="g18"><g
+           transform="translate(51.8912,72.061)"
+           id="g24"><path
+             id="path26"
+             style="fill:#464648;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 v 71.061 c 0,3.629 2.86,6.6 6.6,6.6 3.74,0 6.6,-2.971 6.6,-6.6 V 32.45 h 2.97 c 1.32,0 2.42,0.551 3.52,1.981 L 33.44,52.69 c 1.43,1.981 3.081,3.3 5.72,3.3 3.63,0 6.271,-2.969 6.271,-6.599 0,-1.87 -0.881,-3.411 -1.981,-4.731 L 29.59,27.5 44.44,3.96 C 45.32,2.641 45.76,1.21 45.76,0 c 0,-3.63 -2.97,-6.6 -6.6,-6.6 -2.309,0 -4.4,1.54 -5.5,3.411 L 19.8,19.25 c -0.88,1.431 -1.98,2.091 -3.52,2.091 H 13.2 L 13.2,0 C 13.2,-3.63 10.34,-6.6 6.6,-6.6 2.86,-6.6 0,-3.63 0,0" /></g><g
+           transform="translate(104.9547,86.8013)"
+           id="g28"><path
+             id="path30"
+             style="fill:#464648;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 v 34.65 c 0,3.63 2.97,6.6 6.6,6.6 3.629,0 6.6,-2.97 6.6,-6.6 V 2.86 c 0,-8.47 3.409,-11.44 9.57,-11.44 4.73,0 9.24,2.86 11.33,4.95 v 38.28 c 0,3.63 2.97,6.6 6.6,6.6 3.63,0 6.6,-2.97 6.6,-6.6 v -50.16 c 0,-3.3 -2.53,-5.83 -5.72,-5.83 -2.97,0 -5.06,2.09 -5.72,4.95 l -0.55,2.42 C 32.12,-17.16 26.18,-21.34 18.149,-21.34 5.06,-21.34 0,-11.99 0,0" /></g><g
+           transform="translate(197.5084,90.4312)"
+           id="g32"><path
+             id="path34"
+             style="fill:#464648;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 v 12.65 c 0,8.47 -2.971,12.54 -10.341,12.54 -4.069,0 -8.029,-2.2 -10.559,-4.839 V -7.59 c 2.53,-2.639 6.49,-4.95 10.559,-4.95 C -2.971,-12.54 0,-8.47 0,0 m -34.101,-19.14 v 71.83 c 0,3.63 2.861,6.601 6.6,6.601 3.74,0 6.601,-2.971 6.601,-6.601 V 31.57 c 3.08,3.191 8.359,6.05 14.299,6.05 13.09,0 19.8,-8.8 19.8,-23.54 V -1.319 c 0,-14.741 -6.819,-23.651 -20.13,-23.651 -6.16,0 -11.88,2.97 -14.96,6.491 l -0.66,-2.201 c -0.769,-2.53 -3.08,-4.29 -5.72,-4.29 -3.299,0 -5.83,2.75 -5.83,5.83" /></g><g
+           transform="translate(251.7047,102.311)"
+           id="g36"><path
+             id="path38"
+             style="fill:#464648;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 c 0,9.57 -1.87,14.301 -9.9,14.301 -7.92,0 -9.9,-4.181 -9.9,-14.301 z M -33,-15.069 V 2.2 c 0,14.521 7.479,23.54 23.1,23.54 15.95,0 22.77,-8.689 22.77,-23.54 v -7.37 c 0,-2.859 -2.309,-5.17 -5.17,-5.17 h -27.5 v -5.939 c 0,-4.62 2.86,-9.13 10.89,-9.13 5.72,0 8.8,0.88 13.09,2.97 0.66,0.33 1.54,0.66 2.42,0.66 2.97,0 5.39,-2.42 5.39,-5.391 0,-2.309 -1.429,-3.96 -3.52,-5.17 -5.17,-2.97 -10.23,-4.51 -17.93,-4.51 -15.73,0 -23.54,8.25 -23.54,21.781" /></g><g
+           transform="translate(271.7564,99.4517)"
+           id="g40"><path
+             id="path42"
+             style="fill:#464648;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 c 0,3.3 2.53,5.83 5.721,5.83 h 19.91 c 3.3,0 5.83,-2.53 5.83,-5.83 0,-3.19 -2.53,-5.72 -5.83,-5.72 H 5.721 C 2.53,-5.72 0,-3.19 0,0" /></g><g
+           transform="translate(345.776,90.4312)"
+           id="g44"><path
+             id="path46"
+             style="fill:#464648;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 v 12.65 c 0,8.47 -2.971,12.54 -10.341,12.54 -4.069,0 -8.029,-2.2 -10.559,-4.839 V -7.59 c 2.53,-2.639 6.49,-4.95 10.559,-4.95 C -2.971,-12.54 0,-8.47 0,0 m -34.101,-19.14 v 71.83 c 0,3.63 2.861,6.601 6.6,6.601 3.74,0 6.601,-2.971 6.601,-6.601 V 31.57 c 3.08,3.191 8.359,6.05 14.299,6.05 13.09,0 19.8,-8.8 19.8,-23.54 V -1.319 c 0,-14.741 -6.819,-23.651 -20.13,-23.651 -6.16,0 -11.88,2.97 -14.96,6.491 l -0.66,-2.201 c -0.769,-2.53 -3.08,-4.29 -5.72,-4.29 -3.299,0 -5.83,2.75 -5.83,5.83" /></g><g
+           transform="translate(399.9723,102.311)"
+           id="g48"><path
+             id="path50"
+             style="fill:#464648;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 c 0,9.57 -1.87,14.301 -9.9,14.301 -7.92,0 -9.9,-4.181 -9.9,-14.301 z M -33,-15.069 V 2.2 c 0,14.521 7.479,23.54 23.1,23.54 15.95,0 22.77,-8.689 22.77,-23.54 v -7.37 c 0,-2.859 -2.309,-5.17 -5.17,-5.17 h -27.5 v -5.939 c 0,-4.62 2.86,-9.13 10.89,-9.13 5.72,0 8.8,0.88 13.09,2.97 0.66,0.33 1.54,0.66 2.42,0.66 2.97,0 5.39,-2.42 5.39,-5.391 0,-2.309 -1.429,-3.96 -3.52,-5.17 -5.17,-2.97 -10.23,-4.51 -17.93,-4.51 -15.73,0 -23.54,8.25 -23.54,21.781" /></g><g
+           transform="translate(421.8512,72.061)"
+           id="g52"><path
+             id="path54"
+             style="fill:#464648;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 v 50.16 c 0,3.301 2.53,5.83 5.72,5.83 2.97,0 5.06,-2.09 5.72,-4.949 l 0.55,-2.421 c 3.19,3.191 9.13,7.37 17.16,7.37 13.09,0 18.15,-9.349 18.15,-21.34 V 0 c 0,-3.63 -2.97,-6.6 -6.6,-6.6 -3.63,0 -6.599,2.97 -6.599,6.6 v 31.79 c 0,8.471 -3.411,11.44 -9.571,11.44 -4.73,0 -9.24,-2.86 -11.33,-4.95 L 13.2,0 C 13.2,-3.63 10.23,-6.6 6.6,-6.6 2.97,-6.6 0,-3.63 0,0" /></g><g
+           transform="translate(478.358,89.1118)"
+           id="g56"><path
+             id="path58"
+             style="fill:#464648;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 v 15.29 c 0,14.52 8.36,23.649 24.31,23.649 8.36,0 14.08,-3.08 18.15,-8.029 1.21,-1.54 1.87,-2.75 1.87,-4.511 0,-3.299 -2.53,-5.83 -5.83,-5.83 -1.76,0 -3.08,0.66 -4.4,1.981 -2.75,2.75 -5.39,4.62 -9.79,4.62 -8.69,0 -11.11,-5.83 -11.11,-12.981 L 13.2,1.1 c 0,-7.151 2.75,-12.981 11.44,-12.981 4.4,0 7.04,1.87 9.79,4.62 1.32,1.321 2.31,1.981 4.29,1.981 3.3,0 5.94,-2.531 5.94,-5.83 0,-1.76 -0.66,-2.97 -1.87,-4.51 C 38.72,-20.57 33,-23.65 24.64,-23.65 8.689,-23.65 0,-14.521 0,0" /></g><g
+           transform="translate(530.5396,72.061)"
+           id="g60"><path
+             id="path62"
+             style="fill:#464648;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 v 71.061 c 0,3.629 2.86,6.6 6.6,6.6 3.74,0 6.6,-2.971 6.6,-6.6 v -21.34 c 3.41,2.969 9.02,6.269 16.17,6.269 13.09,0 18.26,-9.349 18.26,-21.34 V 0 c 0,-3.63 -2.859,-6.6 -6.6,-6.6 -3.74,0 -6.6,2.97 -6.6,6.6 v 31.79 c 0,8.471 -3.52,11.44 -9.68,11.44 -4.729,0 -9.46,-2.86 -11.55,-4.95 V 0 C 13.2,-3.63 10.34,-6.6 6.6,-6.6 2.86,-6.6 0,-3.63 0,0" /></g><g
+           transform="translate(249.2096,192.0259)"
+           id="g64"><path
+             id="path66"
+             style="fill:#f1df36;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 h 0.008 l 131.211,0.031 h 0.013 c 3.063,0 6.107,0.66 8.916,1.863 L 65.602,49.549 -8.531,1.7 C -5.83,0.6 -2.923,0 0,0" /></g><g
+           transform="translate(420.2877,374.9341)"
+           id="g68"><path
+             id="path70"
+             style="fill:#faaf42;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 -105.477,-133.359 74.547,-47.655 c 3.392,1.452 6.439,3.697 8.747,6.559 l 75.104,93.431 6.686,8.317 c 1.38,1.714 2.479,3.637 3.289,5.675 0.384,0.965 0.701,1.954 0.95,2.962 z" /></g><g
+           transform="translate(145.3785,311.2251)"
+           id="g72"><path
+             id="path74"
+             style="fill:#faaf42;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 c 0.583,-2.568 1.609,-5.036 3.054,-7.245 0.401,-0.614 0.83,-1.209 1.285,-1.783 l 81.823,-101.735 c 2.396,-2.975 5.588,-5.289 9.138,-6.736 L 169.433,-69.65 62.648,65.424 Z" /></g><g
+           transform="translate(179.4977,457.7324)"
+           id="g76"><path
+             id="path78"
+             style="fill:#9ad7ec;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 c -2.408,-2.762 -4.144,-6.1 -4.985,-9.762 l -29.149,-126.8 c -0.65,-2.826 -0.715,-5.774 -0.239,-8.633 0.073,-0.44 0.155,-0.878 0.254,-1.312 l 62.648,65.424 z" /></g><g
+           transform="translate(484.1334,310.8643)"
+           id="g80"><path
+             id="path82"
+             style="fill:#9ad7ec;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="M 0,0 C 0.837,3.378 0.913,6.943 0.131,10.337 L -29.076,137.21 c -0.791,3.437 -2.374,6.586 -4.566,9.236 L -63.846,64.07 Z" /></g><g
+           transform="translate(317.7506,366.4487)"
+           id="g84"><path
+             id="path86"
+             style="fill:#ffffff;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="M 0,0 40.622,41.329 H 14.038 L -18.791,6.272 V 77.598 H -39.47 V -56.101 h 20.679 v 40.069 l 3.269,3.181 33.46,-43.25 h 27.03 z" /></g><g
+           transform="translate(275.7818,468.8486)"
+           id="g88"><path
+             id="path90"
+             style="fill:#1280c4;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 39.028,53.109 c -0.01,0 -0.022,10e-4 -0.033,10e-4 -0.047,0 -0.094,-0.003 -0.141,-0.003 C 38.521,53.105 38.187,53.099 37.853,53.082 37.814,53.08 37.776,53.072 37.738,53.07 34.783,52.909 31.86,52.166 29.192,50.889 L -89.022,-5.593 c -2.809,-1.342 -5.266,-3.235 -7.262,-5.523 L -67.755,-92.199 0,0.03 Z" /></g><g
+           transform="translate(442.8853,463.2578)"
+           id="g92"><path
+             id="path94"
+             style="fill:#1280c4;fill-opacity:1;fill-rule:nonzero;stroke:none"
+             d="m 0,0 -118.288,56.48 c -3.039,1.455 -6.412,2.215 -9.785,2.22 L -22.598,-88.324 7.606,-5.947 C 5.558,-3.467 2.978,-1.422 0,0" /></g></g></g></g></svg>

integration/docker.go

@@ -0,0 +1,61 @@
+package integration
+
+import (
+	"os"
+	"path/filepath"
+
+	"github.com/pkg/errors"
+
+	"sigs.k8s.io/kind/pkg/cluster"
+	clusternodes "sigs.k8s.io/kind/pkg/cluster/nodes"
+	"sigs.k8s.io/kind/pkg/container/docker"
+	"sigs.k8s.io/kind/pkg/fs"
+	"sigs.k8s.io/kind/pkg/util/concurrent"
+)
+
+func loadImageFromDocker(imageName string, kindCtx *cluster.Context) error {
+
+	// Check that the image exists locally and gets its ID, if not return error
+	_, err := docker.ImageID(imageName)
+	if err != nil {
+		return errors.Errorf("Image: %q not present locally", imageName)
+	}
+
+	selectedNodes, err := kindCtx.ListInternalNodes()
+	if err != nil {
+		return err
+	}
+
+	// Save the image into a tar
+	dir, err := fs.TempDir("", "image-tar")
+	if err != nil {
+		return errors.Wrap(err, "failed to create tempdir")
+	}
+	defer os.RemoveAll(dir)
+	imageTarPath := filepath.Join(dir, "image.tar")
+
+	err = docker.Save(imageName, imageTarPath)
+	if err != nil {
+		return err
+	}
+
+	// Load the image on the selected nodes
+	fns := []func() error{}
+	for _, selectedNode := range selectedNodes {
+		selectedNode := selectedNode // capture loop variable
+		fns = append(fns, func() error {
+			return loadImage(imageTarPath, &selectedNode)
+		})
+	}
+	return concurrent.UntilError(fns)
+}
+
+// loads an image tarball onto a node
+func loadImage(imageTarName string, node *clusternodes.Node) error {
+	f, err := os.Open(imageTarName)
+	if err != nil {
+		return errors.Wrap(err, "failed to open image")
+	}
+	defer f.Close()
+	return node.LoadImageArchive(f)
+}

integration/integration.go

@@ -0,0 +1,144 @@
+package integration
+
+import (
+	"bytes"
+	"fmt"
+	"io"
+	"io/ioutil"
+	"strings"
+	"time"
+
+	batchv1 "k8s.io/api/batch/v1"
+	apiv1 "k8s.io/api/core/v1"
+	corev1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	yaml "k8s.io/apimachinery/pkg/util/yaml"
+	"k8s.io/client-go/kubernetes"
+	"k8s.io/client-go/tools/clientcmd"
+	"sigs.k8s.io/kind/pkg/cluster"
+	"sigs.k8s.io/kind/pkg/cluster/create"
+)
+
+func runWithKind(ctx *cluster.Context, clientset *kubernetes.Clientset, jobName, kubebenchYAML, kubebenchImg string, timeout time.Duration) (string, error) {
+	err := deployJob(clientset, kubebenchYAML, kubebenchImg)
+	if err != nil {
+		return "", err
+	}
+
+	p, err := findPodForJob(clientset, jobName, timeout)
+	if err != nil {
+		return "", err
+	}
+
+	output := getPodLogs(clientset, p)
+
+	err = clientset.BatchV1().Jobs(apiv1.NamespaceDefault).Delete(jobName, nil)
+	if err != nil {
+		return "", err
+	}
+
+	return output, nil
+}
+
+func setupCluster(clusterName, kindCfg string, duration time.Duration) (*cluster.Context, error) {
+	options := create.WithConfigFile(kindCfg)
+	toptions := create.WaitForReady(duration)
+	ctx := cluster.NewContext(clusterName)
+	if err := ctx.Create(options, toptions); err != nil {
+		return nil, err
+	}
+
+	return ctx, nil
+}
+
+func getClientSet(configPath string) (*kubernetes.Clientset, error) {
+	config, err := clientcmd.BuildConfigFromFlags("", configPath)
+	if err != nil {
+		return nil, err
+	}
+	clientset, err := kubernetes.NewForConfig(config)
+	if err != nil {
+		return nil, err
+	}
+
+	return clientset, nil
+}
+
+func deployJob(clientset *kubernetes.Clientset, kubebenchYAML, kubebenchImg string) error {
+	jobYAML, err := ioutil.ReadFile(kubebenchYAML)
+	if err != nil {
+		return err
+	}
+
+	decoder := yaml.NewYAMLOrJSONDecoder(bytes.NewReader(jobYAML), len(jobYAML))
+	job := &batchv1.Job{}
+	if err := decoder.Decode(job); err != nil {
+		return err
+	}
+	job.Spec.Template.Spec.Containers[0].Image = kubebenchImg
+
+	_, err = clientset.BatchV1().Jobs(apiv1.NamespaceDefault).Create(job)
+
+	return err
+}
+
+func findPodForJob(clientset *kubernetes.Clientset, jobName string, duration time.Duration) (*apiv1.Pod, error) {
+	failedPods := make(map[string]struct{})
+	selector := fmt.Sprintf("job-name=%s", jobName)
+	timeout := time.After(duration)
+	for {
+		time.Sleep(3 * time.Second)
+	podfailed:
+		select {
+		case <-timeout:
+			return nil, fmt.Errorf("podList - timed out: no Pod found for Job %s", jobName)
+		default:
+			pods, err := clientset.CoreV1().Pods(apiv1.NamespaceDefault).List(metav1.ListOptions{
+				LabelSelector: selector,
+			})
+			if err != nil {
+				return nil, err
+			}
+			fmt.Printf("Found (%d) pods\n", len(pods.Items))
+			for _, cp := range pods.Items {
+				if _, found := failedPods[cp.Name]; found {
+					continue
+				}
+
+				if strings.HasPrefix(cp.Name, jobName) {
+					fmt.Printf("pod (%s) - %#v\n", cp.Name, cp.Status.Phase)
+					if cp.Status.Phase == apiv1.PodSucceeded {
+						return &cp, nil
+					}
+
+					if cp.Status.Phase == apiv1.PodFailed {
+						fmt.Printf("pod (%s) - %s - retrying...\n", cp.Name, cp.Status.Phase)
+						fmt.Print(getPodLogs(clientset, &cp))
+						failedPods[cp.Name] = struct{}{}
+						break podfailed
+					}
+				}
+			}
+		}
+	}
+
+	return nil, fmt.Errorf("no Pod found for Job %q", jobName)
+}
+
+func getPodLogs(clientset *kubernetes.Clientset, pod *apiv1.Pod) string {
+	podLogOpts := corev1.PodLogOptions{}
+	req := clientset.CoreV1().Pods(pod.Namespace).GetLogs(pod.Name, &podLogOpts)
+	podLogs, err := req.Stream()
+	if err != nil {
+		return "getPodLogs - error in opening stream"
+	}
+	defer podLogs.Close()
+
+	buf := new(bytes.Buffer)
+	_, err = io.Copy(buf, podLogs)
+	if err != nil {
+		return "getPodLogs - error in copy information from podLogs to buf"
+	}
+
+	return buf.String()
+}

integration/integration_test.go

@@ -0,0 +1,150 @@
+// +build integration
+
+package integration
+
+import (
+	"bufio"
+	"bytes"
+	"flag"
+	"fmt"
+	"io/ioutil"
+	"strings"
+	"testing"
+	"time"
+)
+
+var kubebenchImg = flag.String("kubebenchImg", "aquasec/kube-bench:latest", "kube-bench image used as part of this test")
+var timeout = flag.Duration("timeout", 10*time.Minute, "Test Timeout")
+
+func testCheckCISWithKind(t *testing.T, testdataDir string) {
+	flag.Parse()
+	fmt.Printf("kube-bench Container Image: %s\n", *kubebenchImg)
+
+	cases := []struct {
+		TestName      string
+		KubebenchYAML string
+		ExpectedFile  string
+		ExpectError   bool
+	}{
+		{
+			TestName:      "kube-bench",
+			KubebenchYAML: "../job.yaml",
+			ExpectedFile:  fmt.Sprintf("./testdata/%s/job.data", testdataDir),
+		},
+		{
+			TestName:      "kube-bench-node",
+			KubebenchYAML: "../job-node.yaml",
+			ExpectedFile:  fmt.Sprintf("./testdata/%s/job-node.data", testdataDir),
+		},
+		{
+			TestName:      "kube-bench-master",
+			KubebenchYAML: "../job-master.yaml",
+			ExpectedFile:  fmt.Sprintf("./testdata/%s/job-master.data", testdataDir),
+		},
+	}
+	ctx, err := setupCluster("kube-bench", fmt.Sprintf("./testdata/%s/add-tls-kind.yaml", testdataDir), *timeout)
+	if err != nil {
+		t.Fatalf("failed to setup KIND cluster error: %v", err)
+	}
+	defer func() {
+		ctx.Delete()
+	}()
+
+	if err := loadImageFromDocker(*kubebenchImg, ctx); err != nil {
+		t.Fatalf("failed to load kube-bench image from Docker to KIND error: %v", err)
+	}
+
+	clientset, err := getClientSet(ctx.KubeConfigPath())
+	if err != nil {
+		t.Fatalf("failed to connect to Kubernetes cluster error: %v", err)
+	}
+
+	for _, c := range cases {
+		t.Run(c.TestName, func(t *testing.T) {
+			resultData, err := runWithKind(ctx, clientset, c.TestName, c.KubebenchYAML, *kubebenchImg, *timeout)
+			if err != nil {
+				t.Errorf("unexpected error: %v", err)
+			}
+
+			c, err := ioutil.ReadFile(c.ExpectedFile)
+			if err != nil {
+				t.Error(err)
+			}
+
+			expectedData := strings.TrimSpace(string(c))
+			resultData = strings.TrimSpace(resultData)
+			if expectedData != resultData {
+				t.Errorf("expected results\n\nExpected\t(<)\nResult\t(>)\n\n%s\n\n", generateDiff(expectedData, resultData))
+			}
+		})
+	}
+}
+
+func TestCheckCIS15WithKind(t *testing.T) {
+	testCheckCISWithKind(t, "cis-1.5")
+}
+
+func TestCheckCIS16WithKind(t *testing.T) {
+	testCheckCISWithKind(t, "cis-1.6")
+}
+
+// This is simple "diff" between 2 strings containing multiple lines.
+// It's not a comprehensive diff between the 2 strings.
+// It does not inditcate when lines are deleted.
+func generateDiff(source, target string) string {
+	buf := new(bytes.Buffer)
+	ss := bufio.NewScanner(strings.NewReader(source))
+	ts := bufio.NewScanner(strings.NewReader(target))
+
+	emptySource := false
+	emptyTarget := false
+
+loop:
+	for ln := 1; ; ln++ {
+		var ll, rl string
+
+		sourceScan := ss.Scan()
+		if sourceScan {
+			ll = ss.Text()
+		}
+
+		targetScan := ts.Scan()
+		if targetScan {
+			rl = ts.Text()
+		}
+
+		switch {
+		case !sourceScan && !targetScan:
+			// no more lines
+			break loop
+		case sourceScan && targetScan:
+			if ll != rl {
+				fmt.Fprintf(buf, "line: %d\n", ln)
+				fmt.Fprintf(buf, "< %s\n", ll)
+				fmt.Fprintf(buf, "> %s\n", rl)
+			}
+		case !targetScan:
+			if !emptyTarget {
+				fmt.Fprintf(buf, "line: %d\n", ln)
+			}
+			fmt.Fprintf(buf, "< %s\n", ll)
+			emptyTarget = true
+		case !sourceScan:
+			if !emptySource {
+				fmt.Fprintf(buf, "line: %d\n", ln)
+			}
+			fmt.Fprintf(buf, "> %s\n", rl)
+			emptySource = true
+		}
+	}
+
+	if emptySource {
+		fmt.Fprintf(buf, "< [[NO MORE DATA]]")
+	}
+
+	if emptyTarget {
+		fmt.Fprintf(buf, "> [[NO MORE DATA]]")
+	}
+
+	return buf.String()
+}

integration/testdata/cis-1.3/add-tls-kind.yaml

@@ -0,0 +1,19 @@
+---
+apiVersion: kind.sigs.k8s.io/v1alpha3
+kind: Cluster
+networking:
+  apiServerAddress: "0.0.0.0"
+
+kubeadmConfigPatchesJson6902:
+  - group: kubelet.config.k8s.io
+    version: v1beta1
+    kind: KubeletConfiguration
+    patch: |
+      - op: add
+        path: /tlsCipherSuites
+        value: ["TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"]
+
+nodes:
+  # the control plane node config
+  - role: control-plane
+    image: "kindest/node:v1.12.10"

integration/testdata/cis-1.3/job-master.data

@@ -0,0 +1,336 @@
+[INFO] 1 Master Node Security Configuration
+[INFO] 1.1 API Server
+[FAIL] 1.1.1 Ensure that the --anonymous-auth argument is set to false (Scored)
+[PASS] 1.1.2 Ensure that the --basic-auth-file argument is not set (Scored)
+[PASS] 1.1.3 Ensure that the --insecure-allow-any-token argument is not set (Scored)
+[PASS] 1.1.4 Ensure that the --kubelet-https argument is set to true (Scored)
+[PASS] 1.1.5 Ensure that the --insecure-bind-address argument is not set (Scored)
+[PASS] 1.1.6 Ensure that the --insecure-port argument is set to 0 (Scored)
+[PASS] 1.1.7 Ensure that the --secure-port argument is not set to 0 (Scored)
+[FAIL] 1.1.8 Ensure that the --profiling argument is set to false (Scored)
+[FAIL] 1.1.9 Ensure that the --repair-malformed-updates argument is set to false (Scored)
+[PASS] 1.1.10 Ensure that the admission control plugin AlwaysAdmit is not set (Scored)
+[FAIL] 1.1.11 Ensure that the admission control plugin AlwaysPullImages is set (Scored)
+[FAIL] 1.1.12 Ensure that the admission control plugin DenyEscalatingExec is set (Scored)
+[FAIL] 1.1.13 Ensure that the admission control plugin SecurityContextDeny is set (Scored)
+[PASS] 1.1.14 Ensure that the admission control plugin NamespaceLifecycle is set (Scored)
+[FAIL] 1.1.15 Ensure that the --audit-log-path argument is set as appropriate (Scored)
+[FAIL] 1.1.16 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Scored)
+[FAIL] 1.1.17 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Scored)
+[FAIL] 1.1.18 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Scored)
+[PASS] 1.1.19 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 1.1.20 Ensure that the --token-auth-file parameter is not set (Scored)
+[FAIL] 1.1.21 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored)
+[PASS] 1.1.22 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Scored)
+[FAIL] 1.1.23 Ensure that the --service-account-lookup argument is set to true (Scored)
+[FAIL] 1.1.24 Ensure that the admission control plugin PodSecurityPolicy is set (Scored)
+[PASS] 1.1.25 Ensure that the --service-account-key-file argument is set as appropriate (Scored)
+[PASS] 1.1.26 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Scored)
+[FAIL] 1.1.27 Ensure that the admission control plugin ServiceAccount is set(Scored)
+[PASS] 1.1.28 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[PASS] 1.1.29 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[WARN] 1.1.30 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Not Scored)
+[PASS] 1.1.31 Ensure that the --etcd-cafile argument is set as appropriate (Scored)
+[PASS] 1.1.32 Ensure that the --authorization-mode argument is set to Node (Scored)
+[PASS] 1.1.33 Ensure that the admission control plugin NodeRestriction is set (Scored)
+[FAIL] 1.1.34 Ensure that the --experimental-encryption-provider-config argument is set as appropriate (Scored)
+[WARN] 1.1.35 Ensure that the encryption provider is set to aescbc (Scored)
+[FAIL] 1.1.36 Ensure that the admission control plugin EventRateLimit is set (Scored)
+[PASS] 1.1.37a Ensure that the AdvancedAuditing argument is not set to false (Scored)
+[FAIL] 1.1.37b Ensure that the AdvancedAuditing argument is not set to false (Scored)
+[PASS] 1.1.38 Ensure that the --request-timeout argument is set as appropriate (Scored)
+[WARN] 1.1.39 Ensure that the API Server only makes use of Strong Cryptographic Ciphers ( Not Scored)
+[INFO] 1.2 Scheduler
+[FAIL] 1.2.1 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.2.2 Ensure that the --address argument is set to 127.0.0.1 (Scored)
+[INFO] 1.3 Controller Manager
+[FAIL] 1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Scored)
+[FAIL] 1.3.2 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Scored)
+[PASS] 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Scored)
+[PASS] 1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Scored)
+[FAIL] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 1.3.7 Ensure that the --address argument is set to 127.0.0.1 (Scored)
+[INFO] 1.4 Configuration Files
+[PASS] 1.4.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.2 Ensure that the API server pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.3 Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.4 Ensure that the controller manager pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.5 Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.6 Ensure that the scheduler pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.7 Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.8 Ensure that the etcd pod specification file ownership is set to root:root (Scored)
+[WARN] 1.4.9 Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Not Scored)
+[WARN] 1.4.10 Ensure that the Container Network Interface file ownership is set to root:root (Not Scored)
+[FAIL] 1.4.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Scored)
+[FAIL] 1.4.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)
+[PASS] 1.4.13 Ensure that the admin.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.14 Ensure that the admin.conf file ownership is set to root:root (Scored)
+[PASS] 1.4.15 Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.16 Ensure that the scheduler.conf file ownership is set to root:root (Scored)
+[PASS] 1.4.17 Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.18 Ensure that the controller-manager.conf file ownership is set to root:root (Scored)
+[INFO] 1.5 etcd
+[PASS] 1.5.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)
+[PASS] 1.5.2 Ensure that the --client-cert-auth argument is set to true (Scored)
+[PASS] 1.5.3 Ensure that the --auto-tls argument is not set to true (Scored)
+[PASS] 1.5.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Scored)
+[PASS] 1.5.5 Ensure that the --peer-client-cert-auth argument is set to true (Scored)
+[PASS] 1.5.6 Ensure that the --peer-auto-tls argument is not set to true (Scored)
+[WARN] 1.5.7 Ensure that a unique Certificate Authority is used for etcd (Not Scored)
+[INFO] 1.6 General Security Primitives
+[WARN] 1.6.1 Ensure that the cluster-admin role is only used where required (Not Scored)
+[WARN] 1.6.2 Create administrative boundaries between resources using namespaces (Not Scored)
+[WARN] 1.6.3 Create network segmentation using Network Policies (Not Scored)
+[WARN] 1.6.4 Ensure that the seccomp profile is set to docker/default in your pod definitions (Not Scored)
+[WARN] 1.6.5 Apply Security Context to Your Pods and Containers (Not Scored)
+[WARN] 1.6.6 Configure Image Provenance using ImagePolicyWebhook admission controller (Not Scored)
+[WARN] 1.6.7 Configure Network policies as appropriate (Not Scored)
+[WARN] 1.6.8 Place compensating controls in the form of PSP and RBAC for privileged containers usage (Not Scored)
+[INFO] 1.7 PodSecurityPolicies
+[WARN] 1.7.1 Do not admit privileged containers (Not Scored)
+[WARN] 1.7.2 Do not admit containers wishing to share the host process ID namespace (Scored)
+[WARN] 1.7.3 Do not admit containers wishing to share the host IPC namespace (Scored)
+[WARN] 1.7.4 Do not admit containers wishing to share the host network namespace (Scored)
+[WARN] 1.7.5 Do not admit containers with allowPrivilegeEscalation (Scored)
+[WARN] 1.7.6 Do not admit root containers (Not Scored)
+[WARN] 1.7.7 Do not admit containers with dangerous capabilities (Not Scored)
+
+== Remediations ==
+1.1.1 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--anonymous-auth=false
+
+1.1.8 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.1.9 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--repair-malformed-updates=false
+
+1.1.11 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins to
+include AlwaysPullImages.
+--enable-admission-plugins=...,AlwaysPullImages,...
+
+1.1.12 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to a
+value that includes DenyEscalatingExec.
+--enable-admission-plugins=...,DenyEscalatingExec,...
+
+1.1.13 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to
+include SecurityContextDeny.
+--enable-admission-plugins=...,SecurityContextDeny,...
+
+1.1.15 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-path parameter to a suitable
+path and file where you would like audit logs to be written, for example:
+--audit-log-path=/var/log/apiserver/audit.log
+
+1.1.16 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxage parameter to 30 or
+as an appropriate number of days: --audit-log-maxage=30
+
+1.1.17 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxbackup parameter to 10
+or to an appropriate value.
+--audit-log-maxbackup=10
+
+1.1.18 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxsize parameter to an
+appropriate size in MB. For example, to set it as 100 MB:
+--audit-log-maxsize=100
+
+1.1.21 Follow the Kubernetes documentation and setup the TLS connection between the
+apiserver and kubelets. Then, edit the API server pod specification file
+/etc/kubernetes/manifests/kube-apiserver.yaml on the master node and set the --kubelet-certificate-authority
+parameter to the path to the cert file for the certificate authority.
+--kubelet-certificate-authority=<ca-string>
+
+1.1.23 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--service-account-lookup=true
+
+1.1.24 Follow the documentation and create Pod Security Policy objects as per your environment.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to a
+value that includes PodSecurityPolicy :
+--enable-admission-plugins=...,PodSecurityPolicy,...
+Then restart the API Server.
+
+1.1.27 Follow the documentation and create ServiceAccount objects as per your environment.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to a
+value that includes ServiceAccount.
+--enable-admission-plugins=...,ServiceAccount,...
+
+1.1.30 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+
+1.1.34 Follow the Kubernetes documentation and configure a EncryptionConfig file.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml on the
+master node and set the --experimental-encryption-provider-config parameter
+to the path of that file:
+--experimental-encryption-provider-config=</path/to/EncryptionConfig/File>
+
+1.1.35 [Manual test]
+Follow the Kubernetes documentation and configure a EncryptionConfig file. In this file,
+choose aescbc as the encryption provider.
+For example,
+kind: EncryptionConfig
+apiVersion: v1
+resources:
+  - resources:
+    - secrets
+      providers:
+      - aescbc:
+          keys:
+          - name: key1
+            secret: <32-byte base64-encoded secret>
+
+1.1.36 Follow the Kubernetes documentation and set the desired limits in a
+configuration file. Then, edit the API server pod specification file
+/etc/kubernetes/manifests/kube-apiserver.yaml and set the below parameters.
+--enable-admission-plugins=...,EventRateLimit,...
+--admission-control-config-file=<path/to/configuration/file>
+
+1.1.37b Follow the Kubernetes documentation and set the desired audit policy in the
+/etc/kubernetes/audit-policy.yaml file. Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+and set the below parameters.
+--audit-policy-file=/etc/kubernetes/audit-policy.yaml
+
+1.1.39 Edit the API server pod specification file /etc/kubernetes/manifests
+kube-apiserver.yaml on the master node and set the below parameter.
+--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+
+1.2.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.yaml
+file on the master node and set the below parameter.
+--profiling=false
+
+1.3.1 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the --terminated-pod-gc-threshold to an appropriate threshold, for example:
+--terminated-pod-gc-threshold=10
+
+1.3.2 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.3.6 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+controller-manager.yaml on the master node and set the --feature-gates parameter to
+include RotateKubeletServerCertificate=true.
+--feature-gates=RotateKubeletServerCertificate=true
+
+1.4.9 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example,
+chmod 644 <path/to/cni/files>
+
+1.4.10 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example,
+chown root:root <path/to/cni/files>
+
+1.4.11 On the etcd server node, get the etcd data directory, passed as an argument --data-dir ,
+from the below command:
+ps -ef | grep etcd
+Run the below command (based on the etcd data directory found above). For example,
+chmod 700 /var/lib/etcd
+
+1.4.12 On the etcd server node, get the etcd data directory, passed as an argument --data-dir ,
+from the below command:
+ps -ef | grep etcd
+Run the below command (based on the etcd data directory found above). For example,
+chown etcd:etcd /var/lib/etcd
+
+1.5.7 [Manual test]
+Follow the etcd documentation and create a dedicated certificate authority setup for the
+etcd service.
+Then, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml on the
+master node and set the below parameter.
+--trusted-ca-file=</path/to/ca-file>
+
+1.6.1 [Manual test]
+Remove any unneeded clusterrolebindings :
+kubectl delete clusterrolebinding [name]
+
+1.6.2 [Manual test]
+Follow the documentation and create namespaces for objects in your deployment as you
+need them.
+
+1.6.3 [Manual test]
+Follow the documentation and create NetworkPolicy objects as you need them.
+
+1.6.4 [Manual test]
+Seccomp is an alpha feature currently. By default, all alpha features are disabled. So, you
+would need to enable alpha features in the apiserver by passing "--feature-
+gates=AllAlpha=true" argument.
+Edit the /etc/kubernetes/manifests/kube-apiserver.yaml file on the master node and set the KUBE_API_ARGS
+parameter to "--feature-gates=AllAlpha=true"
+KUBE_API_ARGS="--feature-gates=AllAlpha=true"
+Based on your system, restart the kube-apiserver service. For example:
+systemctl restart kube-apiserver.service
+Use annotations to enable the docker/default seccomp profile in your pod definitions. An
+example is as below:
+apiVersion: v1
+kind: Pod
+metadata:
+  name: trustworthy-pod
+  annotations:
+    seccomp.security.alpha.kubernetes.io/pod: docker/default
+spec:
+  containers:
+    - name: trustworthy-container
+      image: sotrustworthy:latest
+
+1.6.5 [Manual test]
+Follow the Kubernetes documentation and apply security contexts to your pods. For a
+suggested list of security contexts, you may refer to the CIS Security Benchmark for Docker
+Containers.
+
+1.6.6 [Manual test]
+Follow the Kubernetes documentation and setup image provenance.
+
+1.6.7 [Manual test]
+Follow the Kubernetes documentation and setup network policies as appropriate.
+For example, you could create a "default" isolation policy for a Namespace by creating a
+NetworkPolicy that selects all pods but does not allow any traffic:
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: default-deny
+spec:
+  podSelector:
+
+1.6.8 [Manual test]
+Follow Kubernetes documentation and setup PSP and RBAC authorization for your cluster.
+
+1.7.1 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.privileged field is omitted or set to false.
+
+1.7.2 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostPID field is omitted or set to false.
+
+1.7.3 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostIPC field is omitted or set to false.
+
+1.7.4 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostNetwork field is omitted or set to false.
+
+1.7.5 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.allowPrivilegeEscalation field is omitted or set to false.
+
+1.7.6 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.runAsUser.rule is set to either MustRunAsNonRoot or MustRunAs with the range of UIDs not including 0.
+
+1.7.7 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.requiredDropCapabilities is set to include either NET_RAW or ALL.
+
+
+== Summary ==
+45 checks PASS
+23 checks FAIL
+21 checks WARN
+0 checks INFO

integration/testdata/cis-1.3/job-node.data

@@ -0,0 +1,104 @@
+[INFO] 2 Worker Node Security Configuration
+[INFO] 2.1 Kubelet
+[FAIL] 2.1.1 Ensure that the --allow-privileged argument is set to false (Scored)
+[PASS] 2.1.2 Ensure that the --anonymous-auth argument is set to false (Scored)
+[PASS] 2.1.3 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 2.1.4 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[FAIL] 2.1.5 Ensure that the --read-only-port argument is set to 0 (Scored)
+[PASS] 2.1.6 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)
+[FAIL] 2.1.7 Ensure that the --protect-kernel-defaults argument is set to true (Scored)
+[PASS] 2.1.8 Ensure that the --make-iptables-util-chains argument is set to true (Scored)
+[PASS] 2.1.9 Ensure that the --hostname-override argument is not set (Scored)
+[FAIL] 2.1.10 Ensure that the --event-qps argument is set to 0 (Scored)
+[FAIL] 2.1.11 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[PASS] 2.1.12 Ensure that the --cadvisor-port argument is set to 0 (Scored)
+[PASS] 2.1.13 Ensure that the --rotate-certificates argument is not set to false (Scored)
+[FAIL] 2.1.14 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[WARN] 2.1.15 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)
+[INFO] 2.2 Configuration Files
+[PASS] 2.2.1 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.2 Ensure that the kubelet.conf file ownership is set to root:root (Scored)
+[PASS] 2.2.3 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.4 Ensure that the kubelet service file ownership is set to root:root (Scored)
+[FAIL] 2.2.5 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)
+[FAIL] 2.2.6 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)
+[WARN] 2.2.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.8 Ensure that the client certificate authorities file ownership is set to root:root (Scored)
+[PASS] 2.2.9 Ensure that the kubelet configuration file ownership is set to root:root (Scored)
+[PASS] 2.2.10 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)
+
+== Remediations ==
+2.1.1 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
+on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--allow-privileged=false
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.5 If using a Kubelet config file, edit the file to set readOnlyPort to 0 .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--read-only-port=0
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.7 If using a Kubelet config file, edit the file to set protectKernelDefaults: true .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--protect-kernel-defaults=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.10 If using a Kubelet config file, edit the file to set eventRecordQPS: 0 .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--event-qps=0
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.11 If using a Kubelet config file, edit the file to set tlsCertFile to the location of the certificate
+file to use to identify this Kubelet, and tlsPrivateKeyFile to the location of the
+corresponding private key file.
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameters in KUBELET_CERTIFICATE_ARGS variable.
+--tls-cert-file=<path/to/tls-certificate-file>
+file=<path/to/tls-key-file>
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.14 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
+on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
+--feature-gates=RotateKubeletServerCertificate=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.15 If using a Kubelet config file, edit the file to set TLSCipherSuites: to TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+If using executable arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter.
+--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+
+2.2.5 Run the below command (based on the file location on your system) on the each worker
+node. For example,
+chmod 644 /etc/kubernetes/proxy.conf
+
+2.2.6 Run the below command (based on the file location on your system) on the each worker
+node. For example,
+chown root:root /etc/kubernetes/proxy.conf
+
+2.2.7 Run the following command to modify the file permissions of the --client-ca-file
+chmod 644 <filename>
+
+
+== Summary ==
+15 checks PASS
+8 checks FAIL
+2 checks WARN
+0 checks INFO

integration/testdata/cis-1.3/job.data

@@ -0,0 +1,440 @@
+[INFO] 1 Master Node Security Configuration
+[INFO] 1.1 API Server
+[FAIL] 1.1.1 Ensure that the --anonymous-auth argument is set to false (Scored)
+[PASS] 1.1.2 Ensure that the --basic-auth-file argument is not set (Scored)
+[PASS] 1.1.3 Ensure that the --insecure-allow-any-token argument is not set (Scored)
+[PASS] 1.1.4 Ensure that the --kubelet-https argument is set to true (Scored)
+[PASS] 1.1.5 Ensure that the --insecure-bind-address argument is not set (Scored)
+[PASS] 1.1.6 Ensure that the --insecure-port argument is set to 0 (Scored)
+[PASS] 1.1.7 Ensure that the --secure-port argument is not set to 0 (Scored)
+[FAIL] 1.1.8 Ensure that the --profiling argument is set to false (Scored)
+[FAIL] 1.1.9 Ensure that the --repair-malformed-updates argument is set to false (Scored)
+[PASS] 1.1.10 Ensure that the admission control plugin AlwaysAdmit is not set (Scored)
+[FAIL] 1.1.11 Ensure that the admission control plugin AlwaysPullImages is set (Scored)
+[FAIL] 1.1.12 Ensure that the admission control plugin DenyEscalatingExec is set (Scored)
+[FAIL] 1.1.13 Ensure that the admission control plugin SecurityContextDeny is set (Scored)
+[PASS] 1.1.14 Ensure that the admission control plugin NamespaceLifecycle is set (Scored)
+[FAIL] 1.1.15 Ensure that the --audit-log-path argument is set as appropriate (Scored)
+[FAIL] 1.1.16 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Scored)
+[FAIL] 1.1.17 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Scored)
+[FAIL] 1.1.18 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Scored)
+[PASS] 1.1.19 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 1.1.20 Ensure that the --token-auth-file parameter is not set (Scored)
+[FAIL] 1.1.21 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored)
+[PASS] 1.1.22 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Scored)
+[FAIL] 1.1.23 Ensure that the --service-account-lookup argument is set to true (Scored)
+[FAIL] 1.1.24 Ensure that the admission control plugin PodSecurityPolicy is set (Scored)
+[PASS] 1.1.25 Ensure that the --service-account-key-file argument is set as appropriate (Scored)
+[PASS] 1.1.26 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Scored)
+[FAIL] 1.1.27 Ensure that the admission control plugin ServiceAccount is set(Scored)
+[PASS] 1.1.28 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[PASS] 1.1.29 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[WARN] 1.1.30 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Not Scored)
+[PASS] 1.1.31 Ensure that the --etcd-cafile argument is set as appropriate (Scored)
+[PASS] 1.1.32 Ensure that the --authorization-mode argument is set to Node (Scored)
+[PASS] 1.1.33 Ensure that the admission control plugin NodeRestriction is set (Scored)
+[FAIL] 1.1.34 Ensure that the --experimental-encryption-provider-config argument is set as appropriate (Scored)
+[WARN] 1.1.35 Ensure that the encryption provider is set to aescbc (Scored)
+[FAIL] 1.1.36 Ensure that the admission control plugin EventRateLimit is set (Scored)
+[PASS] 1.1.37a Ensure that the AdvancedAuditing argument is not set to false (Scored)
+[FAIL] 1.1.37b Ensure that the AdvancedAuditing argument is not set to false (Scored)
+[PASS] 1.1.38 Ensure that the --request-timeout argument is set as appropriate (Scored)
+[WARN] 1.1.39 Ensure that the API Server only makes use of Strong Cryptographic Ciphers ( Not Scored)
+[INFO] 1.2 Scheduler
+[FAIL] 1.2.1 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.2.2 Ensure that the --address argument is set to 127.0.0.1 (Scored)
+[INFO] 1.3 Controller Manager
+[FAIL] 1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Scored)
+[FAIL] 1.3.2 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Scored)
+[PASS] 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Scored)
+[PASS] 1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Scored)
+[FAIL] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 1.3.7 Ensure that the --address argument is set to 127.0.0.1 (Scored)
+[INFO] 1.4 Configuration Files
+[PASS] 1.4.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.2 Ensure that the API server pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.3 Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.4 Ensure that the controller manager pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.5 Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.6 Ensure that the scheduler pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.7 Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.8 Ensure that the etcd pod specification file ownership is set to root:root (Scored)
+[WARN] 1.4.9 Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Not Scored)
+[WARN] 1.4.10 Ensure that the Container Network Interface file ownership is set to root:root (Not Scored)
+[FAIL] 1.4.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Scored)
+[FAIL] 1.4.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)
+[PASS] 1.4.13 Ensure that the admin.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.14 Ensure that the admin.conf file ownership is set to root:root (Scored)
+[PASS] 1.4.15 Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.16 Ensure that the scheduler.conf file ownership is set to root:root (Scored)
+[PASS] 1.4.17 Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.18 Ensure that the controller-manager.conf file ownership is set to root:root (Scored)
+[INFO] 1.5 etcd
+[PASS] 1.5.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)
+[PASS] 1.5.2 Ensure that the --client-cert-auth argument is set to true (Scored)
+[PASS] 1.5.3 Ensure that the --auto-tls argument is not set to true (Scored)
+[PASS] 1.5.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Scored)
+[PASS] 1.5.5 Ensure that the --peer-client-cert-auth argument is set to true (Scored)
+[PASS] 1.5.6 Ensure that the --peer-auto-tls argument is not set to true (Scored)
+[WARN] 1.5.7 Ensure that a unique Certificate Authority is used for etcd (Not Scored)
+[INFO] 1.6 General Security Primitives
+[WARN] 1.6.1 Ensure that the cluster-admin role is only used where required (Not Scored)
+[WARN] 1.6.2 Create administrative boundaries between resources using namespaces (Not Scored)
+[WARN] 1.6.3 Create network segmentation using Network Policies (Not Scored)
+[WARN] 1.6.4 Ensure that the seccomp profile is set to docker/default in your pod definitions (Not Scored)
+[WARN] 1.6.5 Apply Security Context to Your Pods and Containers (Not Scored)
+[WARN] 1.6.6 Configure Image Provenance using ImagePolicyWebhook admission controller (Not Scored)
+[WARN] 1.6.7 Configure Network policies as appropriate (Not Scored)
+[WARN] 1.6.8 Place compensating controls in the form of PSP and RBAC for privileged containers usage (Not Scored)
+[INFO] 1.7 PodSecurityPolicies
+[WARN] 1.7.1 Do not admit privileged containers (Not Scored)
+[WARN] 1.7.2 Do not admit containers wishing to share the host process ID namespace (Scored)
+[WARN] 1.7.3 Do not admit containers wishing to share the host IPC namespace (Scored)
+[WARN] 1.7.4 Do not admit containers wishing to share the host network namespace (Scored)
+[WARN] 1.7.5 Do not admit containers with allowPrivilegeEscalation (Scored)
+[WARN] 1.7.6 Do not admit root containers (Not Scored)
+[WARN] 1.7.7 Do not admit containers with dangerous capabilities (Not Scored)
+
+== Remediations ==
+1.1.1 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--anonymous-auth=false
+
+1.1.8 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.1.9 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--repair-malformed-updates=false
+
+1.1.11 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins to
+include AlwaysPullImages.
+--enable-admission-plugins=...,AlwaysPullImages,...
+
+1.1.12 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to a
+value that includes DenyEscalatingExec.
+--enable-admission-plugins=...,DenyEscalatingExec,...
+
+1.1.13 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to
+include SecurityContextDeny.
+--enable-admission-plugins=...,SecurityContextDeny,...
+
+1.1.15 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-path parameter to a suitable
+path and file where you would like audit logs to be written, for example:
+--audit-log-path=/var/log/apiserver/audit.log
+
+1.1.16 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxage parameter to 30 or
+as an appropriate number of days: --audit-log-maxage=30
+
+1.1.17 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxbackup parameter to 10
+or to an appropriate value.
+--audit-log-maxbackup=10
+
+1.1.18 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxsize parameter to an
+appropriate size in MB. For example, to set it as 100 MB:
+--audit-log-maxsize=100
+
+1.1.21 Follow the Kubernetes documentation and setup the TLS connection between the
+apiserver and kubelets. Then, edit the API server pod specification file
+/etc/kubernetes/manifests/kube-apiserver.yaml on the master node and set the --kubelet-certificate-authority
+parameter to the path to the cert file for the certificate authority.
+--kubelet-certificate-authority=<ca-string>
+
+1.1.23 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--service-account-lookup=true
+
+1.1.24 Follow the documentation and create Pod Security Policy objects as per your environment.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to a
+value that includes PodSecurityPolicy :
+--enable-admission-plugins=...,PodSecurityPolicy,...
+Then restart the API Server.
+
+1.1.27 Follow the documentation and create ServiceAccount objects as per your environment.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to a
+value that includes ServiceAccount.
+--enable-admission-plugins=...,ServiceAccount,...
+
+1.1.30 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+
+1.1.34 Follow the Kubernetes documentation and configure a EncryptionConfig file.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml on the
+master node and set the --experimental-encryption-provider-config parameter
+to the path of that file:
+--experimental-encryption-provider-config=</path/to/EncryptionConfig/File>
+
+1.1.35 [Manual test]
+Follow the Kubernetes documentation and configure a EncryptionConfig file. In this file,
+choose aescbc as the encryption provider.
+For example,
+kind: EncryptionConfig
+apiVersion: v1
+resources:
+  - resources:
+    - secrets
+      providers:
+      - aescbc:
+          keys:
+          - name: key1
+            secret: <32-byte base64-encoded secret>
+
+1.1.36 Follow the Kubernetes documentation and set the desired limits in a
+configuration file. Then, edit the API server pod specification file
+/etc/kubernetes/manifests/kube-apiserver.yaml and set the below parameters.
+--enable-admission-plugins=...,EventRateLimit,...
+--admission-control-config-file=<path/to/configuration/file>
+
+1.1.37b Follow the Kubernetes documentation and set the desired audit policy in the
+/etc/kubernetes/audit-policy.yaml file. Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+and set the below parameters.
+--audit-policy-file=/etc/kubernetes/audit-policy.yaml
+
+1.1.39 Edit the API server pod specification file /etc/kubernetes/manifests
+kube-apiserver.yaml on the master node and set the below parameter.
+--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+
+1.2.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.yaml
+file on the master node and set the below parameter.
+--profiling=false
+
+1.3.1 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the --terminated-pod-gc-threshold to an appropriate threshold, for example:
+--terminated-pod-gc-threshold=10
+
+1.3.2 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.3.6 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+controller-manager.yaml on the master node and set the --feature-gates parameter to
+include RotateKubeletServerCertificate=true.
+--feature-gates=RotateKubeletServerCertificate=true
+
+1.4.9 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example,
+chmod 644 <path/to/cni/files>
+
+1.4.10 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example,
+chown root:root <path/to/cni/files>
+
+1.4.11 On the etcd server node, get the etcd data directory, passed as an argument --data-dir ,
+from the below command:
+ps -ef | grep etcd
+Run the below command (based on the etcd data directory found above). For example,
+chmod 700 /var/lib/etcd
+
+1.4.12 On the etcd server node, get the etcd data directory, passed as an argument --data-dir ,
+from the below command:
+ps -ef | grep etcd
+Run the below command (based on the etcd data directory found above). For example,
+chown etcd:etcd /var/lib/etcd
+
+1.5.7 [Manual test]
+Follow the etcd documentation and create a dedicated certificate authority setup for the
+etcd service.
+Then, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml on the
+master node and set the below parameter.
+--trusted-ca-file=</path/to/ca-file>
+
+1.6.1 [Manual test]
+Remove any unneeded clusterrolebindings :
+kubectl delete clusterrolebinding [name]
+
+1.6.2 [Manual test]
+Follow the documentation and create namespaces for objects in your deployment as you
+need them.
+
+1.6.3 [Manual test]
+Follow the documentation and create NetworkPolicy objects as you need them.
+
+1.6.4 [Manual test]
+Seccomp is an alpha feature currently. By default, all alpha features are disabled. So, you
+would need to enable alpha features in the apiserver by passing "--feature-
+gates=AllAlpha=true" argument.
+Edit the /etc/kubernetes/manifests/kube-apiserver.yaml file on the master node and set the KUBE_API_ARGS
+parameter to "--feature-gates=AllAlpha=true"
+KUBE_API_ARGS="--feature-gates=AllAlpha=true"
+Based on your system, restart the kube-apiserver service. For example:
+systemctl restart kube-apiserver.service
+Use annotations to enable the docker/default seccomp profile in your pod definitions. An
+example is as below:
+apiVersion: v1
+kind: Pod
+metadata:
+  name: trustworthy-pod
+  annotations:
+    seccomp.security.alpha.kubernetes.io/pod: docker/default
+spec:
+  containers:
+    - name: trustworthy-container
+      image: sotrustworthy:latest
+
+1.6.5 [Manual test]
+Follow the Kubernetes documentation and apply security contexts to your pods. For a
+suggested list of security contexts, you may refer to the CIS Security Benchmark for Docker
+Containers.
+
+1.6.6 [Manual test]
+Follow the Kubernetes documentation and setup image provenance.
+
+1.6.7 [Manual test]
+Follow the Kubernetes documentation and setup network policies as appropriate.
+For example, you could create a "default" isolation policy for a Namespace by creating a
+NetworkPolicy that selects all pods but does not allow any traffic:
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: default-deny
+spec:
+  podSelector:
+
+1.6.8 [Manual test]
+Follow Kubernetes documentation and setup PSP and RBAC authorization for your cluster.
+
+1.7.1 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.privileged field is omitted or set to false.
+
+1.7.2 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostPID field is omitted or set to false.
+
+1.7.3 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostIPC field is omitted or set to false.
+
+1.7.4 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostNetwork field is omitted or set to false.
+
+1.7.5 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.allowPrivilegeEscalation field is omitted or set to false.
+
+1.7.6 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.runAsUser.rule is set to either MustRunAsNonRoot or MustRunAs with the range of UIDs not including 0.
+
+1.7.7 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.requiredDropCapabilities is set to include either NET_RAW or ALL.
+
+
+== Summary ==
+45 checks PASS
+23 checks FAIL
+21 checks WARN
+0 checks INFO
+[INFO] 2 Worker Node Security Configuration
+[INFO] 2.1 Kubelet
+[FAIL] 2.1.1 Ensure that the --allow-privileged argument is set to false (Scored)
+[PASS] 2.1.2 Ensure that the --anonymous-auth argument is set to false (Scored)
+[PASS] 2.1.3 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 2.1.4 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[FAIL] 2.1.5 Ensure that the --read-only-port argument is set to 0 (Scored)
+[PASS] 2.1.6 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)
+[FAIL] 2.1.7 Ensure that the --protect-kernel-defaults argument is set to true (Scored)
+[PASS] 2.1.8 Ensure that the --make-iptables-util-chains argument is set to true (Scored)
+[PASS] 2.1.9 Ensure that the --hostname-override argument is not set (Scored)
+[FAIL] 2.1.10 Ensure that the --event-qps argument is set to 0 (Scored)
+[FAIL] 2.1.11 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[PASS] 2.1.12 Ensure that the --cadvisor-port argument is set to 0 (Scored)
+[PASS] 2.1.13 Ensure that the --rotate-certificates argument is not set to false (Scored)
+[FAIL] 2.1.14 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[WARN] 2.1.15 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)
+[INFO] 2.2 Configuration Files
+[PASS] 2.2.1 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.2 Ensure that the kubelet.conf file ownership is set to root:root (Scored)
+[PASS] 2.2.3 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.4 Ensure that the kubelet service file ownership is set to root:root (Scored)
+[FAIL] 2.2.5 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)
+[FAIL] 2.2.6 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)
+[WARN] 2.2.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.8 Ensure that the client certificate authorities file ownership is set to root:root (Scored)
+[PASS] 2.2.9 Ensure that the kubelet configuration file ownership is set to root:root (Scored)
+[PASS] 2.2.10 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)
+
+== Remediations ==
+2.1.1 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
+on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--allow-privileged=false
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.5 If using a Kubelet config file, edit the file to set readOnlyPort to 0 .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--read-only-port=0
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.7 If using a Kubelet config file, edit the file to set protectKernelDefaults: true .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--protect-kernel-defaults=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.10 If using a Kubelet config file, edit the file to set eventRecordQPS: 0 .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--event-qps=0
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.11 If using a Kubelet config file, edit the file to set tlsCertFile to the location of the certificate
+file to use to identify this Kubelet, and tlsPrivateKeyFile to the location of the
+corresponding private key file.
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameters in KUBELET_CERTIFICATE_ARGS variable.
+--tls-cert-file=<path/to/tls-certificate-file>
+file=<path/to/tls-key-file>
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.14 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
+on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
+--feature-gates=RotateKubeletServerCertificate=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.15 If using a Kubelet config file, edit the file to set TLSCipherSuites: to TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+If using executable arguments, edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and set the below parameter.
+--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+
+2.2.5 Run the below command (based on the file location on your system) on the each worker
+node. For example,
+chmod 644 /etc/kubernetes/proxy.conf
+
+2.2.6 Run the below command (based on the file location on your system) on the each worker
+node. For example,
+chown root:root /etc/kubernetes/proxy.conf
+
+2.2.7 Run the following command to modify the file permissions of the --client-ca-file
+chmod 644 <filename>
+
+
+== Summary ==
+15 checks PASS
+8 checks FAIL
+2 checks WARN
+0 checks INFO

integration/testdata/cis-1.4/add-tls-kind.yaml

@@ -0,0 +1,19 @@
+---
+apiVersion: kind.sigs.k8s.io/v1alpha3
+kind: Cluster
+networking:
+  apiServerAddress: "0.0.0.0"
+
+kubeadmConfigPatchesJson6902:
+  - group: kubelet.config.k8s.io
+    version: v1beta1
+    kind: KubeletConfiguration
+    patch: |
+      - op: add
+        path: /tlsCipherSuites
+        value: ["TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"]
+
+nodes:
+  # the control plane node config
+  - role: control-plane
+    image: "kindest/node:v1.14.6"

integration/testdata/cis-1.4/job-master.data

@@ -0,0 +1,327 @@
+[INFO] 1 Master Node Security Configuration
+[INFO] 1.1 API Server
+[WARN] 1.1.1 Ensure that the --anonymous-auth argument is set to false (Not Scored)
+[PASS] 1.1.2 Ensure that the --basic-auth-file argument is not set (Scored)
+[PASS] 1.1.3 Ensure that the --insecure-allow-any-token argument is not set (Not Scored)
+[PASS] 1.1.4 Ensure that the --kubelet-https argument is set to true (Scored)
+[PASS] 1.1.5 Ensure that the --insecure-bind-address argument is not set (Scored)
+[PASS] 1.1.6 Ensure that the --insecure-port argument is set to 0 (Scored)
+[PASS] 1.1.7 Ensure that the --secure-port argument is not set to 0 (Scored)
+[FAIL] 1.1.8 Ensure that the --profiling argument is set to false (Scored)
+[FAIL] 1.1.9 Ensure that the --repair-malformed-updates argument is set to false (Scored)
+[PASS] 1.1.10 Ensure that the admission control plugin AlwaysAdmit is not set (Scored)
+[FAIL] 1.1.11 Ensure that the admission control plugin AlwaysPullImages is set (Scored)
+[INFO] 1.1.12 [DEPRECATED] Ensure that the admission control plugin DenyEscalatingExec is set (Not Scored)
+[WARN] 1.1.13 Ensure that the admission control plugin SecurityContextDeny is set (Not Scored)
+[PASS] 1.1.14 Ensure that the admission control plugin NamespaceLifecycle is set (Scored)
+[FAIL] 1.1.15 Ensure that the --audit-log-path argument is set as appropriate (Scored)
+[FAIL] 1.1.16 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Scored)
+[FAIL] 1.1.17 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Scored)
+[FAIL] 1.1.18 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Scored)
+[PASS] 1.1.19 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 1.1.20 Ensure that the --token-auth-file parameter is not set (Scored)
+[FAIL] 1.1.21 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored)
+[PASS] 1.1.22 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Scored)
+[PASS] 1.1.23 Ensure that the --service-account-lookup argument is set to true (Scored)
+[FAIL] 1.1.24 Ensure that the admission control plugin PodSecurityPolicy is set (Scored)
+[PASS] 1.1.25 Ensure that the --service-account-key-file argument is set as appropriate (Scored)
+[PASS] 1.1.26 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Scored)
+[PASS] 1.1.27 Ensure that the admission control plugin ServiceAccount is set(Scored)
+[PASS] 1.1.28 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[PASS] 1.1.29 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[PASS] 1.1.30 Ensure that the --etcd-cafile argument is set as appropriate (Scored)
+[WARN] 1.1.31 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Not Scored)
+[PASS] 1.1.32 Ensure that the --authorization-mode argument is set to Node (Scored)
+[PASS] 1.1.33 Ensure that the admission control plugin NodeRestriction is set (Scored)
+[FAIL] 1.1.34 Ensure that the --encryption-provider-config argument is set as appropriate (Scored)
+[WARN] 1.1.35 Ensure that the encryption provider is set to aescbc (Scored)
+[FAIL] 1.1.36 Ensure that the admission control plugin EventRateLimit is set (Scored)
+[PASS] 1.1.37a Ensure that the AdvancedAuditing argument is not set to false (Scored)
+[FAIL] 1.1.37b Ensure that the AdvancedAuditing argument is not set to false (Scored)
+[PASS] 1.1.38 Ensure that the --request-timeout argument is set as appropriate (Scored)
+[PASS] 1.1.39 Ensure that the --authorization-mode argument includes RBAC (Scored)
+[INFO] 1.2 Scheduler
+[FAIL] 1.2.1 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.2.2 Ensure that the --address argument is set to 127.0.0.1 (Scored)
+[INFO] 1.3 Controller Manager
+[FAIL] 1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Scored)
+[FAIL] 1.3.2 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Scored)
+[PASS] 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Scored)
+[PASS] 1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Scored)
+[FAIL] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 1.3.7 Ensure that the --address argument is set to 127.0.0.1 (Scored)
+[INFO] 1.4 Configuration Files
+[PASS] 1.4.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.2 Ensure that the API server pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.3 Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.4 Ensure that the controller manager pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.5 Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.6 Ensure that the scheduler pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.7 Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.8 Ensure that the etcd pod specification file ownership is set to root:root (Scored)
+[WARN] 1.4.9 Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Not Scored)
+[WARN] 1.4.10 Ensure that the Container Network Interface file ownership is set to root:root (Not Scored)
+[PASS] 1.4.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Scored)
+[FAIL] 1.4.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)
+[PASS] 1.4.13 Ensure that the admin.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.14 Ensure that the admin.conf file ownership is set to root:root (Scored)
+[PASS] 1.4.15 Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.16 Ensure that the scheduler.conf file ownership is set to root:root (Scored)
+[PASS] 1.4.17 Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.18 Ensure that the controller-manager.conf file ownership is set to root:root (Scored)
+[WARN] 1.4.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Scored)
+[WARN] 1.4.20 Ensure that the Kubernetes PKI certificate file permissions are set to 644 or more restrictive (Scored)
+[WARN] 1.4.21 Ensure that the Kubernetes PKI key file permissions are set to 600 or more restrictive (Scored)
+[INFO] 1.5 etcd
+[PASS] 1.5.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)
+[PASS] 1.5.2 Ensure that the --client-cert-auth argument is set to true (Scored)
+[PASS] 1.5.3 Ensure that the --auto-tls argument is not set to true (Scored)
+[PASS] 1.5.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Scored)
+[PASS] 1.5.5 Ensure that the --peer-client-cert-auth argument is set to true (Scored)
+[PASS] 1.5.6 Ensure that the --peer-auto-tls argument is not set to true (Scored)
+[WARN] 1.5.7 Ensure that a unique Certificate Authority is used for etcd (Not Scored)
+[INFO] 1.6 General Security Primitives
+[WARN] 1.6.1 Ensure that the cluster-admin role is only used where required (Not Scored)
+[WARN] 1.6.2 Create administrative boundaries between resources using namespaces (Not Scored)
+[WARN] 1.6.3 Create network segmentation using Network Policies (Not Scored)
+[WARN] 1.6.4 Ensure that the seccomp profile is set to docker/default in your pod definitions (Not Scored)
+[WARN] 1.6.5 Apply Security Context to Your Pods and Containers (Not Scored)
+[WARN] 1.6.6 Configure Image Provenance using ImagePolicyWebhook admission controller (Not Scored)
+[WARN] 1.6.7 Configure Network policies as appropriate (Not Scored)
+[WARN] 1.6.8 Place compensating controls in the form of PSP and RBAC for privileged containers usage (Not Scored)
+[INFO] 1.7 PodSecurityPolicies
+[WARN] 1.7.1 Do not admit privileged containers (Not Scored)
+[WARN] 1.7.2 Do not admit containers wishing to share the host process ID namespace (Scored)
+[WARN] 1.7.3 Do not admit containers wishing to share the host IPC namespace (Scored)
+[WARN] 1.7.4 Do not admit containers wishing to share the host network namespace (Scored)
+[WARN] 1.7.5  Do not admit containers with allowPrivilegeEscalation (Scored)
+[WARN] 1.7.6 Do not admit root containers (Not Scored)
+[WARN] 1.7.7 Do not admit containers with dangerous capabilities (Not Scored)
+
+== Remediations ==
+1.1.1 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--anonymous-auth=false
+
+1.1.8 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.1.9 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--repair-malformed-updates=false
+
+1.1.11 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins to
+include AlwaysPullImages.
+--enable-admission-plugins=...,AlwaysPullImages,...
+
+1.1.13 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to
+include SecurityContextDeny.
+--enable-admission-plugins=...,SecurityContextDeny,...
+
+1.1.15 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-path parameter to a suitable
+path and file where you would like audit logs to be written, for example:
+--audit-log-path=/var/log/apiserver/audit.log
+
+1.1.16 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxage parameter to 30 or
+as an appropriate number of days: --audit-log-maxage=30
+
+1.1.17 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxbackup parameter to 10
+or to an appropriate value.
+--audit-log-maxbackup=10
+
+1.1.18 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxsize parameter to an
+appropriate size in MB. For example, to set it as 100 MB:
+--audit-log-maxsize=100
+
+1.1.21 Follow the Kubernetes documentation and setup the TLS connection between the
+apiserver and kubelets. Then, edit the API server pod specification file
+/etc/kubernetes/manifests/kube-apiserver.yaml on the master node and set the --kubelet-certificate-authority
+parameter to the path to the cert file for the certificate authority.
+--kubelet-certificate-authority=<ca-string>
+
+1.1.24 Follow the documentation and create Pod Security Policy objects as per your environment.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to a
+value that includes PodSecurityPolicy :
+--enable-admission-plugins=...,PodSecurityPolicy,...
+Then restart the API Server.
+
+1.1.31 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+
+1.1.34 [Manual test]
+Follow the Kubernetes documentation and configure a EncryptionConfig file.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml on the
+master node and set the --encryption-provider-config parameter
+to the path of that file:
+--encryption-provider-config=</path/to/EncryptionConfig/File>
+
+1.1.35 [Manual test]
+Follow the Kubernetes documentation and configure a EncryptionConfig file. In this file,
+choose aescbc as the encryption provider.
+For example,
+kind: EncryptionConfig
+apiVersion: v1
+resources:
+  - resources:
+    - secrets
+      providers:
+      - aescbc:
+          keys:
+          - name: key1
+            secret: <32-byte base64-encoded secret>
+
+1.1.36 Follow the Kubernetes documentation and set the desired limits in a
+configuration file. Then, edit the API server pod specification file
+/etc/kubernetes/manifests/kube-apiserver.yaml and set the below parameters.
+--enable-admission-plugins=...,EventRateLimit,...
+--admission-control-config-file=<path/to/configuration/file>
+
+1.1.37b Follow the Kubernetes documentation and set the desired audit policy in the
+/etc/kubernetes/audit-policy.yaml file. Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+and set the below parameters.
+--audit-policy-file=/etc/kubernetes/audit-policy.yaml
+
+1.2.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.yaml
+file on the master node and set the below parameter.
+--profiling=false
+
+1.3.1 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the --terminated-pod-gc-threshold to an appropriate threshold, for example:
+--terminated-pod-gc-threshold=10
+
+1.3.2 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.3.6 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+controller-manager.yaml on the master node and set the --feature-gates parameter to
+include RotateKubeletServerCertificate=true.
+--feature-gates=RotateKubeletServerCertificate=true
+
+1.4.9 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example,
+chmod 644 <path/to/cni/files>
+
+1.4.10 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example,
+chown root:root <path/to/cni/files>
+
+1.4.12 On the etcd server node, get the etcd data directory, passed as an argument --data-dir ,
+from the below command:
+ps -ef | grep etcd
+Run the below command (based on the etcd data directory found above). For example,
+chown etcd:etcd /var/lib/etcd
+
+1.4.19 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example, chown -R root:root /etc/kubernetes/pki/
+
+1.4.20 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example, chmod -R 644 /etc/kubernetes/pki/*.crt
+
+1.4.21 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example, chmod -R 600 /etc/kubernetes/pki/*.key
+
+1.5.7 [Manual test]
+Follow the etcd documentation and create a dedicated certificate authority setup for the
+etcd service.
+Then, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml on the
+master node and set the below parameter.
+--trusted-ca-file=</path/to/ca-file>
+
+1.6.1 [Manual test]
+Remove any unneeded clusterrolebindings :
+kubectl delete clusterrolebinding [name]
+
+1.6.2 [Manual test]
+Follow the documentation and create namespaces for objects in your deployment as you
+need them.
+
+1.6.3 [Manual test]
+Follow the documentation and create NetworkPolicy objects as you need them.
+
+1.6.4 [Manual test]
+Seccomp is an alpha feature currently. By default, all alpha features are disabled. So, you
+would need to enable alpha features in the apiserver by passing "--feature-
+gates=AllAlpha=true" argument.
+Edit the /etc/kubernetes/manifests/kube-apiserver.yaml file on the master node and set the KUBE_API_ARGS
+parameter to "--feature-gates=AllAlpha=true"
+KUBE_API_ARGS="--feature-gates=AllAlpha=true"
+Based on your system, restart the kube-apiserver service. For example:
+systemctl restart kube-apiserver.service
+Use annotations to enable the docker/default seccomp profile in your pod definitions. An
+example is as below:
+apiVersion: v1
+kind: Pod
+metadata:
+  name: trustworthy-pod
+  annotations:
+    seccomp.security.alpha.kubernetes.io/pod: docker/default
+spec:
+  containers:
+    - name: trustworthy-container
+      image: sotrustworthy:latest
+
+1.6.5 [Manual test]
+Follow the Kubernetes documentation and apply security contexts to your pods. For a
+suggested list of security contexts, you may refer to the CIS Security Benchmark for Docker
+Containers.
+
+1.6.6 [Manual test]
+Follow the Kubernetes documentation and setup image provenance.
+
+1.6.7 [Manual test]
+Follow the Kubernetes documentation and setup network policies as appropriate.
+For example, you could create a "default" isolation policy for a Namespace by creating a
+NetworkPolicy that selects all pods but does not allow any traffic:
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: default-deny
+spec:
+  podSelector:
+
+1.6.8 [Manual test]
+Follow Kubernetes documentation and setup PSP and RBAC authorization for your cluster.
+
+1.7.1 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.privileged field is omitted or set to false.
+
+1.7.2 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostPID field is omitted or set to false.
+
+1.7.3 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostIPC field is omitted or set to false.
+
+1.7.4 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostNetwork field is omitted or set to false.
+
+1.7.5 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.allowPrivilegeEscalation field is omitted or set to false.
+
+1.7.6 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.runAsUser.rule is set to either MustRunAsNonRoot or MustRunAs with the range of UIDs not including 0.
+
+1.7.7 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.requiredDropCapabilities is set to include either NET_RAW or ALL.
+
+
+== Summary ==
+49 checks PASS
+17 checks FAIL
+25 checks WARN
+1 checks INFO

integration/testdata/cis-1.4/job-node.data

@@ -0,0 +1,89 @@
+[INFO] 2 Worker Node Security Configuration
+[INFO] 2.1 Kubelet
+[PASS] 2.1.1 Ensure that the --anonymous-auth argument is set to false (Scored)
+[PASS] 2.1.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 2.1.3 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[FAIL] 2.1.4 Ensure that the --read-only-port argument is set to 0 (Scored)
+[PASS] 2.1.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)
+[FAIL] 2.1.6 Ensure that the --protect-kernel-defaults argument is set to true (Scored)
+[PASS] 2.1.7 Ensure that the --make-iptables-util-chains argument is set to true (Scored)
+[PASS] 2.1.8 Ensure that the --hostname-override argument is not set (Scored)
+[FAIL] 2.1.9 Ensure that the --event-qps argument is set to 0 (Scored)
+[FAIL] 2.1.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[INFO] 2.1.11 [DEPRECATED] Ensure that the --cadvisor-port argument is set to 0
+[PASS] 2.1.12 Ensure that the --rotate-certificates argument is not set to false (Scored)
+[FAIL] 2.1.13 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 2.1.14 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)
+[INFO] 2.2 Configuration Files
+[PASS] 2.2.1 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.2 Ensure that the kubelet.conf file ownership is set to root:root (Scored)
+[PASS] 2.2.3 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.4 Ensure that the kubelet service file ownership is set to root:root (Scored)
+[FAIL] 2.2.5 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)
+[FAIL] 2.2.6 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)
+[PASS] 2.2.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.8 Ensure that the client certificate authorities file ownership is set to root:root (Scored)
+[PASS] 2.2.9 Ensure that the kubelet configuration file ownership is set to root:root (Scored)
+[PASS] 2.2.10 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)
+
+== Remediations ==
+2.1.4 If using a Kubelet config file, edit the file to set readOnlyPort to 0 .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--read-only-port=0
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--protect-kernel-defaults=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.9 If using a Kubelet config file, edit the file to set eventRecordQPS: 0 .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--event-qps=0
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.10 If using a Kubelet config file, edit the file to set tlsCertFile to the location of the certificate
+file to use to identify this Kubelet, and tlsPrivateKeyFile to the location of the
+corresponding private key file.
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameters in KUBELET_CERTIFICATE_ARGS variable.
+--tls-cert-file=<path/to/tls-certificate-file>
+file=<path/to/tls-key-file>
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.13 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
+on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
+--feature-gates=RotateKubeletServerCertificate=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.2.5 Run the below command (based on the file location on your system) on the each worker
+node. For example,
+chmod 644 /etc/kubernetes/proxy.conf
+
+2.2.6 Run the below command (based on the file location on your system) on the each worker
+node. For example,
+chown root:root /etc/kubernetes/proxy.conf
+
+
+== Summary ==
+16 checks PASS
+7 checks FAIL
+0 checks WARN
+1 checks INFO

integration/testdata/cis-1.4/job.data

@@ -0,0 +1,416 @@
+[INFO] 1 Master Node Security Configuration
+[INFO] 1.1 API Server
+[WARN] 1.1.1 Ensure that the --anonymous-auth argument is set to false (Not Scored)
+[PASS] 1.1.2 Ensure that the --basic-auth-file argument is not set (Scored)
+[PASS] 1.1.3 Ensure that the --insecure-allow-any-token argument is not set (Not Scored)
+[PASS] 1.1.4 Ensure that the --kubelet-https argument is set to true (Scored)
+[PASS] 1.1.5 Ensure that the --insecure-bind-address argument is not set (Scored)
+[PASS] 1.1.6 Ensure that the --insecure-port argument is set to 0 (Scored)
+[PASS] 1.1.7 Ensure that the --secure-port argument is not set to 0 (Scored)
+[FAIL] 1.1.8 Ensure that the --profiling argument is set to false (Scored)
+[FAIL] 1.1.9 Ensure that the --repair-malformed-updates argument is set to false (Scored)
+[PASS] 1.1.10 Ensure that the admission control plugin AlwaysAdmit is not set (Scored)
+[FAIL] 1.1.11 Ensure that the admission control plugin AlwaysPullImages is set (Scored)
+[INFO] 1.1.12 [DEPRECATED] Ensure that the admission control plugin DenyEscalatingExec is set (Not Scored)
+[WARN] 1.1.13 Ensure that the admission control plugin SecurityContextDeny is set (Not Scored)
+[PASS] 1.1.14 Ensure that the admission control plugin NamespaceLifecycle is set (Scored)
+[FAIL] 1.1.15 Ensure that the --audit-log-path argument is set as appropriate (Scored)
+[FAIL] 1.1.16 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Scored)
+[FAIL] 1.1.17 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Scored)
+[FAIL] 1.1.18 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Scored)
+[PASS] 1.1.19 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 1.1.20 Ensure that the --token-auth-file parameter is not set (Scored)
+[FAIL] 1.1.21 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored)
+[PASS] 1.1.22 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Scored)
+[PASS] 1.1.23 Ensure that the --service-account-lookup argument is set to true (Scored)
+[FAIL] 1.1.24 Ensure that the admission control plugin PodSecurityPolicy is set (Scored)
+[PASS] 1.1.25 Ensure that the --service-account-key-file argument is set as appropriate (Scored)
+[PASS] 1.1.26 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Scored)
+[PASS] 1.1.27 Ensure that the admission control plugin ServiceAccount is set(Scored)
+[PASS] 1.1.28 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[PASS] 1.1.29 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[PASS] 1.1.30 Ensure that the --etcd-cafile argument is set as appropriate (Scored)
+[WARN] 1.1.31 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Not Scored)
+[PASS] 1.1.32 Ensure that the --authorization-mode argument is set to Node (Scored)
+[PASS] 1.1.33 Ensure that the admission control plugin NodeRestriction is set (Scored)
+[FAIL] 1.1.34 Ensure that the --encryption-provider-config argument is set as appropriate (Scored)
+[WARN] 1.1.35 Ensure that the encryption provider is set to aescbc (Scored)
+[FAIL] 1.1.36 Ensure that the admission control plugin EventRateLimit is set (Scored)
+[PASS] 1.1.37a Ensure that the AdvancedAuditing argument is not set to false (Scored)
+[FAIL] 1.1.37b Ensure that the AdvancedAuditing argument is not set to false (Scored)
+[PASS] 1.1.38 Ensure that the --request-timeout argument is set as appropriate (Scored)
+[PASS] 1.1.39 Ensure that the --authorization-mode argument includes RBAC (Scored)
+[INFO] 1.2 Scheduler
+[FAIL] 1.2.1 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.2.2 Ensure that the --address argument is set to 127.0.0.1 (Scored)
+[INFO] 1.3 Controller Manager
+[FAIL] 1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Scored)
+[FAIL] 1.3.2 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Scored)
+[PASS] 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Scored)
+[PASS] 1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Scored)
+[FAIL] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 1.3.7 Ensure that the --address argument is set to 127.0.0.1 (Scored)
+[INFO] 1.4 Configuration Files
+[PASS] 1.4.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.2 Ensure that the API server pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.3 Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.4 Ensure that the controller manager pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.5 Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.6 Ensure that the scheduler pod specification file ownership is set to root:root (Scored)
+[PASS] 1.4.7 Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.8 Ensure that the etcd pod specification file ownership is set to root:root (Scored)
+[WARN] 1.4.9 Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Not Scored)
+[WARN] 1.4.10 Ensure that the Container Network Interface file ownership is set to root:root (Not Scored)
+[PASS] 1.4.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Scored)
+[FAIL] 1.4.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)
+[PASS] 1.4.13 Ensure that the admin.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.14 Ensure that the admin.conf file ownership is set to root:root (Scored)
+[PASS] 1.4.15 Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.16 Ensure that the scheduler.conf file ownership is set to root:root (Scored)
+[PASS] 1.4.17 Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.4.18 Ensure that the controller-manager.conf file ownership is set to root:root (Scored)
+[WARN] 1.4.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Scored)
+[WARN] 1.4.20 Ensure that the Kubernetes PKI certificate file permissions are set to 644 or more restrictive (Scored)
+[WARN] 1.4.21 Ensure that the Kubernetes PKI key file permissions are set to 600 or more restrictive (Scored)
+[INFO] 1.5 etcd
+[PASS] 1.5.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)
+[PASS] 1.5.2 Ensure that the --client-cert-auth argument is set to true (Scored)
+[PASS] 1.5.3 Ensure that the --auto-tls argument is not set to true (Scored)
+[PASS] 1.5.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Scored)
+[PASS] 1.5.5 Ensure that the --peer-client-cert-auth argument is set to true (Scored)
+[PASS] 1.5.6 Ensure that the --peer-auto-tls argument is not set to true (Scored)
+[WARN] 1.5.7 Ensure that a unique Certificate Authority is used for etcd (Not Scored)
+[INFO] 1.6 General Security Primitives
+[WARN] 1.6.1 Ensure that the cluster-admin role is only used where required (Not Scored)
+[WARN] 1.6.2 Create administrative boundaries between resources using namespaces (Not Scored)
+[WARN] 1.6.3 Create network segmentation using Network Policies (Not Scored)
+[WARN] 1.6.4 Ensure that the seccomp profile is set to docker/default in your pod definitions (Not Scored)
+[WARN] 1.6.5 Apply Security Context to Your Pods and Containers (Not Scored)
+[WARN] 1.6.6 Configure Image Provenance using ImagePolicyWebhook admission controller (Not Scored)
+[WARN] 1.6.7 Configure Network policies as appropriate (Not Scored)
+[WARN] 1.6.8 Place compensating controls in the form of PSP and RBAC for privileged containers usage (Not Scored)
+[INFO] 1.7 PodSecurityPolicies
+[WARN] 1.7.1 Do not admit privileged containers (Not Scored)
+[WARN] 1.7.2 Do not admit containers wishing to share the host process ID namespace (Scored)
+[WARN] 1.7.3 Do not admit containers wishing to share the host IPC namespace (Scored)
+[WARN] 1.7.4 Do not admit containers wishing to share the host network namespace (Scored)
+[WARN] 1.7.5  Do not admit containers with allowPrivilegeEscalation (Scored)
+[WARN] 1.7.6 Do not admit root containers (Not Scored)
+[WARN] 1.7.7 Do not admit containers with dangerous capabilities (Not Scored)
+
+== Remediations ==
+1.1.1 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--anonymous-auth=false
+
+1.1.8 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.1.9 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--repair-malformed-updates=false
+
+1.1.11 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins to
+include AlwaysPullImages.
+--enable-admission-plugins=...,AlwaysPullImages,...
+
+1.1.13 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to
+include SecurityContextDeny.
+--enable-admission-plugins=...,SecurityContextDeny,...
+
+1.1.15 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-path parameter to a suitable
+path and file where you would like audit logs to be written, for example:
+--audit-log-path=/var/log/apiserver/audit.log
+
+1.1.16 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxage parameter to 30 or
+as an appropriate number of days: --audit-log-maxage=30
+
+1.1.17 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxbackup parameter to 10
+or to an appropriate value.
+--audit-log-maxbackup=10
+
+1.1.18 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxsize parameter to an
+appropriate size in MB. For example, to set it as 100 MB:
+--audit-log-maxsize=100
+
+1.1.21 Follow the Kubernetes documentation and setup the TLS connection between the
+apiserver and kubelets. Then, edit the API server pod specification file
+/etc/kubernetes/manifests/kube-apiserver.yaml on the master node and set the --kubelet-certificate-authority
+parameter to the path to the cert file for the certificate authority.
+--kubelet-certificate-authority=<ca-string>
+
+1.1.24 Follow the documentation and create Pod Security Policy objects as per your environment.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to a
+value that includes PodSecurityPolicy :
+--enable-admission-plugins=...,PodSecurityPolicy,...
+Then restart the API Server.
+
+1.1.31 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+
+1.1.34 [Manual test]
+Follow the Kubernetes documentation and configure a EncryptionConfig file.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml on the
+master node and set the --encryption-provider-config parameter
+to the path of that file:
+--encryption-provider-config=</path/to/EncryptionConfig/File>
+
+1.1.35 [Manual test]
+Follow the Kubernetes documentation and configure a EncryptionConfig file. In this file,
+choose aescbc as the encryption provider.
+For example,
+kind: EncryptionConfig
+apiVersion: v1
+resources:
+  - resources:
+    - secrets
+      providers:
+      - aescbc:
+          keys:
+          - name: key1
+            secret: <32-byte base64-encoded secret>
+
+1.1.36 Follow the Kubernetes documentation and set the desired limits in a
+configuration file. Then, edit the API server pod specification file
+/etc/kubernetes/manifests/kube-apiserver.yaml and set the below parameters.
+--enable-admission-plugins=...,EventRateLimit,...
+--admission-control-config-file=<path/to/configuration/file>
+
+1.1.37b Follow the Kubernetes documentation and set the desired audit policy in the
+/etc/kubernetes/audit-policy.yaml file. Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+and set the below parameters.
+--audit-policy-file=/etc/kubernetes/audit-policy.yaml
+
+1.2.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.yaml
+file on the master node and set the below parameter.
+--profiling=false
+
+1.3.1 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the --terminated-pod-gc-threshold to an appropriate threshold, for example:
+--terminated-pod-gc-threshold=10
+
+1.3.2 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.3.6 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+controller-manager.yaml on the master node and set the --feature-gates parameter to
+include RotateKubeletServerCertificate=true.
+--feature-gates=RotateKubeletServerCertificate=true
+
+1.4.9 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example,
+chmod 644 <path/to/cni/files>
+
+1.4.10 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example,
+chown root:root <path/to/cni/files>
+
+1.4.12 On the etcd server node, get the etcd data directory, passed as an argument --data-dir ,
+from the below command:
+ps -ef | grep etcd
+Run the below command (based on the etcd data directory found above). For example,
+chown etcd:etcd /var/lib/etcd
+
+1.4.19 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example, chown -R root:root /etc/kubernetes/pki/
+
+1.4.20 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example, chmod -R 644 /etc/kubernetes/pki/*.crt
+
+1.4.21 [Manual test]
+Run the below command (based on the file location on your system) on the master node.
+For example, chmod -R 600 /etc/kubernetes/pki/*.key
+
+1.5.7 [Manual test]
+Follow the etcd documentation and create a dedicated certificate authority setup for the
+etcd service.
+Then, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml on the
+master node and set the below parameter.
+--trusted-ca-file=</path/to/ca-file>
+
+1.6.1 [Manual test]
+Remove any unneeded clusterrolebindings :
+kubectl delete clusterrolebinding [name]
+
+1.6.2 [Manual test]
+Follow the documentation and create namespaces for objects in your deployment as you
+need them.
+
+1.6.3 [Manual test]
+Follow the documentation and create NetworkPolicy objects as you need them.
+
+1.6.4 [Manual test]
+Seccomp is an alpha feature currently. By default, all alpha features are disabled. So, you
+would need to enable alpha features in the apiserver by passing "--feature-
+gates=AllAlpha=true" argument.
+Edit the /etc/kubernetes/manifests/kube-apiserver.yaml file on the master node and set the KUBE_API_ARGS
+parameter to "--feature-gates=AllAlpha=true"
+KUBE_API_ARGS="--feature-gates=AllAlpha=true"
+Based on your system, restart the kube-apiserver service. For example:
+systemctl restart kube-apiserver.service
+Use annotations to enable the docker/default seccomp profile in your pod definitions. An
+example is as below:
+apiVersion: v1
+kind: Pod
+metadata:
+  name: trustworthy-pod
+  annotations:
+    seccomp.security.alpha.kubernetes.io/pod: docker/default
+spec:
+  containers:
+    - name: trustworthy-container
+      image: sotrustworthy:latest
+
+1.6.5 [Manual test]
+Follow the Kubernetes documentation and apply security contexts to your pods. For a
+suggested list of security contexts, you may refer to the CIS Security Benchmark for Docker
+Containers.
+
+1.6.6 [Manual test]
+Follow the Kubernetes documentation and setup image provenance.
+
+1.6.7 [Manual test]
+Follow the Kubernetes documentation and setup network policies as appropriate.
+For example, you could create a "default" isolation policy for a Namespace by creating a
+NetworkPolicy that selects all pods but does not allow any traffic:
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: default-deny
+spec:
+  podSelector:
+
+1.6.8 [Manual test]
+Follow Kubernetes documentation and setup PSP and RBAC authorization for your cluster.
+
+1.7.1 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.privileged field is omitted or set to false.
+
+1.7.2 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostPID field is omitted or set to false.
+
+1.7.3 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostIPC field is omitted or set to false.
+
+1.7.4 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.hostNetwork field is omitted or set to false.
+
+1.7.5 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.allowPrivilegeEscalation field is omitted or set to false.
+
+1.7.6 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.runAsUser.rule is set to either MustRunAsNonRoot or MustRunAs with the range of UIDs not including 0.
+
+1.7.7 [Manual test]
+Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.requiredDropCapabilities is set to include either NET_RAW or ALL.
+
+
+== Summary ==
+49 checks PASS
+17 checks FAIL
+25 checks WARN
+1 checks INFO
+[INFO] 2 Worker Node Security Configuration
+[INFO] 2.1 Kubelet
+[PASS] 2.1.1 Ensure that the --anonymous-auth argument is set to false (Scored)
+[PASS] 2.1.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 2.1.3 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[FAIL] 2.1.4 Ensure that the --read-only-port argument is set to 0 (Scored)
+[PASS] 2.1.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)
+[FAIL] 2.1.6 Ensure that the --protect-kernel-defaults argument is set to true (Scored)
+[PASS] 2.1.7 Ensure that the --make-iptables-util-chains argument is set to true (Scored)
+[PASS] 2.1.8 Ensure that the --hostname-override argument is not set (Scored)
+[FAIL] 2.1.9 Ensure that the --event-qps argument is set to 0 (Scored)
+[FAIL] 2.1.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[INFO] 2.1.11 [DEPRECATED] Ensure that the --cadvisor-port argument is set to 0
+[PASS] 2.1.12 Ensure that the --rotate-certificates argument is not set to false (Scored)
+[FAIL] 2.1.13 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 2.1.14 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)
+[INFO] 2.2 Configuration Files
+[PASS] 2.2.1 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.2 Ensure that the kubelet.conf file ownership is set to root:root (Scored)
+[PASS] 2.2.3 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.4 Ensure that the kubelet service file ownership is set to root:root (Scored)
+[FAIL] 2.2.5 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)
+[FAIL] 2.2.6 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)
+[PASS] 2.2.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)
+[PASS] 2.2.8 Ensure that the client certificate authorities file ownership is set to root:root (Scored)
+[PASS] 2.2.9 Ensure that the kubelet configuration file ownership is set to root:root (Scored)
+[PASS] 2.2.10 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)
+
+== Remediations ==
+2.1.4 If using a Kubelet config file, edit the file to set readOnlyPort to 0 .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--read-only-port=0
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--protect-kernel-defaults=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.9 If using a Kubelet config file, edit the file to set eventRecordQPS: 0 .
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--event-qps=0
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.10 If using a Kubelet config file, edit the file to set tlsCertFile to the location of the certificate
+file to use to identify this Kubelet, and tlsPrivateKeyFile to the location of the
+corresponding private key file.
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameters in KUBELET_CERTIFICATE_ARGS variable.
+--tls-cert-file=<path/to/tls-certificate-file>
+file=<path/to/tls-key-file>
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.1.13 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
+on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
+--feature-gates=RotateKubeletServerCertificate=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+2.2.5 Run the below command (based on the file location on your system) on the each worker
+node. For example,
+chmod 644 /etc/kubernetes/proxy.conf
+
+2.2.6 Run the below command (based on the file location on your system) on the each worker
+node. For example,
+chown root:root /etc/kubernetes/proxy.conf
+
+
+== Summary ==
+16 checks PASS
+7 checks FAIL
+0 checks WARN
+1 checks INFO

integration/testdata/cis-1.5/add-tls-kind.yaml

@@ -0,0 +1,19 @@
+---
+apiVersion: kind.sigs.k8s.io/v1alpha3
+kind: Cluster
+networking:
+  apiServerAddress: "0.0.0.0"
+
+kubeadmConfigPatchesJson6902:
+  - group: kubelet.config.k8s.io
+    version: v1beta1
+    kind: KubeletConfiguration
+    patch: |
+      - op: add
+        path: /tlsCipherSuites
+        value: ["TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"]
+
+nodes:
+  # the control plane node config
+  - role: control-plane
+    image: "kindest/node:v1.15.0"

integration/testdata/cis-1.5/job-master.data

@@ -0,0 +1,176 @@
+[INFO] 1 Master Node Security Configuration
+[INFO] 1.1 Master Node Configuration Files
+[PASS] 1.1.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Scored)
+[PASS] 1.1.3 Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Scored)
+[PASS] 1.1.5 Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Scored)
+[PASS] 1.1.7 Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Scored)
+[WARN] 1.1.9 Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Not Scored)
+[WARN] 1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Not Scored)
+[PASS] 1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Scored)
+[FAIL] 1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)
+[PASS] 1.1.13 Ensure that the admin.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.14 Ensure that the admin.conf file ownership is set to root:root (Scored)
+[PASS] 1.1.15 Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Scored)
+[PASS] 1.1.17 Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Scored)
+[PASS] 1.1.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Scored)
+[PASS] 1.1.20 Ensure that the Kubernetes PKI certificate file permissions are set to 644 or more restrictive (Not Scored)
+[PASS] 1.1.21 Ensure that the Kubernetes PKI key file permissions are set to 600 (Not Scored)
+[INFO] 1.2 API Server
+[WARN] 1.2.1 Ensure that the --anonymous-auth argument is set to false (Not Scored)
+[PASS] 1.2.2 Ensure that the --basic-auth-file argument is not set (Scored)
+[PASS] 1.2.3 Ensure that the --token-auth-file parameter is not set (Scored)
+[PASS] 1.2.4 Ensure that the --kubelet-https argument is set to true (Scored)
+[PASS] 1.2.5 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Scored)
+[FAIL] 1.2.6 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored)
+[PASS] 1.2.7 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 1.2.8 Ensure that the --authorization-mode argument includes Node (Scored)
+[PASS] 1.2.9 Ensure that the --authorization-mode argument includes RBAC (Scored)
+[WARN] 1.2.10 Ensure that the admission control plugin EventRateLimit is set (Not Scored)
+[PASS] 1.2.11 Ensure that the admission control plugin AlwaysAdmit is not set (Scored)
+[WARN] 1.2.12 Ensure that the admission control plugin AlwaysPullImages is set (Not Scored)
+[WARN] 1.2.13 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Not Scored)
+[PASS] 1.2.14 Ensure that the admission control plugin ServiceAccount is set (Scored)
+[PASS] 1.2.15 Ensure that the admission control plugin NamespaceLifecycle is set (Scored)
+[FAIL] 1.2.16 Ensure that the admission control plugin PodSecurityPolicy is set (Scored)
+[PASS] 1.2.17 Ensure that the admission control plugin NodeRestriction is set (Scored)
+[PASS] 1.2.18 Ensure that the --insecure-bind-address argument is not set (Scored)
+[PASS] 1.2.19 Ensure that the --insecure-port argument is set to 0 (Scored)
+[PASS] 1.2.20 Ensure that the --secure-port argument is not set to 0 (Scored)
+[FAIL] 1.2.21 Ensure that the --profiling argument is set to false (Scored)
+[FAIL] 1.2.22 Ensure that the --audit-log-path argument is set (Scored)
+[FAIL] 1.2.23 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Scored)
+[FAIL] 1.2.24 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Scored)
+[FAIL] 1.2.25 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Scored)
+[PASS] 1.2.26 Ensure that the --request-timeout argument is set as appropriate (Scored)
+[PASS] 1.2.27 Ensure that the --service-account-lookup argument is set to true (Scored)
+[PASS] 1.2.28 Ensure that the --service-account-key-file argument is set as appropriate (Scored)
+[PASS] 1.2.29 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Scored)
+[PASS] 1.2.30 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[PASS] 1.2.31 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[PASS] 1.2.32 Ensure that the --etcd-cafile argument is set as appropriate (Scored)
+[WARN] 1.2.33 Ensure that the --encryption-provider-config argument is set as appropriate (Not Scored)
+[WARN] 1.2.34 Ensure that encryption providers are appropriately configured (Not Scored)
+[WARN] 1.2.35 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Not Scored)
+[INFO] 1.3 Controller Manager
+[WARN] 1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Not Scored)
+[FAIL] 1.3.2 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Scored)
+[PASS] 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Scored)
+[PASS] 1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Scored)
+[FAIL] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Scored)
+[INFO] 1.4 Scheduler
+[FAIL] 1.4.1 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Scored)
+
+== Remediations ==
+1.1.9 Run the below command (based on the file location on your system) on the master node.
+For example,
+chmod 644 <path/to/cni/files>
+
+1.1.10 Run the below command (based on the file location on your system) on the master node.
+For example,
+chown root:root <path/to/cni/files>
+
+1.1.12 On the etcd server node, get the etcd data directory, passed as an argument --data-dir,
+from the below command:
+ps -ef | grep etcd
+Run the below command (based on the etcd data directory found above).
+For example, chown etcd:etcd /var/lib/etcd
+
+1.2.1 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--anonymous-auth=false
+
+1.2.6 Follow the Kubernetes documentation and setup the TLS connection between
+the apiserver and kubelets. Then, edit the API server pod specification file
+/etc/kubernetes/manifests/kube-apiserver.yaml on the master node and set the
+--kubelet-certificate-authority parameter to the path to the cert file for the certificate authority.
+--kubelet-certificate-authority=<ca-string>
+
+1.2.10 Follow the Kubernetes documentation and set the desired limits in a configuration file.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+and set the below parameters.
+--enable-admission-plugins=...,EventRateLimit,...
+--admission-control-config-file=<path/to/configuration/file>
+
+1.2.12 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to include
+AlwaysPullImages.
+--enable-admission-plugins=...,AlwaysPullImages,...
+
+1.2.13 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to include
+SecurityContextDeny, unless PodSecurityPolicy is already in place.
+--enable-admission-plugins=...,SecurityContextDeny,...
+
+1.2.16 Follow the documentation and create Pod Security Policy objects as per your environment.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to a
+value that includes PodSecurityPolicy:
+--enable-admission-plugins=...,PodSecurityPolicy,...
+Then restart the API Server.
+
+1.2.21 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.2.22 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-path parameter to a suitable path and
+file where you would like audit logs to be written, for example:
+--audit-log-path=/var/log/apiserver/audit.log
+
+1.2.23 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxage parameter to 30 or as an appropriate number of days:
+--audit-log-maxage=30
+
+1.2.24 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxbackup parameter to 10 or to an appropriate
+value.
+--audit-log-maxbackup=10
+
+1.2.25 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxsize parameter to an appropriate size in MB.
+For example, to set it as 100 MB:
+--audit-log-maxsize=100
+
+1.2.33 Follow the Kubernetes documentation and configure a EncryptionConfig file.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --encryption-provider-config parameter to the path of that file: --encryption-provider-config=</path/to/EncryptionConfig/File>
+
+1.2.34 Follow the Kubernetes documentation and configure a EncryptionConfig file.
+In this file, choose aescbc, kms or secretbox as the encryption provider.
+
+1.2.35 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+
+1.3.1 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the --terminated-pod-gc-threshold to an appropriate threshold,
+for example:
+--terminated-pod-gc-threshold=10
+
+1.3.2 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.3.6 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the --feature-gates parameter to include RotateKubeletServerCertificate=true.
+--feature-gates=RotateKubeletServerCertificate=true
+
+1.4.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.yaml file
+on the master node and set the below parameter.
+--profiling=false
+
+
+== Summary ==
+44 checks PASS
+11 checks FAIL
+10 checks WARN
+0 checks INFO

integration/testdata/cis-1.5/job-node.data

@@ -0,0 +1,77 @@
+[INFO] 4 Worker Node Security Configuration
+[INFO] 4.1 Worker Node Configuration Files
+[PASS] 4.1.1 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)
+[PASS] 4.1.2 Ensure that the kubelet service file ownership is set to root:root (Scored)
+[FAIL] 4.1.3 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)
+[FAIL] 4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)
+[PASS] 4.1.5 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 4.1.6 Ensure that the kubelet.conf file ownership is set to root:root (Scored)
+[PASS] 4.1.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)
+[PASS] 4.1.8 Ensure that the client certificate authorities file ownership is set to root:root (Scored)
+[PASS] 4.1.9 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)
+[PASS] 4.1.10 Ensure that the kubelet configuration file ownership is set to root:root (Scored)
+[INFO] 4.2 Kubelet
+[PASS] 4.2.1 Ensure that the anonymous-auth argument is set to false (Scored)
+[PASS] 4.2.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 4.2.3 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[PASS] 4.2.4 Ensure that the --read-only-port argument is set to 0 (Scored)
+[PASS] 4.2.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)
+[FAIL] 4.2.6 Ensure that the --protect-kernel-defaults argument is set to true (Scored)
+[PASS] 4.2.7 Ensure that the --make-iptables-util-chains argument is set to true (Scored)
+[PASS] 4.2.8 Ensure that the --hostname-override argument is not set (Not Scored)
+[WARN] 4.2.9 Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture (Not Scored)
+[FAIL] 4.2.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[PASS] 4.2.11 Ensure that the --rotate-certificates argument is not set to false (Scored)
+[FAIL] 4.2.12 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 4.2.13 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)
+
+== Remediations ==
+4.1.3 Run the below command (based on the file location on your system) on the each worker node.
+For example,
+chmod 644 /etc/kubernetes/proxy.conf
+
+4.1.4 Run the below command (based on the file location on your system) on the each worker node.
+For example, chown root:root /etc/kubernetes/proxy.conf
+
+4.2.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--protect-kernel-defaults=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+4.2.9 If using a Kubelet config file, edit the file to set eventRecordQPS: to an appropriate level.
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+4.2.10 If using a Kubelet config file, edit the file to set tlsCertFile to the location
+of the certificate file to use to identify this Kubelet, and tlsPrivateKeyFile
+to the location of the corresponding private key file.
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameters in KUBELET_CERTIFICATE_ARGS variable.
+--tls-cert-file=<path/to/tls-certificate-file>
+--tls-private-key-file=<path/to/tls-key-file>
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+4.2.12 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
+on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
+--feature-gates=RotateKubeletServerCertificate=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+
+== Summary ==
+17 checks PASS
+5 checks FAIL
+1 checks WARN
+0 checks INFO

integration/testdata/cis-1.5/job.data

@@ -0,0 +1,424 @@
+[INFO] 1 Master Node Security Configuration
+[INFO] 1.1 Master Node Configuration Files
+[PASS] 1.1.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Scored)
+[PASS] 1.1.3 Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Scored)
+[PASS] 1.1.5 Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Scored)
+[PASS] 1.1.7 Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Scored)
+[WARN] 1.1.9 Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Not Scored)
+[WARN] 1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Not Scored)
+[PASS] 1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Scored)
+[FAIL] 1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)
+[PASS] 1.1.13 Ensure that the admin.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.14 Ensure that the admin.conf file ownership is set to root:root (Scored)
+[PASS] 1.1.15 Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Scored)
+[PASS] 1.1.17 Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Scored)
+[PASS] 1.1.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Scored)
+[PASS] 1.1.20 Ensure that the Kubernetes PKI certificate file permissions are set to 644 or more restrictive (Not Scored)
+[PASS] 1.1.21 Ensure that the Kubernetes PKI key file permissions are set to 600 (Not Scored)
+[INFO] 1.2 API Server
+[WARN] 1.2.1 Ensure that the --anonymous-auth argument is set to false (Not Scored)
+[PASS] 1.2.2 Ensure that the --basic-auth-file argument is not set (Scored)
+[PASS] 1.2.3 Ensure that the --token-auth-file parameter is not set (Scored)
+[PASS] 1.2.4 Ensure that the --kubelet-https argument is set to true (Scored)
+[PASS] 1.2.5 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Scored)
+[FAIL] 1.2.6 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored)
+[PASS] 1.2.7 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 1.2.8 Ensure that the --authorization-mode argument includes Node (Scored)
+[PASS] 1.2.9 Ensure that the --authorization-mode argument includes RBAC (Scored)
+[WARN] 1.2.10 Ensure that the admission control plugin EventRateLimit is set (Not Scored)
+[PASS] 1.2.11 Ensure that the admission control plugin AlwaysAdmit is not set (Scored)
+[WARN] 1.2.12 Ensure that the admission control plugin AlwaysPullImages is set (Not Scored)
+[WARN] 1.2.13 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Not Scored)
+[PASS] 1.2.14 Ensure that the admission control plugin ServiceAccount is set (Scored)
+[PASS] 1.2.15 Ensure that the admission control plugin NamespaceLifecycle is set (Scored)
+[FAIL] 1.2.16 Ensure that the admission control plugin PodSecurityPolicy is set (Scored)
+[PASS] 1.2.17 Ensure that the admission control plugin NodeRestriction is set (Scored)
+[PASS] 1.2.18 Ensure that the --insecure-bind-address argument is not set (Scored)
+[PASS] 1.2.19 Ensure that the --insecure-port argument is set to 0 (Scored)
+[PASS] 1.2.20 Ensure that the --secure-port argument is not set to 0 (Scored)
+[FAIL] 1.2.21 Ensure that the --profiling argument is set to false (Scored)
+[FAIL] 1.2.22 Ensure that the --audit-log-path argument is set (Scored)
+[FAIL] 1.2.23 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Scored)
+[FAIL] 1.2.24 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Scored)
+[FAIL] 1.2.25 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Scored)
+[PASS] 1.2.26 Ensure that the --request-timeout argument is set as appropriate (Scored)
+[PASS] 1.2.27 Ensure that the --service-account-lookup argument is set to true (Scored)
+[PASS] 1.2.28 Ensure that the --service-account-key-file argument is set as appropriate (Scored)
+[PASS] 1.2.29 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Scored)
+[PASS] 1.2.30 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[PASS] 1.2.31 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[PASS] 1.2.32 Ensure that the --etcd-cafile argument is set as appropriate (Scored)
+[WARN] 1.2.33 Ensure that the --encryption-provider-config argument is set as appropriate (Not Scored)
+[WARN] 1.2.34 Ensure that encryption providers are appropriately configured (Not Scored)
+[WARN] 1.2.35 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Not Scored)
+[INFO] 1.3 Controller Manager
+[WARN] 1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Not Scored)
+[FAIL] 1.3.2 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Scored)
+[PASS] 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Scored)
+[PASS] 1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Scored)
+[FAIL] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Scored)
+[INFO] 1.4 Scheduler
+[FAIL] 1.4.1 Ensure that the --profiling argument is set to false (Scored)
+[PASS] 1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Scored)
+
+== Remediations ==
+1.1.9 Run the below command (based on the file location on your system) on the master node.
+For example,
+chmod 644 <path/to/cni/files>
+
+1.1.10 Run the below command (based on the file location on your system) on the master node.
+For example,
+chown root:root <path/to/cni/files>
+
+1.1.12 On the etcd server node, get the etcd data directory, passed as an argument --data-dir,
+from the below command:
+ps -ef | grep etcd
+Run the below command (based on the etcd data directory found above).
+For example, chown etcd:etcd /var/lib/etcd
+
+1.2.1 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--anonymous-auth=false
+
+1.2.6 Follow the Kubernetes documentation and setup the TLS connection between
+the apiserver and kubelets. Then, edit the API server pod specification file
+/etc/kubernetes/manifests/kube-apiserver.yaml on the master node and set the
+--kubelet-certificate-authority parameter to the path to the cert file for the certificate authority.
+--kubelet-certificate-authority=<ca-string>
+
+1.2.10 Follow the Kubernetes documentation and set the desired limits in a configuration file.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+and set the below parameters.
+--enable-admission-plugins=...,EventRateLimit,...
+--admission-control-config-file=<path/to/configuration/file>
+
+1.2.12 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to include
+AlwaysPullImages.
+--enable-admission-plugins=...,AlwaysPullImages,...
+
+1.2.13 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to include
+SecurityContextDeny, unless PodSecurityPolicy is already in place.
+--enable-admission-plugins=...,SecurityContextDeny,...
+
+1.2.16 Follow the documentation and create Pod Security Policy objects as per your environment.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --enable-admission-plugins parameter to a
+value that includes PodSecurityPolicy:
+--enable-admission-plugins=...,PodSecurityPolicy,...
+Then restart the API Server.
+
+1.2.21 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.2.22 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-path parameter to a suitable path and
+file where you would like audit logs to be written, for example:
+--audit-log-path=/var/log/apiserver/audit.log
+
+1.2.23 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxage parameter to 30 or as an appropriate number of days:
+--audit-log-maxage=30
+
+1.2.24 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxbackup parameter to 10 or to an appropriate
+value.
+--audit-log-maxbackup=10
+
+1.2.25 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --audit-log-maxsize parameter to an appropriate size in MB.
+For example, to set it as 100 MB:
+--audit-log-maxsize=100
+
+1.2.33 Follow the Kubernetes documentation and configure a EncryptionConfig file.
+Then, edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the --encryption-provider-config parameter to the path of that file: --encryption-provider-config=</path/to/EncryptionConfig/File>
+
+1.2.34 Follow the Kubernetes documentation and configure a EncryptionConfig file.
+In this file, choose aescbc, kms or secretbox as the encryption provider.
+
+1.2.35 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
+on the master node and set the below parameter.
+--tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+
+1.3.1 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the --terminated-pod-gc-threshold to an appropriate threshold,
+for example:
+--terminated-pod-gc-threshold=10
+
+1.3.2 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the below parameter.
+--profiling=false
+
+1.3.6 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
+on the master node and set the --feature-gates parameter to include RotateKubeletServerCertificate=true.
+--feature-gates=RotateKubeletServerCertificate=true
+
+1.4.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.yaml file
+on the master node and set the below parameter.
+--profiling=false
+
+
+== Summary ==
+44 checks PASS
+11 checks FAIL
+10 checks WARN
+0 checks INFO
+[INFO] 2 Etcd Node Configuration
+[INFO] 2 Etcd Node Configuration Files
+[PASS] 2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)
+[PASS] 2.2 Ensure that the --client-cert-auth argument is set to true (Scored)
+[PASS] 2.3 Ensure that the --auto-tls argument is not set to true (Scored)
+[PASS] 2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Scored)
+[PASS] 2.5 Ensure that the --peer-client-cert-auth argument is set to true (Scored)
+[PASS] 2.6 Ensure that the --peer-auto-tls argument is not set to true (Scored)
+[PASS] 2.7 Ensure that a unique Certificate Authority is used for etcd (Not Scored)
+
+== Summary ==
+7 checks PASS
+0 checks FAIL
+0 checks WARN
+0 checks INFO
+[INFO] 3 Control Plane Configuration
+[INFO] 3.1 Authentication and Authorization
+[WARN] 3.1.1 Client certificate authentication should not be used for users (Not Scored)
+[INFO] 3.2 Logging
+[WARN] 3.2.1 Ensure that a minimal audit policy is created (Scored)
+[WARN] 3.2.2 Ensure that the audit policy covers key security concerns (Not Scored)
+
+== Remediations ==
+3.1.1 Alternative mechanisms provided by Kubernetes such as the use of OIDC should be
+implemented in place of client certificates.
+
+3.2.1 Create an audit policy file for your cluster.
+
+3.2.2 Consider modification of the audit policy in use on the cluster to include these items, at a
+minimum.
+
+
+== Summary ==
+0 checks PASS
+0 checks FAIL
+3 checks WARN
+0 checks INFO
+[INFO] 4 Worker Node Security Configuration
+[INFO] 4.1 Worker Node Configuration Files
+[PASS] 4.1.1 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)
+[PASS] 4.1.2 Ensure that the kubelet service file ownership is set to root:root (Scored)
+[FAIL] 4.1.3 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)
+[FAIL] 4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)
+[PASS] 4.1.5 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored)
+[PASS] 4.1.6 Ensure that the kubelet.conf file ownership is set to root:root (Scored)
+[PASS] 4.1.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)
+[PASS] 4.1.8 Ensure that the client certificate authorities file ownership is set to root:root (Scored)
+[PASS] 4.1.9 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)
+[PASS] 4.1.10 Ensure that the kubelet configuration file ownership is set to root:root (Scored)
+[INFO] 4.2 Kubelet
+[PASS] 4.2.1 Ensure that the anonymous-auth argument is set to false (Scored)
+[PASS] 4.2.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
+[PASS] 4.2.3 Ensure that the --client-ca-file argument is set as appropriate (Scored)
+[PASS] 4.2.4 Ensure that the --read-only-port argument is set to 0 (Scored)
+[PASS] 4.2.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)
+[FAIL] 4.2.6 Ensure that the --protect-kernel-defaults argument is set to true (Scored)
+[PASS] 4.2.7 Ensure that the --make-iptables-util-chains argument is set to true (Scored)
+[PASS] 4.2.8 Ensure that the --hostname-override argument is not set (Not Scored)
+[WARN] 4.2.9 Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture (Not Scored)
+[FAIL] 4.2.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
+[PASS] 4.2.11 Ensure that the --rotate-certificates argument is not set to false (Scored)
+[FAIL] 4.2.12 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 4.2.13 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)
+
+== Remediations ==
+4.1.3 Run the below command (based on the file location on your system) on the each worker node.
+For example,
+chmod 644 /etc/kubernetes/proxy.conf
+
+4.1.4 Run the below command (based on the file location on your system) on the each worker node.
+For example, chown root:root /etc/kubernetes/proxy.conf
+
+4.2.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+--protect-kernel-defaults=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+4.2.9 If using a Kubelet config file, edit the file to set eventRecordQPS: to an appropriate level.
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+4.2.10 If using a Kubelet config file, edit the file to set tlsCertFile to the location
+of the certificate file to use to identify this Kubelet, and tlsPrivateKeyFile
+to the location of the corresponding private key file.
+If using command line arguments, edit the kubelet service file
+/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
+set the below parameters in KUBELET_CERTIFICATE_ARGS variable.
+--tls-cert-file=<path/to/tls-certificate-file>
+--tls-private-key-file=<path/to/tls-key-file>
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+4.2.12 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
+on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
+--feature-gates=RotateKubeletServerCertificate=true
+Based on your system, restart the kubelet service. For example:
+systemctl daemon-reload
+systemctl restart kubelet.service
+
+
+== Summary ==
+17 checks PASS
+5 checks FAIL
+1 checks WARN
+0 checks INFO
+[INFO] 5 Kubernetes Policies
+[INFO] 5.1 RBAC and Service Accounts
+[WARN] 5.1.1 Ensure that the cluster-admin role is only used where required (Not Scored)
+[WARN] 5.1.2 Minimize access to secrets (Not Scored)
+[WARN] 5.1.3 Minimize wildcard use in Roles and ClusterRoles (Not Scored)
+[WARN] 5.1.4 Minimize access to create pods (Not Scored)
+[WARN] 5.1.5 Ensure that default service accounts are not actively used. (Scored)
+[WARN] 5.1.6 Ensure that Service Account Tokens are only mounted where necessary (Not Scored)
+[INFO] 5.2 Pod Security Policies
+[WARN] 5.2.1 Minimize the admission of privileged containers (Not Scored)
+[WARN] 5.2.2 Minimize the admission of containers wishing to share the host process ID namespace (Scored)
+[WARN] 5.2.3 Minimize the admission of containers wishing to share the host IPC namespace (Scored)
+[WARN] 5.2.4 Minimize the admission of containers wishing to share the host network namespace (Scored)
+[WARN] 5.2.5 Minimize the admission of containers with allowPrivilegeEscalation (Scored)
+[WARN] 5.2.6 Minimize the admission of root containers (Not Scored)
+[WARN] 5.2.7 Minimize the admission of containers with the NET_RAW capability (Not Scored)
+[WARN] 5.2.8 Minimize the admission of containers with added capabilities (Not Scored)
+[WARN] 5.2.9 Minimize the admission of containers with capabilities assigned (Not Scored)
+[INFO] 5.3 Network Policies and CNI
+[WARN] 5.3.1 Ensure that the CNI in use supports Network Policies (Not Scored)
+[WARN] 5.3.2 Ensure that all Namespaces have Network Policies defined (Scored)
+[INFO] 5.4 Secrets Management
+[WARN] 5.4.1 Prefer using secrets as files over secrets as environment variables (Not Scored)
+[WARN] 5.4.2 Consider external secret storage (Not Scored)
+[INFO] 5.5 Extensible Admission Control
+[WARN] 5.5.1 Configure Image Provenance using ImagePolicyWebhook admission controller (Not Scored)
+[INFO] 5.7 General Policies
+[WARN] 5.7.1 Create administrative boundaries between resources using namespaces (Not Scored)
+[WARN] 5.7.2 Ensure that the seccomp profile is set to docker/default in your pod definitions (Not Scored)
+[WARN] 5.7.3 Apply Security Context to Your Pods and Containers (Not Scored)
+[WARN] 5.7.4 The default namespace should not be used (Scored)
+
+== Remediations ==
+5.1.1 Identify all clusterrolebindings to the cluster-admin role. Check if they are used and
+if they need this role or if they could use a role with fewer privileges.
+Where possible, first bind users to a lower privileged role and then remove the
+clusterrolebinding to the cluster-admin role :
+kubectl delete clusterrolebinding [name]
+
+5.1.2 Where possible, remove get, list and watch access to secret objects in the cluster.
+
+5.1.3 Where possible replace any use of wildcards in clusterroles and roles with specific
+objects or actions.
+
+5.1.4 Where possible, remove create access to pod objects in the cluster.
+
+5.1.5 Create explicit service accounts wherever a Kubernetes workload requires specific access
+to the Kubernetes API server.
+Modify the configuration of each default service account to include this value
+automountServiceAccountToken: false
+
+5.1.6 Modify the definition of pods and service accounts which do not need to mount service
+account tokens to disable it.
+
+5.2.1 Create a PSP as described in the Kubernetes documentation, ensuring that
+the .spec.privileged field is omitted or set to false.
+
+5.2.2 Create a PSP as described in the Kubernetes documentation, ensuring that the
+.spec.hostPID field is omitted or set to false.
+
+5.2.3 Create a PSP as described in the Kubernetes documentation, ensuring that the
+.spec.hostIPC field is omitted or set to false.
+
+5.2.4 Create a PSP as described in the Kubernetes documentation, ensuring that the
+.spec.hostNetwork field is omitted or set to false.
+
+5.2.5 Create a PSP as described in the Kubernetes documentation, ensuring that the
+.spec.allowPrivilegeEscalation field is omitted or set to false.
+
+5.2.6 Create a PSP as described in the Kubernetes documentation, ensuring that the
+.spec.runAsUser.rule is set to either MustRunAsNonRoot or MustRunAs with the range of
+UIDs not including 0.
+
+5.2.7 Create a PSP as described in the Kubernetes documentation, ensuring that the
+.spec.requiredDropCapabilities is set to include either NET_RAW or ALL.
+
+5.2.8 Ensure that allowedCapabilities is not present in PSPs for the cluster unless
+it is set to an empty array.
+
+5.2.9 Review the use of capabilites in applications runnning on your cluster. Where a namespace
+contains applicaions which do not require any Linux capabities to operate consider adding
+a PSP which forbids the admission of containers which do not drop all capabilities.
+
+5.3.1 If the CNI plugin in use does not support network policies, consideration should be given to
+making use of a different plugin, or finding an alternate mechanism for restricting traffic
+in the Kubernetes cluster.
+
+5.3.2 Follow the documentation and create NetworkPolicy objects as you need them.
+
+5.4.1 if possible, rewrite application code to read secrets from mounted secret files, rather than
+from environment variables.
+
+5.4.2 Refer to the secrets management options offered by your cloud provider or a third-party
+secrets management solution.
+
+5.5.1 Follow the Kubernetes documentation and setup image provenance.
+
+5.7.1 Follow the documentation and create namespaces for objects in your deployment as you need
+them.
+
+5.7.2 Seccomp is an alpha feature currently. By default, all alpha features are disabled. So, you
+would need to enable alpha features in the apiserver by passing "--feature-
+gates=AllAlpha=true" argument.
+Edit the /etc/kubernetes/apiserver file on the master node and set the KUBE_API_ARGS
+parameter to "--feature-gates=AllAlpha=true"
+KUBE_API_ARGS="--feature-gates=AllAlpha=true"
+Based on your system, restart the kube-apiserver service. For example:
+systemctl restart kube-apiserver.service
+Use annotations to enable the docker/default seccomp profile in your pod definitions. An
+example is as below:
+apiVersion: v1
+kind: Pod
+metadata:
+  name: trustworthy-pod
+  annotations:
+    seccomp.security.alpha.kubernetes.io/pod: docker/default
+spec:
+  containers:
+    - name: trustworthy-container
+      image: sotrustworthy:latest
+
+5.7.3 Follow the Kubernetes documentation and apply security contexts to your pods. For a
+suggested list of security contexts, you may refer to the CIS Security Benchmark for Docker
+Containers.
+
+5.7.4 Ensure that namespaces are created to allow for appropriate segregation of Kubernetes
+resources and that all new resources are created in a specific namespace.
+
+
+== Summary ==
+0 checks PASS
+0 checks FAIL
+24 checks WARN
+0 checks INFO