Merge pull request #206 from westonsteimel/no_runningversion_if_version_set

Only get runningVersion if --version has not been provided

NOTICE

@@ -0,0 +1,5 @@
+kube-bench
+Copyright 2017-2019 Aqua Security Software Ltd. 
+
+This product includes software developed by Aqua Security (https://aquasec.com).
+

README.md

@@ -13,7 +13,7 @@ Tests are configured with YAML files, making this tool easy to update as test sp
 
 ## CIS Kubernetes Benchmark support
 
-kube-bench supports the tests for multiple versions of Kubernetes (1.6, 1.7, 1.8, and 1.11) as defined in the CIS Benchmarks 1.0.0, 1.1.0, 1.2.0, and 1.30 respectively. It will determine the test set to run based on the Kubernetes version running on the machine.
+kube-bench supports the tests for multiple versions of Kubernetes (1.6, 1.7, 1.8, and 1.11) as defined in the CIS Benchmarks 1.0.0, 1.1.0, 1.2.0, and 1.3.0 respectively. It will determine the test set to run based on the Kubernetes version running on the machine.
 
 ## Installation
 
@@ -25,36 +25,54 @@ You can choose to
 
 ### Running inside a container
 
-You can avoid installing kube-bench on the host by running it inside a container using the host PID namespace.
+You can avoid installing kube-bench on the host by running it inside a container using the host PID namespace and mounting the `/etc` and `/var` directories where the configuration and other files are located on the host, so that kube-bench can check their existence and permissions.
 
 ```
-docker run --pid=host -t aquasec/kube-bench:latest <master|node>
+docker run --pid=host -v /etc:/etc:ro -v /var:/var:ro -t aquasec/kube-bench:latest <master|node>
 ```
 
 You can even use your own configs by mounting them over the default ones in `/opt/kube-bench/cfg/`
 
 ```
-docker run --pid=host -t -v path/to/my-config.yaml:/opt/kube-bench/cfg/config.yaml aquasec/kube-bench:latest <master|node>
+docker run --pid=host -v /etc:/etc:ro -v /var:/var:ro -t -v path/to/my-config.yaml:/opt/kube-bench/cfg/config.yaml aquasec/kube-bench:latest <master|node>
 ```
 
 > Note: the tests require either the kubelet or kubectl binary in the path in order to know the Kubernetes version. You can pass `-v $(which kubectl):/usr/bin/kubectl` to the above invocations to resolve this.
 
 ### Running in a kubernetes cluster
-Run the master check
 
-```
-kubectl run --rm -i -t kube-bench-master --image=aquasec/kube-bench:latest --restart=Never --overrides="{ \"apiVersion\": \"v1\", \"spec\": { \"hostPID\": true, \"nodeSelector\": { \"kubernetes.io/role\": \"master\" }, \"tolerations\": [ { \"key\": \"node-role.kubernetes.io/master\", \"operator\": \"Exists\", \"effect\": \"NoSchedule\" } ] } }" -- master --version 1.11
+You can run kube-bench inside a pod, but it will need access to the host's PID namespace in order to check the running processes, as well as access to some directories on the host where config files and other files are stored. 
+
+To run the tests on the master node, the pod needs to be scheduled on that node. This involves setting a nodeSelector and tolerations in the pod spec.
+
+The supplied `job-node.yaml` and `job-master.yaml` files can be applied to run the tests as a job. For example:
+
+```bash
+$ kubectl apply -f job-master.yaml 
+job.batch/kube-bench-master created
+
+$ kubectl get pods
+NAME                      READY   STATUS              RESTARTS   AGE
+kube-bench-master-j76s9   0/1     ContainerCreating   0          3s
+
+# Wait for a few seconds for the job to complete
+$ kubectl get pods
+NAME                      READY   STATUS      RESTARTS   AGE
+kube-bench-master-j76s9   0/1     Completed   0          11s
+
+# The results are held in the pod's logs
+k logs kube-bench-master-j76s9
+[INFO] 1 Master Node Security Configuration
+[INFO] 1.1 API Server
+...
 ```
 
-Run the node check
-
-```
-kubectl run --rm -i -t kube-bench-node --image=aquasec/kube-bench:latest --restart=Never --overrides="{ \"apiVersion\": \"v1\", \"spec\": { \"hostPID\": true } }" -- node --version 1.11
-```
+The default labels applied to master nodes has changed since Kubernetes 1.11, so if you are using an older version you may need to modify the nodeSelector and tolerations to run the job on the master node.
 
 ### Installing from a container
 
 This command copies the kube-bench binary and configuration files to your host from the Docker container:
+** binaries compiled for linux-x86-64 only (so they won't run on OSX or Windows) **
 ```
 docker run --rm -v `pwd`:/host aquasec/kube-bench:latest install
 ```
@@ -81,6 +99,7 @@ go build -o kube-bench .
 ```
 
 ## Configuration
+
 Kubernetes config and binary file locations and names can vary from installation to installation, so these are configurable in the `cfg/config.yaml` file.
 
 For each type of node (*master*, *node* or *federated*) there is a list of components, and for each component there is a set of binaries (*bins*) and config files (*confs*) that kube-bench will look for (in the order they are listed). If your installation uses a different binary name or config file location for a Kubernetes component, you can add it to `cfg/config.yaml`.

cfg/1.11/config.yaml

@@ -9,39 +9,21 @@
 
 master:
   apiserver:
-    confs:
-      - /etc/kubernetes/manifests/kube-apiserver.yaml
-      - /etc/kubernetes/manifests/kube-apiserver.manifest
     defaultconf: /etc/kubernetes/manifests/kube-apiserver.yaml
 
   scheduler:
-    confs:
-      - /etc/kubernetes/manifests/kube-scheduler.yaml
-      - /etc/kubernetes/manifests/kube-scheduler.manifest
     defaultconf: /etc/kubernetes/manifests/kube-scheduler.yaml
 
   controllermanager:
-    confs:
-      - /etc/kubernetes/manifests/kube-controller-manager.yaml
-      - /etc/kubernetes/manifests/kube-controller-manager.manifest
     defaultconf: /etc/kubernetes/manifests/kube-controller-manager.yaml
 
   etcd:
-    confs:
-      - /etc/kubernetes/manifests/etcd.yaml
-      - /etc/kubernetes/manifests/etcd.manifest
     defaultconf: /etc/kubernetes/manifests/etcd.yaml
 
 node:
   kubelet:
-    confs:
-     - /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
-     - /etc/kubernetes/kubelet.conf
-    defaultconf: /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
+    defaultconf: /etc/kubernetes/kubelet.conf
+    defaultsvc: /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
 
   proxy:
-    confs:
-      - /etc/kubernetes/addons/kube-proxy-daemonset.yaml
     defaultconf: /etc/kubernetes/addons/kube-proxy-daemonset.yaml
-
-

cfg/1.11/master.yaml

@@ -179,7 +179,7 @@ groups:
       Edit the API server pod specification file $apiserverconf
       on the master node and set the --enable-admission-plugins to
       include AlwaysPullImages.
-      --enable-admission-plugins...,AlwaysPullImages,...
+      --enable-admission-plugins=...,AlwaysPullImages,...
     scored: true
 
   - id: 1.1.12
@@ -220,12 +220,15 @@ groups:
     text: "Ensure that the admission control plugin NamespaceLifecycle is set (Scored)"
     audit: "ps -ef | grep $apiserverbin | grep -v grep"
     tests:
+      bin_op: or
       test_items:
       - flag: "--disable-admission-plugins"
         compare:
-          op: has
+          op: nothave
           value: "NamespaceLifecycle"
         set: true
+      - flag: "--disable-admission-plugins"
+        set: false
     remediation: |
       Edit the API server pod specification file $apiserverconf
       on the master node and set the --disable-admission-plugins parameter to
@@ -496,7 +499,7 @@ groups:
     remediation: |
       Edit the API server pod specification file $apiserverconf
       on the master node and set the below parameter.
-      --tls-cipher- suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM _SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM _SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM _SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+      --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
     scored: false
 
   - id: 1.1.31
@@ -607,15 +610,15 @@ groups:
     text: "Ensure that the AdvancedAuditing argument is not set to false (Scored)"
     audit: "ps -ef | grep $apiserverbin | grep -v grep"
     tests:
-      bin_op: and
+      bin_op: or
       test_items:
       - flag: "--feature-gates"
         compare:
           op: nothave
           value: "AdvancedAuditing=false"
         set: true
-      - flag: "--audit-policy-file"
-        set: true
+      - flag: "--feature-gates"
+        set: false
     remediation: |
       Follow the Kubernetes documentation and set the desired audit policy in the
       /etc/kubernetes/audit-policy.yaml file. Then, edit the API server pod specification file $apiserverconf
@@ -790,6 +793,7 @@ groups:
     text: "Ensure that the --address argument is set to 127.0.0.1 (Scored)"
     audit: "ps -ef | grep $controllermanagerbin | grep -v grep"
     tests:
+      bin_op: or
       test_items:
       - flag: "--address"
         compare:
@@ -1421,7 +1425,7 @@ groups:
     scored: false
 
   - id: 1.7.5
-    text: " Do not admit containers with allowPrivilegeEscalation (Not Scored)"
+    text: "Do not admit containers with allowPrivilegeEscalation (Not Scored)"
     type: "manual"
     remediation: |
       Create a PSP as described in the Kubernetes documentation, ensuring that the .spec.allowPrivilegeEscalation field is omitted or set to false.

cfg/1.11/node.yaml

@@ -164,6 +164,8 @@ groups:
           op: eq
           value: true
         set: true
+      - flag: "--make-iptables-util-chains"
+        set: false
     remediation: |
       If using a Kubelet config file, edit the file to set makeIPTablesUtilChains: true .
       If using command line arguments, edit the kubelet service file
@@ -309,7 +311,7 @@ groups:
     remediation: |
       If using a Kubelet config file, edit the file to set TLSCipherSuites: to TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 ,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 ,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 ,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
       If using executable arguments, edit the kubelet service file $kubeletconf on each worker node and set the below parameter.
-      --tls-cipher- suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM _SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM _SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM _SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
+      --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
     scored: false
 
 - id: 2.2
@@ -362,7 +364,7 @@ groups:
     - id: 2.2.3
       text: "Ensure that the kubelet service file permissions are set to 644 or
       more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %a $kubeletconf; fi'"
+      audit: "/bin/sh -c 'if test -e $kubeletsvc; then stat -c %a $kubeletsvc; fi'"
       tests:
         bin_op: or
         test_items:
@@ -384,12 +386,12 @@ groups:
       remediation: |
         Run the below command (based on the file location on your system) on the each worker
         node. For example,
-        chmod 755 $kubeletconf
+        chmod 755 $kubeletsvc
       scored: true
 
     - id: 2.2.4
       text: "Ensure that the kubelet service file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %U:%G $kubeletconf; fi'"
+      audit: "/bin/sh -c 'if test -e $kubeletsvc; then stat -c %U:%G $kubeletsvc; fi'"
       tests:
         test_items:
         - flag: "root:root"
@@ -397,7 +399,7 @@ groups:
       remediation: |
         Run the below command (based on the file location on your system) on the each worker
         node. For example,
-        chown root:root $kubeletconf
+        chown root:root $kubeletsvc
       scored: true
 
     - id: 2.2.5
@@ -460,7 +462,7 @@ groups:
 
     - id: 2.2.9
       text: "Ensure that the kubelet configuration file ownership is set to root:root (Scored)"
-      audit: "/bin/sh -c 'if test -e $/var/lib/kubelet/config.yaml; then stat -c %U:%G $/var/lib/kubelet/config.yaml; fi'"
+      audit: "/bin/sh -c 'if test -e /var/lib/kubelet/config.yaml; then stat -c %U:%G /var/lib/kubelet/config.yaml; fi'"
       tests:
         test_items:
         - flag: "root:root"
@@ -472,7 +474,7 @@ groups:
 
     - id: 2.2.10
       text: "Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)"
-      audit: "/bin/sh -c 'if test -e $/var/lib/kubelet/config.yaml; then stat -c %a $/var/lib/kubelet/config.yaml; fi'"
+      audit: "/bin/sh -c 'if test -e /var/lib/kubelet/config.yaml; then stat -c %a /var/lib/kubelet/config.yaml; fi'"
       tests:
         bin_op: or
         test_items:

cmd/common.go

@@ -44,9 +44,12 @@ func runChecks(nodetype check.NodeType) {
 		file = federatedFile
 	}
 
-	runningVersion, err := getKubeVersion()
-	if err != nil && kubeVersion == "" {
-		exitWithError(fmt.Errorf("Version check failed: %s\nAlternatively, you can specify the version with --version", err))
+	runningVersion := ""
+	if kubeVersion == "" {
+		runningVersion, err = getKubeVersion()
+		if err != nil {
+			exitWithError(fmt.Errorf("Version check failed: %s\nAlternatively, you can specify the version with --version", err))
+		}
 	}
 	path, err := getConfigFilePath(kubeVersion, runningVersion, file)
 	if err != nil {
@@ -79,11 +82,13 @@ func runChecks(nodetype check.NodeType) {
 	typeConf = viper.Sub(string(nodetype))
 	binmap := getBinaries(typeConf)
 	confmap := getConfigFiles(typeConf)
+	svcmap := getServiceFiles(typeConf)
 
 	// Variable substitutions. Replace all occurrences of variables in controls files.
 	s := string(in)
 	s = makeSubstitutions(s, "bin", binmap)
 	s = makeSubstitutions(s, "conf", confmap)
+	s = makeSubstitutions(s, "svc", svcmap)
 
 	controls, err := check.NewControls(nodetype, []byte(s))
 	if err != nil {

cmd/util.go

@@ -172,8 +172,6 @@ func decrementVersion(version string) string {
 }
 
 // getConfigFiles finds which of the set of candidate config files exist
-// accepts a string 't' which indicates the type of config file, conf,
-// podspec or untifile.
 func getConfigFiles(v *viper.Viper) map[string]string {
 	confmap := make(map[string]string)
 
@@ -204,6 +202,37 @@ func getConfigFiles(v *viper.Viper) map[string]string {
 	return confmap
 }
 
+// getServiceFiles finds which of the set of candidate service files exist
+func getServiceFiles(v *viper.Viper) map[string]string {
+	svcmap := make(map[string]string)
+
+	for _, component := range v.GetStringSlice("components") {
+		s := v.Sub(component)
+		if s == nil {
+			continue
+		}
+
+		// See if any of the candidate config files exist
+		svc := findConfigFile(s.GetStringSlice("svc"))
+		if svc == "" {
+			if s.IsSet("defaultsvc") {
+				svc = s.GetString("defaultsvc")
+				glog.V(2).Info(fmt.Sprintf("Using default service file name '%s' for component %s", svc, component))
+			} else {
+				// Default the service file name that we'll substitute to the name of the component
+				glog.V(2).Info(fmt.Sprintf("Missing service file for %s", component))
+				svc = component
+			}
+		} else {
+			glog.V(2).Info(fmt.Sprintf("Component %s uses service file '%s'", component, svc))
+		}
+
+		svcmap[component] = svc
+	}
+
+	return svcmap
+}
+
 // verifyBin checks that the binary specified is running
 func verifyBin(bin string) bool {
 

cmd/util_test.go

@@ -289,6 +289,81 @@ func TestGetConfigFiles(t *testing.T) {
 	}
 }
 
+func TestGetServiceFiles(t *testing.T) {
+	cases := []struct {
+		config      map[string]interface{}
+		exp         map[string]string
+		statResults []error
+	}{
+		{
+			config: map[string]interface{}{
+				"components": []string{"kubelet"},
+				"kubelet":    map[string]interface{}{"svc": []string{"kubelet", "10-kubeadm.conf"}},
+			},
+			statResults: []error{os.ErrNotExist, nil},
+			exp:         map[string]string{"kubelet": "10-kubeadm.conf"},
+		},
+		{
+			// Component "thing" isn't included in the list of components
+			config: map[string]interface{}{
+				"components": []string{"kubelet"},
+				"kubelet":    map[string]interface{}{"svc": []string{"kubelet", "10-kubeadm.conf"}},
+				"thing":      map[string]interface{}{"svc": []string{"/my/file/thing"}},
+			},
+			statResults: []error{os.ErrNotExist, nil},
+			exp:         map[string]string{"kubelet": "10-kubeadm.conf"},
+		},
+		{
+			// More than one component
+			config: map[string]interface{}{
+				"components": []string{"kubelet", "thing"},
+				"kubelet":    map[string]interface{}{"svc": []string{"kubelet", "10-kubeadm.conf"}},
+				"thing":      map[string]interface{}{"svc": []string{"/my/file/thing"}},
+			},
+			statResults: []error{os.ErrNotExist, nil, nil},
+			exp:         map[string]string{"kubelet": "10-kubeadm.conf", "thing": "/my/file/thing"},
+		},
+		{
+			// Default thing to specified default service
+			config: map[string]interface{}{
+				"components": []string{"kubelet", "thing"},
+				"kubelet":    map[string]interface{}{"svc": []string{"kubelet", "10-kubeadm.conf"}},
+				"thing":      map[string]interface{}{"svc": []string{"/my/file/thing"}, "defaultsvc": "another/thing"},
+			},
+			statResults: []error{os.ErrNotExist, nil, os.ErrNotExist},
+			exp:         map[string]string{"kubelet": "10-kubeadm.conf", "thing": "another/thing"},
+		},
+		{
+			// Default thing to component name
+			config: map[string]interface{}{
+				"components": []string{"kubelet", "thing"},
+				"kubelet":    map[string]interface{}{"svc": []string{"kubelet", "10-kubeadm.conf"}},
+				"thing":      map[string]interface{}{"svc": []string{"/my/file/thing"}},
+			},
+			statResults: []error{os.ErrNotExist, nil, os.ErrNotExist},
+			exp:         map[string]string{"kubelet": "10-kubeadm.conf", "thing": "thing"},
+		},
+	}
+
+	v := viper.New()
+	statFunc = fakestat
+
+	for id, c := range cases {
+		t.Run(strconv.Itoa(id), func(t *testing.T) {
+			for k, val := range c.config {
+				v.Set(k, val)
+			}
+			e = c.statResults
+			eIndex = 0
+
+			m := getServiceFiles(v)
+			if !reflect.DeepEqual(m, c.exp) {
+				t.Fatalf("Got %v\nExpected %v", m, c.exp)
+			}
+		})
+	}
+}
+
 func TestMakeSubsitutions(t *testing.T) {
 	cases := []struct {
 		input string

job-master.yaml

@@ -0,0 +1,38 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: kube-bench-master
+spec:
+  template:
+    spec:
+      hostPID: true
+      nodeSelector: 
+        node-role.kubernetes.io/master: "" 
+      tolerations:
+      - key: node-role.kubernetes.io/master
+        operator: Exists
+        effect: NoSchedule
+      containers:
+      - name: kube-bench
+        image: aquasec/kube-bench:latest
+        command: ["kube-bench","master"]
+        volumeMounts:
+        - name: var-lib-etcd
+          mountPath: /var/lib/etcd
+        - name: etc-kubernetes
+          mountPath: /etc/kubernetes
+          # /usr/bin is mounted to access kubectl / kubelet, for auto-detecting the Kubernetes version. 
+          # You can omit this mount if you specify --version as part of the command.
+        - name: usr-bin
+          mountPath: /usr/bin
+      restartPolicy: Never
+      volumes:
+      - name: var-lib-etcd
+        hostPath:
+          path: "/var/lib/etcd"
+      - name: etc-kubernetes
+        hostPath:
+          path: "/etc/kubernetes"
+      - name: usr-bin
+        hostPath:
+          path: "/usr/bin"

job-node.yaml

@@ -0,0 +1,37 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: kube-bench-node
+spec:
+  template:
+    spec:
+      hostPID: true
+      containers:
+      - name: kube-bench
+        image: aquasec/kube-bench:latest
+        command: ["kube-bench","node"]
+        volumeMounts:
+        - name: var-lib-kubelet
+          mountPath: /var/lib/kubelet
+        - name: etc-systemd
+          mountPath: /etc/systemd
+        - name: etc-kubernetes
+          mountPath: /etc/kubernetes
+          # /usr/bin is mounted to access kubectl / kubelet, for auto-detecting the Kubernetes version. 
+          # You can omit this mount if you specify --version as part of the command.           
+        - name: usr-bin
+          mountPath: /usr/bin
+      restartPolicy: Never
+      volumes:
+      - name: var-lib-kubelet
+        hostPath:
+          path: "/var/lib/kubelet"
+      - name: etc-systemd
+        hostPath:
+          path: "/etc/systemd"
+      - name: etc-kubernetes
+        hostPath:
+          path: "/etc/kubernetes"
+      - name: usr-bin
+        hostPath:
+          path: "/usr/bin"