fixed bug in import loop

Fix typo

.github/workflows/publish.yml

@@ -77,9 +77,10 @@ jobs:
           python-version: '3.9'
 
       - name: Install dependencies
+        shell: bash
         run: |
-          python -m pip install -U pip
+          pip install -U pip
-          python -m pip install -r requirements-dev.txt
+          make deps
 
       - name: Build project
         shell: bash

.github/workflows/release.yml

@@ -10,7 +10,7 @@ name: Release
 jobs:
   build:
     name: Upload Release Asset
-    runs-on: ubuntu-16.04
+    runs-on: ubuntu-18.04
     steps:
       - name: Checkout code
         uses: actions/checkout@v2
@@ -18,12 +18,14 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v2
         with:
-          python-version: '3.9'
+          python-version: '3.8'
 
       - name: Install dependencies
+        shell: bash
         run: |
-          python -m pip install -U pip
+          pip install -U pip
-          python -m pip install -r requirements-dev.txt
+          pip install pyinstaller
+          make deps
 
       - name: Build project
         shell: bash

.github/workflows/test.yml

@@ -13,7 +13,7 @@ jobs:
       fail-fast: false
       matrix:
         python-version: ["3.6", "3.7", "3.8", "3.9"]
-        os: [ubuntu-20.04, ubuntu-18.04, ubuntu-16.04]
+        os: [ubuntu-20.04, ubuntu-18.04]
 
     steps:
       - uses: actions/checkout@v2
@@ -38,11 +38,11 @@ jobs:
             ${{ matrix.os }}-${{ matrix.python-version }}-
 
       - name: Install dependencies
+        shell: bash
         run: |
-          python -m pip install -U pip
+          pip install -U pip
-          python -m pip install -U wheel
+          make dev-deps
-          python -m pip install -r requirements.txt
+          make install
-          python -m pip install -r requirements-dev.txt
 
       - name: Test
         shell: bash

Makefile

@@ -31,7 +31,7 @@ lint-check:
 
 .PHONY: test
 test:
-	pytest
+	python -m pytest
 
 .PHONY: build
 build:

README.md

@@ -18,7 +18,8 @@ kube-hunter hunts for security weaknesses in Kubernetes clusters. The tool was d
 
 **Run kube-hunter**: kube-hunter is available as a container (aquasec/kube-hunter), and we also offer a web site at [kube-hunter.aquasec.com](https://kube-hunter.aquasec.com) where you can register online to receive a token allowing you to see and share the results online. You can also run the Python code yourself as described below.
 
-**Explore vulnerabilities**: The kube-hunter knowledge base includes articles about discoverable vulnerabilities and issues. When kube-hunter reports an issue, it will show its VID (Vulnerability ID) so you can look it up in the KB at https://aquasecurity.github.io/kube-hunter/
+**Explore vulnerabilities**: The kube-hunter knowledge base includes articles about discoverable vulnerabilities and issues. When kube-hunter reports an issue, it will show its VID (Vulnerability ID) so you can look it up in the KB at https://aquasecurity.github.io/kube-hunter/  
+_If you're interested in kube-hunter's integration with the Kubernetes ATT&CK Matrix [Continue Reading](#kuberentes-attck-matrix)_
 
 **Contribute**: We welcome contributions, especially new hunter modules that perform additional tests. If you would like to develop your modules please read [Guidelines For Developing Your First kube-hunter Module](https://github.com/aquasecurity/kube-hunter/blob/main/CONTRIBUTING.md).
 
@@ -28,6 +29,7 @@ Table of Contents
 =================
 
 - [Table of Contents](#table-of-contents)
+  - [Kubernetes ATT&CK Matrix](#kubernetes-attck-matrix)
   - [Hunting](#hunting)
     - [Where should I run kube-hunter?](#where-should-i-run-kube-hunter)
     - [Scanning options](#scanning-options)
@@ -37,8 +39,9 @@ Table of Contents
     - [Nodes Mapping](#nodes-mapping)
     - [Output](#output)
     - [Dispatching](#dispatching)
-    - [Advanced Usage](#advanced-usage)
+  - [Advanced Usage](#advanced-usage)
-      - [Azure Quick Scanning](#azure-quick-scanning)
+    - [Azure Quick Scanning](#azure-quick-scanning)
+    - [Custom Hunting](#custom-hunting)
   - [Deployment](#deployment)
     - [On Machine](#on-machine)
       - [Prerequisites](#prerequisites)
@@ -48,9 +51,21 @@ Table of Contents
     - [Pod](#pod)
   - [Contribution](#contribution)
   - [License](#license)
-         
-## Hunting
 
+---
+## Kubernetes ATT&CK Matrix
+
+kube-hunter now supports the new format of the Kubernetes ATT&CK matrix.
+While kube-hunter's vulnerabilities are a collection of creative techniques designed to mimic an attacker in the cluster (or outside it)
+The Mitre's ATT&CK defines a more general standardised categories of techniques to do so.
+
+You can think of kube-hunter vulnerabilities as small steps for an attacker, which follows the track of a more general technique he would aim for.
+Most of kube-hunter's hunters and vulnerabilities can closly fall under those techniques, That's why we moved to follow the Matrix standard.  
+ 
+_Some kube-hunter vulnerabities which we could not map to Mitre technique, are prefixed with the `General` keyword_ 
+![kube-hunter](./MITRE.png)
+
+## Hunting
 ### Where should I run kube-hunter?
 
 There are three different ways to run kube-hunter, each providing a different approach to detecting weaknesses in your cluster:
@@ -61,6 +76,7 @@ You can run kube-hunter directly on a machine in the cluster, and select the opt
 
 You can also run kube-hunter in a pod within the cluster. This indicates how exposed your cluster would be if one of your application pods is compromised (through a software vulnerability, for example). (_`--pod` flag_)
 
+
 ### Scanning options
 
 First check for these **[pre-requisites](#prerequisites)**.
@@ -141,11 +157,49 @@ Available dispatch methods are:
     * KUBEHUNTER_HTTP_DISPATCH_URL (defaults to: https://localhost)
     * KUBEHUNTER_HTTP_DISPATCH_METHOD (defaults to: POST)
 
-### Advanced Usage  
+
-#### Azure Quick Scanning 
+## Advanced Usage
+### Azure Quick Scanning 
 When running **as a Pod in an Azure or AWS environment**, kube-hunter will fetch subnets from the Instance Metadata Service. Naturally this makes the discovery process take longer.
 To hardlimit subnet scanning to a `/24` CIDR, use the `--quick` option. 
 
+### Custom Hunting
+Custom hunting enables advanced users to have control over what hunters gets registered at the start of a hunt. 
+**If you know what you are doing**, this can help if you want to adjust kube-hunter's hunting and discovery process for your needs.
+
+Example: 
+```
+kube-hunter --custom <HunterName1> <HunterName2>
+``` 
+Enabling Custom hunting removes all hunters from the hunting process, except the given whitelisted hunters.
+
+The `--custom` flag reads a list of hunters class names, in order to view all of kube-hunter's class names, you can combine the flag `--raw-hunter-names` with the `--list` flag.  
+
+Example: 
+```
+kube-hunter --active --list --raw-hunter-names
+```
+
+**Notice**: Due to kube-huner's architectural design, the following "Core Hunters/Classes" will always register (even when using custom hunting):
+* HostDiscovery 
+  * _Generates ip addresses for the hunt by given configurations_
+  * _Automatically discovers subnets using cloud Metadata APIs_
+* FromPodHostDiscovery
+  * _Auto discover attack surface ip addresses for the hunt by using Pod based environment techniques_
+  * _Automatically discovers subnets using cloud Metadata APIs_
+* PortDiscovery
+  * _Port scanning given ip addresses for known kubernetes services ports_
+* Collector
+  * _Collects discovered vulnerabilities and open services for future report_
+* StartedInfo 
+  * _Prints the start message_
+* SendFullReport 
+  * _Dispatching the report based on given configurations_
+
+
+
+
+
 ## Deployment
 There are three methods for deploying kube-hunter:
 

kube_hunter/__main__.py

@@ -1,6 +1,7 @@
 #!/usr/bin/env python3
 # flake8: noqa: E402
 
+from functools import partial
 import logging
 import threading
 
@@ -28,6 +29,8 @@ config = Config(
     k8s_auto_discover_nodes=args.k8s_auto_discover_nodes,
     service_account_token=args.service_account_token,
     kubeconfig=args.kubeconfig,
+    enable_cve_hunting=args.enable_cve_hunting,
+    custom=args.custom,
 )
 setup_logger(args.log, args.log_file)
 set_config(config)
@@ -72,16 +75,20 @@ def interactive_set_config():
     return True
 
 
-def list_hunters():
+def list_hunters(class_names=False):
     print("\nPassive Hunters:\n----------------")
     for hunter, docs in handler.passive_hunters.items():
         name, doc = hunter.parse_docs(docs)
+        if class_names:
+            name = hunter.__name__
         print(f"* {name}\n  {doc}\n")
 
     if config.active:
         print("\n\nActive Hunters:\n---------------")
         for hunter, docs in handler.active_hunters.items():
             name, doc = hunter.parse_docs(docs)
+            if class_names:
+                name = hunter.__name__
             print(f"* {name}\n  {doc}\n")
 
 
@@ -94,7 +101,10 @@ def main():
     scan_options = [config.pod, config.cidr, config.remote, config.interface, config.k8s_auto_discover_nodes]
     try:
         if args.list:
-            list_hunters()
+            if args.raw_hunter_names:
+                list_hunters(class_names=True)
+            else:
+                list_hunters()
             return
 
         if not any(scan_options):

kube_hunter/conf/__init__.py

@@ -1,7 +1,11 @@
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from typing import Any, Optional
 
 
+def get_default_core_hunters():
+    return ["FromPodHostDiscovery", "HostDiscovery", "PortDiscovery", "SendFullReport", "Collector", "StartedInfo"]
+
+
 @dataclass
 class Config:
     """Config is a configuration container.
@@ -21,6 +25,7 @@ class Config:
     - remote: Hosts to scan
     - report: Output format
     - statistics: Include hunters statistics
+    - enable_cve_hunting: enables cve hunting, shows cve results
     """
 
     active: bool = False
@@ -39,6 +44,10 @@ class Config:
     k8s_auto_discover_nodes: bool = False
     service_account_token: Optional[str] = None
     kubeconfig: Optional[str] = None
+    enable_cve_hunting: bool = False
+    custom: Optional[list] = None
+    raw_hunter_names: bool = False
+    core_hunters: list = field(default_factory=get_default_core_hunters)
 
 
 _config: Optional[Config] = None

kube_hunter/conf/parser.py

@@ -46,6 +46,22 @@ def parser_add_arguments(parser):
         help="One or more remote ip/dns to hunt",
     )
 
+    parser.add_argument(
+        "-c",
+        "--custom",
+        nargs="+",
+        metavar="HUNTERS",
+        default=list(),
+        help="Custom hunting. Only given hunter names will register in the hunt."
+        "for a list of options run `--list --raw-hunter-names`",
+    )
+
+    parser.add_argument(
+        "--raw-hunter-names",
+        action="store_true",
+        help="Use in combination with `--list` to display hunter class names to pass for custom hunting flag",
+    )
+
     parser.add_argument(
         "--k8s-auto-discover-nodes",
         action="store_true",
@@ -76,6 +92,12 @@ def parser_add_arguments(parser):
 
     parser.add_argument("--active", action="store_true", help="Enables active hunting")
 
+    parser.add_argument(
+        "--enable-cve-hunting",
+        action="store_true",
+        help="Show cluster CVEs based on discovered version (Depending on different vendors, may result in False Positives)",
+    )
+
     parser.add_argument(
         "--log",
         type=str,

kube_hunter/core/events/handler.py

@@ -62,7 +62,7 @@ class EventQueue(Queue):
     ######################################################
     """
 
-    def subscribe(self, event, hook=None, predicate=None):
+    def subscribe(self, event, hook=None, predicate=None, is_register=True):
         """
         The Subscribe Decorator - For Regular Registration
         Use this to register for one event only. Your hunter will execute each time this event is published
@@ -74,12 +74,12 @@ class EventQueue(Queue):
         """
 
         def wrapper(hook):
-            self.subscribe_event(event, hook=hook, predicate=predicate)
+            self.subscribe_event(event, hook=hook, predicate=predicate, is_register=is_register)
             return hook
 
         return wrapper
 
-    def subscribe_many(self, events, hook=None, predicates=None):
+    def subscribe_many(self, events, hook=None, predicates=None, is_register=True):
         """
         The Subscribe Many Decorator - For Multiple Registration,
         When your attack needs several prerequisites to exist in the cluster, You need to register for multiple events.
@@ -99,12 +99,12 @@ class EventQueue(Queue):
         """
 
         def wrapper(hook):
-            self.subscribe_events(events, hook=hook, predicates=predicates)
+            self.subscribe_events(events, hook=hook, predicates=predicates, is_register=is_register)
             return hook
 
         return wrapper
 
-    def subscribe_once(self, event, hook=None, predicate=None):
+    def subscribe_once(self, event, hook=None, predicate=None, is_register=True):
         """
         The Subscribe Once Decorator - For Single Trigger Registration,
         Use this when you want your hunter to execute only in your entire program run
@@ -125,7 +125,8 @@ class EventQueue(Queue):
 
             hook.__new__ = __new__unsubscribe_self
 
-            self.subscribe_event(event, hook=hook, predicate=predicate)
+            self.subscribe_event(event, hook=hook, predicate=predicate, is_register=is_register)
+
             return hook
 
         return wrapper
@@ -256,7 +257,33 @@ class EventQueue(Queue):
             self.hooks[event].append((hook, predicate))
             logging.debug("{} subscribed to {}".format(hook, event))
 
-    def subscribe_event(self, event, hook=None, predicate=None):
+    def allowed_for_custom_registration(self, target_hunter):
+        """
+        Check if the partial input list contains the hunter we are about to register for events
+        If hunter is considered a Core hunter as specified in `config.core_hunters` we allow it anyway
+
+        Returns true if:
+         1. partial hunt is disabled
+         2. partial hunt is enabled and hunter is in core hunter class
+         3. partial hunt is enabled and hunter is specified in config.partial
+
+        @param target_hunter: hunter class for registration check
+        """
+        config = get_config()
+        if not config.custom:
+            return True
+
+        hunter_class_name = target_hunter.__name__
+        if hunter_class_name in config.core_hunters or hunter_class_name in config.custom:
+            return True
+
+        return False
+
+    def subscribe_event(self, event, hook=None, predicate=None, is_register=True):
+        if not is_register:
+            return
+        if not self.allowed_for_custom_registration(hook):
+            return
         if not self._register_hunters(hook):
             return
 
@@ -267,9 +294,13 @@ class EventQueue(Queue):
         else:
             self._register_hook(event, hook, predicate)
 
-    def subscribe_events(self, events, hook=None, predicates=None):
+    def subscribe_events(self, events, hook=None, predicates=None, is_register=True):
+        if not is_register:
+            return
+        if not self.allowed_for_custom_registration(hook):
+            return
         if not self._register_hunters(hook):
-            return False
+            return
 
         if predicates is None:
             predicates = [None] * len(events)

kube_hunter/modules/discovery/hosts.py

@@ -166,7 +166,9 @@ class FromPodHostDiscovery(Discovery):
                 return True
         except requests.exceptions.ConnectionError:
             logger.debug("Failed to connect AWS metadata server v1")
-            return False
+        except Exception:
+            logger.debug("Unknown error when trying to connect to AWS metadata v1 API")
+        return False
 
     def is_aws_pod_v2(self):
         config = get_config()
@@ -189,7 +191,9 @@ class FromPodHostDiscovery(Discovery):
                 return True
         except requests.exceptions.ConnectionError:
             logger.debug("Failed to connect AWS metadata server v2")
-            return False
+        except Exception:
+            logger.debug("Unknown error when trying to connect to AWS metadata v2 API")
+        return False
 
     def is_azure_pod(self):
         config = get_config()
@@ -206,7 +210,9 @@ class FromPodHostDiscovery(Discovery):
                 return True
         except requests.exceptions.ConnectionError:
             logger.debug("Failed to connect Azure metadata server")
-            return False
+        except Exception:
+            logger.debug("Unknown error when trying to connect to Azure metadata server")
+        return False
 
     # for pod scanning
     def gateway_discovery(self):

kube_hunter/modules/hunting/cves.py

@@ -3,7 +3,8 @@ from packaging import version
 
 from kube_hunter.conf import get_config
 from kube_hunter.core.events import handler
-from kube_hunter.core.events.types import Vulnerability, Event, K8sVersionDisclosure
+
+from kube_hunter.core.events.types import K8sVersionDisclosure, Vulnerability, Event
 from kube_hunter.core.types import (
     Hunter,
     KubectlClient,
@@ -15,6 +16,7 @@ from kube_hunter.core.types import (
 from kube_hunter.modules.discovery.kubectl import KubectlClientEvent
 
 logger = logging.getLogger(__name__)
+config = get_config()
 
 
 class ServerApiVersionEndPointAccessPE(Vulnerability, Event):
@@ -199,7 +201,7 @@ class CveUtils:
         return vulnerable
 
 
-@handler.subscribe_once(K8sVersionDisclosure)
+@handler.subscribe_once(K8sVersionDisclosure, is_register=config.enable_cve_hunting)
 class K8sClusterCveHunter(Hunter):
     """K8s CVE Hunter
     Checks if Node is running a Kubernetes version vulnerable to
@@ -224,6 +226,7 @@ class K8sClusterCveHunter(Hunter):
                 self.publish_event(vulnerability(self.event.version))
 
 
+# Removed due to incomplete implementation for multiple vendors revisions of kubernetes
 @handler.subscribe(KubectlClientEvent)
 class KubectlCVEHunter(Hunter):
     """Kubectl CVE Hunter

kube_hunter/modules/report/dispatchers.py

@@ -12,10 +12,7 @@ class HTTPDispatcher:
         dispatch_url = os.environ.get("KUBEHUNTER_HTTP_DISPATCH_URL", "https://localhost/")
         try:
             r = requests.request(
-                dispatch_method,
+                dispatch_method, dispatch_url, json=report, headers={"Content-Type": "application/json"}, verify=False
-                dispatch_url,
-                json=report,
-                headers={"Content-Type": "application/json"},
             )
             r.raise_for_status()
             logger.info(f"Report was dispatched to: {dispatch_url}")

requirements-dev.txt

@@ -1,5 +1,3 @@
--r requirements.txt
-
 flake8
 pytest >= 2.9.1
 requests-mock >= 1.8

tests/core/test_handler.py

@@ -1,6 +1,6 @@
 # flake8: noqa: E402
 
-from kube_hunter.conf import Config, set_config
+from kube_hunter.conf import Config, set_config, get_config
 
 set_config(Config(active=True))
 
@@ -23,7 +23,9 @@ from kube_hunter.modules.hunting.apiserver import (
 from kube_hunter.modules.hunting.arp import ArpSpoofHunter
 from kube_hunter.modules.hunting.capabilities import PodCapabilitiesHunter
 from kube_hunter.modules.hunting.certificates import CertificateDiscovery
-from kube_hunter.modules.hunting.cves import K8sClusterCveHunter, KubectlCVEHunter
+
+from kube_hunter.modules.hunting.cves import K8sClusterCveHunter
+from kube_hunter.modules.hunting.cves import KubectlCVEHunter
 from kube_hunter.modules.hunting.dashboard import KubeDashboard
 from kube_hunter.modules.hunting.dns import DnsSpoofHunter
 from kube_hunter.modules.hunting.etcd import EtcdRemoteAccess, EtcdRemoteAccessActive
@@ -40,6 +42,8 @@ from kube_hunter.modules.hunting.mounts import VarLogMountHunter, ProveVarLogMou
 from kube_hunter.modules.hunting.proxy import KubeProxy, ProveProxyExposed, K8sVersionDisclosureProve
 from kube_hunter.modules.hunting.secrets import AccessSecrets
 
+config = get_config()
+
 PASSIVE_HUNTERS = {
     ApiServiceDiscovery,
     KubeDashboardDiscovery,
@@ -56,7 +60,6 @@ PASSIVE_HUNTERS = {
     ApiVersionHunter,
     PodCapabilitiesHunter,
     CertificateDiscovery,
-    K8sClusterCveHunter,
     KubectlCVEHunter,
     KubeDashboard,
     EtcdRemoteAccess,
@@ -67,6 +70,9 @@ PASSIVE_HUNTERS = {
     AccessSecrets,
 }
 
+# if config.enable_cve_hunting:
+#     PASSIVE_HUNTERS.append(K8sClusterCveHunter)
+
 ACTIVE_HUNTERS = {
     ProveAzureSpnExposure,
     AccessApiServerActive,