ignore E402 flake8 on test_cloud

* removed arp and dns hunters usage due to it's violations of the scapy GPL2 license

* added installation of arp and dns hunters to Dockerfile

* added explicit new version to plugins in dockerfile installation

* ignore B020 flake8

.flake8

@@ -1,5 +1,5 @@
 [flake8]
-ignore = E203, E266, E501, W503, B903, T499
+ignore = E203, E266, E501, W503, B903, T499, B020
 max-line-length = 120
 max-complexity = 18
 select = B,C,E,F,W,B9,T4

.github/workflows/publish.yml

@@ -77,9 +77,10 @@ jobs:
           python-version: '3.9'
 
       - name: Install dependencies
+        shell: bash
         run: |
-          python -m pip install -U pip
-          python -m pip install -r requirements-dev.txt
+          pip install -U pip
+          make deps
 
       - name: Build project
         shell: bash

.github/workflows/release.yml

@@ -10,7 +10,7 @@ name: Release
 jobs:
   build:
     name: Upload Release Asset
-    runs-on: ubuntu-16.04
+    runs-on: ubuntu-18.04
     steps:
       - name: Checkout code
         uses: actions/checkout@v2
@@ -18,12 +18,14 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v2
         with:
-          python-version: '3.9'
+          python-version: '3.8'
 
       - name: Install dependencies
+        shell: bash
         run: |
-          python -m pip install -U pip
-          python -m pip install -r requirements-dev.txt
+          pip install -U pip
+          pip install pyinstaller
+          make deps
 
       - name: Build project
         shell: bash

.github/workflows/test.yml

@@ -13,7 +13,7 @@ jobs:
       fail-fast: false
       matrix:
         python-version: ["3.6", "3.7", "3.8", "3.9"]
-        os: [ubuntu-20.04, ubuntu-18.04, ubuntu-16.04]
+        os: [ubuntu-20.04, ubuntu-18.04]
 
     steps:
       - uses: actions/checkout@v2
@@ -38,11 +38,11 @@ jobs:
             ${{ matrix.os }}-${{ matrix.python-version }}-
 
       - name: Install dependencies
+        shell: bash
         run: |
-          python -m pip install -U pip
-          python -m pip install -U wheel
-          python -m pip install -r requirements.txt
-          python -m pip install -r requirements-dev.txt
+          pip install -U pip
+          make dev-deps
+          make install
 
       - name: Test
         shell: bash

Dockerfile

@@ -26,4 +26,7 @@ RUN apk add --no-cache \
 COPY --from=builder /usr/local/lib/python3.8/site-packages /usr/local/lib/python3.8/site-packages
 COPY --from=builder /usr/local/bin/kube-hunter /usr/local/bin/kube-hunter
 
+# Add default plugins: https://github.com/aquasecurity/kube-hunter-plugins 
+RUN pip install kube-hunter-arp-spoof>=0.0.3 kube-hunter-dns-spoof>=0.0.3
+
 ENTRYPOINT ["kube-hunter"]

Makefile

@@ -31,7 +31,7 @@ lint-check:
 
 .PHONY: test
 test:
-	pytest
+	python -m pytest
 
 .PHONY: build
 build:

README.md

@@ -18,7 +18,8 @@ kube-hunter hunts for security weaknesses in Kubernetes clusters. The tool was d
 
 **Run kube-hunter**: kube-hunter is available as a container (aquasec/kube-hunter), and we also offer a web site at [kube-hunter.aquasec.com](https://kube-hunter.aquasec.com) where you can register online to receive a token allowing you to see and share the results online. You can also run the Python code yourself as described below.
 
-**Explore vulnerabilities**: The kube-hunter knowledge base includes articles about discoverable vulnerabilities and issues. When kube-hunter reports an issue, it will show its VID (Vulnerability ID) so you can look it up in the KB at https://aquasecurity.github.io/kube-hunter/
+**Explore vulnerabilities**: The kube-hunter knowledge base includes articles about discoverable vulnerabilities and issues. When kube-hunter reports an issue, it will show its VID (Vulnerability ID) so you can look it up in the KB at https://aquasecurity.github.io/kube-hunter/  
+_If you're interested in kube-hunter's integration with the Kubernetes ATT&CK Matrix [Continue Reading](#kuberentes-attck-matrix)_
 
 **Contribute**: We welcome contributions, especially new hunter modules that perform additional tests. If you would like to develop your modules please read [Guidelines For Developing Your First kube-hunter Module](https://github.com/aquasecurity/kube-hunter/blob/main/CONTRIBUTING.md).
 
@@ -28,6 +29,7 @@ Table of Contents
 =================
 
 - [Table of Contents](#table-of-contents)
+  - [Kubernetes ATT&CK Matrix](#kubernetes-attck-matrix)
   - [Hunting](#hunting)
     - [Where should I run kube-hunter?](#where-should-i-run-kube-hunter)
     - [Scanning options](#scanning-options)
@@ -37,8 +39,9 @@ Table of Contents
     - [Nodes Mapping](#nodes-mapping)
     - [Output](#output)
     - [Dispatching](#dispatching)
-    - [Advanced Usage](#advanced-usage)
-      - [Azure Quick Scanning](#azure-quick-scanning)
+  - [Advanced Usage](#advanced-usage)
+    - [Azure Quick Scanning](#azure-quick-scanning)
+    - [Custom Hunting](#custom-hunting)
   - [Deployment](#deployment)
     - [On Machine](#on-machine)
       - [Prerequisites](#prerequisites)
@@ -48,9 +51,21 @@ Table of Contents
     - [Pod](#pod)
   - [Contribution](#contribution)
   - [License](#license)
-         
-## Hunting
 
+---
+## Kubernetes ATT&CK Matrix
+
+kube-hunter now supports the new format of the Kubernetes ATT&CK matrix.
+While kube-hunter's vulnerabilities are a collection of creative techniques designed to mimic an attacker in the cluster (or outside it)
+The Mitre's ATT&CK defines a more general standardised categories of techniques to do so.
+
+You can think of kube-hunter vulnerabilities as small steps for an attacker, which follows the track of a more general technique he would aim for.
+Most of kube-hunter's hunters and vulnerabilities can closly fall under those techniques, That's why we moved to follow the Matrix standard.  
+ 
+_Some kube-hunter vulnerabities which we could not map to Mitre technique, are prefixed with the `General` keyword_ 
+![kube-hunter](./MITRE.png)
+
+## Hunting
 ### Where should I run kube-hunter?
 
 There are three different ways to run kube-hunter, each providing a different approach to detecting weaknesses in your cluster:
@@ -61,6 +76,7 @@ You can run kube-hunter directly on a machine in the cluster, and select the opt
 
 You can also run kube-hunter in a pod within the cluster. This indicates how exposed your cluster would be if one of your application pods is compromised (through a software vulnerability, for example). (_`--pod` flag_)
 
+
 ### Scanning options
 
 First check for these **[pre-requisites](#prerequisites)**.
@@ -141,11 +157,49 @@ Available dispatch methods are:
     * KUBEHUNTER_HTTP_DISPATCH_URL (defaults to: https://localhost)
     * KUBEHUNTER_HTTP_DISPATCH_METHOD (defaults to: POST)
 
-### Advanced Usage  
-#### Azure Quick Scanning 
+
+## Advanced Usage
+### Azure Quick Scanning 
 When running **as a Pod in an Azure or AWS environment**, kube-hunter will fetch subnets from the Instance Metadata Service. Naturally this makes the discovery process take longer.
 To hardlimit subnet scanning to a `/24` CIDR, use the `--quick` option. 
 
+### Custom Hunting
+Custom hunting enables advanced users to have control over what hunters gets registered at the start of a hunt. 
+**If you know what you are doing**, this can help if you want to adjust kube-hunter's hunting and discovery process for your needs.
+
+Example: 
+```
+kube-hunter --custom <HunterName1> <HunterName2>
+``` 
+Enabling Custom hunting removes all hunters from the hunting process, except the given whitelisted hunters.
+
+The `--custom` flag reads a list of hunters class names, in order to view all of kube-hunter's class names, you can combine the flag `--raw-hunter-names` with the `--list` flag.  
+
+Example: 
+```
+kube-hunter --active --list --raw-hunter-names
+```
+
+**Notice**: Due to kube-huner's architectural design, the following "Core Hunters/Classes" will always register (even when using custom hunting):
+* HostDiscovery 
+  * _Generates ip addresses for the hunt by given configurations_
+  * _Automatically discovers subnets using cloud Metadata APIs_
+* FromPodHostDiscovery
+  * _Auto discover attack surface ip addresses for the hunt by using Pod based environment techniques_
+  * _Automatically discovers subnets using cloud Metadata APIs_
+* PortDiscovery
+  * _Port scanning given ip addresses for known kubernetes services ports_
+* Collector
+  * _Collects discovered vulnerabilities and open services for future report_
+* StartedInfo 
+  * _Prints the start message_
+* SendFullReport 
+  * _Dispatching the report based on given configurations_
+
+
+
+
+
 ## Deployment
 There are three methods for deploying kube-hunter:
 
@@ -191,7 +245,7 @@ python3 kube_hunter
 _If you want to use pyinstaller/py2exe you need to first run the install_imports.py script._
 
 ### Container
-Aqua Security maintains a containerized version of kube-hunter at `aquasec/kube-hunter`. This container includes this source code, plus an additional (closed source) reporting plugin for uploading results into a report that can be viewed at [kube-hunter.aquasec.com](https://kube-hunter.aquasec.com). Please note, that running the `aquasec/kube-hunter` container and uploading reports data are subject to additional [terms and conditions](https://kube-hunter.aquasec.com/eula.html).
+Aqua Security maintains a containerized version of kube-hunter at `aquasec/kube-hunter:aqua`. This container includes this source code, plus an additional (closed source) reporting plugin for uploading results into a report that can be viewed at [kube-hunter.aquasec.com](https://kube-hunter.aquasec.com). Please note, that running the `aquasec/kube-hunter` container and uploading reports data are subject to additional [terms and conditions](https://kube-hunter.aquasec.com/eula.html).
 
 The Dockerfile in this repository allows you to build a containerized version without the reporting plugin.
 

docs/_kb/KHV002.md

@@ -2,6 +2,7 @@
 vid: KHV002
 title: Kubernetes version disclosure
 categories: [Information Disclosure]
+severity: low
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV003.md

@@ -2,6 +2,7 @@
 vid: KHV003
 title: Azure Metadata Exposure
 categories: [Information Disclosure]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV004.md

@@ -2,6 +2,7 @@
 vid: KHV004
 title: Azure SPN Exposure
 categories: [Identity Theft]
+severity: medium
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV005.md

@@ -2,6 +2,7 @@
 vid: KHV005
 title: Access to Kubernetes API
 categories: [Information Disclosure, Unauthenticated Access]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV006.md

@@ -2,6 +2,7 @@
 vid: KHV006
 title: Insecure (HTTP) access to Kubernetes API
 categories: [Unauthenticated Access]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV007.md

@@ -2,6 +2,7 @@
 vid: KHV007
 title: Specific Access to Kubernetes API
 categories: [Access Risk]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV020.md

@@ -2,6 +2,7 @@
 vid: KHV020
 title: Possible Arp Spoof
 categories: [IdentityTheft]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV021.md

@@ -2,6 +2,7 @@
 vid: KHV021
 title: Certificate Includes Email Address
 categories: [Information Disclosure]
+severity: low
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV022.md

@@ -2,6 +2,7 @@
 vid: KHV022
 title: Critical Privilege Escalation CVE
 categories: [Privilege Escalation]
+severity: critical
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV023.md

@@ -2,6 +2,7 @@
 vid: KHV023
 title: Denial of Service to Kubernetes API Server
 categories: [Denial Of Service]
+severity: medium
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV024.md

@@ -2,6 +2,7 @@
 vid: KHV024
 title: Possible Ping Flood Attack
 categories: [Denial Of Service]
+severity: medium
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV025.md

@@ -2,6 +2,7 @@
 vid: KHV025
 title: Possible Reset Flood Attack
 categories: [Denial Of Service]
+severity: medium
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV026.md

@@ -2,6 +2,7 @@
 vid: KHV026
 title: Arbitrary Access To Cluster Scoped Resources
 categories: [PrivilegeEscalation]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV027.md

@@ -2,6 +2,7 @@
 vid: KHV027
 title: Kubectl Vulnerable To CVE-2019-11246
 categories: [Remote Code Execution]
+severity: medium
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV028.md

@@ -2,6 +2,7 @@
 vid: KHV028
 title: Kubectl Vulnerable To CVE-2019-1002101
 categories: [Remote Code Execution]
+severity: medium
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV029.md

@@ -2,6 +2,7 @@
 vid: KHV029
 title: Dashboard Exposed
 categories: [Remote Code Execution]
+severity: critical
 ---
 
 # {{ page.vid }} - {{ page.title }}
@@ -12,4 +13,5 @@ An open Kubernetes Dashboard was detected. The Kubernetes Dashboard can be used
 
 ## Remediation
 
-Do not leave the Dashboard insecured.
+Do not leave the Dashboard insecured.
+

docs/_kb/KHV030.md

@@ -2,6 +2,7 @@
 vid: KHV030
 title: Possible DNS Spoof
 categories: [Identity Theft]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV031.md

@@ -2,6 +2,7 @@
 vid: KHV031
 title: Etcd Remote Write Access Event
 categories: [Remote Code Execution]
+severity: critical
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV032.md

@@ -2,6 +2,7 @@
 vid: KHV032
 title: Etcd Remote Read Access Event
 categories: [Access Risk]
+severity: critical
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV033.md

@@ -2,6 +2,7 @@
 vid: KHV033
 title: Etcd Remote version disclosure
 categories: [Information Disclosure]
+severity: medium
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV034.md

@@ -2,6 +2,7 @@
 vid: KHV034
 title: Etcd is accessible using insecure connection (HTTP)
 categories: [Unauthenticated Access]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV036.md

@@ -2,6 +2,7 @@
 vid: KHV036
 title: Anonymous Authentication
 categories: [Remote Code Execution]
+severity: critical
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV037.md

@@ -2,6 +2,7 @@
 vid: KHV037
 title: Exposed Container Logs
 categories: [Information Disclosure]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV038.md

@@ -2,6 +2,7 @@
 vid: KHV038
 title: Exposed Running Pods
 categories: [Information Disclosure]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV039.md

@@ -2,6 +2,7 @@
 vid: KHV039
 title: Exposed Exec On Container
 categories: [Remote Code Execution]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV040.md

@@ -2,6 +2,7 @@
 vid: KHV040
 title: Exposed Run Inside Container
 categories: [Remote Code Execution]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV041.md

@@ -2,6 +2,7 @@
 vid: KHV041
 title: Exposed Port Forward
 categories: [Remote Code Execution]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV042.md

@@ -2,6 +2,7 @@
 vid: KHV042
 title: Exposed Attaching To Container
 categories: [Remote Code Execution]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV043.md

@@ -2,6 +2,7 @@
 vid: KHV043
 title: Cluster Health Disclosure
 categories: [Information Disclosure]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV044.md

@@ -2,6 +2,7 @@
 vid: KHV044
 title: Privileged Container
 categories: [Access Risk]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV045.md

@@ -2,6 +2,7 @@
 vid: KHV045
 title: Exposed System Logs
 categories: [Information Disclosure]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV046.md

@@ -2,6 +2,7 @@
 vid: KHV046
 title: Exposed Kubelet Cmdline
 categories: [Information Disclosure]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV047.md

@@ -2,6 +2,7 @@
 vid: KHV047
 title: Pod With Mount To /var/log
 categories: [Privilege Escalation]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV049.md

@@ -2,6 +2,7 @@
 vid: KHV049
 title: kubectl proxy Exposed
 categories: [Information Disclosure]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV050.md

@@ -2,6 +2,7 @@
 vid: KHV050
 title: Read access to Pod service account token
 categories: [Access Risk]
+severity: medium
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV051.md

@@ -2,6 +2,7 @@
 vid: KHV051
 title: Exposed Existing Privileged Containers Via Secure Kubelet Port
 categories: [Access Risk]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV052.md

@@ -2,6 +2,7 @@
 vid: KHV052
 title: Exposed Pods
 categories: [Information Disclosure]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

docs/_kb/KHV053.md

@@ -2,6 +2,7 @@
 vid: KHV053
 title: AWS Metadata Exposure
 categories: [Information Disclosure]
+severity: high
 ---
 
 # {{ page.vid }} - {{ page.title }}

kube_hunter/__main__.py

@@ -1,6 +1,7 @@
 #!/usr/bin/env python3
 # flake8: noqa: E402
 
+from functools import partial
 import logging
 import threading
 
@@ -21,6 +22,7 @@ config = Config(
     log_file=args.log_file,
     mapping=args.mapping,
     network_timeout=args.network_timeout,
+    num_worker_threads=args.num_worker_threads,
     pod=args.pod,
     quick=args.quick,
     remote=args.remote,
@@ -28,6 +30,8 @@ config = Config(
     k8s_auto_discover_nodes=args.k8s_auto_discover_nodes,
     service_account_token=args.service_account_token,
     kubeconfig=args.kubeconfig,
+    enable_cve_hunting=args.enable_cve_hunting,
+    custom=args.custom,
 )
 setup_logger(args.log, args.log_file)
 set_config(config)
@@ -72,16 +76,20 @@ def interactive_set_config():
     return True
 
 
-def list_hunters():
+def list_hunters(class_names=False):
     print("\nPassive Hunters:\n----------------")
     for hunter, docs in handler.passive_hunters.items():
         name, doc = hunter.parse_docs(docs)
+        if class_names:
+            name = hunter.__name__
         print(f"* {name}\n  {doc}\n")
 
     if config.active:
         print("\n\nActive Hunters:\n---------------")
         for hunter, docs in handler.active_hunters.items():
             name, doc = hunter.parse_docs(docs)
+            if class_names:
+                name = hunter.__name__
             print(f"* {name}\n  {doc}\n")
 
 
@@ -94,7 +102,10 @@ def main():
     scan_options = [config.pod, config.cidr, config.remote, config.interface, config.k8s_auto_discover_nodes]
     try:
         if args.list:
-            list_hunters()
+            if args.raw_hunter_names:
+                list_hunters(class_names=True)
+            else:
+                list_hunters()
             return
 
         if not any(scan_options):

kube_hunter/conf/__init__.py

@@ -1,7 +1,11 @@
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from typing import Any, Optional
 
 
+def get_default_core_hunters():
+    return ["FromPodHostDiscovery", "HostDiscovery", "PortDiscovery", "SendFullReport", "Collector", "StartedInfo"]
+
+
 @dataclass
 class Config:
     """Config is a configuration container.
@@ -16,11 +20,13 @@ class Config:
     - log_file: Log File path
     - mapping: Report only found components
     - network_timeout: Timeout for network operations
+    - num_worker_threads: Add a flag --threads to change the default 800 thread count of the event handler
     - pod: From pod scanning mode
     - quick: Quick scanning mode
     - remote: Hosts to scan
     - report: Output format
     - statistics: Include hunters statistics
+    - enable_cve_hunting: enables cve hunting, shows cve results
     """
 
     active: bool = False
@@ -31,6 +37,7 @@ class Config:
     log_file: Optional[str] = None
     mapping: bool = False
     network_timeout: float = 5.0
+    num_worker_threads: int = 800
     pod: bool = False
     quick: bool = False
     remote: Optional[str] = None
@@ -39,6 +46,10 @@ class Config:
     k8s_auto_discover_nodes: bool = False
     service_account_token: Optional[str] = None
     kubeconfig: Optional[str] = None
+    enable_cve_hunting: bool = False
+    custom: Optional[list] = None
+    raw_hunter_names: bool = False
+    core_hunters: list = field(default_factory=get_default_core_hunters)
 
 
 _config: Optional[Config] = None

kube_hunter/conf/logging.py

@@ -4,10 +4,6 @@ DEFAULT_LEVEL = logging.INFO
 DEFAULT_LEVEL_NAME = logging.getLevelName(DEFAULT_LEVEL)
 LOG_FORMAT = "%(asctime)s %(levelname)s %(name)s %(message)s"
 
-# Suppress logging from scapy
-logging.getLogger("scapy.runtime").setLevel(logging.CRITICAL)
-logging.getLogger("scapy.loading").setLevel(logging.CRITICAL)
-
 
 def setup_logger(level_name, logfile):
     # Remove any existing handlers

kube_hunter/conf/parser.py

@@ -46,6 +46,22 @@ def parser_add_arguments(parser):
         help="One or more remote ip/dns to hunt",
     )
 
+    parser.add_argument(
+        "-c",
+        "--custom",
+        nargs="+",
+        metavar="HUNTERS",
+        default=list(),
+        help="Custom hunting. Only given hunter names will register in the hunt."
+        "for a list of options run `--list --raw-hunter-names`",
+    )
+
+    parser.add_argument(
+        "--raw-hunter-names",
+        action="store_true",
+        help="Use in combination with `--list` to display hunter class names to pass for custom hunting flag",
+    )
+
     parser.add_argument(
         "--k8s-auto-discover-nodes",
         action="store_true",
@@ -76,6 +92,12 @@ def parser_add_arguments(parser):
 
     parser.add_argument("--active", action="store_true", help="Enables active hunting")
 
+    parser.add_argument(
+        "--enable-cve-hunting",
+        action="store_true",
+        help="Show cluster CVEs based on discovered version (Depending on different vendors, may result in False Positives)",
+    )
+
     parser.add_argument(
         "--log",
         type=str,
@@ -111,6 +133,14 @@ def parser_add_arguments(parser):
 
     parser.add_argument("--network-timeout", type=float, default=5.0, help="network operations timeout")
 
+    parser.add_argument(
+        "--num-worker-threads",
+        type=int,
+        default=800,
+        help="In some environments the default thread count (800) can cause the process to crash. "
+        "In the case of a crash try lowering the thread count",
+    )
+
 
 def parse_args(add_args_hook):
     """

kube_hunter/core/events/handler.py

@@ -62,7 +62,7 @@ class EventQueue(Queue):
     ######################################################
     """
 
-    def subscribe(self, event, hook=None, predicate=None):
+    def subscribe(self, event, hook=None, predicate=None, is_register=True):
         """
         The Subscribe Decorator - For Regular Registration
         Use this to register for one event only. Your hunter will execute each time this event is published
@@ -74,12 +74,12 @@ class EventQueue(Queue):
         """
 
         def wrapper(hook):
-            self.subscribe_event(event, hook=hook, predicate=predicate)
+            self.subscribe_event(event, hook=hook, predicate=predicate, is_register=is_register)
             return hook
 
         return wrapper
 
-    def subscribe_many(self, events, hook=None, predicates=None):
+    def subscribe_many(self, events, hook=None, predicates=None, is_register=True):
         """
         The Subscribe Many Decorator - For Multiple Registration,
         When your attack needs several prerequisites to exist in the cluster, You need to register for multiple events.
@@ -99,12 +99,12 @@ class EventQueue(Queue):
         """
 
         def wrapper(hook):
-            self.subscribe_events(events, hook=hook, predicates=predicates)
+            self.subscribe_events(events, hook=hook, predicates=predicates, is_register=is_register)
             return hook
 
         return wrapper
 
-    def subscribe_once(self, event, hook=None, predicate=None):
+    def subscribe_once(self, event, hook=None, predicate=None, is_register=True):
         """
         The Subscribe Once Decorator - For Single Trigger Registration,
         Use this when you want your hunter to execute only in your entire program run
@@ -125,7 +125,8 @@ class EventQueue(Queue):
 
             hook.__new__ = __new__unsubscribe_self
 
-            self.subscribe_event(event, hook=hook, predicate=predicate)
+            self.subscribe_event(event, hook=hook, predicate=predicate, is_register=is_register)
+
             return hook
 
         return wrapper
@@ -256,7 +257,33 @@ class EventQueue(Queue):
             self.hooks[event].append((hook, predicate))
             logging.debug("{} subscribed to {}".format(hook, event))
 
-    def subscribe_event(self, event, hook=None, predicate=None):
+    def allowed_for_custom_registration(self, target_hunter):
+        """
+        Check if the partial input list contains the hunter we are about to register for events
+        If hunter is considered a Core hunter as specified in `config.core_hunters` we allow it anyway
+
+        Returns true if:
+         1. partial hunt is disabled
+         2. partial hunt is enabled and hunter is in core hunter class
+         3. partial hunt is enabled and hunter is specified in config.partial
+
+        @param target_hunter: hunter class for registration check
+        """
+        config = get_config()
+        if not config.custom:
+            return True
+
+        hunter_class_name = target_hunter.__name__
+        if hunter_class_name in config.core_hunters or hunter_class_name in config.custom:
+            return True
+
+        return False
+
+    def subscribe_event(self, event, hook=None, predicate=None, is_register=True):
+        if not is_register:
+            return
+        if not self.allowed_for_custom_registration(hook):
+            return
         if not self._register_hunters(hook):
             return
 
@@ -267,9 +294,13 @@ class EventQueue(Queue):
         else:
             self._register_hook(event, hook, predicate)
 
-    def subscribe_events(self, events, hook=None, predicates=None):
+    def subscribe_events(self, events, hook=None, predicates=None, is_register=True):
+        if not is_register:
+            return
+        if not self.allowed_for_custom_registration(hook):
+            return
         if not self._register_hunters(hook):
-            return False
+            return
 
         if predicates is None:
             predicates = [None] * len(events)
@@ -335,4 +366,5 @@ class EventQueue(Queue):
             self.queue.clear()
 
 
-handler = EventQueue(800)
+config = get_config()
+handler = EventQueue(config.num_worker_threads)

kube_hunter/modules/discovery/hosts.py

@@ -166,7 +166,9 @@ class FromPodHostDiscovery(Discovery):
                 return True
         except requests.exceptions.ConnectionError:
             logger.debug("Failed to connect AWS metadata server v1")
-            return False
+        except Exception:
+            logger.debug("Unknown error when trying to connect to AWS metadata v1 API")
+        return False
 
     def is_aws_pod_v2(self):
         config = get_config()
@@ -189,7 +191,9 @@ class FromPodHostDiscovery(Discovery):
                 return True
         except requests.exceptions.ConnectionError:
             logger.debug("Failed to connect AWS metadata server v2")
-            return False
+        except Exception:
+            logger.debug("Unknown error when trying to connect to AWS metadata v2 API")
+        return False
 
     def is_azure_pod(self):
         config = get_config()
@@ -206,7 +210,9 @@ class FromPodHostDiscovery(Discovery):
                 return True
         except requests.exceptions.ConnectionError:
             logger.debug("Failed to connect Azure metadata server")
-            return False
+        except Exception:
+            logger.debug("Unknown error when trying to connect to Azure metadata server")
+        return False
 
     # for pod scanning
     def gateway_discovery(self):

kube_hunter/modules/hunting/__init__.py

@@ -2,12 +2,10 @@
 from . import (
     aks,
     apiserver,
-    arp,
     capabilities,
     certificates,
     cves,
     dashboard,
-    dns,
     etcd,
     kubelet,
     mounts,

kube_hunter/modules/hunting/arp.py

@@ -1,71 +0,0 @@
-import logging
-
-from scapy.all import ARP, IP, ICMP, Ether, sr1, srp
-
-from kube_hunter.conf import get_config
-from kube_hunter.core.events import handler
-from kube_hunter.core.events.types import Event, Vulnerability
-from kube_hunter.core.types import ActiveHunter, KubernetesCluster, ARPPoisoningTechnique
-from kube_hunter.modules.hunting.capabilities import CapNetRawEnabled
-
-logger = logging.getLogger(__name__)
-
-
-class PossibleArpSpoofing(Vulnerability, Event):
-    """A malicious pod running on the cluster could potentially run an ARP Spoof attack
-    and perform a MITM between pods on the node."""
-
-    def __init__(self):
-        Vulnerability.__init__(
-            self,
-            KubernetesCluster,
-            "Possible Arp Spoof",
-            category=ARPPoisoningTechnique,
-            vid="KHV020",
-        )
-
-
-@handler.subscribe(CapNetRawEnabled)
-class ArpSpoofHunter(ActiveHunter):
-    """Arp Spoof Hunter
-    Checks for the possibility of running an ARP spoof
-    attack from within a pod (results are based on the running node)
-    """
-
-    def __init__(self, event):
-        self.event = event
-
-    def try_getting_mac(self, ip):
-        config = get_config()
-        ans = sr1(ARP(op=1, pdst=ip), timeout=config.network_timeout, verbose=0)
-        return ans[ARP].hwsrc if ans else None
-
-    def detect_l3_on_host(self, arp_responses):
-        """returns True for an existence of an L3 network plugin"""
-        logger.debug("Attempting to detect L3 network plugin using ARP")
-        unique_macs = list({response[ARP].hwsrc for _, response in arp_responses})
-
-        # if LAN addresses not unique
-        if len(unique_macs) == 1:
-            # if an ip outside the subnets gets a mac address
-            outside_mac = self.try_getting_mac("1.1.1.1")
-            # outside mac is the same as lan macs
-            if outside_mac == unique_macs[0]:
-                return True
-        # only one mac address for whole LAN and outside
-        return False
-
-    def execute(self):
-        config = get_config()
-        self_ip = sr1(IP(dst="1.1.1.1", ttl=1) / ICMP(), verbose=0, timeout=config.network_timeout)[IP].dst
-        arp_responses, _ = srp(
-            Ether(dst="ff:ff:ff:ff:ff:ff") / ARP(op=1, pdst=f"{self_ip}/24"),
-            timeout=config.network_timeout,
-            verbose=0,
-        )
-
-        # arp enabled on cluster and more than one pod on node
-        if len(arp_responses) > 1:
-            # L3 plugin not installed
-            if not self.detect_l3_on_host(arp_responses):
-                self.publish_event(PossibleArpSpoofing())

kube_hunter/modules/hunting/cves.py

@@ -3,7 +3,8 @@ from packaging import version
 
 from kube_hunter.conf import get_config
 from kube_hunter.core.events import handler
-from kube_hunter.core.events.types import Vulnerability, Event, K8sVersionDisclosure
+
+from kube_hunter.core.events.types import K8sVersionDisclosure, Vulnerability, Event
 from kube_hunter.core.types import (
     Hunter,
     KubectlClient,
@@ -15,6 +16,7 @@ from kube_hunter.core.types import (
 from kube_hunter.modules.discovery.kubectl import KubectlClientEvent
 
 logger = logging.getLogger(__name__)
+config = get_config()
 
 
 class ServerApiVersionEndPointAccessPE(Vulnerability, Event):
@@ -199,7 +201,7 @@ class CveUtils:
         return vulnerable
 
 
-@handler.subscribe_once(K8sVersionDisclosure)
+@handler.subscribe_once(K8sVersionDisclosure, is_register=config.enable_cve_hunting)
 class K8sClusterCveHunter(Hunter):
     """K8s CVE Hunter
     Checks if Node is running a Kubernetes version vulnerable to
@@ -224,6 +226,7 @@ class K8sClusterCveHunter(Hunter):
                 self.publish_event(vulnerability(self.event.version))
 
 
+# Removed due to incomplete implementation for multiple vendors revisions of kubernetes
 @handler.subscribe(KubectlClientEvent)
 class KubectlCVEHunter(Hunter):
     """Kubectl CVE Hunter

kube_hunter/modules/hunting/dns.py

@@ -1,90 +0,0 @@
-import re
-import logging
-
-from scapy.all import IP, ICMP, UDP, DNS, DNSQR, ARP, Ether, sr1, srp1, srp
-
-from kube_hunter.conf import get_config
-from kube_hunter.core.events import handler
-from kube_hunter.core.events.types import Event, Vulnerability
-from kube_hunter.core.types import ActiveHunter, KubernetesCluster, CoreDNSPoisoningTechnique
-from kube_hunter.modules.hunting.arp import PossibleArpSpoofing
-
-logger = logging.getLogger(__name__)
-
-
-class PossibleDnsSpoofing(Vulnerability, Event):
-    """A malicious pod running on the cluster could potentially run a DNS Spoof attack
-    and perform a MITM attack on applications running in the cluster."""
-
-    def __init__(self, kubedns_pod_ip):
-        Vulnerability.__init__(
-            self,
-            KubernetesCluster,
-            "Possible DNS Spoof",
-            category=CoreDNSPoisoningTechnique,
-            vid="KHV030",
-        )
-        self.kubedns_pod_ip = kubedns_pod_ip
-        self.evidence = f"kube-dns at: {self.kubedns_pod_ip}"
-
-
-# Only triggered with RunningAsPod base event
-@handler.subscribe(PossibleArpSpoofing)
-class DnsSpoofHunter(ActiveHunter):
-    """DNS Spoof Hunter
-    Checks for the possibility for a malicious pod to compromise DNS requests of the cluster
-    (results are based on the running node)
-    """
-
-    def __init__(self, event):
-        self.event = event
-
-    def get_cbr0_ip_mac(self):
-        config = get_config()
-        res = srp1(Ether() / IP(dst="1.1.1.1", ttl=1) / ICMP(), verbose=0, timeout=config.network_timeout)
-        return res[IP].src, res.src
-
-    def extract_nameserver_ip(self):
-        with open("/etc/resolv.conf") as f:
-            # finds first nameserver in /etc/resolv.conf
-            match = re.search(r"nameserver (\d+.\d+.\d+.\d+)", f.read())
-            if match:
-                return match.group(1)
-
-    def get_kube_dns_ip_mac(self):
-        config = get_config()
-        kubedns_svc_ip = self.extract_nameserver_ip()
-
-        # getting actual pod ip of kube-dns service, by comparing the src mac of a dns response and arp scanning.
-        dns_info_res = srp1(
-            Ether() / IP(dst=kubedns_svc_ip) / UDP(dport=53) / DNS(rd=1, qd=DNSQR()),
-            verbose=0,
-            timeout=config.network_timeout,
-        )
-        kubedns_pod_mac = dns_info_res.src
-        self_ip = dns_info_res[IP].dst
-
-        arp_responses, _ = srp(
-            Ether(dst="ff:ff:ff:ff:ff:ff") / ARP(op=1, pdst=f"{self_ip}/24"),
-            timeout=config.network_timeout,
-            verbose=0,
-        )
-        for _, response in arp_responses:
-            if response[Ether].src == kubedns_pod_mac:
-                return response[ARP].psrc, response.src
-
-    def execute(self):
-        config = get_config()
-        logger.debug("Attempting to get kube-dns pod ip")
-        self_ip = sr1(IP(dst="1.1.1.1", ttl=1) / ICMP(), verbose=0, timeout=config.network_timeout)[IP].dst
-        cbr0_ip, cbr0_mac = self.get_cbr0_ip_mac()
-
-        kubedns = self.get_kube_dns_ip_mac()
-        if kubedns:
-            kubedns_ip, kubedns_mac = kubedns
-            logger.debug(f"ip={self_ip} kubednsip={kubedns_ip} cbr0ip={cbr0_ip}")
-            if kubedns_mac != cbr0_mac:
-                # if self pod in the same subnet as kube-dns pod
-                self.publish_event(PossibleDnsSpoofing(kubedns_pod_ip=kubedns_ip))
-        else:
-            logger.debug("Could not get kubedns identity")

kube_hunter/modules/report/dispatchers.py

@@ -12,10 +12,7 @@ class HTTPDispatcher:
         dispatch_url = os.environ.get("KUBEHUNTER_HTTP_DISPATCH_URL", "https://localhost/")
         try:
             r = requests.request(
-                dispatch_method,
-                dispatch_url,
-                json=report,
-                headers={"Content-Type": "application/json"},
+                dispatch_method, dispatch_url, json=report, headers={"Content-Type": "application/json"}, verify=False
             )
             r.raise_for_status()
             logger.info(f"Report was dispatched to: {dispatch_url}")

requirements-dev.txt

@@ -1,5 +1,3 @@
--r requirements.txt
-
 flake8
 pytest >= 2.9.1
 requests-mock >= 1.8

setup.cfg

@@ -32,7 +32,6 @@ packages = find:
 install_requires =
     netaddr
     netifaces
-    scapy>=2.4.3
     requests
     PrettyTable
     urllib3>=1.24.3

tests/core/test_cloud.py

@@ -1,11 +1,13 @@
+# flake8: noqa: E402
 import requests_mock
 import json
 
 from kube_hunter.conf import Config, set_config
-from kube_hunter.core.events.types import NewHostEvent
 
 set_config(Config())
 
+from kube_hunter.core.events.types import NewHostEvent
+
 
 def test_presetcloud():
     """Testing if it doesn't try to run get_cloud if the cloud type is already set.

tests/core/test_handler.py

@@ -1,6 +1,6 @@
 # flake8: noqa: E402
 
-from kube_hunter.conf import Config, set_config
+from kube_hunter.conf import Config, set_config, get_config
 
 set_config(Config(active=True))
 
@@ -20,12 +20,12 @@ from kube_hunter.modules.hunting.apiserver import (
     AccessApiServerActive,
     AccessApiServerWithToken,
 )
-from kube_hunter.modules.hunting.arp import ArpSpoofHunter
 from kube_hunter.modules.hunting.capabilities import PodCapabilitiesHunter
 from kube_hunter.modules.hunting.certificates import CertificateDiscovery
-from kube_hunter.modules.hunting.cves import K8sClusterCveHunter, KubectlCVEHunter
+
+from kube_hunter.modules.hunting.cves import K8sClusterCveHunter
+from kube_hunter.modules.hunting.cves import KubectlCVEHunter
 from kube_hunter.modules.hunting.dashboard import KubeDashboard
-from kube_hunter.modules.hunting.dns import DnsSpoofHunter
 from kube_hunter.modules.hunting.etcd import EtcdRemoteAccess, EtcdRemoteAccessActive
 from kube_hunter.modules.hunting.kubelet import (
     ProveAnonymousAuth,
@@ -40,6 +40,8 @@ from kube_hunter.modules.hunting.mounts import VarLogMountHunter, ProveVarLogMou
 from kube_hunter.modules.hunting.proxy import KubeProxy, ProveProxyExposed, K8sVersionDisclosureProve
 from kube_hunter.modules.hunting.secrets import AccessSecrets
 
+config = get_config()
+
 PASSIVE_HUNTERS = {
     ApiServiceDiscovery,
     KubeDashboardDiscovery,
@@ -56,7 +58,6 @@ PASSIVE_HUNTERS = {
     ApiVersionHunter,
     PodCapabilitiesHunter,
     CertificateDiscovery,
-    K8sClusterCveHunter,
     KubectlCVEHunter,
     KubeDashboard,
     EtcdRemoteAccess,
@@ -67,11 +68,12 @@ PASSIVE_HUNTERS = {
     AccessSecrets,
 }
 
+# if config.enable_cve_hunting:
+#     PASSIVE_HUNTERS.append(K8sClusterCveHunter)
+
 ACTIVE_HUNTERS = {
     ProveAzureSpnExposure,
     AccessApiServerActive,
-    ArpSpoofHunter,
-    DnsSpoofHunter,
     EtcdRemoteAccessActive,
     ProveRunHandler,
     ProveContainerLogsHandler,