two types of zone outage

Signed-off-by: Paige Patton <prubenda@redhat.com>

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,10 @@
+## Description  
+<!-- Provide a brief description of the changes made in this PR. -->  
+
+## Documentation  
+- [ ] **Is documentation needed for this update?**
+
+If checked, a documentation PR must be created and merged in the [website repository](https://github.com/krkn-chaos/website/).
+
+## Related Documentation PR (if applicable)  
+<!-- Add the link to the corresponding documentation PR in the website repository -->  

.github/workflows/require-docs.yml

@@ -0,0 +1,45 @@
+name: Require Documentation Update
+on:
+  pull_request:
+    types: [opened, edited, synchronize]
+    branches:
+        - main
+jobs:
+  check-docs:
+    name: Check Documentation Update
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Check if Documentation is Required
+        id: check_docs
+        run: |
+          echo "Checking PR body for documentation checkbox..."
+          # Read the PR body from the GitHub event payload
+          if echo "${{ github.event.pull_request.body }}" | grep -qi '\[x\].*documentation needed'; then
+            echo "Documentation required detected."
+            echo "docs_required=true" >> $GITHUB_OUTPUT
+          else
+            echo "Documentation not required."
+            echo "docs_required=false" >> $GITHUB_OUTPUT
+          fi
+
+      - name: Enforce Documentation Update (if required)
+        if: steps.check_docs.outputs.docs_required == 'true'
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          # Retrieve feature branch and repository owner from the GitHub context
+          FEATURE_BRANCH="${{ github.head_ref }}"
+          REPO_OWNER="${{ github.repository_owner }}"
+          WEBSITE_REPO="website"
+          echo "Searching for a merged documentation PR for feature branch: $FEATURE_BRANCH in $REPO_OWNER/$WEBSITE_REPO..."
+          MERGED_PR=$(gh pr list --repo "$REPO_OWNER/$WEBSITE_REPO" --state merged --json headRefName,title,url | jq -r \
+            --arg FEATURE_BRANCH "$FEATURE_BRANCH" '.[] | select(.title | contains($FEATURE_BRANCH)) | .url')
+          if [[ -z "$MERGED_PR" ]]; then
+            echo ":x: Documentation PR for branch '$FEATURE_BRANCH' is required and has not been merged."
+            exit 1
+          else
+            echo ":white_check_mark: Found merged documentation PR: $MERGED_PR"
+          fi

ADOPTERS.md

@@ -0,0 +1,7 @@
+# Krkn Adopters
+
+This is a list of organizations that have publicly acknowledged usage of Krkn and shared details of how they are leveraging it in their environment for chaos engineering use cases. Do you want to add yourself to this list? Please fork the repository and open a PR with the required change. 
+
+| Organization | Since | Website | Use-Case |
+|:-|:-|:-|:-|
+| MarketAxess | 2024 | https://www.marketaxess.com/ | Kraken enables us to achieve our goal of increasing the reliability of our cloud products on Kubernetes. The tool allows us to automatically run various chaos scenarios, identify resilience and performance bottlenecks, and seamlessly restore the system to its original state once scenarios finish. These chaos scenarios include pod disruptions, node (EC2) outages, simulating availability zone (AZ) outages, and filling up storage spaces like EBS and EFS. The community is highly responsive to requests and works on expanding the tool's capabilities. MarketAxess actively contributes to the project, adding features such as the ability to leverage existing network ACLs and proposing several feature improvements to enhance test coverage. |

README.md

@@ -9,14 +9,17 @@ Chaos and resiliency testing tool for Kubernetes.
 Kraken injects deliberate failures into Kubernetes clusters to check if it is resilient to turbulent conditions.
 
 
+### Website 
+[Kraken Website](https://krkn-chaos.dev) is the one stop shop for all things Kraken.
+The website contains comprehensive information about the workflow, supported scenarios, and detailed descriptions of each scenario. It also provides the necessary configurations needed to run Kraken, along with insights into performance monitoring and signaling features.
 ### Workflow
-![Kraken workflow](media/kraken-workflow.png)
+![Kraken workflow](media/kraken-workflow.png) 
 
-### Demo
-[![Kraken demo](media/KrakenStarting.png)](https://youtu.be/LN-fZywp_mo "Kraken Demo - Click to Watch!")
+<!-- ### Demo
+[![Kraken demo](media/KrakenStarting.png)](https://youtu.be/LN-fZywp_mo "Kraken Demo - Click to Watch!") -->
 
 
-### Chaos Testing Guide
+<!-- ### Chaos Testing Guide
 [Guide](docs/index.md) encapsulates:
 - Test methodology that needs to be embraced.
 - Best practices that an Kubernetes cluster, platform and applications running on top of it should take into account for best user experience, performance, resilience and reliability.
@@ -25,28 +28,28 @@ Kraken injects deliberate failures into Kubernetes clusters to check if it is re
 - Test environment recommendations as to how and where to run chaos tests.
 - Chaos testing in practice.
 
-The guide is hosted at https://krkn-chaos.github.io/krkn.
+The guide is hosted at https://krkn-chaos.github.io/krkn. -->
 
 
 ### How to Get Started
-Instructions on how to setup, configure and run Kraken can be found at [Installation](docs/installation.md).
+Instructions on how to setup, configure and run Kraken can be found at [Installation](https://krkn-chaos.dev/docs/installation/).
 
-You may consider utilizing the chaos recommendation tool prior to initiating the chaos runs to profile the application service(s) under test. This tool discovers a list of Krkn scenarios with a high probability of causing failures or disruptions to your application service(s). The tool can be accessed at [Chaos-Recommender](utils/chaos_recommender/README.md).
+You may consider utilizing the chaos recommendation tool prior to initiating the chaos runs to profile the application service(s) under test. This tool discovers a list of Krkn scenarios with a high probability of causing failures or disruptions to your application service(s). The tool can be accessed at [Chaos-Recommender](https://krkn-chaos.dev/docs/chaos-recommender/).
 
-See the [getting started doc](docs/getting_started.md) on support on how to get started with your own custom scenario or editing current scenarios for your specific usage.
+See the [getting started doc](https://krkn-chaos.dev/docs/getting-started/) on support on how to get started with your own custom scenario or editing current scenarios for your specific usage.
 
 After installation, refer back to the below sections for supported scenarios and how to tweak the kraken config to load them on your cluster.
 
 
-#### Running Kraken with minimal configuration tweaks
-For cases where you want to run Kraken with minimal configuration changes, refer to [krkn-hub](https://github.com/krkn-chaos/krkn-hub). One use case is CI integration where you do not want to carry around different configuration files for the scenarios.
+<!-- #### Running Kraken with minimal configuration tweaks
+For cases where you want to run Kraken with minimal configuration changes, refer to [krkn-hub](https://github.com/krkn-chaos/krkn-hub). One use case is CI integration where you do not want to carry around different configuration files for the scenarios. -->
 
 
 ### Config
 Instructions on how to setup the config and the options supported can be found at [Config](docs/config.md).
 
 
-### Kubernetes chaos scenarios supported
+<!-- ### Kubernetes chaos scenarios supported
 
 Scenario type               | Kubernetes    
 --------------------------- | ------------- | 
@@ -57,7 +60,7 @@ Scenario type               | Kubernetes
 [Time Scenarios](docs/time_scenarios.md) | :heavy_check_mark: |
 [Hog Scenarios: CPU, Memory](docs/hog_scenarios.md) | :heavy_check_mark: |
 [Cluster Shut Down Scenarios](docs/cluster_shut_down_scenarios.md) | :heavy_check_mark: |
-[Service Disruption Scenarios](docs/service_disruption_scenarios.md.md) | :heavy_check_mark: |
+[Service Disruption Scenarios](docs/service_disruption_scenarios.md) | :heavy_check_mark: |
 [Zone Outage Scenarios](docs/zone_outage.md) | :heavy_check_mark: |
 [Application_outages](docs/application_outages.md) | :heavy_check_mark: |
 [PVC scenario](docs/pvc_scenario.md) | :heavy_check_mark: |
@@ -95,7 +98,7 @@ Information on enabling and leveraging this feature can be found [here](docs/SLO
 
 ### OCM / ACM integration
 
-Kraken supports injecting faults into [Open Cluster Management (OCM)](https://open-cluster-management.io/) and [Red Hat Advanced Cluster Management for Kubernetes (ACM)](https://www.krkn.com/en/technologies/management/advanced-cluster-management) managed clusters through [ManagedCluster Scenarios](docs/managedcluster_scenarios.md).
+Kraken supports injecting faults into [Open Cluster Management (OCM)](https://open-cluster-management.io/) and [Red Hat Advanced Cluster Management for Kubernetes (ACM)](https://www.krkn.com/en/technologies/management/advanced-cluster-management) managed clusters through [ManagedCluster Scenarios](docs/managedcluster_scenarios.md). -->
 
 
 ### Blogs and other useful resources
@@ -114,10 +117,10 @@ Enhancements being planned can be found in the [roadmap](ROADMAP.md).
 ### Contributions
 We are always looking for more enhancements, fixes to make it better, any contributions are most welcome. Feel free to report or work on the issues filed on github.
 
-[More information on how to Contribute](docs/contribute.md)
+[More information on how to Contribute](https://krkn-chaos.dev/docs/contribution-guidelines/contribute/)
 
 If adding a new scenario or tweaking the main config, be sure to add in updates into the CI to be sure the CI is up to date.
-Please read [this file]((CI/README.md#adding-a-test-case)) for more information on updates.
+Please read [this file](https://krkn-chaos.dev/docs/getting-started/#adding-new-scenarios) for more information on updates.
 
 
 ### Scenario Plugin Development

krkn/scenario_plugins/network_chaos_ng/models.py

@@ -0,0 +1,41 @@
+from dataclasses import dataclass
+from enum import Enum
+
+
+class NetworkChaosScenarioType(Enum):
+    Node = 1
+    Pod = 2
+
+@dataclass
+class BaseNetworkChaosConfig:
+    supported_execution = ["serial", "parallel"]
+    id: str
+    wait_duration: int
+    test_duration: int
+    label_selector: str
+    instance_count: int
+    execution: str
+    namespace: str
+
+    def validate(self) -> list[str]:
+        errors = []
+        if self.execution is None:
+            errors.append(f"execution cannot be None, supported values are: {','.join(self.supported_execution)}")
+        if self.execution not in self.supported_execution:
+            errors.append(f"{self.execution} is not in supported execution mod: {','.join(self.supported_execution)}")
+        if self.label_selector is None:
+            errors.append("label_selector cannot be None")
+        return errors
+
+@dataclass
+class NetworkFilterConfig(BaseNetworkChaosConfig):
+    ingress: bool
+    egress: bool
+    interfaces: list[str]
+    target: str
+    ports: list[int]
+
+    def validate(self) -> list[str]:
+        errors = super().validate()
+        # here further validations
+        return errors

krkn/scenario_plugins/network_chaos_ng/modules/abstract_network_chaos_module.py

@@ -0,0 +1,58 @@
+import abc
+import logging
+import queue
+
+from krkn_lib.telemetry.ocp import KrknTelemetryOpenshift
+from krkn.scenario_plugins.network_chaos_ng.models import BaseNetworkChaosConfig, NetworkChaosScenarioType
+
+
+class AbstractNetworkChaosModule(abc.ABC):
+    """
+    The abstract class that needs to be implemented by each Network Chaos Scenario
+    """
+    @abc.abstractmethod
+    def run(self, target: str, kubecli: KrknTelemetryOpenshift, error_queue: queue.Queue = None):
+        """
+        the entrypoint method for the Network Chaos Scenario
+        :param target: The resource name that will be targeted by the scenario (Node Name, Pod Name etc.)
+        :param kubecli: The `KrknTelemetryOpenshift` needed by the scenario to access to the krkn-lib methods
+        :param error_queue: A queue that will be used by the plugin to push the errors raised during the execution of parallel modules
+        """
+        pass
+
+    @abc.abstractmethod
+    def get_config(self) -> (NetworkChaosScenarioType, BaseNetworkChaosConfig):
+        """
+        returns the common subset of settings shared by all the scenarios `BaseNetworkChaosConfig` and the type of Network
+        Chaos Scenario that is running (Pod Scenario or Node Scenario)
+        """
+        pass
+
+
+    def log_info(self, message: str, parallel: bool = False, node_name: str = ""):
+        """
+        log helper method for INFO severity to be used in the scenarios
+        """
+        if parallel:
+            logging.info(f"[{node_name}]: {message}")
+        else:
+            logging.info(message)
+
+    def log_warning(self, message: str, parallel: bool = False, node_name: str = ""):
+        """
+        log helper method for WARNING severity to be used in the scenarios
+        """
+        if parallel:
+            logging.warning(f"[{node_name}]: {message}")
+        else:
+            logging.warning(message)
+
+
+    def log_error(self, message: str, parallel: bool = False, node_name: str = ""):
+        """
+        log helper method for ERROR severity to be used in the scenarios
+        """
+        if parallel:
+            logging.error(f"[{node_name}]: {message}")
+        else:
+            logging.error(message)

krkn/scenario_plugins/network_chaos_ng/modules/node_network_filter.py

@@ -0,0 +1,136 @@
+import os
+import queue
+import time
+
+import yaml
+from jinja2 import Environment, FileSystemLoader
+
+
+from krkn_lib.telemetry.ocp import KrknTelemetryOpenshift
+from krkn_lib.utils import get_random_string
+from krkn.scenario_plugins.network_chaos_ng.models import (
+    BaseNetworkChaosConfig,
+    NetworkFilterConfig,
+    NetworkChaosScenarioType,
+)
+from krkn.scenario_plugins.network_chaos_ng.modules.abstract_network_chaos_module import (
+    AbstractNetworkChaosModule,
+)
+
+
+class NodeNetworkFilterModule(AbstractNetworkChaosModule):
+    config: NetworkFilterConfig
+
+    def run(
+        self,
+        target: str,
+        kubecli: KrknTelemetryOpenshift,
+        error_queue: queue.Queue = None,
+    ):
+        parallel = False
+        if error_queue:
+            parallel = True
+        try:
+            file_loader = FileSystemLoader(os.path.abspath(os.path.dirname(__file__)))
+            env = Environment(loader=file_loader, autoescape=True)
+            pod_name = f"node-filter-{get_random_string(5)}"
+            pod_template = env.get_template("templates/network-chaos.j2")
+            pod_body = yaml.safe_load(
+                pod_template.render(
+                    pod_name=pod_name,
+                    namespace=self.config.namespace,
+                    host_network=True,
+                    target=target,
+                )
+            )
+            self.log_info(
+                f"creating pod to filter "
+                f"ports {','.join([str(port) for port in self.config.ports])}, "
+                f"ingress:{str(self.config.ingress)}, "
+                f"egress:{str(self.config.egress)}",
+                parallel,
+                target,
+            )
+            kubecli.get_lib_kubernetes().create_pod(
+                pod_body, self.config.namespace, 300
+            )
+
+            if len(self.config.interfaces) == 0:
+                interfaces = [
+                    self.get_default_interface(pod_name, self.config.namespace, kubecli)
+                ]
+                self.log_info(f"detected default interface {interfaces[0]}")
+            else:
+                interfaces = self.config.interfaces
+
+            input_rules, output_rules = self.generate_rules(interfaces)
+
+            for rule in input_rules:
+                self.log_info(f"applying iptables INPUT rule: {rule}", parallel, target)
+                kubecli.get_lib_kubernetes().exec_cmd_in_pod(
+                    [rule], pod_name, self.config.namespace
+                )
+            for rule in output_rules:
+                self.log_info(
+                    f"applying iptables OUTPUT rule: {rule}", parallel, target
+                )
+                kubecli.get_lib_kubernetes().exec_cmd_in_pod(
+                    [rule], pod_name, self.config.namespace
+                )
+            self.log_info(
+                f"waiting {self.config.test_duration} seconds before removing the iptables rules"
+            )
+            time.sleep(self.config.test_duration)
+            self.log_info("removing iptables rules")
+            for _ in input_rules:
+                # always deleting the first rule since has been inserted from the top
+                kubecli.get_lib_kubernetes().exec_cmd_in_pod(
+                    [f"iptables -D INPUT 1"], pod_name, self.config.namespace
+                )
+            for _ in output_rules:
+                # always deleting the first rule since has been inserted from the top
+                kubecli.get_lib_kubernetes().exec_cmd_in_pod(
+                    [f"iptables -D OUTPUT 1"], pod_name, self.config.namespace
+                )
+            self.log_info(
+                f"deleting network chaos pod {pod_name} from {self.config.namespace}"
+            )
+
+            kubecli.get_lib_kubernetes().delete_pod(pod_name, self.config.namespace)
+
+        except Exception as e:
+            if error_queue is None:
+                raise e
+            else:
+                error_queue.put(str(e))
+
+    def __init__(self, config: NetworkFilterConfig):
+        self.config = config
+
+    def get_config(self) -> (NetworkChaosScenarioType, BaseNetworkChaosConfig):
+        return NetworkChaosScenarioType.Node, self.config
+
+    def get_default_interface(
+        self, pod_name: str, namespace: str, kubecli: KrknTelemetryOpenshift
+    ) -> str:
+        cmd = "ip r | grep default | awk '/default/ {print $5}'"
+        output = kubecli.get_lib_kubernetes().exec_cmd_in_pod(
+            [cmd], pod_name, namespace
+        )
+        return output.replace("\n", "")
+
+    def generate_rules(self, interfaces: list[str]) -> (list[str], list[str]):
+        input_rules = []
+        output_rules = []
+        for interface in interfaces:
+            for port in self.config.ports:
+                if self.config.egress:
+                    output_rules.append(
+                        f"iptables -I OUTPUT 1 -p tcp --dport {port} -m state --state NEW,RELATED,ESTABLISHED -j DROP"
+                    )
+
+                if self.config.ingress:
+                    input_rules.append(
+                        f"iptables -I INPUT 1 -i {interface} -p tcp --dport {port} -m state --state NEW,RELATED,ESTABLISHED -j DROP"
+                    )
+        return input_rules, output_rules

krkn/scenario_plugins/network_chaos_ng/modules/templates/network-chaos.j2

@@ -0,0 +1,17 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: {{pod_name}}
+  namespace: {{namespace}}
+spec:
+  {% if host_network %}
+  hostNetwork: true
+  {%endif%}
+  nodeSelector:
+    kubernetes.io/hostname: {{target}}
+  containers:
+  - name: fedora
+    imagePullPolicy: Always
+    image: quay.io/krkn-chaos/krkn-network-chaos:latest
+    securityContext:
+      privileged: true

krkn/scenario_plugins/network_chaos_ng/network_chaos_factory.py

@@ -0,0 +1,24 @@
+from krkn.scenario_plugins.network_chaos_ng.models import NetworkFilterConfig
+from krkn.scenario_plugins.network_chaos_ng.modules.abstract_network_chaos_module import AbstractNetworkChaosModule
+from krkn.scenario_plugins.network_chaos_ng.modules.node_network_filter import NodeNetworkFilterModule
+
+
+supported_modules = ["node_network_filter"]
+
+class NetworkChaosFactory:
+
+    @staticmethod
+    def get_instance(config: dict[str, str]) -> AbstractNetworkChaosModule:
+        if config["id"] is None:
+            raise Exception("network chaos id cannot be None")
+        if config["id"] not in supported_modules:
+            raise Exception(f"{config['id']} is not a supported network chaos module")
+
+        if config["id"] == "node_network_filter":
+            config = NetworkFilterConfig(**config)
+            errors = config.validate()
+            if len(errors) > 0:
+                raise Exception(f"config validation errors: [{';'.join(errors)}]")
+            return NodeNetworkFilterModule(config)
+
+

krkn/scenario_plugins/network_chaos_ng/network_chaos_ng_scenario_plugin.py

@@ -0,0 +1,116 @@
+import logging
+import queue
+import random
+import threading
+import time
+
+import yaml
+from krkn_lib.models.telemetry import ScenarioTelemetry
+from krkn_lib.telemetry.ocp import KrknTelemetryOpenshift
+
+from krkn.scenario_plugins.abstract_scenario_plugin import AbstractScenarioPlugin
+from krkn.scenario_plugins.network_chaos_ng.models import (
+    NetworkChaosScenarioType,
+    BaseNetworkChaosConfig,
+)
+from krkn.scenario_plugins.network_chaos_ng.modules.abstract_network_chaos_module import (
+    AbstractNetworkChaosModule,
+)
+from krkn.scenario_plugins.network_chaos_ng.network_chaos_factory import (
+    NetworkChaosFactory,
+)
+
+
+class NetworkChaosNgScenarioPlugin(AbstractScenarioPlugin):
+    def run(
+        self,
+        run_uuid: str,
+        scenario: str,
+        krkn_config: dict[str, any],
+        lib_telemetry: KrknTelemetryOpenshift,
+        scenario_telemetry: ScenarioTelemetry,
+    ) -> int:
+        try:
+            with open(scenario, "r") as file:
+                scenario_config = yaml.safe_load(file)
+                if not isinstance(scenario_config, list):
+                    logging.error(
+                        "network chaos scenario config must be a list of objects"
+                    )
+                    return 1
+                for config in scenario_config:
+                    network_chaos = NetworkChaosFactory.get_instance(config)
+                    network_chaos_config = network_chaos.get_config()
+                    logging.info(
+                        f"running network_chaos scenario: {network_chaos_config[1].id}"
+                    )
+                    if network_chaos_config[0] == NetworkChaosScenarioType.Node:
+                        targets = lib_telemetry.get_lib_kubernetes().list_nodes(
+                            network_chaos_config[1].label_selector
+                        )
+                    else:
+                        targets = lib_telemetry.get_lib_kubernetes().list_pods(
+                            network_chaos_config[1].namespace,
+                            network_chaos_config[1].label_selector,
+                        )
+                    if len(targets) == 0:
+                        logging.warning(
+                            f"no targets found for {network_chaos_config[1].id} "
+                            f"network chaos scenario with selector {network_chaos_config[1].label_selector} "
+                            f"with target type {network_chaos_config[0]}"
+                        )
+
+                    if network_chaos_config[1].instance_count != 0 and network_chaos_config[1].instance_count > len(targets):
+                        targets = random.sample(targets, network_chaos_config[1].instance_count)
+
+                    if network_chaos_config[1].execution == "parallel":
+                        self.run_parallel(targets, network_chaos, lib_telemetry)
+                    else:
+                        self.run_serial(targets, network_chaos, lib_telemetry)
+                    if len(config) > 1:
+                        logging.info(f"waiting {network_chaos_config[1].wait_duration} seconds before running the next "
+                                     f"Network Chaos NG Module")
+                        time.sleep(network_chaos_config[1].wait_duration)
+        except Exception as e:
+            logging.error(str(e))
+            return 1
+        return 0
+
+    def run_parallel(
+        self,
+        targets: list[str],
+        module: AbstractNetworkChaosModule,
+        lib_telemetry: KrknTelemetryOpenshift,
+    ):
+        error_queue = queue.Queue()
+        threads = []
+        errors = []
+        for target in targets:
+            thread = threading.Thread(
+                target=module.run, args=[target, lib_telemetry, error_queue]
+            )
+            thread.start()
+            threads.append(thread)
+        for thread in threads:
+            thread.join()
+        while True:
+            try:
+                errors.append(error_queue.get_nowait())
+            except queue.Empty:
+                break
+        if len(errors) > 0:
+            raise Exception(
+                f"module {module.get_config()[1].id} execution failed: [{';'.join(errors)}]"
+            )
+
+    def run_serial(
+        self,
+        targets: list[str],
+        module: AbstractNetworkChaosModule,
+        lib_telemetry: KrknTelemetryOpenshift,
+    ):
+        for target in targets:
+            module.run(target, lib_telemetry)
+
+    def get_scenario_types(self) -> list[str]:
+        return ["network_chaos_ng_scenarios"]

krkn/scenario_plugins/node_actions/gcp_node_scenarios.py

@@ -224,7 +224,6 @@ class gcp_node_scenarios(abstract_node_scenarios):
     def __init__(self, kubecli: KrknKubernetes, affected_nodes_status: AffectedNodeStatus):
         super().__init__(kubecli, affected_nodes_status)
         self.gcp = GCP()
-        print("selfkeys" + str(vars(self)))
 
     # Node scenario to start the node
     def node_start_scenario(self, instance_kill_count, node, timeout):

krkn/scenario_plugins/zone_outage/zone_outage_scenario_plugin.py

@@ -2,15 +2,21 @@ import logging
 import time
 
 import yaml
+
+from multiprocessing.pool import ThreadPool
+from itertools import repeat
+
+from krkn_lib.k8s import KrknKubernetes
+from krkn_lib.models.k8s import AffectedNodeStatus
 from krkn_lib.models.telemetry import ScenarioTelemetry
 from krkn_lib.telemetry.ocp import KrknTelemetryOpenshift
-from krkn_lib.utils import log_exception
 
-from krkn import utils
+from krkn_lib.utils import get_yaml_item_value
 from krkn.scenario_plugins.abstract_scenario_plugin import AbstractScenarioPlugin
 from krkn.scenario_plugins.native.network import cerberus
-from krkn.scenario_plugins.node_actions.aws_node_scenarios import AWS
 
+from krkn.scenario_plugins.node_actions.aws_node_scenarios import AWS
+from krkn.scenario_plugins.node_actions.gcp_node_scenarios import gcp_node_scenarios
 
 class ZoneOutageScenarioPlugin(AbstractScenarioPlugin):
     def run(
@@ -25,92 +31,138 @@ class ZoneOutageScenarioPlugin(AbstractScenarioPlugin):
             with open(scenario, "r") as f:
                 zone_outage_config_yaml = yaml.full_load(f)
                 scenario_config = zone_outage_config_yaml["zone_outage"]
-                vpc_id = scenario_config["vpc_id"]
-                subnet_ids = scenario_config["subnet_id"]
-                duration = scenario_config["duration"]
                 cloud_type = scenario_config["cloud_type"]
-                # Add support for user-provided default network ACL
-                default_acl_id = scenario_config.get("default_acl_id")
-                ids = {}
-                acl_ids_created = []
-
-                if cloud_type.lower() == "aws":
-                    cloud_object = AWS()
-                else:
-                    logging.error(
-                        "ZoneOutageScenarioPlugin Cloud type %s is not currently supported for "
-                        "zone outage scenarios" % cloud_type
-                    )
-                    return 1
-
                 start_time = int(time.time())
-
-                for subnet_id in subnet_ids:
-                    logging.info("Targeting subnet_id")
-                    network_association_ids = []
-                    associations, original_acl_id = cloud_object.describe_network_acls(
-                        vpc_id, subnet_id
-                    )
-                    for entry in associations:
-                        if entry["SubnetId"] == subnet_id:
-                            network_association_ids.append(
-                                entry["NetworkAclAssociationId"]
-                            )
-                    logging.info(
-                        "Network association ids associated with "
-                        "the subnet %s: %s" % (subnet_id, network_association_ids)
-                    )
-                    
-                    # Use provided default ACL if available, otherwise create a new one
-                    if default_acl_id:
-                        acl_id = default_acl_id
-                        logging.info(
-                            "Using provided default ACL ID %s - this ACL will not be deleted after the scenario", 
-                            default_acl_id
-                        )
-                        # Don't add to acl_ids_created since we don't want to delete user-provided ACLs at cleanup
+                if cloud_type.lower() == "aws":
+                    self.cloud_object = AWS()
+                    self.network_based_zone(scenario_config)
+                else:
+                    kubecli = lib_telemetry.get_lib_kubernetes()
+                    if cloud_type.lower() == "gcp":
+                        affected_nodes_status = AffectedNodeStatus()
+                        self.cloud_object = gcp_node_scenarios(kubecli, affected_nodes_status)
+                        self.node_based_zone(scenario_config, kubecli)
+                        affected_nodes_status = self.cloud_object.affected_nodes_status
+                        scenario_telemetry.affected_nodes.extend(affected_nodes_status.affected_nodes)
                     else:
-                        acl_id = cloud_object.create_default_network_acl(vpc_id)
-                        logging.info("Created new default ACL %s", acl_id)
-                        acl_ids_created.append(acl_id)
-
-                    new_association_id = cloud_object.replace_network_acl_association(
-                        network_association_ids[0], acl_id
-                    )
-
-                    # capture the orginal_acl_id, created_acl_id and
-                    # new association_id to use during the recovery
-                    ids[new_association_id] = original_acl_id
-
-                # wait for the specified duration
-                logging.info(
-                    "Waiting for the specified duration " "in the config: %s" % duration
-                )
-                time.sleep(duration)
-
-                # replace the applied acl with the previous acl in use
-                for new_association_id, original_acl_id in ids.items():
-                    cloud_object.replace_network_acl_association(
-                        new_association_id, original_acl_id
-                    )
-                logging.info(
-                    "Wating for 60 seconds to make sure " "the changes are in place"
-                )
-                time.sleep(60)
-
-                # delete the network acl created for the run
-                for acl_id in acl_ids_created:
-                    cloud_object.delete_network_acl(acl_id)
+                        logging.error(
+                            "ZoneOutageScenarioPlugin Cloud type %s is not currently supported for "
+                            "zone outage scenarios" % cloud_type
+                        )
+                        return 1
 
                 end_time = int(time.time())
                 cerberus.publish_kraken_status(krkn_config, [], start_time, end_time)
-        except (RuntimeError, Exception):
+        except (RuntimeError, Exception) as e:
             logging.error(
                 f"ZoneOutageScenarioPlugin scenario {scenario} failed with exception: {e}"
             )
             return 1
         else:
             return 0
+        
+    def node_based_zone(self, scenario_config: dict[str, any], kubecli: KrknKubernetes ):
+        zone = scenario_config["zone"]
+        duration = get_yaml_item_value(scenario_config, "duration", 60)
+        timeout = get_yaml_item_value(scenario_config, "timeout", 180)
+        label_selector = f"topology.kubernetes.io/zone={zone}"
+        try: 
+            # get list of nodes in zone/region
+            nodes = kubecli.list_killable_nodes(label_selector)
+            # stop nodes in parallel 
+            pool = ThreadPool(processes=len(nodes))
+    
+            pool.starmap(
+                self.cloud_object.node_stop_scenario,zip(repeat(1), nodes, repeat(timeout))
+            )
+
+            pool.close()
+
+            logging.info(
+                "Waiting for the specified duration " "in the config: %s" % duration
+            )
+            time.sleep(duration)
+
+            # start nodes in parallel 
+            pool = ThreadPool(processes=len(nodes))
+            pool.starmap(
+                self.cloud_object.node_start_scenario,zip(repeat(1), nodes, repeat(timeout))
+            )
+            pool.close()
+        except Exception as e:
+            logging.info(
+                f"Node based zone outage scenario failed with exception: {e}"
+            )
+            return 1
+        else:
+            return 0
+
+    def network_based_zone(self, scenario_config: dict[str, any]):
+
+        vpc_id = scenario_config["vpc_id"]
+        subnet_ids = scenario_config["subnet_id"]
+        duration = scenario_config["duration"]
+        # Add support for user-provided default network ACL
+        default_acl_id = scenario_config.get("default_acl_id")
+        ids = {}
+        acl_ids_created = []
+        for subnet_id in subnet_ids:
+            logging.info("Targeting subnet_id")
+            network_association_ids = []
+            associations, original_acl_id = self.cloud_object.describe_network_acls(
+                vpc_id, subnet_id
+            )
+            for entry in associations:
+                if entry["SubnetId"] == subnet_id:
+                    network_association_ids.append(
+                        entry["NetworkAclAssociationId"]
+                    )
+            logging.info(
+                "Network association ids associated with "
+                "the subnet %s: %s" % (subnet_id, network_association_ids)
+            )
+            
+            # Use provided default ACL if available, otherwise create a new one
+            if default_acl_id:
+                acl_id = default_acl_id
+                logging.info(
+                    "Using provided default ACL ID %s - this ACL will not be deleted after the scenario", 
+                    default_acl_id
+                )
+                # Don't add to acl_ids_created since we don't want to delete user-provided ACLs at cleanup
+            else:
+                acl_id = self.cloud_object.create_default_network_acl(vpc_id)
+                logging.info("Created new default ACL %s", acl_id)
+                acl_ids_created.append(acl_id)
+
+            new_association_id = self.cloud_object.replace_network_acl_association(
+                network_association_ids[0], acl_id
+            )
+
+            # capture the orginal_acl_id, created_acl_id and
+            # new association_id to use during the recovery
+            ids[new_association_id] = original_acl_id
+
+        # wait for the specified duration
+        logging.info(
+            "Waiting for the specified duration " "in the config: %s" % duration
+        )
+        time.sleep(duration)
+
+        # replace the applied acl with the previous acl in use
+        for new_association_id, original_acl_id in ids.items():
+            self.cloud_object.replace_network_acl_association(
+                new_association_id, original_acl_id
+            )
+        logging.info(
+            "Wating for 60 seconds to make sure " "the changes are in place"
+        )
+        time.sleep(60)
+
+        # delete the network acl created for the run
+        for acl_id in acl_ids_created:
+            self.cloud_object.delete_network_acl(acl_id)
+
 
     def get_scenario_types(self) -> list[str]:
         return ["zone_outages_scenarios"]

scenarios/kube/network_filter.yml

@@ -0,0 +1,13 @@
+- id: node_network_filter
+  wait_duration: 300
+  test_duration: 100
+  label_selector: "kubernetes.io/hostname=ip-10-0-39-182.us-east-2.compute.internal"
+  namespace: 'default'
+  instance_count: 1
+  execution: parallel
+  ingress: false
+  egress: true
+  target: node
+  interfaces: []
+  ports:
+    - 2049

scenarios/openshift/zone_outage_gcp.yaml

@@ -0,0 +1,4 @@
+zone_outage:                                         # Scenario to create an outage of a zone by tweaking network ACL
+  cloud_type: gcp                                    # cloud type on which Kubernetes/OpenShift runs. aws is only platform supported currently for this scenario.
+  duration: 600                                      # duration in seconds after which the zone will be back online
+  zone: <zone>                    # (Optional) ID of an existing network ACL to use instead of creating a new one. If provided, this ACL will not be deleted after the scenario.