mirror of
https://github.com/krkn-chaos/krkn.git
synced 2026-08-25 09:27:36 +00:00
v5.1.1
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 harshjainnn
|
bf59ed3c9c |
feat(tests): migrate pod error scenario tests to v2 framework (#1440)
* feat(tests): migrate pod error scenario tests to v2 framework Signed-off-by: Harsh Jain <hjain4815@gmail.com> * chore: drop unrelated file from PR Signed-off-by: Harsh Jain <hjain4815@gmail.com> * refactor: rename assert_no_hang_and_correct_logs to assert_failure_logs_contain Signed-off-by: Harsh Jain <hjain4815@gmail.com> * docs: reference pod_disruption_scenario_plugin.py:234 in test comment Signed-off-by: Harsh Jain <hjain4815@gmail.com> --------- Signed-off-by: Harsh Jain <hjain4815@gmail.com> |
||
  
|
cef0d4025d |
test: migrate test_namespace.sh to CI/tests_v2 namespace_deletion (#1406)
* test: migrate test_namespace.sh to CI/tests_v2 namespace_deletion Migrate the legacy CI/tests/test_namespace.sh functional test to the pytest-based v2 framework under CI/tests_v2/scenarios/namespace_deletion/. Covers the service_disruption (namespace deletion) scenario with happy-path cases (single-namespace object deletion, multi-namespace delete_count, multiple runs, wait_time handling, label-selector targeting) and negative cases (no-match regex, namespace/label mutual exclusion, delete_count exceeding available namespaces). - Add scenario assets (resource.yaml with Deployment+Service, scenario_base.yaml) - Register namespace_deletion marker in pytest.ini - Add namespace_deletion execution-evidence marker in lib/utils.py Closes #1405 * test: move namespace_deletion helpers into reusable lib modules Address review feedback: extract the per-test helpers into shared lib modules so other scenarios can reuse them. - lib/namespace.py: add POD_SECURITY_PRIVILEGED_LABELS, create_labeled_namespace, delete_namespace_quietly, and a make_namespace factory fixture (auto-cleanup) - lib/deploy.py: add deployment_exists, wait_for_no_deployment, wait_for_present_deployment_count, deploy_manifest_to_namespace - conftest.py: re-export make_namespace fixture - test_namespace_deletion.py: drop local helpers, use the lib functions/fixture * test: honor --keep-ns-on-fail in make_namespace factory The make_namespace finalizer always deleted ad-hoc namespaces, ignoring --keep-ns-on-fail on failure unlike test_namespace. Extract the keep-on-fail decision into a shared helper and use it in both fixtures so the documented debugging workflow works for scenarios that create extra namespaces. * test: surface FailToCreateError in deploy_manifest_to_namespace Mirror deploy_workload's error handling so manifest-apply failures in multi-namespace tests raise a formatted RuntimeError listing the underlying API exceptions instead of an opaque FailToCreateError. * test: clarify why label-selector test bypasses run_scenario The inline comment claimed run_scenario was bypassed because **overrides would collide with the positional namespace arg. The real reason is that NAMESPACE_IS_REGEX=True wraps an empty namespace as '^$', whereas label-selector mode needs a literal empty string. * test: use actual scenario inputs in negative-test failure context The no-match and mutual-exclusion tests reported context=namespace=self.ns (the ephemeral namespace), not the inputs actually under test. Reference the real namespace (and label_selector) so unexpected-success diagnostics are clear. * test: use non-default wait_time so override patching is exercised wait_time=30 matched the scenario_base.yaml default, so the test passed even if override patching regressed. Use wait_time=5 (non-default) so the override path is actually validated. * test: guard cluster post-checks under KRKN_TEST_DRY_RUN Three namespace_deletion tests asserted cluster side effects (workload deletion) after the Kraken run. Under KRKN_TEST_DRY_RUN=1 Kraken is skipped, so the seeded workload is never deleted and wait_for_no_deployment / wait_for_present_deployment_count would time out and fail. Guard those post-checks, and in test_label_selector_targeting (which bypasses run_scenario and calls run_kraken directly) honor dry-run explicitly by skipping the invocation and post-check. * test: tighten no-match assertion, clarify runs-loop test, dry-run-safe negatives Addresses Deep Code Review feedback on the namespace_deletion suite: - test_no_match_namespace_fails: drop the dead 'no namespaces matching' OR branch; the service_disruption plugin only ever logs 'not enough namespaces matching ...', so the assertion now checks that string directly. - test_multiple_runs_repeat_deletion -> test_multiple_runs_repeat_disruption_loop: rename + docstring make explicit that it verifies the outer runs loop iterates twice, not that object deletion recurs (Krkn does not redeploy between runs, so run 2 re-selects an already-empty namespace). Object removal is asserted in test_single_namespace_object_deletion. - Negative tests now return early under KRKN_TEST_DRY_RUN=1, since run_scenario returns a fake rc=0 and the failure path cannot be exercised; this makes the whole class consistent under make test-dry-run. * test: use UUID-based namespace in no-match test to avoid accidental matches * test: assert correct zero-match error in no-match namespace test A regex matching zero namespaces makes krkn_lib's check_namespaces raise 'there exists no namespaces matching' before the plugin's delete loop, so the 'not enough namespaces matching' branch is never reached for this case. Assert the actual zero-match message instead. * test: poll for async Service deletion in namespace_deletion test Kubernetes deletions are asynchronous, so checking the Service immediately after the scenario run could be flaky. Add wait_for_no_service (mirroring wait_for_no_deployment) and poll until the Service is actually gone before asserting. * test: assert Service deletion in label-selector namespace_deletion test test_label_selector_targeting deploys both a Deployment and a Service but only asserted the Deployment was removed. Add wait_for_no_service so the test fully validates that label-selector mode deletes all objects (including Services), matching test_single_namespace_object_deletion. Signed-off-by: augmentcode[bot] <185243770+augmentcode[bot]@users.noreply.github.com> * test: snapshot namespaces in wait_for_present_deployment_count list(namespaces) consumed the iterable before the poll loop re-iterated over the original, so a one-shot iterable (e.g. generator) would be empty on every poll. Snapshot to a list once and iterate over that snapshot. Signed-off-by: augmentcode[bot] <185243770+augmentcode[bot]@users.noreply.github.com> * ci: free up runner disk space before tests_v2 KinD run The Tests v2 (pytest functional) job intermittently fails with 'System.IO.IOException: No space left on device' on ubuntu-latest runners, which ship with only ~14GB free. Creating the KinD cluster plus pulling and kind-loading nginx:alpine and krkn:tools exhausts the disk, failing even the runner's own diagnostic logging. Reclaim ~20-30GB by removing the bundled .NET/Android/GHC SDKs and pruning docker images before the cluster is created. Signed-off-by: augmentcode[bot] <185243770+augmentcode[bot]@users.noreply.github.com> --------- Signed-off-by: augmentcode[bot] <185243770+augmentcode[bot]@users.noreply.github.com> Co-authored-by: augmentcode[bot] <185243770+augmentcode[bot]@users.noreply.github.com> Co-authored-by: Darshan Jain <darjain@redhat.com> |
||
|
augmentcode[bot]andlnx01
|
03416ddbb6 |
test: migrate node_scenarios to tests_v2 pytest framework (#1400)
* test: add node_scenarios pytest migration under tests_v2 Migrate the node chaos coverage from the legacy CI/tests/test_node.sh into the v2 pytest framework at CI/tests_v2/scenarios/node_scenarios/. - test_node_scenarios.py (TestNodeScenarios extends BaseScenarioTest): reboot and stop/start happy paths, node_name vs label_selector targeting, node recovery with finalizer, a control-plane safety guard, and negative cases (invalid selector, invalid node, missing actions, unsupported cloud type, unknown action). - scenario_base.yaml: single node_scenarios entry (cloud_type docker, worker-only) patched per test. - Register the node_scenarios marker in pytest.ini and document the scenario in CI/tests_v2/README.md. Part of #1398. The coupled legacy move + workflow edit is left for a maintainer (the bot lacks GitHub App workflows permission); see PR description. * test: address node_scenarios review feedback - Reboot happy path now runs with kube_check: True so Krkn waits for the node to go Unknown then Ready, eliminating the race where wait_node_ready could pass against a stale Ready=True before the disruption propagated. - Finalizer ensure_node_container_running now polls for Ready (bounded) when given k8s_core and logs non-zero 'start' exits, matching the docstring contract so a rerun never picks up an unrecovered node. - Clarify the parallelism note: first/last worker separation only applies on multi-worker clusters (CI's 2-worker kind-config.yml); single-worker dev clusters share the node and rely on test ordering. * fix: make node happy-path tests resilient to KinD multi-CP API LB The reboot/stop_start happy paths failed in the Tests v2 job with "Response ended prematurely": with kube_check enabled, Krkn's docker node plugin polls the kube API (wait_for_unknown_status/wait_for_ready_status) for ~40-50s after the disruption, and krkn-lib does not retry a transient connection drop from the multi-control-plane KinD haproxy API load balancer. The docker action itself succeeded in <0.5s. Run both happy paths with kube_check: False so Krkn performs the action and exits cleanly, and prove real disruption deterministically via the node container's State.StartedAt advancing (runtime-level evidence, independent of node-status timing). Recovery is still verified with the resilient test-side wait_node_ready poll. README updated to match. * refactor: move reusable node/container test helpers to lib/utils Per review feedback, relocate the generic node-level helpers out of the node_scenarios test module into the shared CI/tests_v2/lib/utils.py so future node/container tests can reuse them: wait_node_ready, container_runtime, container_started_at, assert_container_cycled, ensure_node_container_running, and assert_kraken_marker. The test module now imports them; behavior is unchanged and all 8 tests still collect. * docs: fix duplicated word in tests_v2 README scenario list * test: skip redundant container start in node finalizer to avoid noisy warnings * test: honor KIND_EXPERIMENTAL_PROVIDER when selecting container runtime * docs: align node finalizer wording with start-if-stopped behavior * docs: align node test module docstring with start-if-stopped finalizer * test: sort schedulable worker nodes for deterministic targeting * test: assert combined node_stop_start_scenario marker in stop/start happy path --------- Co-authored-by: augmentcode[bot] <185243770+augmentcode[bot]@users.noreply.github.com> |
||
|
augmentcode[bot]andlnx01
|
8265c358d3 |
test: migrate memory_hog functional test to tests_v2 (closes #1396) (#1397)
* test: migrate memory_hog functional test to tests_v2 (closes #1396) Add a pytest v2 functional test for the memory hog scenario (hog_scenarios), mirroring the cpu_hog migration. Covers execution success, node-selector targeting, duration/memory-size parameter handling, hog pod lifecycle and cleanup, and graceful failure on invalid selector / invalid config. - CI/tests_v2/scenarios/memory_hog/test_memory_hog.py: TestMemoryHog with functional + memory_hog markers and three no_workload tests. - CI/tests_v2/scenarios/memory_hog/scenario_base.yaml: flat hog config tuned for functional testing (light fixed memory-vm-bytes, short duration). - Register the memory_hog marker in pytest.ini and document the scenario in the tests_v2 README. * test: extract shared hog-pod/node helpers into tests_v2 lib/utils Lift the duplicated pod-prefix and schedulable-node helpers out of the cpu_hog and memory_hog test modules into CI/tests_v2/lib/utils.py as parameterized, reusable functions (list_pods_by_prefix, wait_for_scheduled_pod_by_prefix, wait_for_no_pods_by_prefix, schedulable_worker_nodes) and reuse them from both scenarios. Addresses review feedback on #1397. --------- Co-authored-by: augmentcode[bot] <185243770+augmentcode[bot]@users.noreply.github.com> |
||
|
augmentcode[bot]andlnx01
|
3a77f487da |
test: migrate cpu_hog functional test to tests_v2 (closes #1390) (#1391)
* test: migrate cpu_hog functional test to tests_v2 (closes #1390) Migrate the legacy CI/tests/test_cpu_hog.sh to the v2 pytest framework under CI/tests_v2/scenarios/cpu_hog/, preserving parity with the legacy flow and adding stronger functional and negative coverage. - Add scenario_base.yaml (flat hog_scenarios config, patched per test) - Add test_cpu_hog.py with TestCpuHog(BaseScenarioTest): - success: hog pod created on node-selector target, run exits 0, pod cleaned up - negative: node-selector matching zero nodes fails gracefully - negative: missing mandatory hog-type fails at config parsing - Register cpu_hog marker in pytest.ini - Document cpu_hog coverage in README.md * test: kill background Kraken proc on any cpu_hog success-test failure Broaden the success-path teardown so a poll/assert failure before proc.communicate() also kills the background Kraken process, preventing a lingering cpu-hog- pod from stressing the node and racing --reruns. Addresses Deep Code Review feedback on PR #1391. --------- Signed-off-by: augmentcode[bot] <185243770+augmentcode[bot]@users.noreply.github.com> Co-authored-by: augmentcode[bot] <185243770+augmentcode[bot]@users.noreply.github.com> |
||
|
augmentcode[bot]andlnx01
|
5a532190aa |
test: attach krkn execution logs to HTML report and assert scenario executed (#1395)
* test: attach krkn execution logs to HTML report and assert scenario executed CI/tests_v2 enhancements for observability and false-positive guarding: - run_kraken fixture stashes stdout/stderr/returncode on request.node so the report hook can attach them. - conftest pytest_runtest_makereport renders the krkn log as a timestamp/level/ message table via pytest_html.extras for every test (pass or fail). - conftest pytest_terminal_summary prints an execution-evidence table and writes a markdown summary to $GITHUB_STEP_SUMMARY when running in GitHub Actions. - utils adds SCENARIO_EXECUTION_MARKERS, an EXECUTION_EVIDENCE registry, and assert_scenario_executed() that fails happy-path tests when krkn exits 0 but no scenario-specific marker is present. Skipped under KRKN_TEST_DRY_RUN=1. - pod_disruption, storage_throttle, and application_outage happy-path tests now call assert_scenario_executed. Negative tests remain exempt. Closes #1394 * test: stash krkn logs on timeout and add log-path hint to evidence assert Address Deep Code Review feedback on PR #1395: - run_kraken now catches subprocess.TimeoutExpired, stashes partial stdout/stderr with a synthetic rc=124 so timed-out runs still attach logs to the HTML report. - assert_scenario_executed failure message now includes the 'Full logs:' tmp_path hint, matching assert_kraken_success/failure. --------- Co-authored-by: augmentcode[bot] <185243770+augmentcode[bot]@users.noreply.github.com> |
||
  
|
eb2efa84ea |
feat: storage I/O throttle scenario (cgroups v1/v2) for PVC-backed workloads (#1296)
* feat(storage-throttle): add storage throttle scenario and tests Consolidate the storage-throttle implementation, scenario configs, CI v2 coverage, and krkn-lib 6.0.10 dependency update into a single signed commit for cleaner PR history. Signed-off-by: ddjain <darjain@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: ddjain <darjain@redhat.com> * adding need dco auto review (#1329) Signed-off-by: Paige Patton <prubenda@redhat.com> Signed-off-by: ddjain <darjain@redhat.com> * fix: start_klusterlet_scenario action calls start instead of stop (#1324) The start_klusterlet_scenario branch in inject_managedcluster_scenario was calling stop_klusterlet_scenario on the scenarios object instead of start_klusterlet_scenario. Any user configuring this action would stop the klusterlet (scale to 0) instead of starting it (scale to 3). Fixes #1323 Signed-off-by: v0idheaven <dahiyavarun2007@gmail.com> Co-authored-by: Paige Patton <64206430+paigerube14@users.noreply.github.com> Signed-off-by: ddjain <darjain@redhat.com> * adding links to issue of completed roadmap items (#1328) Signed-off-by: Paige Patton <prubenda@redhat.com> Signed-off-by: ddjain <darjain@redhat.com> * container scenario template image update (#1342) Signed-off-by: Paige Patton <prubenda@redhat.com> Signed-off-by: ddjain <darjain@redhat.com> --------- Signed-off-by: ddjain <darjain@redhat.com> Signed-off-by: Paige Patton <prubenda@redhat.com> Signed-off-by: v0idheaven <dahiyavarun2007@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Paige Patton <64206430+paigerube14@users.noreply.github.com> Co-authored-by: Varun Yadav <dahiyavarun2007@gmail.com> |
||
  Semih702andsemih702
|
9d06239341 |
docs: align Python requirement with 3.11 (#1258)
Signed-off-by: semih702 <semih702@users.noreply.github.com> Co-authored-by: semih702 <semih702@users.noreply.github.com> |
||
 Arpit Raj
|
daa6dc4df9 |
fix: replace hardcoded /tmp paths with secure tempfile.mkdtemp() (#1223)
Signed-off-by: 1PoPTRoN <vrxn.arp1traj@gmail.com> |
||
|
Darshan JainandCursor
|
a14d3955a6 |
feat(ci): add pytest-based CI test framework v2 with ephemeral namespace isolation (#1172) (#1171)
* feat: add pytest-based CI test framework v2 with ephemeral namespace isolation Signed-off-by: ddjain <darjain@redhat.com> * feat(ci): add tests_v2 pytest functional test framework Signed-off-by: ddjain <darjain@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com> * feat: improve naming convention Signed-off-by: ddjain <darjain@redhat.com> * improve local setup script. Signed-off-by: ddjain <darjain@redhat.com> * added CI job for v2 test Signed-off-by: ddjain <darjain@redhat.com> * disabled broken test Signed-off-by: ddjain <darjain@redhat.com> * improved CI pipeline execution time Signed-off-by: ddjain <darjain@redhat.com> * chore: remove unwanted/generated files from PR Signed-off-by: ddjain <darjain@redhat.com> * clean up gitignore file Signed-off-by: ddjain <darjain@redhat.com> * fix copilot comments Signed-off-by: ddjain <darjain@redhat.com> * fixed copilot suggestion Signed-off-by: ddjain <darjain@redhat.com> * uncommented out test upload stage Signed-off-by: ddjain <darjain@redhat.com> * exclude CI/tests_v2 from test coverage reporting Signed-off-by: ddjain <darjain@redhat.com> * uploading style.css to fix broken report artifacts Signed-off-by: ddjain <darjain@redhat.com> * added openshift supported labels in namespace creatation api Signed-off-by: ddjain <darjain@redhat.com> --------- Signed-off-by: ddjain <darjain@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com> |