awesome-kubernetes/docs/qa.md

QA/TestOps - Continuous Testing. Software Quality Test Automation

1. Introduction
2. Blogs
3. Testing Frameworks
4. Release Testing
5. Tools
6. Performance Testing
7. Kubernetes conformance testing tools
8. Codeless Automation Testing
9. Images
10. Tweets

Introduction

• Awesome Test Automation
• Awesome Software Quality
• softwaretestguideforu.com: What is system testing? How to perform system testing?
• linkedin.com/pulse: Importance of API Automation Testing 🌟
• thenewstack.io: Optimizing App Performance in a Multicloud Stack
• botplayautomation.com: Automation Testing Implementation Guide
• botplayautomation.com: Types of Software Testing
• botplayautomation.com: Automation & Manual Testing Best Practices
• botplayautomation.com: How to write a Software Test Plan?
• blog.thundra.io: 4 Software Testing Roles
• lambdatest.com: TestNG vs JUnit : Which testing framework should you choose?
• botplayautomation.com: Common mistakes test teams make in automation testing and how to fix them
• blog.testproject.io: A Complete Guide to Test Automation Framework 🌟
  • Test Automation Framework: TestProject TestProject is a free end-to-end test automation platform for web, mobile, and API testing that’s supported by the #1 test automation community.
• botplayautomation.com: Automation Test Plan
• devops.com: Continuous Testing Practices (Part 1) 🌟
  • devops.com: Continuous Testing Practices – Part 2 🌟
• boozangfromthetrenches.com: Root Cause Analysis in Test Automation
• copyconstruct.medium.com: Testing in Production, the safe way 🌟🌟🌟
• blog.thundra.io: How to Fix Your Failing End-to-End Tests?
• rookout.com: How Real-Time Debugging Improves Reliability
• itnext.io: Software Development Is Misunderstood 🌟 Quality is fastest way to get code into production. Non-developers focus on creating code, developers focus on creating maintainable code
• blog.testproject.io: The 10–10–10 Rule of Test Automation
• blog.testproject.io: End to End Testing
• blog.testproject.io: REST API Automation From Scratch 🌟
• dzone: Checklist for API Verification 🌟 These days where Applications talk to each other using API, the verification of any message between the applications/microservices needs to be verified. This checklist includes some best practices for API verification.
• thenewstack.io: 7 Benefits of Testing in Isolation
• dzone: A Detailed Comparison: Unit Testing vs. Functional Testing The main goal of any testing is to deliver a quality product with find the right job balance between unit testing and functional testing.
• dzone: Microservice Testing Strategies This article contains Microservice testing strategies. Learn more about testing in layered architectures.
• dzone: We Are Testing Software Incorrectly and It's Costly We should write tests to enable developers to move fast with confidence. Code is always evolving, so question everything, collect experience, and judge for yourself.
• thenewstack.io: Error Handling from Backends to the Frontend
• ==lambdatest.com: Top Automation Testing Trends To Look Out In 2021==
• ==betterprogramming.pub: Why We Quit Unit Testing Classes to Focus On a Behavioral Approach== Why you should consider avoiding class-level tests, in favor of a behavior-oriented approach
• dev.to: Test-Driven-Development with Django: Unit Testing & Integration testing with Docker, Flask & Github Actions
• smashingmagazine.com: Testing Pipeline 101 For Frontend Testing
• infoq.com: Maintaining Software Quality with Microservices
• speakerdeck.com/thockin: Code Review in Kubernetes
• adequatica.medium.com: Principles of Writing Automated Tests While working on test automation in different projects, I’ve learned that there are not enough static analyzers and code formatters for writing good tests. The team had to have an agreement on how the tests should be written.
• getxray.app: The top 5 software testing trends of 2022
• linkedin.com: Microservices are testable in isolation 🌟
• dzone: Testing the Untestable and Other Anti-Patterns The productive path to establishing and maintaining effective test automation is not easy. In this post, explore well-intentioned yet harmful anti-patterns.
• softwaretestingsapiens.com: Roadmap to learn API Testing in Just 30 days

Blogs

• https://automationqahub.com/
• https://www.botplayautomation.com/
• https://www.lambdatest.com/
• https://blog.testproject.io/

Testing Frameworks

• JUnit
  • lambdatest.com A Comprehensive Guide On JUnit 5 Extensions
• TestNG
• Spock Framework
• testsvision.com: 6 Popular Automation Testing Frameworks & Tools
• opensource.com: Perform unit tests using GoogleTest and CTest Using unit tests will likely improve your code's quality and do so without disturbing your workflow.

Release Testing

• launchdarkly.com: Release Testing Explained 🌟 Release testing refers to coding practices and test strategies that give teams confidence that a software release candidate is ready for users. Release testing aims to find and eliminate errors and bugs from a software release so that it can be released to users. Let’s dive in and explore several methods used to perform release testing.

Tools

• collabnix.com: The Ultimate Docker Tutorial for Automation Testing
  • TestProject Free Test Automation for All.
• Allure Report 🌟
  • Allure Jenkins Plugin 🌟 This plugin allows to automatically generate Allure Report and attach it to build during Jenkins job run.
• venturebeat.com: Reflect brings automated no-code web testing to the cloud
• Semgrep Static analysis at ludicrous speed. Find bugs and enforce code standards
  • meetup.com: A single open-source security scanner for most languages on Jenkins As software security is becoming an increasingly bigger organization risk, static and dynamic analysis are becoming indispensable. However most popular open-source static analysis scanners cover only a handful of languages, or offer limited security rules. In this talk, we’ll present Semgrep, an open-source static analysis tool that support 12+ languages, and simplifies writing custom rules for organization-specific code patterns. In addition, we’ll show how to integrate Semgrep into a Jenkins Pipeline for scanning every commit or PR.
• testcontainers 🌟 Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
  • testcontainers.org
  • thenewstack.io: Testcontainers Integration Library Gets Commercial Backing with AtomicJar
  • spinscale.de: Using Testcontainers To Test Elasticsearch Plugins
  • testcontainers-spring-boot 🌟 Container auto-configurations for spring-boot based integration tests. If you use Testcontainers with Spring Boot you may be interested in the Playtika_Ltd Testcontainers library that provides auto-configurations for springboot based integration tests. It contains modules e.g. for kafka rabbitmq mongodb
  • atomicjar.com: Announcing Testcontainers Cloud: Integration Testing has never been easier Ever wished for integration tests to be faster, easier, and more efficient? Wish no more! We're happy to announce Testcontainers Cloud - a lightweight, fast, and secure integration testing platform for everyone.
  • vladmihalcea.com: Testcontainers Database Integration Testing
• Metabob Metabob is an AI Static Analysis Tool that enables developers to quickly identify the causes behind errors and performance sinks. An AI-assisted tool to visualize and debug Python code.
• dzone: Top Microservices Testing Tools Testers Should Know About
• SystemTap
  • developers.redhat.com: Opening black boxes with statement tracing (SystemTap)
• auchenberg/volkswagen Volkswagen detects when your tests are being run in a CI server, and makes them pass.
• getxray.app: 7 benefits of using a Test Management App vs. Excel
• ==google/clusterfuzzlite== 🌟 ClusterFuzzLite - Simple continuous fuzzing that runs in CI.
  • thenewstack.io: Google Introduces ClusterFuzzLite Security Tool for CI/CD
• convert.com: Top 10 A/B Testing Tools That Are Good for the Next 5 Years (Vetted by Features, Privacy, Maturity & Price)
• ==K6 Cloud== Managed performance testing for engineering teams
  • grafana.com: How to build performance tests into your CI pipeline with k6, GitHub Actions, and Grafana
• ==circleci.com: Unit testing vs integration testing== 🌟
• ==reviewdog - A code review dog who keeps your codebase healthy.== 🐶 Automated code review tool integrated with any code analysis tools regardless of programming language
• ==dzone: Component Tests for Spring Cloud Microservices== In this tutorial, we discussed guidelines and considerations for Spring Cloud microservices component tests and provided a recipe for common use cases.
• CoolerVoid/codecat: CodeCat - Tool to help in static code analysis CodeCat is an open-source tool to help you find/track user input sinks and security bugs using static code analysis. These points follow regex rules.
• ==beellz.hashnode.dev: Continuous Profiling in Kubernetes Using Pyroscope== In this blog post, you will learn about continuous profiling, and then instrument a few microservices running on Kubernetes using an open-source tool called Pyroscope to find performance issues and bottlenecks in your code.

Performance Testing

• devops.com: Best Practices for Application Performance Testing
• softwarequickguide.com: What is performance testing and performance testing tools

Kubernetes conformance testing tools

• Mastercard, for example, checks every deployment with Kubernetes conformance testing tools such as sonobuoy and kubench.
• sonobuoy is a diagnostic tool that makes it easier to understand the state of a Kubernetes cluster by running a set of plugins (including Kubernetes conformance tests) in an accessible and non-destructive manner. It is a customizable, extendable, and cluster-agnostic way to generate clear, informative reports about your cluster.
• kubench Benchmark different containerized applications within a kubernetes cluster.

Codeless Automation Testing

• botplayautomation.com
• botplayautomation.com: Benefits of Codeless (no code) Automation Testing

Images

??? note "Click to expand!"

<center>
![tests in prod](images/testinprodmeme.jfif)
![10 must haves test automation](images/10-must-haves-test-automation.jfif)
</center>

Tweets

Click to expand!

If your code has no tests, then:

- it is not clean
- it is not complete
- it is not correct
- it is not documented
- it is not verified
- it is not working

And most importantly: It is not quality.

— Daniel Moka⚡ (@dmokafa) February 28, 2021

<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

The term "legacy" is not about time or author. It is about quality.

— Mario Cervera (@macerub) September 23, 2021

<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

Code review does not work (and asynchronous code review is particularly bad). As Deming said: "You can not inspect quality into a product." NO form of code _review_ yields quality. Collaborative programming that assures that the defects don’t exist to begin with does.

— Allen Holub (@allenholub) January 16, 2022

<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

Why refactor your code and keep up with dependency deprecations when you can get a new job every 3 years to avoid your own technical debt?

— Carla Notarobot 🤖👩🏻‍💻 (@CarlaNotarobot) January 16, 2022

<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

Software engineering, a haiku:

fuck fuck fuck fuck fuck
fuck fuck fu-oh hey it works
wait, no - fuck fuck fuck

— Kelly Vaughn (@kvlly) June 18, 2022

<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>