Testing
94 items tagged with "testing"
Standards3
Rust 1.78 Edition 2024
Migrating systems that utilize Rust requires adherence to its safety and performance standards. By leveraging Rust's ownership model, error handling capabilities, and robust tooling, teams can ensure a successful transition while mitigating common challenges. Following these guidelines will lead to safer, more reliable applications post-migration.
Kotlin 2.0 Spec
Understanding and adhering to established migration standards is crucial for successful software transitions. These standards help mitigate risks, ensure data integrity, and maintain compliance, leading to a smoother migration process. By leveraging the right tools and processes, teams can address common challenges and execute migrations with confidence.
Perl 5.38 Syntax
Adhering to Perl standards during software migrations is crucial for ensuring code compatibility, maintaining best practices, and streamlining the transition process. By conducting thorough code audits, implementing robust testing protocols, and utilizing effective tools, teams can navigate the complexities of migration with confidence and achieve successful outcomes.
Best Practices18
Shift-Left Testing Manifesto
Encourages earlier testing (unit, security, performance) in the SDLC to catch defects sooner.
Contract-Driven Development with Pact
Consumer-driven contract testing methodology to ensure micro-service compatibility.
Continuous Integration Best Practices
A development practice where engineers merge code into a shared mainline many times a day, each merge verified by an automated build and test suite.
Hexagonal Architecture (Ports and Adapters)
An architecture that isolates core application logic behind ports, with adapters connecting external concerns like databases and UIs, so the core stays independent of technology.
Data Quality Management
The practice of measuring, monitoring, and improving data across dimensions like accuracy, completeness, consistency, timeliness, and validity so it stays fit for use.
LLM Evaluation and Evals
LLM evaluation measures model and application quality with repeatable tests, scoring accuracy, faithfulness, safety, and cost so teams can ship and improve with evidence.
Component-Driven Development
A development approach that builds UIs bottom-up from isolated, reusable components, developed and tested independently before assembly into pages and apps.
The Test Pyramid
A testing strategy that favors many fast unit tests, fewer integration tests, and a small number of slow end-to-end tests.
The Testing Trophy
A testing model that weights integration tests most heavily, balancing static analysis, unit, integration, and end-to-end tests by confidence-per-cost.
Test-Driven Development (TDD)
A development discipline where you write a failing test first, write minimal code to pass it, then refactor, in short red-green-refactor cycles.
Behavior-Driven Development (BDD)
A collaborative practice that expresses requirements as concrete, executable examples in plain language shared by business, development, and testing.
Property-Based Testing
A technique that asserts general properties of code and lets a framework generate many randomized inputs to find counterexamples and shrink them.
Mutation Testing
A technique that injects small faults (mutants) into code and checks whether tests detect them, measuring how effective the test suite really is.
Flaky Test Management
A disciplined approach to detecting, quarantining, and fixing nondeterministic tests so CI signal stays trustworthy and developers keep merging.
Code Coverage Best Practices
Guidance on using code coverage as a signal of untested code rather than a target, including diff coverage and avoiding coverage gaming.
End-to-End Testing Best Practices
Guidance for writing reliable, maintainable end-to-end tests that exercise critical user journeys without becoming slow and flaky.
Test Data Management
Practices for provisioning realistic, isolated, and compliant test data so tests are reliable, repeatable, and free of production data exposure.
Visual Regression Testing
Automated testing that captures screenshots of UI states and compares them against baselines to detect unintended visual changes.
Anti-Patterns12
Magic Pushbutton
Putting business logic directly in UI event handlers, so a single button click handler holds validation, rules, and persistence with no separation of concerns.
Fat Controller
Web or API controllers that accumulate business logic, validation, and data access instead of delegating, becoming bloated and impossible to test or reuse.
Flaky Pipeline
A CI/CD pipeline that fails intermittently for reasons unrelated to the code change, eroding trust and training teams to ignore red builds.
Ice-Cream Cone (Inverted Test Pyramid)
A test suite dominated by slow manual and end-to-end tests with few unit tests, making feedback slow, brittle, and expensive to maintain.
Flaky Tests
Tests that pass and fail non-deterministically without code changes, eroding trust in the suite and masking real regressions.
Testing Implementation Details
Tests coupled to private internals rather than observable behavior, so harmless refactors break them and real bugs slip through.
Assertion Roulette
A test with many unlabeled assertions, so when one fails it is unclear which condition broke or why, slowing diagnosis.
Mystery Guest
A test that depends on external data or resources not visible in the test itself, making it opaque, fragile, and non-reproducible.
Happy-Path-Only Testing
Tests that exercise only the expected, valid flow and ignore errors, edge cases, and failures — leaving real-world conditions untested.
Excessive Mocking (Mockery)
Replacing nearly every collaborator with mocks so tests verify interactions instead of behavior, becoming brittle and detached from reality.
Slow Test Suite
A test suite so slow that developers stop running it locally and feedback arrives too late, encouraging skipped tests and large risky batches.
Coverage-Driven Testing (Coverage as a Target)
Chasing a code-coverage percentage as the goal, producing tests that execute code without meaningfully asserting behavior — high numbers, low confidence.
Tutorials14
Testing with Vitest
Set up Vitest for fast, modern JavaScript testing
How to run matrix builds in CI
Use a build matrix to test your code across multiple language versions and operating systems in parallel.
How to validate data quality with Great Expectations
Define expectations for a dataset, run validations, and surface failures so bad data is caught before it spreads downstream.
How to evaluate LLM outputs systematically
Build a repeatable evaluation suite for LLM features using reference checks, rubrics, and model-graded scoring.
How to set up a Go project with modules and tests
Start a Go project with Go modules, structure packages, and write table-driven tests using the standard testing package.
How to set up a Rust project with Cargo and tests
Create a Rust project with Cargo, add dependencies from crates.io, and write unit and integration tests.
How to set up a Python project with Poetry and pytest
Create a reproducible Python project using Poetry for dependency management and pytest for testing.
How to set up a Java project with Maven and JUnit
Create a Java project with Maven, manage dependencies, and write unit tests with JUnit 5.
How to set up a .NET project with xUnit tests
Create a .NET solution with a class library and an xUnit test project using the dotnet CLI.
How to set up a Node and TypeScript project with Vitest
Create a TypeScript project on Node.js, configure the compiler, and write fast unit tests with Vitest.
How to set up a Kotlin project with Gradle and JUnit
Create a Kotlin project with the Gradle build tool, manage dependencies, and write tests with JUnit 5.
How to build and test a Node.js REST API with Jest and Supertest
Build a small Express REST API and test its endpoints with Jest and Supertest, including setup and teardown.
How to test a FastAPI application with pytest and TestClient
Test a FastAPI app's endpoints using pytest and the built-in TestClient, with fixtures and dependency overrides.
How to test a Go HTTP API with httptest
Write fast, isolated tests for a Go HTTP handler using the standard library's net/http/httptest package.
Blueprints6
Mocha to Jest Blueprint
Test framework migration from Mocha/Chai to Jest
Python 2 to Python 3 Modernization Blueprint
Migrate end-of-life Python 2 codebases to Python 3 with automated 2to3 fixes, string/bytes correctness, and dependency upgrades.
Ruby on Rails Major Version Upgrade Blueprint
Upgrade a Ruby on Rails application across major versions with incremental dual-boot, deprecation cleanup, and gem compatibility work.
Node.js Callbacks to Async and ESM Blueprint
Modernize callback-based CommonJS Node.js services to async/await with promises and ES modules for cleaner control flow.
Ad-Hoc REST Versioning to API-First Contracts Blueprint
Move from unmanaged REST API changes to an API-first workflow with OpenAPI contracts, contract testing, and a backward-compatibility policy.
WCAG 2.2 Accessibility Remediation Blueprint
Remediate a web application to meet WCAG 2.2 AA through audit, prioritized fixes, automated testing, and governance.
Playbooks9
Python 2 to 3 Program Playbook
A coordinated program for migrating remaining Python 2 codebases to modern Python 3 with type hints and a hardened test suite.
Rails Major-Version Upgrade Program Playbook
A staged program for upgrading Ruby on Rails applications across major versions with dual-boot validation and gem modernization.
PHP Major-Version Upgrade Program Playbook
A program for upgrading PHP applications across major versions with automated rectoring, dependency updates, and staged rollout.
Tech-Debt Paydown Program Playbook
A measurable program to quantify, prioritize, and systematically pay down technical debt across a backend portfolio.
Data Quality Program Playbook
Stand up a data quality program with profiling, automated tests, SLAs, anomaly detection, and incident workflows across pipelines.
AI Red-Teaming Program Playbook
A phased program to build an AI red-teaming capability that adversarially tests LLM systems for jailbreaks, prompt injection, and harmful outputs.
API-First Design Program Playbook
A program to establish API-first practices across teams using OpenAPI contracts, mock-driven development, and contract testing.
LLM Evaluation Program Playbook
A program to build a rigorous LLM evaluation capability with offline evals, online metrics, and regression gating in CI.
WCAG 2.2 Accessibility Program Playbook
A program for bringing a product portfolio into WCAG 2.2 AA conformance through audit, remediation, and embedded accessibility governance.
Checklists7
Disaster Recovery Test Checklist
Plan and run a disaster-recovery test that proves backups, runbooks, and recovery objectives actually work end to end.
LLM Evaluation Readiness Checklist
Verification items for building a trustworthy evaluation harness before releasing changes to an LLM-powered feature.
Microservices API Contract Testing Checklist
Verification items for establishing consumer-driven contract testing across microservices to prevent integration breakage.
WCAG 2.2 Accessibility Audit Checklist
Audit a web application against WCAG 2.2 AA success criteria across perceivable, operable, understandable, and robust requirements.
React Class to Hooks Migration Checklist
Verify patterns, testing, and incremental rollout are ready before migrating React class components to function components with hooks.
jQuery to Modern SPA Migration Checklist
Plan a safe, incremental migration from a jQuery-based UI to a modern single-page-application framework.
TypeScript Adoption Readiness Checklist
Verify tooling, configuration, and an incremental strategy are in place before adopting TypeScript in a JavaScript codebase.
Comparisons2
Jest vs Vitest
Jest is the established JavaScript testing framework; Vitest is a faster, Vite-native test runner with a Jest-compatible API and first-class ESM/TS support.
Playwright vs Cypress
Two leading end-to-end testing frameworks. Playwright offers broad multi-browser and language support; Cypress provides a polished, developer-friendly experience.
Benchmarks10
Test Generation Coverage
Measures quality and coverage of AI-generated test suites for migrated code
HumanEval
A code-generation benchmark of 164 Python programming problems graded by executing unit tests, popularizing the pass@k metric.
MBPP (Mostly Basic Python Problems)
A benchmark of ~1,000 entry-level Python programming tasks with test cases, used to evaluate basic code synthesis from short descriptions.
BenchBase
An extensible Java framework (successor to OLTPBench) for benchmarking relational databases with many built-in workloads via a common JDBC harness.
SAST/DAST Detection Rate Benchmark
Measures how accurately static and dynamic application security testing tools find real vulnerabilities, reporting true-positive, false-positive, and recall rates.
Code Coverage Benchmark
Measures the proportion of code exercised by automated tests, across line, branch, statement, and function coverage, as a test-completeness indicator.
Mutation Testing Score Benchmark
Measures test-suite effectiveness by injecting small faults into code and checking how many are detected, reported as the mutation score.
Flaky Test Rate Benchmark
Measures the prevalence and impact of nondeterministic tests that pass and fail without code changes, eroding trust in the test suite.
Lighthouse Accessibility Benchmark
Measures a web page's automated accessibility conformance using Lighthouse audits mapped to WCAG, producing a 0-100 accessibility score.
Defect Escape Rate Benchmark
Measures the proportion of defects that reach production undetected by pre-release testing, indicating the effectiveness of quality gates.
FAQs6
What is the difference between SAST and DAST?
SAST (Static Application Security Testing) analyzes source code, bytecode, or binaries without running the application, finding flaws like injection o...
What is penetration testing?
Penetration testing is an authorized, simulated attack on a system, network, or application performed by security professionals to find and safely exp...
What is the test pyramid?
The test pyramid is a guideline for balancing automated tests by type and quantity. Its wide base is many fast, cheap unit tests; the middle is a smal...
What is the difference between unit, integration, and end-to-end tests?
Unit tests verify a single function or class in isolation, often with dependencies mocked, and run very fast. Integration tests check that multiple co...
What is Test-Driven Development (TDD)?
Test-Driven Development (TDD) is a practice where you write a failing automated test before writing the code that makes it pass. The cycle is 'red, gr...
What is chaos engineering?
Chaos engineering is the practice of deliberately injecting failures into a system to test its resilience before real incidents expose weaknesses. Tea...
Glossaries6
Continuous Integration (CI)
A practice where developers merge code changes into a shared repository frequently, with each merge automatically built and tested to catch integration problems early.
Pure Function
A pure function always returns the same output for the same input and has no side effects, meaning it does not read or modify any state outside its own arguments.
Dependency Injection
Dependency injection is a design technique in which an object receives the other objects it depends on from an external source rather than creating them itself, improving testability and decoupling.
Dynamic Typing
Dynamic typing is a language approach in which variable types are checked at run time rather than compile time, allowing variables to hold values of any type and offering flexibility at the cost of later error detection.
Test-Driven Development
Test-driven development (TDD) is a software practice in which developers write a failing automated test before writing the code to make it pass, then refactor, repeating in short cycles.
Mocking
Mocking is a testing technique that replaces a real dependency with a controllable stand-in object, letting a test isolate the code under test and verify how it interacts with that dependency.