Scalability
130 items tagged with "scalability"
Best Practices3
Capacity Planning
Forecasting future demand and provisioning resources ahead of need, combining organic growth, launches, and headroom to avoid both outages and waste.
CQRS (Command Query Responsibility Segregation)
An architecture pattern that separates the model that writes data (commands) from the model that reads it (queries), allowing each side to scale and evolve independently.
Cell-Based Architecture
An architecture that partitions a system into independent, self-contained cells, each serving a subset of traffic, to limit blast radius and scale through replication.
Patterns37
Publish-Subscribe
Decouples senders from receivers by routing messages through a broker or channel so publishers and subscribers never reference each other.
API Gateway
A single entry point that routes, aggregates, and secures client requests across many backend microservices.
CQRS (Command Query Responsibility Segregation)
Separates the model that writes data (commands) from the model that reads it (queries) so each can be optimized independently.
Consistent Hashing
Distributes keys across nodes so that adding or removing a node remaps only a small fraction of keys.
Sharding
Horizontally partitions a data store into independent shards so capacity and load scale beyond a single node.
Cache-Aside
Load data into a cache on demand from a data store to improve read performance and reduce load on the backing store.
Competing Consumers
Enable multiple concurrent consumers to process messages from the same queue to increase throughput and improve resilience.
Queue-Based Load Leveling
Use a queue between tasks and a service to smooth intermittent heavy loads and protect the service from being overwhelmed.
Throttling
Control the consumption of resources by an instance, tenant, or service so a system stays within capacity under load.
Claim Check
Store a large message payload externally and pass only a reference through the messaging system to avoid moving bulky data.
Valet Key
Issue a client a token granting scoped, time-limited direct access to a resource, offloading data transfer from the application.
Geode
Deploy independent geographically distributed nodes that each serve any request, placing compute close to users worldwide.
Deployment Stamps
Deploy multiple independent copies of a full application stack to scale, isolate tenants, and contain failures.
Static Content Hosting
Serve static assets from storage or a CDN instead of application servers to cut load, latency, and cost.
Index Table
Create secondary index tables over data stores queried by non-key fields to speed up lookups that would otherwise scan.
Pipes and Filters
Decompose complex processing into a sequence of independent components connected by channels so each step can scale and evolve.
Sequential Convoy
Process related messages in order while still processing unrelated messages in parallel, by grouping them into ordered sets.
Rate Limiting
Constrain the rate of operations against a service or resource to stay within quotas and avoid throttling or overload.
Point-to-Point Channel
Ensures exactly one receiver consumes each message on a channel, even when multiple consumers compete, so a message is processed once.
Publish-Subscribe Channel
Broadcasts each message to all interested subscribers so one event can notify many independent consumers without the publisher knowing them.
Splitter
Breaks a composite message into a series of individual messages so each element can be processed independently downstream.
Read-Through Cache
A caching strategy where the cache itself loads missing data from the backing store on a miss, so application code reads only from the cache.
Write-Behind Cache
A caching strategy where writes update the cache immediately and are flushed to the backing store asynchronously, maximizing write throughput at the cost of durability.
Refresh-Ahead Cache
A caching strategy that proactively reloads hot entries before they expire, so reads of popular keys never pay miss latency.
Polyglot Persistence
An architecture that uses multiple, purpose-fit data stores within one system, matching each store's strengths to each data access pattern.
Command Query Responsibility Segregation (CQRS)
Separates the write model that handles commands from the read model that serves queries, letting each be optimized, scaled, and evolved independently.
Event-Driven Architecture
An architectural style where components communicate by producing and reacting to events, enabling loose coupling, asynchronous flow, and independent scaling.
Lambda Architecture
A big-data design that runs a batch layer for accurate historical views and a speed layer for low-latency recent data, merging both at query time.
Kappa Architecture
A streaming-first data design that uses a single processing path over an immutable log, reprocessing history by replay instead of a separate batch layer.
Producer-Consumer
A concurrency pattern where producers place work on a shared bounded queue and consumers process it independently, decoupling rates and smoothing load.
Thread Pool
A concurrency pattern that reuses a fixed set of worker threads to execute many tasks, avoiding per-task thread creation cost and bounding concurrency.
Actor Model
A concurrency model where independent actors encapsulate state and communicate only by asynchronous messages, avoiding shared memory and locks.
Reactor
An event-handling pattern that demultiplexes I/O events on one or few threads and dispatches them synchronously to registered handlers, enabling scalable non-blocking servers.
Optimistic Concurrency Control
A concurrency strategy that lets transactions proceed without locking and validates at commit, retrying on conflict, assuming conflicts are rare.
Load Shedding
Deliberately rejects or drops lower-priority work when a system nears capacity, preserving stability and protecting high-priority requests under overload.
Micro Frontend
Decomposes a web frontend into independently developed and deployed pieces owned by separate teams, then composes them into one application at runtime or build time.
Pagination
Splits a large result set into smaller pages so APIs and UIs can return and traverse data incrementally instead of loading everything at once.
Anti-Patterns20
Nanoservices
Splitting a system into services so small that coordination, network, and operational overhead vastly exceed the value of each tiny service.
N+1 Query Problem
Loading a list, then firing one extra query per row to fetch related data, turning a single page load into hundreds of round-trips.
Polling the Database
Repeatedly querying a table on a tight loop to detect changes instead of using events or notifications, wasting resources and adding latency.
Chatty Data Access
Making many fine-grained round-trips to the database for one logical operation, so network latency dominates and throughput collapses under load.
No Connection Pooling
Opening a fresh database connection per request or query and closing it after, paying high handshake cost and exhausting connection limits under load.
Unbounded Result Sets
Querying without a LIMIT and loading entire growing tables into memory, so a query that was fine at launch crashes the app as data accumulates.
Lock Contention
Many threads competing for the same lock, serializing work that should run in parallel and turning a multicore machine into an expensive single-core one.
Thread-Per-Request Overload
Spawning a dedicated OS thread for every incoming request, so concurrency is capped by thread count and memory, collapsing under load instead of degrading gracefully.
Chatty I/O
Making many small, fine-grained remote or storage calls where a few coarse-grained calls would do, multiplying latency and overhead per operation.
N+1 Network Calls
Fetching a list, then making one additional remote call per item to enrich it, so a single logical operation fans out into N+1 dependency calls.
Unbounded Cache
A cache with no size limit, eviction, or expiry that grows until it consumes all memory and turns a performance optimization into an out-of-memory failure.
Cache Stampede (Dogpile Effect)
When a hot cache entry expires, many concurrent requests all miss and recompute it at once, hammering the backing store and amplifying load instead of absorbing it.
Thundering Herd
Many clients or threads waking and acting at the same instant — on a recovery, a timer, or a wakeup — creating a synchronized load spike that overwhelms the resource they target.
Head-of-Line Blocking
A single slow or stuck item at the front of a strictly ordered queue holds up everything behind it, even when the later items are unrelated and ready to proceed.
Hot Partition (Hot Shard)
A partitioning key that sends a disproportionate share of traffic to one shard, overloading it while the rest sit idle and capping the system at one node's throughput.
False Sharing
Independent variables that happen to live on the same CPU cache line, so updates by different cores invalidate each other's caches and silently destroy multicore performance.
Missing Back-Pressure
A producer that pushes work faster than the consumer can handle, with no mechanism to slow it down, so queues grow unbounded until memory or the downstream collapses.
Premature Scaling
Building for massive scale before there is load to justify it, paying the cost and complexity of distributed systems to solve problems the product does not yet have.
Missing Pagination (Unbounded Result Sets)
Collection endpoints that return all records at once with no pagination, causing huge payloads, slow queries, and out-of-memory failures as data grows.
Nanoservices (Overly Fine-Grained Services)
Splitting a system into so many trivially small services that coordination, network, and operational overhead dwarf any benefit of separation.
Tutorials9
How to autoscale pods with the Kubernetes HorizontalPodAutoscaler
Scale a Deployment automatically on CPU or custom metrics using the HorizontalPodAutoscaler and metrics-server.
How to autoscale Kubernetes nodes with the Cluster Autoscaler
Add and remove nodes automatically based on pending pods using the Cluster Autoscaler on a managed cluster.
How to configure EC2 Auto Scaling groups on AWS
Set up an EC2 Auto Scaling group with a launch template, target tracking policies, and health checks.
How to build a multi-region active-active app on AWS
Architect a multi-region active-active application on AWS with Route 53, global tables, and health-based failover.
How to set up MySQL primary-replica replication
Configure binary log replication from a MySQL primary to a replica for read scaling and high availability, then verify it.
How to implement the cache-aside pattern with Redis
Add a Redis read-through cache with TTLs and safe invalidation to reduce database load, including stampede protection.
How to partition a large PostgreSQL table by range
Use declarative range partitioning to split a large table by time, improving query pruning, maintenance, and data retention.
How to shard a MongoDB collection for horizontal scale
Enable sharding on a MongoDB cluster, choose a shard key, and distribute a collection across shards with verified balancing.
How to Store Traces at Scale with Grafana Tempo
Run Grafana Tempo as an object-storage trace backend, ingest OpenTelemetry spans, and query traces from Grafana.
Products3
Reference Architectures1
Stacks5
AWS Serverless Stack
Fully managed AWS serverless stack using Lambda, DynamoDB, and API Gateway for event-driven, pay-per-use applications with no server management.
Azure Serverless Stack
Managed Microsoft Azure serverless stack using Azure Functions and Cosmos DB for event-driven, globally distributed, pay-per-use applications.
Event-Driven Microservices Stack
Asynchronous microservices architecture using Kafka as an event backbone with independently deployable services for loose coupling and scalability.
Serverless Containers Stack
Cloud stack running standard OCI containers on managed serverless platforms for scale-to-zero, pay-per-use workloads without cluster management.
Ray Distributed ML
A unified compute stack using Ray to scale Python machine learning workloads from data processing through training to serving.
Comparisons11
Kubernetes vs Nomad
Kubernetes is the dominant, feature-rich container orchestrator; HashiCorp Nomad is a simpler, lightweight scheduler for containers and non-containerized workloads.
Serverless vs Containers
Serverless abstracts away infrastructure and scales to zero; containers give portable, full control over the runtime. The choice balances operational simplicity against flexibility.
EC2 vs Lambda
Amazon EC2 provides full virtual servers you manage; AWS Lambda runs event-driven functions with no servers to manage. Control and steady cost versus elasticity and simplicity.
RDS vs Aurora
Amazon RDS runs standard managed database engines; Amazon Aurora is AWS's cloud-native engine with a distributed storage layer offering higher performance and availability.
MongoDB vs DynamoDB
Document database with rich querying versus a fully managed key-value and document store built for predictable scale on AWS.
DynamoDB vs Cassandra
Managed AWS wide-column key-value store versus open-source Apache Cassandra for write-heavy, distributed workloads.
Postgres vs CockroachDB
The versatile single-node-first relational database versus a distributed, Postgres-compatible SQL database built for global scale.
PostgreSQL vs Cassandra
A relational ACID database for complex queries versus a distributed wide-column store built for write-heavy linear scale.
Async/Await vs Threads
Two models for concurrency: cooperative asynchronous programming with an event loop, versus preemptive OS threads. Each suits different workloads.
Java Virtual Threads vs Reactive Programming
Two approaches to scalable concurrency on the JVM: Project Loom's virtual threads keep simple blocking code, while reactive programming uses non-blocking streams.
Batch vs Real-Time Inference
Batch inference processes data in scheduled bulk jobs; real-time inference serves predictions on demand. They trade latency against throughput, cost, and complexity.
Benchmarks5
Autoscaling Responsiveness Benchmark
Measures how quickly and accurately an autoscaler adds or removes capacity in response to load changes, including reaction time, overshoot, and stabilization.
Kubernetes Scheduling Latency Benchmark
Measures how quickly the Kubernetes scheduler places pods on nodes and how the control plane scales as pod, node, and churn counts grow.
MLPerf Training
Industry-standard benchmark suite measuring how fast hardware and software systems train machine-learning models to a fixed target quality.
SPEC OMP 2012
Benchmark suite measuring shared-memory parallel performance of OpenMP applications across scientific and engineering workloads.
HPCG
High Performance Conjugate Gradients benchmark measuring HPC system performance on memory-bound, sparse computations that mirror real applications.
FAQs10
What is autoscaling in the cloud?
Autoscaling automatically adjusts the number of running resources, such as virtual machines, containers, or pods, in response to demand or defined met...
What is container orchestration?
Container orchestration is the automated management of the lifecycle of containers across a cluster of machines, including scheduling, scaling, networ...
What is the difference between SQL and NoSQL databases?
SQL (relational) databases store data in tables with fixed schemas and use SQL for queries, offering strong consistency and powerful joins; examples i...
What is database sharding?
Sharding is a horizontal partitioning technique that splits a large dataset across multiple database instances, each holding a subset of the rows. A s...
What is database replication?
Replication keeps copies of a database on multiple servers, typically a primary that accepts writes and one or more replicas that receive the changes....
What is data partitioning?
Partitioning divides a large table or dataset into smaller, manageable pieces based on a key such as date, region, or category. In databases it improv...
What is a load balancer?
A load balancer distributes incoming network traffic across multiple servers so no single instance becomes a bottleneck, improving availability and sc...
What is a microservice?
A microservice is a small, independently deployable service that owns a single business capability and communicates with other services over the netwo...
Monolith vs microservices: which should I choose?
A monolith packages all functionality into a single deployable unit, which keeps development, testing, and deployment simple and is usually the right ...
What is event-driven architecture?
Event-driven architecture is a style where components communicate by producing and reacting to events—records that something happened—rather than call...
Glossaries26
Elasticity
Elasticity is the ability of a cloud system to automatically add or remove computing resources in response to changing demand, so capacity tracks load in near real time.
Multi-Tenancy
Multi-tenancy is a software architecture in which a single deployment serves multiple customers, called tenants, while keeping each tenant's data and configuration logically isolated.
Serverless
Serverless is a cloud execution model in which the provider fully manages servers and scaling, running code in response to events and billing only for actual usage.
Function as a Service (FaaS)
Function as a service is a serverless model in which developers deploy individual functions that the cloud provider runs and scales automatically in response to events.
Infrastructure as a Service (IaaS)
Infrastructure as a service is a cloud model that provides on-demand access to fundamental computing resources such as virtual machines, storage, and networking, which customers manage themselves.
Platform as a Service (PaaS)
Platform as a service is a cloud model that provides a managed application platform, handling servers, runtimes, and scaling so developers focus on code rather than infrastructure.
Autoscaling
Autoscaling is the automatic adjustment of the number of running compute instances or resources based on demand, metrics, or schedules, without manual intervention.
Spot Instance
A spot instance is spare cloud compute capacity offered at a deep discount that the provider can reclaim with little notice, suited to fault-tolerant and interruptible workloads.
Object Storage
Object storage is a data storage architecture that manages data as discrete objects with metadata and a unique identifier in a flat namespace, accessed over HTTP APIs and scaling to massive volumes.
Cold Start
A cold start is the added latency incurred when a serverless function or container must initialize a fresh execution environment before handling a request, rather than reusing a warm, already-running one.
Horizontal Pod Autoscaler
The Horizontal Pod Autoscaler is a Kubernetes controller that automatically adjusts the number of pod replicas in a workload based on observed metrics such as CPU utilization or custom metrics.
BASE
BASE (Basically Available, Soft state, Eventual consistency) is a consistency model for distributed systems that favors availability and partition tolerance over the strict guarantees of ACID, allowing data to converge over time.
CAP Theorem
The CAP theorem states that a distributed data store can simultaneously provide at most two of three guarantees — Consistency, Availability, and Partition tolerance — forcing a trade-off when a network partition occurs.
Eventual Consistency
Eventual consistency is a guarantee that, in the absence of new updates, all replicas of a piece of data will converge to the same value over time, though reads may temporarily return stale results.
Sharding
Sharding is a database scaling technique that horizontally splits a dataset across multiple servers (shards), each holding a distinct subset of rows, so that load and storage are distributed.
Partitioning
Partitioning is the practice of dividing a large table or dataset into smaller, more manageable pieces called partitions, which can be queried and maintained independently while appearing as a single logical entity.
Replication
Replication is the process of copying and maintaining database data across multiple servers so that the same data is available on more than one node, improving availability, fault tolerance, and read scalability.
OLAP
OLAP (Online Analytical Processing) refers to systems optimized for complex analytical queries over large volumes of historical data, enabling aggregation, slicing, and multidimensional analysis for reporting and decision-making.
Data Warehouse
A data warehouse is a centralized analytical database that stores integrated, structured data from multiple sources, optimized for querying and reporting rather than transactional processing.
Data Lake
A data lake is a centralized repository that stores large volumes of raw data in its native format — structured, semi-structured, and unstructured — at low cost, with schema applied at read time rather than on ingestion.
Data Lakehouse
A data lakehouse is an architecture that combines the low-cost, flexible storage of a data lake with the management, transactions, and performance of a data warehouse, using open table formats over object storage.
Inference
Inference is the process of running a trained model on new inputs to produce outputs, as opposed to the training phase that creates the model.
Latency
Latency is the time delay between a request being made and the corresponding response beginning to arrive, typically measured as round-trip time in milliseconds.
Monolith
A monolith is an application built and deployed as a single, unified unit, where all functionality runs in one process or codebase rather than being split into independent services.
CQRS (Command Query Responsibility Segregation)
CQRS is a pattern that separates the model used to change data (commands) from the model used to read data (queries), allowing each side to be optimized, scaled, and evolved independently.
Parallelism
Parallelism is the simultaneous execution of multiple computations, typically across several CPU cores or machines, to complete work faster than sequential execution.