Redshift Serverless
/tldr: Automatic, elastic data warehousing with pay-per-use consumption.
1. Core Concept: Redshift Processing Units (RPU)
Redshift Serverless abstracts away the physical clusters and nodes. Instead of provisioning capacity, you define a **base capacity** using Redshift Processing Units (RPUs), which represent the compute power required for your workload.
The most significant benefit is the elimination of sizing and scaling decisions. The system automatically and instantly scales the RPU capacity up or down to meet the demands of concurrent queries and complex workloads, ensuring consistent performance.
How Serverless Works
- **No Clusters to Manage:** You only create a "namespace" and a "workgroup" instead of provisioning specific cluster types (e.g., DC2, RA3).
- **Automatic Scaling:** Redshift Serverless handles all concurrency and scaling automatically, spinning up resources *instantly* to handle unexpected bursts.
- **Storage is Separate:** Like RA3 nodes, storage is automatically managed and scaled using high-performance, durable storage, separating compute costs from storage costs.
2. Consumption-Based Pricing
Pricing for Serverless is purely based on consumption, making it ideal for unpredictable or intermittent analytical workloads, such as development/testing environments or reporting that runs only during business hours.
Cost Transparency
- **RPU Hour:** You are charged only for the RPU-hours consumed while the data warehouse is active. There is no cost when the warehouse is paused (after a period of inactivity).
- **Auto-Pausing:** The service automatically pauses compute resources after a short period of inactivity, immediately stopping RPU charges.
- **Cost Controls:** You can set maximum spending limits (usage limits) to prevent unexpected overages, giving you peace of mind in an auto-scaling environment.
3. When to Choose Serverless
While provisioned clusters give you maximum control and predictable pricing for 24/7 workloads, Serverless is generally recommended for almost all new workloads unless you require specific, highly granular control over node configuration.
Key Scenarios
- **Unpredictable Workloads:** Where usage spikes significantly (e.g., end-of-quarter reporting, Black Friday sales analysis).
- **Rapid Prototyping/Dev/Test:** Instant deployment and easy tear-down without needing to estimate cluster size.
- **Data Sharing:** Serverless makes it exceptionally easy to share live data sets using **Redshift Data Sharing** across different workgroups or AWS accounts.
Code Simplicity (No Configuration)
-- Example of connecting to a Redshift Serverless endpoint.
-- No cluster name, node type, or node count is needed for the connection string.
CREATE EXTERNAL SCHEMA external_data
FROM DATA CATALOG
DATABASE 'my_glue_catalog_db'
IAM_ROLE 'arn:aws:iam::123456789012:role/RedshiftServerlessAccessRole';
-- All analytical queries run seamlessly on the auto-scaling RPU capacity.
SELECT region, COUNT(customer_id)
FROM external_data.transactions
GROUP BY 1;
Redshift Serverless: Get the speed of Redshift without the cluster management overhead.