Bedrock on demand and provisioned thruput inferencing difference

 On Amazon Bedrock: You Don't Need to Handle Idle Time 🎉

​If you are using Amazon Bedrock's standard On-Demand models (like Claude 3.5 Sonnet or Amazon Nova), idle time costs you exactly $0.

​How it works: Bedrock On-Demand is 100% serverless. You do not pay for running servers or active endpoints. AWS manages the underlying infrastructure.

​The Billing: You are charged strictly per token processed (input and output). If your startup platform goes 12 hours overnight with zero user queries, your Bedrock bill for those 12 hours is exactly $0.

​⚠️ The Exception (Provisioned Throughput): If you purchase Provisioned Throughput (allocated hourly capacity for large scale or custom models), you are charged per hour regardless of use. To handle idle time here, you must use Application Auto Scaling policies to programmatically reduce your provisioned model units during off-peak hours (e.g., late at night).