This document in Well-Architected Framework: AI and ML perspective provides an overview of principles and recommendations to optimize the cost of your AI systems throughout the ML lifecycle. By adopting a proactive and informed cost management approach, your organization can realize the full potential of AI and ML systems and also maintain financial discipline.
AI and ML systems can help you unlock valuable insights and predictive capabilities from data. For example, you can reduce friction in internal processes, improve user experiences, and gain deeper customer insights. The cloud offers vast amounts of resources and quick time-to-value without large up-front investments for AI and ML workloads. To maximize business value and to align the spending with your business goals, you need to understand the cost drivers, proactively optimize costs, set up spending controls, and adopt FinOps practices.
The recommendations in this document are mapped to the following core principles:
Define and measure costs and returns
Optimize resource allocation
Enforce data management and governance practices
Automate and streamline with MLOps
Use managed services and pre-trained models
Define and measure costs and returns
To effectively manage AI and ML costs in Google Cloud, you must define and measure the cloud resource costs and the business value of your AI and ML initiatives. To help you track expenses granularly, Google Cloud provides comprehensive billing and cost management tools, such as the following:
Cloud Billing reports and tables
Looker Studio dashboards, budgets, and alerts
Cloud Monitoring
Cloud Logging
To make informed decisions about resource allocation and optimization, consider the following recommendations.
Establish business goals and KPIs
Align the technical choices in your AI and ML projects with business goals and key performance indicators (KPIs).
Define strategic objectives and ROI-focused KPIs
Ensure that AI and ML projects are aligned with strategic objectives like revenue growth, cost reduction, customer satisfaction, and efficiency. Engage stakeholders to understand the business priorities. Define AI and ML objectives that are specific, measurable, attainable, relevant, and time-bound (SMART). For example, a SMART objective is: "Reduce chat handling time for customer support by 15% in 6 months by using an AI chatbot".
To make progress towards your business goals and to measure the return on investment (ROI), define KPIs for the following categories of metrics:
Costs for training, inference, storage, and network resources, including specific unit costs (such as the cost per inference, data point, or task). These metrics help you gain insights into efficiency and cost optimization opportunities. You can track these costs by using Cloud Billing reports and Cloud Monitoring dashboards.
Project-specific metrics. You can track these metrics by using Vertex AI Experiments and evaluation.
Predictive AI: measure accuracy and precision
Generative AI: measure adoption, satisfaction, and content quality
Computer vision AI: measure accuracy
To validate your ROI hypotheses, start with pilot projects and use the following iterative optimization cycle:
Monitor continuously and analyze data: Monitor KPIs and costs to identify deviations and opportunities for optimization.
Make data-driven adjustments: Optimize strategies, models, infrastructure, and resource allocation based on data insights.
Refine iteratively: Adapt business objectives and KPIs based on the things you learned and the evolving business needs. This iteration helps you maintain relevance and strategic alignment.
Establish a feedback loop: Review performance, costs, and value with stakeholders to inform ongoing optimization and future project planning.
Use Cloud Monitoring to collect metrics from various sources, including your applications, infrastructure, and Google Cloud services like Compute Engine, Google Kubernetes Engine (GKE), and Cloud Run functions. To visualize metrics and logs in real time, you can use the prebuilt Cloud Monitoring dashboard or create custom dashboards. Custom dashboards let you define and add metrics to track specific aspects of your systems, like model performance, API calls, or business-level KPIs.
Use Cloud Logging for centralized collection and storage of logs from your applications, systems, and Google Cloud services. Use the logs for the following purposes:
Track costs and utilization of resources like CPU, memory, storage, and network.
Identify cases of over-provisioning (where resources aren't fully utilized) and under-provisioning (where there are insufficient resources). Over-provisioning results in unnecessary costs. Under-provisioning slows training times and might cause performance issues.
Identify idle or underutilized resources, such as VMs and GPUs, and take steps to shut down or rightsize them to optimize costs.
Identify cost spikes to detect sudden and unexpected increases in resource usage or costs.
Use Looker or Looker Studio to create interactive dashboards and reports. Connect the dashboards and reports to various data sources, including BigQuery and Cloud Monitoring.
Optimize resource allocation
To achieve cost efficiency for your AI and ML workloads in Google Cloud, you must optimize resource allocation. To help you avoid unnecessary expenses and ensure that your workloads have the resources that they need to perform optimally, align resource allocation with the needs of your workloads.
To optimize the allocation of cloud resources to AI and ML workloads, consider the following recommendations.
Use autoscaling to dynamically adjust resources
Use Google Cloud services that support autoscaling, which automatically adjusts resource allocation to match the current demand. Autoscaling provides the following benefits:
Cost and performance optimization: You avoid paying for idle resources. At the same time, autoscaling ensures that your systems have the necessary resources to perform optimally, even at peak load.
Improved efficiency: You free up your team to focus on other tasks.
Increased agility: You can respond quickly to changing demands and maintain high availability for your applications.
The following table summarizes the techniques that you can use to implement autoscaling for different stages of your AI projects.
No comments:
Post a Comment