Salesforce utilizza i controlli SageMaker per distribuire i modelli Agentforce tra le zone di disponibilità

Cosa è successo

AWS says Salesforce deployed Agentforce models with SageMaker Inference Components and used the new SchedulingConfig parameter to distribute model copies across Availability Zones and instances. Salesforce reported an 8x reduction in infrastructure costs from co-hosting multiple models on shared GPUs, but the default placement behavior did not guarantee the two-zone resilience required for its production models.

AWS and Salesforce describe a production-serving problem involving Agentforce, Salesforce’s AI foundation for agents. SageMaker Inference Components allow multiple models to share GPU-backed infrastructure, which the source says reduced Salesforce’s infrastructure costs by eight times. The tradeoff was that SageMaker’s default placement algorithm evaluated each deployment operation independently. Copies of a particular model could therefore be unevenly distributed even when the endpoint itself used multiple Availability Zones. This left endpoint-level multi-zone configuration insufficient to guarantee model-level distribution. The issue was specifically the relationship between shared infrastructure and the placement of individual model copies.

The new control is exposed through the SchedulingConfig parameter in the CreateInferenceComponent API. Its AvailabilityZoneBalance setting governs how evenly copies are distributed between zones, while PlacementStrategy controls placement within each zone. SPREAD distributes copies across as many instances as possible to improve fault isolation; BINPACK puts copies on fewer instances to improve utilization. The source says Salesforce selected SPREAD for its production high-availability requirement. These settings make the intended placement behavior explicit at deployment time and connect the choice of distribution to the operational goal being pursued.

AWS gives a two-zone example with four instances and four copies of a model. With SPREAD and a maximum imbalance of one copy, the intended result is two copies in each zone. For a model requiring only two copies, a maximum imbalance of zero targets exactly one copy per zone. The same scheduling settings are intended to remain in effect during scale-out, scale-in, and endpoint or inference-component updates. The source also recommends a consolidation strategy for longer-term cleanup after repeated scaling operations. Together, these details describe placement as an ongoing scheduling concern rather than a setting applied only once at initial deployment.

Dettagli della fonte: aws.amazon.com ↗

Perché è importante

The deployment addresses a practical problem in serving AI systems: a multi-zone endpoint can still leave all copies of one model concentrated in a single zone or instance. The configuration gives enterprise teams explicit controls for balancing availability-zone placement and choosing between fault isolation and higher utilization.

The distinction between endpoint-level and model-level resilience is important for organizations operating many AI models on shared infrastructure. A multi-zone endpoint does not automatically ensure that every model has a copy in every zone. If a model’s copies are concentrated, an instance failure or zone outage can remove that model even though other workloads on the endpoint remain available. This means a broad availability-zone design can appear resilient while leaving a particular model exposed. The placement question therefore has to be evaluated at the level of each model’s copies.

The placement feature connects reliability requirements with an explicit resource tradeoff. SPREAD can reduce the number of model copies lost in an instance failure, while BINPACK can improve accelerator utilization by concentrating workloads. For Salesforce, the source presents the choice as a way to retain the economic benefit of multi-model GPU hosting while meeting an internal requirement that every production model have two-zone support. The configuration does not remove the tradeoff; it gives teams a direct way to choose how their copies occupy the available instances and zones.

The reported result is consequential for enterprise AI operations because it moves high availability from a general architecture goal into deployment settings that can be inspected and managed. The source says Salesforce achieved two-zone compliance for its model fleet, preserved its co-hosting savings, maintained distribution during scaling, and avoided breaking zone balance during model updates. Those are claims from the AWS customer-success account, not independently audited performance findings. The post does not establish how the system behaved during a real zone outage or whether every model and region had identical conditions. The implementation account is therefore useful for understanding the control and its stated outcome, while leaving independent validation open.

Cosa guardare dopo

The source does not provide independent uptime measurements, outage-test results, latency data, or a complete accounting of the reported cost reduction. Teams adopting the pattern will need to verify capacity in each target zone, monitor placement after scaling, and determine whether best-effort placement meets their own compliance requirements.

Capacity remains a central limitation. AWS recommends On-Demand Capacity Reservations in each target Availability Zone for zone-constrained regions, and the source says Salesforce pre-provisioned reserved GPU capacity to help obtain balanced placement. Without sufficient capacity, SageMaker may partially deploy copies on available instances because the enforcement mode described is permissive. That makes the feature usable under constraint, but it can leave the final distribution less balanced than intended. The desired configuration and the placement that is actually achieved can consequently differ when the required capacity is not available in every target zone.

Operators will need to monitor whether the desired placement persists over time. The source points to SageMaker AI Insights and CloudWatch metrics covering availability-zone skew, inference-component copy counts by zone, rebalancing events and duration, and insufficient-capacity errors. It also warns that an HA-critical component should not be reduced to one copy, because one copy cannot span two zones. The practical question is how quickly teams detect and remediate an imbalance before it becomes an availability problem. Monitoring must cover both the number of copies and their distribution, especially as scaling and updates change the deployment.

Important details remain unknown from the source. It does not state the geographic regions involved, the number of production endpoints or models covered, the cost of reserved capacity, the effect on latency and throughput, or the availability target Salesforce was trying to meet. It also does not provide comparative failure testing against the default algorithm. Enterprise teams should therefore treat the post as an implementation pattern and customer-reported result, then validate capacity, failover behavior, monitoring, and total cost in their own environments. Those checks are necessary to determine whether the reported balance between resilience and utilization applies to their own workloads and operating conditions.

Guide e quiz correlati