...deploy AI

Fastest path, uneven traffic, no ML-operations team? → hosted API (pay per use). Use batch APIs for offline jobs and prompt caching for repeated context; both cut cost substantially.
Need enterprise controls (private networking, data residency, quotas, service-level agreements)? → models through your cloud's AI platform: Amazon Bedrock, Microsoft Foundry, or Google's Gemini Enterprise Agent Platform (formerly Vertex AI)
Data must stay in your environment, or volume is high and steady? → self-host open-weight models with an inference server (vLLM, SGLang, NVIDIA NIM / TensorRT-LLM, llama.cpp), often on Kubernetes. Size GPUs for the model weights plus the KV cache for your context lengths and concurrency.
Must work offline, keep data on the device, or respond instantly? → on-device / edge: Apple's Foundation Models framework and Core ML, Android's on-device Gemini Nano, ONNX Runtime, ExecuTorch, LiteRT (formerly TensorFlow Lite), MLX on Macs. Quantize to 4–8 bits (GGUF, AWQ, GPTQ formats).
Prototyping on a laptop? → Ollama, LM Studio or llama.cpp
Classic ML model (forecast, classifier)? → batch scoring on a schedule, or a real-time endpoint (MLflow, BentoML, or your cloud's ML endpoints)

Cost levers

Lever	Typical effect
Smaller model, or routing easy requests to one	Large savings when quality holds (check with evals)
Prompt caching	Cheaper and faster repeated context (system prompts, documents)
Batch processing	Lower price for work that can wait
Quantization (8-bit, 4-bit)	Less memory, faster; small accuracy loss
Mixture-of-experts models	Fewer active parameters per token than dense models of similar quality
Shorter outputs, fewer retries	Fewer tokens per completed task
Caching whole answers	Free repeats for common questions

Version everything: prompts, models, retrieval indexes, datasets.
Trace every call (inputs, outputs, tool calls, latency, cost); OpenTelemetry has conventions for generative AI.
Monitor quality (sampled human or LLM-judge review), drift, latency, cost per task and error rates.
Plan for failure: timeouts, fallback models, rate-limit handling, a kill switch and fast rollback.
Re-run evals before every model, prompt or index change → ...evaluate and trust AI

On the device — Apple Intelligence runs a model of roughly 3 billion parameters on iPhones, iPads and Macs and uses a larger model on Apple's own servers for heavier requests. Through the Foundation Models framework, any app can use the on-device model offline, with no per-request inference cost. Foundation Models framework | Apple Developer
Big model, tight latency — Stripe runs its payments foundation model on the critical path of transactions, within a latency budget of about a tenth of a second. Stripe: foundation models in production | ZenML LLMOps Database
Freshness vs. cost — Netflix retrains its recommendation foundation model monthly and fine-tunes it daily, and lets other teams use it as precomputed embeddings (cheap, slightly stale) or inside their own models (fresh, slower). Integrating foundation models into production personalization | ZenML LLMOps Database

GPU memory is often the real limit: long contexts and many users need room for the KV cache.
Hidden costs: engineers on call, upgrades, security patching and idle GPUs.
Vendor changes: APIs deprecate models; plan for migrations.

Where next? ◀ Build it right · ...choose a model · ...customize a model · ...pick a product or platform · ▶ ...evaluate and trust AI