On-Device AI
On-device inference keeps data local, works offline, and has no per-token cost — genuinely attractive for privacy and regulated contexts. The trade is capability: local models are smaller and slower than frontier APIs. Quantization and distillation are what make it viable at all. In practice: Transcribing a confidential meeting without the audio leaving the laptop.