Mohit Bhat, AI and Digital Technology Leader and Chief Delivery and Innovation Officer at Tenarai.

getty

​Enterprise AI is at an inflection point. What began with centralized, cloud-scale large language models (LLMs) is moving toward distributed, domain-specific intelligence operating closer to data, users and devices.​

Small language models (SLMs) are emerging as a foundational layer in production AI. Gartner predicted that, by 2027, organizations will implement small, task-specific AI models three times more frequently than LLMs. Meanwhile, SLMs are also a major focus for improving on-device experiences for providers such as Microsoft, Meta and Google as they advance edge-native AI capabilities.

In my experience, SLMs provide a more practical balance of performance, cost efficiency, privacy and control than larger models for many enterprise workloads.​

As adoption accelerates, the emphasis is shifting from scale to specialization, making fine-tuning the primary mechanism for embedding domain expertise into these models.​

Fine-Tuning Powers High-Performance SLMs

Fine-tuning is transforming SLMs from efficient components into high-performance, enterprise-grade systems.

While base SLMs can outperform larger models on narrowly defined, repeatable tasks, fine-tuning extends this advantage by enabling consistent, domain-specific outputs aligned to targeted workflows.

Compared to prompt engineering, fine-tuning can deliver greater determinism, lower operational overhead and stronger alignment with enterprise requirements.​Research from Microsoft and the University of Washington shows, for example, that parameter-efficient techniques such as Low-Rank Adaptation (LoRA) and Quantized LoRA (QLoRA) can reduce resource requirements by more than 90%.

By updating only a small subset of model parameters, these approaches enable efficient specialization, often on a single GPU, without the cost and complexity of retraining large models.

Enterprises are using fine-tuning to encode proprietary knowledge, business rules and operational constraints directly into models, creating a durable competitive advantage that cannot be replicated through prompting alone.​

Fine-tuning is now an architectural pattern: systematically specializing smaller models to deliver measurable business outcomes with greater efficiency, control and repeatability.​

How Fine-Tuning SLMs Delivers ROI For Enterprises

In my experience, fine-tuned SLMs can deliver measurable ROI by improving unit economics and operational performance. The reduced compute requirements and token usage lead to lower cost per task, while domain-specific tuning increases accuracy and consistency for more reliable automation.

Meanwhile, the faster inference from SLMs supports real-time decisioning, and lightweight deployment allows AI to scale across high-volume workflows.​​

However, these benefits come with trade-offs. While fine-tuned SLMs deliver strong performance within defined domains, they require clear task boundaries to avoid degradation when applied to broader use cases. They also introduce upfront investment in data curation and model development, with outcomes highly dependent on training data quality.​

In one internal evaluation that I conducted using a LLaMA 3.1 8B base model, fine-tuned SLMs outperformed both the base model and larger LLMs on structured, domain-specific tasks, demonstrating higher output consistency and template fidelity.

While inference latency was higher in a controlled environment compared to cloud-based LLMs, the SLM achieved near-zero marginal cost, highlighting a key trade-off between speed and economics.​

Over time, models must be maintained to reflect evolving business rules and avoid drift. Organizations must also determine when to encode knowledge through fine-tuning versus accessing it dynamically through retrieval-based approaches.​

When managed effectively, these trade-offs can be outweighed by the operational and economic advantages fine-tuned SLMs provide.​

Fine-Tuned SLMs In Production Across Industries

Across industries, a hybrid operating model is developing. Fine-tuned SLMs handle high-volume, deterministic tasks locally, while larger models or human review serve as escalation paths for ambiguous, creative or high-risk scenarios.​

In healthcare, fine-tuned SLMs are deployed in HIPAA-compliant environments to support clinical workflows. Common applications include automated discharge summaries, radiology note summarization and evidence-based clinical documentation that adheres to strict template and compliance requirements.​

In financial services, organizations rely on on-premise SLMs for low-latency fraud triage, and they can achieve sub-100 millisecond response times, improving detection speed while reducing operational costs. Additional use cases include Know Your Customer (KYC) automation, compliance screening and structured loan underwriting.​

Legal and corporate teams use fine-tuned SLMs to automate contract analysis, clause extraction and internal knowledge search. When deployed in private infrastructure, these models deliver higher consistency, faster processing and reduced risk of data leakage.​

Operationalizing Fine-Tuned SLMs

To begin adopting fine-tuned SLMs, enterprises should start with narrow, template-driven workflows such as document summarization or compliance extraction, where outputs are structured and measurable.

In this process, the key actions include:​

• Deploying parameter-efficient fine-tuning techniques (e.g., LoRA) on curated domain data to maximize structure adherence.

• Defining and tracking KPIs such as ≥90% schema adherence, <5% escalation rates and cost per validated output.

·• Deploying models locally or in private environments to ensure data sovereignty and regulatory compliance.

• Optimizing latency through quantization, batching and hybrid routing strategies.​

Meanwhile, several emerging trends are expanding the potential of fine-tuning in enterprise SLMs:​​

Model Compression And Transfer

Large, high-capability models are increasingly used to train smaller models, which are then fine-tuned for specific workflows. This approach converts frontier AI capabilities into scalable, cost-efficient operational systems.​

Distillation And Fine-Tuning Hybrids

Distilled models combined with fine-tuning can approximate the performance of larger models while operating with significantly lower latency and cost. In this model, large systems provide intelligence, while smaller models execute.​

Synthetic Data And Teacher Models

Particularly in regulated industries, synthetic data generation is reducing reliance on human labeling. Teacher models guide training, enabling faster iteration cycles and improved control over edge cases and risk scenarios.​

Targeted, Workflow-Level Fine-Tuning

Making fine-tuning more granular, aligned to specific functions, workflows and teams can accelerate ROI, clarify ownership and reduce organizational risk.​​

Conclusion

Fine-tuned SLMs are becoming a foundational layer in enterprise AI, redefining how intelligent systems are built, deployed and scaled. What began as an efficiency play is evolving into a new operating model, defined by specialization, control and distributed execution.​

Organizations that adopt this approach can achieve lower costs, faster decisioning and tighter alignment with domain-specific requirements, while maintaining greater control over data and governance.​​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?