TL;DR: You don’t need to rewrite your .NET application to add AI. Start small by wrapping Azure OpenAI behind a clean interface, then add RAG or agent orchestration only when the use case justifies it. This keeps AI adoption incremental, testable, and aligned to your existing architecture.
Your team just got the brief: add AI to the product. The business wants a timeline by the end of the week.
The immediate instinct is to think about what you need to rebuild. A new service, maybe. A new architecture. A new team!
As fun as this might sound, these options involve too much investment for your first project.
Adding AI to an existing .NET application doesn’t require a rewrite. You can add it by choosing two decisions and then implementing one of three patterns.
The Common Mistake in .NET AI Integration
Teams treat AI like an entirely separate system, a “side-car” service that runs next to the real application and communicates via API.
You can see why. AI demos are usually standalone. The Python notebooks, the Azure OpenAI quickstarts, are all self-contained. So teams assume the pattern is: build AI over there, integrate it from here.
This creates real problems fast:
- Context is lost every time you cross the boundary
- Two systems to maintain, monitor, and deploy
- AI becomes the thing you bolt on rather than bake in
- The latency and cost of every cross-system call compounds quickly
The better pattern is to treat AI as a capability layer inside your existing architecture, not alongside it.
Two Low-Risk Entry Points for Adding AI to .NET
Before writing a line of code, decide where in your application AI actually adds value. There are two entry points that work well in existing apps:
- The service layer: AI as a service behind an interface. Your existing code calls it; nothing else changes.
- The data processing pipeline: AI augments what you’re already doing with data. Think summarisation, classification, extraction. The data’s already moving through your system; you’re just adding a step.
Pick the entry point where the value is obvious and the risk is low.
The three patterns below aren’t tied one-to-one to these entry points; they’re implementation choices you apply once you’ve picked where AI enters. Pattern 1 (wrap Azure OpenAI behind an interface) works for either entry point and is where you should start regardless. Pattern 2 (RAG) is specific to the data processing pipeline. Pattern 3 (orchestration) applies to either entry point once a single call to the model isn’t enough, especially when you need to call multiple services, reason across steps, or decide which tool to use.
Decision guide: SDK vs RAG vs Agent Framework
Use case | Best fit | Choose this when… | Avoid this when… |
Single, well-defined AI task | Azure OpenAI SDK behind an interface | You need summarisation, classification, extraction, rewriting, or another focused model call that fits cleanly behind a service boundary. | The answer depends on fresh or internal data that the model does not already have. |
Answers grounded in your own content | RAG with Azure AI Search + Azure OpenAI | Users need to ask questions over documents, knowledge bases, policies, product data, or changing enterprise content. | The task is just a simple transformation and does not need retrieval. |
Multi-step workflows across services | Microsoft Agent Framework | The AI needs to decide which tools or services to call, chain multiple steps, or coordinate business actions across your existing .NET services. | You only need one predictable model call; the extra orchestration will add complexity without much value. |
Pattern 1: Wrap Azure OpenAI Behind a Clean Interface
Whichever entry point you pick, start here.
Define an interface. Put the Azure OpenAI SDK behind it. Your business logic never talks to the SDK directly.
public interface ITextAnalysisService
{
Task<string> SummariseAsync(string input, CancellationToken ct = default);
}
Your Azure OpenAI implementation:
public class AzureOpenAITextAnalysisService : ITextAnalysisService
{
private readonly AzureOpenAIClient _client;
private readonly string _deploymentName;
public AzureOpenAITextAnalysisService(AzureOpenAIClient client, IConfiguration config)
{
_client = client;
_deploymentName = config["AzureOpenAI:DeploymentName"]!;
}
public async Task<string> SummariseAsync(string input, CancellationToken ct = default)
{
var chatClient = _client.GetChatClient(_deploymentName);
var response = await chatClient.CompleteChatAsync(
[
new SystemChatMessage("Summarise the following text in 2-3 sentences."),
new UserChatMessage(input)
], cancellationToken: ct);
return response.Value.Content[0].Text;
}
}
And a test double for your unit tests:
public class FakeTextAnalysisService : ITextAnalysisService
{
public Task<string> SummariseAsync(string input, CancellationToken ct = default)
=> Task.FromResult($"[Summary of: {input[..Math.Min(50, input.Length)]}...]");
}
Why does this matter? The Azure OpenAI SDK is an implementation detail. Your business logic shouldn’t care whether the summary comes from GPT-4o, a smaller model, or a stub in tests. The interface keeps AI swappable, testable, and isolated from everything else.
Pattern 2: RAG With Your Existing Data
Most enterprise AI use cases boil down to: our users want to query our data using natural language.
The answer is Retrieval Augmented Generation (RAG). You don’t train a model on your data. You store your data in a vector index, retrieve the relevant chunks at query time, and pass them as context to the LLM.
In Azure, the stack is Azure AI Search + Azure OpenAI.
The flow in C# looks like this:
public async Task<string> QueryDocumentsAsync(string userQuestion, CancellationToken ct = default)
{
// 1. Embed the user's question
var embeddingClient = _openAIClient.GetEmbeddingClient(_embeddingDeployment);
var embeddingResult = await embeddingClient.GenerateEmbeddingAsync(userQuestion, ct: ct);
var queryVector = embeddingResult.Value.ToFloats();
// 2. Retrieve relevant chunks from Azure AI Search
var searchOptions = new SearchOptions
{
VectorSearch = new VectorSearchOptions
{
Queries = { new VectorizedQuery(queryVector) { KNearestNeighborsCount = 5, Fields = { "contentVector" } } }
}
};
var searchResults = await _searchClient.SearchAsync<IndexedDocument>(searchOptions, ct);
// 3. Build a grounded prompt
var chunks = new List<string>();
await foreach (var result in searchResults.Value.GetResultsAsync())
chunks.Add(result.Document.Content);
var context = string.Join("\n\n", chunks);
var prompt = $"Answer the question using only the context below.\n\nContext:\n{context}\n\nQuestion: {userQuestion}";
// 4. Generate the answer
return await _textAnalysisService.SummariseAsync(prompt, ct);
}
This is how you query your own documents, knowledge base, or database content without building a custom model, and without sending all your data to the LLM upfront.
When is RAG better than fine-tuning? Almost always, for enterprise apps. Your data changes. RAG handles that. Fine-tuning doesn’t.
Pattern 3: Multi-Step Orchestration with Microsoft Agent Framework
Reach for Microsoft Agent Framework when a single call isn’t enough.
Microsoft Agent Framework (MAF) is Microsoft’s current SDK for AI orchestration in .NET. It succeeds Semantic Kernel and AutoGen by combining both into one framework. It gives you a clean way to expose your existing .NET services as tools an agent can call.
var customerService = new CustomerService(_customerRepo);
var orderService = new OrderService(_orderRepo);
// Wrap your existing .NET services as tools
AIAgent agent = new AzureOpenAIClient(new Uri(endpoint), new DefaultAzureCredential())
.GetChatClient(deploymentName)
.AsAIAgent(new ChatClientAgentOptions
{
Name = "OrderAssistant",
ChatOptions = new ChatOptions
{
Instructions = "You are a helpful assistant for order and customer queries.",
Tools =
[
AIFunctionFactory.Create(customerService.GetCustomerByEmailAsync),
AIFunctionFactory.Create(orderService.GetMostRecentOrderAsync)
]
}
});
// The agent decides which tools to call and in what order
var response = await agent.RunAsync(
"Find the most recent order for customer john@example.com and summarise its status.");
The key thing here: your existing services don’t change. You’re wrapping their methods as tools so the agent can discover and invoke them. The business logic stays where it is.
Use Microsoft Agent Framework when you want AI to orchestrate across multiple services. Use the raw SDK when you have a single, well-defined AI task. Don’t reach for a full agent framework for a simple summarisation call – it’s overkill.
For a deeper look at production agent design, see Arinco’s post on why the Microsoft Agent Framework pipeline matters for production .NET agents.
What Not to Do
A few patterns that cause real problems when implementing AI:
- Don’t skip error handling on LLM calls. The API will fail. Responses will be malformed. Rate limits will be hit. Handle RequestFailedException, set sensible timeouts, and have a fallback for when the AI is unavailable.
- Don’t ignore token costs. Log token usage from day one. Set up Azure Cost Management alerts. A feature that runs fine in development can surprise you in production if you didn’t model the cost per request.
- Don’t pass huge context windows blindly. Sending the entire content of a large document on every request is both expensive and slow. Chunking, caching, and retrieval strategies exist; use them.
Where to Start with AI in an Existing .NET App
There are three patterns here.
Pick one pattern for one high-value use case. Ship it quickly, learn from real usage, then expand with confidence.
If I had to recommend a starting point: wrap Azure OpenAI behind an interface and use it for one well-defined task, such as summarisation, classification, or extraction. That gives you faster delivery because the change is small, lower risk because the AI boundary is isolated, and easier maintainability because the rest of the application stays familiar.
Once that foundation is in production and you understand the costs, failures, and user feedback, RAG or Microsoft Agent Framework becomes an incremental step, not a rearchitect.
You don’t need to rebuild to get business value from AI. You need a small, well-contained first step.
This post was originally published at thegroundeddeveloper.substack.com.
