azure-genai-design-patterns

### Agentic AI System Design for Customer Support Automation

**Pattern Selected:** Hybrid (Orchestrator + Router) — This combines the Orchestrator pattern for high-level task delegation with the Router pattern for dynamic routing of queries to specialized agents.

**System Components:**
- **LLM:** Azure OpenAI GPT-4o (deployed as `gpt-4o-2024-05-13` in Azure AI Studio)
- **Memory:** Azure Cosmos DB for structured conversation history and user preferences
- **Tools:** Azure AI Search for knowledge base retrieval, Azure Functions for ticketing system integration, and a custom sentiment analysis tool
- **API Connections:** Twilio API for SMS interactions, Salesforce REST API for ticket updates

**Decision-Making Logic:**
1. **Query Classification:** The system first routes the user's query using a lightweight classifier (fine-tuned `text-embedding-3-small` model) to determine intent (e.g., billing, technical support, account update).
2. **Specialized Agent Assignment:** Based on intent, the query is passed to the appropriate agent:
   - **Billing Agent:** Handles payment queries using Azure AI Search to retrieve invoice data from SharePoint.
   - **Technical Agent:** Uses a tool-calling LLM to execute diagnostic commands via Azure Functions, pulling logs from Azure Monitor.
   - **Account Agent:** Updates user details by calling the Salesforce API.
3. **Fallback Mechanism:** If confidence scores for intent classification drop below 0.7, the query is escalated to a human agent via a Teams webhook.

**Error Handling:**
- **LLM Failures:** If the LLM returns an empty or invalid response, the system retries with a temperature of 0.3 and falls back to a predefined response template.
- **Tool Failures:** Failed tool calls trigger a retry with exponential backoff (1s, 2s, 4s) before escalating to human support.
- **API Rate Limits:** Implements Azure API Management policies to throttle requests and queue excess calls.

**Python Code Skeleton (Azure AI SDK):**
```python
from azure.ai.inference import ChatCompletionsClient
from azure.core.credentials import AzureKeyCredential
from azure.search.documents import SearchClient
from azure.cosmos import CosmosClient

# Initialize clients
llm_client = ChatCompletionsClient(
    endpoint="https://[YOUR-AZURE-OPENAI-ENDPOINT].openai.azure.com/",
    credential=AzureKeyCredential("[YOUR-AZURE-OPENAI-KEY]")
)
search_client = SearchClient(
    endpoint="https://[YOUR-AZURE-SEARCH-ENDPOINT].search.windows.net",
    index_name="support-knowledge-base",
    credential=AzureKeyCredential("[YOUR-AZURE-SEARCH-KEY]")
)

# Agent orchestrator
class SupportAgent:
    def __init__(self):
        self.memory = CosmosClient("[YOUR-COSMOS-CONNECTION-STRING]").get_database_client("support-db")

    def route_query(self, user_query: str) -> str:
        # Step 1: Classify intent
        intent = self._classify_intent(user_query)
        
        # Step 2: Route to specialized agent
        if intent == "billing":
            return self._handle_billing(user_query)
        elif intent == "technical":
            return self._handle_technical(user_query)
        else:
            return self._handle_account(user_query)

    def _classify_intent(self, query: str) -> str:
        response = llm_client.complete(
            messages=[{"role": "user", "content": f"Classify this query: '{query}'. Options: billing, technical, account."}],
            model="gpt-4o",
            temperature=0.1
        )
        return response.choices[0].message.content.strip().lower()

# Example usage
agent = SupportAgent()
response = agent.route_query("I need help with my recent invoice #INV-2024-00123")
print(response)  # Output: "I see your invoice #INV-2024-00123. Let me retrieve the details..."
```