AI Support Chat
Functional Specification
1. Purpose & Overview
This outlines the functional requirements for the AI-powered chat support feature integrated within the CommandIT platform. This AI agent leverages data directly from CommandIT's internal PSA, RMM, documentation, identity synchronization (AD/M365), and other modules to provide context-aware, first-level IT support to end-users accessing the platform. The agent aims to resolve issues automatically where possible, ensure efficient escalation when necessary, improve user experience, increase support efficiency, ensure consistency, and capture data for continuous improvement of the CommandIT platform and support services.
1a. Core Principles & Goals
This document specifies the CommandIT Support AI agent designed to:
- Provide a helpful, empathetic, confident, and human-like conversational experience with high emotional intelligence (EQ).
- Leverage all accessible data within the CommandIT platform to avoid asking users for information already known.
- Possess superior context awareness compared to human agents by synthesizing CommandIT data from multiple modules proactively.
- Resolve common issues directly and automatically, including potential script execution via the CommandIT Agent where safe and appropriate.
- Minimize the creation of duplicate tickets within the CommandIT PSA module by identifying existing relevant tickets, problems, or incidents.
- Identify recurring or related issues across users and devices using CommandIT data to aid troubleshooting and potentially trigger Problem Management workflows within CommandIT.
- Ensure tickets are created accurately, completely, and efficiently within CommandIT using templates and contextual data.
- Maintain a user-focused approach, respecting requests for human assistance promptly.
- Operate within defined communication policies, avoiding admission of fault or making unauthorized commitments.
- Attempt to de-escalate frustrated or hostile users using defined protocols.
- Operate securely, respecting user privacy and permissions, especially when performing actions or accessing data within the CommandIT platform.
- Accurately track its operational costs (e.g., LLM usage) and associate them with interactions/tickets as non-billable expenses for internal accounting and KPI analysis.
- Continuously improve through feedback loops and analysis of operational data captured by CommandIT and LangSmith, including identifying its own capability gaps.
2. Core Chat Flow
- User Access: User initiates an AI Chat session via the CommandIT Agent (systray application on Windows, menubar application on macOS) or the CommandIT web portal.
- Authentication & Context: User is assumed authenticated within CommandIT. The platform provides initial context (UserID, OrgID, ChatStartTimeUTC).
- AI gathers extended initial context using CommandIT tools.
- AI greets user and determines intent.
- AI gathers necessary information (checking tools first, then asking user if needed).
- AI attempts resolution (KB lookup, automated actions, scripting) OR prepares for escalation/ticketing.
- AI creates/updates PSA ticket accurately within the CommandIT PSA module (following language rules).
- AI determines next steps (resolution confirmation, escalation, scheduling, field visit request, order status query).
- AI executes next steps (provides info, initiates handoff, confirms schedule, may force agent sync).
- AI concludes interaction and performs cleanup (logging time, notes in English, logging AI cost as expense).
- CommandIT platform potentially triggers post-chat actions (survey, human follow-up offer).
3. Initial Context Gathering
- Goal: Provide the AI with immediate context about the user and their environment within CommandIT.
- Mechanism:
- Core Context (CommandIT Platform Provided): Upon chat initiation via Agent or Portal, the CommandIT platform must inject the following based on the user's authenticated session:
- UserID: Identifier for the logged-in CommandIT user (Users.user_id).
- OrgID: Identifier for the user's organization (Users.org_id, Organizations.org_id).
- ChatStartTimeUTC: Timestamp when the chat session was initiated.
- Extended Context (AI Gathered via Tools): Using the provided UserID and OrgID, the AI will perform initial tool calls against the CommandIT database/API at the start of the conversation to retrieve:
- User Details: From Contacts & Users tables - Name, Title, Email, Phone(s), Role/Permissions (via UserRoleAssignments, Roles), Location (Locations), VIP status (is_vip), Preferred Language (preferred_language), Timezone.
- User Group Membership: Via UserTeamAssignments (Teams).
- Organization Details: From Organizations table - Name, Account Manager, Primary Tech, Default Settings.
- User's Assigned Devices: From Devices table - Name, Type, Status, OS, RMM Agent Status (agent_version, last_agent_checkin, agent_health_status). Implementation Detail: Focusing on a device should trigger the agent's 'fast mode' (approx. 30s sync).
- User's Open Tickets: From Tickets table (filtered by requester_user_id, status not closed/resolved).
- Relevant Org-wide Status: From Tickets (is_major_incident = true), possibly Alerts.
4. CommandIT Platform & Integration Tools
- Tool Integration Framework: Assumed framework like LangGraph integrated within CommandIT, managed via LangSmith.
- Guiding Principle: Tool-First Approach: Before prompting the user, the AI MUST first attempt to retrieve information via available CommandIT tools.
- Mixed Language Data Handling: The AI must handle English data from CommandIT tools/database fields while formulating user-facing responses in the detected/supported chat language.
- Available Tools (Conceptual List, accessing CommandIT data):
- Ticket Management (CommandIT PSA Module):
- getUserTickets(userId, status=None, timePeriod='e.g., last 3 months'): Query Tickets.
- getOrgTickets(orgId, status=None, timePeriod='e.g., last month'): Query Tickets.
- getTicketDetails(ticketNumber): Query Tickets and related tables.
- searchTickets(query, orgId=None, userId=None, status=None): Query Tickets, TicketUpdates.
- searchSimilarTickets(query, orgId, userId=None, deviceId=None, timePeriod='e.g., last 6 months'): Query Tickets.
- getProblemRecords(query, orgId, status='Open'): Query Problems.
- getTicketsLinkedToProblem(problemId): Query ProblemIncidents.
- createTicket(userId, orgId, subject, description, ...): Insert Tickets.
- updateTicket(ticketNumber, notes=None, fields=None, internalNotes=None): Update Tickets / Insert TicketUpdates.
- Constraint: ALL text data written by the AI MUST be in English.
- getTicketTemplates(): Query TicketTemplates.
- getTemplateVariables(templateId): Query TicketTemplates.form_questions.
- applyTemplateToTicket(ticketNumber, templateId, variableAnswers): Update Tickets, Insert TicketTasks.
- addTicketTimeEntry(...): Insert TimeEntries.
- startTicketTimeEntry(...): Insert TimeEntries.
- endTicketTimeEntry(...): Update TimeEntries.
- User & Org Information (CommandIT Platform):
- getUserDetails(userId): Query Users, Contacts, Roles, Teams, Locations.
- getOrgDetails(orgId): Query Organizations.
- getServiceAgreementDetails(orgId): Query Agreements.
- getUserADStatus(userId): Query synced data in AdUsers.
- getUserM365Status(userId): Query synced data in AzureAdUsers.
- Device Information (CommandIT RMM/CMDB Module):
- getUserDevices(userId): Query Devices. Impl Detail: Trigger 'fast mode'.
- getDeviceDetails(deviceId): Query Devices, DeviceHardware, DeviceLogicalDisks. Impl Detail: Trigger 'fast mode'.
- getDeviceSoftware(deviceId): Query DeviceSoftware.
- getDevicePerformance(deviceId): Query central DB. May support direct agent query.
- getRecentDeviceAlerts(deviceId, timePeriod='1h'): Query Alerts. May query central DB or live agent.
- getDeviceNetworkShares(deviceId): Query NetworkShares.
- getDeviceServices(deviceId): Query DeviceServices status from central DB. May support direct agent query.
- Knowledge & Troubleshooting (CommandIT Documentation Module):
- searchKnowledgeBase(query): Query Documents / DocumentRevisions.
- suggestKBUpdate(articleId, suggestion): Insert DocumentReviewRequests.
- flagResolutionForKBReview(ticketId, resolutionSummary): Insert DocumentReviewRequests.
- Automated Actions (via CommandIT Agent & Platform - Requires Constraints 4.1):
- resetUserPassword(userId, method): Trigger CommandIT identity action.
- unlockUserAccount(userId): Trigger CommandIT identity action.
- addRemoveUserGroupMembership(...): Trigger CommandIT identity action.
- assignRemoveLicense(...): Trigger CommandIT SaaS management action.
- executeRMMScript(deviceId, scriptId, parameters, requiresConsent=True): Execute script via CommandIT Agent LocalSystem Service (queues via AgentCommandQueue).
- forceAgentSync(deviceId): Instructs the specified CommandIT Agent to perform an immediate delta synchronization. Returns status of request dispatch. Used post-action/pre-query for freshness.
- Scheduling & Handoff (CommandIT Platform):
- checkAgentAvailability(queue/skill): Query CommandIT user status.
- initiateLiveChatHandoff(...): Trigger CommandIT chat transfer.
- getTechnicianAvailability(...): Query TechnicianSchedules, UserAvailability.
- scheduleAppointment(...): Insert TechnicianSchedules.
- checkLocationEligibility(...): Query Locations, ServiceZones, Agreements.
- initiateFieldVisitRequest(...): Insert FieldServiceRequests.
- getScheduleEntryDetails(...): Query TechnicianSchedules.
- getUserCurrentStatus(userId): Query Users.current_status_enum.
- Policy & Compliance (CommandIT Platform):
- checkActionPolicyCompliance(actionDetails): Mandatory check querying CommandIT policy tables.
- getDeviceComplianceStatus(...): Query CommandIT compliance results (Requires results stored).
- getDevicePatchStatus(...): Query CommandIT patch status results (Requires results stored).
- getAppliedPolicyDetails(...): Evaluate CommandIT tag assignments and policy hierarchy.
- Account Management & User Security (CommandIT Platform):
- createAccountManagerRequest(...): Create CommandIT ticket for AM.
- getUserSecurityPosture(userId): Query CommandIT identity security tables (Flags only).
- CMDB & Relationships (CommandIT Platform):
- getRelatedCIs(...): Query ConfigurationItemRelationships.
- Inventory & Stock (CommandIT Platform):
- checkProductStockLevel(...): Query InventoryStockLevels.
- LMS (CommandIT Platform):
- getUserCourseStatus(...): Query CourseEnrollments, Courses.
- Service Management (CommandIT Platform):
- getServiceAssignment(...): Query DeviceServiceAssignments / UserServiceAssignments.
- Telecom & Network (CommandIT Platform):
- getPhoneNumberDetails(...): Query PhoneNumbers.
- getCircuitDetails(...): Query TelecomCircuits.
- Order Tracking (CommandIT Platform):
- getUserSalesOrders(...): Query SalesOrders.
- getSalesOrderDetails(...): Query SalesOrders, SalesOrderLineItems.
- getSalesOrderShipmentInfo(...): Query Shipments, ShipmentLineItems.
- getSalesOrderProcurementInfo(...): Query linked PurchaseOrders etc.
- Billing & Cost Tracking (CommandIT Platform):
- addExpenseEntry(userId, orgId, ticketId, expenseDate, expenseType, description, totalAmount, currencyCode, billingOption='DoNotBill'): Creates a non-billable expense entry against a ticket for internal cost tracking (e.g., AI usage). Inserts into ExpenseEntries.
4.1 Security Constraints for Actions & Scripting
- Mandatory Policy Check: checkActionPolicyCompliance MUST be called before sensitive actions.
- Explicit User Consent: Required before modifying state or running scripts via AI.
- Independent Input Validation: Action tools MUST validate inputs independently.
- Contextual Validation: Actions MUST target relevant entities.
- Risk Assessment Logic: Tools MUST check against known risks.
- User Privilege Check: Actions exceeding user's role require denial or approval.
- Pre-Approved Scripts Only: executeRMMScript prefers scriptIds.
- Least Privilege: AI service accounts use minimum necessary permissions.
- Auditing: All actions/checks logged (LangSmith, AuditLog).
- Fail-Safes: Timeouts/error handling required.
5. Intent Determination
- Primary Intents: Report New Issue, Manage Existing Ticket, Ask General Question, Request Specific Service, Request Human Agent, QueryOrderStatus.
- Mechanism: LLM classification.
- Implicit Triggers: AI identifies needs based on diagnostics.
6. User Experience & Communication
- Use Persona (6a) guidelines.
- Use clear, concise language. Adjust technical depth.
- Provide proactive updates. Confirm understanding. Explain 'why'. Manage expectations.
- Adhere to Policy Constraints (6b). Use conversational memory.
- Maintain Chat Language: Maintain user's supported language.
- (Potential WOW): Offer hyper-proactive assistance.
6a. AI Persona Definition
- Core Persona: "CommandIT Ultra-Competent Technical Assistant"
- Key Traits: Knowledgeable, Confident, Helpful, Efficient, Empathetic, Patient (High EQ), Professional, Calm, Accommodating, Neutral, Objective.
- Implementation: Guided by LLM system prompts.
6b. Communication Policy Constraints
- No Admission of Fault: Never admit fault.
- No Unauthorized Agreements: Do not agree to SLAs, credits, etc.
- Handling Fault/Agreement Queries: Use standard response -> createAccountManagerRequest.
- De-escalation Protocol: Acknowledge -> Maintain -> Refocus -> Offer Handoff -> Disengage if Abusive.
- Security & Privacy: Foundational. Adhere to policies/compliance.
7. Ticket Creation Process
- Language for Ticket Data: All AI-written ticket data MUST be in English.
- Pre-Creation Search: Search CommandIT data.
- Link/Update Option: Offer if relevant existing item found.
- Template Selection: Query TicketTemplates. Match issue.
- Variable Identification: Query TicketTemplates.form_questions.
- Variable Population: Fill from context/tools first. Prompt for mandatory missing info. Associate primary_device_id.
- Priority & Type Determination: Set in Tickets.
- AI Review Step: LLM self-critique.
- User Confirmation: Confirm summary accuracy.
- Ticket Creation/Application: Insert Tickets or applyTemplateToTicket.
- Time Entry: Call startTicketTimeEntry.
- Context Attachment: Add context to TicketUpdates (English, non-sensitive).
8. Determining Next Steps Post-Information Gathering/Ticket Creation
- Decision Logic: Based on issue, template (TicketTasks), tools, diagnostics, user preference, policy checks.
- Initial Troubleshooting Prompt: Ask user preference.
- Mandatory Policy Check Step: Call checkActionPolicyCompliance before sensitive actions.
- Possible Next Steps:
- A. Provide Instructions/KB Article (If User Agrees): Search Documents. Present steps. Test. -> Path 9.1 if resolved.
- B. Perform Automated Troubleshooting (If User Agrees): Execute tools (unlockUserAccount, executeRMMScript), policy check first. Get consent. Call forceAgentSync(deviceId) after action. Query central DB for result verification. Ask user to test. If resolved -> Path 9.1. If not -> Offer Path C/D.
- C. Escalate to Live Chat: Triggered by user/AI/policy. Check checkAgentAvailability. If available -> initiateLiveChatHandoff -> Path 9.2. Else -> Offer D.
- D. Schedule Remote Appointment: Triggered if user requests & chat unavailable, or next step. Use getTechnicianAvailability/scheduleAppointment -> Path 9.2.
- E. Summarize and Conclude (Async Handoff): If next step non-interactive. Provide summary -> Path 9.2.
- F. Initiate On-Site Field Visit: Triggered by request/diagnostics/template. Find template -> Process -> Policy Check -> Apply Template -> initiateFieldVisitRequest -> Inform User -> Path 9.2.
- G. Identify Capability Gap: If AI lacks tool/capability -> Log Wish (13.C) -> Escalate.
- H. Provide Order Status Update: Triggered by QueryOrderStatus. Identify Order -> Get Details -> Provide Summary & Shipment/Procurement info -> Path 9.
9. Chat Conclusion & Cleanup
- Constraint Reminder - English Ticket Updates: All AI-generated content added to the CommandIT ticket (notes, fields, time entries) MUST be in English.
- Final Step (Both Paths): Log AI Costs: Retrieve accumulated AI cost (tracked internally, e.g., via LangGraph state accumulated_llm_cost updated after LLM calls). If cost > 0 and ticket exists, call addExpenseEntry with userId=AI User, ticketId, expenseType='AI_Interaction_Cost', totalAmount=accumulated cost, billingOption='DoNotBill', and relevant currencyCode.
- Path 9.1: Issue Confirmed Resolved by User During Chat
- Update Ticket Status: Call updateTicket(..., fields={'status': 'Resolved', 'FCR_Flag': True}).
- Add Final Note (English).
- (Optional) Offer personalized tip.
- Inform User.
- Final Check.
- End Time Entry: endTicketTimeEntry (notes in English).
- Log AI Costs (See Final Step above).
- Conclude AI Interaction -> Transition to Section 10.
- Path 9.2: Issue Not Resolved / Escalated / Async Handoff
- Add Summary Note (Public, English) to TicketUpdates.
- Add Internal Note (English) to TicketUpdates.
- Attach Transcript (System function, original chat language).
- End Time Entry: endTicketTimeEntry (notes in English).
- Log AI Costs (See Final Step above).
- Conclude AI Interaction: State next step -> Transition to Section 10.
10. Post-Chat Actions (Triggered by CommandIT Platform)
- 1. Present Survey: Offer brief CSAT survey. Link results to InteractionID. Include optional "What couldn't AI do?".
- 2. Offer Human Follow-up (Optional): After survey. Check checkAgentAvailability. Offer live chat or scheduling.
11. Areas for Further Definition (Open Questions)
- Detailed logic for Support vs. Project determination.
- Specific integration details with Dispatcher System (if external).
- Structure/content details for Ticket Templates (SOPs, approvals).
- Handling of edge cases (tool errors, conflicting data).
- Specific KB structure/tagging needed.
- Survey content and integration.
- Trigger mechanism and UI/UX for Post-Chat Actions.
- Specific rules/logic engine for checkActionPolicyCompliance tool.
- Feasibility/design for cross-session memory/user profiles (V2).
- Detailed design of currentStateSummary synthesis logic.
- Definition of supported non-English languages and fluency requirements.
- Strategy for language detection and switching.
- LLM/translation services needed for mixed-language handling.
- Design of results tables for Compliance/Patch status queries.
- Optimal wait/retry logic after forceAgentSync.
- Method for reliably capturing LLM costs per interaction step (via LangSmith/LangChain callbacks) and handling potential non-LLM AI costs.
- Definition of specific expenseType values and currency/pricing source handling for cost tracking.
- Error handling strategy for cost logging.
12. Out of Scope (Initially)
- Direct modification of RMM agent configurations (beyond approved scripts).
- Complex project scoping or quoting.
- Direct billing or payment processing.
- Voice channel support.
- Non-English language support (beyond specified V1 languages).
13. Metrics and Data Capture Requirements
- Purpose: Ensure data capture for MSP KPIs (TruMethods, SLIQ) and AI improvement.
- A. Data Logged by AI System/Platform (e.g., LangSmith, CommandIT Logs): InteractionID, UserID, OrgID, Timestamps (Start, End, Escalation), InteractionDuration, TicketNumber(s), DeviceID(s), InteractionOutcome (Categorical), TemplateIDUsed, ToolsCalled log, LLM_Metrics (incl. cost), SentimentTrend, SurveyID.
- B. Data Written to CommandIT PSA Ticket by AI (All text in English): TicketSource="AI Chat Support", CreationTimestamp, UserID, DeviceID/CI, TicketType, TicketPriority, FCR_Flag(Boolean), ResolvedTimestamp(if AI FCR), AI_TimeEntryDuration, AI_TimeEntryNotes, DetailedNotes(Diagnostics, Actions, KBs), InternalNotes(Context, Escalation Reason, Patterns), RelatedItems, AccountManagerRequestFlag. (AI User record MUST have $0 cost rate in UserCostRates).
- C. Tool/Capability Gap Logging ("AI Wishlist"):
- Purpose: Identify needed capabilities for future development.
- Trigger: When AI cannot proceed due to lack of specific tool/data/action capability.
- Mechanism: Dedicated step (analyzeAndLogCapabilityGap) uses LLM to articulate the gap.
- Logged Data: TimestampUTC, InteractionID, TicketNumber, AIGoal, UserRequestSummary, GapDescription_Wish.
- Storage: Logged separately within CommandIT for analysis.
- D. AI Cost Tracking:
- Method: Actual AI operational costs (primarily LLM token costs captured via LangSmith/callbacks, potentially other tracked AI API costs) accumulated during the interaction are logged as a single non-billable expense entry against the associated ticket using the addExpenseEntry tool (targeting the ExpenseEntries table) with expenseType='AI_Interaction_Cost' (or similar) and billingOption='DoNotBill'.
- KPI Reporting: Downstream reporting systems calculating KPIs like Cost Per Ticket or Agreement Profitability MUST SUM human technician costs (from TimeEntries * UserCostRates) AND actual AI costs (from relevant ExpenseEntries) when determining total service delivery or "labor" costs. The AI's time from TimeEntries can be used for separate efficiency metrics.
Technical Implementation Details
1. Introduction
This document outlines the architectural approach for implementing the CommandIT Support AI agent using LangGraph and LangSmith. The functional requirements, detailed in the "CommandIT Support AI Functional Specification," describe a sophisticated, stateful agent requiring complex conversational logic, flexible control flow, extensive integration with CommandIT platform tools and data, robust security and policy enforcement, and comprehensive observability. LangGraph provides the framework for building the agent's stateful, potentially cyclic execution logic, while LangSmith delivers essential tools for developing, debugging, testing, monitoring, and evaluating this complex AI system.
2. Core Architecture: LangGraph
The AI's core logic will be built as a stateful, potentially cyclic graph using LangGraph, likely extending StatefulGraph. This allows for managing conversational context, handling complex branching logic, executing tools sequentially or conditionally, and enabling iterative processes.
3. State Definition (StatefulGraph State)
A well-defined state is crucial. We will use Python's TypedDict to define the graph's state structure, ensuring type safety and clarity. The state will be updated progressively as the graph executes.
Key State Fields (Illustrative TypedDict Structure):
from typing import TypedDict, List, Optional, Dict, Any, Tuple
from datetime import datetime
class CommandITAiState(TypedDict):
# === Initial Context (Provided by Platform) ===
user_id: str
org_id: str
chat_start_time_utc: datetime
# === Conversation Management ===
chat_history: List[Tuple[str, str]] # List of ("human", message) or ("ai", message) - REQUIRES SIZE MANAGEMENT
current_user_input: Optional[str] # The latest message from the user
detected_language: Optional[str] # Detected language of the chat (e.g., 'en', 'fr')
# === Gathered Context (From Tools) ===
user_details: Optional[Dict] # Result from getUserDetails tool
org_details: Optional[Dict] # Result from getOrgDetails tool
user_devices: Optional[List[Dict]] # Result from getUserDevices tool
focused_device_id: Optional[str] # Device currently being discussed/targeted
focused_device_details: Optional[Dict] # Result from getDeviceDetails tool
user_open_tickets: Optional[List[Dict]] # Result from getUserTickets(status='open')
# ... other context fields ...
synthesized_context_summary: Optional[str] # Optional LLM-generated summary
# === Intent & Workflow State ===
current_intent: Optional[str] # Classified user intent
active_ticket_number: Optional[str] # Ticket being created or discussed
active_ticket_details: Optional[Dict]# Result from getTicketDetails/createTicket
selected_template_id: Optional[str] # ID of chosen TicketTemplate
template_variables_required: Optional[Dict] # {var_name: description}
template_variables_gathered: Optional[Dict] # {var_name: value}
order_tracking_context: Optional[Dict] # State for order tracking flow
# === Action & Policy State ===
requires_user_consent_for: Optional[str] # Description of action needing consent
policy_check_result: Optional[Dict] # Output from checkActionPolicyCompliance
last_action_details: Optional[Dict] # Details of the last action attempted
# === Tool Execution State ===
last_tool_called: Optional[str]
last_tool_output: Optional[Any]
last_tool_status: Optional[str] # 'success', 'failure'
error_message: Optional[str] # Details if last_tool_status is 'failure'
# === Cost Tracking ===
accumulated_llm_cost: float # Running total of LLM costs for the session (e.g., in USD)
# === Output & Control Flow ===
ai_response: Optional[str] # The generated response message for the user (in detected_language)
internal_processing_log: List[str] # Append-only log for debugging
next_node_to_call: Optional[str] # Optional: Explicit routing control
ai_wishlist_entry: Optional[str] # Gap description if capability missing (for logging)
session_outcome: Optional[str] # Final outcome for logging/metrics- State Updates: Nodes return dictionaries with updates. LangGraph merges them.
- History Management: chat_history must be managed (summarized/truncated) before LLM calls.
- Cost Accumulation: accumulated_llm_cost is updated after each LLM call based on token usage/cost data (e.g., from LangSmith trace metadata or LangChain callbacks).
4. Node Implementation Details
Nodes represent units of work.
- LLM Nodes (e.g., determineIntent, generateResponse):
- Use ChatPromptTemplate. Inject persona, policy reminders, language, relevant state slices. Use appropriate model binding.
- Cost Capture: After the LLM call, retrieve cost information (from LangChain run info/LangSmith) and update state['accumulated_llm_cost'].
- Parse output, return state updates.
- Tool Nodes (e.g., WorkspaceUserDetails, executeSecureScript, checkPolicyCompliance, addExpenseEntry):
- Wrap CommandIT API calls using Tool or RunnableLambda.
- Perform input validation before sensitive API calls.
- Handle API errors gracefully, update last_tool_status / error_message.
- Initiate actions and handle results/waits. Use forceAgentSync judiciously.
- addExpenseEntry Node: Called near the end of the graph to log accumulated costs to the ExpenseEntries table.
- Function Nodes: Implement simple Python logic.
- Entry/Exit: Define START and END nodes.
5. Edge Implementation Details
Edges control flow based on state.
- Use conditional edges (Python functions inspecting state).
- Route based on current_intent, last_tool_status, policy_check_result, etc.
- Implement loops by routing back to earlier nodes.
- Route to error handling or analyzeAndLogCapabilityGap nodes as needed.
6. LangSmith Integration & Usage
LangSmith is essential for observability.
- Setup: Configure standard LangChain environment variables.
- Automatic Tracing: Captures graph execution, LLM calls (with token counts/costs), tool calls, state transitions, errors.
- Viewing Traces: Use LangSmith UI to visualize paths, inspect state, analyze prompts/responses, check tool performance/errors, view granular costs.
- Tagging/Metadata: Add userId, orgId, session_id, intent, outcome to traces.
- Datasets & Evaluation: Create datasets from traces for testing and evaluation.
- Monitoring: Set up LangSmith dashboards for latency, errors, tool usage, aggregated costs, feedback correlation.
- AI Wishlist Review: Supplement Section 13.C logging by reviewing traces.
- Performance Analysis: Identify bottlenecks.
7. Interaction with CommandIT Backend API & Frontend
- API Endpoint: Backend service (e.g., FastAPI) hosts LangGraph, exposes /chat/invoke, manages sessions.
- Streaming: Recommended: Use WebSockets/SSE/HTTP streaming from API endpoint to React/TS frontend (in Tauri/browser) for progressive ai_response delivery.
- Tool Calls: LangGraph Tool Nodes call internal CommandIT backend APIs (/api/internal/...) which handle database interaction (including ExpenseEntries) and agent service coordination.
8. Final Step: Logging Aggregated Costs
- Near the END of the graph execution (after other notes/time entries):
- A dedicated step/node retrieves the final state['accumulated_llm_cost'].
- If cost > 0 and a relevant state['active_ticket_number'] exists, this step calls the addExpenseEntry tool node to log the total calculated cost as a single, non-billable expense line against the ticket in the CommandIT ExpenseEntries table.
9. Conclusion
The combination of LangGraph's explicit state management, conditional branching, and cycle support provides a powerful structure for implementing the complex logic defined in the CommandIT Support AI functional specification. LangSmith delivers the essential observability needed to develop, debug, monitor, and iteratively improve the agent. This technical approach, integrating closely with CommandIT's backend APIs and data, enables the creation of a sophisticated, context-aware, and policy-driven support AI, while also providing mechanisms (like cost tracking via ExpenseEntries) to manage its operational aspects effectively.