AI Support Chat

functional specification 1\ purpose & overview this outlines the functional requirements for the ai powered chat support feature integrated within the commandit platform this ai agent leverages data directly from commandit's internal psa, rmm, documentation, identity synchronization (ad/m365), and other modules to provide context aware, first level it support to end users accessing the platform the agent aims to resolve issues automatically where possible, ensure efficient escalation when necessary, improve user experience, increase support efficiency, ensure consistency, and capture data for continuous improvement of the commandit platform and support services 1a core principles & goals this document specifies the commandit support ai agent designed to provide a helpful, empathetic, confident, and human like conversational experience with high emotional intelligence (eq) leverage all accessible data within the commandit platform to avoid asking users for information already known possess superior context awareness compared to human agents by synthesizing commandit data from multiple modules proactively resolve common issues directly and automatically, including potential script execution via the commandit agent where safe and appropriate minimize the creation of duplicate tickets within the commandit psa module by identifying existing relevant tickets, problems, or incidents identify recurring or related issues across users and devices using commandit data to aid troubleshooting and potentially trigger problem management workflows within commandit ensure tickets are created accurately, completely, and efficiently within commandit using templates and contextual data maintain a user focused approach, respecting requests for human assistance promptly operate within defined communication policies, avoiding admission of fault or making unauthorized commitments attempt to de escalate frustrated or hostile users using defined protocols operate securely, respecting user privacy and permissions, especially when performing actions or accessing data within the commandit platform accurately track its operational costs (e g , llm usage) and associate them with interactions/tickets as non billable expenses for internal accounting and kpi analysis continuously improve through feedback loops and analysis of operational data captured by commandit and langsmith , including identifying its own capability gaps 2\ core chat flow user access user initiates an ai chat session via the commandit agent (systray application on windows, menubar application on macos) or the commandit web portal authentication & context user is assumed authenticated within commandit the platform provides initial context ( userid , orgid , chatstarttimeutc ) ai gathers extended initial context using commandit tools ai greets user and determines intent ai gathers necessary information (checking tools first, then asking user if needed) ai attempts resolution (kb lookup, automated actions, scripting) or prepares for escalation/ticketing ai creates/updates psa ticket accurately within the commandit psa module (following language rules) ai determines next steps (resolution confirmation, escalation, scheduling, field visit request, order status query) ai executes next steps (provides info, initiates handoff, confirms schedule, may force agent sync) ai concludes interaction and performs cleanup (logging time, notes in english, logging ai cost as expense) commandit platform potentially triggers post chat actions (survey, human follow up offer) 3\ initial context gathering goal provide the ai with immediate context about the user and their environment within commandit mechanism core context (commandit platform provided) upon chat initiation via agent or portal, the commandit platform must inject the following based on the user's authenticated session userid identifier for the logged in commandit user ( users user id ) orgid identifier for the user's organization ( users org id , organizations org id ) chatstarttimeutc timestamp when the chat session was initiated extended context (ai gathered via tools) using the provided userid and orgid , the ai will perform initial tool calls against the commandit database/api at the start of the conversation to retrieve user details from contacts & users tables name, title, email, phone(s), role/permissions (via userroleassignments , roles ), location ( locations ), vip status ( is vip ), preferred language ( preferred language ), timezone user group membership via userteamassignments ( teams ) organization details from organizations table name, account manager, primary tech, default settings user's assigned devices from devices table name, type, status, os, rmm agent status ( agent version , last agent checkin , agent health status ) implementation detail focusing on a device should trigger the agent's 'fast mode' (approx 30s sync) user's open tickets from tickets table (filtered by requester user id , status not closed/resolved) relevant org wide status from tickets ( is major incident = true ), possibly alerts 4\ commandit platform & integration tools tool integration framework assumed framework like langgraph integrated within commandit, managed via langsmith guiding principle tool first approach before prompting the user, the ai must first attempt to retrieve information via available commandit tools mixed language data handling the ai must handle english data from commandit tools/database fields while formulating user facing responses in the detected/supported chat language available tools (conceptual list, accessing commandit data) ticket management (commandit psa module) getusertickets(userid, status=none, timeperiod='e g , last 3 months') query tickets getorgtickets(orgid, status=none, timeperiod='e g , last month') query tickets getticketdetails(ticketnumber) query tickets and related tables searchtickets(query, orgid=none, userid=none, status=none) query tickets , ticketupdates searchsimilartickets(query, orgid, userid=none, deviceid=none, timeperiod='e g , last 6 months') query tickets getproblemrecords(query, orgid, status='open') query problems getticketslinkedtoproblem(problemid) query problemincidents createticket(userid, orgid, subject, description, ) insert tickets updateticket(ticketnumber, notes=none, fields=none, internalnotes=none) update tickets / insert ticketupdates constraint all text data written by the ai must be in english gettickettemplates() query tickettemplates gettemplatevariables(templateid) query tickettemplates form questions applytemplatetoticket(ticketnumber, templateid, variableanswers) update tickets , insert tickettasks addtickettimeentry( ) insert timeentries starttickettimeentry( ) insert timeentries endtickettimeentry( ) update timeentries user & org information (commandit platform) getuserdetails(userid) query users , contacts , roles , teams , locations getorgdetails(orgid) query organizations getserviceagreementdetails(orgid) query agreements getuseradstatus(userid) query synced data in adusers getuserm365status(userid) query synced data in azureadusers device information (commandit rmm/cmdb module) getuserdevices(userid) query devices impl detail trigger 'fast mode' getdevicedetails(deviceid) query devices , devicehardware , devicelogicaldisks impl detail trigger 'fast mode' getdevicesoftware(deviceid) query devicesoftware getdeviceperformance(deviceid) query central db may support direct agent query getrecentdevicealerts(deviceid, timeperiod='1h') query alerts may query central db or live agent getdevicenetworkshares(deviceid) query networkshares getdeviceservices(deviceid) query deviceservices status from central db may support direct agent query knowledge & troubleshooting (commandit documentation module) searchknowledgebase(query) query documents / documentrevisions suggestkbupdate(articleid, suggestion) insert documentreviewrequests flagresolutionforkbreview(ticketid, resolutionsummary) insert documentreviewrequests automated actions (via commandit agent & platform requires constraints 4 1) resetuserpassword(userid, method) trigger commandit identity action unlockuseraccount(userid) trigger commandit identity action addremoveusergroupmembership( ) trigger commandit identity action assignremovelicense( ) trigger commandit saas management action executermmscript(deviceid, scriptid, parameters, requiresconsent=true) execute script via commandit agent localsystem service (queues via agentcommandqueue ) forceagentsync(deviceid) instructs the specified commandit agent to perform an immediate delta synchronization returns status of request dispatch used post action/pre query for freshness scheduling & handoff (commandit platform) checkagentavailability(queue/skill) query commandit user status initiatelivechathandoff( ) trigger commandit chat transfer gettechnicianavailability( ) query technicianschedules , useravailability scheduleappointment( ) insert technicianschedules checklocationeligibility( ) query locations , servicezones , agreements initiatefieldvisitrequest( ) insert fieldservicerequests getscheduleentrydetails( ) query technicianschedules getusercurrentstatus(userid) query users current status enum policy & compliance (commandit platform) checkactionpolicycompliance(actiondetails) mandatory check querying commandit policy tables getdevicecompliancestatus( ) query commandit compliance results ( requires results stored ) getdevicepatchstatus( ) query commandit patch status results ( requires results stored ) getappliedpolicydetails( ) evaluate commandit tag assignments and policy hierarchy account management & user security (commandit platform) createaccountmanagerrequest( ) create commandit ticket for am getusersecurityposture(userid) query commandit identity security tables ( flags only ) cmdb & relationships (commandit platform) getrelatedcis( ) query configurationitemrelationships inventory & stock (commandit platform) checkproductstocklevel( ) query inventorystocklevels lms (commandit platform) getusercoursestatus( ) query courseenrollments , courses service management (commandit platform) getserviceassignment( ) query deviceserviceassignments / userserviceassignments telecom & network (commandit platform) getphonenumberdetails( ) query phonenumbers getcircuitdetails( ) query telecomcircuits order tracking (commandit platform) getusersalesorders( ) query salesorders getsalesorderdetails( ) query salesorders , salesorderlineitems getsalesordershipmentinfo( ) query shipments , shipmentlineitems getsalesorderprocurementinfo( ) query linked purchaseorders etc billing & cost tracking (commandit platform) addexpenseentry(userid, orgid, ticketid, expensedate, expensetype, description, totalamount, currencycode, billingoption='donotbill') creates a non billable expense entry against a ticket for internal cost tracking (e g , ai usage) inserts into expenseentries 4 1 security constraints for actions & scripting mandatory policy check checkactionpolicycompliance must be called before sensitive actions explicit user consent required before modifying state or running scripts via ai independent input validation action tools must validate inputs independently contextual validation actions must target relevant entities risk assessment logic tools must check against known risks user privilege check actions exceeding user's role require denial or approval pre approved scripts only executermmscript prefers scriptid s least privilege ai service accounts use minimum necessary permissions auditing all actions/checks logged (langsmith, auditlog ) fail safes timeouts/error handling required 5\ intent determination primary intents report new issue, manage existing ticket, ask general question, request specific service, request human agent, queryorderstatus mechanism llm classification implicit triggers ai identifies needs based on diagnostics 6\ user experience & communication use persona (6a) guidelines use clear, concise language adjust technical depth provide proactive updates confirm understanding explain 'why' manage expectations adhere to policy constraints (6b) use conversational memory maintain chat language maintain user's supported language (potential wow) offer hyper proactive assistance 6a ai persona definition core persona "commandit ultra competent technical assistant" key traits knowledgeable, confident, helpful, efficient, empathetic, patient (high eq), professional, calm, accommodating, neutral, objective implementation guided by llm system prompts 6b communication policy constraints no admission of fault never admit fault no unauthorized agreements do not agree to slas, credits, etc handling fault/agreement queries use standard response > createaccountmanagerrequest de escalation protocol acknowledge > maintain > refocus > offer handoff > disengage if abusive security & privacy foundational adhere to policies/compliance 7\ ticket creation process language for ticket data all ai written ticket data must be in english pre creation search search commandit data link/update option offer if relevant existing item found template selection query tickettemplates match issue variable identification query tickettemplates form questions variable population fill from context/tools first prompt for mandatory missing info associate primary device id priority & type determination set in tickets ai review step llm self critique user confirmation confirm summary accuracy ticket creation/application insert tickets or applytemplatetoticket time entry call starttickettimeentry context attachment add context to ticketupdates (english, non sensitive) 8\ determining next steps post information gathering/ticket creation decision logic based on issue, template ( tickettasks ), tools, diagnostics, user preference, policy checks initial troubleshooting prompt ask user preference mandatory policy check step call checkactionpolicycompliance before sensitive actions possible next steps a provide instructions/kb article (if user agrees) search documents present steps test > path 9 1 if resolved b perform automated troubleshooting (if user agrees) execute tools ( unlockuseraccount , executermmscript ), policy check first get consent call forceagentsync(deviceid) after action query central db for result verification ask user to test if resolved > path 9 1 if not > offer path c/d c escalate to live chat triggered by user/ai/policy check checkagentavailability if available > initiatelivechathandoff > path 9 2 else > offer d d schedule remote appointment triggered if user requests & chat unavailable, or next step use gettechnicianavailability / scheduleappointment > path 9 2 e summarize and conclude (async handoff) if next step non interactive provide summary > path 9 2 f initiate on site field visit triggered by request/diagnostics/template find template > process > policy check > apply template > initiatefieldvisitrequest > inform user > path 9 2 g identify capability gap if ai lacks tool/capability > log wish (13 c) > escalate h provide order status update triggered by queryorderstatus identify order > get details > provide summary & shipment/procurement info > path 9 9\ chat conclusion & cleanup constraint reminder english ticket updates all ai generated content added to the commandit ticket (notes, fields, time entries) must be in english final step (both paths) log ai costs retrieve accumulated ai cost (tracked internally, e g , via langgraph state accumulated llm cost updated after llm calls) if cost > 0 and ticket exists, call addexpenseentry with userid =ai user, ticketid , expensetype ='ai interaction cost', totalamount =accumulated cost, billingoption ='donotbill', and relevant currencycode path 9 1 issue confirmed resolved by user during chat update ticket status call updateticket( , fields={'status' 'resolved', 'fcr flag' true}) add final note (english) (optional) offer personalized tip inform user final check end time entry endtickettimeentry (notes in english) log ai costs (see final step above) conclude ai interaction > transition to section 10 path 9 2 issue not resolved / escalated / async handoff add summary note (public, english) to ticketupdates add internal note (english) to ticketupdates attach transcript (system function, original chat language) end time entry endtickettimeentry (notes in english) log ai costs (see final step above) conclude ai interaction state next step > transition to section 10 10\ post chat actions (triggered by commandit platform) 1\ present survey offer brief csat survey link results to interactionid include optional "what couldn't ai do?" 2 offer human follow up (optional) after survey check checkagentavailability offer live chat or scheduling 11\ areas for further definition (open questions) detailed logic for support vs project determination specific integration details with dispatcher system (if external) structure/content details for ticket templates (sops, approvals) handling of edge cases (tool errors, conflicting data) specific kb structure/tagging needed survey content and integration trigger mechanism and ui/ux for post chat actions specific rules/logic engine for checkactionpolicycompliance tool feasibility/design for cross session memory/user profiles (v2) detailed design of currentstatesummary synthesis logic definition of supported non english languages and fluency requirements strategy for language detection and switching llm/translation services needed for mixed language handling design of results tables for compliance/patch status queries optimal wait/retry logic after forceagentsync method for reliably capturing llm costs per interaction step (via langsmith/langchain callbacks) and handling potential non llm ai costs definition of specific expensetype values and currency/pricing source handling for cost tracking error handling strategy for cost logging 12\ out of scope (initially) direct modification of rmm agent configurations (beyond approved scripts) complex project scoping or quoting direct billing or payment processing voice channel support non english language support (beyond specified v1 languages) 13\ metrics and data capture requirements purpose ensure data capture for msp kpis (trumethods, sliq) and ai improvement a data logged by ai system/platform (e g , langsmith, commandit logs) interactionid , userid , orgid , timestamps (start, end, escalation), interactionduration , ticketnumber(s) , deviceid(s) , interactionoutcome (categorical), templateidused , toolscalled log, llm metrics (incl cost), sentimenttrend , surveyid b data written to commandit psa ticket by ai (all text in english) ticketsource ="ai chat support", creationtimestamp , userid , deviceid /ci, tickettype , ticketpriority , fcr flag (boolean), resolvedtimestamp (if ai fcr), ai timeentryduration , ai timeentrynotes , detailednotes (diagnostics, actions, kbs), internalnotes (context, escalation reason, patterns), relateditems , accountmanagerrequestflag (ai user record must have $0 cost rate in usercostrates ) c tool/capability gap logging ("ai wishlist") purpose identify needed capabilities for future development trigger when ai cannot proceed due to lack of specific tool/data/action capability mechanism dedicated step ( analyzeandlogcapabilitygap ) uses llm to articulate the gap logged data timestamputc , interactionid , ticketnumber , aigoal , userrequestsummary , gapdescription wish storage logged separately within commandit for analysis d ai cost tracking method actual ai operational costs (primarily llm token costs captured via langsmith/callbacks, potentially other tracked ai api costs) accumulated during the interaction are logged as a single non billable expense entry against the associated ticket using the addexpenseentry tool (targeting the expenseentries table) with expensetype='ai interaction cost' (or similar) and billingoption='donotbill' kpi reporting downstream reporting systems calculating kpis like cost per ticket or agreement profitability must sum human technician costs (from timeentries usercostrates ) and actual ai costs (from relevant expenseentries ) when determining total service delivery or "labor" costs the ai's time from timeentries can be used for separate efficiency metrics technical implementation details 1\ introduction this document outlines the architectural approach for implementing the commandit support ai agent using langgraph and langsmith the functional requirements, detailed in the "commandit support ai functional specification," describe a sophisticated, stateful agent requiring complex conversational logic, flexible control flow, extensive integration with commandit platform tools and data, robust security and policy enforcement, and comprehensive observability langgraph provides the framework for building the agent's stateful, potentially cyclic execution logic, while langsmith delivers essential tools for developing, debugging, testing, monitoring, and evaluating this complex ai system 2\ core architecture langgraph the ai's core logic will be built as a stateful, potentially cyclic graph using langgraph, likely extending statefulgraph this allows for managing conversational context, handling complex branching logic, executing tools sequentially or conditionally, and enabling iterative processes 3\ state definition ( statefulgraph state) a well defined state is crucial we will use python's typeddict to define the graph's state structure, ensuring type safety and clarity the state will be updated progressively as the graph executes key state fields (illustrative typeddict structure) from typing import typeddict, list, optional, dict, any, tuple from datetime import datetime class commanditaistate(typeddict) \# === initial context (provided by platform) === user id str org id str chat start time utc datetime \# === conversation management === chat history list\[tuple\[str, str]] # list of ("human", message) or ("ai", message) requires size management current user input optional\[str] # the latest message from the user detected language optional\[str] # detected language of the chat (e g , 'en', 'fr') \# === gathered context (from tools) === user details optional\[dict] # result from getuserdetails tool org details optional\[dict] # result from getorgdetails tool user devices optional\[list\[dict]] # result from getuserdevices tool focused device id optional\[str] # device currently being discussed/targeted focused device details optional\[dict] # result from getdevicedetails tool user open tickets optional\[list\[dict]] # result from getusertickets(status='open') \# other context fields synthesized context summary optional\[str] # optional llm generated summary \# === intent & workflow state === current intent optional\[str] # classified user intent active ticket number optional\[str] # ticket being created or discussed active ticket details optional\[dict]# result from getticketdetails/createticket selected template id optional\[str] # id of chosen tickettemplate template variables required optional\[dict] # {var name description} template variables gathered optional\[dict] # {var name value} order tracking context optional\[dict] # state for order tracking flow \# === action & policy state === requires user consent for optional\[str] # description of action needing consent policy check result optional\[dict] # output from checkactionpolicycompliance last action details optional\[dict] # details of the last action attempted \# === tool execution state === last tool called optional\[str] last tool output optional\[any] last tool status optional\[str] # 'success', 'failure' error message optional\[str] # details if last tool status is 'failure' \# === cost tracking === accumulated llm cost float # running total of llm costs for the session (e g , in usd) \# === output & control flow === ai response optional\[str] # the generated response message for the user (in detected language) internal processing log list\[str] # append only log for debugging next node to call optional\[str] # optional explicit routing control ai wishlist entry optional\[str] # gap description if capability missing (for logging) session outcome optional\[str] # final outcome for logging/metrics state updates nodes return dictionaries with updates langgraph merges them history management chat history must be managed (summarized/truncated) before llm calls cost accumulation accumulated llm cost is updated after each llm call based on token usage/cost data (e g , from langsmith trace metadata or langchain callbacks) 4\ node implementation details nodes represent units of work llm nodes (e g , determineintent, generateresponse) use chatprompttemplate inject persona, policy reminders, language, relevant state slices use appropriate model binding cost capture after the llm call, retrieve cost information (from langchain run info/langsmith) and update state\['accumulated llm cost'] parse output, return state updates tool nodes (e g , workspaceuserdetails, executesecurescript, checkpolicycompliance, addexpenseentry) wrap commandit api calls using tool or runnablelambda perform input validation before sensitive api calls handle api errors gracefully, update last tool status / error message initiate actions and handle results/waits use forceagentsync judiciously addexpenseentry node called near the end of the graph to log accumulated costs to the expenseentries table function nodes implement simple python logic entry/exit define start and end nodes 5\ edge implementation details edges control flow based on state use conditional edges (python functions inspecting state) route based on current intent, last tool status, policy check result, etc implement loops by routing back to earlier nodes route to error handling or analyzeandlogcapabilitygap nodes as needed 6\ langsmith integration & usage langsmith is essential for observability setup configure standard langchain environment variables automatic tracing captures graph execution, llm calls (with token counts/costs), tool calls, state transitions, errors viewing traces use langsmith ui to visualize paths, inspect state, analyze prompts/responses, check tool performance/errors, view granular costs tagging/metadata add userid, orgid, session id, intent, outcome to traces datasets & evaluation create datasets from traces for testing and evaluation monitoring set up langsmith dashboards for latency, errors, tool usage, aggregated costs, feedback correlation ai wishlist review supplement section 13 c logging by reviewing traces performance analysis identify bottlenecks 7\ interaction with commandit backend api & frontend api endpoint backend service (e g , fastapi) hosts langgraph, exposes /chat/invoke, manages sessions streaming recommended use websockets/sse/http streaming from api endpoint to react/ts frontend (in tauri/browser) for progressive ai response delivery tool calls langgraph tool nodes call internal commandit backend apis (/api/internal/ ) which handle database interaction (including expenseentries) and agent service coordination 8\ final step logging aggregated costs near the end of the graph execution (after other notes/time entries) a dedicated step/node retrieves the final state\['accumulated llm cost'] if cost > 0 and a relevant state\['active ticket number'] exists, this step calls the addexpenseentry tool node to log the total calculated cost as a single, non billable expense line against the ticket in the commandit expenseentries table 9\ conclusion the combination of langgraph's explicit state management, conditional branching, and cycle support provides a powerful structure for implementing the complex logic defined in the commandit support ai functional specification langsmith delivers the essential observability needed to develop, debug, monitor, and iteratively improve the agent this technical approach, integrating closely with commandit's backend apis and data, enables the creation of a sophisticated, context aware, and policy driven support ai, while also providing mechanisms (like cost tracking via expenseentries) to manage its operational aspects effectively