AI Cost Tracking

1\ introduction 1 1 purpose this defines a standardized approach for tracking the operational costs associated with artificial intelligence (ai) features deployed across the commandit platform this includes the support ai chat agent, ai powered ticket processing, automation assistance, reporting insights, and future ai driven functionalities 1 2 goals the primary goals of this strategy are to accurately capture the actual operational costs incurred by ai service usage (primarily large language models (llms) and potentially other specialized ai apis) enable precise internal cost accounting and profitability analysis for ai features per client, agreement, or feature area, specifically associating costs with relevant commandit tickets where applicable provide reliable data for incorporating ai costs into relevant msp key performance indicators (kpis) , such as those used by frameworks like trumethods and service leadership (sliq) inform pricing and packaging decisions for commandit features that leverage ai identify high cost ai operations or features for potential optimization 1 3 scope this strategy applies to all current and future commandit features that utilize llms or other third party ai services with usage based costs it covers cost tracking associated with specific user interactions, automated processes, and system tasks, with a primary focus on attributing costs back to specific tickets when an interaction relates to one 2\ core concept & rationale track actual costs we will track the actual, variable costs incurred per ai operation (primarily llm token costs) we will avoid using arbitrary internal hourly rates or fixed costs for ai "labor " rationale actual cost tracking provides financial accuracy essential for understanding the true cost to serve, identifying the real economic impact of ai automation, calculating accurate profit margins (especially at the ticket and agreement level), and making data driven decisions unit of association costs should be associated primarily with the relevant commandit ticket ( tickets ticket id ) whenever an ai interaction or process relates directly to a ticket for ai actions not tied to a ticket, association might be linked to other relevant entities if necessary, but the focus is per ticket cost attribution internal operational expense these ai costs are treated as internal operational expenses they will be logged in a way that prevents them from appearing directly on client invoices by default 3\ data source & measurement primary source (llms) langsmith langsmith will be integrated with all llm based features it automatically tracks token usage by configuring model pricing (see section 4 4), we can derive estimated costs per llm call or interaction other ai services costs from non llm ai services (e g , transcription apis) must be captured by the interacting commandit backend module and logged using the same mechanism, potentially with a distinct expense type cost calculation & currency costs are calculated based on provider pricing the standard currency for cost logging should be defined (recommend usd ) currency conversion for financial reporting in other currencies (e g , cad) should be handled by the reporting/accounting system 4\ technical implementation strategy centralized storage expenseentries table we will utilize the existing expenseentries table within the commandit database schema to store the aggregated ai operational costs associated with specific tickets data recorded per ai interaction/task user id the commandit users user id representing the ai agent org id the client organizations org id related to the ticket ticket id the relevant tickets ticket id expense date date the cost was incurred/logged expense type a specific identifier, e g , 'ai interaction cost' , 'ai summarization cost' , 'ai transcription cost' description standardized text, e g , "ai operational cost for chat interaction \[interactionid]" or "ai cost for ticket summarization" quantity 1 unit amount null total amount the calculated actual cost (e g , accumulated llm cost ) for the interaction/task currency code the currency of total amount (e g , 'usd') billing option 'donotbill' (crucial ensures it's treated as an internal cost, not billed to the client via standard invoicing logic) payment method 'systeminternal' or similar indicator requires reimbursement false cost accumulation stateful interactions (e g , support chat ai) maintain a running total (e g , accumulated llm cost ) within the session's state (e g , langgraph state) at the end of the interaction (section 9 of functional spec), log a single expenseentries record for the total accumulated cost, linked to the ticket id discrete ai tasks (e g , ticket summarization) after the task completes, capture the cost and log a single expenseentries record immediately, linked to the relevant ticket id capturing cost data backend services implementing ai features must integrate with langsmith (for llms) or other mechanisms to capture usage/cost data per operation use langchain callbacks or inspect run metadata to get token counts/costs call an internal commandit api endpoint (wrapping the addexpenseentry tool logic) to write the finalized cost record to expenseentries ai user record configuration the "ai user" record ( users where is ai=true ) must have its usercostrates hourly cost rate set to 0 00 model pricing configuration clarification needed the strategy for maintaining the llm pricing data (cost per token for different models) needs definition will this be configured within commandit settings, pulled dynamically from langsmith, or hardcoded (less flexible)? this needs to be defined and kept up to date error handling clarification needed define the system behavior if logging to expenseentries fails should the primary ai operation still be considered successful? should errors be logged separately for reconciliation? recommend logging the error and proceeding, with a mechanism for later review/retry of cost logging 5\ api impact internal logging api the internal api endpoint used by the addexpenseentry tool handles writing cost data to the expenseentries table backend ai services call this endpoint reporting apis internal commandit apis for reporting must query expenseentries (filtering by relevant expense type values) to fetch ai costs associated with tickets, clients, or agreements external apis no direct impact on client facing apis is expected 6\ kpi integration (trumethods/sliq etc ) data source ai operational costs associated with tickets are available in the expenseentries table (e g , where expense type = 'ai interaction cost' ) human labor costs are derived from timeentries and usercostrates calculation logic reporting systems calculating kpis like cost per ticket , agreement profitability , gross margin , etc , where labor cost is a factor, must calculate the total service delivery cost as total cost = sum(human costs) + sum(ai costs) + sum(other direct costs) human costs = sum of ( timeentries duration seconds / 3600 usercostrates hourly cost rate ) for human users ai costs = sum of ( expenseentries total amount ) where expense type indicates an ai cost related to the scope (ticket, agreement, period) rationale this incorporates the actual variable cost of ai automation into profitability calculations accurately 7\ granularity perspective while this approach stores the aggregated cost per interaction/task in expenseentries for accounting/kpi purposes, the underlying granular data (cost per specific llm call, token counts, model used, latency) remains available within langsmith traces langsmith should be used for detailed ai performance analysis, cost optimization efforts, and debugging specific high cost interactions 8\ conclusion using the expenseentries table to log aggregated, non billable ai operational costs per ticket provides a pragmatic and effective strategy for commandit it enables accurate internal cost accounting and integrates ai costs correctly into established msp kpi frameworks like trumethods and sliq, using actual incurred costs rather than arbitrary rates consistent implementation across all ai features, clear definition of pricing sources, and robust error handling for logging are key to the success of this strategy