Token Consumption & Response Time for LLMs
About this model
The diagram models a complex simulation of a natural language processing (NLP) request-response cycle, representing an AI chatbot's operational logic. The system is split into two primary functional sub-diagrams: a prompt processing circuit and a response generation circuit. This flow is synchronized with a `Request Duration` Delay node, simulating the temporal overhead of processing a user's question, which eventually populates the `Cumulative question tokens consumed per session` Pool.
The second phase represents the system's output, triggered by the `Number of Processed requests` Pool. This activates the `Generate response` Source, which similarly uses a Converter to calculate token consumption based on the significantly larger `answer_token_count` ratio. Detailed monitoring is handled by a network of `RegisterNodes` that calculate real-time ratios between words and tokens for both prompts and responses. All consumed resources are aggregated into the `Total number of tokens consumed per session` Pool, providing a comprehensive data set for analyzing the operational cost and efficiency of the simulated AI interaction model.
"Token Consumption & Response Time for LLMs" is a public Machinations diagram shared in the Community by Catalin Ichim, with 43 components.
It was created in March 2026 and last edited in March 2026.
Open the diagram in the Machinations app and press Play to watch the resources move, adjust its values to test how the system responds, or remix it to make an editable copy in My diagrams.
