Medium-term future of LLM Applications

1. Hospitals and support centers utilize triage trees to efficiently manage and prioritize cases.

2. Individuals interact with tools in unique ways, developing personal “flows” for their work processes.

3. Booking agents follow a structured sequence of decisions to achieve desired outcomes for clients.

4. These examples illustrate that people operate within distinct “state machines” when using tools to produce results.

5. The concept of AI “Agents” in 2023 aimed to generate these state machines dynamically based on user queries, using task masters and evaluators.

6. This approach has largely failed in production environments, with agents often trapped in unresolvable loops.

7. The issue persists regardless of AI intelligence, as successful outcomes depend more on defined processes than general intelligence. Even highly intelligent individuals may disagree on optimal methods for common tasks.

8. While future technology might allow for more adaptive workflows through screen recording and flexible deviation, current and near-future AI models (including hypothetical GPT-5 or GPT-6) likely lack the fundamental architecture to achieve this reliably.

9. A more feasible approach involves “generally flexible” state machines, combining elements of agents and traditional chatbots. These would be stateful, progressing through defined stages while allowing for flexible conversation within each stage.

10. This structured approach may seem restrictive, but it addresses the fundamental need for a minimum amount of information to complete tasks effectively. Even simple queries like “buy flight” require specific details to be actionable. Furthermore, effective task completion often requires grounding in real-world information. This could involve:

    • Real-time access to airline databases
    • Up-to-date information on travel regulations and visa requirements
    • Current exchange rates and pricing information
    • Seasonal variations in flight schedules and demand

11. The key advantage of this method is that creators can define clear stages, confirm and save user inputs across multiple interactions, and trigger meaningful actions once sufficient information is gathered. This contrasts with attempting to execute a complete workflow from a single user input, which is often impractical and unintuitive.

12. This approach acknowledges the limitations of AI in mind-reading or inferring unstated preferences, focusing instead on guiding users through necessary information-gathering steps in a flexible, conversational manner.

13. By breaking complex tasks into manageable, defined stages, this system can potentially overcome the limitations of current AI agents while still leveraging the strengths of language models in natural conversation and context understanding.

Products like https://www.gumloop.com/ are providing a horizontal framework for this, but they don’t quite have an easy way to define “stateful” workflows that is part of a whole triage tree and can have intermediate points at which you can take user input of arbitrary length.