Imagine a power management firm that runs entirely on artificial intelligence—no human employees, yet it’s responsible for real money and real crises. Now, watch it face its toughest week in a live experiment that reveals not just what AI can do, but what it cannot.
The Live Experiment: AI as a Company
At Firmulate, a groundbreaking experiment is unfolding in real time. They’ve created a virtual company staffed by 13 synthetic employees—AI models trained to handle every aspect of a small software business. Every decision, every crisis, every customer interaction is simulated, and every move is publicly recorded and versioned.
How Does It Work?
Each AI model is subjected to the same challenging week: same customers, same crises, and the same temptations to cheat or cut corners. These models run through a carefully curated set of scenarios, and their decisions are evaluated against real-world metrics like cash burn and deal closure.
The Results Are Eye-Opening
- All four AI models identified every crisis and refused every manipulation attempt. This demonstrates a high level of integrity and crisis recognition.
- Only two models successfully signed a €55,000 deal that their own analysis had earned—meaning they followed through and completed their tasks.
- Interestingly, the decisive factor in securing the deal was hidden in a document buried deep within the company’s records. The models that read and understood this file won the full-price deal, adding €4,583 MRR to their performance.
Real Money and Real Consequences
The experiment isn’t just a game; it involves real finances. The virtual company burns €105,000 per month against a modest €2,300 monthly recurring revenue (MRR). Its public cash countdown adds urgency, making every decision critical. You can watch this ongoing drama unfold at firmulate.com/live.
Battery Energy Storage System Engineering: Design, Integration, and AI-Optimized Operation of Grid-Scale BESS from Cell Chemistry to Power Markets (Energy Systems Engineering Series Close)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What Does This Mean for Power Management?
For those in the energy space, the lessons are clear. AI can recognize crises and resist manipulation, but it may falter in executing long-term commitments or reading critical, buried information. For example, in the experiment, the most disciplined model—Kimi K3—ran without an effort parameter and was the most honest, yet still lost opportunities due to discipline lapses.
Beyond Chat: Evaluating AI for Critical Tasks
Many AI demos focus on how well they chat or generate content. But in high-stakes environments—like managing your power systems or integrating solar assets—the real questions are: does the AI finish what it starts? Does it read your critical documents? Will it stay honest under pressure? The firmulate experiment highlights that the ability to complete tasks and read deeply buried information is vital, not just conversational skill.
The Cognitive Grid: Artificial Intelligence and the Governance of Delegated Power in Critical Infrastructure
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Why Should You Care?
As the energy industry increasingly adopts AI for automation, decision-making, and customer engagement, understanding these capabilities becomes crucial. Will your AI be able to handle crises? Will it stay truthful and follow through? Or will it leave opportunities on the table due to discipline lapses? This live experiment offers a real-world glimpse into what’s possible—and what still needs work.
The Future of AI in Power Management
Firmulate’s ongoing test shows that AI models can spot crises and refuse manipulation, but closing deals or executing complex tasks remains a challenge. For energy operators, this underscores the importance of testing AI systems thoroughly—before deploying them in critical infrastructure or customer-facing roles.
Decision Making Using AI in Energy and Sustainability: Methods and Models for Policy and Practice (Applied Innovation and Technology Management)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Final Takeaway
This extreme build-in-public experiment proves that AI can recognize and resist manipulation in real time, but it still struggles with execution and deep understanding. Watching this virtual company fight for survival teaches us that AI’s potential isn’t just in chatbots—it’s in its ability to manage complex, high-stakes environments if we understand its limits.
The Firmulate live experiment reveals that while AI can identify crises and refuse manipulations, it still struggles with execution and understanding buried information—crucial traits for managing energy systems and critical infrastructure.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Document Intelligence Made Easy: A Beginner’s Guide to Humata AI
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
