Over the last few months, I’ve been having versions of the same conversation with a bunch of seed and early-stage investors. The specifics differ, but the shape is always similar: the world we underwrite against has changed, and the founders who are building in it look different from the ones we backed in the past.

For most of the history of software, we built deterministic systems. You wrote the code, you tested it, you shipped it, and within well-understood bounds you knew what it did. Given the same input, you got the same output. And because that was the nature of the systems, it was also, whether we said it out loud or not, the nature of the founders we backed to build them. Structured. Methodological. Thoughtful. The kind of operator you could hand a plan to and trust would execute it quarter after quarter. We pattern-matched on rigor, on clarity, on the ability to write a roadmap you could live inside for two years. Backing founders at the seed stage, we evaluated “clarity of vision”.

The systems we’re all building now are not deterministic anymore. And the founders who are winning are changing too.

The era of probabilistic engineering

Tim Davis, the co-founder of Modular, published an essay called Probabilistic Engineering and the 24-7 Employee that, in my view, is a clear articulation of this shift. His central claim is that software is becoming a probabilistic system. Inside the most AI-native teams, large portions of the codebase are generated by stochastic models, reviewed under time pressure against contexts too large to fully hold, and integrated into wholes no single human ever designed end-to-end. The code still runs. It still ships. But the confidence interval around “this works as intended” has widened. Generation has become cheap. Validation has not. The codebase stops being a thing you know works and becomes a thing you believe works, with a probability you can no longer precisely state.

Senior engineers have always lived with some version of this, every large production system is, to some degree, a thing you believe works. But the degree has changed enough to matter. When most of the code you’re shipping wasn’t written by the humans reviewing it, belief-based correctness stops being an edge condition and starts being the default.

If that’s what the systems are becoming, it stands to reason the people building them would start to look the same way. And in some of the most important categories forming right now, the deterministic archetype we used to underwrite against, the operator with the two-year roadmap and the crisp milestone plan, is no longer sufficient. In a few of them, it’s actively a mismatch.

The probabilistic founder

The founders building the best AI-native companies are a lot like the systems they’re building.

They are experimental by default. They are willing to abandon their priors. They’d rather run ten cheap experiments than commit to one expensive plan. Their iteration cycles are measured in days, not quarters. And (this is the part that would have been a red flag two years ago and is now a tell that they’ve read the environment correctly), their roadmaps are deliberately, almost defiantly, light.

More than one founder has told me some version of the same line: “Here’s our long-term vision of the platform, but honestly, this could all change in two or three months.” A model ships, a capability jumps, and the plan they wrote a quarter ago is no longer the plan you’d write today. The best ones aren’t embarrassed by that. They’ve priced it in.

You can see the same shift inside their engineering orgs. For decades, the benchmark for “how much time should engineers spend on exploratory work” was something like Google’s famous 80/20, roadmap-heavy, with a slice carved out for experimentation. In the AI-native teams I spend time with, that ratio has flipped. Something closer to 70% experimental and 30% roadmap is now what I hear from the most forward-leaning shops, not as a formal policy but as the lived reality of how the week actually breaks down. The roadmap bends around the models, not the other way around.

And there’s a philosophical tell I keep hearing that I think matters more than any of the tactical ones. The probabilistic founder operates with what I’d call an agent-default posture: the assumption that anything can be done with an agent in mind, and if the agent doesn’t work, the operator - the human - has failed, not the agent system. That’s a fundamentally different locus of accountability than the one most of us grew up with, where a tool that didn’t work was, well, a bad tool. The probabilistic founder doesn’t blame the tool. They assume the tool is getting better every week and ask whether their own specification, review, and orchestration kept up.

What this means for how we invest

If you’re still pattern-matching founders against the deterministic archetype, you might miss the best ones in this cycle. Some of the signals that would have read as “unstructured” or “not rigorous enough” five years ago are, in this environment, exactly the traits that match the shape of the work - a willingness to kill a feature the day after a model release, to live inside uncertainty without flinching, to treat the roadmap as a hypothesis rather than a promise.

I don’t think rigor is dead. I think it has moved. The rigor that matters now lives in experimentation quality, in selection discipline, in the ability to direct a fleet of agents toward the right problem and tell the brilliant output from the plausible-looking-but-wrong output. That’s a different muscle than five-quarter roadmap adherence, and we’re still learning how to evaluate it.

And it’s worth saying plainly what doesn’t change. Probabilistic doesn’t mean casual. The probabilistic founder still has to be a relentless executor - arguably more so, because an environment that rewards running ten experiments instead of one is unforgiving to anyone who can’t actually finish things. Velocity is the price of admission; the ability to ship, close loops, and compound week over week is as non-negotiable as it ever was.

Likewise the probabilistic founder still has to be an absolute talent magnet. If anything, that bar has gone up. In a world where a small team of the right people plus a fleet of agents can out-ship a team of fifty, the premium on pulling in the top one percent of operators is higher, not lower. The best probabilistic founders are, without exception, the kind of people other great people rearrange their careers to work with. Experimentation without execution is noise and speed without talent is churn.

The bet

Tim Davis ends his essay with a line I keep turning over: the bet of this era is whether the human in the loop stays sharp enough, honest enough, and trained well enough to be worth having in the loop at all.

The founder version of that same bet is whether the operator pointing the fleet has the taste, the speed, and the comfort with uncertainty to compound faster than a competitor still trying to plan their way through.

The probabilistic founder is that bet, in a person. And it’s the one I’m convinced is worth making.

“The Token Threshold is the point at which the cost to build a capability from scratch exceeds the cost to buy it, measured in inference tokens. Every software product sits either above or below this line. Below it, agents build. Above it, agents buy. The line is invisible, unmapped, and moving.”

Every software product now has an invisible line.

The entire SaaS industry was built on a single assumption: the buyers are human. Humans evaluate tools, sit through demos, get approval from procurement, and eventually choose your product over building something internally. The entire model (positioning, pricing, PLG motions, sales cycles, feature roadmaps, steak dinners) is designed to influence that purchasing decision. It works because humans have budgets, time constraints, job requirements, career aspirations, and a limited tolerance for greenfield engineering risk. So they buy. Some more emotionally than others.

Today, the SaaS industry is getting decimated and that assumption is breaking. Because if you play the current trajectory forward and more enterprises complete their agentic transformation (we’re all running Cowork on our desktops now, aren’t we?), the real influencer of a purchasing decision in the AI era is no longer just human.

In February 2026, a research team at Amplifying.ai pointed Claude Code at real production codebases 2,430 times. Blind prompts. Just tasks: add authentication, add feature flags, add caching, set up a job queue.

In 12 of 20 categories tested, the agent skipped the tool entirely. Instead, it chose to build. Authentication from scratch: JWT, bcrypt, session management, every single time. Feature flags: 69% build rate. Caching: 50% build rate in newer models and rising. Task queues: custom implementations, over Celery, over BullMQ.

But on the other side:

GitHub Actions got picked 94% of the time. Stripe, 91%. Vercel, 100% for JavaScript deployments. shadcn/ui, 90%.

The agent chose deliberately. Aggregating individual decisions into efficient outcomes. Adam Smith’s invisible hand at machine speed.

—-

Agentic Procurement Evaluation

In the Amplifying AI study, they looked at individual tasks where the models show specific preferences while converging on Custom/DIY implementations.

But in an enterprise procurement instance with complex requirements and expensive contracts, what factors into the agent’s decision to recommend build vs buy?

One has to think the Agent ICP calculates: can I reach the required outcome more cheaply and reliably by building it myself? With no regard towards long standing vendor relationships, career risk, or boozy conference lunches.

This means that every software vendor will have to exceed that bar in order for the Agent to determine it’s more worth the investment to purchase from you. We call this the Token Threshold.

The Token Threshold is the final estimate after an agent performs a lifecycle comparison:

1. Define the target outcome. What does “done” actually look like?

2. Define the acceptance bar. What level of correctness, reliability, and security is required?

3. Estimate the build path. Spec + implementation + verification + debugging + maintenance + incident handling.

4. Estimate the buy path. Evaluation + integration + adaptation + vendor cost + dependency risk.

5. Compare reachable confidence. Beyond initial buildability, who gets you to the acceptance bar more reliably over time.

6. Buy only if the product materially improves success probability or lowers lifecycle cost.

For JWT and bcrypt? Build, every time. For PCI-compliant payment processing? The math goes the other way entirely.

The line moves

The Token Threshold moves. Continuously on a downward trajectory with every model release and the fact that tokens are deflationary.

Opus 4.6 builds custom solutions 50% more often than Sonnet 4.5 in the same categories. Redis went from a 93% pick rate in Sonnet to 29% in Opus 4.6. Celery went from 100% to 0%. The newer the model, the more it builds and the less it buys.

Open source accelerates this further. Developers have always pulled from OSS rather than rebuilding everything from scratch. Agents will do the same, only faster and more systematically. A product that felt safe because “nobody would build that from scratch” faces a new question: is there a good enough open source version an agent can just leverage? If yes, the threshold just dropped again, and the vendor in the middle gets bypassed entirely.

And here’s where it compounds. As enterprises shape their engineering processes around building with agents, the agent buyer gets more capable and more dominant with every cycle. Internal tooling, approved libraries, preferred patterns all train the next generation of agent decisions. The enterprises moving fastest today are building institutional muscle memory that makes their agents faster and more opinionated tomorrow.

For software vendors, that means the window to establish above-threshold positioning is narrowing.

The Napster parallel is worth sitting with. Napster didn’t make music worse, but it made copying cheaper than buying, for the first time. The industry that survived held something you simply couldn’t copy: live experience, artist relationships, synchronization rights. That’s the question for SaaS. What do you have that tokens genuinely have to purchase?

Bear vs Bull

The bull case: Products that clear the threshold win with near-monopoly conviction. That kind of pick rate in an agent-driven world is a distribution flywheel with no equivalent in human procurement history. Every agent that spins up in a developer’s sandbox is a potential procurement event. The winners win bigger than traditional GTM ever allowed.

The bear case: Most software sits below its own threshold and has yet to find out. The agent economy runs on zero authentication vendors, zero feature flag tools, zero caching layers. It builds them. The long tail of enterprise SaaS is about to discover that human friction was doing all the heavy lifting.

Both are right. The Token Threshold is simultaneously the best thing that’s ever happened to software with genuine moats, and an extinction-level event for software that relied on human inertia.

Core IP

Here’s the thing. The products that will always sit above the line share one property: their value lives in what the code connects to.

Call this core IP. There could be more, but these four came to my mind:

Network effects. The moat here are the participants. You can’t token-generate a two-sided marketplace with real liquidity, or a professional network where the value is who’s actually on it. The value in each case lives in participant behavior, history, willingness to transact. An agent can use any of these platforms. It cannot simply conjure critical mass.

Proprietary data & context arbitrage. Models train on available data. If your product creates, captures, or acts on data with no public equivalent (transaction histories at scale, sensor readings from physical systems, domain signals accumulated over years) that’s a ceiling agents hit every time. No new entrant bootstraps it. A model can’t either.

Regulatory and trust infrastructure. The most underrated moat in software. PCI certification, FedRAMP authorization, SOC 2 Type II, banking relationships, enterprise security reviews that took 18 months: none of that is token-generatable. Agents buy Stripe because the alternative is becoming a payment processor. The compliance layer is years of work even when the code is trivial. That asymmetry holds regardless of how capable the models get.

Risk transference. This one the data doesn’t surface, but enterprise buyers live by it. Even when an agent can build something, large organizations often buy anyway, because they need someone accountable when it breaks. A vendor carries SLAs, support contracts, liability, and a throat to choke. For regulated industries and risk-averse agentic buyers, that accountability layer is the purchase. Technical complexity is beside the point.

Back to hard things

Here’s what I keep coming back to.

The era we’ve been living in: fast traction, fast fundraising, AI-assisted shipping. It created real companies. It also created a generation of software built on the quiet assumption that human inertia was permanent. That switching costs would hold. That procurement cycles would protect you. That being first and fast was enough.

It was always a knife’s edge. AI is just shining a light on it.

As the agent becomes the economic buyer, founders and investors get pushed back toward something that looks a lot like real innovation. Beyond growth hacking. Beyond PLG funnels. Building things that are genuinely hard to replicate. The kind of work that takes years not because the tools are slow, but because the moat itself doesn’t compress.

That feels like where we’re going. And honestly? Good. Maybe it is time to build hard things again.

Data: Amplifying.ai Claude Code Picks benchmark · Feb 2026 · 2,430 responses across 3 models, 4 repos, 20 categories

Las Vegas in August is always intense, but this year the real heat came from breakthroughs not just in cyber tools, but in how we think about offense, defense, simulation, identity, and human sustainability under pressure all under the large cloud cover of AI (obviously). The Decibel team was out in full force hosting a series of events from the Vibes & Cocktails happy hour with the awesome Mike Privette from Return on Security, a Women in Cyber brunch and our usual Founder Oasis suite where founders can hang out and take a breather from the craziness of the BlackHat Expo. Over these glorious few days inside the Mandalay Bay hotel, here are some of the biggest topics and themes that were discussed:

The First AI-in-Cyber Use Case has matured: Toil Reduction

It's taken a lot of hype cycles, but we've finally hit a clear, repeatable AI use case in cybersecurity: getting rid of soul-crushing toil. Sublime Security's ADE (Autonomous Detection Engineer) is a prime example: automating the grunt work of detection engineering so humans can focus on higher-order problems. Dropzone AI is doing the same for the SOC, taking Tier-1 alert handling and triage from hours to minutes without sacrificing quality. The value is obvious because the before/after delta is so stark. No philosophical debates about "will it work". It's already working.

The timeline tells an important story about enterprise AI adoption. Edward Wu, founder of Dropzone AI, first introduced the concept of investigating alerts using LLMs at our Founder Oasis at RSA in March of 2023, only a few months after ChatGPT was released. Dropzone was born in August 2023, and while he long proved that its AI was effective for alert investigations, the human acceptance, trust, and willingness to delegate alerts and reduce toil took until recently to fully materialize. AI SOC dominated every conversation on the expo floor at Black Hat 2025. For me, this marks the moment we can officially declare victory on AI-powered toil reduction in security. What was once a pilot in many enterprises has become a mainstream operational reality.

This raises a critical question for the broader AI-in-security landscape: now that we've moved past the foundational debate of whether AI works in security contexts, how much faster will we adopt other AI use cases? The answer likely depends on whether those use cases can demonstrate the same stark before/after delta that made toil reduction so compelling. With trust barriers lowered and operational patterns established, I think the adoption curve for subsequent AI security applications should accelerate significantly

Proudly Offensive — Deterrence in the AI Era

After this year’s RSA, I wrote an article about "Proudly Offensive", an idea on how in an AI world, security insights increasingly come from offense, not just from watching and waiting. Historically, security posture leaned heavily on detection, with just enough response to feel proactive. Organizations built their security programs around the assumption that visibility and monitoring would give them the edge they needed.

But AI fundamentally changes this calculus. We're already seeing operational proof: startups using AI agents to dominate bug bounty programs, and more concerning, full end-to-end data exfiltrations that run autonomously from initial access to final payload delivery. The offensive capabilities of AI have moved from theoretical to operational.

Bug bounties are a useful proving ground. They demonstrate that AI can find vulnerabilities faster than humans and execute complex attack chains with minimal human oversight. But the real question is whether we can build offensive cyber capabilities that matter when geopolitical tensions escalate. What happens when the call comes in: "We need you to help"?

This is where deterrence becomes front and center. The organizations and nations that figure out how to weaponize AI for cyber operations will have strategic leverage. We're seeing early signals in American defense investments: companies like Palantir and Anduril building AI-powered capabilities at scale, not for corporate penetration testing, but for real-world adversarial engagement.

This shift is technical, political, and extisential. In an AI-driven threat landscape, deterrence requires demonstrable offensive capability. The willingness to develop, deploy, and when necessary, unleash AI-powered cyber operations will define which nations maintain strategic advantage in the coming decade. American dynamism built the internet, scaled cloud computing, and created the AI models powering this revolution.

The next chapter won't just reward those who can defend better, it will reward those who can project power through AI when called upon to do so.

Simulation Is All You Need (…Almost)

One of the more interesting threads was the growing role of digital twins in cyber. Enterprises are interested in building high-fidelity replicas of systems and networks to safely run security experiments, train AI models, and test defenses. This Dark Reading piece on this is worth a read. Today, these simulations excel at controlled, discrete scenarios perfect for training or red-team-blue-team exercises. But the leap to truly continuous, real-time "mirror worlds" of live enterprise environments is still constrained by data freshness, fidelity, and cost.

The most compelling advancement is how these twins are evolving beyond simple network topology modeling to capture dynamic interactions between applications, cascading effects of component failures, and environmental factors that influence security posture. Google shared the actual complexity required in their approach: real-time data streams, comprehensive monitoring capabilities, and computational power to model both normal operations and adversarial scenarios simultaneously.

The next evolution will likely involve AI agents operating within these simulated environments. While today's digital twins rely on human red teams manually probing for weaknesses, the future points toward autonomous agents that could continuously explore attack paths, test defensive responses, and discover novel vulnerability combinations at machine speed. We're not there yet, but the foundation is being built.

This sort of rehearsal logic combined with operationalized threat intel (like GLACIAL PANDA's long-duration persistence) can let defenders explore edge cases and policy effectiveness before reality does. It's a cyber range that no longer approximates, but lets you fight today's adversary with your own specific blueprint.

Other Blackhat Thoughts

Social Engineering - Phishing, Vishing, really all the -ishings

The narrative is changing from phishing campaigns to quantifying and modeling human risk in real-time. The old security awareness training model of quarterly compliance checkboxes is dead. In its place, we're seeing human risk management emerge as a control layer that feeds behavioral telemetry into identity systems, conditional access policies, and even insurance underwriting (I wrote about this recently!). According to Crowdstrike, “Vishing attacks increased 442% from the first to the second half of 2024 and the number of vishing attacks in the first half of 2025 have already exceeded the total number seen in 2024.”

This means that authentication must evolve beyond static credentials to dynamic behavioral baselines and step-up validation that adapts to the sophistication of AI-driven social engineering. The threat is substantial and requires a different approach (See Push’s Phishing Detection Evasion Techniques Matrix) now that adversaries can clone voices or generate convincing deepfakes in real-time that adapt mid-conversation to bypass detection systems and human intuition alike.

Security for AI

Acquisitions continued this year. Protect AI’s acquisition was announced during RSA and SentinelOne announced its intent to acquire Prompt Security at the start of Black Hat. This reminds me of Evident.io’s acquisition in the early cloud days. The vendors getting acquired today are the ones creating the security controls that make AI deployment viable at enterprise scale. In the past year, solutions have moved from simple model governance to AI security platforms that can detect prompt injection, monitor model behavior drift, and protect training data pipelines. But the question is whether or not this is the end game solution to Security for AI since we don’t even know the full attack surface yet.

From Application Security to Product Security

There's a subtle but important shift happening in how organizations think about security ownership. Traditional AppSec focused on finding vulnerabilities in code. Product Security embeds security thinking into the entire product lifecycle: from design decisions to feature rollouts to customer-facing security controls. This isn't just semantic evolution; it's organizational. Product Security teams report to product leadership, not just security leadership, and they're measured on customer outcomes, not just vulnerability counts. AI companies are attacking this problem at different stages: Prime Security (this year's Black Hat Startup Champion) focuses on design-stage risk analysis before code is written, making the call that security needs to be woven into the product development process itself.

Final Thought

Black Hat 2025 made it clear: AI in security is no longer just a thought experiment, it’s operational, shaping how we reduce toil, test defenses, and even think about deterrence. From the expo floor to our own events, the energy, ideas, and debates reminded me why this community is so special. Huge thanks to all the security practitioners who joined us, and to our friends across the Decibel family and beyond; it was truly great to catch up! Looking forward to seeing you all again soon…ideally somewhere with a milder climate.

PS: A big shoutout to my good friend Sean Sun (founder of Miscreants) for hosting an incredible (and much needed off strip) dinner with so many friends, teachers and creators in the security community that I have admired for a long time as I have been a student of the security industry.

One of my favorite books growing up was The Rainmaker. John Grisham's novel is about a young lawyer taking on insurance companies with nothing, but grit and a contingency fee arrangement. There's something brilliant about the economics in that story: the lawyer only gets paid if he wins, and when he does win, he gets paid very well. Perfect alignment of incentives, clear attribution of success, and a business model that puts the service provider's interests directly in line with the customer's outcome.

The outcome-based pricing conversation in regards of AI software is everywhere right now (and for good reason). It promises perfect alignment between buyer and vendor: only pay when value is delivered. In theory, it fixes everything wrong with SaaS seat-based bloat and unpredictable AI usage spikes. Instead of charging for effort, compute, or vague abstractions like "credits," you just charge for the thing the customer wants: results.

But that simplicity hides a deeper complexity.

Here's the real tension: in a sense, usage-based pricing is already a form of outcome-based pricing; just one where the model decides what the outcome is. You pay per token, per function, per interaction. But the "outcome" is inferred, often implicitly. Meanwhile, human-defined outcomes, like "issue resolved" or "churn prevented", are explicit, interpretable, and contract-worthy.

The trouble comes when those two diverge. If the model believes it completed the task, but the human disagrees? Or if a user drops off mid-flow and the vendor counts it as a resolution? That dissonance creates billing disputes, mistrust, and strategic risk.

This post is about that dissonance.

We'll explore the full spectrum of outcome-based pricing from deterministic work-completed to probabilistic success-shared. We'll walk through the four key quadrants of outcome design, define where different companies land, and unpack the risks of pricing outcomes before alignment is achieved.

What Most Companies Are Adopting Today

In the current AI software market, pricing is trending towards the “hybrid” model. Hybrid pricing has taken over because it promises to solve the budgeting chaos of pure usage-based models while maintaining cost alignment with actual AI infrastructure. Most AI companies now combine base platform fees with usage tiers or credits so customers have predictable minimums, but variable final cost. Yet, hybrid models still have issues:

1. It's still hard to budget (usage spikes can blow through credit allowances)

2. It doesn't guarantee value (you pay for platform access whether the AI delivers results or not)

3. It adds complexity (customers now have to understand both subscription tiers and usage calculations)

Outcome-based pricing promises to fix that. But to do it right, you need the model and the human to agree on what success means.

When the Model and the Human Disagree

At the heart of outcome pricing is a question of definition. Who decides what counts as a completed task or a successful resolution?

• In usage-based pricing: The model defines success ("I returned a response, so I charged you X.").

• In outcome-based pricing: The human defines success ("My problem was actually solved, so I pay you X.").

When these are congruent, pricing aligns beautifully. When they're not, all hell breaks loose. This leads us to the core framework of this post.

(Yet Another) 2x2 Framework

The implementation of an outcome-based pricing model should consider the buyer’s objective(s). Is your offer a revenue driver? Or is it a cost center?

So this creates two critical dimensions:

• What are you charging for?

  • Work Completed (Task-Based) → Task finished, regardless of impact

  • Upside Captured (Success-Based) → Pay only when business results land

• What’s the buyer mindset?

  • Cost Center → Budget sensitivity, efficiency metrics

  • Revenue Driver → ROI-focused, outcome-oriented

Put those together and you get four pricing quadrants:

Ops Backbone (Cost Center × Work Completed)

• Think per-resolution pricing in support automation (e.g. Intercom Fin).

• Focuses on replacing headcount and improving deflection.

• Risk: Optimizes for cost reduction, not necessarily customer satisfaction.

Skeptic’s Gambit (Cost Center × Success-Based)

• Classic chargeback or reimbursement models where vendors are only paid when cost avoidance is proven and verifiable.

• Example: Chargeflow only takes a cut (~25%) of recovered funds from successful disputes.

• Risk: Attribution is clear, but the volume is unpredictable. Cash flow is highly variable and may not justify vendor overhead unless win rates are strong.High burden of proof and slow cycles; trust and attribution are hard.

Volume Multiplier (Revenue Driver × Work Completed)

• Pricing tied to agent throughput—e.g. outbound emails, summaries, meetings scheduled.

• Scales with activity, but not necessarily conversion.

• Risk: Over-incentivizes volume over quality; low gross margin if success rates are poor.

Skin-in-the-Game (Revenue Driver × Success-Based)

• The most aligned but riskiest quadrant: pay only when revenue impact is proven.

• Seen in performance-based agents, like chargeback wins or upsell automation.

• Risk: Attribution complexity, lagged revenue, and long sales cycles.

Each quadrant reveals a different risk-reward profile. Your margins and customer trust depend on pricing to match not just what is delivered, but how it's valued by the buyer.

Modeling the Margins — Four Real-World Scenarios

Scenario 1: Customer Support AI

(Work Completed × Cost Center)

The Promise: Intercom Fin charges $0.99 per resolution. Zendesk offers "pay only on successful automated resolutions." The value proposition is mathematical: 10x cost reduction compared to human support costs.

Pricing Model: $1.20 per resolved ticket

The Margin Reality by Resolution Type

Why It's Aspirational: As Decagon discovered, "the vast majority of customers choose per-conversation" over per-resolution because definition disputes become a recurring nightmare. Customer support remains fundamentally a cost center, buyers optimize for predictable savings, not uncertain value alignment. What counts as a "resolution" becomes a recurring billing dispute, while "conversation handled" is black and white.

The procurement logic is simple: why pay for outcome complexity when you're just trying to reduce headcount costs? Even when customers say they want resolution-based pricing, cost center budgets drive them toward whatever delivers the lowest total spend, usually per-conversation pricing that runs 20% cheaper overall.

Tokenomics Takeaway

Token cost is not the limiting factor. It's edge-case behavior (complex tickets + high escalation). Depending on distribution, margin flexs violently: if "complex" tickets rise from 10% → 25%, blended margin crashes.

Cost center buyers choose simplicity over alignment, even when outcome pricing could theoretically offer better value.

Scenario 2: Fraud Detection AI

(Upside Captured × Cost Center)

The Promise: Perfect outcome alignment. Customers only pay when fraud is actually prevented. Like AirHelp's 35% commission model for flight compensation, the attribution is crystal clear.

Pricing Model: 15% of fraud losses prevented

Why It’s Aspirational:
Success-based pricing sounds like a win-win. But fraud is not a binary outcome like a lawsuit, thereby it’s probabilistic. You still pay to run the model during quiet months. And every false positive comes with a cost (missed revenue, user friction).

What We Actually Learn From Modeling It:

• Revenue depends on fraud volume you can't predict or control.

• Fixed costs don't scale down because infrastructure is always running an evaluating for fraud.

• False positive tax compounds: every legitimate transaction blocked creates customer service costs and lost revenue that you subsidize, but don't get paid for.

• Customer’s overall fraud prevention ecosystem improving means less fraud to prevent and monetize.

• Attribution infrastructure is expensive, proving you earned commission requires costly reporting that eats into already thin margins.

• Volatility compounds, monthly swings from 43% to -269% margin make financial planning impossible and investor confidence fragile.

Tokenomics Takeaway:

You've tied revenue to external factors completely outside your control. Quiet fraud months aren't just lower revenue, they're losses while infrastructure costs continue.

You're exposed to seasonality, economic cycles, and ironically, your own success (less fraud = less revenue). This model only works if you can guarantee consistent baseline fraud activity.

Scenario 3: Sales AI SDR

(Work Completed × Revenue Driver)

Company Profile: AI sales development platform for 200+ B2B companies Pricing

Model: $85 per qualified meeting booked + $3,500/month seat fee

Why It’s Aspirational:
The cost to get the job done keeps rising, but the price per meeting doesn’t. AI output is no longer enough—you need multi-agent orchestration, deep research, and adaptive touch sequences just to stay competitive.

What We Actually Learn From Modeling It:

• There’s a personalization arms race. What’s “good enough” today is table stakes tomorrow.

• Every agent you add compounds overhead. 20–30% per handoff = death spiral without pricing power.

• High margin today doesn’t protect you from tomorrow’s customer expectations.

Tokenomics Takeaway: You're in an arms race where the cost to deliver "good enough" work rises faster than your pricing power. Today's 85% margins assume current personalization standards, but buyer expectations compound annually. Each new competitor raises the bar for what counts as a "quality" meeting, forcing you to add more agents, deeper research, and longer workflows, while your $85 per meeting stays fixed. You're essentially shorting your own industry's innovation curve.

Scenario 4: AI Upsell Engine

(Upside Captured × Revenue Driver)

The Promise: 15% commission on AI-generated upsell revenue

Pricing Model: AI identifies opportunities, personalizes offers, executes outreach, only pay on closed deals**

The Attribution War:

Upsell Success Scenarios:

• AI-initiated, AI-closed: 25% of deals, 10% margin

• AI-initiated, sales-closed: 60% of deals, split commission → 5% margin

• Natural sales cycle (AI gets credit): 15% of deals, -130% margin

Why It's Aspirational: Sales teams fight every attribution decision. Did the AI "create" the opportunity or just identify an existing need? When sales reps close deals flagged by AI, who deserves credit? You're running expensive multi-touch campaigns while sales argues they would have found the upsell anyway. The more successful your AI becomes, the more sales pushes back on attribution.

Tokenomics Takeaway: You're building a business where your primary customer (sales leadership) has economic incentives to minimize your contribution. Every dollar you earn reduces sales team commission credit, creating internal political battles over attribution. Your margins get squeezed not by technology costs, but by organizational politics and revenue sharing disputes. You've created a model where success breeds resistance from the people who control your pipeline.

The Bigger Lesson: It's Not About the Numbers

Here's what this modeling exercise actually reveals (as of course all the numbers were illustrative): Outcome-based pricing isn’t just a billing strategy that you can (or should) implement overnight. It requires deeper thinking to make sure model-determined work will match human-defined value. But when those diverge, even slightly, your margins can vanish, your trust can erode, and your contracts can implode.

The truth is, outcomes live on a gradient ranging from atomic actions to economic impact. The further you travel toward success-based pricing, the more you depend on humans to confirm that value was real, contextual, and worth paying for. That’s where risk concentrates.

Each quadrant we explored tells a different story:

• Customer Support AI showed us that resolution counts hide escalating complexity.

• Fraud AI reminded us that even perfect success can be invisible during a quiet month.

• Sales AI SDR revealed that the bar for "done" keeps rising, but the price doesn’t.

• AI Upsell Engine exposed that your biggest customer (sales leadership) has economic incentives to minimize your contribution.

So if you’re building in this world, the real decision isn’t just what to charge for. It’s whose definition of value you trust and how often you expect it to be wrong.

A fresh look at security awareness in the age of AI

On a January morning in Hong Kong, a finance clerk joined what looked like a routine video call with the company’s CFO. The voice, the mannerisms, even the virtual backdrop, flawless. But the “executive” wasn’t real. It was an AI-generated deepfake. Fifteen wire transfers later, $25 million was gone.

Just a few months later, another unsettling moment made headlines: someone used an AI-cloned voice of U.S. Senator Marco Rubio to impersonate him in a Signal call to government officials. The goal? Social engineering at a geopolitical scale.

Incidents like these make one thing clear: people, not endpoints, are the new perimeter. And the old model, reminding employees not to click on links once a quarter, is no defense against adversaries armed with generative AI, voice cloning, and real-time deception.

Security Awareness Was Always a Compliance Checkbox

For most of the past decade, Security Awareness Training (SAT) has been viewed as a formality. A quarterly fire drill. Employees sat through a video, passed a quiz, and forgot the content before lunch. It wasn’t really about behavior change. It was about satisfying audit requirements.

That’s changing. In boardrooms across industries, CISOs are ripping out KnowBe4 and Proofpoint contracts, not because phishing went away, but because it evolved. Today’s attacks are more targeted, more dynamic, and disturbingly convincing. “Awareness” alone just doesn’t cut it.

A Short History of a Long-Suffering Category

SAT took off in the 2010s as regulatory pressure and cyber insurance mandates created budget for anything that looked like training. A few players rode the wave:

• KnowBe4 went public in 2021, then was taken private by Vista Equity in 2023 for $4.6B.

• PhishMe (later Cofense) sold to a PE consortium for around $400M.

• Wombat Security was acquired by Proofpoint for $225M in 2018; Proofpoint itself was later taken private for $12.3B.

What they offered was simple: phishing simulations, templated training, and dashboards to check the compliance box. The moat was content scale, localized modules, integrations with email gateways, and wide distribution via MSPs. It worked, but only because the threat model was basic. Many of us still remember those hilariously bad phishing emails with typos and Comic Sans.

Why the Old Moats No Longer Hold

Fast-forward to 2025, and those same content libraries feel archaic. Users ignore hour-long training. Attackers move faster than update cycles. Phishing-as-a-Service kits now bundle LLM-generated lures, MFA bypass pages, QR-code bait, and more, all for less than a Spotify subscription.

Voice-based attacks are no longer novelty edge cases. Deepfake tools can clone a colleague’s voice from a few seconds of audio. The result? A simple phone call can become a high-stakes social engineering vector.

In this world, “click-to-comply” training is theater. The threats are adaptive and real-time. Defenses need to be too.

From Training to Risk Intelligence

The next generation of vendors aren’t in the training business. They’re in the business of human telemetry, quantifying, modeling, and reducing behavioral risk. This shift is giving rise to a new architecture built on five principles:

1. Per-user risk scoring
Modern platforms ingest identity, privilege, behavioral signals, and phishing test outcomes to create dynamic risk scores. These scores can flow into SIEM, XDR, IAM, even HR.

2. Just-in-time micro-nudges
Instead of annual training, users get real-time interventions: 60–90 second nudges triggered by specific risky behaviors, like credential reuse or clicking a suspicious link.

3. Consequences over certificates
Some platforms integrate with IAM tools to enforce access guardrails. A user who fails three phish tests might have app access temporarily throttled, automatically reducing blast radius without waiting for IT.

4. Behavior-driven content and closed-loop feedback
LLMs are used to auto-generate hyper-relevant training based on role, region, and recent behavior. Some systems allow security teams to feed real-world incidents into the engine, creating simulations that mirror actual threats. Some vendors are even exploring incident replays, interactive walkthroughs of real BEC attempts or deepfake voice scams, to help employees learn directly from modern attack chains.

5. Adaptive learning paths and gamification
Instead of static modules, emerging platforms are using performance data to personalize a user’s learning path over time, reinforcing weak points and skipping what’s already known. Some have even added light gamification elements, like team-based progress dashboards or behavioral leaderboards, to create healthy competition and engagement across departments.

The goal is no longer to “teach better” but rather to reduce the probability, and impact, of human error.

What Makes This a Moat Business Now?

In the 2010s, defensibility came from content scale. In the 2020s, it comes from behavioral depth.

With GenAI, personalized content is cheap. What matters now is the richness of human telemetry, signals across identity, endpoint, cloud, and behavior. That telemetry powers better detection, better response, and better insurance underwriting.

And unlike static training modules, behavioral telemetry compounds. The more signals and systems a platform connects to, the more valuable, and thereby sticky, it becomes. This is why CISOs are exploring whether human risk platforms could either feed UEBA systems or serve as lightweight UEBA alternatives.

Where It’s All Going

Security Awareness Training will likely fade as a standalone category. In its place, we’ll see Human Risk Management emerge as a control layer between identity, detection, and compliance systems. Risk scores will drive conditional access. They’ll show up in board packets. They may influence insurance underwriting and even employee risk evaluation.

To make this a reality, some teams are already pushing the envelope:

• Phishing threat intel feeds are being piped directly into simulation engines to keep lures current.

• Security team playbooks are auto-triggered based on escalating user risk.

• Employees are getting access to their own risk dashboards, giving them agency and visibility into their own posture.

The next $4B exit won’t be for a content library. It’ll be for a telemetry engine that connects human behavior to machine response. Because in an era of deepfakes and AI-driven deception, awareness isn’t protection. Instrumentation is.

Back in 2023, I wrote a post called The Price is AI-ght about how early AI-native startups were starting to wrestle with pricing and how seat-based models were already starting to crack under the weight of unpredictable usage. Since then, the market hasn’t exactly stabilized, but rather has progressed at a rapid rate.

We’ve seen companies experiment with everything from flat rates to token caps, credit systems, and abstracted compute units. Some of the early bets have broken down in public while others are still figuring out how to align real costs with perceived value.

Tokenomics is my attempt to unpack that. It’s a running study on how AI companies make money, where the cost centers really are, and what kind of business models will actually scale.

Things are still in flux, but asking the right questions early matters. If you're a founder building in this world, I hope this is helpful to you!

The Pricing Evolution of AI Coding Agents: From Per-Repo to Compute Units

Pricing in the AI world is still early and unsettled. There are a lot of variables in play: how customers use the product, how often they interact with it, what agents are doing in the background, and what the underlying model providers are charging. Most founders I talk to are still figuring out how to balance predictable revenue with unpredictable costs.

This week, I'm diving deeper into how developer tools have evolved their pricing models over the years, why Cursor's recent pricing change stirred up so much controversy, and what the rise of agent swarms means for the future of devtool economics.

To understand where we're headed, I've taken a look the current pricing strategies of some of the major AI coding tools: Cursor, Devin, Claude Code, v0, Bolt, Codex, Replit, and Lovable.

Each model tells a story about token economics, market positioning, and the search for sustainable AI pricing.

Five Phases of Developer Tool Pricing

Phase 1: Per-Repository Era (2008-2014)

First, a quick turn back in time: GitHub started simple in 2008 with a freemium model based on private repositories. Public repos were free, and you paid for privacy. In 2011, a "Micro" plan cost $7/month for 5 private repos, while a "Medium" plan ran $12 for 10. GitLab followed a similar model, focusing on features gated by tiers rather than user count.

Why it worked: Developers often worked solo or in small teams. Metering access to private code, not people, aligned with real usage patterns of the time.

Phase 2: Per-Seat Enterprise (2015-2020)

As usage shifted from solo developers to teams, both GitHub and GitLab introduced per-seat pricing. GitHub Team and GitHub Enterprise charged $4–$21 per user/month depending on features. The shift made sense: collaboration became the primary value driver, and per-seat pricing mapped cleanly to team structure and enterprise budgets.

Why the shift: Enterprises needed budgeting clarity. As more companies adopted CI/CD and DevOps pipelines, tools scaled across large teams. IT procurement teams preferred clear pricing per developer, and unit economics became more predictable.

This model helped turn devtools into big business. GitHub had around 28 million users when Microsoft acquired it for $7.5 billion in 2018. GitLab went public in 2021 and now serves more than 30 million users with $500M+ ARR. The key thing: pricing was never about compute. It was about developers.

Phase 3: Flat-Fee AI Augmentation (2021-2023)

GitHub Copilot entered in 2021 with a new twist—AI assistance priced at a flat $10/month per user. It didn't matter how much you used it. The value was in seamless integration and predictable cost, abstracting away the complexity of token consumption.

Why it gained traction: Copilot was cheap, worked within VS Code, and offered high perceived ROI for daily coding tasks. Predictability trumped cost control, and developers didn't need to think about usage limits.

Phase 4: Flat-Rate with Usage Caps (2023-2024)

Then came Cursor in 2023.

At $20/month for the Pro plan, users got up to 500 "requests" per month; a flat subscription fee with a usage ceiling. This wasn't true usage-based pricing; users didn't pay per token or per API call. Instead, they paid a predictable monthly fee but were capped at an abstract number of "requests."

Why this worked initially: In the early days of AI coding tools, usage patterns were relatively light and predictable. Most requests were simple completions or small code generations. The abstracted limits felt generous, and few users hit their caps.

The Cursor Controversy: When Flat-Rate Pricing Breaks

By June 2025, Cursor's economics had become untenable. As the product matured, power users started chaining together complex multi-step agents, piping in huge context windows, and running long, compute-heavy sessions. What looked like a flat subscription on the surface was burning through API credits underneath.

In the meantime, Cursor continued to experience breakneck growth and saw its annual recurring revenue breach the $500 million mark by June 2025, after revenue doubling roughly every two months.

The company was forced to respond. Overnight, it dropped request-based pricing entirely and moved to a hybrid usage-based billing system grounded in token consumption. The new Ultra plan, for heavy users was priced at $200/month and offered ~20x more usage, but the Pro plan introduced hard limits based on compute consumption, not requests.

The backlash was swift. Developers were used to Copilot-style predictability. They didn't expect to face massive overages under a $20 plan. Some users reported bills of over $1,000 in a single month. Cursor was forced to introduce more transparent dashboards, offer refunds, and respond to community criticism.

To their credit, the Cursor team responded swiftly and constructively. Within days, they issued public explanations, offered refunds to affected users, launched clearer usage dashboards, and allowed existing annual subscribers to retain the old request-based plan until renewal. They also clarified the new system’s controls, including spending caps and model routing options, to restore trust and transparency.

But what about the others?

Phase 5: Abstracted Usage Units (2024-Present)

The newest evolution attempts to solve the token complexity problem by abstracting usage into more intuitive units that still map to underlying costs. To understand how this plays out in practice, I looked into the other seven leading AI development tools.

Devin: Agent Compute Units

Devin's Agent Compute Unit (ACU) is a normalized measure that bundles together all computational resources (think virtual machine time, model inference, and networking bandwidth) into a single unit representing approximately 15 minutes of active AI development work.

The Core plan charges $2.25 per ACU with pay-as-you-go billing, while the Teams plan includes 250 ACUs at $2.00 each.

As each ACU represents approximately 15 minutes of "active Devin work," the $20 entry plan is equivalent to about 2.25 hours of work.

Which may not seem like much at first glance, but Devin 2.0 now completes over 83% more junior-level development tasks per ACU compared to its predecessor, nearly doubling the output per unit.

Tokenomics Takeaway: Devin has created units that abstract away model complexity while maintaining meaningful cost correlation. Variable task complexity is baked into the unit calculation, making pricing more predictable for users while preserving unit economics.

Claude Code: Tiered Subscription & Model Ownership

Claude Code operates on a tiered subscription model: Pro ($20/month) provides 10-40 prompts every 5 hours for small repositories, while Max plans offer 5x ($100/month) and 20x ($200/month) usage with 200-800 prompts every 5 hours for larger codebases.

Starting at $17 per month for individual developers, with enterprise plans reaching significantly higher price points, Claude Code has seen 300% active user growth and 5.5x revenue growth since launching Claude 4 models in May.

Pro subscribers can only access Sonnet 4, while Max subscribers can switch between Sonnet and Opus 4 models using the /model command. The platform targets organizations with dedicated AI enablement teams and substantial development operations.

Tokenomics Takeaway: Claude Code's tiered approach works because Anthropic owns the underlying models, giving them a significant cost advantage over competitors paying third-party API fees. While we don't know their exact internal compute costs, this vertical integration likely allows much higher gross margins than tools like Cursor that were paying full API rates.

v0: Mapping Pricing to User Value

Vercel’s v0 reveals a strategic use of AI tooling to reinforce their broader platform ecosystem. Their credit-based pricing model aligns costs with actual token consumption while driving adoption of their core infrastructure.

v0 operates on a credit system where users receive monthly credit allowances: Free ($5), Premium ($20), Team ($30/user), and Enterprise (custom). Credits are consumed based on actual token usage across different model tiers, with v0-1.5-lg costing $7.50 per million input tokens and $37.50 per million output tokens, while smaller models like v0-1.5-sm cost significantly less at $0.50/$2.50 per million tokens.

The platform includes relevant context like chat history, source files, and Vercel-specific knowledge when generating responses, with this context counted as input tokens. Crucially, v0 heavily favors Next.js in its code generation (another Vercel product) creating a natural pathway from AI-assisted development to their hosting and deployment platform.

Tokenomics Takeaway: v0's transparent token-based pricing maps directly to user value perception while serving a larger strategic purpose. Users pay for actual compute consumption rather than abstract units, but more importantly, the tool generates Next.js applications that naturally deploy on Vercel's infrastructure. The AI tool becomes a customer acquisition engine for their core hosting business by making Vercel deployment the obvious next step after code generation.

Lovable: Transparent Task-Based Agent Pricing

Lovable has evolved beyond simple message-based pricing to introduce a more sophisticated agent pricing model that charges based on the actual complexity and scope of work performed. While Default and Chat modes still cost 1 credit per message, their Agent Mode uses dynamic pricing that reflects the computational effort required for each task.

The Pro plan ($25/month) provides 100 monthly credits plus 5 daily credits (totaling up to 150 credits per month), with unused monthly credits rolling over. Agent Mode pricing varies significantly based on task complexity: removing a footer costs 0.90 credits, adding full authentication with sign-up and login costs 1.20 credits, while building a complete landing page with generated images runs 1.70 credits.

This is a fundamental shift from uniform pricing to outcome-based billing. Simple edits and removals cost less than a full credit, while complex multi-component builds cost more. Users can see the exact cost of each message by hovering over message options in their history.

Tokenomics Takeaway: Lovable's approach addresses one of the core problems in AI tool pricing: the massive variance in computational work between simple and complex requests. What sets them apart is the radical transparency: users can see the exact cost of each individual message by hovering over message options in their history. By moving to task-complexity pricing in Agent Mode with full cost visibility, they've created a model where users pay more fairly for what they actually get while understanding exactly what each interaction costs.

OpenAI Codex (non-API): The Subscription-Freebie (for now?)

Being still in early access, OpenAI's Codex is included with ChatGPT Pro ($200/month), Enterprise, Team, and Plus subscriptions, providing "generous access at no additional cost" during the initial rollout period, after which rate limits and on-demand pricing will apply.

Tokenomics Takeaway: Time will tell when they release pricing.

Bolt.new: Flat Fee, Token Ceilings

Bolt presents itself as the simple option: $0 to start, $20/month for Pro, and $30/month per user for Teams. But behind the clean UI is a usage model that centered around tokens.

The free plan comes with 1 million tokens per month and a 150K daily limit. Which is enough for basic usage, but you hit the daily cap almost immediately when trying to build anything with complexity. The Pro plan bumps you up to 10 million tokens per month, removes the daily cap, and allows unused tokens to roll over. The Teams tier adds admin features like centralized billing and access controls, but keeps the same usage base.

There’s no per-request or per-agent billing here, just token ceilings. And while “unlimited” isn’t part of the pitch, the framing still feels generous compared to some of the newer credit-based or task-metered models. That said, the token cap means power users (especially those experimenting with background agents or long context windows) will need to track usage, even on paid plans.

What sets Bolt apart is actually the integrated development environment. Unlike tools that operate as plugins or external agents, Bolt provides a complete IDE where you can code, preview, and deploy full-stack applications entirely in the browser. This also means that users can modify code without spending tokens (especially helpful in getting out of those pesky troubleshooting loops that can burn tokens).

Tokenomics Takeaway: Bolt’s pricing looks flat, but it’s grounded in hard token ceilings. It’s a middle ground between flat-rate simplicity and usage-based fairness making it more predictable than credit-based models, but still constrained. Bolt’s approach creates stronger user lock-in than standalone coding assistants, as switching costs include not just the AI tool but the entire development workflow. The token-based pricing works within this model because users are paying for platform access, not just AI interactions.

Replit: Effort-Based Pricing via “Checkpoints”

Replit’s pricing page has an interesting strategy, comparing the money you spend to money you recieve ($20/month for $25 worth of credits). Though, “credits” seems opaque since there isn’t an immediate way to calculate the value or output you’ll get from $25 worth of Replit credits. But diving deeper into their effort-based pricing blog, it’s actually quite interesting. Read it here.

Replit rolled out an effort-based pricing model in mid-2025 that charges users based on the actual complexity of each task, rather than by token or message count. The system uses “checkpoints” to measure work, so instead of every request costing the same, simpler edits (like renaming a variable) might cost a few cents, while more involved tasks (like building a new component) cost more.

Each checkpoint’s price is visible in the interface, and usage rolls up into a monthly credit allowance ($25 on the Core plan, $40 per user on Teams) with the goal to make pricing more aligned with actual work done.

Tokenomics Takeaway: Replit’s effort-based pricing is an attempt at turning usage-based into something closer to “customer value”. But since it’s hard to quantify value, I think “effort” is an apt name. It seems pretty similar to Lovable’s agent and complexity-based pricing, but there’s something there in calling it “effort-based”.

What This All Means for Founders

The evolution of developer tool pricing reveals several critical lessons:

Flat-rate pricing isn't inherently wrong — especially in the early days, it’s still a great way to win when introducing new behaviors. Your first job isn't to maximize revenue; it's to build habits. Both Cursor and GitHub Copilot succeeded initially with flat pricing.

Transparent usage dashboards aren't optional in the AI era. Customers need to (and will soon demand to) understand their consumption patterns before they hit surprise bills. The tools with the smoothest transitions all invested heavily in usage visibility.

Hybrid models (base + credits) give predictability without unlimited exposure. Having predictable costs will still make the deal more tenable for enterprise procurement teams. Many AI-native tools are converging on this approach, though they implement it differently.

Abstraction can solve complexity, but requires careful design. Units should map meaningfully to both real costs and user value perception.

The Next Potential Tipping Point: Agent Swarms

The next wave of disruption won't be from individuals building faster; it'll be from code writing itself in swarms. Agentic development tools are showing us what happens when you give five agents the ability to coordinate, refactor, and reason about an entire codebase simultaneously.

Swarm-based workflows introduce powerful new capabilities like full-codebase refactoring and speculative scaffolding, but they also risk blowing up cost models entirely. What used to be a single developer calling the model a few times per day becomes 5–10 agents each calling different tools, running context windows in parallel, and looping over 1,000+ documents to complete a task. With no human in the loop, token consumption may become unpredictable and uncapped.

Adrian Cockcroft's Example on Swarms at Work

Developer tools are going through a pricing reset similar to its technological counterpart. As AI becomes more embedded, looping in the background, calling tools, writing code autonomously, flat fees and per-seat models start to break down. Each tool is handling it differently: Cursor moved from requests to tokens, Devin bundles compute into time-based units, and Bolt keeps things simple with monthly token caps. There’s no perfect answer yet, but one thing’s clear: pricing now has to reflect real compute costs, not just user count. And as multi-agent workflows become more common, those costs are only going to get harder to predict.

AI is rewriting the economics of enterprise software. Instead of seats and servers, we now measure value in tokens, context windows, and compute. This shift doesn’t just change how you build, it changes how you make money.

In this newsletter, we’ll explore the emerging business models behind AI-native companies: how they price, package, and scale in a world where your cost of goods is driven by inference, not infrastructure. We’ll cover topics from unit economics to go-to-market mechanics, and occasionally challenge some sacred SaaS assumptions along the way.

If you’re building and betting on the future of enterprise AI, Tokenomics is your field guide.

Close to three years after ChatGPT ignited mainstream generative-AI adoption, the offensive side of the security equation is making huge strides. When I first talked about the Era of the Autonomous Defense and the inevitability of Proudly Offensive security, my focus was on how defenders could automate at machine-speed. Since then, real-world vulnerabilities and breaches have shown that attackers are weaponizing the very same tooling. My partner Jess Leão just dropped a must-read on models that happily yank their own kill-switches, blackmail operators, and scheme around oversight for example. Given the rapid developments I wanted to pull together a short field report on some of the emerging styles of AI-enabled attacks, together with the implications each style creates.

Prompt-Injection and Tool-Chain Hijacks

LLMs wired into developer workflows are now a direct path to source-code theft and supply-chain compromise. Last month, Researchers at Legit Security showed that a single base-encoded comment inside a merge request could coerce GitLab Duo (powered by Claude) to dump private repositories, inject rogue HTML and phone home to an attacker-controlled URL.

In a very similar case, Invariant Labs uncovered a critical flaw in the widely-used GitHub MCP server, starred 14 k+ times on GitHub, that lets a malicious Issue trick an AI agent into leaking private repos. The way it works is that the adversary plants a public-repo issue laced with a hidden prompt-injection string. When a developer’s AI assistant, Claude Desktop connected via the GitHub MCP server, pulls the list of open issues, it ingests the booby-trapped text, executes the rogue prompt, and cascades into a malicious agent workflow.

The same pattern hit the orchestration tier: CVE-2025-3248 in the open-source builder Langflow lets an unauthenticated attacker hit the /api/v1/validate/code endpoint and execute arbitrary Python NVD. CISA shoved the bug into its Known Exploited Vulnerabilities catalogue after detecting in-the-wild abuse, and internet scans still found hundreds of exposed servers days later. Once an LLM or low-code workflow gains exec() rights, a stealth prompt or crafted payload can exfiltrate more data in seconds than months of slow, covert theft.

Synthetic Impersonation and Deep-Fake Social Engineering

AI voice and video cloning have demolished the cost of credibility. In May, the FBI warned that adversaries were texting, then phoning, targets with AI-generated voices of senior U.S. officials to harvest credentials. The alert followed an earlier bureau bulletin on AI-driven vishing and smishing campaigns. Data backs the trend: CrowdStrike’s 2025 Global Threat Report logged a 442 % jump in vishing attacks between H2 2024 and H1 2025, fueled by AI voice cloning. Beyond reputational damage, every deep-fake call burns executive time, and one breached mailbox can ripple through partner ecosystems, creating a hidden productivity and trust tax rarely itemized in breach-cost spreadsheets.

Crime-ware-as-a-Service: Malicious LLMs and Phishing Factories

Generative AI is turning script-kiddies into enterprise-grade criminals. As one of our security founders likes to say “What was once nation state is now common place”. Dark-web operators now tout an uncensored chatbot dubbed FraudGPT for $200 per month or $1,700 per year, boasting 3,000+ confirmed sales that bundle turnkey malware and spear-phish generation. Delivery scales just as aggressively: Barracuda telemetry logged more than one million Phishing-as-a-Service attacks in January–through February 2025. 89% of them launched via the Tycoon 2FA kit, with EvilProxy and Sneaky 2FA rounding out the field. When anyone with $200 and a Telegram handle can mint flawless, localized phishing lures, the old “look for bad grammar” user-training tip is becoming more and more obsolete.

AI & IT Infrastructure Attack Vectors

Application-layer exploits are only half the story. Last week, Palisade Research’s red-teamers showed OpenAI’s o3 model disabling its own shutdown routine. If a model can jailbreak itself in the lab, the scaffolding around it is living on borrowed time. In its threat report published in February, OpenAI had reported that it had begun purging ChatGPT accounts tied to Chinese, Iranian and North-Korean state actors after discovering they used the service to debug spyware and craft influence campaigns.

As adversarial campaigns become more sophisticated it is likely it will extend into traditional network and endpoint infrastructure as well. In December of last year, Palo Alto Networks’ Unit 42 had published research that LLMs excelled at transforming existing malicious code into natural-looking, evasive variants, especially in JavaScript. Over time, these transformations can degrade the accuracy of malware classifiers, subtly training them to misidentify threats as benign. Damien Lewke had a great post on how to effectively seek these out by applying behavior, network, automation, artifact and entropy-based hunts, matched with prevention controls.

Conclusion

Adversaries have moved from experimenting with AI to operationalizing it at industrial scale. AI-enabled attacks have moved from one-off proofs of concept to systemic campaigns that breach developer pipelines, con executives with deep-fake voices, and mass-produce phishing kits. Simultaneously, adversaries are trying to find gaps in the AI and IT infrastructure plumbing itself, proving that defenses must extend past applications and down into the platforms that serve them. More than ever these styles of attacks are becoming embedded, pervasive and go beyond code to quite lucrative “as-a-service” businesses. In such a dynamically changing infrastructure and application environment, I am certain we will see even more varieties and styles of attacks. As always if you are building in AI and cybersecurity, please reach out to me!

Dmitri Alperovitch, xCTO and Co-founder of Crowdstrike, speaking at Decibel’s Founder Dinner at RSAC 2025

The World Has Changed, And So Has Cyber

Earlier this year, we co-hosted the National Cyber Innovation Summit in Washington, D.C. with our friends at JP Morgan. It brought together 100+ elite founders, government officials, and security leaders from both the public and private sector. The message was clear: cyber is no longer a commercial afterthought or even a siloed IT concern. It’s geopolitical, it’s economic, and it’s existential.

Cybersecurity has always been deeply shaped by geopolitics. From the APT1 report in 2013 where Kevin Mandia publicly pointed out Chinese nation-state attackers, to the Colonial Pipeline ransomware attack that paralyzed the east coast in 2021, we’ve spent the last two decades reacting to threats born on the international stage. Nation-state attribution used to be controversial — now it’s table stakes. After each geopolitical shock, defenders have scrambled to bolt on controls for mitigation. The structural shift of AI makes that retrospective model untenable. Public LLMs and cheap fine-tuning allow even hobbyists to generate convincing malware and bespoke phishing at will, so threat volume is exploding non-linearly and signature-centric defenses are drowning. The net result is simple: defenders can no longer wait for “known bad” intelligence. Systems must be attacked in simulation, continuously ideally and reinforced in near real time. We’re entering a new chapter on what we call the Proudly Offensive Era.

Offense Is the New Defense

In the past, we operated on a Sequential Kill Chain — an attacker would recon, gain access, escalate, exfiltrate. We responded after the fact. We built playbooks around alerts and symptoms. We were always one step behind and our defenses were built on yesterday’s attacks.

When AI first came onto the scene, we saw a huge opportunity to focus on reducing alert fatigue from security tools. Companies like Dropzone emerged to handle the signal overload in SOCs by automating triage — not by replacing humans, but by relieving them of the tedious, the repetitive, the soul-crushingly obvious.

In this next chapter, both our allies and our adversaries are teaching LLMs to think like attackers. At the same time, the rise of code-gen tools like Bolt and Cursor means that software is being written — and exposed — at unprecedented scale. The result is a multiverse of exploitable vulnerabilities. And in the context of rising global conflict, the implications could be quite serious.

Today, we’re looking to invest in companies that proactively simulate the kill chain before an attack occurs. We expect these companies will use autonomous agents, draw on human expertise, and scale their reach through AI.

RSAC 2025: Proudly Offensive In Action

At RSAC this year, we took this idea public. The Decibel Founder Oasis hosted a packed session titled “Proudly Offensive”. It featured live demos and candid conversations on the topic. Here are three of the teams leading this charge:

🧠 Delphos – Teaching Machines to Reverse Engineer Malware

David Dubick and Caleb Fenton, founders of Delphos

Delphos is pioneering AI-powered reverse engineering, letting defenders deconstruct and see malicious code faster than ever before. Traditional sandboxes rely on behavior; Delphos pushes further, building code semantic models that can detect known and unknown vulnerabilities. The product thereby gives defenders the upper hand against attackers by doing what an elite reverse engineer would do, but in mere seconds.

👁️ SpecterOps – Turning Identity Into an Attack Surface Map

Justin Kohler, Chief Product Officer of SpecterOps

SpecterOps has long been a leader in offensive identity research. At RSA, they showed how their tools map attack paths via Bloodhound — then simulate exploitation. The approach is grounded in real-world tradecraft, helping customers such as Palantir and OpenAI harden their defenses with evidence, not hypotheticals.

☠️ Dreadnode – Simulate, Exploit, Repeat

Will Pearce and Nick Landers, founders of Dreadnode

Dreadnode showed why advancing the state of offensive security through AI isn’t the future but rather the present. Their AI-powered agents simulate attackers with intent: they probe, exploit, exfil, and learn. Dreadnode is building the most advanced adversary that lives inside your test environment and evolves. The best way to secure a system isn't by scanning and monitoring, it best secured by trying to break it, repeatedly.

Continuous Adversarial Validation

We’re seeing adversaries use LLMs to write better phishing campaigns, to craft tailored payloads, to fuzz APIs at scale. In our minds, the antidote is AI-powered Continuous Adversarial Validation, the idea that defense can no longer be episodic. We don’t wait for a red team quarterly or once a year. We spin up a red team every hour.

This approach blends simulation with detection and remediation. Rather than simply hardening endpoints, it war-games against a synthetic enemy that may understand your systems better than you do. As these tools mature, the boundary between defense and offense will blur even further. Put differently, reaction collapses into anticipation: every hour, new attack graphs are explored, and exploitable paths are patched or segmented before production traffic carries real risk

If You’re Building in This Space

If you’re working on AI-powered simulation, adversarial agents, or any other tools that turn offense into insight — we want to meet you. At Decibel, we are keen to invest into a new kind of offense and are proudly backing the companies getting us there.

Decibel and JP Morgan Co-hosting the National Cyber Innovation Summit in DC in March 2025

Not long ago, I wrote about a new wave of Programmable Defense platforms, and how they fit within the broader evolution of cybersecurity. At the time, it felt like a logical progression: We’d seen the slow but steady rise in both sophistication and volume of attacks—and the corresponding call for more adaptive, security engineering-friendly security products. Now, with the advent of LLMs, that progression has taken a major leap forward and is driving cybersecurity into a new era: the Autonomous Defense.

The Autonomous Defense is a collaborative paradigm in cybersecurity where human creativity and AI work together to detect, contextualize, and respond to threats across entire kill chains in real time. It represents the evolution from rigid, static black-box security solutions to adaptive, programmable systems that seamlessly integrate human oversight with AI-driven automation.

The need for Autonomous Defense is a response to more than just an accelerating volume of attacks; it’s also addressing threat actors’ evolution in style, creativity, and deception.

McKinsey’s data on phishing alone highlights the surge in AI-enabled exploits: more intricate social engineering, more convincing synthetics (think deepfake audio and video), and a flood of new email tactics that bypass traditional filters. In one well-known example, threat actors leveraged deepfakes to pose as a company’s chief financial officer and convinced a finance worker to pay out $25M.

And let’s be clear: It’s not just the malicious side harnessing these AI engines. Security teams are actively embedding AI into their workflows—raising the bar for rapid detection, real-time contextualization, and automated responses.

What does all of this mean? If you talk to the folks who came up through the ranks of BlackHat and DEFCON, you’ll see a common theme: security professionals are done waiting for vendors to patch together stale, rigid solutions. They’re demanding tools that can adapt as fast as the threats do–and do so in a way that maximizes both security outcomes and operational efficiency. The Autonomous Defense is responding to that call, empowering teams to reduce the cost and time of defending against increasingly sophisticated attacks, while freeing up human defenders to focus on the highest-value challenges.

Let’s break down the foundational elements of the Autonomous Defense and how they coalesce into this new paradigm:

Programmable: Where AI and Humans Collaborate

Programmable security has long been part of our thesis at Decibel. It represents the critical fusion of flexible toolsets that can adapt as attacks evolve—and, more importantly, do so seamlessly with human oversight. A big part of making that happen lies in how systems, humans, and AI “talk” to each other.

A lot of security products historically hid behind black-box algorithms. If it flagged something, good luck debugging the “why.” Now, we’re seeing a shift to detection tools that expose a domain-specific language (DSL). Think of Sublime Security’s approach, where everything from rule creation to threat hunting can be expressed in a clear, programmable format. It’s a shared language between you and your AI, reducing confusion and “hallucinations” that can happen when an AI tries to guess your intent without a well-defined structure.

This synergy is the same reason code-generation platforms like Cursor and Codeium have been so effective: humans design the logic and context, AI executes the heavy lifting. The result is faster, more accurate rule-writing and detection logic that can be tuned on the fly. We often forget that many cyber risks hinge on “time to detection.” If your detection rules can be automatically refined and iterated within minutes—not weeks—you effectively take away one of the biggest advantages attackers have.

Contextualization: Enriching without reducing signal

Silos are an unfortunate byproduct of the rapid technology sprawl in security. We have specialized tools for threat intelligence, incident response, vulnerability management, threat hunting, endpoint monitoring—the list goes on. But the reality is that kill chains don’t respect product boundaries. AI-driven attacks can pivot quickly from a phishing exploit to privilege escalation on your network, and from there, to data exfiltration in your cloud environment.

To truly defend in real time, we need a holistic vantage point. That’s where contextualization comes in. Imagine if your vulnerability management system and threat intelligence feeds were fully merged. Instead of receiving an alert saying, “Remote code execution vulnerability in product X,” you’d also get context: “APT Group Y has been exploiting this exact vulnerability in the last week, targeting financial institutions similar to yours.”

Now you don’t just know there’s a problem—you understand the likelihood and potential impact. You prioritize a patch. You escalate within your team, because you see the real risk of lateral movement in your environment. It’s basically the difference between an alert and a narrative. And that narrative is what teams need to respond swiftly and effectively.

One big step in that direction is the collapse of multiple security categories into a single, integrated chain. If threat intel informs vulnerability management, which directly ties to patching or exploit-blocking, that reduces both manual handoffs and the risk of something slipping through the cracks. AI models trained on your specific environment can automatically flag the vulnerabilities most likely to be exploited in real time, driving meaningful risk-based prioritization rather than generic best practices.

For instance, Empirical Security is leveraging OSINT and attack telemetry, mining its own proprietary data models and EPSS to build more advanced, localized models. Rather than depending on generic indicators, it aggregates and analyzes data from a worldwide network of sources to stay ahead of emerging threats. This shifts the approach of security teams from a reactive to a predictive and preventive mindset. By combining EPSS-based predictions with their contextual threat analysis engine, it can identify high-probability weaknesses specific to an organization’s profile, industry, and infrastructure. This holistic, intelligence-driven approach transforms raw alerts into actionable narratives, helping security teams not only detect issues faster but also understand the “why” and “how” of each threat—empowering them to prioritize and remediate with far greater precision.

Digital Twins: Real-Time Threat and Defense Simulations

We all know the classic tabletop exercises and third-party pen tests that happen every quarter (or year, if you’re unlucky). They’re valuable, but also resource-intensive and slow. You’re basically simulating known attacks, capturing the results, then hoping to find and fix the gaps before the adversaries do.

Enter digital twins (DTs). Already commonly used in industries like aerospace and manufacturing, digital twins help organizations model complex systems like jet engines or assembly lines, enabling them to simulate performance, predict failures, and optimize operations in real time by testing changes in a virtual environment before implementing them in the real world. In the context of cybersecurity, a digital twin is a virtual model of your organization’s infrastructure—networks, applications, endpoints, and even the workflows your security teams use. By simulating attacks in this environment, you can stress-test new defensive strategies without risking actual systems.

Digital Twins can point out the transformative potential for defenders. Researchers describe how Cyber Digital Twins (CDTs) focus specifically on cybersecurity functions, letting security teams simulate attacks, evaluate patch effectiveness, and predict the downstream impacts of compromises—all without real-world disruption. They’re like living blueprints that track real-time changes, enabling you to validate configurations, hunt for zero-days, and quickly pivot to new defensive measures before the threat actors can adapt.

To me, digital twins represent the next logical step in cybersecurity resilience. Because once you can replicate your entire security environment in code, you can unleash AI to run hundreds—maybe thousands—of “what if” scenarios in parallel. This advanced simulation closes the gap between detection, analysis, and remediation.

AI-Powered Workflows: Driving Down Toil

Even in 2025, so much of cybersecurity still involves repetitive, manual tasks. Security teams are inundated with alerts, logs, tickets—some of which are critical, many of which are false positives or routine. This grind leads to fatigue, burnout, and inevitably, missed signals.

Automation is nothing new, but the level of workflow optimization we can now achieve with AI is next-level. Products like Dropzone reduce toil by automatically categorizing and responding to routine alerts, letting defenders focus on the nuanced tasks that require human judgment. That’s where we want the human defenders—applying their experience, creativity, and gut instinct to tricky edge cases, advanced threat hunting, or major incidents that need strategic input.

There’s a virtuous cycle at play: The more tasks you offload to a well-tuned AI system, the more time you free up for the human side of the collaboration. Your team can invest energy in improving detection logic or analyzing novel threats.

Conclusion

The Autonomous Defense is a convergence of principles that have been brewing for years: programmability for adaptive detection rule-making, contextualization that breaks down data silos, real-time simulation via digital twins, and AI-driven workflow optimization that frees up human defenders to to focus on high-impact work while lowering operational costs and improving the ROI on security.

Not every “autonomous defense” product of the future needs to include every single feature in this list. But these concepts—programmable security, integrated intel, digital twins, and streamlined workflows—serve as a guidepost for what’s possible right now. AI has changed the game for attackers, but it’s also leveling up our defenses in ways we couldn’t have imagined a decade ago.

If you’re building (or dreaming of building) something that fits these principles, I’d love to hear from you. Because I believe they provide a compelling path forward to embrace this era of collaborative AI-human defense—and help shape the tools that define it.

In many ways, the folks who cut their teeth at BlackHat and DEFCON did us a favor by demanding more flexible, programmable solutions. Now that they’re in leadership roles, it’s time for us to answer the call and deliver.

Big thank you to Josh Devon (Flashpoint), Damien Lewke (Arctic Wolf) and Ed Bellis (Kenna Security) for providing massively insightful feedback on this piece and helping me refine my thoughts on this topic.

We recently hosted the inaugural Programmable Defense Summit in New York City, bringing together some of the most forward-thinking cybersecurity leaders and founders. The discussions centered on how the security industry is shifting toward a more transparent, adaptable, and customer-centric, rather than vendor-centric, security posture.

This blog offers a quick recap of both the summit and the investment thesis that inspired it, along with key trends shaping the future of programmable defense. Throughout, we’ll share some of the most thought-provoking quotes from the day—anonymized but too good not to highlight.

A special thank you to the incredible CISOs and security leaders in attendance, including those from Prudential, BNY Mellon, Cribl, Ro, Vanta, Workato, Crowdstrike, Maven Clinic, FanDuel, and Dropbox whose insights helped make this such a dynamic and impactful event.

Security Secular Trends

"Programmable defense puts power back in the hands of security engineers—moving fast is no longer a luxury but a necessity."

The security landscape is shifting rapidly, driven by both technological advances and evolving threat vectors:

1. AI-Enhanced Threat Actors:
   Threat actors are leveraging AI to scale their operations. Tools like ChatGPT make it possible for attackers to generate convincing phishing emails or automate reconnaissance. The barrier to entry for orchestrating complex cyberattacks has plummeted, resulting in a significant uptick in both volume and variety of threats.

2. A New Generation of Security Leaders:
   Today’s security teams have grown up in a world shaped by DEFCON and Black Hat conferences. These leaders are now reaching the C-suite, bringing with them expectations of in-house security engineering expertise. Unlike in previous eras, relying security purely on product vendors is no longer the default—security leadership expects its own sophisticated operations to be able to collaborate with the vendor in the the product.

3. Collaborative Defense Communities:
   Security professionals are organizing on platforms like X and Mastodon to share intelligence and counter common threats. Much like the 1% of social media users who create content consumed by the 99%, a small but growing group of defenders is publishing actionable insights for broader community benefit.

From Click-Ops to Programmable Security

"The black box might scale, but it doesn’t solve enough problems. The future is transparent and programmable."

Security tooling is evolving, and a fundamental shift is underway:

• The Demand for Transparency:
  Black-box solutions are increasingly seen as insufficient for modern security challenges. Instead, forward-thinking organizations expect their tools to be programmable, consumable via APIs, and tailored to their unique environments and needs of an organization.

• Parallels with DevOps:
  Just as open-source software and agile DevOps tooling empowered developers, programmable security will unleash creativity and innovation within security teams. Leaders envision a future where security engineers can adapt tools in real time, fostering collaboration and sharing knowledge across organizations.

Programmable Defense: A Foundation for the Autonomous Defense

"LLMs are the bridge—making security decisions understandable to executives and actionable for engineers."

• The Autonomous Defense: Programmable defense is the cornerstone of a future autonomous cybersecurity ecosystem, as effective AI requires vast amounts of high-quality first-party data. By enabling security experts to contribute directly to the system, programmable defense becomes a powerful, community-driven engine for defense—one capable of evolving in a rapidly changing adversarial environment. However, implementing this strategy today is not without challenges. It demands deep technical expertise, including an understanding of code and detection engineering, skills that not all security teams possess. This gap often limits organizations’ ability to operationalize programmable tools effectively.

Sublime Security, utilizes BERT LLM, enhances Sublime's Natural Language Understanding and brings improved contextual awareness and understanding, language comprehension, and performance to better identify GenAI in phishing attacks.

• Enter AI: LLMs democratize access to sophisticated defense strategies by translating complex, technical threat intelligence into actionable steps that engineers can implement and executives can understand. This collaboration between AI and humans is a critical step toward autonomous defense—where systems can dynamically adapt to threats in real time, powered by first-party data collected from real attacks and responses from a community fighting a common enemy.

Programmable defense is as much about culture as technology, relying on transparency, collaboration, and shared knowledge. AI makes this vision achievable, enabling adaptive, ever-improving cybersecurity systems for the future.

Looking Ahead

The discussions at the Programmable Defense Summit confirmed what many of us already believed: In order to build a truly dynamic autonomous defense that can keep pace with the high variety and volume of attacks of the future, cybersecurity products first have to become more programmable, transparent, and collaborative. While we’re still early in this journey, the momentum is exciting.

To those who attended and contributed to this pivotal conversation—thank you. If you are interested in joining us for the next event, please reach out!

Intro

In the 115 degree heat an impatient but hilarious cab driver asked me if I was also here for the “Hack Rats” Conference and yes, indeed, I was. The only thing hotter than the weather in the lovely August sun in Las Vegas were some of the topics covered at this year’s biggest hacker conference. Over the past few days, the Decibel team and I had a chance to catch up with founders, CISOs and security practitioners at our usual Founder Oasis event. We discussed the latest trends and topics in cybersecurity, and I wanted to summarize the key takeaways here: 

The Decibel Founder Oasis in full swing, a place for founders to take customer meetings and exchange ideas. 

I Spy with My Little AI 

The growing excitement around AI Security and the Security of AI is intensifying, especially as AI tools empower criminals to scale their operations more rapidly. These tools are increasingly being used to socially engineer the most vulnerable layer of security—humans—manipulating them to bypass security controls. For example, attackers might pose as high-level executives to pressure a finance department employee into wiring funds to a fraudulent account by creating a sense of urgency. Similarly, they might manipulate a help desk into disabling MFA for an executive. This highlights the critical importance of helping humans make better security decisions—a key topic of discussion.

As AI becomes more sophisticated, so do the threats it faces and poses. Without continuous research and innovation, we risk falling behind in understanding and mitigating these emerging risks. It is essential to stay ahead of the curve, ensuring that AI systems are not only effective but also resilient against exploitation by malicious actors.

Therefore, in collaboration with our friends at SpecterOps and Dreadnode, we were thrilled to launch our first-ever “Man vs. Machine” competition at Black Hat. This open challenge gave over 100 security researchers the opportunity to test their skills by hacking AI models in real-world simulations. Participants tackled 12 different “capture the flag” (CTF) exercises, generating more than 2 million API requests against widely used LLMs. While more than 50 researchers successfully completed the first challenge, only 3 managed to conquer all 12. We hope that competitions like these will inspire researchers in our community to develop protective measures and guardrails for today’s AI models.

Kenneth Yeung was crowned the winner of the event and was awarded an NVIDIA RTX-4090 by our honorary guest HD Moore (founder of Metasploit and runZero). 

The Programmable Defense: The Crowd Strikes Back

At our annual Black Hat Decibel Founder Happy hour, the July CrowdStrike IT outage was a major topic of discussion. Huge kudos to Crowdstrike founder George Kurtz for showing up to the Innovators and Investors Summit where he sent all of our founders a message that you need to show up when times are tough. There were many lessons learned and open questions on how the industry moves forward and many in our community are working towards a more resilient solution: a Programmable Defense which allows vendors, researchers, and early adopters to collaborate through open security solutions. Our portfolio companies Sublime Security, SpecterOps, Prowler, Push Security, and our friends at Panther continue to lead the way in this very important movement. We expect an even larger group next year - the crowd will definitely strike back! 

Our Decibel Founder Happy Hour at Libertine Social 

Will.”IAM” - Where is the love?

Identity and Access Management (IAM) and Identity Security (IS) took center stage at Black Hat 2024, underscoring its critical role in modern cybersecurity. As the cloud era continues to evolve, IAM and IS has become more than just a security necessity; it's now a frontline defense against increasingly sophisticated credential compromises and identity-related attacks. This year's conference highlighted how security leaders are realigning their IAM and IS strategies to meet the demands of this new landscape, recognizing that effective IAM and IS is vital for protecting sensitive data and ensuring only authorized access to critical systems.

Packed house at the Push Security, Sublime and Panther Happy Hour!

Our portfolio company Push Security –which focuses on identity attack detection and response – held a detailed webinar on the topic, in particular covering the massive recent spike of AitM (Adversary-in-the-Middle) attacks. The growing threat of advanced phishing tools like Evilginx, EvilnoVNC, Muraena, and Modlishka, which attackers use to bypass traditional security measures such as multi-factor authentication (MFA). These tools enable attackers to steal live session logins, making it critical for organizations to adopt robust detection strategies. One way to protect against this is to leverage browser telemetry to identify and block these phishing toolkits.

Conclusion

Attending Black Hat in Las Vegas this year was, as always, an incredible experience, filled with insightful discussions, innovative ideas, and the chance to connect with some of the brightest minds in cybersecurity. While it’s great to be back in the cooler climate of San Francisco, the excitement and energy from the conference are still fresh in our minds. We're already looking forward to next year, eager to reunite with founders, CISOs, and practitioners to continue exploring the cutting edge of security. As always, if you are thinking about or building in security, AI or infra IT space please reach out to me or join our Founder Catalyst community! 

Quick Recap:

In last month’s Founder Catalyst edition, I introduced the concept of the Managed Service-as-Software startup where AI-driven service-oriented startups build their companies according to a new business model blueprint. This requires a fundamental shift in mindset for startups to use AI, rather than to sell AI. Initially labor-intensive with low gross margins, these startups gradually shift to higher SaaS-like gross margins through automation and AI. As described in my previous post, this model had already been proven before in the managed cybersecurity space where companies like Expel and Arctic Wolf started as lower margin managed security service providers and evolved into higher margin software based security platforms using automation.

In that article, I tried to emphasize the need for an initial labor-first approach coupled with robust AI tooling, and operational monitoring to achieve SaaS-like efficiency and profitability while providing high quality service delivery that is human augmented. This follow up piece dives into the financial and valuation implications of this shift, specifically focusing on AI-Operating Margin Gain and AI-Margin Expansion.

AI-Operating Margin Gain

AI-Operating Margin Gain refers to the gap between consulting gross margins and software gross margins, that is exploitable using AI, agents and automation. Traditional consulting businesses typically operate with gross margins in the mid-30% range due to their labor-intensive nature since the service is delivered via consultants. In contrast, software businesses often achieve gross margins around ~70% because of their lower delivery costs, usually COGS in the form of cloud hosting costs. This significant disparity makes consulting companies operationally less attractive. AI-driven M-SaS startups have the potential to bridge this gap by gradually automating their services, thus reducing labor costs and increasing gross margins over time. By leveraging robust AI tools and maintaining operational intelligence, these startups can achieve SaaS-like efficiency and profitability, ultimately driving higher valuations.

AI-Margin Expansion

AI-Margin Expansion (AI-ME) is defined by the gap in valuation multiples between consulting businesses and software businesses that can is exploitable using AI, agents and automation. Consulting companies typically trade at enterprise value to revenue (EV/Rev) multiples of 2-3x, while software companies often achieve multiples of 8-12x, and sometimes as high as 25-30x during bull markets. This difference is primarily due to the scalability and high margins of software businesses, which public market investors find more appealing. By transforming into M-SaS businesses, AI-driven startups can position themselves to capture these higher multiples while starting with a more labor intensive approach in the beginning. This involves not only delivering software but also providing a comprehensive service, thereby raising their gross margin profile and attracting better valuations long term. The old venture capital adage of “don’t do services” is maybe from a bygone time, where the gap between software and consulting businesses was significantly larger, like 10-15x, than it is currently, like 4-5x. For the best performing hybrid software / services companies like Palantir for example the multiple is almost the same or even better than any other software business. With the advancements of AI, the convergence between software and services will likely speed up even more. 

For Managed-Service-as-Software Startups

For startups looking to build Managed Service-as-Software businesses, the implications of AI-OMG and AI-ME might be profound. Starting with low or even negative gross margins is acceptable due to initial investments in labor and compute. However, the journey from 0% to 70% gross margins must be meticulously managed with both a focus on operational intelligence and patience as the process can span several years and the business matures. This fundamental shift from a traditional software business model to an M-SaS model requires a holistic approach to managing engineering, product, sales, marketing, and operations. By offsetting labor costs with GPU costs and leveraging AI to enhance service delivery, startups can significantly improve their margins and achieve software valuations.

As always, if you are thinking about or building in this space please reach out to me or join our Founder Catalyst community!

In November of last year, I first wrote about the new paradigm of AI agents becoming the fundamental technology driving a new era of software: Services-as-Software, where startups provide service-oriented, outcome-driven solutions to their customers using AI agent technology. With this change in how startups would deliver value came many new potential business considerations such as pricing which I discussed in detail here. 

One interesting aspect that is still overlooked in this new paradigm, however, is the potential to build according to a very different business model blueprint. If we quickly review the history of how software business models functioned pre-AI:

On-Premise: As software was installed on-premise at the customer site, software was sold in the form of ELAs (Enterprise License Agreements) with an attached support contract. The ELA was essentially a large dollar amount that allowed customers to use the software into perpetuity while the maintenance/support contract was annually recurring in nature. Traditionally, the breakdown between the perpetual license and support was a 90% / 10% split. For example, a VMware vSphere license, illustratively, would be $900K with a $100K support contract, for a total contract size (TCV) of $1M. The customer could use that version of the VMware license forever, but of course, every 2-3 years there would be a refresh cycle which would allow VMware to charge for their latest version. Since a service would be associated with maintenance, only 10% of the ELA would be annually recurring. 

Cloud: In the era of cloud, we were able to deliver software directly to the customer without coming on site for installation. Since everything ran and still runs through the cloud, the way we charge for software also changed to a fully recurring model. No longer did we split up the perpetual license against an ongoing support license. Support and license became one, charged annually, and thus SaaS was born. Many new SaaS companies emerged as incumbents struggled to keep up, often due to the challenges of replatforming to the cloud and their entrenched ways of selling software. 

With every new technology development comes a new business model disruption. With AI at our fingertips, a new path has been forged for startups to consider when building the next generation of companies, which I call Managed Services as Software. 

Consulting and Software: An “It’s Complicated” Love Story

“Managed” typically implies a human labor component for service delivery, and this is no different. Many large product companies began as consulting businesses, gradually shifting as they achieved product-market-fit with a particular product they initially provided through white-glove service for clients before transitioning to SaaS. FreshBooks, Mailchimp, and UIPath are just a few examples of giants that made that transition. Turning a service business into a SaaS business is now a textbook strategy, provided you can find the right product to build your software company around. 

However, a few companies have taken a different approach, remaining in a traditionally labor-intensive, consulting-led services world, but using technology to drive better margins over time. Expel and Arctic Wolf are both cybersecurity players that have embraced a software-enabled service business model. In a way, both have effectively outsourced the role of Security Operations Center (SOC) and delivered it back to customers through a managed experience, known in the security world as a managed detection and response (MDR) service. 

Internally, however, they have continued to innovate and automate their operations to drive towards SaaS-like margins. Through rigorous operations and automation, these businesses, traditionally with 20-30% gross margins due to high labor costs, have willed themselves to 70-80% gross margins over 8-12 years. The MDR product space became so lucrative that even product companies like CrowdStrike, which originally focused on endpoint agent technology, entered the market with their own MDR offering, CrowdStrike Falcon Complete. 

Sell AI or use AI yourself? Introducing the Managed-Service-as-Software Startup

Philosophically, the biggest change is that instead of trying to sell AI to customers, startups should think about delivering value by using the AI they built themselves. You might ask, “Hey, doesn't that make me a consulting business?” – and the answer is partially yes. M-SaS businesses are AI powered services businesses that over time drive from low (20-30%) gross margins, while labor intensive, to high / SaaS-like (70-80%) gross margins, when AI intensive. The service delivered remains unchanged to the customer throughout the journey.

Following the principles of the Managed Detection and Response market, there is an opportunity for the new generation of startups to mimic the operational prowess of Expel and Arctic Wolf. However, in this time frame these new companies can use their era-appropriate technology of LLMs, agents, and GPUs instead of business logic software, static automation, and CPUs. The net result could mean that accomplishing 70% gross margins as is typical with software companies could be achieved in maybe less than half the time than it took the previous generation, all pending the cost of GPU declining at a faster rate as the cost of labor (which is really only increasing). 

Starting with a labor-intensive service delivery model requires upfront investment in both automation scaffolding for scale, as well as a competent human service desk. In a way, a M-SaS company never has to pass the Turing test because in its initial stage of service delivery, it is largely human. The goal of the engineering team would be to supercharge the human service desk employee and to enable them to service an increasing amount of customers over time, leveraging automation & AI tooling. On the other side, the customer experiences consistent service without necessarily knowing or caring if AI is used in the process. The graphic above demonstrates this (with completely illustrative timeline and figures): initially, each service operations employee can service a small number of customers, but over time, this scales up significantly to ten and then hundreds of customers. Since labor required per customer is decreasing while automation & AI is increasing (assuming falling compute GPU costs), gross margins would improve. The following summarizes the basic building principles for a M-SaS business. 

1. Labor-First Approach: Initially, focus on delivering an excellent client experience through a labor-intensive model. Although gross margins will suffer initially, this sets the stage for long-term success.

2. Build AI Tooling and Automation: Develop AI tools and automation to make your operations & service delivery desk more efficient, aiming to serve more clients per unit of labor over time. This will gradually reduce the reliance on labor and increase efficiency.

3. Increase GPU Utilization: As technology and automation improve, the ratio of labor per client will decrease, allowing for higher margins and better service delivery.

4. Operational Monitoring: Maintain rigorous operational monitoring to ensure uptime and efficiency. High inference costs from GPUs may initially be higher than labor costs, but these are expected to decrease over time, justifying the investment long term.

What this means in practice for M-SaS businesses is that it is acceptable to start with low or even negative gross margins due to upfront investments in both labor and compute costs, especially when customer count is low or non-existent at the beginning. However, the path from 0% to 70% gross margins should be closely monitored, and M-SaS founders should maintain full operational intelligence at every step. This requires a fundamental mindset shift from the typical software business model. You are not just delivering software but a full-fledged service, which involves managing not only engineering, product, sales, and marketing, but in particular operations.

Conclusion

The market is increasingly willing to pay for outcomes rather than traditional software tooling. Whether your AI tools are used internally to drive efficiency or sold to external customers, the key is their usage. Modern startups can potentially do both, making the question more about sequencing rather than where you start and end up. Leveraging the new paradigm of AI and decreasing GPU costs should eventually lead to the creation of more M-SaS driven companies. These companies will transform from labor-intensive operations to technology-enhanced services with SaaS-like margins and, consequently, SaaS-like valuations. 

As always, if you are interested or working on an idea on this topic please reach out to me or join our Founder Catalyst Community!

Special thank you to Amil Naik who helped me clarify my thoughts and pull this piece together.

At Decibel, we have always believed in the power of serendipity. By bringing the right people together, we can create an environment where magic happens. 

A few months ago we started a series of small dinners with both prospective and repeat founders, thinking through new ideas and markets, discussing topics such as the programmable cyber defense or Service-as-Software via AI agents as well as more strategic items like pricing in the AI world.

We named these gatherings Founder Catalyst because our vision for starting a great company involves a few crucial elements. These catalysts can take many forms:

• Meeting the Right Co-Founder

• Discovering the Right Idea

• Receiving Key Insights from Customers

• Securing Capital

Often, the magic lies in the combination of all these elements. Our dinners have sparked lively conversations and even birthed a few company ideas. Today, we're collaborating with several Founder Catalyst members to incubate a variety of companies that originated from these get togethers.

Over the past few months, many of you have asked for a virtual space to continue building relationships and exchanging ideas. In response, we're excited to announce the launch of the Founder Catalyst Community on Slack!

If you are a product, sales, or engineering leader with aspirations of starting a company one day in the field of technical software, this community is for you.

Sign up here (or ping me) to become part of this community which includes product and engineering leaders from Zoom, Duo Security, Rubrik, Meta, Harness, Atlassian and Google among many others and attend our upcoming founder events!

Hackers are just like us: they don't really want to do the work. With the introduction of LLMs into our lives, it became immediately clear that there was a huge gain in labor productivity to be had for all knowledge workers. In the arena of cybersecurity, this has ushered in a new era in an industry where there is already a massive labor shortage to begin with. Of course, the opponent on the other side of the field (or rather web) is not sitting idle. Threat actors have already started to experiment with AI to supplement or enhance their own attack campaigns. And so, much like the ease at which we can now generate essays or complex documents using tools like ChatGPT, the barrier to entry for hackers orchestrating cyber attacks has plummeted. This has led to a significant uptick in both the variety and volume of cyber threats. This phenomenon is already evident today in areas such as phishing, which is up 967% since Q4 of 2022, or endpoint security, where a ChatGPT-based Polymorphic code was able to evade EDR systems. As we enter this new AI-fueled era where hackers are utilizing these new tools, security engineering, a discipline that focuses on detecting malicious activities or unauthorized behaviors, will become more important than ever. 

Modern security organizations have invested heavily into building out personnel and tooling in that department. A member of the Decibel early adopter community –a global banking institution – stated that they were analyzing over 1 petabyte of security data per day with over 1000 security engineers working in the organization. For organizations with limited budgets for direct investment in security engineering, vendors such as Arctic Wolf and Expel have provided assistance. The commercial success of these companies, both having valuations of $4B and $1B respectively, underscores a trend: customers are increasingly seeking security engineering solutions, whether in-house or through outsourcing.  

As the enemy upgrades its arsenal with a broader range and variety of AI-based attacks, it becomes evident that security engineers will need new tools to counter these threats effectively. The traditional reliance on vendors for updates and patches is becoming increasingly untenable in this rapidly evolving landscape. The burgeoning field of security engineering had embodied this shift towards adaptability and self-reliance long before the AI hype cycle, but against the plethora of attacks GenAI might unleash, security teams are craving control, transparency, and explainability over their defensive measures more than ever.

In the following, I wanted to explore a few of the product principles that might be incorporated into the next generation of security tooling and how they might spell the end of Black Box type products as we know them. 

Vendor Bottleneck vs. Adaptability, Transparency & Explainability

It is unsurprising that the sheer volume and variety of email attacks has drastically increased with the adoption of LLMs by hackers. This in turn has prompted email detection teams to take a more proactive stance to keep their inboxes safe. In a vendor-dominant world, customers might find themselves queueing up to get critical security issues addressed, especially if they're not deemed 'important' customers. 

Custom detections by Sublime Security 

Sublime Security has emerged with an alternative approach, offering a unique solution to email security. By allowing detection teams to write, run, and share custom detection rules to threat-hunt phishing attacks, Sublime enables a level of customization and transparency that empowers defenders to respond swiftly to phishing attacks. This open platform approach not only facilitates rapid adaptation to newly observed phishing techniques but also fosters collaboration across organizations, effectively democratizing the fight against email-originated threats. The more detections are contributed by Sublime’s users the more effective the system becomes.

Us Against Them: Community vs. Vendor System

The battle against cyber threats—be they from nation-states or hacker groups—is more effectively fought as a collective rather than in isolation. Security practitioners have long congregated in digital forums like Slack channels, Twitter, and HackerNews, sharing insights and strategies, but implementing them uniformly has always been a challenge. As Mike Schwartz, former Cybersecurity leader at AWS and Target, pointed out to me, “the implementation of this shared knowledge often hits a wall at the product level because everyone has different vendor tooling and thereby can’t be implemented in a prescribed way”. 

Community Rules by Semgrep 

The Semgrep community exemplifies this proactive stance, swiftly sharing code fixes in response to new Common Vulnerabilities and Exposures (CVEs), underscoring the importance of agility and collaboration in modern cybersecurity efforts. This highlights the need for a community-first approach, where enterprises can unite, speak a common language, and implement solutions directly, without being hampered by vendor bottlenecks. Just based on the sheer volume of vulnerabilities that exist today, this is needed more than ever. 

One Size Fits Most vs. Control & Customization

The belief that a one-size-fits-all security solution is effective is becoming increasingly impractical. Modern IT ecosystems, characterized by their complex cloud infrastructures, containers, and continuously evolving attack surfaces, demand a security approach that goes beyond the limitations of traditional, generic solutions. 

Custom checks & dashboards by Prowler 

Within this context, Prowler emerges with an interesting solution, specifically engineered for AWS environments. It distinguishes itself by offering a suite of tools designed for comprehensive security assessments and audits, enabling organizations to conduct in-depth analyses of their own AWS configurations against best practices and compliance standards. By allowing users to customize checks and adapt security configurations to their unique operational environments, Prowler facilitates a more open and granular approach to cloud security. This capability is essential for effectively managing the intricate requirements of today's IT infrastructures, ensuring that security measures are as nuanced and dynamic as the ecosystems they protect. 

Conclusion

As we stand at the cusp of a new era in cybersecurity, it's clear that the traditional vendor-reliant defense mechanisms may not suffice against the sophisticated, AI-driven offensive tactics that are emerging. The future of cybersecurity lies in transparent, programmable, and community-driven systems that can serve as first-party data for a future autonomous defense. This approach will allow our future products to dynamically adapt and respond to new forms of attacks. Tools like Semgrep, Sublime, and Prowler are only a few examples of this trend, providing platforms that not only foster community collaboration, but also enable the tailored, flexible defense mechanisms that tomorrow's security landscape will likely demand.

Speaking with several CISO and security leaders over the last few months, it is clear that Vulnerability Management remains a significant challenge for many organizations. Security teams often find themselves overwhelmed by the sheer number of vulnerabilities detected, struggling to categorize, prioritize and address them effectively. On average, 100,000 vulnerabilities remain unaddressed in an organization’s security backlog. With developers leaning more and more into AI-powered code development tools like Github Copilot, which rely on human trained data, vulnerabilities will likely continue to rise. Indeed, a study at NYU had revealed that 40% of code produced by Copilot was indeed vulnerable, while a study at Stanford showed developers believed their code was more sound when using an AI tool when the code was actually more vulnerable.

With the emergence of SCA (software composition analysis), we have vastly increased the amount of vulnerabilities we detect, but that has left us with a different problem: How do we fix them when there are so many? The reality is that while we have a relatively firm grasp of detection with tools such as Snyk, Contrast Security, and Checkmarx, we have not yet solved the human problem associated with remediation. The issue with remediation falls into 2 categories:

1. Focus and Prioritization: Using the limited developer time available to focus on what vulnerabilities need to be addressed more urgently 

2. Automating Developer & Security work: It's hard to get valuable developer time spent on security issues over product features

In order to address this, a new crop of companies have started to develop over the last few years. In the following, I wanted to highlight some of the really interesting approaches and techniques that are being developed to usher in the next era of application vulnerability management.

Pre-Production Reachability 

In the face of overwhelming vulnerabilities within an application, it's essential to discern their actual impact. Many of these vulnerabilities may reside in third-party code that isn't even executed by the app, rendering them non-exploitable. This raises an important consideration: are some vulnerabilities essentially unreachable and, therefore, less critical to address immediately? By identifying these unreachable vulnerabilities, developers and security teams can prioritize their efforts on those vulnerabilities that genuinely affect the application in its current production state. This approach is called Reachability Analysis. Although eventually, in a perfect world all vulnerabilities warrant attention, understanding where to begin can significantly streamline the remediation process and limit the precious impact to developers.

Reachability-based SCA by Endor Labs

One example of startups trying to address this is Endor Labs. Endor Labs' approach to SCA reachability is centered around the concept of analyzing the code behavior at build time to identify reachable vulnerabilities at the function level. This is achieved through a combination of static analysis of source code, manifest files, file system, and package manager data. Static call graphs are used as a precise instrument for this analysis, enabling the identification of direct and transitive dependencies, including those not declared in manifest files. This approach not only helps in prioritizing reachable risks but also provides a comprehensive view of all dependencies in use, improving the accuracy and reducing the time and cost associated with remediation. Endor Labs' method is distinguished by its ability to map vulnerabilities in open-source libraries in their real-world context, leveraging call graphs to trace relationships between software functions. This enables a more precise identification of vulnerabilities that are actually exploitable in the application's unique context.

The end result of this is self-evident. By using reachability as a core tenant to understanding priority of remediation of a particular vulnerability, security teams can be more targeted in trying to prioritize fixes within their control. Solutions like Slim.ai can then also provide a shared workspace among customers and software vendors to coordinate vulnerability fixes involving third parties. 

Runtime Instrumentation for Vulnerability Prioritization

Runtime Instrumentation, when used in the context of vulnerability prioritization, embeds monitoring capabilities directly into the application code, thus actively observing its interactions with various components like libraries and infrastructure. This method is particularly advantageous for identifying and prioritizing security vulnerabilities in real-time. By monitoring internal operations, runtime instrumentation can detect unusual patterns or behaviors that may indicate security flaws. This immediate detection allows for quicker response to potential threats, reducing the window of opportunity for exploitation.

The detailed insights provided by runtime instrumentation into the application's functioning enable a more nuanced assessment of vulnerabilities. It helps in understanding the context and potential impact of a security flaw within the application's unique environment. This context-specific information is crucial for prioritizing vulnerabilities effectively, ensuring that the most critical issues are addressed first.

One company in the space is called Oligo Security. The company’s product utilizes dynamic library-level analysis and behavior monitoring to identify vulnerabilities in real-time. This approach is particularly effective in detecting vulnerabilities in open-source libraries during runtime. By analyzing each library's behavior, Oligo creates profiles of legitimate activity and generates alerts or blocks suspicious deviations from these patterns. This targeted approach filters out about 85% of alert noise, significantly reducing cybersecurity fatigue.

Oligo Runtime-Based Vulnerability Prioritization 

Oligo's platform is based on the eBPF subsystem, a technology developed for the Linux kernel. This allows Oligo to run efficiently without compromising on performance or stability. The platform's focus is on identifying vulnerabilities based on the context of the running application, thereby helping developers and security teams focus on the actual attack surface, which is often a fraction of the vulnerabilities identified by traditional tools. This method not only streamlines the security process but also accelerates development by allowing teams to concentrate on the most relevant issues​.

Autonomous Product Security Engineering

Autonomous product security engineering represents a significant advancement in the realm of application security. By integrating directly into developers' workflows, this approach can effectively alleviate vulnerabilities in real-time. It operates by analyzing code during the development process and proactively suggesting code fixes. This method not only identifies potential security issues but also offers immediate, actionable solutions, seamlessly blending security practices with the development process. This integration helps maintain the flow of development while ensuring that security is a continuous, integral part of the process rather than an afterthought. Such a system significantly reduces the time and resources typically needed for vulnerability detection and remediation, and it enhances overall application security without disrupting the development cycle.

Pixeebot monitors code repositories and provides high quality fixes instantly

Pixee.ai has pioneered a developer tool for enhancing application security. Their product, Pixeebot, is designed to identify and fix vulnerabilities within code, effectively streamlining the software development process. By integrating directly into the development environment, Pixeebot offers high-quality, instant fixes as developers or AI-assistants work on code. This approach ensures that developers can maintain their productivity and focus while simultaneously enhancing the security and quality of their code. Essentially, Pixeebot acts like an automated security engineer, seamlessly integrating into the development workflow and significantly reducing the backlog of security tickets by transforming code scan results into actionable pull requests for merging. 

Conclusion

The emerging techniques in application vulnerability management, such as Pre Production Reachability, Runtime Instrumentation, and Autonomous Product Security Engineering are clearly attempting to solve one of the biggest problems in cybersecurity in vulnerability overload among security teams. As many breaches still begin with unpatched vulnerabilities, it is still a key obstacle in getting a strong security posture among enterprises. By prioritizing actionable vulnerabilities, integrating directly into developers' workflows, and employing advanced technologies, these approaches are not only addressing the current challenges in vulnerability management but also paving the way for more efficient and secure software development practices.

Special thank you to Sean Cassidy, Mike Schwartz, Arshan Dabirsiaghi, Ed Bellis and Surag Patel who provided incredibly valuable input and helped me think through some of these concepts.

In the ever-changing world of software pricing, the rise of AI agent applications could lead to a significant change in how companies determine the value of their products. To grasp this potential shift, it's insightful to examine the history and recent trends in software pricing models,  particularly focusing on seat-based pricing and consumption-based pricing.

Salesforce, for example, has famously employed a seat-based pricing model. This model is characterized by charging per user, with pricing tiers offering varying levels of functionality, starting with around $25 per user per month with incremental creeping up to their Lighting Unlimited package of $325 per user per month​. 

In contrast, consumption-based pricing, as used by companies like Snowflake and Databricks, charges based on the amount of resources or services consumed by the user. This model aligns well with data and cloud computing platforms where the usage can vary significantly across customers.

The introduction of AI agent applications adds a new dimension to the pricing discussion. Agents’ unique value proposition, coupled with diverse applications across various business functions, significantly complicates pricing strategy. Companies must consider the value delivered by the AI, the costs associated with its operation, and customer preferences. So, if you are an AI startup, how should you address these factors and determine pricing? 

In the following, I try to explore three models around pricing for the next generation AI agent companies, analyzing the potential benefits and drawbacks of each approach. 

AI Copilot ➡️ Seat-Based Pricing

If you are building a AI Copilot company, a seat-based pricing model may be a good fit. This approach aligns with the product’s aim to supercharge an individual's performance. Since a copilot product is intended to enhance the productivity and capabilities of specific roles within an organization, the pricing should be linked to each individual user. An obvious example of this is Github Copilot, which charges $10 per user per month and can go up to $39 per user for enterprise features. 

The per-user pricing of GitHub Copilot aligns with the direct benefit it provides to each individual developer’s productivity, and scales directly with an organization’s user base of engineers. The advantages of this pricing model is clear: it encourages users to use as much of the technology as possible, which has downstream benefits of increased user data for product iteration and virality within an organization. However, with essentially unlimited usage, the provider of the product faces less control over its cost structure. Theoretically, the product could continuously call upon the underlying foundation model, leading to potentially unbounded costs. A possible workaround for this would be implementing rate limits on usage, though it might have an impact on the perception of the product by customers. 

AI Agents ➡️ Usage-Based Pricing

Usage-based pricing in AI startups is a trend that aligns closely with the actual consumption of services, offering a more flexible and fair approach to customers. This model is particularly relevant in the field of AI Agents, where the computational demands can vary significantly based on the application and user engagement. By adopting this model, AI startups are able to offer their services in a way that scales with the customer's actual usage, ensuring a direct correlation between cost and value received.

One notable example is Intercom's AI chatbot Fin, introduced in March. Fin represents a shift in AI service pricing, charging customers 99 cents for each customer request it successfully resolves. This pricing model represents a notable departure from traditional flat-rate or subscription models, aligning more closely with consumption-based strategies similar to those employed by  cloud computing platforms such as Snowflake and Databricks. In these platforms, costs are calculated based on the computational resources used – typically measured in credits – and consumed according to data volume or task complexity. Similarly, Intercom's Fin directly ties the cost to the service's effectiveness, charging per resolved request, thereby aligning the cost with the value delivered.

Usage-based pricing models come with distinct advantages. The companies offering the product benefit from predictable costs as charges are tied to each use of the service, making it straightforward to understand the direct cost of calling the foundation model with each use. This transparency and direct correlation to usage make the model appealing to customers, as they only pay for what they actually use.

However, there are challenges to this model as well. For startups, revenue forecasting becomes more complex and less predictable, as income is directly tied to fluctuating customer usage patterns. Additionally, customers might seek workarounds to minimize their use of credits, potentially leading to underutilization of the service. This can discourage full usage of the AI capabilities, as customers may be motivated to conserve credits to save costs.

Autonomous AI Service ➡️ Capacity-Based Pricing

As we enter the emerging world of AI agents and Service as Software applications, I have had a few interesting conversations around a potentially new developing pricing model, one that is closely linked to the cost savings in human labor. This model, which I’ll call “capacity” or “staff augmentation-based pricing”, aligns the value of AI solutions with the labor costs they offset or replace.

For instance, consider a traditional managed service provider who charges based on the staffing needs they fulfill. A similar approach can be applied to AI agents, where the cost of the AI solution can be benchmarked against the full cost of a human employee performing the same task. This includes not just the salary, but also additional expenses like healthcare benefits. If an AI agent can perform the work of a human IT analyst, who, for example, handles 5,000 IT tickets a year, the cost savings become evident. The AI’s pricing could then be set in proportion to the cost of the human labor it replaces.

This pricing model becomes particularly relevant in two scenarios: One is in areas where labor shortages exist and the availability of human resources is just plain limited. The other is a cost discussion for the customer. Organizations might find adopting AI solutions very appealing when the cost of these AI agents is significantly lower than hiring additional staff – potentially at a ratio of 4:1 or 3:1, meaning the AI is four times cheaper than the human alternative.

By tying the value of AI directly to labor cost savings, this model offers a pragmatic approach for organizations to evaluate and adopt AI solutions, making them a viable alternative. 

Conclusion

For AI startups navigating this landscape, the chosen pricing model should reflect the value delivered by their product. It should be simple, easy to understand, and allow for straightforward forecasting. This approach is vital in the AI-driven software domain, where the value proposition can significantly vary based on the application and its impact on business processes. 

Special thank you to Matt Peters, Peter Silberman, Surag Patel, Edward Wu and Scott Gudmundson for providing incredibly valuable input and helping me think through some of these concepts.

The recent product launches by OpenAI have triggered an immediate reaction among startups, prompting an essential question: In an era which is dominated by a fast moving incumbent, how can a startup establish a defensible position?

This evokes memories from the mid-2010s when AWS reigned supreme in cloud computing, sparking rapid innovation. Their impactful re:Invent keynotes often led startup founders to reconsider their reasons for entering the industry in the first place. And yet, we still have massive infrastructure companies like Datadog, Elastic, and Databricks which were all started during that time. It seems plausible that it will be similar this time around. 

Shortly after OpenAI's Dev Day, and merely a few blocks away, GitHub unveiled new features for Copilot at their Github Universe conference. With over 1 million paid users across 37,000 companies,  GitHub Copilot serves as an interesting case study on how a Service-as-Software product can build and sustain long term differentiation while using foundation LLMs from Open AI. While Copilot itself is of course part of a larger company in Microsoft, a lot of their tactics can be applied to both startups and scale ups trying to build AI products. In the following, I try to highlight a few of the strategies that I believe transfer well and can provide interesting guidance for founders currently building in the AI arena. 

Community = 1st-Party Data, Evangelism and Adoption

The role of community engagement in shaping AI-driven companies has become increasingly crucial. The extensive data generated from open source or freemium users, combined with the rapid pace of feature requests and feedback, is pivotal for crafting an effective and dependable AI service.

In recent years, open-source companies have primarily monetized their user base through cloud-based managed services or enterprise features like Role-Based Access Control (RBAC), high availability, and enhanced security. This approach often involves charging for the convenience of minimizing infrastructure work required to deploy the service efficiently within an organization. For instance, Elastic operates Elastic Cloud, a managed service version of its open-source offering, and MongoDB has a similar approach with MongoDB Cloud, among others. This 'convenience layer' monetization strategy essentially frees initial community users from concerns irrelevant to them but necessary for enterprise requirements.

Looking forward, companies with a strong community foundation are likely to leverage their extensive data to create new services using LLMs and agent technology. These services are both complex technologically and from a use-case perspective. GitHub's Copilot is a prime example: it leveraged repository data to train its initial version and has continually refined it, keeping its core users, the developers, in focus. Every user interaction of accepting or rejecting code suggestions provided it with an opportunity for gathering 1st-party data that not only continuously improved the product but also created a long term defensible mechanism. Crafting a tool that developers not only use but love is a significant achievement, given the complexity and utility of Copilot. 

The strategic advantages of having a built-in community are clear, offering opportunities for data collection, implementing best practices, evangelism and product testing by users. For modern Service-as-Software companies fortunate to have such a community, the challenge and the opportunity lies in balancing complexity and convenience in a single cohesive product strategy, grounded in first principles. In short, if Github was started today, how would it jump directly to Copilot?

Fanatic Obsession on UI / UX + AI = User Happiness

GitHub Copilot's recent developments, highlighted at GitHub Universe last week, reveal a vital insight for AI startups aiming to build a lasting and successful business. The profound integration of UX and UI in their AI offerings, such as GitHub Copilot Chat and Copilot Enterprise, is a testament to balancing complex AI functionalities with user-centric design. For AI startups, this signifies the importance of not just leveraging advanced LLMs, but also embedding it in a way that is intuitive and aligns with user expectations and workflows.

GitHub Copilot Chat, powered by GPT-4, exemplifies their commitment to UX/UI by enabling natural language programming and offering code-aware guidance, inline chatting about specific code lines, and user-friendly slash commands for task shortcuts​​. This tool is not only highly accessible, being integrated into GitHub.com and its mobile app, but it also enhances the developer experience by providing suggestions, summaries, analysis, and answers for coding queries, directly within the platform​​.

The Copilot Enterprise edition tailors the Copilot experience to organizational needs, empowering teams with AI assistance at every step of the software development lifecycle. It offers personalized code suggestions and documentation help, quickly bringing teams up to speed on their specific codebases. This customization, combined with enterprise-grade security and privacy features, illustrates how GitHub Copilot has evolved from a simple autocomplete tool to a comprehensive, AI-powered development aid​​.

AI startups looking to emulate GitHub Copilot's success should prioritize creating AI tools that are technically sophisticated yet intuitive and deeply integrated into users’ workflows. This focus on user-centric AI development, coupled with continuous innovation, forms the cornerstone of building a long-term viable business in the competitive AI industry.

Deep Integrations Into Multiple Core Systems

AI startups can gain significant insights from GitHub Copilot's recent preview release of "Workspaces". This move exemplifies the strategy of deeply integrating into various systems to create a unified, agile user experience. As described in my previous post on the "Many-to-Many Problem", the capability of AI agents and LLMs to navigate through a labyrinth of systems, aggregating and interacting across tools, can vastly improve operational efficiency and decision-making processes​​.

GitHub Copilot's foray into Workspaces highlights this potential, addressing the complexity of modern software development environments. By leveraging the knowledge of the entire codebase and the reasoning capabilities of GPT-4, GitHub Copilot Workspace assists developers in efficiently turning ideas into code. This not only streamlines the development process but also integrates various aspects of software development into a cohesive workflow.

Creating agent systems viable for long-term use depends on integrating systems, workflows, and data​​. GitHub Copilot Workspace embodies this approach, demonstrating how AI can be utilized to solve specific use cases while aligning with existing enterprise budgets and needs.

For AI startups looking to build a sustainable business model, GitHub Copilot's strategy of branching out and integrating into multiple systems provides a valuable blueprint. By focusing on deep integration and addressing specific user needs, startups can create products that not only solve immediate problems but also fit seamlessly into the broader workflow, thereby building a long-term sustainable competitive advantage in the service-as-software domain.

Conclusion

Forging a long-lasting and defensible position in the fast-evolving world of genAI necessitates startups playing to their unique strengths. This could involve:

• Cultivating and leveraging a robust community of engaged users

• Focusing intensely on marrying user interface and experience with AI to meet users where they are

• Developing deeply integrated AI agent systems that provide critical insights and actions across various core systems

Ideally, a combination of these approaches would be most effective. Despite OpenAI's rapid pace of innovation posing a challenge to startups within its ecosystem, the strategies employed by GitHub Copilot, which have cemented it as a leader in the AI -powered software development tooling, offer valuable lessons. Needless to say that Copilot obviously benefited massively from being part of Microsoft, the tactics nonetheless provide an interesting blueprint for AI startups to establish their own strong presence in this new era.

TLDR: With the continued rapid innovation around LLMs and AI Agents, software as we know is getting a major upgrade. From on-premise software to SaaS, we experienced a major productivity boost for knowledge workers and created a massive industry in the process. As we drive forward into the genAI era, we will see the next generation of software applications, that instead of being delivered to us as a service (SaaS), actually be a service themselves, delivered to us in the form of software. This will be the era of Service-as-Software. Undoubtedly, the latest product releases by OpenAI yesterday, regarding agents / GPTs in particular, are major step in making this a reality.

The Opportunity of AI Agents in the Software & IT Services Market

It’s now almost 1 year A.C (After ChatGPT) and it's clear that it has sparked a new wave of excitement in the startup and tech world. This is especially emphasized in light of OpenAI’s product releases announced yesterday of becoming a basically a full fledged UGC platform where users can create and share agents, with everyone brainstorming what's now possible. Reflecting back on the software ecosystem, I've been particularly drawn to revisiting the original purpose behind creating SaaS platforms. Essentially, they were meant to boost our productivity as knowledge workers, and delivering this software through the cloud simply made everything more convenient and effective, which is how SaaS took off.

Software spend globally, whether it’s hosted in the cloud or on-site, has been on a rapid rise, reaching the massive market size of about ~$900 billion. But, alongside this hefty investment in software, is an even bigger bill for IT services.

Source: Gartner

As a matter of fact, for every dollar we spend on software, we end up spending 1.5x times more on IT services to get that software operational and successfully deployed. This includes areas like BPO, outsourcing and consulting, which take up most of that extra cost. You could argue that that number is potentially even understated since a lot of companies use FTEs to do the manual work around tuning, maintaining, integration  and operational work. Bain & Company suggests that around 37% of IT tasks could be automated using genAI.

When you think about the vast $1.4 trillion annual spend on IT services, it's easy to see the huge chance for new companies to step in. There’s a big market out there, maybe even larger than the software market itself, ready for businesses that can use AI to shake things up.

The Era of Service-as-Software 

Automating human labor and turning it into a successful software business is certainly not a new concept in Infra IT. Robotic Process Automation giant UI Path has led the charge in automating every manual task from data entry, invoice processing to supply chain logistics. In cybersecurity, Expel has built a fully transparent Managed Detection and Response (MDR) service that is powered by automation software to help its customers defend against cyber attacks.   

With the rapid innovation around AI Agents, software applications as we know them are in process of getting a major upgrade. Agents are programs that can make decisions or perform actions based on its environment, user feedback or experiences. In short, they mimic the subtle human behavioral characteristics that make us humans quite effective at taking actions or fulfilling tasks. What this means in practice is that instead of software simply being delivered to us as a service, agents and LLMs will power the service to be delivered to us in the form of software. Thereby this ushers in the Era of Service-as-Software. 

At the end of the day this really plays into the most exciting and scariest opportunity of AI and that is the ability of humans to process, reason and build knowledge while using all three effectively to conduct a task autonomously. Over the course of the last year, we have been investing in various companies in the infrastructure IT & cybersecurity space that play on this trend, but all have very similar characteristics that I wanted to summarize here. 

Labor Shortage Creates Urgency

In certain critical sectors, AI's intent isn't to supplant humans, particularly where there's a stark labor deficit preventing roles from being filled. Cybersecurity is a prime example, with a staggering 3.5 million positions lying vacant. The urgency intensifies as we witness a surge in hackers leveraging LLMs, escalating the frequency and sophistication of cyber-attacks. The barrage of alerts from security tools monitoring firewalls, endpoints, cloud assets, and emails can overwhelm even the most sophisticated security dashboards, akin to an unending cascade of Christmas lights. In the trenches are the beleaguered analysts in Security Operations Centers (SOCs), who grapple with the sheer volume of alerts, leaving them stretched thin and increasing the risk of critical breaches slipping through the net.

DropzoneAI, which is pioneering the development of a SOC Tier 1 Analyst powered by LLMs is designing an agent based system to autonomously sift through alerts and field ad-hoc queries, tapping into a host of tools and data repositories to investigate a security alert. Here you can see it conducting a full investigation of an AWS Guard Duty alert that gets fired to Splunk as somebody is trying to potentially conduct a data exfiltration out of S3. 

In realms plagued by skill gaps and labor shortages, LLMs emerge not as replacements but as vital enhancements to human capabilities—acting as force multipliers where the need is most acute. Such strategic augmentation underscores AI’s role as a crucial ally in functions where shortages of labor are apparent. 

The Many-to-Many Problem

The human capability to navigate through a labyrinth of systems to extract answers is nothing short of remarkable. Consider a developer engaged in the debugging process: the journey often involves traversing from an Integrated Development Environment (IDE) to an Application Performance Management (APM) platform, and then to a log management system to pinpoint the root of an issue. This scenario, where insights are nested within a multitude of platforms—be it in data engineering, DevOps, or cybersecurity—signals a prime niche for AI agents and LLMs. They thrive where tools sprawl and interoperability falters, stepping in when a human intermediary is traditionally tasked with piecing together a coherent narrative from disparate data sources. In DevOps, the aspiration to integrate logs, metrics, and events across various platforms has long been a topic of discussion. Data engineering wrestles with the fragmentation of information across silos, standardizing the data, checking its quality and feeding it into diverse analytical tools. Meanwhile, security analysts leapfrog over an extensive array of instruments—from Endpoint Detection and Response (EDR) systems to firewalls and logs—to thoroughly investigate alerts. The question then arises: might a Service-as-Software company adeptly shoulder the more monotonous tasks of aggregation and cross-tool interaction, thus sparing us to focus solely on critical decision-making? Such a solution could significantly transform the efficiency and efficacy of operations across these complex roles.

Large Scale Analysis but with a Personalized Touch

LLMs and AI agents are particularly adept at conducting deep-level analyses on a grand scale, thanks to their capacity to parse and make sense of data from a multitude of sources, all while maintaining the ability to offer a personalized touch. In the field of IT support, for instance, these AI systems can draw from a variety of databases, incident logs, user manuals, and forums to diagnose issues and provide solutions. An AI agent can analyze thousands of tickets from an IT support system, identify common problems, and suggest improvements or automated responses for future incidents. This capability not only streamlines the troubleshooting process but also allows for customized assistance; the AI can learn from each interaction with a user, recognizing patterns in the types of issues a particular user faces or the level of technical language they understand. It can then tailor its communication to match the user's expertise, whether it's a network engineer needing deep technical details to resolve a complex server issue or an end-user requiring step-by-step guidance to reset a password. This individualized approach is the result of sophisticated algorithms that adapt and optimize their output, ensuring that as users engage with the AI, the guidance they receive becomes increasingly focused on their specific needs and preferences, even in the context of a broad and complex IT landscape. 

What’s next?

In the IT infrastructure and security sectors, there's a groundswell of opportunity to innovate with autonomous agents, propelling us toward a more seamless Service-as-Software (SaS) future. Crafting an agent system that is viable for the long haul and ready for production hinges on a deep integration of systems, workflows and data. Thereby I believe for these agents to truly be disruptive and enduring—ready to be deployed at scale—they must be built with a vertical-first and maybe almost role based first strategy (and potentially scale horizontally from there). This approach isn't just about achieving early wins; it’s about solving specific use cases while also aligning with already existing budget line items in the enterprise.

When considering which ideas to pursue first, the guiding star should be the urgency of labor need—roles that are hard to recruit for would allow agents to be adopted at a higher rate and provide a faster journey towards product-market fit. I couldn't be more excited about what this new paradigm shift will bring, so if you are thinking of building a company in this space, please reach out to me!