How Engineering Leaders Measure AI ROI -- Seattle AI Week Recap

Most engineering teams are spending more on AI. The harder question is how to tell what they're getting back. That question drove "The ROI Question: Measuring AI Against Headcount, Quality, and Cost," a Seattle AI Week panel Fuel Talent hosted at Industrious in Seattle on Thursday, October 29, 2026.

Derek Stephens, Managing Director, Revenue & AI Enablement at Fuel Talent, moderated a conversation with Nipun Dureja CTO of iSpot.tv, Sharath Udupa, CTO of SumerSports, and Jeremy Tryba, VP of Engineering at Ai2. The panelists came at the questions from different company sizes and stages, and they didn't always land in the same place.

Tokens and lines of code came up early. The panelists track them, and they described them as only a small piece of what's considered. Here's what else they weigh, and what it means for how they hire.

How do engineering leaders measure the ROI of AI?

Tokens and lines of code are easy to track, and leaning on them too heavily can reward the wrong behavior. Alongside cost, the teams on the panel look at how quickly work moves from idea to production, quality measures such as incidents, bug counts and rework, developer experience and focused time, and the business outcomes the work supports.

Jeremy noted that token cost can also work as an early signal. If a typical change starts costing more over time, it may point to a codebase that's getting harder to work in.

Several panelists described the cost side as a moving target. Pricing and model capabilities change often enough that long-range AI budgets are hard to pin down.

How are teams keeping code quality up as AI output grows?

Teams are producing more code with AI, and the panel was candid that review practices built for human-written code are still catching up. Sharath described an open debate on his team about how much AI-generated code engineers should read line by line, and said there isn't a settled answer yet.

The approaches the panelists are trying include writing a clear spec and design before handing tasks to agents, automated testing that tries to break new code before it ships, closer human review for changes in the most critical parts of a system, and asking engineers to walk through how they built a large change, prompts included.

For Jeremy, culture comes first: teams that care about the quality of what they ship and have room to try new methods, with the best of those experiments becoming shared practice over time.

Where does engineering judgment fit as AI writes more code?

Judgment came up in nearly every part of the conversation. As AI takes on more of the first draft, the panelists see engineers spending more of their time on whether the work solves the right problem, whether it's secure, and whether it serves the business.

Sharath described flatter teams where one person can own more of a product, with the open question being how to keep those people coordinated and working on the right priorities. Jeremy expects engineering teams to put more focus on infrastructure and security as more people across a company start building software. Both framed these as open questions rather than settled answers.

How are technical interviews changing with AI?

An audience member who leads tech recruiting asked how the panelists are adapting their interviews. All three said people should still be interviewing people.

What's shifting is what the interviews are designed to show. Sharath described replacing a traditional coding round with an AI-assisted coding exercise and a separate code review exercise, along with more weight on system design. Nipun described live problems where the focus is on why a candidate made each choice, including how they think about architecture and security. Jeremy said his team uses AI tools mainly early in the process, to confirm that candidates are who they say they are.

Fuel Talent's process has always included engineers evaluating engineers, and that's become more useful as the bar moves toward judgment. Thank you to Nipun, Sharath and Jeremy, to everyone who brought questions, and to Industrious for hosting. For more from our events, read the recap of our Seattle Tech Week panel on hiring in the age of AI.