We’ve just launched our agentic capabilities to make your AI workflows even more powerful. Try it out for a limited time →
Q: Tell the audience a little bit about who you are.
David: I run internal product, Enterprise initiatives at MongoDB, and I'm on the hook for internal generative AI. Basically all the internal use cases for how we adopt it.
Q: What are you guys actually using generative AI for internally?
David: I'll focus on the internal story. Framing this is kind of thinking a little bit through the journey we've taken. Not unlike a lot of other shops, we started with a sort of traditional chatbot experience, not hooked into any internal data. Safe approach - just hook it up to publicly available data. We were literally crawling our own websites at something just to think about like data availability for AI was a really interesting story.
We wrote the web scrapers to crawl our own websites, which is funny when you think about it because it's like almost like you wouldn't think that that's a challenge internally. And there was of course a lot of data we did have - we had our docs team, our education teams, there were things that we could use.
Pretty quickly we recognized that there was a need for some kind of Access Control layer, so we needed permission retrieval. This is where Credal came in. Candidly, we looked at building this ourselves, and we had a lot of starts and stops with that. In the end, the upkeep necessary and the hardening that we needed was just not going to be scalable for us to do ourselves.
Once we addressed that concern, that took us on that quicker trajectory of shipping more valuable applications - things that can talk to your internal data, permission aware, sensitive content. It kind of broke through some of the trust boundaries for us internally because there was a lot of fear with our leaders.
As we progressed through that journey, the next stage was more around "okay we can have conversational AI, we can talk to internal data and content in a secure way, but can we actually perform intelligent operations?" Something that has this cognitive understanding of your business. This took us down the path of graph RAG and all these types of concepts.
We've done a bunch of investment there, and now we're in the age of Agents.
Q: How did you guys actually point to which problems to solve? How did you prioritize?
David: There's a lot of nuance here. The thing that Credal enabled us to do with this concept of a central RAG service was enable a lot of self-service ability to just create. We saw a lot of plumbing happening there.
One of my favorite examples was a GRC bot that was created. There's a Dev prod bot underway, so people can sort of self-help in this context. But when you think about these larger investments, these Enterprise scale "what's the thing we should focus on" - this is much harder.
Somewhat naively, we made certain investments where from a security and standards perspective we just needed to do. So we shipped something called MongoDB GPT, which is like a companywide Chat GPT experience. That was because people were going to Chat GPT, so this is like a shadow IT problem.
Q: How did you guys think about the trade-offs between building your own solution versus buying Chat GPT Enterprise?
David: One, a lot of these Enterprise based solutions are architecturally still something we're not comfortable with. Two, just building it on our infrastructure, behind our corporate boundary makes us comfortable. But also having this surface area for the entire company gives us additional utility to do things that are very overfit to our needs - very bespoke solutions.
When you look at the road map for that particular product, it's really centered on squeezing out more MongoDB specific optimizations.
When I think about how we think about these investments - first it's those table stakes, and then we started pulling a lot of data from usage patterns. What are people actually doing in something like MongoDB GPT?
Interestingly, we found that our largest user group was actually engineering. This was a surprise to us. This centered us on what are engineers doing, which changed how we thought about integrations and which products ultimately mattered.
My best advice in these cases is just ship something quickly and then see what people are doing with it.
Q: How do you translate business goals into product requirements and measure ROI?
David: Even our sort of how we think about the productivity that comes out of these types of systems is constantly evolving. The primary metric with something like MongoDB GPT was Time Savings. Then we started socializing some of those gains back to our leadership and their response was "okay great but Time Savings aren't dollars." You can't just draw a straight line from Time Savings to dollar savings.
What I have found is that most of the time, if you're going to say something is boosting productivity, you really need a handraiser to say "I'm going to suspend growth in my team because I'm taking that leap of faith that I'm going to get that additional capacity from the systems you're building."
It has to translate into real dollar Opex, or conversely, if you don't want to do the Opex approach, then you have to sign up for something like more quota or more topline. Otherwise, you're just going to accumulate all these theoretical savings and say that you're saving three hours a week which translates to astronomical figures that nobody's going to believe.
At the end of the day, you look at the P&L and you're like "wait, where was all of that?"
Q: Can you talk about the Coach GTM use case?
David: Coach GTM is a technical question responder trained on all of these internal technical documentation libraries, code bases, and everything else. If you think about these more expensive capital-T Technical Resources that have to answer questions on the fly coming from customers, that entire support ecosystem can now move faster and have these things at their fingertips.
Once you curate that data and start to see where the quality of response becomes, now you have these golden data sets and you can start to get super creative and build sort of like in-call live prompting.
We've seen a lot of excitement around the Coach GTM product, tons of usage as well. But candidly, even that one is difficult to translate into real dollar savings. We can't even really equate it to shortening of a sales cycle.
A lot of this telemetry and diagnostics is a big part of the problem. You're tracking classic MAU type of stats, but it's hard to get it down to a translatable P&L change.
Q: How do you herd the cats, get alignment, and get stuff done when you have so many different teams thinking about AI simultaneously?
David: There's a natural sort of herding that happens because you're providing a lot of the Baseline tooling necessary for other teams to even develop on top of. You're giving them the frameworks, the models, you're centralizing that aspect of it.
Philosophically, we didn't really start as an Enterprise that was saying "there's only one team that's going to ship AI." It was "everybody go at it, go solve the problem, you're the experts."
My goal is not necessarily to be some kind of governance over what everybody is doing. We continue to encourage use and adoption of AI and do the things that are very specific to your verticals. We're just enablers.
But there are definitely situations where there's synergy. Our marketing Ops Team built a great backend service, and rather than recreate some of that work, we're leveraging it. We use this multi-agentic framework and orchestration within our apps to talk to their service.
We're investing in thinking about how to build custom agents, vertical agents, horizontal platforms. All of those investments are just inputs into where we should be focused.
Q: Where have you had pushback? What have been the hardest things to get done?
David: Copyright law is complicated and basically ultimately requires a human being to have created the thing for it to be copyrightable. If you are shipping code that is materially AI generated, you are jeopardizing your copyrights. This might work for certain kinds of code but not all code.
Then there's cases where you have to detangle the code - you have non-shipped software that is in the same repo potentially. Plus we have an open source product as well.
The next challenge is basically anything where you're introducing a new kind of threat vector. A lot of AI now is at the connector level - it's talking directly to these different SaaS products and systems and pulling data out in a way that's no longer this traditional data lake ETL process. By centralizing information to make it more AI accessible, you're also giving people a single point of failure or threat. There's a lot of work around creating comfort there.
Also, candidly, a lot of the most interesting companies are newer and smaller and less hardened. There's this metacognition that "I'm about to sign up a 15-person company." They're claiming they have all the Type 2s and the pen tests, but no two pen tests are equal. There's a lot of angles that slow innovation unfortunately, but for good reason.
Q: How do you think about business continuity when working with startups?
David: I think of it across a couple of different dimensions. The first is: is it part of my stack and is it critical? If so, how can I not build in a layer of protection? This is where we think a lot about abstraction. That's never perfect, but there's additional plumbing just to make that work.
If you look at how Enterprises are thinking about this, it is so different to how they thought about cloud migration. In the cloud migration, everybody just went to their cloud provider - they weren't thinking about how to get off of AWS. People are just so much savvier about it today, where they want to bet on the technology but don't necessarily want to bet their entire house that this provider is not going to raise the price 10x in a year or two.
Can you get comfortable with a smaller, newer entrant being a fully SaaS solution? That's already a hard thing to do. The company can tell a great story, but ultimately you're shipping your most sensitive, valuable data, and if they go out of business or get acquired, what does that mean for you?
It has happened - we've had elements of our stack that were in that startup category that have gone out of business. We just got an end-of-life notice, and it was like "go sort it out." There goes our roadmap, because not that the product will just fail overnight, but it's not a supported product, which means we have to shift our focus towards now building the capability.
Q: Where did you guys wind up on the "Vibe coding" or AI coding assistants?
David: First of all, any system where people are taking away VS Code from engineers, they're just like "over my dead body." Switching your IDE is just not an enticing thing for a lot of folks, and you want a high degree of buy-in.
We use more of the code suggestion, code complete type of solutions, mostly because there are some knobs in there that are necessary for us. But that's limited in its utility. Most of the research will tell you that acceptance rates are typically in that 20-30% range, and they drastically drop as you move up the seniority stack.
Junior devs are hitting complete, tabbing out code suggestions at a much higher rate (40%), and my staff engineers are doing it at 5%. Should I be super comfortable about that or should I be a little bit wary?
We have looked at everything from the virtual UI to the more advanced integrated cursors and augments. The jury is out - we are very interested, but at our size and scale, it takes us a little bit longer to get comfortable.
David: The thing that Credal enabled us to do with this concept of a central RAG service was enable a lot of self-service ability to just create. We saw a lot of plumbing happening there.
When we looked at building permission retrieval ourselves, we had a lot of starts and stops with that. In the end, the upkeep necessary and the hardening that we needed was just not going to be scalable for us to do ourselves. Once we addressed that concern with Credal, that took us on that quicker trajectory of shipping more valuable applications.
Q: How do you think about the competitive advantage of AI adoption?
David: It used to be the competitive advantage was that you were large, you had scale. But now it's turning into the reverse advantage for the smaller, more nimble companies, the startups that are just "full AI anywhere we can tap into it." They're moving really quickly.
We know that we have to get there. Other Enterprises have figured it out to an extent - Google ships something like 25% of their code as AI-generated. There is an avenue to it, we just have to take our path.
Part of that is approaching it from a few different vectors. I mentioned earlier that our largest persona group in MongoDB GPT is engineers, so the immediate pivot there was to add more models, give them optionality to choose a Claude or other models to interact with as opposed to whatever the default model is behind the scenes.
Q: AGI skeptic or AGI bull? Will AI replace your job?
David: I believe that the best use of AI is the augmented use case, not the replace use case. I think of the Iron Man analogy and the Jarvis suit. Calculators didn't replace accountants - it allowed them to have more clients and more billable hours.
Get on the train, use these tools, learn these tools, get curious about these systems. You don't have to be an expert by any means, but you should be aware. The best solutions, especially the ones that we've been building, are still human-in-the-loop. How do we focus them on the higher value activities versus the lower order activities? That's the main thesis of everything we do.
Credal gives you everything you need to supercharge your business using generative AI, securely.
