We’ve just launched our agentic capabilities to make your AI workflows even more powerful. Try it out for a limited time →
Ramp's Ian Macomber on Building Model-Agnostic Systems
January 22, 2025
We’ve just launched our agentic capabilities to make your AI workflows even more powerful. Try it out for a limited time →
Q: Can you introduce yourself and tell us what you do at Ramp?
Ian: I'm Ian Macomber, I lead the data team at Ramp. Been there for just under three years. All my prior experiences in B2C e-commerce, so came in knowing a lot about product analytics, marketing analytics, very little about credit and underwriting and risk and fraud in fintech and payment rails sales. Within Ramp, a lot of what we did was really building out feature parity with competitors. I think increasingly for us, we now have a data moat and a distribution moat, and so when we think about what we can do with generative AI, there's all the normal use cases around how do we make our own team more effective, but I think increasingly we're trying to think how can we build something that no one else can to help save our customers time and money with Generative AI.
Q: What do you actually use AI for at Ramp?
Ian: What I'm most excited about right now is I think maybe four years ago you're like who can implement large language models in industry and maybe it's 150 people. And two years ago you're like okay at a company like Ramp it'd be the machine learning engineers, the senior ML. I manage an analytics engineering team - we use DBT, we use BI tools, we write in SQL. What I'm excited about now is if you look at the advances of how easy it is to call these models and put them into a workflow, the most junior person on my team can leverage Cortex within Snowflake as part of DBT and in SQL.
The probably strongest opinion I have right now is for an analytics engineering workflow - it's not even applied ML. You've been able to do work in a CSV file with tabular data sets and sums for decades. You've really only been able to work closely with semi-structured data for four or five years. And I think over the last three months and sort of always going forward, just the analytics engineering workflow will increasingly include text and images and videos and contracts and PDFs the exact same way it's always included CSV files.
Q: What's the coolest thing you've seen someone build who isn't necessarily a 10-year AI PhD?
Ian: My favorite - this is not necessarily cool in how cool it is, it's more just cool in how easy it is - is we were able to build a retrieval augmented generation program using 26 lines of SQL. And you think about like what that would mean relative to like a year or two ago, that's incredible.
For us, I'll just give you an example: if you ask Facebook's Llama model what is Ramp, it knows what Ramp is, it's been trained to that point. If you ask what is Ramp Plus pricing and you give it the context around don't hallucinate, it says I do not know Ramp Plus pricing. But if you take our Gong transcripts and you just say find me sentences that are semantically similar to Ramp Plus pricing, finds those, you add that in the context window, you then say here are sentences that are like Ramp Plus pricing, put into the context window "what is Ramp Plus pricing," it says "Ramp Plus pricing is $12 per user per month." All 26 lines of SQL.
Q: What does your AI stack look like and how did you think about which models to use?
Ian: I can talk about some of the evolution that we've had at Ramp and also we have data on a lot of what Ramp customers do. The first thing I'll say is companies are designing their stacks to flip between foundational models extremely quickly. We see that in things like how many OpenAI customers are also spending on Anthropic is going up and up and up.
I think the B2C analogy is I have Uber, I love Uber, I also have Lyft. I check them both almost every time. That's what we do internally and it's what we see a lot of our customers do.
To talk a little bit about receipt matching, this is one of the first times we built out AI, took a dependency on Scale. They had a receipt matching project - we uploaded receipts, they process them, something like 10 cents an invoice. Eventually GPT came out with a model that could do it better and cheaper and it's more generalized. Then eventually 3.5 comes out, and we're plotting points on an axis of how good is the model versus how expensive is the model. Eventually these models on that specific task just start to get to 100% accuracy. At that point really all you're competing on is cost.
Really the only constant for us has been let's assume these models are only going to get faster, more powerful, and cheaper. That's how we design and that's also how we see Ramp customers designed.
Q: Why do you think AI models have been more interchangeable compared to cloud or data infrastructure?
Ian: I think it probably has to do with state versus lack of state. If you call a model, upload a receipt, loosely the assumption is you'll get the same answer and over time it'll be more accurate, but you don't necessarily get a ton of additional benefit on context. These are relatively stateless API calls, versus something like Snowflake or Azure or GCP - once you have built some piece of it, you get a little bit of vendor lock-in.
I think the same way that if I called an Uber earlier today, it doesn't mean that I'm going to call necessarily an Uber or Lyft based on anything other than my next interaction. We'll see if that changes in the future, but I think building out more context such that maybe you do get better answers or more of an understanding as you use the product more - I think that's probably what you'd have to believe to start seeing a little bit more lock-in.
Q: How do you build a defensible business when everything is interchangeable?
Ian: I think an analogy for me might be Hadoop six years ago or maybe streaming data 3 years ago, where you think about could you process a billion rows of event stream data six years ago and the answer was sort of loosely yes, but you probably needed to have a couple of database administrators spin it up and one manage it in perpetuity. So Uber had this stuff but it was not yet turnkey.
Then I think about analytics engineering stack - DBT, Snowflake, Fivetran - you can set up your stack with a credit card over the course of an hour. I think it's a little bit analogous here which is there's a lot that every company wants to do. We're thinking about how do we leverage Slack, how do we leverage emails, how do we leverage calendars for context. As you start to build that out and make it turnkey where you do not need a ton of resources to spin it up, I think that's largely how you sink your fangs into an industry and start to have a little bit more lock-in.
Q: What trends are you seeing in how companies are spending on AI?
Ian: The metrics that we like to track - I love the fact that we have these time series panel data. One of the first things we look at is of those that pay someone in a month, how many of them are paying the same entity 12 months from now. You loosely call that retention. A lot of the point solution AI tools don't actually have super sticky retention. They come out of an experimental budget, you play around with them, and then they go away.
We've really noticed that obviously OpenAI and Anthropic have very high retention. But then the other thing you start to notice is card spend we would loosely consider to be smaller amounts, a little bit more experimental and budget, and most importantly it's sort of after the fact. Versus if you are paying something on bill pay, it actually means that it's going to go through procurement - you're going to think about the capacity you're going to use, you're going to sign DSAs, MSAs, and DPAs.
Mostly we just really see that the spend is becoming operational and it is being managed the exact same way an AWS bill or rent would.
Q: Are experimental budgets still growing or are they being replaced by operational spending?
Ian: I think the experimental budgets are definitely still there. To sort of both plug Ramp and plug some of the services we offer, being good at procurement ends up increasingly being an advantage. When our developer - we've seen Cursor absolutely rip as a new vendor. Like we saw 60 businesses on Ramp paying for it, then 200 the following month, then 600 the month after that.
You think about that - it's a pull from the developers. If you want to have a world class engineering team, I'm not necessarily saying you need to use Cursor, but I am saying if you can't get Cursor in the hands of your engineering teams in two months, what are you even doing? This stuff is moving so fast and engineering salaries are so expensive, and you're not going to be able to get them a $20 a month subscription to supercharge them.
I think being able to buy quickly is extremely important. Unquestionably we're not going to know who the winners are in the space. You don't need to pick the winner - I think you just really need to move quickly on procurement and kind of also recognize in many of these situations, if you pick a tool for two years, build a lot, get it a little bit wrong and flip, it's actually not that much of a negative situation.
Q: What AI tools are you using at Ramp?
Ian: I'm using Cursor. I use it to write documentation. The main difference compared to Copilot is sort of like if you start with writing code and hit tab, or you just sort of hit command K and just start typing in English text. Mostly I just highlight big chunks and I say write documentation for me. Does a pretty good job.
We've decided how much of our total docs we want to index. We use Guru - it's really helpful for our sales team, really helpful for our customer service team. We also have a podcast that's all done by AI where it just takes five minute chunks of various calls with customers over the course of the week, packages them into five minutes, and we just hear the stuff people have really enjoyed about Ramp or the stuff that they've hated. We really like to listen to both of them.
Q: How does your lean, small team philosophy affect your procurement process for AI tools?
Ian: It's definitely a little bit different for the specific tool. The axes would be to what extent does this need to be a companywide decision, what are the ramifications one way or another.
We've always organized more around problems than tooling. An example would be our credit team really needs to have an understanding of what's in someone's bank account because we do cash flow loans. We do not have a Finicity team or a Teller team or a Plaid team - we have an engineering team and they can build in-house, they can buy, they can flip between models if they want. We say no one at Ramp other than you is going to be incentivized and tasked with understanding what's in a customer's bank account, and we'll let you make those build or buy decisions. Generally we like to have engineers make them.
Q: How are you thinking about what data is valuable to connect to AI models?
Ian: This is an area where I'm extremely curious. I don't know the right answer and I think we will in two or three years. You can't not have an opinion if you lead a data team on text to SQL. People are like how are we going to empower people, let them talk, get what they need in SQL.
We read the papers - a lot of the papers are like "we can generate 92% of SQL queries correctly" and you're like awesome. And then you read them and you're like "ah, you have four tables, they have six columns each, and they're named correctly." That's not what our database looks like. We have 133,000 columns. Our bill pay table has 15 timestamps and we need all of them.
I think increasingly we're going to need to say hey, not only will an LLM not be able to answer 100% of our questions, it won't be able to answer 90% of them unless we curate quite a bit. So I think for us we're thinking about how can we pair down the total amount of data we have, write the joins and the documentation in a way that an LLM can read it, and then hopefully empower the next generation of text to SQL on a much smaller subset of our total data.
Q: How are you implementing that curation process?
Ian: We are still very much in the test and learn phase here. The first semantic layer at a lot of companies is in Looker, and it is the case for us. I do not think everyone's semantic layer will be in LookML two years, three years from now.
This is one where to be totally honest with you, I took like the first five text to SQL meetings that people offered me. Now there's like 20 and I'm like we shouldn't take them anymore. I'm gonna wait till someone canonizes.
Q: What would a good text to SQL solution look like to you?
Ian: There's a concept of induced demand from traffic - you're like "there's a bunch of traffic on this two-lane highway, let's build some more lanes" and then you're like "more people drove, traffic got worse." That's what I think self-service data always will be, and that's why data teams are safe. As you make things more self-serve, human nature is to push up the edge of complexity as to what you can do, and once you get stuck you ask the data team some questions.
I think what we're left with is increasingly the hard questions. I don't think text to SQL will be perfect. I think there will always be scenarios where people get it wrong, but for me I think a better way for a median employee on a median topic to be able to answer 95% of their questions would be a great start.
Q: How do you manage the risk of users getting incorrect results from text to SQL?
Ian: I think a decent analogy here is sort of bumper rails. A lot of our tools do give you the ability to sort of softly nudge or aggressively nudge people in the right direction.
Most people at Ramp don't actually ever want to see transactions that didn't clear. An example would be if you put a credit card down in a hotel, $500 goes on your card, $500 comes off when you check out - it's just not really something we count. If you are a certain operator wanting to know something about declines or holds, you might need to answer that, but we nudge people very aggressively into starting their query with those filters on there.
I think the same thing is probably true for something like timestamps. We're like "Hey, we're going to nudge you very aggressively towards this timestamp. If you know the reason you don't want to use this timestamp, use one of the other 14."
Q: What are you most excited to build next with AI?
Ian: I am really excited to elevate Ramp's external data brand. When I think about Zillow, it's like you would never go to an open house without checking Zillow's estimate first. Zillow doesn't sell its estimates - Zillow sells buyer agents leads and they sell mortgage leads.
For us, we actually just hired an economist. We are just starting to work on our ramp.com/data. I do not want you to be able to spend money as a customer without checking to see how Ramp would recommend this happens first. There's a lot of AI to this, there's a lot of SQL to this as well. For the most part it is saying how can we take the context and documents around how a company spends, what the line items are, what processes they go through, who makes the decisions, and bundle that up in a way that's interpretable, lets companies benchmark.
I'll describe our travel product a little bit. We built it, we asked 50 friendly companies "hey, just give us your travel policies." We have a sense of the axes of a travel policy now. We built it to be configurable. Smaller companies come up to us, they're like "I don't have a travel policy - you're Ramp, just tell me what to do. If I IPO one day, I'm not going to think about how my travel policy got there."
That is something where we want to free people up to do work that is necessary and strategic. We want to attack anything where you want a median outcome to occur. You're buying Figma - great, median outcome, just make sure it works, get me a fair price, let me go do something that's strategic. That's the stuff that I think we're really excited about solving with AI.
Q: Do you have a thesis about whether AGI is going to happen in the next 5, 10, or 20 years?
Ian: The only thing I'm sure of is that we'll change the goalposts. I think there was a time period where I look back to when I graduated from college, and it was a time period where everyone could build on their MacBook a program in Python that could beat anyone in the world in chess, but Go was unsolved. And there was that interesting 15 to 20 year period like "chess, solved problem; Go, unsolved problem, we'll never beat Go." Then we beat Go. It'll be something else, we'll see.
I think it for me always comes back to you're always just predicting the median next token at the end of the day. That's what the math behind it is - if you need median outcomes. And that's not to say these things can't do incredible things, they obviously can. A lot of phenomenal PhD work or math proofs - they can clearly discover new things with median outcomes.
Credal gives you everything you need to supercharge your business using generative AI, securely.
