What Five Months of Building a Journey Analytics Prototype Has Taught Me So Far
By Trevor Paulsen
If you've been following my blog series, you know I've been experimenting to see if I (equipped with Claude Code) could build a real journey analytics product in my free time. I started from the architecture principles to syncing warehouse data to identity stitching and the AI assistant. I've been pretty thrilled with what I was able to put together - it works a lot better than I expected! I've arrived at a working prototype that connects to a warehouse, syncs data, stitches identities, runs fast journey queries, has a half-way decent user experience, and lets an AI assistant do a lot of the tedious setup and reporting for you.
But as any true PM knows, building a prototype end to end is the best way to discover where your assumptions are wrong. And with the help of a bunch of very generous people who have been willing to test what I've created, I've learned a ton! So in the spirit of sharing the messy middle and not just the polished wins, I want to walk through what I've learned so far, and where I'm headed next.
What still holds up
Before I get into what I'd change, a few of the original bets turned out to be good ones.
The instinct to treat the warehouse as the source of truth was right. Companies really don't want yet another vendor making independent copies of their data and becoming a second place where privacy rules and corrections have to be managed. Building on top of where the data already lives is still the right starting point.
Another bet that held up was making AI a foundational part of the experience rather than a bolt-on. The setup, configuration, and "help me build this dimension/segment" moments are really magical for people. My favorite moments of people testing Trevorwithdata were really just watching people realize how much an AI assistant can actually do when you give it enough tools and freedom.
I'd also say that ClickHouse as foundational reporting tech layer (and all of the optimizations I've detailed along the way for what we're doing) has been shockingly good for what I'm trying to do. Pretty impressive piece of open source software in my opinion.
So generally I'd say the foundation is solid. That said, there are a few things that have to change if this is going to be a real solution for people.
Lesson 1: A lot of companies don't want their data leaving their cloud at all
This is the big one, and it pretty much redefines almost everything else.
When I designed the prototype, I assumed most companies would connect their warehouse to the Trevorwithdata hosted service, and the analytics engine pulls the data it needs. I still think that model can work really well for many companies out there, however...
What I underestimated is how careful the industry is becoming about data leaving their own cloud, full stop. Across the board, organizations are getting more privacy and security conscious, and "your data never leaves your account" has gone from a nice-to-have to a hard requirement for a major chunk of the market. I also think AI is adding much greater incentives for companies to be cautious with their own data - most are (rightfully) hesitant to have external vendors use their data to feed AI model training or be used in ways that the company didn't intend.
The hard reality is that you can't satisfy that requirement by hosting the engine yourself, no matter how secure your setup is. The data has to stay put, which means the engine has to be able to come to the data and run inside the customer's own cloud. That's a fundamentally different deployment story than "sign up and connect your warehouse," and it's one I didn't design for at the outset.
Seems obvious in retrospect, but a learning for me nonetheless!
Lesson 2: Don't make people bring in a second vendor
In the warehouse syncing post, I leaned on a separately hosted ClickHouse instance as the query engine. ClickHouse is genuinely a fantastic technology for what we're trying to do here, but the tricky part is that you can either use ClickHouse Cloud (which runs into the problem above), or you have to lisence ClickHouse BYOC directly. This means that just to get Trevorwithdata working, a customer would have to take on yet another vendor for the product to work - this increases the setup complexity exponentially (more contracts, more endpoints to configure, more credentials, etc. etc.). A core principle we're chasing is that journey anlaytics tools should be easy to setup, which makes this approach untenable.
The lesson is that the engine should be bundled into the product in a way that the customer never has to think about it. Companies need a solution where ClickHouse is just an invisible part of the thing they're deploying, not a separate vendor they have to integrate with as a prerequisite.
Again, another one that seems obvious now ☺️.
Lesson 3: Modern warehouses do change-data-capture now, so lean on it
I was a big fan of Airbyte at the start, and it definitely helped me get something running quickly. I was thinking I could avoid building and maintaining connectors for the major warehouses so I could focus on the actual analytics product.
What I failed to see is how much the modern cloud data platforms have evolved. BigQuery, Snowflake, and Databricks have all been building out native change-data-capture capabilities, and those let you detect inserts, updates, and deletes much more efficiently and natively than using a separate technology to manage it. I also didn't want to have to lean on another vendor like I mentioned above.
The right way to do this is to use the native CDC capbilities built right into the warehouses directly. It's faster, it's closer to the source of truth, and it fits much better with the "keep the data where it lives" model.
Lesson 4: Pick infrastructure that can run anywhere
This one follows directly from Lesson 1. The prototype runs on Firebase for the backend and Vercel for the frontend, and both are great to build on. For getting a prototype live quickly, I'd recommend them to anyone.
The catch is that neither one can be deployed inside someone else's cloud account. They're managed platforms that live where they live, which is great until a customer says "that all needs to run in our cloud account."
So the lesson here is to choose infrastructure that's portable from day one if you ever expect to run inside a customer's environment. In practice that means container-based, orchestrated infrastructure that can run the same way whether it's in my cloud or another company's owned cloud, across AWS, Azure, or GCP. It's more work to set up than Firebase and Vercel, and for a prototype, that tradeoff wasn't worth it. For where we want to go next, it is.
Lesson 5: Design is the part I most want to push further
This last one is less of a technical correction and more of an ambition.
The prototype's design was the best I could do on my own, and I'm honestly proud of how far it got. User experience design has always been the missing piece in this project. So much of analytics tooling treats the interface as a place to display the data rather than a place that actively makes you better at exploring it, and I've always wanted to build something that genuinely pushes on that.
For a long time, professional-grade design felt out of reach for a solo project. That's the part that's changed. With the launch and evolution of Claude Design, AI has gotten good enough that I think I can finally build the analysis experience I've always had in my head, instead of settling for "good enough for a prototype," and this excites the heck out of me.
Where we go from here
Adding all of this up, the prototype did exactly what a prototype is supposed to do. It taught me where I was wrong cheaply, before I'd built too much on top of the wrong foundations.
So I've decided to take some time to retool. The goal is something that can be deployed two ways from the same codebase:
- As an easy cloud offering, the way the prototype works today. You sign up, connect your warehouse, and go, with nothing to install.
- Or inside your own cloud, running on Kubernetes across AWS, Azure, or GCP as a managed service, so data never has to leave the environment it sits in.
Same product, same engine, two deployment models, with the choice driven by how privacy and security conscious a given company needs to be.
And yes, it's also going to need a name that isn't "Trevorwithdata." 😅 More on that to come.
A heads-up on the cadence
Retooling all of this is a meaningful chunk of work, so I want to set expectations: posting here may slow down a bit while I rebuild, but I'll still share things as I go when there's something worth sharing.
The payoff I'm working toward is simple. On the other side of this, I want anyone to be able to sign up and start using it without having to fill out my survey first. That's the bar I'm setting for myself.
Before I finish this post, I want to say a real thank-you to everyone who has spent their own time playing with the prototype, kicking the tires, and sending me honest feedback. Several of the lessons above exist because someone was generous enough to tell me what didn't work for them. None of those folks had to do that, and it genuinely shaped where this is headed. Thank you. 🙏
If any of this resonates, whether you're wrestling with where to run your own analytics, thinking through the same data-residency questions, or just curious how the rebuild goes, I'd genuinely love to hear from you. Building this in a vacuum isn't nearly as fun as building it alongside people who care about the same problems.
Come say hi on LinkedIn, and I'll keep you posted as the retool comes together. 🚀
Trevor Paulsen is a data product professional at UKG and a former product leader of Adobe's Customer Journey Analytics. All views expressed are his own.