Branching databases like code: a CI/CD pattern for Lakebase, in production at Glaspoort

The problem we couldn't ignore

Glaspoort builds and operates fiber infrastructure in the Netherlands. Everything revolves around growing the number of fiber connections, and for a long time the data team spent its days building BI reports in support of that goal, while the question behind each report was already outdated by the time the report was finished. The result was a sprawl of one-off reports, and users who had nowhere to take their follow-up questions.

So we broke the dependency. Instead of shipping the next report, we built a custom front-end application in which project managers see directly where the opportunities for their projects lie. What is new sits under the hood: we use Databricks products directly as building blocks in the app. Genie, to chat with the data and spin up quick analyses. AI/BI Dashboards, for insights and self-service analytics. Automated workflows with Agent Bricks, which alert project managers the moment something stands out on their projects and Lakebase, the Databricks OLTP database, for the application's transactional data.

That combination brings together two worlds that until recently lived apart: the analytical environment and an operational front-end, where analytical data meets transactional data. The data team now spends its time on Genie spaces and metadata instead of one-off reports. But none of this stays fast without a serious foundation underneath: CI/CD, data-quality testing, Infrastructure as Code, and data governance. One piece of that foundation took the most careful design, and it is the piece the rest of this story is about: how we ship changes to the database behind the app.

Behind that application sits a Databricks Lakebase database. It is serverless Postgres OLTP, running next to the lakehouse rather than bolted onto it. The data flow is simple to describe and, as we found out, more interesting to operate than it looks:

• Curated data from the lakehouse is synced into a Lakebase production branch, where it lands in a read-only application schema.
• The application writes its own state back into a separate schema on that same branch, so the data read from the lakehouse and the data written by the app live side by side without colliding.
• On top of that we run three logical environments: development, acceptance, and production. We ship changes to this database the same way we ship changes to application code, through pull requests, CI, and gated promotion.

That last point is where the interesting question lives. The moment you decide an OLTP database deserves the same rigor as application code, you have to answer one hard question: 

How do we let every PR test against a database that looks like production, without people stepping on each other, and without losing the fresh data that makes the test meaningful in the first place?

The failure modes here are familiar to anyone who has shared a database across a team. PRs that pass in isolation and break when they land together. Dev and acceptance environments that have quietly drifted away from what production actually looks like. And the refresh day nobody wants, where getting clean data back means tearing environments down, re-wiring every connection string, and re-applying every grant by hand.

This post is about how we avoided most of that, and the one decision we are still actively debating.

Lakebase branching, in 60 seconds

If you have not used Lakebase branches yet, here is the only mental model you need for this post.

A Lakebase branch is a copy-on-write Postgres branch off a parent. Creating one does not copy the data; it forks the state cheaply and instantly, and the branch only diverges from its parent as you write to it. Each branch is fully isolated, with its own endpoint and its own data. You can create one in seconds and throw it away just as fast.

If that sounds like git, that is the point. The analogy that organizes everything below is simple: a feature branch in git maps to a database branch in Lakebase. A PR gets its own code branch and its own database branch, tests run against both, and when the work is good it gets promoted toward production.

There is one constraint to carry into the next section, because it is a critical distinction from git branching. To reset a branch from its parent, you first have to delete that branch's own children. A parent cannot be reset out from under the branches that depend on it.

The reset-from-parent trap

Here is the design most teams reach for first, because it mirrors the way we draw environments on a whiteboard:

• A single long-lived development branch off production.
• An acceptance branch off development.
• Feature branches off development.

It is a clean hierarchy: production at the root, then dev, then acceptance and features hanging beneath it. It also introduces two predictable failure modes: the branches drift, and the fix for drift is expensive enough that teams stop applying it.

First, dev and acceptance drift from production. Production keeps receiving fresh synced data and real application writes; the long-lived dev and acceptance branches do not. Within a sprint or two you are testing against a database that no longer resembles the one you are shipping to.

The obvious fix is to refresh dev from production, and this is where the constraint from the last section turns into a tax. To reset development from its parent, you must first delete development's children, which in this topology means acceptance and every feature branch hanging off dev. So a routine "give me fresh data" turns into a cascade: delete acceptance and all feature branches, reset development, re-create the environments, re-wire every connection string that pointed at the old branches, and re-apply every Postgres grant, because grants live on the branch you just deleted.

None of those steps is hard on its own. Together, on a recurring basis, they are a reliable way to make the whole team quietly avoid refreshing, which means everyone goes back to testing against stale, drifted data. That was the original problem. The naive topology does not just cost you a bad afternoon; it discourages the hygiene that keeps the environment honest.

The Glaspoort design: always branch from production

The fix is a one-line change in how you think about the topology, and it has outsized effects: every long-lived environment branch is a child of production, not of another environment.

Development and acceptance are both branches taken directly from production. They sit beside each other under production, not stacked on top of one another. Neither is the other's parent, so refreshing one never forces you to delete the other.

That single change defuses the trap. When dev or acceptance drifts, we reset it from production (a UI operation today), roughly once a sprint or whenever we want fresh data. Because nothing hangs below dev or acceptance, there are no children to delete first, no connection strings to re-thread across the team, and no grant re-apply marathon. The reset is cheap, so we actually do it, so the environments stay honest. If a reset happens to miss a change, the next migration replay simply applies it again.

Animated: the per-PR lifecycle. Plays on the published blog; shown as a still frame inside this doc.

The per-PR flow

The day-to-day developer loop layers ephemeral branches on top of that stable topology:

1. A developer opens an ephemeral PR (TTL set to 1 hour).
2. CI cuts a fresh, ephemeral pr-xxxx branch from production. We branch everything from production and never merge databases back. These per-PR branches are disposable: we archive them the moment the PR closes, which keeps us under Lakebase’s default limit of 10 unarchived branches per project.
3. CI replays the migrations against that fresh branch. A git diff check decides whether a migration rehearsal is actually needed, so PRs that don't change migrations skip the replay.
4. We don't just test the migration in isolation. CI deploys the new app image to a staging slot, points it at the freshly migrated branch, and runs the full test suite against that pair.
5. Every PR validates the migration and the new application image together, before either touches a real environment.
6. When the PR is merged, CI does it again. It re-branches from production, re-replays the migrations, and re-tests, because the world may have moved since the PR first went green. If the migrations succeed, they are then applied to the target branch, in place. For the first merge, that target is development.
7. Promotion to acceptance and then production happens through manually approved gates, each one replaying the migrations against the next branch in line.

One detail is worth calling out for teams coming from a git-centric world. We keep only a master branch in git. There is no long-lived develop or release branch in source control. The promotion path from dev to acceptance to production is expressed entirely through manually approved gates in Azure DevOps, which are what kick off CI and move a change from one branch to the next. The environment topology lives in Lakebase and the pipeline; git stays simple.

Why migration replay is the source of truth

The thing that makes all of this safe is that we never merge databases back into each other. We do not promote a change by copying data from a feature branch up to development. We promote it by re-running the migrations against the target branch. We use vitest to run the smoke test against the staging slot.

Branches, in this model, are deliberately short-lived and disposable. The durable, authoritative description of what the schema should be is not any one long-running database that might have drifted. It is the ordered set of migrations. That is why a reset is safe even when it misses something: if a refreshed branch is missing a change, the next migration replay applies it again. The migrations are central enough that the live databases are always reproducible from them, rather than precious because of them.

That is the real unlock. Branching from production keeps the data fresh. Treating migrations as the source of truth keeps the schema correct. Together they mean no environment is ever too special to throw away and rebuild.

The fork: two promotion models

This is the part we think is most useful to other teams, because it is a genuine fork rather than a best practice with one right answer. Once you have the topology above, you still have to decide when a PR is allowed to merge. We evaluated two models, and they trade off in opposite directions.

The short version: Option 1 optimizes for developer velocity inside a tight-knit team. Option 2 optimizes for independence and a clean hotfix path. If your team works on overlapping changes and trusts CI, stacking gets you moving fastest. If you frequently need to ship a fix that cannot wait behind whatever else is in flight, per-PR promotion pays for itself.

We chose Option 1. We are a small team on overlapping changes. Waiting for acceptance to go green before anyone can build on your work is a tax we'd pay every day, while the stacking risk only materializes occasionally. And when it does, it's contained: every promotion gate re-runs the stacked migrations against a fresh fork of production before touching the real branch, so a stack gets validated as a unit. For the genuinely urgent case we keep a separate crisis pipeline that goes straight to production with the same preflight. Velocity by default, escape hatch on standby.

Sidebars: the things that came up along the way

A few topics came up repeatedly in the working sessions. They are not the spine of the story, but they are the details that decide whether the pattern survives contact with production.

Database auth and token rotation. The app doesn’t hold a long-lived database secret. A static password in Azure Key Vault is used to mint a short-lived database credential (a token with a 60-minute TTL) via `databricks postgres generate-database-credential`. Because the token expires after an hour, the client has to refresh proactively; we handle this with an async password function on the connection pool that caches the credential and rotates it before it lapses.

Grants and access provisioning. We don’t manage table DDL through migrations, and object-level grants are applied by hand today rather than through code. It’s the clearest item on our list to improve. But because we apply those grants at production and environment branches are reset by re-forking from production, child branches inherit them automatically on every refresh. The manual grant step happens once at production, not repeatedly at every child, so it stays rare enough to live with for now.

Database access: a single application user. We don’t provision a Postgres role per end user. The app connects through a single application user, and authorization for individual users lives in the application layer rather than in the database. This keeps provisioning simple; the trade-off is that per-user access control is enforced above Postgres, not by it.

What this unlocked for Glaspoort

A use case that used to take a complete team and months of work now goes live with a small team in days. Our iterations have increased tenfold, and one team continuously delivers new value on a platform that grows with the organization.—Raymon Veldman, Business Control & IT manager

The most visible change is speed with less anxiety. Every PR now gets its own production-shaped database in minutes, so developers stopped coordinating around shared dev and acceptance environments, the "who broke acceptance?" conversations simply disappeared. Because environment branches are always children of production, we never hit a reset-from-parent day again: no deleting child branches, no re-wiring connection strings, no afternoons lost to re-applying grants. Migrations are replayed at every promotion step, and we deploy the new app image to a staging slot against the freshly migrated branch and run the full test suite before anything touches a real environment, so what reaches production has already proven itself twice. And when something urgent comes up, hotfixes travel the same safe path as everything else, just faster, instead of bypassing the pipeline.

Choosing Lakebase for an application database and wrestling with the same CI/CD questions? Start here: long-lived environments branched directly from production, ephemeral per-PR branches, and migrations as the single source of truth. Learn more about Databricks Lakebase