A city guide, a fares table, a list of opening hours: if it is published from data, someone has decided when the checking happens. Some teams look after publishing, some during, some before. This comparison sets the three common options side by side so a small team in Chennai, or anywhere else, can pick one without a long debate. The framing is drawn from the Ops Atlas reference on DataOps and padded out with ordinary working experience.
The three options at a glance
- Option A: publish, then watch. New data goes straight to the live table, and monitors alert you if something looks off.
- Option B: build and test in place. A tool builds the table and runs tests; if a test fails, later steps are skipped.
- Option C: write, audit, publish. Fresh data is parked out of sight, inspected, and swapped over the live version only after it clears.
None of them is wrong in every case. The skill is matching the choice to what a bad table would cost you.
Option A: publish, then watch
What you gain. Speed and simplicity. There is no extra storage and no waiting; data appears as soon as it is ready. For pages where a wrong figure causes a brief annoyance, this is a sensible default.
What you risk. Readers see the problem before you do. The Atlas proposes tracking how many incidents are first found by consumers, and option A pushes that figure up by design. The entry also cautions against relying on a job status alone, since a run can end cleanly with the wrong contents. Monitors must therefore be well designed. Limits built from a mix of weekdays and weekends miss real faults, and a fixed threshold on seasonal numbers fires either too often or never.
Good fit. Exploratory dashboards, scratch analysis and anything in the Atlas lowest tier.
Option B: build and test in place
What you gain. Tests run on every build, and downstream steps stop when something fails. It feels safe, and for many teams it is a large step up from option A.
What you risk. The Atlas points to a subtle catch in dbt. A failed test makes the build skip everything that depends on the failing model, yet that model itself was already rewritten in the target schema by then. Anyone querying that table directly reads the bad data. So the protection covers downstream models, not the table that failed. If a booking page reads straight from that table, it is exposed.
Good fit. Important but not critical datasets, where a short exposure is tolerable and the readers go through downstream models rather than the raw output.
Option C: write, audit, publish
What you gain. Bad data never reaches readers. If a blocking check fails, the new batch is quarantined and yesterday’s version remains in place. Under the Atlas advice you build into a distinct audit area and release as a separate step, which neatly closes the gap noted under option B.
What you pay. Time and storage. The entry notes that this approach adds latency, and that publishing by copying can double the writes. It also needs someone to define which checks are blocking, which means thinking carefully about what really matters.
Good fit. The few outputs that drive money, safety or reputation. The Atlas reserves it for tier one.
A quick comparison table in words
- Speed: A is fastest, B is close behind, C is slowest.
- Protection of readers: A is weakest; B protects downstream models; C protects everyone.
- Extra cost: A has none, B has test run time, C has test time plus duplicate storage.
- Effort to set up: A is trivial, B is moderate, C is the highest.
How to choose without overthinking
Ask three questions about each table:
- If it is wrong for a day, who is harmed and how badly?
- How fast do readers need new data?
- Can we afford the extra step?
Then assign tables to tiers. The Atlas uses critical, important and exploratory. Put C on the first, B on the second, A on the third. If you are not sure which tier a table belongs to, start it in B and revisit after your first incident log review.
A local analogy
It is like picking how to travel across the city. The autorickshaw is quick and flexible (A). The suburban train is dependable but runs on rules (B). The pre-booked car with a confirmed pickup costs more and takes planning (C). Sensible travellers mix all three. If you are laying out your own day, ideas for a fun, memorable Chennai outing, an answer on how long Chennai deserves and a first-visit planning guide for Chennai help you choose well.
Rollback and the history question
Whichever option you choose, think about undoing a mistake. Option C keeps the previous version untouched, so rollback is a matter of not swapping. Options A and B overwrite in place, which means a fix may require a rebuild from source. The Atlas lists rewriting history without snapshots as an anti-pattern, so keep dated snapshots of anything critical before you correct it.
Costs to keep in mind
One more factor is how often and how widely you test. The Atlas gives an example where a daily uniqueness test on a 90 GiB column scans about 2.6 TiB a month under on-demand billing, whereas a partition-scoped version scans about 2.5 GiB. Whichever option you pick, scope your tests to new partitions and run the full-history ones less often.
Source
The comparison follows the Ops Atlas explanation of DataOps, with the labels A, B and C being my own shorthand. Pick one table this week, decide its tier, and match it to an option. You will learn more from one real decision than from three more articles.