Point-in-Time Recovery Without the Enterprise Price Tag

description: A deep dive into HuskHoard's open-source PITR engine — how the catalog tracks version...

Dev.to · Jul 25, 2026

description: A deep dive into HuskHoard's open-source PITR engine — how the catalog tracks version history and byte offsets across physical media, and how you roll back a single file without touching the rest of the volume.

Point-in-Time Recovery Without the Enterprise Price Tag

If you've priced out Veeam, Commvault, or a Zerto license recently, you already know the punchline: granular point-in-time recovery is one of the most aggressively monetized features in enterprise storage. Vendors charge for it because it genuinely solves a hard problem — and because buyers are often in enough pain that they'll pay whatever it costs.

HuskHoard is an open source datatiering archive for Linux that ships PITR as a first-class feature, built directly into its catalog engine, free under AGPL v3. This post is the deep dive that the project's docs don't yet cover: how versioning actually works under the hood, how the catalog tracks byte offsets across physical media (disk, LTO tape, and S3), and how you roll back a single file without touching anything else on the volume.

Versioning vs Backup: Stop Using the Words Interchangeably

Before getting into mechanics, it's worth being precise about terminology, because these two words describe fundamentally different operations with different tradeoffs.

A backup is a complete redundant copy of a dataset at a point in time. The goal is full-loss recovery. If your server catches fire, you restore the backup. The cost is that you're duplicating data — 10 TB of active data requires at least 20 TB of total storage to have one backup copy.

Versioning (or PITR at the object level) is a historical record of how a specific file changed over time. The goal is granular rollback — recovering from an overwrite, a corruption, or a ransomware event that targeted specific files. It doesn't guarantee you can recover from total hardware loss unless your versions live on separate media. But it lets you say "give me /etc/nginx/nginx.conf as it was three weeks ago" without restoring your entire backup set.

The confusion is expensive. People who treat their archive as a backup get burned when the archive goes down (there's no redundant copy). People who treat their backup as a versioning system get burned when they need to roll back a single file and discover they have to restore 2 TB of snapshot data to get at it.

A solid DR strategy uses both:

Scenario	What You Want
Server catches fire / disk fails completely	Backup — full restore from a redundant copy
Developer overwrites critical config file	PITR — roll back that specific file in seconds
Ransomware encrypts 10,000 files	PITR — roll back affected files to pre-infection version
VM guest OS corrupts itself	Backup — restore the whole VM image
VM config file or snapshot is accidentally deleted	PITR — roll back just that object

HuskHoard handles versioning (PITR at the file/object level) natively. For full redundancy, pair it with N-way replication across multiple volumes, which HuskHoard also handles.

How HuskHoard's Versioning Engine Works

The Append-Only Storage Model

HuskHoard's architecture is built on a core constraint borrowed from tape: all writes are append-only and sequential. Even if you're archiving to a spinning disk image or an S3 bucket, HuskHoard writes to the volume as though it were a tape — from the current write head forward, never punching holes or overwriting previous data.

This constraint, which exists primarily to avoid the "write wall" problem on SMR drives and to stay compatible with physical LTO hardware, turns out to give you versioning entirely for free. When a file is re-archived, the old payload stays exactly where it was on the volume. The new version gets appended after everything else. Nothing is overwritten.

The catalog is where this becomes useful.

The Catalog Schema — The Version Column

Here is the actual schema from database.rs, as discussed in The Catalog post:

CREATE TABLE IF NOT EXISTS catalog (
    id               INTEGER PRIMARY KEY AUTOINCREMENT,
    file_path        TEXT    NOT NULL,
    version          INTEGER NOT NULL,
    tape_uuid        TEXT    NOT NULL,
    tape_offset      INTEGER NOT NULL,
    payload_size     INTEGER NOT NULL,  -- uncompressed byte count
    compressed_size  INTEGER DEFAULT 0,
    compression_type INTEGER DEFAULT 0, -- 0=None, 1=Zstd
    uid              INTEGER NOT NULL,
    gid              INTEGER NOT NULL,
    posix_mode       INTEGER NOT NULL,
    archived_at      DATETIME DEFAULT CURRENT_TIMESTAMP,
    original_mtime   INTEGER NOT NULL,
    blake3_hash      TEXT    NOT NULL,
    custom_metadata  TEXT
);

CREATE INDEX IF NOT EXISTS idx_path_version ON catalog (file_path, version);

Enter fullscreen mode Exit fullscreen mode

The key is the (file_path, version) pair. Every time the Janitor archives a file that has been modified since its last archive event, it does not overwrite the previous row. It inserts a new row with version = previous_version + 1. The old row — pointing to the old payload at its old tape_offset — stays in the catalog permanently (until pruned by max_versions).

Because the old payload is still physically present on the append-only volume, that old row is a complete, restorable snapshot of the file at the moment it was archived.

file_path                        | version | tape_uuid | tape_offset | archived_at
---------------------------------+---------+-----------+-------------+---------------------
/data/projects/config.yaml       |       1 | a1b2c3... |        4096 | 2026-04-01 09:00:00
/data/projects/config.yaml       |       2 | a1b2c3... |    52432896 | 2026-04-15 14:22:11
/data/projects/config.yaml       |       3 | a1b2c3... |   104857600 | 2026-06-01 09:30:45

Enter fullscreen mode Exit fullscreen mode

Three archive events. Three catalog rows. Three independently restorable versions. Zero additional tooling required.

Rolling Back a Specific Version

From the CLI:

# Restore the latest version (default behavior)
huskhoard restore --file /data/projects/config.yaml

# Roll back to a specific version
huskhoard restore --file /data/projects/config.yaml --version 2

# List all available versions for a file
huskhoard versions --file /data/projects/config.yaml

Enter fullscreen mode Exit fullscreen mode

The versions subcommand shows you what's available:

/data/projects/config.yaml
  v1  2026-04-01 09:00:00  blake3: a3f1...  44 KB   [tape: a1b2c3..., offset: 4096]
  v2  2026-04-15 14:22:11  blake3: 7c90...  47 KB   [tape: a1b2c3..., offset: 52432896]
  v3  2026-06-01 09:30:45  blake3: d81e...  51 KB   [tape: a1b2c3..., offset: 104857600]  ← current

Enter fullscreen mode Exit fullscreen mode

Under the Hood: The Mechanics of a Byte-Level Rollback

This is the part that separates HuskHoard's approach from snapshot-based alternatives — and it's worth understanding in detail.

Step 1: Catalog Lookup

When you run huskhoard restore --file /path/to/file --version 2, the first thing that happens is a single indexed SQL query:

SELECT tape_uuid, tape_offset, payload_size, compressed_size,
       compression_type, blake3_hash, uid, gid, posix_mode, original_mtime
FROM catalog
WHERE file_path = '/data/projects/config.yaml'
  AND version = 2;

Enter fullscreen mode Exit fullscreen mode

The (file_path, version) composite index makes this microsecond-fast regardless of how many millions of files and versions are in the catalog.

This query returns a tape_uuid and a tape_offset — a complete physical address for the version's payload.

Step 2: Volume Resolution

The tape_uuid isn't a device path. It's a 16-byte UUID burned into the volume's header block when it was first formatted with huskhoard format. This is critical: it means the restore doesn't care that your LTO drive moved from /dev/nst0 to /dev/nst1, or that your disk image was renamed, or that your rclone remote was reconfigured. The daemon resolves the current physical location of any UUID by scanning tapes table entries against live hardware on startup.

SELECT device_path, backend_type
FROM tapes
WHERE tape_uuid = 'a1b2c3...';
-- Returns: /dev/nst0, local

Enter fullscreen mode Exit fullscreen mode

Step 3: Seek to the Exact Byte Offset

Once the device is located, HuskHoard seeks directly to tape_offset. For disk images, this is a standard lseek(). For physical LTO tape, this uses hardware SCSI positioning commands to reach the correct block without reading through intervening data.

Volume [a1b2c3...]
 ┌─────────────────────────────────────────────────────────────────────┐
 │ Header │ v1 payload │ other files... │ v2 payload │ v3 payload │... │
 │  4KB   │  offset:   │               │  offset:   │  offset:   │    │
 │        │  4,096     │               │  52,432,896│ 104,857,600│    │
 └─────────────────────────────────────────────────────────────────────┘
                                             ↑
                               seek here for version 2

Enter fullscreen mode Exit fullscreen mode

No other version's data is read. No other file's data is read. The restore is surgical.

Step 4: Decompress and Verify

The payload at that offset is a Zstd-compressed stream. HuskHoard decompresses it and feeds the output bytes through a BLAKE3 hasher simultaneously. When decompression completes, the computed hash is compared against the blake3_hash stored in the catalog at archive time. If they don't match, the restore is aborted and the corrupted block is flagged.

This means every version restore is automatically integrity-verified. You're not just getting the bytes that were there — you're getting cryptographic confirmation that those bytes are exactly what was archived.

Step 5: Atomic Placement

The restored bytes are written to a temporary file, ownership and permissions are set from the catalog's uid, gid, and posix_mode columns, and then the file is atomically renamed into place at the target path. The previous stub (or the current version, if one existed) is replaced atomically — no partially-written state is ever visible to the OS.

The `object_frames` Table: Partial-Content Rollback Without Full Recall

For large files, there's an additional optimization. When HuskHoard archives a file, it writes the Zstd stream in discrete frames (typically 16 MB each) and records the mapping of uncompressed byte ranges to their compressed positions in the object_frames table:

CREATE TABLE IF NOT EXISTS object_frames (
    file_path           TEXT    NOT NULL,
    version             INTEGER NOT NULL,
    uncompressed_offset INTEGER NOT NULL,
    compressed_offset   INTEGER NOT NULL,
    compressed_size     INTEGER NOT NULL
);

Enter fullscreen mode Exit fullscreen mode

This is primarily what powers the StreamGate HTTP gateway (letting Plex or VLC seek into a 100 GB video on tape without full recall), but it also means that targeted restores of large files don't require reading the entire payload from media. If you need to verify the state of bytes 4,000,000,000 through 4,016,000,000 of a 2 TB database dump at version 5, HuskHoard can seek directly to the relevant frame, decompress only that 16 MB chunk, and verify just the portion you need.

This is fundamentally different from snapshot-based PITR in backup tools, which typically require mounting an entire snapshot volume before you can access any individual file within it.

Real-World Strategies

Strategy 1: File Versioning for Granular DR

The most common use case. You have a home lab or a small team environment where files change regularly — code, configs, media project files, database exports. You want the ability to say "give me that file as it was two weeks ago."

Setup: Point HuskHoard's hot tier at your working directories. Set a max_age_days appropriate to your workflow (7 or 14 days is typical). HuskHoard will archive files that haven't been touched since the threshold, and re-archive modified files, building up a version history automatically.

# husk_config.toml
hot_tier = "/home/user/projects"
primary_volumes = ["/mnt/archive/main.img"]
max_age_days = 14
janitor_interval_secs = 3600
max_versions = 10  # keep the 10 most recent versions per file

Enter fullscreen mode Exit fullscreen mode

Recovery scenario: Your web app's database.conf got overwritten by a botched deployment script at 3 PM on a Tuesday. You want the version from Monday.

# See what versions exist
huskhoard versions --file /home/user/projects/myapp/database.conf

# Output:
#   v1  2026-05-20 08:14:00  blake3: 9a2f...  1.2 KB
#   v2  2026-05-27 11:55:33  blake3: 4d8c...  1.3 KB   ← Monday's version
#   v3  2026-06-03 15:01:44  blake3: f1a0...  1.3 KB   ← broken deployment

# Restore Monday's version directly to path
huskhoard restore --file /home/user/projects/myapp/database.conf --version 2

Enter fullscreen mode Exit fullscreen mode

Total recovery time: seconds. You don't need to find last night's backup tape, mount a snapshot, or navigate a backup UI. You query the catalog, seek to a byte offset, decompress, verify. Done.

Strategy 2: VM Backup + File Versioning for Faster DR

This is where the versioning-vs-backup distinction pays off in a real operational context.

If you're running KVM or Proxmox, your VM disk images (.qcow2 or raw) are just files. You can absolutely archive them with HuskHoard, and every re-archive creates a new versioned copy that you can roll back to. But a full restore of a 500 GB VM image is still going to take time, even with direct seeks to the right offset on tape.

A better hybrid strategy:

Back up the full VM image periodically (weekly or on major changes). This lives as a versioned archive in HuskHoard — your safety net for total VM loss.

Version the individual files that change frequently and are small enough to restore in seconds: VM configs, cloud-init seeds, network configuration files, snapshot manifests, Terraform state files, Ansible inventories.

# Archive the full VM disk image (becomes version 1, then 2, then N)
# HuskHoard handles this automatically when the Janitor runs
ls -lh /var/lib/libvirt/images/production-web.qcow2
# 487G production-web.qcow2

# But for the config files alongside it:
huskhoard versions --file /etc/libvirt/qemu/production-web.xml
#   v1  2026-04-01  blake3: ...  12 KB
#   v2  2026-05-15  blake3: ...  13 KB
#   v3  2026-06-10  blake3: ...  14 KB  ← current

# VM XML config accidentally deleted or corrupted?
huskhoard restore --file /etc/libvirt/qemu/production-web.xml --version 2
# Restored in < 1 second

Enter fullscreen mode Exit fullscreen mode

The result: when something breaks, you reach for the right tool at the right granularity. Config went wrong? PITR the config file. VM guest OS corrupted itself? Restore the full image from the latest VM image archive. You're never force-multiplying a small problem into a large recovery operation.

DR time comparison for "someone edited the wrong network bridge config":

Approach	Recovery Time
Restore full VM from last night's backup	45 minutes to 2 hours
Mount last night's VM snapshot, extract file	10–20 minutes
HuskHoard PITR the specific XML file	< 5 seconds

Strategy 3: Defending Against Ransomware

Ransomware targets active files. A ransomware event that encrypts your working data will create a wave of "modified" files that, if your archive system isn't careful, will get archived as new versions — burying the pre-encryption versions under the corrupted ones.

HuskHoard's max_versions config is your safety net here. As long as max_versions is set high enough that the pre-infection versions haven't been pruned, you can roll back every affected file individually.

# After discovering ransomware encrypted files on June 10th at 14:00

# Find all files modified after that timestamp that have prior versions
huskhoard versions --file /data/documents/proposal.docx

#   v1  2026-05-01 10:00:00  blake3: 3c8a...  45 KB  ← clean
#   v2  2026-06-10 14:03:22  blake3: 9f1b...  45 KB  ← encrypted

# Roll back to clean version
huskhoard restore --file /data/documents/proposal.docx --version 1

Enter fullscreen mode Exit fullscreen mode

The append-only storage model is what makes this safe. The encrypted version was appended to the volume — it didn't overwrite the clean version. Both are present, both restorable.

For a large-scale ransomware event affecting thousands of files, you'd script the rollback:

# Roll back all files in /data/documents/ to their version 1
huskhoard restore --directory /data/documents/ --before "2026-06-10 14:00:00"

Enter fullscreen mode Exit fullscreen mode

Pruning Old Versions

Versions accumulate on append-only media. The max_versions config key controls how many versions per file are retained in the catalog (and eligible for restore):

max_versions = 10  # Default: keep the 10 most recent versions per file

Enter fullscreen mode Exit fullscreen mode

When a file exceeds max_versions, the oldest version's catalog row is deleted. The underlying payload bytes on the storage volume are not immediately freed — they become "dark" space that gets reclaimed the next time huskhoard repack is run on that volume (the garbage collection pass).

This means you always have control over the disk-vs-history tradeoff. More versions = more history = more storage consumed on the archive tier over time. For most file versioning use cases, 5–10 versions is sufficient.

What Enterprise Vendors Charge for This

Just for context. Veeam's granular file recovery requires their Enterprise Plus tier. Commvault's IntelliSnap point-in-time recovery is licensed per agent. Zerto's journal-based PITR (which tracks continuous block-level changes for arbitrary-timestamp recovery) starts at thousands of dollars per month for production workloads. IBM Spectrum Protect (formerly TSM) charges per terabyte of managed data.

HuskHoard's PITR engine — the catalog, the version rows, the byte-offset seeks, the BLAKE3 verification — is in the open-source core, licensed AGPL v3. You can read every line of the implementation in src/database.rs and src/daemon.rs.

The tradeoff is that HuskHoard's versioning is event-driven (archive events) rather than continuous (block-level journal). You get snapshots at the moments the Janitor ran, not arbitrary-timestamp recovery for every write in between. For most use cases — homelab, small teams, media workflows, development environments — that's entirely sufficient. For continuous block-level CDP of a high-transaction OLTP database, you need a dedicated database tool for that layer.

Getting Started

# Clone and build
git clone https://github.com/huskhoard/huskhoard.git && cd huskhoard
cargo build --release

# Grant capabilities (no root daemon needed)
sudo setcap cap_sys_admin,cap_dac_read_search+ep target/release/huskhoard

# Format an archive volume and start the engine
./target/release/huskhoard format --tape-dev my_archive.img
./target/release/huskhoard daemon

# After files are archived, list versions
./target/release/huskhoard versions --file /path/to/your/file

# Roll back
./target/release/huskhoard restore --file /path/to/your/file --version 2

Enter fullscreen mode Exit fullscreen mode

Full documentation is on GitHub. The architecture series on HuskHoard.com/blog.html covers the catalog schema, file stubbing, the fanotify interceptor, and SMR drive handling in depth — good reading if you want to understand the full system before deploying it.

Summary

Versioning and backup are different tools for different failure modes. Use both, not one instead of the other.
HuskHoard's append-only storage model gives you versioning for free — old payloads are never overwritten, so every archive event is inherently a new restorable version.
The catalog is the version index. A (file_path, version) row with a tape_uuid + tape_offset physical address is all you need to restore any specific version of any file in microseconds.
Rolling back a single file is a single seek. No snapshot volumes to mount, no backup sets to restore, no staging areas. Seek to the offset, decompress, verify the BLAKE3 hash, place atomically.
The object_frames jump table means even large files can be partially restored without reading the entire payload from media.
The whole engine is open source, free under AGPL v3, built in Rust, and runs in user-space on any Linux kernel 5.1+.

If you're evaluating backup or archiving software and PITR is on your requirements list, it's worth spinning up HuskHoard before you sign any enterprise contracts.

HuskHoard is an open source data tiering archive. Source code at github.com/HuskHoard/HuskHoard. Project site and architecture blog at huskhoard.com.

Read the original source

Point-in-Time Recovery Without the Enterprise Price Tag

Point-in-Time Recovery Without the Enterprise Price Tag

Versioning vs Backup: Stop Using the Words Interchangeably

How HuskHoard's Versioning Engine Works

The Append-Only Storage Model

The Catalog Schema — The Version Column

Rolling Back a Specific Version

Under the Hood: The Mechanics of a Byte-Level Rollback

Step 1: Catalog Lookup

Step 2: Volume Resolution

Step 3: Seek to the Exact Byte Offset

Step 4: Decompress and Verify

Step 5: Atomic Placement

The object_frames Table: Partial-Content Rollback Without Full Recall

Real-World Strategies

Strategy 1: File Versioning for Granular DR

Strategy 2: VM Backup + File Versioning for Faster DR

Strategy 3: Defending Against Ransomware

Pruning Old Versions

What Enterprise Vendors Charge for This

Getting Started

Summary

0 Comments

The `object_frames` Table: Partial-Content Rollback Without Full Recall