Building a Streaming, Low-Allocation Excel Writer in .NET

Introduction

If you've ever worked with Excel generation in .NET, you've probably encountered the same pain point: generating a simple .xlsx file allocates tens or even hundreds of megabytes of memory. For a 10,000-row dataset with four string columns, popular libraries like EPPlus or ClosedXML routinely allocate 30–100 MB. This problem only compounds when you're serving file downloads in a web API under concurrent load.

This article walks through the key techniques I used to build a zero-dependency Excel writer that stays under 1 MB of allocation for the same 10,000-row workload and completes 8–10× faster. We'll cover OOXML internals, streaming strategies, IBufferWriter<T> integration, and real-world benchmark comparisons.

Why OOXML?

An .xlsx file is just a ZIP archive containing XML files. The structure is well-defined:

• [Content_Types].xml – Content type declarations

• _rels/.rels – Package-level relationships

• xl/workbook.xml – Workbook definition (sheet names)

• xl/_rels/workbook.xml.rels – Workbook relationships

• xl/styles.xml – Cell styles

• xl/sharedStrings.xml – Shared string table (optional)

• xl/worksheets/sheet1.xml – Sheet data

• xl/theme/theme1.xml – Theme definition (optional)

This means we don't need any third-party library to generate a valid .xlsx file. We just need to:

1. Write XML parts to a ZIP stream

2. Handle the OPC (Open Packaging Convention) relationship structure

3. Manage cell data, styles, and shared strings efficiently

Technique 1: Direct-to-Stream Writing

Most Excel libraries build an in-memory DOM first, then serialize it. ClosedXML, for example, builds an entire XLWorkbook object graph with IXLRow, IXLCell, styles, formulas, etc. — all in memory — before writing.

A streaming approach flips this: write XML directly to the output stream as data arrives. No intermediate object model.

Here's the core idea in simplified form:

// Traditional DOM approach: everything lives in memory
var workbook = new XLWorkbook();
var sheet = workbook.Worksheets.Add("Sheet1");
sheet.Cell(1, 1).Value = "Header";
// ... add 10,000 rows
workbook.SaveAs("output.xlsx");  // only now does serialization happen

// Streaming approach: write as we go
using var zip = new ZipArchive(stream, mode);
var sheetEntry = zip.CreateEntry("xl/worksheets/sheet1.xml");
using var sw = new StreamWriter(sheetEntry.Open());
sw.Write("<?xml version=\"1.0\"?>");
sw.Write("<worksheet xmlns=\"...\"><sheetData>");
foreach (var row in data)
{
    // Write row XML directly — no intermediate objects
    sw.Write($"<row><c t=\"inlineStr\"><is><t>{Escape(row.Name)}</t></is></c></row>");
}
sw.Write("</sheetData></worksheet>");

This pattern eliminates the entire in-memory object graph. The tradeoff is that you lose random access (no modifying cell B4 after writing row 100), but for export scenarios, you never need it anyway.

Technique 2: IBufferWriter<byte> for Low-Allocation Paths

.NET has a lesser-known gem: System.Buffers.IBufferWriter<byte>. It lets you write bytes directly into rented pooled buffers, avoiding intermediate byte[] allocations and copies.

Here's how it works in practice:

// Traditional: allocates a byte[]
var ms = new MemoryStream();
Xlsx.Write(ms, data);
byte[] result = ms.ToArray();  // another allocation + copy

// IBufferWriter: writes to pooled buffers
var writer = new ArrayBufferWriter<byte>();
Xlsx.Write(writer, data);
// writer.WrittenSpan is a ReadOnlySpan<byte> over rented memory
// No intermediate allocations

The IBufferWriter<byte> path is the primary low-allocation hot path. It chains through ZipArchive → DeflateStream → XML writer, all using IBufferWriter<byte> under the hood, eliminating allocations at every layer.

This is also the technique that System.Text.Json uses under the hood for its high-performance serialization — the same pattern applies to XML generation.

Technique 3: Inline Strings vs. Shared Strings Table (SST)

OOXML supports two ways to store string values:

Inline strings (t="inlineStr"): The string value lives directly inside the cell element. No shared strings file needed. This is optimal when strings are mostly unique.

Shared Strings Table (t="s" with index): Strings are stored once in xl/sharedStrings.xml, and cells reference them by index. This is optimal when strings are highly repeated (e.g., status values, categories).

Most libraries default to SST (or build it unconditionally), but SST is wasteful when strings are unique — you double the memory: once for the SST, once for the cell references.

My approach uses a heuristic: pre-scan the first 64 rows. If string deduplication ratio is below 70%, stay with inline strings. Only switch to SST when it actually saves space. This is similar to what Excel itself does when saving files.

// Auto-detect based on data characteristics
Xlsx.ToBytes(data, p => p.WithAutoSst(true));

// Benchmarks on 10k rows with 16 unique strings (96% dedup):
// SST path:  2,804 μs, 955 KB  — dedup saves XML space
// Inline:    2,933 μs, 979 KB  — inline avoids SST bookkeeping
// For unique strings, inline is both faster and smaller

Technique 4: Fluent Export Profiles

Rather than a verbose configuration object, a fluent API keeps the code readable and the allocation minimal:

// Fluent profile — no intermediate configuration objects allocated
var bytes = Xlsx.ToBytes(orders, p => p
    .Sheet("Orders")
    .Column(x => x.OrderNo, c => c.WithName("Order #").WithWidth(30))
    .Column(x => x.Amount,  c => c.WithFormat("0.00").WithFontColor("#0000FF"))
    .Ignore(x => x.InternalId)
    .WithFreezeHeader());

// Traditional (ClosedXML) — configuration mixed with data manipulation
using var wb = new XLWorkbook();
var ws = wb.Worksheets.Add("Orders");
ws.Cell(1, 1).Value = "Order #";  // mutating a DOM object
ws.Cell(1, 1).Style.Font.Bold = true;
// ...

The Action<ExportProfile<T>> callback is compiled into a delegate once (via Expression.Compile or source-gen fast paths with [XlsxExportable]), and the configuration is applied during streaming without allocating intermediate objects.

Technique 5: IAsyncEnumerable<T> for True Streaming

For web API scenarios where data comes from an async source (database, gRPC, event stream), IAsyncEnumerable<T> enables true end-to-end streaming:

// Database → stream → HTTP response — no materialized list
app.MapGet("/export/orders", async (HttpContext ctx, AppDbContext db) =>
{
    ctx.Response.ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
    await Xlsx.WriteAsync(
        ctx.Response.Body,
        ReadOrdersAsync(db),  // IAsyncEnumerable<Order>
        p => p.Sheet("Orders"));
});

async IAsyncEnumerable<Order> ReadOrdersAsync(AppDbContext db)
{
    await foreach (var order in db.Orders.AsAsyncEnumerable()
        .Where(o => o.Status == OrderStatus.Active))
        yield return order;
}

This means the HTTP response body stream is written concurrently with the database query. At no point is the full dataset materialized in memory. Contrast this with:

// Traditional: load all data, then write
var orders = await db.Orders.Where(...).ToListAsync();  // materialized in memory
ClosedXML...SaveAs(stream);                              // writes from memory

The API is symmetric — reading also uses IAsyncEnumerable:

await foreach (var order in Xlsx.ReadAsync<Order>(stream))
{
    await db.Orders.AddAsync(order);
}

Note: IAsyncEnumerable reading streams rows as they're parsed from the ZIP/XWL, but the actual XML cell parsing is synchronous CPU work. The async benefit is primarily on the I/O side.

Real-World Benchmarks

All benchmarks run on an Apple M4 with .NET 10, comparing against MiniExcel, ClosedXML, EPPlus, and raw OpenXml SDK. The benchmark code is available in the companion repository.

10,000 rows × 4 string columns

10,000 rows × mixed types (string + number + datetime + bool)

Key takeaways

• Speed: 4× faster than MiniExcel, 8–10× faster than EPPlus/ClosedXML

• Memory: 30–100× less allocation than alternatives

• Gen2 GC: Minimal or zero Gen2 collections vs. competitors that trigger Gen2 on every iteration

The 37–88× allocation difference isn't a small optimization — it's the difference between a server that can handle thousands of concurrent exports and one that falls over under GC pressure.

When to Use Each Approach

Limitations

No single approach fits all use cases. The streaming writer has known tradeoffs:

• No random access: Once a row is written, you can't go back and modify it. This is fine for export, not for interactive editing.

• Reader doesn't evaluate formulas: Read<T> reads cached values, not formula results.

• No ZIP64: The underlying ZIP writer doesn't support ZIP64 yet — files must fit within ZIP32 limits (4 GB raw, effectively ~2 GB compressed for large workbooks).

• Formatting is output-only: You specify styles declaratively; there's no DOM to programmatically modify after creation.

Conclusion

The streaming, low-allocation approach to OOXML generation is not inherently complex — it's just applying well-established .NET patterns (Span<T>, IBufferWriter<T>, IAsyncEnumerable<T>) to a problem that's traditionally been solved with DOM-style libraries. The result is an 8–10× speed improvement and 30–100× less memory allocation, without sacrificing the type safety and fluent API that .NET developers expect.

The techniques described here — direct-to-stream XML writing, IBufferWriter<byte> pooling, SST heuristics, and IAsyncEnumerable integration — are transferable to any XML-based format and serve as a general template for high-performance .NET I/O.

The code described in this article is available as Magicodes.IE.IO on NuGet (MIT licensed). The benchmark suite that produced the numbers above is in the same GitHub repository.