Jun

I run an automated video pipeline that generates images locally, on my Mac. A 16GB M4. The image model is FLUX, a 12B-parameter thing that keeps ~7GB of weights resident in memory even quantized down to 4bit.

I've frozen this machine before (lesson learned the hard way), so there are safety mechanisms in place. A pre-flight gate: don't even start generating if free memory is below 50%. And a watchdog: if free memory crosses into the danger zone mid-generation, kill the generation process — just that process. Never freeze the machine. That's rule one.

And the safety mechanisms worked. Twice. By killing my generation.

Killed. Zero images

The gate passes (51% free). Then the moment generation starts, free memory slides 51→7%, the watchdog goes "nope," and kills it. Zero images produced.

The machine is fine. The safety net is flawless. I just... can't make anything. Protected and completely unproductive at the same time. Ever been there?

Hypothesis 1: "Just lower the resolution" — wrong

The obvious first move: shrink the output size and retry.

...Same monotonic slide into the kill zone.

But that failure taught me something. The culprit wasn't the output size — it was the resident weights themselves. Changing resolution barely changed the memory drop, because 12B worth of weights lands in memory upfront, no matter how small the picture is. My first hypothesis died a quick, satisfying death.

Peeking at actual memory: the machine was already starving

Time to look at the real numbers instead of vibes.

PhysMem: 13G used (3.6G wired — the OS holds it and won't let go; 2.9G already compressed)
Actually free: 2.7G
swap: 7.8G in use

Enter fullscreen mode Exit fullscreen mode

At idle, this machine has no slack left. And my dev environment alone (editor plus its resident processes) eats ~3GB. Drop a 7GB model on top of that? Of course it overflows.

Which is where the classic advice lands: "Close your editor before running heavy stages." Great advice. Except the automation running this generation lives inside that editor. Closing it kills the thing I'm trying to run. A perfect deadlock.

The turning point: "wait, is 7GB negotiable?"

That question changed the whole direction. Is 7GB just what FLUX costs, or is there room to haggle?

I went back and reread every option the generation tool offers, and found the quantization level is selectable: 3, 4, 5, 6, or 8 bit. I was on 4bit. Meaning there was one more notch down — no extra download, the cheapest possible experiment.

(Full disclosure: my first attempt at this experiment failed because I mistyped the option name. Not a memory problem. A typo. In any debugging session, the thing that burns the most time is usually this exact kind of thing.)

Fixed the typo, ran 3bit with the editor open. ...It passed. Finished with 19% free at the lowest point. The first image ever produced without shutting down my own workspace.

Before writing the victory blog post: A/B the actual culprit

At this point I really wanted to write "3bit was the silver bullet!" But hold on. Right before that run, I had also closed a heavy browser. So was it the 3bit, or the freed memory? I hadn't isolated the variable.

So I deliberately reran the original 4bit, this time with plenty of free memory.

Condition Free at start Lowest free during run Result Peak usage (approx.)
4bit / no headroom 51% stopped at 7% killed ~9GB
4bit / headroom 71% 12% completed ~9.4GB
3bit / medium 62% 19% completed ~6.9GB

Now it reads clearly. The real culprit was insufficient free memory at start. Close the browser, raise the starting line from 51% to 71%, and even 4bit completes. Freeing memory was the biggest lever.

That said, 3bit measurably shaves ~2.5GB off the peak (9.4→6.9GB). Not strictly required — but the lowest-free point rises from 12% to 19%, which means it runs safely with less headroom, with more apps left open. And I couldn't spot any quality degradation by eye.

One bonus discovery: the 50% pre-flight gate was slightly too generous for this setup. If the real peak is ~9.4GB, surviving requires starting at ~57% free or better. The watchdog's two rescues were exactly this mismatch between the gate and the real peak. At 3bit the peak drops to ~6.9GB, and the 50% gate becomes honest again. That's the real argument for making 3bit the default.

What I learned

  • A system that refuses safely turns a freeze into a measurement. Because it killed the process instead of hanging, I got a lowest-free reading on every run. A frozen machine tells you nothing.
  • Kill the most plausible hypothesis (resolution) first, and kill it cheaply. A confirmed miss is still progress.
  • A/B before you write the victory post. If you changed two things right before the win, you will credit the wrong one. I nearly did — the main effect wasn't 3bit, it was freed memory.
  • The question isn't "is the model too big?" It's the gap between peak usage and free-at-start. You only see the real bottleneck after measuring both.

When local generation "mysteriously fails or dies," don't panic about model size. Measure the peak, measure the free memory, and look at the difference. That's where the culprit usually hides.