Featuring
Sponsors
Tiger Data – Postgres for Developers, devices, and agents The data platform trusted by hundreds of thousands from IoT to Web3 to AI and more.
Depot – 10x faster builds? Yes please. Build faster. Waste less time. Accelerate Docker image builds, and GitHub Actions workflows. Easily integrate with your existing CI provider and dev workflows to save hours of build time.
Augment Code – Adam loves “Auggie” – Augment Code’s CLI that brings Augment’s context engine and powerful AI reasoning anywhere your code goes. From building alongside you in the terminal to any part of your development workflow.
Framer – Design and publish in one place. Get started free at framer.com/design, code CHANGELOG for a free month of Pro.
Notes & Links
Chapters
| 1 | 00:00 | Let's talk! | 00:38 |
| 2 | 00:38 | Sponsor: Tiger Data | 01:40 |
| 3 | 02:17 | Start the show! | 00:28 |
| 4 | 02:45 | How are you, Alex? | 03:32 |
| 5 | 06:17 | Ok, this is the start for reals | 02:07 |
| 6 | 08:24 | Docker ot Podman? | 07:09 |
| 7 | 15:33 | Let's talk bootc | 01:07 |
| 8 | 16:40 | Rootless vs root | 00:12 |
| 9 | 16:52 | Texas Linux Fest!! | 03:54 |
| 10 | 20:46 | 50,000 subs (and counting) | 07:45 |
| 11 | 28:31 | Sponsor: Depot | 02:51 |
| 12 | 31:22 | Adam, how do you do your backups?! | 01:35 |
| 13 | 32:57 | Hetzner for backups? | 00:49 |
| 14 | 33:46 | zfs.rent | 03:27 |
| 15 | 37:12 | The trouble with BSD | 01:45 |
| 16 | 38:57 | Immich instead of Google Photos | 02:27 |
| 17 | 41:24 | From Red Hat to full-time content for Tailscale | 02:08 |
| 18 | 43:31 | Doing content as a job | 05:23 |
| 19 | 48:54 | How honest can you be?? | 02:28 |
| 20 | 51:22 | Alex's homelab (from gateway to clients) | 01:41 |
| 21 | 53:03 | I don't trust myself with k8s yet | 00:21 |
| 22 | 53:24 | Plex!!! | 05:44 |
| 23 | 59:08 | Physical media is off the shelf | 07:44 |
| 24 | 1:06:51 | Sponsor: Augment Code | 02:54 |
| 25 | 1:09:46 | Sponsor: Framer | 01:15 |
| 26 | 1:11:00 | Now we have to talk about AI | 07:32 |
| 27 | 1:18:33 | Alex built QuickSync Benchmarks with AI | 15:45 |
| 28 | 1:34:17 | Favorite Linux distro? | 02:08 |
| 29 | 1:36:25 | Declarative vs Bash script | 01:37 |
| 30 | 1:38:02 | The 4 DIMM problem | 02:06 |
| 31 | 1:40:08 | BBQ and Texas Linux Fest 2026!! | 01:52 |
| 32 | 1:41:59 | Watch Silicon Valley | 00:47 |
| 33 | 1:42:47 | Closing thoughts and stuff | 02:32 |
Transcript
Don’t mind me, I’ve got a little cough drop in my mouth there… I’ve got this nasal drip, so if I sound a little nasally, that’s why. But I do have hot water, a tablespoon-ish of honey, and a little lemon. So…
That’s what you need. A spoonful of sugar helps the medicine go down, so they tell me.
Wow… Gosh, that’s so good. So good. Alex, how are you, man? How’s life? You good?
I mean, you see me here in my attic studio in Raleigh, and we’ve had the house on the market for three, four months now… We’re trying to move back to England.
But nobody’s buying houses right now. So we’re kind of living in this perfect show home of a house, and –
Because you’ve got it all ready to sell… It’s always clean… It’s always clean, so you can show it…
You’re in the middle of a recording… You’ve gotta show it in an hour, “Gotta end the recording and go”, right? Is that how it works?
Well, London’s calling… Is this on air material, a little bit of it, or no?
Yeah. Well, if we don’t end up selling, we’ll stay. What choice do I have…
…if the place doesn’t sell?
It is what it is, right? I hate that phrase, “It is what it is…” I feel like it’s just such a – such a give up moment, you know what I mean?
You know, many years ago when I was working at the Apple store, we used to have these little bets on the genius bar about how our conversation would go from the opening line… And if someone used the phrase “What it is, is”, we’d be like “Uh-oh… This is going to be a good one.”
Oh… “Tell me your story about how your photos don’t sync to grandma’s device”, or “There’s offloading happening”, or “Where are my apps?”
I’ll tell you, there are only so many times you can reset someone’s iCloud password within the same 10-minute appointment without losing your sanity.
Yeah, that’s – how long ago was that for you? About a decade, or more?
iPhone 4 era, so… When was that? A long time ago.
Yeah, maybe. ’10, ’11, ‘12, maybe… I don’t know. Something like that. A while.
Well, I guess 2007 was the first one… So –
I was there when the iPhone 4 launched. I remember we had a… Because it wasn’t available in many –
I think so… It was certainly a really good one.
I mean, the current version is an iteration of the iPhone 4, right? It’s the same blocky feels…
Really? But I remember it wasn’t available in many countries at that point. So what ended up happening was we’d get a bunch of what we called resellers queuing up outside the store first thing in the morning, with thousands of pounds in cash in their pockets, buying as many iPhones as we could sell them, to ship them off to Dubai, or Abu Dhabi, or just… It was all Arabic customers that we had purchasing these things for resale. It was kind of crazy, actually.
Yeah. And that was back in London?
Yeah. Well, Manchester, but yeah.
Manchester. Forgive my lack of geography understanding of the United Kingdom. I just, I don’t know where things are at – I’ve never been there to know where things are at…
It’s [unintelligible 00:05:50.07] I say London to Manchester like it’s a big deal, but it’s probably no further than Houston to Dallas, in reality.
So like four or five hours, kind of thing?
Yeah. It’s pretty close, really.
Well, if you said Dallas, and I lived in Austin, I would say Austin, because… It’s Austin. Or Houston. So I get the correction. I get the correction. Interesting, interesting. Well, friends, if any of that made it into the show, which I imagine some of it might - maybe all of it, who knows…? This is my good friend, Alex. We’ve become friends over the years.
Big fan, as you know, Alex, of your podcast, that is no longer in place… Man, self-hosted is such a wild world these days, bro… Obviously, you know… I was the last – I don’t think the last time, but like one of the last times we talked, I was surprised by your creation of… What was it? Linux Server IO. Is that right? Is that what it was?
Gosh. And we found that out, and you were the very first entry in there, because obviously, you started it…
Yeah, it was my personal blog, and then it morphed into something else.
And now it’s like where the trusted images of all your favorite things as a home labber, that’s usually your source. Maybe if you’re running Plex in a container, you’re running that Plex container… I’m personally not; I’m using the official Plex media server.
Well, fun story with that one… Plex actually approached us back in the day to help them write that Docker container. So technically, you’re just running a fork of the Linux server container, technically speaking.
Technically. Okay… A derivative.
Yes. A derivative is more accurate. Yeah, it was a fun project, because you know, 10-15 years ago, whenever it was now - I’m getting old - there was just no standardization in the containerization space. People weren’t writing – if they existed, they weren’t writing coherent documentation at all, and there was no kind of standardized base image pattern, or anything. And there is some weirdness in how the Linux server images ended up architecting around S6 to have like this init system inside the container… Because obviously, if PID 1 dies, then the container dies with it… And some of the services need multiple things running inside the same container. So it’s a bit of a weird thing trying to containerize some of these apps that were written before containers were really the native deployment format for server software in general now.
[00:08:23.16] Yeah. How do you feel about the Podman/Docker war? Do you feel like it’s a war that’s won? Are you a Podman person? What’s your feels?
I’m not. You know, I’m ex Red Hat, so I really should be pro Podman, shouldn’t I?
But the thing that Podman misses for me is it’s a very purist implementation. So it’s extremely technically sound… What it misses for me though is some of the user spice on top. That last 10% that Docker closed in terms of usability. Because all the primitives for Docker pretty much were there before Docker came along. It was just the packaging of Docker with the standardized image format that really made it take off. The trouble with Podman though is when you want to do basic stuff, like even just mounting a volume, you’ve got to do this user ID shuffle, because –
I’m talking specifically about rootless Podman here. You’ve got to do this user ID shuffle, because the user IDs inside the container don’t map to the IDs on the host properly, unless you do this shift… And so it ends up being this world of complicated UID scripts, shuffling nonsense that I just haven’t got time for.
Huh. I guess I’m not that much of a Docker user to know exactly what you’re talking – I definitely mapped UIDs to UIDs, but I never really hit the issue. I do know if you mismatch on ownership there’s an issue, but that’s – I sort of leave it to the AI, or the Stack Overflows, or something else that’s like slightly above my pay grade.
Well, I’ll break it down as best I can, just really simply… So in a rootless Podman container, you end up shifting the user IDs by roughly 100,000 or so. It depends on how you’ve got it configured. So on your host, you would probably have a user ID of something like 1000.
Inside the container, it would be the user ID 100,000 and 1000. So like 101,000. So you’ve got to find a way to map those IDs from what the container sees to what the host can speak, so that the file permissions for those bind mount volumes actually work. And that whole mess – the fact I’ve just had to explain that to you is exactly my point.
Yes, I’m feeling your point.
With a daemon that Docker has running as root - yes, there are some security implications with that, but also comes simplicity for the user experience.
Right. This is the world of like potentially SE Linux, I think… Like, you maybe mentioned Red Hat… Does that play into it at all? Like, this whole purist security Linux thing that I’m sort of just being exposed to. I’ve just now moved to Fedora 43… First time I’ve ever ran Fedora as a desktop, even.
I’m loving it, honestly. It’s really cool. Traditionally, I’ve been an Ubuntu server kind of guy. I only really ever ran Linux as a server, never as a desktop. And so only the last few months have I actually taken the plunge and began to run a system that’s based on Linux. And honestly, Silicon Valley back there is running on Fedora 43, just so you know…
Alright. Good. Yeah. There’s some other interesting stuff that Podman does as well. They have this thing called the quadlets. Don’t know if you’ve heard of Podman quadlets.
It’s a bit of a silly name… But essentially, they’re SystemD units that let you run your Podman containers as SystemD units, instead of using something like Docker Compose to define the state of the world that container would see.
Okay. Yeah, so you have a .service file for it instead.
[00:12:03.10] Effectively, yeah. I know I give Podman a hard time, but it does do some stuff really well.
Well, I didn’t ask you to talk about Podman at length, but I’m really curious about this… So I think the one thing you mentioned and just glossed over, which may be super-important, which is rootless versus root… That seems to be the divider there, which is the big reason why people think they want to use Podman. It’s more secure because it’s rootless, versus root. That seems to cause problems based on what you’ve just said.
Well, I come at this from the home lab angle; the pragmatist angle of “I just need my Plex [unintelligible 00:12:36.07]
“It’s me, I’m secure”, right?
Whereas I think a lot of the motivation for rootless Podman and that whole movement in terms of Bootc, and a lot of the…
…containerization changes that are happening is because of the enterprise, where that stuff does matter, where SE Linux does matter, AppArmor does matter. But in my home lab, I don’t care.
“I don’t really need that stuff.” Yeah. I mean, even if you had an API locally that you were doing something with, no auth, right? Why do you have auth? No auth? Tiered auth systems. Like, if I’m external –
Yeah. Like, there’s no auth for an API I’m writing locally. That’s just causing problems. I don’t need to authenticate to my own API, in my own home lab. I can trust me, okay? I’m the admin here. We need a shirt that says “I’m the admin here.”
Yeah, it’s true. Yeah, “Look at me. I’m the admin here.” If anything though, my adoption of Tailscale over the last few years has made that stance of mine even stronger, because I have no ports, nada, opening my firewall anymore. Nothing.
Oh yeah, maybe, for the –
That’s my only port open.
…the advertisement of the server for friends, right?
Right. And we do external watching, for my kids mainly. I mean, I do have to watch Silicon Valley in a pinch, here and there, okay…? So I might be on the subway, or something like that, which - we have no subways here; I’m just making a story up. But I’ve gotta get my –
This is America, we don’t do subways. [laughter]
You know…? We don’t have subways around here… No, the ground’s too hard here in Austin to do a subway. Like, it’s rock under there. I mean, I think Elon may be trying to find a way to make a hole, but we’ll see.
My land is a trusted zone, completely.
Do you VLAN a lot then? I imagine you probably have an IoT VLAN at least, right?
I do. Yeah. So when some random piece of IOT junk arrives off Amazon, it’s not getting on the same VLAN as my server. That’s just not happening.
And the issue there is that it might be shipping software onto it that’s got a rootkit or something like that that could be trying to infiltrate… Or like [unintelligible 00:14:46.27]
You just don’t know. It could do that. It could just have an algorithm that crawls open SMB shares and lists the file directories.
Oh, gosh… That’s fearful.
I don’t know. I mean, who knows what these things are doing, or capable of? ESP devices over the years have gotten so capable… I watched a video just last night of a guy that built, using one of these ESP32s, a little display; like a Need for Speed kind of like mini map for his car… And he had like 2.5 million map tiles that he generated for the UK, and had petrol stations on it, and restaurants… It was really cool, actually. So they’re very capable devices, is my point. You just don’t know what they’re up to.
Gosh… Do we want to go to Bootc? I mean, do you have anything good to say about Bootc? What’s your thoughts on Bootc?
I think it’s a really interesting development. I don’t know a huge amount about Bootc technically, but in terms of where it fits in in the space, I think it’s a logical conclusion of where things should go next. The immutable atomic OS…
Are you able to describe Bootc, so that folks can follow? I’ll try, but I won’t do a good job. I’m gonna put it on you.
Yeah. Well, you go first and I’ll look it up.
[00:16:03.15] Okay. Uh, geez… What I know about Bootc - it’s out of Red Hat, it’s part of their world, essentially, and they’re setting this up… It’s essentially a system that can change itself underneath, and it’s… What’s the right word? You can’t change the system. It’s immutable. So what you have is what you have, and you can update underneath your data… But that’s the part I don’t get. And I know that Steam Deck, and some other gaming things are doing on top of it is really cool… I just don’t know what’s cool about it yet, aside from being able to build your own Linux distro for yourself, potentially. That’s kind of where I’ve been dabbling at. And when I say ‘dabbling’, just a moment, and I got scared and I ran away. That was all of it.
Did you go to Red Hat Summit this year, per chance?
I did not. But I did go to… Texas Linux Fest!
Oh… It’s the first one I haven’t been to, in maybe seven or eight years.
Oh, man… I think I reached out last year and I was like “Hey, I might be there” and I didn’t end up going. And me and a buddy went, and thank you to Carl George from Red Hat for the invite…
Yeah, Carl’s a good dude.
…and all the folks who are running that conference. It’s an amazing conference. If you’re even close to Texas during Texas Linux Fest, you’re missing out if you don’t go. Seriously, it was a blast. I was at two workshops, both amazing… It just is like one of those really feel-good, regional Linux conferences, so you get that regional feel, where it’s sort of small and intimate… But all the right people there to sort of still go deep into this world of Linux.
Did Carl share with you any of his pocket meat?
No. Does he have pocket meat?
[laughs] Yeah. Carl George… I met Carl for the first time - it must’ve been 2017 or ’18 at Texas… And he was “Hey Alex, how you doing?” You know, in the way he does… “Hey, Alex, would you like some of my pocket meat?”
He’s a very jolly person.
“You’re a complete stranger. Why are you offering me your meat?” [laughs] But it turns out he lives in San Antonio, and there’s this meat market near his house… And so he came out for All Things Open in October to Raleigh, and I literally placed an order with him to bring me vacuum-packed packages of San Antonio meat market meat to Raleigh, so that I could get my fix of Texas barbecue remotely.
Oh… Yes, Texas barbecue. Yes… Good old Texas barbecue. It is the best…
If I lived in Austin, I’d be the size of a house. I mean, it would be a problem, I think.
It would be a problem. It’s a problem around here, okay? Texas is big, and so are the people. People in Texas are big people. But not everybody… But some of them, pretty big.
I still need to make it out to something like Salt Lake, or something like that. I mean, I’ve done the Franklins, I’ve done the… What’s it called? Down by the river… Terry Black’s. You know, [unintelligible 00:18:50.11] I need to do some of the others.
If you came here, I would happily take you, and we can take Carl, too. He can come, too. You’re invited, Carl.
[unintelligible 00:19:04.28] but you’d find time.
I would probably – so I’ve been wanting to take him… We’ve been meaning to meet, Carl and I, at Salt Lake… Which - I live about 10 minutes from the original Salt Lake. So the one that began it all for Salt Lake. And it’s like going to somebody’s backyard ranch, and their house. It’s so cool that it’s not even like restaurant feels; it’s like somebody stood up a little barbecue in their house and it just started to morph over time.
Well, that’s probably exactly what it was.
Well, that’s exactly what it was. It was like “Hey, let’s make some barbecue. Oh, people come and eat it… Oh, cool. Let’s keep doing this.” And there you go. There’s nothing wrong with Salt Lake. They’re good. It’s not my favorite, though. But it’s still amazing. So you can’t go wrong. I would take you to The Switch, which is near here… I think it’s under-known, and… It’s just so awesome, man. They’ve got a really good platter, they’ve got an amazing chopped brisket baked potato… I mean, just like a little baked potato, with chopped brisket in it. I mean, come on… Is your mouth watering?
[00:20:15.17] I’m hungry now. We’re not far off lunchtime for me.
The Switch, I would take you there.
Yeah, that’s quite – I’m just looking at it on a map. That’s quite a ways outside of the city limits, too.
It is, but it’s really close for me. So I live about 10 minutes West of The Switch.
So I live in that area there. Okay, Texas barbecue, Texas Linux Fest… Bootc…
Well, we were talking about Bootc briefly…
Podman… Rootles… Well, we don’t have to go into Bootc. I’m more – you know what I’m interested in? I’m interested in the number. This number. Tell me if this number matters to you. Five zero, comma, zero zero zero. What does that mean to you?
50,000, right? Congratulations.
Thank you. Yeah, I think I know where you’re going with this, but…
That’s a feat. I mean, you’ve grown the Tailscale channel on YouTube from - I don’t know what number, but… 50,000 is nothing to shake a stick at. That’s awesome.
About 1,600 subscribers when I took over…
Does that make you happy? Does it make you sad? Do you feel content? Discontent? What are your feels right now when I say the number five zero, comma, zero zero zero?
You know, to some degree the subscriber count itself doesn’t matter that much… Because you can put out a video that gets 2000 views, and you can put out a video that gets 50,000 views, or 100,000, or whatever, and there’s just no connection between the number of subscribers to the number of views you’re going to get. But it is a nice vanity metric, and it is a nice thing to turn around to management and say “Look, we’ve got a big number. It now says 50,000.” It’s a nice signal, and I really appreciate everyone that has subscribed to the channel, of course.
It wouldn’t – I mean, I wouldn’t be able to put a roof over my head without people watching, so… It’s quite humbling in a way, but it also in some ways doesn’t matter that much. Like, maybe you understand what I’m trying to say about it…
I know what you mean, yeah. In the grand scheme of things, right?
Yeah. I also kind of just feel like we’re just getting started. I’ve just taken over as the head of developer relations for Tailscale about two weeks ago now… So we’re putting together a plan to build out a team to support me on the backend, because up until today it’s been just me. So I’ve been editing, scripting, filming… Doing everything.
Wow. That’s a lot of work.
So we’re putting together a plan, and there’ll be some job postings going up in the new year… So keep an eye on the Tailscale Careers site - a little pluggy plug. If you want to work on my team, come do some video editing, come do some dev rel stuff, it’ll be a good time.
Yeah. Well, I’ve been following, and actually, when I saw that number, I was like “Let me go check it out.” And I think you’re at like 52,000 now. So I think since that post, a few more thousand have subscribed. And then I saw a button there, and it said Subscribe, and I’m like “Gosh, Adam, you haven’t subscribed to this yet?” Which is so weird, because that’s how the algorithm works, which is probably why you say it doesn’t matter if you have 50,000 subscribers or not… Because I watch your stuff, but I’m not a subscriber.
And that’s a weird thing.
I kind of like that. I like as a consumer not having to subscribe to everything.
[00:23:37.23] Well, because I generally think that good content – there’s a bunch of tropes on YouTube, and the people that rely on the Like/Comment/Subscribe/Smash that Like button, all those kinds of tropes that exist on YouTube. Like, if the content is good enough, you don’t need to say any of that stuff. People watch people, they’re not watching the Tailscale channel. They’re just watching me and what I’m doing. You look at Linus and how he’s had to grow his channel, sort of beyond the cult of just Linus… And obviously, he’s way down the road from where I am, but… It’s really interesting, he’ll just make random cameos in videos that are hosted ostensibly by someone else on his team, and then he’ll just helicopter in for five minutes and say a couple of pieces to camera, and then go away again. And you know that he hasn’t done anything in like post-production, or scripting, or really been involved in the video at all, but they’re really struggling to enter that post PC era, that post Linus era. Hopefully, Tailscale won’t suffer from that problem, because I don’t really chase the fame aspect of it, or I don’t want to be famous, or anything like that. But I do want people to be as excited about stuff that I am as I am… And that’s really my driving goal, is to just – like this week, for example… I’ve set up a self-hosted image instance; you know, the Google Photos replacement.
Done by Alex and his team at Futo, and Lewis Rossman, in Austin, no less. So they’re on your doorstep, dude. So I set this up, and I was like “I can delete Google Photos now.” And I finally did it. I’ve actually finally deleted my quarter of a million pictures in Google Photos. It took me all weekend… You have to go through the web UI and select a maximum of like 10,000 images at a time. It’s –
Really, one million, for real?
Yeah, it was like a quarter of a million pictures. Yeah, it was quite a lot.
But Google makes it really difficult to leave. But anyway, I’m off the Google source, finally. It’s about time the self-hosted guy actually did that, but… I was like “Well–” I bought my family for Christmas when my daughter was born five years ago some of those little Google picture frames. And so we just had this shared album where I just dropped photos in every few months, and then it would update their picture frames remotely… And I’m like “Oh. Oops… I guess I’ve just broken that workflow for them.”
Just the favorites, basically.
Effectively. Yeah. And so I came across this project called Immich Frame. And this lets you turn any device – because it’s just a web view. So you can run it on Android, iOS, and Apple TV, and Android TV… Just a web browser even. And it lets you turn any of those devices into essentially a Google Photos frame. So I can just drop pictures into an Immich album on my self-hosted server, connect this Android tablet that I’m going to give to my family for Christmas back over Tailscale to my house, and the same functionality exists, but it’s all completely self-hosted and completely – I own the whole shebang. That’s what I love about this stuff. Getting people excited about that real world use case.
It’s like - yeah, Tailscale’s cool, Immich is cool, and all that… People don’t wake up in the morning and go “You know what I’m going to do? I’m going to sign up for a new VPN, and…” It’s like the “Now what?” problem. How do I actually use this stuff, make it useful?
Right. I think my usage of Tailscale is pretty simple. I think I just connect my services and I don’t do much, really… I’m not a deep Tailscale user, and I kind of feel bad. So I’m actually thinking about having Claude help me go deeper, because I’m just like – I don’t know how else I can use it personally… Aside from the obvious connected tissue of my home lab; that’s really just it. And it’s really just mainly – I’m gonna show my colors here a little bit, but I think for me I will SSH into Cineplex, which is what I call my Plex machine… So I just type “SSH Cineplex”, because that’s what it’s named in Tailscale. It’s the name of the machine, the hosting. That’s really - sadly, potentially; maybe you’ll be sad by this - my use of Tailscale, is just easier hostname SSH around my home lab.
[00:28:04.13] It’s great. It’s fine.
And the occasional external view of something of a service running. I don’t have a lot of – I could do better with that. I’ve got other problems I’m dealing with in my home lab, which is just writing a few more pieces of fun software for me…
You know, home-labbing.
Break: [00:28:22.09]
So how’d you do your backups, Adam?
Gosh, I don’t have backups, Alex… I’ve got Raid, okay? I’m just kidding… I really don’t have offsite backups, okay…? I’m not practicing 321. I’m in a situation I could – like, it could go down… Right in this moment, it could just like catch on fire over there, and I will have to say “Get the thing, get the extinguisher” and put it out, and my stuff is gone. So I don’t have backups.
Well, there goes one of my pitches for you… It would be an offsite backup.
[00:31:55.08] Well, that could be a pitch, though. I need an offsite backup. So what would you do? How would you do it?
Well, I have a friend in Canada… So I’ll tell you how I do it. And so I have a ZFS array in the basement, which has just got, I think, five 20-terabyte hard drives in it…
…and then I use – it’s a program called zrepl. Z-R-E-P-L, for zreplicate, I guess… And it essentially mirrors all of the ZFS blocks to Canada, which is where my friend lives. And he’s got a similar-sized server – he actually runs an MSP. So he just carved out a few user rack space for me to use there.
He’s a very nice friend. [laughs] I’m sure he would offer you some rack space if you paid him a few Canadian rubles every month…
And I just use Zrepl over Tailscale, because it’s an SSH-based protocol for the replication.
Could you do the same thing with Hetzner, by any chance? Like, I don’t know enough about Hetzner… I know you could do a lot of cool things with production machines, I know you can do some pretty decent sized builds, because you can choose your CPU and stuff like that… But do they have that level, of like ZFS (ZFS to me, because we’re different)?
My brain is so confused about that letter… Because it’s in my name, Alex KTZ. People always give me a hard time about that one, and I’m like “Ahh…”
Yeah… I’m not giving you a hard time. I’m just being clear that I say ZFS. That’s it.
[laughs] I just pick whatever.
Could you do that with Hetzner, though? Could you do that with like a – I don’t have the friend, I wouldn’t mind the friend… But for those who don’t have the friend, how could you do the same thing?
There’s a couple of ways. I mean, the trouble with any cloud provider is that once you get over a few hundred gigs of storage, the cost becomes prohibitive pretty fast. So there’s a service called - and I’m not affiliated with them in any way, but there’s a service called zfs.rent. Go take a look.
What you do is you send them a hard drive, and they put it into a slot in one of their servers - I think in California, but don’t quote me on that… And you pay them 10 bucks a month per slot that you occupy. So if you want 20 terabytes of offsite backup, you send them a 20-terabyte hard drive; it can be direct shipped from Amazon, or Best Buy, or wherever…
They’ll rack the drive for you, and then they’ll give you an IPv4 address and a VM, and then you use that as your ZFS replication target offsite. 10 bucks a month, job done.
That is good… That’s a good find. I should get them on the show. This is a really – I wish I thought about this. zfs.rent is the coolest domain ever… And then any notable person with RAID in their house has got to be running ZFS. And if it’s – I mean, if it’s ButterFS, sure. Okay. BTRFS.
There just aren’t many alternatives. I know Butter gets a bit of a hard time, perhaps unfairly, but.. ZFS, at this point, really – for me, there’s nothing that comes close in terms of the reliability, the development trajectory that it has, with the [unintelligible 00:35:10.21] Klara systems with Allan Jude in Canada as well… TrueNAS, of course, has been industry standard for sort of normal people for a long time…
Non-enterprise deployments, of like iSCSI storage, and… Anything you want to do for a small/medium business, whatever, TrueNAS is the answer. I just can’t think of anything better. BcacheFS is kind of promising, but then there was that whole drama recently with Kent Overstreet and Linus Torvalds in the Linux Kernel mailing lists and development trying to sneak some weirdness there…
Oh, go look it up. It’s a whole tranche of…
[00:35:55.26] Oh, gosh… That’s what we need, is more drama.
Well, Bcache got accepted into the Linux Kernel, which you would think would be an amazing thing, right? We’ve now got another file system native in the Linux Kernel… Because this is one of the downsides of ZFS, is the licensing drama. Are you familiar with all that?
Loosely… I mean, I know about the origination in Sun, and the accidental open-sourcing… I’ve talked to Matt Aarons before… We have a good friend at Oxide, and Bryan Cantril..l. So I kind of have some of the backstory of the Sun days, and… I guess not “Sundays”, like the day of the week, but S-U-N space days… Capital S, u, n.
Yeah, Sun Microsystems. And it’s got an interesting history. And I think even the accidental open-sourcing of it was really wild, and then now you have OpenZFS… I don’t know – and Allan Jude came on the podcast and talked about, and really got me wanting to become a BSD nerd… And I tried for a little bit; I just did enough to be familiar… But I just was like “Okay, Linux is really for me.” I just don’t need what BSD has. And it felt like an uphill battle for me, personally… But I get it. I get it. But the licensing thing is really where it’s at. The issue, at least.
The trouble with BSD is I learned Linux first, so all my muscle memory is Linux.
And just stuff like the way that grep works, or sed works, or just basic command line utilities like that… It’s just different enough in BSD land that I’m like “I’ll just go back to what I know.” There’s a lot to like about the BSD universe. There really is. But it’s just never stuck for me.
Yeah. So offsite backups, zfs.rent… Unaffiliated. I don’t have any experience with it, so if you check it out, y’all… Your mileage may vary. Report back, of course…
The other option you’ve got - it’s just a Raspberry Pi with a USB hard drive, and shove it under the stairs at your mom’s house, or something, you know?
So how does that work then? I mean, ZFS is so RAM-intensive… But I guess for a backup you don’t need a lot of RAM, right?
Is that the idea? It’s just a backup, so the arc can be small.
You are not going to be limited by the performance of the Raspberry Pi, you’re going to be limited by the performance, presumably, of your internet pipe.
That’s going to be the bottleneck.And even if you do end up being limited by some local thing on the Raspberry Pi, it’s a backup. Does it matter? It’s going to run overnight, every night, for the next 5, 10, 20 years, whatever. So if it takes an hour or it takes six hours, who cares?
I have to do this. You’ve given me a Christmas present. Thank you, Alex. This is my new –
This is what motivates me, dude. It’s just, like, solving real problems for people. So I’m glad to hear that.
I like that. Okay. So Immich is what you stood up… You’re off of Google Images, if I understand correctly. There’s no favorites list, like (I presume) in Google anymore; you’ve gotten a whole new hosted system that is Immich all the way… And your family is still happy with their Google frames.
The family - one of the family texts me this morning and said “My photo frames stopped working.” And I’m like “Oops…” I might have pulled the trigger a bit too soon on the deletion, but… I just wanted to stop paying Google eight bucks a month for the two terabytes, you know?
Yeah. How many hard drives did you buy?
Well, for eight bucks a month, amortized over five years, I can buy probably two 20 terabyte drives.
Yeah. It’s not the end of the world… I mean, there is some value there that you don’t have to manage it, but now you’ve got your friend; you’ve got your friend in Canada…
[00:39:49.18] Well, I’m also not training Gemini’s future models with my images, and the images of my daughter. There’s a huge digital sovereignty aspect to my motivations for getting off the big tech train when it comes to really deeply personal data like photos and documents. I mean, I’m sure many of us use Gmail even, and I’ve been a Gmail user since, I guess, 2004, 2005 era, when it was cool and a beta product. And I look back at the original emails, I’m like “Oh, my God…” Every email I’ve had in the last 22 years, they know what it is. It’s a lot of data. It’s a big dataset.
You get paid to run a home lab, though. Would you say that? Is it fair to say you get paid to run a home lab, for the most part, or play with a home lab?
Well, you get paid to make videos…
The phrase “If you do what you love, you never work another day in your life” is very true for me. I’m extremely privileged and lucky to get to do what I do. Thanksgiving weekend was just a couple of weeks ago as we record, and I decided that I was going to learn Kubernetes over the weekend.
And I spent the whole weekend glued to my laptop, and I had a great time, and I’ve now got a three-node Kubernetes cluster in a closet, and it works great. And then I was able to turn that into a piece of content for work. So for me, I’m just very privileged. This is what I’d be doing anyway… So the fact that I get paid to do it is just a huge bonus for me.
Yeah. Can we maybe touch quickly on – I guess, as long as you’d like to, actually; I said ‘quickly’, but however… These are the fields that I think get exposed through vibe-based podcasting, let’s just say. I’ve got this vibe that going to Tailscale from being a podcaster – I don’t know what else you did then, so I’m not going to diminish what you were doing. I don’t know. I know you were running the podcast, then you got the job at Tailscale… What were you doing before that? Were you consulting, or…? What I’m trying to get to is how was it to go from podcaster, that I knew you of at least, and then now you work for Tailscale to produce content there.
I was a Red Hat consultant in Europe for a y
0 Comments
Log in to join the conversation.No comments yet. Be the first to share your thoughts.