Why I self host my servers and what I've recently learned

I do self-host my mail, and I've done so since about 2005! It gets harder and harder to with every passing year. Some notable things that have changed since then:

Many ISPs now block inbound port 25, required to receive mail via SMTP. It's quite hard to get an ISP to unblock this. My university wouldn't at all, and I left a laptop under my parents' couch for four years to do it instead. Some time later, Comcast began blocking it as well, and the only way to get it unblocked was to call support, work your way up the phone tree to someone that realized you were talking about inbound rather than outbound (no, this is _not_ a misconfiguration in Outlook), and get them to push a special config to your cable modem, which would be reset whenever another config was auto-pushed, or your modem lost power. You may notice that implies extended downtime when Comcast, my electric service, or my physical operations (read: I unplugged it by accident) suffers a failure.

Many mail servers (e.g. Gmail) require you to have reverse DNS that matches your forward DNS. Getting your ISP to understand what they're asking you to do is... difficult. The last time I changed ISPs, it took about a week to get this done. Comcast batches these updates weekly, and support wanted to double, triple, and quadruple-check that what I was asking for was, indeed, what I was asking for.

There are a bunch of anti-spam measures in effect that use DNS: SPF and DMARC are table stakes for most mail servers (again, e.g. Gmail) to speak to you. I've so far managed to get by without setting up DKIM, but I suspect that's next.

The worst part, by far, is spam blacklists. Many blacklists will already have your IP address listed by policy - you're a _residential IP_, not to be trusted. The Spamhaus PBL, for instance, automatically blocks all Comcast residential IPs. There is nothing you can do about this, and many mail servers will refuse to speak to you if you're on a blacklist.

These days I am paying Comcast an arm and a leg for business-class service, which both gives me unbridled inbound port 25, and also a (luckily!) clean IP on block lists.

A Linux server (e.g. stock Debian) on a well-reputed VPS is pretty secure by default, in my experience. Use software packages from the Linux distribution whenever possible (certainly for email software) and configure unattended security updates.

Note that you generally can’t host email from a residential IP, so you’ll probably want to use a VPS. Making services on your home network publicly accessible (i.e. not just via VPN) obviously comes with more risks; personally I wouldn’t do that.

I would echo others here and just use a cheap VPS to experiment with. Then you have much less to worry about.

How technical are you?

One thing I strongly agree with you here is being open to the cloud. Self hosting strongly favors running on your own hardware, but indie hosting focuses more about the tangible benefits, ie data ownership, mobility which breeds competition, etc.

That said, I think the VPS marketplace is still too complicated. What about updates, backups, TLS certs, domains, etc?

Hard to take this product/company seriously when every piece of copy on their site feels toxic, condescending, or dismissive.

Umbrel[0] maybe? I posted a list of related services here[1] as well, though most of them are cloud based.

[0]: https://umbrel.com/

[1]: https://forum.indiebits.io/t/open-identities/500/12

and learn on the way

In case you're interested in resources for this: I think _Learn Enough Developer Tools to Be Dangerous_ is a great start. I've been guiding my roommate through it as he studies, a chapter a week. If you do pick it up, just skim the chapters on editors— their examples are overly specific to a choice of editor of few outstanding strengths, and if you prefer a different editor you may struggle to find equivalents for some of the hokey examples the book uses.

https://www.oreilly.com/library/view/learn-enough-developer/...

Special editions/compilations of Linux magazines can also be a very good source of high-quality tutorials, including for CLI stuff. These are nice because while they include general introductions, they're mostly comprised of bite-sized tutorials that you can pick and choose according to your interest. I also like them because they're available in print, colorful, and shiny, and thoughtfully laid out, plus there are no ads— very pleasant compared to the web in many ways. Linux Pro Magazine did one on shell topics this year, back in February: https://www.linuxpromagazine.com/Resources/Special-Editions/...

Such magazines also include step-by-step guides for setting up services that I was able to follow (with some trepidation!) when I was just a kid who was new to Linux and still honestly a bit scared of the command line. Linux Format is really good for this because it's targeted at desktop Linux and computer hobbyists broadly rather than rather than programmers or IT professionals. Their guides assume little to no familiarity with the command line, so they often include reminders of little bits of command line basics rather than just assuming you share that context with the authors: https://linuxformat.com/

Besides web-based management interfaces for servers, like Proxmox, you might consider getting started by just running some services on a spare desktop computer. openSUSE has a long history of emphasizing GUI administration tools, so many relatively 'advanced' tasks for it do not require the command line, which is somewhat different from other distros. (If you give it a try, its GUI configurator, YaST2, will strike you at first glance as having a dated look. This is intentional— continuity is a priority for YaST, so GUI-based tutorials from many years ago will still be accurate.) It's also a distro with good guts and nice CLI tools, so you won't necessarily outgrow it after you get your feet wet with the command line.

Yes there are. I would suggest going through the subreddit mentioned /r/selfhosted. There are GUI tools and NAS products that would let you host docker images. As for CLI ask an LLM, for simple common commands you are going to be dealing with they are pretty good at it.

Definitely worth just sitting down and learning how the command line works. Its not as scary as it looks.

No need to have any fancy comp-sci background, hell I have an arts degree!

As far as turnkey solutions go coolify.io is the one I’ve seen floating around recently.

Fascinating! What didn't you like about caddy?

In NixOS/Guix System there is no need of such package, the configuration language/package manager takes care of anything, configuration included.

Let's say you want Jellyfin?

    jellyfin = {
      enable = true;
      user="whatyouwant";
    }; # jellyfin

    paperless = {
      enable = true;
      address = "0.0.0.0"; 
      port = 58080;
      mediaDir = "/var/lib/paperless/media";
      dataDir = "/var/lib/paperless/data";
      consumptionDir = "/var/lib/paperless/importdir";
      consumptionDirIsPublic = true;
      settings = {
        PAPERLESS_AUTO_LOGIN_USERNAME = "admin";
        PAPERLESS_OCR_LANGUAGE = "ita+eng+fra";
        PAPERLESS_OCR_SKIP_ARCHIVE_FILE = "with_text";
        PAPERLESS_OCR_USER_ARGS = {
          optimize = 1;
          pdfa_image_compression = "auto";
          continue_on_soft_render_error = true;
          invalidate_digital_signatures = true;
        }; # PAPERLESS_OCR_USER_ARGS
      }; # settings
    }; # services.paperless

    chromium = {
      enable = true;
      # see Chrome Web Store ext. URL
      extensions = [
        "cjpalhdlnbpafiamejdnhcphjbkeiagm" # ublock origin
        "pkehgijcmpdhfbdbbnkijodmdjhbjlgp" # privacy badger
        "edibdbjcniadpccecjdfdjjppcpchdlm" # I still don't care about cookies
        "ekhagklcjbdpajgpjgmbionohlpdbjgc" # Zotero Connector
        # ...
      ]; # extensions
     
      # see https://chromeenterprise.google/policies/
      extraOpts = {
        "BrowserSignin" = 0;
        "SyncDisabled" = true;
        "AllowSystemNotifications" = true;
        "ExtensionManifestV2Availability" = 3; # sino a 06/25
        "AutoplayAllowed" = false;
        "BackgroundModeEnabled" = false;
        "HideWebStorePromo" = false;
        "ClickToCallEnabled" = false;
        "BookmarkBarEnabled" = true;
        "SafeSitesFilterBehavior" = 0;
        "SpellcheckEnabled" = true;
        "SpellcheckLanguage" = [
                           "it"
                           "fr"
                           "en-US"
                         ];
      }; # extraOpts
    }; # chromium

NixOS improves the reproducibility of both self-hosted software and configuration state.

However, most self-hosted software is already "pre-packaged" in Docker containers. It's much easier to grab that "off-the-shelf" than have to build out something custom.

Imo, the quality and documentation level of an application's build system is a valuable signal in determining its overall health and competency. It usually (though not always) ends up being that well-maintained software written for modern runtimes is very easy to build from source and run. Even if I do end up using the developer's container image, I generally want to check out their manual deployment documentation.

Yes by design, meaning you do not have an FHS structure where "installing" means copy this in /usr/share, this in /usr/lib ... Alla packages just see a small "virtual" root tree, meaning a network of symlinks.

You can choose for package A to se package B and C, but not D, different versions means simply different packages because you have package alone, let's say the current stable version but also package_major_minor for various supported major or minor version. As well as you can choose to fetch sources from a specific commit from a public repo, build them and link them in the system.

Every package is not "isolated" in the container self, like small userlands on a common kernel, but in the sense of "having different views of the common system", so you avoid wasting gazzilion of storage and ram and CPU plus anything is still from the upstream or yourself, there are no outdated forgotten deps left around.

Plus being anything configured in the config (well, not mandatory, but that's the typical way) you have a fully reproducible system, maybe not binary-reproducible meaning "packageA now is version x.y" when you rebuild the system, but anything could be rebuild in the current state of nixpkgs/guix states from few kb of text, typically versioned in a repo. Every update does not overwrite anything, if put some new versions in a special tree /nix/store o /gnu/store and updates symlinks accordingly, if it does not work you restore the previous state instead/before garbage collection.

Practically it do the same of IllumOS with IPS (the package manager) integrated with zfs (and the bootloader), here instead of zfs clones and snapshots you have a poor man's version with symlinks.

There is no isolation by default, meaning it's a single system, but you can "portion" the system in various ways. Of course on Linux there are no IllumOS zones of FreeBSD jails, the state of paravirtualisation it's very behind those unices.

The first, yes.

The second, you can do this by setting certain options in the systemd service for your apps, but this is true for any systemd distro.

Does it seamlessly handle the case where you have 3 different apps that all require 3 different version of python with different python dependencies?

Ohh yes! That is exactly what it does on a repeatable and documented way..

But it also does a lot of work to do this. And doesn't hide the systems of that so much. So that means you're likely to have yo care about how it does all that work. And if you start doing something weird, might have to do a lot of work yourself..

So you prefer manually handling such containers, that you probably do not really know enough, perhaps even for serious usage, instead of writing a derivation? No reproducibility etc

For quick testing something, nothing to say, but for real, albeit personal homelab, usage...

Unfortunately, you can’t avoid having your information sold by the DMV in California. Likely others too.

How this is legal is beyond my comprehension.

Depends on the city/state laws but vast majority of the time, it's all public domain under FOIA laws. City/State won't say that because it's assumed.

So that should be an indicator that the government offices should just change their internal policies if people are going to go around them anyways.

automatic transcript generation

There is a neat application called "Whisper" which will translate/transcribe just about any media format, locally on your computer.

But I guess that's only necessary if bodycam footage isn't uploaded to YouTube (wow!).

obnoxious things the officer said

My hope is for your swift and proper resolution, whatever the charges/incident. They're allowed to lie/bait/entrap; glad you got the footage.

are all going to be voodoo to my spouse

That's why you really need to rethink your 'if you are hearing this I musta croaked' procedure.

Thing is, 99% of the files on your NAS and whatever never would be accessed after your death. And anything of importance should be accessible even if you are alive but incapacitated but your NAS is dead.

So the best thing to do is to make a list of Very Important Documents and have it in the printed form in two locations, eg your house for the immediate access and someone's parents who are close enough. And update it every year, with a calendar reminder in both of yours calendars. You can throw a flash drive[0] there too, with the files which can't be printed but you think they have a sentimental value.

[0] personally I don't believe SSDs are for the task of the long term storage, but flash drives I've seen to survive at least 5 years

Data recovery instructions can be documented on paper in the same physical location used for financial accounts, e.g. fireproof safe, trusted off-site records, estate attorney. These recovery instructions are also required for data hosted by third parties.

No

Thanks for the additional detail.

Your blog post inspired a discussion on "Coding on iPad using self-hosted VSCode, Caddy, and code-server" (80+ comments), https://news.ycombinator.com/item?id=41448412

It's really not that hard to run a server with a handful of applications on it, especially for personal/home use. It seems to me that when developers talk about this kind of thing they often massively overstate how much work is involved in running a server and keeping it up to date while massively understating the inherent complexities of working with cloud services.

I haven't really witnessed the cloud serving as a big time saver in my career. Cloud-centric ops teams seem to be consistently larger than those running applications on regular, shmegular servers (whether rented or their own), if anything.

For self-hosting purposes, I'd expect serverless deployments of open-source apps to take more time for most people to figure out and get right than just running the same apps on a VPS, unless you had spent years deploying to the cloud at work and also never learned Linux usage basics. And if you deploy to a only a single cloud VM, you're just using an extremely overpriced VPS at that point.

I’ve found the opposite. Google drive costs me less than a VPS, and far less than a physical server to do the same job.

Can you share the web applications and databases you serve from your google drive? since you can "do the same job" and it costs less? You must be into something here.

Ignoring any privacy concerns for a moment. Neither the comment you're replying to, nor the submission is talking about just storing documents in a storage. They're hosting applications, databases, services which you need a "server" for and can't do in Google drive.

You might not the have the same use case, you might not need to host websites, databases, virtual machines, DNS, and other services, but then you're not doing "the same job". You just have a different use case. Just like you don't need an IDE if you just want to take notes.

Do you have any more details such as a blog post on what you did?

Refer to https://news.ycombinator.com/item?id=41242945

For years I just uploaded a lump sum and it was spent as I used the service per year. This way I didn't need to bother what I would be without email when the paid period is over and I didn't need to overpay much in case I would need to cancel early.

Now I need to be sure I be around with a working CC at the time the bought period is over... and what if won't? Do I need to jump through cancel-and-reorder hoops every couple of years? More importantly, am I sure what a couple years later these hoops would work or through some very unlucky coincidence it would wipe my decade+ emails? Sure, Fastmail guys would be so sorry but... that wouldn't help me.

The 5G T-Mobile stuff in my area has often had latency at least as competitive as most cable providers in the area. I've had friends do cloud gaming on it no problem.

Or you build golden images with Packer + Ansible or the like, and swap images out every N months. The work to get everything automated and configured can be a bit daunting, but I much prefer the end result over doing continual upgrades.

If the feedback loop is tight enough, anything is possible! Every crash becomes a dopamine rush!