bit-habit · platform infrastructure
One small server hosts every bit-habit.com website. This page explains how,
in layers: read the first line in 10 seconds, or keep going for an hour. Stop whenever you
know enough.
Each site is checked from the outside every 60 seconds. Full status page → status.bit-habit.com.
Loading live status…
One small server runs every bit-habit.com website. You type an address,
it finds the right app, and sends the page back.
bit-habit.comstartpage.bit-habit.comblog.bit-habit.com
wiki.bit-habit.complane.bit-habit.com… ~20 sites
Want to know how it picks "the right app"? Keep reading.
startpage.bit-habit.comFollow one request from your browser to the page. Six steps.
*.bit-habit.com name points to one server IP: 158.180.71.122.http (port 80), Traefik
sends you to secure https (port 443). It's a permanent redirect (status 308).startpage.bit-habit.com
and forwards the request to the matching app.The server runs k3s, a small version of Kubernetes. Kubernetes is just a manager that starts apps, restarts them if they crash, and routes traffic to them. Here are its parts, in plain terms.
| Part | Think of it as | What it does |
|---|---|---|
| Traefik | the front door | receives every request on ports 80 / 443, adds HTTPS, and sends it to the right app |
| Ingress | the routing list | the table that says "this address → that app" |
| Service | a stable inside address | a fixed name for an app, even after it restarts |
| Pod | the running app | the container doing the actual work |
| cert-manager | the certificate robot | gets and renews the HTTPS certificate automatically |
| k3s | the manager | a small Kubernetes that starts, restarts, and watches every app |
And here is how those parts connect for a single app — a Deployment keeps Pods running, a Service gives them one address, an Ingress maps the hostname to that Service, and Traefik (with a cert from cert-manager) is the door:
About 20 apps share this one server. A few you can visit:
| Address | What it is |
|---|---|
bit-habit.com | the landing page (static site) |
startpage.bit-habit.com | a personal start dashboard |
blog.bit-habit.com | the blog (Ghost) |
wiki.bit-habit.com | the knowledge base (Wiki.js) |
habit.bit-habit.com | the habit tracker (web app + API) |
plane.bit-habit.com | project management (self-hosted) |
fider.bit-habit.com | feature-request voting |
quali-fit.bit-habit.com | an explainable staffing recommender (sign-in required) |
physical-spark.bit-habit.com | a Physical AI school — static site + an auth API |
llm-app-lab.bit-habit.com | an LLM course site |
headlamp.bit-habit.com | the live cluster dashboard (sign-in required) |
*.bit-habit.com. Every subdomain shares it,
so a new site is HTTPS with zero extra setup.
Everything above follows a request inward: browser → Traefik → app. This section follows code outward: laptop → GitHub → server → live. It is the half of the story people most often guess wrong.
Following one git push
main.authorized_keys with a forced command: no matter what command
arrives, only one script runs. Even a leaked key can do nothing else on the box.git reset --hard origin/main
— an exact mirror, so the server never drifts).
Most static sites deploy by baking the HTML into a container image, pushing it to a
registry, and rolling out new Pods. Here, nginx reads the git checkout on the server's
disk directly. So the moment git pull finishes, the new page is being
served — the same Pod, untouched, uptime intact. A build step buys nothing when the
artifact is the source.
Two layers, deployed differently
It matters which of these you changed — they do not travel the same road.
| What changed | Lives in | How it reaches the cluster |
|---|---|---|
| App content pages, code |
each app's own repo | automatic — GitHub Actions on every push (see above) |
| Infrastructure Ingress, certs, Deployments |
this repo, bit-habit-infra |
manual — kubectl apply by hand. It changes rarely, and a bad
Ingress can take every site down at once, so a human stays in the loop. |
Three repos ship themselves this way today:
| Repo | Site | What its deploy does |
|---|---|---|
portfolio-bithabit | bit-habit.com | git pull — that's the whole deploy |
llm-app-lab | llm-app-lab.bit-habit.com | git pull |
physical-spark |
physical-spark.bit-habit.com |
pull; rebuild the auth image only if auth/ changed; then
kubectl apply its own manifests
(ops/deploy.sh) |
kubectl apply is idempotent — re-applying an unchanged file costs one
unchanged line — so physical-spark applies its manifests on
every deploy, not only when they change. Anything someone poked by hand in the
cluster is quietly reset to what git says. The file, not the running cluster, is the
source of truth. That is the core idea behind GitOps tools like ArgoCD and Flux,
reached here with one extra line and nothing new installed.
A fair question to ask of any platform: is the whole thing in one git repo? Almost. Every service that matters is in git — either this repo, or its own. What is left over is small, and worth naming precisely rather than worrying about.
base/ingress.yaml) — plus 17 app manifests live in
bit-habit-infra. Two apps keep their manifests in their own repo alongside their
code (physical-spark, quali-fit), which is a perfectly standard
layout, not a gap. Nothing that matters exists only on this disk.
plane and fider
are off-the-shelf apps — stock images (makeplane/plane,
getfider/fider) driven by a couple of hand-written YAML files that sit in a home
directory and are in no repo. They are also barely used. Losing those files would mean
rewriting a Deployment and a Service from a public project's docs — an hour, not a disaster.
Worth tidying eventually; not worth calling a risk.
The cluster, right now
The standard goal is one line: the running cluster equals what git says, with nothing that exists only on one machine. GitOps tools — ArgoCD or Flux — enforce it by continuously watching a repo and re-applying it, so manual cluster edits are reverted automatically.
For one node run by one person, ArgoCD is deliberately not installed (kept only as reference): it adds more moving parts than it removes, and re-applying manifests on every deploy already buys the self-healing that makes GitOps worth having. Measured against the standard, this platform is closer than it looks — the remaining gap is secrets, not services. See what to grow.
"Where is that configured?" is the question you actually ask at 2am. There are three places a setting can live here, and only one of them is this git repo. Here is all three, with real paths.
1 · In this git repo
On GitHub as
bookseal/bit-habit-infra,
checked out on the server at /home/ubuntu/workspace/bit-habit-infra.
| Path | What it defines |
|---|---|
base/ingress.yaml | the routing table — every subdomain → its Service. The one file that can break every site at once. |
base/cert-manager/cluster-issuer.yaml | how TLS certificates get issued (Let's Encrypt, DNS-01 via Route53) |
base/cert-manager/certificate.yaml | the wildcard *.bit-habit.com certificate request |
base/cert-manager/aws-secret.example.yaml | a redacted template. The real one (Route53 keys) is gitignored and applied by hand |
base/middlewares/ | Traefik middlewares, e.g. strip the /api prefix before forwarding |
apps/<name>/deployment.yaml | one folder per service — 17 of them (ghost, wikijs, gatus, sentinel, llm-app-lab, …) |
apps/argocd/application.yaml | GitOps config, written but never installed — see the next section |
k3s-bootstrap/traefik-config.yaml | the edge — Traefik on host ports 80/443, http→https redirect |
docs/index.html | this page. nginx serves this folder straight off the disk |
2 · In the app's own git repo
Two services keep their Kubernetes manifests next to their code, and deploy themselves. This is a normal, defensible layout — the service stays self-contained, at the cost of this repo no longer being the complete picture.
| Service | Repo · path |
|---|---|
physical-spark |
bookseal/physical-spark
→ k8s/, auth/k8s/, applied by its own
ops/deploy.sh. It even brings its own Ingress. |
quali-fit |
KIBA-Automation/quali-fit
→ k8s/ (namespace, PVC, deployment, service, middleware, ingress).
The image is tagged with the git commit it was built from, so a running Pod always names
its own source. |
3 · On the host — k3s's own directories
These are not in the repo, and mostly cannot be. They belong to k3s itself.
| Path | What it is |
|---|---|
/var/lib/rancher/k3s/server/manifests/ |
A magic folder: k3s applies every YAML you drop in here, automatically.
It holds Traefik, CoreDNS, local-storage — and traefik-config.yaml,
a copy of the repo's file. To change the edge you
sudo cp the file here and k3s redeploys Traefik on its own.
This is a second source of truth, and ArgoCD can never manage it. |
/etc/rancher/k3s/k3s.yaml | the admin kubeconfig — the credentials + address kubectl uses to talk to the cluster. Root-only. |
/etc/rancher/k3s/registries.yaml | where to pull container images from |
~/.kube/config | the same kubeconfig, copied for a normal user |
~/.ssh/authorized_keys | the deploy keys GitHub Actions uses, each pinned to one script (see the deploys section) |
And the leftovers
/home/ubuntu/plane/k8s/ and
/home/ubuntu/fider/fider.yaml deploy stock upstream images
(makeplane/plane, getfider/fider) — a Deployment, a Service, a
Postgres. Nothing bespoke, nothing precious, and both are barely used.
Recreating them from the upstream docs is an hour of work, so this is a tidiness item,
not a risk. The real thing still outside git is secrets — see
what to grow.
GitOps is one idea in one sentence: put the desired state in git, and let a program inside the cluster continuously make reality match it. That's it. ArgoCD is the most common program for doing that. Here is what changes, and what it would cost here.
The one thing that changes: push becomes pull
Today, everything pushes into the cluster from outside — GitHub Actions SSHes in, or
a human runs kubectl apply. The cluster is passive; it never looks at git.
GitOps inverts that: a controller runs inside the cluster and pulls from git forever.
The loop, precisely
| Word | What it means |
|---|---|
| Application | ArgoCD's core config object. It says: "watch this path in this repo, and keep this namespace matching it." One per group of manifests. Already written here: apps/argocd/application.yaml. |
| Sync | The act of applying git to the cluster. Can be automatic or a button you press. |
| Synced / OutOfSync | Does the cluster currently match git? OutOfSync means someone changed one side. |
| Healthy / Degraded | A separate question: are the Pods actually running? You can be Synced and Degraded — git applied fine, but the app is crashing. |
| Drift | The cluster no longer matches git. Usually because a human ran kubectl edit at 2am. |
| selfHeal | Automatically undo that 2am edit — re-apply git over it. This is the property people actually want from GitOps. |
| prune | Delete things from the cluster when they're deleted from git. Powerful and dangerous: a bad file deletion becomes a production deletion. Off (prune: false) in this repo's config. |
| app-of-apps | One Application whose job is to create the other Applications. Adding a new service becomes "add one file", not "click around the UI". |
What installing it actually looks like
Five commands. The wildcard DNS and certificate already cover a new subdomain, so the UI is reachable with no DNS or TLS work.
# 1. a namespace to live in
kubectl create namespace argocd
# 2. install ArgoCD itself (one big official manifest)
kubectl apply -n argocd -f \
https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
# 3. let Traefik handle TLS, so ArgoCD serves plain http internally
kubectl -n argocd patch deployment argocd-server --type json \
-p '[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--insecure"}]'
# 4. route argocd.bit-habit.com to it (wildcard cert already covers this name)
kubectl apply -f apps/argocd/ingress.yaml
# 5. tell ArgoCD which repo and paths to watch — this file already exists
kubectl apply -f apps/argocd/application.yaml
Step 5 is the whole idea, and it is already written in this repo:
spec:
source:
repoURL: https://github.com/bookseal/bit-habit-infra.git
targetRevision: main # watch the main branch
path: base # watch this folder
syncPolicy:
automated:
prune: false # do NOT delete things removed from git (safety)
selfHeal: true # DO undo hand-edits made in the cluster
selfHeal, which is the valuable half — and it costs nothing to run.
ArgoCD adds ~5 Pods (controller, repo-server, api-server, redis, dex) to a free-tier ARM box,
plus one more web UI to lock down. For one node run by one person, that trade is honestly close.
The real argument for adopting it is not the tool — it's that the tool forces
everything into git. You can get that benefit first, and for free, without installing anything.
Honest scorecard: what this platform would need before GitOps is even possible, ranked by what actually matters. There is exactly one real blocker, and it is not the services — it is the secrets.
Today: real secrets are gitignored and applied by hand — the Route53 keys,
quali-fit's basic-auth, oauth2-proxy's client secret. Git holds only redacted
*.example.yaml templates.
Why it blocks GitOps: a controller that syncs "everything in git" cannot sync what is deliberately absent from git. You would keep a manual step forever, and every fresh install would silently come up broken.
The options, cheapest first:
Today: plane and fider are deployed from YAML that
sits in a home directory and is in no repo. Both are stock upstream apps, and barely used.
Why it is low on this list: ArgoCD can only sync what is in a repo, so
strictly these would have to move before it could manage them. But there is nothing bespoke
in them — losing them costs an hour with the upstream docs, not a service.
Drop them in apps/ whenever it is convenient.
Today: almost everything is jammed into the default namespace.
Only plane, quali-fit, fider and headlamp have their own.
Why it matters: a namespace is the unit ArgoCD scopes an Application
to, and the unit prune deletes within. With one shared default,
turning on prune means one controller with permission to delete everything.
Namespaces are what make prune safe to ever enable.
/var/lib/rancher/k3s/server/manifests/ is applied by k3s itself,
before and below ArgoCD. Traefik's own configuration will always be a
sudo cp onto the host. That is fine — but it should be written down, not
discovered. Do not chase 100%: aim for "everything that can be in git, is."
With secrets encrypted in git and namespaces per app, installing ArgoCD becomes the five commands above and it just works. Do it in that order and it is a calm afternoon. Do it in the reverse order and you spend a week fighting a controller that keeps trying to sync a repo that does not contain the truth.
| Dimension | Today | Needed for GitOps |
|---|---|---|
| Edge, TLS, routing | ✅ in git | ✅ already there |
| Most app manifests | ✅ in git (17 apps) | ✅ already there |
| physical-spark · quali-fit | ✅ in their own git repos | ✅ fine — ArgoCD reads many repos |
| plane · fider | ⚠️ stock apps, YAML in no repo | commit them — an hour, low stakes |
| Secrets | ❌ gitignored, applied by hand | encrypted in git (Sealed Secrets) |
| Namespaces | ⚠️ mostly one shared default | one per app, so prune is safe |
| Self-healing | ✅ kubectl apply on every deploy | ✅ already have the property |
| Host-level (Traefik, CoreDNS) | manual sudo cp | stays manual — out of scope, by design |
This page uses jargon that sounds like it assumes you already know it. You don't have to. Here is every word, defined once, with where it actually shows up here.
| Term | In plain words |
|---|---|
| bootstrap | The one-time setup that brings a machine from "empty Linux box" to "running cluster". You do it once and mostly never again — which is exactly why it must be written down. That is what k3s-bootstrap/ is. |
| systemd service | How Linux keeps a program running in the background and restarts it after a reboot. k3s runs as one. systemctl status k3s asks "is it alive?" |
| SSH | The encrypted remote login used to reach the server from elsewhere. |
| deploy key | An SSH key given to an automated system (here, GitHub Actions) rather than a person, so it can log in without a password. |
~/.ssh/authorized_keys | The list of keys allowed to SSH into this server. If a key is in this file, it can log in. |
| forced command | The security trick this platform leans on. Next to a key in authorized_keys, you can pin command="/path/to/one-script.sh". Then no matter what the remote side asks to run, the server runs only that script and nothing else. So a leaked deploy key cannot open a shell, read files, or delete anything — it can only trigger a redeploy. |
sudo | "Run this as the administrator (root)." Needed to touch system directories like /var/lib/rancher/. |
git pull / git reset --hard origin/main | Fetch the latest code. The reset --hard form throws away any local change and makes the folder an exact mirror of GitHub — so the server can never quietly drift from the repo. |
| Term | In plain words |
|---|---|
| Kubernetes (k8s) | A program that runs your other programs: it starts them, restarts them when they crash, and routes traffic to them. k3s is a small single-binary version of it, which is what runs here. |
| manifest | Just a YAML file describing something you want to exist. "Manifest" and "YAML file" are used interchangeably. |
| declarative | You describe the end state ("I want 1 copy of nginx running"), not the steps to get there. Kubernetes figures out the steps and keeps that statement true forever. This is the single biggest idea in the whole system. |
kubectl | The command-line tool you use to talk to the cluster. Everything below is a kubectl subcommand. |
kubectl apply -f file.yaml | The main command in this whole platform. It means: "read this file, and make the cluster match it." Not "create" — match. If the thing doesn't exist it's created; if it exists but differs it's updated; if it already matches, nothing happens. |
| idempotent | Running it twice does the same thing as running it once. kubectl apply is idempotent, which is why a deploy can safely re-apply every manifest every single time. That habit is what quietly gives this platform self-healing without ArgoCD. |
| Pod | One running instance of your app (a container). The smallest unit Kubernetes runs. |
| Deployment | The manifest that says "keep N Pods of this app running." If a Pod dies, the Deployment makes a new one. This is what restarts your crashed app at 4am. |
| Service | A stable internal address for a set of Pods. Pods come and go with new IPs; the Service name never changes. |
| Ingress | The routing rule: "requests for wiki.bit-habit.com go to the wikijs-svc Service." All of them together are the routing table. |
| ingress controller | The program that actually does what the Ingress rules say. Here that's Traefik. The Ingress is the rule; Traefik is the doorman reading it. |
| namespace | A folder for cluster objects. Purely for grouping and permissions. Most things here sit in the one called default. |
| Secret / ConfigMap | Config handed to a Pod. A Secret is for passwords and keys (base64-obscured, not encrypted); a ConfigMap is for everything else. |
| hostPath | Letting a Pod read a folder from the server's actual disk. It's why this page deploys in ~10 seconds: nginx serves the git checkout directly, so a git pull is the deploy — no image to build, no Pod to restart. |
| kubeconfig | The file holding the cluster's address and your credentials, so kubectl knows where to connect and how to prove who you are. Lives at /etc/rancher/k3s/k3s.yaml. |
| CRD (Custom Resource Definition) | How a tool teaches Kubernetes a brand-new kind of object. cert-manager adds Certificate; ArgoCD adds Application. After a CRD is installed, kubectl apply handles the new type like any built-in one. |
| controller / reconcile loop | A program that never stops asking "does reality match what was declared? no? fix it." Every piece of Kubernetes is one of these. ArgoCD is simply one more, whose declared state happens to live in git. |
| desired state vs actual state | What you wrote down, versus what is really running. The gap between them is called drift, and closing it is the entire job. |
| Term | In plain words |
|---|---|
| wildcard DNS | One rule — *.bit-habit.com → 158.180.71.122 — that points every possible subdomain at this server. It's why adding a site needs no DNS change at all. |
| TLS certificate | The file that proves a site is genuinely who it claims to be. It's what produces the padlock, and what makes https possible. |
| wildcard certificate | One certificate valid for *.bit-habit.com, so every subdomain shares it instead of each needing its own. |
| Let's Encrypt | The free authority that issues those certificates, automatically, every 90 days. |
| DNS-01 challenge | How Let's Encrypt checks you really own the domain: it asks you to place a specific secret record in your DNS. It's the only method that can issue a wildcard certificate — which is why this setup needs Route53 API access at all. |
| Route53 | AWS's DNS service, which hosts this domain. cert-manager holds an API key for it so it can write that challenge record on its own. |
| cert-manager | The robot in the cluster that requests the certificate, answers the DNS-01 challenge, and renews it before it expires. Nobody touches certificates by hand. |
| 308 redirect | "This moved permanently — go to the https version." Traefik sends it for every plain http request. |
| Term | In plain words |
|---|---|
| GitOps | The rule that git is the only source of truth: if it isn't in the repo, it shouldn't be in the cluster. A controller enforces it continuously. |
| push vs pull | Push (today): something outside shoves changes in. Pull (GitOps): something inside the cluster fetches from git on its own, forever. |
| ArgoCD / Flux | The two common programs that do the pulling. They are alternatives to each other; neither is installed here. |
| Application | ArgoCD's config object: "watch this repo + this folder, keep this namespace matching it." |
| sync | Actually applying git to the cluster. |
| selfHeal | Automatically re-apply git over any hand-made change. The property everyone actually wants. |
| prune | Delete from the cluster whatever was deleted from git. Genuinely dangerous — deleting a file becomes deleting production. Off here. |
| Sealed Secrets / SOPS | Ways to encrypt a secret so the encrypted file is safe to commit. Needed because GitOps cannot sync what is deliberately kept out of git. |
How a new site gets added
newsite.bit-habit.com → that Service.kubectl apply). That's all: the wildcard DNS and wildcard certificate already
cover the new subdomain, so it's reachable over HTTPS right away.158.180.71.122 (private 10.0.0.61 inside Oracle Cloud).80 and 443 on the host directly.http → permanent 308 redirect to https.*.bit-habit.com (Let's Encrypt, DNS-01 via Route53), stored in
the Secret tls-secret and auto-renewed before its 90-day expiry.
See the real configs
It is all plain YAML in the public repo. A whole small app is often just this:
apiVersion: apps/v1
kind: Deployment # keep N copies of my app running
metadata: { name: myapp }
spec:
replicas: 1
selector: { matchLabels: { app: myapp } }
template:
metadata: { labels: { app: myapp } }
spec:
containers:
- name: web
image: nginx:alpine
---
apiVersion: v1
kind: Service # one stable address for those copies
metadata: { name: myapp-svc }
spec:
selector: { app: myapp }
ports: [{ port: 80, targetPort: 80 }]
The key files that run this platform (click to view on GitHub):
| File | What it defines |
|---|---|
k3s-bootstrap/traefik-config.yaml | the edge — Traefik on host ports 80/443, http→https redirect |
base/ingress.yaml | the routing table — every subdomain → its Service |
base/cert-manager/cluster-issuer.yaml | how TLS is issued — Let's Encrypt + Route53 DNS-01 |
base/cert-manager/certificate.yaml | the wildcard *.bit-habit.com certificate |
apps/llm-app-lab/deployment.yaml | a minimal app — Deployment + Service |
apps/gatus/deployment.yaml | the status page — Deployment + Service + ConfigMap |
Go further