Homelab

The source code for this simple site plus some standout snippets.

I was researching how to monitor systemd services using Grafana/Prometheus when I came across this article. I learned systemd monitoring comes pre-loaded with node_exporter and just needs to be enabled. Then I discovered the Ansible role I use already enables it for me. Neat! No work to do. Or so I thought. Punching node_systemd_unit_state{instance="192.168.1.8:9100"} into Prometheus did not show user services, units that would be shown by systemctl --user. Turns out node_exporter only collects for root. systemd monitoring for USER services I found the community collector systemd_exporter which has this capability, although it isn’t documented anywhere. It’s a completely different binary than node_exporter, and thus has to be installed and enabled. Follow the install instructions or use the cloudalchemy/ansible-systemd-exporter Ansible role. The role starts another root systemd exporter, which is not what we’re interested in. After installing the binary, set up yet-one-more systemd service in ~/.config/systemd/user/systemd_exporter.service which runs the same collector at a user level with some changes: Add --collector.user argument. Change the metrics port (I use 9559). Remove the User= and Group= lines from the [Service]. It should look something like this: ~/.config/systemd/user/systemd_exporter # # Ansible managed # [Unit] Description=Prometheus SystemD Exporter After=network-online.target [Service] Type=simple ExecStart=/usr/local/bin/systemd_exporter \ --collector.user \ --web.listen-address=0.0.0.0:9559 SyslogIdentifier=systemd_exporter Restart=always RestartSec=1 StartLimitInterval=0 [Install] WantedBy=multi-user.target For completeness, this is my install-systemd_exporter.yml ansible playbook: - hosts: all become: true roles: - cloudalchemy.systemd_exporter - hosts: all tasks: - name: Copy user systemd_exporter.service ansible.builtin.copy: src: ../../templates/systemd/systemd_exporter.service dest: ~/.config/systemd/user/systemd_exporter.service - name: Enable systemd_exporter.service ansible.builtin.systemd: daemon-reload: true enabled: true state: started name: systemd_exporter scope: user After running the playbook, or otherwise --user enable --now-ing systemd_exporter, I now have metrics on 0.0.0.0:9559 describing user services. Neat.

As of Dec 2021, this is now outdated. I am no longer using Dendron, nor nomad to deploy the site. I’ll have a follow up shortly. This expands on the DroneCI -> Nomad pipeline I examined earlier I explained why I use Dendron, today I will show how. The glue that binds Dendron, Quantified Self reports, and Hugo is DroneCI. The CI build runs every night and immediately when a commit is made to the Hugo site. I wrote a custom Pod for exporting notes. It’s similar to the built-in markdown pod, but includes frontmatter and renders a hierarchical structure which makes Hugo happy. The server completes these build steps: Clone the required build repos from Github / Gitea: Hugo site workspace Hugo site theme Dendron workspace and notes Dendron <-> Hugo export pod Nomad config files Once the files are in order, we install dendron-cli and link the hugo-pod in an node container Apply the included patch that skips validation of the pod Run the export pod, creating a well formed /garden/ subdirectory in Hugo’s content folder Build the Hugo site Build the final Docker image with the new public directory Publish the final Docker image Send a redeploy command to Nomad using the config cloned in step 1. Prune old Docker images from our registry Done! Simple, no? 😆 The complexity stems from the sources being so scattered (Dendron, Hugo, Nomad…). Obviously this isn’t practical for casual note writing. But in my head, if it’s worth doing it’s also worth overdoing. This is the full .drone.yml spec: kind: pipeline type: docker name: default clone: git: image: plugins/git recursive: true submodule_override: "themes/novela": https://git.quinncasey.com/qcasey/hugo-theme-novela steps: - name: "Clone hugo theme" image: alpine/git commands: - sed -ir 's$:$/$' .gitmodules - sed -ir 's$git@ssh.$https://$g' .gitmodules - git submodule update --recursive --init - name: "Clone Dendron notes and export pod" image: alpine/git commands: - git clone https://git.quinncasey.com/qcasey/hugo-pod.git - git clone https://git.quinncasey.com/qcasey/notes-framework.git notes - cd notes - git clone https://git.quinncasey.com/qcasey/notes.git garden - name: "Export Dendron notes" image: node:current-alpine3.11 commands: - npm install -g @dendronhq/dendron-cli - cd hugo-podand - npm install # Skip validation error "data must NOT have additional properties" - patch node_modules/@dendronhq/pods-core/lib/utils.js < utils.js.patch - npm link - cd ../notes/ - npm link hugo-pod - dendron-cli exportPod --wsRoot ./ --podId hugo --podPkg hugo-pod --podSource custom --config "fname=dendron,vaultName=vault,dest=../content,includeStubs=true" - name: "Build site with hugo" image: klakegg/hugo:ext-alpine commands: - hugo - name: publish image: plugins/docker settings: registry: registry.quinncasey.com repo: registry.quinncasey.com/garden-quinncasey tags: - latest - ${DRONE_BUILD_NUMBER} username: casey password: from_secret: registry_password - name: clone deployment dotfiles image: alpine/git commands: - git clone -b abathur https://git.quinncasey.com/qcasey/dotfiles - name: redeploy nomad image: multani/nomad commands: - nomad run -address=http://192.168.1.196:4646 -var=image_id=${DRONE_BUILD_NUMBER} dotfiles/nomad/webserver/quinncasey.com.nomad - name: prune images image: anoxis/registry-cli:latest commands: - /registry.py -l casey:$PASSWORD -r https://registry.quinncasey.com -i garden-quinncasey --delete --num 10 --keep-tags "stable" "latest" environment: PASSWORD: from_secret: registry_password

Redeploying container images built in CI/CD to a Nomad cluster.

Digital Garden I like the idea of writing with the garage door open. I write notes to help myself a year from now. But if I publish an obscure thing that helps someone do something cool, I wouldn’t complain. Our natural fear of being judged leads most people to build, learn, and think privately. But seeking validation should not be the goal of learning in public. ~ Anne-Laure Le Cunff My sites are hosted in my Homelab, these notes are written in Dendron. I publish the site using Hugo. Plain text is wonderful. It allows me to easily find ideas, references, links, personals notes, tasks, thoughts and everything else I want to keep handy. Even short jots and thoughts deserve their space. Being useful for me is the primary use case for this space on the internet. It’s not that I don’t care about you, but this is for me. It’s here so I can record what I think and know and preserve it in time and space. It’s my garden, but I’m happy for you to hang around and eat tomatos with me. https://joelhooks.com/on-writing-more In it’s current iteration, digital gardens take active effort on the part of the reader to wade through links to related pieces of content. This is very unlike a real garden: you don’t have to be an expert at horticulture or garden design to appreciate the overall landscape. You can easily engage with many different levels of a real garden at the same time, telescoping in and out at will to first examine the minutae of informational placards or specific plants and then returning to a broad perspective of the scenery as a whole. https://vivqu.com/blog/2020/10/18/digital-gardens/ Links to other gardens nikitavoloboev You and your mind garden Building a digital garden A Renaissance of Open Thinking and Curated Writing on the Web Digital gardens Work with the garage door up brendex beepb00p My blog is a digital garden, not a blog Julian’s lifelog and digital playground Alex’s Notes YouTube Creators Linux The Linux Experiment Linux For Everyone Techno Tim ExplainingComputers Makers Estefannie Explains It All JetsonHacks MickMake N-O-D-E Zack’s Lab Strange Parts Maker’s Muse Make Anything

I try to self-host the software and services I use on a daily basis.

Generate boot image from here: https://github.com/kholia/OSX-KVM With this as a guide for proxmox: https://manjaro.site/how-to-install-macos-big-sur-on-proxmox-ve/ GenSMBIOS, ProperTree, Hackintool are REQUIRED! Post Install Use Clover Configurator to mount EFI disk of root and OpenCore install disk. Copy OpenCore install disk EFI files to root efi. Open config.plist into GenSMBIOS, use the settings there. Use Hackintool to check that en0 is built-in. If not, use ProperTree to edit the plist as described here FIXME NSKeyedArchiver Notes