Container on Home

Hermes Agent: A Personal AI That Gets More Useful Over Time

Sat, 02 May 2026 00:00:00 +0000

How Hermes Agent Works: From Closed-Loop Learning to Multi-Platform Deployment - AI generated

Introduction

I came across the Hermes Agent project in early March 2026 and deployed it a couple of days later. A couple of weeks in I am still using it daily, and the use cases keep expanding rather than converging. Most tools settle into a narrow routine or fall off altogether. What keeps this one going is that the agent gets more useful the longer you run it. The project is young and moving fast, with new releases every few days. The initial setup requires patience: getting the configuration to a point where it actually saves time takes effort, and the frequent updates occasionally introduce breaking changes. That said, it is genuinely fun to use, and you learn a fair amount along the way.

Hermes Agent is an open-source, self-hosted AI agent framework built by Nous Research, an independent AI research lab based in New York. Nous Research is best known for the Hermes model family, a series of open-weight models fine-tuned on Llama that are used widely in the open-source AI community. The agent framework shares the name but is a separate project. It is MIT-licensed, model-agnostic, and runs on your own infrastructure, either as a self-hosted Python service or as a containerized deployment.

How It Works

The part that makes Hermes Agent different from most agent frameworks is the skill system. The agent ships with a set of preconfigured skills covering common tasks. Beyond that, you can ask it to create a skill from something it just did: it writes a structured Markdown document capturing the approach, what worked, and describes possible edge cases. The next time a similar task appears, the agent loads the relevant skill rather than starting from scratch. Skills can be triggered directly by asking Hermes to run one, or set on a schedule and executed automatically at defined intervals. Over time this turns completed work into a growing library of reusable operating knowledge. Version v0.12.0 added an Autonomous Curator to keep that library from growing unwieldy. It runs on a seven-day cycle by default, grades skills by usage, consolidates overlapping ones, and removes those that have stopped being useful. A short report is written after each run, so you can see what changed and why.

Alongside the skill system, the agent maintains three layers of memory: a persistent store for completed tasks and notes, a full-text search index across prior sessions, and a user model that accumulates preferences over time, coding style, communication tone, timezone, tools. The idea is that the agent gets more useful the longer you run it, not just better at individual tasks in isolation.

My Setup

Hermes Agent runs in my homelab as a service on a dedicated Linux host. Keeping it on a separate machine gives me direct control over what the agent has access to. Incoming traffic is routed through Traefik. I access it through three entry points depending on where I am and what I am doing. The primary interface is the Matrix chat protocol, which means I can reach the agent from any Matrix client on any device. I also connected it to a dedicated email inbox, so it can handle certain tasks asynchronously. For longer sessions at my desk I use Open WebUI, which gives a more comfortable interface for extended conversations.

The model configuration is versatile: the agent supports various AI services and model providers.

What I Gave It Access To

I gave the agent access to three local knowledge sources: my bookmarks, a structured knowledge base, and a local mirror of Red Hat’s product documentation.

The first is my bookmarks folder. I have been saving links as Markdown files in Obsidian for several years. The agent can search and cross-reference that collection when doing research, which means it draws on context I actually care about rather than training data alone.

The second is a knowledge base built on the LLM Wiki principle described by Andrej Karpathy. The idea is to maintain a curated set of structured Markdown files that an AI agent helps write and update over time. Topics, entities, comparisons, each in its own file. The agent both contributes to this knowledge base and draws from it when working on research tasks.

The third is a local mirror of Red Hat’s product documentation. A team mate built a tool called rh-mastery that pulls documentation from docs.redhat.com, converts it to Markdown, and stores it in a structured local directory. Pointed at that directory, Hermes can query accurate, version-tracked product documentation without touching the internet. For someone who spends a lot of time with Red Hat products, this closes a gap that is easy to overlook until you actually need it. More on rh-mastery in an upcomming post.

Practical Uses

The combination of bookmarks, structured knowledge, Red Hat’s product documentation, and the skill system makes the agent genuinely useful for research. When I ask it to investigate a topic, it starts with what I have already collected: prior notes, bookmarks, and documentation. If that is not enough, and when asked, it reaches out to the web to fill the gaps. The result is something grounded in material I collected and curated myself, which makes the output in most cases very useful.

One use I did not expect to find as useful: slide generation. I integrated Marp, a Markdown-based presentation framework, into the workflow. When I need to put together a presentation and am staring at a blank file, I can ask the agent to draft an initial structure. Getting past that first empty screen is often the hardest part. Whether I keep most of what it produces is a different question, but having something to react to is worth more than nothing to start from.

Skills and Subagents

The agent can develop and add skills on its own as it works, but skills can also be added manually or loaded from the community hub at agentskills.io. More interesting to me is the subagent capability: the agent can delegate tasks to specialized subagents, each backed by a specific AI service or holding a particular context. This makes it possible to compose workflows where different parts of a task go to the most appropriate model.

Conclusion

Several weeks in is not a long track record, and the project is still moving fast enough that some things will break between releases. That said, the architecture is sound and the development pace is truly impressive. Whether I will keep running it long-term, I genuinely do not know. For now, it is pulling its weight. For anyone already running a homelab and looking for a self-hosted agent that gets more useful over time rather than staying flat, Hermes Agent is worth the setup time.

Peter Steinberger, the creator of OpenClaw, another widely-used AI agent framework, put it well in a recent TED talk: “The bottleneck is no longer typing. It’s thinking.” That observation fits. The agent handles the mechanical parts of research and structuring. The judgment about what matters and what to do with it still has to come from someone. For now, a human in the loop is still necessary.

References

Hermes Agent on GitHub - link
Hermes Agent Documentation - link
Nous Research - link
Matrix - link
OpenRouter - link
Andrej Karpathy LLM Wiki concept - link
Marp - link
agentskills.io - link
Peter Steinberger TED talk - link

My Local AI Stack: Open WebUI, LiteLLM, SearXNG, and Docling

Sat, 14 Feb 2026 00:00:00 +0000

Overview of the modular self-hosted AI stack - AI generated

Introduction

In my previous post about my homelab, I described the foundation I use for self-hosted services: a small set of low-power machines, Docker Compose for deployment, Traefik as the reverse proxy, and internal DNS to expose services with clean HTTPS hostnames. I have been running this setup for several years with very little maintenance overhead. That setup turned out to be a good base not only for classic self-hosting, but also for local AI workloads. Over the past two year or so, I started extending it with tools to use and experiment with AI services.

Over time, I wanted more than a single chat UI connected to a single model provider. I wanted a setup that would let me experiment with different models, keep sensitive data inside my own network, enrich prompts with live web results, and work with local documents in a structured way. I also wanted to reuse the same operational patterns I already trusted in the rest of the homelab.

The result is a local AI stack built from four components:

Open WebUI as the browser-based user interface
LiteLLM as the OpenAI-compatible model gateway
SearXNG as the privacy-friendly web search backend
Docling as the document parsing layer for file-based workflows

Individually, each of these tools is useful. Combined, they form a practical self-hosted AI environment that fits neatly into the same Traefik-centered architecture as the rest of my homelab.

Base platform and prerequisites

The AI stack runs on the same infrastructure described in the previous post: refurbished thin clients running CentOS Stream 9, Docker and Docker Compose, Traefik as the reverse proxy, and internal DNS for clean HTTPS hostnames. The key design principle carries over as well: every externally reachable service joins the external Docker network and is exposed through Traefik using labels, giving a consistent way to publish services under HTTPS without managing ports or certificates per application.

My current setup is CPU-only. That matters. It is perfectly usable for orchestration, document processing, and web-augmented prompting, but it is not the right environment for large, latency-sensitive inference workloads. In practice, that constraint pushed me toward an architecture where the user interface, routing, tools, and document workflows run locally, while the model backend remains flexible enough to use either local or remote providers.

Architecture overview

At a high level, the request flow looks like this:

A user opens Open WebUI in the browser.
Open WebUI sends model requests to LiteLLM through its OpenAI-compatible API.
LiteLLM routes the request to the selected backend model.
If a prompt requires live information, Open WebUI can use SearXNG as a search tool.
If a prompt requires document context, uploaded files are parsed with Docling and converted into Markdown.
The model response is returned to Open WebUI and displayed to the user.

This separation of concerns is what makes the stack useful:

Open WebUI handles the human interaction layer
LiteLLM abstracts model backends and credentials
SearXNG provides fresh web context
Docling turns messy source documents into structured text

Traefik remains the single public entry point. From an operations perspective, that is valuable because the AI stack behaves like any other part of the homelab.

Open WebUI as the central interface

Open WebUI is the part of the stack I interact with every day. It provides the browser-based interface for conversations, model selection, file uploads, and tool-assisted prompting. The important point is that Open WebUI does not need to know anything about individual model providers. It only needs a single OpenAI-compatible endpoint, which in this setup is LiteLLM.

That keeps the client configuration simple. If I want to add a new provider, swap one model for another, or change credentials, I do it behind the scenes in LiteLLM without having to reconfigure the user interface. Open WebUI also supports user and group management, making it straightforward to grant access to specific models or restrict certain users to a defined set of backends. A particularly useful feature is the ability to send a single prompt to multiple AI services simultaneously, which makes side-by-side model comparison a natural part of the workflow.

A simplified Docker Compose service definition for Open WebUI in this setup looks like this:

services:
 open-webui:
 image: ghcr.io/open-webui/open-webui:main
 container_name: open-webui
 restart: unless-stopped
 environment:
 - OPENAI_API_BASE_URL=http://litellm:4000/v1
 - OPENAI_API_KEY=${LITELLM_MASTER_KEY}
 volumes:
 - ./data/open-webui:/app/backend/data
 networks:
 - external
 - internal
 labels:
 - "traefik.enable=true"
 - "traefik.docker.network=external"
 - "traefik.http.routers.openwebui.rule=Host(`ai.home.example.com`)"
 - "traefik.http.routers.openwebui.entrypoints=https"
 - "traefik.http.routers.openwebui.tls.certresolver=cloudflare"
 - "traefik.http.services.openwebui.loadbalancer.server.port=8080"

The exact image tag and environment variables may differ depending on the release and your setup, but the pattern stays the same: persistent storage for state, Traefik labels for routing, and a backend API endpoint that points to LiteLLM.

LiteLLM as the model gateway

LiteLLM is the glue that makes the rest of the system flexible. It exposes a single OpenAI-style API while allowing multiple backends underneath. That means I can define logical model names and map them to either local inference backends or remote providers.

This is useful for several reasons:

Open WebUI only has to speak to few API endpoints
I can standardize naming across models
Provider credentials stay centralized
Swapping backends becomes operationally cheap
Logging and usage controls are easier to centralize

The Compose service definition for LiteLLM follows the same pattern:

services:
 litellm:
 image: litellm/litellm:main-v1.83.14-stable.patch.3
 container_name: litellm
 restart: unless-stopped
 command: ["--config", "/app/config.yaml", "--port", "4000"]
 environment:
 - LITELLM_MASTER_KEY=${LITELLM_MASTER_KEY}
 - OPENAI_API_KEY=${OPENAI_API_KEY}
 volumes:
 - ./litellm/config.yaml:/app/config.yaml:ro
 networks:
 - internal
 - external
 labels:
 - "traefik.enable=true"
 - "traefik.docker.network=external"
 - "traefik.http.routers.litellm.rule=Host(`litellm.home.example.com`)"
 - "traefik.http.routers.litellm.entrypoints=https"
 - "traefik.http.routers.litellm.tls.certresolver=cloudflare"
 - "traefik.http.services.litellm.loadbalancer.server.port=4000"

Warning

Security note:
In March 2026, LiteLLM was subject to a suspected supply chain attack in which versions v1.82.7 and v1.82.8 on PyPI contained a malicious payload designed to harvest credentials and exfiltrate them to an external domain. Users running the official LiteLLM Docker image were not affected, as that deployment path pins dependencies and does not rely on the compromised PyPI packages. If you installed LiteLLM via pip during the affected window, treat any secrets on that system as compromised and rotate them immediately. See the official incident report for full details and verified safe versions.

SearXNG for live, privacy-friendly search

One of the biggest limitations of a plain chat interface is the lack of current information. SearXNG solves that problem cleanly. It is a self-hosted metasearch engine that aggregates results from multiple sources and gives me a search API under my own control.

Even outside the AI stack, SearXNG is useful as a search engine. Inside the stack, it becomes more interesting because it can be exposed as a tool for prompts that need fresh information.

A minimal Compose service might look like this:

services:
 searxng:
 image: docker.io/searxng/searxng:latest
 container_name: searxng
 restart: unless-stopped
 volumes:
 - ./searxng:/etc/searxng
 networks:
 - external
 labels:
 - "traefik.enable=true"
 - "traefik.docker.network=external"
 - "traefik.http.routers.searxng.rule=Host(`search.home.example.com`)"
 - "traefik.http.routers.searxng.entrypoints=https"
 - "traefik.http.routers.searxng.tls.certresolver=cloudflare"
 - "traefik.http.services.searxng.loadbalancer.server.port=8080"

Once connected to Open WebUI as a tool, the flow is straightforward:

The user asks a question that requires current information.
The model decides to call the search tool.
SearXNG performs the search.
Titles, snippets, and URLs are returned as context.
The model synthesizes an answer grounded in current results.

Docling for document parsing

The fourth component, Docling, addresses a different problem. Large language models work best with clean text, but many real documents are messy. PDFs, slide decks, and office files often contain broken text flows, layout artifacts, or table structures that are not useful when passed to a model as-is.

Docling converts these documents into a Markdown representation that is much easier to use as model context. That sounds small, but it is a major quality improvement for local document workflows.

The Docling service definition is straightforward:

services:
 docling:
 image: quay.io/docling-project/docling-serve:latest
 container_name: docling
 restart: unless-stopped
 networks:
 - internal
 - external
 labels:
 - "traefik.enable=true"
 - "traefik.docker.network=external"
 - "traefik.http.routers.docling.rule=Host(`docling.home.example.com`)"
 - "traefik.http.routers.docling.entrypoints=https"
 - "traefik.http.routers.docling.tls.certresolver=cloudflare"
 - "traefik.http.services.docling.loadbalancer.server.port=5001"

The typical usage pattern is:

Upload a document in Open WebUI.
Docling parses the file and converts it to Markdown.
Feed that Markdown into the model as structured prompt context.
Ask targeted questions against the extracted content.

This is especially useful for technical notes, whitepapers, internal PDFs, or vendor documentation where the raw file format is not suitable for direct prompting.

Conclusion

This stack did not start as an attempt to build a local alternative to a commercial AI product. It emerged naturally from an existing homelab that already had strong building blocks: containerized services, Traefik, DNS-based routing, and a bias toward self-hosting.

Adding Open WebUI, LiteLLM, SearXNG, and Docling turned that base into a practical local AI environment. It gives me a single interface for model interaction, the ability to swap backends without changing clients, a way to enrich prompts with live web data, and a better workflow for document-driven tasks.

Just as important, it stays operationally consistent with the rest of the homelab. That keeps the setup understandable, maintainable, and worth using day to day.

Future extensions are obvious: adding a vector database, introducing GPU-backed local inference, routing requests to model endpoints running on specialized inference platforms, or using Open WebUI as a gateway to interact with AI agents. But even without those additions, this combination already covers a large share of the AI workflows I actually care about.

References

My Homelab: A Traefik-centered Self-hosting Setup - link
Open WebUI - project site - link
Open WebUI - GitHub - link
LiteLLM - project site - link
LiteLLM - GitHub - link
LiteLLM - Security incident report, March 2026 - link
SearXNG - documentation - link
SearXNG - GitHub - link
Docling - documentation - link
Docling - GitHub - link

My Homelab: A Traefik-centered Self-hosting Setup

Sat, 24 Jan 2026 00:00:00 +0000

Summary of Homelab services - AI generated

Introduction

Several years ago, I began building a small homelab with two primary objectives in mind: gaining hands-on experience with containers and modern application deployment, and running selected services locally to avoid storing certain data in public cloud environments. In hindsight, this environment evolved into a solid foundation for a local AI stack as well, which I now operate alongside the rest of my setup and will detail in a future post. Although the focus here is on a homelab, the technical stack described can be deployed just as easily in any cloud environment, e.g. a VPS or or any hyperscaler, all that is required is a virtual machine running a Linux distribution of your choice and a container engine.

What began as an experiment has turned into a stable setup that I use every day. At the center of this setup is Traefik, which handles all incoming HTTP and HTTPS traffic and lets me access every service over SSL with clean domains like service-name.home.example.com instead of a collection of raw IP addresses and ports.

In this post I will walk through how I structure this homelab, explain how Traefik ties everything together, and outline a selection of the services currently running in my lab.

Hardware and base platform

The homelab does not run on high-end servers. Most of the hosts are refurbished x86 thin clients with the following specifications:

16 to 32 GB of RAM per node
A modest amount of storage for container images, configuration files, and selected data
Low power consumption, which is important for a system that runs 24/7

The environment uses CentOS Stream 9 as the operating system. On top of that, I run Docker and Docker Compose. Nearly every component in the homelab is containerized, with Traefik positioned in front of these containers as a reverse proxy and routing layer.

Architecture overview

At a high level, the architecture looks like this:

Several containers run on the hosts
A dedicated container network called external, where Traefik and all services that are exposed to the home network reside
An internal DNS setup and a private domain, such as home.example.com, where services are exposed as subdomains like:
- https://pihole.home.example.com
- https://ntfy.home.example.com

Clients on the home network resolve these hostnames to the internal IP address of the homelab host, ensuring that traffic remains entirely within the local network. The local DNS server is automatically assigned to clients connected to the internal network, making all services immediately accessible to any device on the same network.
Traefik acts as the single entry point for HTTP and HTTPS. It terminates TLS, routes requests to the appropriate container based on the hostname, and applies middlewares such as redirects and authentication where required.

Traefik as the center of the homelab

Traefik is an open-source reverse proxy and edge router that integrates well with containerized environments. It monitors the container socket, automatically discovers running containers, and uses labels defined on those containers to configure routing.

In my setup, Traefik provides three main benefits:

Automatic TLS for everything
Traefik uses the DNS challenge with my DNS provider to request certificates from Let’s Encrypt. I can issue a wildcard certificate for *.home.example.com, so every internal service gets proper HTTPS without having to manage individual certificates.
Clean hostnames instead of ports
Every service gets its own subdomain, such as pihole.home.example.com or ntfy.home.example.com. This means I do not have to remember that one service is on port 8080, another on 9090, and so on.
Centralized routing and security
Since everything goes through Traefik, I can:
- Redirect all HTTP traffic to HTTPS
- Protect specific endpoints with basic auth or other middleware
- Inspect and debug routes using the Traefik dashboard

Traefik Docker Compose configuration

Here is a simplified version of the Traefik docker-compose.yml I use:

version: "3"

services:
 traefik:
 image: traefik:latest
 container_name: traefik
 restart: unless-stopped
 security_opt:
 - no-new-privileges:true
 networks:
 - external
 ports:
 - 80:80
 - 443:443
 environment:
 - CF_API_EMAIL=${CF_API_EMAIL}
 - CF_DNS_API_TOKEN=${CF_DNS_API_TOKEN}
 volumes:
 - /etc/localtime:/etc/localtime:ro
 - /var/run/docker.sock:/var/run/docker.sock:ro
 - ./data/traefik.yml:/traefik.yml:ro
 - ./data/acme.json:/acme.json
 - ./data/config.yml:/config.yml:ro
 labels:
 - "traefik.enable=true"

 # HTTP router for Traefik dashboard
 - "traefik.http.routers.traefik.entrypoints=http"
 - "traefik.http.routers.traefik.rule=Host(`traefik.home.example.com`)"

 # Redirect HTTP to HTTPS
 - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
 - "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
 - "traefik.http.routers.traefik.middlewares=traefik-https-redirect"

 # Basic auth for the secure dashboard
 - "traefik.http.middlewares.traefik-auth.basicauth.users=user:hashed-password"

 # HTTPS router for Traefik dashboard
 - "traefik.http.routers.traefik-secure.entrypoints=https"
 - "traefik.http.routers.traefik-secure.rule=Host(`traefik.home.example.com`)"
 - "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
 - "traefik.http.routers.traefik-secure.tls=true"
 - "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
 - "traefik.http.routers.traefik-secure.tls.domains[0].main=home.example.com"
 - "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.home.example.com"
 - "traefik.http.routers.traefik-secure.service=api@internal"

networks:
 external:
 external: true

The important ideas are:

Traefik listens on ports 80 and 443 and is connected to the external network.
It uses environment variables to access the DNS provider so it can request certificates from Let’s Encrypt.
The dashboard is exposed at https://traefik.home.example.com, protected by basic auth.
The TLS configuration issues a wildcard certificate for *.home.example.com.

Other services join the same external network and define their own labels, for example:

services:
 ntfy:
 image: binwiederhier/ntfy
 networks:
 - external
 labels:
 - "traefik.enable=true"
 - "traefik.http.routers.ntfy.entrypoints=https"
 - "traefik.http.routers.ntfy.rule=Host(`ntfy.home.example.com`)"
 - "traefik.http.routers.ntfy.tls.certresolver=cloudflare"

With this pattern, every service becomes available over HTTPS under its own subdomain without additional manual configuration in Traefik.

Core services in my homelab

On top of Traefik, I run a set of core services that provide DNS, monitoring, automation, messaging, logging, and secrets management. The key components are:

Pi-hole – DNS: Provides network-wide DNS resolution and ad-blocking, and handles internal DNS for homelab hostnames such as *.home.example.com. Blocking unwanted domains for devices on the network.
Mafl – Dashboard: A minimalistic and flexible homepage for organizing service links, grouping categories, and providing quick navigation. Mafl can perform health checks on linked services, is configured through a simple YAML file, and offers a Progressive Web App for mobile devices. Since each service sits behind Traefik with its own hostname, Mafl serves as a curated entry point to the entire environment.
Ntfy – Messaging / Pub-Sub: A lightweight HTTP-based publish/subscribe notification service used for event-driven messaging across the environment. Typical use cases include sending alerts when backups complete and receiving notifications when containers restart unexpectedly. Ntfy provides mobile and desktop apps, allowing access from phones and laptops both inside and outside the home network, depending on firewall and VPN settings.
Doozle – Container Logs: A simple web-based UI for viewing Docker logs in real time. Logs are accessible through a browser, it is possible to filter by container, and tail logs as they update. This is particularly useful when testing new services or debugging automation workflows.
Beszel – Resource Monitoring: A lightweight monitoring tool for tracking system metrics and container statistics across multiple machines. It provides CPU, memory, and disk usage insights, making it easy to identify overloaded or misbehaving nodes and maintain visibility into the health of thin clients and other devices.
Uptime Kuma – Service Monitoring: A dashboard for monitoring the availability of both internal and external services. It checks defined endpoints, as well as public websites and APIs. If a service becomes unreachable, Uptime Kuma sends alerts, e.g. via Ntfy or other services, providing an early warning system for issues in the homelab.
n8n – Automation Engine: A workflow automation platform used to orchestrate tasks, trigger scripts or containers, and integrate events across services. Typical use cases include reacting to webhooks or scheduled triggers, executing scripts or container actions, and sending notifications through Ntfy when certain conditions are met. Instead of implementing automation logic in custom code, workflows can be modeled visually and integrated directly with containers and external services.
Vaultwarden – Secrets Management: A self-hosted Bitwarden-compatible server for securely managing passwords and sensitive information within the homelab. It stores credentials and secrets for services and accounts, enables secure sharing across devices.

Conclusion

What began as a simple playground for learning containers and avoiding public cloud services for certain use cases has evolved into a practical, resilient platform for running everyday services at home. Centering the setup around Traefik, standardizing on containerized services, and using a wildcard domain with automated TLS have kept the architecture both manageable and extensible. The use of modest, low-power refurbished thin clients has also proven effective in keeping costs and energy consumption low while still offering sufficient resources.

Over time, the homelab has also turned out to be a solid foundation for hosting local AI services, content of a future post. Depending on the criticality of individual services and one’s tolerance for risk, it can be worthwhile to distribute components across independent hosts, monitor services across nodes, or run certain workloads in parallel for redundancy. It is equally important to think carefully about backups to avoid losing data or configurations during failures or experiments. That said, this remains a homelab project rather than a production environment governed by strict service-level agreements; temporary outages are acceptable, and part of the experimentation process.

With these principles such as simple routing, consistent domains and TLS, lightweight hardware, and containerized services, one can build a flexible environment that supports DNS, monitoring, automation, messaging, secrets management, and more, tailored to one’s own needs.

References

CentOS Stream - link
Traefik - reverse proxy - link
Pi-hole - network-wide ad blocking and DNS - link
Mafl - dashboard for homelab services - link
ntfy - publish/subscribe push notifications - link
Doozle - web based interface to monitor logs - link
Beszel - resource monitoring for multiple clients - link
Uptime Kuma - monitoring tool - link
n8n - workflow automation - link
Vaultwarden - Bitwarden-compatible server - link
Youtube Video - Techno Tim: Put Wildcard Certificates and SSL on EVERYTHING - link

Cloud Native Tutorial

Tue, 07 Mar 2023 00:00:00 +0000

Introduction

Last year I created a tutorial which explains the basics of application deployment in a cloud-native environment. I have also recorded videos for the individual chapters for a Dell internal learning platform, these are not publicly available.
With this post I describe the individual chapters of the tutorial, the individual parts of the tutorial follow on from each other. The corresponding code can be found on GitHub. A simple website serves as a sample application and can be run as a container in various cloud environments. The sample application’s source code is part of the tutorial.
For simplicity, do not attempt this while using a corporate firewall. Comments, additions, and collaboration are welcome.

Find the code on GitHub

1. Setup of local environment

The tutorial provides a virtual machine (VM) to ensure a consistent development environment. This VM is deployed using Vagrant. Vagrant allows it to leverage a declarative configuration file that describes the required software, packages, and operating system configuration. The VM used in this tutorial is based on Ubuntu 20.04 LTS. The VM is allocated 1 vCPU and 4 GB of RAM.
We recommend Virtualbox as the virtualization software for this tutorial.
This tutorial uses Visual Studio Code as a code editor.

After installing the three components (Vagrant, VirtualBox, and Visual Studio Code), the GitHub repository can be cloned, and the VM can be started. The first line ensures that that the software tracking tool git is still installed if it is not already present on the PC:

winget install --id Git.Git -e --source winget
git config --global core.autocrlf false
git clone https://github.com/smichard/cloud_bites_tutorial
cd cloud_bites_tutorial
vagrant up

After the VM has started and all software components have been fully installed, you can log into the VM via ssh:

vagrant ssh

The VM has been configured to mount the host PC’s file system, it is available in the vagrant folder:

cd /vagrant

2. Build and run Docker containers locally

In this section, a container with the demo application is created first. This container is started locally can be reached via the host PC’s web browser.

Switch to the directory:

cd cloud_bites_tutorial/1_2_app_sources

View the container images available in the VM:

docker images

Build the container image based on the folder’s Dockerfile:

docker build -t <image_name> .
docker images

Running the Docker container. With the following command the container runs in the background and the VM ports 8082 are mapped to the container port 80. The website is available via the Host PC’s web browser at localhost:8082:

docker run -d -p 8082:80 <image_name>

The sample application’s source code can be modified easily with the help of a small helper script. This helper script is run twice, and the container image is rebuilt twice. The commands shown start new containers with the newly generated container images. The website is then available via the host PC’s web browser under localhost:8083 and localhost:8084:

./update_script.sh
docker build -t <image_name_2> .
docker run -d -p 8083:80 <image_name_2>
./update_script.sh
docker build -t <image_name_3> .
docker run -d -p 8084:80 <image_name_3>

Running containers can be listed and stopped with the following commands:

docker ps
docker kill <container_id>

The created container images can be pushed to the container registry if you have an account on Docker Hub. The container image must contain the Dockerhub username. So, the container image may need to be rebuilt or named appropriately via a tag:

docker login
docker push <dockerhub_username>/<image_name>

3. Deploy a local Kubernetes cluster using K3D

In the following section you are going to use the K3D project to locally deploy a minimal Kubernetes cluster. K3D is a lightweight wrapper to run K3S, Rancher Lab’s minimal Kubernetes distribution, in Docker. K3D makes it very easy to create single- and multi-node K3S clusters in Docker, e.g. for local development on Kubernetes. A good introduction to K3D with some use cases can be found in the DevOps Toolkit YouTube video.

Deploy a local K3D cluster with three control planes and three worker nodes:

k3d cluster create local-cluster --servers 3 --agents 3 -p "8080:80@loadbalancer"

The cluster and the individual nodes can be displayed with the following commands:

k3d cluster list
kubectl get nodes

The following command deploys the demo application to the local Kubernetes cluster that has just been created. This step creates a deployment with multiple pods, a service, and an ingress controller. You can find the declarative configuration in the 1_3_local_deployment/deployment.yml file:

kubectl apply -f 1_3_local_deployment/deployment.yml

The following command displays the deployment, the replica set, the pods, and the service:

kubectl get deployments,replicasets,pods,services

The website is then available via the host PC’s web browser under localhost:8080.

4. Basic operations to handle Pods and Deployments

The following section details a few basic commands for handling pods and deployments. The following commands create two pods named my-pod-1 and my-pod-2 each with the container image nginx:alpine. The last command shows all pods running in the default namespace:

kubectl run my-pod-1 --image=nginx:alpine
kubectl run my-pod-2 --image=nginx:alpine
kubectl get pods

The following command will stop and delete the pod named my-pod-1. Since this pod is not part of a ReplicaSet, it will not be restarted:

kubectl delete pod my-pod-1
kubectl get pods

The following command scales the deployment created in the previous section to six replicas. The second command displays the deployment, the replica sets, and the pods in the default namespace. Now, six pods should be shown for dpl-demo-1:

kubectl scale deployment dpl-demo-1 --replicas=6
kubectl get deployments,replicasets,pods

Now it’s time to change the deployment. The 1_3_local_deployment/deployment.yml file must be modified to do this. For example, in line 24, the container image can be changed from demo-app:sydney to demo-app:london. The adjusted deployment is rolled out again with the following command:

kubectl apply -f 1_3_local_deployment/deployment.yml
kubectl get deployments,replicasets,pods

The second command displayed the deployment, the replica sets, and the pods for the default namespace. By launching the changed deployment, a new replica set was created in which the number of pods is scaled up to the required number of pods. The number of pods for the existing replica set is scaled to zero.
The deployment is changed a second time and rolled out again. How does this occur? The container image is modified from demo-app:london to demo-app:newyork in line 24 of the 1_3_local_deployment/deployment.yml file. The adjusted deployment is rolled out again with the following command:

kubectl apply -f 1_3_local_deployment/deployment.yml
kubectl get deployments,replicasets,pods

The demo application’s website is always available via the host PC’s web browser via localhost:8080.
The following command shows a list of revisions for the dpl-demo-1 deployment. The second command displays details for a specific revision:

kubectl rollout history deployment dpl-demo-1
kubectl rollout history deployment dpl-demo-1 --revision=2

Use the following command to roll back the deployment to a specific revision:

kubectl rollout undo deployment dpl-demo-1 --to-revision=2

If the deployment is rolled back, the number of pods for the corresponding ReplicaSet is scaled, meaning no new ReplicaSet is created, but an existing one is used.

5. Deploy a remote Kubernetes cluster using Google Cloud – part 1

In the following section, a Kubernetes cluster is deployed in a public cloud. For this purpose, we use Google Cloud in this tutorial. For simplicity, we assume that a Google Cloud account already exists.

Warning

Costs are generated at the public cloud provider used in the following tutorial sections. It is, therefore, vital to limit the runtime of the clusters and prevent possible idle time to minimize costs.

First, obtain access to your Google Cloud account:

gcloud auth login

The following command creates a new Google Cloud project, generating a random project name. Last, the newly created project is specified in the active configuration.

PROJECT_ID="gcp-$(($(date +%s%d)/1000000))$(($RANDOM%20))" && echo $PROJECT_ID
gcloud projects create ${PROJECT_ID} --name "${PROJECT_ID}"
gcloud config set project ${PROJECT_ID}

Before proceeding with the tutorial, make sure to enable billing on the new project.

Then some required Google Cloud APIs have to be activated:

gcloud services enable compute.googleapis.com container.googleapis.com

For simplicity let’s define a default compute region and a default compute zone:

gcloud config set compute/region europe-west3
gcloud config set compute/zone europe-west3a

Finally, the Kubernetes cluster can be deployed:

gcloud beta container clusters create my-gke-cluster --zone europe-west3-a

To interact with the newly created Kubernetes Cluster, fetch its credentials. This commands updates the local kubeconfig file with the appropriate credentials and endpoint information:

gcloud container clusters get-credentials my-gke-cluster

Now the demo application can be deployed on the newly created cluster:

kubectl apply -f 1_5_cloud_deployment/deployment.yml

A load balancer was created as a networking service in the previous step. After a certain wait time, a public IP address is displayed. The website of the demo application can then be reached via a web browser:

kubectl get services

The Kubernetes cluster can be deleted with the following command:

gcloud container clusters delete my-gke-cluster

6. Deploy a remote Kubernetes cluster using Google Cloud – part 2

In this section, another Kubernetes cluster is created on Google Cloud. This time, the Cloud Console is used:

Google Cloud Screenshot

Once the Kubernetes Cluster has been created successfully, fetch its credentials:

gcloud container clusters get-credentials my-gke-cluster-2

Now the demo application can be deployed on the newly created cluster:

kubectl apply -f 1_5_cloud_deployment/deployment.yml

kubectl get services

Tip

It is crucial to delete the Kubernetes cluster if unused to keep costs low.

7. Deploy a remote Kubernetes cluster leveraging Terraform

In this section a Kubernetes cluster is created on Google Cloud leveraging Terraform. Terraform is an infrastructure as code tool allowing one to build infrastructure safely and efficiently in a declarative way on various platforms.
First the Google Cloud environment has to be prepared by enabling a couple of API’s:

gcloud services enable compute.googleapis.com container.googleapis.com

The following command authorizes the Google Cloud SDK to access Google Cloud using your user account credentials. This step adds your account to the Application Default Credentials, allowing Terraform to access these credentials to provision resources on Google Cloud:

gcloud auth application-default login

The following command initializes a working directory containing Terraform configuration files:

cd 1_7_terraform_deployment
terraform init

The terraform plan command creates an execution plan, which lets you preview the changes that Terraform plans to make to your infrastructure. By default, when Terraform creates a plan it:

Reads the current state of any already-existing remote objects to make sure that the Terraform state is up-to-date.
Compares the current configuration to the prior state and noting any differences.
Proposes a set of change actions that should, if applied, make the remote objects match the configuration. With the following command, you will create a Terraform plan. You must pass the project ID generated earlier as a variable:

terraform plan -var project_id=${PROJECT_ID}

The terraform apply command executes the actions proposed in a Terraform plan.

terraform apply -var project_id=${PROJECT_ID}

After creating the Kubernetes Cluster successfully, you can fetch its credentials:

gcloud container clusters get-credentials my-terraform-cluster --region europe-west3

Now the demo application can be deployed on the newly created cluster:

kubectl apply -f 1_5_cloud_deployment/deployment.yml

kubectl get services

The terraform destroy command conveniently eliminates all remote objects managed by a particular Terraform configuration:

terraform destroy -var project_id=${PROJECT_ID}

If you don’t need your Google Cloud project anymore, you can delete the project:

gcloud projects delete ${PROJECT_ID}

8. Visualize Kubernetes workloads with VMware Octant

You will explore, in this section, the various Kubernetes clusters created earlier with VMware Octant. Octant is a tool for developers to understand how applications run on a Kubernetes cluster. It aims to be part of the developer’s toolkit for gaining insight and approaching complexity found in Kubernetes. Octant offers a combination of introspective tooling, cluster navigation, and object management. It also provides a plugin system to extend its capabilities further.

Start Octant:

octant

The Octant application can be reached via a web browser of the host PC using localhost:8001.

Summary

The tutorial gives a quick overview of the basics of application deployment in a cloud-native environment. This tutorial is intended to provide an easy introduction with a quick learning curve. There is by far no claim to completeness. There are certainly ways to improve or expand the tutorial.
You can download the slides I used to present the tutorial via the following link:

Download slide deck

References

GitHub - link
Cloud-Native Computing Foundation - link
Virtualbox - link
Vagrant - link
K3D - link
Terraform - link
Visual Studio Code - link
VMware Octant - link
Introduction to K3D on Youtube - link
Introduction Terraform on Youtube - link