From 3efaf3000f2275ac2def25499c37f10e6fa34404 Mon Sep 17 00:00:00 2001 From: Adrien Beaudouin Date: Mon, 8 May 2023 21:41:45 +0200 Subject: [PATCH] wip k8s --- .../10-build-your-kubernetes-cluster/index.md | 95 +++++++++++++------ .../index.md | 2 +- .../index.md | 7 +- .../index.md | 11 ++- .../index.md | 9 +- .../index.md | 2 +- .../index.md | 4 +- .../index.md | 4 +- .../index.md | 2 +- 9 files changed, 84 insertions(+), 52 deletions(-) diff --git a/content/posts/10-build-your-kubernetes-cluster/index.md b/content/posts/10-build-your-kubernetes-cluster/index.md index de1c0ca..0cc9f2f 100644 --- a/content/posts/10-build-your-kubernetes-cluster/index.md +++ b/content/posts/10-build-your-kubernetes-cluster/index.md @@ -1,6 +1,6 @@ --- title: "Setup a HA Kubernetes cluster for less than $60 / month" -date: 2022-12-08 +date: 2023-06-08 description: "Follow this opinionated guide as starter-kit for your own Kubernetes platform..." tags: ["kubernetes"] draft: true @@ -14,16 +14,16 @@ Build your self-hosted Kubernetes cluster and be free from any SaaS solutions by This guide is mainly intended for any developers or some SRE who want a Kubernetes cluster that respect following conditions : -1. Free from any big cloud providers (AWS, GCP, Azure) which offers managed Kubernetes -2. Completely bare-metal and self-hosted, with some GitOps way steps -3. High availability with cloud Load Balancer -4. Not too much expensive (targeting between $30 and $50 depending on your needs). - -I will not risk saying that it's production ready, but I think it's at least a very good way to build your own bare-metal Kubernetes platform and quick learning all his ecosystem with practice. +1. On-Premise management (The Hard Way), no managed Kubernetes provider +2. Follow the GitOps principles +3. High availability with cloud Load Balancer and resilient storage and DB +4. Fully monitored +5. Complete CI/CD pipeline +6. Not too much expensive (from $30 to $70 /month depending on your needs) ### You may don't need Kubernetes -If you prefer to stay away of all overwhelming Kubernetes features, but always interested in a very simple self-hosted orchestration platform, keep in mind that **Docker Swarm** is probably the best solution for you. It should be always supported as long as Docker CE live, as it's built in into the Docker Engine, and it's far easier and cheaper to maintain it than K8S. +If you prefer to stay away of all overwhelming Kubernetes features, but just interested in a very simple self-hosted orchestration platform (as 99% of any personal usage), keep in mind that **Docker Swarm** is probably the best solution for you. Don't listen people that say it's outdated, because [it's not](https://dockerlabs.collabnix.com/intermediate/swarm/difference-between-docker-swarm-vs-swarm-mode-vs-swarmkit.html) and will always be supported as long as Docker CE live, as it's built in into the Docker Engine, and it's far easier and cheaper to maintain it than K8S. The downside is that there is no longer any new features added to Swarm. I wrote a [complete dedicated guide here]({{< ref "/posts/02-build-your-own-docker-swarm-cluster" >}}) that explains all steps in order to have a production grade Swarm cluster. @@ -31,37 +31,70 @@ I wrote a [complete dedicated guide here]({{< ref "/posts/02-build-your-own-dock As a HA Kubernetes cluster can be quickly expensive, a good cloud provider is an essential part. -After testing Digital Ocean, Vultr, Linode, Civo (which is completly optimized for Kubernetes), OVH, Scaleway, it becomes very clear that nothing can really compete with Hetzner in terms of QoS for that price **in my opinion** : +After testing many providers, as Digital Ocean, Vultr, Linode, Civo , OVH, Scaleway, it seems like **Hetzner** is very well suited **in my opinion** : * Very competitive price for middle-range performance (plan only around **$6** for 2CPU/4GB for each node) -* Cloud Load Balancer, VPC and Firewall support, so no need to reinvent the wheel for these cases -* Very good UI, and with dark mode support which of course absolutely mandatory for my taste -* Perfect CLI tool -* cert-manager [DSN01 challenge support](https://github.com/vadimkim/cert-manager-webhook-hetzner) (but not official) +* No frills, just the basics, VMs, block volumes, load balancer, DNS, firewall, and that's it +* Simple nice UI + CLI tool * Official [Terraform support](https://registry.terraform.io/providers/hetznercloud/hcloud/latest), so GitOps ready +* cert-manager [DSN01 challenge support](https://github.com/vadimkim/cert-manager-webhook-hetzner) Please let me know in below comments if you have other better suggestions ! -### Requirements - -## Final goal 🎯 - -TODO - -### 1. Cluster & routing 🌍 - -Cluster + Traefik - -### 3. Databases & testing with some apps πŸ’Ύ - -### 5. Monitoring πŸ“ˆ - -### 6. CI/CD setup πŸ’» - -Concourse / FluxCD - ## Cluster Architecture 🏘️ +Note as this cluster will be intended for developer user with complete self-hosted CI/CD solution. So for a good cluster architecture starting point, we can imagine the following nodes : + +| server | description | +| --------------- | ----------------------------------------------------------------------------------------------------- | +| `controller-0x` | The control planes nodes, use at least 3 or any greater odd number (when etcd) for HA kube API server | +| `worker-0x` | Workers for your production/staging apps, at least 3 for running Longhorn for resilient storage | +| `data-0x` | Data nodes for any DB / critical statefulset pods | +| `monitor-0x` | Workers dedicated to monitoring | +| `runner-0x` | Workers dedicated to CI/CD pipelines execution | + +```mermaid +flowchart TD +lb((Load Balancer)) +subgraph worker-01 + traefik-01([Traefik]) + apps-01[Apps] + longhorn-01[/Longhorn/] + + traefik-01 --> apps-01 + longhorn-01 --> apps-01 +end +subgraph worker-02 + traefik-02([Traefik]) + apps-02[Apps] + longhorn-02[/Longhorn/] + + traefik-02 --> apps-02 + longhorn-02 --> apps-02 +end +subgraph worker-03 + traefik-03([Traefik]) + apps-03[Apps] + longhorn-03[/Longhorn/] + + traefik-03 --> apps-03 + longhorn-03 --> apps-03 +end +lb --> traefik-01 +lb --> traefik-02 +lb --> traefik-03 +subgraph data-01 + postgresql[(PostgreSQL Primary)] +end +subgraph data-02 + postgresql-replica[(PostgreSQL Replica)] +end +apps-01 --> postgresql +apps-02 --> postgresql +apps-03 --> postgresql +postgresql --> postgresql-replica +``` + ## Cheap solution with Hetzner VPS πŸ–₯️ ## Let’s party πŸŽ‰ diff --git a/content/posts/11-build-your-kubernetes-cluster-part-2/index.md b/content/posts/11-build-your-kubernetes-cluster-part-2/index.md index 4275a77..904b67a 100644 --- a/content/posts/11-build-your-kubernetes-cluster-part-2/index.md +++ b/content/posts/11-build-your-kubernetes-cluster-part-2/index.md @@ -1,6 +1,6 @@ --- title: "Setup a HA Kubernetes cluster Part II - Cluster initialization with Terraform" -date: 2022-12-09 +date: 2023-06-09 description: "Follow this opinionated guide as starter-kit for your own Kubernetes platform..." tags: ["kubernetes", "terraform", "hetzner", "k3s", "gitops"] draft: true diff --git a/content/posts/12-build-your-kubernetes-cluster-part-3/index.md b/content/posts/12-build-your-kubernetes-cluster-part-3/index.md index d0ebcb1..31b54eb 100644 --- a/content/posts/12-build-your-kubernetes-cluster-part-3/index.md +++ b/content/posts/12-build-your-kubernetes-cluster-part-3/index.md @@ -1,8 +1,8 @@ --- title: "Setup a HA Kubernetes cluster Part III - Ingress & HA storage" -date: 2022-12-10 +date: 2023-06-10 description: "Follow this opinionated guide as starter-kit for your own Kubernetes platform..." -tags: ["kubernetes", "traefik", "cert-manager", "longhorn", "minio", "s3"] +tags: ["kubernetes", "traefik", "cert-manager", "longhorn"] draft: true --- @@ -12,8 +12,7 @@ Build your self-hosted Kubernetes cluster and be free from any SaaS solutions by 1. Traefik & cert-manager 2. Resilient Storage with Longhorn -3. Samples with Portainer & Minio -4. S3 Backup with longhorn +3. Test with PgAdmin (valid both ingress and storage) ## 2nd check βœ… diff --git a/content/posts/13-build-your-kubernetes-cluster-part-4/index.md b/content/posts/13-build-your-kubernetes-cluster-part-4/index.md index f24a273..6e0f16a 100644 --- a/content/posts/13-build-your-kubernetes-cluster-part-4/index.md +++ b/content/posts/13-build-your-kubernetes-cluster-part-4/index.md @@ -1,8 +1,8 @@ --- title: "Setup a HA Kubernetes cluster Part IV - Databases with HA & backups" -date: 2022-12-11 +date: 2023-06-11 description: "Follow this opinionated guide as starter-kit for your own Kubernetes platform..." -tags: ["kubernetes", "mysql", "postgresql", "minio", "restic"] +tags: ["kubernetes", "postgresql", "longhorn"] draft: true --- @@ -11,10 +11,11 @@ Build your self-hosted Kubernetes cluster and be free from any SaaS solutions by {{< /lead >}} 1. Add data-01 node -2. MySQL and PostgreSQL -3. Web management apps with PMA & PGA +2. PostgreSQL +3. Access through PgAdmin 4. Job backups -5. Clustering with additional data-02 and postgreSQL cluster +5. S3 Backup with longhorn +6. Clustering with additional data-02 and postgreSQL cluster ## 3rd check βœ… diff --git a/content/posts/14-build-your-kubernetes-cluster-part-5/index.md b/content/posts/14-build-your-kubernetes-cluster-part-5/index.md index 861a82b..07d1f86 100644 --- a/content/posts/14-build-your-kubernetes-cluster-part-5/index.md +++ b/content/posts/14-build-your-kubernetes-cluster-part-5/index.md @@ -1,8 +1,8 @@ --- title: "Setup a HA Kubernetes cluster Part V - Installing some apps & NoCode tools" -date: 2022-12-13 +date: 2023-06-13 description: "Follow this opinionated guide as starter-kit for your own Kubernetes platform..." -tags: ["kubernetes", "nocode", "nocodb", "n8n", "redmine", "analytics", "umami"] +tags: ["kubernetes", "nocode", "nocodb", "n8n", "analytics", "umami"] draft: true --- @@ -12,9 +12,8 @@ Build your self-hosted Kubernetes cluster and be free from any SaaS solutions by 1. Build and deploy simple blog 2. Analytics with Umami -3. Redmine -4. NocoDB -5. n8n +3. NocoDB +4. n8n ## 5th check βœ… diff --git a/content/posts/15-build-your-kubernetes-cluster-part-6/index.md b/content/posts/15-build-your-kubernetes-cluster-part-6/index.md index 3e198b8..9865d86 100644 --- a/content/posts/15-build-your-kubernetes-cluster-part-6/index.md +++ b/content/posts/15-build-your-kubernetes-cluster-part-6/index.md @@ -1,6 +1,6 @@ --- title: "Setup a HA Kubernetes cluster Part V - Monitoring Stack" -date: 2022-12-12 +date: 2023-06-12 description: "Follow this opinionated guide as starter-kit for your own Kubernetes platform..." tags: ["kubernetes", "prometheus", "loki", "grafana"] draft: true diff --git a/content/posts/16-build-your-kubernetes-cluster-part-7/index.md b/content/posts/16-build-your-kubernetes-cluster-part-7/index.md index 0d0b095..b6402da 100644 --- a/content/posts/16-build-your-kubernetes-cluster-part-7/index.md +++ b/content/posts/16-build-your-kubernetes-cluster-part-7/index.md @@ -1,8 +1,8 @@ --- title: "Setup a HA Kubernetes cluster Part VI - VCS & CI" -date: 2022-12-14 +date: 2023-06-14 description: "Follow this opinionated guide as starter-kit for your own Kubernetes platform..." -tags: ["kubernetes", "gitea", "concourse"] +tags: ["kubernetes", "ci", "gitea", "concourse"] draft: true --- diff --git a/content/posts/17-build-your-kubernetes-cluster-part-8/index.md b/content/posts/17-build-your-kubernetes-cluster-part-8/index.md index 5488132..01d9082 100644 --- a/content/posts/17-build-your-kubernetes-cluster-part-8/index.md +++ b/content/posts/17-build-your-kubernetes-cluster-part-8/index.md @@ -1,8 +1,8 @@ --- title: "Setup a HA Kubernetes cluster Part VII - CD, GitOps way" -date: 2022-12-15 +date: 2023-06-15 description: "Follow this opinionated guide as starter-kit for your own Kubernetes platform..." -tags: ["kubernetes", "fluxcd", "gitops"] +tags: ["kubernetes", "cd", "fluxcd", "gitops"] draft: true --- diff --git a/content/posts/18-build-your-kubernetes-cluster-part-9/index.md b/content/posts/18-build-your-kubernetes-cluster-part-9/index.md index c6bb5c6..6b86acb 100644 --- a/content/posts/18-build-your-kubernetes-cluster-part-9/index.md +++ b/content/posts/18-build-your-kubernetes-cluster-part-9/index.md @@ -1,6 +1,6 @@ --- title: "Setup a HA Kubernetes cluster Part VIII - Load testing & tracing" -date: 2022-12-16 +date: 2023-06-16 description: "Follow this opinionated guide as starter-kit for your own Kubernetes platform..." tags: ["kubernetes", "k6", "jaeger"] draft: true