Cozystack: Free and open source PaaS framework for building clouds on Cozystack

Monitoring Parameters

Mon, 01 Jan 0001 00:00:00 +0000

Parameters

Common parameters

Name	Description	Type	Value
`host`	The hostname used to access Grafana externally (defaults to ‘grafana’ subdomain for the tenant host).	`string`	`""`

Metrics storage configuration

Name	Description	Type	Value
`metricsStorages`	Configuration of metrics storage instances.	`[]object`	`[...]`
`metricsStorages[i].name`	Name of the storage instance.	`string`	`""`
`metricsStorages[i].retentionPeriod`	Retention period for metrics.	`string`	`""`
`metricsStorages[i].deduplicationInterval`	Deduplication interval for metrics.	`string`	`""`
`metricsStorages[i].storage`	Persistent volume size.	`string`	`10Gi`
`metricsStorages[i].storageClassName`	StorageClass used for the data.	`string`	`""`
`metricsStorages[i].vminsert`	Configuration for vminsert.	`object`	`{}`
`metricsStorages[i].vminsert.minAllowed`	Minimum guaranteed resources.	`object`	`{}`
`metricsStorages[i].vminsert.minAllowed.cpu`	CPU request.	`quantity`	`""`
`metricsStorages[i].vminsert.minAllowed.memory`	Memory request.	`quantity`	`""`
`metricsStorages[i].vminsert.maxAllowed`	Maximum allowed resources.	`object`	`{}`
`metricsStorages[i].vminsert.maxAllowed.cpu`	CPU limit.	`quantity`	`""`
`metricsStorages[i].vminsert.maxAllowed.memory`	Memory limit.	`quantity`	`""`
`metricsStorages[i].vmselect`	Configuration for vmselect.	`object`	`{}`
`metricsStorages[i].vmselect.minAllowed`	Minimum guaranteed resources.	`object`	`{}`
`metricsStorages[i].vmselect.minAllowed.cpu`	CPU request.	`quantity`	`""`
`metricsStorages[i].vmselect.minAllowed.memory`	Memory request.	`quantity`	`""`
`metricsStorages[i].vmselect.maxAllowed`	Maximum allowed resources.	`object`	`{}`
`metricsStorages[i].vmselect.maxAllowed.cpu`	CPU limit.	`quantity`	`""`
`metricsStorages[i].vmselect.maxAllowed.memory`	Memory limit.	`quantity`	`""`
`metricsStorages[i].vmstorage`	Configuration for vmstorage.	`object`	`{}`
`metricsStorages[i].vmstorage.minAllowed`	Minimum guaranteed resources.	`object`	`{}`
`metricsStorages[i].vmstorage.minAllowed.cpu`	CPU request.	`quantity`	`""`
`metricsStorages[i].vmstorage.minAllowed.memory`	Memory request.	`quantity`	`""`
`metricsStorages[i].vmstorage.maxAllowed`	Maximum allowed resources.	`object`	`{}`
`metricsStorages[i].vmstorage.maxAllowed.cpu`	CPU limit.	`quantity`	`""`
`metricsStorages[i].vmstorage.maxAllowed.memory`	Memory limit.	`quantity`	`""`

Logs storage configuration

Name	Description	Type	Value
`logsStorages`	Configuration of logs storage instances.	`[]object`	`[...]`
`logsStorages[i].name`	Name of the storage instance.	`string`	`""`
`logsStorages[i].retentionPeriod`	Retention period for logs.	`string`	`1`
`logsStorages[i].storage`	Persistent volume size.	`string`	`10Gi`
`logsStorages[i].storageClassName`	StorageClass used to store the data.	`string`	`replicated`

Alerta configuration

Name	Description	Type	Value
`alerta`	Configuration for the Alerta service.	`object`	`{}`
`alerta.storage`	Persistent volume size for the database.	`string`	`10Gi`
`alerta.storageClassName`	StorageClass used for the database.	`string`	`""`
`alerta.resources`	Resource configuration.	`object`	`{}`
`alerta.resources.requests`	Resource requests.	`object`	`{}`
`alerta.resources.requests.cpu`	CPU request.	`quantity`	`100m`
`alerta.resources.requests.memory`	Memory request.	`quantity`	`256Mi`
`alerta.resources.limits`	Resource limits.	`object`	`{}`
`alerta.resources.limits.cpu`	CPU limit.	`quantity`	`1`
`alerta.resources.limits.memory`	Memory limit.	`quantity`	`1Gi`
`alerta.alerts`	Alert routing configuration.	`object`	`{}`
`alerta.alerts.telegram`	Configuration for Telegram alerts.	`object`	`{}`
`alerta.alerts.telegram.token`	Telegram bot token.	`string`	`""`
`alerta.alerts.telegram.chatID`	Telegram chat ID(s), separated by commas.	`string`	`""`
`alerta.alerts.telegram.disabledSeverity`	List of severities without alerts (e.g. [“informational”,“warning”]).	`[]string`	`[]`
`alerta.alerts.slack`	Configuration for Slack alerts.	`object`	`{}`
`alerta.alerts.slack.url`	Configuration uri for Slack alerts.	`string`	`""`
`alerta.alerts.slack.disabledSeverity`	List of severities without alerts (e.g. [“informational”,“warning”]).	`[]string`	`[]`

Grafana configuration

Name	Description	Type	Value
`grafana`	Configuration for Grafana.	`object`	`{}`
`grafana.db`	Database configuration.	`object`	`{}`
`grafana.db.size`	Persistent volume size for the database.	`string`	`10Gi`
`grafana.resources`	Resource configuration.	`object`	`{}`
`grafana.resources.requests`	Resource requests.	`object`	`{}`
`grafana.resources.requests.cpu`	CPU request.	`quantity`	`100m`
`grafana.resources.requests.memory`	Memory request.	`quantity`	`256Mi`
`grafana.resources.limits`	Resource limits.	`object`	`{}`
`grafana.resources.limits.cpu`	CPU limit.	`quantity`	`1`
`grafana.resources.limits.memory`	Memory limit.	`quantity`	`1Gi`

Vmagent configuration

Name	Description	Type	Value
`vmagent`	Configuration for VictoriaMetrics Agent.	`object`	`{}`
`vmagent.externalLabels`	External labels applied to all metrics.	`object`	`{}`
`vmagent.remoteWrite`	Remote write configuration.	`object`	`{}`

Monitoring Parameters

Mon, 01 Jan 0001 00:00:00 +0000

Parameters

Common parameters

Name	Description	Type	Value
`host`	The hostname used to access Grafana externally (defaults to ‘grafana’ subdomain for the tenant host).	`string`	`""`

Metrics storage configuration

Name	Description	Type	Value
`metricsStorages`	Configuration of metrics storage instances.	`[]object`	`[...]`
`metricsStorages[i].name`	Name of the storage instance.	`string`	`""`
`metricsStorages[i].retentionPeriod`	Retention period for metrics.	`string`	`""`
`metricsStorages[i].deduplicationInterval`	Deduplication interval for metrics.	`string`	`""`
`metricsStorages[i].storage`	Persistent volume size.	`string`	`10Gi`
`metricsStorages[i].storageClassName`	StorageClass used for the data.	`string`	`""`
`metricsStorages[i].vminsert`	Configuration for vminsert.	`object`	`{}`
`metricsStorages[i].vminsert.minAllowed`	Minimum guaranteed resources.	`object`	`{}`
`metricsStorages[i].vminsert.minAllowed.cpu`	CPU request.	`quantity`	`""`
`metricsStorages[i].vminsert.minAllowed.memory`	Memory request.	`quantity`	`""`
`metricsStorages[i].vminsert.maxAllowed`	Maximum allowed resources.	`object`	`{}`
`metricsStorages[i].vminsert.maxAllowed.cpu`	CPU limit.	`quantity`	`""`
`metricsStorages[i].vminsert.maxAllowed.memory`	Memory limit.	`quantity`	`""`
`metricsStorages[i].vmselect`	Configuration for vmselect.	`object`	`{}`
`metricsStorages[i].vmselect.minAllowed`	Minimum guaranteed resources.	`object`	`{}`
`metricsStorages[i].vmselect.minAllowed.cpu`	CPU request.	`quantity`	`""`
`metricsStorages[i].vmselect.minAllowed.memory`	Memory request.	`quantity`	`""`
`metricsStorages[i].vmselect.maxAllowed`	Maximum allowed resources.	`object`	`{}`
`metricsStorages[i].vmselect.maxAllowed.cpu`	CPU limit.	`quantity`	`""`
`metricsStorages[i].vmselect.maxAllowed.memory`	Memory limit.	`quantity`	`""`
`metricsStorages[i].vmstorage`	Configuration for vmstorage.	`object`	`{}`
`metricsStorages[i].vmstorage.minAllowed`	Minimum guaranteed resources.	`object`	`{}`
`metricsStorages[i].vmstorage.minAllowed.cpu`	CPU request.	`quantity`	`""`
`metricsStorages[i].vmstorage.minAllowed.memory`	Memory request.	`quantity`	`""`
`metricsStorages[i].vmstorage.maxAllowed`	Maximum allowed resources.	`object`	`{}`
`metricsStorages[i].vmstorage.maxAllowed.cpu`	CPU limit.	`quantity`	`""`
`metricsStorages[i].vmstorage.maxAllowed.memory`	Memory limit.	`quantity`	`""`

Logs storage configuration

Name	Description	Type	Value
`logsStorages`	Configuration of logs storage instances.	`[]object`	`[...]`
`logsStorages[i].name`	Name of the storage instance.	`string`	`""`
`logsStorages[i].retentionPeriod`	Retention period for logs.	`string`	`1`
`logsStorages[i].storage`	Persistent volume size.	`string`	`10Gi`
`logsStorages[i].storageClassName`	StorageClass used to store the data.	`string`	`replicated`

Alerta configuration

Name	Description	Type	Value
`alerta`	Configuration for the Alerta service.	`object`	`{}`
`alerta.storage`	Persistent volume size for the database.	`string`	`10Gi`
`alerta.storageClassName`	StorageClass used for the database.	`string`	`""`
`alerta.resources`	Resource configuration.	`object`	`{}`
`alerta.resources.requests`	Resource requests.	`object`	`{}`
`alerta.resources.requests.cpu`	CPU request.	`quantity`	`100m`
`alerta.resources.requests.memory`	Memory request.	`quantity`	`256Mi`
`alerta.resources.limits`	Resource limits.	`object`	`{}`
`alerta.resources.limits.cpu`	CPU limit.	`quantity`	`1`
`alerta.resources.limits.memory`	Memory limit.	`quantity`	`1Gi`
`alerta.alerts`	Alert routing configuration.	`object`	`{}`
`alerta.alerts.telegram`	Configuration for Telegram alerts.	`object`	`{}`
`alerta.alerts.telegram.token`	Telegram bot token.	`string`	`""`
`alerta.alerts.telegram.chatID`	Telegram chat ID(s), separated by commas.	`string`	`""`
`alerta.alerts.telegram.disabledSeverity`	List of severities without alerts (e.g. [“informational”,“warning”]).	`[]string`	`[]`
`alerta.alerts.slack`	Configuration for Slack alerts.	`object`	`{}`
`alerta.alerts.slack.url`	Configuration uri for Slack alerts.	`string`	`""`
`alerta.alerts.slack.disabledSeverity`	List of severities without alerts (e.g. [“informational”,“warning”]).	`[]string`	`[]`

Grafana configuration

Name	Description	Type	Value
`grafana`	Configuration for Grafana.	`object`	`{}`
`grafana.db`	Database configuration.	`object`	`{}`
`grafana.db.size`	Persistent volume size for the database.	`string`	`10Gi`
`grafana.resources`	Resource configuration.	`object`	`{}`
`grafana.resources.requests`	Resource requests.	`object`	`{}`
`grafana.resources.requests.cpu`	CPU request.	`quantity`	`100m`
`grafana.resources.requests.memory`	Memory request.	`quantity`	`256Mi`
`grafana.resources.limits`	Resource limits.	`object`	`{}`
`grafana.resources.limits.cpu`	CPU limit.	`quantity`	`1`
`grafana.resources.limits.memory`	Memory limit.	`quantity`	`1Gi`

Vmagent configuration

Name	Description	Type	Value
`vmagent`	Configuration for VictoriaMetrics Agent.	`object`	`{}`
`vmagent.externalLabels`	External labels applied to all metrics.	`object`	`{}`
`vmagent.remoteWrite`	Remote write configuration.	`object`	`{}`

Requirements and Toolchain

Mon, 01 Jan 0001 00:00:00 +0000

Toolchain

You will need the following tools installed on your workstation:

talosctl, the command line client for Talos Linux.
kubectl, the command line client for Kubernetes.

Talm, Cozystack’s own configuration manager for Talos Linux:

curl -sSL https://github.com/cozystack/talm/raw/refs/heads/main/hack/install.sh | sh -s

Hardware Requirements

To run this tutorial, you will need the following setup:

Cluster nodes: three bare-metal servers or virtual machines. Hardware requirements depend on your usage scenario:

Here are the baseline requirements for running a small installation. The minimum recommended configuration for each node is as follows:

Requirements and Toolchain

Mon, 01 Jan 0001 00:00:00 +0000

Toolchain

You will need the following tools installed on your workstation:

talosctl, the command line client for Talos Linux.
kubectl, the command line client for Kubernetes.

Talm, Cozystack’s own configuration manager for Talos Linux:

curl -sSL https://github.com/cozystack/talm/raw/refs/heads/main/hack/install.sh | sh -s

Hardware Requirements

To run this tutorial, you will need the following setup:

Cluster nodes: three bare-metal servers or virtual machines. Hardware requirements depend on your usage scenario:

Here are the baseline requirements for running a small installation. The minimum recommended configuration for each node is as follows:

Go Types

Mon, 01 Jan 0001 00:00:00 +0000

Go Types

Cozystack publishes its Kubernetes resource types as a Go module, enabling management of Cozystack resources from any Go code. The types are available at pkg.go.dev/github.com/cozystack/cozystack/api/apps/v1alpha1.

Installation

Add the dependency to your Go module:

go get github.com/cozystack/cozystack/api/apps/v1alpha1@v1.2.0

Use Cases

The Go types are useful for:

Building custom automation tools - Create scripts or applications that programmatically deploy and manage Cozystack resources
Integrating with external systems - Connect Cozystack with your own CI/CD pipelines, monitoring systems, or orchestration tools
Validating configurations - Use the types to validate resource specifications before applying them to the cluster
Generating documentation - Parse and analyze existing Cozystack resources
Building dashboards - Create custom UIs for Cozystack management

Available Packages

The module contains packages for each resource type, you can explore it for your specific version in pkg.go.dev/github.com/cozystack/cozystack/api/apps/v1alpha1

Monitoring Setup

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Cozystack provides a comprehensive monitoring and alerting system for Kubernetes clusters and applications. The system is based on Victoria Metrics for storing metrics and logs, Grafana for visualization, Alerta for alerting, and WorkloadMonitor for monitoring application states.

The architecture is divided into two levels:

System level: Monitoring of cluster infrastructure
Tenant-specific level: Isolated monitoring for each tenant

Installation of Monitoring

System Monitoring

System monitoring is installed automatically during platform deployment through the HelmRelease monitoring-agents in packages/apps/kubernetes/templates/helmreleases/monitoring-agents.yaml.

Monitoring Setup

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The architecture is divided into two levels:

System level: Monitoring of cluster infrastructure
Tenant-specific level: Isolated monitoring for each tenant

Installation of Monitoring

System Monitoring

System monitoring is installed automatically during platform deployment through the HelmRelease monitoring-agents in packages/apps/kubernetes/templates/helmreleases/monitoring-agents.yaml.

Network Architecture

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Cozystack uses a multi-layered networking stack designed for bare-metal Kubernetes clusters. The architecture combines several components, each responsible for a specific layer of the network:

Layer	Component	Purpose
External load balancing	MetalLB	Publishing services to external networks
Service load balancing	Cilium eBPF	kube-proxy replacement, in-kernel DNAT
Network policies	Cilium eBPF	Tenant isolation and security enforcement
Pod networking (CNI)	Kube-OVN	Centralized IPAM, overlay networking
VM IP passthrough	cozy-proxy	Passing through external IPs into virtual machines
VM secondary interfaces	Multus CNI	Attaching secondary L2 interfaces to virtual machines
Observability	Hubble (optional)	Network traffic visibility (disabled by default)

flowchart TD
 EXT["External Clients"]
 RTR["Upstream Router / Gateway"]
 MLB["MetalLB<br/>(L2 ARP / BGP)"]
 CIL["Cilium eBPF<br/>(Service Load Balancing + Network Policies)"]
 OVN["Kube-OVN<br/>(Pod Networking + IPAM)"]
 PODS["Pods"]

 EXT --> RTR
 RTR --> MLB
 MLB --> CIL
 CIL --> OVN
 OVN --> PODS

Cluster Network Configuration

Parameter	Default Value
Pod CIDR	10.244.0.0/16
Service CIDR	10.96.0.0/16
Join CIDR	100.64.0.0/16
Cluster domain	cozy.local
Overlay type	GENEVE
CNI	Kube-OVN
kube-proxy replacement	Cilium eBPF

Networking Stack Variants

Cozystack supports several networking stack variants to accommodate different cluster types. The variant is selected via bundles.system.variant in the platform configuration.

Install Talos Linux using boot-to-talos

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to install Talos Linux on a host running any other Linux distribution using boot-to-talos.

boot-to-talos was made by Cozystack team to help users and teams adopting Cozystack with installing Talos, which is the most complex step in the process. It works entirely from userspace and has no external dependencies except the Talos installer image.

Note that Cozystack provides its own Talos builds, which are tested and optimized for running a Cozystack cluster.

Install Talos Linux using boot-to-talos

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to install Talos Linux on a host running any other Linux distribution using boot-to-talos.

Note that Cozystack provides its own Talos builds, which are tested and optimized for running a Cozystack cluster.

Preparing Disks for LINSTOR Storage Pools

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to prepare physical disks for use with LINSTOR when they contain old metadata that prevents automatic detection.

Problem Description

When setting up storage on new or repurposed nodes, physical disks may contain remnants from previous installations:

RAID superblocks
Partition tables
LVM signatures
Filesystem metadata

This old metadata prevents LINSTOR from detecting disks as available storage.

Symptoms

linstor physical-storage list shows empty output or missing disks
Disks appear with unexpected filesystem types (e.g., linux_raid_member)
Storage pools only show DfltDisklessStorPool without actual storage

Diagnostics

Set up LINSTOR alias

For easier access to LINSTOR commands, set up an alias:

Preparing Disks for LINSTOR Storage Pools

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to prepare physical disks for use with LINSTOR when they contain old metadata that prevents automatic detection.

Problem Description

When setting up storage on new or repurposed nodes, physical disks may contain remnants from previous installations:

RAID superblocks
Partition tables
LVM signatures
Filesystem metadata

This old metadata prevents LINSTOR from detecting disks as available storage.

Symptoms

linstor physical-storage list shows empty output or missing disks
Disks appear with unexpected filesystem types (e.g., linux_raid_member)
Storage pools only show DfltDisklessStorPool without actual storage

Diagnostics

Set up LINSTOR alias

For easier access to LINSTOR commands, set up an alias:

Adding External Applications to Cozystack Catalog

Mon, 01 Jan 0001 00:00:00 +0000

Since v0.37.0, Cozystack administrators can add applications from external sources in addition to the standard application catalog. These applications will appear in the same application catalog and behave like regular managed applications for platform users.

This guide explains how to define a managed application package and how to add it to Cozystack.

1. Create an Application Package Repository

Create a repository with the application package sources. For a reference, see github.com/cozystack/external-apps-example.

Adding External Applications to Cozystack Catalog

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to define a managed application package and how to add it to Cozystack.

1. Create an Application Package Repository

Create a repository with the application package sources. For a reference, see github.com/cozystack/external-apps-example.

Hardware requirements

Mon, 01 Jan 0001 00:00:00 +0000

Cozystack utilizes Talos Linux, a minimalistic Linux distribution designed solely to run Kubernetes. Usually, this means you cannot share a server with any services other than those run by Cozystack. The good news is that whichever service you need, Cozystack will run it perfectly: securely, efficiently, and in a fully containerized or virtualized environment.

Hardware requirements depend on your usage scenario. Below are several common deployment options; review them to determine which setup fits your needs best.

Hardware requirements

Mon, 01 Jan 0001 00:00:00 +0000

Hardware requirements depend on your usage scenario. Below are several common deployment options; review them to determine which setup fits your needs best.

Use Talm to bootstrap a Cozystack cluster

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to install and configure Kubernetes on a Talos Linux cluster using Talm. As a result of completing this guide you will have a Kubernetes cluster ready to install Cozystack.

Talm is a Helm-like utility for declarative configuration management of Talos Linux. Talm was created by Ænix to allow more declarative and customizable configurations for cluster management. Talm comes with pre-built presets for Cozystack.

Prerequisites

By the start of this guide you should have Talos Linux installed, but not initialized (bootstrapped), on several nodes. These nodes should belong to one subnet or have public IPs.

Use Talm to bootstrap a Cozystack cluster

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to install and configure Kubernetes on a Talos Linux cluster using Talm. As a result of completing this guide you will have a Kubernetes cluster ready to install Cozystack.

Prerequisites

By the start of this guide you should have Talos Linux installed, but not initialized (bootstrapped), on several nodes. These nodes should belong to one subnet or have public IPs.

System Resource Planning Recommendations

Mon, 01 Jan 0001 00:00:00 +0000

This guide helps you plan system resource allocation per node based on cluster size and tenant count. Recommendations are based on production deployments and provide reasonably accurate estimates for planning purposes.

Important: Values shown are only for system components. Add your tenant workload requirements (applications, databases, Kubernetes clusters, VMs, etc.) on top of these.

Quick start: Allocate at least 2 CPU cores and 6 GB RAM per node for system components. For precise requirements based on your cluster size and tenant count, use the table or calculator below.

System Resource Planning Recommendations

Mon, 01 Jan 0001 00:00:00 +0000

Important: Values shown are only for system components. Add your tenant workload requirements (applications, databases, Kubernetes clusters, VMs, etc.) on top of these.

1. Install Talos Linux

Mon, 01 Jan 0001 00:00:00 +0000

Before you begin

Make sure that you have nodes (bare-metal servers or VMs) that fit the hardware requirements.

Objectives

On this step of the tutorial you will install Talos Linux on bare-metal servers or VMs running some other Linux distribution.

The tutorial is using boot-to-talos, a simple-to-use CLI app made by Cozystack team for users and teams adopting Cozystack. There are multiple ways to install Talos Linux for Cozystack, not used here and covered in separate guides.

1. Install Talos Linux

Mon, 01 Jan 0001 00:00:00 +0000

Before you begin

Make sure that you have nodes (bare-metal servers or VMs) that fit the hardware requirements.

Objectives

On this step of the tutorial you will install Talos Linux on bare-metal servers or VMs running some other Linux distribution.

Installing Cozystack as a Platform

Mon, 01 Jan 0001 00:00:00 +0000

The third step in deploying a Cozystack cluster is to install Cozystack on a Kubernetes cluster that has been previously installed and configured on Talos Linux nodes. A prerequisite to this step is having installed a Kubernetes cluster.

If this is your first time installing Cozystack, consider starting with the Cozystack tutorial.

To plan a production-ready installation, follow the guide below. It mirrors the tutorial in structure, but gives much more details and explains various installation options.

Managed ClickHouse Service

Mon, 01 Jan 0001 00:00:00 +0000

ClickHouse is an open source high-performance and column-oriented SQL database management system (DBMS). It is used for online analytical processing (OLAP).

How to restore backup from S3

Find the snapshot:

restic -r s3:s3.example.org/clickhouse-backups/table_name snapshots

Restore it:

restic -r s3:s3.example.org/clickhouse-backups/table_name restore latest --target /tmp/

For more details, read Restic: Effective Backup from Stdin.

Parameters

Common parameters

Name	Description	Type	Value
`replicas`	Number of ClickHouse replicas.	`int`	`2`
`shards`	Number of ClickHouse shards.	`int`	`1`
`resources`	Explicit CPU and memory configuration for each ClickHouse replica. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`resources.cpu`	CPU available to each replica.	`quantity`	`""`
`resources.memory`	Memory (RAM) available to each replica.	`quantity`	`""`
`resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`size`	Persistent Volume Claim size available for application data.	`quantity`	`10Gi`
`storageClass`	StorageClass used to store the data.	`string`	`""`

Application-specific parameters

Name	Description	Type	Value
`logStorageSize`	Size of Persistent Volume for logs.	`quantity`	`2Gi`
`logTTL`	TTL (expiration time) for `query_log` and `query_thread_log`.	`int`	`15`
`users`	Users configuration map.	`map[string]object`	`{}`
`users[name].password`	Password for the user.	`string`	`""`
`users[name].readonly`	User is readonly (default: false).	`bool`	`false`

Backup parameters

Name	Description	Type	Value
`backup`	Backup configuration.	`object`	`{}`
`backup.enabled`	Enable regular backups (default: false).	`bool`	`false`
`backup.s3Region`	AWS S3 region where backups are stored.	`string`	`us-east-1`
`backup.s3Bucket`	S3 bucket used for storing backups.	`string`	`s3.example.org/clickhouse-backups`
`backup.schedule`	Cron schedule for automated backups.	`string`	`0 2 * * *`
`backup.cleanupStrategy`	Retention strategy for cleaning up old backups.	`string`	`--keep-last=3 --keep-daily=3 --keep-within-weekly=1m`
`backup.s3AccessKey`	Access key for S3 authentication.	`string`	`<your-access-key>`
`backup.s3SecretKey`	Secret key for S3 authentication.	`string`	`<your-secret-key>`
`backup.resticPassword`	Password for Restic backup encryption.	`string`	`<password>`

ClickHouse Keeper parameters

Name	Description	Type	Value
`clickhouseKeeper`	ClickHouse Keeper configuration.	`object`	`{}`
`clickhouseKeeper.enabled`	Deploy ClickHouse Keeper for cluster coordination.	`bool`	`true`
`clickhouseKeeper.size`	Persistent Volume Claim size available for application data.	`quantity`	`1Gi`
`clickhouseKeeper.resourcesPreset`	Default sizing preset.	`string`	`micro`
`clickhouseKeeper.replicas`	Number of Keeper replicas.	`int`	`3`

Parameter examples and reference

resources and resourcesPreset

resources sets explicit CPU and memory configurations for each replica. When left empty, the preset defined in resourcesPreset is applied.

Cozystack ConfigMap Reference

Mon, 01 Jan 0001 00:00:00 +0000

This page explains the role of Cozystack’s main ConfigMap and provides a full reference for its values.

Cozystack’s main configuration is defined by a single Kubernetes ConfigMap. This ConfigMap includes Cozystack bundle and components setup, key network settings, exposed services, and other options.

Example

Here’s an example of configuration for installing Cozystack with bundle paas-full, with root host “example.org”, and Cozystack Dashboard and API exposed and available to users:

apiVersion: v1
kind: ConfigMap
metadata:
 name: cozystack
 namespace: cozy-system
data:
 bundle-name: "paas-full"
 root-host: "example.org"
 api-server-endpoint: "https://api.example.org:443"
 expose-services: "dashboard,api"
 ipv4-pod-cidr: "10.244.0.0/16"
 ipv4-pod-gateway: "10.244.0.1"
 ipv4-svc-cidr: "10.96.0.0/16"
 ipv4-join-cidr: "100.64.0.0/16"

Reference

Value (`data.*`)	Bundles	Description
`bundle-name`	all	Name of bundle to use for installation.
`bundle-enable`	all	Optional bundle components included in the installation. Read more about this option in “How to enable and disable bundle components”.
`bundle-disable`	all	Bundle components excluded (disabled) from the installation. Read more about this option in “How to enable and disable bundle components”.
`values-<component>`	all	JSON or YAML formatted values passed to specific component installation. Read more about this option in “how to overwrite parameters for specific components”.
`root-host`	all	The main domain for all services created under Cozystack, such as the dashboard, Grafana, Keycloak, etc.
`api-server-endpoint`	all	Used for generating kubeconfig files for your users. It is recommended to use a routable FQDN or IP address instead of local-only addresses.
`telemetry-enabled`	all	Enable telemetry feature in Cozystack (default: `true`).
`expose-services`	all	Comma-separated list of services to expose to the internet. Possible values: `api,dashboard,cdi-uploadproxy,vm-exportproxy`.
`expose-ingress`	all	Ingress controller to use for exposing services. (default: `tenant-root`)
`expose-external-ips`	all	Comma-separated list of external IPs used for the specified ingress controller. If not specified, a LoadBalancer service is used by default.
`ipv4-pod-cidr`	`paas-full`, `distro-full`	The pod subnet used by Pods to assign IPs
`ipv4-pod-gateway`	`paas-full`	The gateway address for the pod subnet
`ipv4-svc-cidr`	`paas-full`, `distro-full`	The pod subnet used by Services to assign IPs
`ipv4-join-cidr`	`paas-full`	The `join` subnet, as a special subnet for network communication between the Node and Pod. Follow the kube-ovn documentation to learn more about these options.
`oidc-enabled`	`paas-full`, `paas-hosted`	Enable oidc feature in Cozystack (default: `false`)
`cpu-allocation-ratio`	`paas-full`, `paas-hosted`	CPU allocation ratio: `1/cpu-allocation-ratio` CPU requested per 1 vCPU. Defaults to 10. See Resource Management for detailed explanation and examples.

Monitoring Dashboards

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Cozystack integrates Grafana as the primary visualization tool for monitoring metrics collected by VictoriaMetrics (VM). This section covers accessing pre-built dashboards, creating custom visualizations, and integrating external data sources to provide comprehensive observability for your Cozystack clusters and applications.

Accessing Grafana

To access Grafana and explore dashboards:

Navigate to the Grafana URL: https://grafana.<tenant-domain>, where <tenant-domain> is your tenant’s domain.
Log in using your tenant credentials (OIDC or token-based authentication).
Once logged in, you can browse pre-configured dashboards in the “Dashboards” section.

For initial setup and configuration details, refer to Monitoring Setup.

Monitoring Dashboards

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Accessing Grafana

To access Grafana and explore dashboards:

Navigate to the Grafana URL: https://grafana.<tenant-domain>, where <tenant-domain> is your tenant’s domain.
Log in using your tenant credentials (OIDC or token-based authentication).
Once logged in, you can browse pre-configured dashboards in the “Dashboards” section.

For initial setup and configuration details, refer to Monitoring Setup.

Configuring a Dedicated Network for LINSTOR

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to improve storage reliability and performance by redirecting LINSTOR replication traffic to a dedicated network interface.

Introduction

The Cozystack platform is built to support high-availability (HA) workloads, which means the system must continue operating even if one or more nodes go offline. Kubernetes handles this well for stateless workloads. However, stateful workloads, such as file storage or virtual machine disks, require a reliable storage backend.

When you choose the replicated storage class for a PersistentVolumeClaim (PVC) or DataVolume, LINSTOR creates multiple synchronous replicas of the data across different nodes. This ensures that if a node fails (due to a power outage, hardware issue, etc.), the data remains instantly available on another node.

Configuring a Dedicated Network for LINSTOR

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to improve storage reliability and performance by redirecting LINSTOR replication traffic to a dedicated network interface.

Introduction

Troubleshooting etcd

Mon, 01 Jan 0001 00:00:00 +0000

How to clean up etcd state

To flush the etcd state from a node, you can use talm or talosctl with the following commands:

Talm
talosctl

Replace nodeN with the name of the failed node, for instance, node0.yaml:

talm reset -f nodes/nodeN.yaml --system-labels-to-wipe=EPHEMERAL --graceful=false --reboot

talosctl reset --system-labels-to-wipe=EPHEMERAL --graceful=false --reboot

⚠️ This command will remove the state from the specified node. Use it with caution.

Troubleshooting etcd

Mon, 01 Jan 0001 00:00:00 +0000

How to clean up etcd state

To flush the etcd state from a node, you can use talm or talosctl with the following commands:

Talm
talosctl

Replace nodeN with the name of the failed node, for instance, node0.yaml:

talm reset -f nodes/nodeN.yaml --system-labels-to-wipe=EPHEMERAL --graceful=false --reboot

talosctl reset --system-labels-to-wipe=EPHEMERAL --graceful=false --reboot

⚠️ This command will remove the state from the specified node. Use it with caution.

Tune etcd timeouts

Mon, 01 Jan 0001 00:00:00 +0000

Potential problems

etcd is a reliable key-value store for critical data, the place where the Kubernetes API server stores its data. Values are not considered written until a quorum of nodes reports that data has been written. Also, etcd constantly checks that there is a leader and every node is aware which one it is. Usually, an etcd cluster is run on nodes in the same datacenter (where latency is low) and default timeouts are tuned for the quickest error reporting possible. In a stretched cluster, where nodes are in different datacenters, the RTT is much higher and default timeouts can cause false alarms as if the network was partitioned. In this case, the data is still consistent, but etcd will enter readonly mode to prevent split-brain. This could happen so frequently that many other components of Kubernetes would start to fail with non-descriptive and unhelpful error messages.

Tune etcd timeouts

Mon, 01 Jan 0001 00:00:00 +0000

Potential problems

Troubleshooting Flux CD

Mon, 01 Jan 0001 00:00:00 +0000

Diagnosing `install retries exhausted` error

Sometimes you can face with the error:

# kubectl get hr -A -n cozy-dashboard dashboard
NAMESPACE NAME AGE READY STATUS
cozy-dashboard dashboard 15m False install retries exhausted

You can try to figure out by checking events:

kubectl describe hr -n cozy-dashboard dashboard

if Events: <none> then suspend and resume the release:

kubectl patch hr -n cozy-dashboard dashboard -p '{"spec": {"suspend": true}}' --type=merge
kubectl patch hr -n cozy-dashboard dashboard -p '{"spec": {"suspend": null}}' --type=merge

and check the events again:

Troubleshooting Flux CD

Mon, 01 Jan 0001 00:00:00 +0000

Diagnosing `install retries exhausted` error

Sometimes you can face with the error:

# kubectl get hr -A -n cozy-dashboard dashboard
NAMESPACE NAME AGE READY STATUS
cozy-dashboard dashboard 15m False install retries exhausted

You can try to figure out by checking events:

kubectl describe hr -n cozy-dashboard dashboard

if Events: <none> then suspend and resume the release:

kubectl patch hr -n cozy-dashboard dashboard -p '{"spec": {"suspend": true}}' --type=merge
kubectl patch hr -n cozy-dashboard dashboard -p '{"spec": {"suspend": null}}' --type=merge

and check the events again:

Cluster Autoscaler for Hetzner Cloud

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to configure cluster-autoscaler for automatic node scaling in Hetzner Cloud with Talos Linux.

Prerequisites

Hetzner Cloud account with API token
hcloud CLI installed
Existing Talos Kubernetes cluster
Networking Mesh and Local CCM configured

Step 1: Create Talos Image in Hetzner Cloud

Hetzner doesn’t support direct image uploads, so we need to create a snapshot via a temporary server.

1.1 Generate Schematic ID

Create a schematic at factory.talos.dev with required extensions:

Key Concepts

Mon, 01 Jan 0001 00:00:00 +0000

Cozystack is an open-source, Kubernetes-native platform that turns bare-metal or virtual infrastructure into a fully featured, multi-tenant cloud. At its core are a few foundational building blocks:

the management cluster that runs the platform itself;
tenants that provide strict, hierarchical isolation;
tenant clusters that give users their own Kubernetes control planes;
rich catalog of managed applications and virtual machines;
bundles that assemble these components into a turnkey stack.

Understanding how these concepts fit together will help you plan, deploy, and operate Cozystack effectively, whether you are building an internal developer platform or a public cloud service.

Key Concepts

Mon, 01 Jan 0001 00:00:00 +0000

Cozystack is an open-source, Kubernetes-native platform that turns bare-metal or virtual infrastructure into a fully featured, multi-tenant cloud. At its core are a few foundational building blocks:

the management cluster that runs the platform itself;
tenants that provide strict, hierarchical isolation;
tenant clusters that give users their own Kubernetes control planes;
rich catalog of managed applications and virtual machines;
variants that assemble these components into a turnkey stack.

Understanding how these concepts fit together will help you plan, deploy, and operate Cozystack effectively, whether you are building an internal developer platform or a public cloud service.

Networking Mesh

Mon, 01 Jan 0001 00:00:00 +0000

Kilo creates a WireGuard mesh between cluster locations. When running with Cilium, it uses IPIP encapsulation routed through Cilium’s VxLAN overlay so that traffic between locations works even when the cloud network blocks raw IPIP (protocol 4) packets.

Select the cilium-kilo networking variant

During platform setup, select the cilium-kilo networking variant. This deploys both Cilium and Kilo as an integrated stack with the required configuration:

How it works

Kilo runs in --local=false mode – it does not manage routes within a location (Cilium handles that)
Kilo creates a WireGuard tunnel (kilo0) between location leaders
Non-leader nodes in each location reach remote locations through IPIP encapsulation to their location leader, routed via Cilium’s VxLAN overlay
The leader decapsulates IPIP and forwards traffic through the WireGuard tunnel
Cilium’s enable-ipip-termination option creates the cilium_tunl interface (kernel’s tunl0 renamed) that Kilo uses for IPIP TX/RX – without it, the kernel detects TX recursion on the tunnel device

Talos machine config for cloud nodes

Cloud worker nodes must include Kilo annotations in their Talos machine config:

Platform Package Reference

Mon, 01 Jan 0001 00:00:00 +0000

This page explains the role of the Cozystack Platform Package and provides a full reference for its values.

Cozystack’s main configuration is defined by a Package custom resource. This Package includes the Cozystack variant and component settings, key network settings, exposed services, and other options.

Example

Here’s an example of configuration for installing Cozystack with variant isp-full, with root host “example.org”, and Cozystack Dashboard and API exposed and available to users:

apiVersion: cozystack.io/v1alpha1
kind: Package
metadata:
 name: cozystack.cozystack-platform
spec:
 variant: isp-full
 components:
 platform:
 values:
 publishing:
 host: "example.org"
 apiServerEndpoint: "https://api.example.org:443"
 exposedServices:
 - dashboard
 - api
 networking:
 podCIDR: "10.244.0.0/16"
 podGateway: "10.244.0.1"
 serviceCIDR: "10.96.0.0/16"
 joinCIDR: "100.64.0.0/16"

Reference

Package-level fields

Field	Description
`spec.variant`	Variant to use for installation (e.g., `isp-full`, `isp-full-generic`, `isp-hosted`, `distro-full`).

Platform values (`spec.components.platform.values.*`)

Publishing

Value	Default	Description
`publishing.host`	`"example.org"`	The main domain for all services created under Cozystack, such as the dashboard, Grafana, Keycloak, etc.
`publishing.apiServerEndpoint`	`""`	Used for generating kubeconfig files for your users. It is recommended to use a routable FQDN or IP address instead of local-only addresses. Example: `"https://api.example.org"`.
`publishing.exposedServices`	`[api, dashboard, vm-exportproxy, cdi-uploadproxy]`	List of services to expose. Possible values: `api`, `dashboard`, `cdi-uploadproxy`, `vm-exportproxy`.
`publishing.ingressName`	`"tenant-root"`	Ingress controller to use for exposing services.
`publishing.externalIPs`	`[]`	List of external IPs used for the specified ingress controller. If not specified, a LoadBalancer service is used by default.
`publishing.certificates.solver`	`"http01"`	ACME challenge solver type for default letsencrypt issuer. Possible values: `http01`, `dns01`.
`publishing.certificates.issuerName`	`"letsencrypt-prod"`	`ClusterIssuer` name for TLS certificates used in system Helm releases.

Networking

Value	Default	Description
`networking.clusterDomain`	`"cozy.local"`	Internal cluster domain name.
`networking.podCIDR`	`"10.244.0.0/16"`	The pod subnet used by Pods to assign IPs.
`networking.podGateway`	`"10.244.0.1"`	The gateway address for the pod subnet.
`networking.serviceCIDR`	`"10.96.0.0/16"`	The service subnet used by Services to assign IPs.
`networking.joinCIDR`	`"100.64.0.0/16"`	The `join` subnet for network communication between the Node and Pod. Follow the kube-ovn documentation to learn more.
`networking.kubeovn.MASTER_NODES`	`""`	Comma-separated list of KubeOVN master node IPs. By default, KubeOVN uses `lookup` to find control-plane nodes by label `node-role.kubernetes.io/control-plane`. On fresh clusters, lookup may return empty results. Set this to override.

Bundles

Value	Default	Description
`bundles.system.enabled`	`false`	Enable the system bundle. Managed by the operator based on `spec.variant`.
`bundles.system.variant`	`"isp-full"`	System bundle variant. Options: `isp-full`, `isp-full-generic`, `isp-hosted`. Managed by the operator based on `spec.variant`.
`bundles.iaas.enabled`	`false`	Enable the IaaS bundle. Managed by the operator based on `spec.variant`.
`bundles.paas.enabled`	`false`	Enable the PaaS bundle. Managed by the operator based on `spec.variant`.
`bundles.naas.enabled`	`false`	Enable the NaaS bundle. Managed by the operator based on `spec.variant`.
`bundles.enabledPackages`	`[]`	List of optional bundle components to include in the installation. Read more in “How to enable and disable bundle components”.
`bundles.disabledPackages`	`[]`	List of bundle components to exclude from the installation. Read more in “How to enable and disable bundle components”.

Authentication

Value	Default	Description
`authentication.oidc.enabled`	`false`	Enable OIDC feature in Cozystack.
`authentication.oidc.insecureSkipVerify`	`false`	Skip TLS certificate verification for the OIDC provider.
`authentication.oidc.keycloakExtraRedirectUri`	`""`	Additional redirect URI for Keycloak OIDC client.
`authentication.oidc.keycloakInternalUrl`	`""`	Internal URL for backend-to-backend requests to Keycloak. When set, the dashboard’s oauth2-proxy skips OIDC discovery and routes token, JWKS, userinfo, and logout requests through this URL while keeping browser redirects on the external URL. Example: `http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy`.

Scheduling

Value	Default	Description
`scheduling.globalAppTopologySpreadConstraints`	`""`	Global pod topology spread constraints applied to all managed applications.

Branding

Value	Default	Description
`branding`	`{}`	UI branding configuration object. See the White Labeling guide for available fields and usage. Individual fields (e.g., `titleText`, `logoSvg`) have their own defaults when not specified.

Registries

Container registry mirrors configuration. Allows routing image pulls through local mirrors.

Using Cozystack to build public cloud

Mon, 01 Jan 0001 00:00:00 +0000

You can use Cozystack as backend for a public cloud

Overview

Cozystack positions itself as a kind of framework for building public clouds. The key word here is framework. In this case, it’s important to understand that Cozystack is made for cloud providers, not for end users.

Despite having a graphical interface, the current security model does not imply public user access to your management cluster.

Instead, end users get access to their own Kubernetes clusters, can order LoadBalancers and additional services from it, but they have no access and know nothing about your management cluster powered by Cozystack.

Using Cozystack to build public cloud

Mon, 01 Jan 0001 00:00:00 +0000

You can use Cozystack as backend for a public cloud

Overview

Despite having a graphical interface, the current security model does not imply public user access to your management cluster.

REST API Reference

Mon, 01 Jan 0001 00:00:00 +0000

REST API Reference

Mon, 01 Jan 0001 00:00:00 +0000

Storage Pools

Mon, 01 Jan 0001 00:00:00 +0000

Storage pools let you partition SeaweedFS volume servers by disk type. Each pool creates a separate Volume StatefulSet tagged with a SeaweedFS diskType, and a matching set of COSI resources (BucketClasses and BucketAccessClasses) that buckets can reference.

When to Use Pools

Use storage pools when your cluster has different storage tiers and you want to control which tier a bucket uses. For example, you might have fast NVMe drives for hot data and large HDD drives for archival storage.

Use talos-bootstrap script to bootstrap a Cozystack cluster

Mon, 01 Jan 0001 00:00:00 +0000

talos-bootstrap is an interactive script for bootstrapping Kubernetes clusters on Talos OS.

It was created by Cozystack developers to simplify the installation of Talos Linux on bare-metal nodes in a user-friendly manner.

1. Install Dependencies

Install the following dependencies

talosctl
dialog
nmap

Download the latest version of talos-bootstrap from the releases page or directly from the trunk:

curl -fsSL -o /usr/local/bin/talos-bootstrap \
 https://github.com/cozystack/talos-bootstrap/raw/master/talos-bootstrap
chmod +x /usr/local/bin/talos-bootstrap
talos-bootstrap --help

2. Prepare Configuration Files

Start by making a configuration directory for the new cluster:

Use talos-bootstrap script to bootstrap a Cozystack cluster

Mon, 01 Jan 0001 00:00:00 +0000

talos-bootstrap is an interactive script for bootstrapping Kubernetes clusters on Talos OS.

It was created by Cozystack developers to simplify the installation of Talos Linux on bare-metal nodes in a user-friendly manner.

1. Install Dependencies

Install the following dependencies

talosctl
dialog
nmap

Download the latest version of talos-bootstrap from the releases page or directly from the trunk:

curl -fsSL -o /usr/local/bin/talos-bootstrap \
 https://github.com/cozystack/talos-bootstrap/raw/master/talos-bootstrap
chmod +x /usr/local/bin/talos-bootstrap
talos-bootstrap --help

2. Prepare Configuration Files

Start by making a configuration directory for the new cluster:

Monitoring Troubleshooting

Mon, 01 Jan 0001 00:00:00 +0000

This guide provides troubleshooting steps for common issues with monitoring components in Cozystack, including metrics collection, alerting, visualization, and log collection.

Diagnosing Missing Metrics

If metrics are not appearing in Grafana or VictoriaMetrics, follow these steps:

Check VMAgent Status

Ensure VMAgent is running and collecting metrics:

kubectl get pods -n cozy-monitoring -l app.kubernetes.io/name=vmagent
kubectl logs -n cozy-monitoring -l app.kubernetes.io/name=vmagent --tail=50

Verify Targets

Check if VMAgent can scrape targets:

kubectl exec -n cozy-monitoring -c vmagent deploy/vmagent -- curl -s http://localhost:8429/targets | jq .

Look for targets with health: "up". If targets are down, check network connectivity and RBAC permissions.

Monitoring Troubleshooting

Mon, 01 Jan 0001 00:00:00 +0000

This guide provides troubleshooting steps for common issues with monitoring components in Cozystack, including metrics collection, alerting, visualization, and log collection.

Diagnosing Missing Metrics

If metrics are not appearing in Grafana or VictoriaMetrics, follow these steps:

Check VMAgent Status

Ensure VMAgent is running and collecting metrics:

kubectl get pods -n cozy-monitoring -l app.kubernetes.io/name=vmagent
kubectl logs -n cozy-monitoring -l app.kubernetes.io/name=vmagent --tail=50

Verify Targets

Check if VMAgent can scrape targets:

kubectl exec -n cozy-monitoring -c vmagent deploy/vmagent -- curl -s http://localhost:8429/targets | jq .

Look for targets with health: "up". If targets are down, check network connectivity and RBAC permissions.

Upgrading Cozystack and Post-upgrade Checks

Mon, 01 Jan 0001 00:00:00 +0000

About Cozystack Versions

Cozystack uses a staged release process to ensure stability and flexibility during development.

There are three types of releases:

Alpha, Beta, and Release Candidates (RC) – Preview versions (such as v0.42.0-alpha.1 or v0.42.0-rc.1) used for final testing and validation.
Stable Releases – Regular versions (e.g., v0.42.0) that are feature-complete and thoroughly tested. Such versions usually introduce new features, update dependencies, and may have API changes.
Patch Releases – Bugfix-only updates (e.g., v0.42.1) made after a stable release, based on a dedicated release branch.

It’s highly recommended to install only stable and patch releases in production environments.

Upgrading Cozystack and Post-upgrade Checks

Mon, 01 Jan 0001 00:00:00 +0000

About Cozystack Versions

Cozystack uses a staged release process to ensure stability and flexibility during development.

There are three types of releases:

Alpha, Beta, and Release Candidates (RC) – Preview versions (such as v0.42.0-alpha.1 or v0.42.0-rc.1) used for final testing and validation.
Stable Releases – Regular versions (e.g., v0.42.0) that are feature-complete and thoroughly tested. Such versions usually introduce new features, update dependencies, and may have API changes.
Patch Releases – Bugfix-only updates (e.g., v0.42.1) made after a stable release, based on a dedicated release branch.

It’s highly recommended to install only stable and patch releases in production environments.

Virtual Machine

Mon, 01 Jan 0001 00:00:00 +0000

A Virtual Machine (VM) simulates computer hardware, enabling various operating systems and applications to run in an isolated environment.

Deployment Details

The virtual machine is managed and hosted through KubeVirt, allowing you to harness the benefits of virtualization within your Kubernetes ecosystem.

Docs: KubeVirt User Guide
GitHub: KubeVirt Repository

Accessing virtual machine

You can access the virtual machine using the virtctl tool:

Virtual Machine

Mon, 01 Jan 0001 00:00:00 +0000

A Virtual Machine (VM) simulates computer hardware, enabling various operating systems and applications to run in an isolated environment.

Deployment Details

The virtual machine is managed and hosted through KubeVirt, allowing you to harness the benefits of virtualization within your Kubernetes ecosystem.

Docs: KubeVirt User Guide
GitHub: KubeVirt Repository

Accessing virtual machine

You can access the virtual machine using the virtctl tool:

VPC

Mon, 01 Jan 0001 00:00:00 +0000

VPC offers a subset of dedicated subnets with networking services related to it. As the service evolves, it will provide more ways to isolate your workloads.

Service details

To function, the service requires kube-ovn and multus CNI to be present, so by default it will only work on paas-full bundle. Kube-ovn provides VPC and Subnet resources and performs isolation and networking maintenance such as DHCP. Under the hood it uses ovn virtual routers and virtual switches. Multus enables a multi-nic capability, so a pod or a VM could have two or more network interfaces.

VPC

Mon, 01 Jan 0001 00:00:00 +0000

VPC offers a subset of dedicated subnets with networking services related to it. As the service evolves, it will provide more ways to isolate your workloads.

Service details

Managed VPN Service

Mon, 01 Jan 0001 00:00:00 +0000

A Virtual Private Network (VPN) is a critical tool for ensuring secure and private communication over the internet. Managed VPN Service simplifies the deployment and management of VPN server, enabling you to establish secure connections with ease.

VPN client applications: https://shadowsocks5.github.io/en/download/clients.html

Deployment Details

The VPN Service is powered by the Outline Server, an advanced and user-friendly VPN solution. Internally known as “Shadowbox”, which simplifies the process of setting up and sharing Shadowsocks servers. It operates by launching Shadowsocks instances on demand. Furthermore, Shadowbox is compatible with standard Shadowsocks clients, providing flexibility and ease of use for your VPN requirements.

Managed VPN Service

Mon, 01 Jan 0001 00:00:00 +0000

VPN client applications: https://shadowsocks5.github.io/en/download/clients.html

Deployment Details

Monitoring Logs

Mon, 01 Jan 0001 00:00:00 +0000

Collecting and Storing Logs

Cozystack uses Fluent Bit for log collection and VictoriaLogs for log storage and querying. Logs are collected from various sources within the cluster and stored in dedicated log storages configured per tenant.

Configuring Logs Storages

Log storages are configured through the monitoring hub parameters. Each tenant can have multiple log storage instances with customizable retention periods and storage sizes.

Parameter	Description	Type	Default
`logsStorages`	Array of log storage configurations	`[]object`	`[]`
`logsStorages[i].name`	Name of the storage instance	`string`	`""`
`logsStorages[i].retentionPeriod`	Retention period for logs (e.g., “30d”)	`string`	`"1"`
`logsStorages[i].storage`	Persistent volume size	`string`	`"10Gi"`
`logsStorages[i].storageClassName`	StorageClass for data persistence	`string`	`"replicated"`

For detailed configuration options, see Monitoring Hub Reference.

Monitoring Logs

Mon, 01 Jan 0001 00:00:00 +0000

Collecting and Storing Logs

Configuring Logs Storages

Log storages are configured through the monitoring hub parameters. Each tenant can have multiple log storage instances with customizable retention periods and storage sizes.

Parameter	Description	Type	Default
`logsStorages`	Array of log storage configurations	`[]object`	`[]`
`logsStorages[i].name`	Name of the storage instance	`string`	`""`
`logsStorages[i].retentionPeriod`	Retention period for logs (e.g., “30d”)	`string`	`"1"`
`logsStorages[i].storage`	Persistent volume size	`string`	`"10Gi"`
`logsStorages[i].storageClassName`	StorageClass for data persistence	`string`	`"replicated"`

For detailed configuration options, see Monitoring Hub Reference.

2. Install and Bootstrap a Kubernetes cluster

Mon, 01 Jan 0001 00:00:00 +0000

Objectives

We start this step of the tutorial, having three nodes with Talos Linux installed on them.

As a result of this step, we will have a Kubernetes cluster installed, configured, and ready to install Cozystack. We will also have a kubeconfgig for this cluster, and will have performed basic checks on the cluster.

Installing Kubernetes

Install and bootstrap a Kubernetes cluster using Talm, a declarative CLI configuration tool with ready configuration presets for Cozystack.

2. Install and Bootstrap a Kubernetes cluster

Mon, 01 Jan 0001 00:00:00 +0000

Objectives

We start this step of the tutorial, having three nodes with Talos Linux installed on them.

As a result of this step, we will have a Kubernetes cluster installed, configured, and ready to install Cozystack. We will also have a kubeconfig for this cluster, and will have performed basic checks on the cluster.

Installing Kubernetes

Install and bootstrap a Kubernetes cluster using Talm, a declarative CLI configuration tool with ready configuration presets for Cozystack.

Custom Metrics Collection

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Cozystack tenant monitoring supports scraping custom metrics from your own applications and exporters. The tenant VMAgent discovers scrape targets through Kubernetes namespace labels, allowing you to connect any application that exposes a Prometheus-compatible /metrics endpoint.

This guide explains how to create VMServiceScrape and VMPodScrape resources so that the tenant VMAgent collects your custom metrics and makes them available in Grafana.

Prerequisites

Monitoring is enabled for your tenant (see Monitoring Setup)
Your application or exporter is deployed and exposes a Prometheus-compatible /metrics endpoint
You have kubectl access to the cluster

Using VMServiceScrape

A VMServiceScrape tells the tenant VMAgent to scrape metrics from endpoints behind a Kubernetes Service.

Custom Metrics Collection

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This guide explains how to create VMServiceScrape and VMPodScrape resources so that the tenant VMAgent collects your custom metrics and makes them available in Grafana.

Prerequisites

Monitoring is enabled for your tenant (see Monitoring Setup)
Your application or exporter is deployed and exposes a Prometheus-compatible /metrics endpoint
You have kubectl access to the cluster

Using VMServiceScrape

A VMServiceScrape tells the tenant VMAgent to scrape metrics from endpoints behind a Kubernetes Service.

Local Cloud Controller Manager

Mon, 01 Jan 0001 00:00:00 +0000

The local-ccm package provides a lightweight cloud controller manager for self-managed clusters. It handles node IP detection and node lifecycle without requiring an external cloud provider.

What it does

External IP detection: Detects each node’s external IP via ip route get (default target: 8.8.8.8)
Node initialization: Removes the node.cloudprovider.kubernetes.io/uninitialized taint so pods can be scheduled
Node lifecycle controller (optional): Monitors NotReady nodes via ICMP ping and removes them after a configurable timeout

Install

cozypkg add cozystack.local-ccm

Talos machine config

All nodes in the cluster (including control plane) must have cloud-provider: external set so that kubelet defers node initialization to the cloud controller manager:

Install Talos Linux using PXE

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to install Talos Linux on bare metal servers or virtual machines using temporary DHCP and PXE servers running in Docker containers. This method requires an extra management machine, but allows for installing on multiple hosts at once.

Note that Cozystack provides its own Talos builds, which are tested and optimized for running a Cozystack cluster.

Dependencies

To install Talos using this method, you will need the following dependencies on the management host:

Install Talos Linux using PXE

Mon, 01 Jan 0001 00:00:00 +0000

Note that Cozystack provides its own Talos builds, which are tested and optimized for running a Cozystack cluster.

Dependencies

To install Talos using this method, you will need the following dependencies on the management host:

Use talosctl to bootstrap a Cozystack cluster

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to prepare a Talos Linux cluster for deploying Cozystack using talosctl, a specialized command line tool for managing Talos.

Prerequisites

By the start of this guide you should have Talos OS booted from ISO, but not initialized (bootstrapped), on several nodes. These nodes should belong to one subnet or have public IPs.

This guide uses an example where the nodes of a cluster are located in the subnet 192.168.123.0/24, having the following IP addresses:

Use talosctl to bootstrap a Cozystack cluster

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to prepare a Talos Linux cluster for deploying Cozystack using talosctl, a specialized command line tool for managing Talos.

Prerequisites

By the start of this guide you should have Talos OS booted from ISO, but not initialized (bootstrapped), on several nodes. These nodes should belong to one subnet or have public IPs.

This guide uses an example where the nodes of a cluster are located in the subnet 192.168.123.0/24, having the following IP addresses:

3. Install and Configure Cozystack

Mon, 01 Jan 0001 00:00:00 +0000

Objectives

In this step of the tutorial, we’ll install Cozystack on top of a Kubernetes cluster, prepared in the previous step.

The tutorial will guide you through the following stages:

Prepare a Cozystack configuration file
Install Cozystack by applying configuration
Configure storage
Configure networking
Deploy etcd, ingress and monitoring stack in the root tenant
Finalize deployment and access Cozystack dashboard

1. Prepare a Configuration File

We will start installing Cozystack by making a configuration file. Take the example below and write it in a file cozystack.yaml:

3. Install and Configure Cozystack

Mon, 01 Jan 0001 00:00:00 +0000

Objectives

This tutorial covers installing Cozystack as a ready-to-use platform. If you want to build your own platform by installing only specific components, see the BYOP (Build Your Own Platform) guide.

In this step of the tutorial, we’ll install Cozystack on top of a Kubernetes cluster, prepared in the previous step.

The tutorial will guide you through the following stages:

Install the Cozystack operator
Prepare a Cozystack configuration file and apply it
Configure storage
Configure networking
Deploy etcd, ingress and monitoring stack in the root tenant
Finalize deployment and access Cozystack dashboard

1. Install the Cozystack Operator

Install the Cozystack operator using the Helm chart from the OCI registry. The operator manages all Cozystack components and handles the Platform Package lifecycle.

Bootstrap an Air-Gapped Cluster

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

This guide outlines the steps to bootstrap a Cozystack cluster in an air-gapped environment.

Air-gapped installation means that the cluster has no direct access to the Internet. All necessary resources, such as images and metadata, must be available on the private network.

Configuring Talos Nodes

For installing with Talm, it’s enough to make all mentioned changes once in ./templates/_helpers.tpl and then build the actual node configuration files with talm template.

Bootstrap an Air-Gapped Cluster

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

This guide outlines the steps to bootstrap a Cozystack cluster in an air-gapped environment.

Air-gapped installation means that the cluster has no direct access to the Internet. All necessary resources, such as images and metadata, must be available on the private network.

Configuring Talos Nodes

For installing with Talm, it’s enough to make all mentioned changes once in ./templates/_helpers.tpl and then build the actual node configuration files with talm template.

Cluster Autoscaler for Azure

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to configure cluster-autoscaler for automatic node scaling in Azure with Talos Linux.

Prerequisites

Azure subscription with Contributor Service Principal
az CLI installed
Existing Talos Kubernetes cluster
Networking Mesh and Local CCM configured

Step 1: Create Azure Infrastructure

az login --service-principal \
 --username "<APP_ID>" \
 --password "<PASSWORD>" \
 --tenant "<TENANT_ID>"

1.2 Create Resource Group

az group create \
 --name <resource-group> \
 --location <location>

1.3 Create VNet and Subnet

az network vnet create \
 --resource-group <resource-group> \
 --name cozystack-vnet \
 --address-prefix 10.2.0.0/16 \
 --subnet-name workers \
 --subnet-prefix 10.2.0.0/24 \
 --location <location>

1.4 Create Network Security Group

az network nsg create \
 --resource-group <resource-group> \
 --name cozystack-nsg \
 --location <location>

# Allow WireGuard
az network nsg rule create \
 --resource-group <resource-group> \
 --nsg-name cozystack-nsg \
 --name AllowWireGuard \
 --priority 100 \
 --direction Inbound \
 --access Allow \
 --protocol Udp \
 --destination-port-ranges 51820

# Allow Talos API
az network nsg rule create \
 --resource-group <resource-group> \
 --nsg-name cozystack-nsg \
 --name AllowTalosAPI \
 --priority 110 \
 --direction Inbound \
 --access Allow \
 --protocol Tcp \
 --destination-port-ranges 50000

# Associate NSG with subnet
az network vnet subnet update \
 --resource-group <resource-group> \
 --vnet-name cozystack-vnet \
 --name workers \
 --network-security-group cozystack-nsg

Step 2: Create Talos Image

2.1 Generate Schematic ID

Create a schematic at factory.talos.dev with required extensions:

Buckets and Users

Mon, 01 Jan 0001 00:00:00 +0000

The Bucket application creates an S3 bucket via COSI and provisions per-user credentials as Kubernetes Secrets.

Creating a Bucket

A minimal bucket uses the default BucketClass and creates no users:

apiVersion: apps.cozystack.io/v1alpha1
kind: Bucket
metadata:
 name: my-bucket
 namespace: tenant-example
spec: {}

This provisions a BucketClaim against the default BucketClass (tenant-example). To make the bucket useful, add at least one user (see Users below).

Selecting a Storage Pool

If your SeaweedFS instance defines storage pools, use the storagePool field to target a specific pool:

Build Your Own Platform (BYOP)

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Cozystack can be used in BYOP (Build Your Own Platform) mode — similar to how Linux distributions let you install only the packages you need. Instead of deploying the full platform with all components, you selectively install only what you need from the Cozystack package repository.

This approach is useful when:

You have an existing Kubernetes cluster and only need specific components (e.g., a Postgres operator or monitoring).
Your cluster already has networking (CNI) and storage configured, and you don’t want Cozystack to manage them.
You want full control over which components are installed and how they are configured.

The workflow relies on two Kubernetes resources managed by the Cozystack Operator:

Cozystack Bundles: Overview and Comparison

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Bundles are pre-defined combinations of Cozystack components. Each bundle is tested, versioned, and guaranteed to work as a unit. They simplify installation, reduce the risk of misconfiguration, and make it easier to choose the right set of features for your deployment.

This guide is for infrastructure engineers, DevOps teams, and platform architects planning to deploy Cozystack in different environments. It explains how Cozystack bundles help tailor the installation to specific needs—whether you’re building a fully featured platform-as-a-service or just need a minimal Kubernetes cluster.

Cluster Scaling: Adding and Removing Nodes

Mon, 01 Jan 0001 00:00:00 +0000

How to add a node to a Cozystack cluster

Adding a node is done in a way similar to regular Cozystack installation.

Install Talos on the node, using the Cozystack’s custom-built Talos image.

Generate the configuration for the new node, using the Talm or talosctl guide.

For example, configuring a control plane node:

talm template -e 192.168.123.20 -n 192.168.123.20 -t templates/controlplane.yaml -i > nodes/nodeN.yaml

and for a worker node:

talm template -e 192.168.123.20 -n 192.168.123.20 -t templates/worker.yaml -i > nodes/nodeN.yaml

Apply the generated configuration to the node, using the Talm or talosctl guide. For example:

Cluster Scaling: Adding and Removing Nodes

Mon, 01 Jan 0001 00:00:00 +0000

How to add a node to a Cozystack cluster

Adding a node is done in a way similar to regular Cozystack installation.

Install Talos on the node, using the Cozystack’s custom-built Talos image.

Generate the configuration for the new node, using the Talm or talosctl guide.

For example, configuring a control plane node:

talm template -e 192.168.123.20 -n 192.168.123.20 -t templates/controlplane.yaml -i > nodes/nodeN.yaml

and for a worker node:

talm template -e 192.168.123.20 -n 192.168.123.20 -t templates/worker.yaml -i > nodes/nodeN.yaml

Apply the generated configuration to the node, using the Talm or talosctl guide. For example:

Configuring DRBD Resync Controller in LINSTOR

Mon, 01 Jan 0001 00:00:00 +0000

Cozystack administrators can adjust DRBD synchronization performance by setting tuning parameters for the LINSTOR Controller.

This allows you to optimize the speed of resynchronization without overloading the replication network or storage backend.

For detailed explanations of all available parameters and tuning recommendations, please refer to the official LINBIT guide: Tuning the DRBD Resync Controller.

For a multi-datacenter setup, also read the Multi-DC DRBD configuration.

Recommended Settings for 10G Networks

We consider the following values to be optimal for clusters connected with a 10-Gigabit network:

Configuring DRBD Resync Controller in LINSTOR

Mon, 01 Jan 0001 00:00:00 +0000

Cozystack administrators can adjust DRBD synchronization performance by setting tuning parameters for the LINSTOR Controller.

This allows you to optimize the speed of resynchronization without overloading the replication network or storage backend.

For detailed explanations of all available parameters and tuning recommendations, please refer to the official LINBIT guide: Tuning the DRBD Resync Controller.

For a multi-datacenter setup, also read the Multi-DC DRBD configuration.

Recommended Settings for 10G Networks

We consider the following values to be optimal for clusters connected with a 10-Gigabit network:

Managed FerretDB Service

Mon, 01 Jan 0001 00:00:00 +0000

FerretDB is an open source MongoDB alternative. It translates MongoDB wire protocol queries to SQL and can be used as a direct replacement for MongoDB 5.0+. Internally, FerretDB service is backed by Postgres.

Parameters

Common parameters

Name	Description	Type	Value
`replicas`	Number of replicas.	`int`	`2`
`resources`	Explicit CPU and memory configuration for each FerretDB replica. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`resources.cpu`	CPU available to each replica.	`quantity`	`""`
`resources.memory`	Memory (RAM) available to each replica.	`quantity`	`""`
`resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`micro`
`size`	Persistent Volume Claim size available for application data.	`quantity`	`10Gi`
`storageClass`	StorageClass used to store the data.	`string`	`""`
`external`	Enable external access from outside the cluster.	`bool`	`false`

Application-specific parameters

Name	Description	Type	Value
`quorum`	Configuration for quorum-based synchronous replication.	`object`	`{}`
`quorum.minSyncReplicas`	Minimum number of synchronous replicas required for commit.	`int`	`0`
`quorum.maxSyncReplicas`	Maximum number of synchronous replicas allowed (must be less than total replicas).	`int`	`0`
`users`	Users configuration map.	`map[string]object`	`{}`
`users[name].password`	Password for the user.	`string`	`""`

Backup parameters

Name	Description	Type	Value
`backup`	Backup configuration.	`object`	`{}`
`backup.enabled`	Enable regular backups (default: false).	`bool`	`false`
`backup.schedule`	Cron schedule for automated backups.	`string`	`0 2 * * * *`
`backup.retentionPolicy`	Retention policy.	`string`	`30d`
`backup.endpointURL`	S3 endpoint URL for uploads.	`string`	`http://minio-gateway-service:9000`
`backup.destinationPath`	Path to store the backup (e.g. s3://bucket/path/to/folder/).	`string`	`s3://bucket/path/to/folder/`
`backup.s3AccessKey`	Access key for S3 authentication.	`string`	`<your-access-key>`
`backup.s3SecretKey`	Secret key for S3 authentication.	`string`	`<your-secret-key>`

Bootstrap (recovery) parameters

Name	Description	Type	Value
`bootstrap`	Bootstrap configuration.	`object`	`{}`
`bootstrap.enabled`	Restore database cluster from a backup.	`bool`	`false`
`bootstrap.recoveryTime`	Timestamp (RFC3339) for point-in-time recovery; empty means latest.	`string`	`""`
`bootstrap.oldName`	Name of database cluster before deletion.	`string`	`""`

Parameter examples and reference

resources and resourcesPreset

resources sets explicit CPU and memory configurations for each replica. When left empty, the preset defined in resourcesPreset is applied.

Managed Nginx-based HTTP Cache Service

Mon, 01 Jan 0001 00:00:00 +0000

The Nginx-based HTTP caching service is designed to optimize web traffic and enhance web application performance. This service combines custom-built Nginx instances with HAProxy for efficient caching and load balancing.

Deployment information

The Nginx instances include the following modules and features:

VTS module for statistics
Integration with ip2location
Integration with ip2proxy
Support for 51Degrees
Cache purge functionality

HAproxy plays a vital role in this setup by directing incoming traffic to specific Nginx instances based on a consistent hash calculated from the URL. Each Nginx instance includes a Persistent Volume Claim (PVC) for storing cached content, ensuring fast and reliable access to frequently used resources.

Managed Nginx-based HTTP Cache Service

Mon, 01 Jan 0001 00:00:00 +0000

Deployment information

The Nginx instances include the following modules and features:

VTS module for statistics
Integration with ip2location
Integration with ip2proxy
Support for 51Degrees
Cache purge functionality

Install Talos Linux using ISO

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to install Talos Linux on bare metal servers or virtual machines. Note that Cozystack provides its own Talos builds, which are tested and optimized for running a Cozystack cluster.

Installation

Download Talos Linux asset from the Cozystack’s releases page.

wget https://github.com/cozystack/cozystack/releases/latest/download/metal-amd64.iso

Boot your machine with ISO attached.
Click and fill your network settings:

Next steps

Once you have installed Talos, proceed by installing and bootstrapping a Kubernetes cluster.

Install Talos Linux using ISO

Mon, 01 Jan 0001 00:00:00 +0000

Installation

Download Talos Linux asset from the Cozystack’s releases page.

wget https://github.com/cozystack/cozystack/releases/latest/download/metal-amd64.iso

Boot your machine with ISO attached.
Click and fill your network settings:

Next steps

Once you have installed Talos, proceed by installing and bootstrapping a Kubernetes cluster.

Kube scheduler configuration

Mon, 01 Jan 0001 00:00:00 +0000

Label nodes

kubectl label node <nodename> topology.kubernetes.io/zone=A

Create global app topology spread constraints

apiVersion: v1
kind: ConfigMap
metadata:
 name: cozystack-scheduling
 namespace: cozy-system
data:
 globalAppTopologySpreadConstraints: |
 topologySpreadConstraints:
 - maxSkew: 1
 topologyKey: topology.kubernetes.io/zone
 whenUnsatisfiable: DoNotSchedule

Configure PodTopologySpread

See: https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#cluster-level-default-constraints

For Talm installation: Add to templates/_helpers.tpl.

cluster:
...
 scheduler:
 config:
 apiVersion: kubescheduler.config.k8s.io/v1beta3
 kind: KubeSchedulerConfiguration
 profiles:
 - schedulerName: default-scheduler
 pluginConfig:
 - name: PodTopologySpread
 args:
 defaultConstraints:
 - maxSkew: 1
 topologyKey: topology.kubernetes.io/zone
 whenUnsatisfiable: ScheduleAnyway
 defaultingType: List

Apply changes:

Kube scheduler configuration

Mon, 01 Jan 0001 00:00:00 +0000

Label nodes

kubectl label node <nodename> topology.kubernetes.io/zone=A

Create global app topology spread constraints

apiVersion: v1
kind: ConfigMap
metadata:
 name: cozystack-scheduling
 namespace: cozy-system
data:
 globalAppTopologySpreadConstraints: |
 topologySpreadConstraints:
 - maxSkew: 1
 topologyKey: topology.kubernetes.io/zone
 whenUnsatisfiable: DoNotSchedule

Configure PodTopologySpread

See: https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#cluster-level-default-constraints

For Talm installation: Add to templates/_helpers.tpl.

cluster:
...
 scheduler:
 config:
 apiVersion: kubescheduler.config.k8s.io/v1beta3
 kind: KubeSchedulerConfiguration
 profiles:
 - schedulerName: default-scheduler
 pluginConfig:
 - name: PodTopologySpread
 args:
 defaultConstraints:
 - maxSkew: 1
 topologyKey: topology.kubernetes.io/zone
 whenUnsatisfiable: ScheduleAnyway
 defaultingType: List

Apply changes:

Troubleshooting Kube-OVN

Mon, 01 Jan 0001 00:00:00 +0000

Getting information about Kube-OVN database state

# Northbound DB
kubectl -n cozy-kubeovn exec deploy/ovn-central -c ovn-central -- ovn-appctl \
 -t /var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound
# Southbound DB
kubectl -n cozy-kubeovn exec deploy/ovn-central -c ovn-central -- ovn-appctl \
 -t /var/run/ovn/ovnsb_db.ctl cluster/status OVN_Southbound

Example output:

Name: OVN_Northbound
Cluster ID: abf6 (abf66f15-9382-4b2b-b14c-355d64ae1bda)
Server ID: 8d8a (8d8a2985-c444-43bb-99f6-21c82f05b58d)
Address: ssl:[10.200.1.22]:6643
Status: cluster member
Role: leader
Term: 3
Leader: self
Vote: self

Last Election started 146211 ms ago, reason: leadership_transfer
Last Election won: 146202 ms ago
Election timer: 5000
Log: [2, 1569]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: ->2a66 ->c23f <-2a66 <-c23f
Disconnections: 30
Servers:
 2a66 (2a66 at ssl:[10.200.1.18]:6643) next_index=1569 match_index=1568 last msg 17 ms ago
 8d8a (8d8a at ssl:[10.200.1.22]:6643) (self) next_index=1471 match_index=1568
 c23f (c23f at ssl:[10.200.1.1]:6643) next_index=1569 match_index=1568 last msg 18 ms ago

To kick a node out of the cluster (for example, if it is down, and you want to remove it from the cluster), use:

Troubleshooting Kube-OVN

Mon, 01 Jan 0001 00:00:00 +0000

Getting information about Kube-OVN database state

# Northbound DB
kubectl -n cozy-kubeovn exec deploy/ovn-central -c ovn-central -- ovn-appctl \
 -t /var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound
# Southbound DB
kubectl -n cozy-kubeovn exec deploy/ovn-central -c ovn-central -- ovn-appctl \
 -t /var/run/ovn/ovnsb_db.ctl cluster/status OVN_Southbound

Example output:

Name: OVN_Northbound
Cluster ID: abf6 (abf66f15-9382-4b2b-b14c-355d64ae1bda)
Server ID: 8d8a (8d8a2985-c444-43bb-99f6-21c82f05b58d)
Address: ssl:[10.200.1.22]:6643
Status: cluster member
Role: leader
Term: 3
Leader: self
Vote: self

Last Election started 146211 ms ago, reason: leadership_transfer
Last Election won: 146202 ms ago
Election timer: 5000
Log: [2, 1569]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: ->2a66 ->c23f <-2a66 <-c23f
Disconnections: 30
Servers:
 2a66 (2a66 at ssl:[10.200.1.18]:6643) next_index=1569 match_index=1568 last msg 17 ms ago
 8d8a (8d8a at ssl:[10.200.1.22]:6643) (self) next_index=1471 match_index=1568
 c23f (c23f at ssl:[10.200.1.1]:6643) next_index=1569 match_index=1568 last msg 18 ms ago

To kick a node out of the cluster (for example, if it is down, and you want to remove it from the cluster), use:

Using Cozystack to build private cloud

Mon, 01 Jan 0001 00:00:00 +0000

You can use Cozystack as platform to build a private cloud powered by Infrastructure-as-Code

Overview

One of the use cases is a self-portal for users within your company, where they can order the service they’re interested in or a managed database.

You can implement best GitOps practices, where users will launch their own Kubernetes clusters and databases for their needs with a simple commit of configuration into your infrastructure Git repository.

Using Cozystack to build private cloud

Mon, 01 Jan 0001 00:00:00 +0000

You can use Cozystack as platform to build a private cloud powered by Infrastructure-as-Code

Overview

One of the use cases is a self-portal for users within your company, where they can order the service they’re interested in or a managed database.

ServiceAccount Tokens for API Access

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Before you begin, make sure that:

A tenant already exists in Cozystack. See Create a User Tenant if you haven’t created one yet.
You have access to the tenant namespace — either via OIDC credentials or an administrative kubeconfig.
kubectl is installed and configured.
(Optional) jq is installed.

Retrieving the ServiceAccount Token

Each tenant in Cozystack has a Secret that contains a ServiceAccount token. The Secret has the same name as the tenant and is located in the tenant’s namespace.

ServiceAccount Tokens for API Access

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Before you begin, make sure that:

A tenant already exists in Cozystack. See Create a User Tenant if you haven’t created one yet.
You have access to the tenant namespace — either via OIDC credentials or an administrative kubeconfig.
kubectl is installed and configured.
(Optional) jq is installed.

Retrieving the ServiceAccount Token

Each tenant in Cozystack has a Secret that contains a ServiceAccount token. The Secret has the same name as the tenant and is located in the tenant’s namespace.

Node labels

Mon, 01 Jan 0001 00:00:00 +0000

How topology labels work

When running a Kubernetes cluster in multiple datacenters, it’s important to know when you want to schedule workloads close to each other (for example, a database and a backend application) or when you want to spread them out (for example, multiple replicas of frontend, or database replicas, or volume replicas). The first step to achieving this is to label Kubernetes nodes. Public clouds typically use the zone and region terms. In Kubernetes, the most common way to designate a geographical location is to use topology.kubernetes.io/zone and topology.kubernetes.io/region labels (or only the zone one).

Node labels

Mon, 01 Jan 0001 00:00:00 +0000

How topology labels work

Cozystack Variants: Overview and Comparison

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Variants are pre-defined configurations of Cozystack that determine which bundles and components are enabled. Each variant is tested, versioned, and guaranteed to work as a unit. They simplify installation, reduce the risk of misconfiguration, and make it easier to choose the right set of features for your deployment.

This guide is for infrastructure engineers, DevOps teams, and platform architects planning to deploy Cozystack in different environments. It explains how Cozystack variants help tailor the installation to specific needs—whether you’re building a fully featured platform-as-a-service or need full manual control over installed packages.

Virtual Machine Disk

Mon, 01 Jan 0001 00:00:00 +0000

A Virtual Machine Disk

Parameters

Common parameters

Name	Description	Type	Value
`source`	The source image location used to create a disk.	`object`	`{}`
`source.image`	Use image by name.	`*object`	`null`
`source.image.name`	Name of the image to use (uploaded as “golden image” or from the list: `ubuntu`, `fedora`, `cirros`, `alpine`, `talos`).	`string`	`""`
`source.upload`	Upload local image.	`*object`	`null`
`source.http`	Download image from an HTTP source.	`*object`	`null`
`source.http.url`	URL to download the image.	`string`	`""`
`optical`	Defines if disk should be considered optical.	`bool`	`false`
`storage`	The size of the disk allocated for the virtual machine.	`quantity`	`5Gi`
`storageClass`	StorageClass used to store the data.	`string`	`replicated`

Virtual Machine Disk

Mon, 01 Jan 0001 00:00:00 +0000

A Virtual Machine Disk

Parameters

Common parameters

Name	Description	Type	Value
`source`	The source image location used to create a disk.	`object`	`{}`
`source.image`	Use image by name.	`*object`	`null`
`source.image.name`	Name of the image to use (uploaded as “golden image” or from the list: `ubuntu`, `fedora`, `cirros`, `alpine`, `talos`).	`string`	`""`
`source.upload`	Upload local image.	`*object`	`null`
`source.http`	Download image from an HTTP source.	`*object`	`null`
`source.http.url`	URL to download the image.	`string`	`""`
`optical`	Defines if disk should be considered optical.	`bool`	`false`
`storage`	The size of the disk allocated for the virtual machine.	`quantity`	`5Gi`
`storageClass`	StorageClass used to store the data.	`string`	`replicated`

How to install Cozystack in Oracle Cloud Infrastructure

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

This guide explains how to install Talos on Oracle Cloud Infrastructure and deploy a Kubernetes cluster that is ready for Cozystack. After completing the guide, you will be ready to proceed with installing Cozystack itself.

This guide was created to support deployment of development clusters by the Cozystack team. If you face any problems while going through the guide, please raise an issue in cozystack/website or come and share your experience in the Cozystack community.

1. Upload Talos Image to Oracle Cloud

The first step is to make a Talos Linux installation image available for use in Oracle Cloud as a custom image.

How to install Cozystack in Oracle Cloud Infrastructure

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

1. Upload Talos Image to Oracle Cloud

The first step is to make a Talos Linux installation image available for use in Oracle Cloud as a custom image.

Build Your Own Platform (BYOP)

Mon, 01 Jan 0001 00:00:00 +0000

Cozystack can be used in BYOP (Build Your Own Platform) mode — installing only the components you need from the Cozystack package repository, rather than deploying the full platform.

Overview

Cozystack provides a package management system inspired by Linux distribution package managers. The Cozystack Operator manages PackageSource and Package resources, while the cozypkg CLI tool provides an interactive interface for listing available packages, resolving dependencies, and installing them selectively.

This approach is useful when:

Cozystack Components Reference

Mon, 01 Jan 0001 00:00:00 +0000

Overwriting Component Parameters

You might want to override specific options for the components. To achieve this, you must specify values in JSON or YAML format using the data.values-<component> option in the Cozystack ConfigMap.

For example, if you want to overwrite k8sServiceHost and k8sServicePort for cilium, take a look at its values.yaml file.

Then specify these options in the values-cilium section of your Cozystack configuration, as shown below:

apiVersion: v1
kind: ConfigMap
metadata:
 name: cozystack
 namespace: cozy-system
data:
 bundle-name: "distro-full"
 ipv4-pod-cidr: "10.244.0.0/16"
 ipv4-svc-cidr: "10.96.0.0/16"
 values-cilium: |
 cilium:
 k8sServiceHost: 11.22.33.44
 k8sServicePort: 6443

Enabling and Disabling Components

Bundles have optional components that need to be explicitly enabled (included) in the installation. Regular bundle components can, on the other hand, be disabled (excluded) from the installation, when you don’t need them.

Cozystack Components Reference

Mon, 01 Jan 0001 00:00:00 +0000

Overwriting Component Parameters

You might want to override specific options for the components. To achieve this, modify the corresponding Package resource and specify values in the spec.components section. The values structure follows the values.yaml of the respective system chart in the Cozystack repository.

For example, if you want to enable FRR-K8s mode for MetalLB, look at its values.yaml to understand the available parameters, then modify the cozystack.metallb Package:

apiVersion: cozystack.io/v1alpha1
kind: Package
metadata:
 name: cozystack.metallb
 namespace: cozy-system
spec:
 variant: default
 components:
 metallb:
 values:
 metallb:
 frrk8s:
 enabled: true

Enabling and Disabling Components

How to generate kubeconfig for tenant users

Mon, 01 Jan 0001 00:00:00 +0000

To generate a kubeconfig for tenant users, use the following script. As a result, you’ll receive the tenant-kubeconfig file, which you can provide to the user.

SERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
kubectl get secret tenant-root -n tenant-root -o go-template='
apiVersion: v1
kind: Config
clusters:
- name: tenant-root
 cluster:
 server: '"$SERVER"'
 certificate-authority-data: {{ index .data "ca.crt" }}
contexts:
- name: tenant-root
 context:
 cluster: tenant-root
 namespace: {{ index .data "namespace" | base64decode }}
 user: tenant-root
current-context: tenant-root
users:
- name: tenant-root
 user:
 token: {{ index .data "token" | base64decode }}
' \
> tenant-root.kubeconfig

How to generate kubeconfig for tenant users

Mon, 01 Jan 0001 00:00:00 +0000

To generate a kubeconfig for tenant users, use the following script. As a result, you’ll receive the tenant-kubeconfig file, which you can provide to the user.

SERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
kubectl get secret tenant-root -n tenant-root -o go-template='
apiVersion: v1
kind: Config
clusters:
- name: tenant-root
 cluster:
 server: '"$SERVER"'
 certificate-authority-data: {{ index .data "ca.crt" }}
contexts:
- name: tenant-root
 context:
 cluster: tenant-root
 namespace: {{ index .data "namespace" | base64decode }}
 user: tenant-root
current-context: tenant-root
users:
- name: tenant-root
 user:
 token: {{ index .data "token" | base64decode }}
' \
> tenant-root.kubeconfig

How to configure GitLab as an Identity Provider

Mon, 01 Jan 0001 00:00:00 +0000

You can use Gitlab identity provider for Keycloak

Overview

Create Application in Gitlab

Open https://gitlab.com/groups/<YOUR_GROUP>/-/settings/applications
Click Add new application
Name: cozy, Redirect URI: https://keycloak.<root-host>/realms/cozy/broker/gitlab/endpoint
Enable Confidential, api, read_api, read_user, openid, profile, email
Copy and save Secret

Configure Keycloak Identity Provider

Create a KeycloakRealmIdentityProvider resource with the following configuration:

apiVersion: v1.edp.epam.com/v1
kind: KeycloakRealmIdentityProvider
metadata:
 name: gitlab
spec:
 realmRef:
 name: keycloakrealm-cozy
 kind: ClusterKeycloakRealm
 alias: gitlab
 authenticateByDefault: false
 enabled: true
 providerId: "gitlab"
 config:
 clientId: "YOUR GITLAB APP ID"
 clientSecret: "YOUR GITLAB APP SECRET"
 syncMode: "IMPORT"
 mappers:
 - name: "username"
 identityProviderMapper: "oidc-username-idp-mapper"
 identityProviderAlias: "gitlab"
 config:
 target: "LOCAL"
 syncMode: "INHERIT"
 template: "${ALIAS}---${CLAIM.preferred_username}"

How to configure GitLab as an Identity Provider

Mon, 01 Jan 0001 00:00:00 +0000

You can use Gitlab identity provider for Keycloak

Overview

Create Application in Gitlab

Open https://gitlab.com/groups/<YOUR_GROUP>/-/settings/applications
Click Add new application
Name: cozy, Redirect URI: https://keycloak.<root-host>/realms/cozy/broker/gitlab/endpoint
Enable Confidential, api, read_api, read_user, openid, profile, email
Copy and save Secret

Configure Keycloak Identity Provider

Create a KeycloakRealmIdentityProvider resource with the following configuration:

apiVersion: v1.edp.epam.com/v1
kind: KeycloakRealmIdentityProvider
metadata:
 name: gitlab
spec:
 realmRef:
 name: keycloakrealm-cozy
 kind: ClusterKeycloakRealm
 alias: gitlab
 authenticateByDefault: false
 enabled: true
 providerId: "gitlab"
 config:
 clientId: "YOUR GITLAB APP ID"
 clientSecret: "YOUR GITLAB APP SECRET"
 syncMode: "IMPORT"
 mappers:
 - name: "username"
 identityProviderMapper: "oidc-username-idp-mapper"
 identityProviderAlias: "gitlab"
 config:
 target: "LOCAL"
 syncMode: "INHERIT"
 template: "${ALIAS}---${CLAIM.preferred_username}"

How to configure Google as an Identity Provider

Mon, 01 Jan 0001 00:00:00 +0000

Configure Google

Head over to Google Console, login in to the console using Google account and you will see Google Developer Console. Once logged in, head over the top left drop-down to create new project.
Click on “New Project” to proceed.
Enter the project name of your choice and select the Organisation if you have multiple organisations. Once done click on “Create”
Once the project is created you will get a pop-up suggesting to configure the consent screen. If not then head over to the Dashboard and head over to “Explore and enable APIs” options. Then Click on “Credentials” > “Configure Consent Screen” and head over to the next step.

How to configure Google as an Identity Provider

Mon, 01 Jan 0001 00:00:00 +0000

Configure Google

Head over to Google Console, login in to the console using Google account and you will see Google Developer Console. Once logged in, head over the top left drop-down to create new project.
Click on “New Project” to proceed.
Enter the project name of your choice and select the Organisation if you have multiple organisations. Once done click on “Create”
Once the project is created you will get a pop-up suggesting to configure the consent screen. If not then head over to the Dashboard and head over to “Explore and enable APIs” options. Then Click on “Credentials” > “Configure Consent Screen” and head over to the next step.

How to install Cozystack in Hetzner

Mon, 01 Jan 0001 00:00:00 +0000

This guide will help you to install Cozystack on a dedicated server from Hetzner. There are several steps to follow, including preparing the infrastructure, installing Talos Linux, configuring cloud-init, and bootstrapping the cluster.

Prepare Infrastructure and Networking

Installation on Hetzner includes the common hardware requirements with several additions.

Networking Options

There are two options for network connectivity between Cozystack nodes in the cluster:

Creating a subnet using vSwitch. This option is recommended for production environments.

How to install Cozystack in Hetzner

Mon, 01 Jan 0001 00:00:00 +0000

Prepare Infrastructure and Networking

Installation on Hetzner includes the common hardware requirements with several additions.

Networking Options

There are two options for network connectivity between Cozystack nodes in the cluster:

Creating a subnet using vSwitch. This option is recommended for production environments.

Managed Kafka Service

Mon, 01 Jan 0001 00:00:00 +0000

Parameters

Common parameters

Name	Description	Type	Value
`external`	Enable external access from outside the cluster.	`bool`	`false`

Application-specific parameters

Name	Description	Type	Value
`topics`	Topics configuration.	`[]object`	`[]`
`topics[i].name`	Topic name.	`string`	`""`
`topics[i].partitions`	Number of partitions.	`int`	`0`
`topics[i].replicas`	Number of replicas.	`int`	`0`
`topics[i].config`	Topic configuration.	`object`	`{}`

Kafka configuration

Name	Description	Type	Value
`kafka`	Kafka configuration.	`object`	`{}`
`kafka.replicas`	Number of Kafka replicas.	`int`	`3`
`kafka.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`kafka.resources.cpu`	CPU available to each replica.	`quantity`	`""`
`kafka.resources.memory`	Memory (RAM) available to each replica.	`quantity`	`""`
`kafka.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`kafka.size`	Persistent Volume size for Kafka.	`quantity`	`10Gi`
`kafka.storageClass`	StorageClass used to store the Kafka data.	`string`	`""`

ZooKeeper configuration

Name	Description	Type	Value
`zookeeper`	ZooKeeper configuration.	`object`	`{}`
`zookeeper.replicas`	Number of ZooKeeper replicas.	`int`	`3`
`zookeeper.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`zookeeper.resources.cpu`	CPU available to each replica.	`quantity`	`""`
`zookeeper.resources.memory`	Memory (RAM) available to each replica.	`quantity`	`""`
`zookeeper.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`zookeeper.size`	Persistent Volume size for ZooKeeper.	`quantity`	`5Gi`
`zookeeper.storageClass`	StorageClass used to store the ZooKeeper data.	`string`	`""`

Parameter examples and reference

resources and resourcesPreset

resources sets explicit CPU and memory configurations for each replica. When left empty, the preset defined in resourcesPreset is applied.

Using Cozystack as Kubernetes distribution

Mon, 01 Jan 0001 00:00:00 +0000

You can use Cozystack as Kubernetes distribution for Bare Metal

Overview

We created Cozystack primarily for our own needs, having vast experience in building reliable systems on bare metal infrastructure. This experience led to the formation of a separate boxed product, which is aimed at standardizing and providing a ready-to-use tool for managing your infrastructure.

Currently, Cozystack already solves a huge scope of infrastructure tasks: starting from provisioning bare-metal servers, having a ready monitoring system, fast and reliable storage, a network fabric with the possibility of interconnect with your infrastructure, the ability to run virtual machines, databases, and much more right out of the box.

LINSTOR DRBD Configuration

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

This guide explains the configuration needed to use LINSTOR storage in a stretched (distributed) Cozystack cluster.

DRBD (Distributed Replicated Block Device) is a kernel-level block device replication system that works over the network. LINSTOR server manages DRBD volumes, including their creation, deletion, and orchestration across nodes.

Challenges of using DRBD

DRBD only considers data as written once it reaches a quorum of nodes. But as it presents itself as a block device to the end user, it must return an error within a given timeout if there are not enough nodes to establish a quorum.

LINSTOR DRBD Configuration

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

This guide explains the configuration needed to use LINSTOR storage in a stretched (distributed) Cozystack cluster.

Challenges of using DRBD

Talos Linux in Cozystack

Mon, 01 Jan 0001 00:00:00 +0000

Why Cozystack is Using Talos Linux

Talos Linux is a Linux distribution made and optimized for one job: to run Kubernetes. It is the foundation of reliability and security in Cozystack cluster. Selecting it enables Cozystack to strictly limit the technology stack and make the system stable as a rock.

Let’s see why Cozystack developers chose Talos as the foundation of a Kubernetes cluster and what it brings to Cozystack.

Talos Linux in Cozystack

Mon, 01 Jan 0001 00:00:00 +0000

Why Cozystack is Using Talos Linux

Let’s see why Cozystack developers chose Talos as the foundation of a Kubernetes cluster and what it brings to Cozystack.

Managed TCP Load Balancer Service

Mon, 01 Jan 0001 00:00:00 +0000

The Managed TCP Load Balancer Service simplifies the deployment and management of load balancers. It efficiently distributes incoming TCP traffic across multiple backend servers, ensuring high availability and optimal resource utilization.

Deployment Details

Managed TCP Load Balancer Service efficiently utilizes HAProxy for load balancing purposes. HAProxy is a well-established and reliable solution for distributing incoming TCP traffic across multiple backend servers, ensuring high availability and efficient resource utilization. This deployment choice guarantees the seamless and dependable operation of your load balancing infrastructure.

Managed TCP Load Balancer Service

Mon, 01 Jan 0001 00:00:00 +0000

Deployment Details

Using NFS shares with Cozystack

Mon, 01 Jan 0001 00:00:00 +0000

Enable NFS driver

Add bundle-enable: nfs-driver to your Cozystack configuration:

bundle-enable: nfs-driver

Wait a minute for the platform chart to reconcile, then verify the HelmRelease has been created:

kubectl get helmrelease --namespace cozy-nfs-driver nfs-driver

apt install nfs-server
mkdir /data
chmod 777 /data
echo '/data *(rw,sync,no_subtree_check)' >> /etc/exports
exportfs -a

Configure connection

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
 name: nfs
provisioner: nfs.csi.k8s.io
parameters:
 server: 10.244.57.210
 share: /data
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true
mountOptions:
 - nfsvers=4.1

Order volume

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: task-pv-claim
spec:
 storageClassName: nfs
 accessModes:
 - ReadWriteMany
 resources:
 requests:
 storage: 3Gi

Using NFS shares with Cozystack

Mon, 01 Jan 0001 00:00:00 +0000

Enable NFS driver

Add nfs-driver to bundles.enabledPackages in the Platform Package:

kubectl patch packages.cozystack.io cozystack.cozystack-platform --type=json \
 -p '[{"op": "add", "path": "/spec/components/platform/values/bundles/enabledPackages/-", "value": "nfs-driver"}]'

Wait a minute for the platform chart to reconcile, then verify the HelmRelease has been created:

kubectl get helmrelease --namespace cozy-nfs-driver nfs-driver

apt install nfs-server
mkdir /data
chmod 777 /data
echo '/data *(rw,sync,no_subtree_check)' >> /etc/exports
exportfs -a

Configure connection

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
 name: nfs
provisioner: nfs.csi.k8s.io
parameters:
 server: 10.244.57.210
 share: /data
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true
mountOptions:
 - nfsvers=4.1

Order volume

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: task-pv-claim
spec:
 storageClassName: nfs
 accessModes:
 - ReadWriteMany
 resources:
 requests:
 storage: 3Gi

Velero Backup Configuration

Mon, 01 Jan 0001 00:00:00 +0000

This guide is for cluster administrators who configure the backup infrastructure in Cozystack: S3 storage, Velero locations, backup strategies, and BackupClasses. Tenant users then use existing BackupClasses to create BackupJobs and Plans.

Prerequisites

Administrator access to the Cozystack (management) cluster.
S3-compatible storage: if you want to store backups in Cozy you need enable SeaweedFS and create a Bucket or can use another external S3 service.
Enable disabled by default component cozystack.velero in bundles.enabledPackages of the Platform Package. And for tenant clusters, set spec.addons.velero.enabled to true in the Kubernetes resource.

1. Set up storage credentials and configuration

Create the following resources in the management cluster in the cozy-velero namespace so that Velero can store backups and volume snapshots.

Virtual Machine (simple)

Mon, 01 Jan 0001 00:00:00 +0000

A Virtual Machine (VM) simulates computer hardware, enabling various operating systems and applications to run in an isolated environment.

Deployment Details

The virtual machine is managed and hosted through KubeVirt, allowing you to harness the benefits of virtualization within your Kubernetes ecosystem.

Docs: KubeVirt User Guide
GitHub: KubeVirt Repository

Accessing virtual machine

You can access the virtual machine using the virtctl tool:

Creating and Using Named VM Images

Mon, 01 Jan 0001 00:00:00 +0000

Golden images in Cozystack allow administrators to prepare named operating system images that users can later reuse when creating virtual machines.
This guide explains the benefits of golden images, how to create them, and how to use them when deploying VMs.

By default, every time a user creates a virtual machine, Cozystack downloads the required image from its source URL.
This can become a bottleneck when multiple VMs are created in quick succession.
Golden images solve this problem by caching the image locally, eliminating repeated downloads and speeding up deployment.

Creating and Using Named VM Images

Mon, 01 Jan 0001 00:00:00 +0000

Monitoring Alerting

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The alerting system in Cozystack integrates Prometheus, Alertmanager, and Alerta to provide comprehensive monitoring and notification capabilities. Alerts are generated based on metrics collected by VMAgent and stored in VMCluster, then routed through Alertmanager for grouping and deduplication, and finally managed by Alerta for notifications via various channels like Telegram and Slack.

Alerting Flow

sequenceDiagram
 participant P as Prometheus
 participant AM as Alertmanager
 participant A as Alerta
 participant T as Telegram
 participant S as Slack
 P->>AM: Send Alert
 AM->>A: Forward Alert
 A->>T: Send Notification
 A->>S: Send Notification

Configuring Alerts in Alerta

Alerta is the alerting system integrated into Cozystack’s monitoring stack. It processes alerts from various sources and provides notifications through multiple channels.

Monitoring Alerting

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Alerting Flow

sequenceDiagram
 participant P as Prometheus
 participant AM as Alertmanager
 participant A as Alerta
 participant T as Telegram
 participant S as Slack
 P->>AM: Send Alert
 AM->>A: Forward Alert
 A->>T: Send Notification
 A->>S: Send Notification

Configuring Alerts in Alerta

Alerta is the alerting system integrated into Cozystack’s monitoring stack. It processes alerts from various sources and provides notifications through multiple channels.

Enable OIDC Server

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

OIDC Configuration Your API server must be configured to use OIDC. If you are using Talos Linux, your machine configuration should include the following parameters:

cluster:
 apiServer:
 extraArgs:
 oidc-issuer-url: "https://keycloak.example.org/realms/cozy"
 oidc-client-id: "kubernetes"
 oidc-username-claim: "preferred_username"
 oidc-groups-claim: "groups"

For Talm Add to your values.yaml in talm repo:

oidcIssuerUrl: "https://keycloak.<YOUR_ROOT_DOMAIN>/realms/cozy"

Domain Reachability Ensure that the domain keycloak.example.org is accessible from the cluster and resolves to your root ingress controller.

Enable OIDC Server

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

OIDC Configuration Your API server must be configured to use OIDC. If you are using Talos Linux, your machine configuration should include the following parameters:

cluster:
 apiServer:
 extraArgs:
 oidc-issuer-url: "https://keycloak.example.org/realms/cozy"
 oidc-client-id: "kubernetes"
 oidc-username-claim: "preferred_username"
 oidc-groups-claim: "groups"

For Talm Add to your values.yaml in talm repo:

oidcIssuerUrl: "https://keycloak.<YOUR_ROOT_DOMAIN>/realms/cozy"

Domain Reachability Ensure that the domain keycloak.example.org is accessible from the cluster and resolves to your root ingress controller.

4. Create a User Tenant and Configure Access

Mon, 01 Jan 0001 00:00:00 +0000

Objectives

At this step of the tutorial, you will create a user tenant — a space for users to deploy applications and VMs. You will also get tenant credentials and log in as a user with access to this tenant.

Prerequisites

Before you begin:

Complete the previous steps of the tutorial to get a Cozystack cluster running, with storage, networking, and management dashboard configured.
Make sure you can access the dashboard, as described in the previous step of the tutorial.

4. Create a User Tenant and Configure Access

Mon, 01 Jan 0001 00:00:00 +0000

Objectives

At this step of the tutorial, you will create a user tenant — a space for users to deploy applications and VMs. You will also get tenant credentials and log in as a user with access to this tenant.

Prerequisites

Before you begin:

Complete the previous steps of the tutorial to get a Cozystack cluster running, with storage, networking, and management dashboard configured.
Make sure you can access the dashboard, as described in the previous step of the tutorial.

Backup and Recovery

Mon, 01 Jan 0001 00:00:00 +0000

⚠️ Warning: Backup and restore functionality is currently under development and will be exposed via user‑friendly API interfaces in future releases. The instructions below require manual configuration and are recommended for advanced users only.

Cozystack uses Velero to manage Kubernetes resource backups and restores, including volume snapshots. This guide explains how to configure one‑off and scheduled backups and how to perform restores, with practical examples.

The Velero add‑on is disabled by default. To enable it:

Backup and Recovery

Mon, 01 Jan 0001 00:00:00 +0000

Cluster backup strategies and BackupClasses are configured by cluster administrators. If your tenant does not have a BackupClass yet, ask your administrator to follow the Velero Backup Configuration guide to set up storage, strategies, and BackupClasses.

This guide covers backing up and restoring VMInstance and VMDisk resources as a tenant user: running one-off and scheduled backups, checking backup status, and restoring from a backup using RestoreJobs.

Cozystack uses Velero under the hood for backup storage and volume snapshots.

Cloneable Virtual Machines

Mon, 01 Jan 0001 00:00:00 +0000

To create a cloneable VM, you will need to create a VMDisk and a VMInstance. This guide uses an ubuntu base image as an example.

Create VMDisk

apiVersion: apps.cozystack.io/v1alpha1
kind: VMDisk
metadata:
 name: ubuntu-source
 namespace: tenant-root
spec:
 optical: false
 source:
 http:
 url: https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
 storage: 20Gi
 storageClass: replicated

Since expanding a disk can be complicated, we recommend creating it with extra space to accommodate future growth.

Create VMInstance

Cloneable Virtual Machines

Mon, 01 Jan 0001 00:00:00 +0000

To create a cloneable VM, you will need to create a VMDisk and a VMInstance. This guide uses an ubuntu base image as an example.

Create VMDisk

apiVersion: apps.cozystack.io/v1alpha1
kind: VMDisk
metadata:
 name: ubuntu-source
 namespace: tenant-root
spec:
 optical: false
 source:
 http:
 url: https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
 storage: 20Gi
 storageClass: replicated

Since expanding a disk can be complicated, we recommend creating it with extra space to accommodate future growth.

Create VMInstance

Configuring a Dedicated Network for Distributed Storage with LINSTOR

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

This guide explains how to improve storage reliability and performance in distributed Cozystack clusters.

In hyper-converged clusters, it’s common to dedicate a network to storage traffic. However, it’s not always possible to provision separate storage links between datacenters.

If you lack dedicated inter-datacenter links for storage, you have two options:

make storage nodes in each datacenter isolated,
make storage traffic share the existing uplinks with other workloads.

This guide shows how to configure LINSTOR to use a dedicated network for storage traffic within each datacenter, while falling back to shared links between datacenters when needed.

Configuring a Dedicated Network for Distributed Storage with LINSTOR

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

This guide explains how to improve storage reliability and performance in distributed Cozystack clusters.

In hyper-converged clusters, it’s common to dedicate a network to storage traffic. However, it’s not always possible to provision separate storage links between datacenters.

If you lack dedicated inter-datacenter links for storage, you have two options:

make storage nodes in each datacenter isolated,
make storage traffic share the existing uplinks with other workloads.

This guide shows how to configure LINSTOR to use a dedicated network for storage traffic within each datacenter, while falling back to shared links between datacenters when needed.

Running VMs with GPU Passthrough

Mon, 01 Jan 0001 00:00:00 +0000

This section demonstrates how to deploy virtual machines (VMs) with GPU passthrough using Cozystack. First, we’ll deploy the GPU Operator to configure the worker node for GPU passthrough Then we will deploy a KubeVirt VM that requests a GPU.

By default, to provision a GPU Passthrough, the GPU Operator will deploy the following components:

VFIO Manager to bind vfio-pci driver to all GPUs on the node.
Sandbox Device Plugin to discover and advertise the passthrough GPUs to kubelet.
Sandbox Validator to validate the other operands.

Prerequisites

A Cozystack cluster with at least one GPU-enabled node.
kubectl installed and cluster access credentials configured.

1. Install the GPU Operator

Follow these steps:

Running VMs with GPU Passthrough

Mon, 01 Jan 0001 00:00:00 +0000

By default, to provision a GPU Passthrough, the GPU Operator will deploy the following components:

VFIO Manager to bind vfio-pci driver to all GPUs on the node.
Sandbox Device Plugin to discover and advertise the passthrough GPUs to kubelet.
Sandbox Validator to validate the other operands.

Prerequisites

A Cozystack cluster with at least one GPU-enabled node.
kubectl installed and cluster access credentials configured.

1. Install the GPU Operator

Follow these steps:

Managed MariaDB Service

Mon, 01 Jan 0001 00:00:00 +0000

The Managed MariaDB Service offers a powerful and widely used relational database solution. This service allows you to create and manage a replicated MariaDB cluster seamlessly.

Deployment Details

This managed service is controlled by mariadb-operator, ensuring efficient management and seamless operation.

HowTos

How to switch master/slave replica

kubectl edit mariadb <instance>

update:

Troubleshooting Kubernetes Installation

Mon, 01 Jan 0001 00:00:00 +0000

This page has instructions for resolving typical problems that can occur when installing Kubernetes with talm, talos-bootstrap, or talosctl.

No Talos nodes in maintenance mode found!

If you encounter issues with the talos-bootstrap script not detecting any nodes, follow these steps to diagnose and resolve the issue:

Verify Network Segment

Ensure that you are running the script within the same network segment as the nodes. This is crucial for the script to be able to communicate with the nodes.

Troubleshooting Kubernetes Installation

Mon, 01 Jan 0001 00:00:00 +0000

This page has instructions for resolving typical problems that can occur when installing Kubernetes with talm, talos-bootstrap, or talosctl.

No Talos nodes in maintenance mode found!

If you encounter issues with the talos-bootstrap script not detecting any nodes, follow these steps to diagnose and resolve the issue:

Verify Network Segment

Ensure that you are running the script within the same network segment as the nodes. This is crucial for the script to be able to communicate with the nodes.

Virtual Routers

Mon, 01 Jan 0001 00:00:00 +0000

Starting with version v0.27.0, Cozystack can deploy virtual routers (also known as “router appliances” or “middlebox appliances”). This feature allows you to create a virtual router based on a virtual machine instance. The virtual router can route traffic between different networks.

Creating a Virtual Router

Creating a virtual router requires a Cozystack administrator account.

Create a VM Instance
Use the standard vm-instance and virtual-machine packages to create a virtual machine instance.

Virtual Routers

Mon, 01 Jan 0001 00:00:00 +0000

Creating a Virtual Router

Creating a virtual router requires a Cozystack administrator account.

Create a VM Instance
Use the standard vm-instance and virtual-machine packages to create a virtual machine instance.

Automated Installation with Ansible

Mon, 01 Jan 0001 00:00:00 +0000

The cozystack.installer Ansible collection automates the full deployment pipeline: OS preparation, k3s cluster bootstrap, and Cozystack installation. It is suited for deploying Cozystack on bare-metal servers or VMs running a standard Linux distribution.

When to Use Ansible

Consider this approach when:

You want a fully automated, repeatable deployment from bare OS to a running Cozystack
You are deploying on generic Linux (Ubuntu, Debian, RHEL, Rocky, openSUSE) rather than Talos Linux
You want to manage multiple nodes with a single inventory file

For manual installation steps without Ansible, see the Generic Kubernetes guide.

Support

Sun, 02 Mar 2025 15:45:07 +0400

These companies provide enterprise support for Cozystack.

Organization	Contact	Description of Use
	Site LinkedIn	We provide all types of assistance, including consultations, development of missing features, design, assistance with installation, and integration.
	Site LinkedIn	We handle installation, provide ongoing support, and offer 24/7 managed services.

5. Deploy Managed Applications, VMs, and tenant Kubernetes cluster

Mon, 01 Jan 0001 00:00:00 +0000

Objectives

This guide will walk you through setting up the environment needed to run a typical web application with common service dependencies—PostgreSQL and Redis—on Cozystack, a Kubernetes-based PaaS framework.

You’ll learn how to:

Deploy managed applications in your tenant: a PostgreSQL database and Redis cache.
Create a managed Kubernetes cluster, configure DNS, and access the cluster.
Deploy a containerized application to the new cluster.

You don’t need in-depth Kubernetes knowledge to complete this tutorial—most steps are done through the Cozystack web interface.

5. Deploy Managed Applications, VMs, and tenant Kubernetes cluster

Mon, 01 Jan 0001 00:00:00 +0000

Objectives

You’ll learn how to:

Deploy managed applications in your tenant: a PostgreSQL database and Redis cache.
Create a managed Kubernetes cluster, configure DNS, and access the cluster.
Deploy a containerized application to the new cluster.

You don’t need in-depth Kubernetes knowledge to complete this tutorial—most steps are done through the Cozystack web interface.

Managed ClickHouse Service

Mon, 01 Jan 0001 00:00:00 +0000

ClickHouse is an open source high-performance and column-oriented SQL database management system (DBMS). It is used for online analytical processing (OLAP).

How to restore backup from S3

Find the snapshot:

restic -r s3:s3.example.org/clickhouse-backups/table_name snapshots

Restore it:

restic -r s3:s3.example.org/clickhouse-backups/table_name restore latest --target /tmp/

For more details, read Restic: Effective Backup from Stdin.

Parameters

Common parameters

Name	Description	Type	Value
`replicas`	Number of ClickHouse replicas.	`int`	`2`
`shards`	Number of ClickHouse shards.	`int`	`1`
`resources`	Explicit CPU and memory configuration for each ClickHouse replica. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`resources.cpu`	CPU available to each replica.	`quantity`	`""`
`resources.memory`	Memory (RAM) available to each replica.	`quantity`	`""`
`resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`size`	Persistent Volume Claim size available for application data.	`quantity`	`10Gi`
`storageClass`	StorageClass used to store the data.	`string`	`""`

Application-specific parameters

Name	Description	Type	Value
`logStorageSize`	Size of Persistent Volume for logs.	`quantity`	`2Gi`
`logTTL`	TTL (expiration time) for `query_log` and `query_thread_log`.	`int`	`15`
`users`	Users configuration map.	`map[string]object`	`{}`
`users[name].password`	Password for the user.	`string`	`""`
`users[name].readonly`	User is readonly (default: false).	`bool`	`false`

Backup parameters

Name	Description	Type	Value
`backup`	Backup configuration.	`object`	`{}`
`backup.enabled`	Enable regular backups (default: false).	`bool`	`false`
`backup.s3Region`	AWS S3 region where backups are stored.	`string`	`us-east-1`
`backup.s3Bucket`	S3 bucket used for storing backups.	`string`	`s3.example.org/clickhouse-backups`
`backup.schedule`	Cron schedule for automated backups.	`string`	`0 2 * * *`
`backup.cleanupStrategy`	Retention strategy for cleaning up old backups.	`string`	`--keep-last=3 --keep-daily=3 --keep-within-weekly=1m`
`backup.s3AccessKey`	Access key for S3 authentication.	`string`	`<your-access-key>`
`backup.s3SecretKey`	Secret key for S3 authentication.	`string`	`<your-secret-key>`
`backup.resticPassword`	Password for Restic backup encryption.	`string`	`<password>`

ClickHouse Keeper parameters

Name	Description	Type	Value
`clickhouseKeeper`	ClickHouse Keeper configuration.	`object`	`{}`
`clickhouseKeeper.enabled`	Deploy ClickHouse Keeper for cluster coordination.	`bool`	`true`
`clickhouseKeeper.size`	Persistent Volume Claim size available for application data.	`quantity`	`1Gi`
`clickhouseKeeper.resourcesPreset`	Default sizing preset.	`string`	`micro`
`clickhouseKeeper.replicas`	Number of Keeper replicas.	`int`	`3`

Parameter examples and reference

resources and resourcesPreset

resources sets explicit CPU and memory configurations for each replica. When left empty, the preset defined in resourcesPreset is applied.

FoundationDB

Mon, 01 Jan 0001 00:00:00 +0000

A managed FoundationDB service for Cozystack.

Overview

FoundationDB is a distributed database designed to handle large volumes of structured data across clusters of commodity servers. It organizes data as an ordered key-value store and employs ACID transactions for all operations.

This package provides a managed FoundationDB cluster deployment using the FoundationDB Kubernetes Operator.

Features

High Availability: Multi-instance deployment with automatic failover
ACID Transactions: Full ACID transaction support across the cluster
Scalable: Easily scale storage and compute resources
Backup Integration: Optional S3-compatible backup storage
Monitoring: Built-in monitoring and alerting through WorkloadMonitor
Flexible Configuration: Support for custom FoundationDB parameters

Configuration

Basic Configuration

# Cluster process configuration
cluster:
 version: "7.3.63"
 processCounts:
 storage: 3 # Number of storage processes (determines cluster size)
 stateless: -1 # Automatically calculated
 cluster_controller: 1
 faultDomain:
 key: "kubernetes.io/hostname"
 valueFrom: "spec.nodeName"

Storage

storage:
 size: "16Gi" # Storage size per instance
 storageClass: "" # Storage class (optional)

Resources

# Use preset sizing
resourcesPreset: "medium" # small, medium, large, xlarge, 2xlarge

# Or custom resource configuration
resources:
 cpu: "2000m"
 memory: "4Gi"

Backup (Optional)

backup:
 enabled: true
 s3:
 bucket: "my-fdb-backups"
 endpoint: "https://s3.amazonaws.com"
 region: "us-east-1"
 credentials:
 accessKeyId: "AKIA..."
 secretAccessKey: "..."
 retentionPolicy: "7d"

Advanced Configuration

# Custom FoundationDB parameters
customParameters:
 - "knob_disable_posix_kernel_aio=1"

# Image type (unified is default and recommended for new deployments)
imageType: "unified"

# Enable automatic pod replacements
automaticReplacements: true

# Security context configuration
securityContext:
 runAsUser: 4059
 runAsGroup: 4059

Prerequisites

FoundationDB Operator must be installed in the cluster
Sufficient storage and compute resources
For backups: S3-compatible storage credentials

Deployment

Install the FoundationDB operator (system package)
Deploy this application package with your desired configuration
The cluster will be automatically provisioned and configured

Monitoring

This package includes WorkloadMonitor integration for cluster health monitoring and resource tracking. Monitoring can be disabled by setting:

Deploying Cozystack on Generic Kubernetes

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to deploy Cozystack on generic Kubernetes distributions such as k3s, kubeadm, or RKE2. While Talos Linux remains the recommended platform for production deployments, Cozystack supports deployment on other Kubernetes distributions using the isp-full-generic bundle.

When to Use Generic Kubernetes

Consider using generic Kubernetes instead of Talos Linux when:

You have an existing Kubernetes cluster you want to enhance with Cozystack
Your infrastructure doesn’t support Talos Linux (certain cloud providers, embedded systems)
You need specific Linux features or packages not available in Talos

For new production deployments, Talos Linux is recommended due to its security and operational benefits.

Managed Harbor Container Registry

Mon, 01 Jan 0001 00:00:00 +0000

Harbor is an open-source trusted cloud-native registry project that stores, signs, and scans content.

Parameters

Common parameters

Name	Description	Type	Value
`host`	Hostname for external access to Harbor (defaults to ‘harbor’ subdomain for the tenant host).	`string`	`""`
`storageClass`	StorageClass used to store the data.	`string`	`""`

Component configuration

Name	Description	Type	Value
`core`	Core API server configuration.	`object`	`{}`
`core.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`core.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`core.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`core.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`registry`	Container image registry configuration.	`object`	`{}`
`registry.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`registry.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`registry.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`registry.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`jobservice`	Background job service configuration.	`object`	`{}`
`jobservice.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`jobservice.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`jobservice.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`jobservice.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`nano`
`trivy`	Trivy vulnerability scanner configuration.	`object`	`{}`
`trivy.enabled`	Enable or disable the vulnerability scanner.	`bool`	`true`
`trivy.size`	Persistent Volume size for vulnerability database cache.	`quantity`	`5Gi`
`trivy.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`trivy.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`trivy.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`trivy.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`nano`
`database`	PostgreSQL database configuration.	`object`	`{}`
`database.size`	Persistent Volume size for database storage.	`quantity`	`5Gi`
`database.replicas`	Number of database instances.	`int`	`2`
`redis`	Redis cache configuration.	`object`	`{}`
`redis.size`	Persistent Volume size for cache storage.	`quantity`	`1Gi`
`redis.replicas`	Number of Redis replicas.	`int`	`2`

Managed Kafka Service

Mon, 01 Jan 0001 00:00:00 +0000

Parameters

Common parameters

Name	Description	Type	Value
`external`	Enable external access from outside the cluster.	`bool`	`false`

Application-specific parameters

Name	Description	Type	Value
`topics`	Topics configuration.	`[]object`	`[]`
`topics[i].name`	Topic name.	`string`	`""`
`topics[i].partitions`	Number of partitions.	`int`	`0`
`topics[i].replicas`	Number of replicas.	`int`	`0`
`topics[i].config`	Topic configuration.	`object`	`{}`

Kafka configuration

Name	Description	Type	Value
`kafka`	Kafka configuration.	`object`	`{}`
`kafka.replicas`	Number of Kafka replicas.	`int`	`3`
`kafka.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`kafka.resources.cpu`	CPU available to each replica.	`quantity`	`""`
`kafka.resources.memory`	Memory (RAM) available to each replica.	`quantity`	`""`
`kafka.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`kafka.size`	Persistent Volume size for Kafka.	`quantity`	`10Gi`
`kafka.storageClass`	StorageClass used to store the Kafka data.	`string`	`""`

ZooKeeper configuration

Name	Description	Type	Value
`zookeeper`	ZooKeeper configuration.	`object`	`{}`
`zookeeper.replicas`	Number of ZooKeeper replicas.	`int`	`3`
`zookeeper.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`zookeeper.resources.cpu`	CPU available to each replica.	`quantity`	`""`
`zookeeper.resources.memory`	Memory (RAM) available to each replica.	`quantity`	`""`
`zookeeper.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`zookeeper.size`	Persistent Volume size for ZooKeeper.	`quantity`	`5Gi`
`zookeeper.storageClass`	StorageClass used to store the ZooKeeper data.	`string`	`""`

Parameter examples and reference

resources and resourcesPreset

resources sets explicit CPU and memory configurations for each replica. When left empty, the preset defined in resourcesPreset is applied.

Managed MariaDB Service

Mon, 01 Jan 0001 00:00:00 +0000

The Managed MariaDB Service offers a powerful and widely used relational database solution. This service allows you to create and manage a replicated MariaDB cluster seamlessly.

Deployment Details

This managed service is controlled by mariadb-operator, ensuring efficient management and seamless operation.

HowTos

How to switch master/slave replica

kubectl edit mariadb <instance>

update:

Managed MongoDB Service

Mon, 01 Jan 0001 00:00:00 +0000

MongoDB is a popular document-oriented NoSQL database known for its flexibility and scalability. The Managed MongoDB Service provides a self-healing replicated cluster managed by the Percona Operator for MongoDB.

Deployment Details

This managed service is controlled by the Percona Operator for MongoDB, ensuring efficient management and seamless operation.

Deployment Modes

Replica Set Mode (default)

By default, MongoDB deploys as a replica set with the specified number of replicas. This mode is suitable for most use cases requiring high availability.

Managed NATS Service

Mon, 01 Jan 0001 00:00:00 +0000

NATS is an open-source, simple, secure, and high performance messaging system. It provides a data layer for cloud native applications, IoT messaging, and microservices architectures.

Parameters

Common parameters

Name	Description	Type	Value
`replicas`	Number of replicas.	`int`	`2`
`resources`	Explicit CPU and memory configuration for each NATS replica. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`resources.cpu`	CPU available to each replica.	`quantity`	`""`
`resources.memory`	Memory (RAM) available to each replica.	`quantity`	`""`
`resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`nano`
`storageClass`	StorageClass used to store the data.	`string`	`""`
`external`	Enable external access from outside the cluster.	`bool`	`false`

Application-specific parameters

Name	Description	Type	Value
`users`	Users configuration map.	`map[string]object`	`{}`
`users[name].password`	Password for the user.	`string`	`""`
`jetstream`	Jetstream configuration.	`object`	`{}`
`jetstream.enabled`	Enable or disable Jetstream for persistent messaging in NATS.	`bool`	`true`
`jetstream.size`	Jetstream persistent storage size.	`quantity`	`10Gi`
`config`	NATS configuration.	`object`	`{}`
`config.merge`	Additional configuration to merge into NATS config.	`*object`	`{}`
`config.resolver`	Additional resolver configuration to merge into NATS config.	`*object`	`{}`

Parameter examples and reference

resources and resourcesPreset

resources sets explicit CPU and memory configurations for each replica. When left empty, the preset defined in resourcesPreset is applied.

Managed NATS Service

Mon, 01 Jan 0001 00:00:00 +0000

NATS is an open-source, simple, secure, and high performance messaging system. It provides a data layer for cloud native applications, IoT messaging, and microservices architectures.

Parameters

Common parameters

Name	Description	Type	Value
`replicas`	Number of replicas.	`int`	`2`
`resources`	Explicit CPU and memory configuration for each NATS replica. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`resources.cpu`	CPU available to each replica.	`quantity`	`""`
`resources.memory`	Memory (RAM) available to each replica.	`quantity`	`""`
`resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`nano`
`storageClass`	StorageClass used to store the data.	`string`	`""`
`external`	Enable external access from outside the cluster.	`bool`	`false`

Application-specific parameters

Name	Description	Type	Value
`users`	Users configuration map.	`map[string]object`	`{}`
`users[name].password`	Password for the user.	`string`	`""`
`jetstream`	Jetstream configuration.	`object`	`{}`
`jetstream.enabled`	Enable or disable Jetstream for persistent messaging in NATS.	`bool`	`true`
`jetstream.size`	Jetstream persistent storage size.	`quantity`	`10Gi`
`config`	NATS configuration.	`object`	`{}`
`config.merge`	Additional configuration to merge into NATS config.	`*object`	`{}`
`config.resolver`	Additional resolver configuration to merge into NATS config.	`*object`	`{}`

Parameter examples and reference

resources and resourcesPreset

resources sets explicit CPU and memory configurations for each replica. When left empty, the preset defined in resourcesPreset is applied.

Managed OpenBAO Service

Mon, 01 Jan 0001 00:00:00 +0000

OpenBAO is an open-source secrets management solution forked from HashiCorp Vault. It provides identity-based secrets and encryption management for cloud infrastructure.

Parameters

Common parameters

Name	Description	Type	Value
`replicas`	Number of OpenBAO replicas. HA with Raft is automatically enabled when replicas > 1. Switching between standalone (file storage) and HA (Raft storage) modes requires data migration.	`int`	`1`
`resources`	Explicit CPU and memory configuration for each OpenBAO replica. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`resources.cpu`	CPU available to each replica.	`quantity`	`""`
`resources.memory`	Memory (RAM) available to each replica.	`quantity`	`""`
`resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`size`	Persistent Volume Claim size for data storage.	`quantity`	`10Gi`
`storageClass`	StorageClass used to store the data.	`string`	`""`
`external`	Enable external access from outside the cluster.	`bool`	`false`

Application-specific parameters

Name	Description	Type	Value
`ui`	Enable the OpenBAO web UI.	`bool`	`true`

Tenant Application Reference

Mon, 01 Jan 0001 00:00:00 +0000

A tenant is the main unit of security on the platform. The closest analogy would be Linux kernel namespaces.

Tenants can be created recursively and are subject to the following rules:

Tenant naming

Tenant names must follow DNS-1035 naming rules:

Must start with a lowercase letter (a-z)
Can only contain lowercase letters, numbers, and hyphens (a-z, 0-9, -)
Must end with a letter or number (not a hyphen)
Maximum length depends on the cluster configuration (Helm release prefix and root domain)

Note: Using dashes (-) in tenant names is allowed but discouraged, unlike with other services. This is to keep consistent naming in tenants, nested tenants, and services deployed in them. Names with dashes (e.g., foo-bar) may lead to ambiguous parsing of internal resource names like tenant-foo-bar.

Creating users and add roles for them

Mon, 01 Jan 0001 00:00:00 +0000

Creating users and add roles for them

Overview

When a tenant is created in Cozy (starting with version 1.6.0), roles, RoleBindings and keycloak groups will automatically be created in the Kubernetes cluster.

To create a user, refer to the following documentation: Keycloak Admin Console Documentation

Assigning a Role to a User for a Tenant

Access Keycloak: To retrieve login credentials, check the secret by running the following command:

kubectl get secret keycloak-credentials -n cozy-keycloak -o yaml

Keycloak Address: The Keycloak address will match the value specified in the cozystack ConfigMap. For example, if your ConfigMap looks like this:

 apiVersion: v1
 kind: ConfigMap
 metadata:
 name: cozystack
 namespace: cozy-system
 data:
 root-host: infra.example.org
 api-server-adress: 55.21.33.22
 bundle-name: "paas-full"
 ipv4-pod-cidr: "10.244.0.0/16"
 ipv4-pod-gateway: "10.244.0.1"
 ipv4-svc-cidr: "10.96.0.0/16"
 ipv4-join-cidr: "100.64.0.0/16"

Then Keycloak will be available at: keycloak.infra.example.org

If you are planning to integrate with external services either as clients or as IdPs, your Keycloak address needs to be publicly accessible and reachable by these services.

Configure Roles for Each Tenant in Cozy:

Cluster wide

cozystack-cluster-admin

Creating users and add roles for them

Mon, 01 Jan 0001 00:00:00 +0000

Creating users and add roles for them

Overview

When a tenant is created in Cozy (starting with version 1.6.0), roles, RoleBindings and keycloak groups will automatically be created in the Kubernetes cluster.

To create a user, refer to the following documentation: Keycloak Admin Console Documentation

Assigning a Role to a User for a Tenant

Access Keycloak: To retrieve login credentials, check the secret by running the following command:
```
kubectl get secret keycloak-credentials -n cozy-keycloak -o yaml
```
Keycloak Address: The Keycloak address will match the publishing.host value specified in your Platform Package. For example, if your Package includes:

White Labeling

Mon, 01 Jan 0001 00:00:00 +0000

White labeling allows you to replace default Cozystack branding with your own logos and text across the Dashboard UI and Keycloak authentication pages.

Overview

Branding is configured through the branding field in the Platform Package (spec.components.platform.values.branding). The configuration propagates automatically to:

Dashboard: logo, page title, footer text, favicon, and tenant identifier
Keycloak: realm display name on authentication pages

Configuration

Edit your Platform Package to add or update the branding section:

Running Windows VMs in Cozystack

Mon, 01 Jan 0001 00:00:00 +0000

Cozystack can run Windows virtual machines. This guide explains the prerequisites and steps required to boot up a virtual machine running Windows OS.

Prerequisites

Windows installation ISO image.
Virtio drivers ISO image.
KubeVirt client virtctl installed in your local environment and configured for your tenant’s namespace.
Cozystack version v0.34.2 or later.

Installation

Creating a virtual machine running Windows OS starts with creating VMDisk objects and continues with creating a VMInstance.

1. Create VMDisk objects

You need three disks:

Running Windows VMs in Cozystack

Mon, 01 Jan 0001 00:00:00 +0000

Cozystack can run Windows virtual machines. This guide explains the prerequisites and steps required to boot up a virtual machine running Windows OS.

Prerequisites

Windows installation ISO image.
Virtio drivers ISO image.
KubeVirt client virtctl installed in your local environment and configured for your tenant’s namespace.
Cozystack version v0.34.2 or later.

Installation

Creating a virtual machine running Windows OS starts with creating VMDisk objects and continues with creating a VMInstance.

1. Create VMDisk objects

You need three disks:

Running MikroTik RouterOS in Cozystack

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

MikroTik RouterOS ISO (CHR or NPK install image), for example, mikrotik-7.19.3.iso.
A free static IP or DHCP on the connected tenant network.
KubeVirt client virtctl installed in your local environment and configured for your tenant’s namespace.
Cozystack version v0.34.2 or later.

Installation

1. Prepare disks

You need two disks:

Installation ISO – optical.
System disk – non‑optical.

apiVersion: apps.cozystack.io/v1alpha1
kind: VMDisk
metadata:
 name: mikrotik-iso
spec:
 source:
 http:
 url: https://download.mikrotik.com/routeros/7.19.3/mikrotik-7.19.3.iso
 optical: true
 storage: 1Gi
 storageClass: replicated
---
apiVersion: apps.cozystack.io/v1alpha1
kind: VMDisk
metadata:
 name: mikrotik-system
spec:
 optical: false
 storage: 1Gi
 storageClass: replicated

2. Create the VMInstance

RouterOS does not require a special instance profile. Use a lightweight Linux profile such as ubuntu with a small instance type such as u1.medium:

Running MikroTik RouterOS in Cozystack

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

MikroTik RouterOS ISO (CHR or NPK install image), for example, mikrotik-7.19.3.iso.
A free static IP or DHCP on the connected tenant network.
KubeVirt client virtctl installed in your local environment and configured for your tenant’s namespace.
Cozystack version v0.34.2 or later.

Installation

1. Prepare disks

You need two disks:

Installation ISO – optical.
System disk – non‑optical.

apiVersion: apps.cozystack.io/v1alpha1
kind: VMDisk
metadata:
 name: mikrotik-iso
spec:
 source:
 http:
 url: https://download.mikrotik.com/routeros/7.19.3/mikrotik-7.19.3.iso
 optical: true
 storage: 1Gi
 storageClass: replicated
---
apiVersion: apps.cozystack.io/v1alpha1
kind: VMDisk
metadata:
 name: mikrotik-system
spec:
 optical: false
 storage: 1Gi
 storageClass: replicated

2. Create the VMInstance

RouterOS does not require a special instance profile. Use a lightweight Linux profile such as ubuntu with a small instance type such as u1.medium:

Managed PostgreSQL Service

Mon, 01 Jan 0001 00:00:00 +0000

Deployment Details

This managed service is controlled by the CloudNativePG operator, ensuring efficient management and seamless operation.

Self-Signed Certificates

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to configure Kubernetes API server for OIDC authentication with Keycloak when using self-signed certificates. By default, Cozystack issues certificates via LetsEncrypt, but some environments (e.g., air-gapped or private enterprise networks) may use a custom CA instead.

Prerequisites

Cozystack cluster with OIDC enabled (see Enable OIDC Server)
Talos Linux control plane nodes
talosctl configured for your cluster
kubelogin installed

Step 1: Retrieve the Keycloak Certificate

Get the certificate from the ingress controller:

Self-Signed Certificates

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Cozystack cluster with OIDC enabled (see Enable OIDC Server)
Talos Linux control plane nodes
talosctl configured for your cluster
kubelogin installed

Step 1: Retrieve the Keycloak Certificate

Get the certificate from the ingress controller:

Telemetry

Mon, 01 Jan 0001 00:00:00 +0000

This document outlines the telemetry feature within the Cozystack project, detailing the rationale behind data collection, the nature of the data collected, data handling practices, and instructions for opting out.

Why We Collect Telemetry

Cozystack, as an open source project, thrives on community feedback and usage insights. Telemetry data allows maintainers to understand how Cozystack is being used in real-world scenarios. This data informs decisions related to feature prioritization, testing strategies, bug fixes, and overall project evolution. Without telemetry, decisions would rely on guesswork or limited feedback, which might slow down improvement cycles or introduce features that don’t align with users’ needs. Telemetry ensures that development is guided by actual usage patterns and community requirements, fostering a more robust and user-centric platform.

Telemetry

Mon, 01 Jan 0001 00:00:00 +0000

Why We Collect Telemetry

Managed MongoDB Service

Mon, 01 Jan 0001 00:00:00 +0000

Deployment Details

This managed service is controlled by the Percona Operator for MongoDB, ensuring efficient management and seamless operation.

Deployment Modes

Replica Set Mode (default)

By default, MongoDB deploys as a replica set with the specified number of replicas. This mode is suitable for most use cases requiring high availability.

Migrating Virtual Machines from Proxmox

Mon, 01 Jan 0001 00:00:00 +0000

This guide describes the process of migrating virtual machines from Proxmox VE to Cozystack by exporting VM disk images and uploading them to the target environment.

Note: Migration is performed by exporting VM disks to files and uploading them to Cozystack. VM state and snapshots are not preserved during migration.

Prerequisites

Before starting the migration, ensure you have:

KubeVirt client virtctl installed on your local machine:
- Installation guide: KubeVirt User Guide - Virtctl Client Tool
Upload proxy access configured in your Cozystack cluster:

Migrating Virtual Machines from Proxmox

Mon, 01 Jan 0001 00:00:00 +0000

This guide describes the process of migrating virtual machines from Proxmox VE to Cozystack by exporting VM disk images and uploading them to the target environment.

Note: Migration is performed by exporting VM disks to files and uploading them to Cozystack. VM state and snapshots are not preserved during migration.

Prerequisites

Before starting the migration, ensure you have:

KubeVirt client virtctl installed on your local machine:
- Installation guide: KubeVirt User Guide - Virtctl Client Tool
Upload proxy access configured in your Cozystack cluster:

Managed RabbitMQ Service

Mon, 01 Jan 0001 00:00:00 +0000

Deployment Details

The service utilizes official RabbitMQ operator. This ensures the reliability and seamless operation of your RabbitMQ instances.

Managed Redis Service

Mon, 01 Jan 0001 00:00:00 +0000

Deployment Details

Service utilizes the Spotahome Redis Operator for efficient management and orchestration of Redis clusters.

Tenant Application Reference

Mon, 01 Jan 0001 00:00:00 +0000

A tenant is the main unit of security on the platform. The closest analogy would be Linux kernel namespaces.

Tenants can be created recursively and are subject to the following rules:

Tenant naming

Tenant names must be alphanumeric. Using dashes (-) in tenant names is not allowed, unlike with other services. This limitation exists to keep consistent naming in tenants, nested tenants, and services deployed in them.

Cozystack Internals and Developer Guides

Mon, 01 Jan 0001 00:00:00 +0000

How it works

In short, cozystack is a seed container that bootstraps the entire platform. It consists of:

installer.sh script: Contains minimal business logic, performs migrations, installs FluxCD, and prepares Kubernetes to run the platform chart.
platform chart: A Helm chart that bootstraps the remaining configuration. It reads the state from the Kubernetes cluster using Helm lookup functions and templates all necessary manifests. The installer.sh script continuously runs the platform chart to ensure changes in the cluster are properly reconciled.

Cozystack Internals and Developer Guide

Mon, 01 Jan 0001 00:00:00 +0000

How it works

Cozystack is an operator-driven platform. The bootstrap and ongoing management are handled by a set of controllers that run inside the cluster. The high-level flow is:

Installer chart (packages/core/installer) is applied via helm install. It deploys the cozystack-operator Deployment into the cozy-system namespace.
cozystack-operator starts and performs one-time bootstrap:
- Installs Cozystack CRDs (Package, PackageSource) from embedded manifests (internal/crdinstall).
- Installs Flux components (source-controller, helm-controller, source-watcher) from embedded manifests (internal/fluxinstall).
- Creates the initial OCIRepository (cozystack-platform) from the platformSourceUrl and platformSourceRef values configured in the installer.
- Creates a PackageSource that references the initial OCIRepository.
Reconciliation loop takes over. The operator watches PackageSource and Package CRDs and translates them into Flux HelmRelease objects. Flux then installs and manages the actual Helm charts.

Creating Encrypted Storage on LINSTOR

Mon, 01 Jan 0001 00:00:00 +0000

Cozystack administrators can enable encrypted storage by creating a custom StorageClass. This guide explains how to set up encryption passphrase, create an encrypted storage class, and use it in applications.

LINSTOR provides at-rest encryption for persistent volumes using LUKS. This ensures that data stored on disk is encrypted and can only be accessed when the volume is mounted and unlocked.

Set Up Encryption in LINSTOR

To start using encryption, set up an encryption passphrase in LINSTOR.

Creating Encrypted Storage on LINSTOR

Mon, 01 Jan 0001 00:00:00 +0000

LINSTOR provides at-rest encryption for persistent volumes using LUKS. This ensures that data stored on disk is encrypted and can only be accessed when the volume is mounted and unlocked.

Set Up Encryption in LINSTOR

To start using encryption, set up an encryption passphrase in LINSTOR.

How to relocate etcd replicas in tenant clusters

Mon, 01 Jan 0001 00:00:00 +0000

Tenant Kubernetes clusters are using their own etcd clusters, not the one that is used by the management cluster. Such etcd clusters are deployed in tenants and are available to managed Kubernetes clusters deployed in the tenant and its sub-tenants.

Replicas of a tenant etcd cluster can be relocated between nodes for maintenance reasons. Currently, management operations for tenant etcd clusters are not automated, but such a task can be done manually.

How to relocate etcd replicas in tenant clusters

Mon, 01 Jan 0001 00:00:00 +0000

How to install Talos on a single-disk machine

Mon, 01 Jan 0001 00:00:00 +0000

Default Talos setup assumes that each node has a primary and secondary disks, used for system and user storage, respectively. However, it’s possible to use a single disk, allocating space for user storage.

This configuration must be applied with the first talosctl apply or talm apply — the one with the -i (--insecure) flag. Applying changes after initialization will not have any effect.

For talosctl, append the following lines to patch.yaml:

---
apiVersion: v1alpha1
kind: VolumeConfig
name: EPHEMERAL
provisioning:
 minSize: 70GiB

---
apiVersion: v1alpha1
kind: UserVolumeConfig
name: data-storage
provisioning:
 diskSelector:
 match: disk.transport == 'nvme'
 minSize: 400GiB

For talm, append the same lines at end of the first node’s configuration file, such as nodes/node1.yaml.

How to install Talos on a single-disk machine

Mon, 01 Jan 0001 00:00:00 +0000

This configuration must be applied with the first talosctl apply or talm apply — the one with the -i (--insecure) flag. Applying changes after initialization will not have any effect.

For talosctl, append the following lines to patch.yaml:

---
apiVersion: v1alpha1
kind: VolumeConfig
name: EPHEMERAL
provisioning:
 minSize: 70GiB

---
apiVersion: v1alpha1
kind: UserVolumeConfig
name: data-storage
provisioning:
 diskSelector:
 match: disk.transport == 'nvme'
 minSize: 400GiB

For talm, append the same lines at end of the first node’s configuration file, such as nodes/node1.yaml.

Troubleshooting LINSTOR controller crash loops

Mon, 01 Jan 0001 00:00:00 +0000

Restarting the controller

If the controller is running but appears idle/unresponsive, try restarting it. This operation is idempotent and safe: pending (unfinished) work will resume after the restart.

LINSTOR controller crash loop

If linstor-controller can’t start, but logs do not contain any useful information, you can increase the log level (maximum level is TRACE).

Example of LINSTORCluster CR with increased log level:

apiVersion: piraeus.io/v1
kind: LINSTORCluster
spec:
 controller:
 podTemplate:
 spec:
 containers:
 - name: linstor-controller
 env:
 # both settings are used by linstor-controller
 - name: LS_LOG_LEVEL
 value: TRACE
 - name: LS_LOG_LEVEL_LINSTOR
 value: TRACE

Note: if linstor-controller is not in a crash loop, but you need to increase log level, you can do so temporarily in the runtime using the following command:

Troubleshooting LINSTOR controller crash loops

Mon, 01 Jan 0001 00:00:00 +0000

Restarting the controller

If the controller is running but appears idle/unresponsive, try restarting it. This operation is idempotent and safe: pending (unfinished) work will resume after the restart.

LINSTOR controller crash loop

If linstor-controller can’t start, but logs do not contain any useful information, you can increase the log level (maximum level is TRACE).

Example of LINSTORCluster CR with increased log level:

apiVersion: piraeus.io/v1
kind: LINSTORCluster
spec:
 controller:
 podTemplate:
 spec:
 containers:
 - name: linstor-controller
 env:
 # both settings are used by linstor-controller
 - name: LS_LOG_LEVEL
 value: TRACE
 - name: LS_LOG_LEVEL_LINSTOR
 value: TRACE

Note: if linstor-controller is not in a crash loop, but you need to increase log level, you can do so temporarily in the runtime using the following command:

Virtual Machine Resources

Mon, 01 Jan 0001 00:00:00 +0000

Each virtual machine has these two configuration settings:

instanceType defines the resourced provided to the Virtual Machine.
instanceProfile defines the set of preferences for Virtual Machines, according to the OS being used.

Instance Type Resources

Reference table

The following instancetype resources are provided by Cozystack:

Name	vCPUs	Memory
cx1.2xlarge	8	16Gi
cx1.4xlarge	16	32Gi
cx1.8xlarge	32	64Gi
cx1.large	2	4Gi
cx1.medium	1	2Gi
cx1.xlarge	4	8Gi
gn1.2xlarge	8	32Gi
gn1.4xlarge	16	64Gi
gn1.8xlarge	32	128Gi
gn1.xlarge	4	16Gi
m1.2xlarge	8	64Gi
m1.4xlarge	16	128Gi
m1.8xlarge	32	256Gi
m1.large	2	16Gi
m1.xlarge	4	32Gi
n1.2xlarge	16	32Gi
n1.4xlarge	32	64Gi
n1.8xlarge	64	128Gi
n1.large	4	8Gi
n1.medium	4	4Gi
n1.xlarge	8	16Gi
o1.2xlarge	8	32Gi
o1.4xlarge	16	64Gi
o1.8xlarge	32	128Gi
o1.large	2	8Gi
o1.medium	1	4Gi
o1.micro	1	1Gi
o1.nano	1	512Mi
o1.small	1	2Gi
o1.xlarge	4	16Gi
rt1.2xlarge	8	32Gi
rt1.4xlarge	16	64Gi
rt1.8xlarge	32	128Gi
rt1.large	2	8Gi
rt1.medium	1	4Gi
rt1.micro	1	1Gi
rt1.small	1	2Gi
rt1.xlarge	4	16Gi
u1.2xlarge	8	32Gi
u1.2xmedium	2	4Gi
u1.4xlarge	16	64Gi
u1.8xlarge	32	128Gi
u1.large	2	8Gi
u1.medium	1	4Gi
u1.micro	1	1Gi
u1.nano	1	512Mi
u1.small	1	2Gi
u1.xlarge	4	16Gi

U Series

The U Series is quite neutral and provides resources for general purpose applications.

Virtual Machine Resources

Mon, 01 Jan 0001 00:00:00 +0000

Each virtual machine has these two configuration settings:

instanceType defines the resourced provided to the Virtual Machine.
instanceProfile defines the set of preferences for Virtual Machines, according to the OS being used.

Instance Type Resources

Reference table

The following instancetype resources are provided by Cozystack:

Name	vCPUs	Memory
cx1.2xlarge	8	16Gi
cx1.4xlarge	16	32Gi
cx1.8xlarge	32	64Gi
cx1.large	2	4Gi
cx1.medium	1	2Gi
cx1.xlarge	4	8Gi
gn1.2xlarge	8	32Gi
gn1.4xlarge	16	64Gi
gn1.8xlarge	32	128Gi
gn1.xlarge	4	16Gi
m1.2xlarge	8	64Gi
m1.4xlarge	16	128Gi
m1.8xlarge	32	256Gi
m1.large	2	16Gi
m1.xlarge	4	32Gi
n1.2xlarge	16	32Gi
n1.4xlarge	32	64Gi
n1.8xlarge	64	128Gi
n1.large	4	8Gi
n1.medium	4	4Gi
n1.xlarge	8	16Gi
o1.2xlarge	8	32Gi
o1.4xlarge	16	64Gi
o1.8xlarge	32	128Gi
o1.large	2	8Gi
o1.medium	1	4Gi
o1.micro	1	1Gi
o1.nano	1	512Mi
o1.small	1	2Gi
o1.xlarge	4	16Gi
rt1.2xlarge	8	32Gi
rt1.4xlarge	16	64Gi
rt1.8xlarge	32	128Gi
rt1.large	2	8Gi
rt1.medium	1	4Gi
rt1.micro	1	1Gi
rt1.small	1	2Gi
rt1.xlarge	4	16Gi
u1.2xlarge	8	32Gi
u1.2xmedium	2	4Gi
u1.4xlarge	16	64Gi
u1.8xlarge	32	128Gi
u1.large	2	8Gi
u1.medium	1	4Gi
u1.micro	1	1Gi
u1.nano	1	512Mi
u1.small	1	2Gi
u1.xlarge	4	16Gi

U Series

The U Series is quite neutral and provides resources for general purpose applications.

Public-network Kubernetes deployment

Mon, 01 Jan 0001 00:00:00 +0000

A Kubernetes cluster for Cozystack can be deployed using only public networks:

Both management and worker nodes have public IP addresses.
Worker nodes connect to the management nodes over the public Internet, without a private internal network or VPN.

Such a setup is not recommended for production, but can be used for research and testing, when hosting limitations prevent provisioning a private network.

To enable this setup when deploying with talosctl, add the following data in the node configuration files:

Public-network Kubernetes deployment

Mon, 01 Jan 0001 00:00:00 +0000

A Kubernetes cluster for Cozystack can be deployed using only public networks:

Both management and worker nodes have public IP addresses.
Worker nodes connect to the management nodes over the public Internet, without a private internal network or VPN.

Such a setup is not recommended for production, but can be used for research and testing, when hosting limitations prevent provisioning a private network.

To enable this setup when deploying with talosctl, add the following data in the node configuration files:

How to Rotate Certificate Authority

Mon, 01 Jan 0001 00:00:00 +0000

Talos sets up root certificate authorities with a lifetime of 10 years, and all Talos and Kubernetes API certificates are issued by these root CAs. In general, you almost never need to rotate the root CA certificate and key for the Talos API and Kubernetes API.

Rotation of the root CA is only needed:

when you suspect that the private key has been compromised;
when you want to revoke access to the cluster for a leaked talosconfig or kubeconfig;
once in 10 years.

Rotate CA for the Management Kubernetes Cluster:

See: https://www.talos.dev/v1.9/advanced/ca-rotation/#kubernetes-api

How to Rotate Certificate Authority

Mon, 01 Jan 0001 00:00:00 +0000

Rotation of the root CA is only needed:

when you suspect that the private key has been compromised;
when you want to revoke access to the cluster for a leaked talosconfig or kubeconfig;
once in 10 years.

Rotate CA for Talos API

To rotate the Talos CA for the management cluster, use the following command:

Troubleshooting LINSTOR CrashLoopBackOff related to a broken database

Mon, 01 Jan 0001 00:00:00 +0000

⚠️ Advanced Users Only

This guide is intended for experienced users who are comfortable with low-level troubleshooting and data recovery operations. Corrupted LINSTOR databases are rare and typically indicate a serious underlying issue.

If you encounter this situation in a production environment, we strongly recommend contacting qualified support rather than attempting to fix it yourself. Incorrect actions can lead to permanent data loss.

Introduction

When running outside of Kubernetes, LINSTOR controller uses some kind of SQL database (various kinds). In Kubernetes, linstor-controller does not require a persistent volume or external database. Instead, it stores all its information as custom resources (CRs) right in the Kubernetes control plane. Upon startup, LINSTOR controller reads all CRs and creates an in-memory database.

Troubleshooting LINSTOR CrashLoopBackOff related to a broken database

Mon, 01 Jan 0001 00:00:00 +0000

⚠️ Advanced Users Only

Introduction

How to configure network bonding (LACP)

Mon, 01 Jan 0001 00:00:00 +0000

Network bonding allows you to combine multiple physical network interfaces into a single logical interface. This provides increased bandwidth and link redundancy.

LACP (Link Aggregation Control Protocol, IEEE 802.3ad) is the most common bonding mode, which dynamically negotiates link aggregation with the network switch.

LACP requires configuration on both the server and the network switch. Make sure your switch has a corresponding LACP port-channel configured for the server’s ports.

Identify network interfaces

After running talm template, the generated node configuration file will contain a comment block with discovered network interfaces:

How to Enable KubeSpan

Mon, 01 Jan 0001 00:00:00 +0000

Talos Linux provides a full mesh WireGuard network for your cluster.

To enable this functionality, you need to configure KubeSpan and Cluster Discovery in your Talos Linux configuration:

machine:
 network:
 kubespan:
 enabled: true
cluster:
 discovery:
 enabled: true

Since KubeSpan encapsulates traffic into a WireGuard tunnel, Kube-OVN should also be configured with a lower MTU value.

To achieve this, add the following to the Cozystack ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
 name: cozystack
 namespace: cozy-system
data:
 values-kubeovn: |
 kube-ovn:
 mtu: 1222

How to Enable KubeSpan

Mon, 01 Jan 0001 00:00:00 +0000

Talos Linux provides a full mesh WireGuard network for your cluster.

To enable this functionality, you need to configure KubeSpan and Cluster Discovery in your Talos Linux configuration:

machine:
 network:
 kubespan:
 enabled: true
cluster:
 discovery:
 enabled: true

Since KubeSpan encapsulates traffic into a WireGuard tunnel, Kube-OVN should also be configured with a lower MTU value.

To achieve this, add the following to the networking component of your Platform Package:

How to enable Hugepages

Mon, 01 Jan 0001 00:00:00 +0000

Enabling Hugepages for Cozystack can be done both on initial installation and at any time after it. Applying this configuration after installation will require a full node reboot.

Read more in the Linux Kernel documentation: HugeTLB Pages.

Using Talm

Requires Talm v0.16.0 or later.

Add the following lines to values.yaml:
```
...
certSANs: []
nr_hugepages: 3000
```
vm.nr_hugepages is the count of pages per 2Mi.
Apply the configuration:
```
talm apply -f nodes/node0.yaml
```
Finally, reboot the nodes:

How to enable Hugepages

Mon, 01 Jan 0001 00:00:00 +0000

Enabling Hugepages for Cozystack can be done both on initial installation and at any time after it. Applying this configuration after installation will require a full node reboot.

Read more in the Linux Kernel documentation: HugeTLB Pages.

Using Talm

Requires Talm v0.16.0 or later.

Add the following lines to values.yaml:
```
...
certSANs: []
nr_hugepages: 3000
```
vm.nr_hugepages is the count of pages per 2Mi.
Apply the configuration:
```
talm apply -f nodes/node0.yaml
```
Finally, reboot the nodes:

Troubleshooting Piraeus custom resources

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Piraeus custom resources are the following:

LinstorCluster
LinstorSatellite
LinstorSatelliteConfiguration
LinstorNodeConnection

The Piraeus controller protects these resources from unintended changes and deletion. When you delete the resource, controller will first make sure that all downstream resources are deleted.

But the default setting of the webhook is to reject any changes if something goes wrong with the webhook itself. And, in a disturbed system, webhook may go down a lot, without useful stacktraces.

Troubleshooting Piraeus custom resources

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Piraeus custom resources are the following:

LinstorCluster
LinstorSatellite
LinstorSatelliteConfiguration
LinstorNodeConnection

The Piraeus controller protects these resources from unintended changes and deletion. When you delete the resource, controller will first make sure that all downstream resources are deleted.

But the default setting of the webhook is to reject any changes if something goes wrong with the webhook itself. And, in a disturbed system, webhook may go down a lot, without useful stacktraces.

Scheduling Classes

Mon, 01 Jan 0001 00:00:00 +0000

SchedulingClass is a cluster-scoped custom resource that lets administrators define placement policies for tenant workloads. When a tenant is assigned a scheduling class, all of its pods are automatically routed to the Cozystack custom scheduler, which merges the class-defined constraints with any constraints already present on the pod.

This allows platform operators to pin tenants to specific data centers, availability zones, or node groups — without modifying individual application charts.

SeaweedFS Multi-DC Configuration

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to deploy SeaweedFS over several data centers (“multi-DC”). Multi-zone configuration for SeaweedFS is available since Cozystack v0.34.0.

SeaweedFS Multi-DC Configuration

To span SeaweedFS over several DCs, create a new cluster in multi-DC mode.

By default, SeaweedFS runs in a single data centre (DC), and a running single-DC deployment cannot be switched to multi-DC mode. If you need to change the topology, delete the current SeaweedFS instance and create a new one with the desired mode.

SeaweedFS Multi-DC Configuration

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to deploy SeaweedFS over several data centers (“multi-DC”). Multi-zone configuration for SeaweedFS is available since Cozystack v0.34.0.

SeaweedFS Multi-DC Configuration

To span SeaweedFS over several DCs, create a new cluster in multi-DC mode.

Cozystack Roadmap

Mon, 01 Jan 0001 00:00:00 +0000

You can view our roadmap on GitHub Project.

If you have some questions and comments about roadmap or need specific functionality, feel free to let us know.

Join our community meetings.

Cozystack Roadmap

Mon, 01 Jan 0001 00:00:00 +0000

You can view our roadmap on GitHub Project.

If you have some questions and comments about roadmap or need specific functionality, feel free to let us know.

Join our community meetings.

Game Servers on Cozystack: No April Fools' Joke

Wed, 01 Apr 2026 07:30:00 +0000

Author: Timur Tukaev (Ænix)

hello, world! We are the team behind Cozystack, an open-source platform for building clouds on your own hardware. We want to explain why we decided to target the game server space and what came of it.

What Is Cozystack

Cozystack is a platform that turns ordinary servers into a full-fledged cloud. The project is part of CNCF Sandbox, is distributed under the Apache 2.0 license, and is deployed on bare-metal servers.

Cozystack 1.2: OpenSearch, VPC Peering, and Smarter Tenant Scheduling

Tue, 31 Mar 2026 00:00:00 +0000

Cozystack 1.2: OpenSearch, VPC Peering, and Smarter Tenant Scheduling

The Cozystack 1.2 release line is now available. v1.2.0 was published on March 27, 2026, and v1.2.1 followed on March 31, 2026.

This cycle expands the platform in three important directions: managed search and analytics, secure networking between tenant environments, and better control over where tenant workloads run. The follow-up v1.2.1 release focuses on safety and operational stability.

Main highlights

Managed OpenSearch in the application catalog

Cozystack 1.2 adds OpenSearch as a fully managed service. It supports OpenSearch v1, v2, and v3, can run in a multi-role topology, enables TLS by default, ships with built-in HTTP Basic authentication, and can optionally deploy OpenSearch Dashboards alongside the engine.

Cozystack v1.0 & v1.1: Introducing Package-Based Architecture, Cozystack Operator, Velero Strategy Controller, MongoDB and OpenBAO Support

Mon, 16 Mar 2026 07:30:00 +0000

Author: Timur Tukaev (Ænix)

The last platform release was 0.41. So it came as a surprise when the next release, 0.42, turned out to be the answer to the ultimate question of life, the universe, and everything. The number of serious changes that had piled up was just too great—so much so that 0.42 had to be renamed to 1.0.

With the release of v1.0.0, Cozystack is undergoing a fundamental architectural transition. We’ve built a package system based on FluxCD and OCI artifacts — think of it like apt for Debian/Ubuntu, but made for Kubernetes (see “Package-based Deployment” below). This let us introduce a unique new approach: Build Your Own Platform (BYOP).

Cozystack at KubeCon Europe 2026

Wed, 11 Mar 2026 00:00:00 +0000

Author: Timur Tukaev (Ænix)

Amsterdam, Netherlands — March 25 📍 CNCF Project Pavilion 🪧 Kiosk P-18A | Halls 1–5 🕙 Wednesday, March 25 | 10:00 – 13:30

Building your own public or private cloud?
Running Kubernetes-based services for your customers?
Trying to simplify your stack?
Want to automate operations and stop paying huge bills to big cloud providers?

Come find our COO, Timur Tukaev, at the kiosk — or reach out and schedule a meeting with him: https://aenix.io/links/timur.tukaev/

Invitation to CozySummit Virtual – May 26

Wed, 11 Feb 2026 00:00:00 +0000

Author: Timur Tukaev (Ænix)

Join us on May 26 for 2nd CozySummit Virtual, conference for CozyStack developers and adopters.

📢 CFP is open until March 8 – submit your talk and become speaker at our event!

CozySummit Virtual is organized by CNCF with the support of CozyStack maintainers and project sponsors.

Talm v0.17: Built-in Age Encryption for Secrets Management

Wed, 17 Dec 2025 00:00:00 +0000

Talm v0.17: Built-in Age Encryption for Secrets Management

The latest release of Talm, the configuration manager for Talos Linux, introduces a powerful new feature: built-in encryption using the age encryption tool. This enhancement allows you to securely store sensitive configuration files like secrets.yaml, talosconfig, and kubeconfig in Git repositories while following security best practices.

Why Age Encryption?

Managing secrets in Git repositories has always been a challenge. While storing configuration files in version control is convenient for GitOps workflows, sensitive data like API keys, certificates, and cluster credentials should never be committed in plain text. Traditional solutions like git-crypt or external secret management systems add complexity and dependencies.

Flux-aio, Kubernetes mTLS and the Chicken and Egg Problem

Fri, 12 Dec 2025 00:00:00 +0000

Here at Cozystack, we’re once again solving the chicken-and-egg problem: how to deploy CNI and kube-proxy through Flux, while ensuring Flux itself works without CNI and kube-proxy.

Flux can be started without CNI and kube-proxy using the flux-aio project (by the creator of Flux), which runs a single deployment with all controllers configured to communicate with each other via localhost.

The specific challenge for Cozystack is that we deploy a small HTTP server with Helm charts and other assets used in the platform to each cluster. Flux reads these charts and installs them into the system.

CozySummit lineup is out!

Tue, 14 Oct 2025 00:00:00 +0000

👻 CozySummit lineup is out!

Yaaay! We’ve published the schedule for CozySummit 2025 Virtual — an online conference for Cozystack developers and users, hosted together with the CNCF. The talk lineup looks great. Just look at that!

Wednesday, December 3, 2025

16:00 CET (9:00 AM CT) Opening/Intro by Andrei Kvapil, Cozystack Maintainer, Ænix CEO&Founder
16:05 CET (9:05 AM CT) “How we build a multi-AZ cloud in Switzerland”, by Matthieu Robin, Hidora
16:40 CET (9:40 AM CT) “Home Lab to the Moon and Back”, by Kingdon Barrett, Navteca, LLC
17:20 CET (10:20 AM CT) “Extensibility without chaos: lessons from building Cozystack”, by Timofei Larkin, Ænix
17:55 CET (10:55 AM CT) “From AWS EC2 to Cozystack: A Beginner’s Roadmap to Cloud Independence”, by Kirti Goyal
18:10 CET (11:10 AM CT) “Integrating Proxmox with CozyStack: Advanced Container and Pod Isolation”, by Marian Koreniuk
18:25 CET (11:25 AM CT) “SeaweedFS S3 API in 2025: Enterprise‑grade security and control”, by Chris Lu, SeaweedFS
18:40 CET (11:40 AM CT) “Cozystack Storage Deep Dive”, by Moritz Wanzenböck, LINBIT
19:10 CET (12:10 PM CT) Closing Remarks by Andrei Kvapil

Check out the speakers, register for the event, join us, and share with friends and colleagues!

Cozystack applied to CNCF Incubated

Wed, 08 Oct 2025 00:00:00 +0000

Cozystack applied to CNCF Incubated

We’ve just submitted our application to move from CNCF Sandbox to Incubated. We’d love your support — drop a like to cheer us on. It won’t sway the TOC’s decision, but it means a lot to us.

Why it matters: CNCF Incubating signals a more mature project that’s ready for production use. It also attracts contributors and unlocks extra CNCF opportunities to help us grow.

Cozystack v0.36

Wed, 01 Oct 2025 00:00:00 +0000

😜 Cozystack v0.36: Server-side Encryption for S3, Kube-OVN Cluster Health Monitor, REST API Documentation

The new version of Cozystack focuses on the stability, observability, and flexible configuration of managed applications.

👉 Major Features and Improvements

Per-Namespace Resource Limits for Tenants

Resource management for Cozystack tenants has received a final patch and is now graduated to a stable feature. Platform administrators can define explicit CPU, memory, and storage limits for each tenant’s namespace via the tenant specification. This prevents any single tenant from consuming more than their share of cluster resources, ensuring cluster stability and a guaranteed service level for each tenant.

Protofire Experience Operating Kubernetes with Cozystack

Wed, 10 Sep 2025 00:00:00 +0000

Protofire Experience Operating Kubernetes with Cozystack

In a recent infrastructure transition that spanned several months, our team explored alternative container orchestration platforms to simplify operations and optimize costs. At the time, our environment consisted of nearly a hundred AWS accounts running multiple ECS services, along with managed PostgreSQL, Redis, RabbitMQ, and ALBs.

One of the goals was to consolidate our deployment architecture under Kubernetes while maintaining support for stateful services, without introducing significant operational complexity. After evaluating different options, we decided to adopt Cozystack, primarily due to its all-in-one approach and compatibility with bare-metal infrastructure.

New CNCF Webinar: Building Your Own Cloud Platform with Open Source

Fri, 05 Sep 2025 00:00:00 +0000

New CNCF Webinar: Building Your Own Cloud Platform with Open Source

We’re excited to share Andrey Kvapil’s webinar for CNCF! He dives deep into how to build a powerful cloud platform using open-source components.

Inside, you’ll find:

Architectural approaches & API design
How to select the right components
Strategies to integrate them into a robust infrastructure solution

This is a must-watch for:

Companies looking to migrate from public cloud to their own or leased servers
Hosting & service providers aiming to compete with hyperscalers like AWS

Watch now!

CNCF Webinar: One API to Rule Them All — Building a Unified Platform with Kubernetes Aggregation

Wed, 03 Sep 2025 00:00:00 +0000

CNCF Webinar: One API to Rule Them All — Building a Unified Platform with Kubernetes Aggregation

Speaker: Andrei Kvapil, Ænix CEO, Cozystack maintainer
When: Sep, 4

How do you build a unified product from a stack of open-source tools? In this talk, a Cozystack core maintainer walks through the engineering journey of integrating Helm, Operators, and the Kubernetes Aggregation Layer to build a general-purpose API — without using etcd.

Cozystack v0.35:

Thu, 21 Aug 2025 00:00:00 +0000

Cozystack v0.35: External Application Sources, Dedicated S3 Clusters and Monitoring, Hetzner RobotLB Support

The new version of Cozystack takes a major step forward in its modular (or: decomposed) architecture, enabling users to swiftly integrate custom applications and services. This significantly extends the platform’s out-of-the-box functionality to meet specific business needs. And there’s more!

What is Cozystack?

Cozystack is a free PaaS and framework for building clouds that unifies VMs, containers, and GPU workloads under Kubernetes. Companies can turn hardware into a cloud: offer users or customers managed K8s, VMs, managed data bases, applications and GPU services. With KubeVirt integration, multi-tenancy, and bare-metal simplicity, it lets enterprises deploy AI, databases, or edge apps without vendor lock-in. Cozystack is a CNCF Sandbox project.

Invitation to CozySummit Virtual — December 3

Thu, 14 Aug 2025 00:00:00 +0000

Invitation to CozySummit Virtual — December 3

Join us on December 3 for CozySummit Virtual, the first conference for CozyStack developers and users.

📢 CFP is open until September 14 — submit your talk and become one of the first speakers at our event!

CozySummit Virtual is organized by CNCF with the support of CozyStack maintainers and project sponsors.

Cozystack v0.34:

Mon, 04 Aug 2025 00:00:00 +0000

Cozystack v0.34: K8s Version Selection and PVC Snapshots in Tenants, Windows and RouterOS on VMs, VPA for VPA

Our maintainers and contributors never stand still, and we’re already ready to present the next stable release of Cozystack v0.34. In this release, we continued working on expanding the functionality of the Vertical Pod Autoscaler, improving tenant clusters, enhancing the backup system, and moving toward platform decomposition.

Below, we’ll cover the most important changes, and you can find the full list of fixes in the links at the end of the announcement.

Cozystack v0.31–0.33

Wed, 09 Jul 2025 00:00:00 +0000

Cozystack v0.31–0.33 Releases: Air Gap, Backup System, AI workloads in K8s, replace for Helm and other features

It’s been a while since we last covered Cozystack’s updates — time to fix that! We’re thrilled to showcase a wealth of new features and key improvements in this roundup. For brevity, we’ve curated only the most significant changes here (you’ll find all fixes and enhancements in the release notes, linked throughout the article).

Cozyhr: How We Simplified Local Development with Helm and Flux

Wed, 18 Jun 2025 00:00:00 +0000

Cozyhr: How We Simplified Local Development with Helm and Flux

Hi! I’m Andrei Kvapil CEO of Ænix and developer of Cozystack, an open source platform and framework for building cloud infrastructure. In this article I’ll walk through the way we deliver applications to Kubernetes, explain why regular GitOps can be awkward in local development, an show how the new tool cozyhr fixes those pain points. The article targets engineers who already know Helm and Flux.

Cozystack became a Certified Kubernetes Platform

Fri, 06 Jun 2025 00:00:00 +0000

Cozystack became a Certified Kubernetes Platform

We’re proud to announce: Cozystack has achieved Certified Kubernetes Platform status. Thanks to our community and especially to our good friends from Hidora.

The Evolution of Virtualization Platforms: The Rise of Managed Services and Local Providers’ Edge…

Wed, 04 Jun 2025 00:00:00 +0000

The Evolution of Virtualization Platforms: The Rise of Managed Services and Local Providers’ Edge Against Hyperscalers

Hello everyone! I’m Andrey Kvapil, CEO of Ænix and developer of Cozystack, an open-source platform and framework for building cloud infrastructure. In this article, I want to share my perspective on how modern cloud patterns have transformed infrastructure approaches, the evolving role of service providers and public clouds in this landscape, and most importantly, how virtualization’s purpose has fundamentally changed in today’s infrastructure stack.

Cozystack Recognized in CNCF's CNAI Landscape!

Wed, 21 May 2025 00:00:00 +0000

🚀 Cozystack Recognized in CNCF’s CNAI Landscape!

We’re thrilled to share that Cozystack has been added to the Cloud Native AI (CNAI) Landscape by the Cloud Native Computing Foundation (CNCF)! This is a significant validation of our work in bridging cloud-native infrastructure with AI workloads.

Why This Matters:

Industry Recognition: Being featured alongside major players confirms Cozystack’s role in shaping the future of AI infrastructure
Technical Validation: Highlights our capabilities in GPU provisioning, scalable Kubernetes deployments, and platform engineering for AI/ML
Ecosystem Growth: Strengthens our position in the open source cloud-native community

A huge thank you to our contributors, users, and the CNCF community for making this possible! This is just the beginning of our journey to simplify AI infrastructure deployment.

A Simple Way to Install Talos Linux on Any Machine, with Any Provider

Mon, 28 Apr 2025 00:00:00 +0000

A Simple Way to Install Talos Linux on Any Machine, with Any Provider

Talos Linux is a specialized operating system designed for running Kubernetes. In my opinion, it does that task better than others. First and foremost it handles full lifecycle management for Kubernetes control-plane components.

On the other hand, Talos Linux focuses on security, minimizing the user’s ability to influence the system. A distinctive feature of this OS is the near-complete absence of executables, including the absence of a shell and the inability to log in via SSH. All configuration of Talos Linux is done through a Kubernetes-like API.

Cozystack Now Offers GPU Passthrough for AI/ML Virtual Machines

Fri, 18 Apr 2025 00:00:00 +0000

Cozystack Now Offers GPU Passthrough for AI/ML Virtual Machines

The open-source cloud platform has introduced direct GPU passthrough in its latest release, enabling users to accelerate AI, machine learning, and other compute-intensive workloads on virtual machines. By leveraging physical GPUs from host nodes, teams can now deploy open-source AI stacks without proprietary cloud dependencies.

Upcoming features include vGPU resource partitioning and a Kubernetes-native GPU operator for multi-tenant clusters.

Read the documentation →

Updates to the Open-Source Platform Cozystack 0.24–0.29:

Thu, 10 Apr 2025 00:00:00 +0000

Updates to the Open-Source Platform Cozystack 0.24–0.29: PXE Machine Provisioning, Inter-Datacenter RTT Monitoring, and Dedicated IP Addresses for VMs

We haven’t shared much about Cozystack’s new features lately, even though we’ve released six new versions over the past month and a half: 0.24, 0.25, 0.26, 0.27, 0.28, and 0.29. Let’s take a closer look at the changes, starting from the latest release and going back to version 0.24.

What is Cozystack?

Cozystack Becomes a CNCF Sandbox Project

Thu, 13 Mar 2025 00:00:00 +0000

Cozystack Becomes a CNCF Sandbox Project

On February 28, members of the CNCF Technical Oversight Committee completed their voting and unanimously accepted Cozystack, a platform for building private clouds and PaaS, into the CNCF Sandbox. The project is currently undergoing the onboarding process. Let’s break down what this means in practice, what Cozystack is, and what the CNCF Sandbox represents.

What is Cozystack?

Cozystack is an open-source platform that enables the creation of a bare metal cloud for deploying proven cloud-native and open-source tools: managed Kubernetes clusters, databases as a service, applications as a service, and virtual machines based on KubeVirt (see the full list of components). Cozystack also provides a ready-made stack for observability and alerting based on Victoria Metrics, Victoria Logs, Grafana, and Alerta.

Cozystack v0.22 Release: telemetry, patched Talos v1.9.1, new entities Workload and WorkloadMonitor

Fri, 17 Jan 2025 00:00:00 +0000

Cozystack v0.22 Release: telemetry, patched Talos v1.9.1, new entities Workload and WorkloadMonitor

Main changes

In the latest release was added cozystack-controller and new entities: Workload and WorkloadMonitor, which allow monitoring the state of pods managed by operators and evaluating the service level according to predefined rules.

Since different applications in Cozystack are managed by different operators, we decided to create a unified format for displaying the status of each service.

Introducing the Pre-New Year Release of open source platform Cozystack v0.21:

Sat, 28 Dec 2024 00:00:00 +0000

Introducing the Pre-New Year Release of open source platform Cozystack v0.21: New User Dashboard, Talos Linux, etc.

The dashboard now works directly with the Cozystack API instead of relying on FluxCD resources. This enhancement enables the platform to provide a user-friendly graphical interface while integrating with Kubernetes’ standard RBAC model for managing deployment permissions.

Each tenant now includes four default groups:
view: Read-only access.
use: Access to virtual machines and service usage.
admin: Ability to deploy core services (MySQL, PostgreSQL, Redis, Kubernetes, virtual machines, etc.).
super-admin: Manage child tenants and deploy service-level components (monitoring, etcd, ingress, seaweedfs, etc.).

Cozystack v0.20 Release: Terraform, Keycloak, and Stability & Security Improvements

Thu, 12 Dec 2024 00:00:00 +0000

Cozystack v0.20 Release: Terraform, Keycloak, and Stability & Security Improvements

This release focuses on enhancing stability while addressing a significant number of bugs and introducing new features.

What’s new

Kube-OVN updated to the latest stable release.
Improved logic in KubeVirt CCM, delivering more reliable load balancers for tenant Kubernetes clusters.
Resolved user permissions issues in OIDC.
Added a dedicated cluster admin group.
Fixed alerts and dashboards in Grafana.
NATs now supports enabling JetStream and passing configuration files.
Introduced Terraform support for interacting with our API.

In v0.19, we introduced OIDC support, along with the integration of Keycloak. However, due to the need for stability improvements, we did not announce v0.19 separately. With this release, Keycloak is bundled with Cozystack, providing seamless OIDC support.

How we built a dynamic Kubernetes API Server for the API Aggregation Layer in Cozystack

Thu, 12 Dec 2024 00:00:00 +0000

How we built a dynamic Kubernetes API Server for the API Aggregation Layer in Cozystack

Hi there! I’m Andrei Kvapil, but you might know me as @kvaps in communities dedicated to Kubernetes and cloud-native tools. In this article, I want to share how we implemented our own extension api-server in the open-source PaaS platform, Cozystack.

Kubernetes truly amazes me with its powerful extensibility features. You’re probably already familiar with the controller concept and frameworks like kubebuilder and operator-sdk that help you implement it. In a nutshell, they allow you to extend your Kubernetes cluster by defining custom resources (CRDs) and writing additional controllers that handle your business logic for reconciling and managing these kinds of resources. This approach is well-documented, with a wealth of information available online on how to develop your own operators.

Cozystack v0.18

Thu, 07 Nov 2024 00:00:00 +0000

Cozystack v0.18 Release: Public API Server, Metrics and Logs from Tenant Clusters, and Other Improvements

🔥 Public API for Cozystack

This is the biggest and most anticipated update for us. Cozystack now includes its own Kubernetes API server, which automatically translates all requests to custom resources into HelmReleases.

This means that platform administrators can now provide users with granular access to specific resources (Kuberneteses, VirtualMachines, Postgresses, etc.). Additionally, the API server can be easily extended with additional components by simply listing them in a ConfigMap — no recompilation required.

What’s New in Cozystack v0.17

Thu, 24 Oct 2024 00:00:00 +0000

What’s New in Cozystack v0.17: Windows on VMs, VM image upload app, and web interface for S3 buckets

This update mainly focuses on enhancing the platform’s virtualization features, while also introducing several other improvements.

Today marks the release of an updated version of the free PaaS system, Cozystack. Built on Kubernetes, Cozystack consists of numerous open technologies and provides all the essential tools for running managed services on your own hardware. The platform is distributed under the Apache 2.0 license.

Cozystack on Hacktoberfest: become a part of the global IT event!

Fri, 04 Oct 2024 00:00:00 +0000

Cozystack on Hacktoberfest: become a part of the global IT event!

We’ve decided to participate in Hacktoberfest. If you’re participating too, come visit our GitHub and check out the amazing issues. And if something seems unclear, follow the links below; you’ll find all the answers there :)

🫡 Rules and details https://hacktoberfest.com

😜 Cozystack GitHub https://github.com/aenix-io/cozystack

❤️ Cozystack community for all who wants to be a part of Hacktoberfest and asking questions https://t.me/cozystack

The Open Source Platform Cozystack Version 0.16.0

Thu, 03 Oct 2024 00:00:00 +0000

The Open Source Platform Cozystack Version 0.16.0 Released: Alert System with Telegram Notifications and More Improvements

Key Highlights Cozystack now features an alert system based on the open-source tool Alerta, with the ability to configure notifications directly to Telegram. Additionally, you can receive alerts from k8s-prometheus stack, all Grafana dashboards have been updated, as well as Grafana itself and the grafana-operator.

Alerta interface

Cozystack is an Open Source platform designed for building cloud infrastructure on bare metal, enabling rapid deployment of managed Kubernetes, database as a service, applications as a service, and virtual machines based on KubeVirt. Within the platform, you can deploy services like Kafka, FerretDB, PostgreSQL, Cilium, Grafana, Victoria Metrics, and others with just a single click.

Recent Changes in the Cozystack Open Source Platform: Opencost, Log Collection System, Bridge…

Thu, 26 Sep 2024 00:00:00 +0000

Recent Changes in the Cozystack Open Source Platform: Opencost, Log Collection System, Bridge Binding in Virtual Machines

Over the past couple of months, we have been actively developing our Cozystack Open Source platform, and today we’re presenting the improvements introduced from v0.12 to v0.15.

Cozystack is an Open Source platform that enables building a cloud on bare metal for rapid deployment of managed Kubernetes, database as a service, applications as a service, and virtual machines based on KubeVirt. Within the platform, you can deploy Kafka, FerretDB, PostgreSQL, Cilium, Grafana, Victoria Metrics, and other services with a single click.

Cozystack has officially been included in the CNCF Landscape

Wed, 25 Sep 2024 00:00:00 +0000

Cozystack has officially been included in the CNCF Landscape

You can now find the Cozystack open source platform in the CNCF Landscape categories of Platform and Certified Kubernetes — Installed. Despite being a relatively young platform, Cozystack is experiencing rapid growth, and an active community of developers and users has already formed around it.

Weekly open meetings are held to discuss the platform’s development, and the Telegram chat is constantly buzzing with discussions and questions. Our speakers are regularly invited to industry conferences to share their experiences with integrating and utilizing various Open Source components within the platform. For example, you can check out a recent video from the Talos Linux Install Fest and a series of articles on building your own cloud in the official Kubernetes blog.

Installing a Kubernetes Cluster Managed by Cozystack: A Detailed Guide by Gohost and Ænix

Fri, 16 Aug 2024 00:00:00 +0000

Installing a Kubernetes Cluster Managed by Cozystack: A Detailed Guide by Gohost and Ænix

This article was written by Vladislav Karabasov from Kazakhstani hosting company gohost, therefore the narrative will be conducted in the first person.

At the time of my transition to gohost.kz, the company had already been operating in the Kazakhstan market for 15 years, providing clients with a standard set of services: VPS/VDC, IaaS, virtual hosting, etc. However, clients developed new needs, so I was tasked with developing the direction of Kubernetes as a Service.

Cozystack v0.11

Thu, 15 Aug 2024 00:00:00 +0000

Cozystack v0.11 Open Source platform has been released: S3, improved tenant isolation, UI enhancements, and other features

The Cozystack v0.11 release is now available for download, installation, or updating current installations.

Key changes:
— Added S3 support. Implemented the basic SeaweedFS functionality in Cozystack. Developed a Kubernetes-COSI driver for automatic S3 bucket provisioning. Added support for automatic volume resizing in the SeaweedFS chart.
— Network isolation between tenants. Significant work was done to enhance network isolation between tenants, bugs were fixed, and network policies were completely revamped.
— UI update. All service icons have been replaced. The dashboard has been redesigned to display only the necessary information in ResourceView. There is now an option to specify which htcehcs to display by listing them in a special role -dashboard-resources.
— Added a Development Guide section to the documentation and updated the installation guide for Hetzner.
— Cilium updated to v1.16, which includes our patch for automatic device detection.
— Resolved garbage collector issues in tenant Kubernetes clusters.
— Fixed issues with forwarding HTTP and HTTPS traffic using ingress in tenant Kubernetes clusters.
— Added snapshot-controller and object-storage-controller.
— LINSTOR updated to v1.28.
— Talos Linux updated to v1.7.6.
— Kube-OVN now built from the stable base.
— Refined the logic for substituting image digests in values, resulting in fewer modifications to the original charts.

Introducing Talm, a configuration manager for Talos Linux

Wed, 29 May 2024 00:00:00 +0000

Author: Andrei Kvapil (Ænix)

The Cozystack project has released Talm, a configuration manager for Talos Linux

The developers of the open-source PaaS platform Cozystack have prepared the Talm project, aimed at simplifying the configuration of bare-metal servers for Talos Linux, an operating system designed to run Kubernetes with a Kubernetes-like API and configured via a single Yaml manifest. Although Talm was created to describe the declarative installation of Cozystack, it is not tied specifically to this platform and can be used to manage any Talos Linux configurations. The project is developed under the MPL license.

DIY: Create Your Own Cloud with Kubernetes (Part 3)

Fri, 05 Apr 2024 07:40:00 +0000

Author: Andrei Kvapil (Ænix)

Approaching the most interesting phase, this article delves into running Kubernetes within Kubernetes. Technologies such as Kamaji and Cluster API are highlighted, along with their integration with KubeVirt.

Previous discussions have covered preparing Kubernetes on bare metal and how to turn Kubernetes into virtual machines management system. This article concludes the series by explaining how, using all of the above, you can build a full-fledged managed Kubernetes and run virtual Kubernetes clusters with just a click.

DIY: Create Your Own Cloud with Kubernetes (Part 2)

Fri, 05 Apr 2024 07:35:00 +0000

Author: Andrei Kvapil (Ænix)

Continuing our series of posts on how to build your own cloud using just the Kubernetes ecosystem. In the previous article, we explained how we prepare a basic Kubernetes distribution based on Talos Linux and Flux CD. In this article, we’ll show you a few various virtualization technologies in Kubernetes and prepare everything need to run virtual machines in Kubernetes, primarily storage and networking.

We will talk about technologies such as KubeVirt, LINSTOR, and Kube-OVN.

DIY: Create Your Own Cloud with Kubernetes (Part 1)

Fri, 05 Apr 2024 07:30:00 +0000

Author: Andrei Kvapil (Ænix)

At Ænix, we have a deep affection for Kubernetes and dream that all modern technologies will soon start utilizing its remarkable patterns.

Have you ever thought about building your own cloud? I bet you have. But is it possible to do this using only modern technologies and approaches, without leaving the cozy Kubernetes ecosystem? Our experience in developing Cozystack required us to delve deeply into it.

Introducing Cozystack: A Free PaaS Platform based on Kubernetes

Wed, 21 Feb 2024 00:00:00 +0000

Author: Andrei Kvapil (Ænix)

Published the first release of the free PaaS platform Cozystack, based on Kubernetes. The project positioned as a ready-to-use platform for hosting providers and a framework for building private and public clouds. The code is available on GitHub and is distributed under the Apache-2.0 license.

Cozystack is a system that is installed directly on servers and covers all aspects of preparing infrastructure for providing managed services. The installed platform allows to spawn tenant Kubernetes clusters, databases, and virtual machines on demand.

Configuring routing for MetalLB in L2 mode

Thu, 14 May 2020 00:00:00 +0000

Configuring routing for MetalLB in L2 mode

Not so far ago, I was faced with a quite unusual task of configuring routing for MetalLB. All would be nothing, since MetalLB usually does not require any additional configuration from user side, but in our case there is a fairly large cluster with a quite simple network configuration.

In this article I will show you how to configure source-based and policy-based routing for the external network on your cluster.

BootBox Service Reference

Mon, 01 Jan 0001 00:00:00 +0000

Parameters

Common parameters

Name	Description	Type	Value
`whitelistHTTP`	Secure HTTP by enabling client networks whitelisting.	`bool`	`true`
`whitelist`	List of client networks.	`[]string`	`[]`
`machines`	Configuration of physical machine instances.	`[]object`	`[]`
`machines[i].hostname`	Hostname.	`string`	`""`
`machines[i].arch`	Architecture.	`string`	`""`
`machines[i].ip`	IP address configuration.	`object`	`{}`
`machines[i].ip.address`	IP address.	`string`	`""`
`machines[i].ip.gateway`	IP gateway.	`string`	`""`
`machines[i].ip.netmask`	Netmask.	`string`	`""`
`machines[i].leaseTime`	Lease time.	`int`	`0`
`machines[i].mac`	MAC addresses.	`[]string`	`[]`
`machines[i].nameServers`	Name servers.	`[]string`	`[]`
`machines[i].timeServers`	Time servers.	`[]string`	`[]`
`machines[i].uefi`	UEFI.	`bool`	`false`

BootBox Service Reference

Mon, 01 Jan 0001 00:00:00 +0000

Parameters

Common parameters

Name	Description	Type	Value
`whitelistHTTP`	Secure HTTP by enabling client networks whitelisting.	`bool`	`true`
`whitelist`	List of client networks.	`[]string`	`[]`
`machines`	Configuration of physical machine instances.	`[]object`	`[]`
`machines[i].hostname`	Hostname.	`string`	`""`
`machines[i].arch`	Architecture.	`string`	`""`
`machines[i].ip`	IP address configuration.	`object`	`{}`
`machines[i].ip.address`	IP address.	`string`	`""`
`machines[i].ip.gateway`	IP gateway.	`string`	`""`
`machines[i].ip.netmask`	Netmask.	`string`	`""`
`machines[i].leaseTime`	Lease time.	`int`	`0`
`machines[i].mac`	MAC addresses.	`[]string`	`[]`
`machines[i].nameServers`	Name servers.	`[]string`	`[]`
`machines[i].timeServers`	Time servers.	`[]string`	`[]`
`machines[i].uefi`	UEFI.	`bool`	`false`

Mon, 01 Jan 0001 00:00:00 +0000

SeaweedFS Service Reference

Mon, 01 Jan 0001 00:00:00 +0000

Parameters

Common parameters

Name	Description	Type	Value
`host`	The hostname used to access SeaweedFS externally (defaults to ‘s3’ subdomain for the tenant host).	`string`	`""`
`topology`	The topology of the SeaweedFS cluster.	`string`	`Simple`
`replicationFactor`	Replication factor: number of replicas for each volume in the SeaweedFS cluster.	`int`	`2`

SeaweedFS Components Configuration

Name	Description	Type	Value
`db`	Database configuration.	`object`	`{}`
`db.replicas`	Number of database replicas.	`int`	`2`
`db.size`	Persistent Volume size.	`quantity`	`10Gi`
`db.storageClass`	StorageClass used to store the data.	`string`	`""`
`db.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`db.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`db.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`db.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`master`	Master service configuration.	`object`	`{}`
`master.replicas`	Number of master replicas.	`int`	`3`
`master.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`master.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`master.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`master.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`filer`	Filer service configuration.	`object`	`{}`
`filer.replicas`	Number of filer replicas.	`int`	`2`
`filer.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`filer.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`filer.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`filer.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`filer.grpcHost`	The hostname used to expose or access the filer service externally.	`string`	`""`
`filer.grpcPort`	The port used to access the filer service externally.	`int`	`443`
`filer.whitelist`	A list of IP addresses or CIDR ranges that are allowed to access the filer service.	`[]string`	`[]`
`volume`	Volume service configuration.	`object`	`{}`
`volume.replicas`	Number of volume replicas.	`int`	`2`
`volume.size`	Persistent Volume size.	`quantity`	`10Gi`
`volume.storageClass`	StorageClass used to store the data.	`string`	`""`
`volume.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`volume.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`volume.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`volume.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`volume.zones`	A map of zones for MultiZone topology. Each zone can have its own number of replicas and size.	`map[string]object`	`{}`
`volume.zones[name].replicas`	Number of replicas in the zone.	`int`	`0`
`volume.zones[name].size`	Zone storage size.	`quantity`	`""`
`s3`	S3 service configuration.	`object`	`{}`
`s3.replicas`	Number of S3 replicas.	`int`	`2`
`s3.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`s3.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`s3.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`s3.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`

SeaweedFS Service Reference

Mon, 01 Jan 0001 00:00:00 +0000

Parameters

Common parameters

Name	Description	Type	Value
`host`	The hostname used to access SeaweedFS externally (defaults to ‘s3’ subdomain for the tenant host).	`string`	`""`
`topology`	The topology of the SeaweedFS cluster.	`string`	`Simple`
`replicationFactor`	Replication factor: number of replicas for each volume in the SeaweedFS cluster.	`int`	`2`

SeaweedFS Components Configuration

Name	Description	Type	Value
`db`	Database configuration.	`object`	`{}`
`db.replicas`	Number of database replicas.	`int`	`2`
`db.size`	Persistent Volume size.	`quantity`	`10Gi`
`db.storageClass`	StorageClass used to store the data.	`string`	`""`
`db.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`db.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`db.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`db.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`master`	Master service configuration.	`object`	`{}`
`master.replicas`	Number of master replicas.	`int`	`3`
`master.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`master.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`master.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`master.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`filer`	Filer service configuration.	`object`	`{}`
`filer.replicas`	Number of filer replicas.	`int`	`2`
`filer.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`filer.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`filer.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`filer.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`filer.grpcHost`	The hostname used to expose or access the filer service externally.	`string`	`""`
`filer.grpcPort`	The port used to access the filer service externally.	`int`	`443`
`filer.whitelist`	A list of IP addresses or CIDR ranges that are allowed to access the filer service.	`[]string`	`[]`
`volume`	Volume service configuration.	`object`	`{}`
`volume.replicas`	Number of volume replicas.	`int`	`2`
`volume.size`	Persistent Volume size.	`quantity`	`10Gi`
`volume.storageClass`	StorageClass used to store the data.	`string`	`""`
`volume.diskType`	SeaweedFS disk type tag for the default volume servers (e.g., “hdd”, “ssd”).	`string`	`""`
`volume.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`volume.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`volume.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`volume.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`
`volume.zones`	A map of zones for MultiZone topology. Each zone can have its own number of replicas and size.	`map[string]object`	`{}`
`volume.zones[name].replicas`	Number of replicas in the zone.	`int`	`0`
`volume.zones[name].size`	Zone storage size.	`quantity`	`""`
`volume.zones[name].dataCenter`	SeaweedFS data center name for this zone. Defaults to the zone name.	`string`	`""`
`volume.zones[name].nodeSelector`	YAML nodeSelector for this zone (default: topology.kubernetes.io/zone: ).	`string`	`""`
`volume.zones[name].storageClass`	StorageClass used to store zone data. Defaults to volume.storageClass.	`string`	`""`
`volume.zones[name].pools`	A map of storage pools for this zone. Each pool creates a separate Volume StatefulSet per zone.	`map[string]object`	`{}`
`volume.zones[name].pools[name].diskType`	SeaweedFS disk type tag (e.g., “ssd”, “hdd”, “nvme”).	`string`	`""`
`volume.zones[name].pools[name].replicas`	Number of volume replicas. Defaults to volume.replicas (Simple) or zone.replicas/volume.replicas (MultiZone).	`int`	`0`
`volume.zones[name].pools[name].size`	Persistent Volume size. Defaults to volume.size (Simple) or zone.size/volume.size (MultiZone).	`quantity`	`""`
`volume.zones[name].pools[name].storageClass`	Kubernetes StorageClass for the pool. Defaults to volume.storageClass.	`string`	`""`
`volume.zones[name].pools[name].resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`volume.zones[name].pools[name].resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`volume.zones[name].pools[name].resources.memory`	Amount of memory allocated.	`quantity`	`""`
`volume.zones[name].pools[name].resourcesPreset`	Default sizing preset used when `resources` is omitted. Defaults to volume.resourcesPreset.	`string`	`{}`
`volume.pools`	A map of storage pools. Each pool creates a separate Volume StatefulSet with its own disk type.	`map[string]object`	`{}`
`volume.pools[name].diskType`	SeaweedFS disk type tag (e.g., “ssd”, “hdd”, “nvme”).	`string`	`""`
`volume.pools[name].replicas`	Number of volume replicas. Defaults to volume.replicas (Simple) or zone.replicas/volume.replicas (MultiZone).	`int`	`0`
`volume.pools[name].size`	Persistent Volume size. Defaults to volume.size (Simple) or zone.size/volume.size (MultiZone).	`quantity`	`""`
`volume.pools[name].storageClass`	Kubernetes StorageClass for the pool. Defaults to volume.storageClass.	`string`	`""`
`volume.pools[name].resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`volume.pools[name].resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`volume.pools[name].resources.memory`	Amount of memory allocated.	`quantity`	`""`
`volume.pools[name].resourcesPreset`	Default sizing preset used when `resources` is omitted. Defaults to volume.resourcesPreset.	`string`	`{}`
`s3`	S3 service configuration.	`object`	`{}`
`s3.replicas`	Number of S3 replicas.	`int`	`2`
`s3.resources`	Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.	`object`	`{}`
`s3.resources.cpu`	Number of CPU cores allocated.	`quantity`	`""`
`s3.resources.memory`	Amount of memory allocated.	`quantity`	`""`
`s3.resourcesPreset`	Default sizing preset used when `resources` is omitted.	`string`	`small`