Upstream/ansible-role-k3s
1
0
Fork 0
mirror of https://github.com/PyratLabs/ansible-role-k3s.git synced 2024-11-22 11:48:22 +01:00
Code Issues Projects Releases Wiki Activity

A number of enhancements for v1.19 release.

Browse Source 
- Added option to skip validation checks #47
  - Add SELinux support in containerd #48
  - Added check for Etcd member count #46
  - Moved token to a file #50
  - Added Etcd snapshot configuration options #49
This commit is contained in:
ᗪєνιη ᗷυнʟ 2020-09-21 14:38:51 -04:00 committed by Xan Manning
parent 1438ddde69
commit c447fcec39
12 changed files with 146 additions and 60 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

121
README.md
View File

@ -45,59 +45,65 @@ my spare time so I cannot promise a speedy fix delivery.
Below are variables that are set against all of the play hosts for environment
consistency.
| Variable | Description | Default Value |
|------------------------------------------|-------------------------------------------------------------------------------------|-----------------------------------------|
| `k3s_cluster_state` | State of cluster: installed, started, stopped, restarted, downloaded, uninstalled. | installed |
| `k3s_release_version` | Use a specific version of k3s, eg. `v0.2.0`. Specify `false` for stable. | `false` |
| `k3s_build_cluster` | When multiple `play_hosts` are available, attempt to cluster. Read notes below. | `true` |
| `k3s_github_url` | Set the GitHub URL to install k3s from. | https://github.com/rancher/k3s |
| `k3s_install_dir` | Installation directory for k3s. | `/usr/local/bin` |
| `k3s_install_hard_links` | Install using hard links rather than symbolic links. | `false` |
| `k3s_server_manifests_dir` | Path for place the `k3s_server_manifests_templates`. | `/var/lib/rancher/k3s/server/manifests` |
| `k3s_server_manifests_templates` | A list of Auto-Deploying Manifests Templates. | [] |
| `k3s_use_experimental` | Allow the use of experimental features in k3s. | `false` |
| `k3s_use_unsupported_config` | Allow the use of unsupported configurations in k3s. | `false` |
| `k3s_non_root` | Install k3s as non-root user. See notes below. | `false` |
| `k3s_cluster_cidr` | Network CIDR to use for pod IPs | 10.42.0.0/16 |
| `k3s_service_cidr` | Network CIDR to use for service IPs | 10.43.0.0/16 |
| `k3s_control_node_address` | Use a specific control node address. IP or FQDN. | _NULL_ |
| `k3s_control_token` | Use a specific control token, please read notes below. | _NULL_ |
| `k3s_private_registry` | Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") | _NULL_ |
| `k3s_https_port` | HTTPS port listening port. | 6443 |
| `k3s_use_docker` | Use Docker rather than Containerd? | `false` |
| `k3s_no_flannel` | Do not use Flannel | `false` |
| `k3s_flannel_backend` | Flannel backend ('none', 'vxlan', 'ipsec', 'host-gw' or 'wireguard') | vxlan |
| `k3s_no_coredns` | Do not use CoreDNS | `false` |
| `k3s_cluster_dns` | Cluster IP for CoreDNS service. Should be in your service-cidr range. | _NULL_ |
| `k3s_cluster_domain` | Cluster Domain. | cluster.local |
| `k3s_resolv_conf` | Kubelet resolv.conf file | _NULL_ |
| `k3s_no_traefik` | Do not use Traefik | `false` |
| `k3s_no_servicelb` | Do not use ServiceLB, necessary for using something like MetalLB. | `false` |
| `k3s_no_local_storage` | Do not use Local Storage | `false` |
| `k3s_default_local_storage_path` | Set Local Storage Path. Specify `false` for default. | `false` |
| `k3s_no_metrics_server` | Do not deploy metrics server | `false` |
| `k3s_kube_apiserver_args` | Customized flag for kube-apiserver process | [] |
| `k3s_kube_scheduler_args` | Customized flag for kube-scheduler process | [] |
| `k3s_kube_controller_manager_args` | Customized flag for kube-controller-manager process | [] |
| `k3s_kube_cloud_controller_manager_args` | Customized flag for kube-cloud-controller-manager process | [] |
| `k3s_disable_scheduler` | Disable Kubernetes default scheduler | `false` |
| `k3s_disable_cloud_controller` | Disable k3s default cloud controller manager. | `false` |
| `k3s_disable_network_policy` | Disable k3s default network policy controller. | `false` |
| `k3s_write_kubeconfig_mode` | Define the file mode from the generated KubeConfig, eg. `644` | _NULL_ |
| `k3s_datastore_endpoint` | Define the database or etcd cluster endpoint for HA. | _NULL_ |
| `k3s_datastore_cafile` | Define the database TLS CA file. | _NULL_ |
| `k3s_datastore_certfile` | Define the database TLS Cert file. | _NULL_ |
| `k3s_datastore_keyfile` | Define the database TLS Key file. | _NULL_ |
| `k3s_become_for_all` | Enable become for all (where value for `k3s_become_for_*` is _NULL_ | `false` |
| `k3s_become_for_systemd` | Enable become for systemd commands. | _NULL_ |
| `k3s_become_for_install_dir` | Enable become for writing to `k3s_install_dir`. | _NULL_ |
| `k3s_become_for_usr_local_bin` | Enable become for writing to `/usr/local/bin/`. | _NULL_ |
| `k3s_become_for_package_install` | Enable become for installing prerequisite packages. | _NULL_ |
| `k3s_become_for_kubectl` | Enable become for kubectl commands. | _NULL_ |
| `k3s_become_for_uninstall` | Enable become for running uninstall scripts. | _NULL_ |
| `k3s_etcd_datastore` | Use Embedded Etcd as the database backend for HA. (EXPERIMENTAL) | `false` |
| `k3s_secrets_encryption` | Use secrets encryption at rest. (EXPERIMENTAL) | `false` |
| `k3s_debug` | Enable debug logging on the k3s service | `false` |
| Variable | Description | Default Value |
|------------------------------------------|-------------------------------------------------------------------------------------|--------------------------------------------|
| `k3s_cluster_state` | State of cluster: installed, started, stopped, restarted, downloaded, uninstalled. | installed |
| `k3s_release_version` | Use a specific version of k3s, eg. `v0.2.0`. Specify `false` for stable. | `false` |
| `k3s_build_cluster` | When multiple `play_hosts` are available, attempt to cluster. Read notes below. | `true` |
| `k3s_github_url` | Set the GitHub URL to install k3s from. | https://github.com/rancher/k3s |
| `k3s_skip_validation` | Skip all tasks that validate configuration. | `false` |
| `k3s_install_dir` | Installation directory for k3s. | `/usr/local/bin` |
| `k3s_install_hard_links` | Install using hard links rather than symbolic links. | `false` |
| `k3s_server_manifests_dir` | Path for place the `k3s_server_manifests_templates`. | `/var/lib/rancher/k3s/server/manifests` |
| `k3s_server_manifests_templates` | A list of Auto-Deploying Manifests Templates. | [] |
| `k3s_use_experimental` | Allow the use of experimental features in k3s. | `false` |
| `k3s_use_unsupported_config` | Allow the use of unsupported configurations in k3s. | `false` |
| `k3s_non_root` | Install k3s as non-root user. See notes below. | `false` |
| `k3s_cluster_cidr` | Network CIDR to use for pod IPs | 10.42.0.0/16 |
| `k3s_service_cidr` | Network CIDR to use for service IPs | 10.43.0.0/16 |
| `k3s_control_node_address` | Use a specific control node address. IP or FQDN. | _NULL_ |
| `k3s_control_token` | Use a specific control token, please read notes below. | _NULL_ |
| `k3s_private_registry` | Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") | _NULL_ |
| `k3s_https_port` | HTTPS port listening port. | 6443 |
| `k3s_use_docker` | Use Docker rather than Containerd? | `false` |
| `k3s_no_flannel` | Do not use Flannel | `false` |
| `k3s_flannel_backend` | Flannel backend ('none', 'vxlan', 'ipsec', 'host-gw' or 'wireguard') | vxlan |
| `k3s_no_coredns` | Do not use CoreDNS | `false` |
| `k3s_cluster_dns` | Cluster IP for CoreDNS service. Should be in your service-cidr range. | _NULL_ |
| `k3s_cluster_domain` | Cluster Domain. | cluster.local |
| `k3s_resolv_conf` | Kubelet resolv.conf file | _NULL_ |
| `k3s_no_traefik` | Do not use Traefik | `false` |
| `k3s_no_servicelb` | Do not use ServiceLB, necessary for using something like MetalLB. | `false` |
| `k3s_no_local_storage` | Do not use Local Storage | `false` |
| `k3s_default_local_storage_path` | Set Local Storage Path. Specify `false` for default. | `false` |
| `k3s_no_metrics_server` | Do not deploy metrics server | `false` |
| `k3s_kube_apiserver_args` | Customized flag for kube-apiserver process | [] |
| `k3s_kube_scheduler_args` | Customized flag for kube-scheduler process | [] |
| `k3s_kube_controller_manager_args` | Customized flag for kube-controller-manager process | [] |
| `k3s_kube_cloud_controller_manager_args` | Customized flag for kube-cloud-controller-manager process | [] |
| `k3s_disable_scheduler` | Disable Kubernetes default scheduler | `false` |
| `k3s_disable_cloud_controller` | Disable k3s default cloud controller manager. | `false` |
| `k3s_disable_network_policy` | Disable k3s default network policy controller. | `false` |
| `k3s_write_kubeconfig_mode` | Define the file mode from the generated KubeConfig, eg. `644` | _NULL_ |
| `k3s_datastore_endpoint` | Define the database or etcd cluster endpoint for HA. | _NULL_ |
| `k3s_datastore_cafile` | Define the database TLS CA file. | _NULL_ |
| `k3s_datastore_certfile` | Define the database TLS Cert file. | _NULL_ |
| `k3s_datastore_keyfile` | Define the database TLS Key file. | _NULL_ |
| `k3s_become_for_all` | Enable become for all (where value for `k3s_become_for_*` is _NULL_ | `false` |
| `k3s_become_for_systemd` | Enable become for systemd commands. | _NULL_ |
| `k3s_become_for_install_dir` | Enable become for writing to `k3s_install_dir`. | _NULL_ |
| `k3s_become_for_usr_local_bin` | Enable become for writing to `/usr/local/bin/`. | _NULL_ |
| `k3s_become_for_package_install` | Enable become for installing prerequisite packages. | _NULL_ |
| `k3s_become_for_kubectl` | Enable become for kubectl commands. | _NULL_ |
| `k3s_become_for_uninstall` | Enable become for running uninstall scripts. | _NULL_ |
| `k3s_etcd_datastore` | Use Embedded Etcd as the database backend for HA. (EXPERIMENTAL) | `false` |
| `k3s_etcd_disable_snapshots` | Disable Etcd snapshots. | `false` |
| `k3s_etcd_snapshot_schedule_cron` | Etcd snapshot cron schedule. | "`* */12 * * *`" |
| `k3s_etcd_snapshot_retention` | Etcd snapshot retention. | 5 |
| `k3s_etcd_snapshot_directory` | Etcd snapshot directory. | `/var/lib/rancher/k3s/server/db/snapshots` |
| `k3s_secrets_encryption` | Use secrets encryption at rest. (EXPERIMENTAL) | `f alse` |
| `k3s_debug` | Enable debug logging on the k3s service | `false` |
| `k3s_enable_selinux` | Enable SELinux in containerd. (EXPERIMENTAL) | `false` |
#### Important note about `k3s_release_version`
@ -228,7 +234,7 @@ a Production workload.
If multiple hosts have `k3s_control_node` set to true, you must also set
`k3s_datastore_endpoint` as the connection string to a MySQL or PostgreSQL
database, or etcd cluster else the play will fail.
database, or external Etcd cluster else the play will fail.
If using TLS, the CA, Certificate and Key need to already be available on
the play hosts.
@ -241,7 +247,12 @@ configuration you will need to set `k3s_use_unsupported_config` to `true`.
Since K3s v1.19.1 it is possible to use Etcd as the backend database, and this
is done by setting `k3s_etcd_datastore` to true. As this is an experimental
feature you will also need to set `k3s_use_experimental` to true.
feature you will also need to set `k3s_use_experimental` to `true`. The best
practice for Etcd is to define at least 3 members to ensure quorum is
established. In addition to this, an odd number of members is recommended to
ensure a majority in the event of a network partition. If you want to use 2
members or an even number of members, please set `k3s_use_unsupported_config`
to `true`.
#### Important note about `k3s_flannel_interface`

20
defaults/main.yml
View File

@ -16,6 +16,9 @@ k3s_build_cluster: true
# URL for GitHub project
k3s_github_url: https://github.com/rancher/k3s
# Skip all tasks that validate configuration
k3s_skip_validation: false
# Installation directory for k3s
k3s_install_dir: /usr/local/bin
@ -53,9 +56,21 @@ k3s_datastore_endpoint: false
# k3s_datastore_certfile: /path/to/cert.crt
# k3s_datastore_keyfile: /path/to/key.pem
# Use Etcd for HA Datastore? (EXPERIMENTAL)
# Use embedded Etcd for HA Datastore? (EXPERIMENTAL)
k3s_etcd_datastore: false
# Disable Etcd snapshots
k3s_etcd_disable_snapshots: false
# Etcd snapshot cron schedule. Example below is every 12 hours.
# k3s_etcd_snapshot_schedule_cron: "* */12 * * *"
# Etcd snapshot retention
# k3s_etcd_snapshot_retention: 5
# Etcd snapshot directory
# k3s_etcd_snapshot_directory: /server/db/snapshots
# HTTPS Listening port
k3s_https_port: 6443
@ -107,6 +122,9 @@ k3s_default_local_storage_path: false
# Use secret encryption at rest (EXPERIMENTAL)
k3s_secrets_encryption: false
# Enable SELinux in containerd (EXPERIMENTAL)
k3s_enable_selinux: false
# with become privileges for
k3s_become_for_all: false
k3s_become_for_systemd: null

1
molecule/highavailability/playbook-etcd.yml
View File

@ -12,6 +12,5 @@
- name: Set each node to be a control node
set_fact:
k3s_control_node: true
when: inventory_hostname in ['node2', 'node3']
roles:
- role: xanmanning.k3s

18
tasks/build/configure-k3s-cluster.yml
View File

@ -19,6 +19,24 @@
check_mode: false
when: k3s_control_token is not defined and ansible_check_mode
- name: Ensure the cluster NODE_TOKEN file location exists
file:
path: "{{ k3s_token_location }}"
state: directory
mode: 0755
become: "{{ k3s_become_for_systemd | ternary(true, false, k3s_become_for_all) }}"
- name: Ensure k3s cluster token file is present on workers and secondary control nodes
template:
src: cluster-token.j2
dest: "{{ k3s_token_location }}/cluster-token"
mode: 0600
become: "{{ k3s_become_for_systemd | ternary(true, false, k3s_become_for_all) }}"
when: (k3s_control_node and not k3s_primary_control_node)
or not k3s_control_node
notify:
- restart k3s
- name: Ensure k3s service unit file is present
template:
src: k3s.service.j2

2
tasks/state-installed.yml
View File

@ -1,6 +1,7 @@
---
- import_tasks: validate/check-environment.yml
when: not k3s_skip_validation
- import_tasks: build/preconfigure-k3s.yml
@ -12,6 +13,7 @@
or k3s_release_version is not regex('\\+k3s[1-9]$')
- import_tasks: validate/main.yml
when: not k3s_skip_validation
- import_tasks: build/get-systemd-context.yml

1
tasks/state-uninstalled.yml
View File

@ -21,3 +21,4 @@
and (k3s_non_root is not defined or not k3s_non_root)
- import_tasks: validate/check-uninstalled.yml
when: not k3s_skip_validation

1
tasks/validate/check-experimental-variables.yml
View File

@ -10,6 +10,7 @@
or (k3s_dqlite_datastore is defined and k3s_dqlite_datastore)
or (k3s_etcd_datastore is defined and k3s_etcd_datastore)
or (k3s_secrets_encryption is defined and k3s_secrets_encryption)
or (k3s_enable_selinux is defined and k3s_enable_selinux)
- name: Check if experimental dqlite is being used and k3s_use_unsupported_config is configured
assert:

9
tasks/validate/check-master-count.yml
View File

@ -22,3 +22,12 @@
success_msg: "Control plane configuration is valid."
fail_msg: "Control plane configuration is invalid. Please see notes about k3s_control_node and HA in README.md."
when: k3s_controller_count | length >= 2
- name: Check the conditions when embedded etcd is defined
assert:
that:
- (k3s_controller_count | length >= 3)
and (((k3s_controller_count | length) % 2) == 1)
success_msg: "Control plane configuration is valid."
fail_msg: "Etcd should have a minimum of 3 defined members and the number of members should be odd. Please see notes about HA in README.md"
when: k3s_etcd_datastore and not k3s_use_unsupported_config

8
tasks/validate/check-variables.yml
View File

@ -278,3 +278,11 @@
success_msg: "Secrets encryption at rest supported in {{ k3s_release_version }}"
fail_msg: "Secrets encryption at rest is not supported in {{ k3s_release_version }}"
when: k3s_secrets_encryption is defined and k3s_secrets_encryption
- name: Check k3s_enable_selinux against k3s version
assert:
that:
- (k3s_release_version | replace('v', '')) is version_compare('1.17.4', '>=')
success_msg: "SELinux supported in {{ k3s_release_version }}"
fail_msg: "SELinux is not supported in {{ k3s_release_version }}"
when: k3s_enable_selinux is defined and k3s_enable_selinux

1
templates/cluster-token.j2 Normal file
View File

@ -0,0 +1 @@
{{ k3s_control_token }}

23
templates/k3s.service.j2
View File

@ -1,7 +1,8 @@
[Unit]
Description=Lightweight Kubernetes
Documentation=https://k3s.io
After=network.target
Wants=network-online.target
After=network-online.target
[Service]
Type={{ 'notify' if k3s_control_node else 'exec' }}
@ -83,7 +84,20 @@ ExecStart={{ k3s_install_dir }}/k3s
--cluster-init
{% else %}
--server https://{{ k3s_control_node_address }}:{{ k3s_https_port }}
--token {{ k3s_control_token }}
--token-file {{ k3s_token_location }}/cluster-token
{% endif %}
{% if k3s_etcd_disable_snapshots %}
--etcd-disable-snapshots
{% else %}
{% if k3s_etcd_snapshot_schedule_cron is defined %}
--etcd-snapshot-schedule-cron "{{ k3s_etcd_snapshot_schedule_cron }}"
{% endif %}
{% if k3s_etcd_snapshot_retention is defined %}
--etcd-snapshot-retention {{ k3s_etcd_snapshot_retention }}
{% endif %}
{% if k3s_etcd_snapshot_directory is defined %}
--etcd-snapshot-dir {{ k3s_etcd_snapshot_directory }}
{% endif %}
{% endif %}
{% endif %}
{% if k3s_secrets_encryption is defined and k3s_secrets_encryption %}
@ -120,7 +134,10 @@ ExecStart={{ k3s_install_dir }}/k3s
{% else %}
agent
--server https://{{ k3s_control_node_address }}:{{ k3s_https_port }}
--token {{ k3s_control_token }}
--token-file {{ k3s_token_location }}/cluster-token
{% endif %}
{% if k3s_enable_selinux %}
--selinux
{% endif %}
{% if k3s_resolv_conf is defined and k3s_resolv_conf %}
--resolv-conf {{ k3s_resolv_conf }}

1
vars/main.yml
View File

@ -35,3 +35,4 @@ k3s_controller_count: []
k3s_systemd_context: system
k3s_systemd_unit_directory: "/etc/systemd/{{ k3s_systemd_context }}"
k3s_token_location: "/etc/rancher"