Break out packages + substitutions docs (#4095)

Co-authored-by: Jesse Hills <3060199+jesserockz@users.noreply.github.com>
Co-authored-by: H. Árkosi Róbert <robreg@zsurob.hu>
This commit is contained in:
Keith Burzinski 2024-08-05 23:43:32 -05:00 committed by GitHub
parent 8620a20f83
commit 6c4e581ec3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 381 additions and 420 deletions

View File

@ -134,7 +134,7 @@ Other notable changes:
(:ref:`wifi-networks`)
- GPIO Switches have a new option ``restore_mode`` to configure how their values should be restored on boot.
(:doc:`/components/switch/gpio`)
- Added :ref:`substitutions <config-substitutions>` to reduce repeating across configs.
- Added :doc:`/components/substitutions/index` to reduce repeating across configs.
- Validation error messages are now displayed even better. Now all errors are shown with the exact context
where the error appeared. Try it, it's so much better. Next step will be to upgrade to a better YAML reader
to provide better error messages when the YAML syntax is invalid.

View File

@ -34,4 +34,6 @@ Components
text/index
update/index
valve/index
packages/index
substitutions/index
*

View File

@ -0,0 +1,203 @@
Packages
========
.. seo::
:description: How to use packages in ESPHome
:image: settings.svg
When you have many ESPHome devices (or are producing and distributing them at scale), a common need tends to surface:
configuration modularization. You'll likely want to break your configuration into common (groups of) elements, building
it into reusable pieces which can subsequently be used by many/all devices. Only unique pieces of your configuration
remain in any given device's YAML configuration file.
This can be accomplished with ESPHome's ``packages`` feature.
All definitions from packages will be merged with your device's main configuration in a non-destructive way. This
allows overriding (parts of) configuration contained in the package(s). Substitutions in your main configuration will
override substitutions with the same name in a package.
Dictionaries are merged key-by-key. Lists of components are merged by component ID (if specified). Other lists are
merged by concatenation. All other configuration values are replaced with the later value.
ESPHome uses ``!include`` to "bring in" packages; this is a syntax brought over from
`Home Assistant's YAML configuration directives <https://www.home-assistant.io/docs/configuration/splitting_configuration/>`__.
Local Packages
--------------
Consider the following example where the author put common pieces of configuration (like Wi-Fi and API) into base files
and then extends it with some device-specific configuration in the main configuration.
Note how the piece of configuration describing ``api`` component in ``device_base.yaml`` gets merged with the services
definitions from main configuration file.
.. code-block:: yaml
# In config.yaml
packages:
wifi: !include common/wifi.yaml
device_base: !include common/device_base.yaml
api:
services:
- service: start_laundry
then:
- switch.turn_on: relay
# any additional configuration...
.. code-block:: yaml
# In wifi.yaml
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
.. code-block:: yaml
# In device_base.yaml
esphome:
name: ${node_name}
esp32:
board: wemos_d1_mini32
logger:
api:
encryption:
key: !secret api_encryption_key
.. _config-git_packages:
Remote/Git Packages
-------------------
Packages can also be loaded from a Git repository by utilizing the correct configuration syntax.
:doc:`/components/substitutions/index` can be used inside the remote packages which allows users to override
them locally with their own substitution value.
.. note::
Remote packages cannot have ``secret`` lookups in them. They should instead make use of substitutions with an
optional default in the packaged YAML, which the local device YAML can set using values from the local secrets.
.. code-block:: yaml
# Git repo examples
packages:
# shorthand form github://username/repository/[folder/]file-path.yml[@branch-or-tag]
remote_package_shorthand: github://esphome/non-existant-repo/file1.yml@main
remote_package_files:
url: https://github.com/esphome/non-existant-repo
files: [file1.yml, file2.yml] # optional; if not specified, all files will be included
ref: main # optional
refresh: 1d # optional
Configuration variables:
------------------------
For each package:
- **url** (**Required**, string): The URL for the repository.
- **username** (*Optional*, string): Username to be used for authentication, if required.
- **password** (*Optional*, string): Password to be used for authentication, if required.
- **files** (**Required**, list of strings): List of files to include.
- **ref** (*Optional*, string): The Git ref(erence) to be used when pulling content from the repository.
- **refresh** (*Optional*, :ref:`config-time`): The interval at which the content from the repository should be refreshed.
Packages as Templates
---------------------
Since packages are incorporated using the ``!include`` system, variables can be provided to them. This means that
packages can be used as *templates*, allowing complex or repetitive configurations to be stored in a package file
and then incorporated into the configuration more than once.
Packages may also contain a ``defaults`` block which provides subsitutions for variables not provided by the
``!include`` block.
As an example, if the configuration needed to support three garage doors using the ``gpio`` switch platform and the
``time_based`` cover platform, it could be constructed like this:
.. code-block:: yaml
# In config.yaml
packages:
left_garage_door: !include
file: garage-door.yaml
vars:
door_name: Left
vars:
door_name: Middle
vars:
door_name: Right
.. code-block:: yaml
# In garage-door.yaml
switch:
- name: ${door_name} Garage Door Switch
platform: gpio
# ...
Extend
------
To make changes or add additional configuration to included configurations, ``!extend config_id`` can be used, where
``config_id`` is the ID of the configuration to modify.
For example, to set a specific update interval on a common uptime sensor that is shared between configurations:
.. code-block:: yaml
# In common.yaml
captive_portal:
sensor:
- platform: uptime
id: uptime_sensor
update_interval: 1min
.. code-block:: yaml
packages:
common: !include common.yaml
sensor:
- id: !extend uptime_sensor
update_interval: 10s
Remove
------
To remove existing entries from included configurations, ``!remove [config_id]`` can be used, where ``config_id`` is
the ID of the entry to modify.
For example, to remove a common uptime sensor that is shared between configurations:
.. code-block:: yaml
packages:
common: !include common.yaml # see above
sensor:
- id: !remove uptime_sensor
To remove captive portal for a specific device:
.. code-block:: yaml
packages:
common: !include common.yaml # see above
captive_portal: !remove
See Also
--------
- :doc:`ESPHome index </index>`
- :doc:`/guides/getting_started_command_line`
- :doc:`/guides/faq`
- :ghedit:`Edit`

View File

@ -0,0 +1,162 @@
Substitutions
=============
.. seo::
:description: How to use substitutions in ESPHome
:image: settings.svg
ESPHome has a powerful way to reduce repetition in configuration files: substitutions.
With substitutions, you can have a single generic source file for all nodes of one kind and
substitute expressions in as required.
.. code-block:: yaml
substitutions:
bme280_temperature_offset: "-1.0"
sensor:
- platform: bme280_i2c
temperature:
name: BME280 Temperature
filters:
- offset: ${bme280_temperature_offset}
In the top-level ``substitutions`` section, you can put as many key-value pairs as you want. Before
validating your configuration, ESPHome will automatically replace all occurrences of substitutions
by their value. The syntax for a substitution is based on bash and is case-sensitive: ``$substitution_key`` or
``${substitution_key}`` (same).
Two substitution passes are performed allowing compound replacements.
.. code-block:: yaml
substitutions:
foo: yellow
bar_yellow_value: !secret yellow_secret
bar_green_value: !secret green_secret
something:
test: ${bar_${foo}_value}
.. _substitute-include-variables:
Substitute !include variables
-----------------------------
ESPHome's ``!include`` accepts a list of variables that can be substituted within the included file.
.. code-block:: yaml
binary_sensor:
- platform: gpio
id: button1
pin: GPIOXX
on_multi_click: !include { file: on-multi-click.yaml, vars: { id: 1 } } # inline syntax
- platform: gpio
id: button2
pin: GPIOXX
on_multi_click: !include
# multi-line syntax
file: on-multi-click.yaml
vars:
id: 2
``on-multi-click.yaml``:
.. code-block:: yaml
- timing: !include click-single.yaml
then:
- mqtt.publish:
topic: ${device_name}/button${id}/status
payload: single
- timing: !include click-double.yaml
then:
- mqtt.publish:
topic: ${device_name}/button${id}/status
payload: double
.. _command-line-substitutions:
Command line substitutions
--------------------------
You can define or override substitutions from the command line by adding the ``-s`` switch with arguments ``KEY`` and
``VALUE``. This will override the substitution ``KEY`` and assign it the value ``VALUE``. This switch can be included
multiple times. Consider the following ``example.yaml`` file:
.. code-block:: yaml
substitutions:
name: my_default_name
esphome:
name: $name
...and the following command:
.. code-block:: bash
esphome -s name my_device01 config example.yaml
You will get something like the following output:
.. code-block:: yaml
substitutions:
name: my_device01
esphome:
name: my_device01
# ...
Command line substitutions take precedence over those in your configuration file. This can be used to create generic
"template" configuration files (like ``example.yaml`` above) which can be used by multiple devices, leveraging
substitutions which are provided on the command line.
.. _YAML-insertion-operator:
Bonus: YAML insertion operator
------------------------------
Additionally, you can use the YAML insertion operator ``<<`` syntax to create a single YAML file from which a number
of nodes inherit:
.. code-block:: yaml
# In common.yaml
esphome:
name: $devicename
# ...
sensor:
- platform: dht
# ...
temperature:
name: Temperature
humidity:
name: Humidity
.. code-block:: yaml
# In nodemcu1.yaml
substitutions:
devicename: nodemcu1
<<: !include common.yaml
.. tip::
To hide these base files from the dashboard, you can
- Place them in a subdirectory (dashboard only shows files in top-level directory)
- Prepend a dot to the filename, like ``.base.yaml``
See Also
--------
- :doc:`ESPHome index </index>`
- :doc:`/guides/getting_started_command_line`
- :doc:`/guides/faq`
- :ghedit:`Edit`

View File

@ -163,423 +163,6 @@ There are several ways of doing this. See below examples to see how you can spec
update_interval: never # never update
update_interval: 0ms # update in every loop() iteration
.. _config-substitutions:
Substitutions
-------------
ESPHome has a powerful new way to reduce repetition in configuration files: Substitutions.
With substitutions, you can have a single generic source file for all nodes of one kind and
substitute expressions in.
.. code-block:: yaml
substitutions:
devicename: livingroom
upper_devicename: Livingroom
esphome:
name: $devicename
# ...
sensor:
- platform: dht
# ...
temperature:
name: ${upper_devicename} Temperature
humidity:
name: ${upper_devicename} Humidity
In the top-level ``substitutions`` section, you can put as many key-value pairs as you want. Before
validating your configuration, ESPHome will automatically replace all occurrences of substitutions
by their value. The syntax for a substitution is based on bash and is case-sensitive: ``$substitution_key`` or
``${substitution_key}`` (same).
Two substitution passes are performed allowing compound replacements.
.. code-block:: yaml
substitutions:
foo: yellow
bar_yellow_value: !secret yellow_secret
bar_green_value: !secret green_secret
something:
test: ${bar_${foo}_value}
.. _YAML-insertion-operator:
YAML insertion operator
***********************
Additionally, you can use the YAML insertion operator ``<<`` syntax to create a single YAML file from which a number
of nodes inherit:
.. code-block:: yaml
# In common.yaml
esphome:
name: $devicename
# ...
sensor:
- platform: dht
# ...
temperature:
name: ${upper_devicename} Temperature
humidity:
name: ${upper_devicename} Humidity
.. code-block:: yaml
# In nodemcu1.yaml
substitutions:
devicename: nodemcu1
upper_devicename: NodeMCU 1
<<: !include common.yaml
.. tip::
To hide these base files from the dashboard, you can
- Place them in a subdirectory (dashboard only shows files in top-level directory)
- Prepend a dot to the filename, like ``.base.yaml``
.. _substitute-include-variables:
Substitute !include variables
*****************************
ESPHome's ``!include`` accepts a list of variables that can be substituted within the included file.
.. code-block:: yaml
binary_sensor:
- platform: gpio
id: button1
pin: GPIOXX
on_multi_click: !include { file: on-multi-click.yaml, vars: { id: 1 } } # inline syntax
- platform: gpio
id: button2
pin: GPIOXX
on_multi_click: !include
# multi-line syntax
file: on-multi-click.yaml
vars:
id: 2
``on-multi-click.yaml``:
.. code-block:: yaml
- timing: !include click-single.yaml
then:
- mqtt.publish:
topic: ${device_name}/button${id}/status
payload: single
- timing: !include click-double.yaml
then:
- mqtt.publish:
topic: ${device_name}/button${id}/status
payload: double
.. _command-line-substitutions:
Command line substitutions
**************************
You can define or override substitutions from the command line by adding e.g. ``-s KEY VALUE``
which overrides substitution KEY and gives it value VALUE. This can be issued multiple times,
so e.g. with the following ``example.yaml`` file:
.. code-block:: yaml
substitutions:
name: default
platform: ESP8266
esphome:
name: $name
platform: $platform
board: $board
and the following command:
.. code-block:: bash
esphome -s name device01 -s board esp01_1m example.yaml config
You will get something like the following output (please note the unchanged ``platform``,
added ``board``, and overridden ``name`` substitutions):
.. code-block:: yaml
substitutions:
name: device01
platform: ESP8266
board: esp01_1m
esphome:
name: device01
platform: ESP8266
board: esp01_1m
includes: []
libraries: []
esp8266_restore_from_flash: false
build_path: device01
platformio_options: {}
arduino_version: espressif8266@2.2.3
We can observe here that command line substitutions take precedence over the ones in
your configuration file. This can be used to create generic 'template' configuration
files (like the ``example.yaml`` above) which can be used for multiple devices,
using substitutions which are provided on the command line.
.. _config-packages:
Packages
--------
Another way to modularize and reuse your configuration is to use packages. This feature allows
you to put common pieces of configuration in separate files and keep only unique pieces of your
config in the main yaml file. All definitions from packages will be merged with your main
config in non-destructive way so you could always override some bits and pieces of package
configuration. Substitutions in your main config will override substitutions with the same
name in a package.
Dictionaries are merged key-by-key. Lists of components are merged by component
ID if specified. Other lists are merged by concatenation. All other config
values are replaced with the later value.
Local packages
**************
Consider the following example where the author put common pieces of configuration like WiFi and
I²C into base files and extends it with some device specific configurations in the main config.
Note how the piece of configuration describing ``api`` component in ``device_base.yaml`` gets
merged with the services definitions from main config file.
.. code-block:: yaml
# In config.yaml
substitutions:
node_name: mydevice
device_verbose_name: "My Device"
packages:
wifi: !include common/wifi.yaml
device_base: !include common/device_base.yaml
api:
services:
- service: start_laundry
then:
- switch.turn_on: relay
- delay: 3h
- switch.turn_off: relay
sensor:
- platform: mhz19
co2:
name: "CO2"
temperature:
name: "Temperature"
update_interval: 60s
automatic_baseline_calibration: false
.. code-block:: yaml
# In wifi.yaml
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
domain: .yourdomain.lan
fast_connect: true
.. code-block:: yaml
# In device_base.yaml
esphome:
name: ${node_name}
platform: ESP32
board: wemos_d1_mini32
build_path: ./build/${node_name}
# I²C Bus
i2c:
sda: GPIOXX
scl: GPIOXX
scan: true
frequency: 100kHz
# Enable logging
logger:
level: ${log_level}
api:
encryption:
key: !secret api_encryption_key
reboot_timeout: 1h
sensor:
- <<: !include common/sensor/uptime.config.yaml
- <<: !include common/sensor/wifi_signal.config.yaml
binary_sensor:
- <<: !include common/binary_sensor/connection_status.config.yaml
switch:
- <<: !include common/switch/restart_switch.config.yaml
.. _config-git_packages:
Remote/git Packages
*******************
Packages can also be loaded from a git repository by utilizing the correct config syntax.
:ref:`config-substitutions` can be used inside the remote packages which allows users to override
them locally with their own subsitution value.
.. note::
Remote packages cannot have ``secret`` lookups in them. They should instead make use of substitutions with an
optional default in the packaged YAML, which the local device YAML can set using values from the local secrets.
.. code-block:: yaml
packages:
# Git repo examples
remote_package:
url: https://github.com/esphome/non-existant-repo
ref: main # optional
files: [file1.yml, file2.yml]
refresh: 1d # optional
# A single file can be expressed using `file` or `files` as a string
remote_package_two:
url: https://github.com/esphome/non-existant-repo
file: file1.yml # cannot be combined with `files`
# files: file1.yml
# shorthand form github://username/repository/[folder/]file-path.yml[@branch-or-tag]
remote_package_three: github://esphome/non-existant-repo/file1.yml@main
Packages as Templates
*********************
Since packages are incorporated using the ``!include`` system,
variables can be provided to them. This means that packages can be
used as `templates`, allowing complex or repetitive configurations to
be stored in a package file and then incorporated into the
configuration more than once.
Additionally packages could contain a ``defaults`` block which provides
subsitutions for variables not provided by the ``!include`` block.
As an example, if the configuration needed to support three garage
doors using the ``gpio`` switch platform and the ``time_based`` cover
platform, it could be constructed like this:
.. code-block:: yaml
# In config.yaml
packages:
left_garage_door: !include
file: garage-door.yaml
vars:
door_name: Left
door_location: left
open_switch_gpio: 25
close_switch_gpio: 26
middle_garage_door: !include
file: garage-door.yaml
vars:
door_name: Middle
door_location: middle
open_switch_gpio: 27
close_switch_gpio: 29
right_garage_door: !include
file: garage-door.yaml
vars:
door_name: Right
door_location: right
open_switch_gpio: 15
close_switch_gpio: 18
open_duration: "1min"
close_duration: "50s"
.. code-block:: yaml
# In garage-door.yaml
defaults:
open_duration: "2.1min"
close_duration: "2min"
switch:
- id: open_${door_location}_door_switch
name: ${door_name} Garage Door Open Switch
platform: gpio
pin: ${open_switch_gpio}
- id: close_${door_location}_door_switch
name: ${door_name} Garage Door Close Switch
platform: gpio
pin: ${close_switch_gpio}
cover:
- platform: time_based
name: ${door_name} Garage Door
open_action:
- switch.turn_on: open_${door_location}_door_switch
open_duration: ${open_duration}
close_action:
- switch.turn_on: close_${door_location}_door_switch
close_duration: ${close_duration}
stop_action:
- switch.turn_off: open_${door_location}_door_switch
- switch.turn_off: close_${door_location}_door_switch
Extend
------
To make changes or add additional configuration to included configurations ``!extend config_id`` can be used, where ``config_id`` is the ID of the configuration to modify.
For example to set a specific update interval on a common uptime sensor that is shared between configurations:
.. code-block:: yaml
packages:
common: !include common.yaml
sensor:
- id: !extend uptime_sensor
update_interval: 10s
Remove
------
To remove existing entries from included configurations ``!remove [config_id]`` can be used, where ``config_id`` is the ID of the entry to modify.
For example to remove a common uptime sensor that is shared between configurations:
.. code-block:: yaml
packages:
common: !include common.yaml
sensor:
- id: !remove uptime_sensor
To remove captive portal for a specific device:
.. code-block:: yaml
packages:
common: !include common.yaml
captive_portal: !remove
See Also
--------

View File

@ -48,7 +48,7 @@ Tips for using ESPHome
payload: double
For even more configuration templating, take a look at :ref:`config-substitutions`.
For even more configuration templating, take a look at :doc:`/components/substitutions/index`.
2. If you want to see how ESPHome interprets your configuration, run
@ -66,7 +66,7 @@ Tips for using ESPHome
5. You can view the full list of command line interface options here: :doc:`/guides/cli`
6. Use :ref:`substitutions <config-substitutions>` to reduce repetition in your configuration files.
6. Use :doc:`/components/substitutions/index` to reduce repetition in your configuration files.
.. |secret| replace:: ``!secret``
.. _secret: https://www.home-assistant.io/docs/configuration/secrets/

View File

@ -174,6 +174,17 @@ ESPHome-specific components or components supporting ESPHome device provisioning
Interval, components/interval, description.svg, dark-invert
Script, components/script, description.svg, dark-invert
ESPHome Configuration
---------------------
Streamline your ESPHome configuration and/or use components provided by other contributors.
.. imgtable::
External Components, components/external_components, external_components.svg, dark-invert
Packages, components/packages/index, description.svg, dark-invert
Substitutions, components/substitutions/index, description.svg, dark-invert
Network Hardware
----------------