Using Operations

Operations tell pyinfra what to do, for example the server.shell operation instructs pyinfra to execute a shell command. Most operations define state rather than actions - so instead of “start this service” you say “this service should be running” - pyinfra will only execute commands if needed.

For example, these operations will ensure that user pyinfra exists with home directory /home/pyinfra, and that the /var/log/pyinfra.log file exists and is owned by that user:

# Import pyinfra modules, each containing operations to use
from pyinfra.operations import server, files

server.user(
    name="Create pyinfra user",
    user="pyinfra",
    home="/home/pyinfra",
)

files.file(
    name="Create pyinfra log file",
    path="/var/log/pyinfra.log",
    user="pyinfra",
    group="pyinfra",
    mode="644",
    _sudo=True,
)

Uses Files Operations and Server Operations. You can see all available operations in the Operations Index. If you save the file as deploy.py you can test it out using Docker:

pyinfra @docker/ubuntu:20.04 deploy.py

Important

Operations that rely on one another (interdependency) must be treated with caution. See: deploy limitations.

Global Arguments

Global arguments are covered in detail here: Global Arguments. There is a set of arguments available to all operations to control authentication (_sudo, etc) and operation execution (_shell_executable, etc):

from pyinfra.operations import apt

apt.update(
    name="Update apt repositories",
    _sudo=True,
    _sudo_user="pyinfra",
)

The host Object

pyinfra provides a global host object that can be used to retrieve information and metadata about the current host target. At all times the host variable represents the current host context, so you can think about the deploy code executing on individual hosts at a time.

The host object has name and groups attributes which can be used to control operation flow:

from pyinfra import host

if host.name == "control-plane-1":
    ...

if "control-plane" in host.groups:
    ...

Host Facts

Facts allow you to use information about the target host to control and configure operations. A good example is switching between apt & yum depending on the Linux distribution. Facts are imported from pyinfra.facts.* and can be collected using host.get_fact(...):

from pyinfra import host
from pyinfra.facts.server import LinuxName
from pyinfra.operations import yum

if host.get_fact(LinuxName) == "CentOS":
    yum.packages(
        name="Install nano via yum",
        packages=["nano"],
        _sudo=True
    )

See Facts Index for a full list of available facts and arguments.

Host & Group Data

Adding data to inventories is covered in detail here: Inventory & Data. Data can be accessed within operations via the host.data attribute:

from pyinfra import host
from pyinfra.operations import server

# Ensure the state of a user based on host/group data
server.user(
    name="Setup the app user",
    user=host.data.app_user,
    home=host.data.app_dir,
)

The inventory Object

Like host, there is an inventory object that can be used to access the entire inventory of hosts. This is useful when you need facts or data from another host like the IP address of another node:

from pyinfra import inventory
from pyinfra.facts.server import Hostname
from pyinfra.operations import files

# Get the other host, load the hostname fact
db_host = inventory.get_host("postgres-main")
db_hostname = db_host.get_fact(Hostname)

files.template(
    name="Generate app config",
    src="templates/app-config.j2.yaml",
    dest="/opt/myapp/config.yaml",
    db_hostname=db_hostname,
)

Operation Changes & Output

All operations return an operation meta object which provides information about the changes the operation will execute. The meta object provides changes (integer) and changed (boolean) attributes initially. This can be used to control subsequent operations:

from pyinfra.operations import server

create_user = server.user(
    name="Create user myuser",
    user="myuser",
)

# If we added a user above, do something extra
if create_user.changed:
    server.shell( # add user to sudo, etc...

Operation Output

pyinfra doesn’t immediately execute normal operations meaning output is not available right away. It is possible to access this output at runtime by providing a callback function using the python.call operation.

from pyinfra import logger
from pyinfra.operations import python, server

result = server.shell(
    commands=["echo output"],
)
# result.stdout raises exception here, but works inside callback()

def callback():
    logger.info(f"Got result: {result.stdout}")

python.call(
    name="Execute callback function",
    function=callback,
)

Nested Operations

Important

Nested operations are currently in beta. Nested operations should be kept to a minimum as they cannot be tracked as changes prior to execution.

Nested operations are called during the execution phase within a callback function passed into a python.call. Calling a nested operation generates and immediately executes it on the target machine. This is useful in complex scenarios where one operation output is required in another.

Because nested operations are executed immediately, the output is always available right away:

from pyinfra import logger
from pyinfra.operations import python, server

def callback():
    result = server.shell(
        commands=["echo output"],
    )

    logger.info(f"Got result: {result.stdout}")

python.call(
    name="Execute callback function",
    function=callback,
)

Include Multiple Files

Including files can be used to break out operations across multiple files. Files can be included using local.include.

from pyinfra import local

# Include & call all the operations in tasks/install_something.py
local.include("tasks/install_something.py")

See more in examples: groups & roles.

The config Object

Like host and inventory, config can be used to set global defaults for operations. For example, to use sudo in all operations following:

from pyinfra import config

config.SUDO = True

# all operations below will use sudo by default (unless overridden by `_sudo=False`)

Enforcing Requirements

The config object can be used to enforce a pyinfra version or Python package requirements. This can either be defined as a requirements text file path or simply a list of requirements:

# Require a certain pyinfra version
config.REQUIRE_PYINFRA_VERSION = "~=1.1"

# Require certain packages
config.REQUIRE_PACKAGES = "requirements.txt"  # path relative to the current working directory
config.REQUIRE_PACKAGES = [
    "pyinfra~=1.1",
    "pyinfra-docker~=1.0",
]

Examples

A great way to learn more about writing pyinfra deploys is to see some in action. There’s a number of resources for this: