Salt Super-Proxy

Salt plugin to automate the management
and configuration of network devices at scale, without running (Proxy) Minions.

Using salt-sproxy, you can continue to benefit from the scalability,
flexibility and extensibility of Salt, while you don't have to manage thousands
of (Proxy) Minion services. However, you are able to use both salt-sproxy
and your (Proxy) Minions at the same time.

Why `salt-sproxy`

salt-sproxy can be used as a standalone tool to manage your devices without
having any further requirements, as well as an extension to your existing Salt
environment (if you already have). In other words, if you have a Salt
installation where you manage some network devices and servers, installing
salt-sproxy on your Master will allow you to run any Salt command as always,
e.g., executing salt \* test.ping and salt-sproxy \* test.ping will have
the exact same effect, and result. On top of that, using salt-sproxy allows
you to manage other devices for which you don't run (Proxy) Minions for.

Of course, if you don't already have Salt, no problem, you can start managing
your devices straight away, check out the quick
start steps.

In brief, here are some benefits you can get by using salt-sproxy:

Say goodbye to the burden of managing hundreds of system services for the
Proxy Minion processes.
Reuse your existing extension modules, templates, Pillars, States, etc., you
may have already developed in your Salt environment, transparently.
You can run it locally, on your own computer.
You can use salt-sproxy to uniformly manage network devices, servers
(either using regular Minions, or SSH), applications
(e.g., Docker containers, VMWare ESXi clusters and vCenters, Marathon or
Chronos clusters, etc.), and virtually anything that has a programmable
interface.
Python programming made a breeze - might go well with the
ISalt package.
Integrates easily with your existing Salt environment (if you have), by
installing the package on your Salt Master.
Can continue to leverage the event-driven automation and orchestration
methodologies.
Can continue using any of the usual targeting mechanisms.
REST API, see also
the Salt REST API
documentation.
By sending events to a Salt Master, you are able to implement whatever
auditing you need (e.g., what command was executed by who and when, etc.).
Benefit from inheriting all the native Salt features and integrations
contributed by thousands of users, and tested in hundreds of different
environments, over almost a decade of development.

Is `salt-sproxy` a wrapper around `salt-ssh`?

No, nothing to do with salt-ssh. The core of salt-sproxy is a Runner loaded
dynamically on runtime, that spins up a pool of child processes, each running
a temporary light version of the Proxy Minion underneath; as soon as the
execution is complete for a device, its associated Proxy Minion is shut down,
and another one takes its place into the child processes bucket.

A source of confusion may also be the usage of the Roster interface, which,
historically has only been used by salt-ssh, although the Roster is not
tightly coupled with salt-ssh: it just happened to be the only use case so
far. Essentially, the Roster simply provides a list of devices together with
their credentials (e.g., similar to the inventory as dubbed in other
automation frameworks) - and now has another use case in salt-sproxy.

Prerequisites

The package is distributed via PyPI, under the name salt-sproxy. If you
would like to install it on your computer, you might want to run it under a
virtual environment.

Besides the CLI, the usage remains the same as when you're running a Salt
environment with Proxy or regular Minions. See the following documents on how
to get started and fully unleash the power of Salt:

Install

Install this package where you would like to manage your devices from. In case
you need a specific Salt version, make sure you install it beforehand,
otherwise this package will bring the latest Salt version available instead.

Execute:

pip install salt-sproxy

To install a specific Salt version, execute, e.g.,

pip install salt==2018.3.4
pip install salt-sproxy

See https://salt-sproxy.readthedocs.io/en/latest/install.html
for more installation details.

Documentation

The complete documentation is available at
https://salt-sproxy.readthedocs.io/en/latest/.

On Unix distributions you can also check the documentation locally, by
executing man salt-sproxy.

Quick Start

See this recording for a live quick start:

In the above, minion1 is
a dummy
Proxy Minion, that can be used for getting started and make the first steps
without connecting to an actual device, but get used to the salt-sproxy
methodology.

The Master configuration file is /home/mircea/master, which is why the
command is executed using the -c option specifying the path to the directory
with the configuration file. In this Master configuration file, the
pillar_roots option points to /srv/salt/pillar which is where
salt-sproxy is going to load the Pillar data from. Accordingly, the Pillar
Top file is under that path, /srv/salt/pillar/top.sls:

base:
  minion1:
    - dummy

This Pillar Top file says that the Minion minion1 will have the Pillar data
from the dummy.sls from the same directory, thus
/srv/salt/pillar/dummy.sls:

proxy:
  proxytype: dummy

In this case, it was sufficient to only set the proxytype field to
dummy.

salt-sproxy can be used in conjunction with any of the available Salt
Proxy modules,
or others that you might have in your own environment. See
https://docs.saltstack.com/en/latest/topics/proxyminion/index.html
to understand how to write a new Proxy module if you require.

For example, let's take a look at how we can manage a network device through
the NAPALM Proxy:

In the same Python virtual environment as previously, make sure you have
NAPALM installed, by executing pip install napalm (see
https://napalm.readthedocs.io/en/latest/installation/index.html
for further installation requirements, depending on the platform you're running on). The
connection credentials for the juniper-router are stored in the
/srv/salt/pillar/junos.sls Pillar, and we can go ahead and start executing
arbitrary Salt commands, e.g.,
net.arp
to retrieve the ARP table, or
net.load_config
to apply a configuration change on the router.

The Pillar Top file in this example was (under the same path as previously, as
the Master config was the same):

base:
  juniper-router:
    - junos

Usage

First off, make sure you have the Salt
Pillar Top file is
correctly defined and the proxy key is available into the Pillar. For more
in-depth explanation and examples, check
this
tutorial from the official SaltStack docs.

Once you have that, you can start using salt-sproxy even without any Proxy
Minions or Salt Master running. To check, can start by executing:

$ salt-sproxy -L a,b,c --preview-target
- a
- b
- c

The syntax is very similar to the widely used CLI command salt, however the
way it works is completely different under the hood:

salt-sproxy <target> <function> [<arguments>]

Usage Example:

$ salt-sproxy cr1.thn.lon test.ping
cr1.thn.lon:
    True

You can continue reading further details at
https://salt-sproxy.readthedocs.io/en/latest/,
for now, check out the following section to see how to get started with
salt-sproxy straight away.

See also https://salt-sproxy.readthedocs.io/en/latest/examples/index.html
for more usage examples.

Event-Driven Automation and Orchestration

It is still possible to use the salt-sproxy functionality in the event-driven
context, even without running Proxy or regular Minions. To see how to achieve
this, see this section of the documentation:
https://salt-sproxy.readthedocs.io/en/latest/events.html.

Using the Salt REST API

Salt has natively available an HTTP API. You can read more at
https://docs.saltstack.com/en/latest/ref/netapi/all/salt.netapi.rest_cherrypy.html#a-rest-api-for-salt
if you haven't used it before. The usage is very simple; for salt-sproxy
specifically you can follow the notes from https://salt-sproxy.readthedocs.io/en/latest/salt_sapi.html how to set it up and
use. Usage example - apply a small configuration change on a Juniper device, by
executing an HTTP request via the Salt API:

$ curl -sS localhost:8080/run -H 'Accept: application/x-yaml' \
  -d eauth='pam' \
  -d username='mircea' \
  -d password='pass' \
  -d client='sproxy' \
  -d tgt='juniper-router' \
  -d fun='net.load_config' \
  -d text='set system ntp server 10.10.10.1'
return:
- juniper-router:
    already_configured: false
    comment: ''
    diff: '[edit system]
      +   ntp {
      +       server 10.10.10.1;
      +   }'
    loaded_config: ''
    result: true

See the documentation
for explanation, and this example
for a quick start.

What's included

When installing salt-sproxy, besides the core files (i.e., cli.py,
parsers.py, scripts.py, and version.py), you will find the
following directories and files, which provide additional features and
backwards compatibility with older Salt versions:

  |-- cli.py
  |-- parsers.py
  |-- _roster/
  |   |-- ansible.py
  |   |-- file.py
  |   |-- netbox.py
  |   `-- pillar.py
  |-- _runners/
  |   `-- proxy.py
  |-- scripts.py
  `-- version.py

The extension modules under the _roster and _runner directories are
documented at https://salt-sproxy.readthedocs.io/en/latest/roster/index.html
and https://salt-sproxy.readthedocs.io/en/latest/runners/index.html,
respectively.

Docker

A Docker image is available at
https://github.com/mirceaulinic/salt-sproxy/pkgs/container/salt-sproxy,
and you can pull it, e.g., docker pull ghcr.io/mirceaulinic/salt-sproxy:develop. See
https://salt-sproxy.readthedocs.io/en/latest/#docker
for further usage instructions and examples.

Community

Get updates on the salt-sproxy development, and chat with the project
maintainer(s) and community members:

Follow @mirceaulinic
Google Groups
Use the salt-sproxy tag on Stack Overflow.
The #salt-sproxy IRC channel on freenode.net.
The #saltstack channel under the
networktocode Slack.

License

This project is licensed under the Apache 2.0 License - see the
LICENSE
file for details.

Acknowledgments

Thanks to Daniel Wallace for the
inspiration.

Use Case

Salt is an agent-based software that automates the management and configuration of infrastructure and application at scale. It requires each Salt-managed node to have a Salt Minion service. Usually, this is not a blocker on a server, but in the networking world, it is not possible to install custom software on the network gear that you want to manage. To solve this problem, https://www.saltstack.com/ introduced the Proxy Minion, a derivative of the regular Minion that does not need to be installed on the targeted device and can run anywhere.

As with regular Minions, you will need to manage the same number of Proxy Minion services as network devices you have. But this comes with a considerable cost of infrastructure management, operations, and time. There are also many cases in the networking world where administrators only need to interact with the device once or twice a year, and those interactions must be consistent and safe, and constantly keeping a service up and running during this time is not an ideal solution. Automation is the only way to manage these scenarios.

salt-sproxy enables you to manage your infrastructure using Salt, without the need to manage thousands of services. With this approach, you also have access to the Salt REST API, can leverage the event-driven automation, and more, as well as inheriting the usual flexibility and extensibility of Salt.

Business Summary

Using salt-sproxy, you can continue to benefit from the scalability, flexibility and extensibility of Salt and Proxy Minions. The plugin enables you to do this without the need to individually manage thousands of Proxy Minion services.