how to use variables in a pypyr pipeline permalink

You can use variables to parameterize your pypyr pipelines and pass values between steps. pypyr lets you pass values seamlessly between the pipeline yaml, the cli (or api) and the steps in the pipeline.

pypyr stores variables in the context. The context is a dictionary that stays in scope for the duration of the entire pipeline.

Variables can be simple types like string or int, and they can also contain complex nested structures like dictionaries or lists. So for example, a variable could be a dictionary mapping, which in turn contains lists/arrays, and those lists in turn contain other dicts or lists.

global variables permalink

inject arguments from the cli permalink

You can inject cli arguments into a pipeline by using a context parser in the pipeline.

$ pypyr mypipeline these are all context input arguments
$ pypyr mypipeline var1=value1 var2="value 2"

These values stay in the scope for the duration of the pipeline’s execution, unless you explicitly unset with clear or explicitly overwrite them in a step.

set variable in a step permalink

You can set variables using the following steps:

These variables are available to all subsequent steps.

The difference between default and contextsetf is that default only sets a variable if it does not exist already. It will not overwrite existing variables. This makes default especially handy if you want to provide default values for optional arguments coming from the cli.

By comparison, contextsetf will always set the variable, overwriting a variable even if it exists already.

steps:
  - name: pypyr.steps.contextsetf
    comment: set some arbitrary variables
    in:
      contextSetf:
        var1: my value
        var2: 2
        var3: True

  - name: pypyr.steps.echo
    comment: use the variables in subsequent steps
    run: '{var3}'
    in:
      echoMe: |
              var1 is '{var1}'
              var2 is {var2}              

environment variables permalink

You can set a pypyr variable from an environment variable.

steps:
- name: pypyr.steps.envget
  comment: get env $MACAVITY
           assign it to pypyr variable theHiddenPaw
           if env $MACAVITY doesn't exist, use the default value
  in:
    envGet:
      env: MACAVITY
      key: theHiddenPaw
      default: but macavity wasn't there!

- name: pypyr.steps.echo
  comment: theHiddenPaw was set from env $MACAVITY
  in:
    echoMe: the master criminal {theHiddenPaw}

load variables from a file permalink

You can load & initialize variables from a json or yaml file.

If you want to set your config file path as a cli argument, you can use the following context parsers:

$ pypyr mypipeline ./myvars.json
$ pypyr mypipeline ./myvars.yaml

You can also use a step explicitly to load your variables from a file at a specific point in the pipeline’s execution:

local variables permalink

You can set a variable that is only in scope for the current step using the in decorator.

Unlike setting a “global” variable, arguments that you pass to a step using in are not available to subsequent steps.

steps:
  - name: mystep
    comment: variables A, B and C only in scope for this step.
    in:
      A: value 1
      B: 2
      C: true
  - name: pypyr.steps.echo
    in:
      echoMe: done! A, B & C are NOT available in this step.

unset variables permalink

You can unset or delete variables using the following steps:

read variables permalink

You can read, format and combine static values & variables using substitutions.

In this example, we set 2 local variables A and B, and then use a substitution expression to pass these to the echo step echoMe input argument:

- name: pypyr.steps.echo
  comment: concat static string and 2 vars.
           will output "piping down the valleys wild".
  in:
    A: down the
    B: valleys
    echoMe: piping {A} {B} wild

Instead of assigning A and B here in the step’s in, you could also set A and B in any previous step as “global” variables using contextsetf or default:

steps:
  - name: pypyr.steps.contextsetf
    in:
      contextSetf:
        A: down the
        B: valleys

  - name: pypyr.steps.echo
    comment: concat static string and 2 vars.
             will output "piping down the valleys wild".
    in:
      echoMe: piping {A} {B} wild       

see also

last updated on .