conditional logic

selectively run or skip step

You can control the flow of execution in your pipeline by selectively running or skipping a step based upon whether a conditional statement evaluates to True.

You use the run or skip decorators on any step to set your condition whether to execute the step.

By default, unless you explicitly tell pypyr differently, every step will run.

# getting-started/basic-conditional.yaml
steps:
  - name: pypyr.steps.echo
    in:
      echoMe: begin

  - name: pypyr.steps.cmd
    run: False
    in:
      cmd: echo this will not run

  - name: pypyr.steps.cmd
    skip: True
    in:
      cmd: echo this will not run

  - name: pypyr.steps.echo
    in:
      echoMe: end

When you run this pipeline, the 2nd & 3rd steps will NOT execute:

$ pypyr getting-started/basic-conditional
begin
end

skip is the inverse of run.

If instead you want to switch between different branches like an IF-THEN-ELSE check out switch branching.

use variables to control if step runs

You can parameterize your conditional statements with variables. You can set variables in preceding steps, pass variables from the cli, use environment variables or even load your variables from a separate configuration file using something like fetchjson or fetchyaml.

set conditional logic in pipeline

You can set the variables that control if a step runs or not in the pipeline itself in a preceding step.

# ./getting-started/conditional-variable-in-pipeline
steps:
  - name: pypyr.steps.echo
    in:
      echoMe: begin

  - name: pypyr.steps.set
    comment: set variables for use in run/skip later
    in:
      set:
        arbVar1: False
        arbVar2: arbitrary str
        # arbVar3 will eval True
        arbVar3: !py arbVar2 == 'arbitrary str'

  - name: pypyr.steps.cmd
    run: '{arbVar1}'
    in:
      cmd: echo this will not run

  - name: pypyr.steps.cmd
    skip: '{arbVar3}'
    in:
      cmd: echo this will not run

  - name: pypyr.steps.echo
    in:
      echoMe: end

When you run this pipeline, the 2nd & 3rd steps will NOT execute:

$ pypyr getting-started/conditional-variable-in-pipeline
begin
end

use cli arguments to run step conditionally

pypyr allows you to pass your own arguments from the cli to the pipeline using a context parser.

You can use these variables directly in a step condition to control if the step should run or not.

In this example, we will use the keys context parser because a frequent pattern to create a natural syntax for cli applications is to pass verbs as arguments to execute select functionality.

# ./ci.yaml
context_parser: pypyr.parser.keys
steps:
  - name: pypyr.steps.default
    comment: set default values for optional cli inputs
    in:
      defaults:
        lint: False
        build: False

  - name: pypyr.steps.echo
    in:
      echoMe: begin - always runs

  - name: pypyr.steps.cmd
    run: '{lint}'
    in:
      cmd: echo this will only run if you pass lint from cli

  - name: pypyr.steps.cmd
    run: '{build}'
    in:
      cmd: echo this will only run if you pass build from cli

  - name: pypyr.steps.echo
    in:
      echoMe: end - always runs

You can control which steps execute by passing the appropriate verbs as arguments from the cli. You can also combine multiple steps by passing more than one verb.

$ pypyr ci
begin - always runs
end - always runs

$ pypyr ci lint
begin - always runs
this will only run if you pass lint from cli
end - always runs

$ pypyr ci build
begin - always runs
this will only run if you pass build from cli
end - always runs

$ pypyr ci lint build
begin - always runs
this will only run if you pass lint from cli
this will only run if you pass build from cli
end - always runs

use python expressions for control flow

You can use !py strings to use any valid Python expression to control if a step executes or not.

# ./getting-started/python-expression-conditional
steps:
  - name: pypyr.steps.echo
    in:
      echoMe: begin

  - name: pypyr.steps.set
    comment: set arbitrary variables to use in later step run/skip conditions
    in:
      set:
        breakfasts:
          - fish
          - bacon
          - spam

        numbersList:
          - 3
          - 4

  - name: pypyr.steps.cmd
    run: !py "'eggs' in breakfasts"
    in:
      cmd: echo this will not run

  - name: pypyr.steps.cmd
    skip: !py sum(numbersList) < 42
    in:
      cmd: echo this will not run

  - name: pypyr.steps.echo
    in:
      echoMe: end

Running this pipeline results in:

$ pypyr getting-started/python-expression-conditional
begin
end

multiple steps with the same condition

If you want to control multiple steps’ conditional execution with the same conditional statement, you can either just duplicate the same conditional statement on each step, or you can group your steps together in a custom step-group and govern the entire step-group’s execution with only the one conditional evaluation on a call step.

# ./ci.yaml
context_parser: pypyr.parser.keys
steps:
  - name: pypyr.steps.default
    comment: set default values for optional cli inputs
    in:
      defaults:
        lint: False
        build: False

  - name: pypyr.steps.echo
    in:
      echoMe: begin - always runs

  - name: pypyr.steps.call
    run: '{lint}'
    in:
      call: lint

  - name: pypyr.steps.call
    run: '{build}'
    in:
      call: build

  - name: pypyr.steps.echo
    in:
      echoMe: end - always runs

lint:
  - name: pypyr.steps.cmd
    in:
      cmd: echo lint cmd 1 here

  - name: pypyr.steps.cmd
    in:
      cmd: echo lint cmd 2 here

build:
  - name: pypyr.steps.cmd
    in:
      cmd: echo build cmd 1 here

  - name: pypyr.steps.cmd
    in:
      cmd: echo build cmd 2 here

You can control this pipeline from the cli like this:

$ pypyr ci
begin - always runs
end - always runs

$ pypyr ci lint
begin - always runs
lint cmd 1 here
lint cmd 2 here
end - always runs

$ pypyr ci build
begin - always runs
build cmd 1 here
build cmd 2 here
end - always runs

$ pypyr ci lint build
begin - always runs
lint cmd 1 here
lint cmd 2 here
build cmd 1 here
build cmd 2 here
end - always runs

call invokes a step-group in the same pipeline. If you want to invoke another pipeline instead you can use a pype step and similarly set the conditional statement on that.

switch branching

If you want to switch between calling different step-groups based on an input expression in IF-THEN-ELSE branch style, use pypyr.steps.switch.

This is useful if you have mutually exclusive execution paths you want to take selectively depending on whether a certain condition obtains.

In this example, pypyr.parser.string will put whatever you pass from the cli after the pipeline name into the string variable argString. The switch then uses the argString to decide which branch to execute:

# ./my-pipeline.yaml
context_parser: pypyr.parser.string
steps:
  - name: pypyr.steps.switch
    comment: argString is set by pypyr.parser.string context parser
    in:
      switch:
        - case: !py argString == 'a'
          call: A
        - case: !py argString == 'b'
          call: B
        - case: !py argString == 'bc'
          call: [B, C]
        - default: D

  - name: pypyr.steps.echo
    in:
      echoMe: done!

A:
  - name: pypyr.steps.echo
    in:
      echoMe: A

B:
  - name: pypyr.steps.echo
    in:
      echoMe: B

C:
  - name: pypyr.steps.echo
    in:
      echoMe: C

D:
  - name: pypyr.steps.echo
    in:
      echoMe: D

Running this pipeline will look like this:

term

$ pypyr my-pipeline # default case, argString is empty ''
D
done!

$ pypyr my-pipeline a
A
done!

$ pypyr my-pipeline b
B
done!

$ pypyr my-pipeline bc
B
C
done!

$ pypyr my-pipeline xyz # no matches, default
D
done!

dynamic step execution

A compact pattern to create an extensible custom cli for your pipeline is to use the call step in conjunction with the list context parser.

This way you are not hard-coding which step-groups to run - whichever group names you pass from the cli will run in order.

# ./ops/build.yaml
context_parser: pypyr.parser.list
steps:
  - name: pypyr.steps.call
    comment: lint & test code. this runs on every pipeline invocation.
    in:
      call: 
        - lint
        - test
  - name: pypyr.steps.call
    comment: optionally do extras like package & publish after lint & test.
    run: '{argList}'
    in:
      call: '{argList}'


lint:
  - name: pypyr.steps.cmd
    in:
      cmd: echo lint command here

test:
  - name: pypyr.steps.cmd
    in:
      cmd: echo test command here

package:
  - name: pypyr.steps.cmd
    in:
      cmd: echo package command here

publish:
  - name: pypyr.steps.cmd
    in:
      cmd: echo publish command here

For the sake of clarity, each step-group only has one step in it, but you can (hopefully obviously) specify multiple steps if you want.

# only run the common functionality, no extras
$ pypyr ops/build
lint command here
test command here

# run common + package
$ pypyr ops/build package
lint command here
test command here
package command here

# run common + package + publish
$ pypyr ops/build package publish
lint command here
test command here
package command here
publish command here

conditional logic permalink

selectively run or skip step permalink

use variables to control if step runs permalink

set conditional logic in pipeline permalink

use cli arguments to run step conditionally permalink

use python expressions for control flow permalink

multiple steps with the same condition permalink

switch branching permalink

dynamic step execution permalink

see also

conditional logic

selectively run or skip step

use variables to control if step runs

set conditional logic in pipeline

use cli arguments to run step conditionally

use python expressions for control flow

multiple steps with the same condition

switch branching

dynamic step execution