pipeline look-up order
the working directory
pypyr first looks for pipelines, any custom steps & other code in the current working directory. This is the directory from which you invoke pypyr.
Simply put, by default, this is the directory you’re currently in when you invoke pypyr from the cli.
You can change the default working directory by passing the --dir
flag to
pypyr CLI. pypyr will use whatever path you specify in --dir
as the working
directory base path. If you’re using the API, use the working_dir
argument
for the same effect.
Changing the working directory not only means that pypyr looks for pipelines in that location first, but will also resolve any custom code modules like your own steps or context-parsers from there.
directory locations lookup order
By default pypyr uses a file loader to find & load pipelines from the filesystem. pypyr searches a specific sequence of locations for pipelines matching the pipeline name:
{working dir}/
{working dir}/pipelines
{pypyr install dir}/pipelines
{pypyr install dir}
is where-ever you installed the currently running
instance of pypyr. Very likely, this is either in your default python path, or
the currently active virtual environment:
{your python environment}/lib/python3.8/site-packages/pypyr/pipelines/
{pypyr-install-dir}
is for pypyr built-in pipelines. You
probably shouldn’t be saving your own pipelines there, they might get
over-written or wiped by upgrades or re-installs.pipeline name matching
To make your life easier when you edit your pipelines in your favorite yaml editor, do use the .yaml extension for your pipeline files.
Still to make your life easier, don’t add the .yaml when you invoke pypyr with your pipeline-name. pypyr always appends .yaml to your input pipeline name for you under the hood.
When you run:
$ pypyr pipeline-name
pypyr will look in these locations in this order:
./pipeline-name.yaml
./pipelines/pipeline-name.yaml
{pypyr-install-dir}/pipelines/pipeline-name.yaml
pipelines in sub-directories
To organize your pipelines, you can save your pipelines in whatever directory structure you please.
./
|-- pipe-0.yaml
|-- dir
|-- subdir1
|-- pipe-1.yaml
|-- pipe-2.yaml
|-- subdir2
|-- pipe-3.yaml
|-- pipe-4.yaml
You can run these pipelines like this:
$ pypyr pipe-0
$ pypyr dir/subdir1/pipe-1
$ pypyr dir/subdir1/pipe-2
$ pypyr dir/subdir2/pipe-3
$ pypyr dir/pipe-4
You can achieve the same effect for everything in the dir
sub-directory by
using the --dir
flag. The difference is that if you use the --dir
flag
pypyr will also load any custom python modules from this directory.
$ pypyr --dir dir subdir1/pipe-1
$ pypyr --dir dir subdir1/pipe-2
$ pypyr --dir dir subdir2/pipe-3
$ pypyr --dir dir pipe-4
shared pipeline directories
The --dir
switch can refer to any accessible directory on your file-system,
it doesn’t have to be a sub-directory of from wherever you execute pypyr.
So if you had a shared pipeline library on your file-system, for example at
/Users/captainhook/shared-pipelines/
, you can run your pipelines from
anywhere else like this:
$ echo $PWD
/git/myproject
$ pypyr --dir ~/shared-pipelines subdir1/my-shared-pipe
The above example will run ~/shared-pipelines/subdir1/my-shared-pipe.yaml
, and
resolve all pipeline names and custom code from the directory specified by the
--dir
input.