filesystem

pypyr has many ready-made functions to read, write & format files on the filesystem.

You can convert between text, json, yaml & toml formats without you having to write code. This is especially handy when you’re generating configuration or template files on-the-fly that you want to inject into other tools that you are automating with pypyr.

cross-platform paths

Windows users, you can specify paths with / or \ as directory separator. pypyr will resolve either correctly for you. Pretty much all platforms except MS Windows use /.

When you’re working with files and you want to have a cross-platform compatible pipeline and you’re using relative paths, front-slash / is the way to go.

encoding

pypyr will use the platform’s default encoding to deal with text-based files. This is utf-8 for most sensible systems, but be aware on Windows it’s still cp1252.

steps

All steps that work with files have an encoding input where you can override the default and explicitly set the encoding:

- name: pypyr.steps.filewrite
  in:
    fileWrite:
      path: out/utf8-example.txt
      payload: "€ ∮ E⋅da = Q,  n → ∞, ∑ f(i) = ∏ g(i), ∀x∈ℝ: ⌈x⌉ = −⌊−x⌋, α ∧ ¬β = ¬(¬α ∨ β)"
      encoding: utf-8

See here for a list of available encodings.

You cannot set encoding in binary mode.

in-out steps

All steps with an in and out input allow you to override the default encoding with three possible inputs:

• encoding - set the encoding for both in + out.
• encodingIn - set the encoding for the in file.
• encodingOut - set the encoding for the out file.

These are all the “format” and “replace” steps.

You can convert files between different encodings by setting a different encoding for the input than the output.

If you don’t specify encodingIn pypyr will default to the value of encoding for the input file.

If you don’t specify encodingOut pypyr will default to the value of encoding for the output file.

If you did not specify encoding either, pypyr will use the default encoding.

context parsers

The file-based context parsers always use the default encoding when reading input files. This is the case when you:

• initialize variables from a json file
• initialize variables from a yaml file

A toml file must always be in utf-8, per the TOML Spec.

default encoding

The platform default is utf-8 for most systems, but be aware on Windows it’s still cp1252. To check your platform’s default encoding, do:

import locale
locale.getpreferredencoding(False)

If you’re getting annoyed with having to change the encoding on each step, you can change the default if you either:

• Set pypyr’s default encoding in config.
  • This changes the encoding just for pypyr.
• Set Python to run in utf-8 mode.
  • See PEP 0540.
  • This sets the default encoding to utf-8 for your entire Python environment.
• Set Python encoding to something other than utf-8.
  • Maybe don’t do this.
  • Check the Python docs if you want to go this route.

If you explicitly set encoding on any filesystem step it will override the default for that particular step.

filesystem operations