pypyraws.steps.s3fetchjson

Fetch a json file from s3 and put the json values into context.

input

Required input context is:

s3Fetch:
  clientArgs: # optional
    arg1Name: arg1Value
  methodArgs: # mandatory
    Bucket: '{bucket}'
    Key: arb.json
  key: 'destination pypyr context key' # optional

• clientArgs go to the aws s3 client constructor. These are optional.
• methodArgs go to the the s3 get_object call. The minimum required values are:
  • Bucket
  • Key
• key writes fetched json to this context key. If not specified, json writes directly to context root.

The s3Fetch input context supports text {substitution} formatting expressions.

method arguments

The only required inputs for methodArgs are Bucket and Key, but you can use any of the following inputs to enable extra functionality like cache, version and SSE service side encryption:

Bucket='string',
IfMatch='string',
IfModifiedSince=datetime(2015, 1, 1),
IfNoneMatch='string',
IfUnmodifiedSince=datetime(2015, 1, 1),
Key='string',
Range='string',
ResponseCacheControl='string',
ResponseContentDisposition='string',
ResponseContentEncoding='string',
ResponseContentLanguage='string',
ResponseContentType='string',
ResponseExpires=datetime(2015, 1, 1),
VersionId='string',
SSECustomerAlgorithm='string',
SSECustomerKey='string',
RequestPayer='requester',
PartNumber=123

Check here for all available arguments with detailed explanations of each: http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Client.get_object

output

pypyr will merge json parsed from the file into the pypyr context. Use the key input to control whether you want the resulting structure to merge into the root or into its own key.

This will overwrite existing values if the same keys are already in there.

I.e if file json has {'eggs' : 'boiled'}, but context {'eggs': 'fried'} already exists, returned context['eggs'] will be the updated value ‘boiled’.

If you do not specify key, the json should not be an Array [] at the root level, but rather an Object {}.

example

See a worked example for pypyr aws s3fetch.