AWS related - notes to self

Uploading files to AWS

A python interface to AWS

e.g. code using the interface

How to decrypting S3 data before EMRing

The below discussion demonstrates how to decrypt S3 data as a bootstrap action to the EMR cluster: https://forums.aws.amazon.com/thread.jspa?threadID=50189

Another example is to use the S3 Java client side encryption in Map/Reduce jobs: http://aws.typepad.com/aws/2011/04/client-side-data-encryption-using-the-aws-sdk-for-java.html

AWS streaming job flow

See link below re how to create a streaming job flow (note: can use gzip + password as part of the streaming job)

http://docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/CLI_CreateStreaming.html