Month: June 2016

Bugs with Non-Alphanumeric Credentials for AWS

I've recently had a chance to be working with AWS and S3 and encountered a somewhat frustrating bug that I wanted to share. I was using boto3, an AWS SDK for Python. I had created a script that would list the EC2 instances that make up an EMR cluster. It worked fine with my credentials, but another user would be a SignatureDoesNotMatch error every time. We spent a lot of time comparing the environment setup and permissions on the two accounts, but everything lined up. We started guessing a little more wildly when we noticed that my AWS credentials (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) were entirely alphanumeric, but the other user's were not. My first thought was that there was some kind of encoding error, but I had no luck with that. After some searching I found this longstanding bug with the same issue. As it turns out having non-alphanumeric credentials can cause this error and has for at least a couple of years. The only fix is to regenerate the credentials until getting a set that is entirely alphanumeric.

I later ran into the same signature error trying to distcp some data from HDFS to S3, but I knew what to look for this time. HADOOP-3733 is a very similar issue to that in boto3, but this is fixed in Hadoop 2.8.0.

This was a troublesome bug for me, so hopefully this post helps someone else who encounters the same issue.

Blogging More Regularly

It has been a while since I have written anything substantial here. General busyness with life and work have kept me from getting around to writing anything. However, I am going to try and blog more regularly going forward. Starting out I'm going to try and post something once a week and see how that goes.

Writing Setup

For my first post in this attempt I am going to talk a little about my writing setup.

I've found that I really like distraction-free writing tools as opposed to a standard text editor or word processor. Being able to focus only on the text helps keep me in the zone. I've tried a few of these and I've come to like Ulysses the best. It's less minimalistic than some others that I have tried but has a lot of really useful features. You write with Markdown, which I'm already familiar with, and it has great export functionality. It also syncs via iCloud so I can work on either my iPad or my desktop computer.

I used to write exclusively on a computer at my desk, but I found this an impediment. If I have an idea of something to write about I don't necessarily want to go sit at my desk to put down the words. Instead I'm now usually using a keyboard attached to my iPad to write. It's a little smaller than a full-sized keyboard so my typing speed isn't up to par yet, but it's great to be able to sit on the couch and write.