At Etsy I recently worked on a project to redo how we handled the submission of ad hoc Hadoop jobs since we were building out our second production Hadoop cluster. I’ve written up a description of the approach as well as some alternatives for Code as Craft.
Reflecting on Open-Sourcing statsd-jvm-profiler
Back in January I open-sourced the statsd-jvm-profiler project. A lot has happened with the project since then, so I took the time to reflect on how open-sourcing it affected the project. I’ve written up my thoughts (and included some exciting new features!).
Suppressing ad-handle-definition Warnings in Emacs
As I have been tweaking my Emacs configuration, I noticed that I was getting warnings like the following in *Messages*
during Emacs startup:
ad-handle-definition: `tramp-read-passwd' got redefined
I looked into it and these warnings are generated when functions are redefined with defadvice
. Most of the warnings I received were coming from third-party packages I had installed, not my own configuration. In any case, they were not helpful, just noise during Emacs startup.
Like everything in Emacs, this is configurable. Turning off the warnings is as easy as adding:
(setq ad-redefinition-action 'accept)
To your .emacs
file. I found I had to add it fairly early to ensure that all the warnings were removed.
New Year, New Tools
New Year, New Tools
It’s a new year, so I thought it would be a good time to review some new tools I started using over the last year.
Mosh
Mosh is a replacement for SSH that handles intermittent connectivity. It will warn you if your connection drops and automatically reconnect. This handles situations like putting your laptop to sleep or changing between different connections (e.g. switching from Ethernet to Wifi). I’m connected to a bunch of different servers over the course of the day, so being able to put my laptop to sleep and have it wake back up with my remote sessions intact is incredibly useful. It’s hard to believe I used to work without it.
Mosh doesn’t support X11 forwarding or non-interactive uses of SSH, like port forwarding. However, I don’t use these particularly often, so in practice I’m not bothered by this. In any case, regular SSH is still available!
zsh
I had resisted changing shells for a long time. Bash is available pretty much everywhere. I had a nice .bashrc that I could put anywhere and have the environment I was used it. zsh had been recommended to me several times, but I was concerned about the effort needed to get up and running on a new machine. I liked being able to just drop a few dotfiles in my home directory – would I also have to install zsh?
I decided to try the switch after getting fed up with an issue with my bash prompt and long lines – characters would occasionally get doubled. I am really glad I made the switch. Needing to install zsh wasn’t as big of a deal as I had feared. I used oh-my-zsh to get started. This gave me a lot of the configuration I had done manually in my .bashrc out of the box. Plus it provides plugins for just about anything you might use in the shell. My favorite feature of zsh itself is the improved tab completion. It’s hard to explain without experiencing it – just give zsh a try.
tmux
Much like bash, I had used screen for a while and it just worked. I had a configuration I liked and I didn’t need to think about it. When I was switching to zsh, I decided to reevaluate screen as well and give tmux a try. There aren’t a lot of major differences here, but I’m sticking with tmux. The documentation is better, window splitting feels nicer, and the memory usage is lower. The biggest gain from switching is the configuration format. I find tmux’s configuration file much easier to understand and modify. Just compare the status line from my .screenrc to the same status line from my .tmux.conf.
Helm
The scope of customization for Emacs is astoundingly broad. It’s one of my favorite parts of using Emacs – anything I want to do can be made better with the judicious application of some Lisp. The downfall of this is that it can be hard to keep up with all the options. When I first starting using Emacs in 2008, I was looking for a library for completing file names. I tried out two candidates: ido-mode and anything.el. ido was easy to set up and use, so I went with it. I kept using ido because it worked great. I hadn’t even considered switching until I came across this article on Helm. Helm is a fork of anything.el that cleans up the code and makes it more modular.
I read through that article and was immediately sold on using Helm. Switching to Helm has been the best change to my text editing workflow since switching to Emacs! It provides a consistent interface for completing anything. The fuzzy matching is far ahead of anything else I’ve used. Despite this power, it’s still incredibly fast. Even with very large lists of completion candidates there is no noticeable slowdown.
I encourage you to try out Helm rather than just reading about it. It is a bit different than other completion libraries for Emacs, and it did take me a couple of days to really get used to it. It was time well-spent, however.
statsd-jvm-profiler
I’ve just open-sourced statsd-jvm-profiler, a JVM profiler for Hadoop that sends data to StatsD! Check out the blog post introducing it and get the code on Github.
Code as Craft Post!
I’ve written my first post on Etsy’s engineering blog, Code as Craft, on the process of switching from Ant to SBT. Check it out here!
Configuring Emacs for Go Development
Setting Up Emacs for Go Development
I’ve recently started learning Go, and, as with any language, I want to configure Emacs as much as possible for Go development. After scouring the Internet, this is what I’ve come up with.
Prerequisites
You’ll need Emacs and Go installed. I’m using OS X, so I use Emacs from http://emacsformacosx.com/ and this Go package.
Once you’ve installed everything, you’ll need to set a few environment variables:
export GOROOT=/usr/local/go export GOPATH=$HOME/Documents/projects/go export PATH=$PATH:$GOROOT/bin
You can set $GOPATH
to a different directory if you like.
go-mode
The first thing you’ll need for Go development in Emacs is go-mode
. You can install this easily in Emacs 24 and up with M-x package-install go-mode
.
Installing go-mode
will also let you pull up documention on the standard library or third-party packages with M-x godoc
. To make godoc
work correctly, you’ll need to make sure Emacs has the correct values of $PATH
and $GOPATH
. This shouldn’t be an issue unless you’re using GUI Emacs on OS X. If you are, I’d recommend using my env-var-import
package to import their values from the shell. That package will also set exec-path
correctly from your $PATH
, which will be important in later steps.
go-eldoc
If you’ve done any Elisp development, you’re probably familiar with eldoc. If not, eldoc shows you the argument list of the function at your point in the minibuffer. go-eldoc
is a package that does this for Go. You can install it with M-x package-install go-eldoc
. Once you’ve done that, you’ll need to enable it:
(defun go-mode-setup () (go-eldoc-setup)) (add-hook 'go-mode-hook 'go-mode-setup)
gofmt
One of the things I really like about go is the automatic enforcement of a consistent coding style via gofmt
. You could manually run this on your code, but go-mode
lets you configure Emacs to run it automatically when you save a buffer:
(defun go-mode-setup () (go-eldoc-setup) (add-hook 'before-save-hook 'gofmt-before-save)) (add-hook 'go-mode-hook 'go-mode-setup)
goimports
There is also a tool called goimports
that not only formats your code like gofmt
but also automatically updates all your imports. I’d recommend using this instead of gofmt
.
You’ll need Mercurial installed first in order to install it. Then you can run go get code.google.com/p/go.tools/cmd/goimports
. Now you can tell Emacs to use goimports
instead of gofmt
:
(defun go-mode-setup () (go-eldoc-setup) (setq gofmt-command "goimports") (add-hook 'before-save-hook 'gofmt-before-save)) (add-hook 'go-mode-hook 'go-mode-setup)
godef
Installing go-mode
will also give you godef
, which will let you jump to the definition of a function like in an IDE. You can jump with M-x godef-jump
and then return to the original point with M-*
. I bound godef-jump
to M-.
for ease of use:
(defun go-mode-setup () (go-eldoc-setup) (setq gofmt-command "goimports") (add-hook 'before-save-hook 'gofmt-before-save) (local-set-key (kbd "M-.") 'godef-jump)) (add-hook 'go-mode-hook 'go-mode-setup)
Custom compile command
You can use M-x compile
to compile and test your code from Emacs. You’ll just need to set the command that compile
runs:
(defun go-mode-setup () (setq compile-command "go build -v && go test -v && go vet") (define-key (current-local-map) "\C-c\C-c" 'compile) (go-eldoc-setup) (setq gofmt-command "goimports") (add-hook 'before-save-hook 'gofmt-before-save) (local-set-key (kbd "M-.") 'godef-jump)) (add-hook 'go-mode-hook 'go-mode-setup)
I also bound C-c C-c
to run compile
.
Autocomplete
The last step is to set up autocompletion for Go. I use the auto-complete
package for other languages, so it’s natural to use it here too. For Go-aware autocomplete, you’ll need to install the gocode
tool first with go get -u -v github.com/nsf/gocode
. Then you can install auto-complete
with M-x package-install auto-complete
and go-autocomplete
with M-x package-install go-autocomplete
. Once you’ve installed everything add the following to your Emacs configuration:
(require 'auto-complete-config) (require 'go-autocomplete)
Now you’re ready to start writing some Go in Emacs!
env-var-import
env-var-import
Writing my last post on setting environment variables in GUI Emacs got me thinking about a more general solution to that problem. Rather than requiring everyone to define a function like set-exec-path-from-shell-PATH
, it would be nice if there were a package that would handle importing everything for you given a list of environment variables.
To that end I wrote env-var-import. It’s available in Marmalade for easy installation: package-install env-var-import
.
Once you’ve installed it, add (require 'env-var-import)
to your Emacs configuration and you’re ready to do some importing!
You have two options for using it:
(env-var-import)
: In this mode, it will only import the environment variable defined byenv-var-import-exec-path-var
and setexec-path
to the value of that variable.env-var-import-exec-path-var
defaults to toPATH
but can be customized.(env-var-import '("VAR1" "VAR2"))
: In this mode it will still import the value of the environment variable defined inenv-var-import-exec-path-var
and setexec-path
to that value. However, it will also import the values ofVAR1
andVAR2
.
Setting Environment Variables in GUI Emacs
I use Emacs for most of my development work. When I’m not SSHed into a remote machine I generally stick with the GUI Emacs. On OS X this has the downside of not inheriting environment variables from your shell.
I could work around this by using setenv
, but that would require me to duplicate the values of those variables. Thankfully there is a better solution:
(defun set-exec-path-from-shell-PATH () (let ((path-from-shell (shell-command-to-string "$SHELL -i -c 'echo $PATH'"))) (setenv "PATH" path-from-shell) (setq exec-path (split-string path-from-shell path-separator))))
This will invoke the shell to get the value of $PATH
and then use setenv
to set it. Then all you need to do is add (if window-system (set-exec-path-from-shell-PATH)) to your Emacs configuration.
There is nothing special about $PATH
in this regard. You could easily extend this function to set other environment variables as needed.
I originally found this code on the Clojure mailing list.
Winds of Change
When I started this blog in early 2013, I intended it to be primarily about technology. I never wrote anything in that vein, so several months later I started posting short horror stories.
I plan to keep writing fiction, but I would like to have a normal tech blog as well. As such, I’ve moved all my fiction to Library of Shadows. All new stories will be posted there.