Eliminating my trivial inconveniences building Discourse
almost 11 years ago
Many developers spend a fair amount of the day waiting:
Waiting for a build to end
Waiting for search results to come up
Waiting for an editor to launch
Waiting for tests to run
Waiting for a boss to give instructions
There is also a more insidious form of waiting. The zero value work you tend to do repetitively, also known as trivial inconveniences.
These issues don’t stop you from working, they just make your job slightly more annoying.
For example:
- Every time you hit the “refresh” button to see how your CSS change modified the site you are working on.
- Every time you manually run your test suite.
- Every time you “compile” your project and wait for it to finish.
Computers could do these tasks automatically; instead, like trained monkeys, we often repeat these pointless tasks tens and hundreds of times daily.
While developing Discourse I pay very special attention to waiting and trivial inconveniences. I work tirelessly to clear friction. In fact, I will spend days to eliminate particularly bothersome bottleneck.
I use Vim
Vim is tough editor to pick up, especially late in your career. It is notoriously difficult to master.
Fortunately, people getting started today with Vim have some amazing resources such as Practical Vim by Drew Neil, Vimcasts and VimGolf that make the process less painful.
Vim helps me eliminate many trivial inconveniences.
- It launches instantly, I am never waiting for my editor to start up.
- I use a fair amount of Vim plugins. Syntastic, rails.vim, NERDTree, surround.vim, tcomment and many others. The plugins I use help meld Vim so it works really well with my workflow.
- I have some Discourse specific bindings.
- I bind CTRL-A to touch a restart file, I can bounce my web server in a key stroke if needed.
- I bind CTRL-S to dump a message in the Discourse messaging bus that forces all web browsers pointing at local Discourse to refresh.
- I have a variety of navigation hot keys to cut through the JavaScript files,
:Rdmodel t<tab>… takes me to the topic js model.
I am not afraid to switch tools
I have used Ack for a very long time to search through code files, recently I discovered the silver searcher. Silver searcher is faster than Ack, to me this makes all the difference. To quote Geoff Greer, the author.
I can already hear someone saying, “Big deal. It’s only a second faster. What does one second matter when searching an entire codebase?” My reply: trivial inconveniences matter.
I am not afraid to spend money on software
I have Windows 8 Desktop (with 2 monitors) and a MacBook Pro Retina on my desk. Initially, I used to reach over to the laptop to type in commands in order to test stuff out. Reaching over to the laptop is not a mammoth effort but it is very inconvenient. Due to this trivial inconvenience I rarely used my laptop during the day.
I downloaded Synergy and tried sharing my keyboard with my Mac laptop. It worked… sometimes. It crashed and failed a lot and was a monster to configure. Many developers would stop here. Free does not work, go shopping, for something else free. I disagree with this attitude.
There are a couple of paid alternatives. Share mouse worked very well for me. In a blink I spent the 50 or so bucks to eliminate these inconveniences. Why save 50 bucks just to have a $2000 laptop sit on your desk collecting dust?
Live CSS refresh
Whenever I save a CSS file my browsers automatically refresh with the amended visual style. This is accomplished with a very Spartan amount of code.
In our Guardfile:
module ::Guard
class AutoReload < ::Guard::Guard
require File.dirname(__FILE__) + '/config/environment'
def run_on_change(paths)
paths.map! do |p|
hash = nil
fullpath = Rails.root.to_s + "/" + p
hash = Digest::MD5.hexdigest(File.read(fullpath)) if File.exists? fullpath
p = p.sub /\.sass\.erb/, ""
p = p.sub /\.sass/, ""
p = p.sub /\.scss/, ""
p = p.sub /^app\/assets\/stylesheets/, "assets"
{name: p, hash: hash}
end
# target dev
MessageBus::Instance.new.publish "/file-change", paths
end
def run_all
end
end
end
guard :autoreload do
watch(/tmp\/refresh_browser/)
watch(/\.css$/)
watch(/\.sass$/)
watch(/\.scss$/)
watch(/\.sass\.erb$/)
watch(/\.handlebars$/)
end
In our devlopment js file it does something like:
return Discourse.MessageBus.subscribe("/file-change", function(data) {
return data.each(function(me) {
var js;
if (me === "refresh") {
return document.location.reload(true);
} else {
return $('link').each(function() {
if (this.href.match(me.name) && me.hash) {
if (!$(this).data('orig')) {
$(this).data('orig', this.href);
}
this.href = $(this).data('orig') + "&hash=" + me.hash;
}
});
}
});
});
}
There are plenty of solutions out there that achieve the same goal, there is LiveReload. The folks at Live Reload also documented some other options.
I chose to roll out our own solution cause I wanted better control. Our code knows how to reload Ember templates and will grow to do more sophisticated things.
I spend time making the development environment fast
Discourse has a very large number of JavaScript and CSS files. This means that during development the web server has to serve out over 370 assets to the clients. This is certainly an edge case for Rails. However, as client side frameworks like Ember and Angular become more popular this is going to be a problem more people are going to face.
I committed a fix to Discourse that cuts down the time it takes to refresh a page down from 4.5 seconds to 1 second in Development. I also opened an issue to help decide if we want to include the optimisation in Rails or not. Unlike more common attempts, that bundle up all assets in Dev to “work around” the problem, this fix is transparent and does not compromise debugging. In future when source maps are more pervasive, hacks like this may not be needed. I am a pragmatist though, this solves our problem now. Why wait?
I also committed a fix that dramatically improved sprockets (the Rails Asset Pipeline) in dev mode.
I have effective Rails development environment
I have a reasonably fast workstation, though I am due for a Haswell upgrade next month. I have been running dedicated SSDs for development for years now. I have a multi-monitor setup primary one is a 30 inch Dell.
Most Rails developers out there are probably using Ruby 1.9.3 (or 1.8.7) untuned to work locally. If I start a Rails console to Discourse on an untuned stack it takes 16 seconds to boot up. I run Ruby 2.0 in Development with the following environment vars:
export RUBY_GC_MALLOC_LIMIT=1000000000 export RUBY_HEAP_SLOTS_GROWTH_FACTOR=1.25 export RUBY_HEAP_MIN_SLOTS=800000 export RUBY_FREE_MIN=600000 export LD_PRELOAD=/usr/lib/libtcmalloc_minimal.so
This reduces Discourse startup time to just under 4 seconds. That is a 4x improvement just by tuning my dev machine. Additionally, I have experimented with Zeus and Spring (Discourse is pre-configured to work with Spring) with varying degrees of success. When these forking apps work you get 1 second startups.
We use better errors that works many times better than the default Rails error page and MiniProfiler that allows me to quickly debug through issues.
I really hate waiting for tests
When I started working with Discourse we had a lightning fast test suite. I could easily work on an area of code and get immediate feedback about my specs.
Fast forward a few months, we now have a test suite that takes 3 minutes to run, after you pull some ninja moves to get it that fast.
I agree it takes too long, I would like to refactor and speed it up. However, this is not the point of this section.
There are currently 2 common spec runners used by the Ruby community, autotest and Guard. Guard is more modern and actively developed.
Trouble is both runners are very troublesome for my workflow.
Both runners insist on running every single spec in the queue before obliging and running the spec you want them to run. So, my typical workflow used to be.
- Launch
bundle exec guard - Write my broken test
- Save
- Wait up to 3 minutes for the old run to finish so I can see it really failed
- Fix it
- Save
- See it passed
- GOTO (2)
So, a lot of waiting. But its critical I run the entire test suite often cause many of our tests unfortunately integrate.
To resolve my large amount of pain I took the time to write a test runner that works really well for me. https://github.com/discourse/discourse/blob/master/lib/autospec/runner.rb .
- It will interrupt the current testing when it detects I want to run a spec (cause I saved a file)
- It will resume the old testing after it finished running the spec I wanted it to run.
- It will automatically focus on a broken test (only run a single test) something that allows me to spring
putsstatement around my code to quickly inspect state for the broken spec. - It allows me to switch focus to another spec (by saving a spec) even in a failed state.
For me, having a test runner that operates as I would expect it to has completely transformed the way I work. I no longer approach testing as a drudge, it is fun again.
Eliminating trivial inconveniences makes me a happier developer. Go ahead, find one inconvenience you have and eliminate it. You will not regret it.
Nice reading, thank you. I’m really interested (not holy war or trolling) about why Windows, and why Windows 8. And can you please tell more about Rails environment inside of your Win box – vagrant? cygwin?