Profiling your website like a true Ninja
over 13 years ago
After a mammoth effort by Jarrod Dixon the team’s production profiler is now ready for an open source release.
http://code.google.com/p/mvc-mini-profiler/
Let me start with a bold claim. Our open-source profiler is perhaps the best and most comprehensive production web page profiler out there for any web platform.
There I said it, so let me back up that statement.
The stone-age of web site profiling
Ever since the inception of the web people have been interested in render times for web pages. Performance is a feature we all want to have.
To help keep sites fast many developers include the time it takes to render a page in the footer or an html comment. This is trivial on almost any platform and has been used for years. However the granularity of this method sucks. You can tell a page is slow, but you really want to know is, why?
To overcome this issue often frameworks use logs, like the famous Rails development.log. The trouble is that it is often very hard to understand log files and this information is tucked away in a place you often do not look at.
Some people have innovated and taken this to the next level Rack Bug is a good example for Rails and L2S Prof is a good example for .Net. Additionally some products like NewRelic take a holistic view on performance and give you a dashboard on the cloud with the ability to investigate particular perf issues down to the SQL.
The trouble has always been that the trivial profilers don’t help much. The nice ones are often not designed to work in production and often involve external tools, installs and dependencies. One clear exception is NewRelic, an excellent product. However when dealing with a single web page I think our profiler has an edge.
An ubiquitous production profiler
Our “dream” with our profiler was to have a way for us to get live, instant feedback on the performance characteristics of the pages we are visiting - in production. We wanted this information to be only visible to developers and have no noticeable impact on the site’s performance. A tricky wish to fill seeing our network sees millions of page views a day.
The ubiquity of the profiler is key, developers have become aware of slowness and the reasons for it in every day usage of the site. Analyzing the source of performance problems is trivial since you are able to drill-down on and share profile sessions. We have become much more aware of performance issues and mindful of slow ajax calls (since they are also displayed on the page as they happen)
Production and development performance can vary by a huge margin
Ever since we started using our profiler in production we noticed some “strange” things. Often in dev a page would be fast and snappy, but in production the same page had very uneven performance. Often we traced this back to internal locking in LINQ-2-SQL and ported queries to dapper. This does however bring up a very important fact.
Development performance may be wildly different to production performance.
Page profiling vs. the holistic view
Internally we use 2 levels of production profiling. We log every request with it’s total render time in a database (via haproxy log), this gives us a birds-eye view of performance. When we need to dig in on actual issues we use our profiler.
Both approaches are complimentary and in-my-view necessary for a high performance, high scale website. Efforts are much better spent optimizing a page that is hit 100k times a day vs. an equally slow page that is only accessed a handful of times.
This kind of functionality should be baked in to web frameworks
I find it strange that web frameworks often omit basic functionality. Some do not include basic loggers, most do not offer an elegant log viewer. None seem to provide a comprehensive approach to page profiling out of the box. It’s a shame, if we were all looking at this kind of information from day-1 we could have avoided many pitfalls.
Play with our profiler … today
Our profiler is open source, so is Data Explorer. All logged in users can experience the profiler first-hand by browsing the web site.
Ease of deployment
The profiler is neatly packaged up in a single DLL. No need to copy any CSS, JS or other files to get it all working. Internally we use the excellent razor view engine to code our profiling page,this is compiled on the fly from the embedded resource using this handy trick. Our CSS is all in awesome LESS, we translate it to CSS on the fly in JavaScript. All the resources are embedded into the DLL.
Profiling SQL is achieved by introducing a bespoke DbConnection that intercepts all DB calls. This interception only happens when a profiling session is in progress.
Profiling blocks are ludicrously cheap since we use a fancy trick around extension methods. You may call extension methods on null objects.
public static IDisposable Step(this MiniProfiler profiler, string name, ProfileLevel level = ProfileLevel.Info)
{
return profiler == null ? null : profiler.StepImpl(name, level);
}
If there is no MiniProfiler in play the cost is a simple null check.
Hope you enjoy the MiniProfiler be sure to tweet a thank you to @marcgravell and @jarrod_dixon if it helps you.
Very cool… one question tho, does the MiniProfiler work with Spark view engine?