Comments on: Just Look at it, Man! http://vgable.com/blog/2009/11/11/just-look-at-it-man/ my weblog. Wed, 08 Feb 2012 13:49:22 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Jason Petersen http://vgable.com/blog/2009/11/11/just-look-at-it-man/comment-page-1/#comment-879 Jason Petersen Wed, 11 Nov 2009 11:40:32 +0000 http://vgable.com/blog/?p=495#comment-879 I probably look at tens to hundreds of graphs a day, for temporal data sets with time ranges measured in days and weeks. Multiple lines and hundreds of data points of various dimensional projections of our data set. I can sit here and reason about the results of a code change all I want, but nothing really compares to pushing a code change out and really jumping into the graphs. Seeing in what ways certain statistics change gives you an intuitive way of reasoning about correctness/performance/regression that no unit test can offer. I don't know how many times I've been called over (or called over people to my desk) for the question: "Why does this look like this?" Humans are astonishingly good at finding patterns, and impeccable at seeing/feeling where those patterns change. Often we'll know something's up way before any of the crude statistics (percentiles/averages) have technically changed significantly. I probably look at tens to hundreds of graphs a day, for temporal data sets with time ranges measured in days and weeks. Multiple lines and hundreds of data points of various dimensional projections of our data set. I can sit here and reason about the results of a code change all I want, but nothing really compares to pushing a code change out and really jumping into the graphs. Seeing in what ways certain statistics change gives you an intuitive way of reasoning about correctness/performance/regression that no unit test can offer.

I don’t know how many times I’ve been called over (or called over people to my desk) for the question: “Why does this look like this?” Humans are astonishingly good at finding patterns, and impeccable at seeing/feeling where those patterns change. Often we’ll know something’s up way before any of the crude statistics (percentiles/averages) have technically changed significantly.

]]>