continuing the discussion with Stephen Few - my response

Before responding to Steve’s points, I want to take a step back and clarify that my original intent in critiquing his InfoVis capstone presentation was not to attack him personally, but rather his position on the topic of “infovis for the masses.” Unfortunately, it may have come off that way, and Steve’s responses to my issues, phrased as responses to “Few does X,” may further personalize the criticism, so I want to emphasize that this is ultimately a discussion about approaches to understanding infovis “for the masses,” and not some kind of flame war. As Steve suggested, I think it is clear that we both care a great deal about the “state of information visualization” today, so I apologize, particularly to Steve, if any part of this conversation has seemed or seems personal. It most certainly isn’t intended to be. I also want to point out that I completely agree with a good deal of the “low level” design issues Steve rails against (problematic use of color, charts designed so poorly that it is actually impossible to read values off of them, etc.); That’s not really what I’m taking issue with in this conversation. Again, I’m more interested in conceptualizing infovis design “for the masses.” So, that said: onwards and upwards!

Rather than responding point-by-point to Steve’s post, I will try to address what I see as the underlying disconnect in our positions. This disconnect, I think, revolves around our differing conceptions of the scope of information visualization, as well as our differing definitions of certain key terms. I’ll elaborate from some of the statements he makes in his response:

What is perhaps not obvious, based on my capstone presentation alone, is
the fact that I spend a fair amount of time trying to understand what draws
people to ineffective visualizations—those that fail to serve the needs of the
audience while managing to appeal to that audience on some level.

[…]

Many-Eyes has managed to make the process of data exploration and analysis interesting and fun, without resorting to features that undermine the effectiveness of the activity.

[…]

I hope that I’m never guilty of writing off meaningful aspects of visualization as useless, simply because they don’t match my own preferences—aesthetic or otherwise. If you ever catch me doing so, I want you to call me on it. Just make sure that what you point out as useful in a
visualization is actually useful and not just appealing to your own preferences.

[…]

When a great deal of evidence indicates that certain visualization practices work better than others, I believe that it’s helpful to teach people to follow the best practices and avoid those that fail.

[…]

I define the “right way” as the way that best satisfies the needs of people—the way that works. I’m a pragmatist. What I don’t do is define the “right way” as the way that people desire things to be done. Our desires, our notions of how things should be done, often conflict with the way that really works.

[…]

The real “disservice to the goal of popularizing information visualization” is the existence of (1) ineffective or irrelevant infovis projects and products that represent our work poorly, and (2) the unfortunate inability of many experts in the field to present their work to those who need it in a way that they can relate to, care about, and understand.

What makes these statements problematic to me, and what was part of what I was trying to get at in my original critique, are Steve’s definitions of terms like “useful,” “(in)effective,” and “[user] needs,” with regard to information visualization. If I were to try to identify, right off the bat, the fundamental disconnect between the two “camps” that Steve and I represent, it would be that we don’t necessarily agree on what these terms mean.

Steve, coming from the area of business intelligence, presumably values the clarity of the data above all else, which I think is a perfectly reasonable position given the “needs” of that field. I’m not totally familiar with the world of business intelligence, but the use of information graphics and visualization in that area is undoubtedly motivated by a need to understand the meaning of quantitative data and make critical decisions based on that understanding. So, for him, having the data presented as plainly, directly, and efficiently as possible is the prime consideration in the design of their visualizations. Anything that distracts from that is considered ineffective or not useful.

My contention is that this sort of traditional, “scientific” understanding of information visualization, while certainly valuable in some domains (such as business intelligence), is too restrictive when considering its broader, more popular uses. For one thing, there is the obvious point that “popular visualization” does not necessarily share the same critical goals. Many of the infovis examples that Steve criticized from the Smashing Magazine article exemplify this, in that they present information that is not necessarily “mission critical” in the same way BI information might be – people are not necessarily viewing these visualizations because they need to make critical decisions based on the meaning of the data they present. Rather, they are perhaps more “casual” forms of information visualization in which directness and efficiency of transmission are not the primary goal, which then complicates our conception of “usefulness” or “effectiveness.”

Now, I am the first to admit that, at this moment, I don’t have formal definitions of these terms that stand in opposition to the ones applied in Steve’s conception of infovis, but work to define them is starting to emerge: at InfoVis, we saw Zack Pousman’s (et al.) presentation and paper on “casual infovis,” Martin Wattenberg’s discussion of “vernacular visualization,” Fernanda Viegas’ description of the many expected uses of Many Eyes, Ola Rosling’s emphasis on visualization as storytelling, Brent Fitzgerald on Swivel as “big (social) database in the sky,” and Robert Kosara suggesting new uses for infovis (I’m sure I’m leaving someone out, but these are some I just blogged about). Across the internet, the discussion continues: Andrew Vande Moere, Andrea Lau, and Nick Cawthon ponder information aesthetics, the guys at Stamen Design discuss “useless” visualization, as does Matthew Hurst of Microsoft Live Labs… the list goes on!

What bothers me about Steve’s position, which I sensed mirrored in a lot of the discussion at InfoVis, is the suggestion that this new research, or the type of visualization that it describes, is somehow “not infovis.” While it’s true that infovis, as a field, grew out of a “strict” scientific tradition (ie. computer science) that informs its theories and methodologies, it is going to have to broaden its understanding of the ways in which “normal” people interact with information if it wants to present itself as accessible to the masses. I think the field will need to start thinking more in terms of “engagement design” rather than the highly quantified metrics of efficiency, time on task, etc., that have traditionally characterized user studies in HCI and interface design. Those metrics may make sense (and I think give rise to, at least in part, the “great deal of evidence” supporting best practices that Steve refers to) when describing visualization usage by users who spend their lives (and make their livelihoods by) working with charts, graphs, and other visualizations, but almost certainly don’t capture what is important about the way “the masses,” as I define them, use information visualization. For their purposes, engagement (another ambiguous term) should perhaps trump efficiency; for instance, maybe a particular design should sacrifice some of the “directness” of the data for other elements, such as “playfulness,” that encourage continued use. I can understand why Steve and others might be skeptical of this sort of approach to infovis, as there doesn’t yet exist a large body of research support it, but I would argue that this is because we haven’t really starting thinking about visualization in these terms yet, rather than because there is something fundamentally wrong with it. I’m concerned that the traditional infovis field believes too strongly that their understanding of their domain is the final word on information visualization. This is a problem.

Steve himself admitted that, if given a choice of visualization design, most users would always pick the “flashier” one. I’ll just reiterate that instead of assuming that “flashier” equals “less effective,” we should try hard to understand the value of “flashiness” (or “novelty,” or “aesthetics,” or whatever we want to call it) against the expanded definitions of “useful” and “effective” I’ve referenced here. At his tutorial session at InfoVis, I made the (admittedly somewhat outlandish) comparison to Jim Cramer and his highly-rated CNBC program, Mad Money. Cramer uses an almost ridiculous amount of (literally) bells-and-whistles to “decorate” his presentation of financial information, and yet the information gets across. Is it presented as thoroughly as it might be by a more Spartan “talking head” presentation? Probably not, but the show’s popularity says something about what engages non-experts. Coming back to the realm of infovis, there is something undeniably important and valuable about the fact that visualization examples such as Jonathan Harris’ “We Feel Fine,” or Stamen Design’s visualizations of Digg.com, or, yes, the Ambient Orb, are popular and engaging. What’s interesting about Many Eyes, an example Steve does like, is that it supports many types of representation, from traditional analytic uses to the more unusual ones that Fernanda described in her presentation at InfoVis. Arguing that the mere existence of these sorts of projects undermines information visualization is a real problem. It implicitly assumes a very narrow definition of what “information visualization” is, which in turn rejects the type of understanding that I think is absolutely necessary to promoting the field as a medium that anyone can use. This kind of work should be taken seriously when considering information visualization design “for the masses.” I’m not arguing that its design principles should replace the ones that Steve talks about, but I’m completely certain that the two perspectives are not mutually exclusive; the reality, I think, is that both could benefit from a more robust understanding of one another.

So, in the end, Steve and I may be talking about two different things. As I stated at the beginning of this post, I certainly agree with most of the principles he employs to promote more effective design of business-related information visualization. What troubled me about his capstone at InfoVis, and a lot of the other discussions there, was perhaps that this was as far as his conception of “infovis for the masses” seemed to go. I applaud him for a lot of what he tried to convey to the audience there (they needed to hear it!), but I would suggest that “the masses,” to me and many others, includes a much wider population of users – most of whom are not “data analysts” of any sort.