Sunday, November 25, 2007

continuing the discussion with Stephen Few - my response

Before responding to Steve’s points, I want to take a step back and clarify that my original intent in critiquing his InfoVis capstone presentation was not to attack him personally, but rather his position on the topic of “infovis for the masses.” Unfortunately, it may have come off that way, and Steve’s responses to my issues, phrased as responses to “Few does X,” may further personalize the criticism, so I want to emphasize that this is ultimately a discussion about approaches to understanding infovis “for the masses,” and not some kind of flame war. As Steve suggested, I think it is clear that we both care a great deal about the “state of information visualization” today, so I apologize, particularly to Steve, if any part of this conversation has seemed or seems personal. It most certainly isn’t intended to be. I also want to point out that I completely agree with a good deal of the “low level” design issues Steve rails against (problematic use of color, charts designed so poorly that it is actually impossible to read values off of them, etc.); That’s not really what I’m taking issue with in this conversation. Again, I’m more interested in conceptualizing infovis design “for the masses.” So, that said: onwards and upwards!

Rather than responding point-by-point to Steve’s post, I will try to address what I see as the underlying disconnect in our positions. This disconnect, I think, revolves around our differing conceptions of the scope of information visualization, as well as our differing definitions of certain key terms. I’ll elaborate from some of the statements he makes in his response:


What is perhaps not obvious, based on my capstone presentation alone, is
the fact that I spend a fair amount of time trying to understand what draws
people to ineffective visualizations—those that fail to serve the needs of the
audience while managing to appeal to that audience on some level.

[…]

Many-Eyes has managed to make the process of data exploration and analysis interesting and fun, without resorting to features that undermine the effectiveness of the activity.

[…]

I hope that I’m never guilty of writing off meaningful aspects of visualization as useless, simply because they don’t match my own preferences—aesthetic or otherwise. If you ever catch me doing so, I want you to call me on it. Just make sure that what you point out as useful in a
visualization is actually useful and not just appealing to your own preferences.

[…]

When a great deal of evidence indicates that certain visualization practices work better than others, I believe that it’s helpful to teach people to follow the best practices and avoid those that fail.

[…]

I define the “right way” as the way that best satisfies the needs of people—the way that works. I’m a pragmatist. What I don’t do is define the “right way” as the way that people desire things to be done. Our desires, our notions of how things should be done, often conflict with the way that really works.

[…]

The real “disservice to the goal of popularizing information visualization” is the existence of (1) ineffective or irrelevant infovis projects and products that represent our work poorly, and (2) the unfortunate inability of many experts in the field to present their work to those who need it in a way that they can relate to, care about, and understand.



What makes these statements problematic to me, and what was part of what I was trying to get at in my original critique, are Steve’s definitions of terms like “useful,” “(in)effective,” and “[user] needs,” with regard to information visualization. If I were to try to identify, right off the bat, the fundamental disconnect between the two “camps” that Steve and I represent, it would be that we don’t necessarily agree on what these terms mean.

Steve, coming from the area of business intelligence, presumably values the clarity of the data above all else, which I think is a perfectly reasonable position given the “needs” of that field. I’m not totally familiar with the world of business intelligence, but the use of information graphics and visualization in that area is undoubtedly motivated by a need to understand the meaning of quantitative data and make critical decisions based on that understanding. So, for him, having the data presented as plainly, directly, and efficiently as possible is the prime consideration in the design of their visualizations. Anything that distracts from that is considered ineffective or not useful.

My contention is that this sort of traditional, “scientific” understanding of information visualization, while certainly valuable in some domains (such as business intelligence), is too restrictive when considering its broader, more popular uses. For one thing, there is the obvious point that “popular visualization” does not necessarily share the same critical goals. Many of the infovis examples that Steve criticized from the Smashing Magazine article exemplify this, in that they present information that is not necessarily “mission critical” in the same way BI information might be – people are not necessarily viewing these visualizations because they need to make critical decisions based on the meaning of the data they present. Rather, they are perhaps more “casual” forms of information visualization in which directness and efficiency of transmission are not the primary goal, which then complicates our conception of “usefulness” or “effectiveness.”

Now, I am the first to admit that, at this moment, I don’t have formal definitions of these terms that stand in opposition to the ones applied in Steve’s conception of infovis, but work to define them is starting to emerge: at InfoVis, we saw Zack Pousman’s (et al.) presentation and paper on “casual infovis,” Martin Wattenberg’s discussion of “vernacular visualization,” Fernanda Viegas’ description of the many expected uses of Many Eyes, Ola Rosling’s emphasis on visualization as storytelling, Brent Fitzgerald on Swivel as “big (social) database in the sky,” and Robert Kosara suggesting new uses for infovis (I’m sure I’m leaving someone out, but these are some I just blogged about). Across the internet, the discussion continues: Andrew Vande Moere, Andrea Lau, and Nick Cawthon ponder information aesthetics, the guys at Stamen Design discuss “useless” visualization, as does Matthew Hurst of Microsoft Live Labs… the list goes on!

What bothers me about Steve’s position, which I sensed mirrored in a lot of the discussion at InfoVis, is the suggestion that this new research, or the type of visualization that it describes, is somehow “not infovis.” While it’s true that infovis, as a field, grew out of a “strict” scientific tradition (ie. computer science) that informs its theories and methodologies, it is going to have to broaden its understanding of the ways in which “normal” people interact with information if it wants to present itself as accessible to the masses. I think the field will need to start thinking more in terms of “engagement design” rather than the highly quantified metrics of efficiency, time on task, etc., that have traditionally characterized user studies in HCI and interface design. Those metrics may make sense (and I think give rise to, at least in part, the “great deal of evidence” supporting best practices that Steve refers to) when describing visualization usage by users who spend their lives (and make their livelihoods by) working with charts, graphs, and other visualizations, but almost certainly don’t capture what is important about the way “the masses,” as I define them, use information visualization. For their purposes, engagement (another ambiguous term) should perhaps trump efficiency; for instance, maybe a particular design should sacrifice some of the “directness” of the data for other elements, such as “playfulness,” that encourage continued use. I can understand why Steve and others might be skeptical of this sort of approach to infovis, as there doesn’t yet exist a large body of research support it, but I would argue that this is because we haven’t really starting thinking about visualization in these terms yet, rather than because there is something fundamentally wrong with it. I’m concerned that the traditional infovis field believes too strongly that their understanding of their domain is the final word on information visualization. This is a problem.

Steve himself admitted that, if given a choice of visualization design, most users would always pick the “flashier” one. I’ll just reiterate that instead of assuming that “flashier” equals “less effective,” we should try hard to understand the value of “flashiness” (or “novelty,” or “aesthetics,” or whatever we want to call it) against the expanded definitions of “useful” and “effective” I’ve referenced here. At his tutorial session at InfoVis, I made the (admittedly somewhat outlandish) comparison to Jim Cramer and his highly-rated CNBC program, Mad Money. Cramer uses an almost ridiculous amount of (literally) bells-and-whistles to “decorate” his presentation of financial information, and yet the information gets across. Is it presented as thoroughly as it might be by a more Spartan “talking head” presentation? Probably not, but the show’s popularity says something about what engages non-experts. Coming back to the realm of infovis, there is something undeniably important and valuable about the fact that visualization examples such as Jonathan Harris’ “We Feel Fine,” or Stamen Design’s visualizations of Digg.com, or, yes, the Ambient Orb, are popular and engaging. What’s interesting about Many Eyes, an example Steve does like, is that it supports many types of representation, from traditional analytic uses to the more unusual ones that Fernanda described in her presentation at InfoVis. Arguing that the mere existence of these sorts of projects undermines information visualization is a real problem. It implicitly assumes a very narrow definition of what “information visualization” is, which in turn rejects the type of understanding that I think is absolutely necessary to promoting the field as a medium that anyone can use. This kind of work should be taken seriously when considering information visualization design “for the masses.” I’m not arguing that its design principles should replace the ones that Steve talks about, but I’m completely certain that the two perspectives are not mutually exclusive; the reality, I think, is that both could benefit from a more robust understanding of one another.

So, in the end, Steve and I may be talking about two different things. As I stated at the beginning of this post, I certainly agree with most of the principles he employs to promote more effective design of business-related information visualization. What troubled me about his capstone at InfoVis, and a lot of the other discussions there, was perhaps that this was as far as his conception of “infovis for the masses” seemed to go. I applaud him for a lot of what he tried to convey to the audience there (they needed to hear it!), but I would suggest that “the masses,” to me and many others, includes a much wider population of users – most of whom are not “data analysts” of any sort.

8 comments:

Robert Kosara said...
This comment has been removed by the author.
Robert Kosara said...

Sorry about the previous comment. I didn't realize that logging in would post what I had written thus far.

You make some good points, and I largely agree - but you also have to give us some time. This is the first year that this topic has been on the radar at all, and it already was the dominating theme throughout the conference. Not everybody will enthusiastically embrace this new idea, and many of those that do don't yet have a good idea where the limits of InfoVis are, and how to deal with this new thing.

I think that over time, different foci will develop, like InfoVis for analysis/visual analytics, InfoVis for presentation and communication, InfoVis for ambient information, etc. We will have to figure out how to evaluate and compare methods if not by task completion time and accuracy (and I agree that there are definitely other relevant criteria). What is more, we will have to decide what trade-offs we are willing to accept for different types of visualization.

In any case, there will have to be discussions about these things, and there will have to be limits. But those will be very helpful for the field to develop, and perhaps there will also be more of a continuum between the InfoVis and CHI communities - which would make a lot of sense especially for this topic, IMHO.

Unknown said...

I think this is a very worthwhile discussion, and if you don't mind, let me add a few thoughts.

First, let me say I actually really enjoyed Stephen's presentation. Having attended this conference for many years, I think it served a very useful purpose which was to educate many of us about what is going on out there in the real world. Despite all our enthusiasm for our subject, information visualization is still underappreciated and not practiced well by the majority of users. And most software designers are not tuned in to the subtle distinctions that both of you are making.

Having said, I actually agree with Mike's points. I think there are many dimensions by which to judge visualizations. And even those that seem very subjective can lead to measurable and important effects. For example,

o Aesthetics - Attractive things are perceived as more usable than unattractive things. Everyone has heard of Noam Tractinsky's studies as popularized by Don Norman in his recent book Emotion and Design. Similar results have been found by Byron Reeves and Cliff Nass. What is effective is often affective.

o Style - Style is an important communication mechanism. Style communicates subtle contextual cues that are absent from minimalist graphics. They brand the graphics and in the process tells us something about who is saying what. The importance of subtle contextual cues are recognized by any strong communicator trained in rhetoric. Egoless designs without styles lose this important communication chennel. In fact, the minimalist style that Stephen practices, says a lot about who he is and what he stands for.

o Playfulness is important. Playfulness encourages experimentation and exploration. Active exploration will lead to better understanding than passive viewing. The fun and engaging interactive visualizations created by Martin Wattenberg have a big effect on how those using them understand the data.

o Vividness of a graphic can cause an image to be much more memorable. This is one of the standard complaints about Tufte's minimal approach. Icons don't convey much information about a file, but they do speed up visual search.

Stephen's quest for the principles of effective design is an important one, and is underappreciated by many practitioners. But a modernist, minimalist approach removes a lot of richness of visual representation which makes the field so fascinating. We need to raise the level of discourse to the level in your blogs in order to make progress.

Stephen Few said...

Pat,

To the degree that aesthetics, style, playfulness, and vividness support the purpose of informing, I enthusiastically support them as worthwhile aspects of information visualization. It is only when they are applied in ways that undermine the user's understanding of the information that I find them harmful. The effectiveness of an information visualization can only be judged in light of its objectives, and the primary objective should be to inform. If a visualization has a different primary objective, perhaps it should be called something other than an "information visualization."

I'm familiar with Don Norman's argument for the utility of aesthetically pleasing design, which I accept, and I have reason to believe that style as useful, based on my general design studies, but I'm not familiar offhand with any experimental evidence that playfulness or vividness help to inform. Are you aware of studies that support this? I am very open to the possibility, but prefer to see the evidence and understand how these factors work before advocating the practice.

Regarding vividness in particular, I advocate the use of vivid icons to draw attention to particular items in a display, such as particular metrics on a dashboard. Vividness is a useful way to make something stand out and thus easier to find, but making everything in a display vivid serves no purpose that I'm aware of and can actually undermine a visualization, because when everything stands out, nothing in particular stands out. Also, overly vivid displays, such as those filled with fully saturated primary colors, are not aesthetically pleasing.

Steve

mike_d said...

Thanks for the comments, everyone! I'm really pleased that there's a lot of interest in this discussion. And as Robert suggests, I think this is an exciting time to be thinking about infovis.

I have a couple of follow-up thoughts:

Pat, I'm intrigued by the categories you suggest, as they remind me how difficult it is to "quantify" these sorts of elements of visualization design. For instance, I hadn't really considered "style" (except maybe as an application of the principles of graphic design) and "vividness" as something separate from "aesthetics," but you bring up great points with those categorizations. I'm really interested in your idea of "branding" effects in visualization.

Similarly, the point you make about icons is important as well. I think we actually spoke about this in person at InfoVis at one point, but this references the idea of considering visualization design semiotically. In many cases, it seems like infovis systems are designed to be data-agnostic; I see the value in this, but for a broader audience I would argue that having elements of the visual representation strongly coupled to the semantic content of the data is useful as well, if for no other reason than it ostensibly reduces the "abstractness" of the visuals. Some of the New York Times graphics shown at the InfoVis keynote seemed to support this, where small photographs and icons were embedded within the graphics to remind readers what they were about.

That said, again, these elements are difficult to quantify, and I completely understand Steve's concern about the lack of evidence to support their value. A huge part of what needs to be done in "popularizing" infovis, as many have suggested, is rethinking how we evaluate visualization systems.

In response to Steve, I agree with your concerns about how the application of these design principles might interfere with informing the user. However, I think that when designing "for the masses," you have two responsibilities: One is to inform, but the other is the sort of "meta" function of encouraging users to spend time with your visualization in the first place. If we are conceptualizing "the masses" in the way that I do, we are talking about users who almost certainly don't normally interact with visualization, and thus may need more "encouragement" to make them comfortable with the idea. This is kind of what I was getting at with my reference to "engagement design." On the surface level, this stuff could be considered "decoration" or "chartjunk" to be written off as distracting, but I think we can (and should) try to think of it as something to be more subtly balanced against our need to inform.

Anyways, thanks again for the comments, guys!

Ted Cuzzillo said...

Sorry if I’m chiming in a little late. I’ve been following this discussion, I wrote about it for a business intelligence publication and on my weblog, but I haven’t had enough chance to throw in my two cents here until now.

I agree with Stephen Few’s search for practical infovis standards. Every medium has them, so why not this one? The purpose is not to set down rules but to figure out the basics of what works. If enough infovis works, the medium will enter the mainstream sooner.

I’m a newcomer to infovis, but I’m not so new to journalism and marketing in the tech industry. Most people who write for a living are at least familiar with basic techniques, conventions and style.

In fact, every medium I’m familiar with has its standards. I find that I hear the same ones over and over in different forms. For example, I like the one from Chekhov about stage theater that says something like “if you show a shotgun in the first act, you have to shoot it by the third.” That is, tell the story economically, and don’t distract with junk.

I just want to follow the story. Some of what I saw at InfoVis and in Smashing Magazine left me wondering what the point was. If that’s OK with you, you look at it, but it won't do much to popularize infovis.

Perhaps all kinds of playfulness, richness, graphical vividness and style have their place as long as they tell a story well to the intended audience--and they don't turn off too many newcomers. Even the Ambient Orb might work in the way a jungle gym works for future gymnasts if no one mistakes it for the real thing.

Play! Experiment! But know that the business world will head for their Blackberries if if doesn't work.

TrendCompass said...

Dear Sir,

Epic Systems together with Beemode (www.beemode.com) have developed a Data Visualization software "Trend Compass" almost ready to be released soon. It is an extension to Gapminder which was invented by a Swedish Professor. You can view it :

- www.gapminder.org

We are looking to promote that software in various sectors. It is a new concept in viewing statistics and trends in an animated way. It could be used in presentation, analysis,research, decision making, etc.

Here is one link for part of what we did with some Governmental institution:
www.epicsyst.com/visual.swf

Here is another link for a project we did with Princeton University on US unemployment :

www.epicsyst.com/main3.swf

I hope you could evaluate it and give me your comments. So many ideas are there.

In a few days you can test the software by uploading data on our website and getting the corresponding Flash charts. This is for a limited number of users.

Regards.

Eng. Hisham Abdel Maguid
Managing Director
Epic Systems
www.epicsyst.com

Unknown said...

If you have read Stephen's books, you simply must read this article that questions his expertise in visualization...
www.fyivisual.com

Cheers,
Kim