One mandate of good information architecture is finding concrete, measurable answers. So, my objectivity radar goes off when I see people trying to slip questions about aesthetics or subjective preferences into a round of Web site usability testing.  It’s against best practice, but it’s not as uncommon as you might think.

Companies want (and sometimes need) user feedback to validate visual design decisions. As a result, they often resort to a seemingly straightforward approach – simply asking usability participants which design they prefer and why.

But if you’ve ever tried it, you know that the answer isn’t always as easy as asking the question. In fact, asking the question often results in contradictory findings, confusing recommendations…and, ultimately, lots and lots of noise. Many users don’t know why a design makes them feel a certain way, and those that do often struggle to articulate it. The few who can provide reasons often introduce feedback too subjective to wager a business decision on.

A recent article at UXmatters.com addresses the practice of asking usability participants which visual design option they prefer. The author shares my belief that such feedback creates more noise than it does clarity — but he proposes an interesting alternative and offers a case study on how the method played out at his company.

The approach they followed was based on a white paper by Joey Benedek and Trish Miner of Microsoft. Summarized, the process would be something like this:

• In the creative brief, identify the top three to five descriptors that the creative design would ideally invoke.
• After information design is complete, prepare multiple creative concepts that include Web page comps with the same layout and content but differing aesthetics.
• Arrange one group of test participants for each visual concept you intend to test.
• Develop a set of adjectives that participants might use to describe the site. Be sure that these adjectives are words your participants will understand, are salient to your research, and include a mix of descriptions that people might consider positive (around 60%) and negative (around 40%). Microsoft used nearly 120 descriptors in their testing.
• Ask each participant to view a comp, then select the three to five of these adjectives they think best described it.
• Ask them why they selected each of the words they did.
• Track the responses.

I’ve often been faced with this research dilemma, but I have never found the solution– so I’m eager to try this method. Have you ever tried a test technique similar to this one? Tell us about your experience in the comments.

Editor’s note: this is part of a series of Point/Counterpoint entries that present differing views on a single topic.

I can relate to concerns about the viability of Twitter for all personal users. Although I was an early adopter of the technology, nearly a year of slow use passed before I understood its value. I think there’s a standard ramp-up for Twitter that takes time and interest to gain steam.

In fact, here’s my weekly tweet frequency over the ages. Betcha can’t tell which week I was at SXSW!?

Despite the sometimes slow engagement ramp and high rate of attrition, Twitter is still a meaningful way for users to engage with content and each other, distinct from blogs, RSS and Facebook in a few key ways.

Tweets are, in fact, “microblogs.” But the character limit, single protocol for account creation and singular starting point (twitter.com) open Twitter up to a whole new fleet of potential content contributors (and readers) who are not cut out for the blog world: those who don’t have the patience, discipline or narcissism for full-fledged blogging.

Twitter is distinct from RSS in that while it does function as a real-time feed, it also offers community, two-way conversation and one particular type of content. RSS is a powerful way to harness and filter all of the content we consume, yet it remains a burdensome technology for some, with low adoption rates (<10%) for all but the most tech-savvy users. Twitter is actually one way of filtering the firehose of information that RSS can be; by following the right people, you’ll often find a nicely curated selection of content, mixed with just the right touch of voyeurism. (“Great article. Oh, and you like Cheerios? Awesome.”)

As for the role of Twitter alongside Facebook, that’s definitely evolving (note the changes Facebook has made in an effort to keep up). While tweets and status updates might have lots in common (and are sometimes the same), there is a more open social protocol in the Twittersphere. In contrast to the abyss of shared personal information that Facebook strives to be, Twitter is defined by brevity and openness. In lieu of multi-tiered privacy controls, Twitter offers two options: private, or public (which, not surprisingly, is also the default.)

The changes that Twitter made to their homepage this week take Twitter even further into a space of differentiation. The new homepage is intended to make the value proposition more clear to new users. The prominent offer to “Share and discover what’s happening right now, anywhere in the world” distances the platform from the traditional “social networking” space (think: Facebook/MySpace) by reducing the emphasis on friends or followings. Instead, the site now underscores the most compelling differentiators of the platform – immediacy, sharing and discovery.

While Twitter may not be for everybody, it’s proving itself to be a unique and powerful medium — a real-time reflection of the collective consciousness and a relatively big reward for both user and reader for their 140-character investment.

Although the Manhattan weather was grey and drizzly, bright discussion abounded late last month inside the Hyatt Grand Hotel, where I got the chance to attend the 2009 Forrester Customer Experience Forum.

I’ve whittled a week of great insights down to a few top takeaways about the state of customer experience today.

• The pendulum is swinging. The correlation between customer satisfaction and loyalty has always been high, but has increased in recent months. Studies show that consumers crave human connection now more than ever. Companies who treat their customers like just a number may not realize that it is reflected, somewhat ironically, in their numbers. Bruce Tempkin, Vice President at Forrester, shared this slide, which shows the jaw-dropping revenue potential that modest customer experience improvements could bring to a $10 billion company.*

• You do know who I am, right? Smart companies realize that consolidated customer interaction data across all touch-points (phone number, e-mail address, physical address) is necessary to provide convenience and personalization that tomorrow’s customers will expect.
• Heed the voice of the consumer. Companies who think they already know what their customers need are often sorely mistaken. And there is a growing variety of ways to uncover your customer’s unmet needs. Popular “listening posts” include call center data, sales reps, user surveys, usability tests, and error reports, but monitoring the social web introduces new listening opportunities. David Cush, CEO of Virgin America reported that customers who tweet from the air about bad experiences are often met at the gate by representatives empowered to resolve the situation. Follow @VirginAmerica to see Virgin’s Twitter account in action.
• Authenticity precedes love. Just like in the dating world, pretending to be who you think your prospect wants you to be is a big, fat turnoff. Only through identifying exactly who your organization actually is and being true to that identity can your company create authentic connections, which are rare and highly desired by the consumer.
• What’s rewarded is accomplished. Achieving a better customer experience requires a clear shared vision and accountability throughout the organization. Hiring mechanisms that help the company identify the most passionate, engaged employees is one part of achieving this. Setting project metrics and employee goals around customer experience are also great ways to cultivate internal ownership. T. Michael Roberts successfully lobbied to get new deliverables into required project documentation at JP Morgan so that the customer experience is better protected as new initiatives roll out. He also requires his employees to put in time monthly to listen to customer service calls. After all, he says, every service call reflects an unmet customer need.

A big thanks Forrester for an insightful week in NY.

See pictures from the event on Flickr.

*Prediction based on data from the North American Technographics® Customer Experience Online Survey conducted in Q4 2008.