Category Archives: annotation

Videolyzing Pharmaceutical Ads

There are just two countries in the world where Direct-To-Consumer (DTC) advertising is allowed for pharmaceuticals: the US and New Zealand. The ostensible motivation? To educate consumers, to raise awareness of medical conditions, to get people talking to their doctors, or to reduce the stigma associated with certain conditions (e.g. Viagra)

Since the laws changed back in 1997 in the US opening the floodgates for big pharma to peddle their wares directly to patients, there has been a debate about the efficacy and value of DTC advertising. Even today the FDA lists several ongoing studies evaluating the understandability and effects of DTC advertising. But the debate is political too. Congress has recently started floating proposals to limit the marketing powers of pharmaceutical companies for the first 2 years after a drug has been approved by the FDA. This would give regulators additional time to evaluate a new drug’s broader risks once it were available on the market.

Drugs aren’t the only DTC advertising issue generating controversy either. DTC medical device advertising is already generating a debate about the ethics of advertising products to people that can’t possibly understand the medical risks and decisions necessary for a medical device implant.

This is not to mention that DTC could be pushing up the overall costs of health care by directing people toward brand name “designer” drugs that may not be any more effective than alternative treatments. Obama’s $1 billion stimulus funding for Comparative Effectiveness Research (CER) should help with this somewhat by doing real comparisons of which treatments are “worth it” both in $$$ and patient value.

But big pharma is big business. Huge sums of money are invested in pharameutical advertising ($5.2 Billion in 2007), with spending growing at an annual rate of about 20% from 1997 to 2005. And with huge returns on investment, who can blame big pharma for wanted to drive traffic for new drugs by going straight to the people who would need treatment. The birth-control pill, Yaz, increased its sales from $262 million in 2007 to  $616 million in 2008, utilizing a few high profile (and misleading) broadcast ads.

Misleading or inaccurate information could lead consumers to make poor health decisions, or take risks that they may not fully understand.
So how does the government keep consumers safe and pharmaceutical advertisers honest? Right now the process is managed by the FDA Division of Drug Marketing, Advertising, and Communications (DDMAC). Advertisers are required to submit promotional materials to the DDMAC when they are first used or published, but not before. This means the FDA’s role is purely to “check up on” what advertisers publish, ex post facto. Ads can be circulating for months before they are critiqued and evaluated. And if an ad is found to be misleading, the FDA sends a warning letter to the offender asking them to retract the ad. That’s it most of the time.

What does the FDA check? According to their website, “advertisements cannot be false or misleading or omit material facts. They also must present a fair balance between effectiveness and risk information. FDA has consistently required that appropriate communication of effectiveness information includes any significant limitations to product use.” They require that all drug advertisements contain information as a brief summary relating to side effects, contraindications, and effectiveness. For instance, the law states that, “an advertisement may be false, lacking in fair balance, or otherwise misleading if it: “Fails to present information relating to side effects and contraindications with a prominence and readability reasonably comparable with the presentation of information relating to effectiveness of the drug, taking into account all implementing factors such as typography, layout, contrast, headlines, paragraphing, white space, and any other techniques apt to achieve emphasis.” The FDA has a very specific set of guidlines for how ads can be used in the video domain as well; including different categories of ads such as “product-claim ads”, “reminder ads” and “information seeking” ads.

The current FDA procedures for the evaluation of DTC video (broadcast) ads are wholly unwieldy. They include the submission of TEN (!!!!) copies of an annotated storyboard with each sequentially numbered frame and associated annotated references and precribing information (PI) supporting claims. Isn’t there a better way to do this?

This got me thinking about how an application like Videolyzer, that I originally built as a tool for bloggers and journalists to critique and debate online video, could be used by someone like the FDA (or the pharma companies) to streamline and digitize the evaluation and sourcing of video advertisements. This is in addition to exisiting journalism outfits, like Consumer Reports Ad Watch, which could use the tool to add back context to an overly curt video advertisement. Yaz, a birth control pill marketed by Bayer gained notoriety in late 2008 for two ads that were deemed misleading by the FDA and for which they had to run corrective ads in 2009. I’ve added the original version of one of the Yaz ads to Videolyzer for anyone interested in seeing how the tool can be used to critique a pharaceutical ad.

Fact Checking Source Contextualization

I ran across this round-up of some of the most prominent Political Fact Checking sites online including non-partisan FactCheck, Politifact, and¬† Washington Post Fact Checker Blog, as well as the partisan counter-parts: Newsbusters and MediaMatters. One of my criticisms of such sites is that oftentimes the fact-checking is decontextualized from the orginal document, especially for multimedia such as video. The presentation is usually done as a block of text explaining the “fact” in question. But what’s missing is the context of the claim or statement within the original source document. A far more compelling information interface for this would be to present an annotated document so that segments of the document (or video) are precisely delineated and critiqued. This is something I worked into the Videolyzer for video and text, but more generally this type of thing needs to happen for all fact-checked texts online.

Music Recommendation & HerdIt

This week I had the chance to attend a tutorial at the ACM Conference on Multimedia on Music Recommender Systems presented by Oscar Celma. It was a very informative talk, touching on some of the foundational issues in music recommenders: relevancy, serendipity, transparency, and context. There was also some discussion of the tradeoffs between content based recommendations versus those made based on human added metadata. For instance, content based recommendations have the cross-genre problem of potentially recommending songs from a different genre which share some similar musical features. The assumption in the presentation is that this is bad, though in some sense, serendipity may call for some cross genre pollination.

I wanted to pick at a point that bothered me a bit: the tension between relevancy and serendipity. Relevancy on the one hand calls for a user centric model which takes into account how interesting a particular recommendation is for a particular user. Relevancy means that the recommendations made are in fact meaningful and perhaps “useful” or at least appreciated by the user. On the other hand the virtue of serendipity is espoused as something to strive for. The value judgement is that people shouldn’t be constrained to things they already know or are familiar with, but should also be exposed to things outside of their comfort zone. And music aside (especially in an information domain like politics) I think serendipity IS something major to strive for. But doesn’t this compete with relevancy for attention? A personalized / recommended news page that includes “serendipitous” results risks presenting results to the user that are in fact not relevant at all. I would have appreciated a more earest discussion of the tradeoffs between these factors.

There are various methods that commercial systems are using to make music recommendations. The two big ones discussed in this tutorial were and pandora. Pandora relies on an “army” of paid specialists who listen to each indexed some and rate it based on 400 attributes on a 10 point scale. This clearly cannot scale as there is simply too much labor involved in the process, but the result is impressive. Another tack on the problem is to produce these annotations using non-experts, something I’ve thought about in my design of games like PhotoPlay and AudioPuzzler. Some folks at UCSD have designed a social game for Facebook called HerdIt to try to collect affective music data in the process of people playing the game. The hope is that this data can inform machine algorithms and eventually produce better recommendation systems.

Videolyzer Alpha Online

Version of Videolyzer is now online! Videolyzer is a tool designed for journalists and bloggers to be able to collaboratively assess the information quality of a video, including its transcript. Information quality involves things like credibility, validity, and comprehensivness among other things.  Videolyzer was designed to support the analysis, collection, and sharing of criticisms of online videos and is initially geared toward politics. To try it out with some of the recent presidential debate content go to

Comment Press – Paragraph level comments

Comment Press is a WordPress plugin that allows for paragraph level commenting of texts and pages on the blog. Produced by the Institute for the Future of the Book, Comment Press is meant to facilitate collaboration around longer, more complex texts which require a finer degree of granularity in annotation and commenting.

The interface for comment press is fairly straightforward, the primary text runs along the left and the comment box along the right. The comment box tracks up and down as the page is scrolled. Small icons are associated with each paragraph of the main text and allow someone to click and apply a comment to just that paragraph; or an ueber icon allows the user to comment on the text as a whole. While this is quite an interesting improvement, I wonder why the paragraph was chosen as the optimal unit of commenting. Sure, it’s a good trade-off because paragraphs aren’t usually too long, but still, how would this look if people could really get down to the sentence or even the phrase or word level? To support the broadest range of texts and behaviors it seems like the user should be able to select an arbitrary length of text and then apply a comment at whatever level of granularity makes most sense. This of course greatly increases the complexity of the interface needed to display and browse what could become quite a messy set of annotations.

Timed Comments in Video

There’s a lot of interest from new video startups in making video into a first class web 2.0 citizen by bringing tagging, commenting, and responses to videos at a sub-video level of granularity. While the old skool video sites like YouTube, Revver, Metacafe, Magnify etc. let you add tags and comments to a video, the new breed of video services such as Viddler, YouTube Streams, and The Click facilitate timed tags and comments to video. There has also been some recent academic interest in how people can interact with one another through commenting and chatting around video. This CHI 2007 paper from CMU is a good first step in understanding chatting behavior around videos.

Today I took a closer look at one of the newer attempts at highly granular commentable video, viddler. Here’s a screenshot of their interface, which despite some great features also suffers from some usability issues. All in all the timeline is pretty good though, you can clearly see where people have left behind comments and an easy to understand “+” graphic allows the user to access a menu and select the addition of a comment, tag, or video response. The downside here is there is no time extent associated with the comments or tags, they are simply added as point anchors. At the same time, this does simplify the interface by not requiring the use to select and in and out point; ideally I think the point anchor should be extendable to cover a time period if the user so chooses.

One of the problems that the interface suffers from is that comments sometimes obscure the video. Although they can be expanded and contracted as necessary, this just seems tedious. Furthermore, to see all of the comments that have been added to the video (as a list), almost half of the video is obscured. Perhaps the goal is to have the comment track and video mutually exclusive as they distract from one another anyway? What is nice is the voting mechanism for comments which is used to determine which comment shows up while someone is watching if there are several comments or responses at that point in the video.

Viddler Screenshot