Skip to main content

Tracking Hulu's viewership

The NY Times is running a piece on the difficulty of counting the number of unique visitors to streaming video sites, like Hulu. Apparently, Nielsen's estimates of the total viewership for Hulu are far lower than that of other measurement firms.

Nielsen seems to be extending the model they use for tracking TV viewership to online video. In this case, they monitor the online viewing habits of some 200,000 users and weight them to extrapolate the total viewership of the site. This is clearly error-prone and potentially leads to substantial bias in the reported results.

I don't understand why Hulu does not simply track the number of unique users to their site directly, using IP addresses and cookies -- just like most other websites. Hulu should have all of the information it needs at its fingertips; why get Nielsen in the loop? I recognize that tracking IPs and cookies not a perfect approach, but I am surprised that Hulu needs to rely on a third party ratings company for its metrics when there are well-established technical solutions out there. The only guess I have is that it's to ensure that the ratings numbers can be independently verified, but there must be a better way than tracking viewership of a relatively tiny slice of the population. Can someone explain this?

Comments

  1. I'm sure Hulu has exact numbers, but independent traffic reporting is very important for advertisers. One of the better ways is through a toolbar, like what ComScore uses, but all of these mechanisms have bias and it can be very easy to miscount. Personally, I think the Nielson method is totally outdated and doesn't even work for regular television.

    ReplyDelete
  2. Yes, I work in this space, and it's all about third party measurement. Nobody cares about Hulu's self reported numbers if they're trying to tell whether the ads they paid for are showing to the right amount of eyeballs. The problem is not a technical one.

    ReplyDelete
  3. That's what I assumed. What's interesting is that the NY Times article never mentioned anything about Hulu's own estimate of their viewership (just estimates from two third-party companies). There must be a good way of generating verifiable host-side usage statistics that doesn't rely on a third party observer.

    ReplyDelete
  4. Here's a straw man approach: have Hulu run a middlebox that uses a trusted platform module to attest to a small piece of software that just does the logging of streams and produces reports signed with a key sealed under the platform configuration registers corresponding to that software. Advertisers can audit the software to make sure it does the right thing, and they can query the middlebox whenever they want to have the TPM attest that the box is running this software.

    Additionally, have Hulu post a bond for $10MM that forfeits to charity if any evidence of physical tampering with the middlebox is found. Have an auditor paid by the advertisers perform random inspections.

    What obstacles stand in the way of doing this?

    ReplyDelete
  5. So the box would have to sit between the user and Hulu servers for every bit of traffic, adding a hop?

    It seems more reasonable for akamai (or any other content distribution network) to build in traffic reporting, and if the site agrees, release those numbers or an estimate using those numbers to determine total traffic.

    ReplyDelete

Post a Comment

Popular posts from this blog

Why I'm leaving Harvard

The word is out that I have decided to resign my tenured faculty job at Harvard to remain at Google. Obviously this will be a big change in my career, and one that I have spent a tremendous amount of time mulling over the last few months.

Rather than let rumors spread about the reasons for my move, I think I should be pretty direct in explaining my thinking here.

I should say first of all that I'm not leaving because of any problems with Harvard. On the contrary, I love Harvard, and will miss it a lot. The computer science faculty are absolutely top-notch, and the students are the best a professor could ever hope to work with. It is a fantastic environment, very supportive, and full of great people. They were crazy enough to give me tenure, and I feel no small pang of guilt for leaving now. I joined Harvard because it offered the opportunity to make a big impact on a great department at an important school, and I have no regrets about my decision to go there eight years ago. But m…

Rewriting a large production system in Go

My team at Google is wrapping up an effort to rewrite a large production system (almost) entirely in Go. I say "almost" because one component of the system -- a library for transcoding between image formats -- works perfectly well in C++, so we decided to leave it as-is. But the rest of the system is 100% Go, not just wrappers to existing modules in C++ or another language. It's been a fun experience and I thought I'd share some lessons learned.

Why rewrite?

The first question we must answer is why we considered a rewrite in the first place. When we started this project, we adopted an existing C++ based system, which had been developed over the course of a couple of years by two of our sister teams at Google. It's a good system and does its job remarkably well. However, it has been used in several different projects with vastly different goals, leading to a nontrivial accretion of cruft. Over time, it became apparent that for us to continue to innovate rapidly wo…

Running a software team at Google

I'm often asked what my job is like at Google since I left academia. I guess going from tenured professor to software engineer sounds like a big step down. Job titles aside, I'm much happier and more productive in my new role than I was in the 8 years at Harvard, though there are actually a lot of similarities between being a professor and running a software team.

I lead a team at Google's Seattle office which is responsible for a range of projects in the mobile web performance area (for more background on my team's work see my earlier blog post on the topic). One of our projects is the recently-announced data compression proxy support in Chrome Mobile. We also work on the PageSpeed suite of technologies, specifically focusing on mobile web optimization, as well as a bunch of other cool stuff that I can't talk about just yet.

My official job title is just "software engineer," which is the most common (and coveted) role at Google. (I say "coveted&quo…