TiVo Awarded Patent For Closed Captions On A DVR
TiVo doesn’t know whether or not their injunction against Dish’s DVR will hold up yet, but that hasn’t stopped them from adding to their patent portfolio in the meantime. In a remarkable filing with the USPTO, TiVo appears to have now won an important patent for displaying closed caption information to DVR customers.
In addition to covering this important feature, TiVo’s latest addition to their portfolio, also appears to encompass enhanced TV services, including a “clip and sling” type technology, that could eventually allow TiVo users to automatically remove commercials from time shifted programs.
According to patent 7,661,121, TiVo now owns the right to use existing closed caption and Enhanced Television (ETV) signaling data to create an interactive experience for their customers. ETV data is metadata that content owners have started to embed into their programming. It’s been used by Cablelabs and is part of the fundamental architecture behind big cable’s sinking “Canoe” DVR advertising venture. While I would suspect that the cable companies also have patents related to how ETV data can be used, it will undoubtedly be another series of rapids that the long delayed project will have to maneuver through.
While the abstract for TiVo’s latest patent is a little vague, if you delve into the details, you start to understand why TiVo would try to seize this particular piece of intellectual property. Essentially, the patent allows TiVo to sync closed caption information (and metadata) from broadcast programs recorded on a DVR and then display that data in an interactive format. This data can be as simple as a menu or closed captioned text or can be as advanced as digital video and sound effects.
From the patent,
“A multimedia device may use closed-caption data patterns to recognize and synchronize to multimedia content streams. The types of data patterns available in closed-caption data are numerous. For instance, distinct data patterns may exist within the actual closed-caption text, the closed-caption control data, as well as well as any other event defined by the closed-caption data. By recognizing distinct patterns within the closed-caption data, a DVR may identify events within the multimedia content stream. One way of recognizing patterns within closed-caption data is by computing hash values representing closed-caption text and identifying patterns of hash value sequences. Thus, according to one embodiment, at a multimedia device, such as a DVR or server, the closed-caption data is parsed and hash values are generated corresponding to the closed-caption data. The hash values are then compiled into hash value sequences associated with particular video programs or segments, and further combined with metadata defining command and control information for processing at multimedia devices. These hash sequences and metadata are provided to multimedia devices such as DVRs in the form of hash value sequence data. The multimedia devices use the hash value sequence data for recognizing and synchronizing to closed-caption data. A matching algorithm is used by the multimedia device to sequentially compare generated hash values from the closed caption data with multiple hash sequences that the multimedia device has stored locally. According to one embodiment, the matching algorithm is implemented through a state machine that processes the generated hash values and reports whether or not a match has occurred with a hash sequence identified by the hash value sequence data. ”
While the patent was only awarded on February 9th, TiVo actually implemented part of this technology years ago. Not only do they already offer a number of different ways to view closed caption data, but if you’ve ever watched a commercial on TiVo from one of their corporate partners, you’ve probably noticed a thumbs up icon that lets you easily subscribe to a show or order more product information.
While TiVo’s current implementation of this technology is admittedly pretty limited, the patent hints that there may be more powerful features going forward.
“”A user can mark off sections of a multimedia program or place points of interest relating to content within the multimedia program. For example, a user may want to mark the best plays of a recording of a football game. Once the user marks the plays, he can send the resulting hash sequences to a friend’s DVR.” [Bold added by me]
One of the reasons why Dish was never able to launch their clip and sling technology was because the content owners threw a hissy fit and threatened to sue the bejesus out them, if they made it easy for consumers to share content. While I don’t know the technical details behind what Sling was trying to accomplish, my sense of the project was that they wanted to let users edit other people’s content and then redistribute digital copies of those clips. The genius behind TiVo’s method is that their method wouldn’t allow anyone to “share content” that wasn’t entirely owned by the customer or TiVo.
“DVR users can distribute their own sets of points of interest for programs to other users. Users can further attach metadata to each point of interest that may cause the DVR to display text to the viewer, e.g., “Isn’t this a great action scene?” The user may also attach metadata to a point of interest that tells the DVR to skip x seconds into the program from that point of interest or display x seconds of the program before skipping to the next point of interest. This allows users to create their own condensed versions of a program that they can distribute to their friends, family, classmates, students, interest group, etc.”
While TiVo uses the example of a clipping highlights from a football game in their patent filing, I’m much more interested in how TiVo customers could potentially use this technology to remove commercials from YOUR programs. If both you and I have already recorded a particular show, there is no copyright violation because we’re both recording content that we already own. By allowing their users to create hash tag data, TiVo would technically own that data and would have the right to distributed that hash tag data to other DVRs without having to worry about content owners accusing them of stealing.
This would make it easy for me (or more likely someone else since I time shift everything 
) to easily tag all the commercials in a program and TiVo would then know to auto-skip past the content (commercials) when it saw the tags. TiVo could also use this as a way for live subscribers to tag the end of football and basketball games, so that they wouldn’t get cut off if the game went to overtime.
Another interesting embodiment of this patent would be the combination of live information with time shifted programming. Whenever I watch something a few months old, I always see commercials for upcoming TV shows or movies that are way past their expiration date. Instead of advertisers wasting their money on DVR subscribers, they could use sponsored hash tags to replace an old ad with something more current. Alternatively, if you were time shifting the news, TiVo could use your internet connection to create a live scrolling ticker that could update you on any new developments in the story.
Other potential uses that appear to be covered by the patent would include shows that have choose your own adventure type storylines. Upset about Chuck and Sarah not hooking up? the producers could give fans an alternate storyline to explore and allow viewers to vote on how they want to see the story move forward or TiVo could use this patent to create Blind Date type pop-ups around recorded television. While I tend to prefer my TV clutter free, for events like the state of the union, I can see why people would be interested in having fun facts pop up, addressing the issues that are being discussed.
While we haven’t seen any of these implementations take place today, the mere fact that TiVo was thinking about these options when they filed the patent would seem to suggest that they’ve been quietly innovating behind the scenes. This new patent award won’t necessarily help them in their case against Verizon or AT&T, but it could offer the telcos yet another reason to settle their dispute with TiVo, instead of being forced to place limits on the future of TV.
Posted on February 11th, 2010 by Davis
Filed under: Disclosure - I own stock in co. mentioned, SA, Slingbox, TV, TiVo
he problem is none of the people on the patent actually invented what is claimed. I did. When everyone thought this was impossible, I explained in detail to Barton, the CTO, how to do it. He actually got energized enough to start coming to work and created a team to implement it. And apparently took credit for the invention. Lovely.
Hi Paul, thanks for sharing things from your perspective, but while I’ve no doubt that your work added to the TiVo experience, I think it’s a pretty bold statement to claim that you singlehandedly invented the technology listed in this patent, especially when TiVo filed the application six years after you left the company. Certainly some of your work at TiVo created a foundation for what is claimed, but very few technologies are created in a bubble and to ignore the contribution that other members of the TiVo team made to this patent (and the underlying technology that I hope we’ll see in use one day) devaulues the work that others did alongside of you. I can understand if you would feel sleighted if your contribution was ignored, but certainly you can acknowledge that many other people helped make TiVo what it is today? While I wasn’t there to know who invented what, I think that to credit this patent to anyone other than TiVo is unfair to all of the employees who do show up day after day so that they can make television better for the rest of us.
Davis, did you ever work at TiVo?
Not unless you count beta testing.
I think you are naive about TiVo, patents and how inventions happen.
The problem with making TiVo interactive was embedding meta-data in a broadcast video stream without having it stripped by broadcasters and MSOs. My insight was that line 21 is always passed through and TiVo could use the blank unused portion of closed caption to embed discrete tagged sequences of characters that Tivo could intercept and act upon them. Kevin et. al. wrote the code that implemented this and we started ‘tagging’ NBC promos in 2000. You see this in commercials when “Thumbs Up” suddenly appears, prompting you to record the show or get more info. Before I drew it all out on Barton’s white board it was regarded as impossible.
And my issue is not that Kevin, Paul or Jim are listed, but that I was not. Without me, the principle claims of the invention would not exist. Unless you are arguing that they have tried to get around their own prior art.
Paul, Actually this doesn’t sound like the VBI tagging used for the promos. Instead of inserting TiVo-specific content into line 21, this patent refers to using data that is already there – CC & ETV – and hashing that data to produce a unique signature. So you’d be able to mark sections of *any* program, without needing to insert the flag ahead of time. So this does seem to be different from TiVo’s own prior art. I think Davis was in error when he referred to the promo system in the TiVo today – as that uses the Line 21 tagging and not this hashing system.
What the patent covers may be different or could be the same, but in the filing TiVo includes a screenshot of the tagging system that they currently use as part of the application.