How the Daily Show's PVRs work

A former Daily Show researcher has weighted in on this PVRBlog post, detailing the PVR setup used by the show:

When TiVo footage is needed for TDS that day (i.e., every day), the clips are dubbed off to Beta tape and brought to an editing bay. Yup, sneakernet. Sounds like a lot of work, right? It is. I wouldn't be surprised if the show upgrades to a networked PVR system -- especially with an imminent move to HD -- but I don't know what their plans are.
Re: "I would think if they are archiving all their footage" -- They're not. There are already services that do this for them. The show would rather pay for those services than pay for the equipment + staff necessary to reinvent the wheel. The show does have a vast tape library, much of it stock footage provided by the AP et al. -- all the stock footage tapes get saved and logged in a database available to everyone in the office.
But since the only way to save things under the current setup is to dub it off to Beta in real time, there is no way to archive all the footage. A lot of stuff does get saved to Beta, particularly major events that are likely to remain relevant. Yet it's not the News Clip Library of Alexandria that people might think.

The Daily Show and their TiVos

I write books. My latest is a YA science fiction novel called Homeland (it's the sequel to Little Brother). More books: Rapture of the Nerds (a novel, with Charlie Stross); With a Little Help (short stories); and The Great Big Beautiful Tomorrow (novella and nonfic). I speak all over the place and I tweet and tumble, too.

MORE: Copyfight • Gadgets

More at Boing Boing

Eurovision 2013: An American in London

The technology that links taxonomy and Star Trek

JFlex

*cough*

What’s PVR?
cherry shiva

not surprising at all. having worked for years in tv land, i can attest to the sluggish juggernaut of combined momentum, habit, investment, bureaucratic purchasing protocols, and union labor rules that is professional television.

it will eventually happen, and when it does let’s hope someone in charge is savvy enough to send those tapes to a recycling plant.

the electronic entertainment industry is a huge polluter.
i’ve been shocked for years at the disposal volume of components and media. i’ve tried encouraging my employers to recycle but the bottom line hasn’t made it much of an option so far.
jccalhoun

Interesting to hear from them. I would like to hear from them some reason why all the Stewart Daily Shows are online but none of the Craig Kilborn episodes are on.
wynneth

I second @17. I like Stewart, but I miss the Kilborn shows. WTH is he doing now anyway? He had late night then Ferguson got it. Is he just a film actor now?
eggbert

I can’t tell you the number of times people suggest “why don’t they just switch over to digital equipment”, etc. in almost any tv situation. Obviously they’ve never worked from the budgetary and management side of things. It takes a LONG time to plan and implement a digital transition. Not just how to carry it out and use it but pay for it. To switch over in a _professional_ environment, it isn’t just a stack of Tivos. Professional equipment that deals with anything using the mpeg codec costs a forture. You could easily spend half a million and still end up with an archive that is way too small to be practical. That sort of money doesn’t just appear because a few folks think its a good idea.
The Lizardman

@17 It wouldn’t be surprising if it had something to do with contracts
historyman68

Working in media (collegeish radio) after the station transitioned to digital, I learned the hard way that it’s possible, and sometimes easy, to erase years of data during upgrades. We’ve lost years of valuable production when we accidentally wiped the hard drive of one of our ProTools machines. Going with what works can be a great way to make sure it continues to work.
Adam Weiss

Betamax vs. VHS is a classic story of inferior technology winning market adoption over superior technology.

It’s far superior to VHS and is still used to this day in newsgathering. There are new digital formats that are starting to inch in on it’s turf, but Beta is still the king in news.
Drew from Zhrodague

#2 Tape isn’t really immutable, and I’d expect someone to pipe-up with more accurate information — I’m too lazy to look stuff up now. Having been a child of producers, I’ve heard many anecdotes about dropping a tape and causing the sound to change (with analog recordings, of course). Obviously the tape technology has changed over the years, but it doesn’t quite last 50 years. For instance, if you’re using a helical scan head technology (DAT, SVHS, etc), you can lose all of your playable signal if you’re the heads recording to tape are out of alignment with the heads used for playback. For DAT and MiniDV this makes the recording less than useless. I think analog video gets “warbly” IIRC.

I used to put Computerworld on CD a while back, so I know how some companies have the most bizarre and obtuse procedures for making even the poorest of excuses for product. I’ll spare y’all the painful details.

As a sysadmin, I’d say that cron/bash + ffmpeg/mencoder + whatever-OSS-encodes-and-burns-DVDs would save time, effort, and even automate such a simple procedure. You should be able to encode it into the final product while the DVD is burning. Should be able to do this on any BSD, Linux, or OSX spare computer in the old-stuff closet.

Which leads me to say that NetApp makes bitch’n online storage for almost every purpose, and I think ILM has the largest installation (Yay, Spinnaker!). Spend the bucks if you got ‘em, they’re quite worth it. Kernel RAID with Linux for the rest of us.
Jack

@#1 POSTED BY ZUZU , AUGUST 17, 2008 11:47 AM

How is it 2008 and professional television shows aren’t recording everything to a massive RAID array of hard drives? I mean, dubbing to tape, really???

Other people have said it as well, but most media monoliths are incredibly sluggish to adapt to new technology and so invested in the “capital” they pumped into that infrastructure they won’t toss it out unless there is a good profit/loss reason.

Also, TVâ€”and especially huge companies like Viacom which owns Comedy Centralâ€”have an unlimited pool of interns they can call on to run the “sneaker net”. So from a tech perspective it seems odd, but from a bottom line standard the cost of training a few underpaid (if paid at all) interns through the dubbing wringer is a lot less than buying new hardware and incurring a larger cost.

And frown on the process all you want, but The Daily Show is still funny and highly rated. Good tech is what works for you to get your job done; not the most “133t” tech.
jsz

having interned at citytv and cbc, i can safely say that these kinds of makeshift setups are in use in fairly prominent shows for all the reasons people have previously mentioned.

most editors feel a great unease in their gut when their media doesn’t exist anywhere physically.
Jack

Oh, and also, folks you’d be shocked at how many places still use AVID setups despite Final Cut clearly being the new standard.

We can also discuss PageMaker and QuarkXPress 3.3 still being used to this day in some print publications.
Nora

@15 Personal Video Recorder. And yeah, I had to look it up too– maybe in English next time, Cory? :)
Nelson.C

Nora @25: Yay, lazyweb! I knew someone would look it up for me if I waited long enough.

So is a PVR the same as a VCR?
Sean Grimm

Betamax? I didn’t know people still used that, but I also grew up using VHS. I never even heard of Beta until my teenage years, and all I knew was that it lost to VHS. Is Beta use common in television?
zuzu

How is it 2008 and professional television shows aren’t recording everything to a massive RAID array of hard drives? I mean, dubbing to tape, really???

As storage follows the accelerating rate of return, while even 1080p HDTV bitrates are likely to remain static (or DVB-S2 MPEG-4 are likely to compress more into the same bitrate), I’m rapidly looking forward to, as a home user, having many HD HomeRun network tuners and Dreambox satellite network tuners archiving “everything all of the time” (at least for a week long buffer + keepers).

As an individual I’m a few terabyte hard drives shy of running my own Daily Show or Meet The Press. So what’s holding back these full-time and commercially budgeted operations from long since having done the same?
Anonymous

Wow – broadcast geek news. Ha! I work in TV – for CBC News in the past, and for the Cdn Discovery channel now, and I can answer Zuzu’s question above in two words -ahem- “Daily broadcast”. The Daily Show is news – it’s new content generated daily (like regular news), and time is of the essence. And yeah – while a raid full of digitally-acquired material would save the dubbing time for sure, it doesn’t solve the pro0blem of technological reliability (ask ANY network how their tape-to-digital-media-server change over went. They’ll all say it sucked, but finally got its shit together).

And besides, tape is pure, immutable (sorta – you can drop it and it still works). It’s quick to work with, needs to rendering or other processing, and is cheap! Since we’re in that odd change-over time between SD and HD, you’ll see a lot of this kinda digital/analogue newschool/oldschool mutant approach. Personally – I love it.

Thanks for the story, Cory. (Ugh -shudder-) The rhyming… ack..

-g-
Doug Nelson

“an imminent move to HD”

Yay!
jccalhoun

Betamax (the consumer VCR format) should not be confused with Betacam (the professional format). The cassettes may look similar but they aren’t the same format.
justONEguy

@#1 I mean, dubbing to tape, really???

Tape, and Beta tape at that, is still the standard in many prod. houses. Having worked in a stock footage house, having dubs is a way to manage what goes out to who especially when you’re licensing footage.

Film archives are treading lightly on digital right now because of the inherent costs of eventual technology upgrades and overhauls. You can stick a 35mm print in cold storage and still have a good print 100 years later. Archiving stuff digitally is a crapshoot because of the rapid technological changes inherent in doing so.

Fox News recently went “all-digital” and I’m curious to see how that works for them in a few years.
pAULbOWEN

I should care because?
jccalhoun

I will say that this does explain why so often the clips on the Daily Show look like ass. I’ve always thought “man, they can’t get a better copy of that clip than that?” and now I know that no, they can’t. kind of sad.
zarl

Nobody cares that you don’t care.
Uncle Geo

“I’d expect someone to pipe-up with more accurate information”

OK, over morning coffee-

A PVR is a Personal Video Recorder

The Beta is indeed not the ill fated Betamax (analog standard def composite video), is it Betacam (analog component standard def video). Betamax lost in the consumer world, Betacam won for many years in the pro world.

Short to long in archival longevity
Recordable optical media (as little as 5-10 years)
Pressed optical media (CD’s DVDs you buy)
Hard Discs
Magnetic tape
Film (as long as 100 years)

Scrambled vs not scrambled
Often studios pay for rights so scrambling is not much of an issue as they get it directly from the source. Over the air or satellite news video is generally not scrambled.

Why not put everything to hard discs on servers?
-Faster, reliable drives capable of the sustained data throughput you need for video are more expensive than the drives you buy at Circuit City. Also more, smaller drives are preferred over fewer large drives (again for speed).
-Reliable software to manage those servers (clustered storage)is still very expensive.
-Video takes a ton of room and no one puts everything on hard drives. Media libraries, some going back almost 60 years, are still primarily tape, though other formats (DVDs, optical) find their way in nowadays. Many people still want something that can sit on a shelf.

Standards
The nice thing about standards is there are so many to choose from! While 1080i (or 720P or now 1080p or 525i) might be standards for display in your home, there are a whole zillion file formats for video.

Final Cut vs Avid
The story goes: Steve Jobs gets to NAB one year and sees, to his surprise, Avid running on NT. Apparently Avid chose to ignore the fact that Jobs owned a software company and that Jobs knew Avid was insanely expensive software running on Truevision NuVista cards. Final Cut was his thumb in their eye. Postscript -Avid was not on the NAB showfloor this year. The shift has been huge to Final Cut (and others) not only for price but also because you can’t wipe your ass without Avid asking for another 12,000 bucks.

Archiving
Archival software has to reliably write all the formats to servers, often while creating proxy video for database searching and perhaps transcoding to other file formats.

“most media monoliths are incredibly sluggish to adapt to new technology and so invested in the “capital” they pumped into that infrastructure they won’t toss it out unless there is a good profit/loss reason.”

Indeed. Video gear has been very expensive for a very long time and P&Ls are important. Video has been undergoing rapid change for years and you need to be very careful when you make investments of millions of dollars. Plus, HD has been a moving target since 1985 but everyone needs to eventually buy it. Broadcast HD gear is still very expensive and has only just begun to settle out a little (just in time for our economy to be in the shitter). Many production companies have put off buying HD for years waiting for the price to come down and formats to settle. HD must also be considered in any archiving plan.

With the proliferation of both “reality TV” and inexpensive, fairly nice cameras ($6K) that are accepted for “reality TV” broadcast, we see the competition like the print world saw when any Bozo could buy Quark and take a chunk of low end business from the pros.

Database
Ingesting all that footage is one huge undertaking but managing a database of that footage is a big effort requiring staff and a very comprehensive plan for tagging each video with relevant info so it can be reliably searched say 20 years from now.

What’s important? Whose in the video? What’s the subject? HD or cel phone video? Date, location… How many staffers do you need to service the archive? Who is going to digitize all that stuff? How much will all the gear cost? Will the companies that make the gear be around in ten years? Software like Pictron will use voice and image recognition to parse the video, fill in data for you (but you’ll still need to tweak) and create proxies and even serve it up online -but none of this is cheap. I.e., it ain’t the relative cheapness of hard drives that determines the cost of reliable archiving systems.

Reliability
…ain’t cheap!
mdhatter

sneakernet = my daily moment of zen.
Jake0748

I care.
ulor

@1

Dubbing to tape has a lot of benefits including being more stable. Also when you have to play something over and over again, but it may be days or weeks between plays. Do you want to keep all the video on a server for something thats not going to play for a month or just dump it to beta and then right before its going to air reingest it back into the system. In order to archive everything digitally would be beyond many smaller TV stations, but a Beta Digital Solution provides a good alternative. That being said I love the word sneakernet, reminds me of an article once about a guy who load up the backup tapes for his company in his stationwagon and drive across town since it was faster usually than download the relevant tapes. (This is slightly different than the original I am sure since I only sort of remember the article) Anyways digital is not always better.
Anonymous

Well the US uses wiered non-standard transmission systems like ATSC for which it is hard to get hard- and software. Plus they have a lot of stuff scrambled.

If there was unsrambled DVB in the US, the problem would be trivial. Just set up a VDR system and set timers for every channel and every hour. You’d have it as easy to backup files.

BTW, it’s not Betamax, it’s most likely one of the modern professional formats starting with beta.
Takuan

I’ll pretend to care for sex.
DanGoodwin

Hey Cory, you should also link to the New York Times story (“Is Jon Stewart the Most Trusted Man in America,” a solid piece) that the TiVo tidbit originates from.

I intern at Late Night with Conan O’Brien and am privileged enough to realize how a set-up like The Daily Shows is so mind-boggling. At Late Night we have three TiVo’s that are set-up to record every other late night talk show, several morning to afternoon shows and a few special events that happen now and again. Every morning we burn DVD copies of the previous day’s shows, a process that takes less then 2 hours, depending on the temperament of the recorders.

Most of the time we use the DVDs to see what future guests have done for other shows, what questions they’ve been asked and if they’ve done anything ripe for comment by Conan. Secondly, the DVDs are used to find clips (more precisely the time for when something occurs) that the writers have asked for. Once we’ve located a clip we can go to our video library and ask for the tape (ready to air on a HD tape). Other times we call up the services described in the post and get the tapes that way.

Since The Daily Show happens in its own offices I doubt that they have access to as vast a library as we do at NBC, Late Night is shot at 30 Rock along with MSNBC, CNBC, the local New York City Affiliate and more, but surprisingly I bet we have to go through as much red-tape as they do (even though everything is owned by the same company there are still fees to be paid and the gray areas that come hand-in-hand with parody).

I’m fairly certain that most other talk shows have a similar set-up, however I had always wondered about The Daily Show and The Colbert Report since they are much more clip intensive then the norm. To read that they have 15 TiVos at work was staggering.
Faceless

Stable, and a lot more cost effective. A media server like an Avid Unity isn’t a bad idea, but they’ve likely already invested in one just for editing, and it likely already has to be in a mirrored configuration that requires twice the amount of storage. You’d STILL end up backing all of your archived footage up to blu ray or something similar. In the end, it’s a huge expense for a small operation like the Daily Show, and for the 20 second clips they use, probably not something that’s high up on the priority list.
pAULbOWEN

Oh well thanks Zarl, that’s my week ruined.
Adam Weiss

I always expected some kind of custom setup built on a bunch of cheap Linux boxes that captures and indexes based on closed captions.

Hmm.. Putting that together would be a fun job!
Anonymous

I think S.Nelson said it best

“I work as an editor and replay operator on live sports broadcasts. Are standard DVR is a machine called an EVS-XT, it is essentially a high end TIVO with often over 30 hours of HD recording and playback. The machines are networkable and a companion device, the X-file, allows you to back up footage to an external hard drive that is searchable by many different tags and variables. It’s hard to believe that popular nationally televised show is relying on consumer products and Beta stock.”
ck

All right, credentials come up top, do they? Ok, I’ve been editing for 18 years, in NYC since 1994…

Comedy Central is notoriously cheap. Ask anyone in the editorial industry working in NY. When John Stewart jokes about basic cable, he’s being more honest than funny. Also, there’s big resistance to change in the post-production industry. It’s expensive and the technology changes too quickly to keep up. You really only upgrade when something breaks, the standards change, or there is a big demand for it.
Since TDS is its own entity, meaning they don’t have clients who are bringing in 7 different HD formats, they don’t need to upgrade yet…”If it ain’t broke, don’t fix it.”
Rodney

Wow, that’s extremely saddening, for some reason. They have HUGE army of fans who would record, upload, tag, transcribe, and otherwise make available, every second of broadcast television if they just asked for it. Instead they tape record shows. Old people alert!!