Mars Curiosity Rover: Boing Boing's $2.5 billion dollar question about image file types, answered by JPL

Photo: Two of the first images transmitted back by Curiosity, as seen on monitors at JPL 20 minutes after the rover landed on Mars. (Xeni Jardin)

NASA's Jet Propulsion Laboratory was a magical place to be last night, as engineers, flight specialists, NASA administrators, space celebrities, and scientists from many fields gathered to witness the landing of the Mars Science Laboratory (MSL) Curiosity rover. Those seven minutes of terror ended in a picture-perfect landing: an amazing machine went through a crazy Rube Goldbergian descent sequence, and plopped down about two meters away from its planned destination on the Red Planet's surface.

We witnessed history. It seemed impossible. It was awesome.

I sat in on the post-landing press conference, and live-tweeted the evening at @boingboing. During the press conference, after the high-fives and screams of joy subsided, I asked MSL engineer Adam Steltzner a question about those first two all-important thumbnail images Curiosity sent back—critical because the data they contained would tell NASA if the rover had touched down in a safe spot.

[Video of that Q&A moment here.]

Given the great distance and technical challenges involved in transmitting timely data back from Mars, what file type and image compression algorithm(s) did they use for those first "rush" thumbnails? There's a 14 minute delay involved for any signals from Mars to Earth.

A dorky question, perhaps, but I was curious, and figured nobody else would ask. Things like, "Hey how do you guys feel right now," and "What will Curiosity do next," I knew others would tackle.

Mr. Steltzner didn't have details handy about the image file types used, and he referred me to Mars mission image specialist Justin Maki. Today I checked in with Mr. Maki and his JPL colleagues whose work focuses on data compression and interplanetary data transmission. Here's what I learned.

What space reporter Bill Harwood recounts on SpaceFlightNow is what I witnessed right there in the press room, too:

While engineers did not expect pictures right away, blurry low-resolution thumbnails from the rover's rear hazard avoidance cameras were transmitted within minutes of touchdown showing a wheel on the surface of Mars.

"Odyssey data is still strong," Chen reported. "Odyssey is nice and high in the sky. At this time we're standing by for images..."

"We've got thumbnails," someone said.

"We are wheels down on Mars!" Chen reported.

"Oh my God," someone said in the background.

They arrived faster than any of us in the audience expected, and they were of great significance in the moment. So how did they get to us? JPL imaging specialist Justin Maki, tells Boing Boing:

The images are wavelet-compressed, much like JPEG 2000. The main difference is that the algorithm used on MSL (and MER) use is computationally less complex than JPEG-2000.

The compression software was written at JPL by Aaron Kiely and Matt Klimesh.

Matt tells Boing Boing:

I don't have much to add beyond Justin's answer. It is a custom file format and the compression algorithm is in many ways similar to the algorithm for JPEG-2000 compression, but with lower computational complexity.

No name for the format (and I wouldn't necessarily characterize it as proprietary), but we call the compressor "ICER" (not an acronym, just a rearrangement of the letters of "Rice"; the Rice algorithm is a data compression algorithm first used decades ago).

And Boing Boing reader Darryl Lee points us to this JPL document (PDF) which contains specs for the lower-res cameras used on the MER rovers, Spirit and Opportunity.

"Curiosity has a much higher-resolution camera (MastCam), but the photographs sent last night were from the Hazcams, which are much lower-resolution," Darryl correctly notes.


3.2. Image Compression

[48] To maximize the number of images acquired during the mission, virtually all image data will be compressed by the rover CPU (using either lossy or lossless compression) prior to placement into the telemetry stream. To perform this task the rovers will utilize a software implementation of the JPL-developed ICER wavelet-based image compressor [Kiely and Klimesh, 2003], capable of providing lossy and lossless compression. In cases where lossless compression is desired and speed is particularly important, compression will be performed (in software) by a modified version of the low-complexity (LOCO) lossless image compression algorithm [Klimesh et al., 2001; Weinberger et al., 1996]. The MER mission is utilizing state of the art image compression technology by flying compressors that deliver compression effectiveness comparable to that achieved by the JPEG- 2000 image compression standard [Adams, 2001], but with lower computational complexity [Kiely and Klimesh, 2003].

There's more details about ICER and LOCO in there, too.

And there you have it.

More about Curiosity in Boing Boing's archives.


  1. Those first two thumbnails came so fast I hadn’t finished posting my first SQUEEEEE tweet before we had imagery! Even with lo-fi thumbnails, I think it’s incredibly important because we are a species built for these cues.

    All the hoopin’ and hollerin’ that came after the initial landing was nearly duplicated when those first images appeared. Even the tech folk wanted to see something RIGHT AWAY! Where in past missions, we would have to wait hours or days before the first images, having the MSL send images literally within seconds of landing marks an important difference with those earlier mission.

    1. Agree about the pictures.  There was also a real-timeyness to the whole thing because the information stream was continuous right through the touchdown confirmation.  In the past landings that I remember, the first exciting event was a re-establishment of comms that had been blacked out during the re-entry.  “Oh, by the way, I landed a while back.  Check back tomorrow for a picture.”

          1. So let me see if I can struggle to follow you, bumblebeeeeeee.  You’re saying that, because of something having to do with the speed of light, the data from Curiosity was not immediately presented to us, but was in fact delayed in some way?

  2. Was excited to see you in the audience last night Xeni! You had one of those huge grins of excitement that were darn common there.. was sad they didn’t have an answer, but thanks for posting the follow-up.

  3. Saw the presser, thought Xeni’s was a great (and relevant) question.

    I’m curious what others think about something else:  I thought the flag waving  seemed a little more like, well, chest thumping.  It’s an odd reversal for me because I’m normally all about flag waving (military, Boy Scout, yada yada, ) but these science events don’t seem like the right time to me.  “We came in peace for all mankind,” after all.

    Excepting, of course, the first moon landing.  We totally kicked ass there.

    1. I think it’s a bit of this – we’re running somewhere around 40% success rate as ‘humanity’ on Mars landings – the US is getting up around 90% now with Curiosity safely down. Sure, it’s for everyone, but these guys managed to do it. Pretty consistently even.

      1. I guess I figure everyone knows it’s a U.S. mission, and the accomplishment should speak for itself.  The presser could focus on the amazing technical accomplishment, not the greatness of the country that did it.

        Shorter: “Look at this great thing we did,” rather than, “Look how great we are because we did this thing.”

          1. I thought the President’s Science Advisor did a good job of quietly noting the USA’s dominance in this area, considering the rising star of China in space travel and the Olympics.

          2. true, but it is a dominance built from a previous economic cycle. will this current economic meltdown allow china to take over ?
            i think so.
            and they have the shoulders of giants to stand on, so to speak.

    2. I found it sad that I wondered the same thing. Was a little worried that folks would find a way to ding Obama for “claiming responsibility”.

      But I think we might just be jaded by the jingoism in the service of Bad Things which is so common. This, however, is demonstrably a Good Thing. That was a US agency that pulled this off, supported by (most of) congress and the president, and payed for by US tax dollars. If there was ever a reason to be patriotic and flag-waving, this would be it.

      1. And it’s a good time to draw people towards sports and science and away from a focus on military dominance

    3. As an Australian, I can say that the only time I’m NOT sick of hearing about the might and power of the US is when you land stuff on distant astral bodies.

      You pull off shit like that landing, then you be as cocky as you like about it, with my full blessing. You earned it.

      On a side note, as an Australian, I’m really glad Michael Phelps has retired.

    4.  I think the jingoism was specifically intended to mollify the members of the U.S. congress who control the budget. We (aka globalists, science geeks) were not the intended audience for those remarks.

      It’s part of the cost of doing business in an ossified democracy.

  4. and this is why I read Boing Boing… this very question crossed my mind.  Thank You!
    I figured the 3 year wait for a layered PSD file might be a bit much

  5. “They arrived faster than expected”
    I am having some trouble understanding this… Knowing the time to snap, compress and send the photo (I assume at the speed of light) shouldn’t they have known exactly when the first photo would arrive?

    How could they time and calculate everything else so perfectly, but not this?

    1. I’ve edited for clarity. I have no idea what the JPL guys and gals expected, but they arrived a hell of a lot faster than any of us in the press room expected.

    2. shouldn’t they have known exactly when the first photo would arrive?

      Does Curiosity actually transmit an hour glass, or does that just reside in the processor on the vehicle?

    3. If I remember correctly, they were worried the orbiter that’s relaying the images over UHF would transit behind Mars from our perspective… they knew they’d be able to get probably one thumbnail, two was a win.

    4.  The rover doesn’t have a high bandwidth long range transmitter, instead it sent the data to the Odyssey (a spacecraft that is orbiting Mars). Odyssey then sends the images to earth using it’s high bandwidth connection to relay the signal to earth. When they where landing the Odyssey was just visible over the horizon and so they didn’t know if they where going to be able to get an immediate signal and how strong it would be. Those two factors established how fast the images got back to earth.

      1. Exactery.  It’s possible that just by a fluke, with Curiosity landing down range a little bit, that it had a good view of Odyssey for a few seconds longer than expected.  We’ve learned from past missions to always be landing with mouth and ears fully open, in case something goes wrong we have scooped up as much data as we possibly can.

  6. Great question – I love to find out what the tech behind obvious problems like this is about. Xeni – write us a book all about curiosity tech!

  7. I have watched space missions ever since the middle 60’s.  My Dad became an Astrophysicist because of Sputnik.  I sent as many links to as many people I know to watch this incredible achievement on so many levels.  And much thanks to Xeni and the humans at Boing Boing for getting such a large fan base so that such moments don’t go unnoticed.

    However, I wish that such events were covered live like in the early days with Walter C. at the controls.  I think it would make people aware of the tremendous accomplishments that groups of humans are capable of doing.  And who knows maybe some other kid will be motivated to become an astrophysicist, instead of a infamous vacuum formed reality show creature.

  8. Xeni, excellent work on this!  And honestly, I want to know more about the data transmission techniques used for something like this so please, keep on asking questions like this.

  9.  I was getting tired of weeping because of the Olympics. Now I have a Mars Car to weep over.

  10. saw your question live last night. Didn’t understand it ’cause i have a small brain or something

    but…. love the hair, Xeni!!! And you got a nice compliment from Adam Steltzner ! Woah!

    1. Age vacillations again! How about just the Urusei Yatsura where they summon a cab dancing in a circle with ‘Ventura, Ventura, Space People…’ to shave a decade, if not Iczer 7, Mos Def, or works of a Takahashi who uses more CG? 

      They’re just working out their fluid head pan and lens clearing across the 200-270 K range, before they fire up 20mw of 4320p encoding, drop thumpers, and see if the dead city runes open a path for the cryomotes, then go mudding in the sentient mic grease. At that point they can put it in a piping bag and spell out ‘o hai–i can has metal acid clathrate drainage?’ You should see the XSGML for the piping bag being on-message…

    2.  Get the soundtrack from Due South and you’ll be able to hear the original Canadian band performing it :-)   Meanwhile, the Mars orbiter was on the top of the world, looking down on creation, and ….

  11. Oooh! How about a question about their error correction algorithm next? That’s got to be a pretty noisy channel.

  12. Cue a copyright lawsuit for infringing JPEG compression IP in 3… 2… 1…

    Awesome moment, made even more special when better when the pics popped up so quickly.

    A whole lotta geeks hereabouts are jealous as hell of you being there, though the web feed and live simulation was a pretty decent alternative for us Aussies in the office gobbling peanuts and bandwidth in equal volumes. Nice work all round.

    Nice to see the Geeks putting one over on the Jocks at the Olympics.

  13. I was falling asleep in front of the TV after the landing when I heard Xeni’s unmistakable voice and was snapped back awake. I was wishing she would ask a big juicy nerdy question, and was not disappointed! Even if it wasn’t answered right away, thanks Xeni!

  14. Makes me Extra happy I jumped in on the Kickstarter for that NASA MMO, I Really wish it was out already!

    1. That is a good point @boingboing-7cf498e42c1a1d06960376beb34127bc:disqus and I wondered about it. They show an ellipse-designated “estimated landing region” and the rover is within that ellipse. I wondered if they “landed exactly where they wanted to” by simply landing in the ellipse.

  15. Oh man—you do NOT want to transmit a full-res, layered Photoshop-formatted file from mars. Not only does that take forever, but odds are good the email server for whoever you’re sending it to will have a 20mb size limit. I’m glad they built that rover smart enough to use the “save for web” option.

  16. Thanks Boing Boing for the heads up about the landing.  And thanks Xeni for being there.  It was such a nice surprise to see you and made me feel connected through my Boing Boing  “family”.
    I can’t remember how I first found Boing Boing, but I’ve been hooked for the past several years.  You’ve upped the ‘cool’ quotient a lot for this 64 year old woman.

    1. Hang on a sec – it seems you didn’t need Boing Boing to make you cool; your handle is already grokstar.

      I’m not sure how much cooler than that you can get…

  17. Yes more tech info, please.  How about screenshots of each of those techie’s screens.  What were/are they seeing?  With annotations – eg this is the “speed of descent, a very important value since we need to have to slow down or we crash” or whatever.

  18. Neat.

    I spent an all-too-brief summer before my last year of college working on the USGS Astrogeology Team’s ISIS project, which involved image processing for the Lunar Lander, Mars Orbiter, and suchlike.  (It was right when Cassini entere Saturn orbit — an exciting time to be working there!)

    Unfortunately I don’t remember much about compression specifics; I was mostly doing grunt work on makefiles to make sure that old-ass Fortran and C code would compile across various platforms.  Still, great just to be on the team; best job I ever had.

  19. I was happy to see you get chosen to ask a question; wasn’t sure what you’d say but was pleasantly dork-surprised by your question and was disappointed they didn’t have a good answer (though understandable, that’s not really their department I guess). 

    You, and that question, perfectly represented BoingBoing (of course) but also happy mutants in general and all the other geek havens on the internet :)

    p.s.: it is imperative that you see this

  20. they didn’t tell the exact compression/format of the image because of the fact the anybody from the globe can intercept the signal coming and going to mars.. to connect it to the format of the pictures, the signal intercepted will be analyzed like the format of the picture and its possible that if you can decode the signals, you can recreated your own signals that will let you manipulate the rover and read the signals that it is sending back to earth.. 

    For an example, a official online game is out in the internet and someone wants to make an emulator of the game server, they only need to sniff the packets of the connection from the game client to gameserver and vice versa and then decode it using a format that can be read. So they can now create server emulation. In some ways its relatively the same concept of hacking.

  21. Going by hairstyles it looks like a lot of the MSL operations team are fans of the Golden era of California the 50s -60s.
    I tweeted a couple of questions to Mr. Seltzner What’s his favorite Band? What’s his favorite Car? I expect something like Beach Boy’s and ’57 Chevy hot rod. Too stereotypical? If he doesn’t (understandably) reply maybe you can ask him sometime?

Comments are closed.