Regressive JPEGs: how a whole video hides inside one image file

A post on maurycyz.com shows how the progressive mode of the JPEG format — the feature that makes a partially downloaded image appear as a low-resolution preview instead of a cut-off mess — can be turned into a way of storing a whole short video, or several different pictures, inside a single file.

How progressive scans work

Progressive JPEG splits the compressed data into a series of scans, each with its own header. The first scan carries only the lowest (DC) Fourier bin of all three colour channels, which is enough to draw a very low-resolution preview of the picture. The eight scans that follow fill in the rest: extra luminance detail first, then full low-precision data, and finally the last missing bits. Colour is stored in YCbCr at half resolution, so the complete chrominance data of an image weighs about as much as its luminance alone.

Overwriting a picture while it loads

Because every scan declares its own spectral range, it is possible to build a JPEG whose later scans overwrite image data that has already been rendered. The construction is crude: concatenate several JPEGs of the same resolution and delete the start-of-image, start-of-frame and end-of-image markers, which the author did with a short C program. Served over a slow connection, such a file visibly morphs from one picture into another as it downloads. Most decoders, however, stop after a certain number of scans — apparently as a defence against decompression-bomb style attacks — which limits the naive approach to roughly nine frames.

Ninety frames in one file

To pack more frames in, each frame has to be as small as possible. A progressive scan cannot contain AC (bins above zero) and DC data at the same time, and the baseline mode stops after its first scan, so the smallest legal progressive JPEG is a single DC-only scan. Since the transform runs on 16×16 blocks, such a frame is not a flat colour: it renders at one sixteenth of the original resolution. Chrome gives up after about ninety of them, and Firefox is more patient. The frames are standards-compliant and can be produced with the jpegtran tool by declaring a scans file.

There is no practical application: a JPEG carries no timing information, so playback speed depends entirely on network delay. The author notes the technique is mostly good for unconventional rickrolls, and links to related experiments — an HTML video built only from dialog elements, and an interactive single-page application with no CSS or JavaScript. The merge program is published next to the post.