Transparent videos

Starting with version 1.0 beta 17, PinballY is capable of showing videos with "alpha" transparency information embedded in the video, as well as animated GIF files with transparency.

Alpha transparency is probably a familiar concept if you've done any work with image editors. Image formats like PNG can represent partially transparent pixels, so that one image can be superimposed on top of another one, with areas of the background image showing through areas of the top image.

There's a separate kind of transparency used in GIF images, where a pixel color in the image is designated as the "invisible ink" color. All pixels in the image that match the designated invisible ink color are drawn as though there were completely transparent, and all other pixels are opaque. This is less sophisticated than alpha transparency, since it only allows an all-or-nothing kind of transparency, whereas alpha allows for continuous gradations of partial transparency.

Where can transparent videos be used in PinballY?

Quick answer: transparent videos can be used anywhere videos can be used.

PinballY uses transparent rendering for all videos that contain alpha channel information, regardless of context. Alpha transparency is a property of the video itself, so PinballY simply uses alpha for a video if the video has alpha information in the first place.

This applies to all videos, including the background videos in the playfield window and other windows. However, it's not particularly useful for background videos, because there's nothing "behind" them to show through the transparent areas in the video - just a black background. Any completely transparent areas in a background video will thus be filled with black, and partially transparent areas will effectively look like they're shown at reduced brightness, as they'll be mixed proportionally with the black background.

Transparency is therefore really only useful for videos that are layered on top of other display areas. Currently, PinballY uses layered videos in the following contexts:

Animated GIFs

The most popular current "video" format with transparency support isn't really a video format at all: it's the GIF image file format. GIF files support a simple sort of animation where a single GIF file can store a series of image frames that are played back sequentially on a timer. It's not exactly a "video" format in the usual sense, and it's not useful for anything lengthy because it doesn't compress well, but it's widely used on the Web for creating animated icons and short video loops. GIF also has a simple form of transparency, so it can be used for animated overlays - just the sort of thing you'd use alpha-channel video for.

Starting in 1.0 beta 17, PinballY supports animated GIF files, including transparent animated GIFs. You can use an animated GIF anywhere a regular still image can be used.

It might seem a bit strange, but note that all GIF files are considered image files in PinballY, never video files - even when animated. That's sometimes important when you're deciding where to store a particular media file. The rule is really simple: if it's a GIF file, treat it like an image file. Always put GIF files in the PinballY media folders designated for image files; don't put GIF files in video folders.

Supported video formats with alpha channels

As of this writing, alpha channel support in the standard video formats is quite limited. It's not supported at all in the most common formats and codecs, including MP4 and H.264. (That's not because of PinballY limitations or libvlc limitations - it's inherent in the formats.)

The only video file formats that currently support alpha appear to be MOV (Apple's QuickTime video format), FLV (Adobe Flash Video), and AVI (Windows). However, that's hardly the full story, because you also have to use a codec that supports alpha. Here's a list of what works now:

Note that "PNG" in this context is a video codec, not a file format. (It does happen to use the PNG image format to encode each frame within the video, which is why the name is reused in this somewhat confusing way.) A PNG-encoded video stream can be stored in an AVI or FLV file. You can encode videos this way with ffmpeg, for example.

PNG and QTRLE are usable only for very short clips, because they use lossless encodings that result in extremely poor compression ratios compared to what we're accustomed to with modern video codecs like MP4 or H.264. So at the moment, the ProRes YUVA 4:4:4:4 format is really the only good option for full video. Fortunately, ProRes seems to be a high-quality format with great alpha support, so there's at least one good option.

This list might eventually grow, and it sure seems primed for growth given how sparse it is, but I actually don't see it growing quickly or soon. Alpha channel support in video seems to be a difficult technical problem that's of little interest to the video engineering community. It doesn't seem that anyone except the Apple ProRes group has given it much thought. (That makes ProRes a curious anomaly: computer technologies usually develop incrementally, with a series of false starts and half measures before a good solution appears. But in this case, it seems like no one else was even trying, and along comes Apple to just solve it for real.)

Creating alpha channel video from chroma-keyed ("green screen") material

Even though alpha channel support in video formats seems to be a work in progress that's barely begun, the concept of motion overlays is hardly new. It goes back to film-based special effects. Traditionally, layered film and video composition was handled with an entirely different process, called chroma keying. This is also known as the "green screen" or "blue screen" technique, because it's widely used in film and video productions to shoot a live subject against a green or blue backdrop that's replaced in the final editing process with a special effects background.

In chroma keying, there isn't a separate "alpha" channel that records which pixels are opaque and which are transparent. Instead, you simply treat all pixels that match the key color as transparent. Every pixel that matches the key color is transparent, and the rest are opaque.

The reason I mention chroma keying is that it's been used for a long time in film and video production, so there's a lot of stock footage on the Web that uses this technique. If you're looking for material to use for overlay videos, you'll probably find lots of chroma-keyed videos and few videos using an alpha-channel format. But PinballY can only use alpha-channel videos, so chroma-keyed videos will have to be converted. That's something that FFMPEG (the popular open-source video converter program) can accomplish. Here's a fairly simple procedure you can use: