When FFMpeg uses an encoder, there are three important steps: This means image formats can be encoded fairly easily (without modifying too much existing FFMpeg code) and it also means, if you understand the various steps that FFMpeg takes to encode an image, you can fairly quickly read and understand an encoder. One reason FFMpeg is so powerful is that all encoders declare a struct (of type AVCodec) that defines the various steps necessary to translate an image into the desired format. This file describes a series of processing steps needed to take FFMpeg’s internal representation of video and translate it into a final GIF output. Since this bug seemed to be a problem with encoding (given individual pixels are broken, rather than the whole file), the GIF codec (defined in ) was a natural place to start searching. While we usually think of a file format (such as GIF) as telling us how an image is encoded, a format can actually specify several different methods for compressing and encoding image data. While this doesn’t solve our problem, it does tell us something important: this bug happens after all filtering, and occurs while converting to the final output. The stuck pixels were gone, but unfortunately the MP4 format does not support transparent pixels, and so every transparent pixel was set to a default color (green in this case). The emoji converted to MP4 and then a GIF. I started by converting the image to an MP4 file, and then converting it to GIF: ffmpeg -i in.gif -filter_complex "scale=100:-1, split palettegen paletteuse" out.mp4 & ffmpeg -y -i out.mp4 out.gif I decided to narrow down where this problem was originating before diving into the code. Using a nifty tool in the repo called graph2dot, I generated a plot of each step in the process: The FFmpeg codebase is large and complex, and the filter above is fairly complicated as well.
What was strange, however, was that non-transparent pixels were being affected. This indicated something was going wrong when processing transparent pixels. This bug only affected Stickers, which are GIFs with some transparent pixels. The buggy renditions were traced back to an FFmpeg command that we use to re-scale GIFs while retaining the original color palette: ffmpeg -i in.gif -filter_complex "scale=100:-1, split palettegen paletteuse" out.gif Unwilling to let the world go without pixel-perfect poop, I started hunting down the bug.
Notice that some parts of the image get stuck (eg, the top of this GIF)
0 kommentar(er)
