Lair Of The Multimedia Guru


FFmpegs Huffyuv

In the last few weeks FFmpegs huffyuv codec has grown the ability to encode and decode a much broader list of pixel formats. From planar rgb variants 4:4:4 YUV to 4:1:0 YUV and bit depths up to 16bit and alpha support.
One might ask why anyone cares about all this, the awnser is simple

1 thread YUV420 10bit, matrixbench
FFv1 Huffyuv
encode fps 63fps 281fps
decode fps 82fps 418fps
filesize 478mb 765mb

Encoding parameters used:
-an -threads 1 -pix_fmt yuv420p10le -strict -2 -vcodec ffvhuff -context 1
-an -threads 1 -pix_fmt yuv420p10le -strict -2 -vcodec ffv1

6 threads YUV420 10bit, matrixbench
FFv1 Huffyuv
encode fps 269fps 1173fps
decode fps 386fps 2675fps
filesize 480mb 771mb

Encoding parameters used:
-threads 6 -an -pix_fmt yuv420p10le -strict -2 -vcodec ffvhuff -pass 2 test.nut
-threads 6 -an -pix_fmt yuv420p10le -strict -2 -vcodec ffv1 -level 3 -slices 6 test1.nut

As can be seen above huffyuv while it doesnt even get close to ffv1 in terms of compression even though i used per frame huffman tables and 2pass mode in the 2 tests, its simply much faster.

Filed under: Entropy Coding,FFmpeg,VideoCoding — Michael @ 02:27



About a month ago we (the ffmpeg team) voted about which name our NGO to which people soon will be able to donate should have. A sideeffect of that is that the world now has one more application to count votes using various condorcet methods, borda count and instant runoff voting. Get it from svn:// with your favorite command line svn client while its still fresh ;).

The vote itself went pretty well, we had a huge participation and it was quite fun :). Less funny was that murphy hit us and the winning choices “FFmedia/Foundation for free multimedia” domain name was already taken by some anonymous person through domaimsbyporxy, who asked us for cash even to just forward a message. Luckily the 2nd most popular choice “FFMTech/Foundation for free multimedia technolgies” didnt had that problem and was just by 1 vote behind so we picked that.

About which voting method is best, i dont know, but for our vote at least they all produced the same result most of the time while people added their votes. In that sense i wonder if any vote in reality that used a condorcet method actually ever needed rules beyond condorcet like the popular schulze method? Anyone knows? i checked debians past votes but it seemed they all had a condorcet winner

Filed under: FFmpeg — Michael @ 01:03


Little Reverse engeneering puzzle

There are SVQ3 files that ffmpeg cannot decode yet, the ones iam speaking of contain a image or watermark, often a logo in a global header (extradata in ffmpeg, QT has its own funky terminology for it). The binary decoder displays this watermark over the actually decoded video.

FFmpeg is in principle fully capable to decode these videos (and of course without these watermarks), the only problem is that 32bit of the header and following bitstream are modified by xoring them with a per file constant. Our problem is we do not know how the binary decoder calculates this constant. A example SVQ3 video and the corresponding constant of 0xA2A2A2A2 for this file as well as a bugreport on our tracker exist as well

The puzzle is to figure out how the binary decoder finds the constant, its likely not very hard for one knowing how to use a debugger, hint: memory breakpoint at the input data and a cup of coffee or tea or a can with cola.

Filed under: FFmpeg,Reverse Engineering — Michael @ 22:10


FFmpeg weekly news #4

Another week has passed in the ffmpeg universe.

  • Mans added -Bsymbolic, everyone wonders why this isnt default and what the ELF designers were smoking. Shared libav* becomes smaller, faster and more secure due to this change.
  • Mike fixes a security issue in the mov/mp4 demuxer. Noone writes a security advisory, like always.
  • Dave Yeo restores OS/2 support
  • RV30/40 work by kostya
  • VC1 improvments by kostya
  • removing globals used in API by me and mans (due to issues with -Bsymbolic)
  • export sha1 implementation in libavutil to the world (actually we just forgot to install the header) by Diego Pettenò
  • Various h264 fixes by jeff
  • Make mov useable over http by elupus and baptiste
  • Support overriding codec id for input files by me
  • Generic global header support for mov by baptiste
  • User adjustable dynamic range compression for AC3 by justin
  • djgpp support by Michael Kostylev
  • PC Paintbrush PCX image decoder by ivo
  • Sun Rasterfile decoder by ivo
  • Extract aspect ratio from ODML-avis by me
  • TechnoTrend PVA Demuxer by ivo
  • support removing elements from the AVL tree in libavutil and make it independant of malloc() by me
  • improve CRC API and install crc.h (another forgotten header) by aurel
  • various request_channels related fixes by justin
  • Linux Media Labs MPEG-4 (LMLM4) demuxer by ivo
  • support for libdc1394 v.2 by Alessandro Sappia
  • Do not install rtp.h (which should have never been part of the public API) by luca#1
  • User specifyable maximum amount of memory to use for the index. by paul kelly
  • libav-user was created by root
  • Add support for H.264 video in the RTP muxer by luca#1
  • ugly hack to support multicodec stsd in mov by me
  • subtitle support in mov improvmets by me and reimar
  • support for Matroska attachments by eugeni stepanov
  • ICC support by reimar
  • SMPTE 421 Annex L format demuxer by kostya
  • The monthly flames, accusations and insults, a vote about fixing warnings and me, baptiste and many others fixing many. Some gcc bugs in -Werror being found, all decoders being changed to have const input by me …
  • support for ogg text subtitles by reimar
  • mbaff spatial direct support by loren
  • many optimizations by loren
  • parallel regression tests by mans
  • various nut fixes by oded
  • support for speex in ogg by reimar
  • The first parts of libavfilter hit svn by bobby and vitor
  • MANY bugfixes, ive ignored most due to lazyness and i think its not that interresting for most readers anyway
  • Lots of cleanup by various people
  • myself loosing the ability to reliably capitalize words correctly as i move toward the end of this entry, also note, nothing has been spellchecked as its almost 6 in the morning here …
Filed under: FFmpeg — Michael @ 05:44


FFmpeg weekly news #3

What has happened in the last ehm week since the previous weekly news, well alot …

  • some ffserver-IPv6-linux fix by Nicolas George
  • some ffserver-IPv6-macosx fix by Ronald S. Bultje
  • flv files with invalid headers work again after a fix by me
  • a vorbis decoder crash after “floor0 dec: booknumber too high” has been fixed by me
  • make ffmpeg stop if writing fails instead of continuing by me
  • split adx into adxenc.c and adxdec.c by aurel
  • improvements to the mpeg-ps detection which fix several mp3 files which where misdetected as mpeg-ps, by me
  • make the NellyMoser decoder use our generic mdct, finding out how to do this and doing it by fabrice
  • Electronic Arts XAS ADPCM decoder by peter ross and aurel
  • Electronic Arts .cdata demuxer by peter ross and aurel
  • RICE2 entropy coding and variable block size for flac support by Josh Coalson
  • reorder codec registration to prefer native implementations by diego
  • Warn user if bitrate parameter is too low by ramiro (using 100 instead of 100k was a pretty common cause of silly bug reports …)
  • List enabled code in configure output by ramiro
  • Remove libvorbis Vorbis decoding support by diego (our vorbis decoder IS bugfree :) at least as far as we know)
  • fix interlaced_frame flag for h.264 by jeff downs and Reinhard Nissl
  • dont send hundreads of RTCP packets by luca abeni
  • adpcm-ima encoder bugfix by Timofei V. Bondarenko
  • some of the MMS patches by Björn Axelsson hit svn, still quite a few to go
  • various h.264 fixes by Jeff Downs
  • Musepack SV8 demuxer and decoder by kostya
  • a-law/mu-law/16bit PCM support in SDP by luca abeni
  • ipv6/ipv4 udp cleanup by luca abeni
  • MPEG2 in RTP muxer support by luca abeni
  • split audio and video grabbing code into libavdevice by luca abeni
  • split wmv2 in its own files by aurel
  • intrax8 decoder by “someone”, now finally we can decode all wmv2 files
  • export top_field_first flag for h.264 by Reinhard Nissl
  • our own ogg muxer written by Baptiste, now finally you can store your music and videos in the worst container ever designed without having to depend on libogg
  • remove libogg support by Baptiste
  • 44.1kHz support and various other small improvements for our nellymoser decoder by alex
  • flv v9 32bit pts support by alex
  • remove important functions from snow by diego
  • mpegts demuxer segfault fix by jeff
  • Optimize memory management and some cleanup of the rm demuxer by roberto
  • remove perror() usage and take meassures against its reintroduction by luca abeni
  • dynamically allocate ByteIOContext in AVFormatContext so changes to it dont break the ABI by Björn Axelsson
  • VC-1 MMX DSP functions by Christophe GISQUET
  • part of the RV30/RV40 patchset from kostya hit svn, the rest should likely hit svn soon as well assuming i and kostya arent too lazy
  • pcm_s16le_planar support for electronicarts files by aurel
  • some stuff needed for OS/2 support by Dave Yeo
  • split vc1dsp_mmx out into its own compilation unit by aurel
  • MLP/TrueHD parser by Ian Caulfield
  • pause/play/seek support for the protocol API by Björn Axelsson
  • wma sound artefact fix by reimar
  • change british english to amerikan by diego and others, yes we all hate british english :)
  • make our rm muxer generate files playable by realplayer by kostya, i guess this once worked already in the past …
  • dnxhd 720p encoding and decoding support by baptiste
  • improvments to the mp3 detection by me
  • fix asf muxer so that asf files work better on win ce by me
  • Adpcm_swf regressions tests by benjamin
  • several segfault fixes in the mov demuxer by baptiste and takis
  • Many small fixes by various people ive been too lazy to list

Missing things, duplicated entries and wrong entries as well as spelling errors are unintended, if you find anything missing/wrong/duplicated tell me and ill fix it, dont bother telling me that every second word contains a typo i do know that already :)

Filed under: FFmpeg — Michael @ 01:54


FFmpeg weekly news #2

After the extreemly popular recent changes in ffmpeg blog post, here are the next news, precissely 1 well something ;) afterwards

Note, errors and ommissions are unintended and if you spot any tell me and ill fix them (except spelling and grammer i wont bother fixing these)

  • More work and patches by ronald and luca to get the IPv6 related code closer to being functional
  • diego removed Metrowerks Codewarrior support due to it being unmaintained, broken and making the code more ugly (also not a single person complained so there is likely no one using it)
  • Altivec runtime detection fix/cleanup and messup
  • The remaining H.264 PAFF patches from jeff downs hit svn, sadly this also caused a slowdown of the decoder, so this needs more work to prevent this slowdown
  • reimar removed the extreemly broken SIGILL based cpu extension detection code, the SIGILL code should have never been commited in the first place …
  • luca barbato added support for a user specifyable maximum number of frames per RTP packet
  • a optimized VP3 IDCT for blackfin by marc hoffman
  • mans changed configure to use pr instead of cat -n as later is not standard, this will likely cause some minor annoyance to some users of non standard systems well fix your system, dont expect every lib and application to workaround it!
  • a DNxHD (SMPTE VC-3) encoder by baptiste
  • DNxHD 10 bit depth and 36mbit decoding support by baptiste
  • user specifyable zlib compression level for PNG by reimar
  • ogg seeking simplification and bugfix by reimar
  • a very small part of the MMS patch hit svn
  • some minor h.264 optimization by jeff downs
  • infinite loop and negtive memcpy in the ac3 and aac parser fix by myself
  • RC4, DES and encrypted asf support by reimar
  • VP6 with huffman encoded blocks support by aurel
  • deblocking for H.264 PAFF fix by Martin Zlomek
  • nellymoser ASAO decoder by a840bda5870ba11f19698ff6eb9581dfb0f95fa5, 539459aeb7d425140b62a3ec7dbf6dc8e408a306, 520e17cd55896441042b14df2566a6eb610ed444
    Loic Minier and Benjamin Larsson
  • support for electronic arts demuxer and decoders by aurel and peter ross
  • speling, gramer and warning fixed by diego
  • streaming to XBox360 fix by patric stout
  • a regression fix related to url_split() by ronald
  • display and sample aspect ratios display by michel
  • WMV3 FASTTX=0 fix by kostya
  • AVProgram API to export mpeg ts program information to the user app by nico
  • rm demuxer changed to output frames instead of slices by kostya and roberto, this simplifies alot of related code
  • Beam Software SIFF demuxer and video decoder by kostya
  • support for reading duration from Xing and VBRI tag in mp3 files by andreas
  • more flac encoding optimizations by loren
  • moving the framecrc muxer to its own file by aurel
  • and theres a patch for a MLP/TrueHD decoder by Ian Caulfield on ffmpeg-dev which i should review
  • many other things ive missed …
Filed under: FFmpeg — Michael @ 00:34


Recent changes in ffmpeg

Maybe you wonder what is currently happening in ffmpeg development, no nothing special i just thought to summarize what i remember and maybe i or someone else could/should write some weekly whats new report from now on? :)

  • heaps of flac encoder optimizations by loren
  • a H.264 PAFF patch, so dont give up hope yet, maybe we soon will have that in svn
  • amv audio and video support
  • IPv6 related fixes by ronald
  • some minor h.264 optimizations by andreas
  • some experimental h.264 multithreading code which splits entropy coding into its own thread by andreas, though dont expect that in svn in the near future
  • a MMS patch ohh darn i just realize i forgot about that one :(
  • various improvements and bugfixes in the rt(s)p code by the luca brothers ;) (sorry couldnt resist)
Filed under: FFmpeg — Michael @ 00:48


storing xiph codecs in containers other than ogg

Storing vorbis, theora and all the other xiph codecs in containers other than ogg is in principle not difficult, the primary problem is just that xiph codecs have several global headers while most generic containers expect just 1 or no global header. Mapping from several (3 normally) to 1 can be done in many ways and as there is no official recommandition from xiph which says how to do this, people have come up with various incompatible solution. Furthermore people have done other insane things like storing vorbis in ogg and that in avi and similar very very broken things …

So a long time ago ive decided to write a proposal for a recommandition on how to store xiph codecs in containers other than ogg, sadly it has been mostly ignored at the time of its writing

But yesterday someone from xiph contacted me and alex and asked if we where interrested to work on this oggless recommandition :) which lead to The oggless wiki page and a posting on ogg-dev so if you want to contribute to this / disscuss it (on ogg-dev), now is the right time …

Filed under: Container Formats,FFmpeg — Michael @ 21:32


Reviewing Patches

Whats a patch

A patch can be many things if you look at wikipedia, but the one i mean here is of course whats generated by the diff program. What diff outputs are simply differences between 2 text files in a terse form which is human and machine readable. The machine readable part means that the diff output together with one of the 2 input text files can be used by the patch program to generate the other text file. This makes patches a very convenient method of communicating changes between people, as people can read and understand them and apply them automatically to a file.

The following is an example patch which corrects a serious mistake of shakespeare. Sadly shakespeare did know about the patch program so he apparently failed to correct that …

--- mytext	2007-05-19 00:46:37.000000000 +0200
+++ mytext2	2007-05-19 00:47:12.000000000 +0200
@@ -1,3 +1,3 @@
 Shakespear said or wrote or whatever ...
-"To be or not to be,
+"To pee or not to pee,
 that is the question"

What are patches used for

Well they are used for communicating changes between software developers, at least in the open source world. So for example if you change the ffmpeg source code and fix a bug or add a feature then you could make a patch and submit that to us.

Why a patch and not the whole text

Well just think about the following: 2 people send you a patch, you look at the 2 small patches and if you are ok with them, you just apply both and have both changes in your text (which may be the source code of a program, or it might be just a love letter, the patch program doesnt care). On the other hand if the 2 people would have sent you the complete files not only would it be tricky to see what they have changed, integrating the changes would be even less fun. Note there are of course cases where applying 2 patches from 2 people conflict with each other so that they cannot both be applied automatically but in practice this is rare if the patches are clean


Now after this little introduction lets look at the actual reviewing of patches. People send us patches with their modifications of the ffmpeg source code and we review and comment on them. … Well, its sadly more i than we but anyway

The primary goal of the reviewing is to prevent bad, broken, buggy, inefficient, stupid, unneeded, … changes from being applied to the official ffmpeg source, so as to keep the source code small, efficient, simple, fast and working. This might sound like an easy task but it is absolutely not easy, if it where people would not subimit crappy changes in the first place but rather realize the problems and correct them before submitting.

Fixing problems in patches ourselfs vs. forcing the submitter to fix them

Fixing all issues which are found during review is always the problem of the person who submitted the patch. The first reason for that is it works very well in practice as the submitter wants the change to be accepted. The second reason is simply that we dont have the time to fix the issues considering the number of patches we receive and applying patches without all issues fixed would cause the quality, stability and maintainability of ffmpeg to decline steeply so that isnt an option either.

Sanity check

This is the first simply review level which checks if the patch could be applied at all if we wanted to apply it and if it does work

  • Is it a patch at all?
  • Is it undamaged?
  • does it apply to the current latest ffmpeg source?
  • do the regression tests pass? (We have a series of automated tests which check if various things like encoding, decoding and seeking still work as they did before)
  • does the patch not introduce any trailing whitespace or tabs? (we have agreed that these dont do any good and so the ffmpeg source doesnt contain any, this is arguable a philosophical issues but its easy to conform to)
  • does it not mix whitespace/cosmetic changes with functional changes? (patches which do are very hard to review so we always require them to be split)
  • does it do what the author claims it does?
Low level review

Low level here means reviewing the code in a line per line / function per function way without thinking too much about how the lines or functions are used

  • Are there any obvious bugs in the code?
  • Is the code minimal, that is can the same thing be done with less code? Less code generally means fewer bugs, faster code and less to read and understand …
  • Is the code fast? (does speed even matter where its used)
  • Is the indention consistent?
  • Is the code portable or would it fail on a big endian architecture?
  • Does the change not break anything?
  • Are all function/struct/… comments doxygen compatible so that nice html source code documentation can be automatically build?
  • Is the code not duplicated? Duplicated code is hard to maintain and wastes space in the source, the object file and in memory at runtime
  • Does the code have proper prefixes on the names of non static functions to avoid namespace issues?
  • Does the code not break the API/ABI and if it does break it does it update the version numbers properly and is the change really needed?
  • Are all input values properly checked to prevent crashes and security issues with invalid input files?
  • Are the types of large arrays minimal so as to safe space?
High level review
  • Is the choosen algorithm wisely choosen or is there a faster/simple one? does speed or size matter more in the case where its used …
  • Can the patch be split into seperate or incremental changes? Splited patches are easier to review, they are easier to understand, they are easier to debug as they can be seperately reverted and applied
  • Does the implementation make sense or are there serious conceptual mistakes in it, like messing up codec/container seperation or doing not really needed things
  • Are Things properly split into files? (for example decoder and encoder should be in seperate files)
  • Can big and seperate features be disabled at compile time?
  • Do the advantages from applying the patch outweight the disadvantages? Some rare patches add features noone needs but to do that add alot of complexity though that is rare and often such things can be made optional at compiletime

There are probably more things i forgot, but iam getting tired and this blog entry has become much longer than i wanted it to become …

Filed under: FFmpeg — Michael @ 01:48


Small tasks #2 (Iterative motion estimation)

FFmpeg supports iterative motion estimation, but only for snow. It would be interresting to implement/port this code to the generic mpeg framework (motion_est.c/mpegvideo.c/h) in ffmpeg

This task requires:

  1. Reading and understanding the generic motion estimation code
  2. Reading and understanding the iterative motion estimation code in snow.c
  3. Porting/Implementing iterative ME for the generic code

The advantage of iterative ME is higher quality per bitrate (but at higher computational cost).

Normal motion estimation tries to optimize each motion vector so as to minimize the weighted sum of the distortion (difference to the input picture) and the rate (bits for the motion vector). And after doing that for vector i, its done with i+1, then i+2 and so on, where all past vectors are held constant and all future ones are ignored. This is of course not correct, iterative motion estimation OTOH does not ignore any vectors but rather considers the bits of the future vectors too and performs several passes over all blocks until no further improvement is achived (=the search found a local minimum)

Filed under: FFmpeg — Michael @ 01:07
« Previous PageNext Page »

Powered by WordPress