Little Reverse engeneering puzzle
There are SVQ3 files that ffmpeg cannot decode yet, the ones iam speaking of contain a image or watermark, often a logo in a global header (extradata in ffmpeg, QT has its own funky terminology for it). The binary decoder displays this watermark over the actually decoded video.
FFmpeg is in principle fully capable to decode these videos (and of course without these watermarks), the only problem is that 32bit of the header and following bitstream are modified by xoring them with a per file constant. Our problem is we do not know how the binary decoder calculates this constant. A example SVQ3 video and the corresponding constant of 0xA2A2A2A2 for this file as well as a bugreport on our tracker exist as well
The puzzle is to figure out how the binary decoder finds the constant, its likely not very hard for one knowing how to use a debugger, hint: memory breakpoint at the input data and a cup of coffee or tea or a can with cola.
Is there other samples ?
The bug tracker refers to a hl2_small.mov .
Comment by mat — 2008-08-16 @ 17:46
http://samples.mplayerhq.hu/V-codecs/SVQ3/b-frames/e32k3-halflife2_pce32003_8dn_qt.mov
and svq3_watermark.mov on mphq incoming for example
Comment by Michael — 2008-08-17 @ 00:12
The xor key is calculated in QuickTimeEssentials.qtx in the Windows player (see function at 681F1420 in current version).
The algorithm is essentially:
uint16_t get_xor(uint8_t *table2, int passes, uint16_t initial_key)
{
int i;
uint16_t key;
if (passes <= 0)
return initial_key;
key = initial_key; // always 0
for (i = 0; i < passes; i++)
{
key = (key <> 8) ];
}
return key;
}
The return value of this function is multiplied by 0x10001 to replicate the key in the top 16 bits.
The only call site I could find always passes 0 as the initial key.
table2 is different per file; not sure where it is calculated yet. Passes is also different per file, and is the size of table2. Passes for lucy_letters_to_satan_2007.mov is 0x5348; for e32k3-halflife2_pce32003_8dn_qt.mov, it is 0x2940.
table1 is a static 256-element 16-bit int array (stored as 32-bit ints at offset 40A10 in QuickTimeEssentials.qtx); first 16 elements seem to be index * 0x1021, not sure after that… I can dump it somewhere if desired.
Comment by DrV — 2008-08-18 @ 23:11
And of course the blog ate the most important line of code… :)
Hopefully this will work…
key = (key <> 8) ];Comment by DrV — 2008-08-18 @ 23:14
Third time’s the charm (maybe) – sorry for all the noise.
Remove underscores to get the real code.
key = (key <__> 8 ) ];
Comment by DrV — 2008-08-18 @ 23:16
Now I feel totally silly… apparently arbitrary tag-looking things are treated as HTML. Here we go:
key = (key << 8) ^ table1[ table2[i] ^ ((key) >> 8) ];Comment by DrV — 2008-08-18 @ 23:18
Is that smiley face a substitute for “eight right-paren”?
If the key is only 16 bits, a brute-force search shouldn’t take long, assuming verification is straight-forward.
Comment by Mans — 2008-08-19 @ 13:29
Interesting. Would that be how the secure watermark is added to videos created for this device: http://www.cylonglobal.com/ ? It’s supposed to add an encrypted watermark to every frame of the video when encoding such that it is somehow permissible as evidence in court.
Comment by Robert Swain — 2008-08-19 @ 14:09
@DrV
Thanks alot for the analyis, this moves us a step further, the function you describe seems to match svq1_packet_checksum() that is used by svq1dec.c
Actually ive tried to feed this one already with random pieces of extradata before this blog post but had no luck …
We still need to know what is used as its input
And iam sorry for the blog trashing code in comments, if anyone knows of a way i can fix this …
@mans
Decoding a picture 65536 times to find the key for a file is somewhat inconvenient (besides it may need more than 1 picture.) The problem is not to find the keys, the problem is to do it quickly and automatically.
@rob
i know nothing about cylong-lobal-whatever, but it sounds unrelated to these QT watermarks
Comment by Michael — 2008-08-19 @ 15:50
a small test about trashing
real HTML tags: XXX
X
named special chars: & < &rt; ∀ π
escaped 8 paren:”8)” ‘8)’ 8\) 8)
8 paren under code and blockquote:
8)Comment by Michael — 2008-08-19 @ 16:09
Things under quotes:
‘a >= < = != == << >> 8) > < ” > < a‘
“a >= < = != == << >> 8) > < ‘ > < a”
and over several lines
‘a
>= < = != == << >> 8)
> < ” >
< a‘
Comment by Michael — 2008-08-19 @ 16:16
hmm & gt ; under quotes “>”
8 paren under quotes “8)”
8 paren under quotes with spaces ” 8) “
Comment by Michael — 2008-08-19 @ 16:19
bye bye stinking smily trasher
:) ;) *g* :> ^-^ :P 8)
Comment by Michael — 2008-08-19 @ 16:28
Indeed, the svq1_packet_checksum() is exactly the same; the table it uses (checksum_table in svq1dec.c) also exactly matches the one used here.
Comment by DrV — 2008-08-20 @ 17:08
table2 is probably the uncompressed logo to display with.
it seems the logo is compressed in extradata, I don’t know using which method.
Comment by bcoudurier — 2008-08-26 @ 23:42
uploaded RAW512K_Stream_003.mov
key: 0x6ee36ee3
extradata right after SEQH:
00000000 00 00 00 2c e2 38 1e 19 8a 0b 41 62 df f0 12 78 …,.8….Ab…x
00000010 9c ed c1 01 0d 00 00 08 03 a0 37 d7 e8 e6 b8 03 ……….7…..
00000020 12 00 00 da 2c c0 5b 03 00 40 9d 03 ….,.[..@..
Source logo test1.bmp, 50×50 full black picture:
00000000 42 4d e6 1d 00 00 00 00 00 00 36 00 00 00 28 00 |BM……..6…(.|
00000010 00 00 32 00 00 00 32 00 00 00 01 00 18 00 00 00 |..2…2………|
00000020 00 00 b0 1d 00 00 13 0b 00 00 13 0b 00 00 00 00 |…………….|
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|
*
00001de0 00 00 00 00 00 00 |……|
The magic between test1.bmp and extradata is to be found :>
Comment by bcoudurier — 2008-08-26 @ 23:57
How does extradata look with a all white bmp of identical size?
Comment by Michael — 2008-08-27 @ 00:09
The first 32bit of extradata seem to be the size in byte of the whole extradata, at least in this example here
Comment by Michael — 2008-08-27 @ 00:15
test2.bmp, 50×50 full white:
00000000 42 4d e6 1d 00 00 00 00 00 00 36 00 00 00 28 00 |BM……..6…(.|
00000010 00 00 32 00 00 00 32 00 00 00 01 00 18 00 00 00 |..2…2………|
00000020 00 00 b0 1d 00 00 13 0b 00 00 13 0b 00 00 00 00 |…………….|
00000030 00 00 00 00 00 00 ff ff ff ff ff ff ff ff ff ff |…………….|
00000040 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |…………….|
*
000000c0 ff ff ff ff ff ff ff ff ff ff ff ff 00 00 ff ff |…………….|
000000d0 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |…………….|
*
00000160 ff ff ff ff 00 00 ff ff ff ff ff ff ff ff ff ff |…………….|
00000170 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |…………….|
*
[…]
RAW512K_Stream_004.mov:
key: 0xa71aa71a
extradata:
00000000 00 00 00 2c e2 38 1e 19 8a 0b 41 62 df f0 12 78 …,.8….Ab…x
00000010 9c ed c1 01 0d 00 00 08 03 a0 e8 9a fc 33 c7 1d ………….3..
00000020 90 00 00 d0 66 01 de 1a 00 00 ea 1c ….f…….
Yes 4 bytes at the beginning is what is just after SEQH, already RE’d in svq3 decoder.
Comment by bcoudurier — 2008-08-27 @ 02:37
RAW512K* and svq3_watermark.mov moved to:
http://samples.mplayerhq.hu/mov/watermark/
Comment by compn — 2008-09-29 @ 21:30
http://pastebin.com/f1156072
Comment by chrono — 2008-10-14 @ 16:00
small update http://pastebin.com/f574891c
Comment by chrono — 2008-10-14 @ 18:19
patch for lavc:
http://pastebin.com/m1a1dd3b6
Offset to compressed data in extradata still needs to be computed exactly.
Comment by bcoudurier — 2008-10-14 @ 20:33
Great work everyone …
Ive applied a few hunks of the patch :)
pastebin though seem to have eaten the at-at from the patch, requireing them to be added back manually :(
anyway some comments about the patch
The way the key is applied can lead to unaligned accesses and might break on some systems, notably ARM. Also it needs that ugly bswap on bigendian, AV_R/WL32() should work better.
And the width/height code looks similar to h263_format and the code using it. And case 5 looks like w/h is fliped.
Comment by Michael — 2008-10-14 @ 21:58
http://pastebin.com/d13e59c22
case 5 it’s probably bug in QT ;)
Comment by chrono — 2008-10-14 @ 23:53
http://pastebin.com/f36a93bb6
Comment by chrono — 2008-10-15 @ 01:44
Could get_bits_var() be the same as svq3_get_ue_golomb() from golomb.h or another function from there?
Comment by Michael — 2008-10-15 @ 02:24
yes, it’s svq3_get_ue_golomb()
Comment by chrono — 2008-10-15 @ 02:46
Your table sounds like a CRC16 table, the standard polynom being 0x11021.
OG.
Comment by Olivier Galibert — 2009-01-08 @ 01:18