From 70c1a78cd840e18ec75a724d0610ea9610b3153c Mon Sep 17 00:00:00 2001 From: Nigel Tao Date: Sat, 23 Sep 2023 22:57:40 +1000 Subject: [PATCH] test/3pdata: add etc-blinksuite.commentary.md --- test/3pdata/.gitignore | 5 +- test/3pdata/asset/182.0-original.djpeg.png | Bin 0 -> 2542 bytes test/3pdata/asset/182.0-original.jpg | Bin 0 -> 887 bytes test/3pdata/asset/182.0-original.wuffs.png | Bin 0 -> 1839 bytes test/3pdata/asset/182.1-byte-change.jpg | Bin 0 -> 887 bytes test/3pdata/asset/182.1-byte-change.png | Bin 0 -> 2280 bytes test/3pdata/asset/182.2-byte-change.jpg | Bin 0 -> 887 bytes test/3pdata/asset/182.2-byte-change.png | Bin 0 -> 2014 bytes .../nia-checksums-of-blinksuite.commentary.md | 243 ++++++++++++++++++ 9 files changed, 245 insertions(+), 3 deletions(-) create mode 100644 test/3pdata/asset/182.0-original.djpeg.png create mode 100644 test/3pdata/asset/182.0-original.jpg create mode 100644 test/3pdata/asset/182.0-original.wuffs.png create mode 100644 test/3pdata/asset/182.1-byte-change.jpg create mode 100644 test/3pdata/asset/182.1-byte-change.png create mode 100644 test/3pdata/asset/182.2-byte-change.jpg create mode 100644 test/3pdata/asset/182.2-byte-change.png create mode 100644 test/3pdata/nia-checksums-of-blinksuite.commentary.md diff --git a/test/3pdata/.gitignore b/test/3pdata/.gitignore index f4213d5b..e1242284 100644 --- a/test/3pdata/.gitignore +++ b/test/3pdata/.gitignore @@ -1,5 +1,4 @@ -# Ignore everything without a dot. +# Ignore everything ending in "suite". # # This ignores "pngsuite" but not "fetch-pngsuite.sh". -* -!*.* +*suite diff --git a/test/3pdata/asset/182.0-original.djpeg.png b/test/3pdata/asset/182.0-original.djpeg.png new file mode 100644 index 0000000000000000000000000000000000000000..7eb3af4e25f179385f0419b650db19cd87de79fe GIT binary patch literal 2542 zcmeAS@N?(olHy`uVBq!ia0y~yU{GLSV36ftVqjo!`SbGv0|Vz)PZ!4!4_@1YoCQ;! zN-{k7{&(O1TW8by-{di|2ss5X8FB^&a%*v2(Nb6t!E$7=+((Di!N&!+YQ=4R8?aQv zE74}dickSpKG(nvGMO9`T%1~D6x>g2oSm}ydVQ_?#F^hJ1!ghYFD_qIQ{y% zsOY`xE6+#Os_EVS8Tr8_^KIL&L${p;M62!B{l2?iJayLYD?K7}lou@w2n`iw(R`ZR zZ*%EhRmtaN`p?hI`+Dip^Hr-?r=Ophgy5xrH;p&EZe-6RW_Q#k->E8sum_LfluAN^J8nDE;;D3 z=-Is8j7JW4i|_yc?zi>6`;K=1o_)>zeLrjI@}NKod$t2#W2>*1dKx@G9}yA${qncB zw?&($iHFX~vD5X5j4x+kP&pVMz{JAp%6MRD#NMo_+>fS3KYzP&b?xJ|Q}@Rj%xBK| zmnD60Z%4Pmuhl2c?w=X`^^WaC)rqH<$6GzUr#ZVn?@nFSXKnBK5q}TbKhv3YWBP5| z%y&;!A9u&52ssK(VXX2oNK&t{n!9%X;pK->W?Sw))$<%GYP7o}Q+wocHZRxBi;><=4+{ zUN>uF>d|Sde!rhnb@M{4yxEiL_MGU?CzH?X@cwfCSn;P()m!Yk&cA1~*DrnY$wskd zO3<9+7X5CEA7*SxcyS?nx$oM1ixW#C{@%0x{pz*Ax<+2>=)C!{pU-{zzHMvnDQUT` z?O(*(z2jmxJn+~aY_}|{{BEOoeDtbcQ%-bv6xK0X&zw>9Gj!|LtxK0K4Jy)14KGjE znyR&4WBMd+#U)RKEdJfU|Kxf9`Q4d!Hzl|4%MH?UwYh4|)7{~kzc(?hGVZ}*d8t>g zUaeZc%Bwp>N3wse%~av^$8J5_WOgrJtSoRMH#>Xxr&XPk5)>YYA9$+2gQf4k^ZuU& z&v(_!-P)WQpBkEa`{A}l7d)~LJX{uG@p+kW>|FV}EiZl2pLb3^9{cRwt*x6=ZgAWc zF?!wNw9vyM#wcsay4S2fFMj*>Eki|V$pj}aO%)G@1D`XZ%z)!gixpsm<%+O9C1EZD`Fp5PJan0KK3rl|0> z{C`_E?o^!U^=HTbgv<<%QWq0_87{VdIm<;WSBcoz3oT_~RPj<^U}+Rsq~PfEqB-71 z`t=*_4cr%(NwBp?h=?q)sQI$t{PsMS2IgM&tk>Ut_ZR;9%&)7dsp~iYo$vQ=p`jrv z!p+I2kEIw#x>(NeU~zVIQK;Y(XyUT%7qBXNwS?nrxN)2Pi}&ZsB(>9h{`+;B@2+j{eXcvy*Vo>yo3Gz)IL&l(V}gSD^>!8x zB}E4fOZSB#3#Y_=crsT|NaKAVv)O;kjHM>#S*K(=ODaxC^P0V1mAbOzsj9fSiP^So zTWmi4IBb3U8=vH}{Vf&hF#(qzdRM!#R?nKHw^F0~>5Ry`A335``&Jt-GhS5?B5`P) z`e}dZ`u}$BQ-6F(S$$nQFMFZgqIYWybmoT4E4Z@cs`N!qh2zDk z@87?ERrz_E!4#KO*X3njosW*6;I(?wFO%z=&#jDkn4zK)q8~WrQ1FsX?4OLx)F;e; z660~zwd>CN1^rry3`#pwxjbcUPdi=OqjBI*XJZ3H^y#GfuxFO5&OeMg^*ott>Q2i$ z3P$IZWnmtyXT91ObG*>PYp&LvEyjIEZG@N{{om;L&oVHYv+0uJ+0~x!sw}Q5 zx(7LN1W!=b;LZ>3`h8^T+cO(ngBia(U2u$*%VPCT&n{LO9w(pqlQv2+SQ?%A?B{t& zQ=of>^(oqc{Zzx)?r6YNoagj&7o0P{TE7n#o&6y9pY}QoHFkaEDnR=(2lb>aC zQ0gSX6GrOF4MDLQE=^1fcb3mTd8=yQ^5vILT|OxJ=)Ljp%Xb8W)Yj@mtuwG>Zxvu% zVQ?(xnU??CaCXV}7Zo3}h9;eG^K4kyu|Usfa&dR%ie(x>LhTNPLhoN(2vL4N#lt}P zwRl|o8uPhl1(=TRvr*|=R4{jyTz`9pSKIDs$=Y>(xt=ntx~v^6z6%%}I(M#0U|R^YaUJcdy;6dr`Lg>$hK@4@d928&`X@#q_rQI(f4V z=2!E0W~`}xa3tZ(lXY+WPcNx__C)hokoF#+`DcMqT@^asq+hQ)KZF_9>-0=Fdo?^Y02{8=fTN2Ja*;lg3Wc>r5+aEuiI5f|t ze$Vdd0aE_zu>!5G$}=ADb)1`*x9Gy5O^1|cUgJ4grL%6AlIC5W&7BiMLMQcg&Pp<# l&go}s#e8z90ng+8w$lEJ&psb^V_;xl@O1TaS?83{1OUA6>a_p> literal 0 HcmV?d00001 diff --git a/test/3pdata/asset/182.0-original.jpg b/test/3pdata/asset/182.0-original.jpg new file mode 100644 index 0000000000000000000000000000000000000000..75b5fb101f48ab6be5146914cbe0e1eccce70f48 GIT binary patch literal 887 zcmex=fLVZnf$=6&=H!}0tIOUKBg@JI>i-{MkP~2FVq|4tMrdPXVrF0w z6cQFtGW>swfrF8efsvIdO|hpnRG%~Be)6{@v+&I^(dv@Xg6G&)u5Q-A3BG^|2x8`f!_rn~$zlvM!d)k&LhAr-#W@hFR?E5!0IhyBVW0ElI z(I8PpEh*91nh@`&i2@UrpLoQir*oclT5D_VS)cQ7PIyhVoq6X@<(>D1``*v9{&J$b zeSetUueGlVU!7LZH2p8L`uF#c*pn}RU6NlHyW`HqipzJlX+J-6F@0B4+~Msr)w_P( zOS`ALC!#?8_p!HgqFZ;i?y9+a*k-QY_j`NQuZG^2w~9#5S)Gx2W>4A5B^ys!T&iEF zFkMT8%S-dHL5qN+K*hZBoiTpPi>n`fd-LYao;`c+T9+L^dUWda>FR1~kEZ!C8FR8V zKa9wVs;jFjEG+c!=m=Zw>*p7@v*_vW{*>1Zx8IgkPM9_C*YStK(cANM_4Lo3I@Q(J zw=HUI^|v<@nSU{CFvz{NWj>#5%#MPGmzH{)ue(2E_Uzg7=H0t?&8?C@A!-i8M}^Y+ z4{KN2$L}mkz5D#Nn0}m%g++$Z%m-s&z8zsZ2Eot?rkBVk2e3hD^eajJ(3;8w|e>V^o$G* zb#-+$HMTzuSB2C48dSB~-^8l1bLd2E(NNX=`Tfk}>hJH)>h8LD^=j$ALzQ#=o^RD~ zp5c&@p}}-_YxeaMCp;n|B#iiYc=l9%)k?LU5tCJ^Ixol9&+DgNYE6xdtGC~s&i2(| zYk%F?)#nCR&%gg=ErFxl}nOC^K`WSNS{lNzI6Wq#qzkmHY z;OSPz=5_tXjgC)c<>iYF1(f#{TF4abKiPTfZCOpjm50pv6Ahnn#IL`uAHA)|@16X6 znaSJp?>B`#X=Ll~?Y+AtGq|F{BCM~s_vF^MZ{Mz6w{G3qwXOSCtk8JG95RFJ1LM65 z{8?M2elGev&%Qn`DvGQAhSG+P?Vq2Y@9*z--`)`C^7!(>;~P&ixGJ#o2?+^ZTNBxQ z%z-UKcL}TCy*-u9+c$7WZQyGC^76p-Zr$i@2Limky@P`D0wnV(#axyZrsC+1IOk-mR;=8vJ|L z{(U?5JKn#2;m-9NLdUY4IBx%Xv~+iQf!Xr41|jpd+waZOli5Eze|O=}X)nLUK8)P{ z@BOq*uO-i)TQW8Ej)qO~?`JQ&cLhI}Tk^m1`;(>C>ML)l7Wg^JS*ouu>s-?vzSE^o z+|qoe&8iB+6YqL9_8g1Qan$Q}+rnb4%eW^rB@>k}9)}8(mU7_D}orL%6+ZhofxX4}doqOE(=nhPKFUQBuOLR#v@s~0)hxu##gOZmvI=c!7b-uWn{;fQ|x zKAHUz0qb&aZ+m-xzr5W7MvK`dQoJnehd3LX4P4yZdp90gIK}B&MbJUjToLm`S2xXl z>M2f7B)CkEPi(4hy1F_%`O<|86Q)dQF=mKjn|b6T%XgXMj#mnzSbMuq`##y6x6+Zd z&1bW>=>r=b&$XHc3wT8|JFhJM(9XGb{rdYG5}k{SjV*5OsWhGK44~B4YDj{Xb&_|Guqi(b?M= Q7#J8lUHx3vIVCg!09Z0~ng9R* literal 0 HcmV?d00001 diff --git a/test/3pdata/asset/182.1-byte-change.jpg b/test/3pdata/asset/182.1-byte-change.jpg new file mode 100644 index 0000000000000000000000000000000000000000..da60825a34188735d1de32ab307df1d512566dc5 GIT binary patch literal 887 zcmex=fLVZnf$=6&=H!}0tIOUKBg@JI>i-{MkP~2FVq|4tMrdPXVrF0w z6cQFtGW>swfrF8efsvIdO|hpnRG%~Be)6{@v+&I^(dv@Xg6G&)u5Q-A3BG^|2x8`w<_a<+rb+OrF+}-z$0+8+r$9dYziwGd)!OB);;$Y&X?>ob%V zb}x@{y&weeUtQmF zl?SZ3xu9=FRpi_ZKSA4|`xjj181C8h=DGO!dA_sF&Z_oo=P6@%l~P@L40b8(St za&q$G#fuYI*x0<520g!GB*DY-RK5FX($=WQj~`E-G-=oG-QxNYckbPi$umi7RbL&V zHP>xv;40BYN3u4)=xsalva=~h>pGj1vXk5K9I@H1+p4~1EjnzatLv*by;*UhM~9q< z$dTHzGP5tQU%$>Zn|(9q-^w*QuFK;mS$)2>^YdY4=CXFvLn1%VZhvk+cY5)=tWNJ& zg;g=J%C%98!}5)I`uU>PiW%zZ@y*hk?!Eqccu2?(&DSMXGkx65Y|fuO8>%7FF>T&F zxqJ@opUEFTe>&c3y=~e(?Ylpo%HMjWU7mlx=H2eq_ov@8n5lHcvomga^tAcwdBRSp z+AR*#U9?G~E927T%f;W`L{2KOkhySwce%cV$g#ZblLJyxmISRd(Y(sobmQT?6}$hK zJA2*vCOSXg?#I*aS*P|-n^LpOB)=eI%g4~gLAp~<+~NDZA?Cx%sf}`r`X6f?}e7&HuLW z-*Hj@|Ax+T%vwqmT7j{`FVG*UvHOBJ2z)#*y`-8 ztP>6Ex98vg_we;)wq7suS2GNa_h$HAyJkP%KZM0fde*9cKj+@Hy&3fJ^Y8Tkq04W& zA3k)Jb7R)tgBSF2OF*%Ywb8R zg|T0&e&zZ6{@H!Ub*5DFmiv?Brl;_k%Iex_)qZ&)=;ZIC%kYA2f|8Qbjz+sXi4Q*i>~T}B zE9E@&J>P2nO_u(s04vPxk9nsA65|q*cs*LRD+6%gR|3d~eN> zi;Rnt+o*8+>Q&S1Yis7rk;&n0X%KQX}}$?4Ai`ucNoEJH+pJJ0b- zh>=p#(&Cb^IhL{M)||Pv*YetZm1g>MbawgmbNp9{ojm2r|8-}@<(KkaX_g$c8xawb#3P6xw(S8xQ{I?Cg}3loZ*sW)06f6-Kv;iXSIVa4;W!`l+SO#WzbW zMreEV$~tbr_N(h=hzR*e&UAfPwK#6=@5~j;7j9j;>Su0-PspM$?MuNe@7}#zwnxG6 zft0j#c6PSl+}2+$$2=saXI{K%ziOfv%haqGt#S)l+rQ3!({)W$EbLTw?%XH0dw1>R zt-PgA?f>kqPf$>^to$~|=H|7v(d%P&9-86Ixa!u03kLQ7{-mCs7C7Vba{pqdH@CLF zJ~PueAz|C{r~aqp?|qM8-z62Fs($R7>7HFbuFlsvyY`@6=jzqv4`Q>w_WO%Hee!4Z zl9brQ#4gP(dzVE`n?Il5h}&9BRP^ngos)n5`tdP&k$wF?nMYHnPL(##OJTl|eQizL z$D2XxPd_Y-*l$%7q1{#g|69`M(^k(fd{|!}I$`FlG)W1GcW-MW!nSGI+nXytHfLqs zwA6dLQN6_piC^Y^vSJPqFv90K-*Z!$HmmA#4zp{PH=X7i@49*F>!+uu zdF*6LKRj^sGdGIfl)~APabVRdt)>m_tk=)ZHV-y+6qemb_jYy*+>_?Tb#C z`m&s#pPsfn{@o*KY;b;S_H~DIr%ojuI5@qy{&=5k`1-h~G0x_f=9=frX~|lby?J!B zTVgufk>gjdUOnjaLuH*Q4STf*SO9mDPZ>ZHh{4+mGe_si`qd3kA9 z>1z%~<+L9sC#!RFbBC{ua-BAXG5*Kw!itI+J%REsCH5Xkk@09?n!oAJ?(+QG+j3KG z-MUrv-~i*P15X@WCd`^W+fZ-H$+__h-413Rn>1-s?eA|Nf3q{?mUZm+TU#^%GlTaQD*qf!OIrT#bNiRKz!4r9sdiy>EC0YXST#($K)nM6OQA% zb^gn9UTasM9L&u#?f<-^EG~s<$G*M2UCGDe`1jeuTN%76xBt8hQg`{i;MkRdl%JBD zA2zG}>yTghGx0~`G)I<-iHw3j+1cX%EiH7pe)Z}^MyJPBUF|DB*sb~NSTUW0VZ)DK zigW*;f1oT>wnXuObJ7)c^%xGetxG>y^x8e2S6=DCCh&d#>*c4r|E>SeEO}Dc`^~31 RWd;TY22WQ%mvv4FO#pmyUGM+^ literal 0 HcmV?d00001 diff --git a/test/3pdata/asset/182.2-byte-change.jpg b/test/3pdata/asset/182.2-byte-change.jpg new file mode 100644 index 0000000000000000000000000000000000000000..302a9995cb6aa50b2e1b4200c98b6c568ac96dd8 GIT binary patch literal 887 zcmex=KnUR4>fLVZnf$=6&=H!}0tIOUKBg@JI>i-{MkP~2FVq|4tMrdPXVrF0w z6cQFtGW>swfrF8efsvIdO|hpnRG%~Be)6{@v+&I^(dv@Xg6G&)u5Q-A3B5>xPaqf4N!~K8plzskvwEqIb7qP#SpYuoB#LiW0TKwCSOZn`Y<*^Hd$61G4?k-hNnml=O*>2f=52mmeSk2X%dg}2Z)RP~D3~1n+$3ubTRlUFWa)t?4-PiF z_sQJco*!RRQjJT}j!=CvO}zcj8)H_qp)I`PD`L;{LvG z(w*;eb=w`?)jGCQXGH|moJ_d(<3wiJE7ncFnHiW{of=PGTN}MTZf{gT>J|0SIK+$ru{ABUxgh4rkBZ&rM_ z*K1&{w|JwV!SsRu+r}wQS;L7dheVj0^|X z3jrq-)YO)3+H~pp=T6UrKAVb$TeqS--}LoAGxq#&=gNlg%TD4cx^-VqT{^lwe9a2Z z`rkU$FaNw@UFscPYiR!A(0ltO@ANLM?AY9Kdvp5vvbVQF!@}5ZEyzns|KMSuBUb3cvHm0Vfe}8@5vc*J%WyzJStunbiFA7x5g>!byw_z*tpXmHHYwdKk`6vG0{WT?C z$8(nS3`2E|x9Qy6)7SY${0Ix3({iRvh^77Nfgg;Dq8faY9`rpBKYFl3?np{Sg+zvb zIFHbSZ(L^61D)ASSBk8g_H5yvKW3guhxEhe@ca#Z_gh|$_N!h)1uV*v6fAUkY+~Q`Uo|3@EY*HK(BlE@K4f8uG$q()yZmi!lxvIav z$81)KK-}x)yR*-%lDpdTqJ-yt$_7ym#LG{~1vk)34rH7uq*5)c;*x==_7ZMU#x*opw$X6tw>S)_&f+y3Gq4 z7q%N48GUhHUuvukOwH^XJ4Q=)7m^hrhqRzK)KHs<1cgll$c1<<+!h>-O!R z{kgfgl$4ddy}f@bXB_(~^Z49c>q^Fjoik_7oURwkU+*jFP}Ro19!Q>R|Nb?euMht7434}_k{_We;%-=25ZsiG!0FtBlRx8bSNr!8ME zNIQA*ZbwFTE`<&eS`((@c!mUn-T))W9FPGpFA|t%AV1>)FuV24@ z6unY>d71CQg9kOD0s{pbSYFQbaOq%q5Va-a;yl~x0*Au6%^|_TpC8#Wo_r+EDH7nL zYHcl@aoD;mCL*GP`Jwc&*&H|KS(SF3=6kiH>}`|?OQizy3iIQSzrDR3>@#@&X$5EuCAumRB2uI=Ek1N;)n=| /dev/null +Premature end of JPEG file +``` + +That file is in the test suite presumably to check that decoders don't crash or +trigger buffer overflows, and `example/convert-to-nia` does neither here. + +--- + +`test/3pdata/blinksuite/182.jpg` is our final example (first in alphabetical +order but last in discussion order) of where Wuffs and libjpeg-turbo differ. + +Once again, different JPEG implementations have some leeway on how to treat +*invalid* JPEGs, and this JPEG is invalid. + +``` +$ djpeg test/3pdata/blinksuite/182.jpg > /dev/null +Corrupt JPEG data: 29 extraneous bytes before marker 0xc4 +``` + +As of Wuffs' [commit +186bdc5f](https://github.com/google/wuffs/commit/186bdc5fe2eedaa2869123aa7954ed9a4582e895) +"Roll back 'decode truncated... on MCU boundaries'" and libjpeg-turbo v3.0.1, +here are the two decodings (Wuffs' and djpeg's): + +![Wuffs decoding of 0-original](./asset/182.0-original.wuffs.png) +![djpeg decoding of 0-original](./asset/182.0-original.djpeg.png) + +When zooming in, the two are indeed different, but both decodings look like +garbage and it's not clear that one is better than the other (although djpeg's +output seems to have more 'detail' that Wuffs' output is missing). + +Let's look at the hex dump of the `test/3pdata/blinksuite/182.jpg` file, copied +into this repo as `test/3pdata/asset/182.0-original.jpg` + +``` +$ hd test/3pdata/asset/182.0-original.jpg +00000000 ff d8 ff e0 00 10 4a 46 49 46 00 01 01 01 00 48 |......JFIF.....H| +00000010 00 48 00 00 ff db 00 43 00 10 0b 0c 0e 0c 0a 10 |.H.....C........| +00000020 0e 0d 0e 12 11 10 13 18 28 1a 18 16 16 18 31 23 |........(.....1#| +00000030 25 1d 28 3a 33 3d 3c 39 33 38 37 40 48 5c 4e 40 |%.(:3=<9387@H\N@| +00000040 44 57 45 37 38 50 6d 51 57 5f 62 67 68 67 3e 4d |DWE78PmQW_bghg>M| +00000050 71 79 70 64 78 5c 65 67 63 ff db 00 43 01 11 12 |qypdx\egc...C...| +00000060 12 18 15 18 2f 1a 1a 2f 63 42 38 42 63 63 63 63 |..../../cB8Bcccc| +00000070 63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 |cccccccccccccccc| +* +00000090 63 63 63 63 63 63 63 63 63 63 63 63 63 63 ff c0 |cccccccccccccc..| +000000a0 00 11 08 00 1d 00 20 03 01 22 00 02 11 01 03 11 |...... .."......| +000000b0 01 ff c4 00 18 00 00 03 01 01 00 00 00 00 00 00 |................| +000000c0 00 00 00 00 00 00 02 03 04 05 01 ff c4 00 16 01 |................| +000000d0 01 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| +000000e0 00 02 03 fe da 00 0c 03 01 00 02 10 03 10 00 00 |................| +000000f0 01 d9 02 69 93 7c 61 3a d3 4b 91 59 76 77 50 7f |...i.|a:.K.YvwP.| +00000100 ff c4 00 1e 10 00 02 01 05 00 03 00 00 00 00 00 |................| +00000110 00 00 00 00 00 01 02 03 00 04 11 12 13 10 22 31 |.............."1| +00000120 ff da 00 08 01 01 00 01 05 02 66 21 8c 85 55 2f |..........f!..U/| +00000130 09 68 df 63 f6 62 36 57 b3 5c 5b 27 19 5b 11 ce |.h.c.b6W.\['.[..| +00000140 06 a9 ae 40 f7 9a b9 25 72 4f 1f ff c4 00 19 11 |...@...%rO......| +00000150 00 01 05 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| +00000160 01 00 02 10 12 21 ff da 00 08 01 03 01 01 3f 01 |.....!........?.| +00000170 4e a1 d1 3f ff c4 00 17 11 00 03 01 00 00 00 00 |N..?............| +00000180 00 00 00 00 00 00 00 00 00 00 01 10 11 ff da 00 |................| +00000190 08 01 02 01 01 3f 01 16 df ff c4 00 23 10 00 01 |.....?......#...| +000001a0 03 02 05 05 00 00 00 00 00 00 00 00 00 00 01 00 |................| +000001b0 11 21 02 71 10 12 22 41 51 03 31 32 61 62 ff da |.!.q.."AQ.12ab..| +000001c0 00 08 01 01 00 06 3f 02 14 d2 1c 9e 4a 24 f4 ea |......?.....J$..| +000001d0 8b 26 a8 52 15 41 e9 2d c2 1f 21 10 7b 15 a0 b1 |.&.R.A.-..!.{...| +000001e0 f6 b5 90 e4 44 ac cd 1b ad 25 ee 86 63 22 61 66 |....D....%..c"af| +000001f0 1e 22 1f 63 84 06 b1 65 21 ee 5f 0f ff c4 00 23 |.".c...e!._....#| +00000200 10 01 00 02 01 03 02 07 00 00 00 00 00 00 00 00 |................| +00000210 00 01 00 11 21 31 41 71 b1 d1 10 51 61 81 a1 f0 |....!1Aq...Qa...| +00000220 f1 ff da 00 08 01 01 00 01 3f 21 28 00 5d 0c 15 |.........?!(.]..| +00000230 de 08 25 2d a7 ba 66 38 d1 ba 0e 60 dc 5d f2 9f |..%-..f8...`.]..| +00000240 43 9f c8 60 ac 29 8e c8 f7 17 16 a2 7c 92 51 55 |C..`.)......|.QU| +00000250 62 a8 5a 7a ae db c7 42 da a8 bb d6 2a f8 26 a5 |b.Zz...B....*.&.| +00000260 13 76 68 00 3e 67 4f 0c d9 39 dd 13 36 4e 77 54 |.vh.>gO..9..6NwT| +00000270 31 a4 ff da 00 0c 03 01 00 02 00 03 00 00 00 10 |1...............| +00000280 41 c3 28 ff c4 00 18 11 01 00 03 01 00 00 00 00 |A.(.............| +00000290 00 00 00 00 00 00 00 00 01 00 10 11 61 ff da 00 |............a...| +000002a0 08 01 03 01 01 3f 10 80 95 ce 5f ff c4 00 18 11 |.....?...._.....| +000002b0 00 02 03 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| +000002c0 00 01 10 11 21 ff da 00 08 01 02 01 01 3f 10 29 |....!........?.)| +000002d0 c7 3f fe c4 00 22 10 01 01 00 02 01 03 04 03 00 |.?..."..........| +000002e0 00 00 00 00 00 00 00 01 11 00 21 41 31 61 71 10 |..........!A1aq.| +000002f0 51 81 91 a1 b1 f0 ff da 00 08 01 01 00 01 3f 10 |Q.............?.| +00000300 db 2f 88 48 1d 83 ba 38 c3 8a 32 a0 00 aa 28 b8 |./.H...8..2...(.| +00000310 dc 83 de bb 15 bb e7 59 72 c8 db a8 9c 95 8d 13 |.......Yr.......| +00000320 2f 6a 51 63 2d c0 f1 bf e3 2d 98 0a ca 24 71 35 |/jQc-....-...$q5| +00000330 70 8b a3 d8 76 f3 fb c5 8f d0 00 f4 16 fd 93 e7 |p...v...........| +00000340 0f 59 3c 5d 9d 21 82 a3 fb 49 0a 23 cd ac a7 1b |.Y<].!...I.#....| +00000350 d6 49 65 a5 91 38 7b 76 c5 aa 60 a8 1a 3b 4b c0 |.Ie..8{v..`..;K.| +00000360 bd 2d 2f a6 e6 cf a8 7b f0 8c 5c 78 f3 fe 93 80 |.-/....{..\x....| +00000370 00 00 34 1e d9 ff d9 |..4....| +00000377 +``` + +The JPEG markers within this (including a corrupted one) are: + +``` +pos = 0x00000000 = 0 marker = 0xFF 0xD8 SOI +pos = 0x00000002 = 2 marker = 0xFF 0xE0 APP0 +pos = 0x00000014 = 20 marker = 0xFF 0xDB DQT +pos = 0x00000059 = 89 marker = 0xFF 0xDB DQT +pos = 0x0000009E = 158 marker = 0xFF 0xC0 SOF0 (Sequential/Baseline) +pos = 0x000000B1 = 177 marker = 0xFF 0xC4 DHT +pos = 0x000000CB = 203 marker = 0xFF 0xC4 DHT +pos = 0x000000E3 = 227 marker = 0xFE 0xDA SOS *** corrupted *** +pos = 0x00000100 = 256 marker = 0xFF 0xC4 DHT +pos = 0x00000120 = 288 marker = 0xFF 0xDA SOS +pos = 0x0000014B = 331 marker = 0xFF 0xC4 DHT +pos = 0x00000166 = 358 marker = 0xFF 0xDA SOS +pos = 0x00000174 = 372 marker = 0xFF 0xC4 DHT +pos = 0x0000018D = 397 marker = 0xFF 0xDA SOS +pos = 0x00000199 = 409 marker = 0xFF 0xC4 DHT +pos = 0x000001BE = 446 marker = 0xFF 0xDA SOS +pos = 0x000001FC = 508 marker = 0xFF 0xC4 DHT +pos = 0x00000221 = 545 marker = 0xFF 0xDA SOS +pos = 0x00000272 = 626 marker = 0xFF 0xDA SOS +pos = 0x00000283 = 643 marker = 0xFF 0xC4 DHT +pos = 0x0000029D = 669 marker = 0xFF 0xDA SOS +pos = 0x000002AB = 683 marker = 0xFF 0xC4 DHT +pos = 0x000002C5 = 709 marker = 0xFF 0xDA SOS +pos = 0x000002F6 = 758 marker = 0xFF 0xDA SOS +pos = 0x00000375 = 885 marker = 0xFF 0xD9 EOI +``` + +Note that this JPEG has multiple `0xFF 0xDA` Start Of Scan markers, but the +`0xFF 0xC0` at position `0x0000009E` denotes a *sequential* JPEG, not a +*progressive* JPEG. It's highly unusual for a non-progressive JPEG to have +multiple scans. + +At position `0x000000E3` there are the two bytes `0xFE 0xDA` which is *not* a +Start of Scan (because it's `0xFE`, not `0xFF`). This (continuing up to but +excluding the `0xFF 0xC4` Define Huffman Table marker at position `0x00000100`) +causes the "Corrupt JPEG data: 29 extraneous bytes before marker 0xc4" error +message seen above. + +The first recognized `0xFF 0xDA` Start of Scan is actually at position +`0x00000120` and the `0x01` byte at position `0x00000124` means that the *scan* +has only one component. In contrast, the `0x03` byte at position `0x000000A7` +means that the *frame* has three components (the typical Y, Cb, Cr). + +Thus, we have a *sequential* JPEG with multiple scans, but also whose first +scan doesn't cover all of the components. Highly, highly unusual, and the cause +of why Wuffs and libjpeg-turbo differ in its output. + +Specifically, decoding progressive JPEGs require more memory than decoding +sequential JPEGs, since decoding progressive JPEGs needs somewhere to store 'in +progress' (partially reconstructed) pre-IDCT coefficients. Wuffs-the-language +doesn't allow Wuffs code to allocate memory at arbitrary times. Instead, for +Wuffs image decoding, there is only one point in time where the Wuffs-callee +can advise its caller how much memory it will need for its "work buffer". This +point in time is after the image configuration (width, height, color model, +etc) has been decoded but before actual pixels have been decoded. For the JPEG +file format, "decode the image configuration" corresponds to parsing the `0xFF +0xC0` Start of Frame (Baseline Sequential) marker. + +When Wuffs decodes `182.jpg`, the JPEG file's Start Of Frame says "I'm a +*sequential* JPEG", so Wuffs *doesn't* ask its caller for that extra memory. +Even though it turns out that the JPEG file has multiple scans, those scans +*aren't* accumulated (in that non-existent extra memory). + +When libjpeg-turbo decodes `182.jpg`, it sees "I'm a *sequential* JPEG" but, +when it also sees that the first scan has incomplete component coverage, it +goes ahead and allocates that extra memory. The multiple scans *are* +accumulated and the final pixel output is different (and more 'detailed'). + +In theory, it'd be possible to modify Wuffs' JPEG decoder so that "decode the +image configuration" actually peeked ahead to the first scan. But that adds +complexity to the implementation. It's hard to justify that additional cost +just to handle highly, highly unusual JPEGs like the invalid (corrupted) +`182.jpg` differently, especially just to produce different *garbage*. + +A final remark: sequential versus progressive JPEGs also differ in how to +handle the Huffman data in each scan. If we modify only 1 byte in +`182.0-original.jpg` to give `182.1-byte-change.jpg`, changing the byte at +position `0x0000009F` from `0xC0` to `0xC2`: + +```diff +- 00000090 63 63 63 63 63 63 63 63 63 63 63 63 63 63 ff c0 |cccccccccccccc..| ++ 00000090 63 63 63 63 63 63 63 63 63 63 63 63 63 63 ff c2 |cccccccccccccc..| +``` + +This changes the JPEG from sequential to progressive. After doing that, Wuffs +and libjpeg-turbo agree *exactly* on what its decoding is (as both ask for and +use that extra memory). It also no longer looks like garbage. It looks like a +kitty! + +![Decoding of 1-byte-change](./asset/182.1-byte-change.png) + +If we also fix the corruption by changing the `0xFE` to `0xFF` at position +`0x000000E3`, so that we've changed 2 bytes overall, it looks like an even +better kitty (and both decoders still agree). + +![Decoding of 2-byte-change](./asset/182.2-byte-change.png)