JPEG XL

Normally on Linux it should default to the recompression but It doesn't seem to be the case on ffmpeg Windows

So I have to figure out the parameters which I still don't know

ah, sorry for explaining something you already knew yeah that sounds about right, I remember reading in here that ffmpeg doesn't support lossless JPEG recompression yet, but that was a while ago

It's fine maybe I could forget something too and learn it again 👍

ffmpeg indeed does not support lossless jpeg recompression

there is also the nefarious lossless VarDCT mode but I think I would get stern looks from the core devs if I mentioned it

I think I get stern looks from the core devs a lot actually <:KekDog:805390049033191445>

Lossless vardct? I guess that's what jpeg recompression does...

true, but I'm referring to the idea of taking in 8 bit per channel pixel data, and encoding with high enough precision and without quantization to result in the exact same pixel values in a sufficiently high precision decoder

I vaguely remember libjxl got a but report from someone that tried this but ran into issues when decoding

I guess technically it could work...

I've always wondered how that would compare to lossless Modular for a photographic image in terms of compression ratio. I don't have a good intuition at all for how efficiently a non-quantized DCT representation can be compressed

If I understand correctly, this is the basis of how lossless AVIF works, though its 'base' image is not entirely lossless, so it additionally stores residuals to get back to the original, and that does about as well as JXL's lossless modular on photographic, somewhat noisy imagery

(I know this because at one point I had this idea and I was told I had just invented lossless AVIF)

I guess with the small DCT transforms like 2x2 or Hornuss it would be most likely to work

yes, that's more or less how lossless AVIF works

and also.. JPEG-LS?

these 'lossless VarDCT' images would be useful for tracking how much precision is lost due to accumulating rounding errors in the decoding process

I don't know how you would even go about producing that, but setting the quant tables to 1 is the obvious first step

no restoration filters, no decorrelation tools that aren't mathematically reversible

in jxl you can set quant tables to 1e-8 or so, too

or 0.00006103515625 more precisely

(lowest possible non-subnormal fp16 value)

(... I *think*, maybe you can go lower)

I might be misunderstanding how values in quant tables affect quantization. I thought 1 meant no quantization since it's the multiplicative identity

what happens is that you divide by the quant table value then round

if the input were an integer, you'd be right

but dct coefficients (in jxl) can and will be arbitrary floats

a very destructive process, isn't it? I believe it is safer to assume the antialiasing should be preserved. seems like an ideal situation for jpeg2png decoding.

here's an example jpeg2png decode and a derivative jxl at the same byte size as the input jpeg

or here's an example at 17217 B, closer to the transcoded jpeg at 17210 B

Here we go again: https://github.com/victorvde/jpeg2png

Or https://github.com/ilyakurdyukov/jpeg-quantsmooth

(or waifu2x)

Everytime I use jpeg2png, I feel indeed "JPEG is Alien Technology from the Future" Theoretical, DCT is bad for hard edge But with jpeg2png, it still can!

Too ancient

Also anime only

upscayl works fine

Yee dispite their age, they work well

Thio made an updated one btw https://github.com/ThioJoe/jpeg2png

Is that 1x?

it can work 1x

Hmm 🤔 esrgan? Or?

wonder why we don't see a decent diffusion model to do this around

different models available

2024 ncnn and esrgan kk

jpeg2png is literal gradient descent, not machine learning

Can do but it won't be faithful

it can absolutely be though

Gan nn's will be close

Haven't seen 1 latent diffusion based upscaler that doesn't invent details

well, as long as it invents details that are compatible with the JPEG data you have ... (i.e. it creates a JPEG that would recompress to the same pixels)

I suppose but the goal right, is to de-jpeg, not transform the img too much

Similar to what competent de-h.264 gan models do for video

Latent diffusion is also heavily limited by the vae

it is really good for digital content, and the above one is a perfect example of where waifu2x excels

Gans have their own limits and flaws ofc but are at least very faithful

also, I don't understand why "oldness" is used as an argument to not use a software

and jpeg2png's last commit is 11years ago <:KekDog:805390049033191445>

Old means it can't take advantage of newer better techniques

That's the acception 🤣

There's wizardry going on with jpeg2png/qs

In w2x's case it lacks a transformer

Specifically a derivative of the swin transformer

Tldr: it can't compensate for lack of training to a certain degree

Causing warping and tons of artifacts

https://github.com/chaiNNer-org/spandrel?tab=readme-ov-file#single-image-super-resolution we have many many more competent architecture's now xD

I tried several "alternatives", they all were worse than waifu2x for digital 2d cartoon images. Seems like they all are trained on photographic content. (Also I remember there was some shareware application that didn't use NNs at all, and it still worked very good for cartoon images) If you have a list of alternatives, I would test them out (normal exe GUI programs, not raw python, CLI, or web-interface).

https://imgsli.com/NzA3NjI

But yea really depends what ur trying to do

In this case 0 jpegness

Literally chainner for sisr, and there's a few others for video

Some use trt/onnx

But if ur primary goal is toon restoration/upscale https://github.com/Sirosky/Upscale-Hub/releases/tag/AniToon

avif seem to also work well with flat color I assume? I can get it to as low as 3.08KB while still visually lossless

Also I kinda noticed ImageGlass show color space changed from Grey/- to sRGB/-

yep AVIF has some coding tools that do particularly well for lossy compression of flat color shapes

the lossless JXL at effort 7 was somewhere between 1 and 2 kilobytes iirc

I tried effort 8 and distance 10 and get around 7.46 KB

lossless?

Good question

It's lossy <:kekw:808717074305122316>

anything above distance 0 is lossy

A minute or two and I got this

`-d 0 -g 3 -e 9 -P 0`

certainly that was not the source for the avif

2.23GB <:PepeGlasses:878298516965982308>

the transparent one is damn bright

why was that so big to begin with?

oh shoot, there was a larger one?

the 8000x8000 actually seemed downscaled because edges were sharpened, I'm glad I know there's a 25000x25000 one now

there's also other files for it too as well like a depth map and IIRC a normal map

uploaded as different entires

maybe was encoded without any predictors? it's around 3.5bpp

it's incredibly noisy unlike the downscaled one, that's probably why

the slight macroblocking effect when you zoom in on that screenshot seems to be due to how XnView is rendering it, it doesn't appear like that in the image

early results

interestingly, effort 2 was a lot faster than effort 1

probably in part because effort 1 used so much memory that memory compression kicked in, whereas for effort 2 that didn't happen

Bouta ran out of space before all tested😂

Also do you see the transparent one

e7 with fd4 allows for better density and even loads faster than the source PNG

fd4?

ah, faster decoding

I might imagine thing but it seem to have higher color-depth

oh interesting

it looks brighter too, I wonder if that's because of how PNG makes assumptions about associated alpha/premultiplication and I know rendering software sometimes gets fucky with that

I wish I had more RAM

use swapfiles <:galaxybrain:821831336372338729>

I use memory compression which is a lot faster

that's the yellow line

yeah, but it can't make use of storage media

I think memory compression is backed by a final swap file on disk, but I haven't verified that. I use whatever the default setup is for Pop!_OS and they were pretty meticulous about tuning the swap behavior so I trust they did it right

zram is not backed by storage, it's in-memory-compression

I have 20 GB "swap", which I'm pretty sure is 16 GB (uncompressed) worth of compressed memory, which usually takes up about 1/5th of the space, and a final 4 GB swap file on disk

yeah I know, but once zram is full it starts using disk swap

ah, that's what you meant

Assuming 16-bit is not really needed for this image, you can gain quite a lot by just using a lower precision, for example: $ cjxl 20241111151449\!Mandelbox_mit_farbigem_Nebel_und_Licht_20241111_\(color\).png -d 0 -e 1 20241111151449\!Mandelbox_mit_farbigem_Nebel_und_Licht_20241111_\(color\).png.jxl8 --override_bitdepth 10 JPEG XL encoder v0.11.1 0.11.1 [NEON_BF16,NEON] Encoding [Modular, lossless, effort: 1] Compressed to 770763.7 kB (9.866 bpp). 25000 x 25000, 110.078 MP/s [110.08, 110.08], , 1 reps, 12 threads. $ cjxl 20241111151449\!Mandelbox_mit_farbigem_Nebel_und_Licht_20241111_\(color\).png -d 0 -e 1 20241111151449\!Mandelbox_mit_farbigem_Nebel_und_Licht_20241111_\(color\).png.jxl8 --override_bitdepth 8 JPEG XL encoder v0.11.1 0.11.1 [NEON_BF16,NEON] Encoding [Modular, lossless, effort: 1] Compressed to 448212.0 kB (5.737 bpp). 25000 x 25000, 145.433 MP/s [145.43, 145.43], , 1 reps, 12 threads.

I went overboard with testing large JXLs, so my setup is like this for swapfiles. top one is zram

Shouldn't Pallete already reduce the effective bitdepth?

since it's a generated image, it has noise in all bits of that 16-bit image so Palette cannot do anything here

(cleaned up a bit for clarity) ``` $ cjxl big.png -d 0 -e 1 big.png.jxl8 --override_bitdepth 10 JPEG XL encoder v0.11.1 0.11.1 [NEON_BF16,NEON] Encoding [Modular, lossless, effort: 1] Compressed to 770763.7 kB (9.866 bpp). 25000 x 25000, 110.078 MP/s [110.08, 110.08], , 1 reps, 12 threads. $ cjxl big.png -d 0 -e 1 big.png.jxl8 --override_bitdepth 8 JPEG XL encoder v0.11.1 0.11.1 [NEON_BF16,NEON] Encoding [Modular, lossless, effort: 1] Compressed to 448212.0 kB (5.737 bpp). 25000 x 25000, 145.433 MP/s [145.43, 145.43], , 1 reps, 12 threads. ```

Ah right, so the image *is* 16bit, but most isn't useful information

I didn't expect the 25000x25000 to be 16 bit as well

I guess 16-bit is useful if you want to do further editing, like adjusting brightness/contrast

but for delivery of SDR images, 8-10 bits is enough.

That definitely doesn't sound right... Effort 1 uses 16bit buffers instead of 32float, so memory should be lower if anything

I feel like this would also be an interesting image to test VarDCT lossy on, especially since with the recent buffering changes to libjxl you can now get a proper progressive group ordering without having to disable chunked encoding

yeah I'm not sure why either, maybe it's because effort 2 wrote chunks back to the file while effort 1 kept them in memory or something?

I was surprised too

on 0.11.1

Hmm

I don't think it would be smaller, but if you want the fastest loading you could try -d 0 -g 0 -P 0 -I 0 -e 6 (could go higher, but will already be very slow to encode)

I did encode it, was decently fast at 50MP/s and ~400MiB smaller

and loads in ~2-3 seconds compared to 14-15sec of the original in nomacs

The 0000 command?

just `-e 7 -d 0 --faster_decoding 4`

libjxl from main

``` JPEG XL encoder v0.12.0 98f8988c [_AVX3_ZEN4_,AVX3_DL,AVX2,SSE4,SSE2] {GNU 15.2.1} Encoding [Modular, lossless, effort: 7] Compressed to 1947411.8 kB (24.927 bpp). 25000 x 25000, 49.373 MP/s, 32 threads. ```

I'm interested in creating a 16-bit 8000x8000 one form the 25000x25000 one, with a downscaling method that doesn't introduce sharpened edges, to give the predictors a good shot

try integer scaling <:galaxybrain:821831336372338729> /s

I meant I doubt that command would be smaller, but should decode in less than a second (maybe)

really the author should've let the renderer run for a couple more months so we'd have a nice and smooth image

let me try, one sec

though I'm wondering if they ran into precision issues in the renderer itself somewhere, because weirdly this image seems to have posterization on a coarse level *despite* the image being very noisy and high bit depth on a fine level...

smells of floating point limits to me...

`convert -scale 25%` produces a nice 6250x6250 image with just averaging of 4x4 pixels

ooo nice, I'll use that

`convert-im6.q16: cache resources exhausted` we're gonna need a bigger boat

actually, I remember it saying that when it doesn't read the input properly too

still there's a lot of entropy remaining in those least significant bits of the 16-bit version

``` bash-3.2$ cjxl 20241111151449\!Mandelbox_mit_farbigem_Nebel_und_Licht_20241111_\(color\).png.png -d 0 20241111151449\!Mandelbox_mit_farbigem_Nebel_und_Licht_20241111_\(color\).png.png.jxl JPEG XL encoder v0.11.1 0.11.1 [NEON_BF16,NEON] Encoding [Modular, lossless, effort: 7] Compressed to 97335.3 kB (19.934 bpp). 6250 x 6250, 3.773 MP/s [3.77, 3.77], , 1 reps, 12 threads. bash-3.2$ cjxl 20241111151449\!Mandelbox_mit_farbigem_Nebel_und_Licht_20241111_\(color\).png.png -d 0 20241111151449\!Mandelbox_mit_farbigem_Nebel_und_Licht_20241111_\(color\).png.png.jxl --override_bitdepth 10 JPEG XL encoder v0.11.1 0.11.1 [NEON_BF16,NEON] Encoding [Modular, lossless, effort: 7] Compressed to 27917.8 kB (5.718 bpp). 6250 x 6250, 7.522 MP/s [7.52, 7.52], , 1 reps, 12 threads. bash-3.2$ cjxl 20241111151449\!Mandelbox_mit_farbigem_Nebel_und_Licht_20241111_\(color\).png.png -d 0 20241111151449\!Mandelbox_mit_farbigem_Nebel_und_Licht_20241111_\(color\).png.png.jxl --override_bitdepth 8 JPEG XL encoder v0.11.1 0.11.1 [NEON_BF16,NEON] Encoding [Modular, lossless, effort: 7] Compressed to 14440.7 kB (2.957 bpp). 6250 x 6250, 11.923 MP/s [11.92, 11.92], , 1 reps, 12 threads. ```

also loads in 2-3 seconds but is 2.5GiB instead of the 1.8GiB with e7 and fd4

the 8-bit per sample version nicely compresses to 1-bit per sample (2.957 bpp), while the 10-bit version takes almost 2 bits per sample and the 16-bit version takes more than 6 bits per sample.

So those 8 msb can be compressed well but the 8 lsb is pretty noisy even after downscaling 4x

Huh, interesting

I cannot get the convert command to work, any chance you could get me that through some file sharing service?

Not atm, on my phone now

alright, no worries

https://files.catbox.moe/jk8fz0.png

thanks :3

for some reason scale and resize with 25% got me a 2000x2000 image, no idea how they calculate 25% from 25000

so had to specify the 6250 manually

oh wait

I had the wrong image

<@384009621519597581> one sec lmao, I'll get you the right one

Here's the correct one https://files.catbox.moe/8jsozz.png

yep that file size seems much more realistic :p

anyone wanna try putting the depth channel to use? https://commons.wikimedia.org/wiki/File:Mandelbox_mit_farbigem_Nebel_und_Licht_20241111_(zbuffer).png?useskin=vector

tbh I don't even remember how you would put this in with cjxl or even if you can

I don't think you can lol

you could probably use this as a gainmap instead

You can, but you need to give input as a generalized PAM file, and there is nothing other than cjxl/djxl that can handle those

you could cheat by using a PNG source and setting the depth buffer as the alpha layer. that's just for if you want to test how it would compress. it's semantically wrong, of course

make sure it doesn't disregard color information it deems invisible due to the alpha value (I think by default it doesn't do this for lossless so you should be good)

unrelated'ish but I thought of a probably bad idea: lossy depth (would have to be lossy modular cause it's a extra channel)

tbh, not having vardct for extra channels is a little unfortunate

probably not too big of a deal, since extra channels will probably be most useful for scientific purposes and those will probably tend to prefer lossless regardless

still, I don't think it would have hurt

unless not having it is overall better in ways I don't realize

IMO something interesting is that pik (what VarDCT is based on) supports alpha so I wonder if it was DCT based back then?

probably complicates the spec/format a bunch having it or something

I think there might have been a convo in here a few years back about it?

that's what I'm guessing, you can say "but why not also this" to every limitation the format has, and while in isolation every one of those would be a good to have, in total it becomes a kitchen sink mess, is what I'm thinking

actually, I wonder if it has to do with block selection, because I know at least the block and XYB channels are encoded is unfortunately 'channel per block' rather than 'block per channel'. in other words, every color channel is subject to the same block selection

hell yeah nailed it

here's also an older message saying a similar thing

a less fun fact is that the decision to go 'channels per block' rather than 'blocks per channel' was done for performance at the cost of compression efficiency, while after the fact it turned out it didn't improve performance at all, so now it's just there, limiting compression efficiency 😔

but you know, hindsight...

oh yeah I heard/saw this in here a bit ago. guess it's just another thing to add to the imaginary hypothetical "JPEG XXL"

JPEG XXL will so much better than AV36 <:Stonks:806137886726553651>

I guess a question for another channel would be: when? because it's intended to be a long-term format, but as we invent new methods and create new needs, *eventually* we're going to want to move on

that's gotta be on the order of dozens of years, I wager

Lossless pik was used for alpha. Hardcoding the DCT mode for 3 channels allows a somewhat more efficient implementation and also allows signaling things like adaptive quantization only once per block instead of having extra signaling cost per component.

I don't think compression improvements alone can still justify a new codec after jxl, the gains just won't be worth the hassle. New functionality would be needed. No idea what that would be though.

Yeah that's fair. Also, the JPEG committee tends to identify new functionality needs and dedicates a custom little format to them, and given that JPEG XL is general purpose, a need would need to be very widespread and very important to be justified in a general purpose format

btw, my effort 10 attempt has been going for at least 40 minutes at this point :)

nice

diminishing returns so far, but maybe trying all predictors will make a bit of a dent

are you sure this is downscaled the right way? these edges look suspiciously artificially sharpened

I think Imagemagick defaults to bicubic? but no idea

ah, well, at least it's a lot better than the one provided by the author which is a lot more aggressive

There's the hacked simple-lossless binary we made that encodes PNG Alpha as Depth

Lossy Alpha works fine, so depth should too

too few options/control compared to full libjxl, I wanted to make a VarDCT image, I mean i guess simple-lossless would work if I just wanted a simple way to make an image with a depth channel and didn't care about much else. Also I'm not 100% sure but I think there *might* be a small error with that patch although I haven't investigated.

You can use the JXL as input for cjxl

oh damn i didn't think of that lol

guess I will do that since I kinda don't want to try and interact with stuff like PAM

Scrolling past that gave me a thought... Does dithered scaling exist? Averaging the pixels almost certainly doesn't land on a full number, so dithering could be applied. But it might end up worse than a consistent but less accurate result...

if you rescale in 16-bit precision, then you can apply dithering when converting to a lower bit depth

your wonderfully small files are now live :D https://slabserver.org/

At some point we should try expanding the predictor sets to include low color and progressive optimized choices

I guess it can be as large as you want

The image can be reproduce

It seem to be one of sample file from software called Mandelbulber

And the parameter they use for the image is also on the wiki commons too

Might be due to the old v0.9 quality regression, but in a quick test EPF 3 seemed to improve every image I tried it on compared to the default level of 1 at distance 1

is this normal? aren't the upsampling modes available in cjxl baked into the format, requiring like a byte to signal?

Only non-seperable is default

It uses an upsampling matrix, so there's a default one which you can override with whatever you want but it's a bunch f16 matrices

Though that does seem like a very large jump, normally it's only around 100 bytes for NN

if it's at least a 8x upsample I don't see how it can be less than 400 bytes higher

damn, nearest neighbor should've been baked in, how else will I brag about my tiny pixel art's small file sizes

have this 4x4 image I stole from a JXL thread somewhere

even a direct encoding of the 8x8 pixels per pixel image is smaller than the image with 1/64 the pixels but upsampled...

30, 80 and 300 bytes of overhead for 2, 4 and 8x NN

So I was probably thinking of 4x when I last tested it

I don't see how it would be only 300 bytes, it's 210 F16 values?

(I guess you're adding it up? 300+80+30 = 410 bytes, which is closer)

Right yeah, it's additive. Sorry, getting late here xP

noting this down for JPEG XXL

I have previously convinced myself that resampling is optimal for highest density in general. I would like to produce a new document carefully clarifying this.

I'm not sure if it's what you mean, but I did enable automatic 2x resampling at distance 10 or lower, since it usually gave a cleaner image at a smaller or equal size. Unfortunately 4x was just too blurry for distance 25, ect

No, referring to AccessViolation_'s use case just above. I guess [this](https://discord.com/channels/794206087879852103/803645746661425173/1240228597005025331) is the best surviving document. The tried cjxl commands included jixel's general default, so any density gains came from the resampling alternative. You might also ask questions like: which scale is densest? which options are broadly Pareto optimal? is there distinguishing behavior between consumption and production? what are the densest commands tailored to consumed resampled images vs untouched images specifically (noting resampled images may not end up 1:1)?

A random thought, but it's worth pointing out lossless JXL only uses 'global' group decisions (not Global MA). It would be possible to change the available predictor set per group, or use LZ77 only if a block is very repetitive/single color, ect. The same goes for per-channel settings, like encoding Alpha with optimal parameters compared to RGB, ect

Was messing around with lossy modular, seeing if I can give it a quality bump, and I found one of the most obvious examples of the B quantization yet It's turning green into teal because of all the extra blue caused by the precision loss. Lowering the B quantization brings back the green again

I wonder how well would "barely lossy modular" work, where you still convert to XYB but then do no (or almost no) quantization

for cases when you don't care about being bit-perfect, but don't want to lose precision anyways

oh wow, v0.11.2 came out recently

XYB is inherently lossy, but you'd essentially be encoding a progressive lossless image without actually being lossless if you used no quantization

I assume this *finally* has the fix for the lossless `-e 1`?

Nope

what?!

why was that not included?

https://discord.com/channels/794206087879852103/804324493420920833/1470736585388724379

I mean it's only as lossy as float computations are

Also <https://github.com/libjxl/libjxl/pull/4027#issuecomment-2854185744>

well, my benchmarking will wait for 0.12.0 then

https://discord.com/channels/794206087879852103/806898911091753051/1270048431737540669

I'm a bit surprised if it has that much of an affect tbh

Near-lossless was the main point of lossy modular, distances below 0.1 or 0.05 roughly (0.05 is the limit for VarDCT currently too)

But it also does pretty well on non-photo content, and if I can get this working, it might actually be competetive/beat VarDCT at distance 1... For better or worse

Initial testing, lossy modular in my branch is both smaller and slightly higher quality than VarDCT at distance 1. Only did 10 minutes of tweaking, so plenty more testing to do though

woah

is that for non-photo?

it must be

I tried two images with two target sizes each between latest modular, vardct and resaturation modular. One image was clear win for latest, other image was convincing win for resaturation (i.e. important improvements, but some tradeoff).

https://litter.catbox.moe/c2doj5kofl4vvbwb.zip

converting DNG to jxl cause incorrect color and also created artifact at edge

is this ffmpeg only issue?

and somehow this image also turned 90 degree counter-clockwise

I didn't even know FFMPEG supported DNG input

It's most likely that ffmpeg isn't reading any metadata that might be present in the dng. And it's almost always recommended to directly use a proper raw development program.

Could give this a shot, but support might be spotty https://tinydng.com/

Has a lossless option in the settings

DNG is not just an image format, it's also a syntax for a chain of image processing operations that is to be applied to it.

I wonder if oiiotool would convert it properly

iirc oiio uses libraw

I'm drive-by dropping this website that has some images that make artistic use of a low color palette, if anyone wants some easy images for benchmarking palette compression https://nchrs.xyz/

dithering?

Just did a bit of testing with the desaturation again. Subtracting all blue from the image definitely fixes the yellows, some pixels had the blue increased all the way up to 42 from 0 in the source image