JPEG XL

I can't wait to crash my phone viewing it again

AV1 is for normies, embrace Motion JXL

using inter-frame copy?

how would it work with seeking? does it have to decode the entire file for each frame?

block skiping

no, decoding of all frames need. JXL don't have keyframes, although I can make it, for example, every 500 frames. only there's no way to mark it. therefore, it is necessary to decode the entire file

> although I can make it, for example, every 500 frames You could implement pseudo-scene detection by saying if, say, 75% or more blocks are new and a keyframe hasn't happened in the last few frames, or a keyframe hasn't happend for the last 3 00 or whatever frames, then it should be a keyframe > only there's no way to mark it. therefore, it is necessary to decode the entire file The `jxli` box lets you list an index of frames, the frames listed are seekable keyframes

when the scene changes, the all blocks update I don't need to search for a keyframe, I just need to mark those frames in which 100% of the blocks have changed. but how? jxli is what?

> I don’t need to search for a key frame I’m thinking more like if all but 4 blocks update it’s probably worth it making the entire frame a key frame instead of trying to reuse those 4 blocks (it would also allow predictors to probably work better as it has the whole context) > jxli is what? The jxli box is a seek table, like the one found in FLAC files. It lists the offsets (bytes and time) of frames that can be reliably seeked to, ie key frames. Look at the screenshot file I sent for the specific implementation of it

No. firstly, I'm not at home right now and no one has answered whether there is a jpeg xl decoder for android with animation support. There doesn't seem to be one. Not exist? Second, I did not understand the principle. I have a ready-made jpeg xl animation and a list of frames that are marked as key. How do I work with this? what should be done and where? which software should I use?

Not sure about the first question as I don’t use android. For the second one, I’m not sure if any such software exists, you might have to write it yourself. You would just have to find each of those frame’s offsets into the JXL code steam (NOT the entire file) and write the bytes as the spec says to

if there is no such software. then why should exist software that can work with this for playback from keyframes?

I don’t think any software as of this time actually uses the index but it’s nice to have and could be implemented in other software if/when JXL mass adoption occurs

I think having occasional keyframes is good even not for just specifically seeking but for memory efficiency as it doesn’t have to keep storing old frames

Unless libjxl is either way smarter or dumber than i thought

I will not be programming the format itself, my work is outside the field of libjxl itself. I give it an apng and get the result until such functionality is added to libjxl, I will not do this mpv can't do key play, ffplay can't either. xnview is not working properly. and there is no software for android at all. I will not waste my strength on this. in addition, it is possible to do this by post-processing when the animation is already fully ready, comparing the past and current frame until there is a pixel change in the entire image. in general, let someone else do it. I'm interested in something more relevant.

considering that I haven't even found a normal software to create such an apng with an overlay. to wait for such people for another 5 years

<:Xjxl:822166843313225758>

finally, the algorithm I wrote for skipping blocks saves about 30% of size. Shrek is now at 3 fps instead of 2 fps <:JXL:805850130203934781><:jxl:1300131149867126814><:Stonks:806137886726553651><:Xjxl:822166843313225758><:logo:829708783336816671><:pancakexl:1283670260209156128>

just wondering what are the frames that are completely black?

for some reason, Windows displays frames with zero transparency in this way. this is RGBA 0 0 0 0

so those frames are effectively skip frames?

yes, empty frames are skipped

This is one of my last things that I did all day so that the frame would not be updated in parts if there were no updates, but would transfer the changes to the next frame. So I saved a lot if the frame moves a little, but entirely

global camera movement

Behold, the first episode of The Amazing Digital Circus in a JXL. -e 5 -d 14.1 @ 2 fps speed up 50%.

``` leo@gauss ~/Development/ffmpeg-devel/ffmpeg :) $ ./ffmpeg_g -y -i ~/Videos/test-clips/lossless-out.mkv -c libjxl_anim -f image2pipe ../test-images/bbb.jxl ffmpeg version N-117995-g56419fd096 Copyright (c) 2000-2024 the FFmpeg developers built with gcc 14.2.1 (GCC) 20240910 configuration: --disable-doc --disable-stripping --assert-level=2 --enable-libjxl libavutil 59. 47.101 / 59. 47.101 libavcodec 61. 27.100 / 61. 27.100 libavformat 61. 9.100 / 61. 9.100 libavdevice 61. 4.100 / 61. 4.100 libavfilter 10. 6.101 / 10. 6.101 libswscale 8. 12.100 / 8. 12.100 libswresample 5. 4.100 / 5. 4.100 Input #0, matroska,webm, from '/home/leo/Videos/test-clips/lossless-out.mkv': Metadata: title : Big Buck Bunny, Sunflower version COMMENT : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net GENRE : Animation MAJOR_BRAND : isom MINOR_VERSION : 1 COMPATIBLE_BRANDS: isomavc1 COMPOSER : Sacha Goedegebure ARTIST : Blender Foundation 2008, Janus Bager Kristensen 2013 ENCODER : Lavf60.7.100 Duration: 00:00:04.00, start: 0.000000, bitrate: 35329 kb/s Stream #0:0: Video: vp9 (Profile 0), yuv420p(pc, bt709, progressive), 1280x720, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 1k tbn (default) Metadata: HANDLER_NAME : GPAC ISO Video Handler VENDOR_ID : [0][0][0][0] ENCODER : Lavc60.20.100 libvpx-vp9 DURATION : 00:00:04.000000000 Stream mapping: Stream #0:0 -> #0:0 (vp9 (native) -> jpegxl_anim (libjxl_anim)) Press [q] to stop, [?] for help Output #0, image2pipe, to '../test-images/bbb.jxl': Metadata: title : Big Buck Bunny, Sunflower version COMMENT : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net GENRE : Animation MAJOR_BRAND : isom MINOR_VERSION : 1 COMPATIBLE_BRANDS: isomavc1 COMPOSER : Sacha Goedegebure ARTIST : Blender Foundation 2008, Janus Bager Kristensen 2013 encoder : Lavf61.9.100 Stream #0:0: Video: jpegxl_anim, rgb24(pc, gbr/bt709/bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn (default) Metadata: encoder : Lavc61.27.100 libjxl_anim HANDLER_NAME : GPAC ISO Video Handler VENDOR_ID : [0][0][0][0] DURATION : 00:00:04.000000000 [out#0/image2pipe @ 0x5c1569001500] video:12944KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.000000% frame= 120 fps=5.2 q=-0.0 Lsize= 12944KiB time=00:00:04.00 bitrate=26509.9kbits/s speed=0.172x leo@gauss ~/Development/ffmpeg-devel/ffmpeg :) $ jxlinfo -v ../test-images/bbb.jxl JPEG XL animation, 1280x720, lossy, 8-bit RGB num_color_channels: 3 num_extra_channels: 0 have_preview: 0 have_animation: 1 ticks per second (numerator / denominator): 30 / 1 num_loops: 0 have_timecodes: 0 Intrinsic dimensions: 1280x720 Orientation: 1 (Normal) Color space: RGB, D65, sRGB primaries, 709 transfer function, rendering intent: Relative frame: full image size, duration: 33.3 ms ``` <:PauseChamp:755852977574510632>

libjxl_anim coming soon

<:megapog:816773962884972565>

tomorrow I will post my algorithm for compressing animation by updating jpeg xl blocks. with use instructions. I must say right away that I did the calibration of the entire algorithm for 128x64 and 3 fps. Therefore, you will most likely have to do a large recalibration of values for different sizes and frame rates. And it's also python code that's half written in a neural network, well, that's so that you know

in fact, I've done more than 2000 parameter iterations comparing the resulting size of the jpeg xl file after full compression. And the pixel difference of the resulting image compared to the original.

Here is the code that takes PNG files from a folder and encodes the APNG in frame-overlap-frame mode. Using python pillow. PngImagePlugin.Blend.OP_OVER aka APNG_BLEND_OP_OVER

128x64 3 fps -d 1.7

wait..... what the 422 on all apng output 💀

ffmpeg input vp9 yuv420p output png, rgb24 but is not <:NotLikeThis:805132742819053610>

another day when software that is used by hundreds of thousands of people does perfect bullshit because it is broken

fuck png now i use jpeg (because for some reason RGB output does work like 4:2:2 in FFMPEG 7.0.2 but jpeg yuvj444p works well) - considering that the result will still be much more compressed in jxl, I think this is not so important

Compress with jpegli in low quality and then recompress with jxl for 20% smaller <:galaxybrain:821831336372338729>

No blurryness

<:Poggers:805392625934663710> jpeg with alpha channel

in any case, this is a good reason to add the input option for jpeg and webp before the release

fixed

Could we just take the Jpegli encoder as JXL lower effort ?

but classic jpeg is -e 3

jpegli is basically e3 without xyb

Could probably make e1 and e2 'JPEG Compatible' like was mentioned long ago, allowing decoding to a regular JPEG without futher loss

So literally just jpegli > cjxl?

Pretty much, an old idea was having JXL files that only use JPEG compatible features, so you can transcode back on the fly at the highest quality

Maybe have JXLs that are mostly JPEG transcode able but have very slight EPF or Gaborish so they look slightly better as JXL then JPEG

not sure about that, IIRC jpegli does adaptive quantization that e3 doesn't

ah nvm then

I don't seem to have anything else to add to my code, and since it's unlikely that anyone needs it anywhere, I'll post the sources Only here as soon as I run the compression cycle for the last time and make sure it works

I hope there are normal python programmers here exist

When creating live wallpapers, it turned out that Saving PNG requires all uncompressed images in memory, for 1920x1080 it came out to about 5 gigabytes per 300 frames. The cjxl has the same requirements, but slightly lower

moreover, the processor load when decoding jpeg xl 1080p animation is greater than if it were AV1 1080p yuv444 10 bit <@853026420792360980>

ffplay refused to decode it as animation, it works in MPV

sounds like proper video codecs are better at video

do you have a (small) sample that ffplay refused to decode but mpv played fine?

perfectly

Yes, I was just able to do that

late I realized that I should have done a larger block size analysis

so well

so I believe ffplay is failing because the video stream isn't able to be decoded in realtime

and mpv is able to handle that better

VIDEO to Jpeg XL ``` 1. create folders input_images output_images 2. ffmpeg -i input_video.mkv -vf "fps=3,zscale=w=128:h=-2:filter=spline16,crop=128:64,format=gbrp" "input_images/frame%05d.png" 3. pip install pillow 4. python shrek-to-jxl_v1.0.py 5. python APNG-BLEND.py 6. cjxl -e 7 -d 2 APNG.apng output.jxl ```

why not just us apngasm

<:JXL:805850130203934781> Jpegxl Video Codec release

not sure why you're splitting the input into jpeg frames

I think this

laziness. Besides, I'm python enjoyer and I understand how to do it the way I did. I think you can write a script yourself to use apngasm because everything works in stages

I found a bug yesterday. ffmpeg converts RGB png in 4:2:2 sampling format

so https://discord.com/channels/794206087879852103/794206170445119489/1313707744204951563

huh? what are you talking about

you can't have 4:2:2 subsampling for RGB

``ffmpeg.exe -ss 1:00 -i "D:\MURDER DRONES\THE AMAZING DIGITAL CIRCUS： PILOT [HwAPLk_sQ3w].webm" -vf "scale=128:-1:flags=spline+full_chroma_inp,crop=128:64" OUT.png``

yet I get

because you're telling it that you have full chroma input and it doesn't

i telling where?

the flag `full_chroma_inp` tells libswscale that you're using full chroma input

but the input doesn't have full chroma

remove that flag

But it doesn't matter.

what kind of place should the 422 format make it? yes, even logically. Of course, I checked it yesterday, it gives the same result. FFMPEG is just broken

there shouldn't be 4:2:2 anywhere

FFmpeg isn't "just broken"

what version are you even using

7.0.2

can you provide a sample that fails?

sample of what?

any image?

a sample file and command line that doesn't do what it's supposed to

when you operate on that file

and the full, uncut output of the command

because the PNG encoder doesn't do anything "4:2:2" related, because it's just an encoder

and switching to jpeg won't fix that

i use https://www.gyan.dev/ffmpeg/builds/packages/ffmpeg-7.0.2-full_build.7z from https://www.gyan.dev/ffmpeg/builds/ for windows

okay, but can you provide a sample file

because I cannot reproduce what you are describing

Okay now. Wait a minute. I'll find an old ffmpeg to extract the sample

because "out.png" doesn't have subsampled chroma

it's a PNG

it can't have subsampled chroma

But wait. are you talking about the file that shows the color?

no, I'm talking about `OUT.png` that you posted

it isn't subsampled. PNG doesn't support subsampling.

end of discussion.

because obviously the error is happening with ffmpeg

there's no error

if you think that the PNG encoder is converting RGB to and then to 4:4:4 to 4:2:2 and then converting back to 4:4:4 and then back to RGB then

well I don't know what you think is happening

but that is not happening

I have bad news, ffmpeg had the same error six months ago

there is no error

I don't know why you think there is an error

ok go <#794206087879852106>

I posted my code to create the jxl animation, but no one will see it now <:PepeHands:808829977608323112>

color fix

`-e 11` go brr

<@226977230121598977> does your code work on images other than 128:64?

im running it on a 640:360 24fps animation just to see what happens

Yes, of course

the problem is only in determining the limits like skipping frames from the level of changes or visually sufficient changes to change frames. It all requires testing output with the original. To understand which parameters are better, I made comparisons of 2000 combinations of parameters over days

but it also works

256x144 @ 24fps

Is there a recent version of the specification available here? The latest I can find is https://discord.com/channels/794206087879852103/1021189485960114198/1169087534060556339 from a year ago

iirc once some guy created a discord account just to share that spec freely (?)

https://discord.com/channels/794206087879852103/794206170445119489/1288880183603888304 And there is a newer draft here: https://discord.com/channels/794206087879852103/1021189485960114198/1256224842894540831

why would there be a two DHT scans in a row?

``` STRUCTURE OF JPEG FILE: cover.jpg address | marker | length | data 0 | 0xffd8 SOI 2 | 0xffe0 APP0 | 16 | JFIF......... 20 | 0xffdb DQT | 67 89 | 0xffdb DQT | 67 158 | 0xffc2 SOF2 | 17 177 | 0xffc4 DHT | 28 207 | 0xffc4 DHT | 24 233 | 0xffda SOS ``` e.g.

likewise, two different DQTs

Different Huffman trees

ye, but why would there be two of them?

Software was probably not smart enough to join them together

it was jpegtran that just created this

some encoders seem to emit separate markers for each table

Some encoders like to waste 2 bytes

is it actually just a 2byte waste?

maybe there are some compatibility concerns, idk

When were multi lane highways added to the JPEG spec?

Multiple Huffman trees*

Too much urban planning chat

Is this related to duplicate quant tables not being removed? There are no current programs exist that can detect and strip it out iirc.

Found the convo: https://discord.com/channels/794206087879852103/805176455658733570/1270873751021682841

unrelated, I'm just curious because I found one in the wild like so

there are a lot of weird jpegs out in the wild

ye but this one was created by turbojpeg's jpegtran

is why I was confused

have you tried mozjpeg's jpegtran?

not yet

/usr/bin/jpegtran is libjpeg-turbo on my system

I can't replace the system libjpeg with mozjpeg yet (or jpegli) because on arch there's some new functions used that make certain things (e.g. libtiff) fail

i see

well post the results when you can

my imagemagick does the same thing, maybe it's libjpeg-turbo

looks like mozjpeg probably doesn't

there is a python library that takes a bytes() object and uses it to optimize a jpeg using mozjpeg

and I wrote a stupid garbo wrapper for it

```sh #!/bin/sh for arg; do if [ -r "$arg" ] && [ -w "$arg" ]; then if [ "$(file --brief --mime-type "$arg")" = "image/jpeg" ] ; then slurp <"$arg" | python3 -c 'from mozjpeg_lossless_optimization import optimize; import sys; sys.stdout.buffer.write(optimize(sys.stdin.buffer.read()))' | slurp "$arg" else printf >&2 '%s: %s: not a jpeg\n' "${0##*/}" "$arg" fi fi done ```

slurp is a C program I wrote

yes we're doing this

https://0x0.st/XhfE.c/slurp.c

probably it's leaky

¯\_(ツ)_/¯

slurp reads in from stdin in full until EOF, and *then* it opens the output for writing, which is either stdout or the first argument

so if you run `slurp <file | some_filter | slurp file` it slurps the entire thing in, sends it through the filter, and then when the second slurp sees EOF it writes it back

allows you to do things like in-place-editing if the thing doesn't provide it

without temp files

cool, can you do this with jpeg xl or any supported image format I imagine?

IIRC there's a huffman table for the DC coeffs and another one for the AC coeffs, and you can define them in two separate DHT markers if you want

Like how Luna and chroma get different DQTs sometimes?

same with quant tables, you can have up to 4 of those (you can use a different one for each component, if you want), and you can choose whether to put them all in one DQT marker or to use multiple markers

Is there a known way to estimate the size of JPEG XL output given width, height, bit depth and the quality value?

It would depend on the content as well

You could get a rough BPP and multiply by width*height but that’s

Thinking about something akin to the preview here when saving a JPEG in Photoshop

<:monkaHmmm:738585281061453936> ic

Doesn't that just encode the image and read the file size?

I'm not completely sure but I don't think doing that with JXL is feasible <:peepothink:715229420838846466>

or maybe it is

more like its a pain <:cutegiggle:758819454518951936>

I know ImageToolbox on mobile has some kind of filesize estimation too, where it adjusts once the file is finished encoding, though it acts... Strangely....

I mean, you could encode at effort 1. The BPP from that is pretty close to higher efforts

That could work as a "preview" mode

Could also downscale the image and then multiply the bpp back for a rough estimate

Since e1 is only a lossless thing, lossy is still the same as e3

I don't know if downscaling is a good idea. The internal metrics in jxl might overly deviate depending on the downscaler used.

Due to how context-sensitive Jpeg XL is, I'm not sure how consistent that would be

Fair point, especially with noise and fine detail in the image that vastly increase filesize

I think effort 1 is fast enough tbh

Does egg 1 even do anything for lossy

I guess maybe the internal modular bitstream?

Actually yeah what effort do the internal modular bitstreams use?

Effort 1 is the same as effort 2 for lossy

There was talk about implementing a super fast lossy mode iirc

Similar to fast lossless

Can lossy lossless modular speed?

I like your funny words magic man

Found it: https://discord.com/channels/794206087879852103/794206170445119489/1297608882444701728

Am I understanding correctly that if I have a recompressed JPEG file that I want to edit I would need to edit it as JPEG, save as JPEG then recompress again?

otherwise I would either save it as lossless JXL which would go from lossy -> lossless and increase the size a bunch or just save as lossy jxl

what specifically do you mean by editing a JPEG? do you mean purely lossless operations with special tools that don't involve decoding the JPEG into pixels such as cropping, converting to grayscale, or editing metadata? or do you mean like decoding the JPEG to pixels and then editing it?

decoding jpeg to pixels, editing it

lossy to lossy

Then just save it as lossy jxl

I guess I can do that <:yep:772595738391740426>

Assuming your editing software supports JXL, it's also better to decode the JXL directly rather than reconstructing a JPEG.

can you do what

jxl -> slurp

slurp doesn't interact with the contents of the file at all

it just reads and writes them in a specific way

interesting

it's just a C file

it reads from stdin entirely into memory, calls fclose, and only then opens the output for writing

If I losslessly convert say a PNG to a JXL, is there a possibility of losing some color information thanks to quantization when it transforms RGB into XYB or is it always mathematically exact?

just happened to be looking around ffmpeg jxl code and found this line. Seems to be about your question

also jxlinfo uses this check to identify if it is defintely lossy or not

so it seems that if you use original profile, it wont convert to XYB and can be mathematically exact

correct, lossless does not typically use XYB (only if you specifically request that via the API)

if you just use `cjxl -d 0` or whatever it won't use XYB

Ah, so I guess modular can just take the take the three channels from the original and copy them exactly instead of turning them into XYB

RCTs are separate and can store it as different colorspaces, but are fully reversible

right, gotcha

I thought the transformation into XYB was what RCT referred to. Is it not?

I *think* even VarDCT can maintain colorspace with `uses_original_profile=true`, but I'm pulling this out of the depths of my brain

RCT is a modular feature, Reversable Color Transform, purely helps in decorrelating the planes rather than perceptual quality like in VarDCT

Does this seem reasonable or not?

trying to use sane values for my encoding flow

Probably best if <@794205442175402004> fact checks us, in case I'm wrong

Do JXL files (lossy or lossless) support HDR colors while png ones do not?

nah it's okay dude I got the information I came for

xD

RCT is reversible and not useful when used on XYB data — it's only used when doing lossless. When doing XYB, there is no point in also doing RCT...

16-bit PNG with proper colorspace tagging (cICP or a modern HDR ICC profile) can certainly do HDR.

where did you get that at?

here are some examples: https://sneyers.info/hdrtest/

What about this? https://en.wikipedia.org/wiki/Comparison_of_graphics_file_formats#Technical_details

"No, see discussion" unclear what that's supposed to mean... PNG can certainly do HDR in i.e. Chrome, I wrote that code 😛

https://en.wikipedia.org/wiki/Talk:Comparison_of_graphics_file_formats#PNG_is_not_a_High_Dynamic_Range_Image_Format Found the original change, some dude in 2010 thought images must store data in floating point to be considered HDR <:DogWhat:806133035786829875>

I can't believe misinformation like this hasn't been corrected for so long 😅

yeah that was true in 2010

or true enough

it is not true today though

<@179701849576833024> <@207980494892040194> That was the discussion? So can PNG support HDR or not?

it does, see this

Or the message directly in-between your question and the message in your follow-up question ;P

Yes

When there is no ICC profile should I be encoding with SRGB or LinearSRGB? <:peepothink:715229420838846466> Right now my logic is if ICC then use ICC otherwise use SRGB

depends on input, for instance, with exr input you should default to using linear

would always be 8-bit or 16-bit RGB and never floating point

sRGB is probably a safe assumption for most input formats, not all but most

ofc you should be reading cicp too

an ICC alternative <:monkaHmmm:738585281061453936>

thanks for the info

**Update for JPEG XL Plugin for Photoshop** (https://gitlab.com/SuperSaltyGamer/jpegxlformat) * Now supports lossy encoding. * Now supports EXIF and XMP. * Added save dialog with options for saving EXIF, XMP, compressing boxes and image quality. * Added info dialog when opening lossy images. * Added about dialog with build info. * Fixed crash with >2GB images.

If nothing crazy surfaces I think this will be the last update for a long while. Highly recommend anyone who already started using the plugin to update

Can JPEG1 have cicp tags?

And hdr transfer curves

I don't think so, but you can embed an ICC profile with CICP...

Most decoders will decode to 8-bit though, which will cause banding for a PQ image...

I wonder how properly dithered 8 bit pq looks

Like with libplacebo

Also I wonder if libjxl dithers correctly taking nonlinear quantization into account

Most software dithers incorrectly by not taking linear light scale into account

Not sure how much that difference matters when dithering with 3x8-bit colors, as opposed to dithering with a small palette of colors...

Well, dithering isn't supposed to change the brightness... I don't know how bad it looks in that particular 8 bit scenario, but libplacebo does it right.

Maybe someone ought to just integrate libplacebo into libjxl

It would be a justifiable dependency

The difference should be small, since the sRGB curve looks close to linear when you're looking at 1/255 increments

Possibly noticeably bad near 0. Going from sRGB value 1/255 to 2/255 is a 4.6x linear brightness increase, while going from 2/255 to 3/255 is a 2.4x increase. Though here the absolute values are very tiny.

The absolute values but human vision is more sensitive to relative differences.

Absolute differences mean nothing.

Either way, there's a right way and a wrong way and most software does it the wrong way. And it's known that the wrong way is lower fidelity.

it’s a bit of both since we are less sensitive to equal relative differences when the absolute values are small (per the Barten ramp)

Well, that sounds quite complex to account for that, afterall the quantization error to diffuse is in whatever colorspace you happen to be in.

Would also necessarily add more noise, no?

Does anything do it this right way?

Most software do ordered dither incorrectly when using a fixed palette. Though I'm not sure how would you do that. I guess you can consider color space as 3D space, and palette colors as points, using tetrangulation locate 4 points inside of which original color is located, and do ordered dither based on percentages of those colors? I'm not sure how it would handle points outside of convex hull of those points, or how badly it would look if tetrahedron is very stretched.

When converting 16-bit to 8-bit, gimp gives you the option to dither in linear light.

Yes. Like libplacebo for example, or libimagequant. But most people are too lazy and dither without taking into account actual brightness

It's rare unless high fidelity image processing is the main focus of the software

You could argue that's the main focus of libjxl too

Idk if libjxl dithers in linear light or not

But I would be surprised?

No, it wouldn't add any more noise, and it's not that complicated. You would just be calculating in linear light. Doing it the wrong way adds more noise/distortion because it changes the brightness of the image. Just like resizing an image in linear light vs nonlinear

Basically, when reducing precision, and you have a pixel that falls somewhere in between 2 possible values, you do the comparison in linear light instead of the raw values.

So if a pixel is 0.45 but the only values are 0.4 or 0.5, if you look at the raw value it's 50% of the way between... but if you convert that nonlinear value to linear light value first, then maybe it's actually closer to 90% between the two not 50%

for a simplified hypothetical explanation

Let's say 0.5 represents a brightness that's 3x as bright as 0.4

Then 0.45 is not halfway between the two

But that's how most lazy dithering works

In software that doesn't really care about high quality like builtin os image viewers or ms paint style apps

So in ordered dither 50% of the pixels will be 0.4 and 50% will be 0.5 and the result when looking at it, is a different amount of light and brightness compared to the actual 0.45 value the dither is supposed to be approximating

Since the brightness values were not compared by how bright they actually are, but instead by some arbitrary, nonlinear number that doesn't have a linear correlation with how it actually looks

No, that does not work. Simply doing that would not get rid of the quantization error you get in non-linear light.

Well you can't get rid of quantization error but you CAN properly dither and get a best effort approximation without accidentally shooting yourself in the foot by comparing the wrong number that isn't the actual brightness

Well, getting rid of simple quantization error is the whole point of dithering.

Well, you technically still have quantization error. But it's spread out.

You are spreading it out and statistically and perceptually shaping it

But anyways it's the difference between comparing the correct number (brightness) and comparing some other, irrelevant number that's the wrong thing to compare.

Yes, but simply applying the normal dither noise from linear light to non-linear light won't get rid of the banding entirely.

You're comparing the brightness, not the binary encoding of the number

I guess ideally you want to do error diffusion dither where the error that is to be diffused is measured in linear light, while it's always applied to non-linear light?

but error diffusion dither is slow of course

What dither method you use makes no difference to what I'm saying

well, it does, makes the whole implementation way more complex

Either way you want to actually compare the brightness values, not some arbitrary and wrong number

Not really... it's just comparing the correct number instead of an incorrect number

like, simple random noise dithering is extremely simple, but doing so while targeting noise distribution in linear light, if that's the correct way to put it, becomes a rather compelx issue to solve

simple dithering doesn't compare anything, it's just noise

You are comparing the original value to the available, possible quantized values

for random dithering you just add noise

before rounding down to the target bit depth

There are lots of comparisons

if you do error diffusion dither, but that's pretty slow

Ordered dither has a dither matrix and the level/intensity of the matrix at that pixel depends on how far between the 2 possible values that pixel is

right

but ordered dither is pretty imperfect anyway

Maybe I'm not the best at explaining it clearly but ordered dither is super fast and to do it properly you need to compare brightness values

Not some other wrong value

The dither matrix has a certain number of steps or levels depending on how close a value is to value A or B

Jon is gifted at explanations

If the difference between the two nearest values is 1/255 and the transfer curve is sRGB then I doubt the difference between dithering correctly and dithering naively (nonlinearly) is important in practice. Doing it incorrectly would basically darken things by a small fraction of an off-by-one in 8-bit...

When doing PQ in dithered 8-bit it will make a somewhat larger difference, but the difference between no dithering at all and lazy dithering will still be vastly larger than the difference between lazy and correct dithering.

yes, that was also my thought, either target precision is high enough that it barely matters, or ordered dithering could barely compensate for it due to lack of available color values, it's also quite restricted spatially afterall

When doing something like halftoning for printing, which is kind of like dithering to 1-bit, the difference between correct and incorrect dithering is going to be much more substantial

Anyway, not sure how libjxl does it, it should in principle be possible to do it correctly since conversion from. XYB to the output space passes through linear RGB anyway...

Here's a good article. https://threadlocalmutex.com/?p=93

You'll notice dithering incorrectly causes darkened stripes similar to banding or quantization noise

But slightly softened by the incorrectly applied dither

It's better than no dither at all but it's an error to use an incorrect arbitrary value instead of the linear brightness value.

But like Jon says, the amount of distortion caused is limited and always less than the difference between discrete quanta

Still, an error is an error... if you ask me...

The incorrect dither looks halfway between undithered and correct dither

At the end of the article it says: > Anyway, the result using an approximate (and cheaper) gamma exponent of 2.0 is visually completely indistinguishable from applying the proper sRGB conversion for the purposes of dithering, so there's that. But then he doesn't elaborate on that. That's interesting...

XYB uses a gamma exponent of 3 so I wonder...

> Linearizing color values before computing and distributing the error, these artifacts disappear completely: not _completely_, as far as I can tell

but yeah, it’s not necessarily _that_ trivial

in audio, for nonsubtractive dither, triangular noise is generally preferred over rectangular (uniform) noise because it not only makes the mean of the quantisation error zero but also decorrelates its standard deviation from the signal, preventing noise modulation

(https://www.researchgate.net/publication/236340550_Quantization_and_Dither_A_Theoretical_Survey)

(subtractive dither for comparison – that’s when the original dither noise that was added to the signal before quantising is then subtracted from the quantised version)

one image reminded me of one of my mistakes when I was making an algorithm for converting a spectrogram back to sound

what I hear in my head when I see those waveforms: https://youtu.be/gVWiE2dHEdU?t=39

This sound reminded me of my "song" from 3 years ago https://youtu.be/q2h9LfMx_qs

I just tried to comvert a jpg image thats saved with CMYK instead of RGB but well seems like thats not supported. will it be supported eventually or will it be like this forever?

there are several issues on GH for CMYK support see https://github.com/libjxl/libjxl/issues/3884

yup that command line error mesage is from cjxl

CMYK (YCCK) jpeg transcoding is not supported in current spec because Black channel cannot be VarDCT-encoded

so i gues if i want these images in jxl i have to convert them first to rgb and accept the conversion quality loss and thats not going to change anytime soon

or well... maybe original Black channel data could be stored as unknown data in jxl, not sure

image formats are much more complicated than i initally thought

You can store the image in CMYK natively in JXL, you just can't have *lossless* JPEG transcoding

So, `-j 0`?

yeah

I suppose technically you could losslessly transcode CMY as DCT data and then K decoded to pixels and stored losslessly.

would that be better than defining an annex N extension?

to the frame header

Or maybe K as a layer?

But it's better to just not have it encoded as cmyk lol

The K channel has to be an extra channel so not VarDCT, though I suppose you could make a layer that pretends to be another CMY layer but it actually encodes K in its VarDCT (say using the Y of YCbCr applied to CMY, with Cb=Cr=0), and use kAdd blending or something, maybe that would kind of work

It's not just a header thing, generalizing VarDCT to deal with 4 channels would be a bit of a pain because everything is hardcoded for 3 channels: the quant tables, chroma from luma, the definition of gaborish/epf/noise/splines etc

I meant encoding the K channel in a custom way and sticking it in an Annex N extension

which would maybe be some sort of modular rectangle of coefficients or something?

I guess it wouldn't be impossible to allow using some kind of VarDCT for extra channels, but that would be an extension that is totally not backwards-compatible

The idea of the extensions was to use them only for backwards compatible stuff, like storing info that can help decode faster (but without it, it still decodes), or maybe some subtle new ways to post-process (where it looks better if you do it like that, but it still decodes to something reasonable without knowing about the extension)

I think it's a bad idea to try to losslessly transcode a CMYK image when CMYK is a terrible and inefficient way to store the image in the first place.

No point in losslessly preserving stupidity

CMYK is not meant for long term storage, it's meant for sending to a printer

but the CMYK can contain more information than an RGB version that looks the same on-screen. E.g. deep blacks and decisions about how to separate K cannot necessarily just be derived from an RGB version...

though I kind of agree that it's basically one big confusion between something that should be handled at the level of the image reproduction technology and something that should be considered the actual image data

(arguably RGB has the same problem since it's basically representing an image according to how screens work)

how many CMYK workflows actually involve thinking about how ink work and how many just have an artist drawing according to what looks good on the screen?

I was thinking of the layer method too. Then reconstruction could *probably* still be possible with something like the jbrd additions already in the work for UltraHDR, ect

Found it: https://perso.liris.cnrs.fr/ostrom/publications/pdf/SIGGRAPH99_MultiColorDithering_600dpi.pdf Ordered dithering for fixed palette. It's a paper from 1999. I wonder why it's not impletented anywhere, if it's a solved problem. I expected at least ImageMagick to do it correctly.

I was reading a 1994 paper of perceptually guided quantization matrices for JPEG encoding that NASA made, yet it seems like jpegli is the only encoder to actually use it and not just a general 'optimized' table for all images based on a metric

But CMYK data is usually automatically generated by a printer driver...

Keyword "can"

Thanks!

Obviously if CMYK is just algorithmically generated from RGB then it doesn't contain any extra information. But this is not what happens when e.g. illustrators are authoring images in a CMYK working space in tools like Adobe Illustrator or Photoshop.

how often does that happen though

and then how often is that work saved as a jpeg

and then how often are those JPEGs required to be losslessly converted

<https://cloudinary.com/tools/jpg-to-jxl>

Hmmmmm

no loss in *visual* quality maybe

no loss in [some arbitrary measure of quality]

a compelling case could be to enable rich black scenarios

https://en.wikipedia.org/wiki/Rich_black

where all four CMYK values are set to something greater than 0%

and such values can be determined by the author, not algorithmically from RGB by the printer/converting software

Hey everyone! Has everyone tried to use JXL for HDR images? I'm trying to save space by converting my EXR photos to JXL images. The EXR converted to JXL via cjxl looks good on some software and on other ones the images are completely messed up! To my understanding a lot of software handles HDR in an inconsistent way... The only formats I found safe to use at this time of writing are PNG (exported from Krita) and EXR itself... Has anyone some experience or any tip about this topic?

I honestly would prefer to use JXL since a EXR image weighs about 200MB, a PNG one about 100-120MB and a JXL weights about 20-30MB

Talking of XYB color space, it recently occured to me that whenever I have looked at various JXL files I've created using different cmd line options both lossless/lossly and later inspecting them with jxlinfo *.jxl or identify *.jxl (imagemagick). I've only ever seen sRGB mentioned in the output, never actually seen mention of XYB in the output anywhere on any of my files. Should I be seeing XYB on at least some?

JXL handles EXR really well since it wont truncate ranges greater then 0.0..1.0, but yes, a lot of software will handle HDR weirdly, especially if your image containes values greater then 1.0

it's probably worth stating which images break

JXL uses XYB internally but the decoder has it's own color management to avoid inconsistent results, so it gets converted back to sRGB or whatever the input was when viewed

I'm assuming you're fine with it being lossy? Because that's what it'll be doing by default

Mainly JPEGXL, trying to have back and EXR or a PNG with HDR from a HDR JXL in most software is a dead end

oops, im dumb

I meant image viewer

Yeah! They're images taken with a camera

mb

IrfanView (completely off) surely

JXL to EXR is surely broken both in Affinity Photo 2 and Krita triyng to reopen the EXR in Luminance HDR brokes almost all tone mapping modes In order to have a 'correct' the only way was to convert the jxl to png and then the converted png back to exr

Huh, I'm surprised you have EXRs, they store the data as floats rather than integer numbers. Cameras are usually around 12bit integer

Do you have color management enabled?

I build the file from Luminance HDR (using three camera JPEGs). Luminance HDR only exports EXR files and tone-mapped (SDR) files

I also have the RAWs but the camera in-chip processing is better than any software I have installed on the pc

Never tried Lightroom Classic however

Have you tried RawTherapee's 'Match JPEG' button? Tried to find the settings to match the in-camera processing based on the JPEG preview

I never knew about that, I opened RawTherapee but I cannot find it lmao

Any clue?

I think it's under the exposure tab, just moved from my computer annoyingly

does using djxl to go from jxl to exr not work?

Same problems as above, trying to tone mapping the reconverted EXR in luminance HDR results in a completely black image

If, instead, the exr is generated from a BT 2100 with PQ PNG the error is not present

even if the PNG comes from the JXL

that’s odd, `djxl` should generate an EXR with the same range as the original, and the same `whiteLuminance` if present

Almost sounds like incorrect padding bits again

But that was only an input error from people padding the wrong side of PNGs

can you provide a sample problematic image?

Just out of curiosity, is there a mode in JPEG XL that will VarDCT encode the source image, then store the residuals of the source image minus the compressed image, and modular mode compress those, effectively giving you a VarDCT-based lossless mode? I had a thought that you could basically turn any existing lossy image or video compression format into a lossless format by just entropy coding the residuals after lossy compression for every frame

I think you might be able to make an encoder that does this by using a VarDCT layer and one layer for the residuals and blending them in some way?

This wouldn't work as VarDCT is inexact

so different decoders or even the same decoder on different hardware may give results that are slightly different

due to floating point rounding issues and optimisations

the spec explicitely allows a certain variation

also for lossless encoding a specialised lossless encoded will usually work better, especially given how advanced JPEG XL's Modular mode is

most lossy encoding systems also allow tolerances

This also already exists for video, but I can't recall the name... It's a side payload to the standard file that adds back the missing residuals to improve quality

it can work if you specially design a format where its lossy mode is exact, like WavPack

or retroactively define an exact decoding procedure, like IIRC JPEG XL

Ahh that's an important point. Reminds me of the decisions around WebAssembly, they had to weigh the importance of portability with identical behavior everywhere and floating point performance

in audio its not uncommon, DTS-HD MA, MPEG-4 SLS, WavPack Hybrid all come to mind

With AVIF having pretty poor lossless performance on synthetic images I actually considered that they might be doing exactly this lol. DCT compressing a flat color would add artifacts which it would then have to store the residuals for to undo

Found it https://www.lcevc.org/how-lcevc-works/

If you converted the image to float 32, it *should* have enough exponent bits to be lossless. I know Jon mentioned similar with subtracting frames to encode residuals in animations https://discord.com/channels/794206087879852103/804324493420920833/1294404142713798657

Yeah, that's pretty much what they do. It doesn't have a lossless path, it just does lossy and uses the low quality residual encoding

oh dear...

what a beautifully designed format

No way lmao

welcome to AVIF 🤷‍♂️

> Lossless is a variable which, if equal to 1, indicates that the block is coded using a special 4x4 transform designed for > encoding frames that are bit-identical with the original frames. I stand corrected, they do change the encoding... To only use 4x4 blocks instead....

I just invented lossless AVIF and it sucks. I'm putting this on my resume

so it becomes worse JPEG but lossless?

"Reinvented Lossless AVIF, but I realised it was stupid before trying to market it"

"Just use JPEG XL"

Looking at the 700 page spec, that was the only section actually specifying a difference. The other 40 mentions were just "If Lossless, don't deblock. If Lossless, don't use the loop filter"

"If lossless, disable half of the coding features that make AV1 good"

Oh, and this > Note: When using both delta quantization and lossless segments, care should be taken that get_qindex returns 0 > for the lossless segments. One approach is to set FeatureData[ segmentId ][ SEG_LVL_ALT_Q ] to -255 for the > lossless segments.

So you need to make sure the lossless is actually lossless

On top of setting it to RGB and all the other nonsense outside the spec itself

But anyway, Kampidh's JXL frame tool might let you do that, or at least with minimal tweaking

https://github.com/kampidh/JXL-Frame-Stitcher

I wish it had a way to directly insert a JXL file though... I'm pretty sure it re-encodes JXL art, ect

Ah neat. Aside from it not being technically lossless because of allowed imprecision, it would be interesting to see how this method conceptually would compare to lossless modular mode encoding of *photographic* images

Again, if you convert it to float first, you actually can make it lossless, just not good

The link there has some more info

comics book with a single JXL file [⠀](https://cdn.discordapp.com/emojis/852007419474608208.webp?size=48&name=av1_woag)

even seekable! (if you assume one jxlc box and no cross-references)

VIPSview has pages support IIRC

Correction, vipsdisp

Surprised Irfanview doesn't, since it already has pages for PDF

Do splines "belong" to a particular layer?

Is there a special "spline layer" or do they have to be assigned to an existing frame/layer?

existing layer, as far as I remember

Is there currently a tool that can join multiple jxl images into one larger image? Like jpegtran but ideally without that pesky factor of 8 thing? I know the answer is probably no haha

The answer is yes

I think jxl-frame-stitcher does this

With JXL multiple frames can be stitched together losslessly into a larger composite image.

No 8x8 limitation

not larger in dimension tho, as it depends on the first frame

But the canvas and frame dimensions are different

Idk if jxl-frame-stitcher supports it but there is no technical reason why you cannot join smaller images into a larger one

You could just as easily stitch a smaller image into a larger, blank, transparent image.

In other words no, not currently. You'd want a tool that increases the canvas size to the joined size, then layer the JXLs next to each other with an offset to do it the most simply. It could probably be done with a raw codestream though using 2048 DC blocks

If jxl-frame-stitcher doesn't let you change the canvas size, then you can simply create a blank, transparent image of whatever canvas size you want.

And stitch whatever images you want onto that

As a workaround

And maybe it will even let you delete the blank base canvas image as the last step after you finish positioning the other frames

I have not tested jxl-frame-stitcher yet though

One thing to keep in mind is when stitching multiple frames together the image will progressively load one frame at a time unless the frames are written in an interleaved way

Like how libjxl interleaves the metadata after the preview image

Like I said before, IIRC it re-encodes the images, so if you're using lossy then it'll degrade further

That sounds extremely unlikely. That would defeat the entire purpose of the frame stitcher.

Unless it's primarily an animation tool...

I thought a crucial purpose was to be able to losslessly stitch frames...

There isn't an option to avoid reencoding?

<@274048677851430913> ?

Nope, it always reencode sadly

While as a workaround to make larger image, yes you can make a blank transparent image for the first layer that represents the final size. Maybe there should be a knob to override the canvas size though, thanks for the idea :p

Stitching existing jxl frames together without reencoding would be a useful feature but maybe there isn't an easy way to reuse existing frames in the libjxl api...?

I don't think it's possible at all in the API. Best you could do is extract the codestream and insert it at the correct point

And putting the xy pos in frame headers

Thanks for the ideas, yeah frame-stitcher re-encodes

The other problem would of course be that for a webtoon you'd end up with a 940x200000 big image which most image viewers crap out on