Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] master from SerenityOS:master #1

Open
wants to merge 10,000 commits into
base: master
Choose a base branch
from

Conversation

pull[bot]
Copy link

@pull pull bot commented Nov 22, 2021

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

@pull pull bot added the ⤵️ pull label Nov 22, 2021
spholz and others added 29 commits January 10, 2025 12:31
This pseudoinstruction causes text relocations, as it stores the target
address in literal pools inside .text.

The bss clearing code now also matches the RISC-V implementation.
We don't have any text relocations in the kernel anymore.
This linker argument was added in bd6dc5c as a temporary workaround.

Removing this argument won't cause text relocations to be reported for
aarch64 and riscv64, as we currently compile the kernel with `--no-pie`
on those architectures.
Some assembly files were previously inconsistently formatted.

- Use 4 spaces for indentation
- Use `.L` as the local label prefix
- Remove trailing whitespace
If we're drawing a transform with a negative scale, we have to go
through the slow path since draw_scaled_bitmap() is unable to flip
images.

Fixes a regression from the 2nd commit in #24465.
in portable version of SHA-1, SHA-256 and SHA-512.
ee89146 accidentally changed the path to the Pi 3 devicetree blob to
an incorrect one.
On some architectures like ARM, using mismatched memory types for
different aliases of the same physical memory can lead to unexpected
behavior.
(https://developer.arm.com/documentation/102376/0200/Memory-aliasing-and-mismatched-memory-types)

Flushing the data cache and using appropriate memory barriers might be
enough, but should have worse performance.
We would otherwise crash here on the Raspberry Pi 4.
Make the MemoryType used by mmap() configurable by
File::vmobject_and_memory_type_for_mmap overrides (previously called
vmobject_for_mmap).

All mmap()s were previously MemoryType::Normal.
status = "okay" means the device is operational. We should ignore
devices that have a different status property value.

The BCM2835TimerDriver force disable hack can be removed, since that
timer is disabled in the Pi 4 devicetree.

But we have to introduce another hack that force enables the Pi 3 system
timer, as we otherwise wouldn't have any timer that we support for the
Pi 3.
This will allow us to add support for xHCI controllers that are not
connected via PCI.
simple-framebuffer nodes describe pre-initialized framebuffers that we
can use without any special setup code.
nico and others added 30 commits February 24, 2025 09:22
Else it's not copied into the image. I forgot to do this when I
added the file.
We now create an FunctionBasedShading object and read all required
parameters and build objects for them, but we don't use them
for any rendering yet.
This is covered by tests, and we seem to do the right thing.
The PDF spec doesn't explicitly mention supporting comments here,
but other implementations (I checked Preview.app and PDFium) seem
to allow and skip them, so follow suit.
I have chosen `centered_on` to prevail for it is the older one.
The GOP framebuffer (if present) is passed to the kernel so it can use
it as the boot framebuffer.
On Meteor Lake this range covers the GFX address space reservations,
at a very high address. Nothing actually ends up using it, so we can
ignore it. Moreover if we don't we end up with highest_address at the
4 TiB mark which would need an enormous page table to map this
huge (0-4TiB) range. Until we support mapping I/O space piecewise we
need to skip this range.
When running with 32GiB of system RAM, 2MiB is not enough for the
page usage bitmap. New value should be enough for 128GiB of RAM
Thsi will now handle when the JPEG2000 decoder starts
returning CMYK data.
`image` can now operate on CMYK data in jpeg2000 files. This also means
you now have to explicitly convert them to RGB with a profile to store
them in most file formats we can write:

        Build/lagom/bin/image -o tmp.webp \
            Tests/LibGfx/test-inputs/jpeg2000/cmyk-small.jpf \
            --assign-color-profile \
                Build/lagom/Root/res/icc/Adobe/CMYK/USWebCoatedSWOP.icc \
            --convert-to-color-profile serenity-sRGB.icc

We _can_ write CMYK .jpg files, so `image` can now transcode CMYK data
in JPEG2000 files to CMYK jpg files, which is kind of cool.

We can now also show CMYK JPEG2000 files in PDFs.

The big challenge here was to create test files. I haven't found a good
way to create a reference CMYK image. So instead I figured I'd make a
small 3x2 test image that has a few well-known colors, but Photoshop
has trouble writing 3x2 CMYK JPEG2000 files:

https://community.adobe.com/t5/photoshop-ecosystem-bugs/photoshop-writes-cmyk-jpeg2000-file-in-a-way-that-macos-s-preview-app-does-not-decode-correctly/idc-p/15178650#M119489

(...and jasper, openjpeg, and grok can't handle CMYK data in JPEG2000
files at all.)

I then tried writing a 3x2 image that has the CMYK data I want in
the RGBA channels, converting that to a JPEG2000 using opj_compress,
and then changing the colr and cdef boxes. But that required putting
0 into the alpha channel for several pixels, and Photoshop then was
smart and set the color values to 0 for these pixels too.

So then I figured I'd write a small program to write this 3x2 RGBA
image using Serenity's libraries. This worked
(https://github.com/nico/serenity/tree/mkcmyk), and I modified the
boxes, and the file showed up fine in Preview.app -- but not in
Photoshop 😱

In the end, I noticed that if I make the image 4x2 instead of 3x2,
Photoshop's JPEG2000 saving code path works fine. So that's how
I created the test files in the end 😅
Created by opening the existing test image in Photoshop, converting
transparency to a layer mask and then that to a channel (and
deleting the layer mask again), Save as..., BMP, Advanced Modes,
A4 R4 G4 B4, and then giving that bmp to opj_compress like:

    ../openjpeg/build/bin/opj_compress -i bmp-a4.bmp -o file.jp2
Images with > 8bpp still aren't supported for now. They currently
overflow the bitplane decoding buffer. (The fix for that is probably
to only decode ~9 bits and then look at the clamped bpp after bitplane
decoding. We need to add a lossy 16bpp test image to make sure we
get this right in copy_and_dequantize_if_needed().)
Do this by making decode_code_block() write to a float buffer,
instead of an i16 buffer.

Alternatively, we could make decode_code_block() stop after some
number of bits since we only need 8 anyways and update a few places
in JPEG2000.cpp that currently read bit_depth(), but this here seems
perf-neutral (even slightly positive) and is simpler, so let's go
with this for now.

(16 bpp images are very rare, and I haven't seen images with more than
16 bpp, even though the spec allows up to 38 bpp.)
This commit partially reverts 7db8ccc, which was added to remove a
DynamicLoader hack to ignore relocations for gcc's transactional
memory functionality, which Serenity didn't support.

Neither hack seem to be required anymore and the existing stubs
will cause ODR violations on the next commit, where the
GNU toolchain starts being statically linked.
This is done as an effort to support building Serenity on Intel macs,
which runs into link errors when dynamically linking binutil's libctf.

This is a known problem and has been reported on gnu's issue tracker
and on the spack package manager's github issues:
https://lists.nongnu.org/archive/html/bug-binutils/2023-11/msg00050.html
spack/spack#35817
spack/spack#42947

spack also "solves" it by simply *not* dynamically linking.

We even already had an in-tree hack to workaround a different, intl
related, linking error for Intel macs, which this patch removes.
The previous code assumed that, under a Darwin system, you would either
have the GNU coreutils version of `du` installed through brew, which
adds a 'g' prefix, or the default, BSD-like version.

This assumption can be easily broken by simply directly installing GNU's
coreutils or through a package manager that doesn't prepend a 'g' to it.

We now instead always use whatever `du` is in the user's $PATH and try
to detect which flavor it is and, consequently, which flag should be
used to retrieve the "apparent size" of something in disk.

The heuristic is not perfect and may lead to the wrong flag being
selected in edge cases, but it can always be amended in the future.
Appending "id_" to the datetime part gives you a raw copy without the
navigational toolbar:
https://en.wikipedia.org/wiki/Help:Using_the_Wayback_Machine#Specific_archive_copy
Otherwise we get an HTML file.
This effectively keeps a failing `du --help` from printing user-visible
messages.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.