v0.8.4 - New Scanner + Browse Authors/Artists + Tons of Bugfixes

[majora2007]

After rewriting 50K lines of code last release, I decided it was best to take a break and focus on the bugs that resulted and a few that had piled up. This release is bug focused, but of course I stuck a few big features in with it.

Scanner

One issue that has been cropping up as we attract more users with larger libraries or libraries laid out in the Publisher/Series style is the scanner. Users with large libraries modify their library quite a bit and the old scanner had issues with change detection. v0.8.3 had quite a nasty bug which required forced library scans to ingest new data. There was also the notorious FK issue that was more common for larger libraries.

To fix these issues, I rewrote the scanner from the ground up to be more reliable. This was a (painful) 3 week effort working closely with a dedicated set of testers on a canary release to be nightly and stable ready. The new scanner will be slower than the old one. As of now, I feel that once the main slow first scan is done, it's actually performing much better. With this new scanner, there were some other major changes.

People Re-architecture

In order to reduce the FK issue in the scanner, I also needed to tackle the 80+ hour re-architecture of how people work within Kavita. When I started, with my limited knowledge of the scene or what Kavita would grow to become, I implemented people as tags, much like how ComicInfo does. This is constrained when you want to make people more entity-focused, like having a dedicated page for them.

With this release, you can now view individual details of a person, set a cover and description about the person and see their works and individual issues they worked on. You can also browse authors and artists, like other software has. This is just the start, please leave feedback for me via Feature Requests, so I can build this feature out more and make it a richer experience. Note: Any Smart Filters that relied on people will need manual intervention to correct. A force scan will be required to regenerate writer information.

Special Keyword Removal

The last thing with the scanner was removing Kavita's ability to detect and classify certain keywords as Special. Like Omake or Annual would classify a file as a special. While this is extremely handy in the Manga scene, there were numerous reports over the years that required manual workarounds by users. In order to behave more reliably, I opened a discussion this year for removal and made the final call. Specials going forward will need the SP marker on the filename (or tagged internally).

CoversDB

Kavita has had the ability to download and display favicon and publisher images with a fallback repository being hosted on the main Kavita website. In this release, I've moved all that functionality and those images over to a new CoversDB repo. This repo will be the home for anything related to pulling images for Kavita instances. I plan to enhance this further as new features come to Kavita.

Small QoL Features

Last Read Filter is a new feature that allows you to pass a number that will be subtracted from TODAY, upping the dynamic function and allowing users to build Smart Filters like "Haven't read in a month".

Colorscape Polish The colorscape system has been polished even further. I spent a good amount of time adding some randomness, fixed a bug with colorscape resetting between navigation, and tried to tweak where colors go to avoid harsh contrasts. It's not perfect, but it's a step in the right direction. If you're passionate about the colorscape system, I welcome collaboration.

Bulk Library Actions This is something @DieselTech has been pushing me to add and the time is finally here. For users that have a lot of libraries (like Diesel and his 200+), sometimes it's quite a pain to update the settings of them all or perform some actions. Now in Kavita there are some limited bulk options, like refreshing covers or copying settings from one library to another (Scan currently disabled).

Recap

This release has been a ton of work on my part, but I felt it was important to squash the bugs first and ensure Kavita remains stable while I continue to add more features and bring a richer experience to users. My next release is still aimed at finishing the UX Overhaul with attention on Kavita+. With the holidays coming up, it is expected that this will take longer than most releases.

For those that have seen, we have hit over 65K active installs. It's absolutely amazing that a small project I started due to Ubooquity not having metadata has turned into such a massive project with a huge userbase. I want to give my thanks again to those in the community that have raised bugs, feature requests, give support in the discord/reddit, those who give me donations via OC/Kavita+, the translators, and of course my wife that helps support me pouring so much time into the project. I'm excited to continue building Kavita and bring about more of our visions into the software and see what is possible.

People metadata will be removed on update. A force scan is required to repopulate in the new system. A force scan/first scan will be much slower than before.

Added

• Added: Manage Libraries now has bulk actions. You can trigger multiple scans (not file scans, but refresh covers or colorscapes, etc) or copy settings from one library to multiple. Shift+select works as well.
• Added: Added a new page for all People within Kavita. From this page, you can see their works across Kavita.
• Added: Ability to see individual chapters/issues (limited to 20) a person has.
• Added: Ability to add images to the person via UI/API. A more automated method is planned in the future.
• Added: Added the ability to browse all Writers and Artists (cover artist) in one page. This should help book users that like to browse by author. (Feature Request: [Kavita] Browse by Author ( & penciller, letterer, editor, etc.?) #2556 - 23 votes) (Thanks @MrRobotjs for some css on the Person detail page)
• Added: Added the ability to trigger the general cleanup (which normally runs at midnight) manually in case you need to clear unlinked people, covers, etc.
• Added: Reading List and Collection cards now show a badge showing the number of series/items underneath.
• Added: New Filter option called Last Read which allows a number to be passed in and will dynamically calculate a date from TODAY() matched against reading time. This allows for users to build even more dynamic smart filters to find things not read in a month or read within the past 7 days.
• Added: Added back a lot of the metadata that had to be removed from the top of the series detail page in the UX overhaul, like Language, Release Year, Format, and Reading Time
• Added: Added Details tab to volume details page
• Added: (Kavita+) New Sync Details button on smart collections that shows an easier to digest view of Series and missing series.
• Added: Bulk send to is now available for chapters/volumes
• Added: Added bulk deletion for chapters/volumes

Changed

• Changed: Optimized a number of methods within the Scanner to reduce memory and CPU time
• Changed: Moved a lot of the chapter scanner logging to Trace.
• Changed: Scanner can now choose to parallel parse files when there are over 100 in a directory
• Changed: Changed how detection and scanning of dirty directories works. The scanner will now parse bottom-up to reduce any potential misses and avoid different layouts working differently. This has extra I/O checks but much greater reliability and should reduce the amount of work needed to ingest changes after the first scan.
• Changed: LocalizedSeries merging with Series is now done at a higher level and performs much better (in terms of reliability)
• Changed: People are now entity based which allows for a richer experience in Kavita.
• Changed: Clicking on any person in Kavita will now open their dedicated page which showcases their different works.
• Changed: (Performance) Some large refactors to the scanner to help with the FK bug. The scanner should be slower but consume much less memory, especially for well tagged systems.
• Changed: Kavita no longer parses keywords from filenames and groups as a Special. The way to mark a special going forward will be explicitly with SP01, SP02, etc. Specials can be flat in the series folder or ideally in Specials/ folder.
• Changed: Changed logging level on some of the loggers that could become noisy
• Changed: Refactored some of the post-series work to a separate thread.
• Changed: Ensure mobile actionable modal is fullscreen and centered vertically
• Changed: Update to Angular 18 and other backend dependencies
• Changed: When a setting is invalid, clicking outside the setting item will not close and edit is disabled.
• Changed: Changed the smart filters page to open in the same window (but still available to open in new tab)
• Changed: (Performance) Made the pipe that turns the language codes into full names use a dedicated API which is faster
• Changed: Improved the wording for Single Volume chapters on Series detail page (and person detail page)
• Changed: Kavita now uses a different repo for publisher images and favicons (https://github.com/Kareadita/CoversDB). Any changes should be made here.
• Changed: You can now edit the person name in the UI. If not corrected in the metadata, a duplicate person will be created. This will be tweaked in the future once Alias support is added.
• Changed: Significantly improved the feel of the colorscape experience. When transitioning between colorscapes, locations of accent points will randomize. Lighter colors will try to avoid the bottom right where there could be bad contrast.
• Changed: Changed Colorscape to use non-linear animation for transitioning between 2 colorscapes.
• Changed: If the user somehow wrote bad cron notation for custom jobs, on startup, Kavita will detect and default to daily
• Changed: Changed the overall look and feel of the preference side nav and main side nav (Thanks @MrRobotjs)
• Changed: Moved Test email settings to the top to align with the UX patterns
• Changed: Moved Reset to Defaults to the top to align with UX patterns and removed a few instances since it resets everything.
• Changed: Changed a lot of modals to be more consistent in how they open and their size to be slightly larger.
• Changed: When failing to get a page count on an archive, report it in media issues
• Changed: Improved memory consumption on some filtering conditions
• Changed: Tweaked some logging messages to be more clear when using logs to debug

Fixed

• Fixed: Fixed a bug where the UnauthorizedAccessException is not correctly ignored when using the docker container as non-root. (Thanks @Fesaa)
• Fixed: Fixed a bug where any media that was less than an hour in length wouldn't be included in stats. This requires Analyze Files to be ran on all libraries to retroactively fix
• Fixed: Fixed a bug where chapter order wasn't properly sorting on Volume Detail page
• Fixed: Fixed an issue where bookmarking wouldn't check if the source file was already in the target encoding and thus had issue when trying to convert. Also added another check to not do anything if target encoding is PNG (aka Kavita doesn't convert webp -> png).
• Fixed: Fixed a bug where Series Cover image could choose a Volume 0 instead of Volume 1
• Fixed: Fixed a bug where teams and locations wouldn't trigger updating people on update series metadata api
• Fixed: Fixed a bug where calculating the Lowest Folder for a Series could be incorrect when there are nested folders. This includes a migration to clear out existing entries to avoid scan series not seeing files.
  Example:
  / <- Lib root assume
  /love hina/love hina/v01.cbz
  /love hina/specials/sp01.cbz
  The lowest series folder was /love hina/love hina/ for some reason, meaning series scan wasn't getting the full series (all folders) scanned in.
• Fixed: Fixed up a check for using lowest folder path to lookup from a Folder Watcher task. This should find series much better from a lower file path.
• Fixed: Fixed a bug where teams and locations wouldn't trigger updating people on update series metadata api
• Fixed: Fixed a bug where users could change their age rating when they didn't have the permission
• Fixed: Fixed a bug where download wasn't functioning from the on deck cards
• Fixed: Fixed a typo on the localization string for Create Multiple Reading List/Collection pages
• Fixed: Fixed Promote/Unpromote not working from reading list detail page and collection detail page
• Fixed: Ensure we show details page when weblinks are present, including chapter detail page
• Fixed: Fixed related tab not having enough bottom padding
• Fixed: Fixed a few different combinations where IsEmpty was not working
• Fixed: Fixed broken Send To on mobile and Send To not actually functioning for chapter/volume cards.
• Fixed: Tweaked read-time-left pipe to have better control on how we present the now decimal number.
• Fixed: Fixed a bug where continue from was choosing ch instead of vol
• Fixed: Fixed read-more text not displaying newlines correctly in some situations (Thanks @Fesaa)
• Fixed: Fixed the search bar closing when toggling show chapters and files toggle (Thanks @Fesaa)
• Fixed: Fixed a bug where validation wasn't showing on Hostname
• Fixed: Next expected card had layout issues and missing localization
• Fixed: Don't close a modal when a typeahead is open and pressing Escape key
• Fixed: Fixed tooltips not rendering on some cards, like reading list/collection cards
• Fixed: Fixed a missing case to show storyline tab on Light Novel libraries when users use chapters (not common)
• Fixed: Ensure the edit series modal is showing the localized publication status
• Fixed: Updated a lot of missing localization keys
• Fixed: Fixed the fade between colorscape so there isn't a reset to default anymore
• Fixed: Fixed an issue with how specials render on reading list detail page
• Fixed: Fixed the duplicate Dark theme issue once and for all
• Fixed: Fixed custom tasks not being possible
• Fixed: Fixed Irish not working when selected as locale
• Fixed: Fixed edit chapter modal failing to save when setting Age Rating
• Fixed: Fixed a bug where when a card doesn't have a format, 22px was still beating taken up.
• Fixed: Ensure More In Genre doesn't choose from a Genre that's not exposed to the dashboard
• Fixed: Fixed a bug where bulk mode on customize side nav wasn't laying out the bulk update well.
• Fixed: Fixed delete chapter not being hooked up from Volume details page
• Fixed: Fixed user rating filter not working at all. It was accepting a percentage, but should have been a number 0-5 for the star rating.
• Fixed: Fixed the bulk actions bar laying out in the wrong position
• Fixed: In the rare occasion someone turns off all the file types for a library, the scanner will now respect that.
• Fixed: Fixed Delete and Analyze Files on the library detail actionables not being hooked up
• Fixed: Fixed an issue with doujin filenames causing series to all parse as (
• Fixed: (Kavita+) Fixed a bug where old Book libraries could be stuck in Scrobbling On state
• Fixed: Fixed age rating icon not updating upon series metadata changes
• Fixed: Ensure Jumpkeys are grouped non-case sensitive
• Fixed: Fixed a bug where want to read page was using a custom query passed through the filtering pipeline, thus breaking some combos.
• Fixed: Fixed a bug where manage email settings was spamming the backend due to an infinite loop
• Fixed: Fixed select all on edit user modal affecting the role/library control as well.
• Fixed: On Age Rating filter, the comparisons could be duplicated
• Fixed: Fixed a bug with bulk selection not pruning submenu actions when there are no children left. (aka empty send to)

API

• Search now returns a list of PersonDto instead of strings for People results.
• Removed metadata/chapter-summary api since it hasn't been used in years

Theme

• Side nav selected tab now uses --side-nav-item-active-text-color on the text instead of just being white
• Added --side-nav-width variable to control side nav bar width
• Added --navbar-header-margin, set it to 8px and --nav-offset to 56px to get the Plex floating navbar.
• Added --navbar-border-radius to fix the navbar but let users customize for that Plex look.
• Added --navbar-header-mobile-x-margin and --navbar-header-mobile-y-margin to allow for the Plex navbar on mobile

Known Issues

• (Kavita+) MAL Stack imports/Smart Collection total series count is not correct. This will be fixed with v0.8.5.
• Uploading some dimensions for person cover can cause the image to be squished. This will be fixed with v0.8.5.
• When using multiple reading progress filters, some series with 100% reading progress will show (clicking On Deck). After 14 hours, I can't solve it. I will loop around in v0.8.5.

---

This discussion was created from the release v0.8.4 - New Scanner + Browse Authors/Artists + Tons of Bugfixes.

[Aloe-recite]

Hi there,
I just updated to 0.8.4 and did a rescan of my only library but am facing a few big issues!
The authors page is not populated at all and all my authors disappeared from the "edit" → "people" mask, despite the line having the green lock. If I add them in again, they are not saved if I save, close and reopen the edit screen, not to mention that a re-scan doesn't pick them up. However, I noticed that if I add them to a comic that didn't have any people before, it works correctly.
Also, I'm missing some strings (in Italian - especially the left side bar on the edit screen) and the "bust cache" option didn't do anything, had to switch to English for everything to work (which is not a big deal, I prefer English anyway but still felt like reporting). I tried deleting browser cache and all to no avail. Is there anything I am missing? Thanks and keep up the good work!

[scare376]

For the first issue did you run a force scan by going to the library settings and hitting the force scan button?
For the second please try all the steps here #3177 (reply in thread)

[Aloe-recite]

yep, force scan done twice followed by multiple normal library scans, still the same. No authors in "browse authors" and authors disappeared from "edit screen" (also not visible in the metadata below title and plot in the comic page).

[majora2007]

Do you have any metadata in your files or did you manually tag them prior?

[Aloe-recite]

@majora2007 previously manually tagged in Kavita under the previous version, up to 50% or so of my library. They're a mixture of CBZ (downloaded with HakuNeko if that matters) and PDF, not sure about files metadata though?