Split composite status into status and quality, add Unknown as valid quality #4221

p0psicles · 2018-05-19T13:33:42Z

This PR contains major changes to our quality system.

It splits the composite qualities into status (Downloaded, Skipped, etc.) and quality (HDTV, 720p WEB-DL, etc.). This allows us to remove a lot of superfluous code and will make a better quality system possible in the future.
Existing qualities are being shifted by one spot. This means that we can use Unknown as a valid quality (the lowest available), thus eliminating a bunch of annoying hacks and allowing it to be set even as Preferred (e.g. for x265 content)

It goes without saying that these are massive changes that should be tested thoroughly over a minimum period of time. It would be great if anyone could give this a spin in the next couple of days. The upgrade process is automated and backup files for main.db, failed.db and cache.db will be created.

PR is based on the DEVELOP branch
Don't send big changes all at once. Split up big PRs into multiple smaller PRs that are easier to manage and review
Read the contribution guide

p0psicles · 2018-05-19T14:21:52Z

medusa/databases/main_db.py

+        self.connection.action("DROP TABLE IF EXISTS new_tv_episodes;")
+
+        log.info(u'Split composite status in to ep_status and ep_quality')
+        sql_results = self.connection.select("SELECT status from tv_episodes GROUP BY status HAVING status > -1")


@medariox Im not sure if we also need to translate status of -1

* Missed self.quality.

…tributes.

p0psicles · 2018-05-19T22:19:29Z

Pff this will require a lot of testing.

Action will include the status, quality (hasn't changed) but will include quality. Action will only have status, quality has been removed from it. * Changed show/history.py, to include quality. As we can't get it from action anymore.

Replace other occurrences of Quality.STATUS.

…d-db-status-quality-fields

This to keep the db sane, while changing all the composite statusses.

* Set status to app.EP_DEFAULT_DELETED_STATUS if episode file was delete and old status was ARCHIVED or DOWNLOADED or IGNORED or SKIPPED * Add some episode attributes that should be reset if a file is deleted

sharkykh

Looks good to me, but I also committed to this branch.

I would prefer it if the compact history "duplicate" qualities could be fixed, but I don't think it should stop you from merging.
Continued in a comment.

sharkykh · 2018-06-02T03:11:56Z

I tried to fix the history "bug" myself but hit a snag.
FWIW I found that because the action database column was a composite status, Action could be defined by it (the quality could be extracted from the action), and Index was defined by the quality column.

But because of the composite statuses split, now Action has quality (because it has to), and Index doesn't have it, so when the items are compacted it's compacting all of the history items of the same indexer, show, season and episode - ignoring the quality.

Edit: as I was writing this comment, I figured out what was the issue I encountered, the 2 newest commits should restore the compact layout to what it was (without affecting detailed)

+ fix quality sorting

p0psicles · 2018-06-03T09:54:21Z

Both History and episode status management where broken for me.

…elds' into feature/add-db-status-quality-fields

medariox

I think this is good to go 🎉

p0psicles · 2018-06-03T14:53:04Z

I dont know. I feel like we should really do more testing. The last two bugs i noticed where really obvious. And i couldnt test it sooner. As my db is a mess.

medariox · 2018-06-03T14:55:29Z

That's why we should merge it to develop ASAP, to have some more people actually test it.

p0psicles · 2018-06-03T15:03:52Z

Good point

p0psicles added 3 commits May 19, 2018 15:33

Added ep_status and ep_quality to tv_episodes table

c6c6453

Replaced ep_status and ep_quality with status and quality.

be96bac

Adapted Episode class to use separated status and quality.

13ac681

p0psicles commented May 19, 2018

View reviewed changes

p0psicles added 5 commits May 19, 2018 21:01

-1 is also a valid status.

f2b6a13

* Missed self.quality.

Refactored most of the composite statussus / quality into separate at…

d1fccb1

…tributes.

Missed quality field in query

ce65654

-1 is not a valid default quality.

7208c0d

Fixed default init of quality.

d9c45af

medariox changed the title ~~Added ep_status and ep_quality to tv_episodes table~~ Added status and quality to tv_episodes table May 20, 2018

p0psicles and others added 19 commits May 20, 2018 13:13

Fix call to overview status.

c6c351f

Adapted the history table.

d38f701

Action will include the status, quality (hasn't changed) but will include quality. Action will only have status, quality has been removed from it. * Changed show/history.py, to include quality. As we can't get it from action anymore.

Synced template changes.

dba2e97

Fix trying to iterate int

5d834c1

Update should_search. Requires 4 arguments now

36c0c08

Fix schedule page

ffb764b

Fix backlog overview

c7c3cf3

Fix queries with references to Quality.STATUS.

f823350

Replace other occurrences of Quality.STATUS.

Merge remote-tracking branch 'remotes/origin/develop' into feature/ad…

f132e66

…d-db-status-quality-fields

Fixed some typos.

8fac878

Reset quality UNKONWN to NONE.

3383c5b

Fixed get_overview() calls.

273434b

Removed some more composite_status method calls.

7e25812

Fix snatchSelection page. Missed quality in select.

bb8a8c5

Changed sql query DDL, to uppercase.

3283493

Convert all status on each startup to status/quality.

d6cb232

This to keep the db sane, while changing all the composite statusses.

Finish conversion of APIv1 funny code

64aac18

Remove last few split_composite_status

e9bc1a2

Completely remove Quality.{status} from mako

cf631cf

This comment has been minimized.

Sign in to view

sharkykh added 3 commits June 2, 2018 02:56

Fix inconsistency with "file name" column and episode number tooltip

665c5e8

Update Series.refresh_dir

fe1c9e8

* Set status to app.EP_DEFAULT_DELETED_STATUS if episode file was delete and old status was ARCHIVED or DOWNLOADED or IGNORED or SKIPPED * Add some episode attributes that should be reset if a file is deleted

More inconsistencies in displayShow

25346d9

sharkykh force-pushed the feature/add-db-status-quality-fields branch from 55e01c5 to 25346d9 Compare June 2, 2018 02:25

sharkykh previously approved these changes Jun 2, 2018

View reviewed changes

Restore compact history functionality

3ab82f7

+ fix quality sorting

sharkykh dismissed their stale review via 3ab82f7 June 2, 2018 04:00

Remove redundant quality from CompactItem

aa8b431

sharkykh previously approved these changes Jun 2, 2018

View reviewed changes

Should fix N/A quality in displayShow, thanks @medariox

3f089ae

sharkykh dismissed their stale review via 3f089ae June 3, 2018 00:22

medariox and others added 4 commits June 3, 2018 08:45

Remove superfluous check

ed0da46

Merge branch 'develop' into feature/add-db-status-quality-fields

0087179

Fix Overview referenced but not imported.

6e27c39

Fix allowed_qualities referenced before assigned.

ef10b06

p0psicles and others added 5 commits June 3, 2018 12:05

Cast whichStatus to int.

2e1e7da

Fix DB upgrade with older DBs, improve upgrade logging

b519468

Merge remote-tracking branch 'origin/feature/add-db-status-quality-fi…

5835a9c

…elds' into feature/add-db-status-quality-fields

Indent manage_episodeStatuses.mako

2f39b87

Gulp sync

ad44aef

medariox approved these changes Jun 3, 2018

View reviewed changes

p0psicles merged commit 55b472b into develop Jun 3, 2018

p0psicles deleted the feature/add-db-status-quality-fields branch June 3, 2018 15:03

sharkykh mentioned this pull request Sep 12, 2018

The database migrations bug #5198

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split composite status into status and quality, add Unknown as valid quality #4221

Split composite status into status and quality, add Unknown as valid quality #4221

p0psicles commented May 19, 2018 •

edited by sharkykh

Loading

p0psicles May 19, 2018

p0psicles commented May 19, 2018 •

edited

Loading

This comment has been minimized.

sharkykh left a comment

sharkykh commented Jun 2, 2018 •

edited

Loading

p0psicles commented Jun 3, 2018

medariox left a comment

p0psicles commented Jun 3, 2018

medariox commented Jun 3, 2018

p0psicles commented Jun 3, 2018

Split composite status into status and quality, add Unknown as valid quality #4221

Split composite status into status and quality, add Unknown as valid quality #4221

Conversation

p0psicles commented May 19, 2018 • edited by sharkykh Loading

p0psicles May 19, 2018

Choose a reason for hiding this comment

p0psicles commented May 19, 2018 • edited Loading

This comment has been minimized.

sharkykh left a comment

Choose a reason for hiding this comment

sharkykh commented Jun 2, 2018 • edited Loading

p0psicles commented Jun 3, 2018

medariox left a comment

Choose a reason for hiding this comment

p0psicles commented Jun 3, 2018

medariox commented Jun 3, 2018

p0psicles commented Jun 3, 2018

p0psicles commented May 19, 2018 •

edited by sharkykh

Loading

p0psicles commented May 19, 2018 •

edited

Loading

sharkykh commented Jun 2, 2018 •

edited

Loading