Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added initial IMDB indexer. #3603

Merged
merged 90 commits into from
Mar 1, 2022
Merged

Added initial IMDB indexer. #3603

merged 90 commits into from
Mar 1, 2022

Conversation

p0psicles
Copy link
Contributor

@p0psicles p0psicles commented Jan 12, 2018 •
edited
Loading

Working: Searching + basic adding of show.

  • PR is based on the DEVELOP branch
  • Don't send big changes all at once. Split up big PRs into multiple smaller PRs that are easier to manage and review
  • Read the contribution guide

image

image

TODO:

Adding these if possible:

  • searching shows
  • basic adding imdb show (including seasons/episodes)
  • shows poster
  • show firstaired
  • Fix continuing / ended (double check this!)
  • show airs_time
  • show airs_dayofweek (maybe need to calculate)
  • show network
  • show status
  • episode airdate
  • episode description (nice to have)
  • image rating (don't think imdb has this)
  • image banner and artwork.
  • episode thumbnail creation
  • show updating
  • add the indexer icon
  • add actors/cast
  • imdb api exception handling. Just using RequestException for now.
  • add shows without any episodes (yet) For exampe D-day to berlin but this is a 3 part documentary?

@duramato
Copy link
Contributor

duramato commented Jan 14, 2018 •
edited
Loading

On every search

2018-01-14 17:16:26 WARNING  Thread-34 :: [dc25c36] Exception trying to parse attribute: {u'year': u'2017', u'type': u'TV series', u'imdb_id': u'tt4396630', u'title': u'The Gifted'}, with exception: AttributeError("'NoneType' object has no attribute 'split'",)

When trying to add http://www.imdb.com/title/tt0407400/, I can't add it:

2018-01-14 17:21:54 INFO     SHOWQUEUE-ADD :: [dc25c36] Starting to add show by ShowDir: D:\Inspector Max
2018-01-14 17:21:54 INFO     SHOWQUEUE-ADD :: [dc25c36] IMDB: {'session': <medusa.session.core.MedusaSession object at 0x07D74ED0>, 'cache': u'C:\\Users\\gonca\\Desktop\\Medusa\\cache\\indexers\\IMDB', 'language': u'en', 'use_zip': True}
2018-01-14 17:21:54 DEBUG    SHOWQUEUE-ADD :: [dc25c36] Getting all series data for 407400
2018-01-14 17:21:54 DEBUG    SHOWQUEUE-ADD :: [dc25c36] Getting all show data for tt0407400
2018-01-14 17:21:55 DEBUG    SHOWQUEUE-ADD :: [dc25c36] HEAD URL: http://www.imdb.com/title/tt0407400/ [Status: 200]
2018-01-14 17:21:55 DEBUG    SHOWQUEUE-ADD :: [dc25c36] User-Agent: Medusa/0.1.23 (Windows; 10; 24279a70-f94e-11e7-b43d-4ccc6aff7cef)
2018-01-14 17:21:57 DEBUG    SHOWQUEUE-ADD :: [dc25c36] GET URL: https://api.imdbws.com/title/tt0407400/auxiliary [Status: 200]
2018-01-14 17:21:57 DEBUG    SHOWQUEUE-ADD :: [dc25c36] User-Agent: IMDb/8.3.1 (iPhone9,4; iOS 11.2.1)
2018-01-14 17:21:57 WARNING  SHOWQUEUE-ADD :: [dc25c36] Exception trying to parse attribute: {u'plot': {u'totalSummaries': 0, u'outline': {u'text': u'A police duo fight crime with the help of a dog named Max.', u'id': u'/title/tt0407400/plot/po2636102', u'author': u'Jo\xe3o Ferreira'}}, u'ratings': {u'rating': 4.9, u'title': u'Inspector Max', u'year': 2004, u'ratingCount': 340, u'canRate': True, u'titleType': u'tvSeries', u'id': u'/title/tt0407400/'}, u'similarities': [{u'title': u'Inspector Max', u'nextEpisode': u'/title/tt6360828/', u'image': {u'url': u'https://images-na.ssl-images-amazon.com/images/M/MV5BNDA5ZjU4ZTAtZjFhYi00ODk5LThjZTItNzgxN2VhZjllNDQyL2ltYWdlL2ltYWdlXkEyXkFqcGdeQXVyMzQ4ODczODQ@._V1_.jpg', u'width': 1024, u'id': u'/title/tt6162526/images/rm3154000640', u'height': 767}, u'year': 2017, u'numberOfEpisodes': 26, u'seriesStartYear': 2017, u'titleType': u'tvSeries', u'id': u'/title/tt6162526/'}, {u'runningTimeInMinutes': 50, u'nextEpisode': u'/title/tt1025340/', u'image': {u'url': u'https://images-na.ssl-images-amazon.com/images/M/MV5BNzE2MzFjNTAtMDQxOS00MGFlLTg1OWQtZjNlMjc3YzQ5Njk2XkEyXkFqcGdeQXVyMjUyMTE3MTc@._V1_.jpg', u'width': 400, u'id': u'/title/tt1018269/images/rm2896159744', u'height': 546}, u'title': u'Conta-me Como Foi', u'year': 2007, u'numberOfEpisodes': 105, u'seriesStartYear': 2007, u'titleType': u'tvSeries', u'id': u'/title/tt1018269/'}, {u'runningTimeInMinutes': 30, u'nextEpisode': u'/title/tt2222542/', u'image': {u'url': u'https://images-na.ssl-images-amazon.com/images/M/MV5BNzUxYTkwOTYtMTBjMi00MGJiLWFmYTEtMDljMmVmZjMzNGUxXkEyXkFqcGdeQXVyNTAyODkwOQ@@._V1_.jpg', u'width': 500, u'id': u'/title/tt0294023/images/rm3229824512', u'height': 665}, u'title': u'Captain Tsubasa', u'year': 1983, u'numberOfEpisodes': 128, u'seriesStartYear': 1983, u'seriesEndYear': 1986, u'titleType': u'tvSeries', u'id': u'/title/tt0294023/'}, {u'runningTimeInMinutes': 30, u'nextEpisode': u'/title/tt0515455/', u'image': {u'url': u'https://images-na.ssl-images-amazon.com/images/M/MV5BMTQ4MjA2MzgxN15BMl5BanBnXkFtZTgwODI3ODM0MjE@._V1_.jpg', u'width': 500, u'id': u'/title/tt0169414/images/rm1119928576', u'height': 409}, u'title': u'Arthur', u'year': 1996, u'numberOfEpisodes': 246, u'seriesStartYear': 1996, u'titleType': u'tvSeries', u'id': u'/title/tt0169414/'}, {u'runningTimeInMinutes': 94, u'title': u'Wizards of Waverly Place: The Movie', u'image': {u'url': u'https://images-na.ssl-images-amazon.com/images/M/MV5BNzhmNzJlYzEtN2NkNS00MTM2LTlhNWEtMmQxNTM2NDRjNGRlXkEyXkFqcGdeQXVyMTkzODUwNzk@._V1_.jpg', u'width': 1073, u'id': u'/title/tt1369845/images/rm3062586624', u'height': 1650}, u'year': 2009, u'titleType': u'tvMovie', u'id': u'/title/tt1369845/'}, {u'runningTimeInMinutes': 22, u'nextEpisode': u'/title/tt0985517/', u'image': {u'url': u'https://images-na.ssl-images-amazon.com/images/M/MV5BMTY1NTEyNzEwOF5BMl5BanBnXkFtZTcwNjkwMTI3MQ@@._V1_.jpg', u'width': 353, u'id': u'/title/tt0799922/images/rm4007433984', u'height': 500}, u'title': u'Wizards of Waverly Place', u'year': 2007, u'numberOfEpisodes': 106, u'seriesStartYear': 2007, u'seriesEndYear': 2012, u'titleType': u'tvSeries', u'id': u'/title/tt0799922/'}, {u'runningTimeInMinutes': 30, u'nextEpisode': u'/title/tt1283448/', u'image': {u'url': u'https://images-na.ssl-images-amazon.com/images/M/MV5BYjI2MGUxOWMtNDM2NS00ZTI0LWJmZmUtMDM0ZDk3YzkzNjJhL2ltYWdlL2ltYWdlXkEyXkFqcGdeQXVyMTEwMTkwOTI@._V1_.jpg', u'width': 440, u'id': u'/title/tt1181917/images/rm357762304', u'height': 640}, u'title': u'The Suite Life on Deck', u'year': 2008, u'numberOfEpisodes': 71, u'seriesStartYear': 2008, u'seriesEndYear': 2011, u'titleType': u'tvSeries', u'id': u'/title/tt1181917/'}, {u'runningTimeInMinutes': 30, u'nextEpisode': u'/title/tt0632548/', u'image': {u'url': u'https://images-na.ssl-images-amazon.com/images/M/MV5BYjcxM2ZlN2MtMzBlYS00MzVjLWJmNzMtOGVjZDczYTVjNWUzXkEyXkFqcGdeQXVyNjk1Njg5NTA@._V1_.jpg', u'width': 701, u'id': u'/title/tt0364774/images/rm3547278592', u'height': 1000}, u'title': u'Lilo & Stitch: The Series', u'year': 2003, u'numberOfEpisodes': 67, u'seriesStartYear': 2003, u'seriesEndYear': 2006, u'titleType': u'tvSeries', u'id': u'/title/tt0364774/'}, {u'runningTimeInMinutes': 30, u'nextEpisode': u'/title/tt0712956/', u'image': {u'url': u'https://images-na.ssl-images-amazon.com/images/M/MV5BMTk5MTgxMzAzN15BMl5BanBnXkFtZTcwMjUyMTk0MQ@@._V1_.jpg', u'width': 355, u'id': u'/title/tt0426371/images/rm612733696', u'height': 500}, u'title': u'The Suite Life of Zack and Cody', u'year': 2005, u'numberOfEpisodes': 87, u'seriesStartYear': 2005, u'seriesEndYear': 2008, u'titleType': u'tvSeries', u'id': u'/title/tt0426371/'}, {u'title': u'Rebelde Way', u'nextEpisode': u'/title/tt1350327/', u'image': {u'url': u'https://images-na.ssl-images-amazon.com/images/M/MV5BMGMzNjEyOGUtZDFiZS00ZWQ5LTgxMzQtMDJiMTFlMjQ0YzNkXkEyXkFqcGdeQXVyMTk4NDg3MzU@._V1_.jpg', u'width': 720, u'id': u'/title/tt1277987/images/rm502863872', u'height': 576}, u'year': 2008, u'numberOfEpisodes': 221, u'seriesStartYear': 2008, u'titleType': u'tvSeries', u'id': u'/title/tt1277987/'}], u'metacriticScore': {u'reviewCount': 0, u'userRatingCount': 0, u'@type': u'imdb.api.title.metacritic.score'}, u'filmingLocations': [{u'id': u'/title/tt0407400/filminglocations/lc0807047', u'location': u'Estudios Contra Campo, Portugal'}, {u'attributes': [u'studio'], u'id': u'/title/tt0407400/filminglocations/lc0465922', u'location': u'Fealmar Studios, Lisbon, Portugal'}, {u'id': u'/title/tt0407400/filminglocations/lc0465923', u'location': u'Lisbon, Portugal'}, {u'id': u'/title/tt0407400/filminglocations/lc0465924', u'location': u'Set\xfabal, Portugal'}, {u'id': u'/title/tt0407400/filminglocations/lc0465925', u'location': u'Vila Nogueira de Azeit\xe3o, Portugal'}], u'base': {u'runningTimeInMinutes': 50, u'nextEpisode': u'/title/tt1264462/', u'image': {u'url': u'https://images-na.ssl-images-amazon.com/images/M/MV5BNDVkYTJjNDYtYmRiMi00MDRiLTgzOTMtNjlmNmQzNWFkZjJiXkEyXkFqcGdeQXVyMjUyMTE3MTc@._V1_.jpg', u'width': 480, u'id': u'/title/tt0407400/images/rm4154450944', u'height': 626}, u'title': u'Inspector Max', u'year': 2004, u'numberOfEpisodes': 106, u'seriesStartYear': 2004, u'seriesEndYear': 2005, u'titleType': u'tvSeries', u'id': u'/title/tt0407400/'}, u'@type': u'imdb.api.title.auxiliary.v1'}, with exception: TypeError("'NoneType' object has no attribute '__getitem__'",)
2018-01-14 17:21:58 DEBUG    SHOWQUEUE-ADD :: [dc25c36] HEAD URL: http://www.imdb.com/title/tt0407400/ [Status: 200]
2018-01-14 17:21:58 DEBUG    SHOWQUEUE-ADD :: [dc25c36] User-Agent: Medusa/0.1.23 (Windows; 10; 24279a70-f94e-11e7-b43d-4ccc6aff7cef)
2018-01-14 17:21:59 DEBUG    SHOWQUEUE-ADD :: [dc25c36] GET URL: https://api.imdbws.com/title/tt0407400/releases [Status: 200]
2018-01-14 17:21:59 DEBUG    SHOWQUEUE-ADD :: [dc25c36] User-Agent: IMDb/8.3.1 (iPhone9,4; iOS 11.2.1)
2018-01-14 17:22:00 DEBUG    SHOWQUEUE-ADD :: [dc25c36] HEAD URL: http://www.imdb.com/title/tt0407400/ [Status: 200]
2018-01-14 17:22:00 DEBUG    SHOWQUEUE-ADD :: [dc25c36] User-Agent: Medusa/0.1.23 (Windows; 10; 24279a70-f94e-11e7-b43d-4ccc6aff7cef)
2018-01-14 17:22:01 DEBUG    SHOWQUEUE-ADD :: [dc25c36] GET URL: https://api.imdbws.com/title/tt0407400/companies [Status: 200]
2018-01-14 17:22:01 DEBUG    SHOWQUEUE-ADD :: [dc25c36] User-Agent: IMDb/8.3.1 (iPhone9,4; iOS 11.2.1)
2018-01-14 17:22:01 DEBUG    SHOWQUEUE-ADD :: [dc25c36] Getting all episodes of 407400
2018-01-14 17:22:02 DEBUG    SHOWQUEUE-ADD :: [dc25c36] GET URL: https://api.imdbws.com/title/tt0407400/episodes [Status: 200]
2018-01-14 17:22:02 DEBUG    SHOWQUEUE-ADD :: [dc25c36] User-Agent: IMDb/8.3.1 (iPhone9,4; iOS 11.2.1)
2018-01-14 17:22:02 ERROR    SHOWQUEUE-ADD :: [dc25c36] 407400 Error while loading information from indexer IMDB. Error: "global name 'seasnum' is not defined"
Traceback (most recent call last):
  File "C:\Users\gonca\Desktop\Medusa\medusa\show_queue.py", line 376, in run
    s = indexer_api[self.indexer_id]
  File "C:\Users\gonca\Desktop\Medusa\medusa\indexers\indexer_base.py", line 318, in __getitem__
    self._get_show_data(key, self.config['language'])
  File "C:\Users\gonca\Desktop\Medusa\medusa\indexers\imdb\imdb_api.py", line 513, in _get_show_data
    self._get_episodes(imdb_id, specials=False, aired_season=None)
  File "C:\Users\gonca\Desktop\Medusa\medusa\indexers\imdb\imdb_api.py", line 257, in _get_episodes
    log.warning('An episode has incomplete season/episode number (season: {0!r}, episode: {1!r})', seasnum, epno)
NameError: global name 'seasnum' is not defined

@p0psicles
Merge remote-tracking branch 'remotes/origin/develop' into feature/ad…
8b4e251 
…d-imdb-indexer

# Conflicts:
#	medusa/image_cache.py
@p0psicles
Fix network and images.
2223f45
@p0psicles
Added episode rating, overview and airdate.
80a607b
@p0psicles
Fixed bug where adding show without a summary, throwed an error.
cad5dea
@p0psicles
Added actors.
d3a4e7d 
* Don't display imdb icon, when it's an imdb show, as there is already an icon shown.
* added cross-env to package.json for building.
@p0psicles
Added fanart and production_art (as fanart).
06a99f4
@p0psicles
Use the poster initially provided with the show, as the images api, d…
730620a 
…oesn't have any ratings attached to the images.

Use the production art as fanart.
* Rename the exceptions to be used as imdb exceptions.
@p0psicles
Copy link
Contributor Author

@duramato what are we going to do with all the new networks? Just add them?

@duramato
Copy link
Contributor

You mean regarding timezones? They will get added overtime

@p0psicles
Copy link
Contributor Author

yes, they all need to be added

@p0psicles
Refactor indexers.
eace45e 
* Use get_nested_value function for tvmaze.
@p0psicles
Copy link
Contributor Author

@duramato you wanna start doing some testing? Already working pretty stable.

@duramato
Copy link
Contributor

Sure I'll start testing

@p0psicles
Use get_nested_value() function for tmdb.
4d9986b 
* Fixed get_nested_value for when it's getting 0 or empty strings.
@p0psicles
tvmaze exceptions got removed.
817e695
@p0psicles
Get airs day of week, from last 10 airdates.
f3e87f3
@p0psicles
Added episode thumbnails with kodi_12plus.
20b0d48
@p0psicles
Merge remote-tracking branch 'remotes/origin/develop' into feature/ad…
203ffbd 
…d-imdb-indexer

# Conflicts:
#	medusa/show/recommendations/anidb.py
#	medusa/show/recommendations/imdb.py
#	themes-default/slim/yarn.lock
@p0psicles
Fix issues with shows missing info.
f1bfbc1 
* correct merge conflict.
@p0psicles
Fix exception for when search doesn't return any results.
f956eec 
The raise, was removed somewhere.
@p0psicles
Make searching more flexible
ff23600
@p0psicles
added externals component
bf3ad02
@p0psicles
Get the thumb version for poster.
8ec6e04
@p0psicles
Fix imdb status
fb6874f
@p0psicles
Extended tmdb exception handling.
1303532 
For AttributeError: 'NoneType' object has no attribute 'raise_for_status'
@p0psicles
Fix error with lastaired
fd5f013
@p0psicles
Adding log for when tt id's are cleaned from indexer_mapping.
76b8246
@p0psicles
Fix get_episode error
b8c133b
@p0psicles
Clean recommended_db
baaf043
@p0psicles
Improve searching
e14be33
@p0psicles
add logs
d783c2a
@p0psicles
simplify clean imdb tt
63b7444
@p0psicles
Fix parsing images tmdb
77e7632
@p0psicles
Fix exception handling for show_updater when updating through indexer…
41a688b 
…s without a per season update list like tvmaze.
@p0psicles
Get posters and banners by aspect_ratio
a6a8226
@p0psicles
New imdb sorting logic
b8e6fff
@p0psicles
Improve the show season updates.
19e3363 
* Limit the amount of info pulled for checking if we need a season update.
* Only pull the season that we need.
@p0psicles
Also applied to the other indexers
6db36ff
@p0psicles
Refactored the get_last_updated_seasons method.
aaefba0
@p0psicles
comment
f68957b
@p0psicles p0psicles mentioned this pull request Feb 26, 2022
Merged
3 tasks
@p0psicles p0psicles merged commit f51a2eb into develop Mar 1, 2022
@p0psicles p0psicles deleted the feature/add-imdb-indexer branch March 1, 2022 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Feature in progress
Projects
None yet
Milestone
backlog
Development

Successfully merging this pull request may close these issues.
3 participants
@p0psicles @duramato @OmgImAlexis