Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3.3 upgrade: config loading error when loading admin page #4371

Closed
decentral1se opened this issue Sep 3, 2021 · 9 comments
Closed

v3.3 upgrade: config loading error when loading admin page #4371

decentral1se opened this issue Sep 3, 2021 · 9 comments
Labels
Status: Waiting for answer Waiting issue author answer

Comments

@decentral1se
Copy link
Contributor

decentral1se commented Sep 3, 2021

Describe the current behavior

When loading the admin page, there is an error shown. It looks to be a configuration breakage but I would expect a v3.2 -> v3.3 to not break configs.

Steps to reproduce:

  1. Load https://tv.lumbung.space/admin/config/edit-custom#instance-homepage
  2. See You provided 'undefined' where a stream was expected. You can provide an Observable, Promise, Array, or Iterable.
  3. Open console and see Uncaught SyntaxError: Unexpected identifier, Cannot load config locally. Fallback to API., Failed to load resource: the server responded with a status of 500 (), main.c9eac874d7bd3b1cbab9.js:3 Backend returned code 500, errorMessage is: relation "actorCustomPage" does not exist and then main.c9eac874d7bd3b1cbab9.js:3 Backend returned code 500, errorMessage is: relation "actorCustomPage" does not exist
  4. Hitting https://tv.lumbung.space/api/v1/custom-pages/homepage/instance gives {"type":"https://docs.joinpeertube.org/api-rest-reference.html#section/Errors/SequelizeDatabaseError","detail":"relation \"actorCustomPage\" does not exist","status":500,"code":"SequelizeDatabaseError","error":"relation \"actorCustomPage\" does not exist"}

Digging a bit further, I found that it is trying to load the following before it goes to the API:

<script type="application/javascript">window.PeerTubeServerConfig = '{"instance":{"name":"tv.lumbung.space","shortDescription":"A self-organized space for the lumbung inter-lokal","isNSFW":false,"defaultNSFWPolicy":"do_not_list","defaultClientRoute":"/videos/local","customizations":{"javascript":"","css":""}},"search":{"remoteUri":{"users":true,"anonymous":true},"searchIndex":{"enabled":false,"url":"","disableLocalSearch":false,"isDefaultSearch":false}},"plugin":{"registered":[{"name":"auth-openid-connect","version":"0.0.5","description":"Add OpenID connect support to login form in PeerTube.","clientScripts":{}},{"name":"video-annotation","version":"0.0.6","description":"PeerTube plugin video annotation","clientScripts":{"dist/embed-client-plugin.js":{"script":"dist/embed-client-plugin.js","scopes":["embed"]},"dist/video-edit-client-plugin.js":{"script":"dist/video-edit-client-plugin.js","scopes":["video-edit"]},"dist/video-watch-client-plugin.js":{"script":"dist/video-watch-client-plugin.js","scopes":["video-watch"]}}}],"registeredExternalAuths":[{"npmName":"peertube-plugin-auth-openid-connect","name":"auth-openid-connect","version":"0.0.5","authName":"openid-connect","authDisplayName":"login.lumbung.space"}],"registeredIdAndPassAuths":[]},"theme":{"registered":[],"default":"default"},"email":{"enabled":true},"contactForm":{"enabled":false},"serverVersion":"3.3.0","serverCommit":"","transcoding":{"hls":{"enabled":true},"webtorrent":{"enabled":false},"enabledResolutions":[240,480,720],"profile":"default","availableProfiles":["default"]},"live":{"enabled":true,"allowReplay":true,"maxDuration":18000000,"maxInstanceLives":20,"maxUserLives":3,"transcoding":{"enabled":true,"enabledResolutions":[480],"profile":"default","availableProfiles":["default"]},"rtmp":{"port":1935}},"import":{"videos":{"http":{"enabled":true},"torrent":{"enabled":true}}},"autoBlacklist":{"videos":{"ofUsers":{"enabled":false}}},"avatar":{"file":{"size":{"max":4194304},"extensions":[".png",".jpeg",".jpg",".gif",".webp"]}},"banner":{"file":{"size":{"max":4194304},"extensions":[".png",".jpeg",".jpg",".gif",".webp"]}},"video":{"image":{"extensions":[".png",".jpg",".jpeg",".webp"],"size":{"max":4194304}},"file":{"extensions":[".webm",".ogv",".ogg",".mp4",".mkv",".mov",".qt",".mqv",".m4v",".flv",".f4v",".wmv",".avi",".3gp",".3gpp",".3g2",".3gpp2",".nut",".mts",".m2ts",".mpv",".m2v",".m1v",".mpg",".mpe",".mpeg",".vob",".mxf",".mp3",".wma",".wav",".flac",".aac",".m4a",".ac3"]}},"videoCaption":{"file":{"size":{"max":4194304},"extensions":[".vtt",".srt"]}},"user":{"videoQuota":-1,"videoQuotaDaily":-1},"trending":{"videos":{"intervalDays":7,"algorithms":{"enabled":["best","hot","most-viewed","most-liked"],"default":"best"}}},"tracker":{"enabled":true},"followings":{"instance":{"autoFollowIndex":{"indexUrl":""}}},"broadcastMessage":{"enabled":true,"message":"Welcome to **tv.lumbung.space**, bear with us as we're still setting everything up!","level":"info","dismissable":true},"homepage":{"enabled":false}}'</script>

I can't quite spot the error?

Here is our production deployment configuration: https://git.coopcloud.tech/coop-cloud/peertube/src/branch/main/compose.yml where we are using the env var substitution functionality. It seem the only change to that has been #4290? I made sure our custom-environment-variables.yml is up to date.

We upgraded from the v3.2.1-buster image

Describe the expected behavior

  1. Load https://$my_instance/admin/config/edit-custom#instance-homepage
  2. The page loads successfully

Additional information

  • PeerTube instance:

    • URL: https://tv.lumbung.space
    • version: same as v3.3 docker
    • NodeJS version: same as v3.3 docker
    • Ffmpeg version: same as v3.3 docker
  • Browser name, version and platforms on which you could reproduce the bug: chrome latest stable

  • Link to browser console log if relevant: (provided above)

  • Link to server log if relevant (journalctl or /var/www/peertube/storage/logs/): no error seen

@decentral1se
Copy link
Contributor Author

decentral1se commented Sep 3, 2021

Oh yeah, here is our local-production.yaml file:

root@b5c9f675e1b4:/config# cat local-production.json 
{
  "instance": {
    "name": "tv.lumbung.space",
    "short_description": "A self-organized space for the lumbung inter-lokal",
    "description": "Welcome to [tv.lumbung.space](https://tv.lumbung.space)!\n\nThis is all a big work in progress.\n\nWe'll have more soon.",
    "is_nsfw": false,
    "default_nsfw_policy": "do_not_list",
    "terms": "No terms for now.",
    "code_of_conduct": "",
    "creation_reason": "",
    "moderation_information": "",
    "administrator": "",
    "maintenance_lifetime": "",
    "business_model": "",
    "hardware_information": "",
    "categories": [],
    "languages": [],
    "default_client_route": "/videos/local",
    "customizations": {
      "javascript": "",
      "css": ""
    }
  },
  "theme": {
    "default": "default"
  },
  "services": {
    "twitter": {
      "username": "@foo",
      "whitelisted": false
    }
  },
  "cache": {
    "previews": {
      "size": 500
    },
    "captions": {
      "size": 500
    },
    "torrents": {
      "size": 500
    }
  },
  "signup": {
    "enabled": false,
    "limit": 10,
    "requires_email_verification": false
  },
  "import": {
    "videos": {
      "concurrency": 2,
      "http": {
        "enabled": true
      },
      "torrent": {
        "enabled": true
      }
    }
  },
  "trending": {
    "videos": {
      "algorithms": {
        "enabled": [
          "best",
          "hot",
          "most-viewed",
          "most-liked"
        ],
        "default": "best"
      }
    }
  },
  "admin": {
    "email": "foo@example.com"
  },
  "contact_form": {
    "enabled": false
  },
  "user": {
    "video_quota": -1,
    "video_quota_daily": -1
  },
  "transcoding": {
    "enabled": true,
    "threads": 4,
    "allow_additional_extensions": true,
    "allow_audio_files": true,
    "profile": "default",
    "concurrency": 2,
    "resolutions": {
      "0p": false,
      "240p": true,
      "360p": false,
      "480p": true,
      "720p": true,
      "1080p": false,
      "1440p": false,
      "2160p": false
    },
    "hls": {
      "enabled": true
    },
    "webtorrent": {
      "enabled": false
    }
  },
  "live": {
    "enabled": true,
    "max_duration": 18000000,
    "max_instance_lives": 20,
    "max_user_lives": 3,
    "allow_replay": true,
    "transcoding": {
      "enabled": true,
      "threads": 4,
      "profile": "default",
      "resolutions": {
        "240p": false,
        "360p": false,
        "480p": true,
        "720p": false,
        "1080p": false,
        "1440p": false,
        "2160p": false
      }
    }
  },
  "auto_blacklist": {
    "videos": {
      "of_users": {
        "enabled": false
      }
    }
  },
  "followers": {
    "instance": {
      "enabled": false,
      "manual_approval": false
    }
  },
  "followings": {
    "instance": {
      "auto_follow_back": {
        "enabled": false
      },
      "auto_follow_index": {
        "enabled": false,
        "index_url": ""
      }
    }
  },
  "broadcast_message": {
    "enabled": true,
    "level": "info",
    "dismissable": true,
    "message": "Welcome to **tv.lumbung.space**, bear with us as we're still setting everything up!"
  },
  "search": {
    "remote_uri": {
      "users": true,
      "anonymous": true
    },
    "search_index": {
      "enabled": false,
      "url": "",
      "disable_local_search": false,
      "is_default_search": false
    }
  }

@decentral1se
Copy link
Contributor Author

decentral1se commented Sep 3, 2021

Renaming /config/local-production.json to /config/local-production-bkup.json and restarting made the error go away but I am still not sure why this error was happening? It would be great to have a way to validate the config outside of the web interface.

@Chocobozzz
Copy link
Owner

Hello,

It seems there was an issue in 3.3 migration script: actorCustomPage should have been created by https://github.com/Chocobozzz/PeerTube/blob/develop/server/initializers/migrations/0650-actor-custom-pages.ts

@decentral1se
Copy link
Contributor Author

@Chocobozzz is there a way I can run that migration script myself via npm run ... or? To give some detail on my upgrade process, I simply changed the tag of the image in my compose configuration and then ran a docker stack deploy. I assume the entrypoint runs this script?

@decentral1se
Copy link
Contributor Author

@Chocobozzz
Copy link
Owner

Please check your logs about this migration

@Chocobozzz Chocobozzz added the Status: Waiting for answer Waiting issue author answer label Sep 6, 2021
@decentral1se
Copy link
Contributor Author

decentral1se commented Sep 8, 2021

[tv.lumbung.space:443] 2021-09-03 11:42:58.926 info: Begin migrations.
[tv.lumbung.space:443] 2021-09-03 11:42:58.930 info: Executing 0650-actor-custom-pages.js migration script.
[tv.lumbung.space:443] 2021-09-03 11:42:58.951 info: Migrations finished. New migration version schema: 650
[tv.lumbung.space:443] 2021-09-03 11:42:59.075 info: Database peertube is ready.

Feels to me that the potential bug is in the generation of the local-production.json from version to version? I can show you the config that the new version generated that is being accepted now? Maybe you can see the issue:

{
  "instance": {
    "name": "tv.lumbung.space",
    "short_description": "...",
    "description": "...",
    "is_nsfw": false,
    "default_nsfw_policy": "do_not_list",
    "terms": "No terms for now.",
    "code_of_conduct": "",
    "creation_reason": "",
    "moderation_information": "",
    "administrator": "",
    "maintenance_lifetime": "",
    "business_model": "",
    "hardware_information": "",
    "categories": [],
    "languages": [],
    "default_client_route": "/videos/local",
    "customizations": {
      "javascript": "",
      "css": "_ngcontent-jkj-c171 {\n display: none;\n}\n\n.external-login-blocks > div.block-title {\n   display: none;\n}\n\n.title-page.title-page-single::after {\n    content: \"with\";\n}\n"
    }
  },
  "theme": {
    "default": "default"
  },
  "services": {
    "twitter": {
      "username": "@foo",
      "whitelisted": false
    }
  },
  "cache": {
    "previews": {
      "size": 500
    },
    "captions": {
      "size": 500
    },
    "torrents": {
      "size": 500
    }
  },
  "signup": {
    "enabled": false,
    "limit": 10,
    "requires_email_verification": false,
    "minimum_age": 16
  },
  "import": {
    "videos": {
      "concurrency": 1,
      "http": {
        "enabled": true
      },
      "torrent": {
        "enabled": false
      }
    }
  },
  "trending": {
    "videos": {
      "algorithms": {
        "enabled": [
          "best",
          "hot",
          "most-viewed",
          "most-liked"
        ],
        "default": "most-viewed"
      }
    }
  },
  "admin": {
    "email": "foo@example.com"
  },
  "contact_form": {
    "enabled": false
  },
  "user": {
    "video_quota": -1,
    "video_quota_daily": -1
  },
  "transcoding": {
    "enabled": true,
    "threads": 2,
    "allow_additional_extensions": true,
    "allow_audio_files": true,
    "profile": "default",
    "concurrency": 1,
    "resolutions": {
      "0p": true,
      "240p": false,
      "360p": true,
      "480p": false,
      "720p": true,
      "1080p": false,
      "1440p": false,
      "2160p": false
    },
    "hls": {
      "enabled": true
    },
    "webtorrent": {
      "enabled": false
    }
  },
  "live": {
    "enabled": true,
    "max_duration": 18000000,
    "max_instance_lives": 20,
    "max_user_lives": 3,
    "allow_replay": true,
    "transcoding": {
      "enabled": true,
      "threads": 4,
      "profile": "default",
      "resolutions": {
        "240p": false,
        "360p": false,
        "480p": true,
        "720p": false,
        "1080p": false,
        "1440p": false,
        "2160p": false
      }
    }
  },
  "auto_blacklist": {
    "videos": {
      "of_users": {
        "enabled": false
      }
    }
  },
  "followers": {
    "instance": {
      "enabled": true,
      "manual_approval": true
    }
  },
  "followings": {
    "instance": {
      "auto_follow_back": {
        "enabled": false
      },
      "auto_follow_index": {
        "enabled": false,
        "index_url": ""
      }
    }
  },
  "broadcast_message": {
    "enabled": true,
    "level": "info",
    "dismissable": false,
    "message": "..."
  },
  "search": {
    "remote_uri": {
      "users": true,
      "anonymous": false
    },
    "search_index": {
      "enabled": false,
      "url": "",
      "disable_local_search": false,
      "is_default_search": false
    }
  }
}

@Chocobozzz
Copy link
Owner

The migration seems to have been a success, and there is nothing wrong with the local configuration. Closing this issue since the homepage (https://tv.lumbung.space/api/v1/custom-pages/homepage/instance ) seems to work as expected.

I also fixed a config bug in b4c945f but I don't think it's related to your issue.

@rscmbbng
Copy link

rscmbbng commented Sep 9, 2021

The migration seems to have been a success, and there is nothing wrong with the local configuration. Closing this issue since the homepage (https://tv.lumbung.space/api/v1/custom-pages/homepage/instance ) seems to work as expected.

Hi @Chocobozzz, that is not completely true because we ended up setting the vanilla config and I just entered the values through the interface by hand since we needed to use the instance. So it might mean that the migration still does not work properly, just a fyi!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Waiting for answer Waiting issue author answer
Projects
None yet
Development

No branches or pull requests

3 participants