feature request: download pages behind login/paywalls

[sanzoghenzo]

Request by email from Yannick Gauthier's: be able to convert pages that needs authentication to be accessed/downloaded.

As of now I don't have the need, ability and time to pursue this, but I opened this to see if there's some interest, and maybe some help/hints by anybody!

[graphit0]

I think this feature will require extensively rewriting and making the app much more complicated.

As for the ideas, Joplin web clipper implemented the feature pretty well. However it's written with JavaScript so cannot be reused, only serve as general guideline

[sanzoghenzo]

I think this feature will require extensively rewriting and making the app much more complicated.

Exactly my thoughts 😉

As for the ideas, Joplin web clipper implemented the feature pretty well. However it's written with JavaScript so cannot be reused, only serve as general guideline

Joplin web clipper, and all the other browser extensions, can load the html/dom of the currently opened page.
This is not the case in android, where the share intent of a browser contains only the url, and there's no way (that I know of) to get the contents.

IIRC, apps like feedly (or was it pocket?) store the login info to access subscription sites, but I would like to keep this app as simple as possible and avoid storing sensitive data.

[Fmstrat]

@sanzoghenzo Could you not use a WebView to pull the text out of the outer HTML post JavaScript rendering, then run that through html2md?

const String url_unext = 'https://video.unext.jp/title/SID0050925';
WebViewController _controller;
Stack(
  children: [
    WebView(
      onWebViewCreated: (controller) {
        _controller = controller;
      },
      javascriptMode: JavascriptMode.unrestricted,
      initialUrl: url_unext,
      onPageFinished: (_) async {
        html = await _controller.evaluateJavascript("window.document.firstElementChild.outerHTML;");
        // Use Metadata_fetch to parse
        final data = getOpenGraphDataFromResponse(html);
        print(data);
      },
    ),
    // else widget here
    Container(),
  ],
),

Source: https://stackoverflow.com/a/66615734

Edit: I realize the original request is for post-authentication vs popup paywalls. Perhaps this is not needed.