Q: Will destreamer work on the new Microsoft SharePoint Stream? #487

WG- · 2023-06-11T20:05:51Z

So Microsoft is phasing out Microsoft Stream and is moving it towards SharePoint.

Will destreamer be modified such that it supports SharePoint Stream?

cgonIT · 2023-06-13T07:40:17Z

+1

hakt0-r · 2023-06-24T15:26:42Z

The short answer is yes! I've just managed to get this working, and that too on my M1 mac running macOS Ventura 13.4.1.
I'll try to lay out what I've done to fix and hopefully can create a pull request and fix this too for all.

These are macOS specific instructions, though you can ignore macOS bits and make it work for other OS's.

brew install chromium
To avoid running into "chromium.app is damaged error" (see Can't install Chromium without --no-quarantine Homebrew/homebrew-cask#112986 (comment))
Run these two commands. (it'll set the right flags for macOS SIP/watchdog/whatever)

xattr -cr /opt/homebrew/bin/chromium
xattr -cr /Applications/Chromium.app

git clone --branch sharepoint https://github.com/snobu/destreamer.git
cd into that directory
edit the package.json file to change
change "@types/puppeteer" : "5.5.0",
to "puppeteer" : "^19.11.1",

change "puppeteer" : "5.5.0",
to "puppeteer" : "^19.11.1",

edit the tsconfig.json file to add a line after "sourceMap"...
"noImplicitAny": false,
now run npm install
(going for gold!) We'll also update all the node packages and their versions. Run npm update --save
at this stage you can now try and run npm run build
You'll get a few errors!
First kind...
few of these... src/LoginModules.ts:14:20 - error TS2503: Cannot find namespace 'puppeteer'.
and few of these... src/TokenCache.ts:60:20 - error TS2503: Cannot find namespace 'puppeteer'.
to Fix this, just edit these two files and change
this line => import puppeteer from 'puppeteer';
to this => import * as puppeteer from 'puppeteer';
running npm run build again, gives you one more error...
src/TokenCache.ts:99:24 - error TS2339: Property 'waitFor' does not exist on type 'Page'.
edit src/TokenCache.ts file line 99 and change waitFor to waitForTimeout (see: [Bug]: TypeError: page.waitFor is not a function when using pupeteer puppeteer/puppeteer#9762)
Finally you are ready to build. npm run build will work.

I've successfully tested today downloading two recordings. 😄

nfontan · 2023-09-22T00:08:38Z

The short answer is yes! I've just managed to get this working, and that too on my M1 mac running macOS Ventura 13.4.1. I'll try to lay out what I've done to fix and hopefully can create a pull request and fix this too for all.

These are macOS specific instructions, though you can ignore macOS bits and make it work for other OS's.

brew install chromium

To avoid running into "chromium.app is damaged error" (see Can't install Chromium without --no-quarantine Homebrew/homebrew-cask#112986 (comment))
Run these two commands. (it'll set the right flags for macOS SIP/watchdog/whatever)

xattr -cr /opt/homebrew/bin/chromium
xattr -cr /Applications/Chromium.app

git clone --branch sharepoint https://github.com/snobu/destreamer.git

cd into that directory

edit the package.json file to change
change "@types/puppeteer" : "5.5.0",
to "puppeteer" : "^19.11.1",

change "puppeteer" : "5.5.0", to "puppeteer" : "^19.11.1",

edit the tsconfig.json file to add a line after "sourceMap"...
"noImplicitAny": false,

now run npm install

(going for gold!) We'll also update all the node packages and their versions. Run npm update --save

at this stage you can now try and run npm run build
You'll get a few errors!
First kind...
few of these... src/LoginModules.ts:14:20 - error TS2503: Cannot find namespace 'puppeteer'.
and few of these... src/TokenCache.ts:60:20 - error TS2503: Cannot find namespace 'puppeteer'.
to Fix this, just edit these two files and change
this line => import puppeteer from 'puppeteer';
to this => import * as puppeteer from 'puppeteer';

running npm run build again, gives you one more error...
src/TokenCache.ts:99:24 - error TS2339: Property 'waitFor' does not exist on type 'Page'.
edit src/TokenCache.ts file line 99 and change waitFor to waitForTimeout (see: [Bug]: TypeError: page.waitFor is not a function when using pupeteer puppeteer/puppeteer#9762)

Finally you are ready to build. npm run build will work.

I've successfully tested today downloading two recordings. 😄

Here not work!

Here the results:
destreamer.cmd -i "https://xxxxxoutlook-my.sharepoint.com/:v:/g/personal/xxxx_xxxxx_com/EU8PrthK4gdGuXXxi1CY8AEBKhUcSjnfHmD-SayKpvBv2w?nav=eyJyZWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
Parsing video/group urls
Downloading SharePoint videos...

Launching headless Chrome to perform the OpenID Connect dance...
Navigating to login page...
Waiting for target!
We are logged in.

[FATAL ERROR] Could not evaluate sessionInfo on the page

cr2007 · 2023-09-23T10:55:13Z

Hey @hakt0-r

I tried these steps you had mentioned on Windows (running via WSL)
Even fixed all the lines of code before running npm run build

But I get this new error when I try to install from MS Stream:

$ ./destreamer.sh -i <Video-URL> -o <preferred-directory>
/bin/sh: 1: aria2c: not found


[FATAL ERROR] Aria2 is missing!
Destreamer requires a fairly recent release of Aria2 to download videos

If you encountered this, could you share the steps on how to fix this
Or refer me to a similar issue where it has been resolved?

nfontan · 2023-09-23T15:01:40Z

Hey @hakt0-r

I tried these steps you had mentioned on Windows (running via WSL) Even fixed all the lines of code before running npm run build

But I get this new error when I try to install from MS Stream:
$ ./destreamer.sh -i <Video-URL> -o <preferred-directory>
/bin/sh: 1: aria2c: not found


[FATAL ERROR] Aria2 is missing!
Destreamer requires a fairly recent release of Aria2 to download videos
If you encountered this, could you share the steps on how to fix this Or refer me to a similar issue where it has been resolved?

You need to install aria2
apt install -y aria2

cr2007 · 2023-09-23T15:07:12Z

I fixed that, and now a fatal error is appearing

$ ./destreamer.sh -i <VIDEO-URL> -o <DEST-DIRECTORY>
Parsing video/group urls
Downloading Microsoft Stream videos...

[WARNING] .token_cache not found.

Launching headless Chrome to perform the OpenID Connect dance...

[ERROR] Error: Failed to launch the browser process!
/home/<user>/.cache/puppeteer/chrome/linux-1108766/chrome-linux/chrome: error while loading shared libraries: libatk-1.0.so.0: cannot open shared object file: No such file or directory


TROUBLESHOOTING: https://pptr.dev/troubleshooting

    at Interface.onClose (/home/<user>/destreamer/node_modules/@puppeteer/browsers/lib/cjs/launch.js:262:24)
    at Interface.emit (node:events:525:35)
    at Interface.close (node:internal/readline/interface:536:10)
    at Socket.onend (node:internal/readline/interface:262:10)
    at Socket.emit (node:events:525:35)
    at endReadableNT (node:internal/streams/readable:1359:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)



[FATAL ERROR] Unhandled error!
Timeout or fatal error, please check your downloads directory and try again

hakt0-r · 2023-09-23T15:14:11Z

The key is…error while loading shared libraries: libatk-1.0.so.0: cannot open shared object file: No such file or directory Since you are running it via WSL, you’ll have to debug a bit more but here is what I found…Run this…. sudo apt install ca-certificates fonts-liberation libappindicator3-1 libasound2 libatk-bridge2.0-0 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgbm1 libgcc1 libglib2.0-0 libgtk-3-0 libnspr4 libnss3 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 lsb-release wget xdg-utilsHth.On 24 Sep 2023, at 01:07, CSK ***@***.***> wrote: I fixed that, and now a fatal error is appearing $ ./destreamer.sh -i <VIDEO-URL> -o <DEST-DIRECTORY> Parsing video/group urls Downloading Microsoft Stream videos... [WARNING] .token_cache not found. Launching headless Chrome to perform the OpenID Connect dance... [ERROR] Error: Failed to launch the browser process! /home/<user>/.cache/puppeteer/chrome/linux-1108766/chrome-linux/chrome: error while loading shared libraries: libatk-1.0.so.0: cannot open shared object file: No such file or directory TROUBLESHOOTING: https://pptr.dev/troubleshooting at Interface.onClose ***@***.***/browsers/lib/cjs/launch.js:262:24) at Interface.emit (node:events:525:35) at Interface.close (node:internal/readline/interface:536:10) at Socket.onend (node:internal/readline/interface:262:10) at Socket.emit (node:events:525:35) at endReadableNT (node:internal/streams/readable:1359:12) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) [FATAL ERROR] Unhandled error! Timeout or fatal error, please check your downloads directory and try again —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

nfontan · 2023-09-23T15:16:16Z

I fixed that, and now a fatal error is appearing

$ ./destreamer.sh -i <VIDEO-URL> -o <DEST-DIRECTORY>
Parsing video/group urls
Downloading Microsoft Stream videos...

[WARNING] .token_cache not found.

Launching headless Chrome to perform the OpenID Connect dance...

[ERROR] Error: Failed to launch the browser process!
/home/<user>/.cache/puppeteer/chrome/linux-1108766/chrome-linux/chrome: error while loading shared libraries: libatk-1.0.so.0: cannot open shared object file: No such file or directory


TROUBLESHOOTING: https://pptr.dev/troubleshooting

    at Interface.onClose (/home/<user>/destreamer/node_modules/@puppeteer/browsers/lib/cjs/launch.js:262:24)
    at Interface.emit (node:events:525:35)
    at Interface.close (node:internal/readline/interface:536:10)
    at Socket.onend (node:internal/readline/interface:262:10)
    at Socket.emit (node:events:525:35)
    at endReadableNT (node:internal/streams/readable:1359:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)



[FATAL ERROR] Unhandled error!
Timeout or fatal error, please check your downloads directory and try again

In my case I tried many fixes with wsl, installing other dependencies to make it work, anyway I couldn't get it to work with wsl.
Try again directly on windows with node for windows. but it didn't work either (error that I left above)

Finish downloading with ffmpeg -i "videomanifest" -codec copy downloadedVideo.mp4

hakt0-r · 2023-09-23T15:40:23Z

Here not work!

Here the results: destreamer.cmd -i "https://xxxxxoutlook-my.sharepoint.com/✌️/g/personal/xxxx_xxxxx_com/EU8PrthK4gdGuXXxi1CY8AEBKhUcSjnfHmD-SayKpvBv2w?nav=eyJyZWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" Parsing video/group urls Downloading SharePoint videos...

Launching headless Chrome to perform the OpenID Connect dance... Navigating to login page... Waiting for target! We are logged in.

[FATAL ERROR] Could not evaluate sessionInfo on the page

First of all, which branch did you clone?
SharePoint URLs work best with this branch https://github.com/hakt0-r/destreamer/tree/sharepoint

Clone that branch and follow all the instructions again please.

If you still encounter problems please paste the verbose output so that we can debug.

nfontan · 2023-09-23T15:49:05Z

Here not work!
Here the results: destreamer.cmd -i "https://xxxxxoutlook-my.sharepoint.com/✌️/g/personal/xxxx_xxxxx_com/EU8PrthK4gdGuXXxi1CY8AEBKhUcSjnfHmD-SayKpvBv2w?nav=eyJyZWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" Parsing video/group urls Downloading SharePoint videos...
Launching headless Chrome to perform the OpenID Connect dance... Navigating to login page... Waiting for target! We are logged in.
[FATAL ERROR] Could not evaluate sessionInfo on the page

First of all, which branch did you clone? SharePoint URLs work best with this branch https://github.com/hakt0-r/destreamer/tree/sharepoint

Clone that branch and follow all the instructions again please.

If you still encounter problems please paste the verbose output so that we can debug.

Yes, I'm testing with that branch.
I have also tried with the 3.0 branch but the error in that case is invalid url, but the login behavior seems more accurate

hakt0-r · 2023-09-23T15:58:31Z

Yes, I'm testing with that branch. I have also tried with the 3.0 branch but the error in that case is invalid url, but the login behavior seems more accurate !

🤨 all I can say is that

try back-to-back attempts.
See if it’s an issue with .cache or .token_cache
Look at Sharepoint.com #380 please.

recolic · 2024-01-06T03:14:21Z

The short answer is yes! I've just managed to get this working, and that too on my M1 mac running macOS Ventura 13.4.1. I'll try to lay out what I've done to fix and hopefully can create a pull request and fix this too for all.

dp: It worked for me on Arch Linux. I forked this repo and applied these fixes, so you don't need to do everything again. Just update executablePath and enjoy.
https://github.com/recolic/destreamer-sharepoint

cvee112 · 2024-01-09T12:37:44Z

The short answer is yes! I've just managed to get this working, and that too on my M1 mac running macOS Ventura 13.4.1. I'll try to lay out what I've done to fix and hopefully can create a pull request and fix this too for all.

Thanks a lot! This worked for me and I was even able to download one 18-min video because of it! But then when I tried running it again, Chromium now crashes a few seconds after launch. It has been like that ever since (even after restarting), and I don't know why. I'm on an M2 Mac with macOS Sonoma and I also have Arc, but when my first run succeeded, I had Arc open so I'm not sure what else could be causing the issue. Any guess on why it's happening and what I can do about it?

It shows either of the following errors:

Launching headless Chrome to perform the OpenID Connect dance...
Navigating to login page...


[ERROR] Error: Navigation failed because browser has disconnected!
    at new LifecycleWatcher (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/LifecycleWatcher.js:83:223)
    at Frame.goto (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Frame.js:183:25)
    at CDPPage.goto (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:438:91)
    at doShareLogin (/Users/cvee/Stream/destreamer/build/src/LoginModules.js:121:20)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async downloadShareVideo (/Users/cvee/Stream/destreamer/build/src/Downloaders.js:146:21)
    at async main (/Users/cvee/Stream/destreamer/build/src/destreamer.js:48:9)



[FATAL ERROR] Unhandled error!
Timeout or fatal error, please check your downloads directory and try again

or

Launching headless Chrome to perform the OpenID Connect dance...
Navigating to login page...

[ERROR] Error: Requesting main frame too early!
    at assert (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/util/assert.js:28:15)
    at FrameManager.mainFrame (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:173:32)
    at CDPPage.goto (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:438:79)
    at doShareLogin (/Users/cvee/Stream/destreamer/build/src/LoginModules.js:121:20)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async downloadShareVideo (/Users/cvee/Stream/destreamer/build/src/Downloaders.js:146:21)
    at async main (/Users/cvee/Stream/destreamer/build/src/destreamer.js:48:9)



[FATAL ERROR] Unhandled error!
Timeout or fatal error, please check your downloads directory and try again

When I launch Chromium by typing chromium on my console, it's the same - it opens then exits very soon after. It does show the following message but I'm not sure if it's related:

[3307:259:0109/204329.397809:ERROR:policy_logger.cc(157)] :components/enterprise/browser/controller/chrome_browser_cloud_management_controller.cc(161) Cloud management controller initialization aborted as CBCM is not enabled. Please use the `--enable-chrome-browser-cloud-management` command line flag to enable it if you are not using the official Google Chrome build.

UPDATE: I got it to work by switching the browser to Brave, but Chromium is still facing the same issue.

pedro-stanaka · 2024-02-19T21:20:14Z

Worked for me on Chromium (with Macbook Air M2). Steps:

Had to install with brew install --cask chromium --no-quarantine.
Changed some stuff to make it work with my URLs: https://github.com/pedro-stanaka/destreamer/tree/sharepoint
Profit!

philCryoport · 2024-02-23T00:28:33Z

Two notes:

@hakt0-r thank you for the above step-by-step process.

FYI on my M2-Max running Sonoma (MacOS 14), I didn't need to do the xattr commands -- but everything else was taken care of with your suggested fixes.

When I copy/paste the Microsoft Stream URL onto my Oh-My-Zsh command line, it's automatically escaping with backslashes the ?, =, and &.

Is anybody else having this issue? I'm half tempted to insert code to look for those escaped characters and remove the escaping...

pedro-stanaka · 2024-02-23T08:33:57Z

Two notes:

@hakt0-r thank you for the above step-by-step process.

FYI on my M2-Max running Sonoma (MacOS 14), I didn't need to do the xattr commands -- but everything else was taken care of with your suggested fixes.

When I copy/paste the Microsoft Stream URL onto my Oh-My-Zsh command line, it's automatically escaping with backslashes the ?, =, and &.

Is anybody else having this issue? I'm half tempted to insert code to look for those escaped characters and remove the escaping...

Just paste the url in a text editor with ' (single quotes) wrapping the url. It should work then.

snowcross · 2024-03-22T14:44:47Z

I had this working for a month or so, now I keep getting an "[ERROR] Invalid url" I think the URLs changed from sharpoint recently. as in the lat week.

philCryoport · 2024-04-22T18:04:14Z

@pedro-stanaka wrote:

Just paste the url in a text editor with ' (single quotes) wrapping the url. It should work then.

Yep, I figured that out after posted...

This was referenced Jun 24, 2023

Mac M1 stopped working #393

Open

Getting destreamer working on M1/M2 macs + using all latest packages + sharepoint videos. Tested + working. #489

Open

hostar mentioned this issue Oct 30, 2023

Support for New Stream (on SharePoint) #493

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Q: Will destreamer work on the new Microsoft SharePoint Stream? #487

Q: Will destreamer work on the new Microsoft SharePoint Stream? #487

WG- commented Jun 11, 2023

cgonIT commented Jun 13, 2023

hakt0-r commented Jun 24, 2023

nfontan commented Sep 22, 2023 •

edited

Loading

cr2007 commented Sep 23, 2023

nfontan commented Sep 23, 2023

cr2007 commented Sep 23, 2023

hakt0-r commented Sep 23, 2023 via email

nfontan commented Sep 23, 2023

hakt0-r commented Sep 23, 2023

nfontan commented Sep 23, 2023

hakt0-r commented Sep 23, 2023

recolic commented Jan 6, 2024

cvee112 commented Jan 9, 2024 •

edited

Loading

pedro-stanaka commented Feb 19, 2024 •

edited

Loading

philCryoport commented Feb 23, 2024

pedro-stanaka commented Feb 23, 2024

snowcross commented Mar 22, 2024

philCryoport commented Apr 22, 2024

Q: Will destreamer work on the new Microsoft SharePoint Stream? #487

Q: Will destreamer work on the new Microsoft SharePoint Stream? #487

Comments

WG- commented Jun 11, 2023

cgonIT commented Jun 13, 2023

hakt0-r commented Jun 24, 2023

nfontan commented Sep 22, 2023 • edited Loading

cr2007 commented Sep 23, 2023

nfontan commented Sep 23, 2023

cr2007 commented Sep 23, 2023

hakt0-r commented Sep 23, 2023 via email

nfontan commented Sep 23, 2023

hakt0-r commented Sep 23, 2023

nfontan commented Sep 23, 2023

hakt0-r commented Sep 23, 2023

recolic commented Jan 6, 2024

cvee112 commented Jan 9, 2024 • edited Loading

pedro-stanaka commented Feb 19, 2024 • edited Loading

philCryoport commented Feb 23, 2024

pedro-stanaka commented Feb 23, 2024

snowcross commented Mar 22, 2024

philCryoport commented Apr 22, 2024

nfontan commented Sep 22, 2023 •

edited

Loading

cvee112 commented Jan 9, 2024 •

edited

Loading

pedro-stanaka commented Feb 19, 2024 •

edited

Loading