-
Notifications
You must be signed in to change notification settings - Fork 31
/
Copy pathGet-EventSession.ps1
1891 lines (1692 loc) · 96.2 KB
/
Get-EventSession.ps1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<#
.SYNOPSIS
Script to assist in downloading Microsoft Ignite, Inspire, Build or MEC contents, or return
session information for easier digesting. Video downloads will leverage external utilities,
depending on the used video format. To prevent retrieving session information for every run,
the script will cache session information.
Be advised that downloading of OnDemand contents from Azure Media Services is throttled to real-time
speed. To lessen the pain, the script performs simultaneous downloads of multiple videos streams. Those
downloads will each open in their own (minimized) window so you can track progress. Finally, CTRL-C
is catched by the script because we need to stop download jobs when aborting the script.
THIS CODE IS MADE AVAILABLE AS IS, WITHOUT WARRANTY OF ANY KIND. THE ENTIRE
RISK OF THE USE OR THE RESULTS FROM THE USE OF THIS CODE REMAINS WITH THE USER.
Michel de Rooij
http://eightwone.com
Version 4.23, November 22th, 2024
Special thanks to: Mattias Fors, Scott Ladewig, Tim Pringle, Andy Race, Richard van Nieuwenhuizen
.DESCRIPTION
This script can download Microsoft Ignite, Inspire, Build and MEC session information and available
slidedecks and videos using MyIgnite/MyInspire/MyBuild techcommunity portal.
Video downloads will leverage one or more utilities:
- YouTube-dl, which can be downloaded from https://yt-dl.org/latest/youtube-dl.exe. This utility
needs to reside in the same folder as the script. The script itself will try to download this
utility when the utility is not present.
- ffmpeg, which can be downloaded from https://ffmpeg.zeranoe.com/builds/win32/static/ffmpeg-latest-win32-static.zip.
This utility needs to reside in the same folder as the script, or you need to specify its location using -FFMPEG.
The utility is used to bind the seperate video and audio streams of Azure Media Services files
in single files.
When you are interested in retrieving session information only, you can use the InfoOnly switch.
Note: MEC sessions are not published through the usual API, so I worked around it by digesting its playlist as
if it were a catalog. Consequence is that filtering might be limited, eg. no Category or Product etc.
.PARAMETER DownloadFolder
Specifies location to download sessions to. When omitted, will use 'systemdrive'\'Event'.
.PARAMETER Format
The Format specified depends on the media hosting the source videos:
- Direct Downloads
- Azure Media Services
- YouTube
Azure Media Services
====================
For Azure Media Services, default option is worstvideo+bestaudio/best. Alternatively, you can
select other formats (when present), e.g. bestvideo+bestaudio. Note that the format requested
needs to be present in the stream package. Storage required for bestvideo is significantly
more than worstvideo. Note that you can also provide complex filter and preference, e.g.
bestvideo[height=540][filesize<384MB]+bestaudio,bestvideo[height=720][filesize<512MB]+bestaudio,bestvideo[height=360]+bestaudio,bestvideo+bestaudio
1) This would first attempt to download the video of 540p if it is less than 384MB, and best audio.
2) When not present, then attempt to downlod video of 720p less than 512MB.
3) Thirdly, attempt to download video of 360p with best audio.
4) If none of the previous filters found matches, just pick the best video and best audio streams.
For Azure Media Services, you could also use format tags, such as 1_V_video_1 or 1_V_video_3.
Note that these formats might not be consistent for different streams, e.g. 1_V_video_1
might represent 1280x720 in one stream, while corresponding to 960x540 in another. To
prevent this, usage of filters is recommended. For Azure Streams, you can use the following format:
478 mp4 320x180 30 │ ~237.16MiB 478k m3u8 │ avc3.4d4016 478k video only
628 mp4 384x216 30 │ ~311.58MiB 628k m3u8 │ avc3.4d401e 628k video only
928 mp4 512x288 30 │ ~460.43MiB 928k m3u8 │ avc3.4d4020 928k video only
1428 mp4 640x360 30 │ ~708.50MiB 1428k m3u8 │ avc3.4d4020 1428k video only
2128 mp4 960x540 30 │ ~ 1.03GiB 2128k m3u8 │ avc3.4d4020 2128k video only
3128 mp4 1280x720 30 │ ~ 1.52GiB 3128k m3u8 │ avc3.640029 3128k video only
6128 mp4 1920x1080 30 │ ~ 2.97GiB 6128k m3u8 │ avc3.64002a 6128k video only
YouTube
=======
For YouTube videos, you can use the following formats:
160 mp4 256x144 DASH video 108k , avc1.4d400b, 30fps, video only
133 mp4 426x240 DASH video 242k , avc1.4d400c, 30fps, video only
134 mp4 640x360 DASH video 305k , avc1.4d401e, 30fps, video only
135 mp4 854x480 DASH video 1155k , avc1.4d4014, 30fps, video only
136 mp4 1280x720 DASH video 2310k , avc1.4d4016, 30fps, video only
137 mp4 1920x1080 DASH video 2495k , avc1.640028, 30fps, video only
18 mp4 640x360 medium , avc1.42001E, mp4a.40.2@ 96k
22 mp4 1280x720 hd720 , avc1.64001F, mp4a.40.2@192k (best, default)
You can use filters or priority when selecting the media:
- Filters allow you to put criteria on the media you select to download, e.g.
"bestvideo[height<=540]+bestaudio" will download the video stream where video is 540p at
most plus the audio stream (and ffmpeg will combine the two to a single MP4 file). It
allows you also to do cool things like "bestvideo[filesize<200M]+bestaudio".
- Priority allows you to provide additional criteria if the previous one fails, such as
when a desired quality is not available, e.g. "bestvideo+bestaudio/worstvideo+bestaudio"
will download worst video and best audio stream when the best video and audio streams
are not present.
Format selection filter courtesey of Youtube-DL; for more examples, see
https://github.com/ytdl-org/youtube-dl/blob/master/README.md#format-selection-examples
Direct Downloads
================
Direct Downloads are downloaded directly from the provided downloadVideoLink source.
.PARAMETER Captions
When specified, for Azure Media Services contents, downloads caption files where available.
Files are usually in VTT format, and playable by VLC Player a.o. Note that captions might not always
be accurate due to machine translation, but at least will help in following the story :)
.PARAMETER Subs
When specified, for YouTube and Azure Media Services, downloads subtitles in provided languages by
specifying one or more 2-letter language codes seperated by a comma, e.g. en,fr,de,nl. Downloaded
subtitles may be in VTT or SRT format. Again, the subtitles might not always be accurate due to machine
translation. Note: For Azure Media Services, will only download caption in first language specified
.PARAMETER Language
When specified, for Azure Media hosted contents, downloads videos with specified audio stream where
available. Note that if you mix this with specifying your own Format parameter, you need to
add the language in the filter yourself, e.g. bestaudio[format_id*=German]. Default value is English,
as otherwise YouTube will download the last audio stream from the manifest (which often is Spanish).
.PARAMETER Keyword
Only retrieve sessions with this keyword in their session description.
.PARAMETER Title
Only retrieve sessions with this keyword in their session title.
.PARAMETER Speaker
Only retrieve sessions with this speaker.
.PARAMETER Product
Only retrieve sessions for this product. You need to specify the full product, subproducts seperated
by '/', e.g. 'Microsoft 365/Office 365/Office 365 Management'. Wildcards are allowed.
.PARAMETER Category
Only retrieve sessions for this category. You need to specify the full category, subcategories seperated
by '/', e.g. 'M365/Admin, Identity & Mgmt'. Wildcards are allowed.
.PARAMETER SolutionArea
Only retrieve sessions for this solution area. You need to specify the full
name, e.g. 'Modern Workplace'. Wildcards are allowed.
.PARAMETER LearningPath
Only retrieve sessions part of this this learningPath. You need to specify
the full name, e.g. 'Data Analyst'. Wildcards are allowed.
.PARAMETER Topic
Only retrieve sessions for this topic area. Wildcards are allowed.
.PARAMETER ScheduleCode
Only retrieve sessions with this session code. You can use one or more codes.
.PARAMETER NoVideos
Switch to indicate you don't want to download videos.
.PARAMETER NoSlidedecks
Switch to indicate you don't want to download slidedecks.
.PARAMETER NoGuessing
Switch to indicate you don't want the script to try to guess the URLs to retrieve media from MS Studios.
.PARAMETER NoRepeats
Switch to indicate you don't want the script to download repeated sessions.
.PARAMETER FFMPEG
Specifies full location of ffmpeg.exe utility. When omitted, it is searched for and
when required extracted to the current folder.
.PARAMETER MaxDownloadJobs
Specifies the maximum number of concurrent downloads.
.PARAMETER Proxy
Specify the URI of the proxy to use, e.g. http://proxy:8080. When omitted, the current
system settings will be used.
.PARAMETER Start
Item number to start crawling with - useful for restarts.
.PARAMETER Event
Specify what event to download sessions for.
Options are:
- Ignite : Ignite contents (current)
- Ignite2024,Ignite2023,Ignite2022,Ignite2021 : Ignite contents from that year/time
- Inspire : Inspire contents (current)
- Inspire2023,Inspire2022,Inspire2021 : Inspire contents from that year
- Build : Build contents (current)
- Build2023,Build2022,Build2021 : Build contents from that year
- MEC : MEC contents (current)
- MEC2022 : MEC contents from that year
.PARAMETER OGVPicker
Specify that you want to pick sessions to download using Out-GridView.
.PARAMETER InfoOnly
Tells the script to return session information only.
Note that by default, only session code and title will be displayed.
.PARAMETER Overwrite
Skips detecting existing files, overwriting them if they exist.
.PARAMETER PreferDirect
Instructs script to prefer direct video downloads over Azure Media Services, when both are
available. Note that direct downloads may be faster, but offer only single quality downloads,
where AMS may offer multiple video qualities.
.PARAMETER Timestamp
Tells script to change the timestamp of the downloaded media files to match the original
session timestamp, when available.
.PARAMETER Locale
When supported by the event, filters sessions on localization.
Currently supported: de-DE, zh-CN, en-US, ja-JP, es-CO, fr-FR.
When omitted, defaults to en-US.
.PARAMETER Refresh
When specified, this switch will try fetch current catalog information from online, ignoring
any cached information which might be present.
.PARAMETER ConcurrentFragments
Specifies the number of fragments for yt-dlp to download simultaneously when downloading videos.
Default is 4.
.PARAMETER TempPath
This will allow you to specify a folder for yt-dlp to store temporary files in, eg fragments.
When omitted, the folder where the videos are saved to will be used.
.NOTES
The youtube-dl.exe utility requires Visual C++ 2010 redist package
https://www.microsoft.com/en-US/download/details.aspx?id=5555
Changelog
=========
2.0 Initial (Mattias Fors)
2.1 Added video downloading, reformatting code (Michel de Rooij)
2.11 Fixed titles with apostrophes
Added Keyword and Title parameter
2.12 Replaced pptx download Invoke-WebRequest with .NET webclient request (=faster)
Fixed titles with backslashes (who does that?)
2.13 Adjusts pptx timestamp to publishing timestamp
2.14 Made filtering case-insensitive
Added NoVideos to download slidedecks only
2.15 Fixed downloading of differently embedded youtube videos
Added timestamping of downloaded pptx files
Minor output changes
2.16 More illegal character fixups
2.17 Bumped max post to check to 1750
2.18 Added option to download for sessions listed in a schedule shared from MyIgnite
Added lookup of video YouTube URl from MyIgnite if not found in TechCommunity
Added check to make sure conversation titles begin with session code
Added check to make sure we skip conversations we've already checked since some RSS IDs are duplicates
2.19 Added trimming of filenames
2.20 Incorporated Tim Pringle's code to use JSON to acess MyIgnite catalog
Added option to select speaker
Added caching of session information (expires in 1 day, or remove .cache file)
Removed Start parameter (we're now pre-reading the catalog)
2.21 Added proxy support, using system configured setting
Fixed downloading of slidedecks
2.22 Added URL parameter
Renamed script to IgniteDownloader.ps1
2.5 Added InfoOnly switch
Added Product parameter
Renamed script to Get-IgniteSession.ps1
2.6 Fixed slide deck downloading
Added Overwrite switch
2.61 Added placeholder slide deck removal
2.62 Fixed Overwrite logic bug
Renamed to singular Get-IgniteSession to keep in line with PoSH standards
2.63 Fixed bug reporting failed pptx download
Added reporting of placeholder decks and videos
2.64 Added processing of direct download links for videos
2.65 Added option to specify multiple sessionCode codes
Added note in source that format only works for YouTube video downloads.
Added youtube-dl returncode check in case it won't run (e.g. missing VC library).
2.66 Added proper downloading of session info using UTF-8 (no more '???')
Additional trimming of spaces and CRLF's in property values
2.7 Added Event parameter to switch between Ignite and Inspire catalog
Renamed script to Get-EventSession
Changed cached session info name to include event
Removed obsolete URL parameter
Added code to download slidedecks in PDF (Inspire)
Cleanup of script synopsis/description/etc.
2.8 Added downloading of Azure Media Services hosted streaming media
Added simultaneous downloading of AMS hosted OnDemand streams
Added NoSlidedecks switch
2.9 Added Category parameter
Fixed searching on Product
Increased itemsPerPage when retrieving catalog
2.91 Update to video downloading routine due to changes in published session info
2.92 Fix 'Could not create SSL/TLS secure channel' issues with Invoke-WebRequest
2.93 Update to slidedeck downloading routine due to changes in published session info
2.94 Fixed cleanup of finished jobs
2.95 Fixed encoding of filenames
2.96 Fixed terminating cleanup when no slidedecks are being downloaded
Added testing for contents to show contents is not available rather than generic 'problem'
2.97 Update to change in video downloading location (YouTube)
Changed default Format due to switch in video hosting - see YouTube format table
2.971 Changed regex for YouTube matching to skip 'Coming Soon'
Made verbose mode less noisy
2.98 Converted background downloads to single background job queue
Cosmetics
2.981 Added cleanup of occasional leftovers (eg *.mp4.f5_A_aac_UND_2_192_1.ytdl, *.f1_V_video_3.mp4)
2.982 Minor tweaks
2.983 Added OGVPicker switch
2.984 Changed keyword search to description, not abstract
Fixed searching for Products and Category
Added searching for SolutionArea
Added searching for LearningPath
2.985 Added Proxy support
2.986 Minor update to accomodate publishing of slideDecks links
3.0 Added Build support
3.01 Added CTRL-Break notice to 'waiting for downloads' message
Fixed 'No video located for' message
3.1 Updated to work with the Inspire 2019 catalog
Cosmetics
3.11 Some more Cosmetics
3.12 Updated to work with current Ignite & Build catalogs
Bumped the download retry limits for YouTube-dl a bit
3.13 Updated Ignite catalog endpoints
3.14 Removed superfluous testing loading of main event page
Fixed LearningPath option verbose output
Some code cosmetics
3.15 Added Topic parameter
3.16 Corrected prefixes for Ignite 2019
3.17 Added NoGuess switch
Added NoRepeats switch
3.18 Added Ignite2018 event
3.19 Fixed video downloading
3.20 Fixed background job cleanup
3.21 Added Timestamp switch
Updated file naming to strip embbeded name of format, e.g. f1_V_video_3
Added stopping of Youtube-DL helper app spawned processes
3.22 Added skipping of processing future sessions
3.23 Added Captions switch and Subs parameter
Added skipping of additional repeats (schedule code ending in R2/R3)
Fixed filename construction containing '%'
Added filtering options to description of Format parameter
Decreased probing/retrieving video URLs from Azure Media Services (speed benefit)
3.24 Added PreferDirect switch
Enhanced Format parameter description
3.25 Updated Youtube-DL download URL
3.26 Updated mutual exclusion for PreferDirect & other parameters/switches
Added workaround for long file names (NT Style name syntax)
Added PowerShell ISE detection
Added Garbage Collection
3.27 Reworked jobs for downloading videos
Added status bars for downloading of videos
Failed video downloads will show last line of error output
Added replacement of square brackets in file names
Removed obsolete Clean-VideoLeftOvers call
3.28 Uncommented line to cleanup output files after downloading video
Changed 'Error' lines to single line outputs or throws (where appropriate)
3.29 Added 'Stopped downloading ..' messages when terminating
3.30 Increased wait cycle during progress refresh
Added schedule code to progress status
Revised detection successful video downloads
3.31 Corrected video cleanup logic
3.32 Do not assume Slidedeck exists when size is 0
3.33 Fixed typo when specifying format for direct YouTube downloads
3.34 Updated for Build 2020
Added NoRepeat filtering for Build 2020
Made Event parameter mandatory, and not defaulting to Ignite
Added filtering example to Format parameter spec
3.35 Updated for Inspire 2020
3.36 Small fix for Inspire repeat session naming
3.37 Added ExcludecommunityTopic parameter (so you can skip 'Fun and Wellness' Animal Cam contents)
Modified Keyword and Title parameters (can be multiple values now)
3.38 Added detection of filetype for presentations (PPTX/PDF)
3.39 Added code to deal with specifying <Event><Year>
3.40 Modified API endpoint for Ignite 2020
Changed yearless Event specification to add year suffix, eg Ignite->Ignite2020, etc.
Fixed Azure Media Services video scraping for Ignite2020
3.41 Fixed: Error message for timeless sessions after downloading caption file
Fixed: Downloading of caption files when video file is already downloaded
3.42 Changed source location of ffmpeg. Download will now fetch current static x64 release.
3.43 Fixed Ignite 2020 slidedeck 'trial & error' URL
3.44 Fixed downloading of non-PDF slidedecks
3.45 Help updated for -Event
3.46 Changed downloading of caption files in background jobs as well
Optimized caption downloading preventing unnecessary page downloads
3.47 Added Captions to PreferDirect command set
3.50 Updated for Ignite 2021
Small cleanup
3.51 Updated for Build 2021
3.52 Updated NoRepeats maximum repeat check
Added Language parameter to support Azure Media Services hosted videos containing multiple audio tracks
3.53 Updated for Inspire 2021
3.54 Fixed adding Language filter when complex Format is specified
3.55 Fixed audio stream selection when requested language is not available or only single audio stream is present
3.60 Added support for Ignite 2021; specify individual event using Ignite2021H1 (Spring) or Ignite2021H2 (Fall)
3.61 Added support for (direct) downloading of Ignite Fall 2021 videos
3.62 Added Cleanup video leftover files if video file exists (to remove clutter)
Changed lifetime of cached session information to 8 hours
Fixed post-download counts
3.63 Fixed keyword filtering
3.64 Changed filter so that default language is picked when specified language is not available
3.65 Updated for Build 2022
Added Locale parameter to filter local content
Fixed applying timestamp due to DateTime formatting changes
3.66 Fixed filtering on langLocale
Default Locale set to en-US
3.67 Added removal of placeholder deck/video/vtt files
3.68 Fixed caching when specifying Event without year tag, eg. Build vs Build2022
Removed default Locale as that would mess things up for Events where data does not contain that information (yet).
3.69 Updated for Inspire 2022
3.70 Added MEC support
3.71 Fixed MEC description & speaker parsing
3.72 Fixed usage of format & subs arguments for direct YouTube downloads
3.73 Added MEC slide deck support
Fixed MEC parsing of description
3.74 Fixed MEC processing of multi-line descriptions
3.75 Added Ignite 2022 support
3.76 Removed session code uniqueness when storing session data, as session data now can contain multiple entries per language using the same code
3.77 Corrected API endpoints for some of the older events
3.78 Fixed content-based help
3.79 Fixed issue with placeholder detection
Fixed path handling, fixes file detection and timestamping a.o.
Added PowerShell 5.1 requirement (tested with)
3.80 Fixed redundant passing of Format to YouTube-dl
3.81 Moved to using ytl-dl, a fork of Youtube-DL (not maintained any longer)
3.82 Fixed new folder creation
3.83 Updated for Build 2023
Removed Ignite 2018 and Ignite 2019
3.9 Fixed retrieval of API-based catalogs for events
Switched to using REST calls for those API-based catalogs
Added Refresh Switch
Removed archived events (<2021) as MS archives sessions selectively from previous years
Merged Ignite2021H1 and Ignite2021H2 to Ignite2021
3.91 Fixed output mentioning youtube-dl instead of actual tool (yt-dlp)
3.92 Added .docx caption support for Build2023
3.93 Fixed scraping streams from Azure Media Services for Build2023+
Reinstated caption downloading with VTT instead of docx (can use Sub to download alt. language)
3.94 Added ytp-dl's --concurrent-fragments support (default 4)
3.95 Fixed localized VTT downloading for Build 2023+ from Azure Media Services
3.96 Removed hidden character causing "Â : The term 'Â' is not recognized .." messages.
3.97 Added Inspire 2023
3.98 Fixed retrieval of Inspire 2023 catalog
3.99 Fixed reporting of element when we cannot add language filter
4.00 Updated yt-dlp download location
Changed checking yt-dlp.exe presence & validity
4.01 Updated Event parameter help
4.02 Added Ignite 2023
4.10 Added Build 2024
4.11 Fixed bug in downloading captions
4.20 Added Ignite 2024
4.21 Fixed date-range for Ignite 2024 ao
4.22 Fixed download locations for Ignite 2024 content
Added Azure Stream format guidance
4.23 Added TempPath parameter to specifying yt-dlp temporary files location
Fixed overwrite mode when calling yt-dlp
Added parameter description for ConcurrentFragments
Fixed reporting of failed downloads
Some minor code cleanup
.EXAMPLE
Download all available contents of Ignite sessions containing the word 'Teams' in the title to D:\Ignite, and skip sessions from the CommunityTopic 'Fun and Wellness'
.\Get-EventSession.ps1 -DownloadFolder D:\Ignite-Format 22 -Keyword 'Teams' -Event Ignite -ExcludecommunityTopic 'Fun and Wellness'
.EXAMPLE
Get information of all sessions, and output only location and time information for sessions (co-)presented by Tony Redmond:
.\Get-EventSession.ps1 -InfoOnly | Where {$_.Speakers -contains 'Tony Redmond'} | Select Title, location, startDateTime
.EXAMPLE
Download all available contents of sessions BRK3248 and BRK3186 to D:\Ignite
.\Get-EventSession.ps1 -DownloadFolder D:\Ignite -ScheduleCode BRK3248,BRK3186
.EXAMPLE
View all Exchange Server related sessions as Ignite including speakers(s), and sort them by date/time
Get-EventSession.ps1 -Event Ignite -InfoOnly -Product '*Exchange Server*' | Sort-Object startDateTime | Select-Object @{n='Session'; e={$_.sessionCode}}, @{n='When';e={([datetime]$_.startDateTime).ToString('g')}}, title, @{n='Speakers'; e={$_.speakerNames -join ','}}
.EXAMPLE
Get all available sessions, display them in a GridView to select multiple at once, and download them to D:\Ignite
.\Get-EventSession.ps1 -ScheduleCode (.\Get-EventSession.ps1 -InfoOnly | Out-GridView -Title 'Select Videos to Download, or Cancel for all Videos' -PassThru).SessionCode -MaxDownloadJobs 10 -DownloadFolder 'D:\Ignite'
#>
[cmdletbinding( DefaultParameterSetName = 'Default' )]
param(
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string]$DownloadFolder,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string]$Format,
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string[]]$Keyword,
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string[]]$Title,
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string]$Speaker,
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string]$Product,
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string]$Category = '',
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string]$SolutionArea,
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string]$LearningPath,
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string]$Topic,
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string[]]$ScheduleCode,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string[]]$ExcludecommunityTopic,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[switch]$NoVideos,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[switch]$NoSlidedecks,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[string]$FFMPEG,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[ValidateRange(1,128)]
[int]$MaxDownloadJobs=4,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[uri]$Proxy=$null,
[parameter( Mandatory = $true, ParameterSetName = 'Download')]
[parameter( Mandatory = $true, ParameterSetName = 'Default')]
[parameter( Mandatory = $true, ParameterSetName = 'Info')]
[parameter( Mandatory = $true, ParameterSetName = 'DownloadDirect')]
[ValidateSet('MEC','MEC2022','Ignite', 'Ignite2024','Ignite2023', 'Ignite2022', 'Ignite2021', 'Inspire', 'Inspire2023', 'Inspire2022', 'Inspire2021', 'Build', 'Build2024', 'Build2023','Build2022', 'Build2021')]
[string]$Event='',
[parameter( Mandatory = $true, ParameterSetName = 'Info')]
[switch]$InfoOnly,
[parameter( Mandatory = $true, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[switch]$OGVPicker,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[switch]$Overwrite,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[switch]$NoRepeats,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[switch]$NoGuessing,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[switch]$Timestamp,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[string[]]$Subs,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[string]$Language='English',
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[ValidateSet('de-DE','zh-CN','en-US','ja-JP','es-CO','fr-FR')]
[string[]]$Locale='en-US',
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'Info')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[switch]$Refresh,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[switch]$Captions,
[parameter( Mandatory = $true, ParameterSetName = 'DownloadDirect')]
[switch]$PreferDirect,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
$ConcurrentFragments= 4,
[parameter( Mandatory = $false, ParameterSetName = 'Download')]
[parameter( Mandatory = $false, ParameterSetName = 'Default')]
[parameter( Mandatory = $false, ParameterSetName = 'DownloadDirect')]
[ValidateScript({ Test-Path -Path $_ -PathType Container})]
[string]$TempPath
)
# Max age for cache, older than this # hours will force info refresh
$MaxCacheAge = 8
$YouTubeEXE = 'yt-dlp.exe'
$YouTubeDL = Join-Path $PSScriptRoot $YouTubeEXE
$FFMPEG= Join-Path $PSScriptRoot 'ffmpeg.exe'
$YTlink = 'https://github.com/yt-dlp/yt-dlp/releases/download/2023.07.06/yt-dlp.exe'
$FFMPEGlink = 'https://www.gyan.dev/ffmpeg/builds/ffmpeg-release-essentials.zip'
# Fix 'Could not create SSL/TLS secure channel' issues with Invoke-WebRequest
[Net.ServicePointManager]::SecurityProtocol = "tls12, tls11, tls"
$script:BackgroundDownloadJobs= @()
Function Iif($Cond, $IfTrue, $IfFalse) {
If( $Cond) { $IfTrue } Else { $IfFalse }
}
Function Fix-FileName ($title) {
return (((((((($title -replace '\]', ')') -replace '\[', '(') -replace [char]0x202f, ' ') -replace '["\\/\?\*]', ' ') -replace ':', '-') -replace ' ', ' ') -replace '\?\?\?', '') -replace '\<|\>|:|"|/|\\|\||\?|\*', '').Trim()
}
Function Get-IEProxy {
If ( (Get-ItemProperty -Path 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings').ProxyEnable -ne 0) {
$proxies = (Get-ItemProperty -Path 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings').proxyServer
if ($proxies) {
if ($proxies -ilike "*=*") {
return $proxies -replace "=", "://" -split (';') | Select-Object -First 1
}
Else {
return ('http://{0}' -f $proxies)
}
}
Else {
return $null
}
}
Else {
return $null
}
}
Function Clean-VideoLeftovers ( $videofile) {
$masks= '.*.mp4.part', '.*.mp4.ytdl'
ForEach( $mask in $masks) {
If( $TempPath) {
$FileMask= (Join-Path -Path $TempPath -ChildPath (Split-Path -Path $videofile -Leaf)) -replace '.mp4', $mask
}
Else {
$FileMask= $videofile -replace '.mp4', $mask
}
Get-Item -LiteralPath $FileMask -ErrorAction SilentlyContinue | ForEach-Object {
Write-Verbose ('Removing leftover file {0}' -f $_.fullname)
Remove-Item -LiteralPath $_.fullname -Force -ErrorAction SilentlyContinue
}
}
}
Function Get-BackgroundDownloadJobs {
$Temp= @()
ForEach( $job in $script:BackgroundDownloadJobs) {
switch( $job.Type) {
1 {
$isJobRunning= $job.job.State -eq 'Running'
}
2 {
$isJobRunning= -not $job.job.hasExited
}
3 {
$isJobRunning= $job.job.State -eq 'Running'
}
default {
$isJobRunning= $false
}
}
if( $isJobRunning) {
$Temp+= $job
}
Else {
# Job finished, process result
switch( $job.Type) {
1 {
$isJobSuccess= $job.job.State -eq 'Completed'
$DeckInfo[ $InfoDownload]++
Write-Progress -Id $job.job.Id -Activity ('Slidedeck {0} {1}' -f $Job.scheduleCode, $Job.title) -Completed
}
2 {
$isJobSuccess= Test-Path -LiteralPath $job.file
$VideoInfo[ $InfoDownload]++
Write-Progress -Id $job.job.Id -Activity ('Video {0} {1}' -f $Job.scheduleCode, $Job.title) -Completed
}
3 {
$isJobSuccess= Test-Path -LiteralPath $job.file
Write-Progress -Id $job.job.Id -Activity ('Captions {0} {1}' -f $Job.scheduleCode, $Job.title) -Completed
}
default {
$isJobSuccess= $false
}
}
# Test if file is placeholder
$isPlaceholder= $false
If( Test-Path -LiteralPath $job.file) {
$FileObj= Get-ChildItem -LiteralPath $job.file
If( $FileObj.Length -eq 42) {
If( (Get-Content -LiteralPath $job.File) -eq 'No resource file is available for download') {
Write-Warning ('Removing {0} placeholder file {1}' -f $job.scheduleCode, $job.file)
Remove-Item -LiteralPath $job.file -Force
$isPlaceholder= $true
Switch( $job.Type) {
1 {
# Placeholder Deck file downloaded
$DeckInfo[ $InfoDownload]--
$DeckInfo[ $InfoPlaceholder]++
}
2 {
# Placeholder Video file downloaded
$VideoInfo[ $InfoDownload]--
$VideoInfo[ $InfoPlaceholder]++
}
3 {
# Placeholder VTT file downloaded
}
}
}
Else {
# Placeholder different text?
}
}
}
If( $isJobSuccess -and -not $isPlaceholder) {
Write-Host ('Downloaded {0}' -f $job.file) -ForegroundColor Green
# Do we need to adjust timestamp
If( $job.Timestamp) {
If( Test-Path -LiteralPath $job.file) {
Write-Verbose ('Applying timestamp {0} to {1}' -f $job.Timestamp, $job.file)
$FileObj= Get-ChildItem -LiteralPath $job.file
$FileObj.CreationTime= Get-Date -Date $job.Timestamp
$FileObj.LastWriteTime= Get-Date -Date $job.Timestamp
}
Else {
Write-Warning ('File {0} not found for timestamp adjustment' -f $job.file)
}
}
If( $job.Type -eq 2) {
# Clean video leftovers
Clean-VideoLeftovers $job.file
}
}
Else {
switch( $job.Type) {
1 {
Write-Host ('Problem downloading or missing slidedeck of {0} {1}' -f $job.scheduleCode, $job.title) -ForegroundColor Red
$job.job.ChildJobs | Stop-Job | Out-Null
$job.job | Stop-Job -PassThru | Remove-Job -Force | Out-Null
}
2 {
$LastLine= (Get-Content -LiteralPath $job.stdErrTempFile -ErrorAction SilentlyContinue) | Select-Object -Last 1
Write-Host ('Problem downloading or missing video of {0} {1}: {2}' -f $job.scheduleCode, $job.title, $LastLine) -ForegroundColor Red
Remove-Item -LiteralPath $job.stdOutTempFile, $job.stdErrTempFile -Force -ErrorAction Ignore
}
3 {
Write-Host ('Problem downloading or missing captions of {0} {1}' -f $job.scheduleCode, $job.title) -ForegroundColor Red
$job.job.ChildJobs | Stop-Job | Out-Null
$job.job | Stop-Job -PassThru | Remove-Job -Force | Out-Null
}
default {
}
}
}
}
}
$Num= ($Temp| Measure-Object).Count
$script:BackgroundDownloadJobs= $Temp
Show-BackgroundDownloadJobs
return $Num
}
Function Show-BackgroundDownloadJobs {
$Num=0
$NumDeck= 0
$NumVid= 0
$NumCaption= 0
ForEach( $BGJob in $script:BackgroundDownloadJobs) {
$Num++
Switch( $BGJob.Type) {
1 {
$NumDeck++
}
2 {
$NumVid++
}
3 {
$NumCaption++
}
}
}
Write-Progress -Id 2 -Activity 'Background Download Jobs' -Status ('Total {0} in progress ({1} slidedeck, {2} video and {3} caption files)' -f $Num, $NumDeck, $NumVid, $NumCaption)
ForEach( $job in $script:BackgroundDownloadJobs) {
If( $Job.Type -eq 2) {
# Get last line of YT log to display for video downloads
$LastLine= (Get-Content -LiteralPath $job.stdOutTempFile -ErrorAction SilentlyContinue) | Select-Object -Last 1
If(!( $LastLine)) {
$LastLine= 'Evaluating..'
}
Write-Progress -Id $job.job.id -Activity ('Video {0} {1}' -f $job.scheduleCode, $Job.title) -Status $LastLine -ParentId 2
$progressId++
}
}
}
Function Stop-BackgroundDownloadJobs {
# Trigger update jobs running data
$null= Get-BackgroundDownloadJobs
# Stop all slidedeck background jobs
ForEach( $BGJob in $script:BackgroundDownloadJobs ) {
Switch( $BGJob.Type) {
1 {
$BGJob.Job.ChildJobs | Stop-Job -PassThru
$BGJob.Job | Stop-Job -PassThru | Remove-Job -Force -ErrorAction SilentlyContinue
}
2 {
Stop-Process -Id $BGJob.job.id -Force -ErrorAction SilentlyContinue
Start-Sleep -Seconds 5
Remove-Item -LiteralPath $BGJob.stdOutTempFile, $BGJob.stdErrTempFile -Force -ErrorAction Ignore
}
3 {
$BGJob.Job.ChildJobs | Stop-Job -PassThru
$BGJob.Job | Stop-Job -PassThru | Remove-Job -Force -ErrorAction SilentlyContinue
}
}
Write-Warning ('Stopped downloading {0} {1}' -f $BGJob.scheduleCode, $BGJob.title)
}
}
Function Add-BackgroundDownloadJob {
param(
$Type,
$FilePath,
$DownloadUrl,
$ArgumentList,
$File,
$Timestamp= $null,
$Title='',
$ScheduleCode=''
)
$JobsRunning= Get-BackgroundDownloadJobs
If ( $JobsRunning -ge $MaxDownloadJobs) {
Write-Host ('Maximum background download jobs reached ({0}), waiting for free slot - press Ctrl-C once to abort..' -f $JobsRunning)
While ( $JobsRunning -ge $MaxDownloadJobs) {
if ([system.console]::KeyAvailable) {
Start-Sleep 1
$key = [system.console]::readkey($true)
if (($key.modifiers -band [consolemodifiers]"control") -and ($key.key -eq "C")) {
Write-Host "TERMINATING" -ForegroundColor Red
Stop-BackgroundDownloadJobs
Exit -1
}
}
$JobsRunning= Get-BackgroundDownloadJobs
}
}
Switch( $Type) {
1 {
# Slidedeck
$job= Start-Job -ScriptBlock {
param( $url, $file)
$wc = New-Object System.Net.WebClient
$wc.Encoding = [System.Text.Encoding]::UTF8
$wc.DownloadFile( $url, $file)
} -ArgumentList $DownloadUrl, $FilePath
$stdOutTempFile = $null
$stdErrTempFile = $null
}
2 {
# Video
$TempFile= Join-Path ($env:TEMP) (New-Guid).Guid
$stdOutTempFile = '{0}-Out.log' -f $TempFile
$stdErrTempFile = '{0}-Err.log' -f $TempFile
$ProcessParam= @{
FilePath= $FilePath
ArgumentList= $ArgumentList
RedirectStandardError= $stdErrTempFile
RedirectStandardOutput= $stdOutTempFile
Wait= $false
Passthru= $true
NoNewWindow= $true
#WindowStyle= [System.Diagnostics.ProcessWindowStyle]::Normal
}
$job= Start-Process @ProcessParam
}
3 {
# Caption
$job= Start-Job -ScriptBlock {
param( $url, $file)
$wc = New-Object System.Net.WebClient
$wc.Encoding = [System.Text.Encoding]::UTF8
$wc.DownloadFile( $url, $file)
} -ArgumentList $DownloadUrl, $FilePath
$stdOutTempFile = $null
$stdErrTempFile = $null
}
}
$object= New-Object -TypeName PSObject -Property @{
Type= $Type
job= $job
file= $file
title= $Title
url= $DownloadUrl
scheduleCode= $ScheduleCode
timestamp= $timestamp
stdOutTempFile= $stdOutTempFile
stdErrTempFile= $stdErrTempFile
}
$script:BackgroundDownloadJobs+= $object
Show-BackgroundDownloadJobs
}
##########
# MAIN
##########
#Requires -Version 5.1
If( $psISE) {
Throw( 'Running from PowerShell ISE is not supported due to requirement to capture console input for proper termination of the script. Please run from a regular PowerShell session.')
}
If( $Proxy) {
$ProxyURL= $Proxy
}
Else {
$ProxyURL = Get-IEProxy
}
If ( $ProxyURL) {
Write-Host "Using proxy address $ProxyURL"
}
Else {
Write-Host "No proxy setting detected, using direct connection"
}
# Determine what event URLs to use.
# Use {0} for session code (eg BRK123), {1} for session id (guid)
Switch( $Event) {
{'MEC','MEC2022' -contains $_} {
$EventName= 'MEC2022'
$EventType='YT'
$EventYTUrl= 'https://www.youtube.com/playlist?list=PLxdTT6-7g--2POisC5XcDQxUXHhWsoZc9'
$EventLocale= 'en-us'
$CaptionExt= 'vtt'
}
{'Ignite','Ignite2024' -contains $_} {
$EventName= 'Ignite2024'
$EventType='API'
$EventAPIUrl= 'https://api-v2.ignite.microsoft.com'