Skip to main content

Podcaster, Broadcaster, Tech Pundit. Chief TWiT at the TWiT Netcast Network. The Tech Guy on the Premiere Networks.

twit.tv

leoville.com

techguylabs.com

 

Using Scripts To Download All Security Now Episodes

4 min read

Once again the TWiT Army rides to the rescue. A number of you have been looking for automated ways to download every single Security Now episode. I've received tweets and emails from several listeners who have provided their own scripts. I haven't tried them, but a cursory glance tells me they'll probably work and are non-lethal.

First for those of you using the B-word (bash) here's a script from @sethleedy. Pythonistas will want to try this script from@thepunkgeek.

Finally, thanks to to Gary Nevills for the following two Powershell scripts. Gary writes:

I thought I’d share my PowerShell scripts I use to download TWiT podcasts every week (and build up my library of shows). The download scripts work best in PowerShell 5.0 (Windows 10).

This one’s for downloading various shows in audio or video from TWiT’s website, you can also use it to view the download links (without downloading the shows), and highlight links that don’t redirect through PodTrac (see line 16). 

 

 

 

 

 

 


########################### ##### Begin Functions ##### ########################### Function find-url { param ( [Parameter(Mandatory=$true)] [string] $DownloadType ) the webpage for the appropriate url $fileUrl = "$DownloadType not available for episode $num" $results | foreach {Select-String -InputObject $_ -Pattern "download>$DownloadType</a>"} | foreach {$fileUrl = $_} $fileUrl = $fileUrl -replace '.*(http.*\.mp[3|4]).*', '$1' below to show each url that doesn't go through podtrac ($fileUrl -notlike "*podtrac*" -and $fileUrl -notlike "$DownloadType not available for episode $num") {Write-Host $fileUrl} below to display each url Write-Host $fileUrl the file if it is available and download was set to true if ($fileUrl -notlike "$DownloadType not available for episode $num" -and $downloadShow) { download-show -URL $fileUrl } } Function download-show { param ( [Parameter(Mandatory=$true)] [string] $URL ) the filename from the url and format it $fileName = [System.IO.Path]::GetFileName($fileUrl) $fileName = $fileName -replace "(.*[a-z]).*[0]($num).*(\.mp[3|4])", '$1$2$3' the full output filename $outputFile = "$outputDirectory\$DownloadType\$fileName" if the directory to save file in exists, create it if necessary $fileDirectory = [System.IO.Path]::GetDirectoryName($outputFile) if (!(Test-Path $fileDirectory)) {New-Item -Path $fileDirectory -ItemType Directory | Out-Null} if file has already been downloaded, skip if it has if (!(Test-Path $outputFile)) { and save the file Invoke-WebRequest $URL -OutFile $outputFile } } ############################################## ###############Begin Parameters############### ############################################## episode range $numOfEpisodes = 1 .. 20 1 show at a time $show = 'Security Now' #$show = 'Windows Weekly' #$show = 'TWiET' #$show = 'Coding 101' the output directoy $baseOutputDirectory = 'E:\user\documents\podcasts' which formats to download $audio = $true $videoSdSmall = $false $videoSdLarge = $false $videoHd = $false shows true/false - useful if just auditing download URL's $downloadShow = $true ############################################## ##############End Paraemeters################# ############################################## ################ Begin Script ################ ############################################## $outputDirectory = "$baseOutputDirectory\$show" foreach ($num in $numOfEpisodes) { switch -regex ($num) { {1..9 -contains $num} {$num = "{0:D1}" -f $num} default {$num = "{0:D2}" -f $num} } $searchResults = $null $downloadPage = $null switch ($show) { 'Security Now' {$showUrl = "https://twit.tv/shows/security-now/episodes/"+$num+"?autostart=false"} 'Windows Weekly' {$showUrl = "https://twit.tv/shows/windows-weekly/episodes/"+$num+"?autostart=false"} 'TWiET' {$showUrl = "https://twit.tv/shows/this-week-in-enterprise-tech/episodes/"+$num+"?autostart=false"} 'Coding 101' {$showUrl = "https://twit.tv/shows/coding-101/episodes/"+$num+"?autostart=false"} } Try { $downloadPage = Invoke-WebRequest $showUrl $results = $downloadPage.Content -split "`n" } Catch { Write-Error "Unable to find $showUrl" Continue } if ($audio) {find-url -DownloadType "Audio"} if ($videoSdSmall) {find-url -DownloadType "SD Video Small"} if ($videoSdLarge) {find-url -DownloadType "SD Video Large"} if ($videoHd) {find-url -DownloadType "HD Video"} }

And this one's for getting all of the files from grc.com for security now:

Download A Range of Security Now episodes from grc.com

$numOfEpisodes = 554 .. 557
$outputDirectory = 'E:\user\documents\podcasts\security now\'


foreach ($num in $numOfEpisodes) {

$num3Digit = "{0:D3}" -f $num
$num4Digit = "{0:D4}" -f $num
Write-Host $num3Digit
-Host $num4Digit

$url = "http://www.podtrac.com/pts/redirect.mp3/twit.cachefly.net/audio/sn/sn$num4Digit/sn$num4Digit.mp3"
$outputFile = "$outputDirectory\audio\sn$num4Digit.mp3"
try {if (!(Test-Path $outputFile)) {Invoke-WebRequest $url -OutFile $outputFile}}
catch {Write-Host "$num3Digit not available in audio"}

$url = "https://www.grc.com/sn/sn-$num3Digit-notes.pdf"
$outputFile = "$outputDirectory\show notes\sn$num3Digit.pdf"
try {if (!(Test-Path $outputFile)) {Invoke-WebRequest $url -OutFile $outputFile}}
catch {Write-Host "$num3Digit not available in show notes"}

$url = "https://www.grc.com/sn/sn-$num3Digit.pdf"
$outputFile = "$outputDirectory\transcripts\pdf\sn$num3Digit.pdf"
try {if (!(Test-Path $outputFile)) {Invoke-WebRequest $url -OutFile $outputFile}}
catch {Write-Host "$num3Digit not available in pdf transcript"}

$url = "https://www.grc.com/sn/sn-$num3Digit.txt"
$outputFile = "$outputDirectory\transcripts\text\sn$num3Digit.txt"
try {if (!(Test-Path $outputFile)) {Invoke-WebRequest $url -OutFile $outputFile}}
catch {Write-Host "$num3Digit not available in txt transcript"}

}

Thanks, Gary! I haven't tried these, but I want to put them here so you Powershell gurus can access them and use them for yourselves. 

 

The Joy of Rift

Editor Anthony Nielsen tries the Oculus Rift while a visiting team from CUNY, Annie Berman and Tim Whitney, look on.

 

Talk Less, Blog More

I don't need another iPad. No one does. That's probably why sales are plummeting. But I bought the new 9.7 inch iPad Pro anyway. Hey, it's my job. I also have a secret agenda. I'm betting it will be the perfect blogging platform.

Two new features intrigued me.

It's the first iPad to have a real camera. The same excellent camera, in fact, that comes in the iPhone 6s. That means I'll be able to shoot stills and videos right from my blogging tool. I've tried that with the iPhone 6s plus and the Galaxy S7, but typing on those things is impossible for those of us with sausage finger syndrome (SFS).

And that's where the new iPad's itty bitty keyboard comes in. Now I can actually type my posts. It's too early to tell whether it's workable for SFS sufferers, but I'm using it right now and it's not too bad. If I were an artist like Serenity Caldwell I'd even be able to sketch a little. I had hoped to do the same thing with Google's Pixel C and the skinny Macbook before that, but neither quite suited.

With the Smart case the new iPad still is small enough to fit into my man bag, and light enough that I won't mind carrying it all the time. And that means I'll have the means to mash those keys wherever I am.

I can use the tiny Shure Motiv microphone to record high quality audio right into the beast. Which means I can audio blog, too. And I find it much easier to talk than type.

As the good book says, Alexander Hamilton wrote "like he was running out of time." And, as a result, we know more of his thoughts and life than any of the other founding fathers. Hearing that has inspired me to talk less, blog more. Or maybe do both and in a couple of hundred years someone will write a hit musical about me. Maybe I should challenge Dvorak to a duel? Nah. I am not throwing away my shot!

 

Hour of the Wolf Blogging

The moon and Jupiter are out tonight. (Shot with the Galaxy S7 - talk about low light!) Yes, that little dot to the right of the moon is Jupiter!

I am up every night at this time, around 4am. Jung called it the hour of the wolf. Either I'm going polyphasic or just getting old. But... I'm thinking if I published one blog post every night at this time instead of scanning Twitter and Instagram I might actually get something accomplished. So here's my first Hour of the Wolf post. 

  • Tip of the night: transfer.sh - a free command line based file transfer service. Written in Golang by the Dutch Coders. I'm working on getting a self-hosted version of this running on my doorstop server (more about that later). How handy!
  • 24 years ago on this night my first child was born. Happy birthday, Abby. I love you so much.
  •  - how heartbreaking. What's wrong with this world?

Good night.

 <iframe width="100%" height="166" scrolling="no" frameborder="no" src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/254263561&amp;color=ff5500&amp;auto_play=false&amp;hide_related=false&amp;show_comments=true&amp;show_user=true&amp;show_reposts=false"></iframe>

 

Doin The TWiT

One of the original TWiT themes from 10 years ago. I use this as my ringtone, but I can't remember who did the remix - can you?

TWiT Chat Room to the rescue. It was Ashley Witt, the Hollywood composer who did many themes for us including The Tech Guy theme. You'll find more of his work on Last.fm

 
 

Mendocino morning

Shot on the Galaxy S7 with an Instagram filter. 

 
 
 
 
 

Fastmail Auto-Configure Chooses Wrong Port for Thunderbird

2 min read

I thought I'd stick this discovery here in the hopes that others would find it useful. I love Mozilla's Thunderbird email client. It does everything I want it to, including PGP/GPG via Enigmail, Calendering via Lightning, and Google contacts and Tasks via the Provider for Google Calendar extension. 

But I just couldn't get it to get all my folders to sync on Mint GNU/Linux. Only the top level INBOX would download. 

Turns out a "smart" engineer at Fastmail changed the auto-configure IMAP SSL port from 993 to 992 to get Thunderbird to work better with some non-compliant email programs, including Apple's Mail. I had tried every possible solution including subscribing to specific folders and renaming the root folder until I found this post:

Alternate Namespace in IMAP

It's a clever hack that makes Fastmail work better for Apple, Windows, and Blackberry users but it confuses the heck out of compliant client users who allow Fastmail to auto-configure settings.

Changing SSL port 992 to 993 fixes the problem.

It's okay to use the autoconfig - it works fine otherwise - but if you're not getting all your mail try switching ports. 993 is standard, 992 is a hack for some non-compliant clients. 

 

 

 
 

TWiT Meetup

TWiT Fan Meetup with Leo, Lisa, Dick DeBartolo, Paul Thurrott, and Mary Jo Foley

Location: Rattle 'n' Hum, 14 E 33rd St, New York City

Time:

Ends:

Leo Laporte, Dick DeBartolo, Mary Jo Foley, and Paul Thurrott will all be there. We hope you will, too. (Rattle 'n' Hum is a bar but it serves great pub food so under 21s are welcome. Space is limited so come early!)

http://www.rattlenhumbarnyc.com/

 

Just set up an automatic connection between this blog and my Telegram bot (https://telegram.me/ChiefTWiTBot).

 

Monkey See, Monkey Do

I feel like such an idiot. Today on MacBreak Weekly I tried to illustrate a problem with the Samsung Galaxy Note 5 discovered by Phil Nickinson at Android Central. I inserted the stylus wrong end up for what I thought was a safe distance. It wasn't.

Video of the incident: https://www.youtube.com/watch?v=dQB0TFcdFIs

Phil says

DO NOT PUT IT IN BUTT-FIRST. IT WILL GET STUCK.

I should have listened. It's well and truly stuck. Next stop: The ATT store. 

UPDATE: ATT says they won't replace it because it's accidental damage. However, they did say I could send it back to Samsung. Or that if I restored it to original condition I could return it (I've only had it a week). I pulled a tiny bit on the pen and was able to ease it out, but the sensor no longer works. Not the end of the world - I can launch the S-Note app by hand - but somewhat annoying.