2 Comments

In the new Azure Portal you create all your resources in Resource Groups, there is also as part of the Azure SDK's a module called AzureResourceManager  by default the module loaded for the Azure SDK is AzureServiceManagement. A blurb from one of the Azure documentation page reads

"The Azure and Azure Resource Manager modules are not designed to be used in the same Windows PowerShell session. To make it easy to switch between them, we have added a new cmdlet, Switch-AzureMode, to the Azure Profile module."

Azure Resource Manager Commands

To get a list of all commands that are available for the AzureResourceManager module you can run the command

Get-Command -Module AzureResourceManager | Get-Help | Format-Table Name, Synopsis

this will return

Name

Synopsis

----

--------

Add-AzureAccount

Adds the Azure account to Windows PowerShell

Add-AzureEnvironment

Creates an Azure environment

Clear-AzureProfile

Clears an Azure profile

Disable-AzureSqlDatabaseDirectAccess

Disables the option to directly access to an Azure Sql database (without auditing)

Disable-AzureSqlDatabaseServerDirectAccess

Disables direct access to all Azure Sql databases that use the audit policy of a Sql databa...

Enable-AzureSqlDatabaseDirectAccess

Enables the option to directly access to an Azure Sql database (with auditing)

Enable-AzureSqlDatabaseServerDirectAccess

Enables direct access to all Azure Sql databases that use the audit policy of a Sql databas...

Get-AzureAccount

Gets Azure accounts that are available to Azure PowerShell.

Get-AzureADGroup

Filters active directory groups.

Get-AzureADGroupMember

Get a group members.

Get-AzureADServicePrincipal

Filters active directory service principals.

Get-AzureADUser

Filters active directory users.

Get-AzureBatchAccount

 

Get-AzureBatchAccountKeys

 

Get-AzureDataFactory

Gets information about Data Factory.

Get-AzureDataFactoryGateway

Gets information about logical gateways in Data Factory.

Get-AzureDataFactoryHub

Gets information about hubs in Data Factory.

Get-AzureDataFactoryLinkedService

Gets information about linked services in Data Factory.

Get-AzureDataFactoryPipeline

Gets information about pipelines in Data Factory.

Get-AzureDataFactoryRun

Gets runs for a data slice of a table in Data Factory.

Get-AzureDataFactorySlice

Gets data slices for a table in Data Factory.

Get-AzureDataFactoryTable

Gets information about tables in Data Factory.

Get-AzureEnvironment

Gets Azure environments

Get-AzureLocation

Gets the resource types and the Azure data center locations that support them.

Get-AzurePublishSettingsFile

Downloads the publish settings file for an Azure subscription.

Get-AzureRedisCache

Gets details about a single cache or all caches in the specified resource group or all cach...

Get-AzureRedisCacheKey

Gets the accesskeys for the specified redis cache.

Get-AzureResource

Gets Azure resources

Get-AzureResourceGroup

Gets Azure resource groups

Get-AzureResourceGroupDeployment

Gets the deployments in a resource group.

Get-AzureResourceGroupGalleryTemplate

Gets resource group templates in the gallery

Get-AzureResourceGroupLog

Gets the deployment log for a resource group

Get-AzureRoleAssignment

Filters role assignments.

Get-AzureRoleDefinition

Filters role definitions.

Get-AzureSqlDatabaseAuditingPolicy

Gets an Azure Sql database's auditing policy.

Get-AzureSqlDatabaseServerAuditingPolicy

Gets an Azure Sql server's auditing policy.

Get-AzureSubscription

Gets Azure subscriptions in Azure account.

Get-AzureTag

Gets predefined Azure tags

Import-AzurePublishSettingsFile

Imports a publish settings file that lets you manage your Azure accounts in Windows PowerSh...

New-AzureBatchAccount

 

New-AzureBatchAccountKey

 

New-AzureDataFactory

Creates a data factory.

New-AzureDataFactoryEncryptValue

Encrypts sensitive data.

New-AzureDataFactoryGateway

Creates a gateway for Data Factory.

New-AzureDataFactoryGatewayKey

Creates a gateway key for Data Factory.

New-AzureDataFactoryHub

Creates a hub for Data Factory.

New-AzureDataFactoryLinkedService

Links a data store or a cloud service to Data Factory.

New-AzureDataFactoryPipeline

Creates a pipeline in Data Factory.

New-AzureDataFactoryTable

Creates a table in Data Factory.

New-AzureRedisCache

Creates a new redis cache.

New-AzureRedisCacheKey

Regenerates the access key of a redis cache.

New-AzureResource

Creates a new resource in a resource group

New-AzureResourceGroup

Creates an Azure resource group and its resources

New-AzureResourceGroupDeployment

Add an Azure deployment to a resource group.

New-AzureRoleAssignment

Create a role assignment to some principals at a given scope.

New-AzureTag

Creates a predefined Azure tag or adds values to an existing tag

Remove-AzureAccount

Deletes an Azure account from Windows PowerShell.

Remove-AzureBatchAccount

 

Remove-AzureDataFactory

Removes a data factory.

Remove-AzureDataFactoryGateway

Removes a gateway from Data Factory.

Remove-AzureDataFactoryHub

Removes a hub from Data Factory.

Remove-AzureDataFactoryLinkedService

Removes a linked service from Data Factory.

Remove-AzureDataFactoryPipeline

Removes a pipeline from Data Factory.

Remove-AzureDataFactoryTable

Removes a table from Data Factory.

Remove-AzureEnvironment

Deletes an Azure environment from Windows PowerShell

Remove-AzureRedisCache

Remove redis cache if exists.

Remove-AzureResource

Deletes a resource

Remove-AzureResourceGroup

Deletes a resource group.

Remove-AzureRoleAssignment

Removes a role assignment.

Remove-AzureSqlDatabaseAuditing

Disables an Azure Sql database's auditing.

Remove-AzureSqlDatabaseServerAuditing

Disables auditing of all the databases that rely on the auditing policy of the given databa...

Remove-AzureSubscription

Deletes an Azure subscription from Windows PowerShell.

Remove-AzureTag

Deletes predefined Azure tags or values

Resume-AzureDataFactoryPipeline

Resumes a suspended pipeline in Data Factory.

Save-AzureDataFactoryLog

Downloads log files from HDInsight processing.

Save-AzureResourceGroupGalleryTemplate

Saves a gallery template to a JSON file

Select-AzureSubscription

Changes the current and default Azure subscriptions

Set-AzureBatchAccount

 

Set-AzureDataFactoryGateway

Sets the description for a gateway in Data Factory.

Set-AzureDataFactoryPipelineActivePeriod

Configures the active period for data slices.

Set-AzureDataFactorySliceStatus

Sets the status of slices for a table in Data Factory.

Set-AzureEnvironment

Changes the properties of an Azure environment

Set-AzureRedisCache

Set redis cache updatable parameters.

Set-AzureResource

Changes the properties of an Azure resource.

Set-AzureResourceGroup

Changes the properties of a resource group

Set-AzureSqlDatabaseAuditingPolicy

Sets an Azure Sql database's auditing policy.

Set-AzureSqlDatabaseServerAuditingPolicy

Sets an Azure Sql database server's auditing policy.

Set-AzureSubscription

Creates or changes an Azure subscription

Stop-AzureResourceGroupDeployment

Cancels a resource group deployment

Suspend-AzureDataFactoryPipeline

Suspends a pipeline in Data Factory.

Switch-AzureMode

Switches between the Azure and Azure Resource Manager modules

Test-AzureResourceGroupTemplate

Detects errors in a resource group template or template parameters

Use-AzureSqlDatabaseServerAuditingPolicy

Marks an Azure Sql database as using its server's auditing policy.

Step 0

As a step 0 lets open up everything we need.

Azure SDK

So to get started you need to install the Azure SDK which you can get from the SDK downloads page. I am using Azure SDK 2.5 version for this post.

PowerShell ISE

Open the PowerShell ISE using Win + R and then %WINDIR%\system32\WindowsPowerShell\v1.0\powershell_ise.exe. You can also use the standard PowerShell window if you want.

Azure Management Portal

Open and sign in to the Azure Management Portal (https://manage.windowsazure.com/)

Azure Portal

Open and sign in to the Azure Portal (http://portal.azure.com/)

Step 1

Let's start by getting the Azure Management Portal pieces out the way. In the Azure Management Portal we will just be creating a new AAD user that we can use to automatically login through PowerShell. If you want to use a MSA just leave the credentials bit off in Step 3 and you will receive a prompt for credentials at which time you can use MSA or AAD credentials and you can now move to step 2. If you want to create the new user follow my other post Creating a new Azure Active Directory User to create a user for this demo.

Step 2

In the PowerShell ISE we will kick off by switching the Azure SDK to use the resource manager module.

Switch-AzureMode -Name AzureResourceManager

Step 3

After we have switched to the AzureResourceManager module we are able to use the commands that are part of it. Let's start off by adding our Azure Account we just created using the snippet below (I keep my username and password in txt files and reference from multiple sample scripts for ease of use but you can place them straight in the script if you wanted

[string]$currentUsername = Get-Content "Z:\_PowerShell\Azure\currentUser.txt"
[string]$currentPassword = Get-Content "Z:\_PowerShell\Azure\currentPass.txt"
$secpasswd = ConvertTo-SecureString $currentPassword -AsPlainText -Force
$mycreds = New-Object System.Management.Automation.PSCredential ($currentUsername, $secpasswd)
Add-AzureAccount -credential $mycreds

 

This would then return the account added

image

At this point if you ran Get-AzureAccount it will show you this account and any others you have previously added.

image

Step 4

Next we'll explore some meta data for Step 5.

Getting a Resource Group Template

We can run the command

Get-AzureResourceGroupGalleryTemplate

and it will return a list of all the current resources group templates that we can create. For the list at the time of writing this post you can refer to one of my GitHub Gists at https://binary-stuff.com/gist/ea0884f5ba00c62a83e4. We are going to be using one of the templates found around line 2220 which is the website and sql database.

image 

Get a list of Azure Resource Locations

When creating Azure Resources you need to specify were those resources are located geographically. Now you could guess or Bing what the locations are or you could use the handy command

Get-AzureLocation

which will return you a list of every location that every resource is available in. This list as with the one above can be found as one of my GitHub Gists at https://binary-stuff.com/gist/87aad8bfbbbcd7421b83. From the long list of locations we are just going to use West Europe for all our resources

Step 5

At this point we have all the info we need to create our resources using the resource manager in powershell so we'll use the snippet

$ResourceManagerTest = "ResourceManagerTest"
$AzureDataCenterLocation = "West Europe"
$administratorLoginPassword = ConvertTo-SecureString "$($ResourceManagerTest)DbP@ssw0rd" -AsPlainText -Force
New-AzureResourceGroup -Name "$($ResourceManagerTest)" `
-Location "$AzureDataCenterLocation" `
-GalleryTemplateIdentity Microsoft.WebSiteSQLDatabase.0.2.2-preview `
-siteName "$($ResourceManagerTest)Site" `
-hostingPlanName "$($ResourceManagerTest)Plan" `
-siteLocation "$AzureDataCenterLocation" `
-serverName "$($ResourceManagerTest.ToLowerInvariant())dbserver" `
-serverLocation "$AzureDataCenterLocation" `
-administratorLogin "$($ResourceManagerTest)DbLogin" `
-administratorLoginPassword $administratorLoginPassword `
-databaseName "$($ResourceManagerTest)DbName" `
-Verbose

This will then go off and create our website and database along with an Application Insights resource that we can use for our website when we deploy it. When the command finishes you should see an output similar to the one below which because we specified the -verbose flag tells us the status of each resource created

image

At this point we really are finished with what the subject of the blog post is but we'll continue on to explore what we have just created, how we would have had to create it using the Azure Portal and how we can remove the resource group using PowerShell.

Step 6

Open the Azure Portal. The first thing you should notice is that you already have a notification and when you open that it says that a deployment has succeeded.

image

Clicking on the success notification will open that resource group

image

From here you can use the resources 100% as if you created them in the portal.

Step 7

Before we see how we would have had to create those resources manually let's remove the resource group from our subscription. To do this we run the simple command below keeping in mind that the $ResourceManagerTest variable should still be set from the previous command in step 5

Remove-AzureResourceGroup -Name $ResourceManagerTest -Force -Verbose

this will then proceed to remove the resource group, again because of the -verbose flag we don't get as much info but rather just that it's deleting the resource group and then comes back when it's done.

image

Step 8

The last thing that I'll show on this post is how we would of have to do this manually (or at least where to find template). Back in the Azure Portal click new in the bottom left corner and then click on Everything

image

Next click on the Web category/section and then you'll see the Website + SQL option which is what we created

image

clicking on that option will give you a little info about the template and from here you'd just click on create

image

From here you will configure the Resource Group Name, all the settings for your Website resource and SQL resource and then click create.

image

When that process completes you will be in the sample place as that small PowerShell script gets you too Smile

Thoughts and comments

If you have any thoughts or comments about any of the pieces of this post please do share below.

0 Comments

I use to make a lot of TFS customizations and had to apply the template changes to multiple team projects which took a bit of time. Depending on the method you use it could be a quick or loooong process Smile. When I first started doing customizations I used the TFS Power Tools to upload changes which is a lot of effort because you are uploading one work item definition at a time into one team project.

Using Command Line

After a while I started using command line (witadmin importwitd), this was slightly faster but I found myself keeping a list commands in a txt file and then searching for the one I need when needed and run it.

A Basic PowerShell Script

I follow Martin Hinshelwood on various social media and one day he posted a blog post titled Upgrading to Visual Studio Scrum 3.0 process template in TFS 2013, although I had been at this point playing a lot with upgrading from TFS 2012 to TFS 2013 there was one piece of magic in that post that changed the way I applied process template changes up until today. It was a script that simple looped through the work item definitions in a set folder and imported them into TFS

Param(
[string] $CollectionUrlParam = $(Read-Host -prompt "Collection (enter to pick):"),
[string] $TeamProjectName = $(Read-Host -prompt "Team Project:"),
[string] $ProcessTemplateRoot = $(Read-Host -prompt "Process Template Folder:")
)

$TeamProjectName = "teamswithareas"
$ProcessTemplateRoot = "C:\Users\mrhinsh\Desktop\TfsProcessTemplates\Microsoft Visual Studio Scrum 3.0 - Preview"
$CollectionUrl = "http://kraken:8080/tfs/tfs01"

$TFSConfig = "${env:ProgramFiles}\Microsoft Team Foundation Server 11.0\Tools\TFSConfig.exe"
$WitAdmin = "${env:ProgramFiles(x86)}\Microsoft Visual Studio 12.0\Common7\IDE\witadmin.exe"

witds = Get-ChildItem "$ProcessTemplateRoot\WorkItem TrackingType\Definitions"

foreach ($witd in $witds)
{
Write-Host "Importing $witd"
& $WitAdmin importwitd /collection:$CollectionUrl /p:$TeamProjectName /f:$($witd.FullName)
}
$WitAdmin importcategories /collection:$CollectionUrl /p:$TeamProjectName /f:"$ProcessTemplateRoot\WorkItem Tracking\Categories.xml"
$WitAdmin importprocessconfig /collection:$CollectionUrl /p:$TeamProjectName /f:"$ProcessTemplateRoot\WorkItem Tracking\Process\ProcessConfiguration.xml"

Small Script Evolution

This worked for a while but I still had keep a couple of PowerShell files for the different projects I want to import the process templates into. I ended up adding over the next while adding a couple of additions to the script like publishing new global lists

#if there is a file with the name GlobalLists-ForImport.xml import it as Global List info for the current collection
if (Test-Path "$ProcessTemplateRoot\GlobalLists-ForImport.xml")
{
Write-Host "Importing GlobalLists-ForImport.xml"
& $WitAdmin importgloballist /collection:$CollectionUrl /f:"$ProcessTemplateRoot\GlobalLists-ForImport.xml"
}

and imported linked types

#import each Link Type for the $CollectionName
foreach($witd_LinkType in $witd_LinkTypes)
{
Write-Host "Importing $($witd_LinkType.Name)"
& $WitAdmin importlinktype /collection:$CollectionUrl /f:$($witd_LinkType.FullName)
}

ALM Rangers - vsarUpgradeGuide & vsarSAFe

vsarUpgradeGuide

The first project I joined after joining the ALM Rangers was the TFS Upgrade Guide. The last part of my contributions for the upgrade guide was a PowerShell script that could help you easily upgrade your process templates (or at least publish them) after you have made the changes required to make them compatible with TFS 2013. And for some reason it wasn't until then that I made the script target multiple team projects in the same collection.

vsarSAFe

The latest small modifications that were made to the script were for the project vsarSAFe which looks at how to modify your process template to make them SAFe aware. If you aren't familiar with SAFe it stands for Scaled Agile Framework. As I'm writing this we are showing up as Delayed on the Flight Plan but will be landing soon Open-mouthed smile

 image_thumb[3]

Most of the changes included here were just around adding comments and cleaning the script up a bit to make it easier to read.

So what does the script look like?

The final script (as it is now) looks like below

# Copyright © Microsoft Corporation.  All Rights Reserved.
# This code released under the terms of the
# Microsoft Public License (MS-PL, http://opensource.org/licenses/ms-pl.html.)
#
#config
$server = "MyTfsServer"
$port = 8080
$virtualDirectory = "tfs"
$CollectionName = "DefaultCollection"
$TeamProjectNames = @("Team Project 1", "Team Project 2", "Team Project 7", "Sample Scrum Project 1")
$ProcessTemplateRoot = "C:\templates\Microsoft Visual Studio Scrum 2013.3"

$CollectionUrl = "http://$($server)$(if ($port -ne 80) { ":$port" })$(if (![string]::IsNullOrEmpty($virtualDirectory)) { "/$virtualDirectory" })/$($CollectionName)"
$API_Version = "12.0"

#----------------------------
# don't edit below this line
#----------------------------

#get a reference to the witadmin executable path for the current api version
$WitAdmin = "${env:ProgramFiles(x86)}\Microsoft Visual Studio $API_Version\Common7\IDE\witadmin.exe"

#if there is a file with the name GlobalLists-ForImport.xml import it as Global List info for the current collection
if (Test-Path "$ProcessTemplateRoot\GlobalLists-ForImport.xml")
{
Write-Host "Importing GlobalLists-ForImport.xml"
& $WitAdmin importgloballist /collection:$CollectionUrl /f:"$ProcessTemplateRoot\GlobalLists-ForImport.xml"
}

#get a reference to all work item type definitions
$wit_TypeDefinitions = Get-ChildItem "$ProcessTemplateRoot\WorkItem Tracking\TypeDefinitions\*.*" -include "*.xml"

#get a reference to all work item link types
$witd_LinkTypes = Get-ChildItem "$ProcessTemplateRoot\WorkItem Tracking\LinkTypes\*.*" -include "*.xml"

#import each Link Type for the $CollectionName
foreach($witd_LinkType in $witd_LinkTypes)
{
Write-Host "Importing $($witd_LinkType.Name)"
& $WitAdmin importlinktype /collection:$CollectionUrl /f:$($witd_LinkType.FullName)
}

foreach ($TeamProjectName in $TeamProjectNames)
{
Write-Host "Upgrading $TeamProjectName."

#import each Type Definition for the $TeamProjectName
foreach($wit_TypeDefinition in $wit_TypeDefinitions)
{
Write-Host "Importing $($wit_TypeDefinition.Name)"
& $WitAdmin importwitd /collection:$CollectionUrl /p:$TeamProjectName /f:$($wit_TypeDefinition.FullName)
}

#import work item categories for the $TeamProjectName
& $WitAdmin importcategories /collection:$CollectionUrl /p:$TeamProjectName /f:"$ProcessTemplateRoot\WorkItem Tracking\Categories.xml"

#import work item process configuration for the $TeamProjectName
& $WitAdmin importprocessconfig /collection:$CollectionUrl /p:$TeamProjectName /f:"$ProcessTemplateRoot\WorkItem Tracking\Process\ProcessConfiguration.xml"
}
Write-Host "Done upgrading team projects"

This script now targets unlimited team projects in 1 team project collection, updates the categories, configuration, global lists and link types. You can grab the script off GitHub as well under my Gists (upgrade-tfs-2013-process-templates.ps1).

This takes care of all the things I need when making process template changes as I now make what ever changes I need run the script and check my changes in the browser. It doesn't get much easier than this but if you have a easier way do let me know Smile.

0 Comments

I was looking for a PowerShell script that would remove all media tags (post coming) from a folder of music that I have and along the way I came across a blog post called PowerShell - Automatically organizing your mp3-collection. Although this wasn't exactly what I was looking for right now I gave it a try and then thought to share it after I made a couple of changes to make it work on my machine Smile with tongue out and categorize a bit more . You can grab the script off GitHub Gist organise-music.ps1.

Basically the script will take any structure of music and organize it into the structure below

%root%

%root% / %Album Artist%

%root%/ %Album Artist% / %Album Year%

%root%/ %Album Artist% / %Album Year%/ %Album Name%

You also don't need to have all your files in a single folder for this to work, it will recursively find matching file extensions ("*.m4a", "*.m4b", "*.mp3", "*.mp4", "*.wma", "*.flc") and then work out where each file should be placed, creating directories where needed and cleaning up empty directories and extra files when finished Smile

1 Comments

I watch and download a lot of videos from Channel 9 and because I’m a developer and always looking for ways to speed up anything I do I searched for a PowerShell script. I eventually found one (can’t remember where) and immediately set it up to download This Week on Channel 9 and Ping Show. Over time I have added many other shows, events and series. The initial script I was found with some modifications was

$url="http://channel9.msdn.com/Shows/This+Week+On+Channel+9/feed/mp4high"
$rss=invoke-webrequest -uri $url
$destination="Z:\Media\Videos\ch9\This Week on ch9\"
[xml]$rss.Content|foreach{
  $_.SelectNodes("
rss/channel/item/enclosure")
}|foreach{
    "
Checking $($_.url.split("/")[-1]), we will skip it if it already exists in $($destination)"
  if(!(test-path ($destination + $_.url.split("
/")[-1]))){
    "
Downloading: " + $_.url
    start-bitstransfer $_.url $destination
  }
}

Because of the amount of shows growing that I was interested in I added another piece to this. On the root level of all my Channel 9 videos I added a PowerShell file that would run all the other PowerShell files that looked like

cd Z:\Media\Videos\ch9

$psScripts = Get-ChildItem -Recurse -Filter "*.ps1" | ForEach-Object -Process { if($_.Name -ne "Download All.ps1"){ Write-Text "$($_.Name)"; . $_.FullName; } }

Write-Output "Done"

This was working great as I wouldn’t need to go and run multiple files in order to update all my videos. Recently because of all the new stuff that was released because of the announcements  at #VS2013Launch I had to add about 5 new feeds to my collection. This took quite a while as I needed to create a new folder for the show, then get a copy of the first PowerShell script above and then alter the feed url and location to save the videos to. This lead to yet another script, the way I saw it I could either write a script that automates the create of the folder and script or I could do the longer but right thing and start “fresh”. The script I now run takes in an array of the url of the show, series, event or any other type of channel that has an RSS feed, basically everything after http://channel9.msdn.com/. The new and improved script looks like

cls

Write-Output "Starting"

$feedTypeNameList = $("Events/Build/2014",
"Shows/PingShow",
"Shows/This+Week+On+Channel+9",
"Series/Microsoft-Research-Luminaries",
"Shows/Windows-Azure-Friday",
"Blogs/One-Dev-Minute",
"Shows/Visual-Studio-Toolbox",
"Events/Visual-Studio/Launch-2013",
"Series/Application-Insights-for-Visual-Studio-Online",
"Series/Visual-Studio-Online",
"Series/Visual-Studio-Online-Monaco",
"Series/Visual-Studio-2012-Premium-and-Ultimate-Overview",
"Blogs/MadsKristensen",
"Blogs/C9Team",
"Series/PerfView-Tutorial",
"Blogs/IE",
"Shows/Edge")
$baseSaveLocation = "N:\s\ch9`$"

$mediaFormat = "high"
$mediaType = "mp4"
$fileExtension = "mp4"

$pathToRemoveInvalidFileNameCharsScript = "$baseSaveLocation\Remove-InvalidFileNameChars.ps1"

#------------Don't edit below here----------------#

. "$pathToRemoveInvalidFileNameCharsScript"
foreach($feedTypeName in $feedTypeNameList)
{
$channelType = $feedTypeName.Split("/")[0]
$feedUrl="http://channel9.msdn.com/$($feedTypeName.Trim("/"))/RSS/$($mediaType + $mediaFormat)"
Write-Output @"

Downloading Feed: $feedUrl

"@
$rss=invoke-webrequest -uri $feedUrl
$destination="$baseSaveLocation\$($feedTypeName.Replace("/","\").Trim("\"))\"

if (!(Test-Path $destination)) {
New-Item -ItemType directory -Path $destination
}

$videos = @()
[xml]$rss.Content|foreach{
$_.SelectNodes("rss/channel/item")|foreach{
[Array]$array = @($_.SelectSingleNode("enclosure").url,$_.SelectSingleNode("title").InnerText)
$videos += , $array
}
}
[Array]::Reverse($videos)
foreach($video in $videos){
$url = $video[0]
$title = $video[1]
if (![string]::IsNullOrEmpty("$url"))
{
$fileName = $($url.split("/")[-1])
$mp4fileName = $($fileName.Replace("." + $fileExtension,"") + "-" + (Remove-InvalidFileNameChars $title.Replace(" ","-")) + ".$fileExtension")
if ($mp4fileName.Contains("_"))
{
$pptxFileName = $mp4fileName.Remove($mp4fileName.LastIndexOf("_")) + ".pptx"
$pptxFileNameSaveAs = $mp4fileName.Remove($mp4fileName.LastIndexOf("_")) + "-" + (Remove-InvalidFileNameChars $title.Replace(" ","-")) + ".pptx"
}
else
{
$pptxFileName = $mp4fileName.Remove($mp4fileName.LastIndexOf(".")) + ".pptx"
$pptxFileNameSaveAs = $mp4fileName.Remove($mp4fileName.LastIndexOf(".")) + "-" + (Remove-InvalidFileNameChars $title.Replace(" ","-")) + ".pptx"
}

if (![string]::IsNullOrEmpty("$mp4fileName"))
{
"Checking $mp4fileName, we will skip it if it already exists in $($destination)"
#if we have the file from the previous script rename it or delete it
if(Test-Path ($destination + $fileName))
{
if(!(Test-Path ($destination + $mp4fileName)))
{
"Renaming: " + $fileName
Rename-Item $($destination + $fileName) $($destination + $mp4fileName)
}
else
{
"Deleting: " + $fileName
Remove-Item $($destination + $fileName)
}
}
else
{
#download media if it doesn'
t exists
if(!(Test-Path ($destination + $mp4fileName)))
{
$dest = $($destination + $mp4fileName)
"Downloading: '$url' to '$dest'"
Start-BitsTransfer $url $dest
}
}
}
#download pptx if it doesn't exists
if ($channelType -eq "Events") #only attempt to get pptx file for events
{
if (![string]::IsNullOrEmpty("$pptxFileName"))
{
"Checking $pptxFileName, we will skip it if it already exists in $($destination)"
$dest = $($destination + $pptxFileNameSaveAs)
if(!(Test-Path ($dest)))
{
$urlToDownload = $url.Remove($url.LastIndexOf("/")) + "/" + $pptxFileName
"Downloading: '
$urlToDownload' to '$dest'"
Start-BitsTransfer $urlToDownload $dest -ErrorAction SilentlyContinue
}
}
}
}
}
}

Write-Output "
Done"

This script caters for renaming of media files from the old just file name format to including the title in the file name. In order to add this extra script that I used for trimming funny characters from the file names. 

#source - http://gallery.technet.microsoft.com/scriptcenter/Remove-Invalid-Characters-39fa17b1

Function Remove-InvalidFileNameChars {
    <#
    .SYNOPSIS
    This is a PowerShell function to remove invalid characters from strings to be used as file names.

    .DESCRIPTION
    The function takes a string parameter called Name and returns a string that has been stripped of invalid file name characters, i.e. *, :, \, /.  The Name parameter will also receive input from the pipeline.

    .PARAMETER Name
    Specifies the file name to strip of invalid characters.

    .INPUTS
    Parameter Name accepts System.String objects from the pipeline.

    .OUTPUTS
    System.String.  Outpus a string object

    .EXAMPLE
    Remove-InvalidFileNameChars -Name "<This/name\is*an:illegal?filename>"
    PS C:\>Thisnameisanillegalfilename

    .NOTES
    It would be easiest to copy the function from the script file and place it in your profile.  However, you may also dot-source the script to load the function into PowerShell:
    i.e. PS C:\>. .\Remove-InvalidFileNameChars.ps1
    #>

    [CmdletBinding()]

    param([Parameter(Mandatory=$true,
        Position=0,
        ValueFromPipeline=$true,
        ValueFromPipelineByPropertyName=$true)]
        [String]$Name
    )

    return [RegEx]::Replace($Name, "[{0}]" -f ([RegEx]::Escape([String][System.IO.Path]::GetInvalidFileNameChars())), '')
}

Hope this helps others that download Channel 9 videos to watch offline as well Smile.

0 Comments

image_thumb2So I'm starting to go through some of the content on Microsoft Virtual Academy but I don't have time to do this when I'm by a good internet connection and at the same time don't have time when by the good internet connection to run though all the content I need to download it to watch in my spare time.

Enter PowerShell Smile

As with any scenario like this you turn to code, I use to create all these types of functionality in C# apps but have recently decided that it's too easy in C# and wanted to learn PowerShell better so now these are all PowerShell scripts.

The Code

2 files

Remove-InvalidFileNameChars.ps1

#source - http://gallery.technet.microsoft.com/scriptcenter/Remove-Invalid-Characters-39fa17b1
#changed slightly for MSVA script

Function Remove-InvalidFileNameChars {
<#
.SYNOPSIS
This is a PowerShell function to remove invalid characters from strings to be used as file names.

.DESCRIPTION
The function takes a string parameter called Name and returns a string that has been stripped of invalid file name characters, i.e. *, :, \, /. The Name parameter will also receive input from the pipeline.

.PARAMETER Name
Specifies the file name to strip of invalid characters.

.INPUTS
Parameter Name accepts System.String objects from the pipeline.

.OUTPUTS
System.String. Outpus a string object

.EXAMPLE
Remove-InvalidFileNameChars -Name "<This/name\is*an:illegal?filename>"
PS C:\>Thisnameisanillegalfilename

.NOTES
It would be easiest to copy the function from the script file and place it in your profile. However, you may also dot-source the script to load the function into PowerShell:
i.e. PS C:\>. .\Remove-InvalidFileNameChars.ps1
#>

[CmdletBinding()]

param([Parameter(Mandatory=$true,
Position=0,
ValueFromPipeline=$true,
ValueFromPipelineByPropertyName=$true)]
[String]$Name
)

$result = [RegEx]::Replace($Name, "[{0}]" -f ([RegEx]::Escape([String][System.IO.Path]::GetInvalidFileNameChars())), '')
$result = $result.Replace("Microsoft-Virtual-Academy", "")
while($result.Contains("--"))
{
$result = $result.Replace("--", "-")
}

return $result.Trim("-")
}

Download-Content.ps1

cls

Write-Output ""
Write-Output ""
Write-Output ""
Write-Output ""
Write-Output ""
Write-Output ""
Write-Output ""
Write-Output ""
Write-Output ""
Write-Output ""
Write-Output "Starting"

$certPages = $("mcsd-application-lifecycle-management","mcsd-windows-store-apps-certification","mcsd-web-apps-certification")
$baseSaveLocation = "Z:\Learning\Microsoft Virtual Academy"

$pathToRemoveInvalidFileNameCharsScript = "$baseSaveLocation\Remove-InvalidFileNameChars.ps1"

#------------Don't edit below here----------------#

. "$pathToRemoveInvalidFileNameCharsScript"
foreach($certPage in $certPages)
{
$certUrl="http://www.microsoft.com/learning/en-us/$certPage.aspx"
Write-Output "Downloading Page: $certUrl"
$pageHtml=invoke-webrequest -uri $certUrl
$destination="$baseSaveLocation\$(Remove-InvalidFileNameChars $certPage.Replace(" ","-"))"

if (!(Test-Path $destination)) {
New-Item -ItemType directory -Path $destination
}

$filteredLinks = $pageHtml.Links | Where-Object { $_.href.ToLower().StartsWith("http://www.microsoftvirtualacademy.com/training-courses/") }

foreach($link in $filteredLinks)
{
Write-Output "Downloading Page: $($link.href)"
$trainingPageHtml=invoke-webrequest -uri $link.href
[string]$trainingTitle = $trainingPageHtml.AllElements | Where-Object { $_.tagName.ToLower().StartsWith("title") } | Select-Object -ExpandProperty innerText {$_}
$trainingTitle = $trainingTitle.Trim()
$trainingDestination="$destination\$($link.innerText)-$(Remove-InvalidFileNameChars $trainingTitle.Replace(" ","-"))"

if (!(Test-Path $trainingDestination)) {
New-Item -ItemType directory -Path $trainingDestination
}

$idEducationTypePattern = [regex]"parent='(?<idEducationType>[^']*)'"
[int]$idEducationType = $idEducationTypePattern.Match($trainingPageHtml.Content).Groups["idEducationType"].Value
$jsonPostDataPattern = [regex]'showNextPanelEducationTypes((?<idEducationTypeSelected>[^"]*),(?<idEducationLevelSelected>[^"]*),(?<isApproved>[^"]*),(?<numControl>[^"]*),this)'
foreach($jsonPostData in $jsonPostDataPattern.Matches($trainingPageHtml.Content))
{

[int]$idEducationTypeSelected = $jsonPostData.Groups["idEducationTypeSelected"].Value.TrimStart("(")
[int]$idEducationLevelSelected = $jsonPostData.Groups["idEducationLevelSelected"].Value
[int]$numControl = $jsonPostData.Groups["numControl"].Value
$isApproved = $jsonPostData.Groups["isApproved"].Value
$jsonPostUri = "http://www.microsoftvirtualacademy.com/Studies/EducationDetails.aspx/ShowNextPanelEducationTypes"
$jsonPostBody = @{
idEducationTypeSelected = "$idEducationTypeSelected";
idEducationLevelSelected = $idEducationLevelSelected;
isSubscribe = $true;
selectedIsEnabled = "true";
numControl = $numControl;
idEducationType = "$idEducationType";
isApproved = "$isApproved";
culture="en-US"
}| ConvertTo-Json

$jsonInvokeResult = Invoke-RestMethod -Method Post -Uri $jsonPostUri -ContentType "application/json" -Body $jsonPostBody

$ctIDPattern = [regex]'openEmbeddedVideo((?<Id>.+));'
foreach($ctID in $ctIDPattern.Matches($jsonInvokeResult.d.HtmlResponseMaterials))
{
$Id = $ctID.Groups["Id"].Value.TrimStart("(").Split(',')[0]
$uriForTrainingVideo = "http://www.microsoftvirtualacademy.com/Content/ViewContent.aspx?et=$idEducationType&m=$idEducationTypeSelected&ct=$Id"
Write-Output "Downloading Page: $uriForTrainingVideo"

$trainingVideoPageHtml=invoke-webrequest -uri $uriForTrainingVideo

[string]$trainingVideoTitle = $trainingVideoPageHtml.AllElements | Where-Object { $_.tagName.ToLower().StartsWith("title") } | Select-Object -ExpandProperty innerText {$_}
$trainingVideoTitle = $trainingVideoTitle.Trim()
[string]$videoMp4Link = $trainingVideoPageHtml.Links | Where-Object {$_.href.EndsWith(".mp4") -and $_.outerText -eq "MP4"} | Select-Object -ExpandProperty href {$_}
$urlFileName = $($videoMp4Link.split("/")[-1])
$videoMp4LocalFullName = "$($trainingDestination +"\" + $(Remove-InvalidFileNameChars $trainingVideoTitle.Replace(" ","-"))).mp4"

if(!(Test-Path ($videoMp4LocalFullName)))
{
Write-Output "Downloading Video: $trainingVideoTitle to $videoMp4LocalFullName"
Start-BitsTransfer "$videoMp4Link" $videoMp4LocalFullName
}
}
}
}
}

Write-Output "Done"

Configuration

$baseSaveLocation - Set this to the location you want to save the videos to

$certPages - This is an array of certification page names that you want to download content from, i.e.: for MCSD: Application Lifecycle Management which has a url http://www.microsoft.com/learning/en-us/mcsd-application-lifecycle-management.aspx, the page name would be mcsd-application-lifecycle-management. Note that you will exclude the extension.

$pathToRemoveInvalidFileNameCharsScript - This can stay the same unless you have placed the first script in a difference location to the base location.

Next Steps

The next steps for this script will be to try get it to download the PowerPoint slides for each video, currently you need to be logged in to download slides so this might not be ready for a while Smile