Tag: PnP Provisioning

Provision Microsoft Teams with Azure Automation – Part II

Provision Microsoft Teams with Azure Automation – Part II

I am a big fan of Azure Automation and its helpful capabilities around provisioning and administrating around Office 365 and its components such as SharePoint, Office 365 Groups or Microsoft Teams. In this little series I want to show you how you can provision your Teams with Azure Automation in a quick and simple way but the same would be scalable to a much more complex solution depending on your needs.

In this second part of my little series I want to show you the way currently supported in production to first create a Group and then a Team based on that. Also we will have a look how we can further adjust that Team by execute some PnP Provisioning templates against it and its corresponding SharePoint site. Finally some adjustments to the Team (not possible with PnP Provisioning so far) will be done.

To remember what I started to explain in Part I. We have a modularized approach consisting of several runbooks where one needs to be called first.

  • Parent Runbook Module (calls:)
    • Create Team runbook
    • PnP Provision against site
    • Execute further PS manipulations …
All our Runbooks for creating Microsoft Teams (Part I & II of this series)

Our parent runbook you might remember from Part I but here once again and a bit more complete.

Our parent runbook to create a team from a group

After our input parameters we retrieve our assets (credentials, AppIDs, Urls and so on) from the current Automation Account (refer to another of my posts in case you need more information on Automation Account assets). Once we have them we can start to call our first runbook to create the Team (see next) which we already did in part I but there with another approach.

Our (sub)runbook to create an Office365 Group first and afterwards a team from it

Most code parts are the same as in part 1. We first connect to Microsoft Graph and get the access token. Then we create our Rest header. Also the block to receive the user id for the given login is the same. What’s new is the first While loop where we check if the desired alias is already in use. If so we manipulate it by attaching a “2”, next a “3” and so on. This pattern is only an example of course but gives us a bit more control in case we cannot handle something more comfortable in a potential UI.

As in this part we want to create a Group first it’s now time to create a GroupRequest body object. Additionally to an intro video on this approach by Paolo Pialorsi I already referenced in part 1. I also provide the alias here as we already managed that above and furthermore it is good practice to not provision Groups without any owners.

Next is to convert the request object to Json and execute a Rest call against Microsoft Graph. Having the Group and it’s ID as a result we can immediately go on with preparing the next request to create a Team from that Group (“teamify”). The request is a bit smaller than in part 1 but thats obvious because many parameters we already provided for the Group. The request follows the same pattern, converting request object to Json, execute Rest request and grabbing the ID from the result (which is not really necessary because Team and Group ID are in fact the same!)

After the new Team is created (back in our parent runbook!) we can retrieve a Group object and with it its site url. With a small waiting loop we check that the team and its site is really ready as provisioning might take some time. Having that we can call our further runbooks to go on provisioning with a PnP template and further PS cmdlets.

Next step is applying a PnP Provisioning template with that we want to create some fields, a content type and a list. The list will also get some DataRows so we have content for demonstration purposes. The runbook for that you find next:

Our (sub)runbook to apply PnP Provisioning template

First you find a function to download our existing provisioning template from an Azure Blob storage secured by a key.

PnP Provisioning template, stored in Azure Blob storage

Next we establish the authentication against SharePoint. You can either grab credentials from our AutomationAccount as a credential asset or use modern authentication with an app reg and certificate as described here.

After Connect-PnPOnline we construct our template-name, download it to local runbook runtime environment (C:\Users\… or C:\temp\… are our options) and then we simply apply our provisioning template.

Unfortunately I was still not able to output PnP Provsiioning debugging information in a runbook output so the following line of code is obsolete. Any hint would be greatly appreciated.

Set-PnPTraceLog	-On -Level Debug 

Next we come to our third runbook. Not everything can be solved with PnP Provisioning unfortunately (or not yet). So we decided in our scenarios to isolate further code steps in another runbook, typically called afterwards. In this scenario lets assume we want to create a new tab in our team and show the just created SharePoint list in a website tab. This is what the third (sub)runbook will do for us.

First we connect to Microsoft Graph again. Might not be necessary as we did in the first (sub)runbook but better than the connection got lost. As we need to add a new Tab to a Channel we first need to identify our Channel by retrieving the “General” one.

Afterwards we create a request url based on the IDs of our Team and the Channel we just identified. Build an object for a Website Tab, converting it to Json and executing a POST request against Microsoft Graph is all we need to do to achieve our result.

The custom ShrePoint list, provisioned with PnP Provisioning to a Teams’ SharePoint site
And the same list embedded in a Teams Website tab for instance

I hope this post illustrated a bit how you can create even more complex provisioning solutions with Azure Automation and PnP PowerShell. The same is applicable for SharePoint only or Office 365 Groups of course. In the create runbook you already saw here how to create Groups. Replace this by creating SharePoint sites is an obvious possibility.

Depending on some feedback next might be to point out a bit more the basics of Azure Automation or an architectural discussion when it might make more or less sense to use compared with/combined to alternatives such as Azure (Durable) Functions, Site Designs and so on. Looking forward to receive some questions on that.

Markus is a SharePoint architect and technical consultant with focus on latest technology stack in Office 365 and SharePoint Online development. He loves the new SharePoint Framework as well as some backend stuff around Azure Automation or Azure Functions and also has a passion for Microsoft Graph.
He works for Avanade and is based in Munich.
Although if partially inspired by his daily work opinions are always personal.
Provision Microsoft Teams with Azure Automation – Part I

Provision Microsoft Teams with Azure Automation – Part I

I am a big fan of Azure Automation and its helpful capabilities around provisioning and administrating around Office 365 and its components such as SharePoint, Office 365 Groups or Microsoft Teams. In this little series I want to show you how you can provision your Teams with Azure Automation in a quick and simple way but the same would be scalable to a much more complex solution depending on your needs.

Inspired by a video from Paolo Pialorsi I wanted to establish some base runbooks in Azure Automation for provisioning Microsoft Teams. In the past I did lots of them for Sharepoint Online and Office 365 Groups.

As always striving for the latest and greatest I wanted to omit the two-step approach that also Paolo is showing in his video: First create an Office 365 Group and then enhance this Group to a Team. And indeed there is at least a beta Api in Microsoft Graph for directly creating a Team, based on an existing template:
POST https://graph.microsoft.com/beta/teams

The pattern is quite simple and similar to any other POST request, you need to

  • Care for authentication by creating a token and providing it in your request header
  • Create a body to provide information for the object to be created
  • Execute a POST request by handing in the header and body

While the token creation is similar to others and shown in the video of Paolo as well lets have a detailed look at the request body:

{
  "template@odata.bind": "https://graph.microsoft.com/beta/teamsTemplates('standard')",
  "displayName": "My First Team",
  "description": "My First Team’s Description",
  "owners@odata.bind": [
    "https://graph.microsoft.com/beta/users/<UserGuid>"
  ]
}

The first thing I missed here was a “mailNickname” or alias. I checked and tried out but you cannot provide that this way, at least at the moment, you need to rely on what Microsoft creates from your requested displayName. While on Groups provisioning I clearly preferred to give the user a chance to enter own wishes.

The second thing I found out (and here I moved away from Microsoft’s example in the beta documentation, as that wasn’t working) that you need to provide a user Guid. Same in Groups by the way if you want to provide at least one owner from the start (what you should!). So a user Guid for sure is no good idea to request from a user directly so we need some additional lines of code but lets come to that right now:

At first we connect against Microsoft Graph. A simple way to do that is with PnP PowerShell of course. I assume you already registered an app for creating Groups as documented here and we already grabbed our id, secret, domain (see later in another runbook). After connecting we get our access token and put that in a header object.

Next we come to the problem with the user Guid. Here we assume our user request provides a login name. With a simple Graph request we get the corresponding user and it’s id which we store in a variable.

Next we can fill our requestBody from above. Not only with the user Guid but also with displayName, description and also our template. I won’t go into details here but there are several Microsoft templates available and you can even override properties of the available templates. One problem we have are the escape charaters ” which enclose the template name!

"template@odata.bind" = "https://graph.microsoft.com/beta/teamsTemplates('standard')"

As in the next step we convert the just created body object to Json this wouldn’t work if we wouldn’t pipe the Json creation to a regex method called Unescape from Microsoft’s standard regex class. I found this simple but effective tip in another blogpost.

ConvertTo-Json -InputObject $teamRequest | % { [System.Text.RegularExpressions.Regex]::Unescape($_) }

The final thing now should be to invoke a Rest request of type Post to the teams Url and providing our just created header and body. We do that and store the response in another variable.

We would expect now to have our most recent created team in a variable and could use it’s Id to request it and do further things with that. Unfortunately this is the next insufficiency in the beta version we currently use: It returns NOTHING.

So for reasons of completeness I added some “weak” lines of code to get the “latest” Team created with our requested displayName. For sure such a method wouldn’t be 100% reliable. But hopefully this gets fixed before this Api is supported for use in production.

Finally I would start to show my regular concept on provisioning with runbooks: I always use a modularized approach because I hate PS scripts with tons of code lines. So I always use one parent runbook to be called that calls further runbooks for significant steps in my provisioning process usch as:

  • Create my site/Group/Team (that runbook I showed above)
  • Provision my artefacts with PnP Provisioning
  • Post Provisioning stuff, modifications where I need additional code and what doesn’t work with PnP Templates
  • Have Shared or Helper runbooks to be called from different runbook modules …

So in our case here is a simple parent runbook that calls the just mentioned “Create” runbook, retrieves back the created Team’s Id and processes this further.

First we grab our assets such as Graph access credentials and store them in $global:variables so we have them available in all called (sub) runbooks as well in case we need to share variables …

That was it for now. Hopefully there will be some progress with this Api on Teams creation soon. I will observe it and potentially update this post by then.

In the next part I will show you the approach which is already available for productional use (v1.0 Graph Api) to first create a Group and then a Team out of it and further modify the just created Team. There you will see additional things to note when handling Provisioning with Azure Automation runbooks. So stay tuned.

Markus is a SharePoint architect and technical consultant with focus on latest technology stack in Office 365 and SharePoint Online development. He loves the new SharePoint Framework as well as some backend stuff around Azure Automation or Azure Functions and also has a passion for Microsoft Graph.
He works for Avanade and is based in Munich.
Although if partially inspired by his daily work opinions are always personal.
Modern SharePoint authentication in Azure Automation (Runbook) with PnP PowerShell

Modern SharePoint authentication in Azure Automation (Runbook) with PnP PowerShell

Modern authentication (not only) in SharePoint Online becomes more and more relevant as more and more organizations turn off LegacyAuthentication. In fact this means the classic Credential authentication with UserName and Password does not work anymore.

The issue is, when connecting with SharePoint Designer or PowerShell with classic credentials you will receive a “Cannot contact web site  or the web site does not support SharePoint Online credentials” error.

The setting that handles this is the following:

Connect-SPOService -Url "https://<Your-Tenant>-admin.sharepoint.com"

$tenantsettings = Get-SPOTenant

$tenantsettings.LegacyAuthProtocolsEnabled

If this setting is False you cannot login with classic credentials. To change this you could run

Set-SPOTenant -LegacyAuthProtocolsEnabled $true
But for security reasons your organization might think different.

For Microsoft.Online.SharePoint.PowerShell the story ends here for now as it does not work with modern authentication especially in an unattended mode such as Azure Automation runbooks.

Luckily the more popular PowerShell module in case of SharePoint Online is PnP-PowerShell. This module provides capabilities for an unattended authentication towards SharePoint Online. In this blogpost I will show you how to handle this inside an Azure Automation runbook.

According to Microsoft’s PnP documentation for an app only permission authentication you would need the following things to get this scenario up and running. I will show you each of them and point out the “Azure Automation specifics”.

 

Create a self-signed certificate

Here you can either use a script or (much simpler) the new New-PnPAzureCertificate cmdlet. For instance run the following:

$Password = "******"
$secPassword = ConvertTo-SecureString -String $Password -AsPlainText -Force
$cert = New-PnPAzureCertificate -Out "AzureAutomationSPOAccess.pfx" - `
 -ValidYears 10 `
 -CertificatePassword $secPassword
 -CommonName "AzureAutomationSPOAccess" `
 -Country "DE" `
 -State "Bavaria"

This would create a self-signed certificate with the needed values and, enforced by the “-out” parameter, write it to a .pfx file in the current directory.

After that simply run the following command and keep the window open. We would need the JSON result (shortened here) a in a minute.

$cert.KeyCredentials

{
    "customKeyIdentifier": "zUFQhchR6FJ0...",
    "keyId": "4d2fe8fc-0dbb-45a7-...",
    "type": "AsymmetricX509Cert",
    "usage": "Verify",
    "value":  "MIIDJDCCAgygAwIBAgIQV9qo..."
}

Register an Application

The next thing would be to register an app in your Office 365 Azure AD. Simply go to your Office 365 Admin Center and from there down below on the left side under “Admin Centers” to the “Azure Active Directory” admin center.

Here I encourage you now to use the new “App registrations (preview)” version and register an application.

RegisterApp1

Here only the “Name” is really relevant for us. Once you have that, you would need to give permissions to that app registration.

RegisterApp2_Permissions

Assuming you want to create lots of runbooks with several admin scenarios the selected permissions would be necessary. If you only have specific needs you might reduce it to a lower level. Pay attention to check “Application permissions” as we cannot handle a delegated scenario.

RegisterApp3_GrantPermissions

Anyway, any given SharePoint permission needs an administrator consent. Simply click the button but make sure you are a tenant administrator.

RegisterApp4_Granted

Then your permissions should be granted.

Next we would need to add our “KeyCredentials”. This “relates” our app registration to the recently created certificate. So switch back to our left open PowerShell window and copy the JSON output. Then switch to the app registration’s manifest and insert it there inside the [] .
At the moment of writing there was an issue with the “Preview” version of the App registration. If you receive the error “Failed to update application SPAccess. Error details: KeyValue cannot be null or empty” you can simply re-open your app registration in the classic mode, insert it there and save. Switch back to the preview mode and there are even more parameters now:

RegisterApp5_Manifest

The final thing would be to copy the App Id. Therefore switch to the “Overview” tab. Here you can copy the ID to the clipboard once we need it in another minute.

RegisterApp6_CopyID

Automation Account settings

PnP PowerShell module

I assume you already have an Automation Account configured. If not follow the documentation from Microsoft. On top you should install the PnP PowerShell module under modules. Simply install it on your local machine and then ZIP the local folder “C:\Program Files\WindowsPowerShell\Modules\SharePointPnPPowerShellOnline” and upload it under “Modules” in your Automation Account.

Upload pfx certificate

Now under “Certificates” upload your recently created certificate (the .pfx file). Therefore you would need the given name and the password.

AzureAutomation1_AddCert

Pay attention to check “Yes” for Exportable as we later need exactly this capability!

Finally I would like to store our App ID as well as the password for the certificate inside the Azure Automation Account assets. Although both values do not exactly belong together, you can store them as a simple credential pair. We later use them independently.

AzureAutomation1_AddCredentials

The Runbook – PnP PowerShell Commands

Now we have everything in place to create our first runbook with modern app only authentication. Therefore create a new PowerShell runbook and insert the following code:

$azureSPOcreds = Get-AutomationPSCredential -Name 'AzureAutomationSPO'

$siteUrl = "https://.sharepoint.com/teams/MMTeamNo1"

$aadDomain = ".onmicrosoft.com"

$Password = $azureSPOcreds.GetNetworkCredential().Password

$secPassword = $azureSPOcreds.Password

$clientID = $azureSPOcreds.UserName

$cert = Get-AutomationCertificate -Name 'AzureAutomationSPOAccess'

$pfxCert = $cert.Export(3 ,$Password) # 3=Pfx

$certPath = Join-Path "C:\Users" "SPSiteModification.pfx"

Set-Content -Value $pfxCert -Path $certPath -Force -Encoding Byte 
    

if (Test-Path $certPath)

{

    $pnpCert = Get-PnPAzureCertificate -CertificatePassword $secPassword `
					-CertificatePath $certPath

    Write-Output "Connecting to $siteUrl"

    Connect-PnPOnline   -CertificatePath $certPath `

                        -CertificatePassword $secPassword `

                        -Tenant $aadDomain `

                        -ClientId $clientID `

                        -Url $siteUrl 

    $web = Get-PnPWeb

    Write-Output $web.Title

}

else

{

    Write-Output "No Cert"

}

For simplicity reasons I put some other values in as hardcoded which you would also put to different Automation variables (or ONE as a config XML).

The first interesting thing is getting the $azureSPOcreds. After retrieval we extract the UserName on the one hand, that is our AppID. On the other hand with two commands we extract the password, once simply as a SecureString by retrieveng the Password attribute and give it to the $secPassword variable. Then with the help of the GetNetworkCredential() method we also extract the password in PlainText (!!!) and give it to the $Password variable. We need it that way for the certificate.

But here you see a disadvantage of the Azure Automation assets. Once you have some Admin credentials in there AND access to it, you also have the clear text password! 😉 

The next trick we use is to export the certificate to the “local filesystem of the Automation account” which is only valid during runtime of a runbook execution.

This is because the Connect-PnpOnline command we use next expects it that way and cannot handle a X509Certificate2 object directly (that is what we get with the Get-AutomationCertificate cmdlet).

Once we have that exported we simply ‘Test-Path’ our export and if everything is fine we can Connect-PnPOnline to our SharePoint and execute a simple demo task to get the current web’s title.

Setting default values on SharePoint Taxonomy columns with CSOM Powershell

Programmatically or with PnP Provisioning setting a default value for a column is not that problem but with Taxonomy columns and its corresponding Termstore termsets things get a bit more complex.

Let my shortly explain why. Assume we have a simple termset (german locations here):

Termset_GER

Assume further we have a taxonomy column and apply it a default value of “Hamburg”

TaxonomyColumn

then accessing the column with a simple REST call (/_api/web/fields I often use this for investigation) would show us the default property in the following way:

<d:DefaultValue>2;#Hamburg|220ac9fa-64f2-4eaf-9a6d-787aefb3a499</d:DefaultValue>

So what about those three values? The second value “Hamburg” indeed is the Label and the third the GUID we might know from the termstore (With PnP Provisioning for instance we can provision whole termsets with pre-created GUIDs. This gives us great control for further dealing with those termsets.
But what about the first number (“2”)?

This is an item ID from a hidden list called “TaxonomyHiddenList“. Each time you select a term from a termset, this term will be written to this list. We can check this by accessing this list. Although it is hidden, we can open it once we know its name (and if we forget the name /_api/web/lists helps us to remember)

TaxonomyHiddenList

The problem now is: A termset exists once in our tenant (we don’t talk about site collection termsets here) but the list is new in every new site collection. And as we do not control the order we are not sure that “Hamburg” will be the 2nd item in the next site collection, too. Or will it be picked at all? That’s the most important problem! We cannot rely that the termset we want to make to the default one already exists in our site collection’s TaxonomyHiddenList.

So we need some kind of an ensure process and the following CSOM-driven Powershell script will do this for us:

We need a site and its Url, the name of our column, the label and the GUID of the term we want to have as default. With that we call our function.

In the function we first access our column. After that we create a TaxonomyFieldValue with our Label and our GUID, as a WssID we use -1.

Finally, and that’s the “heart” of this function, we use the GetValidatedString method of our Taxonomy field instance. The return value will be what we need and have seen above. If the term already existed in the TaxonomyHiddenList it will contain the existing ID of the list item. If it didn’t it would have been inserted now.

The returned string we can use in our code to set the default value or insert in our Provisioning Template if we use PnP Provisioning or another tool. In PnP we can simply transfer the just evaluated string with a parameter like I did with evaluated default groups in my last post, too.

Dealing with default SharePoint groups in PnP Provisioning

Together with a SharePoint site several default groups are created. The most important ones are:

  • Owners
  • Members
  • Visitors

Unfortunately from a provisioning point of view the names are not fix and depend on the language. Furthermore I discovered this week how those groups get renamed during the creation process of modern Groups sites.

The simplest step is to add additional members to those groups. This is quite easy as you only need to add to your PnP Provisioning template:

<pnp:Security>
 <pnp:AdditionalOwners>
   <pnp:User Name="user1@yourdomain.com"/>
   <pnp:User Name="group1@yourdomain.com"/>
 </pnp:AdditionalOwners>
</pnp:Security>

But what about those attributes where you need the exact group names?

Here you can use a combination of PnP Powershell and the Parameters. Ran the following Powershell script:


$siteUrl = "https://mytenant.sharepoint.com/sites/mysite"
$templatePath="c:\Scripts\in\Template_Groups.xml"
$credentials = Get-Credential -UserName 'Admin@mytenant.onmicrosoft.com' -Message "Enter SPO credentials"
Connect-PnPOnline -Url $siteUrl -Credentials $credentials
$spoOwnerGroup = Get-PnPGroup -AssociatedOwnerGroup
Apply-PnPProvisioningTemplate -Path $templatePath -Parameters @{"SPOOwnerGroup"=$spoOwnerGroup.Title}

The main aspect here is the Get-PnPGroup -AssociatedOwnerGroup command and the parameter “SPOOwnerGroup” we provide to our template.

In our template we pick this up and use it in our Principal attribute when providing different RoleAssignments to our libraries or lists for instance.

In our example we only want to have Read access for Owners to the Site Pages library.

In the new schema 2017-05 it will be also possible to remove permissions. At the time of writing this does not work yet. So I show the way to not copy existing permissions but add new ones from the scratch. The problem with adding new permissions for existing groups is that they will be added and do not overwrite existing permissions. And if a group already has Edit permissions adding Read additionally has absolutely no effect.

When you don’t provision with Powershell but from CSOM code, the way is also valid as you have the AssociatedOwnerGroup property .

Programmatically set default values for Managed Metadata Columns

It is quite easy to set default values for SharePoint columns programmatically or with PnP Provisioning but not for every column type.

Especially Managed Metadata columns are not that easy. The problem is the technical architecture of those columns.

I guess you know about managing the termstore and assume we have the following termset attached to our column:

But once we check the content of a column in an existing list item we see something like this:

‘2;#Hamburg|{5722CAB5-58D1-48B3-B09D-378C67E2B938}’

What we see are three parts of content. The 2nd and the 3rd part are easy to guess:
Hamburg is the text value of the term as we know from the termstore and
{5722CAB5-58D1-48B3-B09D-378C67E2B938} is the GUID of the term also existing in the termstore.

But what about the 1st part? We know such syntax from lookup columns and that’s exactly the case here: The content is alos pointing to a lookup list.

Checking a site collection for all existing lists by a simple REST call like the following
http://sp2013.sp.com/sites/our_test_site/_api/web/lists
we can find a list called “TaxonomyHiddenList”.

Once we have found content like that one shown above we will find a list item with the ID 2 in that list.

The problem is, we will only find items for terms that were already used on that site but not for terms that also exist but were not used before.
So how to set default values for values that were not used already for instance in a use case of an automatically provisioned site with still no content at all?

There is the  TaxonomyField.GetValidatedString  method that helps us. It will produce a value like that one above. If an item in the “TaxonomyHiddenList” already exists or not. The following function is a CSOM Powershell script. But you can easily transfer it to managed .Net code if that fits better to your situation:

We provide a site Url, the column name and a term label and term guid to our function. The function creates a client context to the site. After that we get our column by the column name provided to the function and cast it to a TaxonomyField.

Then we create a TaxonomyFieldValue. It contains our label and GUID and a dummy item ID of -1. (As we don’t know the correct one or even if it already exists.

Finally we ensure this with our GetValidatedString method and return the whole value with all three correct parts as shown in the example above. That’s it.

XsltListViewWebPart on a Wiki page with PnP Provisioning

Here I want to show a simple and reusable example to provision a Wiki page containing a simple XsltListViewWebPart:

You can see (if aware of the list of properties a XsltListViewWebPart has) that we removed most of them. It is very important to not have properties such as ListId or ListUrl present. The properties to identify the list (and view, if you do not want just the Default view) are only ListDisplayName and ViewName which is used by the PnP engine. [Link]

What else matters?

We provision a page with the Layout TwoColumnsHeader. Our webpart we place in Row=”2” (so not in the Header) and Column=”1” so on the left side.

The Webpart’s Title is also “Tasks” and has a TitleUrl pointing to the list which you can also remove if you do not want.