Tag: CSOM

Secure Azure Functions Part 2 – Handle certificates with Azure KeyVault when accessing SharePoint Online

Secure Azure Functions Part 2 – Handle certificates with Azure KeyVault when accessing SharePoint Online

This is the second post of my little series on secure Azure Functions working with Office 365. The first one was about “simple” credential (user/password or ID/secret) access. Now we need to use an additional certificate.

Recently I spent lots of time with modern SharePoint authentication used in either Azure Automation or Azure Functions. For most of the parts there is some documenation but not as a whole and step-by-step guide. This is what this blog post wants to accomplish. One of the best existing posts is this one from Jeremy Hancock.

The Architecture Scenario

As in part 1 we use an Azure Function to be securely called from a SPFx webpart with AADHttpClient for instance. We do not use the user authentication but an impersonation. That is, an own Azure AD App Registration with own permissions. This can be a necessary scenario when you need elevated privileges (as SPFx directly only can use user permissions when calling SharePoint).

Example scenarios can be around provisioning or post-provisioning site modifications (where you want to allow specific users to handle stuff that needs elevated privileges).

01securelyaccessspofromazurefunction

The Azure Function is once again MSI enabled so it can authenticate “itself” against the Key Vault (which gave access to the function, see part 1). With the Client ID of an registered app, which is given SharePoint Api permissions, the Azure Function will access SharePoint.

Create a self-signed certificate

For this step we can reuse the certificate / description from my last blogpost.

But in short again:

  • Create a self signed certificate with New-PnPAzureCertificate cmdlet
  • Keep the PS window open, we need the certificate’s KeyCredentials in a minute
$Password = "******"
$secPassword = ConvertTo-SecureString -String $Password -AsPlainText -Force
$cert = New-PnPAzureCertificate -Out "AzureAutomationSPOAccess.pfx" - `
 -ValidYears 10 `
 -CertificatePassword $secPassword
 -CommonName "AzureAutomationSPOAccess" `
 -Country "DE" `
 -State "Bavaria"

$cert.KeyCredentials
{
    "customKeyIdentifier": "zUFQhchR6FJ0...",
    "keyId": "4d2fe8fc-0dbb-45a7-...",
    "type": "AsymmetricX509Cert",
    "usage": "Verify",
    "value":  "MIIDJDCCAgygAwIBAgIQV9qo..."
}

App registration and SharePoint Online Api

For this step we can reuse the certificate / description from my last blogpost.

In short again:

  • Go to your Azure Portal 
  • Switch to Azure Active Directory
  • Choose “App registrations (preview)” version (the standard works either, but…)
  • Register an App with a name of your choice
  • Provide adequate SharePoint Api permissions (Application permissions for an elevated scenario!)
  • Grant your permissions with an admin account 
  • Insert Key Credentials in app registration`s manifest settings
Register Azure AD App registration
Provide SharePoint Api permissions
Paste KeyCredentials from certificate to app registation manifest

Import certificate in Azure Key vault

This is also quite the same than loading the certificate to an Azure Automation account:

You have to import the .pfx file under “Certificates” to your Azure Key vault by entering a name and the given password (to make sure you are the one who “controls” the certificate)

Example Code

We need three parts for this:

  • Our key vault controller as we had it in part 1 as well
  • An authentication helper class for establishing our authentication / client context
  • The Azure Function itself using both parts mentioned beforehand

Retrieve certificate from Key vault

It might not seem obvious but this step is quite similar to that one in part 1. Although we imported a certificate above (and not a secret), we now need to retrieve the secret of exactly that certificate first. And this works quite the same than in part 1.

The ‘magic’ happens afterwards: While in part one we simply returned the retrieved secret value, we NOW use that value to create a X509Certificate2 from it and return that one for further usage.

Access SharePoint Online

To access SharePoint Online we use a simple CSOM / MSAL combination.

In our helper class this time (in part 1 of this series I retrieved it via the Azure Key vault but for an ID this is not 100% necessary) we retrieve our client ID from configuration manager (your local.settings.json for local debugging, respectively the “Application settings” of your Azure function).

Then we create our MSAL Authentication context quite similar to my last blog post. Next step is once again retrieving an access token but this time we provide a combination of our ID and a X509Certificate2.

Final thing is to create a CSOM client context and attach the access token to every request.

Using it in an Azure Function

To use that stuff is also no rocket science anymore. For simplicity reasons I minimized the SharePoint operation itself to a simple web.Title retrieval. Of course that part is much more code in other scenarios but here I wanted to put attention on Key vault access and SharePoint authentication only.

Only two lines are really important to mention here. At first we retrieve our certificate by using our KeyVaultAccess-controller from above in line 8. Next we retrieve our SharePoint ClientContext from our SPOAuthHelper in line 10.

Conclusion

Modern SharePoint authentication becomes more and more relevant. Furthermore there is a necessity for a secure but comfortable handling of secret artefacts such as credentials, app secrets or private keys. The combination of Azure Function, Azure Key vault and modern SharePoint authentication addresses this. 

 

Markus is a SharePoint architect and technical consultant with focus on latest technology stack in Office 365 and SharePoint Online development. He loves the new SharePoint Framework as well as some backend stuff around Azure Automation or Azure Functions and also has a passion for Microsoft Graph.
He works for Avanade and is based in Munich.
Although if partially inspired by his daily work opinions are always personal.

Set the site classification with PnP PowerShell and PnP Core component SiteExtension method

In the last weeks and months after Microsoft pushed the option to work with site classifications a bit there has been some confusion about the right way to handle that property programmatically.

Recently (to the time of writing this small post) the PnP team has released its new core component as well as its November update on PnP PowerShell. And now there are new capabilities to handle it better.

In the past we had two options to create an Office 365 Group respectively a modern site:

  • First and for Groups (Team Sites) only: The New-PnPUnifiedGroup
  • Second and for both current modern sites (Teamsites as well as modern Communication Sites): The New-PnPSite

The first command isn’t able to modify the site classification at all by now. The second didn’t really work so far. Especially not for Team Sites (means Groups).

The latter issue might depend on the case that for Communication sites, as they are “sites only” it seems enough to edit the site property site.Classification
For Groups we checked this in the past and it was overwritten by the system quite soon which made me assume quite early there must be another (more major) place where this is mainly maintained.
A short look to the new SiteExtension method in the PnP Core library shows this: Here different lines of code handle the classification of Communication (SITEPAGEPUBLISHING#0) and Team Sites (GROUP#0)

I wish I could show you a way now to simply use some PnP cmdlets and set the classification as long as you want but at the moment I found now way.

But what I found out is a combination of the use of PnP PowerShell together with some kind of “native PnP Core CSOM driven” approach 😉
In genereal that kind of approach might also work in the future once you have another capability only available in a CSOM or PnP Core method.

Some notes on the script:

  • I am adding the 3 .Net libraries from my PnP PowerShell module installation. This works on Azure Runbooks (I am currently heavily using them) as well. Slightly different paths of course …
  • I am using the Connect-PnPMicrosoftGraph with the parameters of an registered (and consented) Azure App that has the right permissions. See here how this works if you are not already familiar with it
    • Please also note that this command just got deprecated and its function was moved to the Connect-PnPOnline  as well. I was just not able to get it running by now so I kept the old way here so far
  • After that I simply store the accessToken in a variable for later use
  • The next block simply gets CSOM driven access to the site object
  • At last I am simply calling the static extension method, coming from the OfficeDevPnP.Core.dll to update the site classification. Here I provide the site object, a value for the classification and the stored accessToken.

That’s all

Maybe for those who are not yet familiar with the topic:

You first need to enable it and provide a short list of classifications to your tenant. Fortunately with PnP PowerShell this is now a two-liner (ref documentation)

Once you have done this (note you can also provide an underlying link to your user guidance, so the value is linked to that)

SelectSiteClassification

Your site owners can manually set the classification of the site or you can provide it programmatically as seen above and the result looks like

SiteClassificationDisplay

Setting default values on SharePoint Taxonomy columns with CSOM Powershell

Programmatically or with PnP Provisioning setting a default value for a column is not that problem but with Taxonomy columns and its corresponding Termstore termsets things get a bit more complex.

Let my shortly explain why. Assume we have a simple termset (german locations here):

Termset_GER

Assume further we have a taxonomy column and apply it a default value of “Hamburg”

TaxonomyColumn

then accessing the column with a simple REST call (/_api/web/fields I often use this for investigation) would show us the default property in the following way:

<d:DefaultValue>2;#Hamburg|220ac9fa-64f2-4eaf-9a6d-787aefb3a499</d:DefaultValue>

So what about those three values? The second value “Hamburg” indeed is the Label and the third the GUID we might know from the termstore (With PnP Provisioning for instance we can provision whole termsets with pre-created GUIDs. This gives us great control for further dealing with those termsets.
But what about the first number (“2”)?

This is an item ID from a hidden list called “TaxonomyHiddenList“. Each time you select a term from a termset, this term will be written to this list. We can check this by accessing this list. Although it is hidden, we can open it once we know its name (and if we forget the name /_api/web/lists helps us to remember)

TaxonomyHiddenList

The problem now is: A termset exists once in our tenant (we don’t talk about site collection termsets here) but the list is new in every new site collection. And as we do not control the order we are not sure that “Hamburg” will be the 2nd item in the next site collection, too. Or will it be picked at all? That’s the most important problem! We cannot rely that the termset we want to make to the default one already exists in our site collection’s TaxonomyHiddenList.

So we need some kind of an ensure process and the following CSOM-driven Powershell script will do this for us:

We need a site and its Url, the name of our column, the label and the GUID of the term we want to have as default. With that we call our function.

In the function we first access our column. After that we create a TaxonomyFieldValue with our Label and our GUID, as a WssID we use -1.

Finally, and that’s the “heart” of this function, we use the GetValidatedString method of our Taxonomy field instance. The return value will be what we need and have seen above. If the term already existed in the TaxonomyHiddenList it will contain the existing ID of the list item. If it didn’t it would have been inserted now.

The returned string we can use in our code to set the default value or insert in our Provisioning Template if we use PnP Provisioning or another tool. In PnP we can simply transfer the just evaluated string with a parameter like I did with evaluated default groups in my last post, too.

Dealing with default SharePoint groups in PnP Provisioning

Together with a SharePoint site several default groups are created. The most important ones are:

  • Owners
  • Members
  • Visitors

Unfortunately from a provisioning point of view the names are not fix and depend on the language. Furthermore I discovered this week how those groups get renamed during the creation process of modern Groups sites.

The simplest step is to add additional members to those groups. This is quite easy as you only need to add to your PnP Provisioning template:

<pnp:Security>
 <pnp:AdditionalOwners>
   <pnp:User Name="user1@yourdomain.com"/>
   <pnp:User Name="group1@yourdomain.com"/>
 </pnp:AdditionalOwners>
</pnp:Security>

But what about those attributes where you need the exact group names?

Here you can use a combination of PnP Powershell and the Parameters. Ran the following Powershell script:


$siteUrl = "https://mytenant.sharepoint.com/sites/mysite"
$templatePath="c:\Scripts\in\Template_Groups.xml"
$credentials = Get-Credential -UserName 'Admin@mytenant.onmicrosoft.com' -Message "Enter SPO credentials"
Connect-PnPOnline -Url $siteUrl -Credentials $credentials
$spoOwnerGroup = Get-PnPGroup -AssociatedOwnerGroup
Apply-PnPProvisioningTemplate -Path $templatePath -Parameters @{"SPOOwnerGroup"=$spoOwnerGroup.Title}

The main aspect here is the Get-PnPGroup -AssociatedOwnerGroup command and the parameter “SPOOwnerGroup” we provide to our template.

In our template we pick this up and use it in our Principal attribute when providing different RoleAssignments to our libraries or lists for instance.

In our example we only want to have Read access for Owners to the Site Pages library.

In the new schema 2017-05 it will be also possible to remove permissions. At the time of writing this does not work yet. So I show the way to not copy existing permissions but add new ones from the scratch. The problem with adding new permissions for existing groups is that they will be added and do not overwrite existing permissions. And if a group already has Edit permissions adding Read additionally has absolutely no effect.

When you don’t provision with Powershell but from CSOM code, the way is also valid as you have the AssociatedOwnerGroup property .

Programmatically set default values for Managed Metadata Columns

It is quite easy to set default values for SharePoint columns programmatically or with PnP Provisioning but not for every column type.

Especially Managed Metadata columns are not that easy. The problem is the technical architecture of those columns.

I guess you know about managing the termstore and assume we have the following termset attached to our column:

But once we check the content of a column in an existing list item we see something like this:

‘2;#Hamburg|{5722CAB5-58D1-48B3-B09D-378C67E2B938}’

What we see are three parts of content. The 2nd and the 3rd part are easy to guess:
Hamburg is the text value of the term as we know from the termstore and
{5722CAB5-58D1-48B3-B09D-378C67E2B938} is the GUID of the term also existing in the termstore.

But what about the 1st part? We know such syntax from lookup columns and that’s exactly the case here: The content is alos pointing to a lookup list.

Checking a site collection for all existing lists by a simple REST call like the following
http://sp2013.sp.com/sites/our_test_site/_api/web/lists
we can find a list called “TaxonomyHiddenList”.

Once we have found content like that one shown above we will find a list item with the ID 2 in that list.

The problem is, we will only find items for terms that were already used on that site but not for terms that also exist but were not used before.
So how to set default values for values that were not used already for instance in a use case of an automatically provisioned site with still no content at all?

There is the  TaxonomyField.GetValidatedString  method that helps us. It will produce a value like that one above. If an item in the “TaxonomyHiddenList” already exists or not. The following function is a CSOM Powershell script. But you can easily transfer it to managed .Net code if that fits better to your situation:

We provide a site Url, the column name and a term label and term guid to our function. The function creates a client context to the site. After that we get our column by the column name provided to the function and cast it to a TaxonomyField.

Then we create a TaxonomyFieldValue. It contains our label and GUID and a dummy item ID of -1. (As we don’t know the correct one or even if it already exists.

Finally we ensure this with our GetValidatedString method and return the whole value with all three correct parts as shown in the example above. That’s it.