5. Dezember 2018

Setting up Windows Authentication with containers connected to Azure AD Domain Services

At DockerCon Europe 2018, Israel Vega from Microsoft and Steven Follis from Docker (@steven_follis) had a session called „Avoiding an identity crisis“ talking about Windows / Active Directory authentication for containers. A really good introduction and also a lot of interesting information if you are already working with win auth in containers. For example I was fully convinced that delegation is not working, but that is just wrong (fortunately). So as soon as the recordings appear, I would suggest to watch that session. At the same time user PleachiM opened an issue in the GitHub repo of nav-docker stating that win auth against Azure Active Directory Domain Services (AAD DS) does work, which was only quickly covered in the DockerCon session, so I decided to give it a try

The TL;DR

It works¹.

Ok, maybe a bit more detail: If you setup AAD DS and make sure that the VM hosting your container is in the right virtual network and subnet, you can then create gMSAs and use them in your containers to get win auth. Through that setup you can also verify that for win auth with Windows Server 2019, the gMSA name and the container hostname no longer need to be identical.

The walkthrough of my setup

As Israel and Steven did a very good job explaining (almost) everything happening in my walkthrough, I won’t try to replicate that and instead just point you to the recording. I will only highlight things specific to my setup. Here are the steps²:

Create a resource group in Azure, in my case I name it „aad“
Install AAD DS as described here https://docs.microsoft.com/en-us/azure/active-directory-domain-services/active-directory-ds-getting-started. After that has finished, we have an Azure AD that we can use to create our gMSAs and connect containers to it. Make sure that you add a user to the administrators group.
Create a „Windows Server 2019 Datacenter with Containers“ VM which we will use to run the containers

On that VM do the following:

Join the AAD domain through Server Manager as you would do with an on prem AD and reboot.

Run the following scripts with reboots as mentioned. This is well explained in the DockerCon session, the only thing special is that we are creating our own OU as we can’t create gMSAs otherwise (see https://docs.microsoft.com/en-us/azure/active-directory-domain-services/active-directory-ds-create-gmsa). I am using „gMSAs“ as the name of my OU, which you can change if you want and „DC=ARSSOLVENDI,DC=ONMICROSOFT,DC=COM“ is the name of my domain, so you need to adapt that to your domain. Also „test19-1“ is the name of the VM.

Install-WindowsFeature RSAT-ADDS
New-ADOrganizationalUnit -Name "gMSAs" -Path "DC=ARSSOLVENDI,DC=ONMICROSOFT,DC=COM"

#Create a group to hold the container hosts
$containerHostName = "test19-1"
$containerHostGroupName = "ContainerHosts"
$containerHostGroupDisplayName = "Windows Container Hosts"
New-ADGroup -GroupCategory Security -Path "OU=gMSAs,DC=ARSSOLVENDI,DC=ONMICROSOFT,DC=COM" -Name $containerHostGroupName -DisplayName $containerHostGroupDisplayName -GroupScope Universal 

#Add the container host machines to the group
Add-ADGroupMember -Members (Get-ADComputer -Identity $containerHostName) -Identity $containerHostGroupName

#reboot

Get-ADGroup -Identity $containerHostGroupName

With that we are set up and can create the gMSA. Again, I am very creatively naming it „gmsa“ but of course you can also change that. In the next part we create the gMSA, install and test it and then download the credential spec module which is in turn used to create the credential spec file. If you are not sure what is happening here, please watch the recording

$gmsaAccount = "gmsa"
$gmsaAccountFQDN = "gmsa@arssolvendi.onmicrosoft.com"
$containerHostGroupLdapCN = "CN=ContainerHosts,OU=gMSAs,DC=ARSSOLVENDI,DC=ONMICROSOFT,DC=COM"
New-ADServiceAccount -Name $gmsaAccount -DNSHostName $gmsaAccountFQDN `
-PrincipalsAllowedToRetrieveManagedPassword "Domain Controllers", "Domain Admins", $containerHostGroupLdapCN `
-KerberosEncryptionType RC4, AES128, AES256 `
-ServicePrincipalNames HTTP/$gmsaAccount, HTTP/$gmsaAccountFQDN

Install-ADServiceAccount -Identity $gmsaAccount
Test-ADServiceAccount -Identity $gmsaAccount

Invoke-WebRequest "https://raw.githubusercontent.com/Microsoft/Virtualization-Documentation/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1" -UseBasicParsing -OutFile $env:TEMP\cred.psm1
Import-Module $env:temp\cred.psm1
New-CredentialSpec -Name "$gmsaAccount-gmsa" -AccountName $gmsaAccount -Domain $(Get-ADDomain -Current LocalComputer)
Get-CredentialSpec

Now we are prepared to actually run a container using the credential spec. After it startd, you can run the basic checks if win auth is correctly set up: Using nltest to check the parent domain and find out if querying works:

docker run --security-opt "credentialspec=file://gmsa-gmsa.json" -ti --rm mcr.microsoft.com/windows/servercore:1809 

Microsoft Windows [Version 10.0.17763.107]
(c) 2018 Microsoft Corporation. All rights reserved

C:\>nltest /parentdomain
arssolvendi.onmicrosoft.com. (1)
The command completed successfully

C:\>nltest /query
Flags: 0
Connection Status = 0 0x0 NERR_Success
The command completed successfully

To make sure there is nothing special with the first machine, I created a second one (called test19-2) and joined it to the same AAD domain

On that we run the following commands to reuse the same gMSA there as well:

Add the machine to the appropriate group to make sure the gMSA works there as well and then reboot:

# Add it to the container hosts group
Install-WindowsFeature RSAT-ADDS
$containerHostName = "test19-2"
$containerHostGroupName = "ContainerHosts"
Add-ADGroupMember -Members (Get-ADComputer -Identity $containerHostName) -Identity $containerHostGroupName

Install and test the gMSA, download the credential spec module and use it to create the file:

$gmsaAccount = "gmsa"
Install-ADServiceAccount -Identity $gmsaAccount
Test-ADServiceAccount -Identity $gmsaAccount

Invoke-WebRequest "https://raw.githubusercontent.com/Microsoft/Virtualization-Documentation/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1" -UseBasicParsing -OutFile $env:TEMP\cred.psm1
Import-Module $env:temp\cred.psm1
New-CredentialSpec -Name "$gmsaAccount-gmsa" -AccountName $gmsaAccount -Domain $(Get-ADDomain -Current LocalComputer)
Get-CredentialSpec

Run a container using the gMSA and verify win auth is working

docker run --security-opt "credentialspec=file://gmsa-gmsa.json" -ti --rm mcr.microsoft.com/windows/servercore:1809

Microsoft Windows [Version 10.0.17763.107]
(c) 2018 Microsoft Corporation. All rights reserved.

C:\>nltest /parentdomain
arssolvendi.onmicrosoft.com. (1)
The command completed successfully

C:\>nltest /query
Flags: 0
Connection Status = 0 0x0 NERR_Success
The command completed successfully

With that you should be able to use Kerberos and NTLM based authentication for users in the AAD DS. In a followup post I will show how to use this with NAV / BC.

if you ever installed apache, you will notice that this is a quote
I unfortunately forgot to write down all reboots, so if something doesn’t work, you might have to sprinkle in a reboot here and there… You know, „have you tried turning it off and on again?“