[BUG] Sync-PnPSharePointUserProfilesFromAzureActiveDirectory cannot handle special signs like Ü,Ä,Ö

[aThuemmel]

Reporting an Issue or Missing Feature

Special Signs like you have in Nürnberg (-> ü) show up in the userprofile as wrong encoding.

Expected behavior

The cmdlet itself works without an error.

Actual behavior

Special Signs like ß,Ü,ü,Ö,ö etc. do not work (but in germany they are so often used).

Steps to reproduce behavior

$connection = Connect-PnPOnline -Url $spourl -Tenant onmicrosoft.com -ClientId "" -thumbprint "*****"

$sync = Sync-PnPSharePointUserProfilesFromAzureActiveDirectory -UserProfilePropertyMapping @{"Street"="StreetAddress";"Company"="CompanyName";"Location"="City";"ZIP-Code"="PostalCode";"CellPhone"="Mobile";"Pager"="extension_3d7216cb6e624d74b9c87434f80aea59_pager";"Fax"="FacsimileTelephoneNumber";"Division"="extension_3d7216cb6e624d74b9c87434f80aea59_division"} -Folder "User Profile Sync"

What is the version of the Cmdlet module you are running?

1.9.0.0

Which operating system/environment are you running PnP PowerShell on?

• Windows
• Linux
• MacOS
• Azure Cloud Shell
• Azure Functions
• Other : please specify

I executed this in my own Shell and in Azure runbook. Both executions get the same result.

Maybe integrate a paramater to assign an encoding, i think utf-8 should do it.

[EA12]

May be it helps, if you tell your Shell to do so? - See

$PSDefaultParameterValues['*:Encoding']='utf8'

[aThuemmel]

I think the way from Azure AD into that json file is corrrect, because in the json file everything is written corrrect. Transfer into the user profile is what ais not working correctly.

I checked which encoding is used in the shell, it is already utf-8

# Initial

Preamble          : 
BodyName          : utf-8
EncodingName      : Unicode (UTF-8)
HeaderName        : utf-8
WebName           : utf-8
WindowsCodePage   : 1200
IsBrowserDisplay  : True
IsBrowserSave     : True
IsMailNewsDisplay : True
IsMailNewsSave    : True
IsSingleByte      : False
EncoderFallback   : System.Text.EncoderReplacementFallback
DecoderFallback   : System.Text.DecoderReplacementFallback
IsReadOnly        : True
CodePage          : 65001

# [Text.Encoding]::Default
Preamble          : 
BodyName          : utf-8
EncodingName      : Unicode (UTF-8)
HeaderName        : utf-8
WebName           : utf-8
WindowsCodePage   : 1200
IsBrowserDisplay  : True
IsBrowserSave     : True
IsMailNewsDisplay : True
IsMailNewsSave    : True
IsSingleByte      : False
EncoderFallback   : System.Text.EncoderReplacementFallback
DecoderFallback   : System.Text.DecoderReplacementFallback
IsReadOnly        : True
CodePage          : 65001

# [Text.Encoding]::UTF8
Preamble          : 
BodyName          : utf-8
EncodingName      : Unicode (UTF-8)
HeaderName        : utf-8
WebName           : utf-8
WindowsCodePage   : 1200
IsBrowserDisplay  : True
IsBrowserSave     : True
IsMailNewsDisplay : True
IsMailNewsSave    : True
IsSingleByte      : False
EncoderFallback   : System.Text.EncoderReplacementFallback
DecoderFallback   : System.Text.DecoderReplacementFallback
IsReadOnly        : True
CodePage          : 65001

[EA12]

Are you able to donwload the json file from the library?
Get-pnpfile -Url "/User Profile Sync/userprofilesyncdata-20220111101720-guidWithoutHyphens.json"
Is the output OK?
Is the output OK if you download the file manually and view it in Notepad++ ?

[EA12]

BTW: I wonder that the -Folder parameter seems to work for you. I was without luck. See here

[aThuemmel]

Output in Chrome, Edge and Notepad++ (if UTF-8 is selected, which is default on my side) is correct. Of course if i switch to code to ANSI it looks like this

"Location":"Nürnberg"

[aThuemmel]

Because it is not really a folder, ment is a document library.

[EA12]

Regarding -Folder parameter: Yes, of course it is a library in my case ("User Profile Sync")

[aThuemmel]

Can this be fixed by a update? I dont see the possibility for me to manually fix it while synchronising. Only workaroundn i can think is to go through all profiles and update wrong entries by another script (which then have to run also daily)

[schneika]

We observe the same behavior. The Sync-PnPSharePointUserProfilesFromAzureActiveDirectory Commandlet is not working with national Characters. Special characters are garbled. Obviously no proper UTF8-Encoding or false BOM (byte order mask)?
Can you confirm this bug?

[KoenZomers]

Thanks guys for reporting this. Let me try to reproduce it on my end.

[KoenZomers]

I can reproduce it. Let me see how this can be fixed.

[KoenZomers]

Fixed! Will be available in the prerelease version of PnP PowerShell of February 15, 2022 at 3 AM Central European Time onwards.

PR reference #1634