← script library

Convert Malformed UTF-8 Characters in JSON with PowerShell

System AdministrationDecember 26, 2024

Handling data correctly is of utmost importance, especially when it comes to text encoding. Malformed UTF-8 characters in JSON can cause errors in data processing and application performance issues. ServerEngine offers advanced server and data management features, making it easy to deal with such challenges effectively. In this post, we will explore a PowerShell script that identifies and fixes malformed UTF-8 characters in JSON data, maintaining data integrity and improving process reliability. The script reads a JSON file, identifies malformed UTF-8 characters and fixes them to ensure the data is correctly processed. This is particularly useful for data administrators who handle JSON data exchanges between disparate systems.

powershell
# PowerShell Script to Fix Malformed UTF-8 Characters in JSON
# Path to the JSON file
$pathToJsonFile = "C:\Path\To\Your\File.json"
# Function to fix malformed UTF-8 characters
function Fix-MalformedUTF8 {
    param (
        [string]$filePath
    )
    # Read the content of the JSON file as bytes
    $fileContent = Get-Content -Path $filePath -Encoding Byte
    # Convert byte array to UTF-8 encoding
    try {
        $decodedString = [System.Text.Encoding]::UTF8.GetString($fileContent)
    } catch {
        Write-Error "Failed to decode JSON due to malformed UTF-8 characters."
        return
    }
    # Check for malformed characters and fix them
    $fixedJsonString = $decodedString -replace [char]0xFFFD, ""
    # Write the fixed JSON string back to the file
    Set-Content -Path $filePath -Value $fixedJsonString -Encoding UTF8
    Write-Host "Malformed UTF-8 characters fixed successfully in $filePath"
}
# Execute the function
Fix-MalformedUTF8 -filePath $pathToJsonFile

Run it across your fleet

This script runs as-is on a single host. Paste it into ServerEngine to schedule it, run it on a whole server group in parallel, and keep the credentials out of the file — see the scripts documentation and the credential store.

Ready when you are.

Try ServerEngine free for 7 days.