Not really rocket science, but couldn't find anything on the web and needed to decompile to find the solution, so decided to drop it here. The ClientWebPartProperty does the job:
$webpart = New-Object Microsoft.SharePoint.WebPartPages.ClientWebPart
$newProperty = New-Object Microsoft.SharePoint.WebPartPages.ClientWebPartProperty
$newProperty.Name = "nameoftheproperty"
$newProperty.Value = "valueoftheproperty"
Don't forget the set the FeatureId, WebPartName and ProductWebId properties as explained here.