Powershell Find Unique and/or Duplicate entries

Yesterday I had a case where a co-worker was trying to determine how many licenses for a particular package were in use.  Unfortunately, users are spread across multiple independent systems, but from a licensing perspective, we only need to could each individual once.

The source files were in CSV format, and each contained a column called “name”.  Some of them had additional columns, but they were ignored in this particular example.


$UserList1=Import-Csv UserList1.CSV
$UserList2=Import-Csv UserList2.CSV


$UserList1 | % { $names += $_.name }
$UserList2 | % { $names += $_.name }

# At this point $names contains all the names from both files

write-output "Total names: " ($names | measure).count
Total names:
write-output "Unique names: " ($names | group | measure).count
Unique names:

In order to interpret the results, lets look at what those last couple lines do.

write-output "Total names: " ($names | measure).count

This line takes the list of names, passes it through Measure-Object, and retrieves the .count property, so it’s a total count.

write-output "Unique names: " ($names | group | measure).count

This line takes the list of names, groups them by common name, and then counts how many groups.  Since each duplicate becomes a single row now, this is your unique count.  Since the difference is 130, that means 130 “accounts” show up 2 or more times.  The “real” unique license count is 4922.

Taking this one step further, who are your duplicates?  One more line gives a nice output (actual names deleted)

PS C:\temp\notes> $names | group | where-object { $_.count -gt 1 } | ft Count,Name -autosize

Count Name
----- ----
    2 B***** , L*****
    4 B***** , S***
    2 B***** , L*****
    2 C*** , H**
    3 J**** , M***

As you can see, here the output shows how many accounts each individual shows up!

P.S. I leave it up the the reader to change if they want the output sorted by Count or Name.

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply