This is a bit of an oldie, but I figured it might help one or two random readers.
Let me start by saying something right off the bat: you should never need what I’m about to share. Of course, how many times have you heard “you shouldn’t ever really need this” when it comes to SharePoint? I’ve been at it a while, and I can tell you that things that never should happen seem to find a way into reality – and into my crosshairs for troubleshooting.
Disclaimer
The story and situation I’m about to share is true. I’m going to speak in generalities when it comes to the identities of the parties and software involved, though, to “protect the innocent” and avoid upsetting anyone.
The Predicament
I was part of a team that was working with a client to troubleshoot problems that the client was encountering when they attempted to run some software that targeted SharePoint site collections. The errors that were returned by the software were somewhat cryptic, but they pointed to a problem handling certain objects in a SharePoint site collection. The software ran fine when targeting all other site collections, so we naturally suspected that something was wrong with only one specific site collection.
After further examination of logs that were tied to the software, it became clear that we had a real predicament. Apparently, the site collection in question contained two or more objects with the same identity; that is, the objects had ID properties possessing the same GUID. This isn’t anything that should ever happen, but it had. SharePoint continued to run without issue (interestingly enough), but the duplication of object GUIDs made it downright difficult for any software that depended on unique object identities being … well, unique.
Although the software logs told us which GUID was being duplicated, we didn’t know which SharePoint object or objects the GUID was tied to. We needed a relatively quick and easy way to figure out the name(s) of the object or objects which were being impacted by the duplicate GUIDs.
Tackling the Problem
It is precisely in times like those described that PowerShell comes to mind.
My solution was to whip-up a PowerShell script (FindDuplicateGuids.ps1) that processed each of the lists (SPList) and webs (SPWeb) in a target site collection. The script simply collected the identities of each list and web and reported back any GUIDs that appeared more than once.
The script created works with both SharePoint 2007 and SharePoint 2010, and it has no specific dependencies beyond SharePoint being installed and available on the server where the script is run.
[sourcecode language=”powershell”]
########################
# FindDuplicateGuids.ps1
# Author: Sean P. McDonough (sean@sharepointinterface.com)
# Blog: http://SharePointInterface.com
# Last Update: August 29, 2013
#
# Usage from prompt: ".\FindDuplicateGuids.ps1 <siteUrl>"
# where <siteUrl> is site collection root.
########################
#########
# IMPORTS
# Import/load common SharePoint assemblies that house the types we’ll need for operations.
#########
Add-Type -AssemblyName "Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
###########
# FUNCTIONS
# Leveraged throughout the script for one or more calls.
###########
function SpmBuild-WebAndListIdMappings {param ($siteUrl)
$targetSite = New-Object Microsoft.SharePoint.SPSite($siteUrl)
$allWebs = $targetSite.AllWebs
$mappings = New-Object System.Collections.Specialized.NameValueCollection
foreach ($spWeb in $allWebs)
{
$webTitle = "WEB ‘{0}’" -f $spWeb.Title
$mappings.Add($spWeb.ID, $webTitle)
$allListsForWeb = $spWeb.Lists
foreach ($currentList in $allListsForWeb)
{
$listEntry = "LIST ‘{0}’ in Web ‘{1}’" -f $currentList.Title, $spWeb.Title
$mappings.Add($currentList.ID, $listEntry)
}
$spWeb.Dispose()
}
$targetSite.Dispose()
return ,$mappings
}
function SpmFind-DuplicateMembers {param ([System.Collections.Specialized.NameValueCollection]$nvMappings)
$duplicateMembers = New-Object System.Collections.ArrayList
$allkeys = $nvMappings.AllKeys
foreach ($keyName in $allKeys)
{
$valuesForKey = $nvMappings.GetValues($keyName)
if ($valuesForKey.Length -gt 1)
{
[void]$duplicateMembers.Add($keyName)
}
}
return ,$duplicateMembers
}
########
# SCRIPT
# Execution of actual script logic begins here
########
$siteUrl = $Args[0]
if ($siteUrl -eq $null)
{
$siteUrl = Read-Host "`nYou must supply a site collection URL to execute the script"
}
if ($siteUrl.EndsWith("/") -eq $false)
{
$siteUrl += "/"
}
Clear-Host
Write-Output ("Examining " + $siteUrl + " …`n")
$combinedMappings = SpmBuild-WebAndListIdMappings $siteUrl
Write-Output ($combinedMappings.Count.ToString() + " GUIDs processed.")
Write-Output ("Looking for duplicate GUIDs …`n")
$duplicateGuids = SpmFind-DuplicateMembers $combinedMappings
if ($duplicateGuids.Count -eq 0)
{
Write-Output ("No duplicate GUIDs found.")
}
else
{
Write-Output ($duplicateGuids.Count.ToString() + " duplicate GUID(s) found.")
Write-Output ("Non-unique GUIDs and associated objects appear below.`n")
foreach ($keyName in $duplicateGuids)
{
$siteNames = $combinedMappings[$keyName]
Write-Output($keyName + ": " + $siteNames)
}
}
$dumpData = Read-Host "`nDo you want to send the collected data to a file? (Y/N)"
if ($dumpData -match "y")
{
$fileName = Read-Host " Output file path and name"
Write-Output ("Results for " + $siteUrl) | Out-File -FilePath $fileName
$allKeys = $combinedMappings.AllKeys
foreach ($currentKey in $allKeys)
{
Write-Output ($currentKey + ": " + $combinedMappings[$currentKey]) | Out-File -FilePath $fileName -Append
}
}
Write-Output ("`n")
[/sourcecode]
Running this script in the client’s environment quickly identified the two lists that contained the same ID GUIDs. How did they get that way? I honestly don’t know, nor am I going to hazard a guess …
What Next?
If you’re in the unfortunate position of owning a site collection that contains objects possessing duplicate ID GUIDs, let me start by saying “I feel for you.”
Having said that: the quickest fix seemed to be deleting the objects that possessed the same GUIDs. Those objects were then rebuilt. I believe we handled the delete and rebuild manually, but there’s nothing to say that an export and subsequent import (via the Content Deployment API) couldn’t be used to get content out and then back in with new object IDs.
A word of caution: if you do leverage the Content Deployment API and do so programmatically, simply make sure that object identities aren’t retained on import; that is, make sure that SPImportSettings.RetainObjectIdentity = false – not true.
Additional Reading and References
I am unfortunately one of the few people who managed to have this issue on one of my sites. I used the Central Admin Backup and Restore facility to export a Web and then imported it onto a site on our Prod app server. For some reason all the document libraries were perfectly duplicated such that if I deleted a file from the Pages library, it was reflected in both Pages libraries. So strange. Who ever said that SharePoint was anything but straight forward lol.
I also found a script that checked for duplicate documents which may be useful for someone – http://blog.pointbeyond.com/2011/08/24/finding-duplicate-documents-in-sharepoint-using-powershell/
Anyhow, thanks for the scipt, it’s very helpful in determining whether I had any other sites that had duplicated information. Luckily it was only one site that had this issue.
However, because the document libraries that were duplicated couldn’t be deleted (the Documents, Pages and Images libraries), I had to export these libraries individually off my dev server via Central Admin. Then on my Prod server, using powershell, I imported these libraries into the same site and voala it fixed the issue.
So if you do have this issue, my advice would be to export the libraries manually, then import them using powershell to the same site. The duplicated libraries will disappear like magic :)
Cheers
Danielle
Danielle,
Thank you so much much for you comment, link, and practical experience on resolving these issues in your environment. Ugh – what a nightmare!
When I wrote this post, I honestly thought it would be the sort of thing that only one or two edge-case environments would ever encounter … but more and more, I’m talking with customers and friends in the community and pointing them to the URL for this post given error messages they see and behaviors they describe in their SharePoint environments. Although the Content Deployment API can be used to import objects and retain identities in the process, I honestly cant believe that “bad imports” are the cause of *all* of the duplicate ID issues that people are encountering.
I’m glad that you got things fixed, and I do appreciate you sharing your experience. The steps you provided here may help someone else in the future :-)
Hi Sean,
When trying to execute the script I get the error “Unexpected Token ‘Add-Type’ in expression or statement.”
Do you have any clue why? What am I doing wrong?
Thanks!
Silvain,
Add-Type is basic PowerShell cmdlet that imports assemblies so that they can be referenced by the script. In this case, the script is referencing the SharePoint assembly so that SharePoint object types can be accessed.
More information on Add-Type: http://technet.microsoft.com/en-us/library/hh849914.aspx
It doesn’t look like Add-Type existed prior to PowerShell 2.0. Is there any chance you have an older version of PowerShell on your server or in your environment?
You might also try typing the offending line of PowerShell directly into a PowerShell console to see what happens. If the Add-Type cmdlet causes problems, and you’re on PowerShell 2.0 or later, then there’s something else going on. I’m afraid I don’t have more to suggest beyond that :-(
I hope that helps!
HI Sean,
Thanks so much line actually works in power shell. Is there a way you could email that script?
Silvain.mak@itesso.com?
Thanks!
Silvain,
You can copy the script directly from the listing above. If you hover over the script window at the top, you should see a few buttons appear near the top right-hand side of the script. One of those buttons allows you to open the script source in another window (as text-only), and yet another copies the script directly to your clipboard. Either of those to should be all you need to get the script in a usable form; just save the contents as a .ps1 file and run the file from a PowerShell prompt :-)
Thanks.
Thanks for the info, I just found some dupe SPWebs in different site collections on the same web application, but I am not fully sure how this happens. Likely due to some kind of content migration tool that retained GUIDs.
Also, a quick tip: You should probably call Dispose on your SPWeb objects at the end of each loop iteration, and then call Dispose on the SPSite at the end of the script
tafs7: I’m glad to hear that the script was of some use to you. Good catch on the disposals, as well; not sure why I omitted them, but I’ve updated the script as a result of your feedback. Thank you!
Thx Dude! Perfect script! Works great for me.
Glad it did the trick. Thanks for the feedback, and happy holidays!