SharePoint 2007 : The incremental crawl never ends!

This is the second time my crawl sticks on indexing files. It shows that it has been crawling for 85 hours. To solve this problem, the first time, I remember, I had to restart the SharePoint services and initialize the index. This time, I do not know how that happened but I found out that on one WFE no SharePoint service was running! So, I started all needed services. When I went back to my Search settings, I found that the stuck crawl has ended. I launched an incremental crawl for all my content sources. Every thing has worked as expected. I am glad I did not have to reinitialize my index this time. I have more than 100000 items in it. It would take me hours to reconstruct it. I tested my search and it is working fine.

Hope this helps.

System.InvalidOperationException: This access control list is not in canonical form and therefore cannot be modified

In this post I want to share a solution for a problem that may occur when you want to copy pages or sites from a location to another inside a site collection, using the content and structure 'copy' option. You may receive this error :

System.InvalidOperationException: This access control list is not in canonical form and therefore cannot be modified.


At System.Security.AccessControl.CommonAcl.ThrowIfNotCanonical()
At System.Security.AccessControl.CommonAcl.RemoveInheritedAces()
At System.Security.AccessControl.CommonSecurityDescriptor.SetDiscretionaryAclProtection(Boolean isProtected, Boolean preserveInheritance)
At System.Security.AccessControl.ObjectSecurity.SetAccessRuleProtection(Boolean isProtected, Boolean preserveInheritance)
At Microsoft.SharePoint.Deployment.DataFileManager.<>c__DisplayClass1.<CreateDirectoryAsProcessAccount>b__0()
At Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode)
At Microsoft.SharePoint.Deployment.DataFileManager.CreateDirectoryAsProcessAccount(String strDir, Boolean allowCurrentUserWriteAccess)
At Microsoft.SharePoint.Deployment.SPExport.AutoGenerateDataFolder()
At Microsoft.SharePoint.Deployment.SPExport.Run()
At Microsoft.SharePoint.Publishing.Internal.DeploymentWrapper.Copy(String[] sourceSmtObjectIds, String destSmtObjectId)
At Microsoft.SharePoint.Publishing.Internal.WebControls.CopyObjects.Copy()
At Microsoft.SharePoint.Publishing.Internal.WebControls.CopyObjects.DoWork()
At Microsoft.SharePoint.Publishing.Internal.LongRunningOperationJob.<>c__DisplayClass16.<ThreadEntryPoint>b__11()
At Microsoft.Office.Server.Diagnostics.FirstChanceHandler.ExceptionFilter(Boolean fRethrowException, TryBlock tryBlock, FilterBlock filter, CatchBlock catchBlock, FinallyBlock finallyBlock)

The problem is that the ACL for the 'Windows\temp' directory is corrupted and therefore, the accounts WSS_WPG and WSS_ADMIN_WPG have no access to the folder to put temporary stuff. 

The solution is to redefine the ACL:

1. Right-Click on the folder Windows\temp
2. Click on the Security tab
3. Click on the Advanced button
4. Ensure that the account WSS_ADMIN_WPG as the full control permission
5. Ensure that the account WSS_WPG has the Read & execute permission.
6. Select both 'Allow inheritable permissions...' and 'Replace permission entries...'
7. Click OK. 

A 1000 thanks for my colleague Frédéric Lévesque the 'Security guy' for helping troubleshooting the issue.

Hope this helps.

Extracting SSP user profiles in a xml file

Have you ever wanted to get all user profiles in one listing, especially that in the SSP UI, profiles are displayed in a paginated list? If you happen to have thousands of users then expect to have hundred of pages. The SSP user profiles list is not very intuitive that's why I developed a console application to get all users in a single xml file that I can sort an filter as I wish. Here is the code :

Imports System.Text
Imports System.IO
Imports System
Imports Microsoft.SharePoint
Imports System.Xml
Imports System.Web
Imports Microsoft.Office.Server
Imports Microsoft.Office.Server.UserProfiles

Module SSPUserProfiles

    Sub Main()
        EnumUserProfiles()
    End Sub
    Private Sub EnumUserProfiles()
        Try
            Using Site As New SPSite("http://ServerName")
                Dim siteContext As ServerContext = ServerContext.GetContext(Site)
                Dim ProfileManager As New UserProfileManager(siteContext)
                ' Open a new XML file stream for writing
                Dim stream As IO.FileStream
                stream = File.OpenWrite("UserProfiles.xml")
                Dim writer As XmlTextWriter = New XmlTextWriter(stream, Encoding.UTF8)

                ' Causes child elements to be indented
                writer.Formatting = Formatting.Indented

                writer.WriteProcessingInstruction("xml", "version=""1.0"" encoding=""utf-8""")
                writer.WriteStartElement("UserProfiles")
                writer.WriteAttributeString("Count", ProfileManager.Count.ToString)
                For Each Userp As UserProfile In ProfileManager
                    
                    writer.WriteStartElement("UserProfile")
                    writer.WriteAttributeString("AccountName", Userp("AccountName").Value.ToString())
                    writer.WriteAttributeString("PreferredName", Userp("PreferredName").Value.ToString())
                    If Userp("WorkEmail").Value IsNot Nothing Then
                        writer.WriteAttributeString("E-Mail", Userp("WorkEmail").Value.ToString())
                    End If
                    writer.WriteEndElement() 'UserProfile
                Next
                writer.WriteEndElement() 'UserProfiles
                ' Flush the writer and close the stream
                writer.Flush()
                stream.Close()
            End Using
        Catch exp As Exception
            Console.WriteLine(exp.Message)
        End Try
    End Sub
End Module

For a full list of user profiles properties, please read Anne Stenberg's Blog.

Hope this helps.

SharePoint log viewer

Finally, I had some time to develop a tool that I always wanted. A tool that will let me go trough the SharePoint log (where unfortunately I spend lot of time) with ease to find the information I want. Actually it is very simple : It is a small windows application that read the text log file, convert it to a xml file and then, bind it to DataGridView where it can be filtered By :

- Time stamp
- Process
- Area
- Category
- Level of severity


You can find it on CodePlex at : http://mosslogviewer.codeplex.com. It is still a beta version but many ideas are coming.

Your feedback is welcome.

Enjoy!

Document icons missing in the search results

This week I had mounted a new server. When I configured Search I found that the documents icons (all types of documents) do not show in the search results page. When it comes to documents icons we always have to check the ...\12\TEMPLATE\XML\DocIcon.xml. The problem was that the file had two instances of the pdf mapping key and the gif file name which is in the mapping value had a blank space in it. I deleted one instance, corrected the gif file name and performed an IISRESET. After that, the icons displayed fine.

To summarize here is what I did :

- Verify that there are no double mapping keys
- Check the icons file names
- IIS reset or recycle the pools

Hope this helps someone.

SharePoint 2010 Public Beta available for download

Microsoft SharePoint team has announced the availability of SharePoint 2010 Public Beta for download. More information and links are available on the SharePoint team blog : http://blogs.msdn.com/sharepoint/archive/2009/11/18/sharepoint-2010-public-beta-is-now-available-for-download.aspx.

Also, Bill Baer has posted a nice article on how to install SharePoint 2010 and SharePoint Foundation 2010 (aka WSS 4). You can read his article here : http://blogs.technet.com/wbaer/archive/2009/11/18/installation-notes-for-microsoft-sharepoint-server-and-microsoft-sharepoint-foundation-2010-beta.aspx

It is not recommended to use SharePoint 2010 Beta in a production environment because it is not supported. Use it only for evaluation purposes. The release version of SharePoint 2010 is expected in first half of the year 2010.

Happy SharePointing...

SharePoint Search : Some files are not crawled (indexed)!

Recently, I was asked to find out why some documents are not indexed in a particular library and therefore, not showed in the search results. I began by inspecting the crawler log to see if there are any errors or warnings. No errors nor warnings I found. Better, I found that some of the documents are crawled correctly.
Comparing the crawled documents with the non-crawled ones, it appeared that the non-crawled ones are in minor versions (draft). By default, the crawler account is granted 'Full Read' permission. Which mean that it just cannot see draft documents which are visible only to authors who have 'Edit' permission.

So what is the solution? You have to :

- Either grant the crawler account the 'Edit' permission to let him see unpublished files and crawl them. In this case, all draft documents will show in search results to everyone, even to visitors who are not supposed to see them. The search results are not security trimmed (1). However, if you do not have access to a document, you still be denied the access even if it shows in the search result.
- Or keep the crawler account with 'Full Read' and publish the draft documents into major versions.
- Otherwise, accept to not index draft documents

I cannot recommend a solution or another. Every company must have a documents management policy, and its according to this policy that we can decide if we have to raise the right of the crawler account or keep draft documents out of the search scope.

Here are some interesting links to better understand SharePoint search behaviour :

What Does the Crawler Crawl and When?
SharePoint indexing/search behavior on major and minor versions
MOSS Enterprise Search - 16 things you might not know

Hope this helps.


(1) The search results are not trimmed only for draft items. That's what I noticed. For the other items, the results shown are trimmed at query time according to the permissions the user has.