28 October 2010

Knowledge Base - Migrate KB Documents to new a 2010 KB library

Problem
In 2007 we had a Knowledge Base document library based on the old MS Fab 40 templates, which as you may or may not know do not work in SP 2010. I found that "Khalil" at TechSolutions had rewritten some of the templates to work in 2010 (http://techsolutions.net/Blog/tabid/65/EntryId/17/Fab-40-Templates-for-MOSS-2010.aspx) so I was able to create a new KB site and document library. Now, the problem I had was getting my current KB documents and articles into my new KB library, which had a number of category/keywords/related articles meta-data that we wanted to retain.

Investigation
The old KB library was based on two content types: Knowledge Base Articles and Knowledge Base Documents. The former contained a richtext field called "Article Content" (internal name KBContentField). When I migrated my 2007 KB list into 2010 and tried to view a KB article the Article Content field is not displayed, the page only displays the "Wiki Content" field (internal name WikiField) which is part of the new Wiki Page content type. Similarly, when trying to create a new "Article" in 2010 it would default to creating a new page in the SitePages library (there is a way around that using the folder parameter) but seemed there were too many reasons to just make a new KB library rather than hacking the current one.
The new KB library was based on two content types: Wiki Page and Knowledge Base Document. I tried doing a "move" and "copy" from "Site Content & Structure" page but found that when copying the
"Knowledge Base Articles" items it simply added that content type to my new KB list and I still had the problem that the Article Content field isn't displayed when viewing the article.
So, I decided to write a little Winforms application to migrate the docs and articles to the new KB library.

Solution:
I created a c# Windows Forms application (used this as opposed to a console app just for better control and readability of output) using the SharePoint 2010 Client Object Model (COM). This is my first COM app so took my a little longer than expected but it all worked out well in the end.

Here's my code for the whole procedure. It is rather procedural and unrefined as it was a quick run-once job, but hopefully you can pick out the bits that are useful to you (error trapping and result output code removed for brevity):


using Microsoft.SharePoint.Client;

string serverName = "spserver";
string sourceRelSiteUrl = "sites/kbsource";
string destRelSiteUrl = "sites/kbdest";
string sourceList = "Knowledge Base";
string destList = "Knowledge Base";

// Create client content for source and destination site
using (ClientContext sourceClientContext = new ClientContext(string.Format("http://{0}/{1}/", serverName, sourceRelSiteUrl)))
{
    using (ClientContext destClientContext = new ClientContext(string.Format("http://{0}/{1}/", serverName, destRelSiteUrl)))
    {

            // Get reference to source and destination KB libraries
            var sourceKBList = sourceClientContext.Web.Lists.GetByTitle(sourceList);
            var destKBList = destClientContext.Web.Lists.GetByTitle(destList);

            // Create and run CAML query to get all items from source KB library
            CamlQuery camlQuery = new CamlQuery();
            camlQuery.ViewXml = "<View />";
            ListItemCollection listItems = sourceKBList.GetItems(camlQuery);
            sourceClientContext.Load(sourceKBList);
            // make sure you call back all the list item fields and properties you will need later on
            sourceClientContext.Load(listItems, li => li.Include(pi => pi.Id, pi => pi.DisplayName, pi => pi.ContentType, pi => pi.FieldValuesAsText, pi => pi.FieldValuesAsHtml, pi => pi["KBContentField"], pi => pi["Category"], pi => pi["Description0"], pi => pi["KBRelatedArticles"], pi => pi["KBKeywords"], pi => pi["FileRef"], pi => pi["ContentTypeId"]));
            sourceClientContext.ExecuteQuery();

            // Set destination content type IDs (get these codes from URL ctype param when viewing teh list content type in browser)
            string wikiCTID = "0x0101080026185C104CE31843BF8B563CB5CA3DD3";
            string docCTID = "0x01010091E04918BF0B9A4D9A94E6D01E34FC19000F2C4D385DC79C408EF48636F2A2CE36";
            ContentTypeCollection listCTID = destKBList.ContentTypes;
            ContentType wikiCType = listCTID.GetById(wikiCTID);
            ContentType docCType = listCTID.GetById(docCTID);
            destClientContext.Load(listCTID);
            destClientContext.Load(wikiCType, ct => ct.Id);
            destClientContext.Load(docCType, ct => ct.Id);
            destClientContext.ExecuteQuery();

            foreach (var sourceListItem in listItems)
            {

                    if (sourceListItem.ContentType.Name.ToLower().Equals("knowledge base article"))
                    {
                    // KB Article Item

                        // Create new Wiki page in detination using name of source item
                        File newKBArticle = destKBList.RootFolder.Files.AddTemplateFile(string.Format("/{0}/{1}/{2}.aspx", destRelSiteUrl, destList, sourceListItem.DisplayName), TemplateFileType.WikiPage);

                        // Now get reference to your wiki page list item and set your field values
                        var wikiListItem = newKBArticle.ListItemAllFields;
                        wikiListItem["Article_x0020_Title"] = sourceListItem.DisplayName;
                        wikiListItem["Category"] = sourceListItem["Category"];
                        wikiListItem["Description0"] = sourceListItem["Description0"];
                        wikiListItem["ContentTypeId"] = wikiCType.Id;

                        // This is the bit where we push the old Article Content field into the new Wiki Content field
                        wikiListItem["WikiField"] = sourceListItem.FieldValuesAsHtml["KBContentField"].ToString();
                        wikiListItem.Update();

                        // Don't forget to execute the change to the destination client content
                        destClientContext.ExecuteQuery();
                                
                    }
                    else if (sourceListItem.ContentType.Name.ToLower().Equals("knowledge base document"))
                    {
                    // KB Document item

                        if (sourceListItem.File != null)
                        {
                            // Get source document name
                            string fileName = (string)sourceListItem["FileRef"];
                            if (fileName.LastIndexOf("/") != -1)
                                fileName = fileName.Substring(fileName.LastIndexOf("/") + 1);

                            // Set source document UNC path
                            string fileFullPath = string.Format(@"\\{0}\{1}\{2}\{3}", serverName, sourceRelSiteUrl, sourceList, fileName);
                            FileCreationInformation newFile = new FileCreationInformation();
                                
                            // Get byte array of the document via UNC
                            newFile.Content = System.IO.File.ReadAllBytes(fileFullPath);
                                
                            // Set the Destination URL of the document
                            newFile.Url = string.Format("http://{0}/{1}/{2}/{3}", serverName, destRelSiteUrl, destList, fileName);
                                
                            // Now add the document to the destination library
                            File newKBDoc = destKBList.RootFolder.Files.Add(newFile);

                            // Get a reference to the list item for the document we just added
                            var wikiListItem = newKBDoc.ListItemAllFields;

                            // Set the field values
                            wikiListItem["Article_x0020_Title"] = sourceListItem.DisplayName;
                            wikiListItem["Category"] = sourceListItem["Category"];
                            wikiListItem["Description0"] = sourceListItem["Description0"];
                            wikiListItem["ContentTypeId"] = docCType.Id;
                            wikiListItem.Update();

                            // Don't forget to execute the change to the destination client content
                            destClientContext.ExecuteQuery();

                        }
                    }
 
            }


    }
}

15 October 2010

Issue when redeploying a BCS solution to SharePoint 2010

Problem:
I created a BCS solution within Visual Studio 2010 and deployed it to my SharePoint farm which worked fine. I later made some changes to the BDCM (BDC Model file) within my project and redeployed it to the server. I then got errors when calling the BDC connection in a  BDC data item web part, which referred to variables in the BDCM file that no longer existed. I did global searches in my solution and found no references anywhere to those old variables. Example error message:


TypeDescriptor with Name 'Identifier1' (found in Parameter with Name 'returnParameter', Method with Name 'ReadList', Entity (External Content Type) with Name 'Job' and Version '1.0.0.1' in Namespace 'Org.JobList') refers to an Identifier with Name 'Identifier1' of Type 'System.String' which is supposed to exist on Entity with Name 'Job' in Namespace 'Org.JobList'. This Identifier cannot be found.


Investigation:
After a bit of messing around I found that when I redeployed the solution, whilst it took care of retracting and deploying the solution then replacing the "External Content Type", it did NOT update the "BDC Model", which is the fundamental part, the XML definition, of your connection. So, the references to old variables were coming from the old version of the BDC Model.

Solution:
If you are deploying a new version of a BCS solution ,which includes changes to the BDCM file, you must do the following:
  1. Go to Central Admin > Application Management > Manage Service Applications
  2. Select Business Data Connectivity Service and click the Manage button
  3. In the drop down list at the top of the page select BDC Models
  4. Find the BDC Model that relates to your project and select Delete from its context menu
  5. Now Deploy your package from Visual Studio
  6. Don't forget to set permissions again on the External Content Type after deployment and check the "Propogate permissions to all methods...." checkbox when doing so
NOTE: If you haven't changed the BDCM file then there's no need to do this before deployment

How To Map The SharePoint 2010 'PictureURL' User Profile Property to a BDC Attribute Using Forefront Identity Manager 2010

[ UPDATE 08-Jun-11: I first used the below approach against SPS 2010 RTM and it worked fine on 3 different environments. I subsequently tried to set this up on another environment that was running version 14.0.5128.5000​ and this failed. The SPS_MV_String_PictureURL value was getting populated but the value was not mapping through to PicureURL property. This may or may not have been due to the fact that this particular instance was also configured to export those pictures to Active Directory (I never had time to get to the bottom of it - but that UPS also had another bunch of issues which led to us recreating the UPS, so the PictureURL sync failure may have been specific to that environment). In any case, if you are not using RTM (which I assume is most of you) then I can't guarantee this will work. There's a comment below from Johann who says this approach worked perfectly for him, so I'll check which version he is running and report back soon. ]

A bit of a problem...

My company store and manage all staff images in an intranet application and those images are available via the url http://company/images/XXXXXX.jpg where XXXXXX is a 6 digit number. That intranet application surfaces a web service method that returns a string called imageURL for a given staff ID.

In SharePoint (MOSS) 2007 I was able to map the PictureURL User Profile property to the imageURL property of my BDC connection. However, in SharePoint 2010 you are unable to map the property.

A bit of investigation...

They said it couldn't and/or shouldn't be done, that you could not map the URL string value from a BDC connection (or any other data connection such as an AD attribute) because the PictureURL property has a data type of PropertyDataType.URL (add reference). Now, that is half right I suppose, in that you cannot do so through the web interface; the BDC image URL string property that I want to map simply doesn't appear in the attributes drop down list in the Add New Mapping section, but....

After plenty of fruitless Googling I came to the familiar conclusion that I'm just going to have to find my own way to squirt my company's staff image URLs into the PictureURL property.

A couple of blogs suggested using FIM to map the extensionAttributeX (http://goodbadtechnology.blogspot.com/2010/05/setting-up-pictureurl-user-profile.html) so I took that a step further and did some investigations. I wanted to see what happened when you try to map a different User Profile property, which also has a URL data type, to my BDC property. So as a test I mapped the Outlook Web Access URL property, real name SPS-OWAUrl, to my BDC property (which I was able to do just fine through the web interface) and, using FIM, saw that my BDC property (imageURL) is mapped to a metaverse attribute called SP_MV_String_SPS-OWAUrl, which in turn is mapped to the real User Profile property SPS-OWAUrl.

So, using the same approach via FIM I manually created the metaverse attribute for PictureURL using the same naming convention, SP_MV_String_PictureURL, and set up the mappings in the same way. To my amazement a full crawl later voila! All my users had their corporate photo in SharePoint. Here's exactly what I did...

A resolution...

1) Open the FIM console: C:\Program Files\Microsoft Office Servers\14.0\Synchronization Service\UIShell\miisclient.exe

2) Click the Metaverse Designer button at the top

3) Select “person” in the Object Types list

4) Click Add Attribute on the right

5) In the dialog box that appears click the New Attribute button

6) Enter “SPS_MV_String_PictureURL” into the Attribute Name field, leave the other fields as they are then click OK

7) Click OK again

8) Click on the Management Agents button at the top. You should all agents listed, in my there were 4:

   a) ILMMA: the FIM service mgmt agent

   b) MOSSAD_Global: the Active Directory Domain Services agent

   c) MOSSBDC-People Image BDC: my corporate BDC connection

   d) MOSS-[someGUID]: the SharePoint Extensible Connectivity agent


9) Double click the MOSS-[someGUID] management agent

10) The properties dialog box will open, select Configure Attributes on the left menu bar

11) This will list all attributes in the user profile store. Check the list for an attribute called “PictureURL”. If it does not exist then:

   a) click the New button

   b) Enter “PictureURL” (case-sensitive) in the Name field, leave other fields as they are and click OK

12) Check that the PictureURL attribute now appears in the attribute list

13) Now select Define Object Types on the left menu bar

14) Select “user” in the Object Types list in the middle. The attributes for “user” should appear in the right pane

15) Click the Edit button:

16) Select our new attribute called PictureURL from the Available Attributes list on the left

17) Now click the lower Add button to add the attribute to the 'May have attributes' list on the right

18) Check the PictureURL attribute appears correctly on the right:

19) Click OK

20) Select Configure Attribute Flow from the left menu bar

21) From the Data Source Attribute list on the left select PictureURL

22) From the Metaverse attribute list on the right select “SPS_MV_String_PictureURL

23) Click the New button to add this new attribute flow. NOTE: do not simply click OK here as it will not add the attribute flow we just selected, you must click New to confirm it

24) Check that the attribute flow was added correctly by expanding the “Object Type: user” section at the top of the dialog box. You will have to scroll down to see the mapping we just created

25) Now click OK to close the Properties dialog box

26) Now double click the management agent called “MOSSBDC-Profile_Image_BDC” (or whatever you BDC connection agent is called) to open the properties dialog box

27) Select Configure Attribute Flow from the left menu bar

28) From the Data Source Attribute list on the left select “ImageURL” (or whatever your BDC attribute is called)

29) From the Metaverse attribute list on the right select “SPS_MV_String_PictureURL

30) Click the New button to add this new attribute flow. NOTE: again, do not simply click OK here as it will not add the attribute flow we just selected, you must click New to confirm it.

31) Finally, check that the attribute flow was added correctly by expanding the “Object Type: PeopleImageEntity” (or whatever your BDC entity class is called) section at the top of the dialog box

32) Click OK to close the properties dialog box

33) Now run a full crawl from the User Profile Service in Central Admin

That’s it!

Good Luck!