SharePoint @ MSANALYSTS.COM

Architecting a Solution – Best Practice Web Applications Configuration for Content Deployment

2010-10-12T14:53:00.001-04:00

Overview

Since SharePoint 2007, content deployment has been part of SharePoint. Its not a commonly used feature in most collaboration farms, but its essential for web publishing farms. Makes the tasks of moving data from Development to Staging to Production an automated process.

As automated processes go, its not bullet proof and the larger and more complex your site gets the more likely you will have to republish the full site to correct errors. Depending on your infrastructure and the size of the site, this could take a long time and your site is essentially down during the process. To keep your site up you should configure two Web Applications in production for content publishing: a Primary and a Secondary. The Secondary should not be accessible on the internet. Each of these Web Applications will have its own content Database and its own Site Collection. Follow the steps below if you need to fully republish the website with quick turn around.

Steps

1. Create a new blank site collection on the Secondary Web Application (to the Secondary Database)

2. Pause all incremental publishing to the Primary Web Application

3. Publish the site to the Secondary Web Application

4. Disconnect the database from the Secondary Web Application

5. Disconnect the database from the Primary Web Application

6. Connect the Secondary Database to the Primary Web Application (Now the Switch is complete)

7. Resume incremental publishing to the Primary Web Application

Remember that the site will only be down while you disconnect and reconnect the databases in step 5 and 6. I recommend that you run a full crawl from your index server to make sure your indexes are fully up to date.

Diagnosing the Problem - Troubleshooting Search Issues

2010-10-01T11:26:00.002-04:00

Overview

When I run into issues searching a particular site, there are a few steps that I go through every time.

Steps:

1. Go to Search Administration check the crawl log you should be able to find the url and from here you will quickly be able to diagnose most access is denied or site is unavailable type of issues.

2. If you can't find the url in the crawl log or the error is not apparent, create a new content source with only the site you are having difficulty with, monitor the results of the full crawl, if necessary turn up logging on the SharePoint.

3. If your SharePoint is heavily customized, check the osssearchresults.aspx page for modification from default. If the page is modified, copy the original back in place to test the search results. If there is not an error the fault is within the customization.

Troubleshooting the Timer Service - Resetting the Timer Service Cache

2010-09-24T13:00:00.002-04:00

Overview

Occasionally I run into problems with the SharePoint 2007 timer service and one of the tricks I have in my bag is to reset the timer service cache on the server with the issue.

Steps:

Start with the server that is running the Central Administration site (complete all steps then proceed to the next server in the farm and repeat)
Stop the Timer service, To do this, follow these steps:

a. Click Start, point to Administrative Tools, and then click Services.
b. Right-click Windows SharePoint Services Timer, and then click Stop.
c. Close the Services console.

3. click Start, click Run, type explorer, and then press ENTER.

4. In Windows Explorer, locate and then double-click the following folder:
Drive:\Documents and Settings\All Users\Application Data\Microsoft\SharePoint\Config\GUID

5. Back up the Cache.ini file.

6. Delete all the XML configuration files in the GUID folder. Do this so that you can verify that the GUID folder is replaced by new XML configuration files when the cache is rebuilt.

Note When you empty the configuration cache in the GUID folder, make sure that you do not delete the GUID folder and the Cache.ini file that is located in the GUID folder.

7. Double-click the Cache.ini file.

On the Edit menu, click Select All.
On the Edit menu, click Delete.
Type 1, and then click Save on the File menu.
On the File menu, click Exit.

8. Start the Timer service. To do this, follow these steps:
a. Click Start, point to Administrative Tools, and then click Services.
b. Right-click Windows SharePoint Services Timer, and then click Start.
c. Close the Services console.

Note The file system cache is re-created after you perform this procedure. Make sure that you perform this procedure on all servers in the server farm.

9. Make sure that the Cache.ini file in the GUID folder now contains its previous value. For example, make sure that the value of the Cache.ini file is not 1.

10. Click Start, point to Programs, point to Administrative Tools, and then click SharePoint 3.0 Central Administration.

11. Click the Operations tab, and then click Timer job status under Global Configuration.
In the list of timer jobs, verify that the status of the Config Refresh entry is Succeeded.

SharePoint 2010 – Datasheet View not working with Office 2010

2010-06-07T13:09:00.001-04:00

Overview

The 64bit version of office 2010 leaves us a bit wanting. For those users that uninstall Office 2007 and install the new 64 bit addition, a host of feature will not work without the addition of the 2007 Office System Driver Data Connectivity components.

Here are the details on what is supported:

http://technet.microsoft.com/en-us/library/cc263526.aspx

“Datasheet view

Requires a 64-bit ActiveX control. Microsoft Office 2010 does not provide a 64-bit version of this control.”

Resolution

Download and install the 2007 Office System Driver: Data Connectivity Components

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=7554f536-8c28-4598-9b72-ef94e038c891

SharePoint 2010 – Account security inconsistencies

2010-05-20T06:21:00.000-04:00

Overview

I think someone was taking a nap when they worked out the least privileged access for SharePoint 2010.

Here are the permissions for the farm service account. You will notice that local administrative access is not listed (which is a secure design):

http://technet.microsoft.com/en-us/library/cc678863.aspx

Farm service account

The server farm account is also referred to as the database access account and is used as the application pool identity for Central Administration, and as the process account for the Microsoft SharePoint Foundation 2010 Timer service. The server farm account requires the following permissions:

It must have domain user account permissions.

Additional permissions are automatically granted to the server farm account on Web servers and application servers that are joined to a server farm.

After you run the SharePoint Configuration Wizard, machine-level permissions include:

Membership in the WSS_ADMIN_WPG Windows security group for the SharePoint Foundation 2010 Timer service.
Membership in WSS_RESTRICTED_WPG for the Central Administration and Timer service application pools.
Membership in WSS_WPG for the Central Administration application pool.

After you run the configuration wizards, SQL Server and database permissions include:

Dbcreator fixed server role.
Securityadmin fixed server role.
db_owner for all SharePoint Server 2010 databases.
Membership in the WSS_CONTENT_APPLICATION_POOLS role for the SharePoint Server 2010 server farm configuration database.
Membership in the WSS_CONTENT_APPLICATION_POOLS role for the SharePoint Server 2010 SharePoint_Admin content database.

Permissions necessary to run the Profile Synchronization service:

http://technet.microsoft.com/en-us/library/ee721049.aspx

To create a new Profile Synchronization connection

Verify that you have the following administrative credentials:
- The Server Farm account, which is created during the SharePoint farm setup, must also be a member of the Administrators group on the server where the User Profile Synchronization service is deployed. For more information, see Account permissions and security settings (SharePoint Server 2010).

All of this I would forgive as a slight miscommunication if there wasn’t also a rule in the Health Checker which throws up a yellow flag:

Because the farm account is also used to host the central administration application pool.

Possible Solution:

It may be that we should change this service account to a separate account that is both a local administrator on SharePoint servers this could be done through CA:

This would have to be tested pretty thoroughly because the documentation is very specific to use the “Server Farm account” if this doesn’t work, the User Synchronization service should never be placed on the server with Central Administration. Typically in our 6 server redundant design, that means placing the User Synchronization service on the WFE servers.

SharePoint 2010 – Opening Port in the firewall for the Document Conversion Load Balancer service

2010-05-13T20:47:00.000-04:00

Overview

If you receive the error below, your Document Conversions Launcher service cannot reach your Load Balancer service, very likely because you haven’t opened a port in the firewall. These errors stack up pretty quickly in the application event log, you will probably notice one every few minutes.

Critical Error 7933

"The Office HTML Viewer Service could not be registered with Office HTML Load Balancing Service."

Procedure:

On the server that has the Load Balancing service, go Control Panel –> Windows Firewall –> Advanced Settings

Under Inbound rules create a rule to allow connection to the Load Balancing Service. Below are the screen shots with the default port: 8093.

SharePoint Antivirus Hooks

2010-05-10T06:23:00.000-04:00

Overview

Occasionally I run into the antivirus-gone-bad scenario. AV Services stop or hang, and can’t be restarted. Most of the time this is caused by a bad antivirus database update, but recently a set of Windows patches left me hanging in a state where user’s could not upload files into the SharePoint site.

The options for antivirus haven’t really changed in the last 7 years. Here is where you can find how SharePoint hooks into AV and disable it if necessary. After making this type of farm change, all servers on the farm are updated, so the rest of your SharePoint services should be started (especially the timer service).

If cannot upload files from an AV failure, you can temporarily unhook it from Central Administration:

SharePoint 2003 From Central Administration:

SharePoint 2007 From Central Administration then Operations:

SharePoint 2010 Central Administration and then Security:

SQL Server – Using SQL Express ADV Reporting Services on a 64bit server

2010-03-03T21:57:00.001-05:00

Overview

Today I was asked to help out setting up Reporting Services for an OCS deployment on a 64bit system with the free express addition. One of the things that I found out during testing is that the free addition is a 32bit operation. In order to run the Reporting Services on a 64bit system we had to:

Enable IIS to run 32bit asp.net 2.0
Register the 32bit version and mark it as allowed to run

We used two separate articles complete the process:

1. http://msdn.microsoft.com/en-us/library/ms143293(SQL.90).aspx

Complete steps in “To install 32-bit Reporting Services on 64-bit IIS 6.0”
Registering asp.net ASP.NET V2.0.50727 (32-bit) needs to be completed before starting step 11, see the link below.

2. http://support.microsoft.com/kb/894435 (How to switch between the 32-bit versions of ASP.NET 1.1 and the 64-bit version of ASP.NET 2.0 on a 64-bit version of Windows”

Run %SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis.exe –i
Mark ASP.NET V2.0.50727 (32-bit) as allowed in the Web Server Extensions folder

Solving SharePoint Issues – WSS 3.0 Indexing Error

2010-02-08T12:51:00.001-05:00

Overview

I spend more time with MOSS Search than WSS, but recently we had to reset the indexes on a WSS 3.0 farm of considerable size. After the service was stopped and started, the new database was successfully created and the full crawl command was executed the engineer called me and told me that there was no indexing activity. In the event logs she noticed:

Event ID 2423

Since there are no crawl rules in an SSP as this is WSS 3.0, I immediately recalled that we needed to set the index server for each content database.

Steps:

1. Find your content databases under Application Management:

2. Select the database

3. Select an Index Server

Solving SharePoint Issues – Search Results errors

2010-01-29T14:03:00.001-05:00

Overview

Sometimes you don’t find what you expect in your search results even though search appears to be working okay for other lists, document libraries and sites. If you are a site collection administrator or even a site administrator, you may have little access to Search Administration, but there are still a few things that you can do.

1. As a Site Collection administrator you should be aware of how often your documents are crawled. If its not published, find out. Users will want to know. Users always want to know! ;-)

2. Try marking the list or document library as not to be indexed. Let the indexing process run for a full or incremental crawl search the list and make sure none of the items return. Uncheck the box to return items to the index and then do a search to verify results.

To enable or disable indexing:

Go to the Document Library or List general settings page by licking on the Settings options form the menu bar:

From General Settings, Click on the Advanced Settings link:

Scroll Down to the bottom until you find Search:

Solving SharePoint Issues – Restoring a farm results in an invalid location

2010-01-27T09:22:00.001-05:00

Overview

I have used the backup feature in SharePoint on and off for a few customers for several years. Perhaps fortunately, I have not had to use this tool for a restore. Most of the time I am migrating data into new environments or new versions. For one of our customers, an engineer ran into an error message during the restore process. The log was quite detailed, just like the backup logs are, and during the database restore he received a generous block of text that amounted to these two statements:

(The system cannot find the path specified.)
Restore Database is terminating abnormally

I was able to solve this one, because I recalled that SQL backup/restore process needs to use a local path (ie c:\etc\etc) not UNC path. you can access the restore from backup option from the Operations page in Central Admin:

SharePoint 2010 – Central Administration feels like a Jumble

2010-01-23T19:32:00.001-05:00

Typically I don’t care to complain much, and I am late to the game for any complains to have a real impact, but I will say that Central Admin for SharePoint 2010 feels like a jumble. The horizontal organization seems to run together in smaller resolutions making the options difficult to read. As usual a search for AAM (alternate access mapping which was under operations and is now under system settings) seems like incorrect categorization for this configuration.

If I am making a change specific to a web application, why isn’t it in application management. Sigh.

I am not sure how we went from 3 relatively well organized pages to 8 difficult to read pages without adding on a significant amount of administrative options, but we did.

Solving SharePoint Issues – Logging not Working

2010-01-20T15:14:00.001-05:00

Overview

I saw a message from one of the engineers I work with that SharePoint logging was not working on a server. They investigated the problem for quite a while, restarted services, the server, etc and found out later that there was a space added to the log file directory in Central Admin. This one is worth further investigation, when I get a chance, I will try to grab some screen shots.

Solving SharePoint Issues – Error 6641 after a service account or password change

2010-01-19T10:05:00.001-05:00

Overview

There are several important steps to changing the service account and/or password on a SharePoint farm. If you are looking at this article you probably:

Changed the service account and password for the SharePoint Services
May have changed the service account and password for the web application pools
Most likely did not read http://support.microsoft.com/kb/934838

Before changing the password, read the article. If you have a simple SharePoint implementation that utilizes a single username and password for all services, you can copy the script in the article and run the batch file to change the username and password. Otherwise note down these two commands:

stsadm -o updatefarmcredentials -userlogin DomainName\UserName -password NewPassword
stsadm.exe -o editssp -title SharedServicesProviderName -ssplogin DomainName\UserName -ssppassword NewPassword

You need to run them from the command line and they are probably the ones that you missed giving you this error message (and perhaps several others)

Error Message:

Architecting a Solution – Planning for Availability (Uptime) SLA

2009-12-29T14:50:00.001-05:00

Overview

There are quite a few different technologies that you can use when designing a solution to meet an Availability SLA. In this post, I will define the highest SLA that my team can offer per the architecture.

One important point is to realize that with any architecture there are two basics to remember with SharePoint:

1. Patching should be done quarterly or semi-annually and patches must be tested thoroughly

2. A change control policy most govern software installations and configuration changes in the environment

The Single Server is the most basic architecture, and it encapsulates all roles. The architecture is scaled out for Availability, Performance, and Recovery reasons.

Availability	Architecture	Redundancy	Explanation
99.00%	Single Server Implementation	Can have redundant Hard Drives, Memory, Processors.	Single Point of Hardware Failure
+0.9%	Utilize a Virtual Environment with High Availability	Full hardware redundancy	Additional Complexity and Requires Engineering Skills
+0.9%	Maintain identical blade or server hardware and utilize SAN volumes	Full hardware redundancy	Additional Complexity and Requires Engineering Skills

The Two Server Implementation is common for additional Performance, however it creates multiple points of failure which overall lowers Availability.

98.00%	Two Server Implementation	Each server can have redundant Hard Drives, Memory and Processors	Multiple Points of Hardware Failure and network Infrastructure failure
+1%	Utilize a Virtual Environment with High Availability	Full hardware redundancy	Additional Complexity and Requires Engineering Skills
+1%	Maintain identical blade infrastructure and utilize all SAN volumes	Full hardware redundancy	Additional Complexity and Requires Engineering Skills

To compensate for a loss of Availability from multiple points of hardware failure, additional hardware is added to create fully redundant SharePoint Farm which requires yet more engineering skills as it increased the complexity of the environment to build and support.

99.90%	Farm with Redundant Roles	Full hardware redundancy	Complex implementation requires knowledge to support both clustering and network load balancing
+0.09%	Utilize a Virtual Environment with High Availability	Full hardware redundancy	Additional Complexity and Engineering Skills
+0.09%	Maintain identical blade infrastructure and utilize all SAN volumes	Full hardware redundancy	Additional Complexity and Engineering Skills

Taking it to the 5 9s. As a service provider, I can increase uptime by having a second set of hardware in a completely different datacenter prepared to receive connections with different mirroring technologies. Microsoft has had SQL Mirroring for a while, but until SharePoint 2010, this was not supported within the product and manual processes take time that a 99.999% Availability SLA just does not require. Essentially SQL Mirroring becomes a Business Continuity (Disaster Recovery) mechanism.

I haven’t had any propositions for this level SLA, but in theory, a Cisco or F5 datacenter load balancer, with Left Hand GeoClustering technology could accomplish a multi-datacenter solution (given a few special procedures and policies) that would allow SharePoint to achieve this kind of SLA.

Configure SharePoint – How SLA’s affect Architecture

2009-12-28T10:47:00.001-05:00

Overview

CFO magazine had an interesting article about SAAS. In the article, 23% of Software as a Service offerings did not have contractual agreements requiring Service Levels. The number surprised me because it was so large, and as an architect I design the SharePoint based on the SLA that I have to meet. Could it be that 23% of all installations are not designed to meet a service level?

There are myriad complexities about designing a system to meet an SLA for corporation. The most minimal SLA is usually Availability (Uptime), followed by Performance, and then typically Recovery.

Availability (Uptime)

Uptime requirements change the Architecture to include more High Availability components. This increases the complexity of the installation, if the installation becomes too complex it may be prone to failure so its a careful balance of available technologies, engineering skills, etc. Mirroring, Clustering, Network Load Balancing, Virtualization, Redundant Services and Servers are all part of increasing Uptime.

Performance

Performance is a measure that requires calculation of Users, Web Servers, Spindles, SQL Servers,latency, bandwidth and most importantly a Usage Policy. Better performance can be as simple as adding more hardware, or applying better policies with lower upload limits, smaller site collections, and fewer features.

Recovery

Recovery includes Maximum Time to recover data and the Maximum Time to recover from a disaster. Recovery SLAs can affect spindle counts on SQL servers, network backup and tape drives, data center mirroring, etc.

In the next several articles I will describe detailed configurations of how these SLAs affect overall architecture. Take a look at Microsoft’s Plan for Availability article: http://technet.microsoft.com/en-us/library/cc748824.aspx.

Configure SharePoint – Windows Server 2008 Requirements for MOSS 2007

2009-12-25T16:29:00.001-05:00

Overview

One of the engineers in my team was doing a new SharePoint 2007 farm for a large pharmaceutical and he was interested in the requirements and differences between SharePoint 2007 on Windows 2008/IIS7 and Windows 2003/IIS6.

Microsoft has a Windows Server 2008 Resource Center for SharePoint at:

http://technet.microsoft.com/en-us/windowsserver/sharepoint/bb735844.aspx

This site has a list of all the specific requirements and you should read it carefully, but the only real show stoppers are:

Install the Web Server Roles and ASP.net 3.0 or 3.51 (R2)
Install IIS 6 Compatibility
You must install WSS 3.0 and MOSS 2007 with SP1

SharePoint 2010 - Prerequisites

2009-12-03T09:56:00.001-05:00

Overview

Microsoft has already published excellent deployment documentation, including hardware and software Prerequisites for the installation. You can find that list here: http://technet.microsoft.com/en-us/library/cc262485(office.14).aspx. You will notice the software prerequisite list is quite long for a web stand alone or web front end server.

The 64-bit edition of Windows Server 2008 Standard with SP2

Web Server (IIS) role

Application Server role

Microsoft .NET Framework version 3.5 SP1

SQL Server 2008 Express with SP1

Microsoft "Geneva" Framework

Microsoft Sync Framework Runtime v1.0 (x64)

Microsoft Filter Pack 2.0

Microsoft Chart Controls for the Microsoft .NET Framework 3.5

Windows PowerShell 2.0 CTP3

SQL Server 2008 Native Client

Microsoft SQL Server 2008 Analysis Services ADOMD.NET

ADO.NET Data Services v1.5 CTP2

Microsoft SQL Server 2008 Cumulative Update package 2

KB Hotfix 971831 for Windows 2008

The Green items are part of Windows Server 2008. These are roles that you can install with little effort, all the necessary files should be on your system.

The Blue items are part of your SQL Server Installation. One disk and a few updates should satisfy those.

The Red items are the ones you have to worry about. A few of these you will recognize from SharePoint 2007 implementations and a few are new. Fortunately SharePoint 2010 comes with a prerequisites installer which makes this process a lot easier.

You have to handle those in Maroon yourself:

Powershell: Before you begin the installer make sure that you uninstall Powershell v1. If you do not uninstall Powershell 1.0 you will get a failure for every prerequisite that requires the 2.0 ctp.

The SQL Server cumulative update is on a request only basis so you will have to register with a valid email address.

KB Hotfix 971831 you can download directly.

Steps for the Prerequisites Installer:

Diagnosing the Problem - “Error updating a list Item”

2009-11-20T10:01:00.001-05:00

Overview

I was working on a site with a fairly simple but multi-part workflow that copied items between lists. The workflow was working fine for a certain group of people but failing for others. After a quick search I was shaking my head, because I should have seen this from the Created by and Modified by fields of the new items:

Workflows run in the context of the user that initiates them

So in summery, if you initiate a workflow from one list to create an item in another list the user must have the creation permission in both lists.

SharePoint 2010 – How will Database Mirroring support change our architecture?

2009-11-18T14:16:00.001-05:00

Overview

If you haven’t been reading up on your SharePoint 2010 deployment guides you are getting behind already. There are many cool new features to SharePoint 2010, but one worth discussion as it relates to architecture is built in database mirroring support. You can start reading up on the logical architecture here: http://technet.microsoft.com/en-us/library/cc263121(office.14).aspx#section7. With mirroring support we can start thinking of our high availability SharePoint without SQL Clustering.

Project Server – SharePoint Installation Options

2009-11-18T10:28:00.001-05:00

Overview

I have been working on and off with Project server for the last few years. Most of the work was with Project server 2003 which was a pain to setup and integrate with SharePoint. Project Server 2007 improved that integration by making project server an application built on WSS 3.0. I am sure this will continue with Project Server 2010, although I haven’t even touched on that one yet.

You can deploy project server in stand alone or farm configurations with WSS 3.0 or you can deploy project server with MOSS as

A Stand Alone server configuration
To an existing MOSS Farm

MOSS can be installed on a farm after Project Server the same way a WSS 3.0 farm can be upgraded to MOSS. Microsoft has the whole process well documented at http://technet.microsoft.com/en-us/library/cc303399.aspx.

Project Application Roles to be started on your Application Server:

Project Server Configuration accessed from your SSP:

Picture of the default Project Web Access Site:

Coding – Setting up your personal development environment

2009-11-16T21:54:00.001-05:00

Overview

Setting up a non-virtualized development environment on a laptop or desktop is a preferred practice when you have many developers and few development servers. An extensive virtual environment may not be available and your developers may spend some of their time working offline.

If you are familiar with SharePoint, you know that this can be something of a challenge because SharePoint will not install on a Vista or Windows 7 computer without some help. In addition to SharePoint there are quite a few development tools I would not be without. (Every time I look in one of our environments, I see someone else has loaded a bunch of programs so I know they have the same feelings!) I will make a list of my personal favs later. Suffice it to say at minimum for Development you need:

Visual Studio 2008
WSS 3.0
Visual Studio 2008 extensions for Windows SharePoint Services 1.3 (minimum for x64)

Recommended Environment is Windows 7 x64 but this works fine on almost any environment with the help of Bamboo SharePoint Installer.

1. Install Visual Studio 2008 as you normally would.

2. Download WSS 3.0 from Microsoft.

3. Download the Bamboo SharePoint Installer helper at http://community.bamboosolutions.com/blogs/bambooteamblog/archive/2008/05/21/how-to-install-windows-sharepoint-services-3-0-sp1-on-vista-x64-x86.aspx

4. Install SharePoint

Bamboo has excellent documentation there. Read it. I won’t bother to back over what’s already there ;-) It’s great as a development environment, but there really should be a server with a similar to production build for testing and staging. If you are sharing your development environment, I recommend creating a procedure for building and testing on the personal pc before installing the wsp on shared development.

Find and Run the SetupLauncher.exe and select your installation files.

5. Install Visual Studio Extensions 1.3 for SharePoint

Some Known Issues I ran into:

If you get an error message saying that the installation ended prematurely, make sure that you install iis 6 metabase compatibility.
If you get an error running the “SharePoint Solution Generator 2008” or deploying webparts to your local SharePoint, make sure that you have Windows Communication Foundation HTTP Activation selected from your .NET 3.5.1 features.

Diagnosing the Problem – SSP provisioning fails with error “User cannot be Found”

2009-11-11T10:06:00.001-05:00

Overview

During the SSP provisioning process the primary site collection administrator from the Central Administration web site is copied into the new SSP as the site collection administrator. I perceive several reasons why this operation could result in a “User cannot be Found” error:

The user account has been removed from Active Directory
The user account is local and has been removed from the local computer
The user account cannot be resolved from Active Directory

You can resolve this provisioning error by changing the site collection administrator in Central Administration. Immediately after the site collection administrator issue is resolved the provisioning will complete if you do not see any changes the condition has not been resolved.

From Central Administration

1. Click on Application Management:

2. Click on SharePoint Site Management

3. Select the Central Administration Web Application and Change the Site Collection Administrator

Diagnosing the Problem – Corrupted Databases and Orphaned Sites

2009-11-02T10:15:00.001-05:00

Overview

I ran into an issue the other day where the target site was corrupted during a publishing attempt. The target was a newly created blank site, and after the publishing attempt the target site was inaccessible from the Web Browser, Central Admin and the command line. We tried to delete the target from all three interfaces without success. We could not create a new site or delete the old site without an error message that said something like “cannot process the request”.

Steps to Resolution

To resolve this issue from the command line examine the database with STSADM database repair command. For the command you will need both the database name and the url. If you don’t keep track of which URLs are in which database you should. You can search the Webs table in your content databases for the site info, but preparing a good report ahead of time is always welcome. After the site is corrupted you may not be able to access this information from Central Administration.

When you run the stsadm –o databaserepair command (http://technet.microsoft.com/en-us/library/cc263282.aspx) you should see output like this. Each orphaned item in the site collection is reported on.

Use the –DeleteCorruption parameter to fix the database and then remove and readd the content database in SharePoint to resolve the issue.

Giving Power Users the Tools – How is my Site Connected to the Portal?

2009-10-30T10:24:00.001-04:00

Overview

Depending on how many farms and portals you have in your environment, Site Collection Administrators may want to know and have a choice of portal sites.

In each site collection there is a link to the Collaboration Portal that you can set with the following instructions:

From your Site Collection, Click on Site Actions

Click on Site Settings
Under Site Collection Administration Click “Portal site connection”

Portal Web Address: Your Portal Web Address
Portal Name: Your Portal Name

Configure SharePoint – Application Pool configuration

2009-10-29T10:31:00.001-04:00

Overview

The default application pool configuration will work for most deployments, but among other configuration it has an arbitrary single scheduled recycle, no physical or virtual memory limit, and no CPU limit. At this point you should be familiar with IIS 7.0 if you aren’t you better start working on it right now ;-). Below are several considerations for controlling the performance of your application pool.

Recommendations:

Recycle Schedule

Recycle once a night at least 30 minutes before peak traffic starts.
Always stagger the recycle schedule of your servers
Utilize warm up scripts to bring your application pool up to speed quickly.

Considerations:

SharePoint servers have been around long enough that enterprises are crowding boxes with multiple web applications and application pools. When you consider your memory and cpu allocation consider the roles the server is running.
For any x86 boxes, Microsoft best practices has specific recommendations that fit a WFE deployment in a medium farm with a single web application. http://technet.microsoft.com/en-us/library/cc850694.aspx
In a x86 all-in-one configuration, allocate a minimum of 1 GB to the operating system and 1 GB for SQL to stay within Microsoft Best Practices you should allocate 1GB to the SSP application pool and 1 GB to the Team Site or Portal application pool.
For x86 corporate WFE servers you can usually squirrel enough memory away for 3 applications pools.
In an x64 environment with lots of memory, the application pool physical memory may be increased considerably or not limited at all, but remember that virtual memory is paged to disk and requires separate spindles configured in Raid 1 or Raid 10 for performance.
I have run into a number of faulty SharePoint applications that did not dispose of objects or that required the SharePoint farm to be recycled more often. In a corporate environment you don’t always have control over the applications that are deployed…even after identifying the problem. Business requirements may indicate that you have to recycle your application pools more frequently.

SQL Server –Setup MSDTC Resource

2009-10-28T13:22:00.001-04:00

Overview

My favorite articles on clustering mention setting up the MSDTC resource but not in detail. If you have requested the resources, I recommend using a separate Group for MSDTC (and with 2008 clustering you will not be able to add it to the quorum group). Make sure you set this up before launching the SQL Server Installation for your Windows 2003 and SQL 2005 Cluster. I expect this will be one of the last view blog entries I have about clustering that concern these old outdated versions ;) but we will see.

(http://www.sql-server-performance.com/articles/clustering/cluster_server_2003_p1.aspx – To setup the Windows 2003 Cluster and http://www.sql-server-performance.com/articles/clustering/cluster_sql_server_2005_p1.aspx – To setup the SQL 2005 Cluster)

MSDTC Steps:

Before installing SQL you need to setup a clustered instance of MSDTC. To do this you will need to:

1. Create a new Group called MSDTC (and the name doesn’t matter, you could call it Bob)

2. Add Resources to the Group. For MSDTC you will need a Disk Resource, a Network Name, a Network IP address. Once these are added you can setup the Distributed Transaction Coordinator resource.

The end result should look like this:

SQL Server – AD configuration of the Computer Account When using Kerberos and a SQL Cluster

2009-10-27T09:59:00.001-04:00

Overview

I use two good sets of step by step instructions for my Windows 2003, SQL 2005 Clusters:

http://www.sql-server-performance.com/articles/clustering/cluster_server_2003_p1.aspx – To setup the Windows 2003 Cluster and

http://www.sql-server-performance.com/articles/clustering/cluster_sql_server_2005_p1.aspx – To setup the SQL 2005 Cluster

Both of these documents are excellent step by step directions for SQL Cluster building. On top of that if you are deploying SharePoint with Kerberos Authentication you have to set SPN for SharePoint and the SQL Server. You also have to create a computer account for the clustered SQL Server in Active Directory and grant the SQL Service Account and the Cluster Service Account permission to the Active Directory Computer Account Object. Here are the exact details: http://technet.microsoft.com/en-us/library/cc738070(WS.10).aspx

Granting permissions to the computer object:

1. Open Active Directory Users and Computers from Administrative Tools

2. Enable advanced features

3. Go to the properties of the computer account

4. Open security and add SharePoint Service Account and SQL Service Account to full control of the computer object

5. Open Managed By and click Change and select the Cluster Service Account.

Diagnosing the Problem – User Alerts affect database size through the Change Log

2009-10-23T14:51:00.001-04:00

Overview

As user alerts are processed several database tables (eventlog and eventcache) can become quite large because of the data retention period of the Change Log. If you find these tables have become unsustainably large, edit your change log retention period by going to:

1. Click on Central Administration

2. Click on Application Management

3. Click on Web Application general Settings

4. Select the web application

5. Scroll down to the Change Log and lower the number of days

6. A timer job will run and delete the extra entries, monitor for the process to complete.

Migrate SharePoint – Setting Databases to Read Only

2009-10-21T09:25:00.001-04:00

Overview

During a migration you may want to set the original databases to read-only especially if you are moving to completely new hardware. You can do this through a number of methods and the one I see most often in articles is using the file system, but I prefer using SQL Management Studio. When databases are set to read only users will not be able to add any new data to SharePoint. Microsoft has a KB article that describes the error messages that you will receive when you attempt specific operations: http://support.microsoft.com/kb/894631. Sites that allow anonymous or NT Authenticated users may have difficulties when new users attempt to logon.

Steps to set your databases as read only in SQL 2008.

1. Right click on the database and then click on Properties

2. Click on Options from the left sidebar:

3. Scroll Down until you find the “State” section. From here you can set your database to Database Read-Only equals True. When you do this all users will be disconnected for a moment.

Giving Power Users the Tools – Connecting to SharePoint Document Libraries with the iPhone

2009-10-21T08:34:00.001-04:00

Overview

People like to carry their documents with them wherever and the iPhone is a great tool for this. In many ways much more powerful that Windows Mobile. First of all I always recommend setting up a VPN from the iPhone to your corporate network before viewing any files, and without doubt a security policy should be placed on the device.

SharePoint uses WebDav protocol for Document Libraries so an application that can take advantage of this can be used with SharePoint. One such application I found in the Appstore is ReaddleDocs by Readdle (http://readdle.com/products/readdledocs). The software is 4.99 on the appstore.

Find and Install Readdle Docs in the Appstore

Open Readdle Docs and touch the online button. Look for the “Other Server” option that will allow you to connect to any WebDav supported online storage:

Enter the URL to the SharePoint Document library and the login information and you will have a direct connection to your SharePoint document library.

Diagnosing the problem – User cannot logon after a migration

2009-10-20T15:08:00.001-04:00

Overview

During a migration, you may get the scenario where the user account has not been migrated fully or where both accounts have been added to the same site collection.

In this case simple remove both user names from the site collection and re-add the correct username. This can be onerous if complex permissions are involved.

Steps:

Go to Site Settings –> People and Groups –> All People.
Remove the Duplicate Users.
Add the correct username.

Diagnosing the Problem – Searching for the missing feature during a migration

2009-10-15T11:01:00.000-04:00

Overview

I have seen some code out there for enumerating features which is great. If you are missing a feature during a migration and all you have is the Feature ID you can search the Features folder in the originating 12 hive for the Feature ID in the .xml files. This will take you to the folder the feature was installed from.

Error:

Diagnostic:

Search the Feature folder in the 12 hive for the Feature for “75a0fea7-775b-4ae1-bbc6-ad5ef0d22413” in an XML file. Make sure that you are search for the ID in the contents of the xml.

Diagnosing the Problem – 403 Errors during Migrations

2009-10-13T08:46:00.001-04:00

Overview

While migrating databases to new farms and new web applications, you may run into 403 errors when browsing the sites. A 403 error is basically indicating that the site is forbidden, usually because of an authentication failure.

Potential Solution:

Check out your Authentication Providers underneath the Central Admin Operations.

Drill into the details of the web application that you have created and are migrating too. You should have at least the default zone.

Check your authentication mechanism and make sure that its consistent with your design, Central Administration and your SSP.

Diagnosing the Problem – SSP Provisioning Errors

2009-09-29T11:31:00.000-04:00

Overview

I have been setting up my new 2008 R2 environment the last month so I haven’t been as busy with my blog. The new test environment is to mimic both internal and external access in a dual SharePoint farm setup. While setting up my SSPs I was being quite modern and using the username@domain.name for my service accounts when I ran into a provisioning error.

Error

"Provisioning failed the specified account name is invalid. Parameter name: account"

It took me about a half an hour to figure out what went wrong. The good news is you don’t need to resetup an SSP to fix this error, go to Operations –> Security Configuration –> Service Accounts:

Change the username and password of your web applications for the SSP and MySite to domainname\username format. After a minute the site should provision.

It’s a little sad that SharePoint is not ready for the username@domain.name style authentication.

Diagnosing the Problem – Wizard Cannot Connect to Database

2009-09-17T09:29:00.000-04:00

Overview

I have run into this one a few times as I try to do installations that are least privileged. You setup a new Web Front End Server and you run the configuration wizard. You checked your network connectivity, made sure you were at the right update level and you can find the configuration database in the wizard but when you complete the wizard you get an error like this:

Solution

When you run the wizard make sure that you login to the new Web Front end an administrator who has access to the Database server. In fine print the Microsoft deployment documentation says that the farm administrator account should have local administrative access during the installation, use that account and you won’t have any difficulty.

When you run the wizard again the server may thing that its already connected to the farm. There is no reason to disconnect just run the wizard through to successful completion by clicking next.

Configure SharePoint – SharePoint Installation With or Without AD?

2009-09-16T11:22:00.000-04:00

Overview

I have run into a handful of straggler installations that don’t use Active Directory. These are usually one-off servers that have another authentication mechanism, but a few days ago someone posited the question to me if SharePoint could be installed in typical farm configuration without AD.

I didn’t see how this was possible, you can install everything (All Roles and SQL) on one server easily. If you use SQL Authentication you could probably do two servers, but how would Search Query role authenticate with your Index Server? I started searching around on the internet to see what the official “Word” on Without AD Installations was.

Both these blogs outline the process for using SQL Authentication:

http://blogs.msdn.com/fooshen/archive/2007/02/01/installing-moss-without-ad.aspx

http://www.jeremytaylor.net/2009/04/24/sharepoint-installed-without-active-directory/

http://josephmdavis.com/sharepoint-without-ad-and-clean-database-names-and-fba.aspx(Added the Steps for Search Configuration)

For the most part they seem to be working with a One SharePoint Server, One SQL Server topology.

The Consensus

I found a couple of posts where supposedly Microsoft Engineers said they supported the configuration, and searching through Microsoft documentation I found documentation that indicated a SharePoint Services Farm could run with local accounts:

“Windows SharePoint Services 3.0 supports SQL Server authentication and local computer process accounts for farms that are not running Active Directory. For example, you can implement local accounts by using identical user names and passwords across all servers within a farm.” (http://technet.microsoft.com/en-us/library/cc288475.aspx)

There is also this document which would seem to say no:

“Windows SharePoint Services 3.0 requires Active Directory directory services for farm deployments. Therefore Windows SharePoint Services 3.0 cannot be installed in a farm on a Windows NT 4.0 domain.” (http://technet.microsoft.com/en-us/library/cc288751.aspx#section3)

In Addition all of the standard deployment guides for MOSS seem to be built around a domain user account model…

“The user account that you specify for this service account must be a domain user account. Because this account does not require a high level privilege, we recommend that you follow the principle of least privilege, and specify a user account that is not a member of the Administrators group on your Web servers or your back-end servers.”http://technet.microsoft.com/en-us/library/cc262079.aspx

In enterprise deployments, you can’t setup a configuration that Microsoft as a company won’t stand behind. Microsoft has published an article on Designing Extranet Farm Topology and Internet Design Topology. All of these architectural documents have Perimeter and DMZ networks that include Domain Controllers. (http://technet.microsoft.com/en-us/library/cc262757.aspx) Microsoft should publish clear documentation on these features, and until they do my answer to any farm installations without an Active Directory is a firm NO.

Diagnosing the Problem – Troubleshooting Web Front End Issues

2009-09-13T08:10:00.000-04:00

Overview

Most of SharePoint farms I work on have several Web Front End servers. After I do an installation all these servers are identical, but sometimes along the way I will find one web server is malfunctioning. There can be several types of errors and it does not have to be a complete failure… i.e. most sites will work on all the Web Front End servers perfectly. However it is possible for Web Front End server to have issues like:

Page Cannot be displayed
Web Part Errors
InfoPath Form errors
Security errors 403.x
Fill in your error here

What to do

In these cases, there are some practical methods and tools that you can use to get the Web Front End working properly. The first step is to Isolate the Problem. Make sure that you are browsing on just that server so by whatever method suits you, stop browsing the load balanced group and focus in on that server (extend the web application, open up a new port, etc). Once the problem has been isolated you can start working on identifying the solution.

5 Tools to fix Web Front End Issues

Run the SharePoint Products and Technologies Wizard

One of the easiest things to do and the first step should be to run the SharePoint Products and Technologies wizard. This tool takes care of all sorts of configuration on the server and having it complete correctly will tell you that the server is connected to the farm.

Run Windiff and verify your folders

You should verify your files and folders against a working server, run windiff on your 12 hive and iis directories, checkout differences in installed solutions and web.config and other supporting files.

Check Solutions in Central Admin

Make sure your solutions deployed successfully to all Web Front End Servers.

Check the Security Groups and User Accounts

Use a working WFE for reference and check the security groups and user accounts (particularly the SharePoint users and groups) IIS_WPG, WSS_ADMIN_WPG, WSS_RESTRICTED_WPG, and WSS_WPG and the farm administrator account.

Check the SharePoint Registry Keys

Use a working Web Front End as an example and compare the registry values.

Configure SharePoint – SQL Server Collation

2009-08-30T12:36:00.000-04:00

This one comes up every few months…the topic never goes away. The issue is whether you should change the default SQL Server Collation on SQL Builds when plan on installing SharePoint.

The default SQL Collation is sql_latin_general_cp1_ci_as. The CI stands for Case Insensitive. If you read the Microsoft document on SQL Server Preparation you will find that the “The SQL Server collation must be configured for case-insensitive.” http://technet.microsoft.com/en-us/library/cc263187.aspx

In the following line, Microsoft describes database configuration for SharePoint: “The SQL Server database collation must be configured for case-insensitive, accent-sensitive, Kana-sensitive, and width-sensitive. This is to ensure file name uniqueness consistent with the Windows operating system.”

THE Database configuration is automatically set to the right Collation when SharePoint Creates a database:

“Note SharePoint Portal Server 2003 and SharePoint Server 2007 will automatically create databases with the correct collation settings. Notice that the collation settings apply to the database level, but not to the SQL Server level.” http://support.microsoft.com/kb/843578

In short, I don’t care what the default server collation is as long as its case insensitive, but out of preference I would leave it to the default collation. This is the configuration that I have seen Microsoft choose in the products that have SharePoint WSS installation such as SBS and WSS.

A Quick Mention – My Top 5 Custom Features for New Sites

2009-08-24T10:14:00.001-04:00

Overview

I don’t spend all my time in site design, but I am responsible for a quite a few of SharePoint sites with Dashboards, Reports and Forms. Here are the Top 5 Custom Features that I use when designing a new website. Most of these can be found at Codeplex.

Top 5 Custom Features for New Sites

1. ChartPart (http://www.codeplex.com/ChartPart)

ChartPart inspired this blog entry. A lot of times I will user SQL reporting services, but sometimes you need something fast for quick KPIs and this is definitely the tool to add a nice graph to your site

2. The Up Folder button (http://blog.thekid.me.uk/archive/2008/11/03/adding-an-up-folder-button-to-a-sharepoint-list-view-webpart.aspx)

Especially Essential for working with large lists in data view mode. I utilize many folders for large lists and I heavily recommend that everyone utilizing SharePoint read the whitepaper on large lists http://technet.microsoft.com/en-us/library/cc262813.aspx

3. Unique Column Policy (part of SharePoint 2007 features http://features.codeplex.com/)

The Unique Column Policy Feature adds an option to Information Policies to require that one metadata column be unique. This is vital for certain lookup operations and great addition to SharePoint.

4. Tool Bar Manager (another great one that is included in SharePoint 2007 features on CodePlex http://features.codeplex.com/)

Remove misleading actions from the toolbar to help lock down a process.

5. Expand and Collapse all Groups at http://www.endusersharepoint.com/?p=1167 from Paul Grenier

I recommend following JQuery for Everyone, Quite a few awesome tools. Overall I have always been highly selective with adding code into Content Editor Web Parts. You have to maintain good documentation and development and staging environments to deploy these tools.

SharePoint SP2 – Pre Upgrade Checker is much more than preparing for SharePoint 2010

2009-08-18T23:10:00.000-04:00

Overview

The Pre Upgrade Checker is much more than just a preparation tool for 2010, its a tool that checks the overall health of your SharePoint environment with a variety of best practices. The best practices are written in as rules that can be changed or updated. The rules are stored in %COMMONPROGRAMFILES%\Microsoft Shared\web server extensions\12\CONFIG\PREUPGRADECHECK directory (http://technet.microsoft.com/en-us/library/dd793609.aspx#bk_rules)

Top 5 uses for the New Pre Upgrade Checker from Top to Bottom of the Report

Total Database Size in Farm
Know which Language Packs Installed
MIA Features
Large Lists and Libraries (Over 5000 Items)
Identify Orphan Sites

Link to the Pre Upgrade Checker kb article: http://support.microsoft.com/kb/960577/

Sample Output:

The tool creates a web-based report for viewing, it would be quite easy to modify this tool to automatically publish this page, and I would recommend setting up a task to run this report on a regular basis.

Raw details on the size and scope of the farm:

Feature List:

I have quite a few features installed on development farm:

Information Only : Feature Information

The following features are either installed or referenced by the content

Name = [PublishingStapling], Feature id = [001f4bd7-746d-403b-aa09-a6cc43de7942], Reference count = [1], Scope = [Farm], Status = [Installed]
Name = [BasicWebParts], Feature id = [00bfea71-1c5e-4a24-b310-ba51c3eb7a57], Reference count = [29], Scope = [Site], Status = [Installed]
Name = [XmlFormLibrary], Feature id = [00bfea71-1e1d-4562-b56a-f05371bb0115], Reference count = [70], Scope = [Web], Status = [Installed]
Name = [AdminLinks], Feature id = [fead7313-ae6d-45dd-8260-13b563cb4c71], Reference count = [1], Scope = [Web], Status = [Installed]

… (About 50 More rows)…

Name = [Unknown], Feature id = [a0469885-9321-ef56-51f8-c945cd70e9ee], Reference count = [1], Scope = [Web], Status = [Missing]

Configuring SharePoint – Add the path to STSADM to the Command Prompt

2009-08-17T21:13:00.000-04:00

Overview

There are a few little things that you should always do when installing SharePoint that save a lot of time. The first and for most is add the path to STSADM to the environment settings of your command prompt. I feel punished whenever this is not setup! ;-)

Steps:

1. Find and copy the path to stsadm:

2. Open up your System Properties (Right click on My Computer and select Properties for System Properties)

3. Click on Advanced then Environmental Variables then under System Variables click on Path and Edit

4. Add the path to STSADM at the end of the statement. Make sure that you use a semicolon to separate paths.

Configure SharePoint – Event ID 7079 on a new build

2009-08-16T11:43:00.000-04:00

Overview

I ran into strange configuration of MOSS 2007 which was getting a whole bunch of even 7078 and 7079 and a whole variety of synchronization errors. The engineer who was investigating this checked to make sure an Index server was a assigned to the SSP.

This was a new build and the SSP was configured on port 80 using the FQDN of one of the web front end servers which near as I can tell could have been causing some issues. Typically when I configure an SSP, I either use a port separate from port 80 or a host header. In this case the resolution was to create a new SSP on a separate port and the old one removed, configuration was applied and no new errors where discovered in a 48 hour period. When troubleshooting this type of error investigate all the configuration of the SSP closely.

Error Message:

Common Resolution:

The common resolution to 7078 errors is to make sure that your index server is configured correctly for your SSP.

1. Assign an Index server to your SSP by click on Shared Services Administration:

2. Click on the downward arrow beside your Shared Service Provider and then click on “Edit Properties”

3. Find the Index Server section and make sure that an index server is filled in:

Additional Troubleshooting

If the common resolution fails, I recommend creating a new SSP, assigning existing sites to the SSP and then removing the old one.

Capacity Planning – HP Proliant SharePoint Sizer tool

2009-08-14T09:05:00.001-04:00

Overview

I wrote a few blogs about the Microsoft System Center Capacity planner and I meant to get back around to HP’s own sizer tool which is definitely first class and I am not just saying that because I work here. The HP Sizer tool is fully updated with current HP hardware. I find that the HP Sizer tool create a more conservative model than the Capacity planner perhaps because of the higher performance hardware. You can download the HP Sizer tool from here: http://h71019.www7.hp.com/activeanswers/Secure/548230-0-0-0-121.html. I recommend utilizing both these tools plus an HP Scientific Calculator to create a model and calculate potential CPU, Memory, and Disk bottlenecks.

Using the HP SharePoint Sizer Tool

After you install and update the tool…

1. Click on the HP Proliant Sizer tool for Microsoft Office SharePoint Server 2007 link

2. Click on the Build a Solution button

3. Click on Next past the info screen and then Name your project. Put in a good name, I have a dozen different saved configurations, but you have to keep them updated. I prefer to build the model fresh for the customer.

4. On the next screen scale your project. SharePoint scales very quickly and the typical single SharePoint server can scale to 500 users. Almost all of my deployments are for users in the thousands so Enterprise it is.

5. Set the number of users, concurrency and operations per user. Concurrency and Operations per user is set at a default to a high level so that inexperienced users will guess high. Most of the time 80% concurrency is highly unusual for an organization. If you are replacing an existing portal its a good time to pull out tracking data.

6. To identity your operations you will need one or many sessions with the customer to figure out there anticipated usage profile. There are allot of details here. If you have no knowledge of SharePoint and want a good guestimate stick with the defaults.

7. Remember that when you are asked about indexing add the anticipated content of the SharePoint PLUS all the other external resources you might index.

8. High Availability Devinitely “Yes” on both the SQL and WFE configuration. This will make sure you have redundant WFEs and a SQL Cluster.

9. I recommend changing the default 0% reserve capacity to minimally 15%. This adds extra resources to the calculation for growth.

10. Configure dedicated application servers, this can be for things like excel services, sql reporting services, or even project servers you want to put in the mix.

11. You should always consider backup with your storage volumes. First, if you are doing anything under 500GB of data, plan for on disk backups. Storage space is less expensive than the backup license for SQL typically. When you start to get to 1 to 2 TB range, consider your tape backup solution and always default to 20% free.

12. I typically go with Rack Mount Server Solutions, but I have done Blade solutions as well. Expect more required SAN space required.

13. If you don’t know which SAN you are using, take the default and pass the recommendation to your SAN person.

14. Review and then Complete your solution.

Giving Power Users the Tools – Creating a Dashboard from a list or document library

2009-08-12T22:35:00.000-04:00

Overview

Many of the sites that I create revolve around a particular business process. There may several lists or document libraries that support the business process but I try to consolidate as much as possible in a single list so that I can make a Dashboard on the front that doesn’t require building a Linked Source. Many sites will start out with a SharePoint list view Web part as the Dashboard. This can easily be customized to include allot functionality.

Quick Note: Dashboards is reactive management unless they are coupled with a communication system so they are usually for big picture people. Most of the time if I create a Dashboard for management the next request is data mining and reporting.

Steps to create a data view Dashboard.

1. Open up a SharePoint .aspx page in SharePoint Designer.

2. Add a document library from the Data Sources pane to an available Web Part Zone. Typically you can find the data source library by clicking on Tools and then Manage Data Sources:

3. For this example, I am clicking on the “Shared Documents” SharePoint Library and dragging it into a Web Part Zone that I created. Your page layout may look different than this one:

4. If you already have a list on the page that you would like to customize, right click on the document library or list and then click on convert to XSLT data view.

5. Find the arrow to the right of your selected data view and open up the Common Tasks, from here you can select your columns:

6. Go back to Common Tasks to Filter and Sort and Group.

7. The most important part of a Dashboard is the conditional formatting. Managers like Dashboards that “light up”. To start working with conditional formatting, it is best to have sample data in your datasheet that represents a wide range of possibilities. Feel free to add additional sample data to the document library at any time and then go to Common Data View Tasks and click Refresh Data to see the results:

9. Open up the conditional formatting side pane by clicking on Data View and Conditional Formatting:

10. Select any cell in your data view and create a conditional formatting rule on that column by clicking on Create and Apply Formatting:

11. Create a rule on the column, for my example I am creating a rule on the Checked Out To column to change the formatting when it equals “Patrick”

12. Add desired formatting, for this example I am changing the Font and the Background.

13. Click Okay, Check your results and Experiment:

Configure SharePoint – Optimizing the Output Cache

2009-08-05T10:11:00.001-04:00

Overview

If you enable the Site Collection Output cache, consider creating different profiles that are optimized for your type of application and utilize Windows Performance Monitor to measure the performance of your cache to see if you have allocated enough memory.

Measures:

Check how your cache is performing:

1. Check at the SharePoint Publishing Cache Performance Object and Select the Total Object discards counter for your Site. If the cache is discarding a high number of objects try increasing the memory for the cache.

2. Check the SharePoint Publishing Cache Hit Ratio. This will tell you if users are utilizing the Cache and be a good measure for improving the cache profile settings and options.

Solving SharePoint Issues – Problem Solving Methodologies - Generalization

2009-08-02T23:24:00.001-04:00

Overview

I am starting up a new section “Solving SharePoint Issues” because I run into so many different SharePoint issues and understanding the process of solving them is more important than the simple fix itself. When analyzing an error message I use a variety of problem solving processes but the most commonly I start with Generalization. Generalization can be used as a deductive proof for cause and effect relationship.

Let me spinout a few words to explain the concept.

Suppose that SharePoint is experiencing an error on a client computer. The user reports that the error message is on one SharePoint Site. Your first response should be to generalize the problem, consider that the browser cannot display any SharePoint Sites. Have the User test other SharePoint Sites. If other SharePoint sites will not display Generalize: Test other Internet Sites that use Active X controls, test shared drives, and test network connectivity. You take a specific error and generalize a condition that could cause a larger set of criteria.

Steps:

Examine error
Hypothesize a more general cause
Test the Hypothesis by looking for additional errors
Identify and resolve the root cause

Configure SharePoint – Take Advantage of SharePoint Caching

2009-08-02T22:36:00.000-04:00

Overview

I wrote an article on caching with an eye to the limitations of caching in a collaboration site. To properly balance the drawbacks and benefits, I thought I post a few words singing the praises of caching publishing sites and custom applications.

There are limitations an Output Caching when you log IIS directly to SQL, depending on your environment that saving logs like that may not be your best option. Here is the reference to the previous article: http://sharepointglobal.blogspot.com/2009/04/architecting-solution-moving-iis-logs.html.

What you can cache VS What you should cache

SharePoint Object / Output Caching is very flexible and Microsoft has outlined what you can cache: http://technet.microsoft.com/en-us/library/cc298466.aspx. This quick blog entry is not a replacement for the article read it now.

Microsoft lists a few caveats, these are things you shouldn’t be concerned about:

Memory: if its a problem, Buy more!
CPU: Ditto!

Hardware should never stand in the way of performance, it’s inexpensive compared to the cost of the service and the risk of poor performance.

Now knowing what you can cache, consider what you should cache by default.
Publishing Sites with anonymous authentication to the internet – the benefits of caching should be obvious
Collaboration Portals with less than 50 concurrent authenticated users – not as obvious, look for benefits in:

Static Pages, Do all users hit a page for content that may not need to be 100% up to date? Cache it!
Reports based off large Lists or Lists that have relationships to other Lists? Cache It!

Here are some sample steps for enabling caching on your site collection.

1. Go to Site Settings at the top level of your Site Collection

2. Click on Site Features under Site Administration

3. Activate the Office SharePoint Server Publishing feature

4. Go back to Site Settings and under Site Collection Administration click on Site Collection Output Cache

5. Enable Output Cache and Select your authentication profiles. Allow Publishing Sites and page layouts to use different profiles to really benefit from the features.

6. The default settings will get you started, based on your configuration you may want to make your own profiles and tweak your object cache settings. You can do this right from Site Collection administration.

Giving Power Users the Tools – Creating an Approval Workflow with SharePoint Designer – Part 2

2009-07-31T12:42:00.002-04:00

Overview

Part 1 has almost all the steps you need to complete the workflow. In Part 2, I will go over the conditions for the next steps and a few possible outcome steps for an approved workflow. It is important to realize that SharePoint Workflows complete Step by Step regardless of the outcome of each step, and that a workflow will not stop unless you tell it to do so.

Conditions for Management Approval Process.

Step 1. Similar to Step 3 in Part 1. Create a New Step and rename it to Collect Management Approval.

Step 2. Set the Condition as Initiation: WorkFlow Status Equals TeamLeadApproved

Step 3. Add an action to Collect Management Approval similar to Step 4 to Step 6 in Part 1.

Step 4. Create a Step to send out email notifications called “Confirm Management Approval” Similar to Step 7 to Step 11 in Part 1. In the conditions look for the ManagementApprovalID instead of the TeamLeadID. Remember that when you compare these values you are looking up a Task by ID and not looking up the value directly.

Step 5. Make sure that you add the condition to Stop the workflow if the Management Approval equals Reject. The end result should look like this:

If you forget to stop the workflows and set the status then un-approved actions will execute.

Step 6. Create a new Step for the approved action. If you wish you can place conditions on this action so that it will only run if both the TeamLeadApproval and Management Approval Tasks are “Approved”, but if you have setup your workflow correctly these conditions are unnecessary. From this point you have all the different options for workflows open. You can use this process to create:

Calendar Entry
List Entry
Update the Status of a Document
Assign a Task

Giving Power Users the Tools – Creating an Approval Workflow with SharePoint Designer – Part 1

2009-07-30T23:14:00.000-04:00

Overview

I use a custom approval workflow with my most of my InfoPath forms. The built in approval workflow is decent however when dealing with an larger organization that has many managers, its typically unacceptable to blanket approval emails and tasks at managers distribution list. With custom workflow, I can target a specific manager and a team lead for every individual in our organization.

Multi-Step Approval Workflow Process:

Sample Steps

1. Open up your SharePoint site with SharePoint Designer, attach the workflow to your list or document library and give it a name:

2. Click “Initiation” and add Parameter “Workflow Status” as a single line of text with the default value of “NotStarted”. I never use spaces in my parameters.

3. Click Okay and then Next to go to “Step 1” and rename “Step 1” to “Collect Team Lead Approval” and Set the Conditions to Initiation: WorkFlow Status equals NotStarted.

4. Add the Action to Collect Team Lead Approval, add a customer form field called Team Lead Approval as a Choice with “Accept” and “Reject” as options.

5. After you complete the Collect Information wizard, it will output the value to a variable. I like to use my own descriptive variable names. Click on Variables at the bottom and Create a new TeamLeadApprovalID variable of type List Item ID. (Remember when you collect information you return the Task ID not the actual form field you are collecting)

6. Set the Output to the new variable. You can delete the old automatically created variable as well to keep your workflow clean and shiny!

7. Create a New Workflow Step and rename it to “Confirm Team Lead Approval”. In this step we are going to send the outgoing emails to the user with a summary.

8. Set the Condition to compare fields and lookup the Tasks list in your site. Look for the field label “Team Lead Approval” This field was automatically created when you ran the Collect Information wizard. (It actually creates a New Task Content Type!) Find the list Item by comparing the Tasks:ID field to your Variable:TeamLeadApprovalID.

9. Set the Condition for Tasks:Team Lead Approval to equals Approve.

10. Create two Actions. The first action set the Initiation:WorkFlow Status to TEamLeadApproved. For the Second Action Submit Send an email message to the person who created the entry letting them now it was approved.

11. Add an Else If condition for Rejected requests using the same instructions as above, but make sure that you add the action to Stop the Workflow or it will KEEP RUNNING. The end result should look like this:

In the Next Post, I will continue with the Management Approval Process. It’s pretty much identical to this one.

Giving Power Users the Tools – Email notification for new Tasks

2009-07-30T09:04:00.001-04:00

Overview

It’s a little strange, but in some of the default Microsoft templates, email notification is turned off for Task lists. As a rule of thumb whenever I make a site template or setup a new Task list I turn this on, its a feature that user’s expect and consistently send me an email about.

Quick Steps

1. Open your Site and List.

2. Click on Settings and List Settings.

3. Click on Advanced Settings.

4. Click on Yes beside “Send email when ownership is assigned?”

We need to work on making this a universal default.

Giving Power Users the Tools – Using InfoPath Web Based Forms to create calendar entries

2009-07-29T16:23:00.001-04:00

Overview

I have been using InfoPath forms with SharePoint workflows for a while now but with this latest workflow, I ran into something interesting I had to create a Calendar item from InfoPath form data.

Creating a Calendar item or a list item from a workflow is easy, you just have to have the required fields with the required data types. Text data must go in text column and date/time fields can be used as start and stop times in a calendar entry.

However, InfoPath 2007 does not have a date/time control just a date control and a time field. In order to pass this information together into SharePoint, the information has to be captured separately and concatenated together in the InfoPath form in a third field. The process looks something like this.

Sample Steps:

1. Open up your InfoPath form and create your fields. You should probably make them all required as without the start and stop and subject fields, you can’t make a calendar entry in SharePoint. StartDate should be a Date type and Start Time should be a Time type. The rest of the types are self explanatory.

2. Two of the fields “StartDateandTime” and “StopDateandTime” are your concatenated fields. Make those two fields date and time type.

3. Enter the default formula for the StartDateandTime fields as concat(StartDate,”T”, StartTime). Its very important to have the T in there so its the recognized date/time format.

4. Promote your new date/time fields. They are ready to be used for new calendar entries.

Configure SharePoint – Disable Loopback on SharePoint WFE Servers

2009-07-27T22:00:00.000-04:00

Overview

I work on a lot of different SharePoint servers and most of the time I don’t have a separate “Client” computer I can use to see how the server is responding. Many times it will appear that SharePoint is broken because of a loopback issue in IIS. When you browse the SharePoint site from the web front end server the server will repeatedly ask for authentication until you get a 401.1 error. The process looks a like this:

Continuous Authentication Boxes:

Eventual failure:

Fortunately you can disable the loopback check, the process is described in Microsoft KB http://support.microsoft.com/kb/896861. I recommend performing this on all your SharePoint servers, but at the very least the server you will be using for administration. My recommendation is use option 2:

“Method 2: Disable the loopback check

Follow these steps:

Click Start, click Run, type regedit, and then click OK.
In Registry Editor, locate and then click the following registry key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
Right-click Lsa, point to New, and then click DWORD Value.
Type DisableLoopbackCheck, and then press ENTER.
Right-click DisableLoopbackCheck, and then click Modify.
In the Value data box, type 1, and then click OK.
Quit Registry Editor, and then restart your computer.”http://support.microsoft.com/kb/896861.

Giving Power Users the Tools - Getting User Information for Web Based Infopath Forms - Part 3

2009-07-21T13:26:00.002-04:00

In the first two parts, I add in the steps for creating a data connection to the profile service and then filtering the results for the current user. The third Part is to display the Manager Information. To do this we will be using the same process that we used in Part 1 and Part 2.

Steps:

Following Part 1, create a new data source with the same configuration as GetUserProfilebyName, but called it GetUsersProfilebyNameMGR. Make sure that this data source does not query when the form is opened.

Add a new text field to your InfoPath form called ManagerAccount

Following Part 2, set the default value from the your GetUserProfilebyName Datasource filter the current User for their Manager’s account information:

Add a rule on the ManagerAccount Field called “QueryManagerDataSource”

Add an Action to set the AccountName fields value to the ManagerAccount field. The ManagerAccount field is the current field selected so it is represented by the underlined dot.

Add an Action to Query using the data connection GetUserProfilebyNameMGR

Click OK until back at the InfoPath form.
Assign default values to the Manager fields the as in Part 2, using the GetUserProfileByNameMGR datasource.

Giving Power Users the Tools - Getting User Information for Web Based Infopath Forms - Part 2

2009-07-20T23:51:00.001-04:00

In the first part we created the data connection, the Second Part Utilize the profile information for the three fields (Full Name, Phone Number, and Email) by filtering out the desired column about the current users.

Steps:

Open the Employee Text Box and click the function ling by the default value box.

Insert a field

Browse to your datasource and select “Value” and then click Filter Data

Add a filter

In the first text box click “Select a Field or Group”, insert the Name field

In the expression column select “is equal to” and in the last column type in “Preferred Name”

Click OK all the way back to the InfoPath form. You have just filter out the data to show the Preferred Name. You can repeat these steps for the “WorkEmail” and “WorkPhone”

Giving Power Users the Tools - Getting User Information for Web Based Infopath Forms - Part 1

2009-07-20T23:50:00.001-04:00

Overview

Somewhere along the way I usually end up building or troubleshooting processes in SharePoint that involve complex workflows and InfoPath forms. I use a pretty basic process for my own forms that has resulted in trouble free operation. Most of my processes require a basic header that gathers user and manager information, this is the trouble free process to do that:

Sample Form “Header”

In this post I will go though the steps to add the User’s info and follow up with Part 2 and Part 3 to create the text field default values and add the manager information.

To automatically populate your form header first Create a Data connection to the SharePoint Site to receive profile information.

Steps:

Open up the data connection wizard in your InfoPath form

Create a new receive data connection

Select the Web Service Option

Enter the following address with your server name to use the SharePoint User Profile Service

Select the operation “Get User Profiles by Name”

Examine the parameters, but there is nothing to add here:

For a web based form there is no need to save an offline copy of the data

Retrieve the data when the form is opened

Click Finish and the data connection is created.

Working with SharePoint Designer - Creating a New Aspx page

2009-07-15T21:42:00.001-04:00

Overview

You can create custom web pages with SharePoint designer and then attach your SharePoint master page with the look and feel of your site.

Sample Process

1. Open up your SharePoint site with SharePoint Designer.
2 . Create a document library for custom web pages.
3. Right click in the document library and select the new aspx command.

4. Right click on your Document and select Properties to change the title. (Just a good practice)

5. After saving the new title, double click on the aspx file to open the page in SharePoint Designer.
6. You can edit the page at this point or go right into attaching the master page. Usually I do my edits first and add my web part zones.
7. Add a web part zone by clicking form#form1 then selecting from the tool bar: Insert then SharePoint Controls then Web Part Zone.

8. Attach the master page by clicking on the tool bar: Format then Master Page then Attach Master Page

9.Typically you want the default master page for your site.

10. The Body is placed in the Main Place Holder, if you have other place holders on your Master page you could modify the content to go there.

11. As SharePoint Designer attaches the page you will see all the typical formatting for your site appear. Save your custom page and you are finished.

Capacity Planning - SharePoint SAN Disk Configuration - Please Make it Right

2009-07-07T08:09:00.003-04:00

Overview

I have a lot of conversations with storage teams, unfortunately when most customers buy a SAN they stack on the largest drives that they can. This is part to reclaim the cost of the frame as quickly as possible, but the problem is that the larger drives have more space but the same IOPS as the smaller drives. With SharePoint Databases I like to achieve 2 IOPS per GB for Log files and Database content. I have found that this is an optimum minimum in collaboration environments with customized pages. As you know, customized pages are stored in SQL Content Databases and not cached on the Web Front End Servers. In this case I am looking at a 512GB volume for SharePoint.

When I write to these teams I try to keep it clear and break it down to:

Understanding IOPS for SharePoint – Understand the requirements

SharePoint Disk Best Practices – Get the info from an Authority

Performance Logs in the Current Configuration – Understand the Impact of what’s been done

After those items I follow it up with specifics that could go in a change request, but I definitely recommend including all elements.

Understanding IOPS for SharePoint

Disk Performance

IOPS are input/output operations per second and a typical 15K RPM SCSI disk has between 150 and 180 IOPS per Disk. A safe measure is to take the lowest value of 150 IOPS. To determine your IOPS per Gigabyte for a specific Disk divide the 150 IOPS by the size of the Disk. In our Current Configuration (see below) we are using 300 GB disks depending on our fill rate we could have as low as 150 IOPS / 300 GB = .5 IOPS per Gigabyte.

Raid Configuration

Raid Configuration is very important to overall performance of a volume, Raids that store parity (raid 5) information decrease overall performance of the disks while raids that stripe data between two volumes (raid 1) increase overall performance per disk. Optimally SharePoint drives are in a Raid 1+0 or a mirrored and Striped configuration.

SharePoint Disk Best Practice

My Best Practice is for SQL Server to access data at a minimum of 2 IOPS per GB. For 512GB we need 1024 IOPS.

Microsoft Recommended Configuration (SharePoint Capacity Planner)

The Microsoft Recommended Configuration for our Database volume of 512GB of content is:

SAN Array\Volume 2 (Data Files),RAID 10 (20 x 146.00 GB SCSI 15,000 RPM)

This configuration is for 10 striped disks = 1500 IOPS which Mirrored on 10 additional Disks.

Current Configuration

One LUSE volume of 512GB constituting 4 logical devices coming from 4 RAID groups.
Each RAID Group has 8 disks( 300GB ,15K RPM) in 7D+1 .

Because this is an uneven number of disks I know that it’s in a parity configuration.

In our current configuration each Raid group is capable of: 656.25 IOPS and 1.8TB of Disk which is .3645 IOPS per GB

A spanned volume does not achieve any higher performance.

http://sharepointglobal.blogspot.com/2009/05/architecting-solution-sharepoint-disk.html

Standard Calculation for Parity Disk IOPS Disks etc:

(0.8 + 4*0.2)( # desired number of IOPS)/(150 IOPS/disk) = # number of Disks

(0.8 + 4*0.2)(X)/(150 IOPS/disk) = 7

(0.8+4*.2)(X) = 1050

(1.6)*(X)=1050

X = 656.25

Performance Logs in the Current Configuration

Even the slightest usage and our disk queue length tops out:

Current Page performance is about 5 seconds to open a page which is slow to failing:

“Estimate user response time

First, determine if your organization can tolerate a slower user response time or if your organization demands a faster user response time. Response times are categorized in the following way:

Slow (3-5 seconds) User response times can slow to this rate without issue.

Recommended (1-2 seconds) The average user response time target.

Fast (<1 second) For organizations whose businesses demand speed. “

http://technet.microsoft.com/en-us/library/cc261795.aspx

The response time is so slow that we are almost off of the chart. When the pages respond slowly all farm operations are slowly which in turn causes errors and make updates difficult.

Architecting a Solution - Best Ways to Grow a Global Solution

2009-06-21T15:07:00.001-04:00

Overview

This is also “Where do I start, if I have only a few servers to start with?” question. All the work in SharePoint is in creating the structure. To create the best structure for your corporation you need to understand: How business processes and culture operate (this will often sell SharePoint), How SharePoint farms work, How to organize your Portal, How to support your implementation and How to scale out your services.

Understanding Business Processes
Business processes are the inputs and outputs of the system and you can map SharePoint technologies to business processes very easily. An example of a business process may be a form that is filled out generates and email that is sent. Example features might include InfoPath Form Services, Outbound Email, Document Approval and Retention.

SharePoint Features

Publishing Portal
Collaboration Portal
Search
Customizable web sites for collaboration via

Document Libraries
Discussion Lists
Custom Lists
Wiki Support
Blog Support
Publishing/Intranet Template

RSS support for each type of library
User controlled e-mail notifications
Microsoft Office 2003 & 2007 Integration
Recycle Bin for deleted list items and documents
Security Trimmed User Interface
File Versioning
Customizable branding of sites
Comprehensive SharePoint Search of:

Entire SharePoint Implementation for data
Search People
Individual Site Search

Custom Search Scopes
Extranet Access
Business Data Catalog Support
MySites
Excel Services
InfoPath Forms Services
SQL Reporting Services Integration
Business Intelligence and Dataview WebParts
Records Management
3^rd Party WebParts

Understanding SharePoint Farms

SharePoint Farms are designed around the usage patterns of the services offered. Microsoft has best practices for the Publishing Portals, Collaborations Sites, Search, and My Sites. When deploying SharePoint, deploy farms around usage patterns so that you can scale out the systems appropriately.

Separate

Portals
Collaborations Sites
Search
MySites

If it’s not feasible to separate these activities consider the resources and usage patterns when designing your SharePoint Farms.

Portals and Search both utilize ram efficiently for caching.
Portals and Search are both accessed by the entire organization.
Collaborations Sites and MySites update files continuously and are read-write intensive.
MySites operate as small site collections.
Typically SharePoint Applications start in Site Collections.

Organizing a Portal

A Portal is a set of mostly static web pages that operate as a gateway to other services. It’s important for the portal interface to be consistent. To build a portal you need to create a taxonomy of the organization. Almost all organizations have a heterogeneous makeup:

Potential Metadata

Region
Division
Department
Function
Audience
Role
Topic
Program

There are many books on portals and they deserve their own paper. Read one!

Supporting your implementation

A Quick Note on High Availability: A popular misconception is that virtualization is high availability. This is not the case, virtualization can create additional hardware availability but does not account for software issues inside the virtual machine for true high availability you have to have redundancy, which means redundant servers, software and data. Consider which components of high availability your organization might need carefully.

When considering your overall implementation of SharePoint it’s important to think of support before and not after production status is “live”. Remember that Portals, Search and MySites are commonly used by the entire organization and may require high availability. Collaboration Sites and SharePoint applications often have third party tools installed which will affect the overall availability of the site. In addition Collaboration sites with large content databases require more maintenance. In addition, Microsoft Best Practices state that some operations need to be completed in the 12 hour off hours interval (read backup and indexing). Utilize a regional support domains whenever possible.

Scaling out your services

There is a misconception out there that you need multiple SQL servers for multiple SharePoint farms and that is not the case. It’s easy to start a multi-farm scenario with a single sql server especially with SQL aliasing. I do recommend multiple instances on that server if possible so that you have separate support domains.

How to start Small – Create your farms and then Scale them Out!

Use Multiple Instances of SQL (Not Requited)
Use SQL Aliases (Easy to Scale out SQL Servers
Add Web Servers

Migrate SharePoint - Migrating Across Versions

2009-06-14T22:22:00.005-04:00

Overview

When you migrate SharePoint Sites typically you would do a backup and restore operation with STSADM. It’s best to migrate to the same version, but this is not always possible. Here are a few things I have found out while doing migrations.

You can go UP but you can’t go Down:

Most of the time you can go to a later version but never an older version, if you try to go to an older version you will receive an error message something like this:

“Your backup is from a different version of Windows SharePoint Services and cannot be restored to a server running the current version. The backup file should be restored to a server with version ‘12.0.0.6318’ or later. “

Beware the Infrastructure Update:

Trying to go past the Infrastructure Update from a server before it will generate an error message about an index property not being set on the content.

It’s fairly safe to migrate from 12.0.0.4518 up to 12.0.0.6303 and also

From 12.0.0.6318 (Infrastructure update) to 12.0.0.6421 (SP2)

Whenever you go across versions you are taking risks that will hopefully pay off in time savings so Test Your Migrations.

SharePoint Capacity Planning -Scaling UP CPU and Memory

2009-06-10T17:50:00.003-04:00

Overview

I am testing SharePoint Scale Up performance for a migration that I have to do for an enterprise customer. To keep the farm in balance I am going to use a farm-in a-box virtual server with a limited data set and scale it up with 3 separate test configurations:

Configuration A is 1 Processor and 4 GB memory
Configuration B is 2 Processors and 8 GB memory
Configuration C is 4 Processors and 16 GB memory

The Disk Subsystem will not change nor will the type and quantity of the data operations performed. All servers will use 6 15K SAS drives configured in a raid 5. The purpose of this test will be to examine farm operations (Adding Content Databases, Adding Site Collections, Indexing and Profile Import) and consider scale up performance.

Numbers in all Graphs represent MINUTES and smaller numbers are better.

Farm Operations

The following measures were implemented with Powershell scripts similar to the one below.

get-date
$N=0
do {
$N=$N+1
stsadm.exe -o addcontentdb -url "http://siteurl" -databasename Contentdb$N
}
while ($N -le 100)
get-date

(Note I used very similar scripts for the rest of the operations.)

Database Creation

A little bit inexplicable, the configuration with the most resources actually took a few minutes longer to create 100 databases.

Disconnect Databases

While disconnecting the databases there did not appear to be any advantage to higher number of CPUs or memory.

Site Collection Creation

I tested Site Collection creation two ways, by having the server create 100 sites sequentially and then having the server create an additional 100 sites in two parallel process.

100 Sequential Site Creation

There was a nominal improvement on single process site collection creation.

50x2 Parallel Site Creation

In this case there was a some improvement between Configuration A and Configuration B.

Indexing

The index server was set to partly reduced as this is a farm in a box configuration. The index server did not have any impact rules applied. Before Each full index I reset all the index content. In each case the same number of results was indexed.

During most operations the CPU Usage was low, however during indexing the web application and SQL Server used the maximum CPU available. This is the nature of the indexing process, even when its set to partly reduced its processing intensive for both the SQL Server and the Web Servers.

During a full crawl our CPU was pegged regardless of the number of CPUs:

Configuration A:

Configuration B:

Configuration C:

An increase in performance from Configuration B to Configuration C.

Profile Import

The script below was used to created 10,000 user accounts for the profile import test:

I tested both incremental and full imports and the server CPU pegged when I started the import pretty much regardless of the number of CPUs:

Configuration A:

Configuration B:

Configuration C:

Definite increase in performance from Configuration A to Configuration B.

Definite increase in performance from Configuration A to Configuration B.

Conclusions

Latency is the number one issue for SharePoint Performance and the biggest gains were seen during the jump from a single cpu to a dual cpu because of a dramatic decrease in latency. SharePoint was noticeably faster in Configuration B and Configuration C. Even with lower latency though single threaded operations can only proceed so fast, site creations speed did not increase significantly regardless of the configuration until the process was multithreaded. Consider that when architecting a solution depending on the size and desired performance response to the user base, two farms in a lesser configuration can outperform one in an greater configuration when it comes to completing single threaded operations.

Capacity Planning - Single Farm for the Global Enterprise

2009-06-08T21:31:00.002-04:00

Overview

I recently downloaded a copy of System Center Capacity Planner 2007 (http://technet.microsoft.com/en-us/sccp/bb969059.aspx). To use this tool you need both the planner and the SharePoint Model. I wanted to compare how a single SharePoint Farm model would play out in this tool when you consider all the services that a global SharePoint implementation has. I would rarely recommend a single farm for an enterprise because most likely it would create a Farm of Unfortunate Proportions.

Steps:

After installation Start the tool and select SharePoint Server 2007:

You will notice with a SharePoint farm deployment you can only select one Usage profile. This is consistent with SharePoint Best Practices and although large enterprises have multiple usage profiles we will continue as if there is just one.

At the Branch offices we get a little more flexibility. These users will be added to the total farm users, to keep a consistent pattern of 20000, I acutally went back to the first screen and changed the local count to 5000 users.

This is the most current chipset and disk model we are using:

I selected High Availability for the Web Front Ends, but not the SQL Server because I think anything we do will be in a Hyper-V Cluster anyway.

You will notice that the SharePoint Model Maximum is 2 TB of data, my own experience is that this is the maximum for a SQL Instance.

Results
“
Topology
Sites with servers: 1
Sites with clients only: 3
Total number of clients: 20000

Site: One Farm
Number of users: 5000
Number of servers: 6
Number of SAN connections: 0

Server: Index Server
Processor: 8-processor, 2.67 GHz, Xeon 5300-Series (2-chip x 4-core)
Minimum memory: 8.0 GB
Disk: DiskArray 1\Volume 1 (File System), 792 GB RAID 5 (12 x 72.00 GB SAS 15,000 RPM)
NIC: 1 x 1,000 Mb/s
Roles: Index

Server: SQL Server 1
Processor: 8-processor, 2.67 GHz, Xeon 5300-Series (2-chip x 4-core)
Minimum memory: 16.0 GB
Disk: DiskArray 1\Volume 1 (Log Files), 216 GB RAID 10 (6 x 72.00 GB SAS 15,000 RPM)
DiskArray 1\Volume 2 (Data Files), 1728 GB RAID 10 (48 x 72.00 GB SAS 15,000 RPM)
NIC: 1 x 1,000 Mb/s
Roles: SQL Server

Server: SQL Server 2
Processor: 8-processor, 2.67 GHz, Xeon 5300-Series (2-chip x 4-core)
Minimum memory: 16.0 GB
Disk: DiskArray 1\Volume 1 (Log Files), 216 GB RAID 10 (6 x 72.00 GB SAS 15,000 RPM)
DiskArray 1\Volume 2 (Data Files), 1728 GB RAID 10 (48 x 72.00 GB SAS 15,000 RPM)
NIC: 1 x 1,000 Mb/s
Roles: SQL Server

Server: SQL Server 3
Processor: 8-processor, 2.67 GHz, Xeon 5300-Series (2-chip x 4-core)
Minimum memory: 16.0 GB
Disk: DiskArray 1\Volume 1 (Log Files), 216 GB RAID 10 (6 x 72.00 GB SAS 15,000 RPM)
DiskArray 1\Volume 2 (Data Files), 1728 GB RAID 10 (48 x 72.00 GB SAS 15,000 RPM)
NIC: 1 x 1,000 Mb/s
Roles: SQL Server

Server: Web Front End 1
Processor: 8-processor, 2.67 GHz, Xeon 5300-Series (2-chip x 4-core)
Minimum memory: 8.0 GB
Disk: DiskArray 1\Volume 1 (File System), 144 GB RAID 10 (4 x 72.00 GB SAS 15,000 RPM)
DiskArray 1\Volume 2 (File System), 792 GB RAID 5 (12 x 72.00 GB SAS 15,000 RPM)
NIC: 1 x 1,000 Mb/s
Roles: Web Front End; Query Server

Server: Web Front End 2
Processor: 8-processor, 2.67 GHz, Xeon 5300-Series (2-chip x 4-core)
Minimum memory: 8.0 GB
Disk: DiskArray 1\Volume 1 (File System), 144 GB RAID 10 (4 x 72.00 GB SAS 15,000 RPM)
DiskArray 1\Volume 2 (File System), 792 GB RAID 5 (12 x 72.00 GB SAS 15,000 RPM)
NIC: 1 x 1,000 Mb/s
Roles: Web Front End; Query Server
“

Diagram Result:

The Total Storage the Capacity tool recommended was 8.5TB for 2TB of content. The Capacity planner also recommended breaking out the content into 3 separate SQL Servers.

SQL Server Management - Breaking up Large Content Databases into Multiple Files

2009-06-05T11:23:00.004-04:00

Overview

In mismanaged environments content databases grow out of control sometimes to hundreds of gigabytes. It’s possible to break up these content databases into several files but there are several considerations and before breaking up a content database you should consider the long term capacity management strategy.

Breakup a large content database with SQL because:

Performance: you can divide up the files across different disks in the san etc
Manageability: keep the files at 50gb or less and you can move them easily with Windows Copy and Paste

Why Not?

Backup/Restore: doesn’t fix backup restore of large databases
SharePoint backup is not aware of SQL file groups

Break up a large content database by managing SharePoint (move around your site collections, subsites into new databases and put in quotas!)

Performance: Microsoft Best Practice is to keep site collection sizes smaller, I have seen numbers ranging from 20 to 50 GB. Consider 50GB your maximum.
Manageability: Ditto
Backup/Restore: Will be within margins

Procedure

Using a test 22GB database I have:

Go to the properties of the database:

Added on 2 addition ndf files.

Execute a shrink on the database:

This command will empty out the original file and place all the contents in the two new files.

You should see the ndf files grow as the processes executes:

And Growing…

During the process expect high disk activity:

I have kept the SharePoint online during the process:

I recommend that if you do so start the process after hours so that there is no competition for the database. If the shrink operation does not complete with the “empty file” parameter you may have to shrink the original data file:

During the operation you will need to plan for sufficient disk space for the Log file (now 14GB, and the 2 additional ndf files (14GB). To complete the operation I recommend having 3x the Free Space of the database available on disk.

Capacity Planning - Farms of Unfortunate Proportions

2009-06-03T18:22:00.001-04:00

Overview

I have worked with many different SharePoint architectures for all the different versions of SharePoint and presented quite a few of my own. I have also run into many poorly architected SharePoints, most of the time these SharePoints are the result of:

Not studying the Microsoft Best Practices before implementation
Piecemeal tactic of cherry picking individual best practices without considering overall impact
Employing a bean counter approach to building a SharePoint
Considering that Best Practices do not apply because of a special customization or consideration

All best practices need to be interpreted with the recognition that they are Best Practices and not Rules, however in a well architected SharePoint, departing from best practices should only be considered when improving performance and user experience. In this blog entry I am summing up some of the best practices and experiences that I have had with SharePoint delivery.

Okay, so where do I go from here

On Microsoft Best Practices site (http://technet.microsoft.com/en-us/office/sharepointserver/bb736746.aspx, which is practically my home page) Microsoft has divided up different best practices based on the type of SharePoint farm that you are deploying. For small deployments the specific guidelines can be followed, but large global enterprise deployments will deploy all of these solutions to a large user base and that requires both a more complex multi-farm, tiered configuration and recognition of how services work together.

Microsoft has best practices for:

Operational Excellence
Team Collaboration Sites
My Sites
Publishing Portals
Search
Capacity Management
Developing Custom Applications

Your typical global enterprise solution will have all these solutions (Publishing Portal, Search, My Sites, Team Collaboration Sites and typically Customized Applications)

Understanding SharePoint Solutions

Single Farm Vs Multiple Farms

There are a number of factors to consider when designing an architecture for a global enterprise that will use SharePoint with all or most SharePoint solutions. Consider the key business drivers behind each solution.

Publishing and Collaboration Portals

Key Business Drivers behind Portals is an always up gateway for internal services. Most organizations would choose availability over performance and typically these are centralized. Microsoft recognizes this and the Microsoft Best Practices is that Portal sites should be optimized for highest availability, with specific limitations on lists and the application pools will be optimized for caching. In addition to this, common sense and experience has taught us that restricting access to SharePoint Designer and 3^rd party applications increase up time for Portals.

My Sites

Most companies recognize the power of Social Networking, My Sites is part of this. Another business drive might be to store and share files. Microsoft has a metric for My Sites I have copied below. For global organizations it is optimal for My Sites to be regionally located to improve performance. Natively MySites and Search are all part of the same SSP. With a large number of My Sites this should be separated.(http://technet.microsoft.com/en-us/library/cc262706.aspx)

“4. Plan for performance

The increased volume in sites, folder structures, drive storage, and processing that can accompany My Sites (even if they are as small as possible and used only to store a profile picture) can impact the backup and recovery processes and the availability of the rest of the SSP farm. Large organizations with 100,000 employees or more should consider putting My Sites into a separate farm from their Search SSP. We recommend a maximum of 150,000 My Sites per SSP. “

Collaboration Team Sites

Key Business drivers for Collaboration sites are to store documents but also drive business processes. Often SharePoint applications start from dedicated team sites. Unlike Publishing Portals, Collaboration Site Collections will often grow quite large, they are usually team based so they are often function, tower or region specific. Ideally the databases should be separated for best query execution. Typically SharePoint Designer is employed on Collaboration Sites and sometimes third party applications to drive particular business processes. Because of the large volume of content maintenance windows for indexing and backup are even more important so whenever possible these should be deployed regionally.

Search

Enterprise search is often one of the top reasons that customers purchase SharePoint. As such it typically requires the highest availability and solid performance.

Custom Applications

Typically custom applications can affect the availability of a SharePoint the most. Microsoft has released best practices around the creation of custom apps and I have written other blogs about it as well: http://sharepointglobal.blogspot.com/2009/06/configuring-sharepoint-disposable.html

In terms of delivering a highly available deployment, it is possible to combine Portal, My Site and Search solutions into a single farm if the anticipated performance requirement is low enough.. Collaboration Team Sites and Custom Applications should be kept separate and local if possible.

Farm Performance

I often think of a SharePoint Farm as a balanced wheel. When we consider guidelines we have to make sure that performance and capacity is in balance across each of the services in the farm.

The number 1 reason for poor SharePoint performance is latency. Latency can be added in to any component of a SharePoint farm or during communication between components. While designing the Architecture for SharePoint in your organization at all costs avoid having Farms of Unfortunate Proportions.

Web Front End Servers

Guidelines for acceptable performance (http://technet.microsoft.com/en-us/library/cc262533.aspx)

“Capacity is directly affected by scalability. This section lists the objects that can compose a solution and provides guidelines for acceptable performance for each type of object. Limits data is provided, along with notes that describe the conditions under which the limit obtain as well as links to additional information where available. Use the guidelines in this article to review your overall solution plans.

If your solution plans exceed the recommended guidelines for one or more objects, take one or more of the following actions:

Evaluate the solution to ensure that compensations are made in other areas.
Flag these areas for testing and monitoring as you build and deploy your solution.
Re-design the solution to ensure that you do not exceed capacity guidelines.

The following tables list the objects by category and include recommended guidelines for acceptable performance. Acceptable performance means that the system as tested can support that number of objects, but that the number cannot be exceeded without some performance degradation. An asterisk (*) indicates a hard limit; no asterisk indicates a tested or supported limit.

The following table lists the recommended guidelines for site objects.

Web server/database server ratio	8 Web servers per database server	The scale out factor is dependent upon the mix of operations.
Web server/domain controller ratio	3 Web servers per domain controller	Depending on how much authentication traffic is generated, your environment may support a greater number of Web servers per domain controller.

Throughput vs. number of Web servers

In our test environment, farm throughput reached a plateau at 5 Web servers per database server, and did not change substantially when additional Web servers were added. Although you can deploy up to 8 Web servers per database server, you may not realize substantial throughput gains after 5 Web servers. This is because as the number of Web servers making calls against a single database server increases, the database server eventually reaches 100% capacity. Results in your environment may vary according to the performance characteristics of your database server. You will need to conduct your own testing to determine the optimum number of Web servers in your farm environment.”

Scaling OUT Vs Scaling UP Web Servers

Here is a quick look at the obvious benefits of each.

Scale OUT Benefits

IIS memory fragmentation (http://technet.microsoft.com/en-us/library/bb742534.aspx) this is an oldie but still good to note that more memory doesn’t equal more performance.
Timer Jobs (Read More below under SharePoint Processes
More concurrent users with better redundancy

Scale UP Benefits

Optimize Network Paths
Licensing Costs

Remember that at least one of your web servers is going to be utilized by your index server and depending on your content that could be a big indexing job.

Index Servers

There are some best practices for index server particular to Collaboration Team sites. Typically Collaboration Team sites have a lot of frequently changing content. It’s essential that the proper maintenance windows are allocated for indexing these sites.

http://technet.microsoft.com/en-us/library/cc262574.aspx

“Estimate crawl window

In a Office SharePoint Server 2007 search environment, crawling content typically is the longest-running operation that is not initiated by users. You will need to perform testing in your own environment to determine the amount of time it takes to crawl content using a particular content source, and whether the throughput consumed by crawling this content interferes with your target user response times. Typically, you should verify that crawling a particular content source can be contained within an overnight time span of 12 hours.

Remote server latency

Server latency is a major factor that affects crawl performance. Performance between farm servers must be balanced for overall crawl performance to reach its potential. For example, a powerful index server can be operating at 25% of its capacity if the database server being crawled is not able to respond quickly enough. In such a case, you can scale up the database server, which will in turn increase crawl speeds across the entire farm.

You should conduct your own testing to evaluate the responsiveness of servers in your environment. The database server serving the target farm is often the bottleneck in cases where crawl performance is poor. To improve crawl performance, you can:

Scale up database server hardware by adding or upgrading processors, adding memory, and upgrading to hard disks with faster seek and write times.
Increase the memory on query servers in the farm
Crawl during non-peak hours so that the database server being crawled can service user traffic during the day, and respond to crawls during off-peak hours.

SQL Database servers

Scaling OUT Vs Scaling UP SQL Server (Topology Recommendations, http://technet.microsoft.com/en-us/library/cc298578.aspx)

When considering scaling up SQL server remember that there is only 1 TEMPDB per SQL Server instance and SharePoint relies heavily upon tempdb operations. Below are Microsoft recommendations, I have found that with current hardware 2TB of data per SQL instance is supportable, I have seen as high as 5 TB per server on Microsoft site for Collaboration Team sites…. But remember the whole farm must be balanced out to handle large data which makes for bigger windows for indexing and backup.

“Consider scaling out in addition to adding resources

It is important to track the following three resource components of a server running SQL Server 2005: CPU, memory, and I/O subsystem. When one or more of the components seem stretched, analyze the appropriate course of action based on the current and projected work load. Then, determine whether to add more resources or to scale out to a new server running SQL Server 2005. In general, we recommend that you consider scaling out in addition to adding more resources.

If your deployment parameters are generally greater than the upper limits of most of the listed values, your deployment can be considered large.

Metric	Value
Content database size	50 GB
Number of content databases	20
Number of concurrent requests to SQL Server 2005	200
Users	1000
Number of items in regularly accessed list	2000
Number of columns in regularly accessed list	20

“

(The calculation is 20 content databases multiplied by 50GB makes 1TB considered Large)

“It is important to track the following three resource components of a server running SQL Server 2005: CPU, memory, and I/O subsystem. When one or more of the components seem stretched, analyze the appropriate course of action based on the current and projected work load. Then, determine whether to add more resources or to scale out to a new server running SQL Server 2005. In general, we recommend that you consider scaling out in addition to adding more resources.

We recommend that you deploy an additional server running SQL Server 2005 when you have more than four Web servers running at full capacity.

Minimal latency on the I/O subsystem that serves the server that runs SQL Server is very important. Slow response from the I/O subsystem cannot be compensated for by adding other types of resources, like CPU or memory, but it can influence and cause issues throughout the farm. Plan for minimal latency before deployment, and monitor your existing systems as described in the section on monitoring.

SQL Server performance depends heavily on the I/O subsystem. Unless your database fits into physical memory, SQL Server constantly brings database pages in and out of the buffer pool. This generates substantial I/O traffic. Similarly, the log records need to be flushed to the disk before a transaction can be declared committed. And finally, SQL Server uses tempdb for various purposes such as to store intermediate results, to sort, to keep row versions and so on. So a good I/O subsystem is critical to the performance of SQL Server.”

Bus bandwidth (http://technet.microsoft.com/en-us/library/cc263261.aspx)

“Greater bus bandwidth helps improve reliability and performance. Consider that the disk is not the only user of bus bandwidth — for example, you must also account for network access.”

When considering scaling out SQL Servers, its common to forget that the system bus can contribute to overall latency in the system and that Scaling Up is not always an acceptable solution.

Content Databases Performance

Microsoft has outlined two best practices for SharePoint around SQL Server performance that I have placed below. You will notice that both guidelines essentially say the same thing, keep your databases consistent:

Here is the Best Practice for Publishing Portals:

(http://technet.microsoft.com/en-us/library/cc850698.aspx)

“This is because Office SharePoint Server 2007 performs best when the types of access and usage patterns for the content in a database are similar. Separating primarily read-only content (publishing) from read-write content (authoring), into different site collections can help.”

(http://technet.microsoft.com/en-us/library/cc262533.aspx)

EXECUTION PATHS are critical when it comes to response and latency in response is critical for indexing.

“Consider performance

When you host team sites on a dedicated Web application, you have several content databases that contain only team site collections. If content databases host sites with similar data characteristics, Microsoft SQL Server database software operates more efficiently because SQL Server chooses a query plan based on the characteristics of a database. By contrast, if a database hosts sites with vastly different data characteristics, the query plan that SQL Server uses might not be the most efficient method for all content in the database. For example, if a database hosts team sites (that is, a large number of medium-sized sites) and portal sites (that is, a small number of very large sites with many requests), the chosen query plan will be inefficient for one of the types of sites. Therefore, by placing content for team sites in dedicated databases, you can optimize performance for SQL Server, which results in better performance for the overall server farm. “

SharePoint Processes that Consume Resources

SharePoint Timer Jobs

Making sure that your collaboration farms are region/function/tower specific will help with this operation. Joel O. Lists this as one of his SharePoint Top Performance Killers:

http://www.sharepointjoel.com/default.aspx
“Misc Timer Jobs – User Sync for large #s of Users – the more users the longer the timer jobs will run. Profile Synchronization - This job runs once every Hour and there is one per Web Application Quick Profile Synchronization - This job runs once every minute as performance permits and there is one per Web Application (MSDN.) More information on miscellaneous timer jobs… “
Operating System issues… reasons why to use more redundant servers and less large ones:

Backup and Restore Operations

Also Listed as a top performance killer by Joel
(http://technet.microsoft.com/en-us/library/cc901593.aspx)

“By design, most backup jobs consume as many I/O resources as they can to finish the job in the available time for maintenance. Therefore, you might see disk queuing, and you might see that all I/O requests come back more slowly than usual. This is typical and should not be considered a problem.”

Index Propagation

Index Propagation will affect the performance of the Index and Query servers depending on the size of the indexes.

Migrate SharePoint - Testing SQL 2008 with SharePoint Portal Server 2003 - Part 3

2009-06-02T15:54:00.003-04:00

Overview – Part 3

Here is Part 3 is screenshots of the SharePoint and errors in the event log.

Testing:

Everything SharePoint Seems to be working okay. This is a SharePoint 2003. Look at old Central Admin:

Portal Comes up fine as well:

I am going to do some more testing with individual sites and you will probably find that in a later blog, but for now lets examine some of the error messages I received during the installation.

Errors

A number of errors during or just after the installation:

Error 30059: This error message may be the result of not changing the service account for the SQL Full-Text Filter Daemon Launcher service to the SQL Service account or local system. I am going to keep an eye on this one and see if its reoccurring:

Error 117: This error may be temporary as well and in the case of an integrated installation I don’t see a reason why I can’t just rebuild the database. Here is what the report server database stores in an integrated installation:

“When created for SharePoint integrated mode, the report server database stores server properties, report execution snapshots, report history, subscription definitions, and schedules. It stores a secondary copy of reports, report models, shared data sources, and resources to improve processing performance on the server. Primary storage for report documents is in the SharePoint content databases. For more information, see Storing and Synchronizing Report Server Content With SharePoint Databases.” http://msdn.microsoft.com/en-us/library/bb283151.aspx

Another Report Server error, I will probably have to reconfigure that:

Error 1101 I see a few of these error messages have been reported on the Internet as possibly installation related. I will see if they re-occur.

Migrate SharePoint - Testing SQL 2008 with SharePoint Portal Server 2003 - Part 2

2009-06-02T15:52:00.005-04:00

Overview – Part 2

Here is Part 2 where the actual installation begins.

Installation

SQL Server does a lot of testing and checking….before the installation

Some more processing

More testing:

Select the instance to upgrade:

Upgrade so features are grayed out:

The usual. This is a test machine, most of the time you would see this on another drive.

Considering these storage space requirements some of my older servers are going to need a cleanup…

I elected to import the Full-Text indexes to see what would happen. You will notice some of the error messages below might have been from this choice:

The installation went through smoothly. Check out Part 3 for Testing and Error Events.

Migrate SharePoint - Testing SQL 2008 with SharePoint Portal Server 2003 Part 1

2009-06-02T15:42:00.004-04:00

Overview – Part 1

I am working out different strategies for a migration project and realized I had not tested a SQL 2008 upgrade on my SharePoint 2003 server. By all accounts the SQL upgrade should not cause any issues, although I have read that SharePoint 2003 will not run on Windows 2008 (IE IIS 7). SQL 2008 will upgrade a 2005 or 2000 instance pretty painlessly and its smart enough to load in its prerequisites first. I took a quite a few screenshots of the process, if you are interested in the event logs got to Part 3 and scroll to the bottom.

Loading the Pre-Reqs:

Needed to Restart:

I installed the Upgrade Advisor. SharePoint 2010 will have an Upgrade Advisor as well.

Once installed you can find it in the Start Menu under Microsoft SQL Server 2008 then SQL Server 2008 Upgrade Advisor

I don’t know why they provide hyper links to the messages when it just tells you to open it in the Report Viewer.

Reporting Services Issues

Some features have been deprecated in Reporting Services, but I don’t believe they are critical or even part of the SharePoint Integrated Configuration.

SQL Server Database Server Issues

The analyzer had a lot to say about full-text indexing. Full-text indexing went through a pretty good upgrade in SQL 2005, adding new logging features, etc. In SQL 2008, it looks like word breakers and filters have been improved. SharePoint 2007 won’t take advantage of this.

Configuring SharePoint - Disposable Objects Affecting Site Performance

2009-06-01T15:42:00.002-04:00

Overview

I am very particular about separating Collaboration environments from Application environments for SharePoint because different Best Practices apply to different types of SharePoints. This issue came to my attention quite a while ago but this is the first I have documented it. This is one of the very rare issues that require adding a registry key to change to the environment.

Understand what type of environment you have. If part of the answer is an environment for custom applications than look into this.

Customize Code for SharePoint:

I am not a fan of writing custom code for SharePoint, the software has so much capability rarely does custom code offer a unique solution. However Microsoft has released excellent documentation for coding with SharePoint using disposable objects to improve system performance: http://msdn.microsoft.com/en-us/library/aa973248.aspx.

Testing the Solution:

Microsoft has also released a tool to check the solution: http://code.msdn.microsoft.com/SPDisposeCheck.

Configuring SharePoint:

Potentially custom code using disposable objects can create too man SPRequest objects. My team has run into this several times and the details can be found under the Best Practices: http://msdn.microsoft.com/en-us/library/aa973248.aspx same as above:

“Because the very low default threshold may not apply to many sites, you can change this threshold by editing the following registry subkey:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\HeapSettings LocalSPRequestWarnCount = desired threshold value“

Configuring SharePoint - Upgrading SQL to a Different Edition

2009-06-01T10:11:00.004-04:00

Overview

As our SharePoint Farms get older, occasionally there is reason to upgrade our database servers. Sometimes for New Features, sometimes to Scale Up the database server, and infrequently to Scale Down a server.

Microsoft has a versions and upgrade link that you can find here: http://msdn2.microsoft.com/en-us/library/ms143393.aspx
If you look through this document and its predecessors you will see that you can upgrade from older standard editions to newer standard and enterprise editions, but you cannot go from enterprise edition to a newer standard edition.

When upgrading is not possible, an uninstallation/reinstallation or new installation is most likely. Depending on your comfort level with your server, you may actually want to go to an entirely new machine(s). In this case if you have observed Microsoft’s best practices of aliasing your SQL Server installation the operation should be painless: http://technet.microsoft.com/en-us/library/cc298578.aspx. Be aware that SQL Alias will prevent MS DPM.

“

Use SQL Server connection aliases when you configure your server farm. A connection alias is an alternate name that can be used to make a connection to a SQL Server instance. If a database server fails, you can adjust the alias on the front-end Web server to point to another server. For more information, see How to: Set a SQL Server Alias (SQL Server Management Studio) (http://msdn.microsoft.com/en-us/library/ms175176.aspx).

“

Make sure that SQL Server Configuration Manager is installed on your SharePoint Server(s):

Microsoft SQL 2008 makes upgrading easy

With SQL 2008 you can actually tell it to upgrade from the command prompt. The list of command prompt settings is here: http://msdn.microsoft.com/en-us/library/ms144259.aspx the actual command line being:

Setup.exe /q /ACTION=editionupgrade /INSTANCENAME=MSSQLSERVER /PID="PID key for new edition"

Fortunately Microsoft has well documented upgrading Failover clusters for SQL 2008 as well: http://msdn.microsoft.com/en-us/library/ms191295.aspx

Testing Tools and Addons - Metalogix SSMM with Large Document Libraries continued

2009-05-29T13:13:00.003-04:00

Overview

I have increased the ram on my test machines to 4GB each as I continue to test MetaLogix SharePoint Site Migration Manager with large files. I have a document library in SharePoint 2003 that is about 20GB in size and has files up to 650MB. This is not the typical load for SharePoint, but the engineering SharePoints that I am looking at migrating have several of these.

Testing a copy list operation:

Lots of options for list copying:

I want to get an idea on performance although many factors come into play fortunately the logging is very thorough and easy to access so I don’t have to spend time grabbing start and stop times.

I am still seeing memory exceptions when copying files although the system has plenty of memory:

I remember issues around webdav and memory and looked up the article from the SharePoint Blogs (http://www.sharepointblogs.com/smc750/archive/2009/05/19/uploading-content-into-sharepoint-let-me-count-the-ways.aspx) here is an excerpt:

“Ok this seems simple enough. As you can see it is not as complex as using the Copy web service. However, it does not support sending any metadata long with the file content. This can be a major problem if the document library has multiple content types, so the new file will be put into the document library with the default content type. Another big issue is if the default content type has required fields. The file will remain checked out until the fields are populated. This prevents other users from seeing the document or from being returned in any searches. It is a great solution if you are just bulk migrating data from an external data store to SharePoint. You more than likely will have to do extra work afterwards. Adding metadata after uploading will also cause the creation of extra versions of the document being created unnecessarily. The fact that it does not use the soap protocol but straight http makes it more scalable than the copy web service. Unfortunately, it still suffers from the fact that it uses a byte array to upload the file. So sooner or later you will run into “out of memory “ exceptions. So how can I create a folder before using WebDav? You can use the lists web service to accomplish this:”

I am sending a quick question to Jeremy at Metalogix to see if they use Webdav for the copy process. The next step is to see if I can copy the files that it fails to copy one at a time through SharePoint Site Migration Manager. With good logging I can setup a process to catch the errors and have them manually moved, ideally though I do not want the people who are executing the migration to have to leave the Migration Manager window. In a large migration, two of the most important factors are the migration execution plan and the responsiveness of the team.

A Quick Mention - SP2 Bug Trial License Issue

2009-05-29T09:56:00.003-04:00

A Quick Mention

The SP2 License bug has been documented quite a few times already and I am not aware of any downtime that has resulted from the issue. To fix the bug all that you need to do is put the key into the software again. As long as you have good documentation of your installation that should not be a problem. (PS always document your implementation) I held of on this quick mention until the actual Microsoft KB was posted. Click here for all the details: http://support.microsoft.com/kb/971620

I have seen the license synchronizer job stall out on servers, if necessary you may have to go to each server and force it through with stsadm.

Testing Tools and Addons - Metalogix SSMM with Large Document Libraries continued

2009-05-27T10:28:00.002-04:00

Overview

Continuing testing with some large site collections. Just to reiterate, I am working on a migration that involves engineering content that is large in both number of items (site size) and size of items so I have created a 20gb site collection full of 600mb files.

During testing I was getting errors attempting to move larger files so I increased the RAM on my test Virtual Machines to 3GB each. I emailed Jeremy at Metalogix as well and he suggested there could be a resource issue and to attempt moving just the Document Library or the item. I am looking at a 10TB farm so I can’t really move one file at a time unless it’s the occasional exception… But overall I have had fairly good results so let’s proceed:

I am beefing up my servers to 4gb of ram:

To go any higher wouldn’t help, none of the old servers have more than 4GB so if I can’t do it now I’m out of luck.

I have more than enough disk and io available for the operation:

For some reason it wanted to be activated again… I thought I had a few more days. Guess not I received the key on the 5/11 and it was good until 5/24. 13 days may seem like a lot of time for testing but with a large migration and many variables that’s a pretty small window. I am sending out for another key.

Metalogix has been very responsive so far whenever I sent a message so I am sure they will not disappoint.

Implementing a Help Desk - Stopping SharePoint Services Quickly

2009-05-22T10:19:00.005-04:00

Overview

I am presently working on a datacenter transformation project where all of the SharePoint servers are being moved to a new location. Since there is replication going on, I need to shutdown all the SharePoint services and verify that the program files have been replicated to a new server. (We are using Database Mirroring for the SharePoint databases.) To help out this process I created a fast and dirty PowerShell script to stop processes. I examined several methods on the net but found I like using sc.exe best. This is the kind of operation the help desk should be able to initiate at need.

'Stop Services on SharePoint Servers'

$SharePointServers = Import-Csv SharePointServers.csv

$SharePointServices = Import-Csv SharePointServices.csv

foreach ($entry in $SharePointServers)

{

foreach ($entry2 in $SharePointServices)

{

sc.exe $entry.Server stop $entry2.name

*sc.exe $entry.Server config iisadmin start=disabled

}

A test in my lab found this to be the best method to avoid error handling issues, etc. If the service is already stopped the script will continue without issue.

*Update I had to actually disable iisadmin so the webservices would not start up again. I also had some issues with our monitoring software comming up behind me and restarting the sql services.

Testing Tools and Addons - Metalogix SSMM with Large Document Libraries continued

2009-05-21T12:46:00.002-04:00

Overview

Continuing my testing with large document libraries. I was able to move the sites but with some issues. From where I left off last:

I was able to get just over 20GB of ISO images in the document library on my test site and I started the copy processes. I followed the logs pretty closely during and after, here is how it starts:

The end result of the process was that:

The errors related to System Out of Memory issues:

During the process I watched the Task Manager and the MLMigrationLL.exe grab memory. I believe that when this process fails to grab memory in sufficient time the file copy fails.
I did not see this activity on the SharePoint 2007 server, but I might have just missed it.

I increased my test box to 3gb of memory and set SQL server to max at 1GB. I notices that the SQL server was picking up memory (as it should) during the last process and I want to make sure that there is sufficient for MLMigrationLL.exe.

You can see the memory ramp up when it gets to those files:

Plenty available:

Still getting out of memory exceptions though. This is a bummer. I will have to send these results to Jeremy:

Testing Tools and Addons - Metalogix SSMM with Large Document Libraries

2009-05-20T10:55:00.003-04:00

Overview

I am looking at using Metalogix SharePoint Site Migration Manager in a new 2003 to 2007 migration project. I like the interface and flexibility of the program, but this client has large files and large document libraries which is a consideration for the migration process. In a typical migration I would move entire databases, with Metalogix I can rebuild the sites in to SharePoint 2007 one element at a time. I am on a tight schedule once the migration starts so I need to factor in how much time an item by item rebuild will take and what Metalogix will do when it encounters a 20 GB Document Library. First thing I need to do is prepare the test site in my VM environment.

I am putting 127GB drives on my test VMs for the migration:

Set the defaults in SQL Server to make sure databases are going to the E:\ I created:

Added a new content database in Central Admin and set the portal content db to “offline” so that new sites would be on my e: database.

Most of the content I am migrating is WSS 2.0 sites within the Portal so that is what I am creating:

Better make sure I ease up on upload limits, the sites that I am migrating have a 650mb upload limit because of the type of data they store. I am going to use iso images of various sizes in my test some of them really large. Let’s see if it stutters with a bunch of 600mb isos.

Copying files into the Document Library may take a while: