SharePoint Interface

There have been some great SharePoint events recently, and quite a few more are coming up.  Here are some of the events I have been (or will be) involved in recently/soon:

SharePoint Cincy

On March 18th, Cincinnati saw it’s first (arguably) “major” SharePoint event.  SharePoint Cincy was put together by Geoff Smith, the Cincinnati CIO Roundtable, and MAX Training … and it was a huge success by any measure.  I took the picture on the right during the introductory speech by Geoff Smith, and it should give you an idea of well-attended the event was.

I was fortunate enough to deliver a talk on disaster recovery during the day, and I also helped Geoff and the organizing committee in advance of the event with some of the speaker round-up for the IT professional / administrator track.

I enjoyed the event because the audience composition was different than one would typically find at a SharePoint Saturday event.  Many of those in attendance were IT decision makers and managers rather than implementers and developers.  I attribute the high numbers in my DR session (typically not a big pull with technical crowds) to that demographic difference.

The next SharePoint Cincy event is already planned for next year (on March 12th, I believe), so keep your eyes peeled at the beginning of next year!

SharePoint Saturday Twin Cities

Some fine folks in the Minneapolis and St. Paul area (Jim Ferguson, Sarah Haase, and Wes Preston)  have worked to assemble the upcoming SharePoint Saturday Twin Cities on April 9, 2011.  Like all SharePoint Saturdays, the event is free for attendees.  There’s plenty of good education and giveaways to make it worth your while to spend a Saturday with other SharePoint enthusiasts.

I’ll be heading out to the event to deliver my IT pro caching talk titled “’Caching-In’ for SharePoint Performance”  The abstract for the session appears below

Caching is a critical variable in the SharePoint scalability and performance equation, but it’s one that’s oftentimes misunderstood or dismissed as being needed only in Internet-facing scenarios.  In this session, we’ll build an understanding of the caching options that exist within the SharePoint platform and how they can be leveraged to inject some pep into most SharePoint sites.  We’ll also cover some sample scenarios, caching pitfalls, and watch-outs that every administrator should know.

If you’re in the Twin Cities area and available on Saturday April 9th, come on by the following address …

Normandale Community College
9700 France Avenue South
Bloomington, MN 55431

… for a day of high-quality and free training.  You can register here on Eventbrite!

Lunch and Learn with Prinomic

Idera partners with a number of different companies in order to make its software more available, and one of those companies is Prinomic Technologies.  Prinomic is a consulting company based out of Denver, Colorado, and they focus on the creation of solutions that employ and target SharePoint.  They are somewhat unique in that they offer a combination of both services and products, and it affords them a great deal of flexibility when addressing customer needs.

I’ll actually be traveling out to Denver to deliver a lunch-and-learn in conjunction with Prinomic titled “SharePoint Disaster Recovery Options” on Wednesday, April 13th, 2011.  The lunch and learn is open to the public; simply follow the link (above) to register if you’re interested.

Prinomic is located at the following address:

4600 S Syracuse
9th floor
Denver, CO 80237

Colorado Springs SharePoint User Group

Knowing that I’d be out in the Denver area on April 13th, I reached out to some of the folks I know there to see if I might coordinate something with one of the local user groups.  I love speaking, and it was my hope that someone would actually grant me some more time with additional SharePoint geeks!

I was very fortunate to get a reply back from Dave Milner.  Dave and Gary Lapointe run the Colorado Springs SharePoint User Group, and they mentioned that it would be okay for me to come by and present to their group on the evening of the 13th.  So, it looks like I’ll be heading down to Colorado Springs after the lunch and learn with Prinomic!

I’ll be presenting my 2010 DR talk titled “SharePoint 2010 and Your DR Plan: New Capabilities, New Possibilities!”

Disaster recovery planning for a SharePoint 2010 environment is something that must be performed to insure your data and the continuity of business operations. Microsoft made significant enhancements to the disaster recovery landscape with SharePoint 2010, and we’ll be taking a good look at how the platform has evolved in this session. We’ll dive inside the improvements to the native backup and restore capabilities that are present in the SharePoint 2007 platform to see what has been changed and enhanced. We’ll also look at the array of exciting new capabilities that have been integrated into the SharePoint 2010 platform, such as unattended content database recovery, SQL Server snapshot integration, and configuration-only backup and restore. By the time we’re done, you will possess a solid understanding of how the disaster recovery landscape has changed with SharePoint 2010.

If you’re in the Colorado Springs area on Wednesday, April 13th, come on by the user group!  The user group meets at the following address:

Cobham Analytics
985 Space Center Drive
Suite 100
Colorado Springs, CO

Meet-and-greet activities are from 5:30pm until 6pm, and the session begins at 6pm!

TechNet Events: Transforming IT from Virtualization to the Cloud

Finally, I wanted to mention a series of events that are both going on right now and coming soon.  My good friend Matt Hester, our region’s IT Pro Evangelist with Microsoft, is traveling around putting on a series of Technet events titled “Transforming IT from Virtualization to the Cloud.”  These events are free training and center on cloud computing, why it is important, private vs. public cloud options, etc.

The event looks really cool, and it’s being offered in a number of different cities in the region.  I’ve already signed up for the Cincinnati event on April 6th (a Wednesday).  Check out Matt’s blog post (linked above) for additional details.  If you want to sign up for the Cincinnati event on April 6th, you can use this link directly.

Additional Reading and References

1. Event: SharePoint Cincy
2. People: Geoff Smith
3. Blog: Jim Ferguson
4. Twitter: Sarah Haase
5. Blog: Wes Preston
6. Event: SharePoint Saturday Twin Cities
7. Registration: SharePoint Saturday Twin Cities on Eventbrite
8. Company: Idera
9. Company: Prinomic Technologies
10. Lunch and Learn: “SharePoint Disaster Recovery Options”
11. LinkedIn: Dave Milner
12. LinkedIn: Gary Lapointe
13. Technet Event: “Transforming IT from Virtualization to the Cloud”
14. Technet Event: Cincinnati cloud event

I first presented (in some organized capacity) on SharePoint’s platform caching capabilities at SharePoint Saturday Ozarks in June of 2010, and since that time I’ve consistently received a growing number of questions on the topic of SharePoint BLOB caching.  When I start talking about BLOB caching itself, the area that seems to draw the greatest number of questions and “really?!?!” responses is the use of the max-age attribute and how it can profoundly impact client-server interactions.

I’d been promising a number of people (including Todd Klindt and Becky Bertram) that I would write a post about the topic sometime soon, and recently decided that I had lollygagged around long enough.

Before I go too far, though, I should probably explain why the max-age attribute is so special … and even before I do that, we need to agree on what “caching” is and does.

Caching 101

Why does SharePoint supply caching mechanisms?  Better yet, why does any application or hardware device employ caching?  Generally speaking, caching is utilized to improve performance by taking frequently accessed data and placing it in a state or location that facilitates faster access.  Faster access is commonly achieved through one or both of the following mechanisms:

• By placing the data that is to be accessed on a faster storage medium; for example, taking frequently accessed data from a hard drive and placing it into memory.
• By placing the data that is to be accessed closer to the point of usage; for example, offloading files from a server that is halfway around the world to one that is local to the point of consumption to reduce round-trip latency and bandwidth concerns.  For Internet traffic, this scenario can be addressed with edge caching through a content delivery network such as that which is offered by Akamai’s EdgePlatform.

Oftentimes, data that is cached is expensive to fetch or computationally calculate.  Take the digits in pi (3.1415926535 …) for example.  Computing pi to 100 decimals requires a series of mathematical operations, and those operations take time.  If the digits of pi are regularly requested or used by an application, it is probably better to compute those digits once and cache the sequence in memory than to calculate it on-demand each time the value is needed.

Caching usually improves performance and scalability, and these ultimately tend to translate into a better user experience.

SharePoint and caching

Through its publishing infrastructure, SharePoint provides a number of different platform caching capabilities that can work wonders to improve performance and scalability.  Note that yes, I did say “publishing infrastructure” – sorry, I’m not talking about Windows SharePoint Services 3 or SharePoint Foundation 2010 here.

With any paid version of SharePoint, you get object caching, page output caching, and BLOB caching.  With SharePoint 2010 and the Office Web Applications, you also get the Office Web Applications Cache (for which I highly recommend this blog post written by Bill Baer).

Each of these caching mechanisms and options work to improve performance within a SharePoint farm by using a combination of the two mechanisms I described earlier.  Object caching stores frequently accessed property, query, and navigational data in memory on WFEs.  Basic BLOB caching copies images, CSS, and similar resource data from content databases to the file system of WFEs.  Page output caching piggybacks on ASP.NET page caching and holds SharePoint pages (which are expensive to render) in memory and serves them back to users.  The Office Web Applications Cache stores the output of Word documents and PowerPoint presentations (which is expensive to render in web-accessible form) in a special site collection for subsequent re-use.

Public-facing SharePoint

Each of the aforementioned caching mechanisms yields some form of performance improvement within the SharePoint farm by reducing load or processing burden, and that’s all well and good … but do any of them improve performance outside of the SharePoint farm?

What do I even mean by “outside of the SharePoint farm?”  Well, consider a SharePoint farm that serves up content to external consumers – a standard/typical Internet presence web site.  Most of us in the SharePoint universe have seen (or held up) the Hawaiian Airlines and Ferrari websites as examples of what SharePoint can do in a public-facing capacity.  These are exactly the type of sites I am focused on when I ask about what caching can do outside of the SharePoint farm.

For companies that host public-facing SharePoint sites, there is almost always a desire to reduce load and traffic into the web front-ends (WFEs) that serve up those sites.  These companies are concerned with many of the same performance issues that concern SharePoint intranet sites, but public-facing sites have one additional concern that intranet sites typically don’t: Internet bandwidth.

Even though Internet bandwidth is much easier to come by these days than it used to be, it’s not unlimited.  In the age of gigabit Ethernet to the desktop, most intranet users don’t think about (nor do they have to concern themselves with) the actual bandwidth to their SharePoint sites.  I can tell you from experience that such is not the case when serving up SharePoint sites to the general public

So … for all the platform caching options that SharePoint has, is there anything it can actually do to assist with the Internet bandwidth issue?

Enter BLOB caching and the max-age attribute

As it turns out, the answer to that question is “yes” … and of course, it centers around BLOB caching and the max-age attribute specifically.  Let’s start by looking at the <BlobCache /> element that is present in every SharePoint Server 2010 web.config file.

BLOB caching disabled

<BlobCache location="C:\BlobCache\14" path="\.(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled="false" />

This is the default <BlobCache /> element that is present in all starting SharePoint Server 2010 web.config files, and astute readers will notice that the enabled attribute has a value of false.  In this configuration, BLOB caching is turned off and every request for BLOB resources follows a particular sequence of steps.  The first request in a browser session looks like this:

In this series of steps

1. A request for a BLOB resource is made to a WFE
2. The WFE fetches the BLOB resource from the appropriate content database
3. The BLOB is returned to the WFE
4. The WFE returns an HTTP 200 status code and the BLOB to the requester

Here’s a section of the actual HTTP response from server (step #4 above):

HTTP/1.1 200 OK 
Cache-Control: private,max-age=0 
Content-Length: 1241304 
Content-Type: image/jpeg 
Expires: Tue, 09 Nov 2010 14:59:39 GMT 
Last-Modified: Wed, 24 Nov 2010 14:59:40 GMT 
ETag: "{9EE83B76-50AC-4280-9270-9FC7B540A2E3},7" 
Server: Microsoft-IIS/7.5 
SPRequestGuid: 45874590-475f-41fc-adf6-d67713cbdc85

You’ll notice that I highlighted the Cache-Control header line.  This line gives the requesting browser guidance on what it should and shouldn’t do with regard to caching the BLOB resource (typically an image, CSS file, etc.) it has requested.  This particular combination basically tells the browser that it’s okay to cache the resource for the current user, but the resource shouldn’t be shared with other users or outside the current session.

Since the browser knows that it’s okay to privately cache the requested resource, subsequent requests for the resource by the same user (and within the same browser session) follow a different pattern:

When the browser makes subsequent requests like this for the resource, the HTTP response (in step #2) looks different than it did on the first request:

HTTP/1.1 304 NOT MODIFIED 
Cache-Control: private,max-age=0 
Content-Length: 0 
Expires: Tue, 09 Nov 2010 14:59:59 GMT

A request is made and a response is returned, but the HTTP 304 status code indicates that the requested resource wasn’t updated on the server; as a result, the browser can re-use its cached copy.  Being able to re-use the cached copy is certainly an improvement over re-fetching it, but again: the cached copy is only used for the duration of the browser session – and only for the user who originally fetched it.  The requester also has to contact the WFE to determine that the cached copy is still valid, so there’s the overhead of an additional round-trip to the WFE for each requested resource anytime a page is refreshed or re-rendered.

BLOB caching enabled

Even if you’re not a SharePoint administrator and generally don’t poke around web.config files, you can probably guess at how BLOB caching is enabled after reading the previous section.  That’s right: it’s enabled by setting the enabled attribute to true as follows:

<BlobCache location="C:\BlobCache\14" path="\.(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled="true" />

When BLOB caching is enabled in this fashion, the request pattern for BLOB resources changes quite a bit.  The first request during a browser session looks like this:

In this series of steps

1. A request for a BLOB resource is made to a WFE
2. The WFE returns the BLOB resource from a file system cache

The gray arrow that is shown indicates that at some point, an initial fetch of the BLOB resource is needed to populate the BLOB cache in the file system of the WFE.  After that point, the resource is served directly from the WFE so that subsequent requests are handled locally for the duration of the browser session.

As you might imagine based on the interaction patterns described thus far, simply enabling the BLOB cache can work wonders to reduce the load on your SQL Servers (where content databases are housed) and reduce back-end network traffic.  Where things get really interesting, though, is on the client side of the equation (that is, the Requester’s machine) once a resource has been fetched.

What about the max-age attribute?

You probably noticed that a max-age attribute wasn’t specified in the default (previous) <BlobCache /> element.  That’s because the max-age is actually an optional attribute.  It can be added to the <BlobCache /> element in the following fashion:

<BlobCache location="C:\BlobCache\14" path="\.(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled="true" max-age=”43200” />

Before explaining exactly what the max-age attribute does, I think it’s important to first address what it doesn’t do and dispel a misconception that I’ve seen a number of times.  The max-age attribute has nothing to do with how long items stay within the BLOB cache on the WFE’s file system.  max-age is not an expiration period or window of viability for content on the WFE.  The server-side BLOB cache isn’t like other caches in that items expire out of it.  New assets will replace old ones via a maintenance thread that regularly checks associated site collections for changes, but there’s no regular removal of BLOB items from the WFE’s file system BLOB cache simply because of age.  max-age has nothing to do with server side operations.

So, what does the max-age attribute actually do then?  Answer: it controls information that is sent to requesters for purposes of specifying how BLOB items should be cached by the requester.  In short: max-age controls client-side cacheability.

The effect of the max-age attribute

max-age values are specified in seconds; in the case above, 43200 seconds translates into 12 hours.  When a max-age value is specified for BLOB caching, something magical happens with BLOB requests that are made from client browsers.  After a BLOB cache resource is initially fetched by a requester according to the previous “BLOB caching enabled” series of steps, subsequent requests for the fetched resource look like this for a period of time equal to the max-age:

You might be saying, “hey, wait a minute … there’s only one step there.  The request doesn’t even go to the WFE?”  That’s right: the request doesn’t go to the WFE.  It gets served directly from local browser cache – assuming such a cache is in use, of course, which it typically is.

Why does this happen?  Let’s take a look at the HTTP response that is sent back with the payload on the initial resource request when BLOB caching is enabled:

HTTP/1.1 200 OK 
Cache-Control: public, max-age=43200 
Content-Length: 1241304 
Content-Type: image/jpeg 
Last-Modified: Thu, 22 May 2008 21:26:03 GMT 
Accept-Ranges: bytes 
ETag: "{F60C28AA-1868-4FF5-A950-8AA2B4F3E161},8pub" 
Server: Microsoft-IIS/7.5 
SPRequestGuid: 45874590-475f-41fc-adf6-d67713cbdc85

The Cache-Control header line in this case differs quite a bit from the one that was specified when BLOB caching was disabled.  First, the use of public instead of private tells the receiving browser or application that the response payload can be cached and made available across users and sessions.  The response header max-age attribute maps directly to the value specified in the web.config, and in this case it basically indicates that the payload is valid for 12 hours (43,200 seconds) in the cache.  During that 12 hour window, any request for the payload/resource will be served directly from the cache without a trip to the SharePoint WFE.

Implications that come with max-age

On the plus side, serving resources directly out of the client-side cache for a period of time can dramatically reduce requests and overall traffic to WFEs.  This can be a tremendous bandwidth saver, especially when you consider that assets which are BLOB cached tend to be larger in nature – images, media files, etc.  At the same time, serving resources directly out of the cache is much quicker than round-tripping to a WFE – even if the round trip involves nothing more than an HTTP 304 response to say that a cached resource may be used instead of being retrieved.

While serving items directly out of the cache can yield significant benefits, I’ve seen a few organizations get bitten by BLOB caching and excessive max-age periods.  This is particularly true when BLOB caching and long max-age periods are employed in environments where images and other BLOB cached resources are regularly replaced and changed-out.  Let me illustrate with an example.

Suppose a site collection that hosts collaboration activities for a graphic design group is being served through a Web application zone where BLOB caching is enabled and a max-age period of 43,200 seconds (12 hours) is specified.  One of the designers who uses the site collection arrives in the morning, launches her browser, and starts doing some work in the site collection.  Most of the scripts, CSS, images, and other BLOB assets that are retrieved will be cached by the user’s browser for the rest of the work day.  No additional fetches for such assets will take place.

In this particular scenario, caching is probably a bad thing.  Users trying to collaborate on images and other similar (BLOB) content are probably going to be disrupted by the effect of BLOB caching.  The max-age value (duration) in-use would either need to be dialed-back significantly or BLOB caching would have to be turned-off entirely.

What you don’t see can hurt you

There’s one more very important point I want to make when it comes to BLOB caching and the use of the max-age attribute: the default <BlobCache /> element doesn’t come with a max-age attribute value, but that doesn’t mean that there isn’t one in-use.  If you fail to specify a max-age attribute value, you end up with the default of 86,400 seconds – 24 hours.

This wasn’t always the case!  In some recent exploratory work I was doing with Fiddler, I was quite surprised to discover client-side caching taking place where previously it hadn’t.  When I first started playing around with BLOB caching shortly after MOSS 2007 was released, omitting the max-age attribute in the <BlobCache /> element meant that a max-age value of zero (0) was used.  This had the effect of caching BLOB resources in the file system cache on WFEs without those resources getting cached in public, cross-session form on the client-side.  To achieve extended client-side caching, a max-age value had to be explicitly assigned.

Somewhere along the line, this behavior was changed.  I’m not sure where it happened, and attempts to dig back through older VM images (for HTTP response comparisons) didn’t give me a read on when Microsoft made the change.  If I had to guess, though, it probably happened somewhere around service pack 1 (SP1).  That’s strictly a guess, though.  I had always gotten into the habit of explicitly including a max-age value – even if it was zero – so it wasn’t until I was playing with the BLOB caching defaults in a SharePoint 2010 environment that I noticed the 24 hour client-side caching behavior by default.  I then backtracked to verify that the behavior was present in both SharePoint 2007 and SharePoint 2010, and it affected both authenticated and anonymous users.  It wasn’t a fluke.

So watch-out: if you don’t specify a max-age value, you’ll get 24 hour client-side caching by default!  If users complain of images that “won’t update” and stale BLOB-based content, look closely at max-age effects.

An alternate viewpoint on the topic

As I was finishing up this post, I decided that it would probably be a good idea to see if anyone else had written on this topic.  My search quickly turned up Chris O’Brien’s “Optimization, BLOB caching and HTTP 304s” post which was written last year.  It’s an excellent read, highly informative, and covers a number of items I didn’t go into.

Throughout this post, I took the viewpoint of a SharePoint administrator who is seeking to control WFE load and Internet bandwidth consumption.  Chris’ post, on the other hand, was written primarily with developer and end-user concerns in mind.  I wasn’t aware of some of the concerns that Chris points out, and I learned quite a few things while reading his write-up.  I highly recommend checking out his post if you have a moment.

Additional Reading and References

1. Event: SharePoint Saturday Ozarks (June 2010)
2. Blob Post: We Drift Deeper Into the Sound … as the (BLOB Cache) Flush Comes
3. Blog: Todd Klindt’s SharePoint Admin Blog
4. Blog: Becky Bertram’s Blog
5. Definition: lollygag
6. Technology: Akamai’s EdgePlatform
7. Wikipedia: Pi
8. TechNet: Cache settings operations (SharePoint Server 2010)
9. Bill Baer: The Office Web Applications Cache
10. SharePoint Site: Hawaiian Airlines
11. SharePoint Site: Ferrari
12. W3C Site: Cache-Control explanations
13. Tool: Fiddler
14. Blog Post: Chris O’Brien: Optimization, BLOB caching and HTTP 304s

Holy smokes!  2011 is off to a fast start, and February is already here.  Now that our product release is out (see below), I’m going to make good on my promise to get more “real” blog content out.  Before I do that, though, I want to highlight all of the great SharePoint stuff I’m fortunate enough to be involved with during the month of February.

Idera SharePoint backup 3.0 Release

As some of you know, I’m currently a Product Manager for SharePoint products at Idera.  Although it isn’t something that is strictly community focused, Idera SharePoint backup has been a large part of my life for most of the last year.  We’ve been doing some really cool development and product work, and I want to share a piece of good news: We just released version 3.0 of Idera SharePoint backup!

Idera SharePoint backup is “my” product from a management standpoint, and I’m really proud of all the effort that our team put in to make the release a reality.  There are a lot of people in many different areas who busted their butts to get the product out-the-door: development, quality assurance, information development, marketing, product marketing management, public relations, web site management, sales, sales engineering, and more.

To everyone who contributed to making the release a success: you have my heartfelt thanks.  To the rest of you who might be shopping for a SharePoint backup product, please check out what we’ve put together!

SPTechCon San Francisco

I’ll be heading out to BZ Media’s SPTechCon in San Francisco for most of the week of February 7th.  Although I will be delivering a lightning talk titled “Backup/Restore Knowledge Nuggets: What’s True, What’s Not?” (more-or-less the same talk I delivered at last Fall’s SPTechCon event in Boston) on Monday the 7th, that’s only a small part of why I’ll be at the conference.

The big stuff?  Well, first off is the big “public release” of Idera SharePoint backup 3.0.  I’ll be talking with conference participants, seeing what they like (and what they don’t like), explaining the new capabilities and features, etc.

My good friend (and co-author) John Ferringer and I will also be doing an Idera-sponsored book signing for our SharePoint 2010 Disaster Recovery Guide.  If you’re going to be at the conference and want to get a free (and signed!) copy of our book, come by booth #302 on Wednesday morning (2/9) at 10am during the coffee and donuts.  We’ll be around and easy to find: John will be the thin bald guy, and I’ll be the mostly bald guy next to him shoveling donuts into his mouth.  I have a tremendous weak spot for donuts …

Presenting for the Rochester SharePoint User Group

Rick Black is one of the organizers of the Rochester (New York) SharePoint User Group.  I met Rick late in 2009 at SharePoint Saturday Cleveland in Cleveland, Ohio; he and I were both presenting sessions.  We talked a bit during the event, and we pinged each other now and again on Twitter and Facebook in the time after the event.

At one point in time, Rick tossed out the idea of having me present for the Rochester SPUG.  I told him I’d certainly be up for it; after all, I really enjoy hanging out with SharePoint users and talking shop.  The trick, of course, would be getting me to Rochester.

Recently, I asked Rick if he thought a virtual SPUG presentation might work for his group.  I do quite a bit of time presenting on Live Meeting these days, so I figured it might be an option.  It sounded like a good idea to Rick, so I’m on the schedule to (virtually) present for the Rochester SPUG on Thursday, February 10th, 2011.  I’ll be presenting Saving SharePoint – a time-tested and refined SharePoint disaster recovery talk.  The abstract reads as follows:

In this session, we will be discussing disaster recovery (DR) concepts and strategies for SharePoint in a format that highlights a combination of both business and technical concerns.  We will define some critical planning terms such as “recovery time objectives” and “recovery point objectives,” and we’ll see why they are so important to understand when trying to formulate a DR strategy.  We will also identify the capabilities and limitations of the Microsoft tools that can used for backing up, restoring, and making SharePoint highly available for disaster recovery purposes.  Changes that have arrived with SharePoint Server 2010 and how they affect DR planning and implementation will also be covered.

I’ll be presenting the night that I get home on a red-eye flight from SPTechCon, so I could be a bit weary … but it will be fun.  I’m really looking forward to it!

SharePoint Saturday San Diego

For the last weekend of February, I’ll be heading back out to the west coast for SharePoint Saturday San Diego.  The event itself will be held at the San Diego Convention Center on Saturday, February 26th.  The event has filled-up once already, but Chris Givens (who is organizing the event) was able to add another 75 tickets thanks to some additional support from sponsors.

In addition to my Saving SharePoint session (which is described earlier in this post), I’ll be delivering another session called “Caching-In” for SharePoint Performance.  The abstract for the session reads as follows:

Caching is a critical variable in the SharePoint scalability and performance equation, but it’s one that’s oftentimes misunderstood or dismissed as being needed only in Internet-facing scenarios.  In this session, we’ll build an understanding of the caching options that exist within the SharePoint platform and how they can be leveraged to inject some pep into most SharePoint sites.  We’ll also cover some sample scenarios, caching pitfalls, and watch-outs that every administrator should know.

As always, SharePoint Saturday events are free to the public.  They’re a great way to get a day’s worth of free training, access to SharePoint experts, and plenty of swag and info from vendors.  If you live in or around San Diego and are free on 2/26, consider signing up!

Two trips to the west coast in one month is definitely a first for me, but I’m looking forward to it.  I hope to see you out there!

Additional Reading and References

1. Company: Idera
2. Product: Idera SharePoint backup
3. Company: BZ Media
4. Event: SPTechCon San Francisco
5. Event: SPTechCon SharePoint Lightning Talks
6. Blog: John Ferringer’s My Central Admin
7. Book: SharePoint 2010 Disaster Recovery Guide
8. Twitter: Rick Black (@ricknology)
9. User Group: Rochester SharePoint User Group
10. Events: SharePoint Saturday San Diego
11. Twitter: Chris Givens (@givenscj)

Thanksgiving has passed, December is right around the corner, and it seems that many folks I know are settling into the holiday mindset.  There’s very little slowdown of activity when it comes to SharePoint, though; one need only look at the SharePoint Saturday home page to see that the first half of December is jam-packed with events.

I’ll be participating in a couple of SharePoint events myself in December, and I thought I’d share those.

Cincinnati SharePoint User Group (CincySPUG)

It’s been quite a while since I’ve attended the monthly Cincinnati SharePoint User Group meetings, and I’ve been meaning to get back to some level of involvement for a while now.  Family and work-related obligations have tied me up more often than not, and then there’s the fact that the meeting location is a solid 40 minutes from my house – and that’s before factoring in rush-hour traffic.

All excuses aside, I’m happy to announce that I’ll be speaking at this month’s CincySPUG meeting.  I’ll be delivering the SharePoint 2010 Disaster Recovery: New Capabilities, New Possibilities! talk that I recently delivered at SharePoint Saturday Dallas.  Here’s the abstract:

Disaster recovery planning for a SharePoint 2010 environment is something that must be performed to insure your data and the continuity of business operations. Microsoft made significant enhancements to the disaster recovery landscape with SharePoint 2010, and we’ll be taking a good look at how the platform has evolved in this session. We’ll dive inside the improvements to the native backup and restore capabilities that are present in the SharePoint 2007 platform to see what has been changed and enhanced. We’ll also look at the array of exciting new capabilities that have been integrated into the SharePoint 2010 platform, such as unattended content database recovery, SQL Server snapshot integration, and configuration-only backup and restore. By the time we’re done, you will possess a solid understanding of how the disaster recovery landscape has changed with SharePoint 2010.

I put the talk together using Prezi, so its style and flow is inherently different from the standard PowerPoint deck you’re probably used to seeing.

The December 2010 meeting of the CincySPUG is at MAX Technical Training on Thursday, December 2nd.  Socializing starts at 6pm, and the actual presentation goes from 6:30pm until 8pm.  If you have the time and availability, it would be great to see you.

Oh, and here’s (perhaps) a little extra incentive: I’ll be giving away a copy of the new SharePoint 2010 Disaster Recovery Guide that John Ferringer and I co-authored.  Come on out!

SharePoint Saturday Kansas City

I mentioned that there are a number of SharePoint Saturday events during the first half of December, and I’m fortunate enough to be participating in one them!

SharePoint Saturday Kansas City will be held on Saturday December 11th at the Johnson County Community College Regnier Center (12345 College Blvd., Overland Park, KS 66210-1299).  The event is a full day of free SharePoint training; all you need to do is register through the Eventbrite site and show up!

I’ll be presenting the tried-and-true Saving SharePoint session.  Here’s the abstract:

In this session, we will be discussing disaster recovery (DR) concepts and strategies for SharePoint in a format that highlights a combination of both business and technical concerns.  We will define some critical planning terms such as “recovery time objectives” and “recovery point objectives,” and we’ll see why they are so important to understand when trying to formulate a DR strategy.  We will also identify the capabilities and limitations of the Microsoft tools that can used for backing up, restoring, and making SharePoint highly available for disaster recovery purposes.  Changes that have arrived with SharePoint Server 2010 and how they affect DR planning and implementation will also be covered.

I’m not one to simply leave a presentation alone.  I’ve delivered Saving SharePoint a number of times, but I’ve found a few new goodies to work into it for this go ‘round – should be fun!

Additional Reading and References

1. Site: SharePoint Saturday Home Page
2. Group: Cincinnati SharePoint User Group
3. Event: SharePoint Saturday Dallas
4. Tools: Prezi
5. Training: MAX Technical Training
6. Book: SharePoint 2010 Disaster Recovery Guide
7. Blog: John Ferringer’s MyCentralAdmin
8. Event: SharePoint Saturday Kansas City
9. Site: Eventbrite Registration for SPS KC

Last week, I was in Boston for BZ Media’s SPTechCon Boston event.  It was a great opportunity to see and spend time with many of my friends in the SharePoint community, do a book signing with John Ferringer and Idera, and take in a few sessions.

Although I wasn’t technically a presenter at the conference, I did deliver a “lightning talk” on the first day of the conference.  Lightning talks are five minute presentations that are typically given by sponsors and designed to expose audiences (who are usually chowing-down on food) to the sponsors’ services, products, etc.

I was given Idera’s slot to speak, and I was also given the latitude to basically do whatever I wanted … so, I decided to have some fun with it!

The Lightning Talk Itself

The five minute presentation that appears below is titled Backup/Restore Knowledge Nuggets: What’s True, What’s Not?  If you weren’t at SPTechCon and can spare a few minutes, I think you’ll find the presentation to be both amusing and informative.

Follow the link and try it out!  You’ll find that the play button allows you to step through the presentation from start to finish pretty easily.

Prezi has a very slick mechanism for embedding actual presentations directly into a website, but that isn’t an option here on my blog.  WordPress.com hosts my blog, and they strip out anything with <object> tags.  I tried to embed the presentation directly, but it got me nowhere  :-(

Wait, What’s Prezi?

I recently became hooked on Prezi (the product + service that drove both the lightning talk and the link that I included above) when I saw Peter Serzo use it to deliver his service application session at SharePoint Saturday Columbus.  Prior to Prezi, I did everything with PowerPoint.  Once I saw Prezi in action and got some additional details from Peter, though, I knew that I’d be using Prezi before long.

I don’t see Prezi as a replacement for PowerPoint, but I do see it as a nice complement.  PowerPoint is great for presenting sequential sets of data and information, and it excels in situations where a linear delivery is desirable.

Prezi, on the other hand, is fantastic for talks and presentation where jumping around happens a lot – such as you might do when trying to tie several points back to a central theme or concept.  Prezi isn’t nearly as full-featured as PowerPoint, but I find that it can be more visually engaging and simply “fun.”

Wrapping It Up

The lightning talk at SPTechCon was the perfect arena for a test run with Prezi, and I think the presentation went wonderfully… and was a whole lot of fun to deliver, as well.  I certainly see myself using Prezi more in the future.  SharePoint Saturday Dallas is coming up in just a couple of weeks …

If you take the time to watch the presentation, I’d really love to hear your feedback!

Additional Reading and References

1. Company: BZ Media LLC
2. Book: SharePoint 2010 Disaster Recovery Guide
3. Blog: John Ferringer’s MyCentralAdmin
4. Company: Idera
5. Presentation: Backup/Restore Knowledge Nuggets: What’s True, What’s Not?
6. Product: Prezi
7. Twitter: Peter Serzo (@pserzo)
8. Event: SharePoint Saturday Columbus
9. Event: SharePoint Saturday Dallas

Fall is here, and the SharePoint bus keeps on rolling down the road.  There’s no shortage of events coming up – conferences, SharePoint Saturdays, and more.  Here are a couple of events in which I’ll be participating.

SPTechCon Boston

In a couple of days, I’ll be heading up to Boston for SPTechCon Boston 2010.  The event is put on by Dave Rubenstein of BZ Media, and it promises to be one of the bigger SharePoint conferences of this year.

Although I’m presenting a “lightning talk” on Wednesday the 20th titled Backup/Restore Knowledge Nuggets: What’s True, What’s Not?, it’s only five minutes long … and not the main reason I’m going to the conference.  To tell you the truth, I’m simply looking forward to taking in some of the sessions and seeing many of my friends in the community whom I haven’t seen in a while.

My co-author, John Ferringer, is one of those folks I haven’t seen in a while – since SharePoint Saturday Columbus, I believe. Thanks to the folks at Idera, the two of us will be getting the band back together to do a book signing on Friday morning (the 22nd) at 9:45am during coffee and donuts.  Idera purchased 20 copies of our SharePoint 2010 Disaster Recovery Guide, and they’ll be giving them away (see the poster on the right).  John and I will be signing those books, so if you want to meet a couple of flagship members in the “SharePoint Mr. Clean Team” (to quote SharePoint superstar and all-around great person, Joy Earles), please swing by the Idera booth!

SharePoint Saturday Dallas

I knew that I was going to be down in Houston for some business during the second week of November, so when I learned that Eric Shupps was in the process of pulling things together for SharePoint Saturday Dallas during the same time frame (Saturday, November 13th), I pinged him to see if he could use another speaker.  He pinged me back, and it looks like I’ll be making a stop in Dallas on my way back to Cincinnati.

The session I’ll be presenting is titled SharePoint 2010 and Your DR Plan: New Capabilities, New Possibilities!, and it’s a relatively new one for me.  It’s a disaster recovery talk, but it’s primarily a technology-focused look at the new platform capabilities and improvements that come with SharePoint 2010.  Here’s the abstract:

Disaster recovery planning for a SharePoint 2010 environment is something that must be performed to insure your data and the continuity of business operations. Microsoft made significant enhancements to the disaster recovery landscape with SharePoint 2010, and we’ll be taking a good look at how the platform has evolved in this session. We’ll dive inside the improvements to the native backup and restore capabilities that are present in the SharePoint 2007 platform to see what has been changed and enhanced. We’ll also look at the array of exciting new capabilities that have been integrated into the SharePoint 2010 platform, such as unattended content database recovery, SQL Server snapshot integration, and configuration-only backup and restore. By the time we’re done, you will possess a solid understanding of how the disaster recovery landscape has changed with SharePoint 2010.

The SharePoint Saturday event is being held at the Hilton Dallas Park Cities from 9am until 5:30pm on Saturday, November 13th.  If you work with SharePoint and reside in or around the Dallas area, I strongly encourage you to sign up for the event and come on out.  Like all SharePoint Saturday events, there’s no cost to you – it’s simply a free day of training, food, giveaways, and interaction with the SharePoint community!

Additional Reading and References

1. Event: SPTechCon Boston 2010
2. Company: BZ Media
3. Blog: John Ferringer’s My Central Admin
4. Company: Idera
5. Book: SharePoint 2010 Disaster Recovery Guide
6. Twitter: Joy Earles
7. Event: SharePoint Saturday Dallas
8. Blog: Eric Shupps Blog
9. Venue: Hilton Dallas Park Cities
10. Registration: SharePoint Saturday Dallas

I recently authored a blog post titled “Configuration-Only Backup and Restore in SharePoint 2010,” and in that post I tried to address some of the false hopes and misunderstandings I saw arising around SharePoint 2010’s configuration-only backup and restore capabilities.

While I was putting the post together, I was reminded of another head-scratcher that I’ve seen confuse some folks on a handful of occasions; specifically, what the differences are between the Backup-SPFarm and Backup-SPConfigurationDatabase PowerShell cmdlets in SharePoint 2010 when it comes to configuration-only backup.

Before I go too far, I should probably rewind a bit and explain a few things.

Many Paths to the Destination

If you aren’t yet familiar with configuration-only backup and restore in SharePoint 2010, the basics of it are covered in this TechNet article and in my previous post.  I’d recommend checking both out before continuing.

Configuration-only backups in SharePoint 2010 can be generated in several different ways;

1. Using the “Backup up only configuration settings” option when running a backup using Central Administration’s Farm Backup and Restore capabilities.
2. Through STSADM.exe using STSADM.exe –o backup with the –ConfigurationOnly switch.
3. By running the Backup-SPFarm PowerShell cmdlet along with the –ConfigurationOnly switch.
4. By executing the Backup-SPConfigurationDatabase PowerShell cmdlet.

Option #1 is obviously designed for the “UI-oriented” administrator who wants to accomplish a configuration-only backup with a point-and-click interface.  Option #2 works, but Microsoft has been pretty clear that STSADM.exe is on its way out and should be generally be avoided in favor of the PowerShell cmdlets shown in Options #3 and #4.

Options #3 and #4 are where I’ve actually seen some administrative head-scratching start.  Both options leverage PowerShell, and both produce a configuration-only backup.  The Backup-SPConfigurationDatabase cmdlet would seem most appropriate for the job … but is it?  If it is most appropriate, then why the redundant capability with the Backup-SPFarm cmdlet?

Two Cmdlets, One Function?

We have two PowerShell cmdlets that produce the same type of backup set.  Understanding how the cmdlets differ starts with an analysis of the syntax and parameters for each one.  Let’s start by looking at the syntax for each of the cmdlets in full.

First, the Backup-SPFarm cmdlet.

Backup-SPFarm -BackupMethod <String> -Directory <String> [-AssignmentCollection <SPAssignmentCollection>] [-BackupThreads <Int32>] [-ConfigurationOnly <SwitchParameter>] [-Confirm [<SwitchParameter>]] [-Force <SwitchParameter>] [-Item <String>] [-Percentage <Int32>] [-WhatIf [<SwitchParameter>]] [<CommonParameters>]

Next, the Backup-SPConfigurationDatabase cmdlet.

Backup-SPConfigurationDatabase -Directory <String> [-AssignmentCollection <SPAssignmentCollection>] [-DatabaseCredentials <PSCredential>] [-DatabaseName <String>] [-DatabaseServer <String>] [-Item <String>] [<CommonParameters>]

Each of the cmdlets requires that you specify where the backup set should be created using the –Directory switch, and each cmdlet permits the selection of either the entire configuration hierarchy (the default) or specific subset of it through the –Item switch.

There’s quite a bit of noise in the full syntax for each cmdlet, particularly for Backup-SPFarm, so let’s distill things down a bit and look at each cmdlet in turn.

Backup-SPFarm

For the purposes of this discussion, the following represents the core syntactical elements of interest for configuration-only backup using the Backup-SPFarm cmdlet:

Backup-SPFarm [-ConfigurationOnly <SwitchParameter>]

All you need to do is specify the –ConfigurationOnly switch and you’re ready to go.  There’s really not much more to it than that.

Under the hood, this method of configuration-only backup creation is the same as running a configuration-only backup operation from within SharePoint Central Administration.  Backup-SPFarm assumes that you’ve got a live SharePoint farm, that it’s operating properly, and that you want to capture its configuration with the backup process.

Backup-SPConfigurationDatabase

So then, what’s the deal with Backup-SPConfigurationDatabase?

Backup-SPConfigurationDatabase [-DatabaseCredentials <PSCredential>] [-DatabaseName <String>] [-DatabaseServer <String>]

Clearly there’s something more going on with this cmdlet.  Looking at the parameters, it should be clear that the –DatabaseName and –DatabaseServer switches allow you to specify an alternate configuration database and server location as the target of the configuration-only backup operation you intend to perform.  If you happen to require SQL Server authentication to access the configuration database, then you can specify connection credentials with the –DatabaseCredentials switch.

In most cases where I’ve seen the Backup-SPConfigurationDatabase cmdlet described, these extra database-centric parameters have been passed-off as giving you the ability to backup configuration databases that reside in other (non-local) farms.  I’ve also seen it suggested that it would be possible to centralize configuration-only backups for multiple farms using this cmdlet.  While I think that those are certainly possibilities, I think that they fail to consider the bigger picture and backup/restore as an end-to-end process.

It’s All About Recovery

First, let me close the case on the Backup-SPFarm cmdlet.  Under normal circumstances, Backup-SPFarm is how you should be running a configuration-only backup if you need it.  The TechNet documentation spells this out in the description for both the Backup-SPFarm and Backup-SPConfigurationDatabase cmdlets.  Although Backup-SPConfigurationDatabase can be used to backup the configuration database of an operational farm, that’s not really its intended use (as I see it, anyway).

To understand the real value of the Backup-SPConfigurationDatabase cmdlet, you need to think past the execution of backups and consider the process of recovery.  In many of the organizations I’ve consulted for, the SharePoint environments were not protected using the native backup and restore capabilities that come with the platform.  Quite a few of these mid-size and enterprise organizations handled SharePoint farm protection using SQL Server backups, third-party tools, or a combination of the two.  Usability guidelines for the native backup capabilities were sometimes the reason for avoiding SharePoint’s built-in tools; in other cases, backups were controlled and managed by a different internal group that had already standardized on their own (non-SharePoint) backup tool or product.

In each of the aforementioned backup scenarios, the primary goal was protection of SharePoint’s SQL Server databases.  Content databases were utterly critical targets in these backup scenarios since nothing would bring them back if they were lost.  Whether or not other databases were backed up depended on the recovery strategy.  In the event of catastrophic farm failure, some administrators try to recover all parts of the farm; others prefer to rebuild the farm from scratch and bolt the content databases back in.  Many different approaches to the recovery challenge exist, and each has benefits and disadvantages.

Regardless of the restore approach taken for database backups, the SharePoint farm configuration database has generally been regarded as relatively useless.  After all, farm configuration databases are not normally portable.  When you rebuild a SharePoint farm, you end up with a new farm configuration database.  In SharePoint 2007, it was generally accepted that the farm configuration database was “throwaway;” backing it up wasn’t even necessary unless you had some very specific (and oftentimes proprietary) use case for it.

The Restore Process: SharePoint 2010-Style

With SharePoint 2010, the question of “should I backup the farm configuration database?” should generally be answered with a “yes.”  The same restrictions regarding the use of a farm configuration database backup still apply from SharePoint 2007 (i.e., you can’t simply “drop it into” a live SharePoint farm and go), but we now have configuration-only backup and restore with SharePoint 2010.

If you think about the database or file-based approach to backup, and you consider the process of restoring or rebuilding a SharePoint farm, then you’ll probably understand where the Backup-SPConfigurationDatabase cmdlet actually fits in.  The cmdlet is less about backup than it is about restore.

If you’re trying to put a farm back together after a catastrophic failure using database backups, then you’re probably going to follow a series of steps that starts out like this:

1. Rebuild your servers with their operating systems
2. Install SharePoint
3. Create a new SharePoint farm (with a new farm configuration database)
4. Restore your old farm’s databases

Once you’ve completed step #3, you actually have a working SharePoint farm – it just doesn’t look anything like the farm you’re trying to rebuild/restore yet.  You’ll probably still need to re-provision services and service applications, re-establish all of your farm’s configuration settings, etc.

Assuming you were capturing your farm configuration database as part of the backup cycle for your old farm, then step #4 is where the Backup-SPConfigurationDatabase can be brought in to work its magic.  The Backup-SPConfigurationDatabase does actually require a functional farm to properly operate (even against another configuration database), but it can be used to execute a configuration-only backup against the old (pre-catastrophe) farm configuration database that was restored into SQL Server from backup in step #4.  The configuration-only backup set that is generated through the Backup-SPConfigurationDatabase action can then be brought into the rebuilt farm almost immediately using the Restore-SPFarm cmdlet with the –ConfigurationOnly switch engaged.

The Data Protection Manager 2010 Connection

I’ll be clear and state that I don’t have hard evidence to say with certainty that my take on the Backup-SPFarm and Backup-SPConfigurationDatabase division of responsibilities is what Microsoft envisioned when they created them, but I am relatively confident based on what I’ve seen and know – especially when I factor-in Microsoft’s data protection product offerings and enterprise backup/restore strategy.

In particular, I’m talking about Microsoft’s own System Center Data Protection Manager (DPM) product line.  The DPM product line conducts its backup and restore operations through the Volume Shadow Copy Service (VSS) that is built into the Windows operating system.  VSS is a very powerful mechanism for the creation of consistent, point-in-time file snapshots … but at its core, VSS is file-based.

Even though DPM advertises some integration capabilities with SharePoint, it isn’t “aware” of SharePoint beyond its interface with the SharePoint Foundation Volume Shadow Copy Service Writer (SPF VSS Writer) and the SPF VSS Writer’s subordinate search index writer.  For all practical purposes, this means that DPM can’t really treat SharePoint backups as much more than file-based backups.  DPM’s approach to SharePoint farm protection is to backup SQL Server databases (including the configuration database) and the farm’s search index partitions.  It doesn’t understand farm metadata, IIS configuration settings, the SharePoint Root (aka “14 hive”), etc.  These additional targets can be backed-up using DPM’s file system protection capabilities, but they aren’t associated in any way with other SharePoint backups.

So even though DPM can protect a SharePoint farm configuration database, it can’t do anything more with it than you or I can do with a standard SQL Server database restore.  If you realize that DPM only works with files and doesn’t have much in the way of application-level intelligence, this chart that compares its capabilities against other SharePoint protection mechanisms makes quite a bit of sense.  It should also make it clear why even Microsoft’s products have a need for the Backup-SPConfigurationDatabase cmdlet.

Closing Thoughts

Don’t interpret what I’m saying as DPM 2010-bashing.  On the contrary, I’ve been using DPM since it’s 2007 release in my home network environment, and I think it’s a pretty good product.  I only brought up DPM to make my point about the Backup-SPConfigurationDatabase cmdlet – not to beat-up the product.

Since both SQL Server backups and DPM are capable of restoring a SharePoint configuration database – but not much more than that – the Backup-SPConfigurationDatabase cmdlet fills a very important role in the SharePoint restore process.  It’s the “bridge” from many backup/restore solutions to SharePoint itself for purposes of getting back configuration data.

The general rule of thumb that I give people is this: use Backup-SPFarm if you’re trying to extract configuration data from a live farm, and use Backup-SPConfigurationDatabase if you’re trying to extract configuration data from a restored (and otherwise unassociated) configuration database.

Additional Resources and References

1. Blog Post: Configuration-Only Backup and Restore in SharePoint 2010
2. TechNet: Backup-SPFarm PowerShell cmdlet
3. TechNet: Backup-SPConfiguration PowerShell cmdlet
4. TechNet: Backup and recovery overview (SharePoint Server 2010)
5. TechNet: Restore-SPFarm PowerShell cmdlet
6. Product: Microsoft System Center Data Protection Manager 2010
7. MSDN: SharePoint Foundation VSS Writer
8. TechNet: Plan for backup and recovery (SharePoint Server 2010)

Since my first copy of our new book actually arrived in the mail yesterday (from Amazon.com), I think I can officially announce that the SharePoint 2010 Disaster Recovery Guide is available!  Here’s a picture of it – straight out of the box:

John Ferringer and I apparently didn’t learn our lesson the first time around.  When Cengage approached us about writing another version of the book, we said “yes.”  We were either in denial or had repressed the memories associated with writing the first book.  There were definitely some difficulties and challenges (like trying to learn the relevant pieces of the SharePoint 2010 platform while also writing about them), but we managed to pull it off again.

Of course, we couldn’t have done this without the technical prowess and patience of JD Wade.  JD was our technical editor, and he had a knack for questioning any assumption or statement that wasn’t clearly backed by fact.  He did a fantastic job – I couldn’t have been happier.  The book’s accuracy and quality are a direct result of his contributions.

What’s Inside?

Interested in what we included?  Here’s the table of contents by chapter:

1. SharePoint Disaster Recovery Planning and Key Concepts
2. SharePoint Disaster Recovery Design and Implementation
3. SharePoint Disaster Recovery Testing and Maintenance
4. SharePoint Disaster Recovery Best Practices
5. Windows Server 2008 Backup and Restore
6. Windows Server 2008 High Availability
7. SQL Server 2008 Backup and Restore
8. SQL Server 2008 High Availability
9. SharePoint 2010 Central Administration Backup and Restore
10. SharePoint 2010 Command Line Backup and Restore: PowerShell
11. SharePoint 2010 Disaster Recovery Development
12. SharePoint 2010 Disaster Recovery for End Users
13. Conclusion

As you can see, we’ve included a little something for just about everyone who might work with SharePoint or interface with it for disaster recovery purposes.  SharePoint administrators will probably benefit the most from the book, but there are definitely sections that are of use to SharePoint developers, DR planners, and others who are interested in SharePoint from a business continuity perspective.

If you happen to pick up a copy of the book, please share your feedback with us – good, bad, ugly, or anything else you feel like sending our way!  We poured a lot of time and effort into this book in an attempt to “do our part” for the community, and your thoughts and feedback mean everything to us.

Thanks, and enjoy!

Additional Resources and References

1. Book: SharePoint 2010 Disaster Recovery Guide
2. Blog: John Ferringer’s MyCentralAdmin
3. Blog: JD Wade’s Wading Through

Fall is on its way, and things may be winding down for some folks.  There’s plenty hopping here, though, and I wanted to share a couple of items that are coming up in the near term.

SharePoint Saturday Vancouver

This upcoming weekend, I’ll be traveling to Vancouver, British Columbia for SharePoint Saturday Vancouver.  The event is being organized by the triumvirate of Sherman Woo, Michal Pisarek, and Yaroslav Pentsarskyy, and it’s being held at the BCIT Downtown Campus in Vancouver.  If you live in or around Vancouver and want to soak up some free SharePoint knowledge and community interaction, sign-up – it’s free!

The event is planned with four tracks (Information Worker, Development/Customisation, Administration, and User Experience/Vendor), and each track will have four sessions throughout the day.  Attendees who fancy themselves administrators are going to get a load of Todd Klindt and me, as Todd and I are each delivering two of the four admin sessions.  No worries for potential attendees, though – I think that both Todd and I will be trying to soften the blow of suffering through an entire day with the two of us by giving away a few freebies …

On a non-SharePoint note, this is actually going to be my first trip to Canada … and I’m really looking forward to it!  I probably won’t be in Vancouver long enough to really see anything, but I will get to spend at least a couple of days in the Great White North – land of Geddy Lee and the McKenzie Brothers.

Off The Beaten Path

The bulk of what I blog about is related to SharePoint in some way, shape, or form.  Every now and then, though, I’ll venture off the SharePoint path to share noteworthy (or simply fun) information.

One such non-SharePoint item that I wanted to share with you is the Windows 7 Deployment Firestarter series of TechNet events taking place in the Midwest region (including Cincinnati).  The events are being brought to the community by a group that includes my friend and colleague Matt Hester.  Matt is Microsoft’s IT Pro Evangelist for our region, and he’s been working very hard in the last several months to organize the IT Pro community and build up some steam.

These events promise to be an excellent rundown on Windows 7, the tools that are available to IT Pros for deployment, and how upgrades can be approached.  If you have the time and interest, I encourage you to sign up.  Like so many other great events, they’re free!  Space is limited, though, so you should probably move fast.

Additional Resources and References

1. Event: SharePoint Saturday Vancouver
2. Twitter: Sherman Woo (@spsherm)
3. Twitter: Michael Pisarek (@MichalPisarek)
4. Blog: Todd Klindt
5. Schedule: SharePoint Saturday Vancouver
6. People: Geddy Lee
7. People: Bob and Doug McKenzie
8. Events: TechNet Events: Windows 7 Deployment Firestarter
9. Blog: Matt Hester

Since our SharePoint 2010 Disaster Recovery Guide is written, starched, pressed, and ready to wear, I thought it was time to get back to some of the blogging I promised to start doing again once the book was finished.  I guess that if I didn’t have something to write, I simply wouldn’t know what to do with myself.  <insert smirk here>

Motivation For This Post

SharePoint 2010’s configuration-only backup and restore capabilities are on a long list of topics I’ve been meaning to blog about, but in all honesty it wasn’t at the top of that list.  I’ve been seeing the topic start to get some real attention in a number of forums, though, from folks like Todd Klindt (in one of his recent netcasts) and Benjamin Athawes (in his blog and in the helpful replies he’s been providing out in Microsoft’s TechNet forums).

It seems that many folks in the SharePoint community have heard about configuration-only backup and restore, and I think there’s an awful lot of hope that it will help with some of the problems we faced with SharePoint 2007.  By the time you finish reading this post, I hope to impart a solid understanding of what configuration-only backup and restore will – and won’t – do for you.

The Elephant In The Room

Before I go any further, let me address the question that I suspect the overwhelming majority of you probably want an answer to:

Will configuration-only backup and restore let me clone my SharePoint 2010 farm?

The quick answer: no.  At the risk of being a bit flippant, I’ll include a slightly longer answer: heck no – not even close.

When configuration-only backup and restore was introduced to the world, it promised so much.  I remember hearing the discussion of “farm templates” and of “cloning configuration.”  I remember sitting through Bill Baer’s business continuity management (BCM) session at the SharePoint Conference in 2009 and thinking about all the things I was going to do with the new capability.

In light of what I now know about configuration-only backup and restore, I went back to the recorded SPC sessions (including SPC311 – Bill’s BCM session) to make sure I wasn’t hearing things.  I wasn’t.  My guess is that the initial vision for configuration-only backup and restore had to get scaled-back prior to the product becoming generally available.  Maybe the team ran out of time, maybe they hit technical hurdles, or perhaps it was a combination of the two.  Regardless, the capability in its current form isn’t quite what I had hoped it would be.

Enough with the hand-waving.  Let’s dive in.

High-Level: What Is Configuration-Only Backup and Restore?

For a brief primer on configuration-only backup and restore, check out the “Backup and recovery overview (SharePoint 2010)” article on TechNet.  If you don’t want to take the time to read the article, though, I’ll sum it up for you: a configuration-only backup and restore allows you to extract portable configuration settings from a SharePoint 2010 farm configuration database and apply those settings to a different farm.  The promise, as indicated earlier, is that you could effectively “clone” the configuration of a farm.  The configuration template that would be generated from this process could then be applied to other farms to create copies of the original farm’s settings and configuration.  This would be extremely beneficial when duplicating environments (e.g., creating staging and testing environments that match a production environment), building development and demo virtual machines (VMs), and more.

Those of you who have worked with SharePoint 2007 recognize the leap forward that this represents.  Anyone who has spent any amount of time exploring SharePoint backup and recovery knows that farm configuration databases are tied to their SharePoint environments.  Microsoft doesn’t support transplanting one farm’s configuration database into another farm; most of the time, it simply wouldn’t work.  Even if you could get it to work through some extremely impressive techno-jujitsu, you’d be in a horribly unsupported state as far as Microsoft support was concerned.

What Does It Look Like?

Configuration-only backup and restore in SharePoint 2010 is an extension to the existing catastrophic backup and restore capabilities located in the Microsoft.SharePoint.Administration and Microsoft.SharePoint.Administration.Backup namespaces.  The processes and mechanisms that allow you to create farm-level backups from Central Administration (through “Farm Backup and Restore”), PowerShell (via Backup-SPFarm), and STSADM.exe (via STSADM –o backup in catastrophic mode) are the same ones that are employed in configuration-only backups.

In fact, the backup sets that are generated from a configuration-only backup are basically the same, structurally speaking, as those that are generated from a “normal” (content + configuration) catastrophic backup.  One easy way to determine the nature of a backup, though, is to crack open the backup location’s table of contents file (spbrtoc.xml) and examine the value within the <SPConfigurationOnly /> element for a given backup or restore run (represented by a <SPHistoryObject /> element).

For example, this particular backup run was clearly a configuration-only backup because its <SPConfigurationOnly /> element contains a value of True. 

<SPHistoryObject> 
    <SPId>571d2ad2-f485-46de-918e-653e8868c8bc</SPId> 
    <SPRequestedBy>SPDC\s0ladmin</SPRequestedBy> 
    <SPBackupMethod>Full</SPBackupMethod> 
    <SPRestoreMethod>None</SPRestoreMethod> 
    <SPStartTime>08/18/2010 16:15:12</SPStartTime> 
    <SPFinishTime>08/18/2010 16:15:37</SPFinishTime> 
    <SPIsBackup>True</SPIsBackup> 
    <SPConfigurationOnly>True</SPConfigurationOnly> 
    <SPBackupDirectory>e:\temp\spbr0001\</SPBackupDirectory> 
    <SPDirectoryName>spbr0001</SPDirectoryName> 
    <SPDirectoryNumber>1</SPDirectoryNumber> 
    <SPTopComponent>Farm</SPTopComponent> 
    <SPTopComponentId>7850df11-60ef-460c-ab4a-9b7b9f2f735f</SPTopComponentId> 
    <SPWarningCount>0</SPWarningCount> 
    <SPErrorCount>0</SPErrorCount> 
</SPHistoryObject>

If you browse the folder containing the backup set that is generated from a configuration-only backup, you’ll see the expected array of sequentially numbered hexadecimal .bak files, as well as a log file (spbackup.log) and backup component hierarchy file (spbackup.xml).

The .bak files themselves contain XML-serialized representations of various farm objects that were captured during the configuration backup process:

<object type="Microsoft.Office.Server.Administration.DiagnosticsService, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c">
  <fld type="System.Collections.Hashtable, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" name="m_Throttles" />
  <fld name="m_Versions" type="null" />
  <fld name="m_UpgradeContext" type="null" />
  <fld type="System.Collections.Hashtable, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" name="m_UpgradedPersistedFields" />
  <fld name="m_Properties" type="null" />
  <sFld type="String" name="m_LastUpdatedUser">SPDC\s0ladmin</sFld>
  <sFld type="String" name="m_LastUpdatedProcess">psconfigui (4228)</sFld>
  <sFld type="String" name="m_LastUpdatedMachine">SPDEV</sFld>
  <sFld type="DateTime" name="m_LastUpdatedTime">2010-04-25T17:38:27</sFld>
</object>

Again, this is all very similar to a standard catastrophic farm backup.  The one notable absence in the backup set that is produced during a configuration-only backup is that of SQL Server database backup files that begin (internally) with a telltale TAPE header.  The absence of these files is expected, though, since configuration-only backups operate on farm configuration settings and metadata – not the content and other data that is housed primarily in SQL Server databases.

“Wait,” you might be saying, “service applications typically have quite a bit of configuration data, and much of that data is housed in SQL Server databases.  Wouldn’t those databases be captured by the configuration-only backup process?”  Hold that question – I’ll be addressing it in a short bit.

A Quick Peek At What’s Going On Under The Hood

To better understand how configuration-only backup and restore works, it helps to dive below the backup set and into the SharePoint object model to see what’s actually happening.  If you’re not a developer, no worries – I’ll try to keep this simple.

The type that is the backbone of configuration-only backup and restore operations is the IBackupRestoreConfiguration interface.  Classes in the SharePoint object model can implement this interface (and supply a CanBackupRestoreAsConfiguration property value of true) if they wish to meet the bare minimum requirements for inclusion in configuration-only backup and restore operations.

If you’ve worked with catastrophic backup and restore in the SharePoint object model before, this interface name may seem a little familiar to you – even if it isn’t.  That’s because extending the native catastrophic backup and restore functionality of SharePoint to include new content classes is done through the similarly named IBackupRestore interface.  IBackupRestore came before IBackupRestoreConfiguration, and the latter is actually derived from the former.  The patterns of interaction between the runtime backup objects and objects that implement these two interfaces is very similar – as you might expect given their inheritance relationship.

So you might be wondering, “So what?  I don’t plan to build configuration-only backup and restore-capable components.  Why are you going through all of this.”  The answer to that question is relatively easy to answer: we can get a pretty clear understanding of what is actually included in configuration-only backup and restore operations by looking at the SharePoint classes that implement the IBackupRestoreConfiguration interface.

Hold onto the concept of examining types that implement IBackupRestoreConfiguration; we’ll be coming back to it in just a second.

What Does A Configuration-Only Backup Actually Capture? – Part 1

Let’s leave the SharePoint object model and come back up to ground level for a moment.

In plain English, configuration-only backup and restore is basically supposed to address the “I need to create a template of my farm” pain point we felt with SharePoint 2007.  Does it?  What gap is filled by the capability according to Microsoft?

If you read the TechNet article I linked to earlier, you’ll find just five types of settings (or configuration data items) that are actually listed as included in a configuration-only backup:

• Antivirus
• Information rights management (IRM)
• Outbound e-mail settings (only restored when performing an "overwrite").
• Customizations deployed as trusted solutions
• Diagnostic logging

I don’t know about you, but this doesn’t exactly line up too well with the list of things I’d want to replicate from Farm A to Farm B if I were actually trying to clone Farm A configuration settings.  Don’t get me wrong: several of the items listed are things I would want to bring across (especially the customizations in the farm solution store), but there are a whole host of additional things I’d want to see.

What Does A Configuration-Only Backup Actually Capture? – Part 2

The five bullet points I just supplied aren’t entirely well-defined, and they’re more than a little “light” in terms of farm configuration data.  Let’s define the list a bit more clearly by seeing which classes in the SharePoint object model actually implement the required IBackupRestoreConfiguration interface.

When I fire-up Reflector and analyze the types that use the IBackupRestoreConfiguration interface, I come up with the following classes (ignoring the SPBackupRestore type, since its ImplementsIBackupRestoreConfiguration method only checks to see whether or not other objects themselves actually implement the interface):

• Microsoft.SharePoint.Administration.SPDiagnosticsServiceBase
• Microsoft.SharePoint.Administration.SPFarm
• Microsoft.SharePoint.Administration.SPResourceMeasure
• Microsoft.SharePoint.Administration.SPSolution
• Microsoft.SharePoint.Administration.SPSolutionCollection
• Microsoft.SharePoint.Administration.SPSolutionLanguagePack
• Microsoft.SharePoint.Administration.SPUserCodeExecutionTier
• Microsoft.SharePoint.Administration.SPUserCodeLoadBalancerProvider
• Microsoft.SharePoint.Administration.SPUserCodeProvider
• Microsoft.SharePoint.Administration.SPUserCodeService
• Microsoft.SharePoint.Administration.SPWebService
• Microsoft.SharePoint.UserCode.SPSolutionValidator

Each of these classes is capable of participating in a configuration-only backup because it implements IBackupRestoreConfiguration.  The list is longer than the five bullets I mentioned earlier, but many of the classes cited can be grouped into common areas of functionality.  The SPSolution, SPSolutionCollection, and SPSolutionLanguagePack types are associated with trusted solutions (the farm solution store), for example, while the SPDiagnosticsServiceBase type is tied to trace log and event throttling management (i.e., diagnostic logging).  A simple one-to-one mapping between classes and settings areas (that you might find in Central Administration) doesn’t actually exist.

Identifying what is included in a configuration-only backup isn’t quite a quick and easy affair.

What Doesn’t A Configuration-Only Backup Capture?

Sometimes it’s simply easier to talk about what a thing isn’t rather than what it is.  As you’re probably coming to see, configuration-only backup and restore is one of those things.

For those who hoped that configuration-only backup and restore would deliver us to the promised land of SharePoint farm templates and full configuration replication, the first signs of trouble in paradise come by reading the implementation notes for the IBackupRestoreConfiguration interface.  In essence they state that you shouldn’t be implementing the interface to capture configuration settings unless the following three conditions are true for the settings in question:

1. The settings you’re trying to preserve are only configuration settings – not content like lists, documents, etc.
2. The settings you want to capture are scoped to the entire farm or the Content Publishing Web Service (i.e., they apply equally to all non-Central Admin Web applications and the site collections contained within them – not to just a subset)
3. The settings aren’t tied to server names or your specific SharePoint farm topology

That list starts “simple” and ends “rough.”  With those three bullets, we can instantly rule-out configuration data that is tied to individual Web applications, content databases, site collections, and everything else below them.  Configuration-only backup and restore won’t protect your per-Web application settings, either, including alternate access mappings (AAMs).

I’m making a special point of highlighting AAMs because configuration-only backup and restore was initially advertised as being capable of capturing these mappings.  Sure, you can view AAMs within Central Administration and may think that they’re maintained at the farm level, but they aren’t – they’re tied to specific Web applications.  AAMs for a Web application are represented (within the object model) as an instance of the SPAlternateUrlCollection class.  The SPAlternateUrlCollection isn’t on the list of IBackupRestoreConfiguration implementers provided earlier, nor are its parent types (most notably the SPWebApplication type through its AlternateUrls property).  Net effect: it isn’t included in configuration-only backup and restore operations.

Since cloning a SharePoint farm usually involves taking it from one environment to another, bullet #3 is a rather big sticking point, as well.  Configuration-only backup and restore won’t handle anything that includes a server name, IP address, or any other environmentally-dependent setting.  The reason is pretty simple – how would SharePoint know how to actually re-wire that stuff (in a new environment) on restore?

Ouch.

Okay, What About Service Applications?

The Service Application Framework is new to SharePoint 2010, and it represents a major step forward in correcting many of the performance, configuration, and scalability limits of MOSS 2007’s shared service provider (SSP) model.  If you’ve touched SharePoint 2010 in any form, chances are you’ve at least stumbled into service applications in some form.  Examples include the Managed Metadata Service, Business Data Connectivity (BCS) Services, and Search.

Although the Service Application Framework has been engineered to participate in normal (content+configuration) catastrophic backup and restore operations, it doesn’t do so through the standard IBackupRestore interface.  Developers of service application and related classes can adorn their classes with a couple of different attributes (IisWebServiceApplicationBackupBehaviorAttribute and IIsWebServiceApplicationProxyBackupBehaviorAttribute – not exactly “short and sweet” in the name department), and they get backup and restore integration as a freebie.  This is a big relief for developers, because properly implementing the IBackupRestore interface in their classes is anything but trivial.

There is a downside to the attribute-based backup and restore approach as its implemented, though: the Service Application Framework simply doesn’t participate in configuration-only backup and restore.  When you execute a configuration-only backup, you won’t capture any configuration data tied to search, BCS, managed metadata, web analytics, Excel services, or any of the other service applications.

Double ouch.

The Verdict On Configuration-Only Backup And Restore

I’ll start by apologizing if this post dashed your hopes.  Believe me when I say that I had very high hopes for configuration-only backup and restore, as well.  Cloning farms by hand is painful work; I’ve done it enough times to know that much.

Since configuration-only backup and restore doesn’t actually cover any configuration data tied to service applications and individual Web applications, cloning a farm in SharePoint 2010 is still going to be a largely manual affair.  Scripting can (and probably should) play a large role, and so will documentation.

There, I said it – the ugly “d” word.  Documentation.

Documentation continues to play a big role in capturing configuration data in SharePoint 2010, but that doesn’t mean you have to resort to taking notes or capturing screenshots en masse.

Microsoft has (indirectly) acknowledged that configuration-only backup and restore isn’t going to round up all of our desired configuration settings, and they’ve attempted to lend us a hand through some PowerShell scripting.  If you haven’t yet reviewed Microsoft’s farm documentation script on TechNet, I highly recommend that you check it out.  Saying that the script’s treatment of farm configuration data is “extensive” is kind of like saying that a tsunami is a “big wave” – it doesn’t do it justice.

I also want to be clear and say that despite the limitations I’ve described, I still think that configuration-only backup and restore is worth some serious investigation for anyone trying to do template creation, cloning, and disaster recovery work.  Given my focus on disaster recovery, for example, the ability to get a farm’s solution store backed-up in a form that can be restored easily at a later time is a huge benefit – one that would really ease the process of farm recovery in a true disaster scenario.

Additional Resources and References

1. Book: SharePoint 2010 Disaster Recovery Guide
2. Netcast: Todd Klindt’s Netcast 54
3. Blog: Benjamin Athawes’ Tales from a SharePoint Farm
4. Blog: Bill Baer’s TechNet Blog
5. TechNet: Backup and recovery overview (SharePoint 2010)
6. API: IBackupRestoreConfiguration
7. API: IBackupRestore
8. Product: Red Gate’s Reflector
9. MSDN: What’s New: Service Application Framework
10. TechNet: Document Farm Configuration Settings (PowerShell Script)