The Five-Minute Page Performance Troubleshooting Guide for SharePoint Online

I regularly hear from SharePoint Online customers that their pages are slow … but they don’t know where to start troubleshooting. Is it the SPO servers? The network? Their page(s)? In this post, I’ll show you how to determine the general source of your slow pages in five minutes or less. It won’t solve your slow page(s) problem, but it will give you enough direction to know where to focus further analysis.

“Why is it so slow?” That’s how nearly every performance conversation I’ve ever had begins.

No one likes a slow intranet page, and everyone expects the intranet to just “come up” when they pop the URL into their favorite browser. From an end-user’s perspective, it doesn’t matter what’s happening on the back-end as long as the page appears quickly when someone tries to navigate to it.

SharePoint Online is a big black box to many of its users and consumers. They don’t understand what it takes to build an intranet, nor should they have to. The only thing that really matters to them is that they can bring up a browser, type in a URL, and quickly arrive at a landing page. The burden of ensuring that the site is optimized for fast loading falls to the folks in IT who are supposed to understand how everything works.

If you’re one of those folks in IT who is supposed to understand how everything works with SharePoint Online but doesn’t, then this blog post is for you. Don’t worry – I know there’s a lot to SharePoint Online, but performing some basic troubleshooting analysis for slow pages in SharePoint Online is pretty straightforward. I’ll share with you a handful of techniques to quickly ascertain if the reason for your slow pages is due to the content within the pages themselves, if the issue is network-related, or if there might be something else happening that is beyond your control.

Your Toolset

The first step in your performance troubleshooting adventure begins by opening up your browser from a client workstation. Everyone has a favorite browser, but I’m going to use and recommend Internet Explorer for this exercise because it has a solid set of development tools to assist you in finding and quantifying performance issues. In particular, it is able to chronologically list and detail the series of interactions that take place between your browser and the SharePoint Online web front-ends (WFEs) that are responding to your requests.

When recommending IE, some people ask “how come you don’t use Fiddler?” It’s a good question, and when I first started showing people how to do some quick troubleshooting, I’d do so with Fiddler. If you’re just starting out, though, Fiddler comes with one really big gotcha: operating inside an SSL tunnel. To get Fiddler (which is a transparent proxy) working with SSL, there is some non-trivial setup required involving certificate trusts. Since this is intended to be a quick and basic troubleshooting exercise, I figure it’s better to sidestep the issue altogether and use IE (which requires no special setup).

The Setup

To make this work, let us assume that I am attempting to profile the Bitstream Foundry (my company) intranet home page in order to understand how well it works – or doesn’t. My intranet home page is pretty plain by most intranet standards (remember: I’m a developer and IT Pro – not a designer), but it’s sufficient for purposes of discussion.

Step 1. Open Your Browser

I start by opening Internet Explorer and navigating to the Bitstream Foundry intranet home page at https://bitstreamfoundry.sharepoint.com. Once I move past the sign-in prompts, I’m shown my home page:

Bitstream Foundry Intranet Home Page

My home page has very little on it right now (I’m still trying to decide what would go best in the main region), but it is a SharePoint Online (SPO) page and it does work as a target for discussion purposes.

Step 2. Access the Developer Tools

Accessing the developer tools within Internet Explorer is extremely simple: either press F12, or go to the browser’s gear icon and select F12 Developer Tools from the drop-down that appears as seen below:

Accessing the IE Developer Tools

Doing either of these will pop-open the developer tools as either a stand-alone window or as a pane on the lower half of the browser as shown below:

Internet Explorer F12 Developer Tools

Step 3. PREPARE TO CAPTURE

When the developer tools first open, they’re commonly set to viewing the page structure on the DOM Explorer tab. For purposes of this troubleshooting exercise, we need to be on the Network tab so we can profile each of the calls the browser makes to the SPO WFE.

Select the Network tab and then select the “Always refresh from server” button as highlighted below in red.

Prepare for Capture

The Network tab is going to allow us to capture the series of exchanges between the SharePoint WFE and our browser as the browser fetches the elements needed to render the page. The “Always refresh from server” button is going to remove client-side caching from the picture by forcing the browser to always re-fetch all referenced content – even if it has a valid copy of one or more assets in the browser cache. This helps to achieve a consistent set of timing values between calls, and it’s also going simulate someone’s first-time visit to the page (which typically takes longer than subsequent visits) more accurately.

Step 4: Capture the Exchange

The next step is to capture the series of exchanges between IE and SPO. To do this, simply refresh the page by pressing the browsers Refresh button, pressing <F5>, or going to the browser’s address bar and re-issuing the page request.

The contents of the window on the Network tab will clear, and as content begins to flow into the browser, entries will appear on the screen. For every request that IE makes of SharePoint Online, a new line/entry will appear. It will probably take a handful of seconds to retrieve all page assets, and it’s not uncommon for a SharePoint page to have upwards of 75 to 100 resources (or more) to load.

Capture the Exchange with SPO

Strictly speaking, you shouldn’t have to stop the capture once the page has loaded, but there are several reasons why you would want to.  First, you will eventually retrieve all SharePoint assets necessary to render the page. If you continue to capture beyond this point, you’ll see the number of requests (represented in the bottom bar of the browser – the number is 83 requests in the screenshot above) continue to tick up. It will slowly go up over time and it’s not due to the contents of the SharePoint page – it’s due to Office 365.

If you look at the last entry in the screenshot above, you’ll see that it’s a request to https://outlook.office365.com/owa. In short: this is due to a background process that allows Exchange to notify you when you receive new messages and calendar/event notifications. See how the Protocol and Result/Description columns indicate a (Pending) state?

If you get to this point and additional SharePoint elements are no longer loading, press the red “recording stop” button in the toolbar of Network tab. This will stop the capture. Not only does this help to keep the captured trace “cleaner,” but it also prevents excessive distortion of certain values – like overall time to load and the graphical representation of the page load (shown on the far right of the Network tab) as shown below.

Page Timeline Distortion

Step 5: Find the SharePoint Page Request

At this point, you should have a populated Network tab with the entire dialog of requests that were needed to render your page. Of these requests, the overwhelming majority of them will be for JavaScript files (.js), cascading stylesheets (.css), and images (.png, .gif, and .jpg). Only one of them will be for the actual SharePoint page itself (.aspx) … and, of course, this is the request that you need to find in the list.

My intranet home page is named Home.aspx (as can be seen in the browser address bar), so I need to find the request for Home.aspx on the Network tab. I got lucky with this dialog attempt, because Home.aspx is the first entry listed. Note that this isn’t always the case, and it’s not uncommon to find your page request 10 or 20 down in the list.

Select the ASPX Page

When you locate the entry in the list for your .aspx page, click on it to select it. You can confirm that you’ve selected the right entry by verifying Request URL on the Headers tab to the right of the various requests listed for the exchange with SPO (highlighted in the image above).

Step 6: Analyze the Headers

At this point, we need to shift our focus to the HTTP Response Headers that are passed back with the content of the page. Much like the request headers that the browser sends to the server to provide information about the request being made, the response headers that are sent from the server supply the browser with all sorts of additional information about the page. This can include the size of the page (Content-Length), the payload (Content-Type), whether or not the page can be cached (Cache-Control), and more.

Making sure that you have the Headers tab selected, locate and record the three response headers as shown below:

Response Headers of Interest

The three values we want to record are:

  • SPIisLatency. This is a measure of the amount of time (in milliseconds) that the request spent queued and waiting to be processed by IIS (Internet Information Services – the web server). Ideally, it should be zero or very close to zero. In my example, the SPIisLatency is 3ms.
  • SPRequestDuration. This is the amount of time (again, in milliseconds) that it took to process the request on the server. Basically, this is the end-to-end processing time for the page. Healthy pages range from a couple hundred milliseconds to around a second depending on the content of the page. In my example, the SPRequestDuration is 249ms.
  • X-SharePointHealthScore. This is the value, from zero to ten, that indicates how heavily loaded the SharePoint Server is at the time when the page was served. A score of zero means the server is not under load, while a score of ten means the server is overloaded. As the X-SharePointHealthScore goes up, the server begins to selectively suspend work designated as “low priority,” like some Timer Service jobs, Search requests, and various other low-priority tasks. ideally, this value should be zero – or close to it. In my example, the value is zero.

We can infer a great deal about the page processing and network traversal of our page request with just these three values and a final number.

A quick note (2017-07-06): For some reason, a variety of SharePoint Online sites have been returning pages without the SPIisLatency and SPRequestDuration headers lately. I don’t know why this is happening, and I’ve reached out to Microsoft to see if it’s a bug or part of some larger strategy. I don’t think it’s deliberate, because the headers provide some of the only insight end-users can get into SharePoint Online page performance. When I hear something from the product team, I’ll post it here!

The Magical Trio: SPIisLatency, SPRequestDuration, and Total Trip Time

So, you’ve now got three numbers – two of which are helpful for page profiling (SPIisLatency and SPRequestDuration), and a third number (X-SharePointHealthScore) which will tell you how stressed the server was when it served your page. What can you do with them? As it turns out, quite a bit when you combine two of the three with a fourth number.

What is the fourth number? It’s the total trip time that is reported for the page being loaded, and it represents the elapsed time from the point at which the page was requested until the time when the last byte of the page was delivered. For example, I profiled my Bunker Tuneage site. It’s a SharePoint Online site (yes, I know – I have to get it moved to another location soon), so it makes a good target for analysis:

Bunker Tuneage Page Profile

In the above example, the three numbers we’re most interested in are:

  • Total Trip Time: 847.47ms
  • SPRequestDuration: 753ms
  • SPIisLatency: 0ms

If we think about what the individual values mean, we can now reason that the total amount of time spent to get the page (847.47ms), minus the total amount of time spent waiting or processing the server (753ms), should be roughly equal to the amount of time spent “elsewhere” – either in routing, traversing network boundaries, on proxies and firewalls, etc.

So, considering our numbers above, the equation looks like this:

Performance Equation

Based on our equation, this means that approximately (this isn’t exact) 94.47ms of time was spent getting from from the SharePoint Online server to our browser – not too shabby when we consider it.

The Permutations

The numbers could come out a variety of different ways when doing this, so it’s best if we try to establish a general trend. Variability between any two runs can be significant, so it’s in your best interests to conduct a number of runs (maybe a dozen) and come up with some average values.

Regardless of the specific values themselves, there are some general conclusions we draw about each value by itself – and when it is compared to the others.

  • High Total Time. The total end-to-end times can vary dramatically. The examples I’ve got shown thus far demonstrate sub-second latency (i.e., hundreds of milliseconds), and any time you can get values like that, it’s nothing to complain about. When your total round trip time climbs to two or three seconds, your generally still doing pretty good. If you hit five, six, or seven+ seconds, it’s time to move on to what to see what SPRequestDuration, SPIisLatency, and the time-spent-elsewhere values say.
  • High SPIisLatency. If you observe consistently high SPIisLatency values, they point to there being something wrong server-side, since a high SPIisLatency suggests that requests are backing up on the server. Although I’ve never seen it, I believe you could see high SPIisLatency for a brief period of time … but during that time, I’d also expect SharePoint Online to be spinning-up additional WFEs to deal with the effects of high user load. I’ve only ever seen SPIisLatency values in the single digits before, and they’ve never lasted beyond a request or two.
  • High “Time Lost ‘Elsewhere.'” If you crunch the numbers in the performance equation and come up with a significant amount of time being lost “elsewhere,” it suggests that the traffic between SharePoint Online and your computer is being slowed down for some reason. It doesn’t specifically indicate what is causing the slowdown, but the slowdown could be due to any number of network conditions: excessive routing, web proxies, egressing to the Internet out-of-region (a form of excessive routing), firewall issues, or a whole host of other conditions. What represents “excessive” time spent elsewhere? Again, I can only speak to trends here, but I tend not to get too upset about anything under 1s (1000ms) being lost to other factors. When time lost elsewhere grows to be high – especially compared to SPRequestDuration – that’s when I get concerned. For example, an SPRequestDuration of 800ms with a time-lost-elsewhere value of 2500ms makes me wonder what’s happening between SharePoint Online and my computer.
  • High SPRequestDuration. A high SPRequestDuration value can be caused by a variety of factors, and in truth the diagnosis tends to become a bit contentious. Since a high SPRequestDuration means that a page is taking a long time to process on the server, the most common response I frequently encounter (especially among those new to SPO) is that “there’s something wrong with SharePoint Online.” I hate to be the bearer of bad tidings, but repeat after me: “The problem isn’t with SharePoint Online, it’s with my site.” That 9000ms SPRequestDuration probably has very little to do with SPO and everything to do with how you customized SharePoint, your choice of navigation style, the fact that there are two dozen “expensive” web parts on the page, or something related to that. I’m not willing to rule out a problem with a SharePoint Online tenant, but in truth I have yet to encounter it.

What Can I Do About a High SPRequestDuration?

If you don’t believe me and instead feel that the problem is with the SharePoint Online environment, the good news is that there’s an easy way to tell one way or the other … and I highly recommend doing this before calling Microsoft Support (trust me, they’ll thank you for doing so).

Believe it or not, SharePoint Online is also where OneDrive for Business data is stored. A OneDrive for Business page, at its core, is a SharePoint page with nearly no customization. Using someone’s OneDrive for Business page becomes an excellent A/B test when the performance of SharePoint Online page is sub-par. Simply load up their OneDrive for Business page and compare performance numbers to the page in question.

OneDrive for Business Performance

Revisiting my Bunker Tuneage site example, you can see that the OneDrive for Business landing page is served from the same tenant as the earlier page: https://bunkertuneage-my.sharepoint.com/personal/sean_bunkertuneage_com/_layouts/15/onedrive.aspx#. If I were to compare the SPRequestDuration value of the OneDrive for Business page (223ms) with the SPRequestDuration of the SharePoint page in-question (753ms), I’d note that the values differed … but are they different enough to think something is going awry in the SPO environment?

Roughly half a second (~500ms) is indeed a difference, but it’s not enough for me to think that the online environment has problems. When I see SPRequestDuration values like 9000ms for a SharePoint page but 500ms for OneDrive for Business page, that’s when I begin to suspect something is amiss. And again: with such an extreme disparity in values, SharePoint Online is healthy (500ms), but there’s clearly something wrong with my page (9000ms).

Practical Advice

When it comes to diagnosing the root cause or causes for high SPRequestDuration values, the good news is that there are plenty of fixes that range from the simple to the quite invasive. Microsoft has taken the time to compile some of the more common causes, and I highly encourage you to take a look if you’re interested.

At the end of the day, though, sometimes you just want to know where to begin troubleshooting so that you can focus remediation efforts. If you follow the steps outlined in this blog post, I think you’ll find that the five minutes they take to execute will help to focus you in the right area.

References and Resources

  1. MSDN: Discovering Windows Internet Explorer Developer Tools
  2. Company: Bitstream Foundry
  3. Telerik: Fiddler Web Debugging Proxy
  4. Fiddler: Decrypting HTTPS-protected traffic
  5. Mozilla Developer Network: HTTP Headers
  6. SPO Public Site: Bunker Tuneage Online
  7. Blog Post: Save Your SharePoint Online Public Site from the Chopping Block
  8. Office Support: Tune SharePoint Online Performance

Save Your SharePoint Online Public Site from the Chopping Block

If you’re like me and have one or more SharePoint Online public sites, you may or may not be aware that they’re currently on the chopping block! In this post, I describe what’s going to happen, and I also cover the process you can follow to extend the life of your SPO public site for another year.

The GuillotineI’ve been very concerned about the fate of my SharePoint Online (SPO) public sites as of late. It’s March of 2017, and I know that Microsoft intends to pull the plug on all of those SPO public sites in the not-so-distant future. I have three of them myself: one for my wife’s non-profit organization (for which I’m also the CTO), one for my LLC, and a final one for my musical labor of love.

A while back, I pleaded with Microsoft publicly to give us some help before they shut things down for the SPO public sites. Well, it would seem that we’ve been given some help in the form of an end-of-life reprieve.

I had heard about the possibility of Microsoft pushing the deadline for the “ya gotta move it” date for SPO public sites, but I hadn’t been looking all that closely to see if there was any movement on that front. Since this month is due to close out in the next few days, I decided I’d better actually take a look. So, I went into one of my tenants and found what I’d hoped to find:

Postponing Deletion

Thank the Heavens!

If you’re like me and you haven’t been tracking things as closely as you might have liked, it turns out that you can spare your SharePoint Online public site a cruel and horrible death for roughly another year (i.e., until March 31 of 2018). The process for delaying your site’s demise is relatively straightforward and described in the body of this support article. If you want something a bit more visual, though, then the following walk-through might help you out.

  1. selectAdminSign in to your Office 365 tenant with a set of credentials that have the necessary rights to make changes to SharePoint Online settings. Go ahead – click the link I just supplied.
  2. Click on the waffle menu in the suite links bar near the top of the page. The waffle menu is opened by clicking those nine dots (arranged like a Rubik’s Cube). When you click the waffle menu button, you’ll get a menu with a bunch of tiles that looks something like the image above. You’re interested in the Admin button right now.
  3. Admin CenterClick the Admin button, and you’ll be taken to your tenant’s Admin center as shown on the right. I’ve branded my Bitstream Foundry tenant, so chances are your admin center is going to look different than mine – perhaps with a different color scheme and logo. Note that if your organization hasn’t assigned a logo, you won’t see one in the suite links bar.
  4. Admin centers drop-downAlong the left-hand side of the Admin center will be a set of collapsed drop-downs that represent your various administrative functions and management pages/areas. You’ll want to click on the Admin centers option at the bottom of the list to expand it as shown on the right. When you do this, you should see SharePoint listed between your Skype for Business and OneDrive options.
  5. SharePoint admin optionsClick on the SharePoint option, and you’ll be taken to the SharePoint admin center for your tenant. You’ll see the list of site collections that exist within your tenant in the main window area, and a toolbar will appear above the main window area providing you with options to create a new site collection, buy storage, and quite a bit more. you’ll also see the list of SharePoint-specific admin areas/options appear along the left-hand side of the admin page as shown to the right.
  6. Locate the settings option in the left-hand column and click it. Once you click it, you’ll see a whole host of settings that you can review and change. It is in this list that you’ll find the Postpone deletion of SharePoint Online public websites option buttons that I showed a bit earlier.
  7. Click on the I’d like to keep my public website until March 31, 2018 option button to pull your SPO public site off of death row.
  8. Scroll to the bottom of the page and click the OK button along the right-hand side of the page. This will save your change.

Save your changes!

That’s all there is to it!

Can’t You Just Give Me the Shortcut?

Sure! If you’re not into clicking through all of the admin screens and options I just walked through, you can simply point your browser at https://{tenantName}-admin.sharepoint.com/_layouts/15/online/TenantSettings.aspx to get to the page which is shown in Step #6 above. Note that you’ll need to replace the {tenantName} token in the URL above with the actual name of your tenant to make this work for you.

A Few Notes

This process buys you roughly another year to get your act together and move your SPO public site. You’ll then have until March 31 of 2018 to locate another home for your site and/or its content.

If you don’t follow the process I’ve outlined, Microsoft calls out the following dates:

  • Beginning May 1, 2017, anonymous access for your SPO public site will be removed.
  • On September 1, 2017, Microsoft will be deleting SPO public sites which haven’t been protected via the opt-in I described above. If you haven’t saved your SPO public site content by 9/1, you’re going to lose it!

Hopefully you’ll rest a bit easier (as I have been doing) after opting-in to protect your public site(s). I intend to get my sites moved before next March, and I’ll likely detail that process in a future post. But for now … deep breaths!

References and Resources

  1. Site: The Schizophrenia Oral History Project
  2. Site: Bitstream Foundry LLC
  3. Site: Bunker Tuneage
  4. Post: Help, We Are Stranded on SOPSI (SharePoint Online Public Site) Island
  5. Microsoft Support: Information about changes to the SharePoint Online Public Websites feature in Office 365
  6. Site: Rubik’s Cube

The Day Outlook Became My Secretary

In this post, I share a brief bit of magic that Outlook exhibited for me recently. I don’t know where it came from or if it is even is an indication of things to come … but I liked what I saw!

typewriterI feel that I’ve been tricked. Okay, maybe “tricked” is a harsh word, but let’s put it this way: I’ve seen a bit of the future, I like it, and I’m not sure if and when it’s coming back.

I recently returned from SPTechCon. While I was in San Francisco, I delivered a few sessions (including a new advanced PowerShell session) and managed to make it to Muir Woods to visit the Redwoods once again. The entire time I was in San Francisco, I was riding around in a rental car from Enterprise. I usually get my rental cars from Enterprise, but something weird happened when I was getting this rental car.

Outlook did me a favor.

When I booked the rental car with Enterprise, I received the following email:

Enterprise Pickup

Do you see the part stating “This event was automatically added to your calendar from email by Outlook?” That caught my attention. Outlook had never taken any action on my behalf prior to this trip, and I can’t say that I’ve seen it do anything since. But for some strange reason, this one car reservation got Outlook to do something new and cool.

I checked my calendar, and sure enough, there were events for both pickup and drop off.

Calendar View

I’ll be honest: I don’t know how these events got onto my calendar, and I don’t even know who wrote the code to make the magic happen. But in this one single instance, I feel like I’ve had a taste of what’s to come … and I really like it.

I did a little digging as I was writing this post to see if I could figure out where Outlook got its smarts from. I didn’t find a whole lot, but I did find this one post on Microsoft’s acquisition of Genee to accelerate intelligent experiences in Office 365. Maybe that had something to do with what I was seeing?

I like the idea of Outlook getting some intelligence and being able to look at my email to ascertain when things will happen. Maybe Delta will send me a trip confirmation and my flight times will end up on my calendar. Or maybe Mark will send me an email about a great Baconfest that’s happening in Harrison, Arkansas, and that event will get parsed and entered into my records so that I’ll know when I need to leave my house to make it there on-time.

I see a lot of potential for this sort of processing and assistance, but I think I’d like to understand it all a bit better before things move on. Heck, right now I’m not even sure if what happened to me is something that’s going to roll out more broadly … or if it was just a blip/test. As I indicated, I haven’t seen anything appear on my calendar since the car reservation, so I’m not even sure that it’s something that “someone” is rolling out.

But I like this. If it’s done right, it has the potential to simplify a lot of things we manually push ourselves to do today.

I’m okay with Outlook becoming my secretary. How about you?

ADDENDUM: 12/12/2016

My friend Tom Resing reached out via Facebook after I shared this blog post, and he opened the door to a world of settings I was simply unaware of. He pointed me to a link titled Automatically add travel and package delivery events to your calendar. It discusses how to control the behavior with Outlook online, and it’s definitely worth checking out. I’m always happy to discover new knobs and levers!

References and Resources

  1. Event: SPTechCon San Francisco
  2. Resources: Tapping the Power in PowerShell (slides)
  3. Resources: Tapping the Power in PowerShell (scripts)
  4. Location: Muir Woods
  5. Microsoft Blog: Microsoft acquisition of Genee to accelerate intelligent experiences in Office 365
  6. Blog: Mark Rackley
  7. Blog: Tom Resing
  8. Office Support: Automatically add travel and package delivery events to your calendar

Help, We Are Stranded on SOPSI (SharePoint Online Public Site) Island

In March of 2015, the Doomsday Clock started ticking for SharePoint Online Public Sites. Some have transitioned off of the service, but many of those least able to make the move (non-profits, user groups, small businesses) are stranded and concerned. In this post, I discuss the issue and my conversation with Jeff Teper about it. I also ask Microsoft to provide us with more help and assistance for transitioning away from SharePoint Online Public Sites.

SOPSI IslandA couple of weeks ago, I was down in Nashville, Tennessee speaking at SharePoint Saturday Nashville. The event was a huge success and a lot of fun to boot. Those two qualities tend to go hand-in-hand with SharePoint Saturday events, but the event in Nashville was different for one very important reason: it had a “distinguished guest.”

And Who To My Wondering Eyes Should Appear?

Who was the “distinguished guest” to whom I’m referring? Well, it was none other that Jeff Teper himself. Some of you may know the name and perhaps the man, but for those who don’t: Jeff is Microsoft’s Corporate Vice President for OneDrive and SharePoint. In essence, he’s the guy who’s primarily responsible for the vision and delivery of SharePoint both now and in the future. The Big Kahuna. Top of the Totem Pole. The Man in Command.

Jeff Teper and Sean McDonough

Jeff wasn’t in Nashville specifically for the event, but he took time out of his personal schedule to do an open Q&A session at the end of the SPS event. This was a *HUGE* deal, and it offered us (the speakers, organizers, and attendees) a rare chance to ask questions we’d always wanted to ask directly of the guy at the top.

Some of the questions were softballs, but several weren’t. A few of us(Mark Rackley, Seb Matthews, myself …) took the opportunity to ask questions that we anticipated might be uncomfortable but were nonetheless important to ask. To Jeff’s credit, he did a fantastic job of listening and responding to each question he received.

So, About These SharePoint Public Sites In Office 365 …

I asked Jeff several questions, but only one of them dealt with a topic that had started becoming a true area of concern for me: SharePoint Online Public Websites.

Some of you may be thinking, “Wait – what are you talking about?” If you came to SharePoint Online after March of 2015, then you might not even be aware that most Office 365 plans prior to that point came with a public-facing website that companies and organizations could use for a variety of purposes: public presence, blogging, e-commerce, and more. It was an extremely easy way for small-to-mid-size organizations to hang their shingle on the web for very little money and with little technical know-how.

Unfortunately, Microsoft announced in January of 2015 that it was deprecating SharePoint Online public sites. Beginning on March 9th of 2015, new customers did not receive a public site with their tenant. Those who already had the public sites, though, were allowed to keep them for a minimum of two years. In that two year period, the organizations with the public sites needed to “move on” and find an alternate hosting option. Microsoft eventually offered up a few options for public site owners, but they didn’t go very far.

Before I continue there, though, let me rewind for some additional context.

Public Sites: The Early Days

The Schizophrenia Oral History Project OnlineLike many smaller businesses, non-profits, user groups, and other non-enterprise customers, I bought into the SharePoint Online Public Website vision in a BIG way when it was laid out at the Microsoft’s SharePoint Conference in 2012. I remember thinking, “this is going to simplify the web presence problem for so many folks who are ill-equipped to deal with the burden of a ‘big site’ and web content management platform.”

Shortly after they became available to me, I set up several of the public sites for my own use. I also put my wife’s non-profit organization on one. As of right now (May 27, 2016), these sites are still alive-and-well in SharePoint Online:

I recommended SharePoint Online public sites to everyone who needed “an Internet presence that was both cheap and easy.” That said, it’s probably easy to understand that the bulk of the public site adoptees (that I saw) were organizations who either lacked money, formal IT capabilities, or a combination of the two.

Back To Now: Why Am I Losing Sleep?

The Clock Is TickingIt’s currently late in May of 2016. The plug could get pulled on SharePoint Online public sites as soon as March 2017. The clock is ticking, time is running out, and I don’t yet have a plan for transitioning to something else for the sites I cited above.

I’m not alone. It seems I’m getting into more and more conversations with other Office 365 customers about the topic, and they don’t know what to do either. It’s not that they want to wait until the last minute to make the move; they simply don’t know how to get off the SOPSI Island.

In my estimation, the organizations that have money and IT capabilities have either transitioned to another platform or are in the process of building a viable plan. As I wrote earlier, though, I think the greatest adoption of these public sites was among those who are traditionally the least capable and underfunded: small-to-mid-size companies, non-profits, user groups, and the like.

When I speak with customers in those segments, their concerns echo my own. They’re still on Office 365 Public Sites and haven’t gone to something else because they lack the money and capability to do so. And they’re growing increasingly worried.

Those Are The Alternatives?

Here’s another problem with this situation: the other hosting platforms and options that Microsoft has tossed our way don’t actually provide any sort of bridge or migration option between SharePoint Online public sites and their platforms.

The reality in all of this is that we won’t be migrating: we’ll be rebuilding. We’re going to need to find some way to drag our content out of the pages we’ve created, and then we need to go somewhere else and rebuild from the ground-up.

GrumpySure, Microsoft has provided us with a “migration support” resource, but as I size it up, the “guidance” it provides is more abstract hand-waiving than usable, actionable content. Go read it. Would you feel confident migrating to one of the third party providers mentioned with the instructions as they’re laid-out? I know I wouldn’t – and I work in IT for a living.

And, of course, any time that was spent customizing a SharePoint Online public site is going to go out the window. That tends to happen in migrations (disclosure: I’ve been doing SharePoint migrations in some form for the better part of a decade), and that’s probably acceptable in the grand scheme of things … but the users who truly need help need something more than the guidance provided in the online resource.

Back To My Conversation With Jeff Teper

Fast-forward back to Nashville a couple of weeks ago.

Although I asked Jeff “Hey, what happened with the SPO public sites?,” the question that I really wanted to get an answer to was this: “Why are our options for exiting the SharePoint Online public site platform so … lousy?”

Jeff took the time to respond to the various pieces of my question, but when we got to talking about migration options and the people who were currently “stuck,” the response was something to the effect of this: he thought that most folks had already migrated or were in the process of doing so.

At that point, various other folks in the audience (representing user groups, non-profits, etc.) started sounding-off and explaining that they were stuck, too. Clearly, I wasn’t the only one with sites hanging out on SOPSI Island.

Jeff indicated he’d take our input and concerns back to Microsoft, and I believe that he will. But just to put the request in writing …

My Request To The Microsoft SharePoint Online Team

HopefulOn behalf of all of the non-profits, small-to-mid-sized companies, user groups, and others stranded on SOPSI Island: please build us a reasonable bridge or provide us with some additional hand-holding (or services) to help us safely leave the island.

At a minimum, we need better and more practical, prescriptive guidance. For some, a tool might help – perhaps something to package up assets to take them somewhere else. If I’m allowed to dream, a tool that might actually carry out some form of migration would probably be appreciated tremendously by the smaller, less-capable customers. Regardless of the specific form(s), we need more help and probably more time to make the move.

When SOPSI Island is (likely) wiped-out in 2017, we don’t want to still be stuck on it – watching our sites disappear forever.

References and Resources

  1. Event: SharePoint Saturday Nashville 2016
  2. Events: SPSEvents.org
  3. LinkedIn: Jeff Teper
  4. Twitter: Mark Rackley
  5. Twitter: Seb Matthews
  6. Microsoft Support: Information about changes to the SharePoint Online Public Website feature in Office 365
  7. Channel 9: Deep Dive on the Capabilities of SharePoint Online’s New Public Website
  8. Office Support: Migrate you SharePoint Online Public Website to a partner website

SPTechCon Austin 2016 – The Videos!

In my last post, I promised those who attended my Content Search Web Part session (at SPTechCon Austin 2016) that I’d deliver videos of the demos I normally perform during that session. This post contains links to those demo videos as well as some additional commentary.

video playerAs I discussed in my last post titled “SPTechCon Austin 2016 And Death By Demo,” the demonstrations I intended to deliver at SPTechCon in Austin a week and a half ago didn’t go very well. In fact, they didn’t really go at all due to some extremely odd technical circumstances. To make up for the lack of demo content, I promised attendees that I would put together video walk-throughs for each of the demos I had intended to deliver at SPTechCon.

It took a little longer than initially anticipated, but the half-dozen links below represent the demo material I would normally walk through during a delivery of my “SharePoint’s New Swiss Army Knife: The Content Search Web Part” session. If there’s a silver lining to the fact that I’m doing the demos after the actual presentation, it’s that I was able to take more time than I normally have (within the context of a 75 minute session) to show some extra content and go off the beaten path a bit more.

So, for those of you who have been waiting … here are the goods!

These videos were recorded with Camtasia and rendered directly out to YouTube. I made every attempt to keep the quality high, but if something gets “lost in translation” or you have other issues, please let me know.

I enjoyed putting these videos together, and in the past I’ve tossed around the idea of doing more videos like this. If these CSWP videos were helpful to you and/or you’d like to see more, please let me know. If enough of you find value in these, I’d be willing to put together additional videos for some of the other presentations and workshops I deliver.

Enjoy, and as with everything else, I welcome your feedback!

References and Resources

  1. Blog Post: SPTechCon Austin 2016 And Death By Demo
  2. Resources: SharePoint’s New Swiss Army Knife: The Content Search Web Part (SPTechCon Austin 2016)
  3. Software: TechSmith’s Camtasia Studio
  4. Site: YouTube

SPTechCon Austin 2016 And Death By Demo

I just got back from SPTechCon Austin 2016, and I had some “trouble” (putting it mildly) with demos I gave during one of my sessions. This post is a note – and a promise – to those who attended my Content Search Web Part (CSWP) session during the conference.

This was me after my CSWP session yesterdayI used to write posts to sum-up the various conferences at which I’ve spoken. That was feasible when I was only speaking at a conference or event here or there, but writing about every event is somewhat time-consuming nowadays. And besides, most of the posts would look about the same: “great event,” “lots of fun,” “awesome attendees,” etc.

Well, I got back from SPTechCon Austin 2016 yesterday … and I felt compelled to write something today. Yes, it was a great conference, lots of fun, and filled with awesome attendees. But there was something more to this conference that motivated me – no, compelled me – to write this post.

Compelled By What?

That “thing” that compelled me was this: death by demo.

I delivered two sessions during the event: a new one on performance troubleshooting with SharePoint Online, and one of my “standards” that is an introduction to the Content Search Web Part (CSWP). I delivered the troubleshooting session on Tuesday, and although it went long (I still need to tune it up), it went pretty well – no real issues. I can’t claim the same about the CSWP session yesterday (Wednesday) morning.

Simply put, the demos for my CSWP session were a disaster. I’d gotten everything ready to go on the Tuesday night before the session; despite that, things went off-the-rails almost immediately. I was RDP’ing back to my home desktop system where I had VMware Workstation running, and all of that (i.e., the RDP and VWware Workstation parts) seemed fine. The fashion in which things blew up was not something I’d ever seen before.

Kerplunk? Kerplooey!

What went wrong? Well, it’s hard to describe. The best way to describe it is that left-clicking didn’t work properly in the development VM I was using. Sometimes my clicks would visibly register (e.g., on a window close button) – but nothing would happen. Other times, my left-clicks seemed to register somewhere else on the screen (other than where the mouse pointer was located). And at other times still, a left-click would highlight some weird section in the web browser window.

Because of this aberrant mouse behavior, I couldn’t show the demo material. I certainly tried enough times, and I even hobbled through one demo with the audience members helping me by shouting out keyboard shortcuts when I asked … but it was a total wreck.

Attendees for my CSWP session at SPTechConIf you were in attendance for this session (and there were quite a few folks, as shown in the picture on the right – taken a handful of minutes before I started), I truly apologize. An apology alone, though, isn’t enough (in my opinion).

My Attempt to Make Up

As the demos were slamming into walls and catching on fire, I commented a couple of times that I’d find some way to share the demo materials with the audience at a later time. I was initially thinking I’d try to do a webcast – kind of a do-over – but I thought about it some more on the plane ride home last night and decided on something else.

Here’s what I’m going to do: rather than do the whole session over again, I’m going to work through each of the demos I intended to show and record those as a Camtasia/video that can be viewed whenever someone has the time to do so. Doing this sort of video cuts straight to the chase and is ultimately more flexible than trying to round everyone up for a webcast. It can also be re-watched as desired.

“When is this video going to be ready,” you might ask? I need to do some catching-up after having been out of town for a while, but I’m hoping to find the time this coming weekend to put it together. If I can do that, then the video will be available sometime early next week.

How Will We Know?

Once everything is ready to go, I’ll put together another blog post to announce the availability and provide a link. I’ve also been in contact with David Rubinstein at BZ Media about this, and he said that he’d blast the information out to attendees and newsletter subscribers, as well.

Summary

So, once again: my sincere apologies to those who attended my CSWP session at SPTechCon. It’ll be a few days after the actual session, but hopefully the video will make up for the demos that went nowhere during the session.

Caching, You Ain’t No Friend Of Mine

I love caching and all that it can do to boost performance, but caching for SharePoint in the cloud isn’t the same as it is on-premises. In this post, I explore why that is for Object Caching – and what you can do about it.

I've got a caching-induced headacheI’m a big fan of leveraging caching to improve performance. If you look over my blog, you’ll find quite a few articles that cover things like implementing BLOB caching within SharePoint, working with the Object Cache, extending your own code with caching options, and more. And most of those posts were written in a time when the on-premises SharePoint farm was king.

The “caching picture” began shifting when we started moving to the cloud. SharePoint Online and hosted SharePoint services aren’t the same as SharePoint on-premises, and the things we rely upon for performance improvements on-premises don’t necessarily have our backs when we move out to the cloud.

Yeah, I’m talking about caching here. And as much as it breaks my heart to say it, caching – you ain’t no friend of mine out in SharePoint Online.

Why the heartbreak?

To understand why a couple of SharePoint’s traditional caching mechanisms aren’t doing you any favors in a multi-tenant service like SharePoint Online (with or without Office 365), it helps to first understand how memory-based caching features – like SharePoint’s Object Cache – work in an on-premises environment.

On-Premises

The typical on-premises environment has a small number of web front-ends (WFEs) serving content to users, and the number of site collections being served-up is relatively limited. For purposes of illustration, consider the following series of user requests to an environment possessing two WFEs behind a load balancer:

On-Premises Request Results

Assuming the WFEs have just been rebooted (or the application pools backing the web applications for target site collection have just been recycled) – a worst-case scenario – the user in Request #1 is going to hit a server (either #1 or #2) that does not have cached content in its Object Cache. For this example, we’ll say that the user is directed to WFE #1. Responses from WFE #1 will be slower as SharePoint works to generate the content for the user and populate its Object Cache. The WFE will then return the user’s response, but as a result of the request, its Object Cache will contain site collection-specific content such as navigational sitemaps, Content Query Web Part (CQWP) query results, common site property values, any publishing page layouts referenced by the request, and more.

The next time the farm receives a request for the same site collection (Request #2), there’s a 50/50 shot that the user will be directed to a WFE that has cached content (WFE #1, shown in green) or doesn’t yet have any cached content (WFE #2). If the user is directed to WFE #1, bingo – a better experience should result. Let’s say the user gets unlucky, though, and hits WFE #2. The same process as described earlier (for WFE #1) ensues, resulting in a slower response to the user but a populated Object Cache on WFE #2.

By the time we get to Request #3, both WFEs have at least some cached content for the site collection being visited and should thus return responses more quickly. Assuming memory pressure remains low, these WFEs will continue to serve cached content for subsequent requests – until content expires out of the cache (forcing a re-fetch and fill) or gets forced out for some reason (again, memory pressure or perhaps an application pool recycle).

Another thing worth noting with on-premises WFEs is that many SharePoint administrators use warm-up scripts and services in their environments to make the initial requests that are described (in this example) by Request #1 and Request #2. So, it’s possible in these environments that end-users never have to start with a completely “cold” WFE and make the requests that come back more slowly (but ultimately populate the Object Caches on each server).

SharePoint Online

Let’s look at the same initial series of interactions again. Instead of considering the typical on-premises environment, though, let’s look at SharePoint Online.

Cloud

The first thing you may have noticed in the diagrams above is that we’re no longer dealing with just two WFEs. In a SharePoint Online tenant, the actual number of WFEs is a variable number that depends on factors such as load. In this example, I set the number of WFEs to 50; in reality, it could be lower or (in all likelihood) higher.

Request #1 proceeds pretty much the same way as it did in the on-premises example. None of the WFEs have any cached content for the target site collection, so the WFE needs to do extra work to fetch everything needed for a response, return that information, and then place the results in its Object Cache.

In Request #2, one server has cached content – the one that’s highlighted in green. The remaining 49 servers don’t have cached content. So, in all likelihood (49 out of 50, or 98%), the next request for the same site collection is going to go to a different WFE.

By the time we get to Request #3, we see that another WFE has gone through the fetch-and-fill operation (again, highlighted in green). But, there’s something else worth noting that we didn’t see in the on-premises environment; specifically, the previous server which had been visited (in Request #1) is now red, not green. What does this mean? Well, in a multi-tenant environment like SharePoint Online, WFEs are serving-up hundreds and perhaps thousands of different site collections for each of the residents in the SharePoint environment. Object Caches do not have infinite memory, and so memory pressure is likely to be a much greater factor than it is on-premises – meaning that Object Caches are probably going to be ejecting content pretty frequently.

If the Object Cache on a WFE is forced to eject content relevant to the site collection a user is trying to access, then that WFE is going to have to do a re-fetch and re-fill just as if it had never cached content for the target site collection. The net effect, as you might expect, is longer response times and potentially sub-par performance.

The Take-Away

If there’s one point I’m trying to make in all of this, it’s this: you can’t assume that the way a SharePoint farm operates on-premises is going to translate to the way a SharePoint Online farm (or any other multi-tenant farm) is going to operate “out in the cloud.”

Is there anything you can do? Sure – there’s plenty. As I’ve tried to illustrate thus far, the first thing you can do is challenge any assumptions you might have about performance that are based on how on-premises environments operate. The example I’ve chosen here is the Object Cache and how it factors into the performance equation – again, in the typical on-premises environment. If you assume that the Object Cache might instead be working against you in a multi-tenant environment, then there are two particular areas where you should immediately turn your focus.

Navigation

By default, SharePoint site collections use structural navigation mechanisms. Structural navigation works like this: when SharePoint needs to render a navigational menu or link structure of some sort, it walks through the site collection noting the various sites and sub-sites that the site collection contains. That information gets built into a sitemap, and that sitemap is cached in the Object Cache for faster retrieval on subsequent requests that require it.

Without the Object Cache helping out, structural navigation becomes an increasingly less desirable choice as site hierarchies get larger and larger. Better options include alternatives like managed navigation or search-driven navigation; each option has its pros and cons, so be sure to read-up a bit before selecting an option.

Content Query Web Parts

When data needs to be rolled-up in SharePoint, particularly across lists or sites, savvy end-users turn to the CQWP. Since cross-list and cross-site queries are expensive operations, SharePoint will cache the results of such a query using – you guessed it – the Object Cache. Query results are then re-used from the Object Cache for a period of time to improve performance for subsequent requests. Eventually, the results expire and the query needs to be run again.

So, what are users to do when they can’t rely on the Object Cache? A common theme in SharePoint Online and other multi-tenant environments is to leverage Search whenever possible. This was called out in the previous section on Navigation, and it applies in this instance, as well.

An alternative to the CQWP is the Content Search Web Part (CSWP). The CSWP operates somewhat differently than the CQWP, so it’s not a one-to-one direct replacement … but it is very powerful and suitable in most cases. Since the CSWP pulls its query results directly from SharePoint’s search index, it’s exceptionally fast – making it just what the doctor ordered in a multi-tenant environment.

Quick note (2/1/2016): Thanks to Cory Williams for reminding me that the CSWP is currently only available to SharePoint Online Plan 2 and other “Plan 3” (e.g., E3, G3) users. Many enterprise customers fall into this bucket, but if you’re not one of them, then you won’t find the CSWP for use in your tenant :-(

There are plenty of good resources online for the CSWP, and I regularly speak on it myself; feel free to peruse resources I have compiled on the topic (and on other topics).

Wrapping-Up

In this article, I’ve tried to explain how on-premises and multi-tenant operations are different for just one area in particular; i.e., the Object Cache. In the future, I plan to cover some performance watch-outs and work-arounds for other areas … so stay tuned!

Additional Reading and References

  1. MSDN: Navigation options for SharePoint Online
  2. MSDN: Using Content Search Web Part instead of Content Query Web Part to improve performance in SharePoint Online
  3. SharePoint Interface: Presentations and Materials