I love caching and all that it can do to boost performance, but caching for SharePoint in the cloud isn’t the same as it is on-premises. In this post, I explore why that is for Object Caching – and what you can do about it.
I’m a big fan of leveraging caching to improve performance. If you look over my blog, you’ll find quite a few articles that cover things like implementing BLOB caching within SharePoint, working with the Object Cache, extending your own code with caching options, and more. And most of those posts were written in a time when the on-premises SharePoint farm was king.
The “caching picture” began shifting when we started moving to the cloud. SharePoint Online and hosted SharePoint services aren’t the same as SharePoint on-premises, and the things we rely upon for performance improvements on-premises don’t necessarily have our backs when we move out to the cloud.
Yeah, I’m talking about caching here. And as much as it breaks my heart to say it, caching – you ain’t no friend of mine out in SharePoint Online.
Why the heartbreak?
To understand why a couple of SharePoint’s traditional caching mechanisms aren’t doing you any favors in a multi-tenant service like SharePoint Online (with or without Office 365), it helps to first understand how memory-based caching features – like SharePoint’s Object Cache – work in an on-premises environment.
On-Premises
The typical on-premises environment has a small number of web front-ends (WFEs) serving content to users, and the number of site collections being served-up is relatively limited. For purposes of illustration, consider the following series of user requests to an environment possessing two WFEs behind a load balancer:
Assuming the WFEs have just been rebooted (or the application pools backing the web applications for target site collection have just been recycled) – a worst-case scenario – the user in Request #1 is going to hit a server (either #1 or #2) that does not have cached content in its Object Cache. For this example, we’ll say that the user is directed to WFE #1. Responses from WFE #1 will be slower as SharePoint works to generate the content for the user and populate its Object Cache. The WFE will then return the user’s response, but as a result of the request, its Object Cache will contain site collection-specific content such as navigational sitemaps, Content Query Web Part (CQWP) query results, common site property values, any publishing page layouts referenced by the request, and more.
The next time the farm receives a request for the same site collection (Request #2), there’s a 50/50 shot that the user will be directed to a WFE that has cached content (WFE #1, shown in green) or doesn’t yet have any cached content (WFE #2). If the user is directed to WFE #1, bingo – a better experience should result. Let’s say the user gets unlucky, though, and hits WFE #2. The same process as described earlier (for WFE #1) ensues, resulting in a slower response to the user but a populated Object Cache on WFE #2.
By the time we get to Request #3, both WFEs have at least some cached content for the site collection being visited and should thus return responses more quickly. Assuming memory pressure remains low, these WFEs will continue to serve cached content for subsequent requests – until content expires out of the cache (forcing a re-fetch and fill) or gets forced out for some reason (again, memory pressure or perhaps an application pool recycle).
Another thing worth noting with on-premises WFEs is that many SharePoint administrators use warm-up scripts and services in their environments to make the initial requests that are described (in this example) by Request #1 and Request #2. So, it’s possible in these environments that end-users never have to start with a completely “cold” WFE and make the requests that come back more slowly (but ultimately populate the Object Caches on each server).
SharePoint Online
Let’s look at the same initial series of interactions again. Instead of considering the typical on-premises environment, though, let’s look at SharePoint Online.
The first thing you may have noticed in the diagrams above is that we’re no longer dealing with just two WFEs. In a SharePoint Online tenant, the actual number of WFEs is a variable number that depends on factors such as load. In this example, I set the number of WFEs to 50; in reality, it could be lower or (in all likelihood) higher.
Request #1 proceeds pretty much the same way as it did in the on-premises example. None of the WFEs have any cached content for the target site collection, so the WFE needs to do extra work to fetch everything needed for a response, return that information, and then place the results in its Object Cache.
In Request #2, one server has cached content – the one that’s highlighted in green. The remaining 49 servers don’t have cached content. So, in all likelihood (49 out of 50, or 98%), the next request for the same site collection is going to go to a different WFE.
By the time we get to Request #3, we see that another WFE has gone through the fetch-and-fill operation (again, highlighted in green). But, there’s something else worth noting that we didn’t see in the on-premises environment; specifically, the previous server which had been visited (in Request #1) is now red, not green. What does this mean? Well, in a multi-tenant environment like SharePoint Online, WFEs are serving-up hundreds and perhaps thousands of different site collections for each of the residents in the SharePoint environment. Object Caches do not have infinite memory, and so memory pressure is likely to be a much greater factor than it is on-premises – meaning that Object Caches are probably going to be ejecting content pretty frequently.
If the Object Cache on a WFE is forced to eject content relevant to the site collection a user is trying to access, then that WFE is going to have to do a re-fetch and re-fill just as if it had never cached content for the target site collection. The net effect, as you might expect, is longer response times and potentially sub-par performance.
The Take-Away
If there’s one point I’m trying to make in all of this, it’s this: you can’t assume that the way a SharePoint farm operates on-premises is going to translate to the way a SharePoint Online farm (or any other multi-tenant farm) is going to operate “out in the cloud.”
Is there anything you can do? Sure – there’s plenty. As I’ve tried to illustrate thus far, the first thing you can do is challenge any assumptions you might have about performance that are based on how on-premises environments operate. The example I’ve chosen here is the Object Cache and how it factors into the performance equation – again, in the typical on-premises environment. If you assume that the Object Cache might instead be working against you in a multi-tenant environment, then there are two particular areas where you should immediately turn your focus.
Navigation
By default, SharePoint site collections use structural navigation mechanisms. Structural navigation works like this: when SharePoint needs to render a navigational menu or link structure of some sort, it walks through the site collection noting the various sites and sub-sites that the site collection contains. That information gets built into a sitemap, and that sitemap is cached in the Object Cache for faster retrieval on subsequent requests that require it.
Without the Object Cache helping out, structural navigation becomes an increasingly less desirable choice as site hierarchies get larger and larger. Better options include alternatives like managed navigation or search-driven navigation; each option has its pros and cons, so be sure to read-up a bit before selecting an option.
Content Query Web Parts
When data needs to be rolled-up in SharePoint, particularly across lists or sites, savvy end-users turn to the CQWP. Since cross-list and cross-site queries are expensive operations, SharePoint will cache the results of such a query using – you guessed it – the Object Cache. Query results are then re-used from the Object Cache for a period of time to improve performance for subsequent requests. Eventually, the results expire and the query needs to be run again.
So, what are users to do when they can’t rely on the Object Cache? A common theme in SharePoint Online and other multi-tenant environments is to leverage Search whenever possible. This was called out in the previous section on Navigation, and it applies in this instance, as well.
An alternative to the CQWP is the Content Search Web Part (CSWP). The CSWP operates somewhat differently than the CQWP, so it’s not a one-to-one direct replacement … but it is very powerful and suitable in most cases. Since the CSWP pulls its query results directly from SharePoint’s search index, it’s exceptionally fast – making it just what the doctor ordered in a multi-tenant environment.
Quick note (2/1/2016): Thanks to Cory Williams for reminding me that the CSWP is currently only available to SharePoint Online Plan 2 and other “Plan 3” (e.g., E3, G3) users. Many enterprise customers fall into this bucket, but if you’re not one of them, then you won’t find the CSWP for use in your tenant :-(
In this article, I’ve tried to explain how on-premises and multi-tenant operations are different for just one area in particular; i.e., the Object Cache. In the future, I plan to cover some performance watch-outs and work-arounds for other areas … so stay tuned!
It would appear that things are more or less back to normal. I never got an “everything is okay and live” email, but theming and branding are working properly both on public sites (tick, tock, tick, tock …) and internal sites. No conflicts at this point. Since I like to tie things up when complete, we’ll call this one “done” for now and move on.
Update (Evening 7/23/2015)
Microsoft has been looking at this issue, and progress is being made! My public site looks like it has returned to normal … but I know that we’re not quite out of the woods yet.
John from Microsoft followed-up with me yesterday and said the following:
“We have pulled the flight that was impacting everyone from production. The plan is to address these issues before turning the flight back on. Would you be up for piloting the upgraded flight before we turn it back on for everyone? Also, you mention you know others who are having problems, would they be willing to pilot the new flight as well? If so, please provide their contact information so I can follow up with them.”
Clearly, there’s a strong “flying vibe” in Redmond ….
I told John that I was definitely a “go” for the flight, and that I knew some others who’d experienced problems. And that’s where I’m hoping that some of you can help.
If you have been encountering problems with your Office 365 public site that are similar to mine – and you want to be part of the fix – let me know and I’ll hook you up with John. Shoot me your name, email address, and public site URL; I think that will do the job.
Stay tuned!
Original Post (below)
First of all, let me state that I’m not talking about the fact that Office 365 public sites are going the way of the dinosaur. That’s old news at this point. Instead, I’m talking about a “disruption in the force” that some of you may have observed when recently browsing to your public sites and discovering that they had … changed.
And what do I mean by “changed?” In my case, it was the observation that my public site’s branding had been altered to something I hadn’t chosen. The background image was different, the fonts weren’t the same, a number of the CSS styles I had applied to address scrollbar positioning and what-not weren’t in effect, and more. In essence, my custom branding had been completely steamrolled.
The After And The (Sort-Of Before)
Don’t take my word for it, though. Have a look for yourself. On the left is my public site as I discovered it a couple of weeks back (i.e., near the beginning of July, 2015). On the right is the way it’s supposed to look … sort of. The fonts and aspects of the responsive design are still off in the “corrected” version on the right, but I managed to hack the proper background, scrollbars, and a few other elements back to where they were previously. Even though I didn’t get everything corrected, differences between the two are immediately obvious.
I was confident that it had been months since I had changed anything on my public site, but I verified the branding assets in the Office 365 site against what I was tracking in source control. As expected, they were identical. The changes I was observing were not due to anything I had done to the public site.
Grumble Grumble …
Since it wasn’t the first time I’d had issues with unexpected changes and behavior on my public site, I wasted little time before going to Facebook to complain aloud. Many of my friends in the SharePoint community are also friends on Facebook, so I figured I’d get some support (moral, if nothing else) there.
Shortly after posting the update seen on the left, I tagged a couple of my Microsoft friends (specifically, Jeremy Thake and Chris Johnson) who work with the Office 365 team(s) and asked if I should have known about an update or change that might have affected my public site in this fashion. Even though I was confident that I hadn’t done anything to directly impact my public site’s look-and-feel, I was not about to rule out the possibility that I had missed something that had been communicated to me. In fact, I have been consulting in information technology long enough to know that my “mental glove” doesn’t catch everything thrown to me; anymore, I just kind of assume that I am in error and work from there.
Jeremy got back to me first and indicated that he didn’t have anything specific to share, but he indicated he would take the issue to folks (internally) who should know. Shortly after that, Chris tagged Steve Walker (another Microsoftee with superhuman powers) to bring the issue to his attention. Steve told me to file a service request (SR) and that he would escalate that SR right away. So, I filed the service request with supporting screenshots and documentation … and as he had indicated he would, Steve escalated the SR in less than an hour.
In the meantime, I did what I could to get some of my site’s branding back to the way it had been. How did I do that? With the liberal application of the dreaded !important CSS directive in the custom style sheet I had created to go with my master page. The !important directive is definitely not something I use (or even like to go near) on a daily basis, but in this case, I was trying to achieve results with a minimal investment of time and effort. I needed my styling to trump whatever was being laid-down after my style sheet was being processed.
So, What Happened Next?
Following Steve’s escalation, I started working with an extremely approachable escalation engineer named John. John and I exchanged some emails, did a late-night screen-sharing session, and generally looked things up-and-down. John concurred that what we were seeing shouldn’t have been happening, and he mentioned that he had another customer or two that seemed to be having a similar problem. Some tracing in those cases seemed to implicate a recent client-side theming change that had been made and rolled-out.
An issue with client-side theming “felt” right to me. I had used Internet Explorer’s Developer Tools to do some backtracking into the source of the errant styles that were being applied to my site, and I noticed that the background image was being served from a temporary theme directory on the server. Until I started forcing my styles to override the ones that were being applied (an example of which is shown on the right), the theme styles were trumping my own styles.
John was out for a while, but he came back to me recently with the following explanation. And his explanation makes complete sense:
So the problem you were having was the result of a bad interaction with a third-party CSS minification technique called minisp. By adding HTML comments around our CssLink controls in the master page, the <link> tags we normally render are part of a comment and therefore not part of the DOM.
When Client-Side Theming goes to replace the CSS on these pages, it wants to put the generated <style> blocks immediately before the corresponding <link> tags. Since it can’t find the <link> tags in the DOM, it defaults to adding the <style> blocks to the end of the document head. Since these come after the custom CSS, the rules in our themed CSS take priority over the rules in the custom CSS.
Resolution?
As of July 21st, 2015, there is no official resolution. Since this has been identified as a problem in how client-side theming handles CSS <style> block insertions, though, the ultimate fix needs to come from Microsoft. In the meantime, I’ll be sticking with my hacked CSS style sheet to get back most of the look-and-feel that I need. I could expend additional (development) effort to ensure that my styles are applied after the Office 365 theme styles without using !important, but there are plenty of other more important tasks vying for my attention right now.
So, if you found this post and it’s helping you to realize that you’re not going insane (at least not because of public site branding changes you didn’t make), I’ll feel that my job is done.
As I hear more and changes take place, I’ll try to update this post accordingly. Check back every now and then if you want the play-by-play on this issue.
Parting Thoughts: A Tip Of My Hat To Microsoft
In the past, I’ve been pretty vocal about the way that Microsoft has sort of “rolled” changes onto its Office 365 customer base and failed to communicate problems in a timely and complete fashion – actions (or lack of action) that ultimately caused pain and problems. As vocal as I’ve been in those situations, I want to go on-record as saying (just as loudly) that Microsoft has definitely listened to the critical feedback it has been receiving and has acted to make changes that we have indicated we need.
Even though this public site branding issue is indeed a bug, Microsoft listened and responded quickly – without protest, without claiming that “nothing is wrong,” and without some of the problem behaviors I used to see.
Where there were previously few communications about Office 365 outages, problems, changes, and updates, we now have a boatload of information (with some of it actually being pushed) to us to keep us in-the-know on our tenants, where they stand at any given point in time, and where they are going. I can get both at-a-glance health information and deep explanations for issues using the administrative portal’s Service Health dashboard. I get push notifications whenever something happens in my tenant using the Office 365 Admin application that runs on my Windows Phone. I know when new service features and capabilities are rolling out, if changes have been cancelled, etc., by looking at the Office 365 Roadmap. And these are just some of the channels and information streams that are available.
Is Microsoft “all the way there” yet? No, but they are dramatically further along than when Office 365 first rolled-out. Outages still occur – as they do with any service – but I feel like I know what’s going on now. That’s a huge improvement in my book.
What started as a simple attempt to use the ~appWebUrl token in an image URL became a deep dive into SharePoint’s internal processing of custom actions and the App deployment process. In this post, I cover what will and won’t work for custom action image URLs in your own SharePoint 2013 Apps.
My adventures in SharePoint 2013 App Model Land have been going pretty well, but I recently encountered a limitation that left me sort of scratching my head.
The limitation applies to the creation of custom actions for SharePoint apps. To be more specific: the problem I’ve encountered is that there doesn’t appear to be a way to package and reference (using relative links) custom images for ribbon buttons like the one that’s circled in the image above and to the left. This doesn’t mean that custom images can’t be used, of course, but the work-around isn’t exactly something I’m particularly fond of (nor is it even feasible) in some application scenarios.
If you’re not familiar with the new SharePoint 2013 App Model, then you may want to do a little reading before proceeding with this post. I’m only going to cover the App Model concepts that are relevant to the limitation I observed and how to address/work-around it. However, if you are familiar with the new 2013 App Model and creating custom actions in SharePoint 2010, then you may want to jump straight down to the section titled Where the Headaches Begin.
One more warning: this post does some heavy digging into SharePoint’s internal processing of custom ribbon actions and URL tokens. If you want to skip all of that and head straight to the practical take-away, jump down to the What About the Image32by32 and Image16by16 Attributes section.
Adding a Ribbon Custom Action
First, let me do a quick run-through on custom actions. They aren’t unique to SharePoint 2013 or its new “Cloud App Model.” In fact, the type of custom action I’m talking about (i.e., extending the ribbon) became available when the Ribbon was introduced with SharePoint 2010.
With a SharePoint 2013 App, adding a new button to the ribbon is a relatively simple affair. It starts with choosing the Ribbon Custom Action option from the Add New Item dialog as shown below and to the left. Once a name is provided for the custom action and the Add button is clicked, the Create Custom Action for Ribbon dialog appears as shown below and to the right. There’s a third dialog page that further assists in setting some properties for a custom action, but I’m going to skip over it since it isn’t relevant to the point I’m trying to make.
I want to call attention to one of the selections I made on the Create Custom Action for Ribbon dialog, though; specifically, the decision to expose the custom action in the Host Web rather than in the App Web.
Why is this choice so important? Well, the new App Model enforces a relatively strict boundary of separation between SharePoint sites and any custom applications (running under the new App Model) that they may contain. A SharePoint site (Host Web) can technically “host” applications, but those applications operate in an isolated App Web that may have components running on an entirely different server. Under the new App Model, no custom app code is running in the Host Web.
App Webs (where custom applications exist after installation) don’t have direct access to the Host Web in which they’re contained, either. In fact, App Webs are logically isolated from their Host Web parents. If App Webs want to communicate with their Host Web parent to interact with site collection data, for example, they have to do so through SharePoint’s Client-Side Object Model (CSOM) or the Representational State Transfer (REST) interface. The old full-trust, server-side object isn’t available; everything is “client-side.”
There are some exceptions to this model of isolation, and one of those exceptions is the use of custom actions to allow an App (residing in an App Web) to partially wire itself into the Host Web. The Create Custom Action for Ribbon dialog shown above, for instance, adds a new button to the ribbon for each of the Document Libraries in the Host Web. This gives users a way to navigate directly from Document Libraries (in the Host Web) to a page in the App Web, for example.
The Elements.xml file that gets generated for the custom action once the Visual Studio wizard has finished running looks something like the following:
[sourcecode language=”XML” autolinks=”false”]
<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
<CustomAction Id="1470c964-6b8a-4d79-9817-4d32c898ffbe.RibbonCustomAction1"
RegistrationType="List"
RegistrationId="101"
Location="CommandUI.Ribbon"
Sequence="10001"
Title="Invoke 'LibraryDetailsCustomAction' action">
<CommandUIExtension>
<!–
Update the UI definitions below with the controls and the command actions
that you want to enable for the custom action.
–>
<CommandUIDefinitions>
<CommandUIDefinition Location="Ribbon.Library.Actions.Controls._children">
<Button Id="Ribbon.Library.Actions.LibraryDetailsCustomActionButton"
Alt="Examine Library Details"
Sequence="100"
Command="Invoke_LibraryDetailsCustomActionButtonRequest"
LabelText="Examine Library Details"
TemplateAlias="o1"
Image32by32="_layouts/15/images/placeholder32x32.png"
Image16by16="_layouts/15/images/placeholder16x16.png" />
</CommandUIDefinition>
</CommandUIDefinitions>
<CommandUIHandlers>
<CommandUIHandler Command="Invoke_RibbonCustomAction1ButtonRequest"
CommandAction="LibraryManager\Pages\LibraryDetails.aspx"/>
</CommandUIHandlers>
</CommandUIExtension >
</CustomAction>
</Elements>
[/sourcecode]
Deploying the App that contains the custom action markup shown above creates a new button in the ribbon of each Host Web Document Library. By default, each button looks like the following:
There are a few attributes in the previous XML that I’m going to repeatedly come back to, so it’s worth taking a closer look at each one’s purpose and associated value(s):
Image32by32 and Image16by16 for the <Button /> element. These two attributes specify the images that are used when rendering the custom action button on the ribbon. By default, they point to an orange dot placeholder image that lives in the farm’s _layouts folder.
CommandAction for the <CommandUIHandler /> element. In its simplest form, this is the URL of the page to which the user is redirected upon pressing the custom ribbon button.
The Problem with the Default CommandAction
When a user clicks on a custom ribbon button in one of the Host Web document libraries, the goal is to send them over to a page in the App Web where the custom action can be processed. Unfortunately, the default CommandAction isn’t set up in a way that permits this.
In fact, attempting to deploy the solution to Office 365 with this default CommandAction results in failure; the App package doesn’t pass validation.
To understand why the failure occurs, it’s important to remember the isolation that exists between the Host Web and the App Web. To illustrate how the Host Web and App Web are different from simply a hostname perspective, consider the project I’ve been working on as an example:
Notice that although the /sites/dev2 relative path portion is the same for both the Host Web and App Web URLs, the hostname portion of each URL is different. This is by design, and it helps to enforce the logical separation between the Host Web and App Web – even though the App Web technically resides within the Host Web.
Looking again at the default CommandAction attribute reveals that its value is just an ASPX page that is identified with a relative URL. Rather than pointing to where we want it to point …
And this is exactly what should happen. After all, the custom action is launched from within the Host Web, so a relative path specification should resolve to a location in the Host Web – not the location we actually want to target in the App Web.
Fixing the CommandAction
Thankfully, it isn’t a major undertaking to correct the CommandAction attribute value so that it points to the App Web instead of the Host Web. If you’ve worked with SharePoint at all in the past, then you may know that the key to making everything work (in this situation) is the judicious use of tokens.
What are tokens? In this case, tokens are specific string sequences that SharePoint parses at run-time and replaces with a value based on the run-time environment, action that was performed, associated list, or some other context-sensitive value that isn’t known at design-time.
To illustrate how this works, consider the default CommandAction attribute:
Modifying the attribute as follows changes the destination URL of the button so that the user is redirected to the desired page in the App Web rather than the Host Web:
Since tokens are the key to inserting context-dependent values at run-time, you’d think they’d have been implemented and usable anywhere a developer needs to cross the Host Web / App Web divide.
Apparently not. To be more specific (and fair), I should instead say “not consistently.”
Since this blog post is about image limitations with custom ribbon buttons, you can probably guess where I’m headed with all of this. So, let’s take a look at the Image16by16 and Image32by32 attributes.
By default, the Image16x16 and Image32by32 attributes point to a location in the _layouts folder for the farm. Each attribute value references an image that is nothing more than a little round orange dot:
Much like the CustomAction attribute, it stands to reason that developers would want to replace the placeholder image attribute values with URLs of their choosing. In my case, I wanted to use a set of images I was deploying with the rest of the application assets in my App Web. So, I updated my image attributes to look like the following:
I deployed my App to my Office 365 Preview tenant, watched my browser launch into my App Web, hopped back to the Host Web, navigated to a document library,and looked at the toolbar. I was not happy by what I saw (on the left).
The image I had specified for use by the button wasn’t being used. All I had was a broken image link.
Examining the properties for the broken image quickly confirmed my fear: the ~appWebUrl token was not being processed for either of the Image32by32 or Image16by16 attributes. The token was being output directly into the image references.
I tried changing the image attributes to reference the App Web a couple of different ways (and with a couple of different tokens), but none of them seemed to work.
To see if I was losing my mind, I decided to try a little experiment. Although I knew it wouldn’t solve my particular problem, I decided to try using the ~site token just to see if it would be parsed properly. Lo and behold, it was parsed and replaced. ~site worked. So, ~site worked … but ~appWebUrl didn’t?
That didn’t make any sense. If it isn’t possible to use the ~appWebUrl token, how are developers supposed to reference custom images for the buttons they deploy in their Apps? Without the ~appWebUrl, there’s no practical way to reference an item in the App Web from the Host Web.
Token Forensics
When I find myself in situations where I’m holding results that don’t make sense, I can’t help myself: I pull out Reflector and start poking around for clues inside SharePoint’s plumbing. If I dig really hard, sometimes I find answers to my questions.
After some poking around with Reflector, I discovered that the “journey to enlightenment” (in this case) started with the RegisterCommandUIWithRibbon method on the SPCustomActionElement type. It is in this method that the Image16by16 and Image32by32 attributes are read-in from the XML file in which they are defined. Before assignment for use, they’re passed through a couple of methods that carry out token parsing:
ReplaceUrlTokens on the SPCustomActionElement type
UrlFromPrefixedUrlCore on the the SPUtility type
Although these methods together are capable of recognizing and replacing many different token types (including some I hadn’t seen listed in existing documentation; e.g., ~siteCollectionLayouts), none of the new SharePoint 2013 tokens, like the ~appWebUrl and ~remoteWebUrl~remoteAppUrl tokens, appear in these methods.
Interestingly enough, I didn’t see any noteworthy differences between the path of execution for processing image attributes and the sequence of calls through which CommandAction attributes are handled in the RegisterCommandUIExtension method of the SPRibbon type. The RegisterCommandUIExtension method eventually “punches down” to the ReplaceUrlTokens and UrlFromPrefixedUrlCore methods, as well.
The differences I was seeing in how tokens were handled between the CommandAction and Image32by32/Image16by16 attributes had to be originating somewhere else – not in the processing of the custom action XML.
Deployment Modifications
After some more digging in Reflector to determine where the ~appWebUrl actually showed-up and was being processed, I came across evidence suggesting that “something special” was happening on App deployment rather than at run-time. The ~appWebUrl token was being processed as part of a BuildTokenMap call in the SPAppInstance type; looking at the call chain for the BuildTokenMap method revealed that it was getting called during some App deployment operations processing.
If changes were taking place on App deployment, then I had a hunch I might find what I was looking for in the content database housing the Host Web to which my App was being deployed. After all, Apps get deployed to App Webs that reside within a Host Web, and Host Webs live in content databases … so, all of the pieces of my App had to exist (in some form) in the content database.
I fired-up Visual Studio, stopped deploying to Office 365, and started deploying my App to a site collection on my local SharePoint 2013 VM farm. Once my App was deployed, I launched SQL Management Studio on the SQL Server housing the SharePoint databases and began poking around inside the content database where the target site collection was located.
Brief aside: standard rules still apply in SharePoint 2013, so I’ll mention them here for those who may not know them. Don’t poke around inside content databases (or any other databases) in live SharePoint environments you care about. As with previous versions, querying and working against live databases may hurt performance and lead to bigger problems. If you want to play with the contents of a SharePoint database, either create a SQL snapshot of it (and work against the snapshot) or mount a backup copy of the database in a test environment.
I wasn’t sure what I was looking for, so I quickly examined the contents of each table in the content database. I hit paydirt when I opened-up the CustomActions table. It had a single row, and the Properties field of that row contained some XML that looked an awful lot like the Elements.xml which defined my custom action:
[sourcecode language=”XML” autolinks=”false”]
<?xml version="1.0" encoding="utf-16"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
<CustomAction Title="Invoke ‘LibraryDetailsCustomAction’ action" Id="4f835c73-a3ab-4671-b142-83304da0639f.LibraryDetailsCustomAction" Location="CommandUI.Ribbon" RegistrationId="101" RegistrationType="List" Sequence="10001">
<CommandUIExtension xmlns="http://schemas.microsoft.com/sharepoint/">
<!–
Update the UI definitions below with the controls and the command actions
that you want to enable for the custom action.
–>
<CommandUIDefinitions>
<CommandUIDefinition Location="Ribbon.Library.Actions.Controls._children">
<Button Id="Ribbon.Library.Actions.LibraryDetailsCustomActionButton" Alt="Examine Library Details" Sequence="100" Command="Invoke_LibraryDetailsCustomActionButtonRequest" LabelText="Examine Library Details" Image16by16="~site/Images/sharepoint-library-analyzer_16x16-a.png" Image32by32="~appWebUrl/Images/sharepoint-library-analyzer_32x32-a.png" TemplateAlias="o1"/>
</CommandUIDefinition>
</CommandUIDefinitions>
<CommandUIHandlers>
<CommandUIHandler Command="Invoke_LibraryDetailsCustomActionButtonRequest" CommandAction="javascript:LaunchApp(‘709d9f25-bb39-4e6a-97d5-6e1d7c855f38’, ‘i:0i.t|ms.sp.int|a441fa2c-8c5f-4152-9085-3930239ab21b@9db0b916-0dd6-4d6c-be49-41f72f5dfc02’, ‘~appWebUrl\u002fPages\u002fLibraryDetails.aspx?ListID={ListId}\u0026SiteUrl={SiteUrl}’, null);"/>
</CommandUIHandlers>
</CommandUIExtension>
</CustomAction>
</Elements>
[/sourcecode]
There were some differences, though, between the Elements.xml I had defined earlier and what actually appeared in the Properties field. I narrowed my focus to the differences that existed between the non-working Image32by32/Image16by16 attributes
As suspected, some deployment-time processing had been performed on the CommandAction attribute but not on the image attributes. The CommandAction still contained an ~appWebUrl token, but it was wrapped as part of a parameter call to a LaunchApp JavaScript function that appeared to be handled (or rather, executed) from a client-side browser.
Jumping into my App in Internet Explorer and opening IE’s debugging tools via <F12>, I did a search for the LaunchApp function within the referenced scripts and found it in the core.js library/script. Examining the LaunchApp function revealed that it called the LaunchAppInternal function; LaunchAppInternal, in turn, called back to the SharePoint server’s /_layouts/15/appredirect.aspx page with the parameters that were supplied to the original LaunchApp method – including the URL with the ~appWebUrl token.
To complete the journey, I opened up the Microsoft.SharePoint.ApplicationPages.dll assembly back on the server and dug into the AppRedirectPage class that provides the code-behind support for the AppRedirect.aspx page. When the AppRedirect.aspx page is loaded, control passes to the page’s OnLoad event and then to the HandleRequest method. HandleRequest then uses the ReplaceAppTokensAndFixLaunchUrl method of the SPTenantAppUtils class to process tokens.
The ReplaceAppTokensAndFixLaunchUrl method is noteworthy because it includes parsing and replacement support for the ~appWebUrl token, ~remoteWebUrl~remoteAppUrl token, and other tokens that were introduced with SharePoint 2013. The deployment-time processing that is performed on the CommandAction attribute is what ultimately wires-up the CommandAction to the ReplaceAppTokensAndFixLaunchUrl method. The Image32by32 and Image16by16 attributes don’t get this treatment, and so the new 2013 tokens (like ~appWebUrl) can’t be used by these attributes.
What About the Image32by32 and Image16by16 Attributes?
Now that some of the key differences in processing between the CommandAction attribute and image attributes have been identified, let me jump back to the original problem. Is there anything that can be done with the Image32by32 and Image16by16 attributes that are specified in a custom action to get them to reference assets that exist in the App Web? Since tokens like ~appWebUrl(and ~remoteWebUrl for all you Autohosted and Provider-hosted application builders) aren’t parsed and processed, are there alternatives?
My response is a somewhat wishy-washy “doubtful.” In my estimation, you’d need to hack SharePoint with something like a javascript: tag for an image attribute (which, interestingly enough, doesn’t appear to be expressly blocked), find some way to obtain the App Web URL base, formulate the proper path to the image, and more. If it could be done, you’d be gaming SharePoint … and I could easily see a cumulative update or service pack breaking this type of elaborate work-around.
The safest and most pragmatic way to handle this situation, it seems, is to use absolute URLs for the desired image resources and forget about deploying them to the App Web altogether. For example, I placed the images I was trying to use on the ribbon buttons here on my blog and referenced them as follows:
I had some initial concerns that I might inadvertently bump into some security boundaries, such as those that sometimes arise when an asset is referenced via HTTP from a site that is being served up under HTTPS. This didn’t prove to be the case, however. I tested the use of absolute URLs in both my development VM environment (served up under HTTP) and through one of my Office 365 Preview site collections (accessed via HTTPS), and no browser security warnings popped up. The target image appeared on the custom button as desired (shown on the left) in both cases.
Although the use of absolute URLs will work in many cases, I have to admit that I’m still not a big fan of this approach – especially for SharePoint-hosted apps like the one I’ve been working on. Even though Office 365 entails an “always connected” scenario, I can easily envision on-premises deployment environments that are taken offline some or all of the time. I can also see (and have seen in the past) SharePoint environments where unfettered Internet access is the exception rather than the rule.
In these environments, users won’t see image buttons at all – just blank placeholders or broken image links. After all, without Internet access there is no way to resolve and download the referenced button images.
Wrapping It Up
At some point in the future, I hope that Microsoft considers extending token parsing for URL-based attributes like Image32by32 and Image16by16 to include the ~appWebUrl, ~remoteWebUrl, and other new tokens used by the SharePoint 2013 App Model. In the meantime, though, you should probably consider getting an easily accessible online location (SkyDrive, Dropbox, a blog, etc.) for images and other similar assets if you’re building apps under the new SharePoint 2013 App Model and intend to use custom actions.
Update (1/27/2013)
I need to issue a couple of updates and clarifications. First, I need to be very clear and state that SharePoint-hosted apps were the focus of this post. In a SharePoint-hosted app, what I’ve written is correct: there is no processing of “new” 2013 tokens (like ~appWebUrl and ~remoteAppUrl) for the Image32by32 and Image16by16 attributes. Interestingly enough, though, there does appear to be processing of the ~remoteAppUrl in the Image32by32 and Image16by16 attributes specifically for the other application types such as provider-hosted apps and autohosted apps. Jamie Rance mentioned this in a comment (below), and I verified it with an autohosted app that I quickly spun-up.
I double-checked to see if the ~remoteAppUrl token would even be recognized/processed (despite the lack of a remote web component) for SharePoint-hosted apps, and it is not … nor is ~appWebUrl token processed for autohosted apps. The selective implementation of only the ~remoteAppUrl token for certain app types has me baffled; I hope that we’ll eventually see some clarification or changes. If you’re building provider-hosted or autohosted apps, though, this does give you a way to redirect image requests to your remote web application rather than an absolute endpoint. Thank you, Jamie, for the information!
And now for some good news that for SharePoint-hosted app creators. Prior to writing this post, I had posted a question about the tokens over in the SharePoint Exchange forums. At the time I wrote this post, there hadn’t been any activity to suggest that a solution or workaround existed. F. Aquino recently supplied an incredibly creative answer, though, that involves using a data URI to Base64-encode the images and package them directly into the Image32by32 and Image16by16 attributes themselves! Although this means that some image pre-processing will be required to package images, it gets around the requirement of being “always-connected.” This is an awesome technique, and I’ll certainly be adding it to my arsenal. Thank you, F. Aquino!
After working with the Office 365 Preview over the last several months, I shifted my thoughts on SharePoint in the Cloud. In this post, I share my thoughts and “revelations” about what’s coming with SharePoint 2013, Office 365, and usage of SharePoint in the Cloud.
It was about a year and a half ago when someone dialed-up the volume on “The SharePoint Cloud Message” in my world. It’s not that I hadn’t heard people talking about SharePoint in the Cloud prior to that; I guess it’s just that I started listening more closely because Microsoft was turning into one of the Cloud’s most vocal proponents.
My relationship with Microsoft and Microsoft technologies goes back to the days of MS-DOS. As a result, I’ve always seen Microsoft as a company that was primarily interested in one thing: selling software. I worked for a Microsoft managed systems integration (SI) partner – Cardinal Solutions Group – for several years. During my years with Cardinal, my goal was to help others who had purchased Microsoft software make use of that software. In many cases, customer leads came from Microsoft either directly or indirectly. Microsoft sold the software, and we setup/customized/serviced/configured that software based on what a customer was trying to accomplish. It was a symbiotic relationship, and it was pretty easy for me to grasp.
Then the whole “Cloud thing” started. Cloud-based SharePoint and other Azure-branded services seemed a somewhat confusing move for Microsoft at first – at least to me. Even before Office 365, Microsoft offered hosted SharePoint through BPOS – or the Business Productivity Online Suite. At the time when BPOS was first released, I viewed it as something of a niche market for Microsoft. I had plenty of friends who worked at places like Rackspace and Fpweb.net, so the part I found unusual wasn’t really that “someone else” was hosting SharePoint and focusing on it as a service. The fact that Microsoft itself was getting serious about SharePoint and other services was the eyebrow raiser.
For Microsoft, it wasn’t just about selling software anymore.
The Biggest Hurdle
Of course, when Microsoft wants to succeed at something, they invest considerable planning and resources in it. Since Microsoft is essentially betting the farm (pun intended) on Office 365 and SharePoint in the Cloud, they’re pushing it very hard on multiple fronts. Redmond’s marketing machine has been talking Office 365 frequently and loudly for at least the last year. With each new release, developer tools like Visual Studio get more Cloud-friendly. Partners have incentives to get customers onto Office 365 and Azure services. Competitive price points make it difficult to ignore Microsoft’s Cloud offerings. For me (and I’m sure for many of you), it’s a lot to process.
I’d also be remiss if I didn’t say that I think Office 365 has a very compelling value proposition, even without SharePoint. SharePoint itself is a complex platform, though, and many organizations struggle with administrative needs like data protection, performance optimization, high availability, and basic day-to-day management. The idea of turning these concerns over to someone else (or some other entity) who better-understands them makes sense to me.
After working with SharePoint 2013 for several months now, I can easily say that the platform isn’t getting any easier. SharePoint 2013 has quite a few more “moving parts” relative to SharePoint 2010, just as SharePoint 2010 demonstrated itself to be significantly more complex than SharePoint 2007.
Despite the compelling nature of Office 365, I always seemed to come back around to fixate on one thought. This thought constantly reverberated through my head anytime “SharePoint in the Cloud” became a topic of conversation:
Most companies using SharePoint have made a significant investments in hardware, software, personnel, and services to get SharePoint up-and-running. They aren’t going to simply “dump” those on-premises investments and go to the Cloud tomorrow. The Cloud will happen, but it’s going to take longer than Microsoft thinks.
In discussions with many friends and respected professionals in the SharePoint community, I knew that I wasn’t completely alone in my way of thinking. In the conversations I’d had, there was almost always agreement that a shift to the Cloud and Cloud-based services would happen over time. The greatest debate seemed to be over whether it would happen next year or if it would take the next half a decade.
Breakthrough
I’d say my “breakthrough moment” came after I started playing with the Office 365 Preview more extensively a few months back. I initially set up a preview tenant to familiarize myself with what was coming, how SharePoint 2013 would be exposed, how to configure Office 365 tenants, etc. The more I played with the tenant, the more I thought about how truly useful Office 365 could be, particularly for non-enterprise customers, home users, and others who didn’t fit into SharePoint’s “big deployment picture” previously.
That’s when the pieces started to click into place for me. All along I had been thinking about Office 365 and Cloud-based SharePoint deployments along the lines of the bar chart seen above and to the right. Numbers and proportions are all relative, but the key concept I’m trying to convey with the chart is this: for some reason, I had always thought that the proponents of Cloud-based SharePoint were suggesting that Cloud adoption would come at the cost of on-premises deployments; i.e., on-premises users would “convert” to the Cloud. If Cloud-based deployments grew, that meant that on-premises deployments had to shrink. In short: I was inadvertently assuming that the overall number of SharePoint deployments had hit saturation and was remaining static.
I don’t think that way at all anymore.
After I’d done some playing with my first tenant, it wasn’t long before I was setting up another two Office 365 tenants for other side projects. In conversations with friends in the SharePoint community, I was discovering that “everyone” was setting up tenants for their families, for their spouse’s business, etc. In almost all cases where tenants were being setup, the use cases were ones that didn’t align with traditional enterprise-scale on-premises SharePoint deployment and usage. In fact, the use cases were typically the types of things that would eventually find a home on Google Apps or its equivalent because Microsoft (previously) had nothing strong to offer in that space.
The more I think about it, the more I feel that Office 365 growth – once the new 2013 Preview goes live – will be aggressive and look something more like what I’ve charted above and to the left. While Office 365 might replace some on-premises deployments, particularly for smaller organizations, I don’t see that as its primary market (initially) or its strong suit. The greatest degree of Office 365 traction is going to be obtained with users who need a Google Apps-like solution but for whom buying the required infrastructure and expertise for Exchange, SharePoint, etc., is cost-prohibitive.
So, I stopped thinking “replacement” and started thinking “complement.” That’s my assessment and working outlook for the Office 365 (Preview) right now.
Why Not Everyone?
I’m sure that plenty of folks who’ve believed in “Cloud Power” since Day One probably think that I’m still being too conservative in my outlook for SharePoint on Office 365, and that may be true. However, I still see plenty of concerns that are near-and-dear to most enterprise and larger business customers, and I believe that they will be Cloud adoption blockers until they’re addressed directly and decisively. Here are just a few that come to mind.
2. What about disasters? Many people point to the Cloud as a solution for business continuity and disaster recovery (DR) concerns. The Cloud can certainly help, but I’ll tell you (somewhat authoritatively) that the Cloud doesn’t make DR concerns “go away” – especially for SharePoint. For one thing, you’re locked into your provider’s terms of service; if you need more aggressive RPO and RTO windows, then you need to be looking elsewhere. Even Cloud data centers themselves go down; what’s your plan then?
3. Can I leave my provider? Everyone is quick to talk about moving to the Cloud, and many companies are happy to talk about migration strategies. What if you want to leave or change providers, though? Do those migration strategies work? What do you lose? How long would it even take? These may not seem like important questions now, but they will become increasingly more important as Cloud adoption grows and more companies get in on the action. It stands to reason that some portion of those companies will fail, close-up shop, be bought, etc. When that happens, what do you do … and what happens to your SharePoint?
Wrap Up
Of course, my perspective on Office 365 uptake in the next several years could be completely off-the-mark. After all, I don’t really have any numbers to back up my hypotheses. They’re just my opinions, but they are in-line with my gut feel.
I’m a big fan of leveraging caching to improve performance. If you look over my blog, you’ll find quite a few articles that cover things like implementing BLOB caching within SharePoint, working with the Object Cache, extending your own code with caching options, and more. And most of those posts were written in a time when the on-premises SharePoint farm was king.
Update (Evening 7/23/2015)
So, if you found this post and it’s helping you to realize that you’re not going insane (at least not because of public site branding changes you didn’t make), I’ll feel that my job is done.
My adventures in SharePoint 2013 App Model Land have been going pretty well, but I recently encountered a limitation that left me sort of scratching my head.
Thankfully, it isn’t a major undertaking to correct the CommandAction attribute value so that it points to the App Web instead of the Host Web. If you’ve worked with SharePoint at all in the past, then you may know that the key to making everything work (in this situation) is the judicious use of tokens.
Since tokens are the key to inserting context-dependent values at run-time, you’d think they’d have been implemented and usable anywhere a developer needs to cross the Host Web / App Web divide.
Now that some of the key differences in processing between the CommandAction attribute and image attributes have been identified, let me jump back to the original problem. Is there anything that can be done with the Image32by32 and Image16by16 attributes that are specified in a custom action to get them to reference assets that exist in the App Web? Since tokens like ~appWebUrl 
It was about a year and a half ago when someone dialed-up the volume on “The SharePoint Cloud Message” in my world. It’s not that I hadn’t heard people talking about SharePoint in the Cloud prior to that; I guess it’s just that I started listening more closely because Microsoft was turning into one of the Cloud’s most vocal proponents. 
Of course, when Microsoft wants to succeed at something, they invest considerable planning and resources in it. Since Microsoft is essentially betting the farm (pun intended) on Office 365 and SharePoint in the Cloud, they’re pushing it very hard on multiple fronts. Redmond’s marketing machine has been talking Office 365 frequently and loudly for at least the last year. With each new release, developer tools like Visual Studio get more Cloud-friendly. Partners have incentives to get customers onto Office 365 and Azure services. Competitive price points make it difficult to ignore Microsoft’s Cloud offerings. For me (and I’m sure for many of you), it’s a lot to process.
![http://sharepointinterface.com/wp-content/uploads/2013/01/sharepoint-library-analyzer_16x16-a.png"](http://sharepointinterface.com/wp-content/uploads/2013/01/sharepoint-library-analyzer_16x16-a.png"){kind=link}
![http://sharepointinterface.com/wp-content/uploads/2013/01/sharepoint-library-analyzer_32x32-a.png"](http://sharepointinterface.com/wp-content/uploads/2013/01/sharepoint-library-analyzer_32x32-a.png"){kind=link}