This white paper discusses enabling and using My Tasks for synchronization, both in SharePoint Online and SharePoint 2013 on-premises environments, and it also covers new integration functionality with Exchange Server 2013. This white paper covers configuration and administrative usage, as well as end-user experience, from both the SharePoint and Exchange Server perspectives. Download White Paper Here
Marc Boyer Microsoft Corporation April 2013 Applies to: SharePoint Server 2013, SharePoint Online, Exchange Server 2013, Project Server 2013 Summary: This white paper discusses enabling and using My Tasks for synchronization, both in SharePoint Online and SharePoint 2013 on-premises environments, and it also covers new integration functionality with Exchange Server 2013. This white paper covers configuration and administrative usage, as well as end-user experience, from both the SharePoint and Exchange Server perspectives. Table of Contents The working view of the My Tasks list in SharePoint Server 3 Executive summary. 3 Prerequisites and configuration. 3 SKUs and prerequisites. 3 Turn on Work Management through the Configuration Wizard. 4 Turn on Work Management through the Shared Service UI 5 Cross-farm topologies. 6 Using matching proxies. 6 Supported types of SharePoint tasks. 7 Accessing the My Tasks feature. 7 How the aggregator works. 7 SharePoint and Project Server Provider Refresh Workflow. 7 Refresh the MyTasks page. 8 Reliance on the Search Indexer 9 What could go wrong?. 10 Exchange integration. 10 Executive summary. 10 Prerequisites and configuration. 11 Anti-starvation logic. 11 How long should it take between each sync?. 11 How to disable SharePoint Server and Exchange Server integration. 12 What is the difference between the timer job and MyTasks behavior?. 12 Legacy features. 12 Legacy sync to Outlook SharePoint feature. 13 Opting in. 13 How to opt-in from My Tasks. 13 Other entry points for opting in. 14 Opt in from a SharePoint Task List 14 Opt in from a Project Web Application Task List 14 The Exchange Sync timer job. 15 The Exchange Sync flow. 15 Property mapping. 18 Expected behavior 19 Timer job adjustments. 22 Opt-in errors. 23
The working view of the My Tasks list in SharePoint Server Executive summary In SharePoint Server, the Newsfeed Site’s My Tasks list aggregates and shows all the SharePoint and Project Server tasks assigned to the logged-in user. It can also be used to create personal or public tasks. This feature is available for SharePoint online and SharePoint Server on-premises deployments and is included in the Newsfeeds and Tasks pages.
Prerequisites and configuration
SKUs and prerequisitesServer productsCloud servicesSharePoint Server 2013 Standard or Enterprise Project Server (for Project Server tasks)SharePoint Online Plan P, Plan 1, or Plan 2 Project Online Project Portfolio Management (for Project Server tasks) The Newsfeed My Tasks service requires that the following Shared Service Applications are provisioned and running: • Work Management service application (WMA). • User Profile service application (UPA, required to get SharePoint tasks). o In order to make this work, UPA needs to be configured to import from Active Directory. o http://technet.microsoft.com/en-us/library/jj219646%28office.15%29.aspx • SharePoint Personal/Newsfeed site. • Project Service Application (This service is necessary if you want to aggregate Project Server tasks. Without this service, only SharePoint site and personal tasks would be aggregated). • Search Service Application (This service should be enabled if you want to aggregate SharePoint tasks. Without this service enabled, only Project and personal tasks would be aggregated). Note By default, search isn’t configured to automatically run. An admin must configure Continuous Crawls or Incremental crawls to have this functionality. The admin should configure any crawling as per Plan crawling and federation in SharePoint Server 2013. This does mean that tasks dependent on search, such as custom task list tasks, only appear per the search crawl schedule.
Turn on Work Management through the Configuration Wizard 1. Go to the SharePoint Central Administration website page. 2. Click Configuration Wizards on the Quick Launch. 3. Launch the farm configuration wizard. 4. Choose whether you want to opt into the Customer Experience Improvement Program (if offered as an option), and then click OK. 5. Click the Start the Wizard button. 6. Scroll to the bottom of the list of services, check the Work Management Service check box, and then click Next. (Note: you can also turn on Search Service from here and verify that User Profile Service is on, just check the boxes.) 7. Let the wizard finish. You can skip the site collection creation step, if you want.
Turn on Work Management through the Shared Service UI 1. In Central Administration, under Application Management, click Manage Service Applications. 2. In the Service Applications ribbon, click New and choose Work Management Service Application. 3. Enter a name for the service. If you want to configure this service with least privileges, create a new application pool for the service. Click OK. You should now see the Work Management Service Application and WM Service Proxy in the services applications list. Note In case of Least Privileges, it’s helpful to note the relationship with Project Server. Project Server resides on top of WMA, and so in order for the WMA application pool account to be trusted by Project Server automatically, it’s best to provision the PWA sites AFTER you create the WM service application. If PWA instances exist already, then the administrator must manually add the WM account to the Project Server databases’ PSDataAccess role. For more information on Least Privilege planning, see: http://technet.microsoft.com/en-us/library/cc263445.aspx 4. Go back to the Central Administration home page and under System Settings, click Manage Services on Server. 5. Scroll down to the Work Management Service and make sure it's Started. If not, click Start to start it.
Cross-farm topologies Cross farm aggregation is not supported. The My Tasks page will not report and aggregate tasks from another farm from the same domain or from a different domain, regardless of the existing trust between the domains and farms. If you have Farm 1 and Farm 2, and they are both on the same domain, task aggregation will only work on the farm that has the work management service installed. If you have a single farm, but the web application and site collections it contains are on separate domains (For example, if they’re using host headers), then aggregation will work).
Using matching proxies Work Management uses two different proxy groups: 1) The My Tasks user interface uses the proxy group associated with the web application where the My Tasks user interface is running. 2) The Exchange Sync timer job uses the proxy group associated with the Work Management Service Application itself. If you wish to configure multiple proxies for the Work Management feature set, make sure the proxies for the My Tasks web app and the Work Management Service Appplication match. Note: In theory, the two groups could be different, but the key is that all our dependencies need to be part of both: UPA, Search, and possibly Project Service. This would be a more complicated setup, and it is not discussed further in this white paper.
Supported types of SharePoint tasks WMA aggregates tasks from SharePoint lists created in SharePoint Server (internally called TasksWithTimelineAndHierarchy-171) plus upgraded SharePoint lists from earlier versions of SharePoint (internally called Task-107 or GanttTasks-150). Additionally, WMA aggregates all task items (whether or not in a SharePoint list) created in a library that supports the task content type. For more information, see: Turn on support for multiple content types in a list or library. Note: The Task content type should be marked as the Default Content Type for your custom list in order for the tasks to be aggregated by the service.
Accessing the My Tasks feature If the server is set with all the prerequisites, then the user can access the My Tasks page by navigating to his or her Personal page and then clicking the Task link in the left column.
How the aggregator works The aggregator is the piece of code responsible for gathering all the tasks assigned to the user in SharePoint and Project (or any provider it’s extended to), in order for them to be shown in the My Tasks page.
SharePoint and Project Server Provider Refresh Workflow
Refresh the MyTasks page Refreshing or accessing the My Tasks page launches the aggregator code, if the last time the aggregator ran was more than five minutes prior. This artificial delay is put in place to preserve performance. There's no requirement to refresh the page after inline edits, as inline changes are instantly replicated on the original provider. The five-minute delay doesn’t apply to them. This value can be changed from on-premises deployments by altering the property on the WMA service application object by using Windows PowerShell. Changing this value for SharePoint Online is theoretically possible, but it will affect all tenants (customer subscriptions for using SharePoint Online) in the SharePoint online environment. PropertyDefault ValueCommentsminimumTimeBetweenProviderRefreshes00:05:00This value specifies the minimum amount of time between refreshes for a provider for a given user. There cannot be a refresh of data if this value is not met; all refresh operations will be null before that.
Reliance on the Search Indexer To find a new SharePoint Tasks list that contains a task assigned to a user, the aggregator sometimes has to rely on the Search indexer. To preserve Search’s performance, the aggregator only tries to look for new task lists every three hours, and this, combined with the fact that Search may only index a few times a day, could delay the aggregation of a freshly created tasks list. Keep these points in mind:
Once a task list is found to contain at least one task assigned to the user, any new task on this task list will be found during the next refresh, as the task list location is in memory already.
Most of the time, even new task lists show up on the next refresh. This is because the aggregator is also listening for the new SharePoint events (hints) to be alerted when a new task is assigned to the user. But this doesn’t work in some cases, which explains why we still need to use Search.
One case where the hint will not be triggered: A user is assigned a task in a SharePoint site that he or she doesn’t have access to. Then the user is given access, but this does not trigger a catchable hint.
Also, old hints (hints that are more than 48 hours old are not guaranteed to be used by the aggregator) are deleted to prevent getting a big list of unused hints when a crawl isn’t run for a period of time.
Tasks from legacy SharePoint task lists will not trigger hints.
Project tasks, even for new Project and PWA URLs, are cached for 1-2 hours, so two hours will be the maximum time it would take for a task to be retrieved.Note: You can see what’s currently in the hint path by running this Windows PowerShell cmldlet: $w=Get-SPWeb <My Site URL> (I.e. http://contoso/my/personal/kberg) $w.Properties["wmahint"]
What could go wrong?
The page simply doesn't appear. This is likely not a My Tasks problem, but more a SharePoint or Personal Site provisioning issue. You can contact support to start an investigation.
Even after the Search indexer runs, some tasks don’t show up on the My Tasks page. You can contact support to start an investigation.
Executive summary Integration between SharePoint Server and Exchange Server completes the My Task aggregation by adding Exchange Server and Outlook tasks to the My Tasks view, making it the one place to go to see all your tasks. Additionally, now all the user’s SharePoint, Project and My Task personal tasks will be available in Exchange Server, where the user is able to edit them. Exchange Server is kept in sync with SharePoint and Project via a timer job that periodically (multiple times per hour) pings all servers to gather and replicate changes. The timer job runs on SharePoint and wakes up every minute, but a user might not be synced every time, depending on the freshness of his or her tasks. Typically, sync should occur every five to 20 minutes to ping all the different providers (including Exchange Server) and look for changes, then process these changes if needed. It is always SharePoint that is calling Exchange Server, which makes the configuration (below) easier, because only one trust needs to be established. The amount of opted-in users and the amount of tenants (company subscriptions for SharePoint Online) are the factors that decide how long it takes between two consecutive syncs for a user. For on-premises installations, the time can be changed by adjusting the default Exchange sync values, as explained later in this white paper. Changing this value for SharePoint Online is theoretically possible, but it will affect all tenants.
Prerequisites and configuration Exchange task sync requires that user profile synchronization be configured in the farm. For information about configuring user profile synchronization, see Plan user profiles and identities (SharePoint Server 2013), and Manage user profile synchronizations in SharePoint 2013 Server. Note: Exchange Task Sync requires Exchange Server 2013. Secure Sockets Layer (SSL) is a requirement for web applications that are deployed in scenarios that support server-to-server authentication and app authentication. As a prerequisite for configuring Task Synchronization, the computer that is running SharePoint Server must have SSL configured. For more information, see Create claims-based web applications in SharePoint 2013 and follow the steps for creating an SSL site collection and server certificate. For related information, see Configure Exchange task synchronization in SharePoint Server 2013.
Anti-starvation logic Anti-starvation logic is only needed in online SharePoint deployments, and it cannot be modified Because the online farm contains a very large number of tenants, and because we don’t have access to a global list of opted-in users (we have to look in every “enabled” tenant for opted-in users), it is necessary to have some logic that ensures that all users are synced before a particular user get synced again. This logic ensures that online farms are fair for everyone, and that no one suffers because a large tenant contains a very large number of users. The anti-starvation logic is as follows: At any given time and for any given tenant, the user who’s gone the longest without being synced has the highest priority. This data is used to order tenant need of sync priority (oldest is higher priority) and then users in a tenant also by sync priority. In short, the user who needs to be synced the most will be synced the soonest, regardless of his or her tenant.
How long should it take between each sync? First the timer job runs every minute to select a list of users to sync, following the anti-starvation logic. After this list is created, it asks the WMA service to sync these users. The maximum number of users in this list is governed by NumberOfUsersPerEwsSyncBatch (by default ,it’s 100). The list of users to be synced is processed for 45 seconds (or all the list), then it stops, and the system now waits for the next timer job to wake up. Also, each user can’t be synced more often than every five minutes (governed by minimumTimeBetweenProviderRefreshes). The unknown variable is the time it takes to sync one user. It depends on the load of the server, the number of tasks, and the delay induced by the connection to Exchange Server. By default, the EWS services tries to sync batches of 10 users at a time, and this value can be tweaked by using NumberOfUsersEwsSyncWillProcessAtOnce. This process means that, in very low opt-in usage, it is guaranteed that every user will be marked for syncing on every timer job wake-up, most of the time. This would equate to a sync every five minutes for everyone. With a larger number of users, this delay between syncs can be longer, but it is typically in the order of 20 minutes. This is easily adjustable on-premises by adjusting the properties or by vertically upgrading the servers. For online, it is not controllable by tenant admins, and the goal for online sync is to sync every 30 minutes or better.
How to disable SharePoint Server and Exchange Server integration Integration between SharePoint Server and Exchange Server can only be disabled on-premises. To disable it, you turn off the Farm Level Exchange Task Sync feature by going to Central Administration -> Manage farm features. This action deletes the timer job and disables the UI integration, bringing back the legacy sync behavior to Outlook.
What is the difference between the timer job and MyTasks behavior? They complete each other to ensure that all the data is always the freshest and that performance is maintained by not syncing too often or with unnecessary tasks. The Timer Job only cares about SharePoint and Project Tasks, its job is to check a few times per hour that all the changes happening in one place are deprecated to the other (SharePoint/Project Server to Exchange Server, or Exchange Server to SharePoint/Project Server). My Tasks, meanwhile, focus on personal tasks, which means that no update on a personal task will be synced unless the user refreshes My Tasks. Additionally, every personal task edit in My Tasks is synchronized to Exchange Server instantly.
Legacy sync to Outlook SharePoint feature The new "Sync to Outlook" feature replaces the legacy feature for all editions of SharePoint Server 2013 except for the Foundation SKU. This means that the feature will be installed on all non-Foundation deployments, regardless of the presence of Exchange Server 2013 in the topology. For the cases where Exchange Server 2013 is not going to be available, the administrator can disable the new Exchange Server integration feature to re-enable the legacy feature. With the feature disabled, behavior will revert to the legacy Outlook behavior from previous versions, which is found in the Foundation SKU as well.
Opting in On install, every user is enabled for Exchange Sync, but no one is opted-in. Actually, if no one ever opts-in within one tenant, then the feature will not be activated, and the timer job will pass over that tenant, which is done to improve performance. Every user has to opt-in individually. This can be done from any list in SharePoint or PWA. When a user opts-in, it automatically activates the feature for this tenant in the my site host and adds an entry to the list of users on the My Site host list. There is no way to turn the default to automatically opt-in all users. And it is not possible to do a bulk opt-in operation for all users without knowing everyone’s password. The list of opted-in users is maintained on the WMA service.
How to opt-in from My Tasks When a user navigates to the SharePoint’s My Task area, that user can find the Sync to Outlook button in the Tasks pane of the ribbon. After clicking that button, the opt-in dialog box is shown. Select the Sync tasks check box to opt in. If everything is working, the dialog box disappears after a short time. This is the sign that opting-in was successful. If an error occurred, the dialog box stays on-screen with an error message displayed. See below for descriptions of error cases.
Other entry points for opting in A user can opt-in from any tasks list in SharePoint or Project Server, listed below are the different entry points that can be used. Anyone of these entry points will have the exact same result: Opting-in the user to sync all assigned tasks into Exchange. Note: In each case, clicking the button will show the opt-in dialog described above, and the same behavior will happen.
Opt in from a SharePoint Task List When a user navigates to a SharePoint tasks list, he or she can find the Sync to Outlook button in the List pane of the ribbon.
Opt in from a Project Web Application Task List When a user navigates to a Project Web Application tasks list, he or she can find the Sync to Outlook button in the Tasks pane of the ribbon.
The Exchange Sync timer job The Exchange timer job is the only way to sync tasks between SharePoint/Project and Exchange Server. It has to run at all times in order to keep tasks in Sync. The timer job performance can be altered (see tweaking section) in on-premises deployments only. The job of the timer job is to select which user to sync next. It creates a batch of syncs and sends these users to the WMA code, which processes the sync following the flow detailed below.
The Timer Job asks for the sites which have the WMA feature, this is a way to get the list of all enabled tenants.
This happens once every 30 minutes (and can be tweaked using MinimumTimeBetweenEwsSyncSubscriptionSearches below)
The Timer Job has in-memory the list of all the tenants synced and the next user to process based on oldest one not processed, in priority order (the last to be synced is highest on the list to get synced).
The Timer Job takes the top 50 tenants (the ones with the oldest yet to be synced user timestamp) and processes them.
Syncing per tenant:
The code looks in the My Site host list for the oldest N users who need to be synced.
N is set to 100 users by default and can be configured using NumberOfUsersPerEwsSyncBatch.
For each user, we run Exchange Sync, then move to the next user. This goes on for 45 seconds, and then it stops.
So each time, we retrieve N users or whatever quantity could be retrieved in 45 seconds.
The timestamp of the next user to be synced is used to stamp the tenant.
The Exchange Sync flow When the users to be synced are selected by the code above, the sync is processed. It starts by gathering the list of changes in the providers as described in the My Task section ("How the aggregator works"). Then it follows the steps detailed below to get and process the changes from and to Exchange Server, in successive order. Note: In the vast majority of the cases, there is nothing to process. The aggregator gets no changes from the providers, and the list of changes from Exchange Server (before step 7 below) returns no change.
This step is for the cases where there is a new location returned by the provider. This location will have to be created in Exchange Server.
This step is for the cases where there is a new task from the provider to be sent to Exchange Server.
This step is for the cases where a task was deleted on the original provider. This deletion has to be ported to Exchange Server.
This step is for the cases where an existing task was updated on a provider. This code checks whether the folder exists in Exchange Server and recreates it, if needed.
This step is for the cases where a task was deleted in Exchange. Server. This deletion is ported to the original provider where the task will also get deleted.
When a task is created in Exchange, nothing happens on the original provider, as we do not allow the creation of public tasks from Exchange Server.
When a task is updated in Exchange Server, the updated properties are sent to the original provider.
This step is to solve the cases where a task was updated in both places at the same time. The latest entry wins, this is not per property, but instead per task.
This step is the continuation of step 4. Tasks updated in the provider are synchronized with Exchange Server.