What You Can Learn From the Microsoft Productivity Score ‘Incident’

Microsoft was recently in the news due to a particular product that comes along with Microsoft 365: the Microsoft Productivity Score. This is a score designed to keep track of actions within Microsoft 365, to help inform organisations of engagement throughout the platform. However, it has been embroiled in controversy as of late. Today we take a look at how the Productivity Score works, issues with the data, what exactly fired up the internet so much that it made news around the world, and whether the data will be useful for your organisation.

What is the Microsoft Productivity Score?

As per Microsoft’s documentation: 

“Productivity Score supports the journey to digital transformation with insights about how your organization uses Microsoft 365 and the technology experiences that support it.”

The new feature is designed to give solid metrics and insights into organisational data based on employees’ app usage behaviours.

This is a move towards monitoring metrics from a managerial standpoint, without custom IT solutions to inform managers such as calculating employees’ total emails sent, time spent in X app, etc. While businesses have always been capable of creating custom ‘productivity’ monitoring solutions, never before has such a comprehensive commercial product been so readily available to so many organisations. 

How the score is calculated

The score is based on People Experiences (out of 500) and Technology Experiences (out of 300) for a total score out of 800.

Scoring areas are each weighted at 100 points and are the following:

• Communication (People)
• Meetings (People)
• Content collaboration (People)
• Teamwork (People)
• Mobility (People)
• Endpoint analytics (Technology)
• Network connectivity (Technology)
• Microsoft 365 Apps Health (Technology)

The Productivity Score dashboard shows each of these areas over time and then how each’s score compares with peer organisations in your industry.

Digging further into the metrics: Communication example

When digging further into each of these metrics, we’re able to see how they are calculated, and how each is entwined with Microsoft app usage.

For instance, if we take a look at the Communication score, it includes:

The total number of people within the organisation using more than one method of communication over time (with data collected over the past 180 days).

People communicating

• How many out of the total email enrollees are sending emails (28 days)
• How many out of the total Teams enrollees are sending messages in Microsoft Teams (28 days)
• How many out of the total Yammer enrollees are posting in Communities (28 days)

Email responses

• How many emails with @mentions were responded to out of the total sent (28 days)
• How many emails without @mentions were responded to out of the total sent (28 days)

Messages in Teams

• Number of Chat messages out of total Teams messages (28 days)
• Number of Channel messages out of total Teams messages (28 days)

Questions in Yammer

• Number of questions answered out of total posted in Yammer
• Number of questions marked ‘Best answer’ out of total posted in Yammer

For a good wrap up of what’s involved in the Microsoft Productivity Score, check out Microsoft Mechanics ‘Microsoft Productivity Score | Measure organization productivity via Microsoft 365 admin center’ on YouTube.

The (Microsoft) issue with Microsoft’s Productivity Score

As you can see from the above metrics, each productivity score relies heavily on Microsoft products. For communication, this doesn’t take into account things like:

• Alternate communication forms, like phone calls, physically stopping by an office, or using Slack or Toggl
• The effectiveness of any given email, Teams message, etc.
• Employees not using Microsoft communication methods as expected (for instance, employees not using @mentions may need training to get in the habit)

In effect, it’s only (somewhat) useful if you are purely using Microsoft communications apps to the T within your organisation. Even then, the degree to which this measures effective productivity is highly debatable. If you are concentrating on figures that don’t represent effective communication (in this example), then you are simply wasting your time looking at them. Instead, Microsoft is changing it’s messaging surrounding the Productivity Score to saying the score is more about the degree of digital transformation in a given area, vs actual productivity.

The (other) issue with Microsoft’s Productivity Score

A new feature of the Microsoft Productivity Score has caused alarm bells to go off for privacy and surveillance advocates around the world. The feature was being able to drill down into each of these metrics to the employee (user) level, to see their individual actions.

An alarmed tweet-chain by Wolfie Christl, an Austrian researcher, was picked up by journalists around the world outlining the dangers of included employee-level monitoring in the Productivity Score product:

“Showing data on individuals can be turned off, but it’s activated *by default*. This normalizes extensive workplace surveillance in a way not seen before.

I don’t think employers can legally use it in most EU countries. I’m sure they cannot legally use it in Austria and Germany.”

This type of surveillance is problematic for the reasons outlined above. Having Microsoft as the default arbitrary body to inform productivity at the individual level is dangerous, as you are probably measuring the wrong outputs specific to their work, among other (sometimes illegal!) problems with this system.

Soon after, Microsoft responded to complaints, releasing a note title ‘Our commitment to privacy in Microsoft Productivity Score.’ Within the note, Microsoft makes it clear that they are scrapping the ability to drill down to the individual employee level by removing usernames – and instead reaffirming that the product is designed for organisation-wide metrics monitoring, not individual monitoring.

Should we use Microsoft Productivity Score in our organisation?

Despite its drawbacks, Microsoft Productivity Score can be useful – depending on your approach to the data. When you are tracking KPIs across the organisation, you’ll need to determine just how much particular data points are useful. The Technology Scores can certainly be important, for instance in checking out your network uptime. For People Scores, you need to be much more scrutinous, and include other data points, to really track productivity. It can certainly be useful to track how well your employees are using Microsoft 365 as a technology solution. In terms of the degree of digital transformation of your company, it neglects to measure effectiveness outside of Microsoft product borders.

If you want to learn more about how Microsoft Productivity Score can benefit your organisation, or would like help in productivity or digital transformation tracking with digital data metrics more generally, then make sure to get in contact with us here at A1 Technologies.