The digital tools landscape is undergoing a swift and dynamic transformation. This is driven especially by the surge of generative AI technologies that hold the promise of dramatically enhancing workplace productivity — also very much within fields of management consulting and knowledge work – areas that we at AskBrian concentrate on. The advent of Microsoft Copilot has introduced a potent new player to the market, sparking vigorous debates across consulting firms globally. Amidst this wave of enthusiasm, a pivotal question emerges: Is Copilot already poised for broad-scale adoption in the realms of consulting and knowledge work?
In this blog, we delve deeper than the commonplace discussions and examine the status quo in more detail. We investigate the actual implementation status of Copilot based on the feedback we gathered from leading consultancies and assess its capabilities in contrast with the rising industry expectations. In this post, we’re exploring new ground by taking a close look at the practical choices firms make when considering the broader adoption of advanced AI technologies in general and Microsoft Copilot in particular.
Microsoft Copilot and its Capabilities
What is it about?
When speaking about Microsoft Copilot, it is important to understand the different versions and interpretations. The term “Copilot” serves as an umbrella term for various Microsoft products that utilize advanced generative AI technology based on OpenAI’s GPT models in different Microsoft applications. The most frequently used version is the free Copilot, formerly known as Bing Chat Enterprise. This version is available to all users and acts as both a generative AI and an advanced search engine.
However, when people talk about “Copilot”, they are often referring to Copilot 365, which was launched in September 2023. This paid version integrates seamlessly with Microsoft Office applications, streamlines workflows and provides insightful answers by accessing user’s individual knowledge base.
A cheaper version of Microsoft 365 is Copilot Pro, an integrated solution for selected Office applications that is not integrated with Teams and does not provide access to the knowledge base, has been available since the beginning of 2024. It is one tier below the Copilot 365 version and could prove to be an attractive option for individual entrepreneurs and smaller companies.
As we navigate through this assortment of Copilot offerings, we’ll focus primarily on Microsoft Copilot 365. However, you can unfold the section below and learn more about the different Copilot versions – we will make comparisons between the Free, Pro and 365 versions to illustrate the key differences between them and highlight their respective pros and cons for your consideration. Please consider this is a snapshot of June 2024 and is likely to change.
From now on, let’s focus on Office 365
Available versions:
Understanding the different versions of Microsoft Copilot is essential as they are often confused or mixed up. This section provides an overview of the key differences between the various Copilot offerings and highlights their features and areas of application.
Copilot Free Version:
The free version of Copilot uses the latest GPT-4 Turbo model and is offered free of charge, with a limited number of requests and accessible functions. Nonetheless, its universal availability represents a strategic move by Microsoft to challenge Google’s dominance in the search space by offering AI-powered search capabilities without any barriers. Essentially, this free version is intended for users who want to experience AI integration with certain limitations and without financial commitments. It is likely to be the most popular option among the general population.
Copilot Pro:
Copilot Pro is a practical solution for business professionals and knowledge workers wanting to use Generative AI in their office applications but at the same are hesitant to integrate Microsoft’s generative AI technology with their sensitive data. It improves productivity within the standard Office tools (with the exception of Microsoft Teams) and offers “commercial data protection” instead of “enterprise grade data protection”, which could be an issue for European companies. The price of Copilot Pro is €22.00 per user per month, slightly below the price of the Copilot 365.
Copilot for Microsoft 365:
Larger companies should consider Copilot’s offer for Microsoft 365, which costs €28.10 per user per month. While it comes at a cost, it offers comprehensive integration with all Microsoft applications and leverages an organization’s collective knowledge to extend functionality within the Microsoft 365 environment. However, the introduction of this platform may raise privacy and security concerns especially in consultancies. Why? The particularly critical aspect is the knowledge base of consultancies often comprising confidential information from hundreds to thousands of client engagements. A knowledge database that would be used in a non-transparent way to generate new content – possibly even for a competitor.
(Source: Microsoft Copilot | Microsoft KI ; Source: Expanding Copilot for Microsoft 365 to businesses of all sizes | Microsoft 365 Blog)
The architecture of Microsoft Copilot 365
(Source: Microsoft Copilot for Microsoft 365 overview | Microsoft Learn)
In this overview, we will briefly describe the functionality of Copilot 365 using the graphic provided. Copilot acts as the central orchestrator, connecting Microsoft 365 applications with Microsoft Graph and a large language model. Microsoft Graph provides a structured repository of company knowledge, making it accessible to the software. The large language model then utilizes this knowledge to offer user-specific assistance within Microsoft 365 applications. This integration allows for a seamless process, where work previously confined to individual applications is now enhanced by organizational knowledge and processed by a powerful AI. This complex system combines the advantages of each element, creating a more efficient and intelligent workflow.
What is the impact of Copilot and Generative AI on consulting?
As there is no quantitative data yet on how Copilot is impacting consultants, we refer to a study by the Boston Consulting Group and Harvard Business School on the impact of GPT-4 from September 2023 analyzing the change in performance of 758 BCG consultants when dealing with 18 creative and analytical tasks. As AI transforms different sectors, this study offers specific insights into how it could change the work of consultants – even before being deeply integrated into the digital workspace.
The study by Harvard Business School sheds light on the transformative effects of generative AI, particularly GPT-4, on the consulting industry. The research revealed that task execution speed increased by an impressive 25%, while the task completion ratio saw a 12% uptick. Notably, 90% of consultants produced more varied results, leading to a remarkable 40% boost in creativity. However, while individual creativity soared, the overall team’s diversity of ideas decreased by 41%.
Another notable effect of GPT is its equalizing impact on consultant performance, reducing skill gaps among consultants. Less skilled consultants improved results by 43%, whereas the top performers saw a 17% improvement, suggesting that GPT-4 may diminish differences in qualifications between individuals.
However, there were also downsides observed. The use of GPT-4 sometimes led to a decrease in work quality, particularly in tasks designed to target weaknesses in generative AI, resulting in lower analytical problem-solving performance. In fact, consultants misled by the AI technology performed worse even after training, which is a rather unexpected finding.
Despite these limitations, the study provides insights into the profound effects of generative AI like GPT on consulting, including areas where AI may lower result quality, notably in analytical problem-solving—an area where GPT-4 was expected to excel due to improvements over its predecessor, GPT-3.5. This underscores the ongoing development at the frontier of AI capabilities.
Generative AI powered applications such as Microsoft Copilot hold the potential to drive significant efficiency and productivity gains for organizations. To understand the practical impact, let’s explore the early adoption of Copilot 365 at Dentsu, one of the world’s largest global creative agencies, as a case study.
In August 2023, Dentsu, one of the world’s largest global creative agencies rolled out Microsoft 365 Copilot to a group of 300 users to measure the tool’s impact on working practices and technology adoption in a real-life scenario. The study revealed that 79% of participants responded positively, with 66% routinely using AI support for tasks such as creating summaries and composing messages. This allowed them to spend more time on strategic goals, which had a direct impact on productivity.
In addition to individual efficiency gains, the integration of Copilot has also improved global communication and knowledge sharing within the organization. The most important applications included writing meeting notes (88%), emails (48%), contributing to team chats (43%), composing emails in Outlook (42%), research (34%) and designing PowerPoint slides and content (33%).
One of the most relevant benefits was the time saved: 71% of users reported that they were able to save an average of 15-30 minutes a day – a time saving of 3 – 6%. The observed impact was unexpectedly low, indicating that users may not have fully utilized Copilot’s capabilities in their workflows. This highlights the need for comprehensive user training and engagement to maximize the potential of AI tools such as Copilot to increase organizational efficiency and productivity.
(Source: 2,500% ROI from Microsoft Office AI Copilot (uxtigers.com)
Copilot in Consulting: Good for what, bad for what?
Feedback from leading consultancies
We frequently engage with numerous management consultancies to discuss the latest technologies and their specific use cases. Drawing from these conversations, we’ve summarized the main arguments for and against Copilot, particularly in meeting the unique requirements of management consultants. It’s important to note that this represents a snapshot of the current perspective, which may and probably will evolve over the coming weeks and months.
Copilot is good for Consultancies because…
- Text processing: Wording and rewording of textual content no matter if based on email conversations, meeting notes or keywords
- Idea generation: Brainstorming ideas, building hypotheses to solve client challenges or choosing the right method to conduct a workshop
- Meeting efficiency: Transcribing Microsoft Teams meetings and summarizing expert interviews
- Knowledge search: New efficient way to find relevant documents and presentations using Microsoft graph
- Excel help: Assistance on how to work more efficiently in Excel – drafting a more complex Excel formula or a macro
- Convenience: Seamless integration into the Office 365 work environment via number of business applications
- Compliance: Enterprise grade data seucrity, privacy and efficient permission management
Copilot is not good for Consultancies (yet) because…
- PowerPoint requirements: Not helpful when it comes to creating and adjusting consulting-grade presentations
- Analytical performance: Only narrow use cases when it comes to working with and analyzing data in Excel
- Use case opacity: Lack of transparency what Copilot can and can not do to to increase the individual performance
- Prompting matters: Requires users to improve prompting techniques, with a possible learning curve leading to user frustration
- Knowledge cut-off: Indexing information from the last six months only – potentially limiting historical data access
- File size limits: A 20,000-character limit for document analysis hindering handling larger documents
- Cost and ROI: Perceived high cost per user combined with missing productivity gains transparency leads to difficult decision making
What is the Copilot adoption rate in consulting? A snapshot
Based on our conversations with consulting industry representatives, the adoption of Microsoft Copilot is progressing cautiously. Faster adoption is hindered by Copilot’s shortcomings in PowerPoint and Excel, where consultants spend 80-90% of their digital workplace time, and the high cost of broader implementation. As of Q2 2024, most consultancies choose to wait with a roll out to everyone or advice the emplyoees to use the limited free version first or to use other solutions. Access to Copilot 365 remains restricted to selected test users only. Although initial adoption is slow, the potential benefits are being explored, and the technology’s development is being closely watched. Positive experiences reported by test users may boost confidence in Copilot and accelerate its adoption in the future. We are closely monitoring the next developments and are curious to see how Microsoft will define its priorities for further Copilot enhancements, considering the diverse needs of over 1 billion Office users versus considering the specific needs of management consultants.
Conclusion:
While Generative AI and Copilot offer great potential in transforming the work of management consultants, they currently fall short of meeting the high expectations within the key tools of their trade—PowerPoint and Excel. Consequently, many consultancies opt to develop their own tailored solutions (‘make’ instead of ‘buy’) or turn to ready-to-use specialized solutions such as Brian to address these shortcomings. We invite you to follow us and stay tuned for more insights on how technology is shaping the future of consulting.
If you would like to find out more about Brian, click here: