Balancing the risks and opportunities of LLM technologies
Tldr;
Third-party LLM applications hit the industry by storm in 2023 and forced organizations across industries to consider their approach to adopt the technology
The LLM landscape changes daily and is complex to understand even if you have a technical background. As such, many organizations are making their decisions from a position of fear rather than confidence
Regardless of whether your organization is an early adopter, a third-party LLM prohibitionist, or a laggard, it is worth taking the early steps required to identify the proprietary data sources that you have at your disposal and how they might support a value LLM-powered use case
Organizations should explore architecting their own LLM platforms that allow them to combine third-party software, open source technology, and custom built components.
While architecting these platforms, firms should constantly be considering how to safeguard their proprietary data as it is one of the few sources of sustainable competitive advantage in the LLM landscape
Start by building out your LLM platform one use case at a time, prioritizing those that will are less complex but still valuable, and leveraging internal and expertise to navigate the LLM technology landscape as you go.
Want to learn more about building an LLM platform for your business? Get in touch
Since the emergence of third-party large language model (LLM) applications in 2023, such as OpenAI's ChatGPT and Google's Bard, they have captivated the AI industry and attracted a substantial user base. Users have enthusiastically embraced these applications in their personal lives, leading to a surge in enthusiasm and online content sharing tips on how best to leverage LLM technology. However, in the business environment, adoption behaviour is split between wholehearted endorsement (minority of the landscape), complete prohibition (a small but growing percentage), and indecision (the vast majority of firms). At Arctic, we believe that LLM technology is a tremendous opportunity for businesses in the investment industry and beyond to find new levels of productivity and companies should be acting now to develop systems that meet their needs and work around their constraints. At the same time, we understand the hesitation and think that being deliberate when architecting and deploying LLMs is necessary. This Perspective will examine the use of LLM technologies from a risk lens to better understand some of the fears that are driving companies towards action or indecision.
Whether they like it or not, all companies are in a position where they have to decide how and when to leverage LLM technology. The first natural thought is for leaders to decide whether some of the third-party LLM tools like ChatGPT and Bard meet their needs. As they consider these tools as a possible solution, we have seen three fears drive decision-making:
The fear of missing out (FOMO)
The fear of getting their first LLM roll-out wrong
The fear of data leakage
The fear of missing out (FOMO)
Amid discussions about the transformative potential of LLMs, reputable voices emphasize the positive impact on firm productivity. Small firms fear being left behind by tech-savvy incumbents who can widen the gap in their market by adopting the technology quickly. In contrast, incumbents worry that startups and peers might leverage LLM technology quickly and narrow their market lead faster than was previously contemplated.
Companies that are primarily worried about being left behind or caught flat footed are likely to be early adopters of third-party LLM applications but it is also likely that they did not take enough time to consider the use cases that LLMs should be used for or whether using a third-party solution was the right approach.
The fear of getting their first LLM roll-out wrong
At Arctic, we cover the LLM space in great detail and if there is one thing that everyone in the space can agree on it’s that the market is rapidly evolving, it is complex, and nobody is certain of what will happen next. New LLM application providers constantly emerge with substantial funding, large technology companies announce their new services, acquisitions, or partnerships, the open source community makes incredible leaps releasing new models to the public, and rumours of regulation start to get more substantial (like in this week’s news about AI regulation in Europe). When firms struggle to keep up with all the moving parts and understand the basics of the landscape, it is nearly impossible for them to be confident that their first roll-out of LLM technology will be successful. They also worry about leveraging the technology appropriately. More specifically, they worry about a third-party LLM application providing their employees with a piece of information that is incorrect and leads to a bad decision. Consequently, businesses are apprehensive about making investments when the underlying conditions might change in a week’s time.
Companies that are worried about the success of their initial implementation are likely to wait until a point in time at which they feel the space has stabilized enough to make a sound investment decision or they have upskilled themselves enough to keep up with the pace of change.
The fear of data leakage
Data privacy poses a legitimate concern for many companies, particularly when dealing with sensitive personally identifiable information (PII). Organizations have a duty to protect user data, making it untenable to share it with third-party LLM providers that store and potentially exploit such data for model re-training. Furthermore, strategic considerations come into play, as organizations realize that data is a valuable competitive asset. Handing over data to third parties relinquishes control over access and usage, eroding the competitive advantage derived from proprietary knowledge bases.
Organizations that start LLM discussions by bringing up this risk often ban third-party LLMs entirely leaving employees and teams wondering, “If I can’t use the same tools that other companies have access to, what am I supposed to do and how am I supposed to keep up?”.
Our advice for…
Early Adopters
For early adopters, it is commendable to have taken a lead, but their pace might not be as defensible as they anticipate. Since third-party LLM applications can be sold to anyone willing to pay, the time advantage gained through early adoption of these tools will erode quickly. It is crucial for these companies to not treat LLM adoption as a one-and-done activity. For starters, they need to:
Continuously measure return on investment and assess ongoing productivity improvements
Conduct a comprehensive data sensitivity analysis, if they have not done so already, to ensure that PII is not shared with third party LLM providers and that they are protecting strategic data assets
Do not assume that everyone in your organization is adapting to the new technology well. A recent study by BCG showed there are varying degrees of optimism towards and experience using generative AI (e.g. LLMs) across frontline workers, management, and leadership that could impact how the technology is adopted. See figure 1.
Figure 1: Optimism and regular use of generative AI by job level
The indecisive and third-party LLM prohibitionists
Acknowledging the risk-averse approach of those who are hesitant to adopt or are banning third-party LLM applications, it is important to recognize that forgoing the technology entirely is likely a mistake in the long run. In our experience, there are a couple low-effort activities that you and your team can do to at least get started and set a foundation of principles that allow you to explore the LLM technology landscape with more confidence. These activities are:
Perform a data classification exercise to identify different data types and determine their suitability for sharing with third-party providers.
Create a list of potential use cases for LLM technologies within the business and map the identified data types to each use case.
If you are fortunate, you might have one or two use cases for which you have no data leakage concerns. If that’s the case look at the third-party vendor landscape, create a business case, and consider rolling the software out in waves. However, it has been our experience that these situations are few and far between. It is more common that data leakage concerns span across all use cases and that the use cases most likely to bring your company the value are those that rely on your propriety data.
For all organizations
To build sustainable competitive advantage through LLM technology, we believe that architecting a system that fits your organization is the correct approach. Building an LLM platform affords organizations control over their sensitive data and maximizes its value for specific use cases.
Notice that we say “architecting” and not “building”. We understand that there are very few organizations in the world that have the capabilities to build an LLM platform of their own from the ground up. The role of architecture is to take the firm’s constraints and design an LLM platform that is feasible from a technical and financial standpoint. This means that organizations:
Must define the capabilities that need to be delivered by software and / or machine learning components
Need to understand the delivery approaches that are available to them (i.e. build custom, leverage open source, or buy third party) and be able to articulate the trade-offs associated with each
Most firms will be surprised by how far they can get before needing a deep understanding of the LLM technology landscape. At some point however you will need a small team of internal or external experts who can help you navigate the LLM world. These experts will help you navigate the rapidly changing space, understand the trade-offs between the various LLMs at your disposal, select tooling to monitor your platform, navigate the many deployment considerations, and design the custom software that you will build to deliver on your use cases.
We believe that this architecture-led approach is the right one for organizations in the long term because:
We anticipate that LLMs will accelerate towards commoditization. For the vast majority of use cases, there will be little differentiation between open-source and private models. Signing an agreement with a third-party LLM provider now on the basis that their model outperforms alternatives might prove shortsighted especially when compared against the concessions that accompany such agreements.
Value will be driven primarily through the use of proprietary data. Ultimately, the most valuable LLM-powered use cases rely on the use of proprietary data and firms would being doing themselves a disservice by enabling that use case for their competitors by sharing their data.
It allows you to start small, measure adoption, and control the roll-out within your teams. When you architect your own platform, you are in complete control of the product roadmap and can prioritize use cases however you see fit. That enables you to roll out solutions to address low hanging fruit or to enable teams that are most likely to be open to adoption of the solution first.
In Summary
The LLM landscape (both third-party solutions and open source) is difficult to navigate and is rapidly evolving. The risks that are forcing companies to act and hesitate are real and it’s difficult to pass judgement on any firm based on their decisions to date. In the long-term we believe that leveraging your proprietary data is the only sustainable way to build competitive advantage through LLM technologies and you should start by architecting a platform that provides you with flexibility to deliver solutions through a combination of custom-built, open source, and third-party approaches.
If you want to discuss LLM technologies more and understand what your organization should do to take the next (or first) step, Get in Touch