Articifial Intelligence Primer

Diving Deep into the AI Value Chain

Over the past year, the world has become increasingly aware of the vast potential of AI. The unparalleled pace of AI discoveries, hardware improvements, and model introductions is putting data and AI strategy at the forefront of investors’ and organizations’ priorities. We believe that AI will usher in the next generation of product and software innovation, and we’re already seeing the early signs of this reflected in company fundamentals.

In this report, we examine the different layers of AI. While most people focus on the application layer, there is much more to the AI opportunity. We go over the three layers (1) hardware (2) data infrastructure (3) applications.

We believe that the hardware opportunity is significantly larger and broader than what is currently incorporated into expectations. We see the most underappreciated value in the data layer, and we believe the key to differentiation in the application layer will be access to proprietary data.

The Evolution of Artificial Intelligence

AI is not a new concept. What is generating all the excitement is that AI is evolving from being able to only perform programmed, predictive tasks to being able to put things in context and draw conclusions.

The first wave of AI focused on learned perception and inference, such as recognizing images, understanding speech, and recommending videos or items to buy. The early Natural Language Processing (NLP) models trained in 2018 were only capable of performing visual search and image recognition, and contained fewer than 100 million parameters.

What changed was that the hardware used to train these models became significantly more powerful, and a new model, known as the Transformer Model, provided a unique framework for leveraging these hardware capabilities. The Transformer Model is the foundational model behind Chat GPT/GPT3 and most other Large Language Models (LLMs) trained today. It is a neural network that learns context, making it a breakthrough in AI.

The current wave of innovation is focused on Natural Language Processing (NLP), which enables the ability to understand text and spoken words in a manner similar to how humans do. ChatGPT, introduced in late 2022, is revolutionizing online search and was the first AI application available at a mass scale (or accessible to everyone). However, there are many other relatively easy applications of NLP, such as smart assistants, chatbots, email filtering, and document/sentiment analysis.

Based on the same concept, there is a lot of innovation in coding where AI can assist in writing code (e.g., GitHub), cybersecurity where AI can assist in going through a vast amount of data and identify threats and weak links, etc. Most of the applications to date have been focused on productivity improvement.

The next wave of AI, we believe, will be AI interacting with the physical world. Examples include digital twins, autonomous driving, robots, and avatars. We are still very early in this third wave. Even areas, such as autonomous driving, that many believe are close to reality are closer to the start line than the finish line. This is because the innovations done to date are not based on generative AI, but rather on predictive approaches, where many technological developments hit bottlenecks. Many predictive models have plateaued in terms of performance improvements, despite the massive growth in data.

The Three Different Layers of the AI Opportunity

While most investors have been focused on the application layer (ChatGPT, Copilot 365, GitHub Copilot), there is a lot more to the AI opportunity.

The Hardware Layer was the first to see sizable fundamental improvement in earnings in 2Q23 with Nvidia Data Center growing from close $300 million in 2018 to ~$40-50bn annualized run rate today. With such outsized growth, investors are wondering if demand was pulled forward. We expect a compute opportunity of approximately $1 trillion over the next 4-5 years. We also believe that the opportunity is broader than just compute and there are many relatively smaller markets such as memory, networking, energy management (e.g., liquid cooling) that will see exponential growth. The hardware opportunity is relatively known by investors, and therefore, we do not expect outsized moves (100%+ upside) going forward. We do believe that there is still attractive opportunities as there is potential for upward estimate revisions, which would imply compounding upside.

The Data Infrastructure Layer is where we anticipate seeing the most significant opportunities in 2024. Over the past year, we went through a spend optimization cycle. Companies noted budget cuts by their customers, increased deal scrutiny, etc. While some level of optimizations is a norm and we expect to continue, the cloud service providers (such as AWS and Azure) and companies across the data ecosystem (such as Snowflake and Datadog) noted a meaningful pickup in demand in 4Q23 with new AI workloads offsetting optimizations. While the initial impact has been only in single-digit % of revenue, we believe that AI is a data problem, and we are just in the early innings of data companies seeing an inflection.

The Application Layer is expected to be the largest TAM (total addressable market), but it is also the most crowded and competitive space. It is comprised of startups and incumbent players. The incumbents have large installed bases and access proprietary data, which provides them with a significant advantage.

Let’s dig into each layer.

The Hardware Layer

From a hardware perspective, Model sizes grew from < 100 million parameters to >500 billion over the span of 5 years. The predictive wave was built on hardware with tens to several hundred million parameters.

ELMo trained in 2018 was able to perform a visual search – image recognition and contained 26M parameters
BERT-Large trained in 2019 was able to flag sentiment and provide basic Q&A – contained 340M parameters
The current LLMs contain>500bn parameters.

Consistent with this is growth in model sizes; Nvidia’s Data Center revenues experienced a significant increase. Data Center revenue had already grown at a healthy 40% CAGR, but 2023 marked an inflection.

CPUs have been the dominant data center processor over the past cycle. They are a serial processor, which means that they process one instruction at a time. As the compute requirements for generative AI increased exponentially, the need for parallel processing emerged (performing multiple calculations simultaneously). This is what made GPUs, a parallel processor originally used for computer graphics, the core processor for AI. It is important to understand the size and scale of these chips.

Nvidia’s H100 is comprised of 35,000 parts and nearly 1 trillion transistors in combination. It weighs 70 pounds. It takes a robot to build and takes a supercomputer to test a supercomputer. – Jensen Huang, CEO of Nvidia

The market for GPUs has grown exponentially is expected to be a ~$1trillion market in 4-5 years. Compute (GPUs and CPUs) is where most of the Data Center upgrade budgets are going, but it is not all. To optimize the performance of the data center, the system’s performance is crucial. As GPU performance has significantly improved, bottlenecks are now found in bandwidth, memory, and other areas. These tangential markets, although not as large as the compute layer, are meaningful in size and can offer similar upside in terms of growth.

For example, the Networking Market last year was a $2 billion market expected to grow to $10 billion in five years. Nvidia recently announced on its most recent earnings call that networking now exceeds a $10 billion annualized revenue run rate. Strong growth was driven by exceptional demand for InfiniBand, which grew fivefold year-on-year. InfiniBand is critical to gaining the scale and performance needed for training LLMs.

Similarly, the Liquid Cooling Market was a $2 billion market last year, expected to grow in the teens. We expect to see significantly higher growth rates. Cooling represents 40% of a data center’s energy consumption, and although GPUs require 2x less energy to perform a similar task to a CPU, the exponential growth in GPUs is coming with a significant

The current generative AI wave has been very compute intense. The computational requirements have increased exponentially with model sizes.

The current wave of generative AI has been significantly more compute-intensive. The computational requirements have increased exponentially with model sizes. The current infrastructure (even data centers that were built 2-3 years ago) would need to be upgraded to accommodate AI workflows. The number of models that companies are developing (covered in the next section) has also surged.

However, the cost to run models has significantly decreased.

Despite companies pricing new products at a 2-3x premium to prior versions (e.g., H100 vs. A100), given that these new products offer significantly better performance, the cost to perform tasks is significantly lower.

We anticipate that, over time, costs will continue to decline, making it significantly more affordable to run models, which we expect will spur even more application development.

In conclusion, accelerated computing is a very broad opportunity.

While significant value is in the GPUs, Data Centers are systems, and it is all about optimizing these systems. Memory, bandwidth, and energy consumption are key pressure points.
A big misconception is that GPUs are replacing CPUs. While it is true that GPUs are gaining a larger slice of the pie, there is innovation necessary across the board (including in software that optimizes the use of GPUs vs. CPUs)
Second, while LLM training and other applications that require large computing power will be done at the data center, accelerated computing usage will broaden to the edge and devices. Ultimately, every device will have some form of accelerated computing.
Over time, while the cost of running models will decrease significantly, the size of the pie will continue to grow strongly.

The Data Infrastructure Layer

The generative AI and data infrastructure companies play a pivotal role in enabling the development, deployment, and operation of AI systems and LLMs. The Infrastructure stack has undergone a significant transformation over the past few decades, especially in the realms of data storage and processing.

In the early 2000s-2010s, most enterprises focused heavily on hardware databases and Relational Database Management Systems (RDBMS), which were the backbone of data storage and management. These systems resided on physical servers, only supporting structured data handling and were limited in scalability and flexibility. Subsequently, the advent of message queuing technologies marked a shift toward more distributed computing, facilitating asynchronous data processing and communication between applications —a precursor to today’s cloud-native capabilities.

Presently, the cloud-native paradigm is ushering in a new modern AI stack, powered by the major cloud providers. They provide the core cloud storage and computing services, enabling scalable, flexible, and often more cost-effective data management solutions.

Cloud Service Providers

In the realm of generative AI and LLMs, cloud service providers are indispensable for their computational resources and scalability. These providers offer the essential infrastructure and platforms necessary for training and deploying sophisticated AI models.

Cloud service providers (CSPs) are the core of the AI value chain, as they offer the necessary infrastructure, computing power, and storage capabilities. AI models, particularly the more advanced ones like deep learning models, require significant computational resources that are often not feasible for individual companies to maintain. Cloud service providers offer scalable, on-demand resources that allow businesses to train and deploy AI models without the overhead of building their own infrastructure. This democratizes access to AI, enabling small and large businesses alike to innovate and leverage AI technologies.

Large companies above the AI value chain, such as Snowflake and Databricks, will require massive compute capabilities, primarily served by Azure and AWS. Similar to how Nvidia is a secular beneficiary of who “wins” in AI, the hyperscalers that underpin the compute needs for data platforms stand to gain regardless of who emerges victorious in the AI battle. In addition to the traditional compute and storage capabilities, CSPs have been building an additional layer on top of their infrastructure that provides companies with access to pre-trained models and other tools, enabling AI development (both internally developed and third-party).

Microsoft Azure (OpenAI): Through its partnership with OpenAI, Microsoft ushered in a new enterprise use for LLMs. Meanwhile, Microsoft doesn’t own its own core foundational model; instead, it has chosen to build a strong partnership with OpenAI and has acquired a significant ownership stake. For Microsoft, this partnership enhances its Windows 365, Teams, and Azure offerings. Microsoft has the opportunity to leverage its optimized AI cloud infrastructure, distribution, and data across its offerings to capture a larger share of the overall enterprise IT infrastructure market. Microsoft is adopting a more integrated platform approach to incorporating AI into its existing SaaS and platform services, with a focus on efficiently monetizing the new technology and delivering good value to customers through cloud service bundling.
Google Cloud Platform (GCP) – DeepMind: GCP has long been recognized for its advanced AI capabilities, particularly since acquiring DeepMind. However, in the current race to dominate the generative AI landscape, they have lagged significantly behind Microsoft. GCP is expected to have a better position in capturing the consumer market compared to Microsoft. The company has adopted heterogeneous computing, utilizing its proprietary TPU AI chip in the fourth generation, which offers a more favorable price-to-performance ratio than previous versions. They are also investing in their proprietary LLMs (PaLM 2, Codey, Imagen & Chirp), in conjunction with the Vertex managed cloud platform. GCP’s key strategy is to leverage its consumer appeal and incorporate some of its LLMs into its productivity apps.
Amazon Web Services (AWS) – Bedrock: AWS has been catching up to the earlier players discussed above. They have focused on leveraging partnerships with leading Gen AI companies. They introduced Bedrock, a fully managed service that offers a choice of high-performing foundation models (FMs) from AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon through a single API. Bedrock simplifies the process of building, training, and deploying ML models at scale. At their recent ReInvent event, they launched Amazon Q, a new generative AI assistant for work. They also announced updates for their AWS-Designed chips – AWS Graviton4 and AWS Trainium2, which will improve the speed, cost, and energy efficiency of running generative AI and other workloads. AWS prioritizes becoming the central hosting platform for LLMs and continues to enhance Bedrock while maintaining strong partnerships with leading startups. In the future, we expect AWS to develop its own proprietary models, AI chips, and training/inference accelerators.

Foundational Models

Foundational Model (FM) companies develop large-scale models that serve as the underlying components powering LLMs and generative AI. These AI models can be adapted for various applications and serve as the backbone of numerous AI systems, ranging from natural language processing to creative content generation. Foundational models are trained on vast datasets and require significant resources, both in terms of data and computational power. These are pre-trained models that lower the barrier to entry for AI application development, enabling more organizations to build AI-driven solutions.

There are two broad categories of foundational models;

Closed-source proprietary models
Open-source foundational models.

Closed-source proprietary models are models whose source code is not publicly available, but they can be accessed using APIs. Closed-source models often come with more advanced technical sophistication – more robust support, regular updates, and are tailored for specific commercial applications. Leading FM companies, such as OpenAI, have models with higher performance and offer off-the-shelf services that make it easier for developers to get up and running. Meanwhile, open-source models such as Stability’s Stable Diffusion and Meta’s Llarma2 are built on open-source software, which allows for community-driven changes to source code.

There will be use cases for both open-source and closed-source models. We believe that large companies with the ability to hire AI engineers will lean towards open-source FMs, while many SMBs will lean towards utilizing the existing closed-source models. Large enterprises will likely use both, and will likely lean toward open models because they offer greater customization and flexibility. Users can modify and tailor open-cloud models to their specific needs, offering greater flexibility in application development.

Data Platforms

The role of database companies in the AI value chain is to manage and facilitate the effective use of data, which is the lifeblood of AI. AI models are only as good as the data on which they are trained. Database companies provide the tools and platforms to store, manage, and process large volumes of data efficiently. They are responsible for ensuring data quality, consistency, and accessibility, which are critical for training accurate and reliable AI models.

In Databricks’ latest report on machine learning use cases, it highlighted that natural language processing (NLP) dominated other types of use cases among users. NLP is a rapidly growing field that enables businesses to gain value from unstructured textual data, such as large documents, summarizing content, or extracting sentiment from customer reviews. These NLP use cases dominated 49% of the Python libraries used.

Databricks reported experiencing a significant rise in the usage of transformer-related libraries on their platform, relative to SaaS LLM APIs and adjacent tools. These transformer-related software libraries and frameworks were specifically designed to support the implementation of Transformer models within an enterprise. These libraries provide pre-built modules and functions that simplify the process of building, training, and deploying Transformer-based models. Examples include TensorFlow and PyTorch, which offer extensive support for Transformer architectures, enabling developers to create sophisticated LLMs with relative ease.

SaaS LLM APIs and LLM tools make it easier for developers to integrate advanced language processing features into their applications without needing to build and train their models. These APIs, offered by companies such as OpenAI and Google, among others, provide access to pre-trained models via cloud-based platforms. Users can leverage these powerful models for various applications, including text generation, translation, summarization, and more, typically on a subscription or pay-per-use basis.

LLM Tools, meanwhile, refer to the broader set of tools and platforms that facilitate the use, management, and optimization of LLMs. This includes user interfaces for interacting with LLMs, tools for fine-tuning models on specific datasets, and systems for monitoring and evaluating model performance. These tools are designed to make it easier for a wider range of users, not just AI specialists, to benefit from the advances in language understanding and generation offered by LLMs.

This data suggests that enterprises prefer to implement solutions that allow them to customize, train, and deploy their own transformer model rather than utilize an already existing API from OpenAI. We believe this is an important shift that should serve as a tailwind for companies like Databricks and Snowflake.

Data Lakehouse vs. Data Cloud

Databricks

Databricks is a cloud-based platform designed for large-scale data processing and analytics. They provide an integrated environment for data engineering, data science, and analytics, built on a powerful and optimized version of Apache Spark, a leading open-source data processing framework. The Databricks Lakehouse is central to Databricks’ approach, blending the capabilities of data lakes and data warehouses to support various data types and structures, allowing for efficient data storage at scale. The unified Lakehouse concept consolidates all data management functions into a single platform, encompassing integration, storage, processing, governance, sharing, analytics, and AI.

Source: Databricks “Lakehouse” Platform

While already considered a key player in the AI stack, Databricks has emboldened its position as a leader in GenAI through investments in models like Dolly (an open-source instruction-following LLM) and its big-ticket acquisition of MosaicML. This acquisition further bolstered Databricks’ AI presence within the enterprise, as this product enables companies to efficiently build their large generative AI models on their own data and business processes. Databricks continues to echo the message that their Lakehouse is the best way for gen-native startups to train and deploy their own AI models, leveraging their proprietary data in a cost-effective manner without being tied to Big Tech.

Overall, Databricks is well-positioned to support startups that want to build their own generative AI models without being tied to Big Tech. Their development of Dolly and the acquisition of Mosaic ML further help them capitalize on a future where more and more companies want to build out their transformer models.

Snowflake:

Snowflake is a cloud-based data platform designed to facilitate data storage, processing, and analytics, with a strong focus on scalability and ease of use. At its core, Snowflake separates storage and compute functionalities, allowing users to scale each independently in the cloud. This architecture is particularly beneficial for handling large datasets and complex queries, making it a popular choice among data professionals, including those specializing in AI and ML.

Snowflake began as a data warehousing solution, enabling centralized storage and management of data, supporting both structured and semi-structured data types. However, in recent years, they have expanded their platform to incorporate more data lake features, enabling users to store unstructured data in its native format or unstructured data that is particularly useful for AI/ML use cases.

Source: Snowflake Investor Presentation

Snowflake announced several notable partnerships with NVIDIA, Microsoft, and Weights & Biases. The partnership enables Snowflake to integrate Nvidia’s NeMo enterprise developer framework into its Data Cloud, allowing Snowflake customers to build and deploy LLMs and AI-driven applications that leverage proprietary data residing in Snowflake. They have also established partnerships with companies such as Weights & Biases, Alteryx, Hex, Dataiku, RelationalAI, Pinecone, and others.

Snowflake vs. Databricks

With the proliferation of Generative AI applications, both companies are trying to position themselves as strategic multiproduct data platforms. Below, we highlight a few of the major announcements from the respective conferences and our thoughts on each company’s overall AI strategy.

Databases

Transactional Databases:

MongoDB – Companies like MongoDB provide a document-based database model and schemaless architecture, which plays a key role in Generative AI. Furthermore, the discovery and exploration of new use cases and application building serve as a demand driver for MongoDB’s consumption-based model. Additionally, MongoDB is expanding platform features with vector search, for example. Vector databases have garnered investor interest as a more suitable offering for AI/ML capabilities.

Vector Databases:

Generative AI and LLMs deal with massive amounts of data, often in unstructured forms like text, images, and audio. Vector databases are designed to efficiently handle, index, and retrieve these large datasets. They convert complex data into vector representations, making it easier to manage and process at scale. One of the primary functions of vector databases is facilitating advanced search capabilities. They allow for more nuanced and context-aware searches, which is essential in applications powered by generative AI and LLMs, such as personalized content recommendations, image recognition, and natural language processing tasks. Companies such as Pinecone and Weviate are leading players within this market.

In the world of generative AI, the ability to process and analyze data in real-time is crucial. Vector databases support this need by providing the infrastructure to quickly retrieve and analyze data vectors, enabling real-time responses and interactions in AI applications. As generative AI technologies and LLMs continue to grow, the demand for scalable database solutions that can handle increased loads without compromising performance becomes essential. Vector databases are inherently scalable, making them well-suited for these evolving needs. They play a significant role in MLOps, particularly in the deployment, monitoring, and maintenance of AI models. They facilitate the operational aspects of managing large-scale AI models by providing efficient ways to store and access the data these models need.

These category refers to MLOps and AIOps companies that bridge the gap between AI model development and operational deployment. They address the complexities involved in managing the lifecycle of AI and ML models, around generative AI and LLMs. This management encompasses the automation of model deployment, monitoring model performance, ensuring scalability, and managing the necessary infrastructure for these AI models to function optimally.

Developer Tooling

Companies within this category are primarily focused on operationalizing generative AI models. They include: Data labelling and curation companies, companies that support model hub and fine-tuning, model inferencing, LLM Ops, and monitoring and evaluation.

Data streaming, transformations, and pipeline companies such as Confluent play a key role in the role of data streaming and processing in the AI value chain. Data must be pre-processed before it hits the LLM. This includes extracting relevant context from data stores across the organization, transforming the data into a usable format, and loading the data into the model’s context window. In the context of generative AI and LLMs, Confluent’s real-time data processing capabilities enable the continuous and efficient flow of data, a prerequisite for training and updating LLMs. Generative AI models, especially those used in language processing, sentiment analysis, and content creation, rely heavily on vast, diverse, and often real-time data. Confluent’s platform facilitates the collection and processing of this data from various sources, ensuring that it is readily available for AI models. This is crucial for keeping the models updated and relevant, a key factor in their effectiveness and accuracy.
Data Labelling companies, such as Scale.AI and Snorkej, play a critical role in labelling data for training AI models, including LLMs. These companies specialize in annotating and categorizing data to make it usable for training AI models. The accuracy and quality of data labelling directly impact the effectiveness of AI models. In generative AI, where models are often trained on vast datasets, the role of these companies becomes even more critical. They ensure that the data fed into LLMs is well-structured and accurately labelled, leading to more effective and reliable models.
Fine-tuning companies such as Hugging Face, Databricks (with MosaicML) play a crucial role in adapting pre-trained LLMs to specific tasks or industries. Companies specializing in fine-tuning help customize these models to suit particular use cases, enhancing their relevance and performance. This process involves adjusting model parameters based on additional, often domain-specific data or specialized market needs.
Monitoring and Evaluation companies such as Weight & Biases, Arthur, and Gantry are crucial for maintaining the accuracy, fairness, and reliability of AI models. Companies in this space provide tools and services for the ongoing assessment of model performance, as well as monitoring for biases, drifts, and other issues that can arise over time. This is essential not only for model accuracy but also for ethical considerations, ensuring that generative AI models are used responsibly.
LLM Ops companies such as Langchain and Weight & Biases specialize in the operational aspects of LLMs, providing tools and services tailored to the unique challenges and requirements of these models. Langchain focuses on leveraging LLMs for language-centric applications. Their contribution lies in making it easier for organizations to integrate LLMs into their systems and workflows. Langchain provides tools and frameworks that help businesses harness the power of LLMs for various applications, such as chatbots, content generation, and natural language understanding tasks. They simplify the process of embedding LLMs into existing applications, making it more accessible for companies to adopt these advanced technologies.

Cybersecurity and Observability

As AI becomes more prevalent, the importance of cybersecurity escalates. AI systems process and generate vast amounts of data, some of which is sensitive or proprietary. Cybersecurity ensures the integrity, confidentiality, and availability of this data. Moreover, as AI systems increasingly make decisions that affect the real world, ensuring these systems are secure from tampering or malicious use is paramount. Cybersecurity firms play a crucial role in safeguarding AI systems from external threats and ensuring that AI is used responsibly and ethically.

Proper data governance and LLM security will be critical to enterprise deployment. Domain specialists like Credal (data loss prevention), Calypso (content governance), and HiddenLayer (threat detection and response) allow companies to securely connect internal data stores to third-party foundation models or customer-facing AI applications—ensuring full transparency, auditability, and traceability.

The advances in Generative AI have significantly changed the security landscape. Both attackers and defenders are rapidly adopting the technology, and the results are material in changing the mechanics of responses. Attackers have the edge as they are able to adopt and deploy their techniques much faster than the defenders. Many emerging companies are still developing their responses to these new types of attacks. There are two key ways to think about the impact of AI on cybersecurity.

AI for security
Security for AI

AI for security:

Over the past decade, AI has already been embedded in security. Some of these include security systems for protecting data from unauthorized access and ensuring compliance with regulatory requirements. The industry is facing a vast shortage in talent, and the current talent can’t keep up with the rise of attacks.

Security for AI:

ChatGPT lowers the barrier to entry for threat actors, opening the door to more malware, phishing, and identity-based ransomware attacks. Cybercriminals have already found ways to circumvent ChatGPT’s content filters and policies, which are designed to prevent malware-based code generation. Phishing attacks are another key attack vector in focus, as ChatGPT can quickly produce authentic-looking and highly personalized emails to target an end-user’s identity credentials. One of the effects of the rise of these types of attacks is that we will need more technologies focused on email security, identity security, and threat detection.

Human-written defensive software may not be enough to combat AI-generated malware, which has drastic implications for cybersecurity. The weakest link in the cybersecurity chain, regardless of how much companies spend, is the human users. Cybercriminals have learned how to use malicious code written by ChatGPT to exploit victims. As discussed, there are risks associated with social phishing attacks, where AI chatbots reliably extract sensitive company or personal information via impersonation. Additionally, there are risks of disinformation, whereby false AI-generated messages are disseminated to the public.

Case Studies:

SentinelOne and Crowdstrike

Crowdstrike and SentinelOne are case studies of companies that have implemented AI across their product lines. Both companies have incorporated AI into their products over the past decade. As core endpoint protection platforms, they combine vast amounts of data collected on the endpoint with information from adjacent platforms, such as cloud, identity, and devices.

These platforms have created large data lakes that capture data and respond to threats across the attack surface. They use AI-based telemetry to detect and respond to threats in real-time. CrowdStrike’s threat graph stores over 40 petabytes of data, handles trillions of events per day, and services 70 million requests per second. This makes their AI algorithm powerful and capable of providing an unprecedented level of security with significant automation. Over the past few months, Crowdstrike launched Charlotte AI in August. Similarly, SentinelOne launched its PurpleAI, a generative AI chatbot dedicated to threat-hunting, analysis, and response.

Zscaler highlighted on their recent earnings call, Zscaler Risk 360. This is the industry’s first holistic AI-powered risk quantification and mitigation solution. It delivers up-to-date risk posture and recommends corrective actions to mitigate risk in a timely fashion. Zscaler has already closed over 10 Risk360 deals and is in active evaluation with more than 100 enterprises. For those deals, Zscaler already generates over a 6-figure ACV on average, and they expect it to grow over time. Risk360 provides critical insights to CISOs when reporting on cybersecurity risk, strategy, and governance, particularly in light of new SEC regulations.

There are also many use cases of Palo Alto Networks deploying their AI models in their security co-pilots for network security, XSIAM in security operations, and AI in cloud security. Cloudflare has demonstrated use cases around edge AI inferencing on edge devices. We envision many more use cases around AI being incorporated within the company’s platform.

Observability

Observability in the context of AI refers to the ability to monitor and understand the internal states of AI systems. As AI models become increasingly complex and autonomous, understanding their decision-making processes becomes more challenging. Observability tools help in diagnosing problems, understanding model performance, and ensuring transparency. This is crucial not only for maintaining operational efficiency but also for building trust in AI systems. It involves monitoring model performance, data drift, and other operational metrics to ensure that AI systems function as intended. For generative AI and LLMs, which are inherently complex and data-intensive, observability becomes even more critical.

As more companies adopt AI into their enterprise stacks, they must move more workloads to the cloud to enable those functions. Companies like Datadog, Grafana, and New Relic provide comprehensive monitoring and analytics tools that are essential for the deployment and maintenance of AI models. These tools offer insights into the performance of AI systems, tracking metrics such as response times, error rates, and resource utilization. Such monitoring is vital to ensure that the AI models are functioning as intended and delivering the expected outcomes. On Datadog’s recent earnings call, the company cited examples where it is providing AI observability functionalities for some of the leading model providers. While Datadog isn’t a direct beneficiary, we view it as a secondary beneficiary of the rise in AI applications.

Application Layer

Over the past decade, companies have undergone digital transformation with the rise of the cloud. Rapid software development has been at the heart of this emergence of the cloud. Now, with the emergence of LLMs, we are at the cusp of an exponential rise in the number of applications developed. AI applications have the potential to supercharge this transformation. Nvidia’s CEO, Jensen Huang, famously said earlier this year:

There are five million applications in app stores, there are probably several hundred PC applications, and there are 350,000, maybe 500,000 websites that matter — how many applications are there going to be on large language models? I think it will be hundreds of millions, and the reason for that is because we’re going to write our own! – Jensen Huang, March 2023, An Interview with Nvidia’s CEO on Strategy.

We have transitioned from a world with hundreds of apps during the PC and Internet era to one with millions of apps in the mobile era. It is not a massive stretch to believe that we are now entering a world where apps built on LLMs can be customized to the needs and wants of individual users.

The most widely talked about AI application that people are familiar with is ChatGPT, but we believe enterprises will be the biggest TAM for AI applications. By types of products, this layer can be divided into several sub-groups:

Horizontal AI:

Horizontal applications, in the context of generative AI and LLMs (Language Models), refer to versatile solutions that address common needs or functions applicable to multiple sectors. These applications are characterized by their wide-ranging applicability and can significantly improve workflow efficiency. As AI advances in reasoning, collaboration, communication, learning, and prediction capabilities, next-generation workflow tools will not only automate routine tasks but also take on work that was previously only possible for humans, utilizing advanced approaches such as agents and multi-step reasoning.

Examples of such tools include Jasper and Copy.ai, which are content generation tools capable of creating marketing copy, writing code, or generating reports across various industries. The advantage of horizontal applications lies in their vast market potential, as they cater to a wide range of users and industries. However, their broad applicability often means they are less specialized and may require further customization or adaptation to meet specific industry needs.

Vertical AI:

Vertical applications, on the other hand, are specialized solutions tailored to the specific needs and challenges of a particular industry or sector. These applications are deeply integrated into the workflows, processes, and regulatory frameworks of their target industry, offering more customized solutions.

Companies can utilize AI to develop industry-specific applications, transforming human-machine collaboration and enabling end-to-end automation rather than just serving as a copilot or collaboration platform. We are witnessing examples of this in various sectors. In healthcare, AI-driven applications are revolutionizing diagnostics and patient care. In finance, they are being used for fraud detection and personalized financial planning. The potential impact of AI is vast and diverse, with significant implications. For instance, at the most recent AWS Re-Invent event, Pfizer noted that they expect to achieve annual cost savings of $750 million to $1 billion from some of their priority AI use cases. Pfizer is already leveraging the power of AI across 17 use cases, including scientific and medical content generation, manufacturing, and prototypes using Amazon Bedrock.

Within enterprises, the opportunity set can be divided into two broad areas by use case:

Enhancing growth
Reducing costs

Opportunity to Transform Revenue Potential and Reduce Costs

Reducing costs

This is the area with the most AI use cases today. Several innovative productivity improvement tools have already proven to deliver significant benefits. The first and largest use case based on NLP models was coding assistants. Two large-scale examples:

Microsoft’s GitHub. On its most recent conference call Microsoft noted that the company’s coding assistant Github has now over 1 million paid Copilot users and more than 37,000 organizations that subscribe to Copilot for business (+40% qoq) The number of developers using GitHub has increased 4x since our acquisition 5 years ago and is estimated to be increasing developer productivity by up to 55%, according to Microsoft.
Similarly, GitLab has released products like GitLab AI Code Suggestions (Beta) that provide either the entire code snippet or a complete line of active code when building applications. In the future, there will be cases where AI will help developers automate more mundane tasks and enhance Security. These are examples of how AI enhances the productivity of developers and drives additional growth for software companies.

According to a Bain AI survey this year, nearly 60% of software engineers expect AI to improve productivity by more than 20%.

In addition to coding assistance, the applications for AI-powered assistants are significantly broader (e.g., Microsoft 365 Copilot, chatbots, customer service assistants).

Growing revenues

The revenue enhancement side has two main components (1) increasing sales and marketing effectiveness (2) developing products. We are still in the early innings.

The application category needs to fix the retention:

A piece from Sequoia explored applications that have experienced record sign-ups compared to retention. ChatGPT set the record for the first application ever to reach 100 million users. However, its retention rate has been much lower compared to other peers that reached that milestone earlier.

Users crave AI that makes their jobs easier and their work products better, which is why they have flocked to applications in record-setting droves. User engagement is also lackluster. Some of the best consumer companies have 60-65% DAU/MAU; WhatsApp’s is 85%. By contrast, generative AI apps have a median of 14% (with the notable exception of Character and the “AI companionship” category).

Source: Sequoia Generative AI’s Act Two

This means that users are not yet finding sufficient value in Generative AI products to use them daily. Generative AI’s biggest problem is not finding use cases, demand, or distribution; it is proving value. It is also important to consider what aspect of the enterprise or consumer ecosystem would generate the highest retention over time. Currently, we understand that Horizontal applications offer a broad market reach and versatility, making them attractive for widespread adoption and rapid growth. Vertical applications, by contrast, offer tailored solutions with deep industry integration, appealing to sector-specific needs and often commanding higher value

Who will capture the value?

Although we expect that most of the value will be captured by the customers (i.e., enterprises using these AI tools), AI can still provide a meaningful uplift for product pricing.

There are two criteria that we have identified as key especially when it comes to capturing value at scale:

Possessing large installed bases that can be upsold
Access to proprietary data

1. Large Installed Base

In the first category, large installed bases, companies can charge a small fraction of the ultimate benefit and still generate meaningful revenue uplift. A good example is Microsoft, where we expect the newly introduced co-pilot to add $5-16 billion of incremental revenue, assuming only a 5-16% adoption rate. The $30/month price tag would nearly double the revenue for customers who adopt this solution. Background read on the Office 365 Copilot announcement here: Axios. 2. Proprietary data is crucial, but more is not always better. In the first category, large installed bases, companies can charge a small fraction of the ultimate benefit and still generate meaningful revenue uplift. A good example is Microsoft, where we expect the newly introduced co-pilot to add $5-16 billion of incremental revenue, assuming only a 5-16% adoption rate. The $30/month price tag would nearly double the revenue for customers who adopt this solution. Background read on the Office 365 Copilot announcement here: Axios.

In this category of large installed bases would be CRM systems providers (Salesforce, Hubspot), E-commerce platforms (Shopify, Amazon), and Design software:

More than 195K companies use Hubspot at scale
150K customers use Salesforce
1.75 million merchants sell on Shopify
Over 200 million people use Autodesk

2. Proprietary data is crucial, but more is not always better

The second criterion is access to proprietary data. This category again puts the incumbent players at a significant advantage.

The most important thing to highlight is that the quality of the data is even more important than the quantity. According to Cohere, a leading LLM provider, training models on a fraction of the data set can result in similar to better performance.

While there are thousands of AI applications, we believe that a significant portion of the value will be captured by incumbent players. There is still room for innovation and start-ups, but a tighter interest rate environment and limited capital availability make it tougher for small companies to succeed. We believe that strategic acquisition, rather than IPOs, will be the more common path in business evolution.

Large incumbents will also benefit from a favorable M&A environment where a proven platform approach will be particularly valuable. We have already seen several interesting deals in cybersecurity (Palo Alto/Talon, Crowdstrike), and data infrastructure (Snowflake/Neeva, Databricks/MosaicML).

Conclusion:

We are still in the early innings of the next technology wave amplified by AI. We have only scratched the surface of the ultimate opportunity by starting to upgrade the hardware infrastructure that enables AI model training and development.

We expect the hardware cycle to be much broader and more durable than investors anticipate. We are finding the most underappreciated opportunities in the data infrastructure layer, which is evolving at a rapid pace. In the application layer, we believe that incumbents that have access to proprietary data and large installed bases will capture a significant portion of the value.