For over a decade, the VMware Tanzu team has been helping our customers harness the power of data to enable machine learning tasks and predictive completion. Building on those years of customer success, we are excited to announce new capabilities to help enterprises deliver enterprise-ready GenAI powered applications. VMware Tanzu AI Solutions, is a curated set of capabilities available in VMware Tanzu Platform, which enables organizations to safely accelerate and scale intelligent app delivery.
When leveraging GenAI in applications, organizations encounter several challenges along the journey, such as
-
-
Complexity when managing the multiple AI models needed for various application types
-
Inefficient large language model (LLM) utilization with stateless application data and apps
-
New operational requirements that include model cost management, model performance, and accuracy evaluations, as well as app performance evaluation
-
New scalability and governance concerns when ensuring that applications remain responsive and that models are being used safely
-
Tanzu AI Solutions addresses these challenges by providing a unified Java-native platform that covers the full spectrum of the AI application development lifecycle. Tanzu AI Solutions helps to automatically instantiate the complex patterns required to build and operate intelligent applications so that developers can use Java frameworks that speed delivery, while platform engineering delivers effective governance. In addition, new observability tools are available to monitor and optimize GenAI apps and underlying models.
Faster GenAI app delivery with Spring AI on Tanzu Platform
Spring AI is an open source application framework that was first announced in August 2023 and then refined by the Tanzu Spring team. It can be used by Java developers to build GenAI-enabled applications. Spring AI bridges the AI skills gap, enabling Java developers to seamlessly integrate AI capabilities without needing to cross-train in Python. Whether organizations are building new or retrofitting existing applications, developers can deliver intelligent apps more quickly and easily with Spring AI.
Spring AI enables easy access to the most popular AI models
Spring AI offers developer-friendly Java APIs that can be used to access Large Language Models (LLM) for multi-model approaches. For example, Spring AI APIs are portable across AI providers for chat, text-to-image, and embedded models. Both synchronous and stream API options are also supported. Furthermore, Spring AI offers exceptional portability for model switching, which simplifies the process.
Spring AI is fully integrated into the Spring ecosystem
Spring is the most popular Java framework, with 72% of Java developers using Spring Boot because it allows developers to leverage the extensive Spring ecosystem to build apps quickly (Internal analysis, VMware, 2023; The State of Developer Ecosystem 2023, Jetbrains 2023). Built by Spring contributors from the Tanzu team and community at large, Spring AI works with the familiar patterns that Java developers have come to know and love.
Safer GenAI apps with governance and private data
Today, one of the most significant hurdles for developers is access to safe, and business approved, GenAI models. Developers often do not have the expertise to curate and validate these models, which leads to delayed time to market when developers have to spend time researching models instead of just focusing on getting apps into production. Tanzu AI Solutions addresses this challenge by providing access to a marketplace in Tanzu Platform with self-service access to curated AI models. Developers do not need to be familiar with, or be experts in, model uses as data scientists can pre-evaluate model appropriateness and safety, and then platform engineering can deliver those vetted models to developers through the marketplace.
GenAI on Tanzu Platform enables seamless model access
Tanzu AI Solutions also offers a Java-based AI middleware that provides an enhanced OpenAI compatible API gateway in Tanzu Platform. Organizations can run or connect to 100+ proprietary or open source LLMs in a secure and consistent manner through this feature. It also provides the ability to secure access to models for authorized users. This AI middleware is the latest innovation we have added to the GenAI on Tanzu Platform service.
For our initial feature release for the AI middleware, safety was our biggest concern, and we are pleased to announce that at Explore Las Vegas 2024 we are launching secure and seamless access to AI models with virtual keys based on role based access control. Virtual keys can be managed by platform engineering so that as individuals change roles, teams, or leave the organization, model keys can be deauthorized to protect against unauthorized access. Platform engineers can also use keys to prevent overutilization of the token budget by limiting the number of developers that can access models.
VMware Tanzu Data Solutions enables Retrieval-Augmented Generation (RAG) context while remaining private
Access to private, reliable, on-demand data is critical for the success of RAG applications that use models alongside your own data to provide enterprise context to responses. Given the diverse array of data types, volume, and complex calculations required to empower contextual responses in your apps, a standard database falls short of delivering response accuracy. Vector databases are needed to provide the scale and flexibility to co-mingle data in the privacy of your own datacenter or cloud, and can be used to enable LLMs to search for similarities and make credible inference.
Tanzu AI Solutions is compatible with several VMware-provided vector databases, including VMware Tanzu Greenplum and VMware Tanzu Gemfire. These vector databases provide the flexibility and automation of a relational database with vector capabilities so that you can increase the accuracy of your RAG application responses. Your data remains private in your datacenter or cloud and the LLM calls to the vector database to add context to answers, which improves the relevance to your business and current status. RAG applications provide these answers seamlessly while avoiding any data leakage as long as you run your models on private infrastructure like VMware Private AI Foundation with Nvidia.
More accurate GenAI apps with observability for GenAI models
Tanzu AI Solutions includes a model and app evaluation capability that addresses non-deterministic apps and services for quality control and quality assurance. Observability for GenAI on Tanzu Platform collects essential metrics to observe GenAI app and model behavior so that organizations can monitor for accuracy and performance while having visibility into the usage of GenAI models, thereby expediting root cause analysis.
Figure 1: Apps that use GenAI require regular management and audit because of their non-deterministic properties
Audit capabilities also enable platform engineers to compare and test model outputs. Logs of outputs enable organizations to perform A/B testing of model performance and improve outputs. Token usage is also captured, which means that organizations can understand the costs they are incurring with their model approach. The audit and token usage insights can be exported to the data visualization tool of your choice.
Ready to take your GenAI app development to the next level?
Organizations that are deploying and running AI-powered applications in production will encounter several challenges in getting to production, including complexity when managing multiple models, inefficient resource utilization, new operations requirements such as model cost management, scalability, and governance. Tanzu AI Solutions provides a comprehensive suite of tools and services that streamline the development and deployment of AI-powered applications. The solution includes new technology and improvements to existing features to holistically address intelligent application use cases.
Tanzu AI Solutions includes a unified Java-native platform covering the full spectrum of the AI application development lifecycle. With Tanzu AI Solutions, developers get access to consumable frameworks with built-in governance so that they can cut through the complexity and safely accelerate intelligent app delivery. In addition, Tanzu AI Solutions can be deployed with VMware Private AI Foundation with Nvidia, providing privacy and security, choice of LLMs, optimized cost, excellent performance, and high productivity for generative AI workloads on-premises.
For those that are looking to uplevel their GenAI readiness across teams, tools and processes:
VMware Tanzu Labs has over 30 years of experience helping organizations bring innovative products to market. Tanzu Labs takes an evidence-driven approach to AI product development, integrating a forward-looking design practice with deep experience building high risk systems in regulated environments that include finance, healthcare, and government. Tanzu Labs can help your organization take use cases from idea to production, overcoming barriers in safety, accuracy, and cost management with small, fast, low risk experiments. For those just getting started on their AI Journey, Tanzu Labs offers a free AI Assessment workshop for qualified customers. To learn more, contact your VMware Tanzu Account Executive.
Explore the possibilities with Tanzu AI Solutions and guide our next feature releases today. Apply to become a Tanzu AI Solutions Design Partner.
VMware makes no guarantee that services announced in preview or beta will become available at a future date. The information in this press release is for informational purposes only and may not be incorporated into any contract. This article may contain hyperlinks to non-VMware websites that are created and maintained by third parties who are solely responsible for the content on such websites.
About the Author
More Content by Camille Crowell-Lee
