Carson Long and Aish Nair co-wrote this post.
VMware Tanzu Application Service 2.13 unveils an improved Log Cache, which has been separated into its own virtual machine instance for enhanced scaling options. Historically, Log Cache has been colocated on Doppler virtual machine (VM) instances in order to reduce the footprint of foundations. This separation is critical as Log Cache is no longer subject to the formerly imposed Doppler maximum of 40 VM instances and can continue to scale up based on platform and application requirements.
The History of Log Cache
Log Cache provides an in-memory caching layer for logs and metrics which are emitted by applications and system components in VMware Tanzu Application Service. Each foundation caches logs and metrics in Log Cache for a period of time in order to:
-
Calculate metrics for autoscaling.
-
Allow developers to view recent logs for their applications:
—To ensure that their app is running.
—For troubleshooting purposes if their app has crashed. -
Allow developers to view recent metrics for their applications:
—Exposed via Cloud Foundry Command Line Interface (cf CLI) for CPU/memory.
—For custom metrics using the Log Cache cf CLI plugin. -
Allow platform operators and developers to quickly retrieve system component metrics.
Customers with large production foundations have encountered situations in the past where the Log Cache receives over 10–20 million logs and/or metrics per minute, overwhelming the component to an extent where it has trouble providing those functions. Under such circumstances, decoupling the Log Cache from the Doppler will help Tanzu Application Service customers with large foundations to scale Log Cache independently of Dopplers, thereby providing them more caching capacity at scale.
More Scaling Options for Log Cache
A Log Cache virtual machine instance separate from Doppler is uncoupled from the artificial horizontal scaling limits previously imposed, and can now be scaled to any arbitrary number of instances. Each added instance increases the total memory available to a foundation’s Log Cache to store logs and metrics.
RAM-dependent Log Cache virtual machines and CPU-dependent Doppler virtual machines can also now be vertically scaled to match their separate purposes.
Less Network Hops with Syslog Ingress
Syslog ingress to Log Cache provides a faster, more efficient way to stream logs and metrics from Diego Cells and other component virtual machines to Log Cache. This feature has been available for some time, but has been turned on by default in Tanzu Application Service 2.13.
Additional Configuration Options for Logging
Tanzu Application Service offers the following options to configure system logging:
Syslog and Aggregate Drains:
Customers can configure syslog drains for individual applications as well as aggregate drains for all applications in order to stream logs to the required destination. The “Enable Log Cache Syslog Ingestion” option, which is enabled by default in Tanzu Application Service 2.13, mostly turns the Log Cache into an aggregate drain destination. Syslog agent communication is the basis of the forward-looking shared-nothing architecture, which gets rid of all the intermediary network hops, as well as standardizes the logging format.
Advanced Platform-Wide Configurations:
By default, the maximum number of envelopes stored in Log Cache per source is set to 100,000. Operators can set this to a higher limit in order to not drop logs for noisy apps as quickly. Customers can optionally configure the maximum number of log lines an app instance can generate per second in Operations Manager under App Containers > App Log Rate Limits.
How Do I Get the Improved Log Cache?
Tanzu Application Service customers can upgrade to Tanzu Application Service 2.13 in order to benefit from the scalability improvements introduced in the Log Cache. These improvements enable dependent systems like the App Autoscaler to harvest metrics more reliably in order to make better scaling decisions for their mission-critical applications.
Check out the release notes for more information on the standalone Log Cache.
Explore more great features of Tanzu Application Service 2.13 here.
Download Tanzu Application Service 2.13 here.
