Apache MADlib

Powerful open source library of scalable in-database algorithms for machine learning

Data is at the center of digital transformation—using data to drive action is how transformation happens. Therefore, it's important to efficiently extract patterns from data in order to identify the insights and actions needed. Machine learning algorithms allow organizations to not only identify patterns and trends in their datasets but also enables them to make high-value predictions that can guide better decisions and smart actions in near real time and without human intervention.

Apache™ MADlib® is an open source library for scalable in-database analytics. It provides data-parallel implementations of machine learning, mathematical, statistical, and graph methods on the PostgreSQL family of databases, including VMware Tanzu Greenplum®. MADlib uses the MPP architecture’s full compute power to process very large data sets, whereas other products are limited by the amount of data that can be loaded into memory on a single node. MADlib algorithms are invoked from a familiar SQL interface so they are easy to use.

Massively Parallel Machine Learning

The methods in MADlib are designed to run on shared-nothing, “scale-out” MPP architectures. This allows machine learning computations to be executed close to the data and at very high speed.

MADlib runs analytics on extremely large datasets. It differentiates itself from other analytical packages by not limiting execution to memory-only structures on a single computing node. MADlib users can add more nodes as data scales. Using all data and not a sample significantly improves accuracy.

Rich Portfolio of Analytical Methods

The MADlib community has steadily added new methods in the areas of mathematics, statistics, machine learning, graph analytics, and data transformation. The current library includes a comprehensive collection of algorithms, operators, and utility functions.

Extensive Support to Popular Data Science Interfaces

PivotalR is a R wrapper that allows practitioners who know R but very little SQL to leverage the performance and scalability benefits of MADlib. It translates R model formulas into corresponding SQL statements (via MADlib), executes these statements in the database or on-hadoop and returns summarized model output to R.