The path to production has long been a space of custom pipelines, continuous integration (CI) sprawl, manual intervention, and tribal knowledge. Surely, there must be a better way? Something loosely coupled, more flexible, less error-prone, and that doesn’t need deep integration with the tooling it controls.
These goals motivated us to create Cartographer, an open source supply chain choreographer. Note the word “choreographer” used here, as this approach is critical to achieving the flexibility and reliability required to be successful.
Orchestrated applications often rely on strictly defined RESTful endpoints to connect microservices, but this linkage can be brittle. Choreography and orchestration both have their place; however, choreography, with its event-driven approach, makes for a loosely-coupled application that can be more resilient to failures. Orchestrators work well within a boundary domain, but choreographers allow for greater flexibility and reliability.
So, why not apply that same event-driven approach to the entire application delivery pipeline and not just the application itself? What if we allowed the orchestrators to do what they do best—carry out individual tasks for processes—but use a choreographer to govern the application’s path to production?
In the VMware Tanzu Application Platform, we use Cartographer for precisely this reason. We want the path to production to be robust and flexible. We want to combat CI pipeline sprawl, where there are many slightly different versions of the same pipeline and where every application—or version of an application—needs a custom path to production.
You might wonder then, are supply chains a replacement for your existing CI? Well, no. Not exactly.
Let’s say you have a current Jenkins CI pipeline that builds your application into a binary artifact and stores it for you. You don’t need to throw that away and start again!
Supply chains don’t have to run your entire end-to-end development stack. You can easily take those pre-built binaries and have supply chains consume them and wrap the rest of the path to production tasks around them (e.g., scanning for common vulnerabilities and exposures (CVE), building and signing container images, and running them on your platform).
So you might now be asking yourself: “If the CI is already done, and the artifact delivered, what can supply chains offer?”
Supply chains allow you to codify the path to production for your organization, not just your application. Let’s take a look at an example:
If you think about it, you can probably abstract your pipeline’s work into a very similar set of high-level steps, and that process, from a SecOps perspective, should be the same across all pipelines. But pipelines are rarely implemented in this way.
Supply chains allow a platform team to have one or two supply chains that cater for all of their application delivery needs, even across multiple languages and frameworks!
And because supply chains are choreographed rather than orchestrated, they do not know how those steps work. All they need to know is what the input and output data from each step should be.
To make it more concrete, let’s take our above example and look at the “Test Code” and “Build Deployable Image” steps. These can be different across apps, as well as languages. So how can we support different apps and languages running through the same supply chain without causing a lot of duplication and sprawl?
The supply chain only needs to know that there are “Test” and “Build” steps involved. We can specify these as labels that can be applied to the application workload when it is submitted to the supply chain. Those steps can be swapped out by Cartographer depending on those labels.
You can see from the screenshot below that we have a single supply chain source-test-scan-to-url servicing both a Python and Java app, with their own testing and discrete image builds.
The labels applied to the workloads allow customizations to be made, depending on those labels.
Python apps would have the label "apps.tanzu.vmware.com/language":"python".
tanzu apps workload create python-myles \
--git-repo https://github.com/mylesagray/python-tap-app \
--git-branch main \
--type web \
--label app.kubernetes.io/part-of=python-pipenv \
--label apps.tanzu.vmware.com/has-tests=true \
--param-yaml testing_pipeline_matching_labels='{"apps.tanzu.vmware.com/pipeline":"test", "apps.tanzu.vmware.com/language":"python"}'
Java apps would have the label: "apps.tanzu.vmware.com/language":"java".
tanzu apps workload create java-web-app-myles \
--git-repo https://github.com/mylesagray/tanzu-java-web-app \
--git-branch main \
--type web \
--label app.kubernetes.io/part-of=tanzu-java-web-app \
--label apps.tanzu.vmware.com/has-tests=true \
--param-yaml testing_pipeline_matching_labels='{"apps.tanzu.vmware.com/pipeline":"test", "apps.tanzu.vmware.com/language":"java"}'
The testing configuration for that step is configured to look for the “language” label on all workloads that pass through the supply chain. The application testing method is then controlled by the language label (i.e., Java or Python).
Additionally, because Tanzu Application Platform uses Cloud Native Buildpacks, the “Build Deployable Image” step will automatically build a deployable container image suitable for the language and packages chosen, with no need for custom pipelines or Dockerfiles.
Implementing this approach in a traditional imperative (sequential, step-by-step) pipeline would be very cumbersome, error-prone, and difficult to maintain, particularly as the number of branches increase. Supply chains and choreography are much more than CI, or a replacement for CI. You can either use them as an additional layer to your existing CI jobs, or as a complete end-to-end solution by deploying new code straight from the source.
For a high-level look at supply chains in Tanzu Application Platform, check out my video.
If you want a deeper dive into supply chains and Cartographer, check out this awesome tutorial video by Cora Iberkleid and Corby Page.
Learn more on how VMware Tanzu Application Platform uses Cartographer at its core, and even try it out for yourself!
About the Author
More Content by Myles Gray
