A famous business author once said, “Leadership is the art of giving people a platform for spreading ideas that work.”
Every minute of every day, Pivotal employees get to exercise this kind of platform.
And, we’re looking to hire.
Imagine if your job was to figure out how to make the Spring Framework talk to Hadoop YARN?
Well, that is Janne Valkealahti’s job at Pivotal. As an open source software engineer, he spends his days figuring out how to connect some of Pivotal’s open source software projects to big data, agile methods, PaaS, and cloud platforms.
In this post, we get to crack open his brain and ask a few questions.
Here is what poured out.
1. Let’s start with some housekeeping. How do we pronounce your name and where are you from?
Well, you are out of luck because my name cannot be pronounced correctly : – ). Actually, it is all said just like it is spelled. Janne is pronounced like JAN-NE and the last name is a mouthful, but it’s VAL-KEA-LAH-TI. I’m proud to be from Finland, a beautiful country with an amazingly gracious, well-educated, and self-sufficient people. If you are from anywhere else, you probably know us best from the word sauna.
2. Could you tell us a little bit about how you grew up?
Sure, I grew up in a small town and did what boys do—we played Finnish baseball and drove mopeds. I have a brother, and we had totally different interests. Of course, I was a nerd from the beginning. At college, I ended up studying chemical engineering but quickly shifted my interest to this new internet thing that people were talking about. After serving in the Finnish Army, my brother and I both moved from our home town.
3. How did you end up at Pivotal?
For 11 years, I worked in Helsinki for one of the big Finnish banks and was tasked with rebuilding the infrastructure monitoring system. We ended up choosing Hyperic, the open source monitoring solution, and I really got involved with the open source community. One thing led to another, and I ended up working for Hyperic, then SpringSource, and then VMware as the acquisitions unfolded. About 5 years ago, I moved from Helsinki to London—it’s where I live now. In 2013, I took some time to travel with my family and contributed some open source bits to Spring Android and to help run Spring on Hadoop YARN. Now, I work as an open source software engineer for Pivotal.
4. What is your role and what are you most focused on at Pivotal?
Most of the Spring developers take a similar position when it comes to roles and responsibilities—regardless of the official title on our business cards, we are all here to make the whole Spring ecosystem better. It’s our job to constantly think of and experiment with new, better ways of doing things to make a developer’s life more awesome. Specifically, I spend time on the Hadoop side of our big data efforts with projects like Spring XD and Spring Hadoop and have a specific focus on Spring YARN.
5. How does the Spring YARN sub-project help people and companies do cool stuff?
In a nutshell, I am making it possible to place any Java application on Hadoop YARN’s massively parallel processing model. As a software engineer, this is really cool stuff. In fact, Paul Maritz, our CEO, talked about YARN indirectly in a recent interview, “If you abstract far enough away from Hadoop, you realize that Hadoop is really an instance of that paradigm. It’s a framework for exploiting lots and lots of cheap machines and lots of lots of cheap storage.” YARN is a key part of that framework.
We are still on the road to GA for Spring YARN, but we are trying to bring the whole Spring programming model to a Hadoop world as I wrote about back in September. Hadoop’s YARN basically improves on past performance issues and can sometimes be referred to as MapReduce 2.0. However, MapReduce is really an application running on top of YARN’s distributed computing framework. You can actually have YARN manage other applications too, but it’s not easy to do. Developing a custom app for YARN means working with infrastructure APIs not developer APIs.
So, the Spring YARN sub-project provides the support to “handle the infrastructure so you can focus on application development.” There are basically three Spring components that mirror the YARN architecture and allow you to use Spring to do things like submit new application instances, list them, or kill them. It also makes testing Spring apps much easier. Recently, I explained how we can use Spring Boot and Spring YARN to deploy a custom application on a Hadoop cluster.
6. What do you love about being a Pivotal developer?
Besides the fact that we have open source solutions in our DNA, I am surrounded by people inside and outside the company who are really smart, love programming, and are really good at it. There are some real rockstars here across a huge variety of projects. As a software engineer, this is an amazing opportunity and a great environment and culture.
Also, we are such a small team. It allows us to shift focus very fast. We don’t spend years on different committees to figure out what we should do. The world around us is changing too fast, and we need to provide solutions for current needs not just attend to fixing yesterday’s problems.
7. Our Head of Products, Greenplum founder Scott Yara, said one of our strategies is addressing the developer. What does this mean for your product?
If your company creates software, information, or media—and all companies do—the creation of these things is centered on developers and software development processes. From my experience, developers sit back and think, “How the hell are we supposed to use these designs and plans to develop this app?” I think our stuff takes the “hell” out of the thought process and just makes it easier to develop and solve problems. With Spring YARN, there is no question it will be easier to develop distributed Spring applications that process in parallel.
8. How does your product work with Cloud Foundry PaaS and Pivotal One?
As you may know, Cloud Foundry is now its own independent foundation with tons of great companies involved like Accenture, GE, Intel, IBM, and many more. From day one, Cloud Foundry has built support for Java runtimes into its design and Spring apps are certainly expected to run on Cloud Foundry. In fact, Spring.io is an example of running Spring apps on Cloud Foundry.
As well, Pivotal One, makes our Pivotal MySQL, Pivotal RabbitMQ, and Pivotal HD services available on top of Pivotal CF (Cloud Foundry). For Spring developers, this is powerful for two reasons. One, RabbitMQ is based on AMQP and there is the Spring AMQP project. So, Spring developers get an easy way to implement message-oriented architectures and scale components. Two, Pivotal HD includes three things—Hadoop, a massively parallel SQL engine on Hadoop’s HDFS, and a real-time, in-memory SQL data grid with a two-way integration to Hadoop’s HDFS. For the Spring developer, they can use JDBC, Spring XD, Spring for Hadoop, and Spring YARN for data access.
So, there is basically a quite powerful, agile, scalable stack available to Spring developers—Cloud Foundry can deploy in a single command and scale like a cloud architecture should. With all the Pivotal pieces mentioned, it will run effectively for all types of big data apps—real-time analytics, mobile, internet of things, and stuff like that.
9. How can people get involved with Spring YARN or other projects?
Well, first, you can start to follow some of our folks or go visit a project and get involved with the community via our forums, Stack Overflow, Github, Jira, etc. Of course, Pivotal is also looking for good people.
10. What new Pivotal products are you most excited about working with?
Looking forward, I am really excited about where things are going with Spring Hadoop and Spring XD. Carving a path for the Java community to cross into the big data realm is just cool. It’s the type of stuff you dream of when you first start programming.
11. What is your favorite developer tool and why?
Well, I might say it’s the Eclipse platform, but I’m old school. So, it’s probably my Linux machine and its terminal windows.
12. What do you like to do in your personal time when you aren’t breathing Pivotal?
What? No one told me there is a life outside of Pivotal! Well, I really enjoy seeing and feeling nature. So, I am always up for doing a hike in unfamiliar areas. Once, I almost got lost on a cloud during a 6 hours hike up a mountain. My wife and I are really bad tourists—we like to stay away from all the usual tourism stuff. When we have enough holiday time, we do road trips and get off the beaten path to see real parts of a country where real people live. Certainly, travelling around Europe and Australia is wonderful, but one of the best trips I ever took was a 5-week trip from LA to NY.
13. What is top on your bucket list of things to do while still on this little rock we call earth?
In no particular order, visit Africa, run a marathon, skydive once, and see the world become a no-war zone.
About the Author
