We look at the new GemFire for Pivotal Cloud Foundry Service that is now available and delivering one of the market’s most powerful in memory technologies on Pivotal’s open Cloud Native application platform. In this week’s episode, we explore what it can do and how you might use it.
PLAY EPISODE
SHOW NOTES
- Subscribe to the feed
- Feedback: podcast@pivotal.io
- Links referred to in the show:
TRANSCRIPT
Speaker 1:
Welcome to the Pivotal Perspectives podcast. The podcast at the intersection of Agile, Cloud and Big Data. Stay tuned for regular updates, technical deep dives, architecture discussions, and interviews. Now let’s join Pivotal’s Australia and New Zealand CTO, Simon Elisha, for the Pivotal Perspectives podcast.
Simon Elisha:
Hello, welcome back to the podcast. Great to have you back. Thank you so much for listening. This week’s episode is from the road. I’m in the beautiful city of Sydney visiting some customers and visiting the team, and just generally doing the stuff that has to be done. This week I thought we could talk about a new service that has just gone GA on Pivotal Cloud Foundry. This is the GemFire service. We’ve spoken about GemFire in the past on previous podcasts, but I’ll redirect what it is. What this is bringing to the table is the ability to deploy GemFire automatically from within the context of Pivotal Cloud Foundry. Why is this important? This is important because we want to give developers an efficient, clean, self-service experience when they’re consuming services for their applications.
We want them to be able to grab dev, test, production environments to create different sized environments, to create the service profiles that they need to support what their application is doing, without having to speak to anyone. No service tickets, no emails, no phone calls etc., it’s all done directly via a platform. This is one of the big things that gives more agile development teams the speed that they need, because they don’t get blocked by having to wait for particular resources. On the operations side, the ability to run this service from within the context of Pivotal Cloud Foundry, means that it takes care of a lot of the automation using a lot of the magic of BOSH, again, something we’ve spoken about in the past quite a lot. Again, what is GemFire? For those of you who haven’t been following along at home, it’s a high performance in-memory database with an event-oriented persistent layer.
What it is used for, is really high performance transactions. Things that need to work really fast. Let me give you an example. We have customers who are operating hundreds of nodes to support 40,000 visits a second on websites, all storing terabytes of data in memory, all handling 120,000 concurrent bookings in the one component. What GemFire does, is solves a raft of different data persistence and data access problems that tend to bedevil a lot of modern internet architectures. If you consider traditional architectures that are disk-based, and even if it SSD-based, they cannot achieve the low latency and performance of the in-memory store. There is way more complexity with an in-memory store, because now you’re trying to look after something that is by definition, volatile.
You need to have effective clustering, you need to have resilience, you need to have performance management, performance monitoring, you need to be able to route traffic, etc. The list goes on. It becomes a non-trivial problem. Let me put it this way. You don’t want to have to build it yourself. GemFire delivers a Raft of capabilities that mean you don’t have to make as many trade-offs as you would have had to in the past. Let me run through some of those and then we’ll talk about how you deploy it on Cloud Foundry. The first thing it brings to the table is the ability to have data consistency. If you have a transactional component of your application that needs to support ACID consistency, so that’s atomicity, consistency, isolation and durability, you can configure GemFire to support ACID consistency across distributed nodes using high performance transactions.
This is really bringing together two almost clashing design constraints into one and making it work. It becomes very important for those types of applications. In terms of performance, in-memory data storage is generally the fastest approach that you can take, because it’s not hitting anything that’s got a lot of latency. One of the other things around performance is not just read and write-type effect, but the ability to be event-oriented. To do things based on things that happen in the data set, or when the data set changes. One of the things that GemFire brings to the table, is something called continuous queries. These are queries or sets of event logic that run in the database itself, in the RAM, and can automatically notify using event-driven architecture, the application that’s consuming that data.
Let’s say you’ve got a training application and you’ve got an application alert that needs to listen to when the S&P 500 goes above a certain amount. As soon as that item changes in the in-memory data store, updated by a number of different sources, the event will be triggered and the application will take place. This becomes really important. As I mentioned, availability typically is very important in these types of applications because they’re typically transactional by nature, and often revenue-generating by nature. We need to have the ability to have multiple nodes so that we can cluster, and being able to have geographic distribution as well, can add an additional layer of protection. Scalability is almost, I want to say a given, or a forgotten component, but again, it’s one of those really hard things.
One of the nice things about GemFire is, it is horizontally scalable, so you can add additional nodes and it will automatically and transparently partition the data across those nodes in the most effective way. The application does not have to be data locality or where to make it work. All these things are really cool. What have we done with this service on PCF, is, it is now available as a service. That means you can install it on the Pivotal Cloud Foundry platform and you can automatically deploy GemFire in itself onto Cloud Foundry. What this brings to the table is the ability to have three different service plans and create a pool of instances for GemFire. Developers can draw down on those instances and bind them to their applications.
The service instances themselves are single tenant and they run on dedicated VMs that are fully configurable, so it means that you know you’re getting the absolute exact performance you need from that GemFire cluster for your particular application. There’s also an interesting CLI plugin that allows you to directly manage the service instances. You can do things like cluster configuration, restarting the cluster, accessing the logs, the statistics etc., and there’s a really good GUI for managing GemFire called Pulse, that allows you to share what’s going on in the environment. In fact, there’s a very good video that shows that whole set up process and I will link that in the show notes. What this means is that you can deploy this service on any of the platforms you want to deploy onto. You can have it on vSphere, on vCloud Air, on Amazon Web Services.
The platform itself looks after the virtual machine health monitoring. It’ll replace any failing processes inside the GemFire virtual machines. It replaces the VMs themselves automatically. It really takes the pressure off the operations team and it allows the developers to get access very, very quickly to the services that they need. This essentially becomes a tile in the PCF marketplace that a developer can access. Really excited this is out. This is version 1. As with everything that we do, we’re constantly iterating and improving the services and the capabilities of our software, so at the moment the 1.0 version supports a single Pivotal Cloud Foundry availability zone. You can have HA features like redundancy and persistence but these will be spread across a single availability zone rather than multiple at this stage. That will change in the future. The other thing you’ll see coming in the future is the WAN replication capability.
That’s not in version 1, but that will be coming in the future as well. You’ll see a raft of other components coming in. Remember that GemFire is built off the open source Apache Geode Project, which is the GemFire code that we contributed into the community, so again you’ll see this component evolving and improving and changing all the time. It really is a very powerful technology solution. That’s a very quick update on the new GemFire for Pivotal Cloud Foundry Service. It’s available on PivNet if you’re a customer. You can download it and start using it and see what you think. Thanks so much for listening. Look forward to hearing from you soon, and until then, keep on building.
Speaker 1:
Thanks for listening to the Pivotal Perspectives podcast with Simon Elisha. We trust you’ve enjoyed it and ask that you share it with other people who may also be interested. We’d love to hear your feedback, so please send any comments or suggestions to podcast@pivotal.io. We look forward to having you join us next time on the Pivotal Perspectives podcast.
About the Author
