Developing modern software emphasizes loosely coupled services, resilience, ease of scaling and the like. But these attributes do not “come for free”. How do you handle things like service discovery, configuration management and the tolerance of service failures?
Fortunately, services to do just that for Spring applications are now an “out of the box” part of Pivotal Cloud Foundry® (PCF). In this episode, we dive a little deeper into each service.
PLAY EPISODE
SHOW NOTES
- Subscribe to the feed
- Feedback: podcast@pivotal.io
- Links referred to in the show:
TRANSCRIPT
Speaker 1:
Welcome to the Pivotal Perspectives Podcast, the podcast at the intersection of Agile, Cloud and Big Data. Stay tuned for regular updates, technical deep dives, architecture discussions and interviews. Now lets join Pivotal’s Australia and New Zealand CTO Simon Elisha for the Pivotal Perspectives Podcast.
Simon Elisha:
Hello everyone and welcome back to the Podcast, so glad you could join me today. Coming to you from the road so the audio is going to be a little different, I’ve been in Canberra, the nations capital today as well as Sydney. So I am away from home base at the moment and have failed to bring my super-special microphone so we are going with what the laptop has and we are just going to roll with it. So today I wanted to speak a little bit about some services that can make developing software easier. Often we say that it is easy to see the future from the shoulders of giants and we are fortunate in the software development world that people like to share, particularly these days. I grew up in the days before open source, although before we even had open source we had shareware and good methods of sharing. But in the modern era now we have open source and it is a beautiful thing because it means that organizations are more than happy to share information with each other.
One of the areas that is being shared at the moment is how best to develop applications using microservices and the ability to decouple those applications and make them easier to deploy, easy to change, more resilient, more robust. All that good stuff that I am often banging on about. But you get nothing for nothing, so to have this loosely coupled lovely architecture, you need some connective tissue, you need to ability to send new configurations to it, you need the ability to tolerate failure. There’s stuff you need to do. Now often people look longingly, I should say, at the work they do over at Netflix and all the cool things that they have developed. But fortunately Netflix also wants to open source a lot of their stuff and their innovation and they share it with the community. We have taken that and adopted in the guise of Spring Cloud Services for PCF and this basically helps us encapsulate various projects. Like Spring Cloud Netflix, spring cloud configure, etc. and make them available in the PCF marketplace.
What this means is that you do not have to create and maintain your own managed services to have access to these projects, you just use them as dependencies in your application and you can use the annotations and it is nice and easy. So what I wanted to do is go through each one of the three current services available and explain how they fit and why you would use them and it is interesting because on the face of it some of them are quite “straightforward” you would think but their power when you deploy them correctly is immense. There are three different services available, there is the config server, the service registry and coolest one of all, the circuit breaker.
Now lets start with the config server, config server does something that you think is kind of obvious but is really important, as you are deploying an application you’ll have a need for it to pull different configurations and different components, particularly as it moves through deployment pipeline, from development to test production, etc. There may also be changes you want to do in real time, in the application without redeploying the application. So rather than having the configuration follow the application along in ever more complex YAML files that are baked into the repository of the application. You can create what is called a config server and have the client automatically call that config server to pull the configuration.
What this means is that you can foresee push new configurations under source control into the config server. The config server that we provide supports git and subversion, what that means that when the application starts up will go look for the config server, pull the config and based upon the different tags available to it, so it could be in test production or a particular version tag or it could be a branch that you are choosing, etc. All those will decide where to get the configuration from. By these embedded in the config server the first thing you notice is the configs are consistent and correct through the different levels of the life cycle. It also means you can change them as necessary and they are also under source control and it also avoids the problem of little snowflakes out there that developers tend to create by putting little special configurations in the YAML files that do not end up in the main branch.
So the config server encapsulates all that for you. Now obviously it works very well with Spring applications, but you can actually use this for any application in any language because it just talks HTTP, so it is very accessible. So that is the config server. The next interesting one is the service registry. Now the service registry deals with that whole concept of service discovery, as a producer it says hey, I am available to be called and as a consumer how do I find what I want to get to. Now many people have toyed with DNS and other low balancing technologies but they all sort of fall short to some extent, some of them fall short because they rely upon globally visible systems, others fall short because of non-resilience to failure, etc.
What we have with the service registry is a service based on Netflix Eureka service, discovery, survey and client. And this focuses very much on a nice, simple but elegant way to discover the availability of services to cache information about those services and to be able to rectify yourself in the event of services disappearing or the service registry disappearing as well because you have cached that information. So it is really about metadata, such as hosts, ports, etc., it also looks for regular heartbeat messages from each service instance. So if a service instant starts to consistently fail and you are not getting that heartbeat being sent to the registry, the registry will look after removing the instance from itself and letting everyone else know do not call that anymore. So again this is an example of connective tissue within your application for those loosely coupled applications. An application or component of application or a microservice should not need to know the specifics of the implementation of service that it is calling.
But it should be intelligent enough to know is it available, should I be calling it? Should I continue to call it? Should I call an alternative if it is down? What do I do? So that is the service registry, now this leads us to what I think is the coolest part of this, which is the circuit breaker dashboard. Now the circuit breaker dashboard is really a very cool component to allow you to continue operating in a graceful manner and it is based on the Hystrix project, and this is a latency and fault tolerance library that Netflix created. It really addresses a key problem. In a traditional monolithic architecture we kind of have this world where something in the application is broken, the whole application is down and that is not acceptable anymore. To be frank it has never been acceptable but it was always a kind of an accepted reality of the world in which we live, systems up or down they are kind of like a binary state situation.
Modern application when they are well engineered do suffer from this. Modern applications when well engineered leverage loosely covered services, often called microservices, that talk to one another. An intelligently designed application will be able to present a user experience that degrades rather than fails in the event of some back end issue. So lets say for example you are on a shopping site and the site comes up, and the site will typically call, potentially many hundreds of microservices when it is being presented to you. Now lets say that one of those services is down, let’s say that the recommendation engineer is broken for reason, cannot tell you that customers who bought this book also bought that DVD. Now in the “bad old world” or “bad old days” we would have this big gaping error, maybe a 503 error thrown right in the middle of the website. You have probably seen some of these sites where there is a big chunk of it missing, in an intelligently designed application instead of presenting the failing service, we detect that automatically and we say well lets present the next best thing. So maybe I do not present you with the recommendation engine results, I will present you with the current top 10 DVD sales.
So as a user you are still getting a valuable, interesting experience, you are not perceiving any failure, but failures are happening at the back end. So that is a long way to get to the circuit breaker dashboard, how do we detect whether a service is working or not working, and services are often not well behaved in the way they fail. But I sort of go, I am working, now I am not working, sometimes they will intermittently fail, they will kind of start sliding down, they get a bad result going on and you get bad user experience. The circuit breaker dashboard tracks and manages whether calls are happening appropriately between a service and its consumers and it monitors the number of failures. If failures reach a certain threshold they will open the circuit, which means it will automatically redirect all the calls to a specified fall back mechanism. So that could be returning some default value, could be calling some other service, etc. It essentially says, hey I am calling this service and most times it is failing or a significant number of times it is failing or all the time it is failing, I will do what is logical, I will automatically stop calling the service. What a concept, what an idea.
Now the benefit of this is it gives the failing service time to recover, so you can first monitor it and hopefully you have got good monitoring in place that says, “hey there is something wrong here”, and you can fix the service at which point you can reopen the circuit and allow access to it again. This concept, this circuit breaker concept is super important because it really decouples the applications and allows them to intelligently call one another on an ongoing basis, adapting to the current transaction load and the work load and the state of the different services. Again a very basic foundational component that has to be there to make it work. The nice thing with this service comes with a handy dandy little dashboard and the dashboard is pretty cool the way it displays performance. So it shows you things like latency, shows you the number of calls, it shows you the number of errors, etc. It shows you whether the circuit is open or closed, I will link that in the show notes so you can have a look, but what it means is that you can see what is going on from a performance perspective for your service at any time, that gives you visibility, that gives you control, that gives you maintainability.
These are insights into your application from an operational perspective that you would not have previously in a more monolithic application, it was either working or not and if it was not working then you had to start diving in and figure out what was going on with production. This allows you to decouple components, measure what is working, measure what is not and you can spin that forward and imagine you are deploying a new version of particular service and we suddenly see latency going up. We can crack the circuit breaker, break that open and say “lets roll back to the previous version”, very easy to do if using Cloud Foundry and reconnect the circuit and away you go. So, again it allows you to tinker with things on the fly much more easily, which is what we want in modern highly scalable applications.
So those are the three components of our Spring Cloud Services for PCF, the config server letting you get configuration very easily. The service registry, finding the services you need and linking to them in a robust way and most importantly in my view, the circuit breaker dashboard. The ability to call and access services and degrade gracefully in the event of problems that inevitably happen in all environments. So take a look, I will link them in the show notes, there are code examples. If you are a Spring developer you should definitely be using this and if you do not develop in Spring you should be using the patterns anyway in your language of choice for many similar types of libraries out there. So I hope that was useful and until next time keep on building.
Speaker 1:
Thanks for listening to the Pivotal Perspectives Podcast with Simon Elisha, we trust you’ve enjoyed it and ask that you share it with other people who may also be interested. Now we would love to hear your feedback, so please send any comments or suggestions to Podcast@pivotal.io. We look forward to having you join us next time on the Pivotal Perspectives Podcast.
About the Author
