Pivotal Labs
Also in this series:
We introduced Cloud Foundry’s new “standalone” applications feature in the first post in this four-part series. In this second installment, we will look at the most common use of a standalone application–the worker process. Workers can be used for all kinds of asynchronous background jobs, such as updating search indexes, emailing all users with a password reset approaching, performing a database backup to persistent storage, or uploading new customer data from external storage. In this post, we will walk through an example of deploying workers to Cloud Foundry using Resque.
Resque Workers on Cloud Foundry
Let’s start by cloning the Resque Demo Example.
mycomp:dev$ git clone git://github.com/defunkt/resque.git mycomp:dev$ cd resque/examples/demo
source "http://rubygems.org" gem 'sinatra' gem 'resque' gem 'rake' gem 'json'
We’ll run “bundle install” and “bundle package” to package the gems in vendor/cache, and we’re ready to deploy. The resque server is a Rack app, so we’ll deploy it to Cloud Foundry as such.
mycomp:demo$ vmc push resque-server Would you like to deploy from the current directory? [Yn]: Detected a Rack Application, is this correct? [Yn]: Application Deployed URL [resque-server.cloudfoundry.com]: Memory reservation (128M, 256M, 512M, 1G, 2G) [128M]: How many instances? [1]: Create services to bind to 'resque-server'? [yN]: y 1: mongodb 2: mysql 3: postgresql 4: rabbitmq 5: redis What kind of service?: 5 Specify the name of the service [redis-2a462]: redis-work-queue Create another? [yN]: Would you like to save this configuration? [yN]: y Manifest written to manifest.yml. Creating Application: OK Creating Service [redis-work-queue]: OK Binding Service [redis-work-queue]: OK Uploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (21K): OK Push Status: OK Staging Application 'resque-server': OK Starting Application 'resque-server': OK
mycomp:demo$ mv manifest.yml server-manifest.yml mycomp:demo$ vmc push resque-worker Would you like to deploy from the current directory? [Yn]: Detected a Rack Application, is this correct? [Yn]: n 1: Rails 2: Spring 3: Grails 4: Lift 5: JavaWeb 6: Standalone 7: Sinatra 8: Node 9: Rack Select Application Type: 6 Selected Standalone Application 1: java 2: node 3: node06 4: ruby18 5: ruby19 Select Runtime [ruby18]: Selected ruby18 Start Command: bundle exec rake VERBOSE=true QUEUE=default resque:work Application Deployed URL [None]: Memory reservation (128M, 256M, 512M, 1G, 2G) [128M]: How many instances? [1]: Bind existing services to 'resque-worker'? [yN]: y 1: redis-work-queue Which one?: 1 Bind another? [yN]: Create services to bind to 'resque-worker'? [yN]: Would you like to save this configuration? [yN]: y Manifest written to manifest.yml. Creating Application: OK Binding Service [redis-work-queue]: OK Uploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (0K): OK Push Status: OK Staging Application 'resque-worker': OK Starting Application 'resque-worker': OK
So we’ve pushed resque-worker as a standalone app with a Ruby runtime. We gave the command “bundle exec rake VERBOSE=true QUEUE=default resque:work” to start the worker. It is recommended to use bundle exec to ensure that all required gems are available. Since resque-worker does not have a web front-end, we selected “None” for URL. Lastly, we bound the app to the same Redis service used by resque-server. If you’ve perused the resque demo example, you may have noticed that it is setup to connect to a local Redis service. However, we didn’t change the code before we pushed it. How will the app connect to the provisioned Redis service? Since we used the Ruby runtime provided by Cloud Foundry, the app will benefit from the new Ruby auto-reconfiguration support. Cloud Foundry will automatically replace the local Redis connection with a connection to the Redis service we bound to the application!
mycomp:demo$ vmc logs resque-worker
====> /logs/stdout.log <====
Loading Redis auto-reconfiguration.
*** Starting worker ubuntu:10245:default
Auto-reconfiguring Redis.
*** got: (Job{default} | Demo::Job | [{}])
Processed a job!
*** done: (Job{default} | Demo::Job | [{}])
*** got: (Job{default} | Demo::Job | [{}])
Processed a job!
*** done: (Job{default} | Demo::Job | [{}])
mycomp:demo$ vmc instances resque-worker +2 Scaling Application instances up to 3: OK
And there you have it! We can now deploy Resque workers as standalone apps on Cloud Foundry. Clone the Cloud Foundry resque-sample and try it out for yourself!
Conclusion
Cloud Foundry now provides improved Resque support through standalone applications, as well as support for other Ruby worker libraries or apps. If you can package all the bits and provide a start command, you can run it on Cloud Foundry! In the next installment in this series, we will explore another example of workers in action using Spring integration. Stay tuned!
– Jennifer Hickey The Cloud Foundry Team
Don’t have a Cloud Foundry account yet? Sign up for free today
About the Author
