Firehose at your fingertips
On the Logging and Metrics team, we recently found ourselves in a need to debug the CF Firehose more easily and we wanted to share that tool with all CF operators out there. In case you don’t know what the Firehose is, let me tell you: It is undoubtedly the most powerful feature added to the CF logging system in the last 2 years. That’s why its typically also restricted to be used by just the admin users. You know the saying ‘With great power comes great bragging rights" (or something like that).
What is the firehose?
In all seriousness, the Firehose can be very useful. It is a stream of all application logs and all component metrics from the CF components. Unfortunately, so far it has been really difficult to access (you had to write some code in go to get access to that stream). But with our brand new Firehose CLI plugin, all you have to do is install the plugin from the plugin repository and you are good to go.
How can I obtain admin access?
As mentioned earlier, you need either admin access or some special permission to access the Firehose. Fortunately, the Loggregator README provides good details on how to set this up.
How do I install the plugin?
Installing, the plugin is really simple.
Login to the CF CLI with your admin creds
cf api <your_cf_api_url> cf login
Add the plugin repository
cf add-plugin-repo CF-Community http://plugins.cloudfoundry.org/
Install the plugin
cf install-plugin "Firehose Plugin" -r CF-Community
How do I use the Firehose plugin?
There are two ways of using the plugin. There is an interactive mode and non-interactive mode.
The interactive mode requires some user input and will look something like below. You may select the type of messages you’d like to have streaming on your terminal by selecting an index. If you are curious as to why it doesn’t start with 1, look here and here
$ cf nozzle What type of firehose messages do you want to see? Please enter one of the following choices: hit 'enter' for all messages 2 for HttpStart 3 for HttpStop 4 for HttpStartStop 5 for LogMessage 6 for ValueMetric 7 for CounterEvent 8 for Error 9 for ContainerMetric > 5 Starting the nozzle Hit Ctrl+c to exit
You can also use the
--filter flag to have the plugin operate without user interaction. For example, if you want to filter the log stream to contain just counter events then you can run the command like this
cf nozzle --filter CounterEvent
If you want all logs and not automatically enter interactive mode, you can pass in the
--no-filter flag. This is the same as hitting enter in the interactive mode.
Share the load
So the Firehose has this nifty feature where the logs can be distributed across various sessions in order to reduce load. All you need to do is use the same subscription ID for each session. So what that means is, if you have two terminal windows open and you run the following command
cf nozzle --no-filter --subscription-id zaphod the load would be split evenly among the two sessions. Although having two heads didn’t seem to help Zaphod anyways.
In case you get any errors, just hold on to your towel.
|Unauthorized error: You are not authorized. Error: Invalid authorization||Just regenerate your auth-token like so:
|Unable to recognize filter HtttpStarrrt||Check your spelling|
|Invalid filter choice||Check your index choice. Enter a number from 2 through 9|
How we use this plugin?
cf nozzle -n primarily for debugging and acceptance purposes. It’s a quick way to look for expected component and application logs. Recently, our PM used this tool to look for logs originating from the DEA after pushing an app to PCF. Before this tool, he would have had to
- Setup his GOPATH and
go getthe NOAA library.
- Set up the environment variables CF_ACCESS_TOKEN and DOPPLER_ADDR.
- Follow instructions to build and run the firehose sample.