The problem: Improperly escaped post data
I recently worked on an app that processed xml files. Once a week, a legacy system posted a large xml document to the app. For almost a year the app worked perfectly, and then we updated to rails 2.3.2 and the posts started failing spectacularly. Looking at the log files, I noticed that the params were incorrect:
<code>{"message"=>"hello", "xml"=>"<xml>Foo &", "Bar</xml>"=>nil, "action"=>"not_scrubbed", "controller"=>"examples"}</code>
After looking into it further, I realized that the data that was being posted contained semi-colons:
<code>xml=<xml>Foo %26amp; Bar</xml>&message=hello</code>
It turns out that rails used to only split params on ampersands, but that rack splits on both ampersands and semi-colons. We couldn’t change the legacy system, so we had to remove the semi-colons before the post params got to rails.
The solution: Rack middleware
Using Rack middleware it’s was easy to insert code before rails params parsing code executed. To start, build a class that conforms to the signature of a rack middleware layer, like so:
<code>
# lib/scrubber.rb
class Scrubber
def initialize(app, options)
@app = app
@routes = options[:routes]
end
def call(env)
scrub(env)
@app.call(env)
end
private
def scrub(env)
return unless @routes.include?(env["PATH_INFO"])
rack_input = env["rack.input"].read
params = Rack::Utils.parse_query(rack_input, "&")
params["xml"] = Rack::Utils.unescape(params["xml"])
env["rack.input"] = StringIO.new(Rack::Utils.build_query(params))
rescue
ensure
env["rack.input"].rewind
end
end
</code>
Then register the middleware from environment.rb:
<code>
config.middleware.insert_before ActionController::ParamsParser,
"Scrubber",
:routes => [ "/examples/scrubbed" ]
</code>
To verify that this works, use curl to send the request, like so:
<code>curl -d 'xml=<xml>Foo %26amp; Bar</xml>&message=hello' http://localhost:3000/examples/scrubbed</code>
I’ve put together a sample app on github that gives a working example of the code above which you can find at http://github.com/zilkey/params-scrubber/tree/master.
About the Author
