I’ve been experiencing grief trying to work with Ruby based tools to build a web-based server (in Part One) and a client (read on).
So. I have to write some Ruby code to act as a client to a remote server using HTTP GET/POST requests, where parameters are sent using the usual HTTP techniques. Just like a form in a browser would do. No XML. No JSON. Just good old HTTP.
I touched on one particular aspect of the server side of things in Part One.
For maybe nine months we’d been using the HTTP client built into Ruby but a few weeks ago started experiencing serious perfomance issues. If it’s bad enough to interfere with development then it certainly won’t work in production. So we had a look around and settled on Patron. Patron is a nice little gem wrapping libcurl. It is very fast, certainly fast enough for our purposes, has proved reliable, and easy to use. We’ve been using it in production, well, we were until this afternoon but the timing is co-incidental.
Patron has a flaw. (Yeah, I’m going to pick on Patron, even though I seriously doubt it is alone here.)
You see, Patron takes its post data in a hash map. Of course hash maps have unique keys, so if multiple values are to be sent you have to associate an array with the key. Or so we supposed. Yeah, we supposed wrong. Patron accepts that form of data but does something seriously wrong with it. It basically converts the array to a string, a string that looks exactly like an array looks when printed in Ruby. This means that multiple values are silently converted into a single value. Secondly that the single value looks just like the array would look when printed in a debugging log.
Patron then sends the data to the server, which, correctly, sees a single value. This is not how even the monkey patched Rack recognises multiple values (Rack parses the key looking for indicators of multiple values, not the value itself.)
So the server gets the wrong thing, and since one is normally a perfectly valid count for a multiple valued parameter, there are no errors reported and, once again, the debugging log looks perfectly okay. But nothing works.
Anyway, like I said, Patron is a wrapper around C and after a quick perusal of the code it looks as though it flattens the value in C not Ruby, and anyway I’ve never been successful monkey patching a method that maps to the Ruby FFI.
Off I go looking for an alternative.
We ended up using typhoeus. Very nice indeed, all the good things about Patron including wrapping libcurl, but it actually generates a proper post. And our client works now.
I’ve put some code in this gist that will demonstrate the issue. Have a look if you’re interested.