Edit #2: Does anyone have a good method of testing the “middle” of a client-server application where we can intercept requests and responses, fake the client or server as needed, and which provides self-documentation of the api?
Cucumber might be a good solution in many cases, but it’s not quite what I’m looking for. And this middle layer should be client/server implementation agnostic. (e.g., black-box).
Our client-server model is a ruby-on-rails server with a Flex client, using a RESTish interface with JSON as the data format. So anything the client posts to the server is usually a single JSON parameter. The server does it’s thing and responds with a pure JSON model.
We have standard rails testing on the server and we’re working on getting proper FlexUnit tests completed on the client (it’s a moving target). However, there’s a debate in my team about the effectiveness of the current testing model, since every change on the server seems to break part of the API. This is telling me that there is both a problem with API communication (between team members, self-documentation in code, etc..), and a lack of proper API sanity testing.
So I’ve been questioning whether we need to have a mock client for testing the server at a pure JSON level (without all the other complexities of a rich client), and possibly a mock-server for doing the same thing with the rich client. This would serve two purposes, to document the API and to provide more thorough testing of the API itself.
The reason there’s a debate is that the rails guy claims that the rails integration testing is sufficient for testing all the server requests, and the middle-ground testing environment would simply be redundant.
So the question here is, given our situation, how should be go about self-documenting the API, and how should we test the API itself?
EDIT:
We have routes like /foo/444/bar.js, but the parameters can be virtually any complex JSON string depending on the action, e.g.:
json={
"foo":{
"x":1,
"y":2
},
"bar":[1,2,3,4,5]
}
but besides manually-edited API docs, there’s no self-documentation. The rails controller often just deserializes and applies changes directly to the model. Would be nice to have common tests to tell us when it’s changed, and what’s expected.
I just started looking at this web functional testing tool called Maxq and I think it has the potential to solve your problem, Maxq acts as a proxy server between your web client and server application.
It sits on top of Junit so that means you could do proper unit testing for your API by asserting the behavior and responses of calls to your server app.
It basically captures and records all the requests you make from a web client and the responses you get back from a server, it also has the ability to generate test scripts of your request which you could use to play back and test on any server.
You should try it out http://maxq.tigris.org/