Question

I have some written a number of unit tests that test a wrapper around a FTP server API.

Both the unit tests and the FTP server are on the same machine.

The wrapper API gets deployed to our platform and are used in both remoting and web service scenarios. The wrapper API essentially takes XML messages to perform tasks such as adding/deleting/updating users, changing passwords, modifying permissions…that kinda thing.

In a unit test, say to add a user to a virtual domain, I create the XML message to send to the API. The API does it’s work and returns a response with status information about whether the operation was successful or failed (error codes, validation failures etc).

To verify whether the API wrapper code really did do the right thing (if the response indicated success), I invoke the FTP server’s COM API and query its store directly to see if, for example when creating a user account, the user account really did get created.

Does this smell bad?

Update 1: @Jeremy/Nick: The wrapper is the focus of the testing, the FTP server and its COM API are 3rd party products, presumably well tested and stable. The wrapper API has to parse the XML message and then invoke the FTP server’s API. How would I verify, and this may be a silly case, that a particular property of the user account is set correctly by the wrapper. For example setting the wrong property or attribute of an FTP account due to a typo in the wrapper code. A good example being setting the upload and download speed limits, these may get transposed in the wrapper code.

Update 2: thanks all for the answers. To the folks who suggested using mocks, it had crossed my mind, but the light hasn’t switched on there yet and I’m still struggling to get my head round how I would get my wrapper to work with a mock of the FTP server. Where would the mocks reside and do I pass an instance of said mocks to the wrapper API to use instead of calling the COM API? I’m aware of mocking but struggling to get my head round it, mostly because I find most of the examples and tutorials are so abstract and (I’m ashamed to say) verging on the incomprehensible.

Answer 1

You seem to be mixing unit & component testing concerns.

• If you’re unit-testing your wrapper, you should use a mock FTP server and don’t involve the actual server. The plus side is, you can usually achieve 100% automation like this.
• If you’re component-testing the whole thing (the wrapper + FTP server working together), try to verify your results at the same level as your tests i.e. by means of your wrapper API. For example, if you issue a command to upload a file, next, issue a command to delete/download that file to make sure that the file was uploaded correctly. For more complex operations where it’s not trivial to test the outcome, then consider resorting to the COM API ‘backdoor’ you mentioned or perhaps involve some manual verification (do all of your tests need to be automated?).