I recently installed CouchDB, but haven’t gotten past the test suite yet. It seems that each time I run it, I get different errors than before, even without configuration changes.
The CouchDB Guide says the test suite shouldn’t have “an inordinate number of failures”, but what is “inordinate”? How many errors should I tolerate before I start developing?
For reference, here are the errors I’m seeing.
When installed from Ubuntu repository (Runs 1 & 2):
- oauth
- replication
- rev_stemming
- security_validation
- view_conflicts
Then remedied by removing, then installing from source, as suggested here:
Run 3
- attachments
- jsonp
Runs 4 & 5
- attachments
- auth_cache
- jsonp
- list_views
- rev_stemming
Run 6, after clearing cache, then restarting couchdb
- No errors!
If each test passes individually, you are all set to proceed with development.
The test suite is not as reliable as we’d like due to its dependence on the javascript engine inside the browser and various timing and other issues. It is mostly used to help the developers validate a release.
We are planning to improve this situation in future releases, most likely by moving the test suite out of the browser entirely and running it from a known environment.
If you do want to see all tests pass, then be sure you are using FireFox (the only browser we officially support for running the test suite itself). You might also find it necessary to clear your browser cache too.