I work as a tester for an organisation that has a web service as it’s critical application. Currently we load massive amounts of test data through the web front end, as this is how it would be done in the real world.
This gives the data an amount of legitimacy and prevents errors in the format of the data. However it is very time consuming to load the data in this way and I often wonder if loading data directly to the database would be more productive.
Have other people had this decision, which option did you opt for? Is there another solution that would give both speed and legitimacy of the data.
This comes from a developer’s perspective rather than a tester’s, so it may or may not apply.
I can’t speak for the organization as a whole, but in our project we have spent some time creating “real-world-like” data that we load into the test database using SQL scripts. This data is a combination of real data from the production environment and data that is tailored to represent specific “problem situations” in our product.
The scripts are run automatically as part of building our software and are used by automated integration tests, driven by a unit testing framework. These tests will test finding, creating, editing and deleting data through various interfaces that are available.
During such a build and test run the test database is reset and reloaded with data on a number of occasions. This is done in order to remove dependencies between tests; one test should not rely on data created or modified by another test, and also because the data for some tests might differ from that data of other tests. A majority of all tests are executed based on the same test data though.
Setting up this test data (and maintaining it) has been (and is sometimes) somewhat of a headache, but in the long run it has worked well in our case.