We have the following setup:
- Visual Studio 2012 / SQL Server 2012
- TFS 2010, Build Server runs unit tests automatically on build.
- A solution with lots of code and 2 Test Projects (so 2 separate assemblies at build time), the tests are done with Visual Studio’s unit test framework.
- The first assembly contains all actual unit tests, and they all run ok. Within most tests, we write some data in the db, run some code,
then clean the data. - We use the second assembly for database seeding. Since the tests and assemblies are loaded/executed in alphabetical
order, we have a test in the second assembly whose name starts with
“Z” (the assembly name also starts with “Z”). The build server logs
also confirm that the Z test is the last to be executed. So this test
only writes some data in the db. We preferred this for seeding versus
using a DB project due to an existing infrastructure which
reads data from Excel files and writes to SQL Server.
The problem is that when we run all tests on our dev machines, the Z test is executed last, and the seed data remains in the database. When the tests are executed on the build server, the Z test is also executed last, but the data doesn’t remain in the db.
According to the SQL Server profiler, the inserts/create tables are executed, so the data gets in the db both on our machines and on the build server, but on the build server some delete/drop queries are executed right after the inserts. We couldn’t tell where they are coming from.
We tried running only the Z test assembly and the data remains in the database. So the drops must be caused by the other assembly somehow. But how? And why is there a difference between the ways visual studio and the build server run the tests?
Has anyone encountered something like this before?
We managed to fix it in the end. It turns out that the first assembly had some cleanup code that ran in a [ClassCleanup] method. The first assembly’s ClassCleanup code ran after the Z test (from the second assembly). So the quick fix was just to replace the Z seeding test with an [AssemblyCleanup] method which did the seeding.