I have been given the task of bringing three legacy systems together into one user interface. This will be an Asp.Net Mvc application.
I have a Sql Server 2005 instance on one server, a Sql Server 2008 instance on another, an access database that holds compliance data and is populated through a custom plugin, and a Powerflex dat file database accessed through odbc.
For every user who accesses this new interface all of these databases need to be queried. One of the Sql Server databases and the Powerflex database has millions of records.
My question is what is the most efficient way to handle this situation?
Do I link the Sql Server databases and write a single query with joins for those servers?
Do I use disconnected in memory datasets?
Do I use minimalistic queries with a data reader?
Do I attempt to utilize the Entity Framework (I haven’t looked into a connector for the Powerflex database)?
I have never attempted to bring this many back ends together before and I am concerned about performance. A minimum of four round trips screams poor performance to me without ever writing a line of code. Any tips would be appreciated.
PS: Bringing them all together into a single database is out of the question at this time.
All the things you suggest in your question have good potential for simplifying your code, making it more readable, or easier to maintain. However, none of them will affect performance in any way, simply because you will still have 4 different physical data connections (even a linked server definition from SQL 2005 to 2008 or vice versa will not help with that).
To get any real performance benefits you will have to try and consolidate the data somehow. For example:
If you can do both of those things, you will end up with only 2 physical data connections to worry about (SQL 2008 and Powerflex). You can then optimise data access manually on a case-by-case basis. For example, if you are joining resultsets from both data connections, execute the one that is likely to return the least number of rows first, then use the results of that to narrow the search criteria for the other query.