Question

I’m working on an Excel 2010 VSTO solution (doing code-behind for an Excel workbook in Visual Studio 2010) and am required to interact with a centralised SQL Server 2008 R2 data source for both read and write operations.

The database will contain up to 15,000 rows in the primary table plus related rows. Ideally, the spreadsheets will be populated from the database, used asynchronously, and then uploaded to update the database. I’m concerned about performance around the volume of data.

The spreadsheet would be made available for download via a Web Portal.

I have considered two solutions so far:

1. A WCF Service acting as a data access layer, which the workbook queries in-line to populate its tables with the entire requisite data set. Changes would be posted to the WCF service once updates are triggered from within the workbook itself.

2. Pre-loading the workbook with its own data in hidden worksheets on download from the web portal. Changes would be saved by uploading the modified workbook via the web portal.

I’m not too fussed about optimisation until we have our core functionality working, but I don’t want to close myself off to any good options in terms of architecture down the track.

I’d like to avoid a scenario where we have to selectively work with small subsets of data at a time to avoid slowdowns -> integrating that kind of behaviour into a spreadsheet sounds like needless complexity.

Perhaps somebody with some more experience in this area can recommend an approach that won’t shoot us in the foot?

Answer 1

Having done similar:

1. Make sure that all of your data access code runs on a background thread. Excel functions and addins (mostly) operate on the UI thread, and you want your UI responsive. Marshalling is non-trivial (in Excel ’03 it required some pinvoke, may have changed in ’10), but possible.

2. Make your interop operations as chunky as possible. Each interop call has significant overhead, so if you are formatting programatically, format as many cells as possible at once (using ranges). This advice also applies to data changes- you only want diffs updating the UI, which means keeping a copy of your dataset in memory to detect diffs. If lots of UI updates come in at once, you may want to insert some artificial pauses (throttling) to let the UI thread show progress as you are updating.

3. You do want a middle-tier application to talk to SQL Server and the various excel instances. This allows you to do good things later on, like caching, load balancing, hot failover, etc. This can be a WCF service as you suggested, or a Windows Service.