Question

I have been building applications / web apps for some time, and I guess most of what I have built has been based upon simple principles, i.e. try to keep things loosely coupled, use industry standard techniques and so on.

I try to follow as many tech blogs I can about architectures, and how sites can scale (Facebook and the like).

Right now I am trying to gauge the requirements / components / tools for building a highly available site using .NET/C# running on Windows 2008 R2. As mentioned in the title, I am just trying to get a top level/30,000ft view of what the “parts” would be. For instance, I am (for arguments sake) not worried if I will be using SQL 2008 or mySQL for the datastore, and if mongoDB etc plays a part.

I guess what has prompted me to add this question is that I have been writing a few web apps which have run fine, but handle no where near the amount of users/actions as some of the social based websites. For instance, and this is very broad/overview stuff, the structure I have been using so far:

1. SQL 2008 backend datastore using Stored Procs
2. MS Enterprise Lib / Data blocks / caching etc forming the “business object” layer
   if you will.
3. ASHX pages to act as handlers for the frontend
4. JQuery / Javascript frontend – using XML (although some JSON recently) to get the
   data from the middle tier to the frontend

I realise the above is just a basic view, but it has worked relatively well.. the ASHX pages provide a lightweight means to interact with JQuery ajax calls, and so far the data access blocks in the Enterprise Library (version 5.0), make calling the DB nice and simple and generally fast.

SO WHAT AM I AFTER HERE…

Well, given the above, I sort of know that building some sort of social based web app (for arguments sake let’s say a very very mini version of facebook) needs to have certain fundamental structures in place like:

1. Caching (i.e. memcache etc?)
2. Queuing mechanisms, so you can hand off lengthy processes to background tasks
3. Fast data access techniques.. are stored procs the best way to go?

I think I read somewhere that stackoverflow.com etc uses .NET technologies.. and I guess what I am after is some starting points / reading material to apply to a .NET world (I realise there will be a lot of *nix people churning.. and I want to say I am not adverse to learning anything new, but right now my skill set is primarily .NET).

Hope this sort of makes sense, bit of a ramble, but feel free to comment for further information.

Thanks in advance.

David.

Answer 1

It sounds like you’ve already made a lot of the key architectural decisions – around technology, anyway.

I would say any platform / technology you choose can be made to work; just as any can be perverted and screwed up.

Stored Procs are good for speed – they’re also a good security measure.

I suggest you draw up your solution architecture formally – having it documented will help those who come after you (or yourself after time has gone by), and the process of documenting it will help your thinking process.

Key things to include:

• Architectural Goals. What are the things you want to achieve architecturally? These will probably align well with the high-level Non-Functional Requirements (NFR) set out by the business client. For example: are you aiming for high re-use of existing technology so that the business doesn’t have to diversify into new areas.
• Architectural Decisions. “We choose to use [technology / component / pattern] “x” because it’s well supported”. Or “we choose ‘A’ over ‘B’ because even though B is faster, A is more secure – which aligns with key NFR’s.

Once you’ve done this it should be easier to make decisions; sometimes you might already know the answer in your head – but it’s not that clear, writing it down can help.