Are websites falling in social network category like LinkedIn, StackOverflow, Facebook, distributed applications ?
Does distributed in the web application’s context just means running on multiple web servers ? What are the most important design differences in distributed & non-distributed web applications ?
It depends what you mean by “multiple web servers.” If you mean multiple HTTP servers, multiple proxies, multiple databases, multiple caching mechanisms (memcached, Redis, etc), load-balancing, then yeah, sure.
There’s no way to answer this without more information (what kind of web application are you talking about here?). For one, you may be using different technologies, you’re going to be doing much more rigorous unit testing, you’ll (hopefully) write your code with fault-tolerance in mind.
Edit:
I believe that a website should be designed with expansion in mind. Again, it’s hard to deal in abstractions but I’m assuming a start-up here. You can make a site (for cheap) using “crappy” technologies (lets say PHP – flame suit on) that are harder to scale than, say, Java. You can use a “simple” SQL solution or you can try to combine pgSQL with something like memcached or Redis so when (and if) your company and web presence expand, you won’t have to redesign your entire subsystem.
I made this mistake with my first start-up and I think there’s some value in thinking ahead. Conversely, you don’t want to focus PURELY on “cool” technologies (“ooh, lets use memcached with a Redis cluster and Cassandra”), you also want to focus on the content.