Question

I am working on a project where we are scoping out the specs for an interface to the backend systems of multiple wholesalers. Here is what we are working with,

1. Each wholesaler has multiple products, upwards of 10,000. And each wholesaler has customized prices for their products.
2. The list of wholesalers being accessed will keep growing in the future, so potentially 1000s of wholesalers could be accessed by the system.
3. Wholesalers are geographically dispersed.
4. The interface to this system will allow the user to select the wholesaler they wish and browse their products.
5. Product price updates should be reflected on the site in real time. So, if the wholesaler updates the price it should immediately be available on the site.
6. System should be database agnostic.
7. The system should be easy to setup on the wholesalers end, and be minimally intrusive in their daily activities.

Initially, I thought about creating databases for each wholesaler on our end, but with potentially 1000s of wholesalers in the future, is this the best option as far as performance and storage.
Would it be better to query the wholesalers database directly instead of storing their data locally? Can we do this and still remain database agnostic?
What would be best technology stack for such an implementation? I need some kind of ORM tool.

Java based frameworks and technologies preferred.

Thanks.

Answer 1

If you want to create a software that can switch the database I would suggest to use Hibernate (or NHibernate if you use .Net).

Hibernate is an ORM which is not dependent to a specific database and this allows you to switch the DB very easy. It is already proven in large applications and well integrated in the Spring framework (but can be used without Spring framework, too). (Spring.net is the equivalent if using .Net)

Spring is a good technology stack to build large scalable applications (contains IoC-Container, Database access layer, transaction management, supports AOP and much more).

Wiki gives you a short overview:

http://en.wikipedia.org/wiki/Hibernate_(Java)

http://en.wikipedia.org/wiki/Spring_Framework

Would it be better to query the wholesalers database directly instead
of storing their data locally?

This depends on the availability and latency for accessing remote data. Databases itself have several posibilities to keep them in sync through multiple server instances. Ask yourself what should/would happen if a wholesaler database goes (partly) offline. Maybe not all data needs to be duplicated.

Can we do this and still remain database agnostic?

Yes, see my answer related to the ORM (N)Hibernate.

What would be best technology stack for such an implementation?

“Best” depends on your requirements. I like Spring. If you go with .Net the built-in ADO.NET Entity Framework might be fit, too.