Here’s my problem. I built a web app, and naturally kept the data in a database which describes that app’s domain. Afterwords, I built another web app for the same organization, and used a seperate database to describe that app’s domain and store data… and naturally a couple more projects came up and for each app I’ve isolated it’s data to a single database. Deveolpment wise, I think it’s ok, as I can maintain changes to the data structure and data at the app’s database.
Considering these apps belong to the same organization, there tends to be plenty of data replicated between them, like department names, job titles, shop names, etc. Most of these tables hold the same data, but are not exactly the same in each database, and are not always used by all of the apps. Changes to this data, though, needs to be changed at all the apps (sometimes in a diferent ways) creating a growing management “hassle”.
So I’ve been think of a way to get some syncronization between the data. I want an easier management – update at one app (or a central app) and update all the databases as needed by each app – and also a better way to share data between apps (like maybe mash up data from differnt apps in a new app to alow specific analysis). Most of the data I’m refering to is used as contraints more than being core domain concept, describing the organization rather than describing a particular domain.
I’m looking for opinions on some ways to get this done.
My first idea was to grab comun data structures, like the department names’ table i mentioned, and stick’em in a core database. Any updates to the data would be done at this database, through a dedicated web app, and I’d apply some sort of Observer or Publisher / Suscriber Pattern for these changes – on changes the app would notify observing apps (through there dedicated webservice) that the changes occured and allow for the app to grab the new data and use it as it needs. GUIDs could be user as a reference to identify the same data throughout the apps. Also, I could build web services for read and search operations that don’t need to be in a specific app’s database, but could be useful to it.
A second idea would be that each app manage it’s own data, and the apps could observe one another. A change in one could notify others that share the same data structure that the change occurred. I could still use some GUIDs and even build services on any of the apps. I think this would also be less excessive in terms of duplication of data, but might be harder to manage as each app would eventually be coupled to other apps, and I would some how have to distribute responsabilities as to which app controls what information.
I’m really curious as to something of this genre of data distibuition and syncing would work and even be recomended. Opions and other ideas are more than welcome!
What you describe here is a typical case for a “Master Data Management” system. EAI vendors (Oracle, TIBCO, IBM) offer such products. They resemble your first solution, being centralised databases with synchronization processes, detecting changes in external data sources, grabbing the changes and synchronizing data out to other external databases. They also provide a user interface to change master data directly.
MDM software are expensive, but you can implement a custom solution which will be – at least initially – cheaper than purchasing one. Both of your solutions make technical sense but there is a difference in their manageability.
The first one is better, if you can dedicate a responsible person/organization to take care of it and the business owners of your services can agree on making changes via this new centralised system.
The second solution shares the responsibility between the service owners. The hard task here is to identify the owner of each type of information (business object).
I cannot advise a solution without a deeper knowledge of your systems and organizations, but I hope I could give some ideas.