I curious to how different people solve integration of systems. I have a feeling that the last years more and more work has gone into integrating systems and that this kind of work need will increase as well.
I wondering if you solve it developing your own small services that are then connected or if you use some sort of product (WebSphere, BizTalk, Mule etc). I’d also think it’d be interesting to know how these kind of solutions are managed and maintained (how do you solve security, instrumentation etc, etc), what kind of problems have you experienced with your solution and so on.
wow – Ok – will get a post on this but will be big.
Intergration needs to be backed up with a big understanding by the business on the benefits – Get an opertating model sorted out – as the business may acutally need to standardise instead of intergrate, as this can be costly – its why most SOA fail! Enterprise Architecture: Driving Business Benefits from IT Author(s): Jeanne W. Ross
If intergration is needed you then need to settle on type of integration.
What are the speed and performance metrics?
We have a .NET SOA with a Composite Application that uses BizTalk 2006 and webservices with Line of Business Applications. Performance of the application at the composite end (consuming) – is limited to the speed of the webservices (and their implementation) in the line of business application! We need sub <3 second return on results – list of cases. Could not be acheived in the webservices so we need to get go to the database directly for initial search. Then over the webservices for case creation. Cost implications and maintance becomes an issue here.
The point here is to look at the performance criteria in the specs and business requirements this will help in look at the type of integration that you need to do – WebServices (HTTP), File Drop/ EDI etc
Functionally for intergration you need to then look at the points of failure in the proposed architecture – as this will lead to a chain of responisblity in SLA/OLA. You may need to wrapper the intergration/faliure points into things that you control.
On similar point about integration with Line of Business is with how much do you need to know about the other product before you can integrate? Yeah Webservices are supposed to be design by contract but the implementation is often leaky and you need to understand alot about what is happening – and if this is a product that you dont control the abstraction even with webservices leaks into your intergation technology aka BizTalk.
Couple these two points together and you the best advise is to get a intergration hub type like BizTalk – wrapper the line of business applications in webservices you create – so the BizTalk side can be free from leaky abstractions then you also can reduce the points of failure as the you have decoupled the line of business application from the intergration hub and the point of failure to a single source rather than inside an orchestration.
Instrumentation and diagnosics in SOA and Intergation Porjects are hard to acheive! – Dont let any shiney sales person try and tell you differently! Yeah MOM with MOM Ent can do this UniCenter can do blah.
The main problem is understand what the error aka burps in the intergation mean and how to recover from them… You end up with messages stuck and you need to understand what that means to that busienss process. You can get an alert to say – processers are 100% Ram 100% orchestrations have failed – but no real meaning. You have to engineer this stuff in to the solution from the outset – and hopefully into you points of failure.
Types of intergration patterns and how to do them do need to be considered too.
The above is a real world view of a .NET SOA with BizTalk in a LIVE implementation. But it is also due to the architectural limitations of this – BizTalk mainly is a HUB and SPOKE pattern.
Check out Enterprise Application Patterns by Martin Fowler
There are many ways to skin the task!
Other considerations… Platform/Developer Languages etc.
One of the big factors for us was the skills needed to start this stuff. We had OO devs with Java and C# understanding, but mainly C#. So we went for the MS stack. But when you choose the intergration type and the product to manage this they will need more skills in understanding that technology. But hey this is normall for us Devs right? Wrong many developers regardless of there expereince can come unstuck with the likes of BizTalk! Big shift in paradigm – which in part is due to messaging shift rather than code.
Best bit for last!
Numbers of transactions that are likely to be faced in the integration is probable the single biggest factor in all of this. As this will guide what pattern, points of failure and tolarance for such things.
You need to select best on anticpated volumes the right one. Something that can scale up and scale out! We selected BizTalk since it can scale up and scale out correctly and with better understanding than some others.
If you dont have volumes then look at not getting something to manage them and go for a webservice to webservice type style with no management – performance and failure understanding will need to be coded into them.
If your on windows platform with .net 3 take look at WWF/WCF as this can help in webservice to webservice – lots more in the acutal platform now for all these concerns without the overhead of BizTalk and others.
Hope this helps.