Tuesday, May 24, 2011

Using Audit Logs for Data Integration

Recently we integrated two applicatons with Solr in near real-time: Siebel and Documentum. We achieved this by monitoring the Audit Logs of the applications. Audit logs work great for data integration where you have to push a change in a business object to another system. Audit logs for mature applications like Siebel and Documentum can be configured to write update/delete events. Usually, this is done by reading from the audit log table (without locking to prevent performance issues) to sniff out interesting events/changes to business objects.

Risks

  • The auditing mechanism must be bug-free and consistently record events - which is mostly the case with mature COTS software systems.
  • The audit log is generated by the application. So every-time you need to test your integration solution, you have to make changes in the application to see if they come through. Sometimes, developers don't have access to front-end apps which may cause problems. 
  • Also, thorough testing is required from the application front-end perspective. Care must be taken to capture all event log signatures a particular business action in the application  can generate.

No comments:

Post a Comment