Integration-Architecture-Cookbook_V1.0_ENG.pdf

Integration Architecture Cookbook Architecture Document for Integration Projects Banco de Bogota, 2013

This document defines the Architectural Patterns and Guidelines related to Architectural Solution defined for Integration Projects of Banco de Bogota.

Table of Contents 1. Introduction ..................................................................................................................... 5 2. Architecture Style ............................................................................................................ 6 3. SOA reference architecture .............................................................................................. 6 Vertical Layers .................................................................................................................................... 6 Operational Systems ....................................................................................................................... 6 Service Components ....................................................................................................................... 6 Services ........................................................................................................................................... 7 Business Processes .......................................................................................................................... 7 Consumer Interfaces ....................................................................................................................... 7 Horizontal Layers ................................................................................................................................ 7 Integration ....................................................................................................................................... 7 Quality of Service (QoS)................................................................................................................. 7 Information ..................................................................................................................................... 7 Governance ..................................................................................................................................... 8 4. Architectural Solution for Banco de Bogota .................................................................... 9 Data Power - Externo ...................................................................................................................... 9 Data Power - Interno....................................................................................................................... 9 WebSphere Service Registry and Repository .......................................................................... 10 WebsphereMessageBroker ......................................................................................................... 10 WebSphere Business Process Server ........................................................................................ 10 Oracle RDBMS................................................................................................................................. 10 IBM Websphere Application Server........................................................................................... 10 5. Product Stack ................................................................................................................. 11 6. Message architecture ..................................................................................................... 11 7. Style of Communication ..................................................................................................... 13 8. Governance Architecture .................................................................................................... 14 9. Architecture of Monitoring.................................................................................................. 14 10. Architectural Guidelines ................................................................................................... 16 11. Components of Architectural Solution ................................................................................ 18 Data Power – Security Gateway for all the external communications .............................................. 18 Data Power – Interno - virtualization of services for all the internal communications ..................... 21

IBM Websphere Message Broker - Enterprise Service BUS ............................................................ 23 Introduction ................................................................................................................................... 23 Interaction of components of WMB ............................................................................................. 26 Architectural Guidelines for Websphere Message Broker............................................................ 29 Websphere Message Broker - Framework components................................................................ 31 Service Specific Components ....................................................................................................... 57 Data Power - Message Transformation Engine for IBM Websphere Message Broker ..................... 64 12. Oracle RDBMS................................................................................................................ 66 Introduction ....................................................................................................................................... 66 Database Design ................................................................................................................................ 66 Initial Setup Tables ....................................................................................................................... 67 Design Phase Tables ..................................................................................................................... 67 Development Phase Tables ........................................................................................................... 67 Relationship model in the RDBMS .............................................................................................. 69 Database Stored Procedure................................................................................................................ 69 Mapping of WMB and Data Base Framework components ............................................................. 71

Document information Version

1.0

Author

Goutam Dutta, Tata Consultancy Services Gaston Mejia, Banco de Bogotá

Date of shipment

27 Sep – 2013

Control de Versiones Version

Date

1.0

27-Sep-2013

Additions / modifications Original Version

Prepared by Banco de Bogotá

1. Introduction The purpose of this document is to establish guidelines for architecture, principles, and policies concerning the implementation of projects of integration in Banco de Bogotá. This document should be considered as the fundamental guide to be referred by all application solutions team of Banco de Bogota. This document has been prepared jointly by TCS and Bank of Bogota Architecture Team, in order to establish the principles of architectural solution involving integration of systems. TCS and Banco de Bogota Architecture Team jointly designed and formulated guidelines and principles related to solutions for projects using Services-Oriented Architecture (SOA). Having said this, the architecture team will continue to work together in the definition and refinement of these principles. Changes and updates related to this document will be published properly and communicated to the necessary stakeholders. It should be noted that the modification and addition of this document would be the sole and exclusive responsibility of the TCS and Banco de Bogotá Architecture Team. Below are the responsibilities of the TCS and Banco de Bogotá Architecture team:  Work together to refine and define the principles and guidelines of the architecture.  Provide advice and evaluation of solutions to architecture for various projects of Banco de Bogotá.  Ensure that the architectural style and application is followed and practiced by the various projects of Banco de Bogota.  Create and maintain a centre of excellence (CoE) to provide adequate support in matters related to the definition and use of the architecture.  Ensure the correct implementation of the process of Software Life-Cycle in various projects  Act as a mentor and guide in various projects of Banco de Bogota to ensure definitions of architecture, design, implementation and testing  Ensure the governance and control of architectural artifacts  Provide support to the components of the Framework  Ensure the delivery and control of the following documents  Integration Architecture Cookbook  Integration Architecture Development Manual  Integration Architecture Operations Guide

2. Architecture Style TCS and Banco de Bogota Architecture Team have jointly concluded the use of Service Oriented Architecture (SOA), as the architectural style for integration solutions. As such, it is obligatory for all application solutions team to eliminate the practice of using Point 2 Point Integration style and move towards the use of service based integration using concepts of Enterprise Service Bus.

3. SOA reference architecture Services-oriented architecture (SOA) facilitates the creation of flexible, reusable assets to enable business from end-to-end solutions. The use of the SOA reference architecture is a key factor for the achievement of the proposals of value of an SOA The logical to a SOA Reference Architecture solution is shown below as defined in "The Open Group "

Vertical Layers Operational Systems The Operational and IT Systems layer captures the organization's infrastructure, new and existing, needed to support the SOA solution at design, deploy and run time. In scope of Banco de Bogota, this layer consists of all existing and new assets of Packaged Applications (like SAP, CRM), Legacy Systems (like First Data, HOST) or any other Custom Applications (like X-Press). Service Components The Service Component layer contains software components, each of which provide the implementation or "realization" for a service, or operation on a service In scope of Banco de Bogota, this layer consists of components that are responsible for implementation of the functionality of Services using the existing or new assets

of Banco de Bogota. These are the granular services realized using the assets of Operational Systems. Services The Services layer consists of all the logical services defined within the SOA. This layer contains the descriptions for services, business capabilities, and IT manifestation that are used/created during design time as well as service contracts and descriptions that are used at runtime. In scope of Banco de Bogota, this layer consists of the definitions and design artifacts of all Services which are part of the SOA portfolio for the Banco. Business Processes The Process Layer covers the process representation, composition methods, and building blocks for aggregating loosely coupled services as a sequenced process aligned with business goals. Data flow and control flow are used to enable interactions between services and business processes. In scope of Banco de Bogota, this layer defines the business processes (like Auto Loan processing, Apertura de Cuenta ). Consumer Interfaces The Consumer Layer is the point where consumers, whether if be a person, program, browser or automation, interact with the SOA. It enables an SOA solution to support a client-independent, channel agnostic set of functionality, which is separately consumed and rendered through one or more channels (client platforms and devices.

Horizontal Layers Integration Responsible for enabling communication between consumers and service providers for mediation capabilities, routing and transformation. Usually, the responsibilities associated with this logical layer are delegated to an Enterprise Service Bus (ESB), which in turn is defined as a logical component that provides the infrastructure for Integration Architecture consistent with principles of SOA Quality of Service (QoS) Provides mechanism for monitoring, tracking and managing to control aspects associated with the quality of service, such as availability, reliability, safety and security. Information Provides unified representation of domain-specific data and definitions , and mechanisms necessary to integrate these domains. Sets information models and overall data architecture that can be used as a basis for creating Business Intelligence from data marts and data warehouses. Includes the definition and storage of meta data.

Governance Supports Policy-time and Run-time governance of services. The Policy-Time defines reference architectures about government policies, sources of compliance, conformance definitions, authorizations ,compliance exceptions and general maintenance and vitality of the governance model over time. For its part, the RunTime governance defines characteristics of a service registry and its implications for the life cycle of services

4. Architectural Solution for Banco de Bogota Based on the existing infrastructure and the architectural principles already in place in Banco de Bogota, TCS and Banco de Bogota Architecture Team has jointly arrived in the following below Solution Architecture for Integration Projects of Banco de Bogota.

Figure 1 – Logical Architecture of Solutions Below are the main responsibilities of each component of the above mentioned solutions architecture

Data Power - Externo Security Gateway for all external communications  Use the existing infrastructure of IBM Data Power as a security service gateway with all consumers and external service providers  Ensure the virtualization of services to consumers and providers.

Data Power - Interno Virtualization of services, policy management of services, governance of services, message transformation engine for communications with service providers  Use the existing infrastructure of IBM Data Power to provide virtualization of services in Intranet.

 Integration with IBM Websphere Service Registry and Repository (WSRR) to handle WSpolicies and SOA governance  Provide a single gateway to all Intranet users to consume services.  Provide a fast and robust mechanism for the transformation of message for communication with Service Providers.

WebSphere Service Registry and Repository For governance of services, manage lifecycle of services, policy management of services, publish and register services for service discovery  Use IBM WSRR for service life cycle management  Use IBM WSRR, for management services and governance of services.  Use IBM WSRR for end-point Lookup

WebsphereMessageBroker Enterprise Service Bus for Banco de Bogota to perform Orchestration of Services and Transformation of messages across consumers and providers  Only Enterprise Service Bus environment for Banco de Bogota.  To manage the Transformation of messages between consumers and service providers  Use IBM Websphere Message Broker for the orchestration of services  Use IBM Websphere Message Broker for the management of services in integration layer transactions, provide capabilities for the management of events, monitoring and auditing of services.

WebSphere Business Process Server  Use IBM Websphere Process Server to implement and deploy complex business processes  Use IBM Websphere Process Server to implement features of business complexes, which are included with IBM Websphere Message Broker

Oracle RDBMS  Use the RDBMS, to manage the data associated with the various Framework components of IBM Websphere Message Broker.

IBM Websphere Application Server  Use IBM Websphere Application Server as J2EE server for hosting and deploying the web application to manage the data in RDBMS

5. Product Stack Product IBM Data Power Hardware

   

IBM Websphere Message Broker V8







IBM Websphere Service Registry and Repository V7

 

IBM Websphere Process Server V7



IBM Websphere Application Server Oracle Database

 

Responsibilities Single point of Security Gateway for services of Banco de Bogota for all external communications Single point of Security Gateway for services of Banco de Bogota for all internal communications Transformation engine for transformaiton of messages between consumers and providors Implement WS-Policies, WS-Security, SLA management of Services of Banco de Bogota Single Enterprise Service Bus for all integration solutions of Banco de Bogotá. Transaction management of services implemented using WMB Orchestration of services, enrichment of messages, Protocol Bridge with service providers Manage lifecycle of services and governance of services Registration of services, publication and discovery of services Implementations of business processes. J2EE application server Database to store and manage the data for the correct operation of the Framework components of WMB

6. Message architecture Based on the discussions between TCS and Banco de Bogota Architecture team , the team has arrived to the conclusion that all the exposed services will be using SOAP / HTTP (s) Protocol, and the defined canonical structure of messaging would be a customized version of Industry Standard definitions IFX - "IFX Light "". On the other hand, the Data Power Interno is would be used for transformation of messages from canonical defined structure to the messaging requirements of the service providers. If in the future, services other than SOAP / HTTP are discovered and defined, then Data Powerinterno would be responsible for the transformation of these message formats / Protocol to generate messaging canonical structure "IFX Light "". This would be done before the invocation of the actual services hosted in WMB.

It must be remembered that the integration of Data Power Interno and WMB will always happen through MQ as the channel of transport using XML (IFX Light) as the messaging format. Definition of Governance Model for managing the Canonical Dictionary of Message It is very important to mention that the TCS and Banco de Bogotá architecture team need to work together to define and control the policies and methods related to the governance of the dictionary of Canonical Message Structure. All decisions of update and control of the canonical message dictionary should be coordinated and validated with the architecture team of Banco de Bogotá. Control of the artifacts and the versioning of the canonical message dictionary should be the sole responsibility of the architecture team of Banco de Bogotá and, as such, any application solutions team of Banco de Bogota using these definitions must involve the architecture team in relation to decisions of governance,control and management of the above dictionary. TCS and Banco de Bogotá architecture team are working together to define an operative model and a governance model, in order to manage and control the canonical message dictionary.

7. Style of Communication Based on discussions between TCS and Banco de Bogota Architecture Team, it has been decided that all services exposed to the consumers, both external and internal, will be synchronous style based using SOAP / HTTP (s) Protocol. However, it must be noted, the services exposed by the different service providers can be both synchronous and asynchronous.  Communication between the Data Power Interno and IBM Websphere Message Broker will be asynchronous using MQ as the channel of transport  Currently, all services exposed by Data Power Interno are synchronous using SOAP/HTTP  Currently, all services exposed by Data Power Externo are synchronous using SOAP/HTTP

Figure 2 - Component Interaction Process Diagram Considerations for the selection of Style of Communications To increase the throughput of messages and to always have a better performance in terms of maximum use of resources WMB, style of communication within WMB, is defined as asynchronous using MQ as the transport channel. Data Power Interno, will use the asynchronous communication style for the integration of input and output with WMB (which acts as the only enterprise service bus for Banco de Bogota). For all existing resources, like Process Server, XPRESS - architecture team continues to recommend the use of the existing communication through synchronous style using SOAP / HTTP protocol. For this reason, these services exposed by the Data Power Interno and or Data Power – Externo should also use the same synchronous style using SOAP/HTTP.

Services exposed by Data Power Externo, for both external customers and external suppliers are

synchronous using the SOAP protocol / HTTP. Having said that, the above define Architectural Solution, is mature and can be extended to other styles of communication, using different protocols such as TCP / IP, FTP, MQ, JMS, etc. It is necessary that project teams conduct a thorough analysis before choosing the necessary style of communication and protocol for definition of services. Such decisions should be taken together with the consultancy and assessment from the architecture team. Factors relating to the message format (ISO-8583, ATH BASE 24, SWIFT, IFX, FIX), restrictions on consumers or suppliers, attributes related to performance, latency, payload of messages, traffic volume of a service, the nature of complexity of the orchestration of a service, handling of states of services, human intervention, transactional nature of services and other parameters should be taken into account before taking a decision on the style of communication and its associated communication protocol. In all cases the architecture team of Banco de Bogotá would be responsible to take and approve the decisions related to the architecture style of communications of services.

8. Governance Architecture IBM Websphere Service Registry and Repository (WSRR) would be used as the solution for the governance of SOA and management of life cycle of services. All services must be registered and published in WSRR for Service Discovery. In the future, the architectural team would define the model for SLA management of the services and define the WSRR Governance Enablement Profile (GEP). It should be note that WSRR should be considered to ensure to keep the artefacts related to WSPolicy, WS-Security, SLA management of Services. It must also be used to keep the End Point of a service, so that it can be used by Data Power to route to the appropriate service provider. Services governance model TCS and Banco de Bogotá architecture team is working together to define the processes and artefacts related to the governance model of services for use in WSRR.

9. Architecture of Monitoring As of now, TCS and Banco de Bogota architecture team proposes to use the auditing and monitoring components of the Framework of WMB to provide the necessary information related to monitoring of services hosted by WMB. The information obtained from these two components of the WMB would help the Banco de Bogotá team with the necessary information for the traceability of transactions. Additionally, the architecture team is coordinating with the HP-BSM support team of Banco de Bogotá for the definition of strategies and points of failure associated with the overall monitoring of the entire architectural solution. TCS architecture team will continue its contributions

and recommendations related to the definition of monitoring the above architectural solution model. The monitoring solution includes  Monitoring operating system - capture and monitor information related to the behavior and performance of the O / S in which the entire infrastructure and architecture is implemented.  Monitoring of the infrastructure peripheral - capture and monitor information related to different products and systems in the above mentioned architectural solution. This includes monitoring of the Execution Groups in WMB, depth and size of MQ, WMB JVM performance. Apart from this, both Data Power Externo and Data Power Interno should also be able to generate events that must be tracked to determine any discrepancies related to the behavior of services as defined in WS-Policies and SLA management artifacts of WSRR.  Monitoring of individual services - this would be done using the framework components Auditing and Monitoring of WMB. Likewise, Data Power must be enabled to issue monitoring events to monitor the behaviour of end-to-end service.

To carry out the first 2 tasks, the TCS and Bank of Bogotá architecture team are working with the HP BSM support team of Banco de Bogotá, to define the model and the related points of failure with the monitoring of the operating system and infrastructure. To carry out the task of monitoring the services exposed by the WMB, it is required to use the components Framework of WMB - Auditing and Monitoring. These components record the audit-related events and errors / exceptions in RDBMS with which the traceability of the service can be retrieved easily. This information can be used to monitor the health and performance of WMB exposed services, along with the determination of the behaviour of the services exposed by other service providers. In the future, the TCS and Bank of Bogotá architecture team will work together to define the model of proactive monitoring services exposed by Data Power – Externo and Data Power Interno. For further information please refer to the section of WMB components - Auditing and Monitoring of this document to learn about which data and fields are stored by these components that can aide in monitoring services in WMB.

10. Architectural Guidelines  The architecture of the solution mentioned above should be referenced and implemented for any type of application solution involving integration of systems.  For the requirements of integration, Websphere Message Broker (WMB) always should be used as the Enterprise Service Bus (ESB). No other tool product / must be replaced or added to complement the use of WMB.  As such, all the services that are implemented using IBM Process Server ESB must be migrated in the future to be implemented using Websphere Message Broker.  Application Solutions team must thoroughly study and apply the appropriate integration style (synchronous - asynchronous) of services exposed at the enterprise level before the design and implementation activities. This study must be done with the guidance and mentoring of the Architecture team of Banco de Bogota.  The above architectural solution is based on the principles of Service Oriented Architecture (SOA) using Asynchronous messaging as the integration style.  It must be taken into account that WMB and XPRESS are the only 2 systems that can be used as the point of access for integration for all services exposed by HOST system of Banco de Bogotá.  The above architectural solution should be used in a way so that no more than 2 end to end Webservice invocations occur. The objective of using this architecture must always be to minimize the latency involved in invocation of various other systems using web services without using Point –to-Point integration.  Application Solutions team should follow the steps , guidelines and methodology established by the architecture team of Banco de Bogota in relation to the identification of the service and the creation of services.  Prior to the adoption of the above architecture, application solutions team should carry their own analysis and studies related to the non-functional requirements of the project. This study could help define the style of integration - Synchronous on asynchronous, Webservice over other methods of integration, such as TCP / IP, file transfer, etc. MQ  For all external consumers, Data Power Externo will act as single point of contact to expose services. Resolution of URL and service-level security is implemented by Data Power Externo with respect to external consumers.  For all external service providers, Data Power Externo will act as a channel for the integration of communication. Specific requirements related to the security of service messages, security of transport layer should be implemented using this component.

 All activities related to security implementations based on guidelines of Banco de Bogota must be done using Data Power - Externo. This includes the verification of certificates, encryption of messages, PCI implementations for communications with First Data, etc. It must be noted here that PCI implementations for First Data are implemented in a separate Data Power hardware due to restrictions and policies for PCI governance.  For all internal consumers, Data Power - Interno will act as single point of contact to expose services at the enterprise level.  IBM WSRR will be used for the governance of life cycle, management of policies, service discovery, and end point lookup of a service. It will be integrated with Data Power - Interno.  Currently, all services exposed in Data Power - Externo and / or Data Power - Interno will be based on SOAP / HTTP (s) Protocol. On the basis of future needs, services using other protocols such as TCP / IP, MQ and JMS can be discovered and defined. In such situations, the services exposed by Data Power - Externo and that Data Power - Interno should use the corresponding protocol.  The communication between Data Power Externo and Data Power Interno will be based on SOAP / HTTP protocol. Given that Data Power is deployed in a secured area, no HTTPS connection is required.  The communication between Data Power Interno and IBM WSRR will be based on REST.  The communication between Data Power Interno and IBM Websphere Message Broker will take place using the MQ as the transport channel and XML(IFX Light) as the messaging format over non-secure channel. This means that all services exposed by IBM WebSphere Message Broker are MQ / XML based.  IBM Websphere Message Broker would be used as Enterprise Service Bus providing functionalities related to orchestration of servics, protocol bridging between service providers and WMB, facilitating transformation of message formats.  Data Power - Interno will be used as transformation engine for all message transformations with service provider. Transformation of message can be made using XSLT / WTX to comply with the definition of contract of service providers services.  IBM WebSphere Message Broker Services to be consumed by other internal consumers always should be exposed through Data Power - Internal. This means, Data Power - Internal will provide the virtualization of services for all enterprise level internal services.

 All Enterprise level services which are capable of being consumed directly by the end consumer should be exposed using Data Power Interno and or Data Power Externo. These services does not need to be hosted at WMB level.  Use of WMB in the above solution is restricted only in cases where there is need of orchestration of services and or transformation of messages while invocation of services. Under any case, WMB should not be used to perform Pass-Thru operations during invocation of Service.  WMB should not be used to implement Stateful services that need to store the state of a transaction. In other ways – only stateless services can be hosted and exposed using WMB. If such requirements arise, proper analysis and aproval is required from the Architecture team before the application solutions team incorporates it in it’s design.  WMB should not be used to orchestrate complex business oriented long running services.  Suitable analysis with the Architecture Team must be carried out to define the level of orchestration that can be performed in WMB services. Too many orchestration of services can lead to a poor response time of service which must be mitígated while defining and designing the service.

11. Components of Architectural Solution Data Power – Security Gateway for all the external communications IBM Data Power is used as the single point of communication (gateway) for all external communications of Banco de Bogotá. The Data Power aims to provide virtualization of services for all external communications, providing a single point of entry and exit only. Data Power Externo will act as a gateway for security for all inbound and outbound communications with external consumers and external service providers respectively. The objective of Data Power Externo is for implementation of policies related to WS-Security of Service for incoming and outgoing communications. Drivers for the use of a separate Data Power for external communications Data Power Externo is used as a separate physical and logical solution to be deployed in the DMZ of Banco de Bogota infrastructure to isolate and separate all communications with external entities - consumer and provider. Using Data Power Externo, the above architecture divides and distributes the tasks related to the management of safety and policy of web service management. Data Power Externo is utlized in the management of security constraints defined according to the guidelines of Banco de Bogota in the DMZ security Data Power Externo will provide virtualization and visibility of all services externally exposed. This will also serve as a single control for all external service providers.

By separating Data Power external and Data Power Interno both logically and physically, architectural solution guarantees to provide additional layer of security to Data Power Externo, which is more vulnerable to external security attacks. The additional security and political measures can be implemented in this component without affecting the rest of the architectural solution. The other reason to have Data Power Externo is to provide ease of scalability to the already existing infrastructure of Banco de Bogota.

Please refer to the following schematic diagram illustrating the process of integrating Data Power Externo with the other components of the architectural solution defined above.

Fig 3 - Communication with External Consumers

Fig 4 - Communication with External Providers See below the different guidelines and policies related to the use of Data Power externo  All external communications - incoming and outgoing with external consumers and external service providers must use Data Power externo  For all inbound communications - Data Power Externo must be responsible for the implementation of WS-Security of the service. The need for and details of such policies will be based on the requirements of service and consumption or requirements.  For all outbound communication with service providers, Data Power Externo should be used to implement WS-Security .  Services exposed by Data Power Externo must be used using Web Service Proxies Patterns of Data Power.

 To date, all services exposed by the Data Power Externo are synchronous with SOAP / HTTP protocol for communication. Data Power Externo will integrate with Data Power Interno using SOAP/ HTTP protocol in synchronous integration style.

Data Power – Interno - virtualization of services for all the internal communications IBM Data Power is used as the single point of communication (gateway) for all internal communications within the INTRANET of Banco de Bogotá. The Data Power Interno aims to provide virtualization of services for all internal communications, providing a single point of entry and exit only. Data Power Interno will be responsible for receiving requests of service from internal consumers, as well as from Data Power Externo . Its main responsibility is to implement WSPolicies defined with the service consumed or invoked. It should be noted that all services should be registered and published with WSRR and as such the Data Power Interno will use the WSRR to resolve a service endpoint. Drivers for the use of separate Data Power for all internal communications Data Power Interno is used as the single point of access for all internal consumers and providers. This would be a logical and physical independent component that would separate the tasks of management and control of internal resources in the Banco de Bogotá. Data Power Interno should expose all enterprise-level internal services. To separate the specific needs of security from policy management and SLA management of services, Data Power Interno would be used for the latter. Security would be the sole responsibility of Data Power Externo. In this way, 2 separate components are used to manage and control 2 different tasks. Any service that can be consumed directly without needs of transformation or orchestration should be exposed and invoked directly with Data Power Interno. There is no need to have WMB. Any service which consists of logic related to the orchestration or needs related to the transformation of the message format and Protocol must be implemented using WMB and subsequently exposed by Data Power Interno. It’s very important to understand that WMB should be only and only used in cases where it’s required to have an integration performed between two different systems using the concepts of transformation of messages, translation of messages. In cases, where two systems can directly invoke each other, service provider must expose its services using Data Power Interno.

Please refer to the below schematic diagram illustrating the process of integration between Data Power Interno and other components of the above architectural solution.

Fig 5 – Data Power Interno Integration Diagram See below the different guidelines and policies related to the use of Data Power Interno within the Intranet  All communications within the INTRANET - input and output with internal consumers and service providers should be done using Data Power-interno.  For all inbound communications - Data Power - interno must be responsible for the implementation of WS -Policies, SLA of Services.  For all inbound communications - Data Power Interno must validate the contract of the service through integration with WSRR. WSRR should also be used to discover and route the service in the corresponding endpoint. The endpoint can be MQ URL based for WMB hosted services or HTTP URL based for services hosted by other systems such as XPRESS, PROCESS SERVER.  For all outbound communication, Data Power Interno should be used to implement WS-Security in relation to the internal service provider.  For all providers internal services that have services exposed in the form of SOAP / HTTP, Data Power Interno should be used as point of integration with the other components of the architecture of the individual solution. This means that the services exposed by XPRESS, FIRST DATA or any other entity must be registered and published by the Data Power - Interno.  For all internal integrations with consumers who have services exposed by MQ or other protocol it’s recommended for direct integration without using Data Power Interno. This means that if WMB need to consume to stay exposed services, one can invoke it directly through MQ as the transport channel.

 Web Service Proxies Pattern must be used to implement Webservices exposed by Data Power Interno.  To date, all the services exposed by the internal power of data are nature synchronous through the SOAP protocol / HTTP for communication. The communication between Data Power Interno and WMB would be done using MQ over XML. Communication with other components such as XPRESS, PROCESS SERVER is performed synchronously using the SOAP protocol / HTTP. WS-Policies, SLA Management of Services All the principles and decisions related to the management of SLA, WS-Policies management and service level management of security should be implemented and applied in the Data Power Interno. Data Power Interno must have all services published and registered pursuant to methods and principles of management of the SLA associated to the service and as defined by the Architecture Team.

IBM Websphere Message Broker - Enterprise Service BUS Introduction IBM Websphere Message Broker (WMB) is used as the enterprise service bus to provide a layer of integration to the above mentioned architectural solution. Services deployed in WMB would be asynchronous in nature - all communications within the various components of WMB would be carried out using MQ as the channel of transport. Drivers for using Websphere Message Broker If a service can be invoked directly and consumed by the consumer without the need for transformation, orchestration, or Transaction Management – Data Power Interno and Data Power Externo should be used for the same. The service must be exposed with Data Power so that the final consumer can directly consume this service without passing through enterprise service bus. If a service needs to orchestrate with several service providers, to perform activities related to the transformation of the message format and the message protocol, is always necessary to use WMB as the integration layer to perform the role of the enterprise service bus. In such cases, the services must be implemented in WMB and exposed with DP-Interno, DP Externo as needed. WMB should not be used for services that can be consumed directly. Putting the layer of WMB adds latency in the overall performance of a service and as such, the use of WMB should be properly evaluated and consulted with the Banco de Bogotá architecture team. TCS architecture and Banco de Bogotá architecture team recommends using WMB as the only solution to the enterprise service bus. There are no other products or components must be used for the realization of the function of enterprise service bus. This means that in the future, in the future, services implemented in WPS should migrate to the architectural solution mentioned above, using WMB as ESB layer.

Asynchronous communication in Websphere Message Broker It should be noted that all the components implemented in the Websphere Message Broker must be exposed asynchronously using MQ as the transport channel. Communication between WebSphere Message Broker and the service provider shall be governed by the definitions of contract related to the method of communication among them - synchronous or asynchronous. To facilitate high-performance processing of messages and transactions, any activity WMB with RDBMS persistence must be asynchronously without affecting the transaction.

Below is the representation of the different components of WMB component diagram

Fig 6 - WMB Component Diagram

See the following table for each of the WMB components and its activities.

Fig 7 - WMB Component Matrix

Interaction of components of WMB Below is the diagram of component interaction ìn WMB explaining how all the above components work together in the environment of WMB

Fig 8 - WMB Process Flow Component Diagram Below is a brief description of the flow of execution of a service in WMB i. Canonical Message received by the Data Power Interno is routed to the component CONTROLLER component exposed by WMB using MQ ii. Each service is always assigned to one and only one CONTROLLER component within the WMB, which is responsible for the orchestration and the business implementation of the service. iii. CONTROLLER component after receiving message canonical invokes the Lookup-Registry component provided by the WMB Framework to retrieve all the information related to the orchestration of the service - which FMG is invoked, what services and operation of the service provider that is invoked, that transformation that must be used to transform messages for the service provider, which is the communication adapter component to use, which is the endpoint of the service provider, etc. iv. CONTROLLER component implements the logic of the service orchestration using the CHAINED pattern where in various components performing smaller task is chained in a sequential manner to perform a bigger task using a connector object – in this case MQ. With the information obtained from Framework provided Lookup Registry , CONTROLLER will invoke 1 or many FMG b y dynamically invoking those components using MQ. Each FMG

component is an abstraction of the service being invoked in the service provider level and is exposed as an MQ/XML service for the CONTROLLER to consume. v. MQ/XML exposed FMG component receive the Canonical message and carry out enrichment required for the integration of services with service provider. This is done by invoking the framework provided Cache component - Enricher. vi. Updated and Translated canonical message is thereafter passed to another MQ exposed Framework provided component - Tx. Manager AGENT . This component ensures to update the message with fields / values to ensure the correct transaction of the service to be invoked depending on the specific needs of the service provider - for example: generates and adds fields related to the generation of sequences, next day and jornada code for HOST, add the username, password fields for service invocations with FIRST DATA, generates and adds fields related to unique identification for transactions with REDEBAN etc. This component will also ensure to trigger the process of persistence of response message received from the service provider to ensure the storage of the same so that during the process of REVERSE/ROLLBACK – the message pertaining to the original transaction can be recovered to construct the message/request necessary to perform ROLLBACK with the Service Provider. vii.

Updated canonical message is then sent to be transformed for the specific message format to invoke the service exposed by the service provider. In this case, the Data Power Interno is used as transformation engine to transform the canonical message to specific message format defined by the Service Provider. For example: Data Power Transformation Engine implemented within a separate domain/hardware of Data Power Interno uses XSLT to transform the IFX light based canonical to XML structure as defined by REDEBAN Or WTX is used to transform the IFX based canonical message to host specific TRAMA / BINARY structure.

viii. The Service provider specific message generated out of the Data Power Transformation Engine is then sent to the Framwork provided specific communication adapter to carry out the communication integration with the service provider. Each and every service provider should have its own framework provided communication adapter that ensures the Protocol bridging with the service provider. For example: HOST communications adapter that is used for communication with HOST using MQ/XML as the protocol, First Data communication adapter as a communication adapter for communicating with First Data using MQ/XML as the protocol. ix.

The response obtained from the service provider is processed in this Communication Adapter component to define the status of the transaction. It’s always mandatory for the Communication Adapter to respond back to

the Data Power Transformation Engine to return back the response to the caller component – in this case framework provided Tx. Manager – AGENT component. x. If the CONTROLLER component has more FMG components (read service provider services) to be invoked, the WMB repeat the steps till point V. xi. If there are no more FMG’s required to be invoked, the CONTROLLER component carries out enrichment of message to return the response to the calling party - in this case Data Power Interno that received the message from the actual consumer. xii. It is noteworthy that before sending the response to the consumer, CONTROLLER component ensures the registration of events (with framework provided audit component), registration of any failure (with framework provided monitoring component), reversal of the transaction in the event of technical errors / business errors (using Framework provides components Tx.Manager Logger).

Architectural Guidelines for Websphere Message Broker.  Services implemented by WMB must always be associated with a CONTROLLER. The CONTROLLER component is the entry point of a service exposed in WMB. The CONTROLLER component is always invoked asynchronously using MQ as the channel of transport, both for input and output, and the Banco de Bogota defined canonical structure as message format.  CONTROLLER component acts as an entity of control for the orchestration of invocations of various service provider exposed services which are abstracted as FMG components  There can be no service implemented in WMB without having a CONTROLLER component  CONTROLLER component’s MQ name must be registered as the end point of the service in WSRR. This means that Data Power Interno should consult the end point of the service from WSRR which would resolve as the MQ URL of the CONTROLLER input MQ.  The basic activity of the CONTROLLER component is to provide the orchestration of the service - invocation of 1 or many other services atomic that are implemented using the FMG component and to ensure the functionality of the logging of audit events, logging of failure events and transaction management.  The CONTROLLER component will use the Lookup Registry Framework component to retrieve the meta-data information about the orchestration of the service. The orchestration of service is done by invocation of various FMG’s as defined in the Lookup Registry component and as such the invocation is always done dynamically to ensure integration at run time and abstraction of the service invoked at the level of the service provider. CONTROLLER components should not invoke FMG through static invocations (this means that MQ of FMG should always be dynamically retrieved from the component of Lookup Registry).  The CONTROLLER component is the main component that is responsible for the transaction and the orchestration of the service implemented in WMB, but it must also ensure managing SLA related to appropriate response time for the service. Configuration of the response time of the services should be determined through the Framework based Lookup Registry component  The framework provided Lookup - Registry ensures the management of all the services exposed by the service provider and the corresponding orchestration associated with such services that are implemented using the CONTROLLER component and the FMG component of WMB. This component provides a layer of abstraction and virtualization invoked services at service provider level, because neither the CONTROLLER nor FMG has knowledge of actual service and service provider to invoke.

 CONTROLLER component should be responsible to ensure the ACID properties of transactions and ensure that the State of the transaction through the consumers and suppliers are well maintained. This must be ensured through the framework based Tx. Manager component.  To Ensure the registration of logging events and the registration of error events in WMB, the CONTROLLER component must use the Framework provided Auditing and Monitoring components  The Framework provided Auditing component is responsible for registering the audit events/logs in the RDBMS. The component store log information of the transaction undertaken by capturing and storing the following informaiton service name, CONTROLLER name associated with the service, the information related to the Service / Operation and Service Provider getting invoked, performance data related to the execution of the service – Time In, Time Out, status of transaction, etc.  The Framework provided Monitoring component is responsible for registering of error events related to technical error/exception generated in the execution of the Service in RDBMS. As such the following information like - service name, CONTROLLER associated with the service, technical errors etc are captured and stored in the RDBMS.  Each atomic integration of services with the service provider must be implemented as a stand-alone component using FMG. As such FMG components are an abstraction of the Services exposed by the Service Provider. A FMG component should always be invoked from CONTROLLER component using MQ as the channel of transport.  FMG component provides a layer of wrapping on the exposed services of the service provider. This component must invoke the framework provided Enricher component to perform enrichment messages and translation of messages before invoking the service provider exposed service.  Each FMG component must be exposed using MQ as the channel of transport and after updating the canonical message with the relevant translation data, must send the message to Framework provided TX. Manager - AGENT component using MQ as the channel of transport.  FMG components must be designed and developed based on the need of each requirement. Both the CONTROLLEr and FMG are service specific components.  The Framework provided TX. Manager - AGENT component ensures the transaction management of the services implemented in WMB.  All the specific requirements of handling the transaction of services with service provider must be implemented in Tx. Manager - AGENT component. For example: the generation of sequence, jornada codigo, next day values for

invoking services exposed by Host/ XPRESS,generation of user id and password for invocation of services exposed by FIRST DATA, generation of unique transaction id for invocation of services exposed by REDEBAN etc.  The process of triggering the storage of transactional data associated with invocation of a service provier (this is based on certain business and technical criteria) should be implemented using the Tx. Manager – Agent component. This will ensure to recover the original transaction data to construct the message required to perform the necessary transactions pertaining to REVERSOS / ROLLBACK.  Data Power - Interno along with XSLT and WTX should be used as the transformation engine to transform the Banco de Bogota defined canonical message structure to the service provider specific message structure for proper message and protocol bridging.  WMB should not be under any circusmtances used for performing message transformation activities.  As a thumb-rule, Architecture team specifies to use XSLT for any XML-XML message conversion  As a thumb-rule, Architecture team specifies to use WTX for any XML-Binary message conversion.  Framework provided Communication Adapters should always be used for the communication with the service provider. There cannot be a direct integration of communication with the Service provider from CONTROLLER / FMG components. Architecture team is responsible for the definition and creation of Communication Adapters.  For any requirements pertaining to configuration of business rules and decisions required to be used within the service implemented at WMB environment, Architecture team recommends to use RDBMS parameterized tables to define such dynamic definitions. For complex rule integrations, the Architecture team recommends to use comercial BRMS products and as such , the decisiont to use them must be thoroughly analyzed and justified with the Architectural Team. Websphere Message Broker - Framework components Auditing The Framework provided Auditing component aims to ensure the registration of the audit events related to the execution of a service in the data repository in order to maintain traceability of the transaction and visibility of the orchestration performed by the service. The information recorded in the database can help the support team and the Banco de Bogota infrastructure team to analyze the message used for the execution of services, as well as analyze the metrics data related to the performance of the service.

As such, the audit component consists of a Client component and a Server Component. The client component uses synchronous style and exposes a requestresponse pattern based interface for the consumers to invoke the service to activate the audit process. The server component exposes an asynchronous style “Fire and Forget" pattern based interface for the client component to invoke the process of persisting the auditing data. Please refer to the following schematic diagram showing the process of integration of the WWMB components (CONTROLLER and Communication Adapters ) and Framework provided Auditing component -.

Fig 9 - WMB Framework Auditing Component Features Communication style Communication pattern Message format Message Protocol Consumers of this component Providers of this component

Asynchronous Fire and Forget XML MQ CONTROLLER, Communication Adapters RDBMS Responsibilities

 The component of Audit -Client is implemented in WMB as a Sub - Flow which is invoked only by the CONTROLLER and Communication Adapter components to trigger the process of logging the audit events. This invocation is performed synchronously sending the necessary XML message format for the audit event log.  The CONTROLLER component invokes the component of Audit – Client synchronously at the end of the operation of the service to register the audit event with information - such as the service name, the name of the CONTROLLER, message input, output message, event start time, event end time and final status of the transaction. This information is generated in an XML format as defined in the Framework of WMB.  The Communication Adapters invokes the Audit - client component synchronously at the end of the transaction of the invocation service provider to register the audit event with information - name of service, the name of the CONTROLLER, name of service provider, the name of the service / provider operation, message input, message output, event start time, event end time and final status of the transaction. It is important to mention here that the input message and output message generated by Communication Adapters would be in the same format of the messages as that the provider of services.  Audit – client component triggers the process of storing information of audit events asynchronously by invoking the component of Audit - Server using MQ as the transport channel.  The component of Audit - Server (which is implemented as a separate message flow exposed by MQ) is responsible for the storage of the audit event in the RDBMS. This is done through the invocation of a stored procedure in the RDBMS.  Persistence of the audit event information is configurable and is maintained in RDBMS tables. Such configurations are allowed at CONTROLLER and Communication Adapter level to store meta-data and message data.  Audit - Server component does not send any response to the Consumer. The errors related to the process of storing the audit event data must be registered separately with Framework provided Monitoring component. Considerations  It is necessary to understand that the audit can only be accomplished by the CONTROLLER and the Communication Adapters components for recording events related to the overall execution of the service exposed by the WMB and invocation of the service exposed by the service provider respectively.  The structure of the audit message must conform to the data structure defined below. All the data necessary to generate the XML must be provided as parameters by the CONTROLLER, Communication Adapters.  The integration of the components of Audit-client and Audit-Server component must be done with asynchronously using MQ as the transport channel.

Below is the integration layer of the audit component

Fig 10 - WMB Framework Auditing Component Integration Layer XML Data Structure for the Audit Message Data Unique Transaction Id

Type String

Description It stores a unique identifier related to the uniqueness of the message

Source Based on the request to WMB from the consumer of the service Generated by WMB

ESB unique Id

Numeric

Consumer IP

String

Consumer Application Id

Numeric

Controller Id

String

Legacy Register Id

Numeric

Unique random ID generated in relation to the transaction carried out by the service of WMB IP address of consumer Based on the consuming WMB service request to WMB from the consumer of service ID. the consumption of Based on the service WMB consumer request to application WMB from the consumer of service Unique ID of the main Developer CONTROLLER for the service in WMB Service Provider Id from Lookup RDBMS Registry

Transaction Status

String

Status of Transaction

Start Time

TimeStamp

End Time

TimeStamp

Invocation Sequence

Numeric

TimeStamp when it started operation TimeStamp when the operation ended Invocation sequence

Message In

BLOB

Request message

Message Out

BLOB

Response message

WMB generated based on communication results with Service Provider Generated byWMB Generated byWMB WMBgenerated during development as part of the sequence of Orchestration Generated by WMB Generated by WMB

The above audit component XML information will be recorded in the RDBMS at TBL_TRAN_AUDITORIO and TBL_TRAN_AUDITORIO_DTL. The field "Unique Transaction ID" which is derived from the request message entering into the service invocation of the WMB CONTROLLER must be used to find corresponding records for the traceability of the service from RDBMS. This information would help to Banco de Bogotá team with the function of monitoring of WMB services.

Monitoring The WMB provided Framework based Monitoring component aims to ensure appropriate logging of errors/faults/exceptions related to the execution of a service in the data repository in order to maintain traceability of the transaction and the points of failure of the service. The information recorded in the database can help support team and the Banco de Bogotá team to analyze the message used during the execution of the service, as well as the error / fault / excluding data related to the fact that service (as in case). As such, the monitoring component consists of a client component and a server component. The client component uses synchronous style request-response pattern based interface which the consumers uses to invoke the service to activate the

process of the error log. The server component exposes an asynchronous fire and forgets pattern based interface for the client component to invoke the service to actually persist the error data. Please refer to the following schematic diagram which shows the process of integration of the Framework provided MONITORING c omponent with its consumers - CONTROLLER, Communication Adapters and Tx. Manager - AGENT

Fig 11 - WMB Framework Monitoring Component

Features Communication style Communication pattern Message format Message Protocol Consumers of this component

Asynchronous Fire and Forget XML MQ CONTROLLER, Communication Adapters, Tx. Manager - AGENT Providers of this component RDBMS Responsibilities  The client component - that is implemented in WMB as a Sub - Flow is invoked by the CONTROLLER, Communication Adapters and Tx. Manager - AGENT to activate the process of registration of technical errors / failures / exceptions. This invocation is performed synchronously by sending the necessary XML format for the logging of error events.  The CONTROLLER component which is responsible for managing the overall orchestration and transaction management of a service is also responsible to detect errors / exceptions related to the execution of the service. Any occurrence of technical errors will lead the CONTROLLER component to invoke the component Monitoring-client synchronously by passing the necessary error event information - name of the service, the name of the CONTROLLER, time of Error, Error description together with the error code. The CONTROLLER component must stop processing service and send an immediate error response to the consumer in these cases.  The Communication Adapters are responsible for effective communication and integration with the external service provider from the environment of WMB. Therefore, whenever there are any faults / errors / exceptions pertaining to WMB communication with the provider of such services, this component must invoke the synchronously exposed Monitoring – client component to trigger the error event logging process. The information captured should be the service name, the CONTROLLER name and name of the service provider service / operation be executed, time of error and description of the error along with the error code. The adapter must always return a response to the above components to allow the consumer receive a response from the CONTROLLER component.  Tx. Manager - AGENT component is responsible for the management of the transaction for a service with respect to the Orchestrator and the invocation of services exposed by the service provider. All errors encountered during processing of transformation of messages using the Data Power Interno Transformation Engine must be captured by the Tx.Manager AGENT component and hence it must invoke the component Monitor - client to activate the process of registration of such errors.  The component Monitoring – client triggers the process of storing information of error events by asynchronously invoking the Monitor-Server component using MQ as the transport channel.  The component Monitoring - server (which is implemented as a separate message flow exposed by MQ) is responsible for the actual storage of error in the RDBMS tables. This is done through the invocation of a stored procedure in the RDBMS. Considerations

 It must be understood that the invocation of Monitoring component must be done in all the components which are susceptible towards the generation of errors or exceptions. As such, CONTROLLER,Tx. Manager - AGENT and Communication Adapters are vulnerable to the generation of errors and therefore only these components must invoke the component Monitor.  Component FMG should not have any invocation to Monitoring and Auditing components. FMG component-related errors should be captured and treated within the development and testing of it and should not be registered.  The structure of the monitor message must conform to the data structure defined below.  The integration of the components Monitoring-Client and Monitoring - Server component must be done using MQ as the transport channel  Monitor events must always be registered in database tables Below is the integration layer of the component

Fig 12 - WMB Framework Monitoring Component Integration Layer XML Data Model Data Unique Transaction Id

Type String

Description It stores a unique identifier related to the uniqueness of the message

Source Based on the request to WMB from consumer service Generated by WMB

ESB unique Id

Numeric

Controller Id

String

Unique random ID generated in relation to the transaction carried out by the service of WMB Unique ID of the main Development

CONTROLLER for the service exposed by WMB RDBMS ID of the Service Provider State of exception-related error / error Error code (technical) as stated in WMB

time.

Legacy Register Id

Numeric

Lookup Registry Generated by WMB Generated by WMB

Error Status

String

Error Code

Numeric

Error Message

String

Description of the error as collected in WMB

Generated by WMB

Error Detail

String

Details of the error as captured by WMB

Generated by WMB

Error Object

String

WMB object associated with the error

Generated by WMB

Error Path

String

Stack trace of the error

Generated by WMB

Time

TimeStamp

Time-stamp when the error occurred

Generated by WMB

Severity

Numeric

The severity associated with the error

Generated by WMB

Previous audit component details will be recorded in the RDBMS at TBL_TRAN_MONITOREO. The field "Transaction ID only" which is derived from the request to WMB message should be used to find corresponding records for the traceability of the service. This information would help to Banco de Bogotá team with the function of monitoring services.

Enricher The Enricher component provides the necessary interfaces and API that allows the FMG and CONTROLLER components for enrichment and the translation of the data fields of messages related to the exchange of messages between the provider and the service consumer. Data related to the enrichment of status codes and messages are maintained in the database and this component provides an interface to access these data (by caching the data) which reduces the count of invocations of databases. As such, the Enricher component consists of a client and server components. The client component is a Java class that exposes Synchronous "Request-Response" pattern based API be invoked from FMG components and control to perform the enrichment or the translation of messages. The matrix of enrichment or data translation (which is maintained in the database) is stored in this Java class as

collections of objects in cache that allows rapid recovery of data rather than reading from the database. The server component is an independent asynchronous "Fire and Forget" based pattern message flow components that retrieves the data from the database and fills in the collections of objects in the cache of the client component. The database read only occurs once in life of the execution environment of WMB, which significantly reduces access to the database. Cache Management It should be noted that the Framework based Enricher components is always responsible for providing interfaces and API access to data relating to the translation / homologation of data, as well as the translation of the status codes / error between the service consumer and service provider. These data are loaded and stored in the memory of WMB (to be more specific, these data are loaded and stored in each execution group JVM of WMB so that all message flows deployed throughout the environment of WMB can have access to it. ( to provide faster access and translation of data during the execution of services ;thus reducing significant time in RDBMS invocation) Currently WMB has limited non JVM based that can handle Global Cache / SharedLibrary solutions, as such the Enricher Cache data is replicated and stored in all execution groups of WMB(To be more precise, this data gets stored in JVM of each execution group of WMB). This means that there would be multiple instances of the Enricher – Client component deployed in WMB. However, the Enricher-Server component would have only 1 instance implemented in only 1 execution group of the WMB. The data stored is composed of all the data from the tables TBL_MST_TRANSLATION, TBL_MST_TRANSLATION_ERROR. The volume of data stored in these 2 tables will depend on the number of services, requirements, translations and combinations of consumers with providers and expected that this will increase over the period of time with more services implemented in WMB. On the other hand it should be noted that for every change done in the data of RDBMS, the execution group s at WMB must be restarted to have the cache updated/refreshed again. This process is manually performed by triggering the Enricher Server component that loads the data from RDBMS into the persistent MQ;so that Enricher Client component can load it in its JVM based cache during the first invocation of it’s API's. Therefore, care must be associated with the use of this component in cache with respect to the use of the JVM heap and the overall performance of the services deployed in WMB. Possible solutions related to the optimization of such performance include adjustment and reconfiguration of JVM parameters of execution groups of WMB. TCS architecture team and Banco de Bogotá team would continue to monitor the use of this cache with respect to the performance of WMB environment. The architecture team continues to evaluate and study the performance of the above caching methodology with respect to the records of the above tables that can be stored in the memory. However at this point in time of its important to mention that triggering of the server

component of Enricher is currently based on the manual intervention - which means that the server component must be manually activated to load data from the database.This process of activation must always be done whenever the WMB / Execution groups are restarted or any changes to the data are made in the RDBMS tables. In the future, with the design and implementation of the Web-based administrative application, this process must be automatic so that when the Web application is started or any modification of the data in the RDBMS is performed using the Web application screens, automatic triggers are sent to start the Enricher Server components.

Please refer to the following schematic diagram showing the process of integration between the Enricher component and their corresponding consumers - FMG, CONTROLLER.

Fig 13 - WMB Framework Enricher Component Characteristics Communication style Communication pattern Message format Message Protocol Consumers of this component Providers of this component

Synchronous Request and Reply API

N/A

CONTROLLER, FMG RDBMS Responsibilities  The client component is implemented in WMB as a Java class that exposes the API to be invoked by the CONTROLLER, FMG components.  The client component is invoked by the CONTROLLER and FMG for translation and enrichment of messages. The FMG invokes this client component to make the translation of messages by passing Consumer Id of the application invoking the Service, Application Id of the Service Provider, the domain name of the data that needs to be translated and the value associated with the data.





 

The CONTROLLER component invokes this Enricher client component to perform the translation of status before sending back the response to the consumer who invoked the service. The server component is implemented in WMB as a separate message flow that invokes an exposed database stored procedure that retrieves all the matrix of information related to translation and the translation of status code of messages. This recovered information is stored in a persistent MQ. This process is currently conducted manually. When the client component is invoked from FMG or CONTROLLER components, the Java class is initialized and thus data is loaded from the persistent MQ. This activity occurs only once and after that, the client is always retrieves the data from its Cache – collections object of the Java class. Considerations The server component is currently activated manually to synchronize the data stored in the database with the persistent MQ. It’s is highly recommended for the construction of the web- application to manage the Framework components to automatically load the data from RDBMS in the persistent MQ.

Lookup-Registry The WMB framework provided Lookup-Registry components aims to ensure the governance of service specific components - CONTROLLER and FMG; and also to maintain the information related to Orchestration of a service using CONTROLLER and FMG components. This component provides the necessary API and interfaces which are invoked by the CONTROLLER component to generate the information necessary to make the orchestration of services dynamically. Lookup-Registry component consists of a client component in cache and a server component. The client component is implemented as a Java class that includes information about orchestration of services - information mentioned comprises of the mapping information between CONTROLLER component and its associated FMG’s. The mapping information between a CONTROLLER and its associated FMG are registered in RDBMS tables. This information is loaded by the Server component into a persistent MQ. The client component; when is invoked for the first time; loads this data from the MQ into the Java class memory objects. There-after ; all integration of Lookup-Registry happens from the cache which reduces invocation with Database considerably. Cache Management It should be noted that the Framework based Lookup Registry component is always responsible for providing interfaces and API access to data relating to meta-data for orchestration of controller and its association with FMG’s. These data are loaded and stored in the memory of WMB (to be more specific, these data are loaded and stored in each execution group JVM of WMB so that all message flows deployed throughout the environment of WMB can have access to it. ( to provide faster access and translation of data during the execution of services ;thus reducing significant time in RDBMS invocation)

Currently WMB has limited non JVM based that can handle Global Cache / SharedLibrary solutions, as such the Lookup Registry Cache data is replicated and stored in all execution groups of WMB(To be more precise, this data gets stored in JVM of each execution group of WMB). This means that there would be multiple instances of the Lookup Registry – Client component deployed in WMB. However, the Lookup Registry-Server component would have only 1 instance implemented in only 1 execution group of the WMB. The data stored is composed of all the data from the tables TBL_MST_CONTROLLER_REGISTER, TBL_MST_FMG_REGISTER and TBL_MST_ORCHESTRATOR. The volume of data stored in these tables will depend on the number of services, requirements, number of service provider integrations, number of orchestrations required for each service and expected that this will over the period of time with more services implemented in WMB. On the other hand it should be noted that for every change done in the data of RDBMS, the execution group s at WMB must be restarted to have the cache updated/refreshed again. This process is manually performed by triggering the Lookup Registry Server component that loads the data from RDBMS into the persistent MQ;so that Lookup Registry Client component can load it in its JVM based cache during the first invocation of it’s API's. Therefore, care must be associated with the use of this component in cache with respect to the use of the JVM heap and the overall performance of the services deployed in WMB. Possible solutions related to the optimization of such performance include adjustment and reconfiguration of JVM parameters of execution groups of WMB. TCS architecture team and Banco de Bogotá team would continue to monitor the use of this cache with respect to the performance of WMB environment. The architecture team continues to evaluate and study the performance of the above caching methodology with respect to the records of the above tables that can be stored in the memory. However at this point in time of its important to mention that triggering of the server component of Lookup Registry is currently based on the manual intervention - which means that the server component must be manually activated to load data from the database. This process of activation must always be done whenever the WMB / Execution groups are restarted or any changes to the data are made in the RDBMS tables. In the future, with the design and implementation of the Web-based administrative application, this process must be automatic so that when the Web application is started or any modification of the data in the RDBMS is performed using the Web application screens, automatic triggers are sent to start the Lookup Registry Server components.

Please refer to the following schematic diagram showing the process of integration of the Framework provided Lookup Registry component with other components CONTROLLER.

Fig 14 - WMB Framework Lookup-Registry Component Features Communication style Communication pattern Message format Message Protocol Consumers of this component Providers of this component  





 



Synchronous Request and Reply API

CONTROLLER RDBMS Responsibilities The client component is implemented in WMB as a Java class that exposes the API to be invoked by the CONTROLLER components. The client component is only invoked by the main component of the CONTROLLER during the beginning of the execution of a service within WMB. The invocation is done to retrieve the information of the required metadata needed to accomplish the orchestration of the various FMG components. The necessary information concerning about the integration of FMG with its Service Provider, Transformation Engine artefacts and Communication Adapter names are also retrieved at this stage and passed to the other components in MQ headers. The server component is implemented in WMB as a separate message flow that invokes data base exposed Stored Procedure that retrieves all the matrix information related to orchestration of CONTROLLER and integration of FMG. This recovered information is stored in a persistent MQ. This process is currently conducted manually. When the client component is invoked from the CONTROLLER, the Java class is initialized and is loaded with the information stored in the MQ. This activity occurs only once and after that the Java Class always retrieves data from its Cache during its API invocations. Considerations The server component is currently activated manually to synchronize the data stored in the database with the persistent cache MQ. It is highly recommended for the construction of the web-application to manage the Framework Components which will automatically load the data in the persistent MQ. Only the CONTROLLER component invokes the component of Lookup Registry

during the beginning of the execution of a service to load the meta data information needed for the proper functioning of the service.

Transaction Manager Transaction Manager - AGENT Transaction Manager - AGENT (abbreviated as Tx Manager - AGENT ) is a Framework provided component that provides a facade between the communication of the FMG component and Data Power internal Transformation Engine component. The main function of this component is to ensure the integrity of transactions of services with respect to the service provider. It is the WMB provided shell to manage transactions with the service provider. This component is implemented as a set of 2 message flow in WMB - for incoming and outgoing communications with respect to the request and the response of the transaction. The INBOUND component is responsible for ensuring that each activity and tasks related to the integration of service provider are properly managed and carried out by WMB. These tasks include the generation of sequence number, jornada codigo, NEXT DAY for integration with HOST, the acquisition of user ID and the password required for integration with FIRST DATA, generation of ID for integration with REDEBAN etc. All tasks relating to the management of the singularity and integrity of transactions with the service provider should be done using ONLY this component. The OUTBOUND component is responsible for ensuring the registration of transactions for services that are reversible in nature. This registration process is performed depending on the status of the transaction with theservice provider. This OUTBOUND component should also trigger the process of registration of the errors captured by the Data Power internal Transformation Engine. Please refer to the following schematic diagram which shows the process of integration of this component.

Features Communication style Communication pattern Message format Message Protocol

Asynchronous Request and Reply XML (IFX Light) MQ

Consumers of this Components

FMG , Data Power Transformation Engine Providers of this Components N/A Responsibilities  Tx. Manager - AGENT component includes an input and output components. These two components are asynchronous in nature and uses MQ as the transport channel. The structure of the message is the canonical message structure defined by Banco de Bogota.  The INBOUND component is always invoked by FMG components using a static MQ. Message arriving to this component is the output message generated by FMG components. In addition to this message, FMG passes information of meta-data obtained by the Lookup Registry component in the form of header MQ.  The main responsibility of the component is to generate the necessary data related to the management of the transactional integrity with the service provider. This activity is carried out by passing this component control to other secondary components created for each service provider - HOST, REDEBAN, First Data, XPRESS, etc. These secondary components are responsible for the generation of the data of transactions necessary with the service provider. Please see the developer guide for more information and details related to the behaviour of these components responsible for the generation of data / fields to maintain the integrity of transactions with service providers.  When a FMG has to invoke HOST, the previous INBOUND component invokes another Framework component specific to the needs of integration with host, which generates the sequence No, Jornada code and updates the canonical message structure before passing them to the transformation engine components.  Similarly, when a FMG has to invoke a service of FIRST DATA, the previous INBOUND component invokes another Framework component with the specific needs of integration with the FIRST DATA –acquiring user and password for communication with First Data ; and updates the structure of the canonical message structure  The INBOUND component always stores the structure of the canonical message updated before sending the message for transformation to the Data Power Interno Transformation Engine.  OUTBOUND component is always is invoked by the Data Power Transformation Engine OUTBOUND component after the transformation of the response received from the service provider service is completed.  The OUTBOUND component is also responsible for triggering the process of storing the transactional message based on the status of the transaction. Such process is done using asynchronous invocations of Tx. Manager LOGGER component. Considerations  FMG should invoke Tx. Manager - AGENT component for integration with service provider services.  TX.Manager-AGENT must have all the logic for managing the integrity of transactions with the service provider. This component must ensure the generation of fields / data which are necessary for the proper functioning of the services of the specific service provider.  All communications with Tx. Manager - AGENT should happen with MQ as the

conveyor channel.  Depending on the service provider, Tx. Manager - AGENT must have secondary components needed for integration of each services provider. These components will keep the logic necessary to maintain the integrity of transactions with service providers.  TX. Manager - AGENT should be responsible for logging transaction (if it is necessary for the service as configured in RDBMS for reversible services.).  TX. Manager - AGENT must invoke the component of monitoring in the event of any error encountered in relation transformation errors in Data Power Transformation Engine.

Please refer to the following diagram of Integration Layer of Tx. Manager - AGENT INBOUND component.

Please refer to the following diagram of Integration Layer of Tx. Manager - AGENT OUTBOUND component.

Transaction Manager--REVERSE MANAGER The framework provided Tx. Manager – REVERSE MANAGER is responsible for managing and performing the REVERSOS / ROLLBACK of services in the event of any technical errors / exceptions. This component is responsible for the configuration of the fields / data that are required to be persisted for the original transaction, providing mechanism for actually storing such data related to the original transaction, and also execute the process of REVERSOS / ROLLBACK for the original transaction. The Transaction Manager - REVERSE MANAGER consists of 3 separate components CONFIGURATION MANAGER  this component is responsible for the configuration and retrieval of fields / data for the original transaction that must be stored to perform REVERSOS / ROLLBACK.  TRANSACTION LOGGER  this component is responsible for the storage and retrieval of the data of an original transaction in the RDBMS.  ROLLBACK EXECUTOR

 this component is responsible for the executing REVERSOS / ROLLBACK of a transaction CONFIGURATION MANAGER The Transaction Manager – Configuration Manager is a Framework component in WMB that enables configuration and retrieval of data needed to be persisted during an original WMB transaction so it can later be recovered to facilitate the generation of message for the process of REVERSOS / ROLLBACK. This component is implemented in WMB as an independent message flow that reads data from RDBMS only once and then stored it in a persistent MQ. The components that access the configuration data uses this MQ to read the necessary information. Please refer to the following schematic diagram which shows the process of operation of the Configuration Manager component.

Features Communication style Synchronous Communication pattern Request and Reply Message format API Message Protocol N/A Consumers of this Component Transaction Logger, Rollback Executor Providers of this Component RDBMS Responsibilities  Tx. Manager-Configuration Manager is invoked synchronously by Tx. Manager Logger component during the original transaction data storage process. This is to retrieve the sub-set fields / data that should be stored from the original transaction to retrieve the message for REVERSOS / ROLLBACK.  TX. Manager - Configuration Manager is invoked synchronously by Tx.Manager ROLLBACK EXECUTOR to recover the already saved data (as done earlier) of the original transaction to perform its REVERSOS / ROLLBACK.  TX. Manager - Configuration Manager Server must be invoked manually to load configuration data from the RDBMS in the persistent MQ. In this case, MQ acts as the Caching Mechanism for Tx. Manager – Configuration Manager.

TRANSACTION LOGGER Transaction Manager - Logger component is a Framework provided component in WMB that is responsible for recording the original transaction (the logic depends on if the service has been configured to handle REVERSOS / ROLLBACK in RDBMS – Service Provider Repository table) in order to store the data of the original transaction, which later can be recovered to perform REVERSOS / ROLLBACK. This component is invoked from the Tx. Manager – AGENT component after receiving the response from the service provider. This component is implemented as a client component and a server component. The client component is implemented as sub-flow in WMB; being invoked by the Tx. Manager – AGENT OUTBOUND component to trigger the process of storing data related to the original transaction. The Tx. Manager-AGENT OUTBOUND component prepares the structure of the necessary XML message comprising both the request and the response of the Transaction message. That is activated only for those services that are configured to handle REVERSOS / ROLLBACK in RDBMS tables and the process of saving only occurs when the invocation of the service provider was successful. Please refer to the schematic diagram below to understand the process performed by the component Framework Transaction Manager Logger.

Communication style Communication pattern Message format

Features Asynchronous Fire and Forget XML

Message Protocol Consumers of this Component Providers of this Component  





  

MQ TX. Manager AGENT OUTBOUND RDBMS Responsibilities This component consists of a writing component and a Reader component. The writer component is based on client -server Component. The client component is implemented in WMB as a sub –flow that is invoked from Tx. Manager - AGENT OUTBOUND component after a successful transaction has been completed with the service provider. The invocation is performed only when the service is marked as REVERSIBLE in RDBMS tables. The server component is a message flow exposed by MQ that performs data storage the data stored in the RDBMS. Before storing the data, the server component retrieves the fields / data required to be stored by invoking the Configuration Manager component so that unwanted data are not kept in the RDBMS tables. The Reader component is MQ exposed message flow that invokes the stored procedure RDBMS to retrieve persistent of the original transaction d. The reader component is invoked by the Tx. Manager ROLLBACK EXECUTOR component while performing the actual ROLLBACK to the original transaction. Considerations The component writer that actually saves the data in the RDBMS is performed asynchronously using MQ as the transport channel. The component of reader for the recovery of data from RDBMS is performed asynchronously using MQ as the transport channel. Each service must identify it’s unique ID of the transaction that can be used as a correlation between original transaction requests and the reverse/rollback operations.

Refer to the below data model of the message being stored as part of the Tx. Manager LOGGER component Field Unique Transaction Id

String

Type

Type of Operation

String

Legacy Code

Numeric

CHANNEL

Numeric

Description Unique ID related to the transaction

ID of the CONTROLLER associated with the service Service provider which performed the transaction Consumer Application Id

Source Generated by the CONTROLLER based on the message input and definition of services. Development

Obtained from Lookup Registry by the controller WMB Captured from the request message arriving

REQUEST

BLOB

RESPONSE

ESB TIME STAMP

TIMESTAMP

Request message of the transaction with the Service Provider Response message of the transaction with the Service Provider Date and time of the event

to the CONTROLLER WMB Captured

WMB Captured

Generated by WMB

Communication Adapters Introduction

All Protocol Bridging and communications with Service Providers - both external and internal to Banco deBogota must happen with Framework provided Communication Adapters. These adapter components are specifically designed and implemented in WMB to perform the task of Protocol Bridging, communication and management of transactions with service providers using the message and the transport channel defined by the Provider to integrate with WMB. It must be taken into account that WMB cannot have direct integration with providers without the communications adapters. As such, all service providers must have an associated communication adapter and this must be used by all the application solution teams while developing services in WMB. The mapping between the Service Provider exposed services and their corresponding communication adapter has to be always registered in the RDBMS tables so that Lookup-Registry component can set the meta-data information accordingly in CONTROLLER component (refer to the TBL_MST_LEGACY_REGISTER table that assigns a service provider with its corresponding Framework Communication Adapter to be used.) Drivers to select Communication Adapters Currently, architecture team has identified 3 different communication adapters -to communicate with the HOST, to communicate with service providers that exposes services through web service (SOAP / HTTP) as XPRESS, REDEBAN, to communicate with FIRST DATA. It should be noted that all communication adapters are implemented as a single message flow exposed by MQ as input and output transport channel. The Protocol of communication for integration with service provider must be defined based on various factors like performance, throughput, WMB compatibility with service providers, ease of acquisition of 3rd party industry standard adapters that can be used in WMB, the nature of transaction services, etc. The solutions team should always consult the architecture team prior to select or define a communication adapter. The design and implementation of WMB based communications adapters

are the responsibility of the Architecture Team and as such modification and aggregation to these components must always need the support from the Architecture Team. HOST ADAPTER Communications adapter - HOST is a Framework provided component specially designed and developed to help communication between HOST and WMB. Communications with the host are performed asynchronously with the request / response pattern with MQ as the transport channel. This adapter component is responsible for the integration of communication WMB and HOST using the structure of messaging and protocol defined by HOST. Communication Adapter for HOST It should be noted that this component must always be used for any communication between WMB - HOST for which services can be consumed by the HOST MQ manager. In such cases, the communication with the host is accomplished using MQ as the transport channel and TRAMA/BINARY as the messaging structure. For other services which are not exposed with MQ manager of the HOST, WMB should use XPRESS to communicate with the host. WMB services are invoked using the SOAP protocol / HTTP with XML messages for communication with XPRESS. It must be taken into account that XPRESS is another Service Component of Banco de Bogota, which offers interfaces and services to communicate with the HOST using SOAP / HTTP. This component will continue to be used until confirmation to use MQ as the transport of integration for HOST is decided. Therefore, decisions about using XPRESS or direct communication with MQ to integrate HOST must be assessed jointly by the Architecture and Application Solutions Team. XPRESS is a stable-platform based on Webservice that allows to integrate seamlessly with the host. It also serves as a contingency and measures of security in direct communication in the case HOST fails or is unstable. All decisions related to the use of MQ or XPRESS for communications with the host must be duly analysed with the help of the architecture team.

It must be taken into account that going in the future, all integrations with HOST must always take place using the WMB provided Communications Adapter for HOST.

HOST Communications Adapter is implemented as a WMB message flow that is responsible for the bridging Protocol with HOST using MQ as the transport channel. The adapter receives the output binary structure message from Data Power Transformation Engine and carries out the integration of the communication with the HOST using MQ as the transport channel.

The MQ to communicate with the HOST is determined dynamically from RDBMS using the canal as a search parameter . Canal is a parameter that the Consumer will send when invoking such services of HOST. RDBMS tables maintain a set of matrix data that maps Canal information against the destination MQ name of the HOST MQ manager that will process the request.

After receiving the response from the HOST, this adapter component processes the status of the transaction and sends back the response to the Data Power Transformation Engine that will transform the binary response structure to the Canonical model defined by Banco de Bogota ; ready to be processed by the TX Manager - AGENT OUTBOUND components. The Adapter component is responsible for always triggering the process of registering the audit event by invoking the framework Auditing component. In the case of errors related to errors of communication, errors of analysis of responses, etc., the adapter triggers the process of error logging by invoking the provided framework component Monitoring. Please refer to the following schematic diagram that illustrates the behaviour of HOST communication adapter.

Features Communication style Communication pattern Message format Protocol format Consumers of this Component Providers of this Component

Asynchronous Request - Response Binary / plot MQ Data Power Transformation Engine HOST Responsibilities

 Communication adapter is implemented as a message flow that performs the integration of communication with the host using MQ as the transport channel  The adapter receives the binary message / TRAMA generated by Data Power Transformation Engine using the dynamic invocation of WTX. This is done using the information obtained from Lookup-Registry  Adapter then determines the HOST MQ using the Canal as search parameter. Configuration of Canal vs HOST MQ is maintained in RDBMS.  In the case of the errors related to communication or processing of response from HOST, Adapter fires the error events registration process by invoking the framework component Monitoring.  The adapter must ensure the request / response correlation for asynchronous integration with HOST.  After receipt of the response of the host, adapter must invoke the framework Auditing component to start the process of the audit event logging to maintain traceability of the transaction.  The response obtained in binary format is there after passed to the Data Power Transformation Engine OUTBOUND component to be transformed into canonical message for further processing. Considerations  MQ HOST queue name are selected dynamically based on the canal where the message is coming – canal is a parameter sent by the consumer invoking the service. As such, this mapping information must be in the RDBMS tables to allow for the dynamic selection of HOST MQ.  This adapter should always invoke the audit component to record the transaction with the HOST  This adapter should always invoke the component of monitoring to record the errors related to the communication with HOST.  The adapter must always be sure to return a response to the Data Power Transformation Engine data OUTBOUND component to ensure the continuity of the process.  This adapter should always be used for any communication with the HOST.

WEBSERVICE ADAPTERS - XPRESS, REDEBAN Web service SOAP / HTTP Adapter - is a Framework provided Communication Adapter specially designed and developed to help communication between WMB and any service providing having services exposed using synchronous SOAP/HTTP protocol. This adapter is general purpose adapter for the invocation any SOAP/HTTP based webservices using synchronized mechanism and as such can be used for the invocation of the services exposed by XPRESS, REDEBAN, PROCESS SERVERetc This adapter is implemented as a single WMB message flow having MQ as the channel of transport for both input and output with other WMB components.The communication between Service Providers is done with SOAP/HTTP ( as per the infrastructure of Banco de Bogota, the communication only happens between WMB

and Data Power Interno / Externo since these are the only IP’s visibile to WMB for outbound communication). Such addresses are retrieved from the configuration of the Service Provider Service registered in the RDBMS tables.Messages for such Adapters are generated using XSLT as the transformation mechanism in Data Power Transformation Engine component.

It should be noted that WMB always communicates with Data Power Interno or Externo for invocations of web services exposed by the Service Providers both internal and external, respectively. Data Power components are responsible for routing of service to XPRESS , REDEBAN, PROCESS SERVER etc and as such special requirements related to WS-Security, governance of services, banking related to governance policies should be applied to Data Power components and not on WMB Communication Adapters. WMB components should only be used to ensure the logic of communication and should not have any burdens associated with the additional functional requirements.

Please refer to the schematic diagram below to understand the process of communication between WMB and any Data Power enabled Service Provider (Interno / Externo)

Characteristics Communication style Communication pattern Message format Protocol format Consumers of this Component

Asynchronous Request - Response XML SOAP/HTTP Power Data Transformation Engine INBOUND Providers of this Component HOST Manager MQ Responsibilities  Communication Adapter for Webservices is a framework provided WMB component that performs communication integration with Service Providers using Synchronous SOAP/HTTP protocol.  The Communication Adapter is implemented as single independent message flow in WMB using MQ as the entry-exit channel. Communication with Service Providers are done synchronously.  Adapter receives XML message generated by the Data Power Transformation Engine.  Invocation of the Service Provider is always done using Data Power Interno or Data Power Externo. This is a very important guideline with respect to Service Invocation of Enterprise level services of Banco de Bogota.  Adapter ensures the proper communication with the Service Provider. Any errors related to communication or receipt of response, are properly captured and logged in the RDBMS using the framework provided Monitoring component.  Adapter ensures the correct logging of the audit events of this transaction using the framework provided Auditing Component.  Adapter ensures to send back the response obtained from the Service Provider to the Data Power Transformation Engine for further transformation of response message to reply back to the Consumer. Considerations  It must be noted that all Webservice invocations of Service Providers must be routed through Data Power Interno or Data Power Externo. They are the only 2 systems visible to the WMB for webservice invocations. Service Specific Components Controllers CONTROLLER component is a service specific component that is implemented in the WMB as a message flows. CONTROLLER acts as the single point of entry and exit of all services implemented in the WMB. The CONTROLLER is responsible for the orchestration of the service, the transaction management and trigger of the process of audit and error logs. Depending on the nature and quantity of the orchestration of the individual services that must be invoked for the service provider, the CONTROLLER component must have components of entry, exit and orchestration. In general, for a requirement of WMB service that has N of invocations to the service provider, one must build (N + 1) no. of message flows to generate the CONTROLLER component.

Each CONTROLLER component must have an input message flow which is the gateway of the service of WMB, an output of messages flow that is out of the service of WMB. In addition, it must have other message flows that will perform the service orchestration invoking FMG components as defined and configured in the RDBMS tables related to orchestration configuration. Input MQ of the CONTROLLER – INBOUND flow must be registered and published in WSRR as the URL for service discovery and routing from Data Power - Interno. The MQ of the CONTROLLER's OUTBOUND flow always need to forwad the response to Data Power Interno OUTPUT MQ, which is responsible for processing WMB response messages and direct to consumer or Data Power - Externo based on internal or external invocations of the service.

See the following diagram schematic process that shows the process of integration between the Data Power - Externo - > Data Power - Interno - > WMB (in the case of an external consumer) and between the Power Data - Interno - > WMB (in the case of a internal consumer).

As mentioned in the previous section, services always are published and registered with WSRR and the endpoint of this type of service should always be aimed to the input MQ queue name of the CONTROLLER – INBOUND component.

Please refer to the following diagram that illustrates the use of (N + 1) no.of CONTROLLER component flows and its integration with other components of WMB. Given figure assumes N = 2.

Note in the diagram above that the CONTROLLER component comprises of 3 message flows CONTROLLER-INBOUND (for the input flow), CONTROLLER - OUTBOUND (for the output flow) and a CONTROLLER additional flow - CONTROLLER STEP (for the orchestration of services) for the invocation of 2 specific services provider (2 FMG invocations). It should be noted that the CONTROLLER can be chained. Therefore a CONTROLLER can also invoke another CONTROLLER as well as the way in which an FMG is invoked. Therefore, the CONTROLLER - output flow responds the message to the caller CONTROLLER component or Data Power Interno component depending from which it received the request. This callback is done dynamically by reading the MQ Reply To information stored in the MQ header of the request message.

Features Communication style Asynchronous Communication pattern Request and Reply Message format XML (IFX Light) Message Protocol MQ Consumers of Component CONTROLLER, Data Power Interno Providers of Component FMG Responsibilities  The CONTROLLER - INBOUND message flow component is the entry point for a service deployed in single WMB. Hence the Input MQ nameof this flow should always be registered in WSRR against the service. Messages must reach this component using the canonical message model defined by Banco de Bogotá.  This CONTROLLER - INBOUND component is responsible for loading metadata related to the functionality of the service by invoking the component framework Lookup - Registry. The component of Lookup-Registry loads the entire orchestration information like – which FMG is invoked, how to manage the transaction time out, if the service needs to reverse / ROLLBACK etc  The time out response in WMB is handled in CONTROLLER – INBOUND flow using WMB provided time out controls  The information obtained from Lookup - Registry is used to invoke the FMG that form part of the orchestration of this service. FMG invocation is done dynamically by setting the MQ name associated to the FMG. During the invocation of FMG from this controller component sets MQ response to CONTROLLER – CHAIN or CONTROLLER –OUTBOUND component depending on whether other invocations of the FMG is necessary or not, respectively. The CONTROLLER, FMG, Tx.Manager - AGENT, Data Power - Interno Transformation Engine, and WMB Framework Communication Adapters retain this metadata information from Lookup - Registry in the form of MQ message headers so that all components have access to it.  The CONTROLLER - CHAIN are intermediate message flows that connects the input and output CONTROLLER components for the orchestration of services. These components connect the INBOUND and OUTBOUND flows based on the no. Of service providers and orchestration necessary to be performed.  The CONTROLLER - OUTBOUND component is the exit flow of WMB Service. All services deployed in WMB should have this flow.  This component is responsible for triggering the process of event log - auditing , triggering the error event log process - monitoring (if necessary on the basis of errors in the flow) and to manage the transaction of the service through the activation of the process of REVERSO / ROLLBACK if service encounters Time Out response using the Framework component of Tx. Manager - Logger  The output of the CONTROLLER should always perform out the translation of the status of the transaction using Enricher framework component Considerations  Must have the Input MQ of CONTROLLER INBOUND component registered in WSRR as the service endpoint.  Must implement (N+1) no. Of CONTROLLER components for a service having N no. Of service provider integrations.  Must invoke Lookup-Registry at the beginning of execution of the service to load the information of orchestration in the cache.  Must ensure that Lookup Registry meta-data information is passed through the MQ headers to the other components of WMB.

 Arriving message to CONTROLLER components must be Canonical in nature.  Must have a definition of Unique Transaction ID to uniquely determine the message and transaction it performs  Must have a Unique ID/Name associated with it. This name/ID must be registered in RDBMS as well as WMB for the correct functioning of Lookup Registry component.  Each component of the CONTROLLER should be responsible to generate an ID only for the traceability of the entire transaction.  Must be ensuring the log of auditing events, log of error events and handling of transactions with respect to triggering process of REVERSOS/ ROLLBACK in case of time out errors.

The following diagram shows the Integration layer of the CONTROLLER component.

GenericMessageFlows (FMG)

Generic Flow Message (FMG) component provides a facade and a layer of abstraction to the actual invocation of Service Provider services by performing the functionality of translation of messages. FMG provides this abstraction using canonical message structure over MQ as the transport channel. FMG components are independent reusable components using asynchronous integration sty le using MQ as the channel of transport. The objective of this component is to shield the Service Provider service using the canonical message structure as the language of communication. It can be fully reusable and as such can be invoked by any CONTROLLER component. Each FMG component must have a Service Provider Service/ Operation associated with it and such information are configured in the RDBMS tables accessed by the Lookup Registry component. Each FMG component must be invoked by a CONTROLLER component responsible to perform the orchestration on it. Such information is obtained from the Lookup Registry and CONTROLLER will perform the orchestration by dynamically setting the MQ name of the FMG.

FMG invokes the Service Provider Services using a general purpose framework provided shell component – Tx. Manager AGENT. See below the process diagram that shows the integration of FMG components with CONTROLLER Tx. Manager - AGENT

Characteristics Communication style Communication pattern Message format Message Protocol Consumers of this Component Providers of this Component   

  

       

Asynchronous Fire and Forget XML (IFX Light )) MQ CONTROLLER TX. MANAGER - AGENT Responsibilities FMG’s are abstraction of the Service Provider Services. As such these components are atomic, reusable and independent in natured. FMG components are deployed on WMB autonomously based only MQ message flow. These components have only entry point. FMG components receive canonical message from CONTROLLER components. Their main responsibility is for translation of messages using Enricher component. FMG components carry out the integration of the services of service providers using Tx. Manager - AGENT component. FMG components are created and applied on the basis of Atomic integration of the requirements of the service provider. Each FMG component must have its unique identifier ID / name that is identified in RDBMS and WMB. This ID is used to correlate the information in metadata retrieved from Lookup Registry by the CONTROLLEr component. Considerations FMG components are independent in nature and completely reusable. FMG should be registered in RDBMS using their ID / ID NAME. Each FMG must have its associated service provider service credentials. This information is always recorded in RDBMS for Lookup-Registry FMG's always have the message input as canonical via MQ Protocol. FMG’s are always invoked by CONTROLLER component. FMG should pass all MQ header information as obtained from component CONTROLLER FMG must always invoke the component TX. Manager - AGENT to carry out the integration of the service provider. FMG components are fire and forget components. As such, the response of its

own execution is never received. Instead, the response of the FMG implementation is transmitted to the CONTROLLER component.  FMG components cannot have integration with audit, monitoring components. As such, these components may not emit errors / exceptions.  There can be no transaction management logic built into the component FMG. The sole purpose of the FMG component is to provide abstraction for the final service provider service  FMG components do not have visibility of the calling CONTROLLER component. As such, the implementation of a FMG has no related assumptions with ID / name of the CONTROLLER, etc - this is also relevant to the service provider FMG is invoking. This information is obtained at run time using the WMB Framework component – Lookup Registry. 

Data Power - Message Transformation Engine for IBM Websphere Message Broker IBM Data Power is used as the main processor of the transformation of messages from Canonical structure to the structure of the corresponding message of service providers. We must remember that the structure of the canonical message defined by Banco de Bogotá is IFX Light and, as such, for all integrations with service providers, it’s bound to have the message transformed and translated to the corresponding message contract of the Service Provider.

Data Power Interno will provide the necessary processing capabilities of transformation of messages from different formats. Drivers to use Data Power for Transformation of Messages Data Power Interno (Completely different domain/hardware to be used as a transformation engine) is used as the processor for transformation engine to transform messages from Banco de Bogotá canonical message format to the service provider specific messages. Given that the Data Power is hardware device, the transformation of messages using XSLT or WTX would work much faster in comparison with any conventional ESB WMB. Since Banco de Bogotá already boasts the infrastructure related to accommodation of Data Power, architectural team decided to leverage this advantage to use the hardware to carry out the transformation activity as well – but in this rapidly and over the wire instead of software. By use of Data Power as the transformation engine, architecture team also reduces the footprint of JVM and improves performance of the ESB - WMB layer. It is important to mention here that by doing transformation of messages through the use of XSLT transformation, WTX in WMB layer, there is a considerable burden and impact on performance, especially with regard to the use of WMB JVM. This was the main reason to use Data Power as the transformation engine. In addition, the logic of transformation of XSLT, WTX-related changes can be hot

deployed at runtime without having to restart the WMB or Data Power.

Please refer to the following schematic diagram for the integration of the component with the other components of the architecture solution.

Please see the following policies related to the use of Data Power Transformation Engine - Interno  Must always be used to transform messages between Banco de Bogota defined Canonical message structure with Service Provider message structure.  WMB should not make any type of transformation of messages. Should always be using the Data Power Transformation Engine component.  Data Power Transformation Engine is a framework components implemented using Multi-Protocol Gateway Pattern in Data Power. 

In accordance with the requirement of the transformation of messages for various services, developers should generate XSLT or WTX to carry out the transformation of messages.

 XSLT should be used in the event of transformation of the message from XMLXML.  WTX should be used in the event of transformation of XML - Binary (WEFT) message  Data Power Transformation Engine would be integrated with WMB components using MQ as the cannel of transport.

12. Oracle RDBMS Introduction The RDBMS is necessary for the management of data related to transactions, events, monitoring of services performed in the environment WMB. Apart from it, its related to the management of the data associated with different Framework components such as Enricher, Lookup-Registry. The RDBMS also provides necessary interfaces and API that are exposed in the form of stored procedures to facilitate the management of data from WMB Responsibilities  Master tables to be used during the Initial Setup of the Services – Stores list of applications, default time out response of these applications, default status code responses obtained for these applications  Master tables to be used during the Design Phase of the Services – Stores information regarding the List of Service Providers and its associated services to be invoked, Translation Information of Messages for integrating with the Service Providers.  Master tables to be used during the Development Phase of the Services – Information regarding the CONTROLLER, FMG and its mapping to generate the orchestration of the Services.  Transactional Tables to store the information related to the functioning of Framework components - Auditing, Monitoring, Tx. Manager LOGGER.

The following diagram illustrates the dependence of each of the tables in the RDBMS with the stage is used

Database Design The following sections describe each of these tables and their associated responsibilities. Master tables are named with "_MST_" and transaction tables and are named with "_TRAN_".

Initial Setup Tables Application Portfolio Management TBL_MST_APPLICATION Master table of all applications TBL_MST_APPLICATION_TIMEOUT Master table of all applications with default timeout value when role is consumer and service provider TBL_MST_APPLICATION_STATUS_DEFAULT Master table of all applications with default status code for error/success TBL_MST_REF_ERROR_CODE Master table with registers related to different error codes in WMB. Design Phase Tables Data Translation , Error Translation TBL_MST_DOMAIN Master tables to store the list of Domains – Types of message data for translation TBL_MST_TRANSLATION Master table to store the mapping between domains and the combination of its values across various consumers and service providers TBL_MST_TRANSLATION_ERROR Master table to store the combination of error codes across various consumers and service providers Registration of Service Providers TBL_MST_LEGACY_REGISTRY Master table to configure the various Services exposed by the Service Providers – Virtualization of services Configuration of REVERSE TBL_MST_REV_CONFIG Master table to configure fields necessary to store for REVERSO/ROLLBACK Development Phase Tables Service Orchestration and Service Governance in WMB TBL_MST_CONTROLLER_REGISTRY Master table to register all CONTROLLERS TBL_MST_FMG_REGISTRY Master table to register all FMG’s TBL_MST_ORCHESTRATOR Master table to register Orchestration details – map between CONTROLLER and FMG Execution Phase Tables Transaction tables Transaction table to store Auditing Component meta-data TBL_TRAN_AUDITORIO_DTL Transaction table to store Auditing Component message transaction-data TBL_TRAN_MONITOREO Transaction table to store Monitoring Component meta-data TBL_TRAN_REV_ORIG_TRANSACTION Transaction table to store Original TBL_TRAN_AUDITORIO

TBL_TRAN_REV_STATUS_LOG

Transaction data to recover for REVERSO/ROLLBACK Transaction table to store the status of REVERSO/ROLLBACK execution.

Relationship model in the RDBMS

Database Stored Procedure List of RDBMS Stored Procedure with the WMB Components Package PKG_ESB_AUDITLOG

Procedure SP_TRAN_SAVE_AUDITORIA

SP_GET_DUPLICACY_RESPONSE

PKG_ESB_ERRORLOG

SP_TRAN_SAVE_MONITOREO

PKG_ESB_LOOKUP_REGISTRY

PRC_GET_ORCHESTRATION_MA P

Responsibilities Auditing Component – Registration of Auditing Events Auditing Component – Checks of Duplicacy of Requests Saves the Error Events of Monitoring Components Used by Lookup Registry to generate the Orchestration information

PRC_GET_APPLICATION_MAP

PKG_ESB_MANAGE_REVERSO S

SP_GET_ALL_FIELDS_STORE_MS T

SP_ADD_ORIGINAL_TRANSACTI ONS SP_ADD_REVERSE_STATUS

SP_GET_ORIGINAL_TRANSACTIO NS PKG_ESB_TRANSL_TIPOS_ERR ORS

SP_ESB_GET_TRANSLATIONS

SP_ESB_GET_TRANSLATION_ERR ORS

Lookup Registry generated information List of configuration fields for REVERSO/ROLLBA CK Stores original transaction data in RDBMS To manage the status of the REVERSO /ROLLBACK operations. To retrieve the original transaction data Translation of message data for Enricher Component Translation of Error Status data for Enricher Component.

Mapping of WMB and Data Base Framework components Below is the table showing the relationship between the various components of the Framework of WMB and its associated RDBMS components Component of WMB Enricher

Lookup-Registry

Auditing Monitoring Tx.Manager - AGENT

Tx.Manager - CONFIG MGR Tx.Manager - ROLLBACK EXECUTOR

RDBMS component TBL_MST_DOMAIN TBL_MST_TRANSLATION TBL_MST_TRANSLATION_ERROR TBL_MST_APPLICATION TBL_MST_APPLICATION TBL_MST_APPLICATION_TIMEOUT TBL_MST_APPLICATION_STATUS_DEFAULT TBL_MST_CONTROLLER_REGISTRY TBL_MST_FMG_REGISTRY TBL_MST_LEGACY_SERVICES_REGISTRY TBL_MST_ORCHESTRATOR TBL_TRAN_AUDITORIO TBL_TRAN_AUDITORIO_DTL TBL_TRAN_MONITOREO TBL_MST_LEGACY_SERVICES_REGISTRY TBL_MST_REV_CONFIG TBL_TRAN_REV_ORIG_TRANSACTION TBL_MST_REV_CONFIG TBL_TRAN_REV_ORIG_TRANSACTION TBL_TRAN_REV_STATUS_LOG

TCS and Banco de Bogotá Architecture team must jointly work together to define and publish the processes related to the maintenance of the data stored in the RDBMS tables. It is highly recommended to have transactional data purged or recycled to another Database/Data warehouse for review and audit for future. Strategies related to the identification of the objects of RDBMS tables must be resolved to achieve this task.