January 11, 2017 | Author: Ummed Mahala | Category: N/A
Download PC8x90L1D_StudentGuide_20081124GV9 (1)...
PowerCenter 8.x/9.0 Level I Developer Student Guide Version – L1D_20081124GV9
1
PowerCenter 8.x Level I Developer Student Guide Version 04 April2008 Copyright (c) 2008 Informatica Corporation. All rights reserved. Printed in the USA. This software and documentation contain proprietary information of Informatica Corporation and are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright law. Reverse engineering of the software is prohibited. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. Use, duplication, or disclosure of the Software by the U.S. Government is subject to the restrictions set forth in the applicable software license agreement and as provided in DFARS 227.7202-1(a) and 227.7702-3(a) (1995), DFARS 252.227-7013(c)(1)(ii) (OCT 1988), FAR 12.212(a) (1995), FAR 52.227-19, or FAR 52.227-14 (ALT III), as applicable. The information in this document is subject to change without notice. If you find any problems in the documentation, please report them to us in writing. Informatica Corporation does not warrant that this documentation is error free. Informatica, PowerMart, PowerCenter, PowerChannel, PowerCenter Connect, MX, and SuperGlue are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners. Portions of this software are copyrighted by DataDirect Technologies, 1999-2002. Informatica PowerCenter products contain ACE (TM) software copyrighted by Douglas C. Schmidt and his research group at Washington University and University of California, Irvine, Copyright (c) 1993-2002, all rights reserved. Portions of this software contain copyrighted material from The JBoss Group, LLC. Your right to use such materials is set forth in the GNU Lesser General Public License Agreement, which may be found at http://www.opensource.org/licenses/lgpl-license.php. The JBoss materials are provided free of charge by Informatica, “as-is”, without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Portions of this software contain copyrighted material from Meta Integration Technology, Inc. Meta Integration® is a registered trademark of Meta Integration Technology, Inc. This product includes software developed by the Apache Software Foundation (http://www.apache.org/). The Apache Software is Copyright (c) 1999-2005 The Apache Software Foundation. All rights reserved. This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit and redistribution of this software is subject to terms available at http://www.openssl.org. Copyright 1998-2003 The OpenSSL Project. All Rights Reserved. The zlib library included with this software is Copyright (c) 1995-2003 Jean-loup Gailly and Mark Adler. The Curl license provided with this Software is Copyright 1996-2007, Daniel Stenberg, . All Rights Reserved. The PCRE library included with this software is Copyright (c) 1997-2001 University of Cambridge Regular expression support is provided by the PCRE library package, which is open source software, written by Philip Hazel. The source for this library may be found at ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre. InstallAnywhere is Copyright 2005 Zero G Software, Inc. All Rights Reserved. Portions of the Software are Copyright (c) 1998-2005 The OpenLDAP Foundation. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted only as authorized by the OpenLDAP Public License, available at http://www.openldap.org/software/release/license.html. This Software is protected by U.S. Patent Numbers 6,208,990; 6,044,374; 6,014,670; 6,032,158; 5,794,246; 6,339,775 and other U.S. Patents Pending. DISCLAIMER: Informatica Corporation provides this documentation “as is” without warranty of any kind, either express or implied, including, but not limited to, the implied warranties of non-infringement, merchantability, or use for a particular purpose. The information provided in this documentation may include technical inaccuracies or typographical errors. Informatica could make improvements and/or changes in the products described in this documentation at any time without notice.
Preface
Welcome to the PowerCenter 8 Level I Developer course. Data integration is a large undertaking with many potential areas of concern. The PowerCenter infrastructure will greatly assist you in your data integration efforts and alleviate much of your risk. This course will prepare you for that challenge by teaching you the most commonly used components of the product. You will build a small operational data store (ODS) using PowerCenter to extract from Source tables and files, transform the data, load it into a staging area, and finally into the operational data store. The Instructor will teach you about Mappings, transformations, Sources, Targets, Workflows, Sessions, Workflow tasks, Connections, and the Velocity methodology.
iii
About This Guide Course Objectives After completing this course, you should be able to:
• • • • •
Use PowerCenter 8 Designer to build mappings that move data from sources to targets Use PowerCenter 8 Workflow Manager to build and run a workflow that runs a session based on a mapping Design mappings and workflows based on business needs Perform basic troubleshooting of PowerCenter mappings and transformaitons Use Informatica Support options to resolve questions and problems about Informatica PowerCenter 8
Audience This course is designed for data integration and data warehousing implementers. You should be familiar with data integration and data warehousing terminology, and in using Microsoft Windows software.
Document Conventions This guide uses the following formatting conventions: If you see…
It means…
Example
Æ
Indicates a submenu to navigate to.
Click Repository Æ Connect. In this example, you should click the Repository menu or button and choose Connect.
boldfaced text
Indicates text you need to type or enter.
Click the Rename button and name the new source definition S_EMPLOYEE.
UPPERCASE
Database tables and column names are shown in all UPPERCASE.
T_ITEM_SUMMARY
italicized text
Indicates a variable you must replace with specific information.
Connect to the Repository using the assigned login_id.
Note:
The following paragraph provides additional facts.
Note: You can select multiple objects to import by using the Ctrl key.
Tip:
The following paragraph provides suggested uses or a Velocity best practice.
Tip: The m_ prefix for a mapping name is…
Other Informatica Resources In addition to the student guides, Informatica provides these other resources:
x x x x x x x
Informatica Documentation Informatica Customer Portal Informatica web site Informatica Developer Network Informatica Knowledge Base Informatica Professional Certification Informatica Technical Support
Obtaining Informatica Documentation You can access Informatica documentation from the product CD or online help.
Visiting Informatica Customer Portal As an Informatica customer, you can access the Informatica Customer Portal site at http://my.informatica.com. The site contains product information, user group information, newsletters, access to the Informatica customer support case management system (ATLAS), the Informatica Knowledge Base, and access to the Informatica user community.
Visiting the Informatica Web Site You can access Informatica’s corporate web site at http://www.informatica.com. The site contains information about Informatica, its background, upcoming events, and locating your closest sales office. You will also find product information, as well as literature and partner information. The services area of the site includes important information on technical support, training and education, and implementation services.
Visiting the Informatica Developer Network The Informatica Developer Network is a web-based forum for third-party software developers. You can access the Informatica Developer Network at the following URL: http://devnet.informatica.com
The site contains information on how to create, market, and support customer-oriented add-on solutions based on interoperability interfaces for Informatica products.
Visiting the Informatica Knowledge Base As an Informatica customer, you can access the Informatica Knowledge Base at http://my.informatica.com. The Knowledge Base lets you search for documented solutions to known technical issues about Informatica products. It also includes frequently asked questions, technical white papers, and technical tips.
Obtaining Informatica Professional Certification You can take, and pass, exams provided by Informatica to obtain Informatica Professional Certification. For more information, go to: http://www.informatica.com/services/education_services/certification/default.htm
v
Providing Feedback Email any comments on this guide to
[email protected].
Obtaining Technical Support There are many ways to access Informatica Technical Support. You can call or email your nearest Technical Support Center listed in the following table, or you can use our WebSupport Service. Use the following email addresses to contact Informatica Technical Support:
x x
[email protected] for technical inquiries
[email protected] for general customer service requests
WebSupport requires a user name and password. You can request a user name and password at http://my.informatica.com.
North America / South America
Europe / Middle East / Africa
Asia / Australia
Informatica Corporation Headquarters 100 Cardinal Way Redwood City, California 94063 United States
Informatica Software Ltd. 6 Waltham Park Waltham Road, White Waltham Maidenhead, Berkshire SL6 3TN United Kingdom
Informatica Business Solutions Pvt. Ltd. 301 & 302 Prestige Poseidon 139 Residency Road Bangalore 560 025 India
Toll Free 877 463 2435
Toll Free 00 800 4632 4357
Toll Free Australia: 00 11 800 4632 4357 Singapore: 001 800 4632 4357
Standard Rate United States: 650 385 5800
Standard Rate Belgium: +32 15 281 702 France: +33 1 41 38 92 26 Germany: +49 1805 702 702 Netherlands: +31 306 022 797 United Kingdom: +44 1628 511 445
Standard Rate India: +91 80 5112 5738
Table of Contents Module
Page
0. Course Introduction
0-1
1. PowerCenter Overview
1-1
2. Mapping Fundamentals
2-1
3. Workflow Basics
3-1
4. Expression and Filter Transformations
4-1
5. Joining and Merging Data
5-1
6. Lookup Transformations
6-1
7. Sorter and Aggregator Transformations
7-1
8. Using the Debugger
8-1
9. Updating Target Tables
9-1
10. Mapping Techniques
10-1
11. Mapplets and Worklets
11-1
12. Controlling Workflows
12-1
13. Mapping Design Workshop
13-1
14. Workflow Design Workshop
14-1
15. PowerCenter 9.0 New Features
15-1
vii
0.1
Course Introduction
0
PowerCenter 8.x/9.0 Level 1 Developer
L1D_20081124GV9
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
0.2
Course Introduction
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
0.3
Course Introduction
0
Introductions • Logistics/Site Information • Introductions • About you • How do you expect to benefit from this course?
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
3 of 22
Copyright © 2010 Informatica Corp
0.4
Course Introduction
Course Audience • PowerCenter 8.x Level I Developer is designed for Developers and Consultants • This course enables participants to use the principal features of Informatica PowerCenter 8 for integrating data between disparate applications • This material assumes familiarity with database concepts and technology
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
4 of 22
Copyright © 2010 Informatica Corp
0.5
Course Introduction
0
Course Objectives • When you have completed this course, you should be able to: • Use PowerCenter 8 Designer to build mappings that move data from sources to targets • Use PowerCenter 8 Workflow Manager to build and run a workflow that runs a session based on a mapping • Design simple mappings and workflows based on business needs • Perform basic troubleshooting of PowerCenter mappings and transformations • Use Informatica Support options to resolve questions and problems about Informatica PowerCenter 8
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
5 of 22
Copyright © 2010 Informatica Corp
0.6
Course Introduction
Course Agenda 1. PowerCenter Overview 2. Mapping Fundamentals 3. Workflow Basics 4. Expression and Filter Transformations 5. Joining and Merging Data 6. Lookup Transformations 7. Sorter and Aggregator Transformations
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
6 of 22
Copyright © 2010 Informatica Corp
0.7
Course Introduction
0
Course Agenda Continued 8. Using the Debugger 9. Updating Target Tables 10. Mapping Techniques 11. Mapplets and Worklets 12. Controlling Workflows 13. Mapping Design Workshop 14. Workflow Design Workshop
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
7 of 22
Copyright © 2010 Informatica Corp
0.8
Course Introduction
Other Informatica Resources • In addition to the student guides, Informatica provides these other resources: • Corporate Website • Global Education Services • Customer Portal • Product Documentation • Knowledge Base • Technical Support • Informatica Product Certification
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
8 of 22
Copyright © 2010 Informatica Corp
0.9
Course Introduction
0
Informatica Corporate Website • Can be accessed at www.informatica.com • Website provides: • Corporate information and background • Upcoming events and access to Informatica Communities, Blogs and Tech Net • Nearest sales office location • Product information, literature and partner information • Information on technical support, training, education and implementation services Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
9 of 22
Copyright © 2010 Informatica Corp
0.10
Course Introduction
Informatica Education Services • Products + Services Æ Education Services •
www.informatica.com/products_services/education_services
• Discover new training offerings • New courses added regularly
• Learn about Informatica Certification
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
10 of 22
Copyright © 2010 Informatica Corp
0.11
Course Introduction
0
Informatica Customer Portal •
Available to Informatica customers and partners at http://my.informatica.com
•
Website provides: •
Product
documentation
•
Access to ATLAS system
•
Knowledge Base
•
Access to the User Community
•
User Group info Newsletters
•
Debugging Tools
•
Velocity
Informatica /
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
Support
11 of 22
Copyright © 2010 Informatica Corp
0.12
Course Introduction
Accessing the Customer Portal • Use your company email id to register • Provides default access to: • Most recent documentation • Knowledge base
• Default does not provide online support access • Request using Project ID • Once approved you can create Service Requests
• Available for 11 countries Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
12 of 22
Copyright © 2010 Informatica Corp
0.13
Course Introduction
0
Informatica Documentation • Can be accessed via: • Product CD or Download link • Online help • Documentation Center
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
13 of 22
Copyright © 2010 Informatica Corp
0.14
Course Introduction
Informatica Knowledge Base • Can be accessed via the Customer Portal http://my.informatica.com
• Contains • Documented solutions to known technical issues • Answers to frequentlyasked questions (FAQs) • White papers • Technical tips • Perform generic- or specific-searches Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
14 of 22
Copyright © 2010 Informatica Corp
0.15
Course Introduction
0
Informatica Technical Support • Preferred Support access via: •
Online using the Web Support Service using http://my.informatica.com username and password
• Alternative Support access via: •
Call / email your nearest Technical Support Center • See Student Note
•
Email Technical Support directly (may be delay in response): •
[email protected] •
[email protected] Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
15 of 22
Copyright © 2010 Informatica Corp
0.16
Course Introduction
Creating a Service Request • Check the Knowledge Base to see if a solution exists • If a solution exists, a Service Request is not required
• On the Online Support page, navigate to the Service Requests tab • Click the New button
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
16 of 22
Copyright © 2010 Informatica Corp
0.17
Course Introduction
0
Creating a Service Request Continued • On the service request form, fill out all required fields • Provide as much relevant detail as possible so engineers can recreate the problem, if necessary
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
17 of 22
Copyright © 2010 Informatica Corp
0.18
Course Introduction
PowerCenter Education and Certification Path PowerCenter 8.x
PowerCenter 8 Quickstart
PowerCenter 8.5+ Administrator
PowerCenter 8.x New Features
PowerCenter 8 Data Migration
PowerCenter 8.5 Unified Security
PowerCenter 8.x Upgrade
PowerCenter 8 Team-Based Development
PowerCenter 8 High Availability
PowerCenter 8 XML Support
PowerCenter 8 Data Cleanse and Match
B2B Data Transformation
PowerCenter 8.x Level I Developer
PowerCenter 8.x Level II Developer
v
Additional Recommended Training
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
18 of 22
Copyright © 2010 Informatica Corp
0.19
Course Introduction
0
PowerExchange Education Path
Introduction to PowerExchange
PowerExchange Basics
PowerCenter 8.5 Level I Administrator
PowerCenter 8 High Availability
PowerCenter 8 Team-Based Development
Additional Recommended Training
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
19 of 22
Copyright © 2010 Informatica Corp
0.20
Course Introduction
Data Quality Education and Certification Path
Informatica Data Quality 8.6 Workbench Level I
Informatica Data Explorer 8.6 Level I
Data Quality Assessment Using IDE
Informatica Data Quality 8.5 Cleansing Workshop
Data Quality Assessment Using IDQ
Informatica Data Quality 8.5 Matching Workshop
Data Quality Assessment Using IDQ Informatica Data Quality 8.6 New Features
Additional Recommended Training
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
20 of 22
Copyright © 2010 Informatica Corp
0.21
Course Introduction
0
Data Integration Certification Path Level
Certification Title
Recommended Training
Required Exams
Informatica Certified Administrator
» PowerCenter QuickStart (eLearning) » PowerCenter 8.5+ Administrator (4 days)
»Architecture & Administration; »Advanced Administration
Informatica Certified Developer
» PowerCenter QuickStart (eLearning) » PowerCenter 8.5+ Administrator (4 days) » PowerCenter Developer 8.x Level I (4 days) » PowerCenter Developer 8 Level II (4 days)
»Architecture & Administration; »Advanced Administration »Advanced Mapping Design
Informatica Certified Consultant
» PowerCenter QuickStart (eLearning) » PowerCenter 8.5+ Administrator (4 days) » PowerCenter Developer 8.x Level I (4 days) » PowerCenter Developer 8 Level II (4 days)
»Architecture & Administration; »Advanced Administration »Advanced Mapping Design »Advanced Admistration »Enablement Technologies
» PowerCenter 8 Data Migration (4 days) » PowerCenter 8 High Availability (1 day) Additional Training: » PowerCenter 8.5 New Features » PowerCenter 8.6 New Features » PowerCenter 8 Upgrade
» PowerCenter 8 Team-Based Development » PowerCenter 8.5 Unified Security `
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
21 of 22
Copyright © 2010 Informatica Corp
0.22
Course Introduction
Data Quality Certification Path Level
Certification Title
Recommended Training
Required Exams
Informatica Certified Specialist: Data Quality
» Informatica Data Quality 8.6 Level I (4 days)
»Data Quality 8.6
Informatica Certified Specialist: Data Explorer
» Informatica Data Explorer 8.6 Level I (2 days)
»Data Explorer 8.6
» Informatica Data Quality 8.6 Level I (4 days)
»Data Quality 8.6 »Data Explorer 8.6
Informatica Certified Professional
» Informatica Data Explorer 8.6 Level I (2 days)
Additional Training: » Data Quality Assessment Using Informatica Data Explorer » Data Quality Assessment Using Informatica Data Quality » Informatica Data Quality 8.5 Cleansing Workshop » Informatica Data Quality 8.5 Matching Workshop » Informatica Data Quality 8.6 New Features
Module 0: Course Introduction
PowerCenter 8.x/9.0 Level I Developer
22 of 22
Copyright © 2010 Informatica Corp
1.1
PowerCenter Overview
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
1.2
PowerCenter Overview
Module Objectives After completing this module you will be able to: • Explain the purposes of PowerCenter • Define terms used in PowerCenter • Name major PowerCenter components
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
2 of 32
Copyright © 2010 Informatica Corp
1.3
PowerCenter Overview
The Problem • Large organizations have a lot of data • The data can be stored in many formats, including databases and unstructured files
• This data must be collated, combined, compared, and made to work as a seamless whole • But the different databases don’t talk to each other!
Marketing (ORCL)
Manufacturing (DB2) Accounting (SAP)
Inventory (SQL Server)
Sales (SalesForce) Resource Planning (PSFT) Billing (Sybase)
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
3 of 32
Copyright © 2010 Informatica Corp
1.4
PowerCenter Overview
The Problem Continued • Many organizations have implemented connectors between these databases • Every pair of databases requires a unique connector Manufacturing (DB2) Marketing (ORCL)
Sales (SalesForce)
Accounting (SAP)
Billing (Sybase)
Inventory (SQL Server)
Resource Planning (PSFT)
Module 1: PowerCenter Overview
4 of 32
Connector
A connector is a piece of custom software that performs two functions: it converts data from the format of one application to the format of another application, and it transports the data between the two applications.
Technical Note
To be precise, to connect n databases fully requires (1+2+…+(n-1)) connectors.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
1.5
PowerCenter Overview
The Problem Continued • If you change one database, many connectors have to be upgraded
Manufacturing (DB2) Marketing (ORCL)
Sales (SalesForce)
Accounting (upgraded)
Billing (Sybase)
Inventory (SQL Server)
Resource Planning (PSFT)
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
5 of 32
Copyright © 2010 Informatica Corp
1.6
PowerCenter Overview
The Solution: Data Integration • Data Integration technologies allow applications using different databases and formats to communicate with each other using one connection each Manufacturing (DB2) Marketing (ORCL)
Sales (SalesForce) Data Integration!
Accounting (SAP)
Billing (Sybase)
Inventory (SQL Server)
Resource Planning (PSFT)
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
6 of 32
Copyright © 2010 Informatica Corp
1.7
PowerCenter Overview
Informatica PowerCenter • Informatica PowerCenter is the premium data integration solution available today • “Database neutral” – will communicate with any database • Powerful data transformations convert one application’s data to another’s format Manufacturing (DB2) Marketing (ORCL)
Sales (SalesForce) Informatica PowerCenter
Accounting (SAP)
Billing (Sybase) Inventory (SQL Server)
Resource Planning (PSFT)
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
7 of 32
Copyright © 2010 Informatica Corp
1.8
PowerCenter Overview
Example: Data Migration • A company purchases a new accounts payable application • PowerCenter can move the existing account data to the new application • Preserves data lineage for tax, accounting, and other legally mandated purposes
Accounting (old)
Informatica PowerCenter
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
Accounting (new)
8 of 32
Copyright © 2010 Informatica Corp
1.9
PowerCenter Overview
Example: Application Integration • Company A purchases Company B • To achieve the benefits of consolidation, Company B’s billing system must be integrated into Company A’s billing system
Billing A
Informatica PowerCenter
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
Billing B
9 of 32
Copyright © 2010 Informatica Corp
1.10
PowerCenter Overview
Example: Data Warehousing • Data warehouses put information from many sources together for analysis • Data is moved from many databases to the warehouse Manufacturing (DB2)
Marketing (ORCL)
Inventory (SQL Server)
Accounting (SAP)
Informatica PowerCenter
Sales (SalesForce) Resource Planning (PSFT)
Data Warehouse
Module 1: PowerCenter Overview
Additional Examples of Data Integration
Billing (Sybase)
10 of 32
In addition to the examples given on these slides, PowerCenter is deployed for: • Data Synchronization – ongoing exchange of data between disparate applications • Data Hubs – master data management; reference data hubs; single view of customer, product, supplier, employee, etc. • Business Activity Monitoring – business process improvement, real-time reporting
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
1.11
PowerCenter Overview
Data Integration Methodology: ETL Source(s)
•Extract the data from a source system •Transform it as needed Data Integration!
•Load it to a target system Target(s)
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
11 of 32
Copyright © 2010 Informatica Corp
1.12
PowerCenter Overview
ETL for Data Warehousing Decision Support
Operational Systems RDBMS
Mainframe
Data Warehouse
Other
• Transaction data • Optimized for transaction response time • Current • Normalized or De-normalized data
Extract
• Aggregate data • Cleanse data • Consolidate data • Apply business rules • De-normalize data
Transform ETL
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
• Aggregated data • Historical data
Load 12 of 32
Copyright © 2010 Informatica Corp
1.13
PowerCenter Overview
ETL: Extract • PowerCenter reads data, row by row, from a table (or group of related tables) in a database, or from a file • This database or file is referred to as the source • The structure of the source is contained in a source definition object. Extract
Source
Informatica PowerCenter
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
13 of 32
Copyright © 2010 Informatica Corp
1.14
PowerCenter Overview
ETL: Transform • PowerCenter converts the rows into a format the second (target) system will be able to use • The logic for this conversion is defined in transformation objects
Extract
Source
Transform
Informatica PowerCenter
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
14 of 32
Copyright © 2010 Informatica Corp
1.15
PowerCenter Overview
ETL: Load • PowerCenter writes data, row by row, to a table (or group of related tables) in a database, or to a file • This database or file is referred to as the target • The structure of the target is contained in a target definition object Extract
Source
Transform
Load
Informatica PowerCenter
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
Target
15 of 32
Copyright © 2010 Informatica Corp
1.16
PowerCenter Overview
Mapping • A set of transformations, in sequence or in parallel, that move and transform data from one or more source(s) to one or more target(s) • Mappings exist entirely inside PowerCenter Informatica PowerCenter
Source(s)
Target(s)
transformations
Module 1: PowerCenter Overview
Mappings
mapping
16 of 32
A mapping logically defines the ETL process. It reads data from sources, applies transformation logic to the data, and writes the transformed data to targets.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
1.17
PowerCenter Overview
Transformations • Transformations receive data and transform it • Generate new fields • Modify data • Select and pass data
Transformations
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
17 of 32
Copyright © 2010 Informatica Corp
1.18
PowerCenter Overview
Session • The object that runs a mapping
Session
Mapping
Transformation
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
18 of 32
Copyright © 2010 Informatica Corp
1.19
PowerCenter Overview
Workflow • An ordered set of one or more sessions and other tasks, designed to accomplish an overall operational purpose
Workflow
Other Tasks
Session
Mapping
Transformation
19 of 32
Module 1: PowerCenter Overview
Tasks
A task is an executable set of actions, functions, or commands. A session is a task that runs a mapping. Other tasks include: • Command – runs a shell script • Email – sends an email • Decision – branches a workflow conditionally • Timer – waits for a defined period
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
1.20
PowerCenter Overview
Metadata • Defines data and processes • Examples: • Source and target definitions • Type (flat file, database table, XML file, etc) • Datatype (character string, integer, decimal, etc) • Other attributes (length, precision, etc.)
• Mapping logic • Workflow logic
• Stored in a metadata repository
Repository
Module 1: PowerCenter Overview
20 of 32
Note
The repository is implemented as a schema, which may reside in any of a number of supported relational database management systems.
Metadata
The word metadata literally means “data about data.” It is the information that describes data. Common contents of metadata include the source of a dataset, how it should be accessed, and its limitations.
Metadata in PowerCenter PowerCenter uses metadata to define sources, targets, transformations, mappings, and workflows
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
1.21
PowerCenter Overview
PowerCenter 8.5 Architecture ODBC
Sources
Native drivers/ ODBC
Domain Integration Service
Native drivers/ ODBC
Targets
TCP/IP TCP/IP ODBC TCP/IP
Repository Service
HTTPS
Administration Console
Repository Service Process
Security Domain Native drivers
PowerCenter Client
Native drivers
Repository
Domain Metadata 21 of 32
Module 1: PowerCenter Overview
Sources and Targets
Can be relational tables or heterogeneous files (such as flat files, VSAM files, or XML)
Integration Service
The engine which performs all the ETL logic
Repository Service
Manages connectivity to metadata repositories that contain mapping and workflow definitions
Repository Service Process
Multithreaded process that retrieves, inserts, and updates repository metadata
Repository
Contains all the metadata needed to run the ETL process
Client Tools
Desktop tools used to populate the repository with metadata, execute workflows on the Integration Service, monitor the workflows, and manage the repository
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
1.22
PowerCenter Overview
PowerCenter Client Applications Administration
Development
Administration Console
Repository Manager Manage repository • connections • folders • objects • users and groups (in PowerCenter 8.1)
Note
Administration Console (browser-based)
Designer
Perform domain and repository service tasks: • Create/configure nodes and repository services • Upgrade/delete • Start/stop • Backup/restore • Manage users and groups (In PowerCenter 8.5)
Create ETL mappings
Monitor and Create and start control workflows workflows
Module 1: PowerCenter Overview
22 of 32
Workflow Manager
Workflow Monitor
Designer and Repository Manager access the repository through the Repository Service. Workflow Manager and Workflow Monitor connect to the Integration Service. Each client has its own user interface. The UIs typically have toolbars, a navigation window to the left, a workspace to the right, and an output window at the bottom.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
1.23
PowerCenter Overview
Designer: Create Mappings, Sources, Targets
Transformation toolbar
Client Apps* Designer Tools
Workspace Navigator Window
Output Window Status Bar
* Single login to client applications – click on icons to open other tools – they are already connected to repository Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
23 of 32
Copyright © 2010 Informatica Corp
1.24
PowerCenter Overview
Designer Tools
Target Transformation Mapplet Source Designer: Developer: Designer: Analyzer: create create source create target create reusable objects transformations mapplets objects
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
Mapping Designer: create mappings
24 of 32
Copyright © 2010 Informatica Corp
1.25
PowerCenter Overview
Transformation Views Iconized – shows the transformation in relation to the rest of the mapping Normal – shows the flow of data through the transformation
Edit – shows transformation ports and properties; allows editing Ports represent table columns or file fields
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
25 of 32
Copyright © 2010 Informatica Corp
1.26
PowerCenter Overview
Workflow Manager: Create and Run Workflows Connections
Client Applications
Tasks
Workflow Manager Tools
Navigator Window
Workspace
Output Window
Status Bar
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
26 of 32
Copyright © 2010 Informatica Corp
1.27
PowerCenter Overview
Workflow Manager Tools
Create reusable tasks
Create worklets
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
Create workflows
27 of 32
Copyright © 2010 Informatica Corp
1.28
PowerCenter Overview
Session Tasks: Run a Single Mapping Iconized view – shows the task in relation to the rest of the workflow Edit view – shows task properties, allows editing
Two or more Sessions can point to a single mapping
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
28 of 32
Copyright © 2010 Informatica Corp
1.29
PowerCenter Overview
Workflow Monitor: View Workflow Progress
Navigator Window
Output Window
Gantt Chart View
Task View
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
Time Window
29 of 32
Copyright © 2010 Informatica Corp
1.30
PowerCenter Overview
Repository Manager: Administer Folders
Navigator Window
Status Bar
Output Window
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
Main Window
30 of 32
Copyright © 2010 Informatica Corp
1.31
PowerCenter Overview
Class Scenario
Business Function: OLTP
Business Function: Staging
Business Function: ODS/EDW
Business Function: DDW
Data in an online Transaction processing system
Data format similar to OLTP, used for populating ODS
Operational Data Store or Electronic Data Warehouse is normalized data in an enterprise data Model that aligns data from various OLTP Systems
Dimensional Data Warehouse uses Fact and Dimension tables in second normal form to speed report generation and allow for historical data
DB Schema: SDBU
DB Schema: STGxx
DB Schema: ODSxx
Covered in Level two Developer class
Module 1: PowerCenter Overview
Note
31 of 32
In the labs for this course, we are simulating part of the creation of a (very simple) Dimensional Data Warehouse. In these labs, you will begin with data in OLTP tables and flat files, bring data to Staging, and from Staging (STG) to the Operational Data Store (ODS). Because creation of Staging tables is fairly trivial, you will do more work on moving data from STG to ODS. This will provide more realistic uses of the capabilities of PowerCenter.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
1.32
PowerCenter Overview
Summary This module showed you how to: • Explain the purposes of PowerCenter • Define terms used in PowerCenter • Name major PowerCenter components
Module 1: PowerCenter Overview
PowerCenter 8.x/9.0 Level I Developer
32 of 32
Copyright © 2010 Informatica Corp
2.1
Mapping Fundamentals
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
2.2
Mapping Fundamentals
Module Objectives After completing this module you will be able to • Create source and target definitions from flat files and relational tables • Create a mapping using existing source and target definitions • Use links to connect ports
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
2 of 22
Copyright © 2010 Informatica Corp
2.3
Mapping Fundamentals
PowerCenter Transformations (partial list) Source Qualifier: reads from data sources Expression: performs row-level calculations Filter: uses a condition statement to select rows for inclusion Sorter: sorts data Aggregator: performs aggregate calculations Joiner: joins heterogeneous sources Lookup: looks up values and passes them to other objects Update Strategy: tags rows for insert, update, delete, reject Router: routes rows conditionally Union: Performs a union-all join between two data streams Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
3 of 22
Copyright © 2010 Informatica Corp
2.4
Mapping Fundamentals
Specialized PowerCenter Transformations
Transaction Control: allows data-driven commits and rollbacks Java: allows Java code to be used within PowerCenter Midstream XML Parser: parses XML anywhere in a mapping Midstream XML Generator: creates XML anywhere in a mapping
More Source Qualifiers: read from XML, message queues and applications
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
4 of 22
Copyright © 2010 Informatica Corp
2.5
Mapping Fundamentals
PowerCenter Designer • Provides tools to define and manipulate • • • • •
Sources Targets Transformations Mappings Other objects
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
5 of 22
Copyright © 2010 Informatica Corp
2.6
Mapping Fundamentals
Repositories • Objects are stored as metadata in repositories • Within a repository, objects are organized in folders Repository
Folder
Sub folders
Module 2: Mapping Fundamentals
6 of 22
Repository Management
Repositories are not created and managed in the Designer application. They are created in the Administration Console application, and managed in the Repository Manager application.
Folder Management
Folders are created and managed in the Repository Manager application.
Shortcut Folders
Do not confuse repository folders with the directories visible in Windows Explorer. The folders are PowerCenter repository objects and are not related to Windows directories. Technically, all folders are “shared” with all users who have the appropriate folder permissions, regardless of the “blue arm” icon. The “blue arm” icon indicates that the folder permits shortcuts, dynamic links to the objects contained in that folder used by mappings in other folders.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
2.7
Mapping Fundamentals
Source Definitions • Defines the structure of a data source such as a relational database table or a flat file • Created using the Source Analyzer in the PowerCenter Designer application • Enables you to preview the data in the source
Relational Database Connections
Flat file sources
Tables in DB source OLTP
Module 2: Mapping Fundamentals
Note
7 of 22
Two sources from different systems may use the same name. Placing each source in a folder based on its connection type avoids confusion when this is the case.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
2.8
Mapping Fundamentals
Source Analyzer • The Designer tool for creating source definitions • Import source definitions from: • • • • •
Relational databases Flat files XML sources Cobol sources Applications such as SAP, Siebel, and PeopleSoft • PowerExchange • Mainframes
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
8 of 22
Copyright © 2010 Informatica Corp
2.9
Mapping Fundamentals
Source Qualifier Transformation • Selects records from flat file and relational table sources and converts to PowerCenter internal datatypes
Module 2: Mapping Fundamentals
9 of 22
Type
Active
Description
Mandatory for all flat file and relational sources in a mapping. Selects records from flat file and relational table sources. For relational tables, creates a SQL SELECT statement. Converts native source datatypes to PowerCenter transformation datatypes.
Ports
Represent table columns or file fields. All ports are input/output.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
2.10
Mapping Fundamentals
Lab 2-1 – Creating Source Definitions In the lab you will: • Use wizards to import Source definitions from a flat file and a relational database table • Preview the data in the Sources
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
10 of 22
Copyright © 2010 Informatica Corp
2.11
Mapping Fundamentals
Target Definitions • Define the structure of a target such as a relational database table or a flat file • Created using the Target Designer in the PowerCenter Designer application • Can be • Copied from a source object • Imported from relational database • Imported from flat file • A shortcut to a target in another folder
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
11 of 22
Copyright © 2010 Informatica Corp
2.12
Mapping Fundamentals
Target Designer • The Designer tool for creating targets • Wizards import target definitions from • • • •
Flat files Relational database tables XML Applications such as SAP BF and MQ Series
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
12 of 22
Copyright © 2010 Informatica Corp
2.13
Mapping Fundamentals
Lab 2-2 – Creating Target Definitions In the lab you will: • Create a Target definition from scratch • Create a Target definition from a Source definition and change the Target type
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
13 of 22
Copyright © 2010 Informatica Corp
2.14
Mapping Fundamentals
Shortcuts • A shortcut is a dynamic link to the original object • Usually a target or source definition
• The original object must be in a shortcut folder • Not all folders permit creation of shortcuts • Shortcut folders display with a “hand” (similar to a shared folder in Windows Explorer)
• To create a shortcut drag the object to another open folder or any workspace where the object type is allowed • Shortcuts appear in the Navigator window with a small curved arrow
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
14 of 22
Copyright © 2010 Informatica Corp
2.15
Mapping Fundamentals
Datatypes • PowerCenter must “know” the data types used internally by both source and target systems • These are called native datatypes
• The Source Qualifier converts the source data to a standard format used internally by PowerCenter • This is the integration or transformation datatype
• The target definition object converts it to the native datatype of the target system
Module 2: Mapping Fundamentals
15 of 22
Note
The integration datatype standardizes transformations, and is easily translated from and to the native datatypes of application databases
Note
Integration datatypes are sometimes referred to as “ASCII Neutral”
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
2.16
Mapping Fundamentals
Datatype Conversion NATIVE DATATYPES
TRANSFORMATION DATATYPES
Specific to the source and target database types Display in source and target tables within Mapping Designer
Source DB native datatypes
PowerCenter internal datatypes Display in transformations within Mapping Designer
Transformation datatypes
Target DB native datatypes
Transformation datatypes allow mix and match of source and target database types When connecting ports, native and transformation datatypes must be compatible (or must be explicitly converted) 16 of 22
Module 2: Mapping Fundamentals
Datatype Conversion
Datatypes can be converted by • Passing data between ports with different datatypes • Passing data from an expression to a port • Using transformation functions • Using arithmetic operators The following type conversions are supported: • Numeric datatypes l other numeric datatypes • Numeric datatypes l string • Date/Time l date or string For further information, in the PowerCenter client, consult Help Æ Content Æ Index Æ port-to-port data conversion
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
2.17
Mapping Fundamentals
Transformation Ports • Data passes into and out of transformations through input and output ports Input Ports
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
Output Ports
17 of 22
Copyright © 2010 Informatica Corp
2.18
Mapping Fundamentals
Passive transformations • One row comes in, one row goes out – one-for-one input to output • Same number of rows output as input
• Example: a transformation that converts character data to numerical data 3 rows in
“(510) 555-7283” “(613) 382-2929” “(650) 385-5000”
3 rows out Passive Transformation
5105557283 6133822929 6503855000
Module 2: Mapping Fundamentals
18 of 22
Note
The transformation is considered “passive” regardless of what transformations take place within a row, provided only that the rows going out are a one-for-one match with the rows going in.
Examples
Examples of passive transformations include Expression and Lookup transformations.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
2.19
Mapping Fundamentals
Active Transformations • Many rows in, many rows out • May not be the same number output as input
• Example: a transformation that discards rows with bad data 3 rows in
2 rows out
5105557283
“(510) 555-7283” (613) 382-299 (650) 385-5000
Active Transformation
6503855000
Module 2: Mapping Fundamentals
Examples
19 of 22
Examples of active transformations include Filter, Router, and Sorter transformations.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
2.20
Mapping Fundamentals
Velocity Methodology Informatica’s Velocity methodology includes: • Templates • Mapping specification templates • Source to target field matrix
• Naming conventions • Object type prefixes: m_, exp_, agg_, wfl_, s_, …
• Best practices
20 of 22
Module 2: Mapping Fundamentals
Velocity Phases
Velocity covers the entire data integration project lifecycle: Phase 1: Manage Phase 2: Architect Phase 3: Design Phase 4: Build Phase 5: Deploy Phase 6: Operate For more information, see http://devnet.informatica.com
Note
We do not use the Velocity templates in this course
Note
In Velocity, the standard is to begin the names of all Source Qualifier objects with “SQ_”, followed by the name of the source. The Mapping Designer automatically names the Source Qualifier this way when you drag a Source object onto the Mapping canvas.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
2.21
Mapping Fundamentals
Lab 2-3 – Creating Mappings • In the lab you will: • Create shortcuts to objects in the shortcut (SC) folder • Create a pass-through Mapping that brings data from a single Source to a single Target
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
21 of 22
Copyright © 2010 Informatica Corp
2.22
Mapping Fundamentals
Summary This module showed you how to • Create source and target definitions from flat files and relational tables • Create a mapping using existing source and target definitions • Use links to connect ports
Module 2: Mapping Fundamentals
PowerCenter 8.x/9.0 Level I Developer
22 of 22
Copyright © 2010 Informatica Corp
3.1
Workflow Basics
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
Copyright © 2010 Informatica Corp
3.2
Workflow Basics
Module Objectives After completing this module you will be able to: • Create a basic Workflow and link its tasks • Run a Workflow, monitor its execution, and verify the results
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
2 of 13
Copyright © 2010 Informatica Corp
3.3
Workflow Basics
PowerCenter Tasks Session: Run the logic of a mapping Command: Run external commands Email: Send an email to a defined recipient Decision: Choose between paths in a Workflow Assignment: Assign values to variables Timer: Wait or pause for specified time Control: Terminate or fail a Workflow Event Wait: Wait for an event Event Raise: Cause an Event Wait task to trigger Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
3 of 13
Copyright © 2010 Informatica Corp
3.4
Workflow Basics
Workflow Object • Executes a series of Mappings (as Sessions) and other tasks
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
4 of 13
Copyright © 2010 Informatica Corp
3.5
Workflow Basics
Workflow Manager • The Workflow Manager is the PowerCenter application that enables designers to build and run workflows • Can be launched from Designer by clicking the “W” icon
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
5 of 13
Copyright © 2010 Informatica Corp
3.6
Workflow Basics
Workflow Designer • The tool in Workflow Manager where you create Workflow objects
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
6 of 13
Copyright © 2010 Informatica Corp
3.7
Workflow Basics
Start Task • Is always the first task in a Workflow
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
7 of 13
Copyright © 2010 Informatica Corp
3.8
Workflow Basics
Session Task • Implements the execution of a Mapping
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
8 of 13
Copyright © 2010 Informatica Corp
3.9
Workflow Basics
Links in Workflows • Indicate the flow of control from one task to the next • The flow may branch either: • Unconditionally (multiple links are followed from a single task) • Using Decision tasks (only one branch is followed)
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
9 of 13
Copyright © 2010 Informatica Corp
3.10
Workflow Basics
Running a Workflow • Right-click in the Workflow and select “Start Workflow” • Passes control to Workflow Monitor
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
10 of 13
Copyright © 2010 Informatica Corp
3.11
Workflow Basics
Workflow Monitor • Displays all Workflows and tasks in real time • Provides access to logs and results
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
11 of 13
Copyright © 2010 Informatica Corp
3.12
Workflow Basics
Lab 3-1: Create and Run Workflows • In this lab you will… • Create and run Workflows that run the Mappings you created in Lab 3
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
12 of 13
Copyright © 2010 Informatica Corp
3.13
Workflow Basics
Summary This module showed you how to: • Create a basic Workflow and link its tasks • Run a Workflow, monitor its execution, and verify the results
Module 3: Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
13 of 13
Copyright © 2010 Informatica Corp
Workflow Basics
PowerCenter 8.x/9.0 Level 1 Developer
3.14
Copyright © 2010 Informatica Corp
4.1
Expression and Filter Transformations
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
Copyright © 2010 Informatica Corp
4.2
Expression and Filter Transformations
Module Objectives After completing this module you will be able to: • Use Expression transformations to perform calculations on a row-by-row basis • Use Filter transformations to pass rows based on userdefined conditions
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
2 of 19
Copyright © 2010 Informatica Corp
4.3
Expression and Filter Transformations
Expression Transformation Performs row-level calculations (no aggregate functions)
Click here to invoke the Expression Editor
3 of 19
Module 4: Expression and Filter Transformations
Type
Passive
Description
Modifies individual ports (columns) within a single row. Can add and suppress ports. Cannot perform aggregation across multiple rows.
Business Purpose
Use the logical and arithmetic operators and built-in functions for: • Character manipulation (concatenate, truncate, etc.) • Datatype conversion (to char, to date, etc.) • Data cleansing (check nulls, replace strings, etc.) • Data manipulation (round, truncate, etc.) • Numerical calculations • Scientific calculations • Special functions (lookup, decode, etc.) • Testing (for spaces, number, etc.)
PowerCenter 8.x/9.0 Level 1 Developer
Copyright © 2010 Informatica Corp
4.4
Expression and Filter Transformations
Uses of Expression Transformation: Examples • • • • •
Adjust employee salaries Concatenate first and last names Convert strings to numbers Arithmetic – add, subtract, multiply, divide numeric values Test conditional statements and pass the results to target tables or other transformations
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
4 of 19
Copyright © 2010 Informatica Corp
4.5
Expression and Filter Transformations
Expression Editor • An expression is a calculation or conditional statement for a specific port • Can contain other ports, functions, operators, variables, constants and return values from other transformations
Module 4: Expression and Filter Transformations
Comments
5 of 19
Comments can be added to expressions by prefacing them with //. This allows later developers to understand the logic behind an expression.
PowerCenter 8.x/9.0 Level 1 Developer
Copyright © 2010 Informatica Corp
4.6
Expression and Filter Transformations
Functions Provided Character manipulation e.g. CONCAT, LTRIM, UPPER Datatype conversion e.g. TO_CHAR, TO_DECIMAL Detect & correct errors e.g. ISNULL, REPLACECHR Manipulate dates e.g. GET_DATE_PART, DIFF_DATES Mathematical operations e.g. LOG, POWER, SQRT More mathematical operations e.g. SIN, COS, TAN Special constructs, e.g. IIF, DECODE Test values e.g. ISNULL, IS_DATE, IS_NUMBER Update variables e.g. SETVARIABLE, SETMINVARIABLE Module 4: Expression and Filter Transformations
6 of 19
Tip
Highlighting a function and pressing F1 will launch the online help at the selected function section.
Note
All expressions resolve to a single value of a specific datatype. For example, the expression LENGTH (“HELLO WORLD”) / 2 returns the numerical value 5.5.
PowerCenter 8.x/9.0 Level 1 Developer
Copyright © 2010 Informatica Corp
4.7
Expression and Filter Transformations
Variable Ports Use to: Simplify complex expressions Example: extract month from a date for use in several output ports
Provide storage for temporary values Improve efficiency
Variable ports are not visible in Normal view, only in Edit view
Module 4: Expression and Filter Transformations
7 of 19
Note
Variable ports cannot be output directly. To output the contents of a variable port, create an output port whose value is the variable.
Creating
A transformation variable is created by creating a port and selecting the V check box. When V is checked, the I and O checkboxes are unavailable (grayed out), indicating that a variable port can not be used for input or output.
PowerCenter 8.x/9.0 Level 1 Developer
Copyright © 2010 Informatica Corp
4.8
Expression and Filter Transformations
Variable Ports (Continued)
Variables are initialized (numeric to 0, string to “”) when the Mapping logic is processed
Module 4: Expression and Filter Transformations
Note
8 of 19
When a record is processed, the expression is evaluated and the result is assigned to the variable port. The result must be compatible with the port’s datatype. The variable persists across the set of records, and may be used or modified anywhere in the set of records.
PowerCenter 8.x/9.0 Level 1 Developer
Copyright © 2010 Informatica Corp
4.9
Expression and Filter Transformations
Order of Evaluation • PowerCenter evaluates ports in the following order: • Input and Input/Output ports • Variable ports • Output ports
• Variable ports are evaluated in the order they appear in the Ports tab • Order of evaluation is critical when one variable refers to another • Reference to a variable that has not yet been evaluated will use the value from the previous row • This is always Null for the first row in the datastream – can cause errors!
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
9 of 19
Copyright © 2010 Informatica Corp
4.10
Expression and Filter Transformations
Expression Validation The Validate and OK buttons in the Expression Editor both parse the current expression to: Resolve references to ports in other transformations Parse default values Check spelling, correct number of arguments in functions, other syntactical errors
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
10 of 19
Copyright © 2010 Informatica Corp
4.11
Expression and Filter Transformations
Expression Example 1 • Check, Clean, and Record Errors • Clean up item name • Some item names are in UPPERCASE, some in lower case, some in MiXEd • They should all be in Title Case
• Missing data • Some records are incomplete
• Invalid dates • Sometimes dates are input in an invalid format
• Invalid numbers • Certain numeric fields sometimes contain nonnumeric data
• Reporting • Need a count of the changes to item names • Incorrect and missing data should be tagged and a report generated
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
11 of 19
Copyright © 2010 Informatica Corp
4.12
Expression and Filter Transformations
Expression Example 1 Solution • Item names can be set to title case using the INITCAP function • The ISNULL function will detect null values • The IS_DATE and IS_NUMBER functions can check for correct formatting • Variables can be used to count the number of errors detected or corrected
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
12 of 19
Copyright © 2010 Informatica Corp
4.13
Expression and Filter Transformations
Expression Example 2 • Calculate Sales Discounting and Inventory Days • Discount tracking • Compare the suggested sale price to the actual sale price to determine the level of discounting • Create a field that tracks this comparison for reporting
• Days in Inventory • Determine how long a given item has been in inventory
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
13 of 19
Copyright © 2010 Informatica Corp
4.14
Expression and Filter Transformations
Performance Considerations for Expressions • Avoid multiple conversions of the same data • Decimal Æ String Æ Decimal • Integer Æ Decimal Æ Integer • Date Æ String Æ Date
• Strings are more ‘expensive’ for the server to process than mathematical calculations • Avoid default data type/size for new ports (string, 10) unless applicable
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
14 of 19
Copyright © 2010 Informatica Corp
4.15
Expression and Filter Transformations
Filter Transformation Passes rows conditionally
Module 4: Expression and Filter Transformations
15 of 19
Type
Active
Description
Allows rows which meet the filter condition are passed through to the next transformation. Rows which do not meet the filter condition are skipped.
Business Purpose
A business may choose not to process records which do not meet a data quality criterion.
PowerCenter 8.x/9.0 Level 1 Developer
Copyright © 2010 Informatica Corp
4.16
Expression and Filter Transformations
Filter Example • Existing customer records need to be updated to reflect changes to columns such as address • However, only existing customer records are to be updated – not new customer records
• Use a Filter transformation to pass only customer records with a legitimate master customer ID number
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
16 of 19
Copyright © 2010 Informatica Corp
4.17
Expression and Filter Transformations
Lab 4-1 – Using Filters and Expressions • In this lab you will… • Move data from the Customer staging table to the ODS database • Use an Expression transformation to reformat data • Use a Filter transformation to pass only valid records
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
17 of 19
Copyright © 2010 Informatica Corp
4.18
Expression and Filter Transformations
Lab 4-2 – Features and Techniques • In this lab you will… • Learn some features and techniques that will increase your efficiency as a PowerCenter Developer
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
18 of 19
Copyright © 2010 Informatica Corp
4.19
Expression and Filter Transformations
Summary This module showed you how to: • Use Expression transformations to perform calculations on a row-by-row basis • Use Filter transformations to pass rows based on userdefined conditions
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
19 of 19
Copyright © 2010 Informatica Corp
4.20
Expression and Filter Transformations
Module 4: Expression and Filter Transformations
PowerCenter 8.x/9.0 Level 1 Developer
20 of 19
Copyright © 2010 Informatica Corp
5.1
Joining and Merging Data
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
5.2
Joining and Merging Data
Module Objectives After completing this module you will be able to: • Use pipeline branches in Mappings • Join homogeneous Sources using a Source Qualifier • Define heterogeneous joins using the Joiner transformation • Merge records from multiple places into one record set using the Union transformation • Create and use reusable transformations
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
2 of 22
Copyright © 2010 Informatica Corp
5.3
Joining and Merging Data
Source Pipelines • Each Source Qualifier transformation starts a single Source pipeline • A single Mapping can have multiple Source pipelines • Each pipeline must terminate with at least one Target • Transformations can split one source pipeline into multiple pipeline branches 1 mapping
T Source pipeline
SQ
T
2 pipeline branches
T T
T
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
T T
3 of 22
Copyright © 2010 Informatica Corp
5.4
Joining and Merging Data
Multiple Pipelines • A single Mapping may contain more than one pipeline
SQ
T
T
T
T
2 source pipelines
1 mapping
SQ
T
T
T
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
T
4 of 22
Copyright © 2010 Informatica Corp
5.5
Joining and Merging Data
Homogeneous Joins • Homogeneous joins combine data from tables in the same database related by a common field • The join is specified in the Source Qualifier transformation • The join is performed on the Source database at runtime • When SQL generated by the SQ transformation executes
Source 1
SQ
T
Source 2
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
5 of 22
Copyright © 2010 Informatica Corp
5.6
Joining and Merging Data
Defining Homogeneous Joins • If the tables have a Primary/Foreign Key relationship in the database, a join condition is automatically created • You can enter a join condition, such as tablea.employee_id=tableb.employee_id
• Default is inner join
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
6 of 22
Copyright © 2010 Informatica Corp
5.7
Joining and Merging Data
Example of Homogeneous Join • A business updates its Product ODS table regularly • This requires a join of Product and Product-Cost
• The two tables • Are in the same database • Have a primary/foreign key relationship based on Product_ID
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
7 of 22
Copyright © 2010 Informatica Corp
5.8
Joining and Merging Data
Heterogeneous Joins • Heterogeneous joins are joins using dissimilar sources, such as
• Oracle table and DB2 table • Flat file and database table • Two flat files • Use a Joiner transformation (performs the join within the mapping) • One source is designated the Master, the other Detail • The Joiner selects rows from the two sources based on a join condition, such as a matching ID field
Master Source Joiner Join Condition: Master.CustomerID = Detail.CustomerID
Detail Source
Module 5: Joining and Merging Data
Pipelines
Join Results
8 of 22
A Joiner transformation combines two pipelines into a single transformation. Specifically, the pipeline from the Detail source ends at the Joiner, flowing into the pipeline from the Master source.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
5.9
Joining and Merging Data
Joiner Transformation Performs heterogeneous joins on two data flows
Ports • Input or Input/Output • One Source is designated Master, the other Detail • M property indicates ports from the Master Source when checked
Module 5: Joining and Merging Data
9 of 22
Type
Active
Description
Combines fields from two data sources into a single combined data source, based on one or more common fields called the join condition
Business Purpose
Enables data from different systems to be combined to achieve desired structure and results
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
5.10
Joining and Merging Data
Joiner Example • Sales transaction data resides on a flat file • Product data resides on a relational table • Sales transactions require product data
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
10 of 22
Copyright © 2010 Informatica Corp
5.11
Joining and Merging Data
Join Types • Normal (inner) join – keeps only matching rows based on the condition • Master outer join - keeps all rows from Detail and matching rows from Master • Detail outer join - keeps all rows from Master and matching rows from Detail • Full outer join - keeps all rows from both Master and Detail
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
11 of 22
Copyright © 2010 Informatica Corp
5.12
Joining and Merging Data
Specify Join Type
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
12 of 22
Copyright © 2010 Informatica Corp
5.13
Joining and Merging Data
Specify Join Condition
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
13 of 22
Copyright © 2010 Informatica Corp
5.14
Joining and Merging Data
Joiner Cache • Two types of cache memory: index cache and data cache • All rows from the master Source are read into cache • Index cache contains values for all ports from the master Source which are part of the join condition • Data cache contains those port values not specified in the join condition
• After the cache is loaded, each row in the detail Source is compared to the values in the index cache • Upon a match, the rows from the data cache are included in the outgoing data stream
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
14 of 22
Copyright © 2010 Informatica Corp
5.15
Joining and Merging Data
Enable Automatic Memory Settings • Configure values for: • Maximum Memory Allowed for Automatic Memory Attributes – numeric value • Maximum Percentage of Total Memory Allowed for Automatic Memory Attributes - %ile value
• Must set for both values • The Integration Service compares the two and uses the lower value as the maximum memory limit • If this value is set to 0 for either attribute, the Integration Service disables automatic memory settings and uses default values.
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
15 of 22
Copyright © 2010 Informatica Corp
5.16
Joining and Merging Data
Union Transformation • Merges row sets from multiple pipelines
One output port group
One or more input port groups
Module 5: Joining and Merging Data
16 of 22
Type
Active
Description
Merges data from multiple pipelines or pipeline branches to a single pipeline branch, similar to the SQL statement UNION ALL. Does not remove duplicate rows.
Business Purpose
Enables you to convert data from multiple sources into a single rowset
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
5.17
Joining and Merging Data
Union Groups • Create groups on the Groups tab • Each group has an identical set of ports
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
17 of 22
Copyright © 2010 Informatica Corp
5.18
Joining and Merging Data
Group Ports • Create on the Group Ports tab
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
18 of 22
Copyright © 2010 Informatica Corp
5.19
Joining and Merging Data
Union Ports • Each group port created appears in the Output group (i.e., “PRODUCT_ID” • Copies of each port appear in the Input groups (i.e., PRODUCT_ID2, PRODUCT_ID3)
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
19 of 22
Copyright © 2010 Informatica Corp
5.20
Joining and Merging Data
Reusable Transformations • Create in Transformation Developer • Or create in Mapping Designer and ‘promote’
• Listed in Transformations node of navigator • Drag and drop into any mapping
Module 5: Joining and Merging Data
20 of 22
Warning
Instance of reusable transformations inherit any changes to the reusable transformation. As a result, changing a reusable transformation may invalidate the mappings containing instances of the transformation.
Note
To make a non-reusable copy of a reusable transformation, hold the Ctrl key while dragging and dropping.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
5.21
Joining and Merging Data
Lab 5-1 Joining Data • In this lab you will… • • • •
Create a Mapping with two data flows Create a homogeneous join using a Source Qualifier Make a transformation reusable Create a heterogeneous join using a Joiner transformation
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
21 of 22
Copyright © 2010 Informatica Corp
5.22
Joining and Merging Data
Summary This module showed you how to: • Use pipeline branches in Mappings • Join homogeneous Sources using a Source Qualifier • Define heterogeneous joins using the Joiner transformation • Merge records from multiple places into one record set using the Union transformation • Create and use reusable transformations
Module 5: Joining and Merging Data
PowerCenter 8.x/9.0 Level I Developer
22 of 22
Copyright © 2010 Informatica Corp
6.1
Lookup Transformations
Module 6: Lookup Transformations
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
6.2
Lookup Transformations
Module Objectives After completing this module you will be able to: • Use Lookup transformations to bring in additional data related to a row
Module 6: Lookup Transformations
PowerCenter 8.x/9.0 Level I Developer
2 of 12
Copyright © 2010 Informatica Corp
6.3
Lookup Transformations
Lookup Functionality
Lookup Data Input value Input value
Lookup value(s)
Lookup Transformation
Output with Lookup values
Lookup condition: ITEM_ID = IN_ITEM_ID PRICE 75, DD_INSERT, DD_REJECT)
Expression is evaluated for each row Rows are tagged according to the logic of the expression Appropriate SQL is submitted to the target database: insert, delete or update DD_REJECT means the row will not have SQL written for it. Rejected rows may be forwarded through the mapping
Module 9: Updating Target Tables
Performance Considerations
5 of 7
Update Strategy performance can vary depending on the number of updates and inserts. In some cases there may be a performance benefit to splitting a mapping with updates and inserts into two mappings and sessions, one performing the inserts and one the updates.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
9.6
Updating Target Tables
Lab 9-1: Updating Target Tables • In this lab you will… • Use a single Source definition to read two files and combine their data in a single data stream • Remove duplicate rows • Create logic that • Rejects records if the incoming CUSTOMER_ID is missing • Inserts the record if the customer does not already exist in ODS_CUSTOMERS • Updates the record if the customer already exists in ODS_CUSTOMERS
Module 9: Updating Target Tables
PowerCenter 8.x/9.0 Level I Developer
6 of 7
Copyright © 2010 Informatica Corp
9.7
Updating Target Tables
Summary This module showed you how to: • Use an Update Strategy transformation to determine how the Target should handle records (insert, update, delete)
Module 9: Updating Target Tables
PowerCenter 8.x/9.0 Level I Developer
7 of 7
Copyright © 2010 Informatica Corp
9.8
Updating Target Tables
Module 9: Updating Target Tables
PowerCenter 8.x/9.0 Level I Developer
8 of 7
Copyright © 2010 Informatica Corp
10.1
Mapping Techniques
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
10.2
Mapping Techniques
Module Objectives After completing this module you will be able to: • Set and use system and Mapping variables and parameters • Use unconnected Lookup transformations to provide values on an as-needed basis • Use a Router transformation to divide a single set of records into multiple sets of records
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
2 of 24
Copyright © 2010 Informatica Corp
10.3
Mapping Techniques
System Variables SYSDATE
Provides current datetime on the PowerCenter Server machine • Not a static value
SESSSTARTTIME
Returns the system date value on the PowerCenter Server • Used with any function that accepts transformation date/time datatypes • Not to be used in a SQL override • Has a constant value
$$$SessStartTime
Returns the system date value as a string. Uses system clock on machine hosting PowerCenter Server • Format of the string is database type dependent • Used in SQL override • Has a constant value 3 of 24
Module 10: Mapping Techniques
Description
System variables hold information derived from the system. The user cannot control the content of the variable but can reference the information contained within the variable.
Business Purpose
System variables provide consistency to program execution.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
10.4
Mapping Techniques
Example • The developer needs to set the value of a port to indicate when the record was last updated by PowerCenter • Create a port called LAST_UPDATED and set its value to the expression SYSDATE
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
4 of 24
Copyright © 2010 Informatica Corp
10.5
Mapping Techniques
Mapping Parameters and Variables • Represent declared values: • Variables can change in value during run-time • Parameters remain constant during run-time
• Apply to all transformations within a Mapping • Declared in Mappings menu • Format is $$VariableName or $$ParameterName
• Value is initialized by the specification that defines it • Parameter values can be set to a different value in a parameter file, specified for the session task
• Provide increased development flexibility • Can be used in pre and post-SQL
Module 10: Mapping Techniques
5 of 24
Description
A Mapping can utilize parameters and variables to store information during execution. Each parameter and variable is defined with a specific datatype. Parameters are different from variables in that parameters are fixed for the run of the Mapping, while variables can change (vary). Both can be accessed from anywhere in the Mapping.
Business Purpose
Mapping variables and parameters are used: • To simplify Mappings by carrying information within or between transformations • To improve maintainability by allowing quick changes to values in a Mapping
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
10.6
Mapping Techniques
Declaring Mapping Parameters and Variables Mappings – Declare Parameters and Variables menu option Set aggregation type
User-defined ($$name)
See Note Set datatype
Set optional initial value
Module 10: Mapping Techniques
6 of 24
Scope
Parameters and variables can be used only inside the object in which they are created. A Mapping variable created for Mapping_1 is available only within that Mapping and cannot be used by another Mapping or Mapplet in the same workflow. A parameter or variable’s scope is the object in which it was created.
Aggregation Type
The PowerCenter Server uses the aggregation type of a mapping variable to determine the final current value of the mapping variable. In a session with multiple partitions, the PowerCenter Server combines the variable value from each partition and saves the final value into the repository. Aggregation types include Count (for integer datatype), Max, and Min.
IsExpVar
IsExpVar determines how the Integration Service expands the parameter in an expression string. If true, the IS expands the parameter after parsing the expression. Default is false. If this is true and the parameter type is not String, the IS fails the session.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
10.7
Mapping Techniques
Using Mapping Parameters and Variables
Use parameters or variables in expression
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
7 of 24
Copyright © 2010 Informatica Corp
10.8
Mapping Techniques
Functions to Set Mapping Variables • SETVARIABLE($$Variable,value) • Sets the variable to the value specified • Executes only for rows marked Insert or Update
• SETCOUNTVARIABLE($$Variable) • Increments a counter variable • If the row is marked for Insert, add 1; if it is marked for Delete, subtract 1; otherwise, do not change
• SETMAXVARIABLE($$Variable,value) • Sets the variable to the larger of its current value and the value passed to the function
• SETMINVARIABLE($$Variable,value) • Sets the variable to the smaller of its current value and the value passed to the function Module 10: Mapping Techniques
Variable Persistence
8 of 24
At the end of a successful session, the values of variables are saved to the repository. The SetVariable function writes the final value of a variable to the repository based on the Aggregation Type selected when the variable was defined. The final value written to the repository for a variable that has an Aggregate type of MAX will be whichever is greater, the current value or the initial value. Similarly, the final value for a variable with an Aggregate type of MIN will be whichever is smaller, the current value or the initial value.
Naming Convention
User-defined variable and parameter names always begin with $$ (i.e., $$ParamName or $$VariableName).
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
10.9
Mapping Techniques
Parameters & Variables – Initialization Priority Parameters 1. Parameter file 2. Declared initial value 3. Default value
Variables 1. Parameter file 2. Repository saved value 3. Declared initial value 4. Default value
Module 10: Mapping Techniques
9 of 24
Parameter File
A file that holds information about definitions of variables and parameters
Repository Saved Value
Values for variables that were saved in the Repository after successful completion of a Session
Declared Initial Value
The initial value, as set by the user when creating the variable or parameter
Default Value
Set by the system
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
10.10
Mapping Techniques
Examples of Variable/Parameter Usage Reason/Goal
Potential Value
Param Name or Var
Expression that determines whether a support case has been open greater than 100 days Set the value of the processing center where the session is executed, using a variable defined in the mapping and has its value set in a parameter file Ensure that everyone uses the same value in expressions. Create two parameters that represent “Yes” and “No.” Have all mappings use these values via a parameter file. To perform an incremental extraction, track when a mapping was last executed using a mapping variable
100
Param
$$MAX_NUM_DAYS_OPEN
'US'
Var
$$REG_PROC_LOCATION
'Y' 'N'
Param Param
$$YES_1_CHAR $$NO_1_CHAR
SESSSTARTTIME
Variable
$$LAST_RUN_DT
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
10 of 24
Copyright © 2010 Informatica Corp
10.11
Mapping Techniques
Unconnected Lookup
Commonly used when a Lookup not needed for every record No links from/to other transformations Lookup data is called at the point in the Mapping that needs it Lookup function can be set within any transformation that supports expressions Unconnected Lookup
Function in the Expression calls the Unconnected Lookup
Module 10: Mapping Techniques
11 of 24
Type
Passive
Description
Unconnected Lookups allow the inclusion of additional information in the transformation process from an external source when they are referenced within any transformation that supports expressions.
Business Purpose
A source table may have a small percentage of records with incomplete data. These “holes” in the data can be filled by performing a lookup to another table or tables, on an as-needed basis.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
10.12
Mapping Techniques
Unconnected Lookup Example • A small percentage of records are missing the CUST_ID field
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
12 of 24
Copyright © 2010 Informatica Corp
10.13
Mapping Techniques
Calling the Unconnected Lookup Use Lookup function within a conditional statement Condition
Row keys (passed to Lookup)
IIF ( ISNULL(customer_id),:lkp.MYLOOKUP(order_no), customer_id)
Lookup function
Condition is evaluated for each row but Lookup function is called only if condition is satisfied
Module 10: Mapping Techniques
Key Points
13 of 24
• Use the Lookup function (:lkp.lookupname) within a conditional expression • The condition is evaluated for each row but the Lookup function is only called when the record requires it • Data from several input ports may be passed to the Lookup transformation, but only one port may be returned, as designated by the R(eturn) property in the Lookup transformation • If no port is set as R the mapping will not be invalid, but the session may fail at runtime
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
10.14
Mapping Techniques
Unconnected Lookup Advantages Data lookup is performed only for those rows which require it. Substantial performance can be gained EXAMPLE: A Mapping will process 500,000 rows, each representing one item in an order. For two percent of those orders (10,000) the customer_id value is NULL. The customer_id can be derived from the value of order_no. IIF ( ISNULL(customer_id), :lkp.MYLOOKUP(order_no), customer_id)
Lookup Condition (true for 2 percent of all rows) (called only when condition is true)
Net savings = 490,000 lookups Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
14 of 24
Copyright © 2010 Informatica Corp
10.15
Mapping Techniques
Unconnected Lookup Functionality One lookup port value may be returned for each lookup
Must select a Return port in the Ports tab, or expression that calls lookup is invalid.
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
15 of 24
Copyright © 2010 Informatica Corp
10.16
Mapping Techniques
Key Points • An unconnected lookup can improve performance if the lookup table is static • Use the lookup with a conditional statement • The condition is evaluated only for those rows where the condition evaluates to TRUE
• The transformation is called using the expression :lkp.lookupname • Data from several input ports may be passed to the Lookup transformation but only one port may be returned • The port to be returned is designated by the Lookup transformation’s R (return) port • If a port is not selected as the R port, the mapping will not be invalidated but the session will fail at runtime
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
16 of 24
Copyright © 2010 Informatica Corp
10.17
Mapping Techniques
Connected versus Unconnected Lookups CONNECTED LOOKUP
UNCONNECTED LOOKUP
Part of the mapping data flow
Separate from the mapping data flow
Returns multiple values (by linking output ports to another transformation)
Returns one value - by checking the Return (R) port option for the output port that provides the return value
Executed for every record passing through the transformation
Only executed when the lookup function is called
More visible, shows where the lookup values are used
Less visible, as the lookup is called from an expression within another transformation
Default values are used
Default values are ignored
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
17 of 24
Copyright © 2010 Informatica Corp
10.18
Mapping Techniques
Joins versus Lookups Source Qualifier Join
Joiner
Lookup
Advantages
Advantages
Advantages
Can join any number of tables
Can join heterogeneous sources
Can re-use cache across session runs
Full functionality of standard SQL available
Can join non-relational sources
Can re-use cache w/in mapping
Can join partially transformed data
Can modify cache dynamically
Disadvantages
Can choose to cache or not to cache
Can only join 2 input data streams per joiner
Can query relational table or flat file
May reduce volume of data on network Disadvantages Can only join homogeneous relational tables
Inequality comparisons are allowed Only supports equijoin SQL override supported
Can affect performance on the source database
Does not support “OR” conditions Can be unconnected & invoked as needed Disadvantages Cannot output multiple matches Unconnected can only have 1 return value “OR” conditions not supported
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
18 of 24
Copyright © 2010 Informatica Corp
10.19
Mapping Techniques
Router Transformation Sends rows to different destinations based on filter conditions
Ports • All input/output • Specify filter conditions for each Group Usage • Link source data in one pass to multiple filter conditions
Module 10: Mapping Techniques
19 of 24
Type
Active
Description
Passes row data to different groups based on filter-like conditions. A Router transformation has one input group, and one or more output groups, each of which has its own filter condition.
Business Purpose
Allows you to write records from a single source into multiple targets based on user-defined criteria.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
10.20
Mapping Techniques
Router Groups Input group (always one) User-defined output groups Each group has one condition ALL group conditions are evaluated for each row One row can pass multiple conditions
Unlinked Group outputs are ignored Default group (always one) capture rows that fail all Group conditions
Module 10: Mapping Techniques
Performance Considerations
20 of 24
A Router transformation is functionally equivalent to several Filter transformations in parallel. However, performance can be substantially better, because a row is read once into the input group but evaluated multiple times, once for each condition.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
10.21
Mapping Techniques
Router Functionality
DATE_HIRED < 90 days ago
POSITION_CODE SALES
Note: New employees will be routed to two Targets
Module 10: Mapping Techniques
Note
POSITION_CODE = SALES
21 of 24
In the diagram above, the record for a salesperson hired less than 90 days ago will be routed to both STG_EMPLOYEES_NEW and STG_EMPLOYEES_SALES. If you wish to prevent single records from being routed to multiple Targets, you must ensure that their filter conditions are mutually exclusive. In the example above, you would prefix the conditions for the positionbased tables with “DATE HIRED >= 90 DAYS AGO AND”. Note that multiple target objects can be instances of the same target table. If this is the case, only one INSERT statement will be generated per record.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
10.22
Mapping Techniques
Lab 10-1 – Filling In Missing Data • In this lab you will… • Use a Unconnected Lookup to attempt to find in what city customers purchased their cars • Use a Router to categorize customers based on income level, Postal code, and city into High-Income, Sub-Prime, and Uncategorized categories • Override the relational writer to write out to a .CSV file so that the data can easily be loaded into a spreadsheet
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
22 of 24
Copyright © 2010 Informatica Corp
10.23
Mapping Techniques
Lab 10-2 – Load Employee Staging Table • In this lab you will… • Use a persistent Repository variable to hold and increment the latest employee identification number • Further leverage the reusable transformation re_exp_Format_Person
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
23 of 24
Copyright © 2010 Informatica Corp
10.24
Mapping Techniques
Summary This module showed you how to: • Set and use system and Mapping variables and parameters • Use unconnected Lookup transformations to provide values on an as-needed basis • Use a Router transformation to divide a single set of records into multiple sets of records
Module 10: Mapping Techniques
PowerCenter 8.x/9.0 Level I Developer
24 of 24
Copyright © 2010 Informatica Corp
11.1
Mapplets and Worklets
Module 11: Mapplets and Worklets
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
11.2
Mapplets and Worklets
Module Objectives After completing this module you will be able to: • Describe Mapplets • Use a Mapplet in a mapping • Describe Worklets
Module 11: Mapplets and Worklets
PowerCenter 8.x/9.0 Level I Developer
2 of 18
Copyright © 2010 Informatica Corp
11.3
Mapplets and Worklets
Mapplets • A Mapplet contains transformations and may be embedded into a Mapping
Module 11: Mapplets and Worklets
3 of 18
Type
Passive or Active
Description
Mapplets combine multiple mapping objects for reusability; they can also simplify complex mapping maintenance. A Mapplet receives input data from either an internal Source or the Mapping pipeline that calls the Mapping. A Mapplet must pass data out to the Mapping via a Mapplet Output transformation.
Note
Mapplets are reusable by nature – a Mapping uses an instance of a Mapplet. These instances inherit all changes to the parent Mapplet, which may affect the behavior of the Mappings that use the instances.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
11.4
Mapplets and Worklets
Mapplet Designer
Mapplet Input & Output transformations (pass data from or to mapping)
Mapplet Designer Tool
Module 11: Mapplets and Worklets
Example
4 of 18
A business, as part of its daily sales, needs to apply discounts, performing a number of lookups and aggregating the sales values. This functionality is used in several types of feeds, so the Mapplet shown here was created to provide this functionality, identically, in many Mappings.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
11.5
Mapplets and Worklets
Mapplet Input Transformation Passes data from a Mapping into a Mapplet
Passive Ports • Output ports only Usage • Only those ports connected from an Input transformation to another transformation will display in the resulting Mapplet
Transformation
Transformation
Module 11: Mapplets and Worklets
Type
Passive
Description
Acts as an input to a Mapplet
PowerCenter 8.x/9.0 Level I Developer
• Connecting the same port to more than one transformation is disallowed • Pass to an Expression transformation first
5 of 18
Copyright © 2010 Informatica Corp
11.6
Mapplets and Worklets
Mapplet Input Transformation Example • The Mapplet Input transformation receives sales transactions for processing
Module 11: Mapplets and Worklets
PowerCenter 8.x/9.0 Level I Developer
6 of 18
Copyright © 2010 Informatica Corp
11.7
Mapplets and Worklets
Mapplet Output Transformation Passes data from a Mapplet into a Mapping
Passive Ports •
Input ports only
Usage •
Only those ports connected to an Output transformation (from another transformation) will display in the resulting Mapplet
•
One (or more) Mapplet Output transformations are required in every Mapplet Module 11: Mapplets and Worklets
Type
Passive
Description
Acts as an output from a Mapplet
PowerCenter 8.x/9.0 Level I Developer
7 of 18
Copyright © 2010 Informatica Corp
11.8
Mapplets and Worklets
Mapplet Output Transformation Example • The Mapplet Output transformation returns the processed transactions to the Mapping
Module 11: Mapplets and Worklets
PowerCenter 8.x/9.0 Level I Developer
8 of 18
Copyright © 2010 Informatica Corp
11.9
Mapplets and Worklets
Mapplet with Multiple Output Groups
Can output to multiple instances of the same target table Module 11: Mapplets and Worklets
PowerCenter 8.x/9.0 Level I Developer
9 of 18
Copyright © 2010 Informatica Corp
11.10
Mapplets and Worklets
Unmapped Mapplet Output Groups
Module 11: Mapplets and Worklets
Warning
10 of 18
When the Mapplet is expanded at runtime, an unconnected output group could result in a transformation having no output connections. If that is not permitted, then the mapping will be invalid. For example: • If the Mapplet outputs are fed by an Expression transformation, the mapping is invalid because an Expression requires a connected output. • But if the Mapplet outputs are fed by a Router, the mapping is valid because a Router can have unconnected output groups.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
11.11
Mapplets and Worklets
Example: Data Sources Defined Inside Mapplet Mapping
Sources define source data within Mapplet
Mapplet
Mapplet Output transformation
Note: Mapplet must come first in mapping Module 11: Mapplets and Worklets
Note
11 of 18
Mapplets cannot be nested – that is, you cannot use a Mapplet inside another Mapplet.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
11.12
Mapplets and Worklets
Passive and Active Mapplets Multiple passive Mapplets can populate the same target instance
Passive Mapplet
Active and passive Mapplets cannot populate the same target instance
contains only passive transformations
Multiple active Mapplets cannot populate the same target instance
Active Mapplet
contains one or more active transformations Module 11: Mapplets and Worklets
PowerCenter 8.x/9.0 Level I Developer
12 of 18
Copyright © 2010 Informatica Corp
11.13
Mapplets and Worklets
Mapplet Parameters and Variables • Similar to Mapping parameters and variables • Defined by using the menu option Mapplets Æ Parameters and Variables • A parameter or variable defined in a Mapplet is not visible in any parent mapping • A parameter or variable defined in a Mapping is not visible in any child Mapplet
Module 11: Mapplets and Worklets
PowerCenter 8.x/9.0 Level I Developer
13 of 18
Copyright © 2010 Informatica Corp
11.14
Mapplets and Worklets
Lab 11-1 – Creating a Mapplet • In this lab you will… • Create a Mapplet and use it in a Mapping
Module 11: Mapplets and Worklets
PowerCenter 8.x/9.0 Level I Developer
14 of 18
Copyright © 2010 Informatica Corp
11.15
Mapplets and Worklets
Worklet • • • •
An object representing a set or grouping of Tasks Can contain any Task available in the Workflow Manager Worklets expand and execute inside a Workflow A Workflow which contains a Worklet is called the “parent Workflow” • Worklets CAN be nested (unlike Mapplets) • Reusable Worklets – create in the Worklet Designer • Non-reusable Worklets – create in the Workflow Designer
Module 11: Mapplets and Worklets
15 of 18
Description
Worklets are optional processing objects inside Workflows. They contain PowerCenter tasks that represent a particular grouping of, or functionally related set of, tasks. They can be created directly in a Workflow (nonreusable) or in the Worklet Designer (reusable)
Business Purpose
A Workflow may contain dozens of tasks. During Workflow design they will develop naturally in “groupings” of meaningfully-related tasks, run in the appropriate operational order. The Workflow can run as-is, from start to finish, executing task-by-task, or the developer can place natural groupings into Worklets. A Worklet’s relationship to a Workflow is like that of a Mapplet to a Mapping.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
11.16
Mapplets and Worklets
Creating Worklets In the Worklet Designer, select Worklets | Create
Worklets Node
Module 11: Mapplets and Worklets
PowerCenter 8.x/9.0 Level I Developer
16 of 18
Copyright © 2010 Informatica Corp
11.17
Mapplets and Worklets
Using a Worklet in a Workflow
Worklet used in a Workflow
Module 11: Mapplets and Worklets
PowerCenter 8.x/9.0 Level I Developer
17 of 18
Copyright © 2010 Informatica Corp
11.18
Mapplets and Worklets
Summary This module showed you how to: • Describe Mapplets • Use a Mapplet in a mapping • Describe Worklets
Module 11: Mapplets and Worklets
PowerCenter 8.x/9.0 Level I Developer
18 of 18
Copyright © 2010 Informatica Corp
12.1
Controlling Workflows
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.2
Controlling Workflows
Module Objectives After completing this module you will be able to: • Set and use workflow variables • Use link conditions and Decision tasks to control the execution of a workflow • Use other workflow tasks: Email, Event Wait, Event Raise, Command • Explain the purpose of the pmcmd utility • Schedule workflows to run automatically
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
2 of 30
Copyright © 2010 Informatica Corp
12.3
Controlling Workflows
Link Conditions • You can set conditions on workflow links: • •
If the link condition is True, the next task is executed If the link condition is False, the next task is not executed
• To set a condition, right-click a link and enter an expression that evaluates to True or false. •
You can use workflow variables in the condition (next slide)
Note: the words SUCCEEDED and FAILED are reserved for use in expressions
Module 12: Controlling Workflows
Reserved Words
3 of 30
In addition to SUCCEEDED and FAILED, the words DISABLED, NOT STARTED, STARTED, STOPPED and ABORTED are reserved for use in link conditions. Consult the Workflow Administration guide for valid values for all Predefined variables.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.4
Controlling Workflows
Incoming Links • OR – run the task as soon as any link condition is TRUE • AND – run the task when all link conditions are TRUE
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
4 of 30
Copyright © 2010 Informatica Corp
12.5
Controlling Workflows
Variables Used in a Workflow User-defined variables (next slides)
Pre-defined variables
Task-specific variables
Built-in system variables
Module 12: Controlling Workflows
5 of 30
Types
Workflow variables come in two types: user-defined and pre-defined
Business Purpose
A workflow can contain multiple tasks and multiple pipelines. One or more tasks or pipelines may be dependent on the status of previous tasks. Workflow variables convey that information from one task to another.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.6
Controlling Workflows
Declaring User-Defined Workflow Variables • To declare user-defined workflow variables • Click Workflows Æ Edit and select the Variables tab
• Variables can persist across sessions in a workflow • The value is saved in the repository
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
6 of 30
Copyright © 2010 Informatica Corp
12.7
Controlling Workflows
Pre-defined Workflow Variables • Include system variables and task-specific variables • System variables (SYSDATE and WORKFLOWSTARTTIME) can be used for calculating variable dates and times, e.g., in Assignment task link conditions • Task-specific variables are available in Decision, Assignment, and Timer tasks and link conditions Task-Specific Variables
Module 12: Controlling Workflows
Note
7 of 30
Predefined workflow variables are discussed in more detail in the Workflow Administration Guide
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.8
Controlling Workflows
Pre-defined Workflow Variable Example • Session 2 depends on success of Session 1 • Defined as session status = successful and failed rows = 0 • Code the link from Session 1 to Session 2 so that Session 2 will not run unless all criteria are true • Use the task-specific workflow variables Status, SrcFailedRows, and TgtFailedRows in the Link condition expression
• Session 4 should not run if Session 3 takes more than one hour • Test the system variable WORKFLOWSTARTTIME in the Link condition expression Session 1
Session 2
Session 3
Session 4
Start
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
8 of 30
Copyright © 2010 Informatica Corp
12.9
Controlling Workflows
Assignment Task – Setting User-Defined Workflow Variables Assigns a value to a user-defined workflow variable Note: The variable must have been declared in the workflow Variables tab
General Tab Expressions Tab
Module 12: Controlling Workflows
9 of 30
Description
Can establish the value of a Workflow Variable, whose value can be used at a later point in the workflow, as testing criteria to determine if or when other workflow tasks/pipelines should be run.
Business Purpose
Running a workflow task may depend on the results of other tasks or calculations in the workflow. An Assignment task can do certain calculations to establish the value for a workflow variable. This value may determine whether other tasks or pipelines are run.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.10
Controlling Workflows
Assignment Task Example • Session 4 should run at least one hour after Session 3 completes • The Assignment Task is coded to set a time that the Timer will wait for • To prevent the Assignment task from running until after Session 2 completes, use a Link Condition
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
10 of 30
Copyright © 2010 Informatica Corp
12.11
Controlling Workflows
Decision Task Tests for a condition during the workflow and sets a flag based on the condition Use a link condition (or a Control task) downstream to test the flag and control execution flow Can use workflow variables in condition
Module 12: Controlling Workflows
11 of 30
Description
Decision tasks enable workflow designers to set criteria by which the workflow will or will not proceed to the next set of tasks, depending on whether the set criteria is true or false
Business Purpose
Commonly, workflows have multiple paths. Some are simply concurrent tasks. Others are pipelines of tasks that should only run if the previous tasks are successful. Still others should be run only if those tasks are not successful. What determines the success or failure of a task or group of tasks is user-defined, depending on the business-defined rules and operational rules of processing. The criteria are set as the decision condition in a Decision task, and subsequently tested for a True or False condition
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.12
Controlling Workflows
Decision Task Example • If Session 3 is successful, run Session 4 • If Session 3 fails, send an email to the system administrator
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
12 of 30
Copyright © 2010 Informatica Corp
12.13
Controlling Workflows
Email Task Sends an email within a workflow Note: emails can also be sent post-session in a Session task
Can be used with a link condition to notify success or failure of prior tasks
Module 12: Controlling Workflows
13 of 30
Description
Email tasks enable PowerCenter to send email messages at various points in a workflow. Users can define email addresses, a subject line, and the email message text. When called from within a Session task, the message text can contain variable Session-related metadata – for example, one message for Session success and another for failure.
Business Purpose
Various business and operational staff may need to be notified of the progress of a workflow, the status of tasks (or combinations of tasks) within it, or various metadata results of a session
Performance Considerations
The PowerCenter domain must be configured to use a running, configured email server. However, the impact of the Integration Service sending the emails is minimal
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.14
Controlling Workflows
Event Wait Task
Pauses processing of the pipeline until a specified event occurs Events can be:
Pre-defined – file watch User-defined – created by an Event Raise task elsewhere in the workflow
Module 12: Controlling Workflows
14 of 30
Description
Event Wait tasks wait for either the presence of a named flat file (a predefined event) or some other user-defined event to occur in the workflow processing. Note that the Workflow must be running in order to recognize a pre-defined event.
Business Purpose
An Event Wait task watching for a flat file by name is placed in a workflow because some subsequent processing is dependent on the presence of the file. An Event Wait task waiting for the occurrence of a user-defined event will be strategically placed so that the workflow should not proceed further until some other set of tasks and conditions has occurred. It always works in concert with an Event Raise task.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.15
Controlling Workflows
Pre-defined Event Example • Session 2 cannot run until the daily sales report file has been placed on the server • The Event Wait task is configured to watch for the file to appear • Session 2 does not run until the Event Wait task finds the file
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
15 of 30
Copyright © 2010 Informatica Corp
12.16
Controlling Workflows
Event Wait Task (cont’d) Events Tab
Specify either a pre-defined or user-defined event
User-defined events must be declared in the workflow Events tab
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
16 of 30
Copyright © 2010 Informatica Corp
12.17
Controlling Workflows
Event Raise Task
Sets the location of a user-defined event in the workflow User-defined events are triggered when the PowerCenter Server executes the Event Raise Task User-defined events must be declared in the workflow Events tab
Used with the Event Wait Task Module 12: Controlling Workflows
17 of 30
Description
Event Raise tasks are always used in conjunction with user-defined Event Wait tasks. They send a signal to an Event Wait task that a set of pre-defined events has occurred, along the pipeline from the Start task to the Event Raise task.
Business Purpose
This task allows signals to be passed from one spot in the workflow to another that a particular series of predetermined events has occurred.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.18
Controlling Workflows
User-Defined Event Example • Session 5 should run after both Session 2 and Session 4 complete successfully • The Event Raise task raises a user-defined event when Session 2 completes successfully • The lower pipeline will not proceed beyond the Event Wait task until the event has been raised
Module 12: Controlling Workflows
User-Defined Events
18 of 30
For a user-defined event, the developer: 1. Defines an event in the workflow properties (prior to workflow processing) 2. Includes an Event Wait task at a suitable point in the workflow, where further processing must await some specific event 3. Includes an Event Raise task at a suitable point in the workflow, e.g., after a parallel pipeline has been completed.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.19
Controlling Workflows
Command Task
Specifies one or more UNIX command or shell script, DOS command or batch file for Integration Services to run during a workflow Note: UNIX and DOS commands can also be run pre- or postsession in a Session task
Command task status (success or failure) is held in the taskspecific variable $command_task_name.STATUS
Module 12: Controlling Workflows
19 of 30
Description
Command tasks are inserted in workflows and worklets to enable the Integration Service to run one or more OS commands of any nature. All commands or batch files referenced must be executable by the OS login that owns the Integration Service process.
Business Purpose
OS commands can be used for any operational or business unit related procedure, and can be run at any point in a workflow. Command tasks can be set to run one or more OS commands or scripts/batch files, before proceeding to the next task in the workflow. If more than one command is coded into a Command Task, the entire task can be set to fail if any one of the individual commands fails. Additionally and optionally, each individual command can be set not to run if a preceding command fails.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.20
Controlling Workflows
Command Task (cont’d)
Add Cmd Remove Cmd
Module 12: Controlling Workflows
Examples of Command Tasks
20 of 30
A Session task that produces an output file can be followed by a Command task that copies the file to another directory, or FTPs the file to another box location. The command syntax is the same as that which would accomplish this at the OS command prompt on the Integration Service machine.
A Session task that relies on a flat file as source data can be preceded by a Command task that verifies the presence of the file, opens it and verifies control totals or record counts with some external source of information.
A series of multiple concurrent or sequential Sessions can be followed by a single Command task coded to copy or move all session logs created by the workflow to a special daily backup directory.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.21
Controlling Workflows
Timer Task Waits for a specified period of time to execute the next task General Tab Timer Tab
• Absolute Time • Datetime Variable • Relative Time
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
21 of 30
Copyright © 2010 Informatica Corp
12.22
Controlling Workflows
Control Task Stops, fails, or aborts the Worklet or Workflow
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
22 of 30
Copyright © 2010 Informatica Corp
12.23
Controlling Workflows
Control Options
Control Option
Description
Fail Me
Marks the Control task as “Failed.” The PowerCenter Server fails the Control task if you choose this option.
Fail Parent
Marks the status of the workflow or worklet that contains the Control task as failed after the workflow or worklet completes.
Stops the workflow or worklet that contains the Control task. Stop Parent Abort Parent Aborts the workflow or worklet that contains the Control task. Fail Top-Level Workflow Fails the workflow that is running. Stops the workflow that is running. Stop Top-Level Workflow Abort Top-Level Workflow Aborts the workflow that is running. Note: Fail = Complete but set status to “failed”, Stop = Stop executing after orderly shutdown, Abort = Stop executing immediately Module 12: Controlling Workflows
Note
23 of 30
The Control task can fail, stop, or abort either the parent Workflow or the top-level Workflow. However, stopping or aborting the parent Workflow means that no further progress takes place along that branch in the top-level Workflow. This can cause the top-level Workflow to stop if there is no other branch.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.24
Controlling Workflows
Reusable Tasks • Session, Email and Command tasks can be reusable • Use the Task Developer to create reusable tasks • Reusable tasks appear in the Navigator Tasks node and can be dragged and dropped into any workflow In a workflow, a reusable task is indicated by a special symbol
Module 12: Controlling Workflows
Business Purpose
24 of 30
Occasionally, a certain mapping logic may be required to run in multiple workflows. Since a mapping is reusable, the developer can code multiple sessions, all based on the same mapping. However, it is simpler to create a reusable session based on the mapping. Once created in the Task Developer, an instance of the Reusable Session can be placed in any workflow or Worklet.
Performance Considerations
Use reusable session tasks sparingly. Retrieving the metadata for a reusable session task and its child instances from the repository takes longer than retrieving the metadata for a non-reusable session task.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.25
Controlling Workflows
pmcmd utility • Line command utility providing most Workflow Manager operations, e.g. start workflow • Example of syntax: pmcmd startworkflow -sv MyIntService -d MyDomain -u seller3 -p jackson -f SalesEast wf_SalesAvg
Note: The password can be provided through the PASSWORD environmental variable. To do this, you can encrypt the password using the pmpasswd utility on the PowerCenter Services machine and then enter the encrypted password in pmcmd. Module 12: Controlling Workflows
Description
25 of 30
The pmcmd command line utility allows the developer to perform most Workflow Manager operations outside of the PowerCenter client tool. These commands can be used in batch files.
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
12.26
Controlling Workflows
Workflow Scheduler • Set and customize workflow-specific schedule
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
26 of 30
Copyright © 2010 Informatica Corp
12.27
Controlling Workflows
Workflow Scheduler (cont’d)
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
27 of 30
Copyright © 2010 Informatica Corp
12.28
Controlling Workflows
Lab 12-1 – Using Assignment and Decision Tasks • In this lab you will… • Create a workflow that eliminates some types of bad data from getting into ODS_SALES • Assign workflow variables to keep track of how many times the workflow has run • Increment workflow variables using an Assignment task • Branch in a workflow using Link conditions and a Decision task to choose to run the next session or report an error
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
28 of 30
Copyright © 2010 Informatica Corp
12.29
Controlling Workflows
Lab 12-2 – Using Events and Timers • In this lab you will… • • • •
Create a workflow that raises and uses a User-Defined event Stop the workflow “nicely” if the tables load properly Set a timer to fail the workflow if it takes more than 15 minutes Schedule the workflow to run at a particular time
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
29 of 30
Copyright © 2010 Informatica Corp
12.30
Controlling Workflows
Summary This module showed you how to: • Set and use workflow variables • Use link conditions and Decision tasks to control the execution of a workflow • Use other workflow tasks: Email, Event Wait, Event Raise, Command • Explain the purpose of the pmcmd utility • Schedule workflows to run automatically
Module 12: Controlling Workflows
PowerCenter 8.x/9.0 Level I Developer
30 of 30
Copyright © 2010 Informatica Corp
13.1
Mapping Design Workshop
Module 13: Mapping Design Workshop
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
13.2
Mapping Design Workshop
Module Objectives After completing this module you will be able to: • Follow best practices for mapping design
Module 13: Mapping Design Workshop
PowerCenter 8.x/9.0 Level I Developer
2 of 4
Copyright © 2010 Informatica Corp
13.3
Mapping Design Workshop
Workshop 13-1: Mapping Design Workshop • In this lab you will… • Practice designing your own mappings
Module 13: Mapping Design Workshop
PowerCenter 8.x/9.0 Level I Developer
3 of 4
Copyright © 2010 Informatica Corp
13.4
Mapping Design Workshop
Summary This module showed you how to: • Follow best practices for mapping design
Module 13: Mapping Design Workshop
PowerCenter 8.x/9.0 Level I Developer
4 of 4
Copyright © 2010 Informatica Corp
14.1
Workflow Design Workshop
Module 14: Workflow Design Workshop
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
14.2
Workflow Design Workshop
Module Objectives After completing this module you will be able to: • Follow best practices for workflow design
Module 14: Workflow Design Workshop
PowerCenter 8.x/9.0 Level I Developer
2 of 4
Copyright © 2010 Informatica Corp
14.3
Workflow Design Workshop
Workshop 14-1: Workflow Design Workshop • In this lab you will… • Practice designing your own workflows
Module 14: Workflow Design Workshop
PowerCenter 8.x/9.0 Level I Developer
3 of 4
Copyright © 2010 Informatica Corp
14.4
Workflow Design Workshop
Summary This module showed you how to: • Follow best practices for workflow design
Module 14: Workflow Design Workshop
PowerCenter 8.x/9.0 Level I Developer
4 of 4
Copyright © 2010 Informatica Corp
15.1
Workflow Design Workshop
Module 15: Workflow Design Workshop 15
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
15.2
Workflow Design Workshop
Objectives After completing this module you will be able to: • Describe New Features in PowerCenter 9.0: • • • • • • • •
Lookup transformation enhancements SQL transformation enhancements XML Parser enhancements Verbose Logging enhancement License Management enforcement Integration Service log file rollover Mapping Architect for Visio Additional Transformations infacmd Command Line enhancements
Module 15: Workflow Design Workshop
PowerCenter 8.x/9.0 Level I Developer
2 of 8
Copyright © 2010 Informatica Corp
15.3
Workflow Design Workshop
PowerCenter 9.0 New Features • Lookup Transformation • Cache updates – Update Dynamic Cache Condition. • Database deadlock resilience. • Multiple rows return. • SQL overrides for uncached lookups.
15
Module 15: Workflow Design Workshop
3 of 8
Cache Updates
You can update the lookup cache based on the results of an expression. When an expression is true, you can add to or update the lookup cache. You can update the dynamic lookup cache with the results of an expression
Database Deadlock Resilience
Database deadlock does not cause immediate session failure. The IS attempts to run the last statement in a lookup again. Number of retries and sleep interval are configurable.
Multiple Rows Return
You can configure the Lookup transformation to return all rows that match a lookup condition
SQL Overrides for Uncached Lookups
You can create an SQL override for uncached lookup. You can include lookup ports in the SQL query
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
15.4
Workflow Design Workshop
PowerCenter 9.0 New Features • SQL Transformation • Environmental SQL for SQL Transformation • Set Active or Passive at design time for SQL Transform • Exactly Once semantics for SQL Transformation • XML Transformation • XML Parser Validation • Verbose Log • Log exact query in Verbose mode
4 of 8
Module 15: Workflow Design Workshop
SQL Transform Exactly Once Semantics
When deleting and inserting records with referential integrity constraints, the order in which the operations are performed becomes important. Allows Referential Integrity constraints to be observed.
XML Transformation Parser can validate an XML document against a schema Routes invalid XML to an error port Routes messages to a separate output group
Verbose Log
The exact query executed is logged in Verbose mode
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
15.5
Workflow Design Workshop
PowerCenter 9.0 New Features • License Management • Core license management • Repository license management • Integration Service •
Mapping Architect for Visio • New mapping objects including Normalizer, Custom transformation
Module 15: Workflow Design Workshop
15
5 of 8
License Management Number of cores enforcement: Ensures that licensees do not exceed licensed number of cores Repository Licensing: Ensures that licensees do not exceed licensed number of repositories Integration Service
Session log file rollover: Limit the size of session logs for real-time sessions
Mapping Architect for Visio
New Mapping Objects: Pipeline Normalizer, Custom Transformation, PowerExchange Source Definition, PowerExchange Target definition Can configure a transformation to use a shortcut You can create a Mapping template that contains these objects, shortcuts, or reusable transformations
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
15.6
Workflow Design Workshop
PowerCenter 9.0 New Features • Command Line • Infacmd to include new application services
6 of 8
Module 15: Workflow Design Workshop
Infacmd expanded to infacmd ds (data services) include management infacmd isp (Informatica service manager) of all Informatica application services infacmd oie (Object import and export) infacmd prs (Model Repository services) infacmd rtm (Analyst Tool services) infacmd sql (SQL data services) infacmd help for help on each service type
PowerCenter 8.x/9.0 Level I Developer
Copyright © 2010 Informatica Corp
15.7
Workflow Design Workshop
Summary This module showed you how to: • Describe New Features in PowerCenter 9.0: • • • • • • • •
Lookup transformation enhancements SQL transformation enhancements XML Parser enhancements Verbose Logging enhancement License Management enforcement Integration Service log file rollover Mapping Architect for Visio Additional Transformations infacmd Command Line enhancements
Module 15: Workflow Design Workshop
PowerCenter 8.x/9.0 Level I Developer
15
7 of 8
Copyright © 2010 Informatica Corp
Workflow Design Workshop
PowerCenter 8.x/9.0 Level I Developer
15.8
Copyright © 2010 Informatica Corp
Global Education Services Course Evaluation
Course Title Course Start Date Location Instructor
Required Information
Your Name Company E-mail Phone
Optional Information (print)
(print)
What method did you use to register for this class? Web ____ 800 Number ____ Telemarketer ____ Other ____ 1. What was your level of exposure to the product? 2. What was your level of exposure to SQL query tools? 3. What was your level of exposure to RDB concepts?
None
Minimal
1
2
Strongly Disagree Disagree
Some Technical Advanced
3
4
5
Strongly Not Agree Applicable
Neutral
Agree
2
3
4
5
0
2
3
4
5
0
Course Content and Materials
1. The course content met my expectations 2. The course met the stated objectives
3. The course length was long enough to cover the content 4. The time allotted for labs was long enough 5. The lab exercises helped in learning the course material 6. The visual aids used by the instructor were helpful Instructor
7. The instructor was knowledgeable in the subject area The instructor effectively used class time to enable me to 8. learn the key concepts 9. The instructor encouraged students to ask questions 10. Student questions were answered clearly and completely 11. The instructor was available and helpful during labs Facilities and Registration
12. The quality of the facilities was conducive to learning 13. Classroom equipment was an effective tool in my learning 14. Registration process made it easy to enroll
1
Overall
15. This training will improve my job performance 16. This training was a worthwhile investment for my employer 17. I am satisfied with the Training Overall 1
1/16/2009
Global Education Services Course Evaluation
What Units were the most valuable, least valuable for you and why? Most:
Least:
What recommendations can you suggest for course improvement: materials and or presentation? What topics require more coverage in class? Less coverage? More:
Less:
Should the lab time be lengthened or shortened? Any lab in particular? Should the course be made longer or shorter? Any unit in particular? What suggestions would you offer the instructor to improve his/her delivery of the course? What suggestions do you have for improving the classroom environment? What follow up course(s) would you like to see? What suggestions do you have that would make it easier to do business with us? Would you recommend this course to others?
0
No
5
Yes
Please provide contact information for anyone you feel would benefit from one of our courses. Name Name Phone Number Phone Number e-mail e-mail
1/16/2009
