1
END TO END AUTOMATION FOR ORACLE PBCS USING EPM AUTOMATE Amit Sahasrabudhe Huron Consulting Group
Sumit Deo Huron Consulting Group
[email protected]
[email protected] 2017HURON HURONCONSULTING CONSULTINGGROUP GROUPINC. INC.AND ANDAFFILIATES. AFFILIATES.ALL ALLRIGHTS RIGHTSRESERVED. RESERVED. ©©2017
ABOUT HURON + + + +
+ +
Formed in 2002 with approximately 200 professionals More than 1,800 full-time billable consultants with leading industry experts 2016 revenue of $726 million Headquartered in Chicago with domestic and international offices Publicly traded on the NASDAQ under ticker symbol “HURN” since October 2004 In 2016, Huron had more than 1,000 client engagements, including 200 new clients 2017HURON HURONCONSULTING CONSULTINGGROUP GROUPINC. INC.AND ANDAFFILIATES. AFFILIATES.ALL ALLRIGHTS RIGHTSRESERVED. RESERVED. ©©2017
2017HURON HURONCONSULTING CONSULTINGGROUP GROUPINC. INC.AND ANDAFFILIATES. AFFILIATES.ALL ALLRIGHTS RIGHTSRESERVED. RESERVED. ©©2017
Disclaimer The information in these slides do not represent the thoughts, intentions, plans or strategies of Oracle or any other organization. These are solely the author’s opinions. Oracle frequently updates Cloud features and hence some of the functionality described in this document may become obsolete/change in future updates to cloud products.
2017HURON HURONCONSULTING CONSULTINGGROUP GROUPINC. INC.AND ANDAFFILIATES. AFFILIATES.ALL ALLRIGHTS RIGHTSRESERVED. RESERVED. ©©2017
SESSION AGENDA 1 2
EPM Automate Installation Accessing EPM Automate Pre-requisites and Installing EPM Automate EPM Automate Help and Command Overview
PBCS Automation using EPM Automate Upload data files in Essbase format to PBCS Uploading metadata files and refresh PBCS application Push Data Operations
Data Management
3
Load data to Planning using DM Automation Options RunDMReport
4
5 6
Error Trapping Error handling for metadata import Error handling for data import Error handling for data import for Data Management
Backup Automation using EPM Automate LCM Artifact Information Back up batch
Miscellaneous User Audit Report Run plan type maps
2017HURON HURONCONSULTING CONSULTINGGROUP GROUPINC. INC.AND ANDAFFILIATES. AFFILIATES.ALL ALLRIGHTS RIGHTSRESERVED. RESERVED. ©©2017
1 EPM AUTOMATE INSTALLATION
7
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
2017HURON HURONCONSULTING CONSULTINGGROUP GROUPINC. INC.AND ANDAFFILIATES. AFFILIATES.ALL ALLRIGHTS RIGHTSRESERVED. RESERVED. ©©2017
WHAT IS EPM AUTOMATE? • The EPM Automate utility enables service administrators to remotely perform tasks like loading data, metadata within the Oracle PBCS instances
• It is implemented on top of REST API’s • Can be used on Windows and Unix • Uses Transport Layer Security 1.2 (TLS 1.2) to secure communication between your computer and the cloud service
• Scripts containing EPM Automate commands can be built to automate a wide variety of tasks using a scheduler 8
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
ACCESSING THE UTILITY – STANDARD INTERFACE • For Standard Interface, download EPM Automate utility for Windows or Linux/Unix from Cloud EPM Workspace->Tools
9
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
ACCESSING THE UTILITY – SIMPLIFIED INTERFACE • For Simplified Interface, download EPM Automate utility for Windows or Linux/Unix from User Profile Dropdown->Downloads.
• The ‘Downloads’ page contains link to download EPM Automate, Smartview, FR Studio and Smartview Admin Extension.
10
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
INSTALLING THE UTILITY - WINDOWS Pre-requisites – 64 bit machine. DOES NOT work on 32 bit machine
Simple installation. Only setting is destination folder
11
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
EPM AUTOMATE LIST OF COMMANDS • Browse to the install directory. Default is C:\Oracle\EPM Automate • From the bin folder double click epmautomate.bat. • Type ‘epmautomate help’ and hit enter. Opens a web page with the command reference.
• Web page can also be accessed at http://docs.oracle.com/cloud/latest/epm-common/CEPMA/epm_automate_command_ref_section.htm Pro Tip – Spin off a 64 bit VM that can be used as a maintenance machine on which EPM Automate runs. This machine serves as a data and metadata landing area as well as a store for the artifact backups. 12
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
2 AUTOMATION FOLDER
13
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
2017HURON HURONCONSULTING CONSULTINGGROUP GROUPINC. INC.AND ANDAFFILIATES. AFFILIATES.ALL ALLRIGHTS RIGHTSRESERVED. RESERVED. ©©2017
SETUP AUTOMATION FOLDER • Setup an automation folder to manage the batch and application files.
14
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
SETUP CONTROL FILES AND ENCRYPT PASSWORD • Setup a control file to pass runtime parameters to EPM Automate Jobs
• Encrypt service administrator password using EPM Automate encrypt command. • Use generated password file path in the login command
Usage: epmautomate login USERNAME PASSWORD_FILE URL IDENTITYDOMAIN 15
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
3 PBCS DATA AND METADATA JOBS
16
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
2017HURON HURONCONSULTING CONSULTINGGROUP GROUPINC. INC.AND ANDAFFILIATES. AFFILIATES.ALL ALLRIGHTS RIGHTSRESERVED. RESERVED. ©©2017
IMPORT FORMAT FOR METADATA IMPORTS – STANDARD INTERFACE • Generate an export from the Planning application for a specific dimension to get it’s import format. From the Standard Interface, Open the Planning application. • Navigate to Administration->Import and Export->Export Metadata.
• Select a dimension and then click ‘Run’ to extract the metadata for that dimension
17
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
IMPORT FORMAT FOR METADATA IMPORTS – SIMPLIFIED INTERFACE • • • •
18
Generate an export from the Planning application for a specific dimension to get it’s import format. From the Simplified Interface, browse to Application->Overview->Dimensions. Select a dimension. Click on Export. Click on Create. Select a dimension. Ensure ‘Export Metadata’ location is set to Local. Then, click on Export. Save the zip file.
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
CREATING A METADATA JOB • Within Simplified Interface, browse to Simplified Interface->Application->Overview
• Select the ‘Dimensions’ tab
19
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
CREATING A METADATA JOB • Select ‘Import’
• Click Create
20
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
CREATING A METADATA JOB • From the import metadata location, select ‘Inbox’. This is the Planning inbox folder
• Select the dimension for which you want to create a job. Then add a name for the metadata file along with the relevant selections for the delimiter
• From the options on the top pane, select ‘Save as Job’. Enter a name for the job and check the ‘refresh database check box if you want to refresh the database if import is successful’
21
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
CREATING A METADATA JOB • The job created is now visible in the Import Metadata dialog box. The job will be used in the automation process using EPM Automate.
22
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
RUNNING THE METADATA JOB USING EPMAUTOMATE • Use EPMAUTOMATE to delete existing file, upload new metadata file and then import and refresh the Planning metadata. • Metadata file uploaded to Planning Inbox/Outbox repository. From Simplified Interface, navigate to Application->Overview->Actions->Inbox/Outbox Explorer
23
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
SAMPLE METADATA IMPORT USING EPMAUTOMATE List of key commands used: • deletefile • uploadfile • importmetadata
24
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
CREATING AN ESSBASE DATA IMPORT JOB • Within Simplified Interface, browse to Application->Overview->Actions->Import Data
• Click Create
25
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
CREATING AN ESSBASE DATA IMPORT JOB • Within Simplified Interface, browse to Application->Overview->Actions->Import Data. • Select the import parameters. Enter a name for the import file and click ‘Save as Job’ • Note that for Source Type ‘Essbase’ only a tab delimited file works correctly.
• Enter a name for the job and click ‘Save’. This job will be referenced in the automation process
26
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
SAMPLE ESSBASE DATA IMPORT USING EPMAUTOMATE Note the use of the following EPM Automate commands in addition to importdata • setsubstvars: Set Substitution Variables prior to batch activity. • runbusinessrule: Run Calc Manager Business Rules
27
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
RUNNING A DATA MANAGEMENT LOAD RULE • Pre-Requisite - Create a data management load rule • Use EPM AUTOMATE rundatarule command based on the start period and end period, and import or export options. • EPM Automate DM load rule uses files uploaded/available in Data Management Inbox repository
28
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
RUNNING A DATA MANAGEMENT LOAD RULE Sample data load rule execution using EPM Automate • Usage: epmautomate rundatarule RULE_NAME START_PERIOD END_PERIOD IMPORT_MODE EXPORT_MODE
29
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
4 ERROR TRAPPING
30
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
2017HURON HURONCONSULTING CONSULTINGGROUP GROUPINC. INC.AND ANDAFFILIATES. AFFILIATES.ALL ALLRIGHTS RIGHTSRESERVED. RESERVED. ©©2017
ERROR TRAPPING EPM Automate process execution can utilize the following functionality to manage error handling: • The current version of EPM Automate Utility returns an exit code and message to indicate the status of the operation. • Exit codes are grouped under five code numbers; each code may indicate many error conditions. • The utility creates a log file for each failed command execution. Log files are created in the same directory location where the batch files are maintained.
Error handling can be separated into the following 3 broad categories: • Metadata Import using Simplified Interface jobs • Data Import using Simplified Interface jobs • Data Import using Data Management
31
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
ERROR TRAPPING – EXIT CODES REFERENCE EPM Automate process execution can utilize the following functionality to manage error handling: • The current version of EPM Automate Utility returns an exit code and message to indicate the status of the operation. • Exit codes are grouped under five code numbers; each code may indicate many error conditions.
32
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
METADATA IMPORT ERROR HANDLING • The batch file for Metadata Import process can be used to redirect the output of the EPM Automate execution to a log file. • The generic code EPMAT can be utilized to check if the metadata process generated an error.
33
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
METADATA IMPORT ERROR HANDLING • The log file will contain the exit code (example EPMAT-1)
• This output can be redirected to an error file which can be emailed to the administrator for further troubleshooting. Administrator can directly log onto Simplified Interface job console and find out the member that has errored out.
34
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
METADATA IMPORT ERROR HANDLING • In simplified Interface, to view job status, Navigate to Application->Jobs
Locate the job under the ‘Recent Activity’ section and click on the Job Name
35
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
METADATA IMPORT ERROR HANDLING • Make a note of the error message: “One ore more child jobs have failed”
Locate the job under the ‘Recent Activity’ section and click on the Job Name
36
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
METADATA IMPORT ERROR HANDLING • In the standard interface, the error message is more clearly specified
• Clicking on the “Completed with Errors” link, you see the actual error
37
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
METADATA IMPORT ERROR HANDLING • If the metadata import process fails, the EPM Utility creates a file in the system folder from where the batch file has been executed.
38
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
METADATA IMPORT ERROR HANDLING • Following is an example of the file that has been generated by the system. • For more advanced error reporting, use REST API (next slide)
39
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
METADATA IMPORT ERROR HANDLING – REST API Using REST API, the admin can use the Parent Job Id to find the Child Job Id which can be used to find the exact error message
40
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
METADATA IMPORT ERROR HANDLING – REST API Sample shown below shows the exact reason for the error.
41
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING – FOR DATA IN ESSBASE FORMAT • The batch file for Data Import process can be used to redirect the output of the EPM Automate execution to a log file. • The generic code EPMAT can be utilized to check if the metadata process generated an error from the log file.
42
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING – FOR DATA IN ESSBASE FORMAT • Sample contents of the log file
43
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING • In Simplified Interface, to view job status, Navigate to Application->Jobs
Locate the job under the ‘Recent Activity’ section and click on the Job Name
44
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING • Make a note of the error message: “One ore more child jobs have failed”
45
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING • In the standard interface, the job status can be viewed using the Job Console
• Clicking on the “Error” link, you see the actual error
46
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING – FOR DATA IN ESSBASE FORMAT •
47
Also, the jobid string can be used to get further details (using REST API, see next slide) of the process from the file generated by the EPM Utility in the system folder.
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING – REST API • Using REST API, the exact drop out error can be obtained by using the Job Id.
48
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING FOR DATA MANAGEMENT • For Data Import jobs executed from Data Management, we can utilize the EPM Automate command listfiles to develop a complete error handling mechanism. •
49
EPM Automate Listfiles command is used to get the paths and names of all files under Data Management (DM) folders and the default Inbox/Outbox explorer location to a log file
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING FOR DATA MANAGEMENT • Administrators can use this feature to get the kick-outs for a data import process that is executed using Data Management. • The findstr batch command in combination with the generic DM job name can be used to generate the job names to another file.
50
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING FOR DATA MANAGEMENT Steps (Continued): • EPM Automate downloadfile can be used to download the data management log files.
51
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING FOR DATA MANAGEMENT Steps (Continued): • The latest log file can be identified using the job name suffix and the batch file command findstr can be used with the error code 3303 to generate all the kickouts to an error file.
52
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING FOR DATA MANAGEMENT • Sample batch file which calls the file containing the list files command and other processing
53
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
DATA IMPORT ERROR HANDLING FOR DATA MANAGEMENT • Sample batch file containing the list files command and other processing
54
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
5 ARTIFACT BACKUP
55
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
2017HURON HURONCONSULTING CONSULTINGGROUP GROUPINC. INC.AND ANDAFFILIATES. AFFILIATES.ALL ALLRIGHTS RIGHTSRESERVED. RESERVED. ©©2017
PBCS ARTIFACT BACKUP • Part of service maintenance, automated artifact snapshot process creates a snapshot of the instance. This snapshot is available under Application Management->Application Snapshots. • Snapshot is called ‘Artifact Snapshot’. • Existing snapshot is overwritten the next time the snapshot is taken by the service. • Snapshot time is configured within ‘Daily Maintenance Time’ setting in the Simplified Interface
• Use EPM AUTOMATE downloadfile command to download the artifact snapshot. Use standard batch commands to download, rename and archive the snapshot file epmautomate downloadfile “Artifact Snapshot" 56
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
SCHEDULING PBCS ARTIFACT BACKUP • On Windows Task Scheduler, use the ‘Start In’ option to download the archive in a specific directory. • If not specified, the snapshot is downloaded to the same directory as the bat file
57
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
SAMPLE ARTIFACT BACKUP BATCH FILE
58
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
6 MISCELLANEOUS
59
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
2017HURON HURONCONSULTING CONSULTINGGROUP GROUPINC. INC.AND ANDAFFILIATES. AFFILIATES.ALL ALLRIGHTS RIGHTSRESERVED. RESERVED. ©©2017
USER AUDIT REPORT • The EPM Automate UserAuditReport command can be used to generate the list of users who have logged onto the system for the specified time periods. • Usage: epmautomate userauditreport FROM_DATE TO_DATE REPORT_NAME • Following is an example of a batch that can be executed on a daily basis.
60
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
USER AUDIT REPORT • The file is saved to the Inbox/Outbox Explorer
• The file can be downloaded, renamed with a timestamp and archived to appropriate folder.
61
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
USER AUDIT REPORT • Following is an example of an Audit Report generated and archived.
62
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
MAP REPORTING APPLICATION • Use ‘Map Reporting Application’ under ‘Administration’ (standard interface) OR Application->Data Maps (simplified interface) to create and define the map between two databases Standard Interface
Simplified Interface
63
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
MAP REPORTING APPLICATION • Use runplantypemap to execute the map to copy data from a BSO to a BSO or a BSO to an ASO database • Usage: epmautomate runplantypemap JOB_NAME [clearData=true|false]
64
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
QUESTIONS?
65
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
THANK YOU
66
© 2017 HURON CONSULTING GROUP INC. AND AFFILIATES. ALL RIGHTS RESERVED.
67
