MDM Extract Custom Publish Views

MDM Extract Custom Publish Views

Overview

The task group MDM Extract Custom Publish Views helps to create the Custom Views by executing the CSV file placed in the S3 location (<defaultBucket>/Custom_Views/<CSVLoad), which contain the details about the Connection Name, Schema Name, View Name and sqlQuery.

Download the Pipeline Template

To create the Extract Custom Publish Views, you need to first create a Task Group in IDP OA platform to execute the process. Follow the below steps to create a task group in IDP OA platform.

  1. Using WinSCP or s3 browser, connect to the IDP default s3 bucket and go to the folder <bucket_name>/templates/product.

  2. Find the template MDM_Extract_Custom_Publish_Views_<version>.json and download the latest version template to the local machine. See the figure below.


Create Task group

  1. Login to IDP OA platform and under the Data Management section, click Data Pipeline.

  2. On the Landing Page, click Data Pipeline tile to open the Task Group Pipeline Flow.

  3. Click Task Group from Template, select the latest downloaded template MDM_Extract_Custom_Publish_Views_<version>.json and then click OPEN.

  4. The pipeline task group for MDM_Extract_Custom_Publish_Views will be created. This task group is used for creating the Custom Views.

  5. After the successful import, navigate to the last page in the Task groups section to find the newly added task group MDM_Extract_Custom_Publish_Views.

    1. Click MDM_Extract_Custom_Publish_Views task group and open the Tasks tab.

      You can see the below tasks listed for Custom Views.

      • Load Custom Views Config file

      • Create Custom Views

      Load Custom Views Config File

      This task performs the below steps:

      1. Identifies the data in .CSV file located in S3 bucket.

      2. Loads the data present in .CSV file to create Custom Views as per the sqlQueries listed in the file using s3Connector plugin.

      Before you RUN this task, navigate to Steps tab and then click Sync Files to sync the .CSV file data.

      Note:   

      Make sure to Sync Files whenever you RUN the task.

      The below image shows the Custom Config Views created in a .CSV file:

      Add a View

      Follow the below steps to add a view:

      1. Open the .CSV file.

      2. Enter the fields OBJECT ID, SCHEMA NAME, VIEW NAME, VERSION, ORDER BY, IS PRODUCT BASED and SQL QUERY for the new view that you want to add.

      3. The VERSION must be 1 for new Views, and ORDER BY value must be same as the OBJECT ID.

      4. When the user adds a new view, IS PRODUCT BASED column must be entered as NO or N.

      Modify an Existing View

      Follow the below steps to modify an existing view:

      1. Open the .CSV file.

      2. Copy the view that you want to modify and add it in a new row as shown below.

      3. Change the OBJECT ID.

      4. Leave the SCHEMA NAME and VIEW NAME as is.

      5. Version number indicates the latest version of SQL QUERY. Change the version number whenever you modify the SQL QUERY. In this case, change it to 2.

      6. The ORDER BY field remains the same. In this example, it must be 77.

      7. Modify the SQL QUERY as required and Save the file.

    Create Custom Views

    The task extracts the .CSV data from s3 to creates the Custom Views. To create the Custom Views, RUN the task group MDM_Extract_Custom_Publish_Views.

Troubleshooting

  • Total 2 tasks are present in this task group (MDM_Extract_Custom_Publish_Views).

  • In case of failure in any task, fix the error and restart the task from the failed task till the end. If unable to identify or fix the error, contact MDM support team.