Xponent Data Load

Prerequisite

  1. MDM_Load_Refresh_Reltio_Views_Customer_Master task groups should be successfully executed at least once.

  2. To configure notification channels such as email id or MS teams channel or slack channel, see Adding Alert Notification. Configure at least one notification channel and Subscribe to the task group pipeline.

Xponent Data Load Overview

Note:   

To avoid the mapping issues while extracting the RDM codes from two different tenants in the same IDP environment, each RDM_SOURCEMAPPINGS table name is configured with a suffix specifying the Reltio connection name as IDPSchema.MV_@ReltioConnectionName_RDM_SOURCEMAPPINGS. The RDM_SRCMAPPINGS view references from Staging RDM table for which, this process manages the transaction and sync it from Land to Stage.

Import Pipeline Template

To Import Pipeline Template

  1. Connect to IDP default s3 bucket and go to the folder bucket_name>/templates/product.

  2. Download the MDM_Load_Reltio_Xponent_<version>.json file template to local windows folder. There can be multiple files suffixed with different versions, download the latest version and use it.

  3. Open the downloaded template json file with any text editor to find and replace the below placeholders with appropriate values.

    Placeholder Replaceable String
    <RELTIO_CONN_NAME> Reltio Connection name configured in Entity Collection. For example, RELTIO_MDM

    Note:   

    The search strings given above are case-sensitive, do not enclose with any character, replace as it is.

  4. Login to IDP platform with valid credentials.

  5. Go to Data Pipeline app, click Task Group from Template and import the template using updated template file.

    Note:   

    Process creates the pipeline task group after the successful import. This has to be used for execution.

Operational Steps

The following are the operational steps:

  1. Keep the data files ready as per the layout defined in the Xponent DID.

  2. Login to IDP console and open the MDM_Load_Reltio_Xponent task group.

  3. To configure notification channels such as email id or MS teams channel or slack channel, see Adding Alert Notification. Configure at least one notification channel and Subscribe to the task group pipeline.

  4. Using Winscp or S3 browser, login to default S3 bucket path mentioned and go to the root folder. Place the data files inside the <s3 root folder>/Xponent/input/ folder.

  5. Note that this task group execution has a capability to create XPO HCP Reference data which is used in data bridging application. By default this reference data creation is disabled. If user wants to create reference data then, set the value of parameter Create_XPO_Reference_Table to true in this task group. Also, if enabled this feature then, do not use/execute the other task group Bridge_XPO_Product_Reference_Load which also creates Xponent HCP reference data.

  6. Once files are placed, click Run in IDP platform to trigger the pipeline task.

  7. Click Show Execution Logs which is next to Run button for verifying the task status.

  8. Monitor jobs:

    1. Both Data pipeline Jobs or API call jobs used to post data to Reltio can be monitored through spring cloud data flow (SCDF).

    2. To monitor, Login to IDP portal, click Data Pipeline in Landing Page.

    3. Identify the Task Group/Pipeline name for which monitoring is required (For example. MDM_Load_Reltio_Xponent). Click Show Execution Logs.

    4. Spring cloud data flow page opens in a new tab. Click Jobs.

  9. Once the tasks are completed successfully then execute the below queries to get counts of HCP, HCO, Affiliation and Merge from canonical table present in snowflake data warehouse under the schema name MDM_CANONICAL. These counts are used to validate against the counts of data posted to Reltio.

    • SELECT COUNT(0) FROM MDM_CANONICAL.C_CUSTOMER where LOAD_STATUS = 'SUCCESSFUL' AND CUSTOMER_TYPE = 'HCP' AND SOURCE_NAME='IMSFactory'.

Troubleshooting

  • Mapping failure issues has to be fixed in the staging to canonical configurations. Make sure that Reltio attribute and data type is matching with it.

  • In case of failure in any task, fix the error and restart the task from the failed task till the end. If unable to identify or fix the error, contact MDM support team.

See DID of Xponent data load to Reltio in Data Interface Documents.