Import data into the staging tables

You must ensure that the data that you import into the staging tables does not cause errors in integrity checks.

About this task

Repeat the following steps until the integrity checks report no errors.

Procedure

  1. Run the zone import tool. Run the command prompt tool zone_import or use the IZoneImportAPI web service to import your CSV data.
  2. Set the server to the maintenance run level. Set each PolicyCenter server to the maintenance run level by using the system_tools command prompt tool or the web service SystemToolsAPI.setRunLevel method.
  3. Request integrity checks from the table import tools. Use the TableImportAPI web service method integrityCheckStagingTableContentsAsBatchProcess or the following command prompt command. 
    table_import -password password -integritycheck -batch

    For an Oracle database, take instrumentation snapshots before and after running integrity checks. Generate database instrumentation reports to identify performance problems.

    Note: The -batch option does not wait until the started process completes before returning. Instead, it returns immediately and prints the ID of the started process (PID). The process caller is responsible for waiting for the process to complete before taking further action.
  4. Set the server to the multiuser run level. Set each PolicyCenter server to the multiuser run level by using the system_tools command prompt tool or the web service SystemToolsAPI.setRunLevel method.
  5. Check load errors from the Load History page. This page is available at Server Tools > Info Pages > Load History.
  6. Correct errors in the zone data source files. Edit the zone data CSV file to correct the data.
  7. Decide whether to populate or clear the error and exclusion tables. The error table contains information about the rows that failed the earlier integrity checks. If you want to clear these errors before the next integrity checks run or exclude data rows that cause errors, perform the following steps.
    1. Exclude rows that caused integrity check errors. Populate the load exclusion table with the LUWIDs that contain errors. Use the TableImportAPI web service method populateExclusionTableAsBatchProcess or the table_import command prompt tool.
    2. Remove excluded records from staging tables. Remove all rows from all staging tables for records whose LUWIDs are listed in the exclusion table. You can use the table_import command prompt tool or the TableImportAPI web service method deleteExcludedRowsFromStagingTablesAsBatchProcess.
    3. Clear the error and exclusion tables. Optionally, for clarity, remove the errors and exclusion rows related to earlier integrity checks. Use the TableImportAPI web service method clearErrorTableAsBatchProcess or the table_import command prompt tool. When you next run integrity checks, only the errors from that run appear in the user interface.