Using Prospective Student Import
These topics describe the process for importing suspects, prospects, and test scores into CRM.
|
Page Name |
Definition Name |
Usage |
|---|---|---|
|
RB_CIM_BATCH_SRCH |
Search for batches to import or purge. |
|
|
RB_CIM_BAT_RUN_SEC |
Confirm batches to import or purge. |
|
|
RB_CIM_BATCH_DTL |
Import or purge batches. |
|
|
RB_SM_CFG_MAP |
Maps Test IDs to Configuration IDs. |
|
|
RB_CIM_ROW_DTL |
Review and change import row status. |
The Prospective Student Import feature allows the bulk import of suspects and prospects into CRM for Higher Education. Prospective students can be loaded directly into CRM for Higher Education using the import feature, or sourced from the Campus Solutions test score loads. Campus Solutions test scores can be pushed to CRM for Higher Education without creating new persons in Campus Solutions, therefore allowing CRM to be the system of record for suspects and prospects and preventing thousands of prospective students from cluttering the Campus Solutions database.
The following functionality is included in CRM for Higher Education for the Prospective Student Import:
An out-of-the-box integration with Campus Solutions, by which prospect and test score data sent from Campus Solutions can be processed and stored in a temporary staging area for further import processing.
A standard XML message structure for Higher Education prospects, which allows prospect data from an external source to be imported directly into CRM.
An import process that extracts raw data from the staging area, processes the data (validates it, transforms it, and identifies duplicates), and then posts it into the CRM Customer Data Model (CDM) and customer profile tables.
A purge process that permanently deletes staging area data that is no longer needed.
You can import prospect information into CRM in two ways:
Import from Campus Solutions.
Import directly into CRM.
Image: Importing Prospects and Test Scores into CRM
The following diagram shows the import process, with data coming from Campus Solutions into CRM and from an external source into CRM.
Importing from Campus Solutions
Search Tape (such as CSS) and Test Score (such as SAT or ACT) files, each of which contain a large number of records of individuals, are first loaded into the Campus Solutions staging area. If any errors exist, they are corrected. The Search/Match/Post process is run, one Test ID at a time. This process performs a Search/Match on the records and then posts them either to Campus Solutions tables or to CRM (this option is configurable in Campus Solutions). If data is to be posted to CRM, a standardized message containing each individual's biographical and demographic (“bio/demo”) data and other information if it is available (test scores, program/plan/subplan information, academic interests, and extracurricular activities) is sent to CRM via EIP. The CRM integration broker parses the message and stores it as a batch in the CRM Import Staging Area. Each batch consists of a set of import rows, each of which has associated child records.
Direct Import Into CRM
You can also import a list of prospects directly into the CRM staging area from an external source (such as a spreadsheet created at a recruiting event, or a third-party system). In this case, the external source sends CRM a message conforming to a standard message format that CRM can recognize. At this point, a new batch is created in the CRM staging area.
Staging
After a batch is in the staging area, it can be imported into the final Customer Data Model (CDM) tables by an import process that is run by the CRM Administrator. The import process performs the tasks of validating each of the batch's import rows, calling the Search/Match functionality to identify potential duplicates, transforming the data, optionally populating an audience list, and then posting it to Customer Data Model (and Profile) tables. The administrator manually resolves import rows that are not posted to CDM because of errors, or those that are put in suspended state (because potential duplicates were found in CDM) and the import process is re-executed on the batch. Afterward, a purge process cleans up the staging area tables by deleting data that the import process no longer needs.
Prospect and test score data is received by CRM in the form of XML messages. These messages are processed by the integration broker subscription code and then stored in the staging area for processing by the import process. Messages sent by Campus Solutions after a Search/Match/Post run are in a standardized form so that CRM can process them the same way irrespective of the type Search Tape or Test Score they represent. Messages sent by external sources (other than Campus Solutions) must also conform to a standard message format, and the import process handles them in the same manner as messages from CS.
Importing Test Scores
You can import the following test score data from Campus Solutions (CS) into CRM:
ACT
AP
CRS
DAT
EOS
GMASS
GMAT
GRE
LSAT
SAT
SSS
TOEFL
CRM receives prospect and test score data in the form of XML messages. These messages are processed by the integration broker's subscription code and then stored in the Staging Area for processing by the Import Process. Messages sent by Campus Solutions after a Search/Match/Post run are in a “standardized” form so that CRM can process them the same way regardless of the type of search tape or test score that they represent. Messages sent by external sources (other than Campus Solutions) must also conform to a standard message format, and the Import Process processes them in the same manner as messages from Campus Solutions.
Messages from Campus Solutions
To summarize, the message consists of the following parts. The specifics of these parts are discussed in the Campus Solutions documentation.
The header record SAD_HEADER_CRM
Standardized Bio-Demo and Prospect combined record (SAD_BIO_PRS_SUS)
Mapped Prospect child records (SAD_PRS_INIT_SUS, SAD_PRS_EXT_SUS, SAD_PRS_COM_SUS)
Standardized test score record (SAD_TST_COM_SUS)
Test-specific suspense records (includes the candidate data): Note that the test-specific suspense record (candidate data) is specific to each type of test and is not processed by CRM even if it is sent by Campus Solutions.
Name/Value pairs (SAT_TEST_POST_NAME_VAL_PART_DS).
See PeopleSoft Recruiting and Admissions
See PeopleSoft Campus Solutions Application Fundamentals, "Introducing Customer Relationship Management for Higher Education"
Messages from External Systems
For loading prospects from sources other than Campus Solutions directly into CRM, CRM includes a standard XML message specification that it recognizes as a Higher Education prospect load message. External sources must convert their prospect data into this format before sending it to CRM. This message is parsed, placed in the Import Staging Area, and processed by the Import Process just as the Campus Solutions messages are processed. The mechanism to load prospect data from Campus Solutions and from other external sources into CRM is the same.
A Test ID named EXT is included as system data in CRM to support external prospect loads. Messages from sources other than Campus Solutions must have Test ID set to EXT in the messages that they send to CRM.
The Staging Area
Prospect records (and their associated data) to be imported into CRM are first placed in the common staging area. This is a temporary storage area and consists of a set of standard, related tables that are populated from the message data received by CRM. When it is run, the Import Process uses data from the staging area to create the final “production” tables in the Customer Data Model (and other profile tables). Thus, the staging area is the common “gateway” for bulk importing Higher Education-related prospect data into CRM.
Import Batches
A batch (also called an “import batch”) is a collection of prospects, suspects, and test scores sent by Campus Solutions or a third party to CRM as part of a single message. A batch is also the “unit of work” for Import and Purge processing in CRM. That is, the Import Process and Purge Process run on a per-batch basis. Note that if someone on campus re-posts the same test or tape, a new batch is created because it is a separate post
The Batches entity stores the header information of each batch created in CRM. Every batch can be uniquely identified by a Batch ID, which is the primary key of the Batches entity. A Batch usually belongs to single Test ID. However, multiple batches exist at any time for the same Test ID. A new, unique Batch ID is generated for every new batch created.
Note: Batches usually belong to the same Test ID, but TEST_ID_OVRD (Test ID Override) can be used to load different test score values for a different test (for example, it is used by Campus Solutions for SAT, which has SAT I and SAT II scores within the same load. Additionally, TEST_ID_OVRD must be used by external loads, because TEST_ID is required to be EXT in the message structure, in order to load self-reported scores).
When created, a batch consists of one or more import rows. These rows are stored in an Import Rows entity. When the Purge process deletes all of the rows of a batch, it will be an empty batch. Every batch has a Batch Status, which defines the current state of the batch and determines the processes that can be run on it. The following table shows the statuses in which a batch can exist:
|
State |
Description |
|---|---|
|
Active |
The batch has at least one import row, and the Import or Purge process can be run on it. |
|
Purged |
The batch has no import rows, because all the rows have been purged (deleted by one or more runs of the Purge Process). The Import or Purge process cannot be run on this batch. A batch in this status is only stored for auditing/history reasons. |
|
Import in Progress |
The Import Process is currently being run on the batch. Another Import or Purge Process cannot be run on it. |
|
Purge in Progress |
The Purge Process is currently being run on the batch. Another Import or Purge Process cannot be run on it. |
If a batch is created as a result of data received from a Campus Solutions Search/Match/Post run, then the following other information is also saved:
CS Run Control ID.
CS Loaded Date.
Institution.
Career.
These additional fields are not mandatory; they can be empty for batches created with data received from third-party sources
Each batch file has a Source value, that describes the origin of the prospect data. If the data is loaded from Campus Solutions, then the value Campus Solutions is stored in this field. If the data is loaded from an external source, the value External is stored in this field.
Import Rows
An import row is an individual prospect or suspect imported from a Campus Search Tape/Test Score load or from an external source. Import rows consists of a standard structure that stores bio/demo and recruiting status information for the individual, regardless of the suspect or prospect's origin (for example, SAT load, GRE load, spreadsheet, and so on).
An import row has Test Record number, a Test ID, and always belongs to one and only one batch. It contains the following types of data:
Bio/Demo-related information: Name, Address, Phone, Email (and types), National ID, Sex, and so forth, along with EmplID (if available).
Prospect information: Institution, Career, Recruiting Status, and so on.
Other relevant information such as last school attended, ethnic group code, and so on.
Every row maintains a row status, which defines the current import processing status of the row. The following table shows the possible statuses:
|
Status |
Description |
|---|---|
|
Ready for Processing |
The default initial state of an import row. This status instructs the import process to perform data validation for the row and run Search/Match before posting it to CDM. |
|
Posted |
The import row has been posted to CDM. The row can be purged. |
|
Suspended |
The import process (after running Search/Match) found this row to be a duplicate. The Administrator must review and resolve it. |
|
Add Person |
Instructs the import process to create a new person. This status is set only after a review process after a row has been marked Suspended. |
|
Update Person |
Instructs the import process to update an existing person. This status is set only after a review process after a row has been marked Suspended. |
|
Ignore |
Instructs the import process to ignore the row. This row can be purged. This status is set only after a review process after a row has been marked Suspended. |
|
Error |
The row has encountered an error during the import process. The Administrator must review and resolve it. |
After an import row is created, its row status can be changed automatically by the import process or manually by the Administrator.
Test Results
The Test Results entity stores test results for an import row. It includes Test ID, Test Record number Test Date, Test Component and Data Source.
Note: When Campus Solutions runs Search/Match/Post for SAT I, scores for both SAT I and SAT II type Test IDs are sent over. This means that SAT II scores are sent along with SAT I scores even if SAT II is not explicitly selected in the Search/Match/Post. When these scores arrive on the CRM side, they are all stored under the SAT I Test ID. For SAT II scores, the Test ID field in the Test Results Entity still contains SAT I, but the Test ID Override value is populated with SAT II. This is an indication to the import process to consider the Test Components (and other attributes) as belonging to SAT II (and not SAT I) when posting to the final CRM Test Scores tables.
Other Import Row Information
An import row also contains other entities that can store prospect Program/Plan/Sub Plan details, Academic Interests, and Extracurricular Activities.
This topic describes the process for populating the staging area.
To define configuration mappings, use the Test Score to Results Configuration Mapping component. Use the RB_SM_CFG_MAP component interface to load data into the tables for this component.
Creating Batch and Import Rows
When an import occurs, an input message (a single message or multiple related messages) containing prospect and test score data is received by CRM as a result of a Campus Solutions Search/Match/Post run or from another external source. This message is parsed and processed in the CRM integration broker. The following steps occur:
The incoming message first goes through a data validation step where the batch-level fields are validated. The following fields are verified if to see if they exist in CRM:
Test ID.
Institution.
Career.
If this validation fails, no batch is created and the message is marked as an error (the message can be resubmitted after the errors have been corrected).
If the validation succeeds, a new Batch ID is generated, and then a batch with this Batch ID is created in the staging area
If the batch is created as a result of a Campus Solutions message, then the Created By value is set to the Operator ID (OPRID) sent on the message, but if the batch created from an external source, then the Created By value is set to a System user.
If the batch is created as a result of a Campus Solutions message, then the batch Source is set to Campus Solutions. If the message originated from a third party, then the Source is set to External.
Individual import rows are created for the batch rows. Test Record Numbers are not generated—they are part of the key structure of the message and must be generated by the entity that creates the XML Message (either Campus Solutions or the external loading code). Associated child records (Test Results, Prospect Program/Plan/Sub Plan, Academic Interests, and Extracurricular Activity information) are also created for each import row if they were provided on the incoming message.
All import rows of the batch are defaulted to the Row Status of Ready for Processing. The Batch Status is then set to Active.
Messages From Search/Match/Post in CRM
When Search/Match/Post for a particular Test ID is run in Campus Solutions, it is done in context of a Run Control ID. You can also provide additional run control parameters (such as default prospect institution and career information). Every run of a Search/Match/Post (with the post to CRM option) results in suspect, prospect, and test score (if a test score exists for the person) data being messaged to CRM. When a new batch is created for this data in the CRM staging area, the following additional run control related information is also stored along with it:
CS Run Control ID.
CS Loaded Date.
Institution.
Career.
Operator ID (OPRID), as Created By.
The HE Administrator can use this information as search criteria when querying for a batch (or set of batches) before running the Import or Purge processes.
Messages from External Sources
An external source can also send CRM a list of prospects/suspects conforming to the standard input message. In this case, each prospect in the message can potentially be associated with a different institution and career, so these fields are typically not populated for the batch that is created. Also, in the case of data from an external source, the CS Run Control ID and CS Loaded Date are empty. However, the Created By field is still set to a CRM system user.
A batch is the “unit of work” on which the Import and Purge processes operate. You can run the Import or the Purge processes on an active batch at any time and as many times as needed.
To import, the Administrator identifies the batch or batches to be imported and runs the import process. Typically, not all import rows in the batch are posted the first time, because rows can fail due to errors (for example, data validation errors) or due to their being potential duplicates. The administrator resolves such rows and then reruns the import process on the batch. This process is continued until all rows are posted or marked as Ignored.
To purge, the administrator identifies the batch or batches to be purged and runs the purge process. The purge process deletes only the rows from the batch that are no longer needed (such as those that are already posted or marked Ignored). Thus, it cleans up the staging area of rows which are no longer to be processed by an import execution, thus improving import process performance. After they have been purged, rows are deleted and cannot be recovered.
Use the Manage Import Batches page (RB_CIM_BATCH_SRCH) to search for batches to import or purge.
Navigation
Customers CRM, Prospective Student Import, Manage Import Batches
Image: Manage Import Batches page (1 of 2)
This example illustrates the fields and controls on the Manage Import Batches page (1 of 2). You can find definitions for the fields and controls later on this page.
Image: Manage Import Batches page (2 of 2)
This example illustrates the fields and controls on the Manage Import Batches page (2 of 2). You can find definitions for the fields and controls later on this page.
To perform an import or purge process, you must first identify the batches. The Manage Import Batches page displays a search page where you can search for batches. Note that you must scroll to the right to see all of the information on the Manage Import Batches page.
|
Field or Control |
Definition |
|---|---|
| Update All Bio-Demo |
Select the check box to update all biographical and demographic data for existing consumers in CDM. Note: In the Manage Import Batches Search Results page, the Update All Bio-Demo check box is enabled only for batches with the Active status and only when all of its import rows are in the Ready for Processing status. For all other batch and import row statuses, the check box is disabled. When this check box is selected, all the bio-demo data related fields for the selected batch in CRM will get updated with the bio-demo data from the import rows. When this check box is not selected only some of the bio-demo fields will get updated based on the existing field values. The rules followed in both the cases are detailed later in this section. See Updating Consumer Data. |
| Batch ID |
Click the link to display details about the batch. |
| Test ID |
The identifier for the test. |
| Date/Time Created |
The full date and time when the batch was created. |
| Created By |
The name of the person who created the batch. |
| Source |
Campus Solutions or External. |
| Batch Status |
The current status of the batch: Active, Purged, Import in Progress, or Purge in Progress. |
| Total Count |
The total number of rows in the batch. |
| Awaiting Import |
The sum of the number of rows in Ready for Processing, Add Person, and Update Person statuses. |
| Awaiting Purge |
The sum of the number of rows in the Posted and Ignore statuses. |
| Posted |
The number of rows in the Posted status. |
| Ready for Processing |
The number of rows in the Ready for Processing status. |
| Suspended |
The number of rows in the Suspended status. |
| Error |
The number of rows in the Error status. |
| Ignore |
The number of rows in the Ignore status. |
| Add Person |
The number of rows in the Add Person status. |
| Update Person |
The number of rows in the Update Person status. |
Use the Run Import/Purge page (RB_CIM_BAT_RUN_SEC) to confirm batches to import or purge.
Navigation
Click the Import or Purge button on the Manage Import Batches page.
Image: Run Import/Purge page, Import selected
This example illustrates the fields and controls on the Run Import/Purge page, Import selected. You can find definitions for the fields and controls later on this page.
You can only import or purge batches that are in the Active status. If you want to import or purge more than one active batch, select the check box next to each desired batch, then click the Import or Purge button to go to the Run Import/Purge page. You can select all active batches by clicking Select All, or clear the selection by clicking Clear All.
The Run Import/Purge page acts as a confirmation page, displaying the list of batches, and the Update All Bio-Demo values you have selected. You can remove a batch from the list by clicking the trash can icon next to it, or click Cancel to go back to the search page. )
If you are performing an import, the Import Options section is displayed (this section is not displayed if you are performing a purge).
|
Field or Control |
Definition |
|---|---|
| Generate Audience |
Select this check box if you want to populate an audience as part of the import process. |
| Audience SetID |
Select an audience Set ID to choose an audience name from that Set ID. The Audience SetID option is available only if the Generate Audience check box is selected. |
| Audience Name |
Select an audience name from the specified SetID. This option is available only if the Generate Audience check box is selected. Only audiences of type Internal using Import in the In Design or Designed status are available for selection. |
If you choose to generate an audience, when the import process executes, import rows (from all selected batches) that are posted are also appended to the existing list of the specified audience (after deduplication occurs). The audience itself is set to Generated status during the import process—you must manually set it back to the In Design or Designed status if you want it available to be selected again in another import.
After you have specified the options on the Run Import/Purge page, click OK to launch the process scheduler request page.
Use the Import Batch page (RB_CIM_BATCH_DTL) to import or purge batches.
Navigation
Click a batch link on the Manage Import Batches page.
Image: Import Batch page
This example illustrates the fields and controls on the Import Batch page. You can find definitions for the fields and controls later on this page.
|
Field or Control |
Definition |
|---|---|
| Batch Details |
Displays the attributes of a batch that are common to batches created from Campus Solutions message data and those created from external sources. |
| Campus Run Parameters |
Displays data that is specific to batches created from Campus Solutions message data. For batches created from eternal data, the fields in this section are empty. If the source is Campus Solutions, this section is expanded by default; if the source is external, it is collapsed. |
| Row Status Summary |
This section shows the status summary of the import rows currently belonging to the batch. These counts are computed when the batch component is loaded only if the last process run on the batch ended abnormally. Otherwise, the values are loaded from the counts stored on the batch in the staging area. When a batch is first created, the Total Count value is equal to the total number of rows in the batch; all other statuses have a count of 0. If the batch is in Purged status, the counts for all statuses are set to 0. Each count in the Row Status Summary is linked if it is greater than 0. Clicking on the count link takes you to the Manage Import Rows search page and execute the appropriate search query. You can use these links as a quick way to view rows in a particular status. |
| Import Options |
The Generate Audience, Audience SetID, and Audience Name options were described in the section on importing and purging multiple batches. Select the Update All Bio-Demo Data check box to update all bio-demo data for existing consumers in CDM. Note: The Update All Bio-Demo check box appears enabled only for batches with the Active status and only when all of its import rows are in the Ready for Processing status. For all other batch and import row statuses, the check box appears disabled. Note: If the Higher Education installation option is set on the CRM instance and you try to commit an audience that has the source Internal using Import, the system checks whether the audience has been populated from any of the import batches in the system. If it has, the system displays a warning that setting the audience to Committed will make it permanently unavailable for use in future import process runs. You can then choose whether you want to change the audience's status. Click the Import or Purge button to perform the import or purge process. You cannot click a button if there is nothing to import or purge. |
Viewing Import History on the Run History Page
Access the Run History page (click the Run History tab on the Import Batch page).
The Run History tab shows a summary of all the import and purge process runs that have been performed on the batch. The default display order is reverse chronological order by the Start Time. A row appears on the grid after an import or purge is initiated, but you cannot view details about the process instance until the process has actually begun.
Note: This details on this page are refreshed each time you access it.
|
Field or Control |
Definition |
|---|---|
| Run Type |
This value is either Import or Purge, depending on the process run. |
| Run By |
The name of the person who ran the process. |
| Start Date/Time |
The date and time when the process was initiated. |
| End Date/Time |
The date and time when the process completed. |
| Audience SetID |
The SetID for the audience used in generation. This value can appear only for imports; the field is empty if the Run Type is Purge. |
| Audience Name |
The audience to which the Import is added. This value can appear only for imports; the field is empty if the Run Type is Purge. |
| Rows Processed |
For an import, the total count of the rows in Ready for Processing, Add Person, and Update Person statuses. For a purge, the total count of the rows in the Posted and Ignore statuses. |
| Run Status |
The current status of the process. |
| Process Instance |
The Process Instance ID (from the process scheduler). Click the link to view details of the process scheduler process. |
Handling the Abnormal Termination of the Import or Purge Process
In rare instances, the import or the purge process run on a batch can abnormally end after processing only some rows of the batch. If this occurs, the batch can remain in a locked state (in the Import in Progress or Purge in Progress status); the row counts for the various statuses stored on the batch will not reflect the actual counts because the import or purge process should update them at the end of processing. If this occurs, you might think that the batch is being processed, where in fact the process instance for the import or purge process is dead.
To recover from this situation, the Batch component on load performs the following actions:
If the batch is in Import in Progress or Purge in Progress status, check whether the Process Instance of the process last run on this batch is still running. This process instance information is obtained from the Batch History.
If the Process Instance is not running, this means the process has ended abnormally. The system then does the following:
Updates the latest Row Status Summary counts on the batch (in the staging area) and sets the Batch Status to Active.
Displays a warning message informing you that the import or purge process that was last run on this batch terminated abnormally, the Row Status Summary shows the most current counts, and that you should rerun the import or purge process.
Use the Prospect Import Mapping page (RB_SM_CFG_MAP) to maps Test IDs to Configuration IDs.
Navigation
Image: Prospect Import Mapping page
This example illustrates the fields and controls on the Prospect Import Mapping page. You can find definitions for the fields and controls later on this page.
The import process run for a batch invokes the Search/Match API with a particular Result Action Configuration ID. The Configuration ID used for a batch depends on which Test ID the batch belongs to. The Prospect Import Mapping page maps Test IDs to Configuration IDs. A Test ID cannot be mapped to multiple Configuration IDs, but the same Configuration ID can be used for several Test IDs.
The goal of the import process is to move import row data (bio/demo, prospect, test scores, program/plan/sub plan, academic interests and extracurricular activities) belonging to a batch from the staging area to CDM and profile tables. This is called posting. The import process performs the following activities:
Extract import rows (and their child data) from the staging area, provided that they are in the Ready for Processing, Add Person, or Update Person status.
Validate the data.
Transform the data to the format required by CDM.
Run Search/Match logic on the data to identify duplicates.
Write the transformed data to CDM:
Create new Consumers along with their contact methods, test results, and lifecycle information.
Update existing persons, adding consumer information if needed, and adding or updating their contact information, test results, and lifecycle information.
Append data from import rows to the member list of an existing audience, if this option is specified at the time of import.
Update the Row Status of each import row processed.
Image: Import Process
The illustration below is the functional flow diagram of the import process.
The process performs the following steps:
Ensure that the batch being imported is Active and set it to Import in Progress status. This ensures that no other import or purge process runs can occur on this batch. Clear any error messages logged for the batch due to a previous import process run.
Process only import rows in the batch that have the Row Status of Ready for Processing, Add Person, or Update Person. Each row must processed independently of the others (that is, the status of one row—for example, if an error occurs—should not affect the processing of other rows).
For rows that are in the Ready for Processing status:
Run all data validations on the row. If a data validation fails on a row, then set it to Error status and display the appropriate error message. Continue performing all other data validations that could lead to more error messages to be logged for the same row, in order to gather as many errors for the row as possible at each run. This allows the administrator to resolve all of them before the next run.
Check if EmplID is available on a row.
If EmplID is not available, invoke Search/Match logic to check for potential duplicates. Based on the results provided by Search/Match, perform one of the following operations and set the row status as indicated.
Action
Row Status
Create a new Consumer
Posted
Update an existing person
Posted
No action
Error
If EmplID is available, this means that the imported individual already exists as a Person in CDM and there is no need to call Search/Match. In this case, the system updates the existing person with the matching EmplID and then sets the row to Posted status.
For rows that are in the Add Person status, add a new Consumer to the system and then set the row to Posted status.
For rows that are in the Update Person status, update the person specified on the import row and then set the row to Posted status
For rows that are in the Add Person or Update Person statuses, if an audience is specified in the import options then append this consumer to the member list of the audience.
If any exceptions occur during the processing, set the rows to Error status and display the appropriate error message. Note that import processing does not stop because a row encounters an error. The other rows are still processed.
After all rows are processed, log a run history row for the batch.
Set the batch status back to Active.
Data Validation at the Import Row Level
Every import row (and its associated child records) first goes through a data validation process. If one of the data validations fails for a row, the row is marked as Error and an error message is logged along with it. However, the processing of the row is not aborted immediately after a data validation failure. Other data validations can still be performed and more errors logged on the same row before processing of the row is aborted. This is so that as many data validation errors as possible for the row can be collected and displayed on the row, and the administrator can handle all of them before the next run.
The following validations are performed:
Ensure that entries for control-type fields specified on the import rows exist in control tables synced to CRM from Campus Solutions. This ensures that invalid values are not sent for these fields. For example, a batch is to be imported and the Test ID field in the batches entity has a value of SAT I, then a validation step ensures that SAT I exists in the Test control table. The following control field type data in the staging area is validated against control tables:
Staging Area Entity
Field
Import Rows
Test ID
Import Rows
Institution
Import Rows
Career
Prospect Program/Plan/Sub Plan
Program
Prospect Program/Plan/Sub Plan
Plan
Prospect Program/Plan/Sub Plan
Sub Plan
Import Rows
Admit Type
Import Rows
Admit Term
Import Rows
Academic Level
Import Rows
Referral Source
Test Results
Test Component
Test Results
Test Data Source
Test Results
Test ID Override
Import Rows
Last School Attended
Import Rows
Housing Interest
Import Rows
Campus
Ensure that each of the following fields have valid values.
Staging Area Entity
Field
Import Rows
Recruiting Status is one of the following values: Prospect, Suspect, Inquiry, Applicant
Import Rows
National ID Type
Import Rows
Primary NID
Import Rows
Marital Status
Import Rows
Recruitment Status
Prospect Extracurricular Activities
Extracurricular - Internal/External
Prospect Academic Interests
Academic Interests - Priority
Ensure that the EmplID provided (in the Import Rows entity) exists in CRM. If it does not, this means that Person Basic Sync has not occurred or it has failed.
Ensure that the Academic Information conforms to the Academic Information hierarchy synced from Campus Solutions. This error does not occur if the data source is Campus Solutions.
Ensure that a Test Score (in the Test Results entity) is in the valid range of the Test Component for that Test ID. Valid ranges are sent over via EIP from Campus Solutions. This validation is omitted if the Source is Campus Solutions.
Calling the Search/Match API
Because the import process creates new persons (Consumers), it is necessary to ensure that a person being created already does not exist in CDM. The Search/Match API facilitates duplicate identification and thus discourages the introduction of duplicates. The Search/Match is run on the bio-demo portion of every row by invoking the Search/Match API. The Result Action Configuration ID mapped to the Test ID are passed to the API, along with bio-demo information. The results of the Search/Match API indicate to the import process whether the individual on the row already exists in CDM as a duplicate and what is to be done with the import row:
Add: Create a new consumer and its associated data.
Update: Update an existing person (specified by the Search/Match API result) and its associated data.
Suspend: Do not do anything with the row and let the administrator decide how to resolve it.
Creating Consumer Data
If prospect data is received for a person who is not already a Consumer and who does not exist in CDM as determined by Search/Match, then a new Consumer (person with the Consumer role) is created in the default SetID for inbound EIPs. This default SetID is specified in . Associated data, such as academic information, test scores, and so on, is then created for the person.
The following fields are used and rules are followed when creating consumer data in CDM:
Name related fields: The Last Name, First Name, Middle Name, Name Suffix, and Name Prefix fields from the import row.
Person related fields: The Sex, Birth Date, and Marital Status fields from the import row.
National Identification (NID) related fields: The National ID Type, National ID, and Primary NID fields from the import row. Note that the NID row created is required to be set to Primary even if the Primary NID value is N. This is because no other NID rows exist for the person at this time. The country is always USA. Note also that if the source is Campus Solutions, the Primary NID field is not sent, because it is always set to Y.
Other fields: The Ethnic Group Code, Ethnic Group Set ID, Citizenship Status, Religious Preference, and Regulatory Region fields from the import row.
Contact Methods: An import row can only have up to one address, one email and one phone specified. The system uses this information to create the create Address, Email and Phone type contact methods for the Consumer role. The Address Type, Email Type and Phone Type fields are used to derive the Contact Method Purpose Types for the Contact Methods.
The Address Type, Address1, Address2, Address3, City, State, Country, and Postal import row fields are used to create the Address contact method. The effective date is set to the current system date, with no end date.
The Phone and Phone Type import row fields are used to make up the Phone contact method. The effective date is set to the current system date, with no end date.
The Email Address and Email Address Type import row fields are used to make up the Email contact method. The effective date is set to the current system date, with no end date.
Creating Academic Information
The following fields are used and rules are followed when creating academic information data in CDM:
Institution and Career: The Institution, Career, Admit Type, Admit Term, Campus, Recruiting Status, Academic Level, Housing Interest, Financial Aid Interest, and Referral Source import row fields.
Last School Info: The Last School Attended and Graduation Date import row fields.
Academic Program: The Program, Recruiting Status (which becomes the Lifecycle status), and Campus import fields. The Creation Date is set to the current system date. Note that if Recruiting Status values of Inquiry or Applicant are sent by Campus Solutions, the corresponding Lifecycle Status in CRM is set to Prospect.
Academic Plan and Sub Plan: The Plan and SubPlan import row fields.
Creating Test Score Information
Test Score rows for a person are created in CDM from the Test Results entity tied to the import row. If the Test ID Override field has a value (that is, it is not empty), then create Test Scores for this Test ID.
Note: Note that if you create an external import (that is, not through Campus Solutions), you must create the message. The Test ID for the message must be EXT. If you have gathered self-reported test scores (test scores that did not come from load media from a confirmed organization through the Campus Search/Match/Post), you should use the TEST_ID_OVRD (Test ID Override) to populate the correct TEST_ID that you want to report.
Creating Extracurricular Activities Information
Extracurricular activities for a person are created in the Customer Profile Table for Extracurricular Activities from the Prospect Extracurricular Activities entity tied to the import row. The Start Date is set to the current system date.
Note: Not all fields in the Profile Table are received on the Campus Solutions message. Those that are not received are left empty.
Creating Academic Interests Information
Academic interests for a person are created in the Customer Profile Table for Academic Interests from the Prospect Academic Interests entity tied to the import row. The Effective Date is set to the current system date.
Updating Consumer Data
If prospect data is received for a person who already exists in CDM, then a new person is not created. However, if the person exists but does not belong to the Consumer role, then that role is added to it. Inserts or updates are made to the person’s bio-demo data (name, address, and so on) and its associated information (academic info, test scores, and so on). An Update All Bio-Demo check box is available to specify whether or not to update all the bio-demo data in CDM tables with the respective field values in the import rows. This option is provided at the batch level, enabling the administrator to update all the bio-demo data for selected batches and process another batches without choosing to update the bio-demo data. The following table illustrates the rules that are followed for updates when the Update All Bio-Demo check box is selected.
|
Information to be Updated |
Rules followed when the Bio-Demo check box is selected. |
|---|---|
|
Name related fields |
A new Name entry is created in CDM using the import row for the Person. This entry is marked as Primary name. The earlier Name entry is not deleted. This name can be accessed through the More Names link. |
|
Person related fields |
The CDM fields corresponding to the Birth Date, Sex, and Marital Status are updated with the values from the respective import row fields. If these are empty in import row, the existing values in CDM are maintained. |
|
NID related fields |
If National ID with NID Type already exists on the import row for the Person, it is updated. Otherwise, an NID row is inserted from the National ID Type, National ID, and Primary NID import row fields. Note: If the Source is Campus Solutions, the Primary NID value is not sent because it is always set toY In this case, the newly created NID row is designated primary (and the current primary row is made non-primary). |
|
Contact methods |
If Address Type on the import row matches with an existing, active Address of same contact method Purpose Type as the Consumer, the address is updated with the new address values effective as of the current system date with End Date of 12/31/2999. If Address Type on the import row does not match with an existing, active Address of same contact method Purpose Type as the Consumer, a new Address contact method of that type is added for the person’s Consumer role, effective as of the current system date with End Date of 12/31/2999. If Phone Type on the import row matches with an existing, active Phone of same contact method Purpose Type as the Consumer, the Phone is updated with the import row Phone value, effective as of the current system date with End Date of 12/31/2999 If Phone Type on the import row does not match with an existing, active Phone of same contact method Purpose Type as the Consumer, a new Phone contact method of that type is added for the person’s Consumer role, effective as of the current system date with End Date of 12/31/2999. The newly created contact method is not set to Primary unless there is no other Phone contact method for the Consumer. If Email Type on the import row matches with an existing, active Email of same contact method Purpose Type as the Consumer, the Email is updated with the import row Email value, effective as of the current system date with End Date of 12/31/2999. If Email Type on the import row does not matches with an existing, active Email of same contact method Purpose Type as the Consumer, a new Email contact method of that type is added for the person’s Consumer role, effective as of the current system date with End Date of 12/31/2999. The newly created contact method is not set to Primary unless there is no other Email contact method for the Consumer. The Address Book entries (ABEs) are rebuilt. |
The following table illustrates the rules that are followed for updates when the Update All Bio-Demo check box is not selected.
|
Information to be Updated |
Rules followed when the Bio-Demo check box is not selected. |
|---|---|
|
Name related fields |
Name related fields in CDM are never updated (even if empty). Also, no new Name entry is created. In other words, name related fields are never touched as part of updating consumer information. |
|
Person related fields |
The CDM fields corresponding to the Sex and Birth Date import row fields are updated only if previously empty. The CDM field corresponding to the Marital Status import row is never updated (even if empty). |
|
NID related fields |
If National ID with NID Type already exists on the import row for the Person, it is not updated. Otherwise, an NID row is inserted from the National ID Type, National ID, and Primary NID import row fields. Note: If the Source is Campus Solutions, the Primary NID value is not sent because it is always set to Y. In this case, the newly created NID row is designated primary (and the current primary row is made non-primary). |
|
Contact methods |
If Address Type on the import row matches with an existing, active Address of same contact method Purpose Type as the Consumer, the person's address is not updated. Otherwise, a new Address contact method of that type is added for the person’s Consumer role, effective as of the current system date (no end date). If Phone Type on the import row matches with an existing, active Phone of same contact method Purpose Type as the Consumer, the person's phone information is not updated. Otherwise, a new Phone contact method of that type is created for the person’s Consumer role, effective as of the current system date (no end date). This created contact method is not set to Primary unless there is no other Phone contact method for the Consumer. If Email Type on the import row matches with an existing, active Email of same contact method Purpose Type as the Consumer, then the email information is not updated. Otherwise, a new Email contact method of that type is created for the person’s Consumer role, effective as of the current system date (no end date). This created contact method is not to Primary unless there is no other Email contact method for the Consumer. The Address Book entries (ABEs) are rebuilt. |
Updating Academic Information
The following table shows the rules for updating academic information.
|
Information to be Updated |
Rules |
|---|---|
|
Institution and Career |
If the Institution/Career combination available on the import row does not exist for the person in CDM, that combination is inserted and the Career Detail fields are created from the following import row fields:
If the Institution/Career combination already exists, the Housing Interest and Financial Aid Interest Career Detail fields from the import row are updated only if they are empty. The Recruiting Status field is updated if it is empty or if it is set to a more advanced status than currently exists on the row (for example, if the current status in CDM is Suspect, and the import row is set to Prospect, then the field is updated). Note that if the value Applicant or Inquiry is received on the Recruiting Status field, it is interpreted as the Prospect status. No other fields are updated, even if they are empty. |
|
Last School and Graduation Date |
The Last School Attended and Graduation Date fields from the import row are created for the person in CDM only if they do not already exist in CDM. If they already exist, they are not updated. |
|
Academic Program |
If Academic Program on a Prospect Program/Plan/Subplan entity row of the import row does not exist for person in CDM, then the Program is inserted for the person’s Institution/Career combination. For each Program, Program level fields are created using the Recruiting Status (which becomes Lifecycle Status) and Campus fields. The Creation Date value is set to the current System Date. If the Academic Program already exists, then the Recruiting Status is updated only if it is empty or if it is of a more advanced status than the status currently on the row (for example, if the current status in CDM is Suspect, and the import row status is Prospect, then it is updated. Note that if the value Applicant or Inquiry is received on the Recruiting Status field, it is interpreted as the Prospect status. The Campus field is not updated even if it is empty. |
|
Academic Plan and Sub Plan |
If the Academic Plan/Sub Plan combination that is available on the Prospect Program/Plan/Subplan entity import row does not exist for the person in CDM, the Plan/SubPlan combination for the person’s Institution/Career/Program is inserted. If the Academic Plan/Sub Plan combination already exists, nothing is updated. |
Updating Test Scores
Each Test Result tied to an import row is matched with the Test ID, Test Component, Test Date, and Data Source of the person's CDM Test Scores tables.
If no match is found, a Test Result row is inserted for the person in CDM.
If a match is found, the CDM tables for the person are updated with the Test Result row data, provided that the REV_SCORE_IND flag (Revised Score Indicator flag) has the value of U on the Import Row. If this value is set to N, then no update occurs. If it is set to C, then update only if the score on the import row is greater than that in CDM.
Note: If the Test ID Override field has a value (that is, it is not empty) then this value is used to perform match and insert/update Test Scores for this Test ID.
Updating Extracurricular Activities
For every extracurricular activity associated with the import row (for example, the Prospect Extracurricular Activities entity), the system determines whether it matches an extracurricular activity for the person in the Customer Profile Table for the same Institution and Career (from the import row entity), Internal/External Flag and Start Date (current date). If such a match does not exist, that extracurricular activity is inserted.
Updating Academic Interests
For every academic interest associated with the import row (for example, the Prospect Academic Interests entity), the system determines whether it matches an External Subject Area for the person for the same Academic Career (from the import row) and Effective Date (current date) in the Custom Profile Table for Academic Interests. If such a match does not exist, that is inserted with the current date as the effective date.
The BO_ID of the updated person (Consumer) is saved on the import row.
Errors
Any row level errors encountered during import processing are stored on the row, so that the Administrator can review and resolve them later. Errors can arise in various stages of processing:
Data validation errors.
Errors returned by the Search/Match API.
Errors while building an audience.
Errors returned by the CDM.
The purge process permanently deletes import rows (and the child rows) of a batch from the staging area, provided that they are in the Posted or Ignore status. The purge process cleans up rows that are no longer needed, thus saving storage space and improving the performance of the import process.
Image: Purge process flow
The following diagram shows the logical flow of the purge process.
The process performs the following steps:
Ensures that the batch being purged is Active and sets it to the Purge in Progress status. This ensures no other runs of the import or purge process will affect this batch.
Deletes all the import rows belonging to the batch provided that they have the Posted or Ignore status. This includes all the associated child records of each import row.
Logs a run history row for the batch.
If the batch has no rows left after the deletion of import rows, sets the Batch Status to Purged. This ensures that this batch can no longer be imported or purged. Otherwise, it sets the Batch Status back to Active.
Use the Manage Import Rows page (RB_CIM_ROW_DTL) to review and change import row status.
Navigation
Manage Import Rows
Image: Manage Import Rows search page
This example illustrates the fields and controls on the Manage Import Rows search page. You can find definitions for the fields and controls later on this page.
Image: Manage Import Rows search results page
This example illustrates the fields and controls on the Manage Import Rows search results page. You can find definitions for the fields and controls later on this page.
Managing an import row primarily means reviewing its data and changing its status as needed, so that the import or purge process can process it in its next run. Updates to import rows (and their associated child data) are not allowed.
Note that updating a row status does not instantly run the import or purge process—it simply saves the updated status in the staging area.
To manage import rows, you use the Manage Import Rows component. This displays a Configurable Search screen that allows the Administrator to search for import rows using various search criteria.
Search results are rows from the Import Rows entity in the staging area that match the search criteria. Note that associated child rows like Test Results, Academic Interests, and so on are not displayed. The Search Result fields are separated into different tabs based on their categorization. Each field is read-only and displayed as stored in the Import Rows entity (no lookups, transformations or translations are performed).
Mass Update of Row Statuses
In certain cases, the HE Administrator might want to perform a mass update of certain rows' statuses of certain rows. (for example, change all Error rows to Ready for Processing after the errors have been fixed, or change all rows that are in Ready for Processing status to Ignore.
You do this by using the Reset, Ignore, Suspend, and Add buttons at the bottom of the configurable search results. If one or more import rows are selected (by selecting the check boxes for each row or by using the Check All/Clear All link) and one of the buttons is clicked, then the system attempts to change the status of all the selected rows to the new status indicated by the button you clicked. The following table shows the status to which the selected rows are changed based on the button clicked:
|
Button |
Status Change |
|---|---|
|
Reset |
Ready for Processing |
|
Ignore |
Ignore |
|
Suspend |
Suspended |
|
Add |
Add Person |
Only those rows that can be changed to the selected status are changed. If any rows cannot be changed, a warning message displays to inform you of the number of rows that were not changed.
For example, assume three rows in the following statuses: Ready for Processing, Suspended and Ignore. If the Add button is clicked, a warning message displays informing you that two rows could not be updated to the Add Person status. Selecting OK changes the status of Row 2 (Suspended) to Add Person and its Select check box is cleared. Rows 1 and 3 (Ready for Processing and Ignore) retain their original statuses, and their Select check boxes remain selected.
Click Save on the toolbar to commit the changes you have made.
Note: Update is not available as a mass action because a row cannot be changed to the Update Person status unless the person (Consumer) to be updated is known. Hence, this action can only be performed after reviewing one import row at a time.
Reviewing Import Rows One at a Time
Click the Record Number link of any of the search results to display the Manage Import Rows page.
Image: Manage Import Rows page
This example illustrates the fields and controls on the Manage Import Rows page. You can find definitions for the fields and controls later on this page.
This page allows you to review one import row at a time. The content and the action buttons displayed on this page depend on the current status of the import row. The toolbar and the Import Row Summary group box are common to all statuses; other sections and action buttons are as shown in the following table.
|
Import Row Status |
Sections Displayed |
Buttons Enabled |
Buttons Disabled |
|---|---|---|---|
|
Suspended |
|
|
Suspend |
|
Add Person |
|
|
Add |
|
Update Person |
|
|
None |
|
Error |
|
|
|
|
Ignore |
|
Reset |
|
|
Posted |
|
None |
|
|
Ready for Processing |
|
Ignore |
|
|
Field or Control |
Definition |
|---|---|
| Toolbar |
The toolbar summary shows the Batch ID, Test ID, Record Number, Import Row Status of the import row and a grayed out Update All Bio-Demo Data check box. This check box will appear as selected only if the Update All Bio-Demo option was selected for the respective batch on the Manage Import Batch page; otherwise, it appears unselected. You can use the Previous and Next links to move backward and forward. The Return to Search button takes the user back to the Manage Import Rows search screen. |
| “Select an existing person to update with import row” grid |
Shows the potential duplicates as returned by the Search/Match API, which is executed when this page is displayed for the Suspended, Add Person, and Update Person statuses. If the Update button is enabled, selecting Update immediately changes the status of the import row to Update Person. The page is redisplayed for the new Update Person status. Note: The address displayed on the duplicates grid for each person is the primary address at the role level (in the following order: Consumer, Worker, Contact, Person). The National ID displayed is the person's Primary NID. |
| Import Row Summary |
Displays a summary of import row information. This section appears for all statuses; action buttons are enabled or disabled depending on the status of the import row. Select an enabled action button to immediately change the status of the import row and redisplay the page for the new status. |
| Search/Match Results |
This collapsible section appears only for the Suspended, Add Person, and Update Person statuses. It displays information returned by the Search/Match API invoked for the import row. The first row of the grid is arbitrarily selected if the row is in Suspended or Add Person status; if the row is in Update Person status, then the previously selected row continues to be selected. |
| Errors |
This grid is displayed only if the import row is in Error status. It shows the complete list of errors for the row from previous import runs. Also it shows the process instance number and the date and time of the run corresponding to each error. |