Preparing for Data Conversion

This chapter provides an overview of data conversion and discusses how to:

Click to jump to parent topicUnderstanding Data Conversion

Conversion of data from an existing system is one of the most challenging and critical tasks that you accomplish while installing the Campus Solutions system. The conversion task takes preparation and planning, a programming effort, and robust testing.

Because of many factors—amount of data to convert, business practices, institutional schedule—much of the decisions as to when, how much, and even the method of converting historical data is ultimately up to you. The more you convert the better the new system becomes, but the more challenging the conversion task is. This section contains some best practice recommendations for converting data into Campus Solutions; however, the information presented here is not exhaustive.

Consider these tasks:

There are as many ways to plan a conversion as there are universities. So it is a unique process based upon the legacy system, your needs, and how you do business. It is suggested that you:

Click to jump to parent topicPerforming Data Conversion

This section provides overviews of the conversion process and referential integrity and discusses how to:

See Also

PeopleTools PeopleBook: Data Management

Click to jump to top of pageClick to jump to parent topicUnderstanding the Conversion Process

Here is a review of the scope of the conversion process, so that you can gain an understanding of the phases involved:

Click to jump to top of pageClick to jump to parent topicUnderstanding Referential Integrity

An important topic to consider and test for as part of the conversion effort is referential integrity. You can convert a lot of data into the correct tables, but the overall system may not have the necessary connections. If you use the normal posting processes in the various applications to convert most of the legacy data, you should not have referential integrity problems because those processes contain many of the necessary referential integrity rules.

An example in Student Financials is third-party contract information. If you link a third party or a transaction to a contract, the contract needs to exist. However, if the conversion neglects to convert contracts, then a referential integrity problem can occur.

Click to jump to top of pageClick to jump to parent topicMapping Data for Conversion

When you map data, you are identifying legacy system data and redefining it in PeopleSoft record definitions. This step can be performed while setting the conversion timeline; however, your data mapping efforts will be more focused if the conversion time line has been set.

Issues to consider when mapping data include defining the amount of historical data to be converted, code validation, workflow, reporting, and other processes. The amount of historical data converted may vary across PeopleSoft applications. The complexity of how the legacy system stores history may contribute to less history mapped into the PeopleSoft system. With code validation, valid codes for the institution may have changed over time, and this too must be considered in the conversion effort.

To map data for conversion:

  1. Define each legacy data element.

  2. Determine if the legacy data element is pertinent to the implementation scope.

  3. Map the legacy data element to the Campus Solutions data element.

  4. If the legacy data element cannot be mapped directly within your Campus Solutions product line, determine if the legacy data should be tracked any more or if it could be tracked in another PeopleSoft product line, such as PeopleSoft Human Resources Management.

  5. Reformat the legacy data to fit, for example, the Campus Solutions field size, format or data type.

    If certain legacy data cannot be reformatted, indicate those as gaps in the gap analysis.

Spreadsheets are helpful tools when mapping data. At a minimum, these spreadsheets should include the legacy system data source (if data comes from more than one source), the legacy system data element and data type, the specific Campus Solutions or Contributor Relations record, and the specific field and data type in that record. Optional description fields might include related prompt tables, conversion translation values, and legacy system data position.

Another option is to build a mini-application using PeopleTools to map data. You can build Structured Query Language (SQL) views to link various PeopleTools tables together, such as PSRECFIELD and PSRECDEFN, which creates a robust method of identifying field attributes. You can create special reports to identify key fields, required fields, or legacy system data not yet mapped.

Click to jump to top of pageClick to jump to parent topicPreparing Data for Conversion

After you have identified the legacy data to be converted, you must prepare it for conversion. For example, when coming from the legacy system, data may be packed in EBCDIC format. These fields should be reformatted into a conversion-ready unpacked flat file in ASCII standard format. Usually, a COBOL program is written and run on the database server to create a file placed on the legacy system.

Conversion-ready data does not have to reside in a single file. The program may be designed to create multiple files during the formatting process, with each file only containing data pertinent to a single or related set of Campus Solutions or Contributor Relations destination tables. Smaller file sizes are more manageable and result in faster conversion processing. Also, the data preparation program itself may be broken into multiple programs based on the destination table.

One of the largest time-consuming tasks of the conversion project can be data cleanup. One way to minimize this task is to clean up the data before importing it into the PeopleSoft system. This can be done by using the data preparation and extract program. This program contains more logic to ensure data validity and therefore takes a longer time to program and test, but the program reduces processing time during the data cleanup.

Another option to reduce the data cleanup effort for the conversion team is to create reports for review by appropriate functional experts, technical experts, employees, and managers. These reports can be used for data verification and correction before the data is entered in the database. Correct invalid legacy data before preparing for conversion.

Not all data conversion must occur through background programs. You may determine that online application data entry is necessary for prompt tables and other required PeopleSoft tables for which there was no legacy data.

Click to jump to top of pageClick to jump to parent topicVerifying Converted Data

After data has been converted to the PeopleSoft system, you must verify the data and check its integrity. Two methods are online verification and batch verification. Data verification must be performed to clean up invalid data that may not have been keyed, required, or validated in the preparation or conversion processes. This process can also identify and clean up duplicates in the legacy system before you finish converting and mapping the data to the new PeopleSoft tables.

You can write batch programs or SQL scripts to check for parent-child relationships and other validation checking. Writing the batch programs requires more time commitment but can quickly check more records.

Online data verification is less exact than the background process, because records are randomly accessed and require a longer amount of time. However, online data entry can be used to enter nonrequired data or supplemental data that was not available from the legacy system.

Click to jump to top of pageClick to jump to parent topicUsing Data Dictionaries

PeopleSoft does not issue a data dictionary as part of its documentation. PeopleTools provides the ability for online documentation of page and field relationships. The development team documents its material as much as is feasible online in PeopleTools. You can write an SQR that provides all the data dictionary information that you need.

Click to jump to top of pageClick to jump to parent topicUsing Data Load Programs

After data preparation, design conversion programs that extract the appropriate file layout, to transfer the legacy data to Campus Solutions tables.

Input Format

Often the input format for the conversion program is the flat file created by the data extract program on the legacy system and transferred—by using File Transfer Protocol—to the database server. Consider loading the data into temporary tables within the PeopleSoft system instead of directly into the "live" tables. By performing this task, you can use the tools already available to you to both clean up and properly convert data. The temporary tables would then be the input for the conversion program.

Conversion Program and Program Language

The conversion program transfers the legacy data from the input format to Campus Solutions tables, including reformatting, validating, and prompt table translating not encompassed in the data preparation programs. The conversion program must include defaults for Campus Solutions record keys and required fields whenever legacy data has no data map.

The two most commonly used languages for conversions are COBOL and SQR. Another option is a specific database management system (DBMS) import utility, if one exists—such as the Oracle SQL Loader. Determinants for making this decision include your skill set. For example, if you already know COBOL, there is no associated learning curve downtime, or increased resources if you use COBOL. Regardless of your skill set, consider if it would be advantageous to learn a language that is likely be used in the future for Campus Solutions modifications, conversion file size, level of translations and edits, and DBMS capability.

COBOL

Has a more robust debugger, can process a larger number of records faster, and is more widely recognized in the industry.

SQR

Is more commonly used in reporting and minor modifications. In most cases, SQR code can be written in less time than COBOL code.

DBMS Import Utility

Requires the least conversion effort but is limited to no application-specific edits or translations.

Table Loading Sequence

Conversion programming may include a single program that incorporates all pertinent tables and their sequence or multiple programs that incorporate single tables or a few related tables. Multiple programs can run simultaneously—increasing efficiency and enabling the allocation of programming responsibility.

Errors and Exception Handling

Regardless of the number of conversion programs, there must be adequate and standard error and exception handling. Reports of database action failures, validation logic failures, and pertinent details help ensure a quality and thorough conversion.

Totals Controls

In addition to errors and exceptions, totals controls also help in quality control. Totals controls include the number of records processed on both the source and destination sides, and dollar tallies whenever possible are pertinent for Financial Aid and Student Financials applications.

Required Fields

You can find out what fields are required for any table or page by searching PeopleSoft Application Designer in PeopleTools. You can also write an SQR to list all required data elements from tables that you want to use in the conversion. It is important that you also read and understand the PeopleCode. There are PeopleCode restrictions within records that can mean that in certain circumstances a field is required or a default value is required. Without understanding the PeopleCode, incorrect data could be entered.

Warning! If a field is not required, do not assume that you do not need to convert data into that field. It is recommended that when in doubt, you review the PeopleCode to determine whether you need data or default values for a particular field.

Note. Refer to individual discussions regarding each application in this chapter for specific data conversion and data loading tips for applications within Campus Solutions.

Click to jump to top of pageClick to jump to parent topicKeeping Systems in Sync

Depending on the implementation plan, there are three scenarios for moving data processing from a legacy system to a PeopleSoft system. The first scenario is to cut over everything at once. When all students are fully implemented, you perform a cutover to the PeopleSoft system. Until that point, the legacy system is fully operational. No system synchronization is required.

The second scenario is the mutually exclusive implementation approach. For example, when a student is converted to an application in Campus Solutions, such as Financial Aid, the student is officially taken off of the legacy system's processing in that application. All of the student's other application processing—such as Student Records—is performed on the legacy system. There are two production systems. This approach increases the complexity of joining two data sources for interfaces and reports.

The third scenario is the parallel system approach. An alternative to having two production systems is to test each Campus Solutions application as necessary, but completely delete the application database and rerun a full conversion with the most up-to-date data from the legacy system until full cutover is achieved after full testing. This approach implies that the legacy system is in full use until cutover, but no new programs must be written—for example, the conversion program is reused and all reports continue to be produced from the legacy system. A variance of this approach is to write new data refresh programs that periodically update the various applications with student information entered in the legacy system. This variance provides for more up-to-date information for more accurate reporting and testing, but also implies the additional effort to program the refresh routine. Care must be taken in the refresh routine design to ensure that data in the PeopleSoft system is not duplicated or overlaid by the legacy data.

Click to jump to top of pageClick to jump to parent topicEstimating Disk Usage Space

For Campus Solutions, PeopleSoft has done some preliminary research on the sizing of a production system. PeopleSoft realizes that the theoretical amount of disk space required by an application is different than the actual amount of disk space required by a database. The difference is due to comparing bytes with database blocks. For instance, if a particular table's average row length is 25 bytes and there are 1,000 rows in the table, theoretically the required amount of space is 25,000. However, if the database block size is 4K (4096), the number of database blocks required to hold the data is seven blocks or 28,673. Therefore, the actual amount of disk space required is greater than the theoretical. With this in mind and realizing that each installation may require a different block size, PeopleSoft states the size in theoretical terms.

For 1,000 student prospects, the amount of space needed for tables and indexes would be 40 megabytes (MB).

For enrollment, PeopleSoft needs to work with a specific, albeit theoretical, scenario. PeopleSoft's scenario was to enroll 1,000 undergraduate students over three terms into four classes per term. In the first term, there are no prerequisites to enroll in any of classes; the second term has one prerequisite for each of the classes; the third term has two prerequisites for each of the classes. During enrollment, the students have no time conflicts and there is no lack of facilities (rooms or seats). The amount of disk space excludes description information stored in long text fields. For 1,000 student enrollments, the amount of space needed for tables and indexes would be 20 MB.

Click to jump to parent topicConverting Recruiting and Admissions and Campus Community Data

This section provides an overview of conversion of recruiting and admissions and campus community data and discusses how to populate tables for recruiting and admissions and campus community data.

Click to jump to top of pageClick to jump to parent topicUnderstanding Conversion of Recruiting and Admissions and Campus Community Data

You should convert recruiting and admissions data first along with relevant Campus Community data such as personal data—names, addresses, or phone numbers.

It is up to you how much of this data you convert—for instance, how many addresses you convert for one individual—and how much history you take into consideration. Of course, if the student records team is converting 10 years of student records, make sure that there are 10 years of personal data elements in the system.

You should convert up to two years of recruiting and admissions data for your institution. Again, this may vary depending upon your practices and your needs. Regardless of how much history you do convert, it is important that you keep the prospect stacks of data and the Recruiting and Admissions stacks of data consistent. You do not want to end up with a year of recruiting and admissions data without the prospect data for the same year. This also ensures that the data required for institutional research reporting—for instance, cohort reporting and statistical reporting—remains consistent.

Click to jump to top of pageClick to jump to parent topicPopulating Tables for Recruiting and Admissions and Campus Community Data

Aside from address and other information, there are some tables that are precursors to all application conversions in Campus Solutions. These tables represent the minimum to be converted.

Academic Organization Tables

Populate these tables in this order for academic organizations:

Level and Load and Term and Session Tables

Populate these tables in this order for level and load and term and sessions:

In addition to these tables, you must convert these tables as early in the process as possible in the sequence. Use the name and address load routines delivered with the system, to load data:

Click to jump to parent topicConverting Financial Aid Data

This section provides an overview of conversion of financial aid data and discusses how to populate tables for financial aid.

Click to jump to top of pageClick to jump to parent topicUnderstanding Conversion of Financial Aid Data

This section lists prerequisites and discusses conversion of financial aid data.

Prerequisites

Before you can convert financial aid data, personal data for each financial aid record must already be converted. In addition, financial aid data should be converted prior to student financials data, because posting of financial aid in Student Financials is dependent upon the disbursements in Financial Aid.

You must define item types before you begin conversion of financial aid data. You must decide whether financial aid or student financials has ownership of the PeopleSoft Item Types table.

Conversion of Financial Aid Data

You should convert up to two years of financial aid data for schools, and as much aggregate data as possible, because it has lifetime loan limit information. Ultimately, it is up to you how much data you convert. You may decide to keep the legacy system up until fall of the conversion year to close out Pell Payment and FISAP reporting.

Note. Please refer to the Department of Education's requirements for maintaining financial aid records to assist you in deciding how much legacy data to convert to the PeopleSoft system.

See Also

Converting Student Financials Data

Click to jump to top of pageClick to jump to parent topicPopulating Tables for Financial Aid

At a minimum for Financial Aid, convert the aggregate data for these tables:

You may want to convert additional data depending on what you have in the current system:

Click to jump to parent topicConverting Student Records Data

You enter data in Student Records tables by academic program. How much history data you convert is up to you.

This section lists prerequisites and discusses how to:

See Also

Converting Recruiting and Admissions and Campus Community Data

Click to jump to top of pageClick to jump to parent topicPrerequisites

Some of the tables, as noted in the Converting Recruiting and Admissions and Campus Community Data section, must be loaded before you load the student records data.

Click to jump to top of pageClick to jump to parent topicPopulating Tables for Student Records

At a minimum for Student Records, set up these tables in the sequence they are listed.

For the course catalog and schedule of classes, there is PeopleCode behind the COURSE_ID field to increment a new course ID by 1. To mimic what this PeopleCode does, the conversion program needs to do the +1 logic. You can manually set the next available course ID in the installation table for Campus Solutions. You should update the last course ID assigned in the Installation table, so that you do not have collisions between the course IDs assigned by the conversion and course IDs manually created later.

In addition to the conversion program you write to create the catalog, you should set the Status field on the Catalog Data page to Active so that the course is active as soon as you run the conversion program to create the record.

Course Catalog Tables

Populate these tables for the course catalog:

Schedule of Classes Tables

Populate these tables for the schedule of classes:

Student Careers and Plans

Populate these tables for student careers and plans:

Student Degrees

Populate these tables for student degrees:

Grades

Populate these tables manually for grades:

Student Term Data

Populate these tables for student term data:

See Also

Selecting General Installation Options

Setting Up the Course Catalog

Click to jump to top of pageClick to jump to parent topicConverting Enrollment Data

There are three options to select for converting student enrollment data. Note that when you convert enrollment data you must create enrollment request input transactions and engage the enrollment COBOL engine to actually post the enrollments. This is the only way that the statistics on the STDNT_CAR_TERM table are automatically calculated.

Options for converting enrollment data are:

  1. Convert the data as normal student enrollment data.

    This is the recommended option. The course catalog and schedule of classes must be converted first, and you must have data for these as far back as you want to have enrollment data. The conversion program must create class association groups for the offerings when you convert the schedule of classes. In addition, a student career term must be present for every past term that you intend to convert. For each class section that you convert historically, you need an entry in the CLASS_TBL. This does not mean that you must convert all of the class sections, just the sections that are graded. Even for graded sections the amount of data needed is minimal. For example, you may choose not to convert meeting times and instructors for classes. You may want to have the conversion programs roll all enrollments for a course in a term into one section. For example, if English 101 had 51 sections in fall 1996, you could create a Section 1 for that term and convert all 51 sections into that Section 1.

  2. Convert enrollment data as internal transfer credit or other credit.

    This option is necessary if you have only your course catalog but not your class schedule for as far back as you want to convert data. This method can make transcript printing a challenge, because past credits prior to conversion appear as transfer credit. Having split transcripts may also be an option, if issuing two transcripts for a student is acceptable—one from the old system for prior work and one from the new Campus Solutions system for all work after you bring the system up. In this case, you would still want to convert the enrollment data by summary transfer credit, so that the prior system's term academic statistics roll into Campus Solutions.

  3. Use the Historical Course Enrollment page (Manage Student Records, Manage Academic Records, Use, Historical Course Enrollment) and convert statistics in summary by using the transfer credit engine.

    If you cannot go through the effort to convert prior enrollment history, or the data is not available to you electronically to reconstruct prior course catalog and schedule of classes data, use the Historical Course Enrollment page. Create an SQR or other program to directly load enrollment data onto the Historical Course Enrollment page, and manually clean up this data. If you select this option, convert summary statistics by using the transfer credit COBOL process.

See Also

Creating Historical Enrollment Records

Click to jump to parent topicConverting Student Financials Data

This section lists prerequisites and discusses how to populate tables for student financials.

Click to jump to top of pageClick to jump to parent topicPrerequisites

Student Financials receives data from many of the other Campus Solutions applications. For this reason, you must set up parts of other applications before you can convert the student financials information.

You must convert personal data tables and external organizations data before you convert student financials data.

See Also

Converting Recruiting and Admissions and Campus Community Data

Click to jump to top of pageClick to jump to parent topicPopulating Tables for Student Financials

This list presents information about populating the tables related to Student Financials:

Click to jump to parent topicConverting Contributor Relations Data

This section lists prerequisites and discusses how to:

Click to jump to top of pageClick to jump to parent topicPrerequisites

Contributor Relations uses data from some of the other Campus Solutions applications. For this reason, you must set up Student Financials item types and related general ledger information as well as Human Resources departments before you can convert some of the contributor relations information.

Click to jump to top of pageClick to jump to parent topicPopulating Tables for Contributor Relations

At a minimum for Contributor Relations, set up these tables.

Organization Structure Tables

Populate these tables for organization structure:

Campus Community and Constituent Tables

Populate these tables for Campus Community and constituent data:

Note. The EXT_ORG_ID field should be left blank for PeopleSoft data. This field is used to store data for external organizations. When dealing with an organization, you would leave EMPLID blank and load the EXT_ORG_ID for the organization.

Gift Tables

Populate these tables for gifts:

Pledge Tables

Populate these tables for pledge data:

Note. For converted active pledges to have pledge payments made against them, the AV_SESSION_TBL.SESS_STATUS field for the pledge session must be set to P.

Click to jump to top of pageClick to jump to parent topicCreating Accumulated Records for Gift and Pledge Inquiry

In Contributor Relations, the system runs Giving Profile and Commitment Summary online reports based on summary accumulator records rather than detailed transaction records. For the proper information to appear on these reports, you must initialize the summary accumulator records for all constituents and all gifts and pledges in the system. Contributor Relations delivers a Constituent Accumulator Initialization Application Engine process (AV_ACC) to perform this task. To access the process, select Set Up SACR, Product Related, Contributor Relations, Install Contributor Relations, Initialize CR, Accumulator Initialization. When initialized, incremental accumulations take place for affected constituents during the Constituent Accumulator process. You can run this process either by itself or along with the GL Interface job.

The Constituent Accumulator Initialization process is a resource-intensive process that requires an extended background processing window. The process deletes and recalculates all data from the accumulator records (PS_AV_CNST_ACC, PS_AV_CNST_ACC_DES, and PS_AV_CNST_ACC_FY). If all transactions contained within in-process sessions are first completed (their session status changes to Posted), do not select the Search for unposted Pledge Payments and Search for unposted Matching Gifts options. This improves the performance of the initialization process.

Note. If you do not complete the initialization process, the system does not complete the information displayed online on the Giving Profile and Commitment Summary pages.

If additional groups of constituents are required to allow the process to complete within the particular background processing window, the records in the PS_AV_ACCUM_INIT table can be modified to include more, smaller groups. Remember, however, that additional occurrences of the temporary tables used by the AV_ACC Application Engine process are required if you run more than the delivered five processes at any one time.

See Also

Setting User Defaults

PeopleTools PeopleBook: PeopleSoft Application Engine