 |
| Danny Rivera / flickr |
An organization is only as effective as the quality and accessibility of information needed for its operations. Information is as critical to success as the organization’s personnel and equipment.
This is why data migrations are tasks not to be taken lightly.
The data migrations process involves the transfer of information from a computer system by means of extracting data from a source database, correcting errors, reformatting and loading it into a new software solution.
To prevent problems and ensure efficiency, moving data must be done with thought and planning. This is especially true for organizations involved in right of way. Due to the critical importance of accurate, easily accessible information, data migrations should be a centerpiece for discussions and planning surrounding technology upgrades.
The technology company geoAMPS has performed successful data migrations for a number of its clients.
Think about how the information collected within your right of way activities influences your daily business. Those paper and electronic documents are used by field agents, the accounting office, the planning committee and several other departments, making your company’s information one of its most valuable assets.
Historically, data migrations have been lengthy and costly. Understandably, company representatives want assurance that the process will be completed seamlessly and without damage to the original data. While such concerns are justified, the need to complete data migrations should not create a roadblock for an organization that wants to upgrade its technology. With new techniques and proper planning, the efficient transfer of information can be accomplished in record time, providing right of way organizations the opportunity to improve efficiency and their bottomline by utilizing current technology.
Data Migration Challenges
Data migrations are not without challenges. For example, variations in data entered in an uncontrolled environment, such as a spreadsheet, result in the company’s project database being inconsistent. Also, information typically is stored in different file formats, ranging from scanned documents, PDF files, Word documents, to spreadsheets. These files oftentimes are housed on multiple computers or subject to closely guarded proprietary techniques. Organizations might desire a central repository for this information, as well as consistency in how the information is presented, but still prefer to retain the original format in which there already had been substantial investment.
Issues and challenges associated with data migrations vary from one organization to another. The right technology can identify these issues within the organization’s database and make the necessary modifications so that the data is stored centrally and consistently. Here are four examples of the types of challenges frequently associated with data migrations:
Example 1 – In the process of identifying and acquiring right of way, the organization may run into multiple tracts titled to the same individual. Maintaining separate records for each tract can result in duplication of the data if appropriate precautions are not taken during the migration of this data. Consider the following instance:
|
Tract # 1
|
Tract # 5
|
Tract # 7
|
Comments/Scenario
|
|
John Doe
|
John Doe
|
John Doe
|
There is no way to know if the three John Does are the same person
|
|
John Doe
2522 Johnson Street
Houston, Texas
|
John Doe
2522 Johnson Street
Houston, Texas
|
John Doe
2522 Johnson St
Houston TX
|
It is easy to see that the first two John Does are the same person, but the third one may pass the screener if the matching logic is not advanced to understand that Street and St as well as Texas and TX are one and the same.
|
The right technology can cleanse the information to ensure accuracy and consistency and reduce the need for manual intervention and scrubbing. A more advanced approach can produce a report to identify potential duplications. In this example, the data migration has increased project efficiency. Now, right of way agents and project planners do not have to search multiple files for information on tracts owned by the same individual.
Example 2 – There can be variation of data assigning property ownership. Consider, for instance, one tract which is owned by three individuals, each owning a different percentage of the whole. In one spreadsheet for one project, ownership is displayed as Owner A – 15, Owner B – 25, Owner C - 60, adding up to 100. A second spreadsheet for a separate project displays ownership for tracts as Owner A - .15, Owner B - .25, Owner C - .60, adding up to 1. Such variability in data entry is easy to occur when the users and contractors are entering data in an uncontrolled environment (such as a spreadsheet) and clear guidelines have not been provided for how the data needs to be entered. Although each data set is accurate, migrating the data from the two spreadsheets into a single system may result in inaccuracies depending on how the ownership is interpreted in the target system. A data migration can extract from the two spreadsheets and consistently assign ownership of the tract by percentages, Owner A – 15 percent, Owner B – 25 percent, Owner C – 60 percent, adding up to 100 percent, or vice versa.
Example 3 – Data on the same tracts stored in different formats can be condensed into one file in a single format. For instance, the organization could have a spreadsheet with information on multiple tracts, and also have file-sharing on its network for associated documents for those tracts. Oftentimes, organizations will use a file-naming convention or folder structure to associate the scanned/electronic documents with the tabular/tract data. With proper planning, this data can be brought together. Through data migration, the information is extracted from the two sources and associations are created as data is loaded into the new system. Done correctly, this can streamline the process of accessing all information (tabular and scanned) for your projects and tracts.
However, small oversights can result in missing documents, which can cause chaos and leave bad tastes and a sense of distrust in the new systems for the end users. In this type of migration, extra care must be taken for data validation and appropriate treatment of orphan documents. Orphan documents are ones that cannot be associated with the appropriate projects or tracts because the convention used may not work (typically because of a typo of some sort). In these cases, the information can be manually adjudicated, but building appropriate reports can ease the process of addressing these exceptions.
Example 4 – There might be variations in the data in the current system. Consider, for instance, the different ways telephone numbers are input. The same telephone number can be input in a variety of ways – (614) 555-1212, 614-555-1212, 614.555.1212, 614 555 1212 or 6145551212. A data migration can convert all these variations into a standard format – 614-555-1212 – and ensure consistency in the data.
Success Stories
Despite a myriad of issues and challenges, data migrations can be completed successfully and enhance the integrity of an organization’s database. Here are two real-world examples of data migrations and how the company benefited:
Example 1 - A company wanted to improve the efficiency of its right of way project through a data migration. Its existing files comprised a series of spreadsheets providing information on 14 projects and 5,697 tracts. This information was extracted from 14,575 scanned documents. The organization’s existing spreadsheet system was difficult to use and lacked technical support. The information, such as parcel owner name, corporation, address and telephone number, was inconsistently stated in the various fields and columns of the spreadsheet. The migration process successfully integrated the data into a consistent format which was easy to use. Configuring the appropriate computer software, doing the necessary quality checks and providing the necessary technical support were critical to the successful migration. The data migration was planned over a period of two weeks and executed overnight without issues.
Example 2 – A second case study posed a more complex challenge. This company sought migration of data for 1,148 projects. These projects include 49,633 tracts and 523,879 documents. Not only did this organization want the efficiency of having information stored centrally and presented consistently, it also wanted the information reconfigured back to the original format in which it had invested time and resources. So, as the migration progressed, the software reconfigured the data in real time to preserve the original format while achieving centralization and consistency. The business continues to use the system, which has been a key part of its organizational strategy. The data migration was planned over a period of one month and executed overnight without issues.
Common misconceptions are that information presented in a variety of forms and files cannot be migrated without serious disruption to the company’s workflow, or that the information cannot be retained in its preferable, original format. However, with proper planning and leveraging the right set of technology, right of way organizations can reach effective solutions to address these concerns. With sufficient pre-planning, the switch can be thrown and the migration successfully completed - literally overnight.
Developing a Plan of Action
A data migration should dedicate the time necessary to specify, build and rigorously test a data migration solution. Choosing a provider that starts planning for a data migration early and has an established approach will make a technology upgrade go quickly and efficiently. By addressing the key issues and identifying the pain points, solutions for adopting change and new technology can be reached in a cost-effective and time-sensitive approach.
Data migrations require close collaboration between business and technology teams. There is no one-size-fits-all approach. While each data migration can be somewhat unique, organizations can use a general framework to plan a successful data migration.
Figure 1 shows the various stages of a successful data migration. While the end result is of critical importance to the organization, the data migration itself can proceed on a flexible schedule to deliver a custom-tailored solution:
Example 2 – There can be variation of data assigning property ownership. Consider, for instance, one tract which is owned by three individuals, each owning a different percentage of the whole. In one spreadsheet for one project, ownership is displayed as Owner A – 15, Owner B – 25, Owner C - 60, adding up to 100. A second spreadsheet for a separate project displays ownership for tracts as Owner A - .15, Owner B - .25, Owner C - .60, adding up to 1. Such variability in data entry is easy to occur when the users and contractors are entering data in an uncontrolled environment (such as a spreadsheet) and clear guidelines have not been provided for how the data needs to be entered. Although each data set is accurate, migrating the data from the two spreadsheets into a single system may result in inaccuracies depending on how the ownership is interpreted in the target system. A data migration can extract from the two spreadsheets and consistently assign ownership of the tract by percentages, Owner A – 15 percent, Owner B – 25 percent, Owner C – 60 percent, adding up to 100 percent, or vice versa.
Example 3 – Data on the same tracts stored in different formats can be condensed into one file in a single format. For instance, the organization could have a spreadsheet with information on multiple tracts, and also have file-sharing on its network for associated documents for those tracts. Oftentimes, organizations will use a file-naming convention or folder structure to associate the scanned/electronic documents with the tabular/tract data. With proper planning, this data can be brought together. Through data migration, the information is extracted from the two sources and associations are created as data is loaded into the new system. Done correctly, this can streamline the process of accessing all information (tabular and scanned) for your projects and tracts.
However, small oversights can result in missing documents, which can cause chaos and leave bad tastes and a sense of distrust in the new systems for the end users. In this type of migration, extra care must be taken for data validation and appropriate treatment of orphan documents. Orphan documents are ones that cannot be associated with the appropriate projects or tracts because the convention used may not work (typically because of a typo of some sort). In these cases, the information can be manually adjudicated, but building appropriate reports can ease the process of addressing these exceptions.
Example 4 – There might be variations in the data in the current system. Consider, for instance, the different ways telephone numbers are input. The same telephone number can be input in a variety of ways – (614) 555-1212, 614-555-1212, 614.555.1212, 614 555 1212 or 6145551212. A data migration can convert all these variations into a standard format – 614-555-1212 – and ensure consistency in the data.
Success Stories
Despite a myriad of issues and challenges, data migrations can be completed successfully and enhance the integrity of an organization’s database. Here are two real-world examples of data migrations and how the company benefited:
Example 1 - A company wanted to improve the efficiency of its right of way project through a data migration. Its existing files comprised a series of spreadsheets providing information on 14 projects and 5,697 tracts. This information was extracted from 14,575 scanned documents. The organization’s existing spreadsheet system was difficult to use and lacked technical support. The information, such as parcel owner name, corporation, address and telephone number, was inconsistently stated in the various fields and columns of the spreadsheet. The migration process successfully integrated the data into a consistent format which was easy to use. Configuring the appropriate computer software, doing the necessary quality checks and providing the necessary technical support were critical to the successful migration. The data migration was planned over a period of two weeks and executed overnight without issues.
Example 2 – A second case study posed a more complex challenge. This company sought migration of data for 1,148 projects. These projects include 49,633 tracts and 523,879 documents. Not only did this organization want the efficiency of having information stored centrally and presented consistently, it also wanted the information reconfigured back to the original format in which it had invested time and resources. So, as the migration progressed, the software reconfigured the data in real time to preserve the original format while achieving centralization and consistency. The business continues to use the system, which has been a key part of its organizational strategy. The data migration was planned over a period of one month and executed overnight without issues.
Common misconceptions are that information presented in a variety of forms and files cannot be migrated without serious disruption to the company’s workflow, or that the information cannot be retained in its preferable, original format. However, with proper planning and leveraging the right set of technology, right of way organizations can reach effective solutions to address these concerns. With sufficient pre-planning, the switch can be thrown and the migration successfully completed - literally overnight.
Developing a Plan of Action
A data migration should dedicate the time necessary to specify, build and rigorously test a data migration solution. Choosing a provider that starts planning for a data migration early and has an established approach will make a technology upgrade go quickly and efficiently. By addressing the key issues and identifying the pain points, solutions for adopting change and new technology can be reached in a cost-effective and time-sensitive approach.
Data migrations require close collaboration between business and technology teams. There is no one-size-fits-all approach. While each data migration can be somewhat unique, organizations can use a general framework to plan a successful data migration.
Figure 1 shows the various stages of a successful data migration. While the end result is of critical importance to the organization, the data migration itself can proceed on a flexible schedule to deliver a custom-tailored solution:
- Identify the current storage environment, with known issues.
- Create a data migration plan and timeline, including developing success criteria which is important to identify when you are done.
- Develop and design requirements for moving the data.
- Create the migration architecture for the data migration, addressing any security concerns surrounding protecting the data during the migration process.
- Develop and test extensively, ensuring accuracy and reliability.
- Implement the data migration through a pilot phase with an eventual roll-out into a complete migration.
Figure 2 outlines the estimated timeframes for base implementation, configuration and client training in preparation of successful data migrations, followed by the migration of data in various formats, such as PDFs and paper documents.
Even if your current database is in paper files, PDFs, TIFFs, etc., a solution exists to turn this information into a live, workable database that your agents in the field or in the office can access quickly and, thus, more efficiently. For example, by leveraging Optical Character Recognition (OCR) and Intelligent Character Recognition (ICR), documents (hand-written and scanned) can be quickly migrated into an electronic database.
Use Technology Professionals
Staying up to date with efficient, user-friendly technology just makes good business sense. Information from right of way agents can be entered into a database faster, delivered with less hassle to the next person in a workflow, and projects completed with greater financial return.
Keeping your organization current with the most advanced technology means you’re going to encounter a data migration sooner or later. As we have discussed here, not doing a data migration can leave an organization with information wrought with errors and inconsistencies, leading to decreased productivity. At the same time, data migrations are challenging and can lead to problems if not completed properly. In fact, with the different types of software currently available, moving from one format to another can damage information if the data migration is not completed by competent individuals who are familiar with the needs of right of way professionals.
When your right of way organization decides to implement a new technology solution that is both user-friendly and easy to manage, ask questions, do the research and learn as much as you can about the plan your technology provider is creating.
There is no reason to fear data migrations and technology upgrades. They are simply a part of staying competitive in today’s highly competitive business world.
For more information about geoAMPS and its products and services, or to request a demo, visit www.geoamps.com.
As President and Chief Executive Officer of geoAMPS, LLC, Yogesh Khandelwal is an engineer with a diverse background in technology across various industries. With more than 15 years’ experience in database customization and implementation, his organization helps energy companies implement standardized processes, bringing efficiency and optimization within project and asset management.
Even if your current database is in paper files, PDFs, TIFFs, etc., a solution exists to turn this information into a live, workable database that your agents in the field or in the office can access quickly and, thus, more efficiently. For example, by leveraging Optical Character Recognition (OCR) and Intelligent Character Recognition (ICR), documents (hand-written and scanned) can be quickly migrated into an electronic database.
Use Technology Professionals
Staying up to date with efficient, user-friendly technology just makes good business sense. Information from right of way agents can be entered into a database faster, delivered with less hassle to the next person in a workflow, and projects completed with greater financial return.
Keeping your organization current with the most advanced technology means you’re going to encounter a data migration sooner or later. As we have discussed here, not doing a data migration can leave an organization with information wrought with errors and inconsistencies, leading to decreased productivity. At the same time, data migrations are challenging and can lead to problems if not completed properly. In fact, with the different types of software currently available, moving from one format to another can damage information if the data migration is not completed by competent individuals who are familiar with the needs of right of way professionals.
When your right of way organization decides to implement a new technology solution that is both user-friendly and easy to manage, ask questions, do the research and learn as much as you can about the plan your technology provider is creating.
There is no reason to fear data migrations and technology upgrades. They are simply a part of staying competitive in today’s highly competitive business world.
For more information about geoAMPS and its products and services, or to request a demo, visit www.geoamps.com.
As President and Chief Executive Officer of geoAMPS, LLC, Yogesh Khandelwal is an engineer with a diverse background in technology across various industries. With more than 15 years’ experience in database customization and implementation, his organization helps energy companies implement standardized processes, bringing efficiency and optimization within project and asset management.