Data is the bedrock of every business’ operations, and finding efficient ways of organizing and managing it is key to driving business development and aligning team efforts.

In 2021, Beyond Finance took on the unwieldy task of building a brand-new customer relationship management system to optimize and transform the way we collect, store and access client data.

A major cross-functional effort, the project brought together members of the Data, Business Intelligence, Operations, Engineering and Product teams in the name of achieving a few common goals. The unique Beyond approach to problem-solving paved the way for disparate teams to:

  • Build a CRM that met our specific business needs
  • Develop a system for mapping data from our old CRM to a new one
  • Pull off the migration of hundreds of thousands of data points without breaking anything
  • Completely transform our data infrastructure with no impact to our clients

Two Systems, One Org, Lots of Room for Improvement

“To support that growth and support our clients’ needs, it’s imperative that we have a platform that provides flexibility and scalability.“

Lou Antonelli, Chief Operating Officer

Before the data migration was initiated, Beyond Finance’s legacy system had many flaws. While it dependably supported the business in earlier phases, as operations scaled, the need for a more efficient and streamlined system became apparent. 

Matt Pollack, Head of Product at Beyond Finance at the time, noted a drawback of the old system was that it partitioned client data into sales data and debt solutions/receiving data. “The sales system was on a platform called Velocify.” said Matt. “Meanwhile, the post-enrollment system was built on Salesforce, but used a third-party tool for the debt solutions functionality.” While our teams resourcefully operated around this division in data and learned to navigate the two systems, a single, centralized solution would have been ideal.

“If you wanted to do something that the tool didn’t support, you would either need to request that the third party update the tool on their timeframe or change your business process to work with it, as is.”

Matt Pollack, Head of Product

Our use of a third-party tool compounded the issue. While it initially made implementing the system a fast process, it ultimately turned our CRM into a tool to work around, rather than one that conformed to our needs. “If you wanted to do something that the tool didn’t support, you would either need to request that the third party update the tool on their timeframe or change your business process to work with it, as is,” Matt explained.

After years of adeptly working around these challenges, leaders at Beyond evaluated the need for a change and arrived at a decisive course of action. Lou Antonelli, Chief Operating Officer, was a key decision-maker behind the migration initiative. “We’re in a rapid growth phase,” he remarked thoughtfully. “To support that growth and support our clients’ needs, it’s imperative that we have a platform that provides flexibility and scalability.“

“In the end, we made the decision to build Bedrock.” Matt concluded.

Embarking on a Major Data Migration Project

“Getting a good understanding of what the new system needs to operate is key,”

Jay Hakim, Vice President of Data and Business Analytics

Jay Hakim, Vice President of Data and Business Analytics, brought many years of data migration experience to the drafting table, understanding that the risks of a data migration can be serious.

“Migrations of this size are all about the complexity of the data and ensuring the right data is being populated in the target system,” Jay explained. “If something goes wrong, then the customer can’t be serviced in the new system and can’t be serviced in the old system, which is the worst possible scenario.”

A successful data migration requires extensive planning before even a single piece of data is moved, so that’s where the team started. “Getting a good understanding of what the new system needs to operate is key,” Jay noted. Without a deadline in mind, the Data and Analytics team evaluated the landscape of the old and new CRMs as well as the data itself. 

With a sense of the landscape, the team devised a migration strategy that would mitigate risk and hopefully lead to a clean transfer of data. Jay identified the plan as a crawl-walk-run approach, a classic project methodology in which teams evolve a large-scale project slowly, in phases, rather than pulling the trigger all at once. This approach gives collaborators at all levels time to evaluate progress, spot errors, and make adjustments as the team advances through the stages of execution, making it perfect for moving hundreds of thousands of data points without majorly disrupting the business.

From there, the team mobilized to execute the migration.

Laying the Groundwork for Success

“Afterwards, I switched over to the Product team, where I was able to develop an in-depth understanding of Bedrock with my legacy knowledge.”

Mike Chandler, Technical PRODUCT Manager

Data Mapping

Although our team had a lot of planning under their belts, they wouldn’t be ready to start moving data until a thorough data mapping had taken place. 

Mike Chandler, Technical Product Manager, stepped in to take the lead on this essential step in which teams match data fields from a “source” to a “target” database.

Already familiar with client data through his work in Business Analytics, Mike dove even deeper into the journeys of our current and former clients. “A lot of our early work involved digging into the legacy system data to figure out how to get useful business information out of it.” Mike stated. “Afterwards, I switched over to the Product team, where I was able to develop an in-depth understanding of Bedrock with my legacy knowledge.”

Painstakingly, the team mapped hundreds of thousands of complex data points in phases. Mike credited strategic thinking, a lot of SQL coding, a highly organized source-of-truth document and team diligence for making this phase a success. 

Database Testing

The mapping process presented a safeguard against both misrouted and low-quality data. “A big help was the creation of a test environment where we could load our mapped data into Salesforce and have actual users tell us when it looked wrong,” said Mike.

The quality of the data, itself, was identified as a major issue. “From the initial phase of the project, we realized the importance of testing the data being migrated.” Mike continued. “We knew the consequences of data errors could be bad for operations. So, it was not a surprise that we spent a solid chunk of time and effort on testing the data.”

Testing was kicked back to the data team. Sourav Bhowmik, Senior Data Engineer, described the data validation jobs his team created to catch mapping errors.

“Broadly, we did two kinds of data testing for this project: qualitative and quantitative,” he explained. “We built an automated data validation job that matched data points in the source and target systems, verifying the linkages between migrated objects were preserved at target.”

Working with this much data from every part of the client journey required input from contributors from all parts of the business. Mike highlighted the expertise of others as a major contributing factor to the success of the mapping phase. “The key was knowing who was familiar with which parts of the business so I could bring them in,” he noted.

In addition to the Data and Product teams, many other groups contributed to the testing and validation of the client data. “These were members of the CSD, Debt Solutions, and BizOps teams who were combing through our test system to provide feedback for updating the mappings,” said Mike.

Working with this amount and complexity of data required caution at every step of the process. With patience, coordination and a data-driven mindset, the team was able to create the sturdy infrastructure needed for a smooth migration. Through this test-and-learn approach, they made it possible to execute only when there was data-supported certainty that business operations would be safe.

With data mapping and quality tests behind us, we began the migration. 

Crawl, Walk, Run

Ahead of the migration, our Engineering team built an infrastructure for automating the process. Sourav Bhowmik headed off the project, leading the creation of a system that migrated data in a predictable and replicable way.

“The framework had to be accurate and uncomplicated to use. This helped other engineers who were onboarded to the project and me to build on top of it with ease,”  said Sourav.

With an architecture built out for transporting our data, we were finally ready to begin. 

The team chose to use the classic and effective “crawl, walk, run” methodology, starting small and working with other teams to make sure they weren’t breaking anything as they worked their way through the data. 

“Our initial phases were very small, starting with seven customers, and then ramping up to a few hundred,” Jay illustrated.

During the ramp-up phases, the team periodically stopped to identify and eliminate bottlenecks, eventually working up to moving big chunks of data each time. “Eventually in our run phase we were able to migrate 3,000–4,000 customers in a 6-hour window,” Jay concluded.

A Data Management Best Practices and Collaboration

“I think this project is a good example of how Tech, Product and BizOps teams within an organization can efficiently collaborate with each other.”

Sourav Bhowmik, Senior Data Engineer

The data migration project was a massive undertaking with positive impacts across the business, namely increasing efficiency in data management and access without disrupting operations.

“While there were a few hiccups to the migration process,“ said Mike Chandler. “They had a relatively minor client impact and only affected a small number of clients.” By the conclusion of the project, Beyond Finance migrated over 25,000 clients, with the majority of them seeing no impact or disruption at all.

A well-maintained CRM can have a direct impact on business performance, leading to a:

  • 42% increase in report accuracy
  • 57% increase in quality of communication between sales reps
  • 34% increase in productivity
  • 27% increase in client retention
  • 23% reduction in lead cost

The team is looking forward to seeing such benefits play out in our own operations. Meanwhile, project collaborators consider the project a resounding success. 

“I think this project is a good example of how Tech, Product and BizOps teams within an organization can efficiently collaborate with each other.” reflected Sourav. Bringing subject matter experts from across the business together to improve this instrumental business tool was exemplary of Beyond’s culture of empowering experts and collaborating to find the best solutions.

Many also pointed out that Beyond’s results-first and collaborative culture made a complicated process easier than expected. “An important thing I learned about Beyond’s culture,” expressed Mike. “is that we look forward and don’t dwell on mistakes. As long as this process took, we could have taken a lot longer if we wanted to, but we trusted each other not to point fingers, took a plunge, and got it done.”

To learn more about the team and how we work, check out our Careers page.