Buy or Build? Unlocking the Power of Data Warehouses in Higher Education

A man works at a laptop with data behind him

November 9, 2023

Universities have the critical role of not only safely collecting student and program data, but also ensuring its accessibility. Post-secondary institutions produce an astonishing volume of data – however, the risk of data becoming lost or compartmentalized among various departments and individuals poses a significant challenge. This is where the concept of “data warehouses” comes in. While currently a popular term, data warehouses represent a well-established technical solution that has been in practical use for many years.

This week on the Holistic Success Show we welcomed Adam Tarnowsky, Senior Product Manager of Analytics at Acuity Insights, who shared the importance of data warehouses in higher education. Adam discussed the differences between building your own data warehouse solution or buying a commercial option. 

So, if you’re interested in how to turn your institution’s disparate data into a unified source, read on!

What is a data warehouse?

The primary purpose of a data warehouse is to consolidate data from various sources within an organization, providing a unified view. This enables a comprehensive understanding of all captured data, making it easier to leverage its full potential. Data warehouses address common issues faced by institutions and companies, such as the need to gather data from different departments and systems, streamlining the reporting process.

What are some common challenges when implementing educational data warehouses?

In the context of an educational data warehouse and dashboarding system, the primary aim is to address various challenges faced by educational institutions. These challenges vary depending on the department within the school.

At a high level, the system helps provide a comprehensive overview of data for decision-makers, often initiated by the Dean of the school. 

This includes key performance indicators (KPIs) displayed on dashboards for daily monitoring. The system tackles issues such as breaking down data silos. Data silos are when the various data a company accumulates is ‘siloed’ off into various departments, systems or even users. This makes it difficult for the right people to access and use the data for informed decision-making which is essential for continuous quality improvement processes and identifying early indicators of students who may be struggling academically.

How can data warehouses support accreditation?

Data warehousing proves beneficial in the context of accreditation, where institutions need to provide evidence of compliance with standards. Data warehousing allows for more frequent data collection and easier data management to meet accreditation requirements, reducing the burden on institutions.

What are the challenges and costs of building your own data warehouse?

Building and maintaining data warehouses are complex and resource-intensive endeavors, often requiring substantial budgets and IT capabilities. Educational institutions and organizations that prioritize being data-driven may attempt to address this complexity by developing their data warehouses. Key points to consider in this context are:

  • Complexity and data cleaning: Data warehouses involve pulling data from various systems and require normalization, cleaning, and transforming data to ensure it’s accurate and consistent. Matching identifiers from different systems can be challenging.
  • Alignment and prioritization: Understanding the needs of different departments within the organization and aligning their requirements can be time-consuming. Prioritizing these needs and ensuring that the data warehouse addresses them effectively is crucial.
  • Integration challenges: Integrating with systems can be difficult, especially when some systems lack APIs for easy data access. This complicates the process of extracting data from these systems.
  • Licensing and maintenance costs: Consider the cost of licensing software, such as business intelligence tools for dashboards and visualizations. Budgeting for ongoing maintenance, expansion, and providing support for users is essential.
  • Knowledge transfer: Ensuring that critical knowledge about the data warehouse isn’t lost when the individuals who built it leave the organization is crucial to avoiding costly disruptions.
  • Underestimating time and costs: One of the significant hidden costs is underestimating the time it takes to build and maintain a data warehouse. IT projects, on average, tend to take more time and cost significantly more than initially budgeted.
  • Budget contingency: Organizations should consider budgeting for contingencies, given the likelihood of projects going over budget and taking longer to complete.

While data warehouses offer valuable insights, they come with complexities and costs that require careful planning and resource allocation to be successful.

Should you build or buy your data warehouse?

The most apparent advantage of using a vendor-provided solution, as opposed to developing your own data warehouse, is the immediate return on investment. You can have a functional system within months, significantly faster than the years it might take to build one in-house. This also reduces the risk associated with budget overruns and unknown costs.

While a vendor solution may not fulfill 100% of your organization’s requirements, it typically covers a substantial portion (around 60-80%). The cost savings and the ability to quickly become data-informed make it a compelling option, allowing organizations to address various issues, such as continuous quality improvement and identifying students in need of assistance.

Recommendations for schools: It’s key that you thoroughly research and be transparent when budgeting for a data warehouse project. Consider the following factors:

  • Comprehensive budgeting: Ensure that you have accurately accounted for all necessary features and costs. This includes initial development, ongoing maintenance, and support.
  • Preliminary assessment: Conduct a preliminary assessment to understand the specific needs of each department in your organization. This involves defining the requirements and aligning them with your budget.
  • Stakeholder support: Identify and secure support from stakeholders who control data silos, as breaking down these silos may face resistance.
  • Talent and resources: Assess whether you have the talent and resources needed to build the data warehouse and create dashboards and visualizations. Skilled individuals in this field are in high demand and can be challenging to find.
  • Realistic budget: Develop a realistic budget that considers the costs and the timeline associated with building your own data warehouse versus buying a turnkey solution from a vendor.
  • Organizational goals: Determine whether your institution aims to be a leader in data warehousing and technology trends or if the priority is data literacy and effective data utilization. The choice should align with your institution’s strategic goals.

Careful consideration of these factors is essential for making an informed decision on whether to build or buy a data warehouse, based on your institution’s specific needs and aspirations.

How does Acuity Insights help schools find the right solution?

Acuity provides comprehensive support to their partners throughout the implementation process, recognizing that a successful project depends not only on the technology but also on the support and understanding of data management. They work closely with their partners, guiding them through the steps, educating them on best practices, and assisting with data cleansing and structuring. This approach resembles a light version of data governance, ensuring that the data is well-prepared and useful within the data warehouse, ultimately helping their partners get the most value from their data.

Watch the full Holistic Success Show episode!

Looking for more information?