Challenge
A nationally recognized academic medical center faced a major infrastructure gap as the COVID-19 pandemic surged. Its on-premises data environment—built over more than a decade—had become a tangled web of custom ETL scripts and disparate data processes. These legacy systems were not capable of supporting the scale, speed, or compliance requirements needed for pandemic response.
The institution needed a modern solution that could rapidly integrate protected clinical, research, and public health data while meeting the highest standards of HIPAA and research compliance. Additionally, the solution had to be scalable without increasing technical headcount and robust enough to support coordination with external public health agencies, including city and state COVID-19 task forces.
Solution
Technology Partners assembled a specialized team to architect and deploy a secure, cloud-based data platform on Microsoft Azure. Central to the implementation was the use of Databricks for advanced analytics, enabling rapid data processing and analysis across clinical, research, and public datasets.
Legacy ETL processes were reimagined and migrated to cloud-native pipelines, ensuring scalability and resilience. Enterprise-grade security controls were implemented to protect sensitive health data, while automated ingestion and processing pipelines replaced labor-intensive workflows. The platform was designed to support multi-organizational collaboration, establishing a governance framework that allowed for secure access and shared insights across academic and public health teams.
Over the course of the two-year engagement, the new system evolved into a foundational enterprise data platform, enabling far more than COVID-19 response. The work also led to the development of proprietary automation tools that were later commercialized.
Results
The impact of the project was immediate and far-reaching. The new platform enabled coordinated COVID-19 response across departments and with external partners. Over 30 departments now rely on the platform for daily operations, extending its value into education, finance, research, space management, and clinical services.
This project wasn’t just about data—it was about resilience, rapid collaboration, and future-proofing public health research. We’re proud to have delivered a platform that helped respond to a crisis and continues to drive innovation every day.
Jake Gower
Service Line Manager, Technology Partners
The shift to automated data ingestion eliminated significant manual overhead, allowing internal teams to scale data operations without additional hires. The success of the automation effort also inspired the creation of Databasin—a commercial SaaS product launched in 2024 that now brings similar capabilities to other institutions.
The initiative also earned perfect Microsoft Data & AI partner scores for Technology Partners, highlighting the technical excellence and innovation delivered throughout the engagement.
Key Outcomes
- Supported critical COVID research and response with secure, scalable data architecture
- Migrated a decade of legacy systems to cloud-native infrastructure
- Enabled cross-agency collaboration through centralized governance
- Supported 30+ departments with automated data ingestion and real-time access
- Sparked the creation of Databasin, a commercial SaaS analytics platform
