City of Denver: Forecasting the long-term impact of COVID-19

What now?

We’re all facing the same question: How, when so much is uncertain and changing from week to week, can we plan for the future? Cities worldwide are coming to grips with lost revenue, event cancellations, budget shortfalls, and job losses due to COVID-19—not to mention radical changes in the everyday lives of their citizens. Denver is no exception to these challenges, recently announcing a $226 million gap in its 2020 budget.

With the mayor’s office having prioritized initiatives around equity and championed a modern culture of data, Denver was readier than many other places to respond to the immediate circumstances of the pandemic. The city was already pursuing a strategy of centralizing and organizing its data, and empowering front-line employees to make data-driven decisions. Still, its team didn’t want to lose sight of long-term impacts. Revenues were becoming difficult to project and they knew taxpayer dollars needed to go further in providing for vulnerable residents and sustaining services. How could they prepare for that?

Data at the heart of every discussion

The basic approach was obvious: it had to remain data-driven. “When diagnosing, you expect doctors to make decisions based on data points,” says Paul Kresser, the City of Denver's chief data officer. “You want them to order tests, review the results, and then determine what needs to be done. It's the same with the decisions we make around services. We don't want to say, ‘Oh, I think this might work!’ No. We want the data to prescribe a path forward.”  

In short, they needed to create models and forecasts for different scenarios, and use data to inform as many conversations and decisions as possible. And they needed to start now.

But in the pressure of an emergency response scenario, they also needed resources and support. One of Slalom’s public sector experts, Joe Kuntner, reached out proactively to offer Denver pro bono help, wherever they needed it. “We did it because one of our clients, who we care about a lot, is in a time of need and is getting stretched further than they have ever been stretched,” Kuntner says.

For Kresser, this approach felt like a puzzle piece falling into place. He’d just received a similar email from students at the Harvard College Data Analytics Group and saw a unique opportunity to bring together Denver’s public resources, private sector partners, and partners in academia in a single collaborative effort.

Blended teams with a sense of purpose

Slalom assembled a cross-market group of consultants to partner with Denver and mentor the Harvard students, leveraging the students’ knowledge and skills while giving them real-world consulting experience.

The students’ academic perspective brought new insights and richness to the project, says Vinit Nair, a consultant from Slalom’s Silicon Valley office. “It turned out to be a partnership. We spent hours discussing econometric models that we thought best fit our problem, and these discussions eventually helped us implement solutions that have really sound theoretical reasoning.”

The blended team focused on three departments with large revenue sources for the city: Community Planning and Development (CPD), Transport and Infrastructure (DOTI), and Excise and Licenses (EXL). The aim was to improve forecasting capabilities and predict revenue in a world where COVID-19 persisted. The team also partnered with the city’s Finance department to measure resiliency and identify economic and social vulnerability areas across the city.

Every mini-project had its own dedicated pod of one data scientist, one data engineer, one student, and business sponsors from the city. On a four-week deadline, and sheltering at home, the pods collaborated virtually across locations and time zones, following an agile methodology. Everyone was aware of the importance of the work for Denver’s future and determined to progress quickly.

Custom needs require custom approaches

Each of the pods developed forecasting models and solutions tailored to their departments’ needs and problem sets. After all, the permit fees from a CPD budget are very different to the license fees handled by the EXL department. “Really talking to your stakeholders, understanding the environment they're working in, understanding what they're looking for is a big piece of what any good data scientist will do,” says Payal Rohira, the solution principal for Slalom Denver who oversaw the pod teams.

Because the pandemic is unpredictable, the departments have to adapt to shifting circumstances. The teams built forecasts with dynamic, reusable frameworks so that as conditions change, the departments can reevaluate them in response to real circumstances, update them, and make decisions.

EXL, for example, faces a distinct set of challenges. It manages a wide array of licenses: retail, liquor and marijuana, security guards, and more. Each license has specific rules around renewals, late fees, and application fees. That’s a lot of variables to factor in when predicting revenue. Add in an uncertain economy where businesses might be shutting down temporarily or permanently and the picture becomes even more complex.

Rather than using simple moving averages or cumulative averages to arrive at estimates, the team had the idea to create a simple, user-friendly R Shiny application. “We hard-coded in a couple of different models,” says Jay Garg, a rising sophomore at Harvard who worked on the application. “It looks at the most recent data that they upload to it and then picks the best, most accurate model.” This means the EXL team can more easily assess how revenue streams will change and build estimates that are in tune with evolving circumstances on the ground—all without time-consuming, manual math work.

Sporting events and concerts have historically generated a significant amount of parking revenue for the city. Whether these events would return in 2020 (or beyond), and in what capacity, was a big question for the DOTI department. The pod team created five new Power BI dashboards that visualize the changing revenue, all backed by a forecasting model that offers a detailed level of information, works with multiple parameters, and enables the team to adapt it. As the city moves through phases of recovery and reopening, DOTI teams can adjust their revenue projections in line with the real-world situation. “Now it's a tool in their toolbox,” Kresser says, “not just for a crisis but anytime." 

Taking care of vulnerable residents

In the face of a new normal, the city wanted to be sure to support historically marginalized communities, allocate resources to where they were most needed in the short-term, and make strategic and thoughtful investments for the long-term. 

The Slalom and City of Denver teams created a holistic resiliency analysis for Denver by using two publicly available indexes: the City Resilience Index, which focuses on economies in a global context, and the Social Vulnerability Index, which was developed in the aftermath of Hurricane Katrina and restricts itself to US cities. The analysis identifies communities disproportionately impacted by COVID-19 and helps city teams plan resiliency measures across the city. It’s also a benchmark that will allow the city to track the those investments and assess improvements in residents’ lives. “Resiliency’s really going to inform some of the investments we make,” says Kresser. “Unfortunately, COVID is not going to be the last major shock—economic, financial, or human—that the city faces.”

Data gives shape to the future

The City of Denver has already identified how it will begin to evolve the pods’ work. Kresser’s vision is that someday data will flow freely and easily between departments and teams. Forecasting has the power to identify future needs—school places, new buildings, and parking capacity—and inform planning across city departments based on what today is one touch point in one department.

But for now, with the right forecasting models in place, the City of Denver is equipped to steer the city safely through the initial crisis of the pandemic and into the future.

For one of the Harvard students, this work held a personal dimension: “This is where I grew up,” says Tom Biasi, a rising junior. “This is where I am right now. I really worry about the small businesses around here. I worry about my family members—and I know that the work we did will help my city respond for the better.”