America’s recovery from the pandemic downturn has been rocky: Millions of workers — especially women — remain on the sidelines, while parts of the economy are booming and inflation is rising.

Getting people back into the labor force and into a job is now key to keep the recovery going and stop the economy from overheating, according to Minneapolis Federal Reserve President Neel Kashkari.

“We have to find a way to bring [women] back to work. This is about our economic potential,” Kashkari told CNN’s Poppy Harlow on Tuesday. “It is certainly about fairness about women and families. But it is also about our economic potential as a nation.”

The US Chamber of Commerce said Tuesday that the nation’s worker shortage is getting worse by the day.

Experts are struggling to disentangle the biggest reasons why people might choose to stay home, including access to child care, persistent virus risks and higher-than-usual unemployment benefits. Women have been hit particularly hard during the pandemic, as child care responsibilities still tend to fall on them and they were more likely to be forced to leave the workforce.

“I believe we’re going to bring women back into the labor force and workers who have been displaced,” said Kashkari.

Failing to do so would make the higher inflation readings more concerning, as it would call into question the efficacy of all the government stimulus.

Kashkari believes higher inflation numbers aren’t here to stay and are just a temporary effect from the economy getting back to normal. That has been the Fed’s go-to line for months now. Economists predict that prices might continue to rise faster than normal during the summer months.

Since the recovery picked up steam in the face of the full reopening, economists, investors and lawmakers have been concerned about the economy overheating.

With plenty fiscal and monetary stimulus in the system, some worried that too much of a good thing would turn up inflation so high that people might stop spending. And that would be very bad news for the consumer spending-dependent US economy.

Worrying about this might not be a bad thing.

Millions of Americans still out of work despite recent improvements in labor markets

“There was a lot of complacency in the then years following the Great Financial Crisis when millions of Americans were on the sidelines. So it took ten years to rebuild the labor market,” Kashkari said.

“We cannot have another ten-year recovery. So while I appreciate the fact that some people are worried about inflation, they didn’t seem that worried when millions of Americans were on the sidelines in the last recovery,” he added.

As of April, America was still down more than 8 million jobs compared with February 2020, before the pandemic lockdown. Economists expect to see 664,000 jobs added back in May in next week’s jobs report. While that would be a great number in normal times, it hardly moves the needle in the pandemic economy.