Housing advocates and low-income residents sued Wednesday to stop Mississippi from spending a half-billion federal dollars to expand a damaged port rather than replace homes destroyed by Hurricane Katrina.

The Mississippi State Conference NAACP, Gulf Coast Fair Housing Center and residents sued the U.S. Department of Housing and Urban Development in federal court in Washington, D.C.

Congressional leaders and others slammed HUD when it approved the state’s plan to steer money to the Hurricane Katrina-damaged port despite a lingering housing crisis caused by the 2005 storm.

Gov. Haley Barbour maintains expanding the State Port at Gulfport, the third-busiest container port in the Gulf of Mexico, is key to the region’s economic recovery.

The storm surge devastated Mississippi’s coast, washing away much of the affordable housing. Three years later, about 6,000 families remain in temporary housing. Housing prices have more than doubled and insurance costs have increased, forcing some to relocate.

“They were just leaving us out to defend for ourselves,” said Dorothy McClendon, 59, a disabled former state employee and plaintiff in the lawsuit who still lives in a trailer provided by the federal government after being denied assistance through the Katrina housing program. “Most people like myself are low-income, elderly people who own their homes, but don’t have no other means.”

In January, then-HUD Secretary Alphonso Jackson said he had concerns about the plan to redirect the money but was obligated to OK the project because of congressional rules.

“Our basic claim is that HUD had a duty and responsibility to review the proposal and took a position we believe is inappropriate — that they did not have the discretion to reject it,” said Joe Rich, an attorney helping the housing advocates with their lawsuit.

HUD spokesman Brian Sullivan declined comment because the agency had not received a copy of the lawsuit.

The agency that oversees federally funded Katrina recovery projects has said programs are under way to address the housing situation, including a $350 million plan to build thousands of homes for the region’s working class.

Lee Youngblood, a spokesman for the agency, the Mississippi Development Authority, said the port project will proceed.

“It’s somewhat surprising that they would file a lawsuit, but at the same time, we’re moving forward,” said Youngblood, who said the state has dedicated $700 million to low-income construction projects.

The lawsuit contends that so far Mississippi has devoted only 21 percent of $5.4 billion in federal hurricane recovery funding to projects that benefit low- to moderate-income households. HUD’s Community Block Grant Program, which provided most of the money, requires 50 percent of all spending to benefit such households.

Several congressional Democrats tried unsuccessfully to stop Mississippi from diverting the funds to the port.

“Safe, affordable housing was touted as the hallmark of Mississippi’s recovery efforts,” said Charmel Gaulden, executive director of the Gulf Coast Fair Housing Center. “Now, our own government is seeking to leave the least among us out in the cold.”