Since Fannie Mae and Freddie Mac entered conservatorship in 2008, there has arisen a sort of urban legend surrounding the role affordable housing played in the mortgage crisis.

One of the perennial issues cited frequently by detractors of the government-sponsored enterprise model exemplified by Fannie and Freddie is that these agencies were conduits for overly aggressive affordable housing goals. This mandate, we are told, contributed greatly to the losses sustained by both companies, eventually leading to their current state of financial limbo known as conservatorship.

Unfortunately, in part due to this mindset, housing markets continue to be held hostage to failed policy. Until some of the myths surrounding the GSEs are dispelled and effective housing policy established, the ability of creditworthy first-time homebuyers and low- and moderate-income borrowers to gain broader access to mortgage credit will remain severely limited.

Ample empirical evidence, including actual delinquency and default statistics, suggests that affordable housing goals enforced on the GSEs did not materially contribute to the crisis. Even so, the risk of these loans, while higher than the GSEs’ standard business was far less than traditional subprime loans such as the 2/28 and 3/27 adjustable-rate mortgages found in private label securities of the time. What is lost in the political rhetoric is the fact that the agencies had negligible exposure to traditional subprime loans even though they had sizable positions in riskier loans.

The mortgage industry never satisfactorily defined what constituted a subprime mortgage, but equating low- and moderate-income borrower loans to subprime is a gross mischaracterization of that segment of the market. Moreover, making the GSEs out to be the financial villains of the mortgage crisis is an easy sell to a fearful and angry public, but it doesn’t make it true.

The good news is that we have an opportunity to get this country’s housing market back on its feet and greatly shrink the massive federal support that has existed since 2008. Unwinding the conservatorship for both GSEs is well within the regulatory purview of Fannie and Freddie’s regulator, the Federal Housing Finance Agency. Recapitalizing and reconstituting the agencies as they were — but with strong capital buffers that would ensure taxpayers never again face having to bail out these companies, vigilant oversight in the form of a public utility model and effective practices and controls that would avoid future financial troubles — can be accomplished within the next few years. It can be done without waiting for Congress to act.

An essential ingredient to all of this is forming a sensible housing policy that clearly defines the purpose and roles of the GSEs, the Federal Housing Administration and the private sector, and establishes effective mechanisms to ensuring low- and moderate-income borrowers have access to credit and the ability to remain in their homes. Today, portions of the FHA and GSE mortgage markets compete against one another in a sort of federally supported shell game that only mortgage bankers understand. And for decades housing policy has been conflicted on what types of loans should be offered by the GSEs and FHA. Loan limits have been one way to limit funding mortgages to millionaires, but we still allow the GSEs to insure vacation homes and loans to investors. Does this even make sense from a public policy perspective?

Moreover, setting high level affordable housing targets doesn’t directly help a low- and moderate-income family struggling to make their next month’s payment when they lose their job or their furnace breaks down.

What's lacking in the housing policy dialogue are practical but viable solutions for financing mortgages for low- and moderate-income and middle class borrowers. Instead of establishing broad affordable housing goals, one possibility would be to create a fund supported by GSE guarantee fees. This fund could be used by financially challenged borrowers to bridge temporarily a loss of income or major unexpected housing-related outlays that can send homeowners down a path of mortgage instability and financial ruin. Rather than a fund for experimentation and innovation as the Market Access Fund of the Johnson-Crapo GSE reform bill was intended for, this fund would directly support those borrowers needing the most help and what typically triggers a default in the first place.

Rather than focus on a comprehensive solution to housing finance reform that takes the best of what worked about the GSEs and addresses the deficiencies of these companies and affordable housing policy, we have piecemeal solutions. They come in the form of federally mandated underwriting and loan repurchase rules; federal appropriation of the profits of private investors in the GSEs that destroy incentives for private capital to flow freely to housing; disjointed GSE and FHA policies and overbearing and highly inefficient regulations that hinder rather than promote a real recovery in housing. It's time for the Dark Ages of housing to end and its Renaissance to begin.

Clifford V. Rossi is Professor-of-the-Practice and Executive in Residence at the Robert H. Smith School of Business at the University of Maryland and a former risk management banking senior executive.