Affordability issues along with a lack of knowledge about home buying are holding millennials back from pursuing homeownership, according to a survey from loanDepot.

No longer wanting to pay rent and being ready to start a family are the top two drivers cited for wanting to buy a home, the survey found.

About half of the surveyed millennials are anxious about real estate and mortgage payments, and only 18% feel they can afford to purchase a home.

Millennials also described a number of reasons they are hesitant to enter the housing market. About 63% were concerned about not being able to afford a down payment, 48% were unsure of where to start (with 56% saying they'd resort to a Google search), 43% listed poor credit history, and about 38% think too much existing debt stands in their way.

"It's clear from the survey results that millennials have a lot of anxiety built up about the home-buying process," said David Norris, loanDepot's head of retail lending, in a press release.

"There is good news, however, as there's more flexibility than most millennials think regarding how to qualify for a loan and what's needed for a down payment," said Norris.

Of the 49% of millennials who have a home mortgage, only 27% said they felt knowledgeable about the home purchase process when they started.

An area millennials feel largely misinformed on is the down payment.

Millennials who have not yet taken the plunge are overestimating how much they need for a down payment, with the average guess being 32%. While the industry standard is still 12 percentage points lower at 20% down, there are still other options available.

Fannie Mae and Freddie Mac opened the door for 3% down payment programs back in 2014, prompting big banks and other institutions to follow suit.

"The best advice I have for young buyers is to not believe everything you read on the internet," said John Pearson, a Hoboken, N.J., loanDepot lending officer.

"When talking with a professional, you can discuss your specific financial situation and the lending officer can help you determine how much down you'll need and what a monthly mortgage payment will look like. You'll probably discover you don't have to wait until you reach the point of a 20% down payment," he explained.

Nearly 1.5 million borrowers made a home purchase using down payments of less than 10% in the past 12 months, according to Black Knight Financial Services.