Small investors, not first-time homebuyers, are driving home prices to unaffordable highs, according to a new report from Attom Data Solutions and Clear Capital.

Real estate investors drove the home price recovery, but a different segment of this market is poised to continue to shape the housing markets.

At first, it was the institutional investors that took advantage of the housing downturn, buying properties as home prices hit their bottom. But as prices rose, these investors cashed out and were replaced by smaller investors.

Rather than flip these properties, many of these investors turned to the rental market for money making opportunities.

The report shows though that over time the smaller investors became the main driving force behind home price appreciation, particularly as the homeownership rate remained low and the buyer share comprising Federal Housing Administration borrowers flattened out.

"Because the driving force behind this housing recovery has been real estate investors, home prices have risen higher and more quickly than if the recovery had been driven more heavily by first time homebuyers," Attom and Clear Capital wrote in the report.

How the divide between investors and owner-occupants will shape housing markets and home price appreciation in the future depends on where one looks. The report compares the experience of three metropolitan areas that have recently experience high levels of price growth: Seattle, Nashville and Dallas. In Dallas and Nashville, investors played a much bigger role in the recovery from the crisis. Seattle on the other hand witnessed housing demand soar thanks to growth in the city's tech sector.

Consequently, Seattle's home price movements are more likely to be influenced by the traditional owner-occupant homebuyers, while investors will continue to have a bigger impact in places like Nashville.