No one wants to be thought of as a bank that is top-heavy in mortgages.
Banks with large home-loan operations are struggling with narrower profit margins, a bevy of regulations and accounting rules, and other problems that make the current mortgage downturn feel worse.
Moreover, investors' concerns about the volatility of the banks' earnings are weighing down their stock prices.
"Having a big mortgage operation can be negative in some years, but in others it's a big positive," said Brad Schwartz, the chief executive of the $1 billion-asset Monarch Bank in Chesapeake, Va.
Mortgage-heavy banks often garner lower price-earnings ratios than their peers, said Chris Marinac, the director of research at FIG Partners.
The average price-to-book value of 300 publicly-traded community banks was 149%, compared with an average of 129% at 17 banks that had heavy concentrations of mortgages, Marinac found. He defined "heavy concentration" as those that relied on mortgage fees for 10% or more of revenue.
"It's one of those conundrums because the mortgage business actually had a good second quarter, but investor perception is not good if you get labeled a mortgage company," Marinac said. "There is still ongoing uncertainty about recurring earnings."
Some bankers balked at being stereotyped, but they acknowledge that a heavy dose of mortgages demands an explanation.
"We're an outlier," said Bernard Clineburg, the chairman and CEO of the $3.2 billion-asset Cardinal Financial in McLean, Va.
Clineburg called Cardinal's price-to-earnings multiple of 14.53 "pretty high." He also noted that his bank operates in the Washington metropolitan area, a strong banking market that has four of the top 10 wealthiest counties in the nation.
"We're competitive with other banks that don't have mortgage banking operations," Clineburg said. "The mortgage bank has had great years and not so great years but over a period of time it produces earnings that are a great capital add."
Some banks with core mortgage divisions have worked hard to overcome the label of being too heavily associated with mortgages. The $19.7 billion-asset EverBank in Jacksonville, Fla., bought a large commercial finance company and an equipment-leasing firm in 2010 and shuttered its wholesale-lending business last year.
Scott Verlander, the vice president of corporate development and investor relations, called EverBank "intentionally larger and more diversified." It now derives 55% of revenue from residential loans, compared with 75% before the downturn and various acquisitions.
"You have to appreciate some of the nuances," Verlandersaid. "We're in top wealth markets…targeting a jumbo-purchase customer. … A lot of these guys that are churning and burning tend to get the lower multiple because you have to continuously refill the bucket."
Mark Mason, the CEO of the $3.1 billion-asset HomeStreet Bank, in Seattle, said some banks are getting lower stock valuations because of overcapacity and the dramatic drop in mortgage volume this year. Investors do not have confidence in the future profitability of mortgages until such issues are worked out.
"Margins have been squeezed as lenders have lowered their profit margins to attract and maintain loan volume," Mason said. "All of this is in the face of ever-increasing compliance and investor requirements to improve loan quality, both of which are going in the wrong direction from a profitability standpoint."
HomeStreet sought to grow during the downturn, pushing into California at a time when the largest banks had no choice but to lay off employees and streamline operations, he said. Many lenders found it hard to make money on mortgages in the third and fourth quarters of last year, and in the first quarter of this year, he said.
"The market shrank faster and farther than everyone generally expected," he added. Even the rebound in the second quarter "wasn't as large as we expected."
Schwartz and other bankers are also blaming an accounting rule from 2007 that treats mortgage rate-lock commitments as derivative assets that get recorded at fair value.
It now is hitting banks at a vulnerable time, when there is a downward trend in overall loan volume. Home purchases typically slow in the winter, particularly in the Northeast, when banks may close half as many loans as in the summer months.
"It hits you when it hurts you the most," said Schwartz, adding that banks differ in how they apply the accounting standard.
Schwartz described the accounting treatment like a hill; when rate-lock commitments are positive, "you're adding revenue that's not real but will be real in the future, and when volume declines, you take paper losses."
During the transition to a home purchase cycle, when refinances have run their course and volume slows, banks take a hit to future earnings.
"It creates volatile earnings so mortgages are love or hate depending on the year," Schwartz said.
Brian Koss, a managing partner at Mortgage Network Inc., a Winchester, Mass., mortgage bank, recalls working at Dime Savings Bank years ago when mortgage lending made up 60% of earnings. The mortgage business was growing faster than deposits, which caused "agitas" for bankers, he said. Dime got sold in 2002 to the Seattle thrift Washington Mutual, another big mortgage contender that in 2008 became the largest bank failure in American history.
"Back then everyone knew that if you became top-heavy in mortgages it would kill your stock price because investors think of you as a mortgage company and not a bank," Koss said. "It will be a long time before anyone is proud to say they have a lot of mortgages."