The New York bank will double in size while gaining greater access to Long Island and low-cost deposits that should come in handy as interest rates rise.
The agreement comes three months after Astoria and New York Community Bancorp called off plans to merge.
Caliber will also buy about $36 million of mortgage servicing rights in a deal that is expected to close at the end of March.