Should Retirees Pay Off Their Mortgage Early?

Should I pay off my mortgage early to get rid of the monthly payments?

I retired five years ago at age 55 with a pension and a 401(k) from which I took fixed withdrawals through age 59½. Then, I rolled over the 401(k) to a low-risk annuity with a $200,000 balance. I have $50,000 left to pay on my mortgage (five years left on a 10-year loan with a 5.375% interest rate). With most of the interest paid on this loan, I'm wondering if it would be better to pay off the balance with an annuity withdrawal or continue to make monthly payments. I realize I would have to pay taxes on the annuity withdrawal, but I could use the extra $1,000 a month now rather than in five years.

--James Wilson, Chula Vista, Calif.

Your dilemma is a common one: Should people pay off their mortgage when they retire -- and get rid of that big monthly expense?

Part of the decision is based on what helps you sleep better at night. Some retirees prefer to spend a chunk of their nest egg upfront so they no longer have to worry about a mortgage; others feel more secure keeping the money in their savings accounts as long as possible.

Looking at the numbers rather than the feelings involved, three certified financial planners we consulted all had the same advice: In most cases, your best bet would be using the annuity to fund your monthly mortgage payments. That way, you don't have to take the $1,000 from your regular income; at the same time, you can preserve the bulk of the annuity and give it a chance to continue to grow tax-deferred.

However, you have to watch out for surrender charges, which are penalties charged on some annuities for early or large withdrawals. You'll have to check your specific contract, but many annuities allow at least $10,000 a year in withdrawals without a surrender charge, which means you could make at least 10 payments a year that way.

Again, because of potential surrender charges, withdrawing $50,000 all at once might be tougher. Even if you aren't slapped with a surrender charge, adding $50,000 to your income in one year might bump you into a higher tax bracket, says Gary Cotter, a planner in Sun City Center, Fla. Assuming that your annuity is tax-deferred, you actually would have to withdraw more money than that -- probably at least $65,000 -- to pay the taxes due on $50,000, he adds.

You should also compare the 5.375% interest you're paying on your mortgage to your annuity's return, advises Ernest Hathaway, a planner in Midvale, Utah. If you have a fixed annuity, where you're earning a set interest rate, the comparison is easier; if you have a variable annuity, "it's a function of the stock market," and you'll have to come up with an average, he says. If your annuity is paying lower interest than your mortgage, you may be better off paying the mortgage -- as long as the difference wouldn't be wiped out by a higher income-tax rate or surrender charges, he says.

But if your annuity is earning more than you are paying in mortgage interest, you're better off keeping the money invested rather than using it to pay your mortgage, says James Suits, a planner in Tacoma, Wash. If your annuity isn't earning 6% currently, you might consider rolling it into a long-term certificate of deposit at a credit union that would, he adds. He also points out that by putting a 401(k) rollover in an annuity, "you're paying an extra expense for tax deferral you already have."

One last note: Are your property taxes and homeowner's insurance currently included in that $1,000 monthly payment? Remember that those expenses will remain, even after you pay off the mortgage.

Corrections & Amplifications:

Many annuities allow withdrawals of 10% a year without a surrender charge. This column incorrectly says that many annuities allow withdrawals of $10,000 a year without a surrender charge.