Larry Swedroe

Larry Swedroe is principal and director of research for Buckingham Asset Management, LLC, a Registered Investment Advisor firm in St. Louis, Mo. He is also principal of BAM Advisor Services, LLC, a service provider to investment advisors across the country, most of whom are affiliated with CPA firms. Swedroe's mission is to educate people on [...]


Joe Anderson
ABOUT Joseph

As President of Pure Financial Advisors, Joe Anderson has led the company to achieve over $2 billion in assets under management and has grown their client base to over 2,160 in just ten years of the firm opening. When Joe began working with Pure Financial in 2008, they had almost no clients, negative revenue and no [...]

Alan Clopine

Alan Clopine is the CEO & CFO of Pure Financial Advisors. He currently shares the CEO role with Michael Fenison, the original founder of the company. Alan is primarily responsible for the day-to-day activities of the firm while Michael’s focus is on expansion and hiring. Alan joined the firm about one year after it was [...]

Published On
January 29, 2019
Larry Swedroe: How to Avoid the Five Horsemen of the Retirement ApocalypseLarry Swedroe: How to Avoid the Five Horsemen of the Retirement Apocalypse

Larry Swedroe from Buckingham Strategic Wealth talks about his 16th book, Your Complete Guide to a Successful and Secure Retirement. Larry discusses the two biggest retirement mistakes, and what he calls the five horsemen of the retirement apocalypse. Plus, is there a way to consolidate your investment portfolio before you retire and avoid paying capital gains tax? Also, dancing to stave off dementia, the world of golf, and China and Brexit news – but probably not the kind you’d expect.

Listen to the podcast on YouTube: 

Show Notes


Our friend Larry Swedroe from Buckingham Strategic Wealth has just published his 16th book, Your Complete Guide to a Successful and Secure RetirementLarry joins us today on Your Money, Your Wealth® to talk about this definitive retirement resource, the two biggest retirement mistakes, and what Larry calls the five horsemen of the retirement apocalypse. Plus, is there a way to consolidate your investment portfolio before you retire and avoid paying capital gains tax? But first, dancing to stave off dementia, the world of golf, and China and Brexit news – but probably not the kind you’re expecting. I’m producer Andi Last, and here are the hosts of Your Money, Your Wealth®, Joe Anderson, CFP® and Big Al Clopine, CPA.

:45 – Market Tidbits: China, Brexit, Grass Golf Shoes and Dancing to Fight Dementia

Joe: We have a lot of stuff going on here in the United States – government shut down, tariffs, volatile stock market, that new tax return, the postcard that’s the stupidest thing I’ve ever seen in my life. But China – did you hear about what’s going on over in China, Alan?

Al: Yeah. Apparently, they’ve got a new app that you can put on your phone to find people that are in debt in a big way.

Joe: Have you ever heard of such a thing?

Al: No.

Joe: Neither have I. There are these bad debtors.

Al: Yeah. So they want people to be to point them out. Whistleblowers. “Hey, there’s a deadbeat debtor.”

Joe: Yeah. This guy owes so much money, but look at him, he’s at Bloomingdales. Is there Bloomingdales in China?

Al: (laughs) I doubt it. So anyway I guess they’ve already implemented this and so people have been reporting others.

Joe: More than 6000 people.

Al: They’ve been punished already because of this, because they failed to pay their taxes, or they misbehaved on public transportation.

Joe: They were barred from taking planes and trains in and out of the country between June and January. What is misbehaving, I wonder on public transportation?

Al: I don’t know.

Joe: Interesting. We could have that here, that would be a lot of fun. My phone would be going bananas around Big Al. (laughs)

Al: (laughs) You would keep reporting me.

Joe: Look at the big debt on Big Al!

Al: They would say oh you know what, there’s a glitch, it’s actually Joe Anderson. It’s actually the person sending it.

Andi: So it’s actually part of China’s “social credit” system. Which is pretty creepy.

Al: It is a bit creepy.

Joe: You know what. There’s also something that we need to implement here at Pure Financial Advisors. We call it the Anger Room. They call it Smash there in Beijing. For about $23 American – that’s 158 yuan. Did you know that? See how good I am with making that exchange rate really quickly? (laughs) They will let you spend half an hour destroying household objects like wine bottles, TVs, computers, and furniture with hammers and pads while wearing protective gear and listening to your music of choice.

Al: I think they’d have to charge you double. You would just go through everything they had.

Joe: Smash opened in September to help people deal with the pressures of living in the big city. Or maybe the pressures of having bad social credit. And then there’s also something going on in the European Union. So we’re nine weeks away for Brexit.

Al: And by the way, that’s Britain leaving the European Union in terms of trading.

Joe: Thank you. Got it. So what they’ve done over there is, these Brits are worried they might need a little survival kit. So they’ve got the Brexit Box.

Al: Right. because they’re worried they may not get the imports from other European countries, including some of the food and things that they like.

Joe: Retailing at £295, wait, that’s probably going to be about $380. (laughs)

Al: You know, you’re right on, according to this paper in front of us.

Joe: It provides food rations to last 30 days. So when you’re looking at, what do you think is in this thing? So you pay a few hundred bucks. They got medical supplies, got some food. Then they have their favorite foods, what do you think some of the favorite foods are?

Al: Well, I’m going to say chicken, chili con carne, macaroni and cheese.

Joe: I love macaroni cheese. You’ve been to Britain, London? Have you ever been over there?

Al: Yeah, sure, a couple of times.

Joe: Did you enjoy the food?

Al: No. That’s one thing about Britain is that the food is not the best. (laughs) I like the beer. I like the atmosphere. I like the town. I like the vibe. But you need to bring in your own food.

Joe: Yeah. Taters and mash, whatever that was.

Al: Bangers and mash. Ever had that Andi?

Andi: I have, yeah.

Al: Do you like it?

Andi: No. I like the meat pies, shepherd’s pie.

Al: Yeah I agree. Those were okay.

Joe: I like macaroni and cheese, but…

Al: I did get bangers and mash and it was a little disappointing. So yeah, you can get this Brexit box and you can then survive for the next 30 days, and the reason why they have to have the box is no one likes British food. They’re afraid they’re not going to get their imported chicken.

Andi: So they’re going to be surviving with freeze-dried fajitas.

Joe: So you don’t think that there’s chicken in Britain?

Andi: Chicken tikka.

Joe: What is chicken tikka?

Andi: It’s Indian food which is incredibly popular in Britain.

Al: And they probably don’t have chicken fajitas. That’s part of the kit apparently.

Andi: And it also has mince, which is, in American, that’s ground beef.

Al: And an emergency water filter because maybe the water systems will shut down.

Andi: And fire lighter liquid.

Joe: Last but not least, here, I’m gonna buy these shoes, Nike just unveiled the new version of the Air Mac 1 golf sneakers. If you wear them and you’re in grass you can’t even see your feet.

Andi: Why? What is the purpose of that?

Al: They look like grass.

Joe: They’re grass shoes.

Al: Yeah they look like grass. They’re astroturf, I guess. So you can blend in with your surroundings.

Andi: Wouldn’t it just look like you don’t have feet? (laughs)

Al: Wouldn’t that be cool? All these golfers have no feet.

Joe: That’s that’s the purpose. It gets you really focused on your game.

Al: That’s something they ought to do on Halloween just on a normal round.

Joe: Here’s one last thing. A study was done by the Albert Einstein College of Medicine in New York, Alan since you like to quote Albert Einstein.

Al: I do have a propensity towards that.

Joe: It’s in New York City, 2003, to determine if any physical or cognitive recreational activities influenced mental…

Andi/Al: Acuity.

Joe: OK the microphone got in the way, I thought it was activity. Acuity. The results were published in The New England Journal of Medicine. I’m not sure where we’re going with this but it’s going to be good. They found that dancing was the only physical activity that reduces the risk of dementia. Bicycling, swimming, playing golf offered no protection whatsoever.

Al: Now you should be fine because you’re a great dancer.

Joe: I am a very good dancer.

Al: But you probably never do it. You probably dance once a year?

Joe: I dance…. no.

Andi: You don’t dance as often as you golf though.

Joe: I dance probably three times a week.

Al: You do?

Joe: Absolutely.

Al: Where?

Joe: In my bedroom, in my living room (laughs) I put a Darth Vader mask on and I start dancing in my living room.

Andi: Dancing reduces the risk of dementia by 76%.

Joe: I was getting there. Frequent dancing offers a 76 percent reduction of dementia and increases cognitive acuity to all ages. For cognitive activities, doing crossword puzzles at least four days a week reduces the risk of dementia by 47%, reading reduces dementia risk by 35 percent. Apparently, I need to do more crossword puzzles and read, but dancing. I enjoy dancing quite often. I don’t go to the clubs and dance. I will dance at my house.

Al: Yeah I’ve seen you dance at your house.

Joe: Put a little music right and do a little this, little that. It’s a good time.

Al: Yeah. Usually what happens is we’re all dancing and then you’re so entertaining we stop and just watch you.

Joe: Yeah and throw dollars. It happens.

Al: I don’t think I’ve ever thrown a dollar at you.

Joe: (laughs) Yeah it’s been tens and twenties.

Al: Hundreds. (laughs)

Find links to all of the articles mentioned in the show notes for today’s episode at the brand-new YourMoneyYourWealth.com. If you’re looking for a way to keep the body and mind active in retirement, sounds like dancing like Joe does is the way to go. Maybe a retirement side hustle would work for you as well – check out the Your Money, Your Wealth® TV show on Retiring in a Gig Economy, also in the show notes.

 Speaking of your life in retirement, Larry Swedroe is the Director of Research for Buckingham Strategic Wealth, and he has written 15 different books on various aspects of investing. He’s just published his 16th book, Your Complete Guide to a Secure and Successful Retirement, which goes into all aspects of retirement.

09:48 – Larry Swedroe: The Five Horsemen of the Retirement Apocalypse and the Two Biggest Retirement Mistakes

Joe: Larry’s been on the show many, many times for the last several years. Larry, how many years have you been joining us on this awful program?

Larry: I don’t know, but it does seem like forever.

Al: (laughs) It seems like forever for you.

Joe: He gets the call from us, he’s like, “oh those jerks? Not again.” Larry; It’s always a pleasure to have come on and hopefully be helpful.

Joe: Every time Larry’s on the show too he’s like, “This is my last book.”

Al: Right. We have heard that Larry for about eight years.

Larry: I have to admit that’s true. I had the idea for Your Complete Guide to a Successful and Secure Retirement for about 10 years, having written a guide to the accumulation phase, also the only guide to the right financial plan, the only guide to alternative investments, and the only guide to a winning bond strategy. So what was missing was how do you deal with the rest of your life once you retire and go into withdrawal phase? So I tackled of the book and in two months, with the help of an all-star team I recruited to help write the various chapters that they’re experts on, we finished the book, and I’m real proud. I think it is the most comprehensive book on retirement out there.

Joe: I would agree. I’ve read my fair share of financial planning books and the organization of it just kind of walks you through basically everything that you need to do to have a successful retirement. Let me ask you this Larry because you’ve written so many books, a lot of them on investing. What is the smart way to do it, helping people to become such a better investor? But investing, in a successful retirement, is just a small piece of it. Did you get bored with other areas of the book, saying, “I really don’t want to do this..” Was there a challenge of diving into the other areas? You said you had experts – tell me a little bit about how you organized everything.

Larry: Well, the first chapter of the book is all about planning a meaningful life in retirement. It has nothing to do with the money and financial issues. And I think that’s absolutely critical. Many of us do have financial plans but we forget about planning a meaningful life. And the sad part is, having done the research, having read some books on this subject, and recruited one of the authors of what I think is the best book on this subject, Your Retirement Quest, I have learned how important it was to plan a meaningful life, as that old saying goes, “those who plan to fail, fail to plan.” And the problem is that most of us get much of our joy out of life from work – it’s the social connections we have and the intellectual stimulation, the sense of accomplishment, all of those things. And when you retire, those things are no longer there. So if you don’t plan to have a meaningful life it can create serious problems. We touch in the book on some of them. I’ll just mention some key statistics: the highest suicide rate in the US. I would have guessed was teenage girls, and it is now retired men. The second thing is, the fastest growing rate of divorce, the cohort where it’s the highest is what is now called “silver divorces.” “Honey I married you for better or worse, but not for lunch.”

Joe: Yeah I think so many people understand preparing for retirement as, “I need a certain dollar figure in my retirement account or my brokerage account and Social Security and everything else.” But there’s such a lack of planning of what they’re going to do, how they’re going to spend their time, what their relations are going to look like. What is their purpose in life? And it’s like, yeah I’m going to play golf every day,” and then a couple of months later they’re like, “I’m never playing golf again.”

Larry: Yeah because it becomes work. It’s not a joy, it’s not the exception. It’s fine probably to play once or twice, maybe even three times a week because it does get you out and have social connections, but you also need mental stimulation and a sense of contributing – that might be through charitable work, it could be, I have friends who have become Big Brothers. I have two uncles who started preparing taxes for the elderly and doing it either free or cheap or at very low cost. I have a friend who became a candy striper in a hospital. Others who go on to take classes at the university to keep learning. Whatever is good for you and creates meaning in your life and gives you what I call a reason to get up in the morning, that’s what you need to plan for. Practice it. That’s what I learned from Alan Spector. You need to imagine what that perfect day in retirement looks like. And maybe while you’re on vacation, practice that day and see if that really works for you.

Joe: Yeah you kind of paint a little bit of a gloomy picture in the beginning of the book, of like almost the retirement apocalypse, of the four horsemen of what’s coming down the pike. Share that analogy.

Larry: Yeah. There is a problem for investors who are approaching retirement. And I would say even worse for the younger generation. The people who are right now having just retired or very close to it, have benefited from basically a 36 year run of bull markets in both stocks and bonds. Over the longest period we have data for, from 1927 through 2018, the 60/40 portfolio returned about 8 1/2%, but over the 36 year period, it was about 2% higher. And the problem is, we got those great returns, the end result of it was that U.S. valuations, the PE ratios that are today much higher than they have been, historically. And of course, bond yields have collapsed. So while stock returns in the US have been about 10%, most financial economists today think, because of higher valuations, they’re likely to be more in the neighborhood of 6 or 7. And obviously, you can’t get 5 or 6% from, say, bonds any longer, when the 10-year treasury is yielding about 2.7%. So we think, and most financial economists agree, a 60/40 portfolio is likely to provide a real return of not about 7%, plus that 3% inflation we’ve experienced, which would have been 10, or more like 3% real and 2% inflation, roughly. So that’s 5. So problem number one is we have much higher valuations for stocks, much lower bond yields. That means you should expect much lower returns, which means you need to save more and/or work longer. Third problem, which is a good one to have: we are tending to live much longer, so you need a bigger pot of money because you need it to last longer. And the fourth problem as a result of the third, increased longevity: as we age, the increased risk of very expensive long-term care increases dramatically, including the risk of cognitive decline in high-cost areas that could easily be $100,000, $150,000 a year. And I know you folks live in a high-cost area, so we need to plan for those things. Those are the four horsemen. I added a fifth, which is the fact that Social Security if it isn’t addressed quickly, is going to be short of its ability to meet its obligations. It’s not going bankrupt. People should not be scared of that, but if there are no changes in 13 more years, Social Security will only be able to pay out 75% of promised benefits. So people need to be realistic, take these facts into account, do not rely on historical returns. That’s a big mistake that even professionals make. And you need to develop a plan that incorporates the reality of today’s world and greater longevity.

You can find the transcript of this interview and the link to purchase Larry’s book, Your Complete Guide to a Successful and Secure Retirement, in the show notes for today’s episode at the brand-new YourMoneyYourWealth.com. And be sure you’re subscribed to the Your Money, Your Wealth podcast. Next week, millennial millionaire Grant Sabatier joins us – you may remember he was on the podcast last year. Grant went from having $2.26 to $1 million in just five years, and he’ll tell us about reaching Financial Freedom with a Proven Path to All the Money You Will Ever Need. Now, more with Buckingham Strategic Wealth Director of Research and author of Your Complete Guide to a Successful and Secure Retirement, Larry Swedroe.

19:48 – Larry Swedroe: Your Complete Guide to a Successful and Secure Retirement

Joe: You know there’s all sorts of different strategies, I think, when it comes to retirement-type planning. Accumulation planning is one thing. It’s like, OK well, save as much money as possible. But then when you turn the corner and now you are retired, you need to create income from the portfolio that you have accumulated. And I know there’s the bucket approach that you just wrote about. There’s the total return approach. There are Monte Carlo simulations. There are all sorts of different things – high dividend paying stocks. I think that’s your favorite, Larry. (laughs) MLPs, there are all sorts of, I think, buzzwords and different investment alternatives or strategies that people could use to create income. What should people be thinking about as they turn the corner, as they approach retirement, as they need to start drawing dollars from their portfolio?

Larry: Well this is obviously a huge topic. We could spend hours on it alone. I’ll try to be succinct here and touch on two key issues. First is, a big mistake I find that people make is they often want to take an income approach instead of a total return approach. So they need income from their portfolio. They don’t want to tap into the principle, which makes no sense, because you don’t live forever and you can’t take the money with you. And an income approach in an environment like we had for the last 10 years, where safe bonds got you virtually nothing, caused many people to sell those, say, bonds, buy high dividend paying stocks, MLPs, REITs, things like that. And all you need is a repeat of 2008 and you could see 30, 40, 50% of that money disappear overnight. And that’s a disaster when you’re in retirement because now you’re withdrawing funds from your portfolio. And if the market eventually recovers, you can’t recover because that money has been spent. That creates what’s called sequence risk, and the order of returns matters greatly. So one thing I would advise against is taking an income-based approach. You should take a total return approach, which will prevent you from making the mistake of chasing yield. That’s probably the biggest mistake though right there is dealing with that. And the other mistake, I would say, is the fact that when you retire at 65, the second to die life expectancy of a typical couple is 25 years, which means half the time one of you is going to be living longer, which means you still have to plan for 30 years. And that means you can’t get too conservative, or inflation could become a serious problem. So you have to plan for a very long lifetime, much longer than our parents had to plan for. When I was growing up, I hardly knew anyone who was over 75.

Joe: Right. And it’s hard too, when you look at us talking about long-term, if someone’s in their 60s, and saying, “I need the money tomorrow, Larry. I need the money tomorrow. And I have to live off of this,” and then the volatility of the markets, I think people look at their portfolios a lot more than they probably should. And then they make the dreaded mistake of either selling out, or going into a lot more conservative type portfolio that’s a lot less volatile per se, but it’s not going to give them the yield or the return that they need to last 30 years, because they don’t have that timeframe in their head. They’re thinking, “my timeframe is this month. I need to pay my bills.” So it gets a little challenging.

Larry: There’s no question, we have all kinds of behavioral problems. And you do need to become, generally, more conservative when you’re in retirement because you no longer have your labor capital, which can replace losses in the market. If that labor capital can allow you to live off of that income without having to spend money from your portfolio, that can’t be recovered if and when the market recovers. But as we touched on, you have to find the balance, because you still need to plan for a very long horizon at age 65. So there used to be a rule of thumb that you shouldn’t take out more than 4% of your portfolio the day you retire, and then you can adjust that for inflation. So if you had a million dollars, you should only take out $40,000 the first year. If you have 5% inflation, you could take out $42,000 the next year. And if you did that, the odds of you running out of money would be very low as long as you maintained a moderate equity allocation in the area 40-50% of stocks. But because of the issues we touched on in the four horsemen of the Apocalypse, the new 4% we believe is 3%. So today to be safe you need to keep a reasonable equity allocation and not withdraw more than 3% of your portfolio. And that’s why it’s so important to plan for this before you retire. It may mean you need to work longer, you need to lower your goal in retirement, you may need to move to a lower cost of living area, downsize, all kinds of alternatives are there, which are better than taking too much out of your portfolio or becoming too conservative. Either one of those can cause you to run out of money while you’re still alive, and I can’t think of many things worse than that.

Joe: So you’ve been in this business for a couple of years I suppose, right?

Larry: Couple of decades.

Joe: So as I’m going through the book and I was thinking of you, and I was saying, “Larry has been doing this for several decades.” What surprised you the most of strategies that you would recommend today, that you probably wouldn’t have recommended maybe 20 or 30 years ago?”
Larry: First of all, I think it’s important to recognize that what we like to call the science of investing, or evidence-based investing continues to advance just like it does in medicine. We learn new things. And like all smart people, when you learn something new, you don’t cling to old ideas if you find that there are better alternatives. So we have now begun to add some alternative investments to our portfolios that didn’t even exist years ago until what is called an interval fund has been introduced by the SEC. So interval funds, which allow only partial liquidity redemptions, they’re required to allow you to withdraw 5% every quarter, at a minimum. Doesn’t mean you can’t get out more. But in aggregate, all investors can only get out 5% per quarter. So 20% a year. That allows mutual funds, through these interval structures, to invest in things that are less liquid. Investments that the Yale and Harvards and other sophisticated institutional investors have been able to invest in for decades, diversifying their risks. Because now they don’t have to have daily liquidity. For example, you can’t invest in a reinsurance contract because it’s one year if you need to price it daily. You can’t invest in consumer loans, which are 3, 5, 7 year alone if you need daily liquidity. So we have now added some funds to our menu of investments because we believe they provide unique sources of risk and return and help reduce the downside risk of portfolios. And we cover them in in the book as well. So the science of investing does advance and we try to stay with it. That’s a big part of my job.

Joe: Anything else? I think because we’re living longer, maybe before looking at, I know you had a chapter on annuities. There’s a chapter on reverse mortgages. There are different things that now might be coming into play that before, those were kind of last resorts or never buy an annuity because you’re just paying a huge commission, and both your firm and our firm are fee-only registered investment advisors, we don’t sell any products, but kind of looking at the full scale now of just trying to put this retirement puzzle piece together, I think some unique strategies are coming out, especially in the alternative space.

Larry: Yeah, your point on dealing with the longevity risk is an important one. As I mentioned I hardly knew anyone who was over 75. I think my oldest grandparent didn’t quite make it that age. So we didn’t worry about having to have a large pool of money that would say need to last more than five or 10 years, typically. But today we need to plan for retirement that could be 30 years. And that creates the risk of living longer than even we expect. And one of the ways to address that is an annuity. Now as you touched on, both our firms strongly advise against investing in variable annuities, which typically have high-cost investments inside and big commissions to the sellers. But there is a place for annuities that are immediate payouts called SPIAs, or my preference is to consider what is called the longevity annuity, so it might be you retire at 65. I should be budgeting for my life expectancy, maybe say to age 80 or 85. I don’t need to buy insurance against that. I want to buy insurance only for the unexpected. So I might buy a deferred annuity that starts paying out at age 85. And now because I got this big deductible if you will, the amount of capital I have to put up to buy that annuity is much, much smaller. And that alleviates a big hurdle that many have in buying annuities, they don’t want to give up that liquidity. So that’s a really important tool. I’ll touch on very briefly reverse mortgages. We have a chapter on that. Many people have a lot of their financial or net worth in their homes. Then they may, as they get older, they want to be able to live with dignity and not maybe go to a long term care facility, but they can’t afford to do so, because their assets are tied up in their home equity. A reverse mortgage is still a fairly expensive alternative, but it still may be a good one for those who want and have the priority of staying in their home and being cared for with dignity, can draw on that asset and create the assets to pay for the long term care they need at their home. And that’s one way of doing it without having to rely on your family or having to sell that home. That could be a really good option for many people.

Joe: Awesome stuff Larry. We talked on a couple of different topics and we only scratched the surface. Get his book, Your Complete Guide to a Successful and Secure Retirement. We can they find it, Larry? It just came out right?

Larry: Yeah it just came out a couple of weeks ago. You can go to Amazon.com of course, where you can buy everything, not just books. One thing for your readers to know: it’s always my pleasure, and my co-author Kevin Grogan, both of us, always happy to answer questions from readers. You just have to email me. And I’m sure if you want a copy you might just ask Joe and Al if they’ll get one for you.

Joe: Absolutely. Anyone that wants a copy of Larry’s book, you just ask Alan Clopine. He will definitely get everyone a copy of this phenomenal book.

Larry: Al’s got pretty deep pockets.

Al: I will do it.

Joe: That’s why we call him Big Al, Larry because he’s got big, big pockets. (laughs)

Al: I can hardly keep my pants up, right? Such a big wallet. (laughs)

Joe: (laughs) That’s why his back is all jacked up, he can’t sit down…

Larry: Well as always my pleasure to be with you, and hopefully we’ll get a chance to talk again next month. All right.

Joe: That sounds great Larry. That’s Larry Swedroe, folks.

Hey, it’s time to start thinking about filing your 2018 taxes. Download our free tax checklist by clicking  “Special Offer” at YourMoneyYourWealth.com, or find it in the show notes.

If you still need help and you’ve got a microphone on your computer, scroll a little further down on YourMoneyYourWealth.com, click “Ask Joe and Al On Air” button and record a question for Joe and Big Al to answer on the podcast. No mic? You can submit your question on a text form too. Speaking of taxes, here’s Jack from Atlanta with a mic and a capital gains tax question:

34:09 – How Can I Consolidate My Portfolio and Avoid Capital Gains?

Jack: Hello. I’m trying to figure out the best way to simplify my portfolio. For years I’ve been investing monthly into a number of mutual funds and taxable accounts at four different institutions. Now I want to combine it all under one fund before I retire early next year. How can I do this and avoid a high tax bill on capital gains? This is Jack from Atlanta, by the way. Thanks.

Joe: All right Jack, thank you for doing that. See, this a lot of fun, isn’t it?

Al: It is a lot of fun.

Joe: So he’s trying to consolidate, it sounds like.

Al: Right. He’s got four different institutions, so that’s like different custodians.

Joe: So he’s got an account at maybe Fidelity, T.D. Ameritrade, Merrill Lynch, Vanguard, whatever, and he’s like, “OK, well I’m getting all these statements, I’m looking to retire, I want to simplify my life. How do I go about doing this to avoid capital gains?” Well, I got a first thought here. You can always consolidate your investments and not incur any type of capital gain. So let’s say you want to hold all of your money at, I don’t know, Charles Schwab. Charles Schwab is not paying me for this.  (laughs)

Al: (laughs) Are you sure?

Joe: (laughs )Yeah Chuck and I go way back. All you need to do is just do a transfer. You can transfer all these different accounts and you can consolidate all of your investments under one roof if you choose to.

Al: Yeah. So that would make it much simpler, instead of four statements every month you have one statement from Charles Schwab and so like you say, you don’t sell the assets, you just transfer in kind. So it goes over in kind, the investment you had at Fidelity, you still have a Charles Schwab. There was no sale, there is no tax consequence.

Joe: A lot of times people think that that is diversification. Different custodians. I got $200,000 here, I got $300,000 there, I got $50,000 here and I got ten bucks over here – I’m diversified.  Well no. I mean sure, but when you’re looking at securities, you are an owner of that particular company. People I think might get confused with CDs. Yes, if you have millions of dollars and you want to buy FDIC insured CDs, then you have to use several different institutions.

Al: Right. That is true.

Joe: Not with securities, because Charles Schwab is like a grocery store. Charles Schwab and Fidelity, let’s say you’ve got Albertson’s vs… Kroger. Pretty good huh? I go grocery shopping a lot. (laughs)

Andi: And there is actually Kroger in Atlanta, so there you go!

Joe: And then there’s Vons.

Al: Yeah. I thought you would say Vons and Ralph’s. Those are the common ones here.

Joe: Got it. Or HyVee. (laughs) I think that’s one. So you go to that grocery store. You could buy beans at all of those grocery stores for the same price, roughly.

Al: And that’s that’s usually what you buy first? (laughs) “Does Ralph’s or Vons have a better selection of beans?”

Joe: Or bananas or whatever. If you want frozen chicken, you can just go there and all of those stores will have it.

Al: They got the same stuff.

Joe: So if I want to buy Apple stock, I can buy that Apple stock through Fidelity, through TD Ameritrade, through Charles Schwab, through E-Trade, through whatever.

Al: So I can buy a can of beans at Ralph’s and Vons. That’s not diversification. (laughs)

Joe: (laughs) Right! So if I’m running to Ralph’s and buy one can of beans and then I’m like, “oh, I need another can, but I can’t buy two cans here. (laughs) I gotta go to Vons.”  So what I’m saying is that you go to the supermarket and then you buy all of your stuff at that one supermarket, it’s very simple, it’s very easy. Same with your investments. You can go to one place to buy all of those different – like Charles Schwab doesn’t necessarily have more options for you than a Fidelity is my point.

Al: Agreed. So the other thing that you can do, so now you’ll end up with a bunch of different investments inside, to use your example, Charles Schwab. And maybe you don’t want that many investments, or maybe they’re not the best investments for you going forward.

Joe: Right. So now then you’re looking at selling, then that’s where the capital gains comes in.

Al: And what I like to do in a case like that is look investment-by-investment and look at the amount of gains in each investment relative to the investment itself. And then I come up with a gain percentage. Like if you bought Apple stock, let’s just say, you had the fortune to buy that 10 years ago, it might have gone up five times or whatever. So that’d be a tough one to sell from a tax standpoint. It might be the right thing to sell from an asset diversification standpoint but from a tax standpoint. So if you’re just thinking taxes, that would be at the bottom of the list. On the other hand, you might have an S&P 500 fund that you bought the beginning of last year and it’s actually down. So you could sell that one, you actually create a tax loss that would allow you to sell some other stuff. So you just kind of rank these investments and then come up with a strategy on how to gradually sort of get this reallocated without paying a ton of tax.

Joe: So put it this way: get a spreadsheet. Name your investments on one column of your spreadsheet. And then the second column of the spreadsheet, I would put market value. So if I bought Apple, it’s worth $100,000. Then the next column what you would want to put down is cost basis – what you purchased it for. So my first is what the investment is – Apple. The second column is going to be the market value – $100,000, and then my cost basis, let’s say it’s $50,000. Then on the third column it’s going to say “gain or loss.” And in that particular example, I have a $50,000 gain. So you want to do that with all of your investments. Just put together a simple spreadsheet, put the name of the investment, the market value, the cost basis, and then gain or loss. And then what Al, I think what you’re saying is just rank those from the lowest to the highest. So you could probably diversify – maybe you have 50 different investments. But maybe you could diversify out of 30 of them, and then your full gain on all those 30 investments might be only $20,000 versus if you did everything it could be $150,000.

Al: Right. Or it could be zero if you have some positions that are at a loss and you net that with other positions where you don’t have a lot of gain in it but it’s a lot of proceeds. So anyway, that’s really how you look at it – and I’m proud of you. That’s an accountant’s answer. “Here, let’s get a spreadsheet. Here’s column 1, here’s column 2.”

Joe: Right. But I think that would help someone quite a bit. Just because that’s what we do when we look at this is, it’s like, “OK well what is the most that we can diversify out with the least amount of tax?”

Al: Yeah. And then the next thing we do, once we’ve done that, it’s like now let’s look at your tax bracket. If you’re in the lowest tax bracket, call it the 10 or 12% bracket, your capital gains are taxed at zero. So maybe you can accommodate another $40,000 of gains, still stay in that low bracket, and pay zero taxes – at least zero federal taxes. You’ll still have to pay state taxes, probably.

Joe: So just some things that you can use there to help you. But another piece of advice, Jack from Atlanta, is it’s the beginning of the year, and I would worry more about the diversification. Don’t let the tax dog shake the… tail?

Al: Yeah. You always ask me and I always forget.

Andi: Don’t let the tail wag the dog.

Joe: Don’t let the tax tail wag the investment dog.

Al: (laughs) That sounds weird.

Joe: That sounds bad. (laughs) I don’t think that’s right. Anyway, so it’s the beginning of the year. I would look at, now I did my spreadsheet. I see what the total gain is. I probably would want to diversify out to have the right portfolio for me now that it sounds like you’re transitioning into an early retirement next year. You want to make sure that you have the right portfolio put in place. Then you want to tax manage the heck out of that account because stocks are volatile. This sounds like you’re in a brokerage account because you’re worried about capital gains. You could tax loss harvest along the way, and then that would offset any gain. You have the next 12 months to tax manage the overall account because you’re full gain or loss is not going to be totally calculated until December 31st.

Al: Yeah I think that’s exactly right. Investments are more important, the right investments for you are more important than the taxes. You always consider taxes, but that’s not really the first thing you look at.

Joe: All right, that’s it for us. We’ll see you again next week. The show is called Your Money, Your Wealth®.


Special thanks to today’s guest, Larry Swedroe. check the show notes for today’s episode at YourMoneyYourWealth.com for links to Larry’s latest articles on the markets and to find the link to buy Larry’s latest book, Your Complete Guide to a Successful and Secure Retirement.

Next week, millennial millionaire Grant Sabatier returns to the show to tell us about his new book, Financial Freedom: A Proven Path to All the Money You Will Ever Need. Click to subscribe to the podcast on any of the following apps: 

Google Podcasts |  Apple Podcasts |  Spotify |  Stitcher | Listen on YouTube

 Overcast | Player.FM | iHeartRadio | TuneIn 

Your Money, Your Wealth is presented by Pure Financial Advisors. For your free financial assessment, visit PureFinancial.com.

Pure Financial Advisors is a registered investment advisor. This show does not intend to provide personalized investment advice through this broadcast and does not represent that the securities or services discussed are suitable for any investor. Investors are advised not to rely on any information contained in the broadcast in the process of making a full and informed investment decision.