For all the glamour of living an early retirement lifestyle, there are plenty of negatives I’ve come to discover since I permanently left my job in 2012.
I know why we revert to our baseline state of happiness, no matter how much freedom and money you have.
Let’s go through some of the negatives of retiring early now that I’m a grizzled veteran.
The downside of retiring early
1) You will suffer an identity crisis for an unknown period. When you’ve spent at least a decade working in a profession, you’ll find it incredibly jolting to no longer be identified as the person who is a marketing expert, an investment professional, or the management consultant who can figure out how to optimize a business. It’s only after you leave your job do you truly realize how wound up you were in your profession.
Your identity crisis may last as short as three months or it might last for years. It all depends on how wrapped up you were in your job, how long you spent getting educated after high school, and whether you have a clear plan postretirement. Doctors are some of the people who suffer the most after leaving their occupations. Conversely, high school graduates who somehow struck it rich with a product or an invention seem to adjust much easier in postretirement life.
Job titles can be incredibly addictive. Why else do people get so depressed when passed over for promotion? Why else do people try so hard to get promoted sooner and faster than everybody else? Do not underestimate the importance of being a manager, director, vice president, or even a C-level executive.
After all, the most common question people ask when they first meet each other is: What do you do for a living? And if you tell them you don’t do anything for a living, well then, you might just feel like a sheepish loser. You’ll want to try to explain yourself, but by then, your three-second first impression will no longer hold the other person’s attention.
What happened to me: After working in the Asian equities business for 13 years, it felt hollow to no longer have my Executive Director title or be identified with my investment firm. I felt sad that I could no longer go to Asia for conferences or with clients. For so long, taking a business-class trip to Hong Kong, India, China or Taiwan was part of my quarterly routine. Shallow as it may sound, it felt special to have priority boarding. I felt important when clients would entrust me to show them around in a foreign land.
For the first year after leaving my job, I wondered how the business was doing without me. Could they really survive without my expertise? After all, I was there for 11 years. Surely, they needed my relationships. But after months went by with no email or phone call from my old firm saying they wanted me back, I had to come to terms that I was no longer important to them.
I wanted to believe that my position meant something to the firm and to the people that I serviced. But at the end of the day, the person I trained to replace me as part of my severance agreement, was good enough. And because he was good enough, I concluded that I was no longer any good.
This ego hit took me a full year to get over.
Read: Why early retirement is all it’s cracked up to be
2) You will be stuck in your head. When you suddenly have an extra 10 to 14 hours a day of free time, it’s very difficult to optimize your time wisely.
Your productivity will suffer in retirement. You will no longer feel motivated to achieve great wins. As a result, you may slowly start to get depressed. Only after some really deep soul-searching and some, “what am I doing with my life?” questioning will you begin to organize your time better and become more productive.
Your mind can be very dangerous because it can always second-guess your actions. Did I retire too soon? What if I run out of money? What if people think I’m a loser? What if I can’t ever get back into the workforce if things go wrong? When you have a lot of time to think, your doubts go on and on.
Perhaps one analogy is to compare being stuck in your head with Locked-in syndrome. LIS is a condition in which a patient is aware but cannot move or communicate verbally due to complete paralysis of nearly all voluntary muscles in the body except for vertical eye movements and blinking. This could be one of my worst nightmares. Retiring early may render you inoperable for a while.
What happened to me: Because I left work at age 34, I was worried for about the first two years whether or not I had made the right choice. No rational person leaves a well-paying job to be unemployed in their mid-30s. Your late-30s is when you start to finally make good money. And by the time you reach your 40s, you should be at your maximum earnings power.
During my first year of early retirement, to the outside world I proudly proclaimed I was retired from a career in finance. But on the inside, I was second-guessing my decision to leave. Because of my uncertainty, I decided to do some part-time consulting with a financial technology startup for about 20 hours a week. It was a great way to distract my mind from all my fears, earn some side income, and replug myself into society. I also kept in touch with multiple banks until my Series 7 and 63 licenses expired.
Finally, I dived deep into my writing on Financial Samurai. Writing has always been my most cathartic way to deal with any uncertainty or problems I might have. For example, now that I have a son, I’ve been worried about whether our roughly $200,000 a year in passive income is enough to support a family of three if he doesn’t win the San Francisco public school lottery system. It’s taken almost 20 years for me to generate this passive income level, and it still doesn’t seem like enough.
Given this worry, I did a deep dive budget analysis for a family earning $300,000 a year, and it sure seems like we need to earn $100,000 more to maintain our quality of life in San Francisco. Alternatively, we can always move to a lower cost area of the country or world.
Read: People may be missing the point of early retirement
3) People will treat you like a weird misfit. Whether it’s because retiring early is unconventional or because people are secretly jealous you aren’t grinding away at a day job, people won’t give you the same amount of respect as working-class citizens. After all, if they can’t describe what you do for a living, then they can’t pigeonhole you into an archetype that is comfortable for them.
Having a job means you are a productive member of society. If you retire at a young age, people will assume you are simply slacking off and not paying any taxes. They’ll sometimes look at you as a leech they want to flick off.
Further, if you are an outcast, then you won’t be invited to parties or events that other working people always get to attend. You’re simply not top of mind to them. If you are an extrovert, early retirement will be much more difficult than if you are an introvert.
What happened to me: After the first year of early retirement, I no longer told anybody I retired early. Instead, I told anybody who asked that I was a writer, a tennis teacher, a fintech consultant, or simply in between jobs. Before that, I think a lot of people just assumed I was a trust fund baby who did not have to work. And the last thing this middle-class guy who went to public school wants to be known as is a trust fund baby.
My favorite time of the year was during the winter holidays. I loved going to all the holiday parties and getting tipsy with fellow revelers. Now, I get invited to zero holiday parties because I don’t work for anyone. Nor do I get invited to client holiday parties either, even though I have several partners who are based in the San Francisco Bay Area. It may sound silly, but having a drink with good people with shared interests really means a lot to me.
It takes a lot of effort to build new social networks if you aren’t part of a larger organization. There is no weekend cookout a colleague is hosting on Labor Day Weekend to attend. I’ve had to participate in various meetup events to find new people to hang out with. So far, my social network only revolves around tennis and softball. But even then, it’s not like I’ve found buddies who will come over and just chill in the hot tub over a beer or anything.
4) You’ll be disappointed that you aren’t much happier. So many people think that once they achieve financial freedom or leave a job they dislike, they’ll suddenly be permanently happier. The truth of the matter is, your elevated happiness will only last at most three to six months. Eventually, you’ll revert to your natural state of being.
Think back to your high school or college days when you didn’t have any money compared with now. I’d venture to guess you were just as happy, if not happier when you were a broke college student.
Having the freedom to do what you want is priceless. But you will eventually take your freedom for granted like the air you breathe. On the days you feel angry or sad, you will start questioning what the hell is wrong with you since you’ve got more than the average person. You’ll feel stupid for feeling unhappy when there are literally hundreds of millions of people in the world wondering whether they’ll have enough to eat the next day.
You think, if I can’t be happy when I’m financially independent, surely there must be something seriously wrong with me. And you could be right! Can you imagine being unhappy as a Norwegian? Norway is perpetually ranked as one of the top five happiest countries in the world.
What’s going on with me: I thought I’d be much happier not having to report to a micromanager boss I did not respect. But my increased happiness was fleeting and only lasted for about a week before I was back to my regular self. Instead, my happiness was weighed down by months of uncertainty on whether I had made the right move to leave my job. It was only after about two years did my doubt finally start to dissipate.
Although corporate politics no longer upset me, other things end up filling the void. For example, drivers who decide to double park on a busy street in rush hour traffic really bother me now. So do dog owners who let their dogs poop in front of my house and don’t pick up after them. In the past, I could only allocate a small amount of annoyance to such incidences.
Instead of being permanently at a happier level, I’m simply no longer as annoyed or as angry at things as frequently. Further, the volatility around my steady state of happiness is lower. In other words, I’ve mellowed out. That said, don’t offend me because I still enjoy a really good fight.
5) You constantly wonder whether this is all there is to life. Retiring early is like finishing up your favorite longstanding TV show. You’re glad there’s a conclusion, but you’re also sad that it’s over. You hope to find a show that’s as good or better, but there are no guarantees.
Most of us spend 13 years going to grade school so we can spend four years in college to get a decent job. Then we spend decades trying to earn and save money to provide for our family and then one day retire by 65. With good luck, we’ll live for another 20 years to enjoy all the fruits of our labor.
When you retire at a much earlier age, you are constantly left wondering what’s next. You are mentally twiddling your thumbs waiting for the next big thing while your close friends are all at work. Early retirement can get extremely mundane and boring because you have nobody to spend time with.
As a result, you’re repeatedly forced to will yourself into action. This constant self-starting attitude can become extremely trying to the point where you long to rejoin the workforce and be told what to do.
What’s going on with me: I probably drove my wife nuts during the first two years of early retirement because I constantly told her I was bored. Only boring people get bored right? Wrong. Everybody gets bored at some point. When you’re working, you don’t have time to get bored because you’re working. There’s only so much tennis, golf, and softball I can play before my knees break apart. There are only so many churches to visit in Europe before they all start looking the same.
She used to have vacations from me because I would be away traveling for work every month. Now she was seeing my cherubic face every single day. It’s a good thing we had three bedrooms at the time. Otherwise, I’m pretty sure we’d both have gone crazy from seeing each other so often.
It was only after our son was born in early 2017 that I felt a renewed sense of purpose. Before my boy, I felt my purpose was to help educate as many readers as possible about personal finance to one day be free. After my boy was born, my purpose has expanded to keeping Financial Samurai running long enough to teach him about operating an online business out of fear he may have a tough time getting ahead. In addition, I now need to live long enough until he finds someone who loves him as much as I love my wife.
I don’t think I’d be able to die in peace if there’s nobody to replace his mom or me. As a result, I’m exercising more, eating healthier, and meditating longer.
Early retirement is great, but it doesn’t solve everything
It might sound like I’m depressed. But I’m not. I’m simply highlighting some of the negatives you will probably go through if you decide to leave the workforce early. The more extroverted you are and the higher your position, the more you will have difficulties making the early retirement adjustment.
Having the freedom to do what you want cannot be overstated. However, your mind will play games with your spirit during the first few years after leaving work. Some people won’t be able to handle early retirement life and will go back to work.
Just know that with enough conditioning, you will eventually embrace your freedom. Nobody I know who retired from corporate life early stayed retired. You will find your purpose. Once you do, you will take steps, such as building passive income, to ensure you remain free forever.
This column originally appeared on Financial Samurai. It has been adapted and published with permission.
Sam Dogen started Financial Samurai in 2009 to help people achieve financial freedom sooner. He spent 13 years working in investment banking and retired at age 34 in San Francisco. Everything Dogen writes is based on firsthand experience.