I finally faced up to something I had been dreading.

After years of procrastinating, I logged on to my retirement account. Just working my way through the rigmarole of retrieving lost passwords and locating my investments was bad enough. But once I started to examine my portfolio, I began to feel anxious. Some of my money was in mutual funds, but I had no sense of how I chose them. And the rest of my money was in cash, earning virtually nothing; how had I let it sit there for so long? With trepidation, I began flicking through screen after screen of investment choices, each one filled with a seemingly endless stream of jargon.

I seemed to have fallen into a recurrent nightmare, one in which I am taking a final exam in a class I never attended and a subject I don’t understand. This was even more embarrassing: I am, after all, a trained economist. When new acquaintances learn what I do for a living, they routinely ask, “So how should I invest my money?” I wish I knew.

But this is an ordeal for nearly everyone, whatever their training. No wonder many of us avoid thinking about long-term investments, especially for retirement. A recent survey by Charles Schwab found that most of us spend twice as much time choosing a car as we do choosing investments that are supposed to support us for years.

A relaxing retirement is supposed to be the reward for a lifetime of work, and yet it seems we must pass an entrance exam to reach it.

I’ve been wrestling with how we got to this point. Why is investing so complicated? And what should we do about it in our own lives, and as a society?

Investing isn’t what most of us do for a living. When a mutual fund company asks me what I want to invest in, it seems like the wrong question. It’s like a taxicab driver in a new city asking me, “Which route do you want to take?” Don’t ask. Just take me there. I paid the fare. Is it really my job to figure out the route to my hotel?

I want to reach my retirement with a nest egg that allows me to maintain my current lifestyle and to travel a bit. If I save what I’m supposed to, I would like someone else to figure out the best route.

Employer-provided pensions still do this for you, and if you’re lucky enough to have one — and it’s solid — congratulations. For most of us, though, they are not an option: Pensions have been disappearing, for many reasons. And many of the remaining ones are underfunded and vulnerable. Expanding employer-provided pensions seems to be a non-starter, much as we might like to see that happen.

Can’t the market fix things? If individuals are forced to choose their investments, why doesn’t the market make these choices easier?

By all accounts, the world of mutual funds, which is the basis for many retirement investments, is a competitive market. At the end of 2014, there were more than 20,000 mutual funds. On top of that, there are hundreds of exchange-traded funds. While many of these funds sit in a few big-name companies, one would hardly call this a monopolized industry. We’ve got plenty of choices.

The market seems to work well for consumers in other industries. The producers of smartphones, for example, compete fiercely to make simple and elegant user interfaces. If they can make it a breeze to interact with billions of lines of code, why can’t somebody simplify the alchemy of finance?

What distinguishes the market for investments is our inability to judge whether we have chosen well. Once I’ve used a phone for a few weeks, I can tell whether it was worth the money. By contrast, I may not know for decades (if ever) whether an investment was wise or foolish. Does a low return signal a prudent choice or a missed opportunity? And by the time the answer becomes clear, it’s already too late. You’ve got to live with your bad choice.

What’s more, the invisible hand of competition does not do well by consumers with limited understanding. Rather than eliminating biases, markets often cater to them. For example, many consumers choose a mutual fund by looking at last year’s returns, despite warnings that they should not do so. This creates a winner-take-all situation with the highest-performing funds getting most of the investors. You might think this encourages funds to produce higher returns, and that might seem to be a good thing.

But what it actually produces is a perverse incentive for fund companies to take risks. That’s because investors often choose what to do with their money once, and leave it there for a long time. Faced with that reality, the most profitable strategy for a mutual fund company can be to simply take risks in the hope of gaining high short-term returns. Win this high-return lottery and you can draw many investors from whom you will earn fees for decades. Big families of funds can start new funds regularly, each with a different risk strategy. For the companies, it’s like buying many lottery tickets. It doesn’t matter for the company if many of the funds are clunkers as long as they end up with a few near the top of the performance rankings. For consumers of these funds, though, it’s a losing proposition.

Educating consumers to be better purchasers seems a sensible idea, but an example from recent history illustrates the problem with that. For a long time, the simple investment advice given to consumers has been “buy an index fund.” Index funds are such standardized products — mirroring the Standard & Poor’s 500-stock index does not require much management — that just about all of them were initially low cost while offering wonderful diversification.

Consumers have been buying index funds, and the market has responded by providing hundreds of them. Nearly all E.T.F.s are index funds.

But the market has also responded by charging high fees for this standardized product. In 2004, Ali Hortacsu and Chad Syverson, economists at the University of Chicago, found that index funds had as much variability in fees as their more labor-intensive actively managed counterparts. And these fees are nothing to be scoffed at — paying 1 percent more every single year in fees can compound over a lifetime to noticeably lower returns.

Initially sound rules of thumb create niches for bad products — like expensive index funds — that dilute the value of the advice. In short, the market responds to education with attempts to subvert it. Any simple advice tends to have an Achilles’ heel, and markets are good at finding it.

Why not rely on financial advice? There certainly seems to be a yearning for good advice. Last year, I posted a message on Twitter. “Want financial advice?” it said, adding that “@haroldpollack gives it free … in index card form.” I included Mr. Pollack’s card containing nine pieces of simple advice, such as, “Never buy or sell an individual security. The person on the other side knows more about this stuff than you do.”

The popularity of that tweet shows the thirst for advice.

But while there is a large market for it, financial advice poses many risks.

In one study that I conducted with the economists Markus Noth at Hamburg University and Antoinette Schoar at M.I.T., we tried to quantify the quality of advice on the market. We did this by sending mystery shoppers to financial advisers. Our shoppers received very bad advice, by any measure. They were told to put their money into highly nondiversified portfolios that were also expensive. That’s the worst of both worlds: high risk and low returns. Perhaps most shocking was when our shoppers started with portfolios that were well diversified and inexpensive. Even in those cases, they were told to switch to options that were clearly worse.

But this is not meant to impugn all advisers. The study examined advisers who did not charge clients directly. Their advice was “free,” but under current rules, their advice only had to meet a very low standard — it only had to be “suitable” in a broad sense of that word. So it is O.K., legally, for these advisers to steer you to products that they profit from.

Advisers who explicitly charge for their advice are held to a higher standard. The Department of Labor has recently proposed a rule to require what is known as a fiduciary standard for advice on retirement accounts. At the moment, though, the rules are loose.

If you’re seeking advice, ask yourself one simple question: “If I’m not paying for this adviser, where are they getting their money?” If you don’t know the answer, you may have a problem.

Can we do anything? Regulations requiring that all advisers have a fiduciary duty to their advisees would be a terrific start. But we can do more.

Even if we receive good financial advice, following it can be hard: Saving more and consuming less is on par with going to the gym more and eating less. Some people can do it easily. Many can’t.

A recent study from Ideas42, a behavioral science nonprofit that I helped to start, showed that timely reminders and programs with good default choices can help ensure that good advice is followed. People in well-designed experimental programs who are given sound advice and plenty of reminders tend to save more. We can start more such programs. And in the investment market itself, better standards could make our lives better. When you buy a USB drive or a Wi-Fi router for your computer, for example, you may not really know how they work but you can assume that they will work with each other, because the industry has adopted standards to make sure that this happens. Imagine the nightmare of shopping for electronics products that were incompatible, or that had different names. Standardization is what allows uninformed consumers to shop intelligently for complex products.

For mutual funds, better standards and better labels could simplify choice. For example, suppose the label “Standard S&P 500 Index Fund” was widely used to mean that the fund satisfied certain criteria about transparent pricing, low management fees and limits on trading costs. Consumers buying this fund would know what they were getting.

Firms would be free to introduce new, innovative funds, just as an electronics company can introduce new products that are not USB compatible. The restrictions would only prevent products from claiming to be something they are not.

What’s to be done now? You are on your own, unfortunately.

But I can tell you what I did. I chose a target retirement fund. With these funds, you simply decide on a target retirement year. As you age — and move closer to the target date — these funds automatically shift their holdings from riskier stocks to less risky bonds.

As with the simple advice “buy index funds,” this is also imperfect. If everyone starts buying target date funds, or life-cycle funds, how long before these terms attract companies that create expensive and not-so-effective products? In fact, it may have happened already. As Pierluigi Balduzzi and Jonathan Reuter found and reported in a recent paper, some target date funds may already be taking an expansive view of what it means to invest in equities: They seem to produce a wide range of returns and risks across the year. As a result, I was careful to verify a few things about the fund I chose. First, I made sure that the stocks held by the fund were contained in a simple, broad-based index such as the Standard & Poor’s 500-stock index or the Russell 2000. Second, I verified that the costs were very low: The annual fees came to less than 0.2 percent. If the fees aren’t obvious, or if they are much higher than this, watch out.

Even armed with this knowledge, there were hiccups. First, choosing the retirement date raised existential quandaries. Since these funds come in increments of five years (2025, 2030 and so on) I ended up choosing two dates to leave me with mental wiggle room. (“I’m not that close to retirement,” I told myself, choosing one fund. “I might want to retire earlier,” I said, choosing another.)

Then I developed a second problem: I was afraid that picking only one kind of fund was imprudent. The economists Shlomo Benartzi at U.C.L.A. and Richard Thaler at the University of Chicago refer to the urge to buy many kinds of funds as “naïve diversification.” One fund can be enough if it’s diversified: What feels like one basket is really a great many baskets.

Overcoming this final hurdle was satisfying. I experienced the kind of euphoria usually reserved for major accomplishments.

The biggest lesson, I realized, was one that faces me all of the time: The biggest cost of fear is paralysis.

It is easy to make a mistake in choosing investments. But in an effort to avoid an error, I had been making an even bigger error. As I procrastinated, my money was uninvested and earning zero returns.

That, surely, is not the path to a happy retirement.

Source: The New York Times