Since the market bottom in 2009, more money has likely been lost trying to short the S&P 500 and its various derivatives than any other single equity trade. The trade is so common and so routine that many market participants don't even think about how often it fails to work.
The appeal of the trade is obvious. With the market going up the great majority of the time there must be some pullbacks and dips along the way. People like to believe there is a natural ebb and flow to the trading action and that they can profit from the cycle of ups and downs. How can market timing be that hard?
The market has a positive tendency over the very long term that makes any short trade more challenging, but the biggest challenge of a short is that the market moves down in a much different manner than it moves up. Bear markets and corrective action are not just the inverse of bull markets and uptrends. Downside action occurs more abruptly and doesn't last nearly as long.
The market is an uptrend far more often than it is in a downtrend, which automatically makes shorting more difficult. Upside action tends to be more gradual and lasts much longer. A reactive approach works very well in bull markets because the moves play out over a longer period.
To be successful at shorting it is necessary to be more anticipatory. Generally, you must be willing to be on the wrong side of the trade for a while in order to really profit. Quite often, market players don't have much tolerance for holding a trade that isn't working and they tend to be shaken out far easier than a bull who is building a long position that isn't working.
When a trader does hit on a good short, it usually is a good idea to take profits very quickly as the bounce back up can be fast and furious. Trying to press a short position is one of the hardest things to do because of the strong inclination toward dip-buying that has persisted for so long. Just when it looks like the indices are about to crack they tend to find support as waves of massive liquidity come into play.
The action during the Covid-19 crises illustrates how the short trade required almost perfect timing in order to really profit. While the payoff was quite good in the early stages of the collapse, it didn't last very long and attempts to catch a retest have been extremely costly.
Despite the obstacles to making money with index shorts they are hard to resist as they so often seem like a smart trade. There is an intellectual appeal to trade counter to the foolish bulls who keep driving stocks higher without any real justification. How can compelling and logical arguments be ignored so often for so long? The short trade has to work at some point and I'm the smart person that is going to nail it.
Another problem with the index short is that it can be a great distraction. I've often found that I lose track of some of the good opportunities in individual stocks when I focus too much attention on trying to navigate the indices. Even when I have relatively little money on the line, it is easy to focus on the indices to the exception of other things that offer better potential.
A good index short can be quite satisfying and profitable, but the trade needs to be viewed as something that works rarely and requires careful management. Too many traders turn into serial top callers. They are stopped out repeatedly in their attempts to predict a turn, and when they finally do catch one they rush to cover quickly so that gains don't slip away.
Most market players would be better off if they ignored index trades and focused on other ways to make money.