TikTok’s Secret Sauce

What made TikTok such a success? A commonly given reason is that the app’s advanced AI is really good at figuring out what you want to watch. Many people say TikTok knows them better than they know themselves. Some users think of the algorithm as a divine force that guides them.

I’m not here to question people’s lived experience of TikTok. But there’s no truth to the idea that TikTok’s algorithm is more advanced than its peers. From everything we know—TikTok’s own description, leaked documents, studies, and reverse engineering efforts—it’s a standard recommender system of the kind that every major social media platform uses. Besides, recommender systems are a topic of furious research in computer science, and it would be implausible for TikTok engineers to have made a breakthrough that no one else knows about. Companies stay at the cutting edge by having their researchers and engineers participate in the open culture of knowledge sharing at conferences such as RecSys. A company walling itself off will only get left behind. For all these reasons, I don’t believe TikTok’s algorithm is its secret sauce.

Why, then, does TikTok’s algorithm feel so different? The answer has nothing to do with the algorithm itself: It’s all about the design.

Scrolling makes bad recommendations less annoying

Even with the best recommendation algorithm, only a fraction of your feed will be content that you truly enjoy. For example, the average ratio of hearts to views on TikTok is roughly 5%. People are just not that predictable. On YouTube, every time you select a video but then decide you don’t want to watch it, it’s an annoying process of scrolling to find another one. On TikTok, swiping up is so quick that you don’t consciously notice. So even if YouTube’s and TikTok’s algorithms are equally accurate, it will feel much more accurate on TikTok.

There’s a darker side to this. Eliminating conscious decision-making from the user experience means that videos that cater to our basest impulses do relatively well on TikTok, because people will watch these videos if they show up in their feed but won’t explicitly click on them.

Of course, YouTube has tried to clone TikTok’s design with its Shorts product, and Instagram with Reels. But those just don’t work as well. Why not? Let’s start with YouTube.

Vertical video from the start, not an afterthought

The scrolling paradigm, the vertical format, and short videos are all intimately connected. Each ingredient works well in the presence of the others. Swiping is much easier when holding the phone vertically (assuming you’re holding the phone one handed). The vertical format encourages short videos, especially of everyday moments, while widescreen is more suited to cinematic content. And short videos practically necessitate swiping: having to scroll and select a video, only to watch it for 15 seconds or less, would be incredibly annoying.

TikTok had all three from the beginning: short, vertical videos that you scroll through. YouTube has tried to butt in awkwardly, like Steve Buscemi saying “how do you do, fellow kids?” Unsurprisingly, YouTube Shorts creators are mostly YouTubers trying the new format because of how heavily YouTube has pushed it, and using it in an attempt to grow traffic to their main channel (which, unlike Shorts, is monetizable). Many of the videos on Shorts seem to have been originally created for the widescreen format but then cropped or squished for the new aspect ratio. 

Now let’s turn to Instagram.

Emphasis on the content, not subscriptions

TikTok’s algorithm treats each video more or less independently to assess its viral potential, caring relatively little about how many followers the creator has. This would be a trivial algorithm change for its competitors. What’s stopping them? Only the fact that their top creators, who collectively determine the platform’s fate, would rebel, because they stand to lose the fruits of the following they’ve built up over years. Stratechery explains that this is why Instagram got into trouble recently with its attempts to change its algorithm to compete with TikTok.

The de-emphasis of subscriptions means that there are fewer superstars, and fewer parasocial relationships. This, in turn, has kept creators from getting too powerful or quite as invested: TikTok pays them a pittance, and didn’t pay at all until 2020. The company has faced criticism for this, and it arguably takes advantage of creators. I don’t hold it up as a model to emulate. But the upside (to TikTok) is that it doesn’t have to worry nearly as much about angering creators as it experiments with its design and algorithm. 

What TikTok lacks in superstars it more than makes up for in its “long tail” of creators. The app is far more successful in converting content consumers into creators, in part because its creator tools are superior and more fun. Besides, TikTok has a trick up its sleeve that lowers the barrier to entry for new creators. As many people have observed, every video seems to be guaranteed an audience. How could that be?

Exploration over exploitation

A central challenge for any recommendation algorithm is the tradeoff between safe but somewhat boring recommendations that are similar to recommendations that worked well in the past (“exploitation”), and risky recommendations that are unlikely to be good but have a high payoff if they do turn out to be good (“exploration”). Exploration lets the algorithm learn about users’ interests that it didn’t know before. It may also enable a hitherto obscure video to go viral. 

TikTok is notable for placing a relatively high emphasis on exploration compared to other platforms. Every video seems to be guaranteed a certain minimum number of views. If the video performs well in that initial test, it will be served to successively larger batches of users. And from a user perspective, the algorithm keeps trying new topics even after it has found a set of topics that the user is interested in and will reliably watch. Exploration explains why there are an unending variety of incredibly weird niches on TikTok: the app manages to connect those creators to their niche audiences. YouTube has long had niche content, but TikTok seems to have taken it to a new level.

Again, the secret sauce is not the algorithm, because it would be trivial for other platforms to increase the preference for exploration. It comes back to the scrolling experience: TikTok is able to take risks because all it takes is a swipe. YouTube Shorts also emphasizes exploration, but so far, doesn’t seem to have been able to overcome TikTok’s huge first mover advantage, as well as its half-hearted adoption from the creator side. Instagram, meanwhile, has always pitched itself as being all about the creator-fan relationship, to the point of not allowing re-sharing so as to keep the feed focused on accounts the user follows. Reels is a big shift from that. It’s no surprise that an internal report found that most Reels users have no engagement whatsoever. 

Lessons

Recommender systems are extremely well studied in computer science, and relatively simple to understand, but public comprehension of how they work is poor. That has led to these algorithms being viewed as magic, demonized, or mythologized. (I hope to play a small role in changing this through my ongoing project on algorithmic amplification and society.) TikTok’s recommender system is not its secret: rather, it’s the design, which, of course, isn’t secret at all. More generally, in AI applications, the sophistication of the algorithm is rarely the limiting factor. The quality of the design, the data, and the people that make up the system all tend to matter more.

Despite TikTok’s design innovations being well known, other apps have trouble copying them because they were originally designed for a very different experience, and they are locked into it due to their users’ and creators’ preferences. This is a classic example of the innovator’s dilemma: Clay Christensen’s argument that incumbents tend to be held back by their own success—a lesson that’s been largely forgotten as “disruption” turned into a buzzword. As changes in technology make new user experiences possible, TikTok may one day be the struggling incumbent.

 

Thanks to Katy Glenn Bass and Roy Rinberg for feedback on a draft.

from Hacker News https://ift.tt/OHBUQeI