Opening Remarks

The Hidden Technology That Makes Twitter Huge

Twitter started with a simple form and through tens of billions of repetitions became a network unto itself

Illustration by David Parkins


Behind this week's coverIllustration by David ParkinsBehind this week's cover


While a tweet thrives in its timeline, among the other tweets, it’s also designed to stand on its own, forever. Any tweet might show up embedded inside a million different websites. It may be called up and re-displayed years after posting. For all their supposed ephemerality, tweets have real staying power.

Once born, they’re alone and must find their own way to the world, like a just-hatched sea turtle crawling to the surf. Luckily they have all of the information they need in order to make it: A tweet knows the identity of its creator, whether bot or human, as well as the location from which it originated, the date and time it went out, and dozens of other little things—so that wherever it finds itself, the tweet can be reconstituted. Millennia from now an intelligence coming across a single tweet could, like an archaeologist pondering a chunk of ancient skull, deduce an entire culture.

Twitter’s (TWTR) Nov. 7 initial public offering marks the San Francisco-based company’s coming-out party, the moment when it graduates from its South of Market beginnings and takes its place as one of the Internet’s most valuable properties, without ever turning a profit. What’s perhaps most remarkable about Twitter’s rise is how little the service has evolved from the original core concept of the 140-character tweet—which is to say, not at all. It’s tempting to view tweeting as silly and trivial, and Twitter itself as overhyped and overvalued. But there’s some sophisticated, supple, and even revolutionary technology at work. Appreciating Twitter’s machinery is key to understanding how an idea so simple changed the way millions of people advertise their existences to the world.

How do you look inside a tweet? It’s easy; the structure of a tweet is a matter of public record. Twitter, as a modern Web company, reveals to the world some of the technology it uses, in the form of an application programming interface—an API—which allows external software developers to build tools on top of the service, making it more widely used and thus more valuable for everyone.




This is where things get interesting. With just the places and the times, you can do some database work and learn when people in every corner of the world are in the strange, receptive state of social media engagement. Could be valuable! This information might tell you the best time to update a blog post or communicate with the most human beings at once, or when to release an advertisement. Maybe we learn that certain people tweet most assiduously right before leaving for work, a fine time for an advertiser to pitch them some orange juice or a new car to ease their commute.

This is the sort of combinatorial work that defines the modern Web: There’s so much data that there’s a very good chance that you will be the first of all humankind to find something interesting or unexpected. Whether you find something valuable is another question. But it’s surprisingly easy to become an expert in a very tiny niche as a developer—to become a world-leading expert in Android video or a specialist in Twitter geography—and charge accordingly for your services.

For all the possibilities of APIs, there are also limits. Another tweet field, “withheld_copyright,” if set to “true,” lets you know that a tweet is in trouble—that its content has raised flags and hackles over copyright. The text of the tweet, in that case, may be suppressed. The “withheld_in_countries” field can provide a list of the nations in which the tweet is banned. Another field has a telling name: “possibly_sensitive.” It’s set to either true or false. The field indicates whether a tweet links to potentially offensive things such as “nudity, violence, or medical procedures.” (If ever you wanted a snapshot of our world’s anxieties in three terms, there you have it.) As a user you can check a box on your profile so that the media you link to is automatically flagged this way. If you don’t, you risk having your pictures of your medical procedure marked as objectionable by an offended reader and thus put “in review,” the Twitter version of limbo.

A field like this indicates the inherent difficulty of managing an enormous platform like Twitter. The only way the company survives is if it can safely ignore most of what’s said on Twitter. If it had to use employees to monitor tweets, it wouldn’t last a day. But in order to attract as many users as possible, it must find ways to avoid horrifying them.

There’s a great deal of hedging in both the words “possibly” and “sensitive.” The end result is that Twitter is putting the moral burden on the user. One person’s art is another person’s smut, and Twitter is not going to decide which is which—nor is it going to force you to look at the stuff. This position is both somewhat noble for its acceptance of the range of human expression and also highly expedient, putting the responsibility back on the user: We told you the picture was “possibly sensitive,” so why did you look at it?

Much of the rest of the metadata contained in a tweet is familiar: The number of times people have starred, or “fav’d,” a tweet, the number of times it’s been retweeted. The value for “user” contains a whole huge bundle of stuff—the user’s name, a link to an avatar image, the user’s number of followers, the number of people the user follows, and whether the user is “verified” and deserves one of those blue checkmarks. It’s a pretty full portrait of an individual, given that this portrait is attached to every tweet.

From a single tweet and with no other information, you can extract a sense of social influence—how big a voice an individual has, the number of people they reach, the number of people who engaged with this particular tweet. Tweets themselves are just regular text (although text on a computer is anything but regular; there are dozens of abstractions that make it possible for an “a” to appear on a screen—but it’s safe to gloss over that). Here it is, 140 characters, a plain little beastie. You might be fooled into thinking there’s hardly anything there.



Another of Twitter’s discoveries was that mobile phones could work as a broadcast platform. This was something of a miracle of timing: A massive proportion of its traffic today comes from mobile devices. The short length of the tweet was perfect for celebrities in limousines to communicate with thousands, and later millions, of followers. The tiny payload of tweets could be easily jammed into narrow mobile phone data streams, giving people a real-time flow of information.

Twitter started with a very simple form—a single box on the Web with a limit that kept people from inserting too many characters—and through tens of billions of repetitions became a network unto itself. It’s embedded within the Web’s culture, but it’s also so large that it’s separate from the rest of the Web. The technologies that go into building Twitter today are not the same technologies one uses to build a typical website. The tweet is the social network’s building block in the way that Web pages built the Web in the mid-1990s.

Twitter’s founders recognized that encouraging people to use a very small number of very tightly controlled forms, billions of times over, creates huge, deeply interconnected, highly creative, and potentially profitable new spaces. It’s as if you could, with exactly the right kind of bricks, build a skyscraper that was infinitely tall. Twitter, like its half-sibling Facebook (FB), became so powerful that people now use it to log on to other websites; your Twitter identity is a major component of your Web identity. And today major news properties and blogs increasingly look like Twitter: infinite streams of data, tags, and voices.

The history of Twitter, as it’s been told so far, doesn’t offer a moment where various youngish men rise up from their desks and run naked through San Francisco yelling “Eureka!” It was created, like most things, in meetings. Somewhere in those meetings, Twitter uncovered a latent aspect of human life that had never before been so clearly articulated and turned it into a product that has altered, to various degrees, hundreds of millions of lives. That much of what is tweeted is trivial or silly is an obvious truth—but that’s not Twitter’s fault. That’s on us.

Ford is a programmer and the creator of

The Good Business Issue

Companies Mentioned

  • TWTR
    (Twitter Inc)
    • $37.6 USD
    • -0.01
    • -0.03%
  • FB
    (Facebook Inc)
    • $80.78 USD
    • 0.01
    • 0.01%
Market data is delayed at least 15 minutes.

Sponsored Links

Buy a link now!

blog comments powered by Disqus