Les neuf premiers arguments sont maintenant convenus, mais le dixième mériterait d’être exploré par les chercheurs et les journalistes. Dans la grande quête de sens qui nous fait imaginer des cosmogonies et des panthéons, la préfiguration d’une intelligence artificielle généraliste qui dépasserait les capacités humaines me semble être la religion du siècle. Les ingénieurs sont les grands prêtres qui annoncent la deuxième venue du messie, la singularité serait le jugement dernier qui trierait le bon grain technophile de l’ivraie technophobe, la peur de la grande méchante IA n’est qu’un nouveau millénarisme.

Lanier va plus loin encore, en parlant d’« impérialisme métaphysique », et d’une religion des zélotes drogués aux likes et aux retweets. « If you buy into any of this stuff, explicitly or just through practice », dit-il, « you cannot even call yourself an atheist or agnostic. You are a convert. » Les sacrosaintes conditions d’utilisation pourraient être vues comme un catéchisme, et sont d’ailleurs aussi peu lues. En définissant les prescriptions et les interdictions, elles définissent un cadre moral et même spirituel :

When you use BUMMER, you implicitly accept a new spiritual framework. It is like the EULA agreement—the user agreement—that you clicked “OK” on without reading. You have agreed to change something intimate about your relationship with your soul. If you use BUMMER, you have probably, to some degree, statistically speaking, effectively renounced what you might think is your religion, even if that religion is atheism. You have been inducted into a new spiritual framework.

Alors que je parle au futur, en espérant pouvoir dessiner un avenir différent, Lanier parle au présent. Après tout, Google prétend pouvoir défier la mort et les dieux, et posséder jusqu’à votre âme :

Google’s director of engineering, Ray Kurzweil, promotes the idea that Google will be able to upload your consciousness into the company’s cloud, like the pictures you take with your smartphone. He famously ingests a whole carton of longevity pills every day in the hope that he won’t die before the service comes online. Note what’s going on here. The assertion is not that consciousness doesn’t exist, but that whatever it is, Google will own it, because otherwise, what could this service even be about?

Passionnant.

Notes

La publicité programmatique comme machine de « modification comportementale continue » :

Now everyone who is on social media is getting individualized, continuously adjusted stimuli, without a break, so long as they use their smartphones. What might once have been called advertising must now be understood as continuous behavior modification on a titanic scale.

La rentabilité de la négativité par défaut :

A corpus of academic research compares the powers of positive and negative feedback, but that is not the key question for the design of commercial social media platforms, which are primarily concerned with reducing costs and increasing performance, thereby maximizing profit. Whether or not positive feedback might in theory be more effective in certain cases, negative feedback turns out to be the bargain feedback, the best choice for business, so it appears more often in social media.

Partir faute d’autre choix :

One of the main reasons to delete your social media accounts is that there isn’t a real choice to move to different social media accounts. Quitting entirely is the only option for change. If you don’t quit, you are not creating the space in which Silicon Valley can act to improve itself.

Donald Trump en cinq phrases :

Ordinary people are brought together in a setting in which the main—or often the only—reward that’s available is attention. They can’t reasonably expect to earn money, for instance. Ordinary users can gain only fake power and wealth, not real power or wealth. So mind games become dominant. […] With nothing else to seek but attention, ordinary people tend to become assholes, because the biggest assholes get the most attention.

Répétons encore et encore que l’outil n’a pas de volonté propre :

The problem isn’t any particular technology, but the use of technology to manipulate people, to concentrate power in a way that is so nuts and creepy that it becomes a threat to the survival of civilization.

Une « attaque de la culture par déni de service », quelle merveilleuse manière de présenter les choses :

Whatever you can do, bots can do a million times while you blink. Fake people are a cultural denial-of-service attack.

La rupture du contexte :

Speaking through social media isn’t really speaking at all. Context is applied to what you say after you say it, for someone else’s purposes and profit.

Une vie sans likes vaut la peine d’être vécue :

Here’s a non-geeky framing of the same idea: What if listening to an inner voice or heeding a passion for ethics or beauty were to lead to more important work in the long term, even if it measured as less successful in the moment? What if deeply reaching a small number of people matters more than reaching everybody with nothing?

Une certaine vision de l’enfer :

Can you imagine if Wikipedia showed different versions of entries to each person on the basis of a secret data profile of that person? Pro-Trump visitors would see an article completely different from the one shown to anti-Trump people, but there would be no accounting of all that was different or why.

La boite noire algorithmique bloque l’empathie :

I don’t like Fox News in the United States, for instance, because I think it’s too paranoid, partisan, and cranky. But I watch it sometimes, and it helps me understand what other people who watch it are thinking and feeling. I cherish that ability. I have no way of seeing your social media feed, however. I therefore have lessened powers to empathize with what you think and feel. We don’t need to all see the same thing to understand each other. Only old-fashioned authoritarian regimes try to make everyone see the same thing. But we do need to be able to peek at what other people see. Empathy is the fuel that runs a decent society. Without it, only dry rules and competitions for power are left.

De la monétisation des données personnelles :

One way is to directly monetize services such as search and social media. You’d pay a low monthly fee to use them, but if you contributed a lot—if your posts, videos, or whatever are popular—you could also earn some money. A large number of people, instead of the tiny number of token stars in the present system, would earn money. (I acknowledge, of course, that there would have to be a way of making services available to those who couldn’t afford to pay even a small fee.) […] In the argument about how BUMMER is making you into an asshole, I suggested that bringing in some level of reward system beyond clannish mind games can inspire dignity in online relations. I suggested LinkedIn as one example of how economic engagement, instead of purely social engagement, can have a civilizing effect. This hypothesis needs to be tested more, but it is possible that when we enter into a new era in which people are paid for the value their data brings to the online world, then that world will become less dark and crazy.

L’intelligence artificielle comme excuse pour les mauvais ingénieurs :

AI is a fantasy, nothing but a story we tell about our code. It is also a cover for sloppy engineering. Making a supposed AI program that customizes a feed is less work than creating a great user interface that allows users to probe and improve what they see on their own terms—and that is so because AI has no objective criteria for success.