Choice trees have more than a trunk

              I hate modern algorithms in my life. The only time I notice them (and I’m sure there are a lot of good ones I never notice) are when they’re limiting my choices in the name of personalisation. When I visit YouTube, for example, I’m flooded with junk I don’t want to watch. How do they regard this ability to hide interesting videos as an asset? I see the same on Netflix or Amazon. Amazon shows me the exact same thing I’ve just bought, or others like it. How many boxes of marzipan should I give my sister as a Christmas present? 1 or 20? Netflix tries to offer similar things in categories, but the categories are clogged up with videos I have already seen: it thinks that these are more relevant to my watching habits that something entirely novel. I’m trapped in a bubble: my last video looked like the one before it, so I’m offered the chance to watch the one I watched before, and then re-suggested the second one, ad infinitum.

              So I do. I re-watch good things and bad. I watch one marble run video on YouTube, because I really wanted one when I was a child, and then my homepage is flooded with dozens of marble races (strangely entrancing that they are). The default setting is too tiresome for me to change. I wouldn’t even know how to search for something interesting.

              Defaults affect us the world over. Someone offers you a free trial, but then wants your bank details. I remember that the first time this happened to me I was shocked. “Why do you need bank details if it’s free?”, I asked. So that they can set up a subscription and it’ll run unless I cancel. I ran away from that; I didn’t want a subscription. I wanted a free trial. My youthful naivety is long gone now: we’re all accustomed to such sales tactics. It’s just the way world works. But is it the best way?

              Imagine all the people having money and time leeched from them because they’ve lost track of all these defaults that they need to change… changes that are made deliberately difficult. I’ve complained to numerous companies over the years: van rentals, cancelled flights etc. No complaints process was easy. I can send the complaint, but then I get a robotic reply that doesn’t address the problem… we have to correspond, time goes by, nothing happens, I have to pursue it and forget… and then my complaint has gone away. No refund, bad service justified.

              This is just another way that people try to control our decisions. It’s not advice or nudging: it’s outright manipulation.

              So what would be better? Well, if we return to Amazon or YouTube, it’s easy to imagine what better would look like. Amazon should know when I’ve ordered something and remove it and similar items from display. Maybe it works this way to make money: lots of gullible idiots buy another copy, just in case, or because they forgot they’d already bought it. Or in some rare cases, because they genuinely could use a similar item at the same time. And then, just when Amazon forgets I ordered something, it should be remembering: when I’ve had a chance to use up, break or lose the last order, or decide it wasn’t good enough. That would be helpful. And if I don’t want something, it’d be lovely to tell Amazon to forget that I ever even looked at something like that: to remove the high ranking it has for me.

              I need something similar on YouTube. The flood of videos about X-box games because I once made a mistake and watched a video I thought was a general comment about computer games was a waste of space. I don’t own an X-Box. But I watched another one from that channel, so I must be interested! Well, of course I watched another one. I was looking for some mild diversion while chopping vegetables, and that was all YouTube was showing me. I need a way of deactivating the ranking for those videos: of down-rating them, to say ‘that was just noise, don’t treat it as hugely meaningful signal’. But without saying ‘this is something I would never watch: never show me this again’.

              What I need, in fact, is a tree structure. Broad categories, all available, which open into subcategories, like a folder system on an old computer. Only because we’re modern, we can use metadata and have some things show up in multiple places, but maybe with different rankings. An item might be 100% comedy, 50% news. So when I go to news, I might find pure news sources first, but see the option of a satirical report.

              I don’t want a helpful machine to learn that when I look for news, I really mean comedy. Maybe I really mean news but nothing was interesting enough, or maybe I’m trying to be more informed but the comedy seemed diverting enough this time. In essence, what I say is what my thinking, conscious brain has decided, and what I do is this decision, corrupted by temptation, emotions and the limited options available. If algorithms learn to bypass my thinking free will and feed the beast of my emotional, whimsical self, then they are not helping me: they are destroying what makes me ‘me’.

              Netflix does the same. I can’t simply say ‘reduce the ranking of things I’ve watched by 90%’, or ‘nothing I’ve watched before should be visible before scrolling’. Instead, 90% of my screen is full of things I’ve seen, helpfully displayed as most similar to the things I’ve watched. I assume that this is to hide the lack of content, and because they have found that when faced with the choice of re-watching something or spending 30 minutes searching for something new, people who have decided to watch Netflix rewatch something. So Netflix gets viewers, and can tell itself that it is adding value to their lives.

              But there is no option to customize for myself: to write my own algorithm, to tailor it to reduce temptation, to help me live the way I want. I must submit to someone else’s choices; to the tyranny of how they choose to interpret my history and desires, all based on the assumption that their calculations can understand me better than I understand myself, and that I will never change nor want to change. The only path is deeper into the rabbit hole.

              That’s how defaults are set up at the moment: traps for the unwary. We must devote cognitive attention to escaping them, and if we do escape, it’s usually extreme: complete removal or deletion of the option.

              Humans don’t work like that. There is noise in our decisions, and gradual shifts in our choices. I might watch one thing because it seems like an appropriate diversion right now; or because a friend is visiting and chose it; or because it was talked about at work, or any of a range of reasons. I might want to watch one or two on a subject, but not hundreds; and I might also want to watch things on other subjects. Some things are only just acceptable; others are brilliant. But the binary information of whether I do it or not doesn't reflect that. I'd do brilliant things more, but sadly they are rare. So I drop down to mediocre things. If that's a sale for whichever salesman or company I get it from, that's fine for them. More of that, please: get me on something that's good enough, and save time and effort that might have been wasted on getting me something great. Soon I'll come to accept that good enough is all I can get.

              I can’t let this discussion go without Facebook, which thinks its algorithm is so hugely valuable. Sure, it might help serve advertisements, but it turned Facebook from a brilliant way to see what every friend I had had done recently into a mediocre way to see what the friends I spend most time talking to had done recently… as if I needed that help.

              It still passes my threshold for use: nowhere else has any of my friends’ activities and thoughts displayed for me to engage with and maintain some social contact. But even when I turn this much-vaunted algorithm off, it looks very much like 480 of my 500 or so friends never do anything, until I look directly at their page. That’s not how I want to arrange my life; limited to as few as possible. I want to pick my interests myself.

              Maybe if I repeatedly click through a sequence of folders, I might want a shortcut to develop, as happens at work: Windows gives me five recently used folders, but only uses ones that I use at least a few times. They’ve realized that there’s noise in my activity. My neurons do the same thing: connections are strengthened or weakened over time, not created or severed.

              But even if algorithms learn to adapt properly: to deal with my interests as a network, with hub subjects and spoke subjects, all of which I want to be available, rather than the hub drowning out the spokes I already had, let alone potential new ones; even then, they can’t replace free choice. Humans thrive on a wonderful combination of self-directed spontaneity. Shops are wonderful because we see things we never thought about. That’s why people love Tiger or Pod, or department stores: for the novel nicknacks that ruin the environment but stimulate the mind. Algorithms, as they deal with me, would take my passing glance at a novelty pencil and teleport me to a pencil shop, smugly proclaiming that they were being helpful.

              The same is true in creativity. We need those infamous water-cooler moments where two different ideas collide that do not usually meet. How can an algorithm, based on history, ever hope to create such truly stimulating and productive moments? The actions of chance created evolution and the remarkable panoply of species that we’re killing off today. Chance moments spark people’s minds, bringing meaning and progress. Our choice systems are instead designed to create ruts and dig us deeper into them. It would be better to help us never dig a rut. Maybe when I’ve watched things, or bought things, it would support me to deliberately show me something new!

              The people who help us most are civil servants. Because they are rightly careful not to be illiberal, every nudge comes with requirements to give information and the right to opt out. We can opt out of pension schemes, but we don’t. Evaluating them and then filling in the paperwork is too much of a burden. Most of us can’t spare the time (and we all know that the administration of life has been gradually outsourced to the individual, while working hours and the demands on our time have increased).

              In summary, algorithms are designed to dig us deeper into whatever hole we start digging for ourselves, because it looks to be more profitable to keep us hooked on whatever just about passed our threshold for interest in the first place than to help us out of holes to search for new places to explore by ourselves. They contract our worldview to whatever is easiest to give us that we still accept using. They do not expand our horizons, helping us see further and better and to be more discerning. They make us small-minded, like the idiot programmers (or PR teams) who call them progress.We should be creating tools to make us better, as Steve Jobs said he did.

The topology of truth

This is a post about how we think about others' opinions. We inevitably imagine that we are right and others wrong, and that others are more wrong the more their opinions differ from ours. That's not quite right.

https://medium.com/the-weird-politics-review/covid-19-beyond-caution-and-normality-ad3985dc368b

I was set to thinking about this again by this article describing two different ways of seeing the current situation. Each viewpoint is reasonable if you accept its set of values, but each has illogical extremes, and this gives a good example of a general problem. I have been in many arguments and discussions in my life, and for many people truth feels like a straight line gradient: I’m over here, further down the path to truth, and every ideological direction seems to be higher up. So that person further away is extremely wrong. I’ve drawn this in this first picture

We all know the different points that can be made about the things close to us, and within our own range of understanding we are reasonably sure that we’ve made the right judgements.

It’s all very much like the energy states of a protein when the amino acids are folding up, or the energy state of a collection of chemicals. At school we learn that chemicals naturally progress to the lowest energy state of a system: if by reacting the resultant mix of chemicals are more stable, the reaction will progress.

We can model that as a graph of energy. Below is an open source example of combustion (burning).

But, as with protein folding and chemical energy states, the lowest current energy state isn’t the absolute lowest. Sometimes the local minimum is only local: with some activation energy, the system can be pushed over a boundary and then drop even further.

That’s shown in the above picture: you need to put in the energy to reach the transition state, like rolling a boulder over a hummock before it can roll all the way down the hill.

If we return to our article about two different opinions, we can draw a little map of these two different approaches that looks similar to our energy graphs for chemical reactions.

There are two local minima in this little part of our world. Someone at point B, for example, can see people at points A and C and think ‘those people are wrong; it makes more sense to think like me’. Someone at point E will think the same of those people he can see at D and F.

And, of course, B and E can extrapolate from that knowledge to assume, as we saw right at the beginning, that someone with even more different opinions is even more wrong. And, if the people at A and F are the really newsworthy people doing very stupid things, they will confirm that assumption.
So when B and E meet, each will say “Oh, you’re like F/A; you’re very wrong. You should slide towards the truth just like D/C should”.

I experience this a lot. I make an effort to look broadly at an issue and consider all opinions. In our topology of opinion, I range widely over the map, unafraid of hills and keen to look beyond them. I want to understand the terrain and work out what really is the truth.

I regard myself as the ‘E’ in the above example, but an E who knows not only about position B but also that it’s more wrong than E. But the Bs of the world don’t like to listen; we often argue across each other, with them trying to persuade me that positions F and D are wrong and me trying to explain that E is lower than B.

I’m not the only one who imagines that I’m right, but I hope that the picture above helps people argue their cases better with others. You have to understand someone’s position before you assume that, because it’s more extreme than D, or almost as extreme as F, it’s therefore a bit more wrong than D, or only a bit less wrong than F. Sometimes there really is a local minimum that from a certain perspective seems pretty right. After all, if you’re the E and I’m B, you won’t persuade me by telling me that A and C are wrong and that I’m in the middle. I know I’m in the middle, and I also know that logic should slide them both to me. You are making assumptions about how the world looks over here and making yourself seem ignorant and wrong as a consequence.

And then there’s the ‘split-the-difference’ debaters like C and D. I don’t know if it’s an emotional need for everyone to win and all to have prizes or a blissful inability to recognise truth, but some people see how strongly those in troughs B and E are stuck in their ruts and decide to please everyone by being more wrong than either. I can understand if you start on the left side you might slide down to B, even though E is lower. I can understand that there are forces in the world (external and internal) radicalising people, pushing them to A and F. But C and D really seem plain old stupid: based on the idea in the first picture that there is only one minimum and no bumps or obstacles. So if B is better than A, and E is better than F, surely going to the middle and B and E must be even better?

This is not how the world works. I might prefer to be in London than in Watford, and on holiday in Madrid rather than the Sahara Desert, but the middle of London and Madrid is probably in the ocean in the Bay of Biscay, and that’s not very comfortable either. C and D are positions that come from looking far away but not bothering to judge the position on its own merits, and a failure of self-reflection is a pretty big failure.

It’s also interesting to note that convex curves obscure the peak. Anyone who’s done much mountain-climbing will know that when you climb a gently curving hill you can’t see the top; the curve gets in the way. People at the bottom of a slope can have a very restricted vision of what’s around them. I don’t think we need to stop our analogy there, though.

So I’ve drawn one last map of opinions that illustrates a few possibilities. First of all, it’s possible to hold the same opinion for different reasons, which can be more or less correct.

So someone at P has the same final answer as someone at O, but is nonetheless more wrong. If you’ve met someone at L and you assume that everyone at L, O, P and even H and g are basically the same, you are going to annoy those Os a lot. And fail to persuade them at all. They know that P is above them, but they also know that they’ve come to their opinion via an entirely different route. They’ve gone caving from over at N, and sometimes you need to retrace the route to get them out of whatever dead end they’re in.

Then there’s the example of the people at Q. Someone at Q might be mightily pleased with themselves for finding a narrow gap in all the obstacles and delving deep towards the truth. They know it was hard to see Q from any distance, and are therefore very unwilling to listen; either to those they think are at L or R.

And then there are the successive steps between M and I. Each position is more in-depth than most of the rest of the range of opinion; it’s taken careful thought and some effort to get that far. But even now, there are two attitudes people can take. You can look behind, or look ahead. You might find that almost everyone you meet is an L or R, mixed with extremists at g and U, and a bit like the Qs, you give up and start treating others as dolts if they don’t drop down the mine shaft to N with you.

Or, having got to N, you can remain open to help. People calling from J can help you find your way from N to more truth. The problem is that people calling from H might seem very similar and as the cacophony intensifies you might not have the time or the ability to separate a good signal from all the rest.

I aim to be a J and I, really much closer to the truth than you might think, especially if you do a rough survey from above. Pollsters might ask everyone’s opinions to try to plot them out on a line, but if pollsters don’t understand the cave structures underneath the surface then they will misunderstand what people really think. They might assume that the I-Os are all Ls (or Hs), fitting them onto a blander, less fine map of thought.

When we realize how complex the terrain of thought is, we also realize how important it is to be able to move around and find your way. Navigation, the mental fitness and agility to climb hills, and light to show the way, are all much more important than your starting location. Someone calling to you could be wrong, or could be right, and the more right you are, the likelier it is to be misleading and the harder it is to trust well-meaning advice.

And I haven’t really addressed density yet! There will be a whole city of people collected at R (or E on a previous picture) telling each other that they’re right. The voices from the few cavers (‘experts’) in the depths of J-O will be lost unless people make a conscious decision to listen to them. But they might hear voices from S reassuring them that if they bothered to spend more time on it they’d not change their opinions much; that they are definitely more right than the mass of people over at L.

One final thought: we might see that people over at T clearly haven’t explored much, but they’ve found a nice local minimum. They can see the world in the broad map of people at g and U, and imagine that the centre is that little bump next to them. After all, if they meet a truly central opinion it’ll seem to be about as true as that bump next to them. They’ve taken the red pill and discovered the hidden truth in its little valley. Maybe they haven’t discovered as much truth as they think…

Maybe I’m not at J or even I. Maybe I got stuck in the dead end at O; my musings have taken me down a bad dead end. But if you assume that everyone is up top on our map of A-F, you might mostly be right, but you will never get me to move. Truth, and argument, comes in layers. If you go too deep, someone up top might not follow you. But not deep enough and those people who’ve already delved will dismiss you and feel insulted.

Bullies in the Right

There is something deeply unsettling about the attitude of many right-wingers to their recent democratic victories in Britain. I have heard or seen things like:

“You’ve lost, get over it!”

Or

“There’s nothing quite like salty liberal tears”

              Victory, for them, seems to mean gloating and triumphalism. The joy of winning is in their enemies’ pain, not in getting something else that they wanted, such as prosperity, health, freedom and other trappings of civilisation. For the Right, these weren’t conflicts of ideas, but of tribes. When a Conservative politician lied on national television in support of the cause, it wasn’t a disgrace; it was a moment to be celebrated because the Right won a victory in getting away with it. The all-out war is no longer with hostile foreign powers such as Russia. They are now allies in undermining the rules and culture of our country. The war is with the well-meaning people who actually care about what happens to the country.

              I have seen this sort of devil-may-care attitude before. We will all, nowadays, have seen it online. Trolls excuse every offensive or inaccurate statement they make by claiming that they’re just playing devil’s advocate. But after a few occasions, that word ‘playing’ no longer seems appropriate. They are the devil’s advocate, play or not. Whatever the intention, the effect is to sow dissent and mistrust, and waste the time of people foolish and anxious enough about whatever they’re saying to bother to rebut it.

To ‘play’ devil’s advocate is to abdicate from the responsibility of thinking for yourself; putting forward abusive, ignorant or fallacious opinions is excusable to an extent because everyone is only human and we can all make mistakes or miss some vital bit of information. But adopting it as a habit is (yes, I’m going to say the p-word) a privilege only people who are already safe and secure can afford. Other people are already forced to think through the rights and wrongs of a situation, and if you have a say in how the country is run, you should already have thought those things through as well.

But even in a past life, not quite pre-internet, I’ve seen this behaviour before. It’s how the bullies at school behaved. Bullying was about engendering misery in others to make up for weakness of character in oneself. Bullies were tribal: children had to show fealty to the group by providing entertainment, which meant abasing themselves or others. Most chose to try to humiliate the ‘easy’ targets. That was people like me: academically able and socially awkward. The more able, the bigger the victory; the more socially awkward, the easier the victory.

That is echoed in our politics. The more well-argued and logically sound the left-wing argument (i.e the non-Conservative, non-quitling side of debate), the greater their joy in ‘winning’ whatever discussion is taking place. The more right your opponent, the greater your victory when you win. It’s about your strength, not truth. And that’s the repetitive refrain throughout all of this: it’s people feeling weak and insecure compensating through belonging to a strong group. It’s an entirely different axis of human interaction from the one that many left-wingers are trying to engage in, which is a conflict of ideas in which correct vanquishes incorrect. And when the two axes are both on display, one intellectual and the other emotional and anti-intellectual, viewers can respond to the right-wing victory in a conflict the left wasn’t even trying to fight.

That’s how bullying, or the Right, works. It’s not about winning an intellectual argument; it’s about being strong enough to get away with not winning. The transgression, of being wrong in some way, is essential for the show of strength. You’re not getting away with anything if you’ve followed the rules: there’s nothing pleasingly subversive that gives a feeling of reversing the usual order if the rules are followed, whether those rules are those of logic, decency or of the school. And the supporters of this proto-fascism do feel insecure: the established way of doing things seems not to have served them well, so they crave subversion and the feeling of throwing off those mental shackles.

Of course the Left wants change too. That’s why the Conservatives have their name: they want to conserve the old order and the socialists want to change it to be more pro-social. This is only for objective reality: the Left has been too focussed on wealth and equality, assuming that people will be proud of a well-run country. The Conservatives have realized that people can be proud and feel like they have all they need without any expense or change at all; they just need to feel like they’re part of something that’s doing well, even if that feeling is a lie and they themselves are doing badly too. Their new strategy is to focus again on community: to give people a sense of belonging, to feel like they’re part of something bigger.

Maslow’s infamous hierarchy of needs is a helpful reminder that people want lots of things in life. At the bottom of the pyramid are basic needs such as food and shelter; then as we get higher we get prosperity, social interaction like friendship, family and community, and finally the broad concept of self-actualization. This final need is the need to be something bigger; to have achieved something important. Belonging to the Brexit/Conservative group gives people these upper parts of the pyramid. Yes, at the expense of both the layers below and their own future, but for a time they can feel good.

That’s what people do when desperate; they take short-term decisions that from an outside perspective seem unwise. Some people become addicts; others start fights or act out. Conservative propaganda is a drug to dull the pain of life under the Conservatives; leaving the EU is the temper tantrum of a disturbed child. People used to use religion as the opiate of the masses, to give them something to hope for. Now something more blatantly political is taking the place that fading religions have left.

It is this need for belonging that is driving the lunatic ideas of tolling Big Ben or holding a national festival on ‘Brexit Day’ (the day we move from Article 50 transition period to negotiated transition period). With other political decisions we would never consider a national festival: imagine a national festival of the bedroom tax or the Iraq war. Or a festival celebrating the Suez crisis when Britain proved itself no longer a world power. The idea that this will promote national unity and happiness is ludicrous: you stoke resentment by co-opting something people might love and using it to represent something awful. Remainers aren’t going to be miraculously won over by this display of power over their British symbols.

These ideas aren’t really intended to win over sensible remainers. They are intended to reinforce wavering leavers. If given a chance they will start to have doubts; the passions aroused in them will run dry, leaving only thoughts behind. Those thoughts are scary for the Conservatives: they don’t want their supporters thinking. They must feed the fire of emotion at all times, stoking it continuously. As with the most vocal homophobes, who then turn out to be closet homosexuals, the loud shouting isn’t really for everyone outside: it’s to persuade themselves. If they shout loudly enough, they will drown out their own thoughts; if they shout loudly enough, they can pretend that the rage and hatred they feel at their own inadequacy is actually for someone else: other races, other religions, experts who make them feel inadequate… anything but the pain of reality.

As with bullies, we can be very sympathetic about the gross inadequacy that has some leavers groping for anything that gives them a sense of pride, and at the same time rightly angry about the awful decision that leavers have made that hurts other people. And, as with many bullies, there are ringleaders who are far worse: insidious worms of people, spreading lies about immigrants, stoking resentment, encouraging misbehaviour whilst always not being close enough to it to be punishable themselves, and revelling in the chaos they cause.

These people: the Dacres, Farages, Bankses and Murdochs of the world, who have genuinely carved a bit of power out for themselves - carved it out of the foundations of our civilised society so that it’s falling down around them – and are proud of it; these are the people for whom we should have no sympathy. Anyone can be deceived by a lie from those they (misguidedly) trust. Everyone has anxieties that can be preyed upon. Normal bullies can become friends when treated well. Only the proud manipulators of human lives are beyond redemption.