Contra Scott on Argument Mapping

I've recently been more public about the soft release of my argument mapping app CruxHub/ fenc.es , but I haven't had the time to write about it. Today Scott Alexander has written a post about why my attempt is doomed to fail . I agree with the majority of his points and I think he is wrong.

The idea for my project has been in my head for at least 5-6 years now, and I've kept researching and looking for existing tools like this for that same time, but I encountered the same graveyard of failed and abandoned argument mapping and "debate solving" projects that Scott mentions in his post.

I don't have the time to now detail every project in comparison to mine, but here are some fundamental mistakes that I've seen:

But the biggest mistake of them all is wanting to "solve debate" at all. But isn't that what Scott is saying? No no no, he conflates "solve debate", "disagree better", and "map arguments" while still being himself in the debate mindset.

Debate culture is the mind killer

I fully agree with Scott that debates basically don't work, not on a podium, not on YouTube, maybe sometimes possibly in writing. If at all, a debate's positive impact comes from affecting onlookers views, but nobody watches Slavoj Zizek debate Jordan Peterson or destiny's stream to learn about the actual topic at hand, this is entertainment in a very similar way that watching sports is.

If you want to actually move directionally towards truth, you need to leave the notion of "winning a debate" behind. What you want is the anti-debate: You assume you are mistaken in some way, and the people disagreeing with your main point may have valuable insights that you want to extract from them.

So what about disagreeing better? can we do that?
Scott says unlike dating apps, which came after classifieds, this has never been done before and also there has been no progress on this for 2000 years so the base rate for success is zero.

If you really want to express a base rate we should at least take the printing press as a prerequisite and take the last 200 years. If you narrow the scope to drawing circles and arrows to convince someone they're wrong, yes there has been little progress. But we have had a lot of progress in argument resolution: common law, peer review, formal debate, Bayesian reasoning, Wikipedia, prediction markets, forecasting tournaments. These work to differing extents and some are even app-shaped.

But even mundane circles and arrows are regularly deployed in group settings all over the world to map arguments on flipcharts and whiteboards, just not to prove someone wrong, but to create a shared world model or understanding. No single person has all the arguments.

I have experienced the "dating classifieds" analogy of CruxHub/fenc.es in real life productive disagreement workshops, it very much works when you have real people in a room together, so we just need to figure out a way to make it scalable.

Being wrong on the internet

"I like writing a post saying that my opinion is right. Sometimes people tell me that actually, my opinion is wrong. This frustrates me." – Scott Alexander

You can choose not to be like Scott. He assumes just because he doesn't like arguing on the internet and just wants to be told he is right, that everyone is like that. I don't think that is true.

Many spaces on the internet select for people like that. Not because they have been consciously designed for this but because of the affordances they offer. I believe we can design spaces that select for people that aren't like this or want to become less like this. People that do want to argue in a collaborative way, not to prove someone wrong, but to discover their blind spots and learn true things about the world.

So what I'm looking to build is no canonical large exhaustive argument maps that prove my enemies wrong. Instead I want people to map small snippets of their reasoning, in the way they reason. Maybe just 5 related claims, but with affordances that make it easy for others to surface which of those 5 points is actually worthwhile talking about under a common understanding that we want to find the rest of the elephant together.

Zombie pull requests

Scott says this as "People like taking drive-by potshots on the Internet" and I think this can become a feature not a but. The nice thing about PRs is you don't need to accept them. And thanks to LLMs, filtering out duplicates, insults, non-sequiturs etc. has become basically free.

@dollspace.gay has written up the concept of VerificationDrivenDesign (VDD) and many others have independently discovered similar approaches being useful: You have one LLM create some artifact (code, a policy document, a research agenda) and then have another model harshly criticize it. You review the points for validity, let the first model make improvements and repeat the process until the reviewer model only produces hallucinations.

We use the same approach to harvest signal from the waves of drive-by potshot zombies: We filter out the hallucinations and take the good stuff as improvements.

Figure out where you are wrong

But the main mechanism that fenc.es aims to provide value, is by making it easier to recruit people who actually want to argue in good-faith to help you discover where specifically you are wrong.

This can happen in 1on1s on conferences (send a preparatory survey link so you only discuss relevant points) or in group settings (mapping where the group has the biggest disagreements in their common theory of change).
Even singleplayer mode can be helpful to figure out your own thinking around a topic (with or without an endlessly patient LLM adding their confidences, importances and counterarguments on your map via MCP). Not everyone is good at writing their thoughts down in text form, for many people some predefined structure can help with that.

Do I believe that CruxHub/fenc.es will definitely be a huge success and revolutionize online disagreement? Well no, otherwise I would focus full time on it, I consider my main job way more impactful. But I very much do believe that good apps for mapping arguments that actually help with disagreeing better can be built, and r/changemyview is not the ceiling, it's the baseline0.