Whose rules?
On the immanent recursive judicial ossification imparted by automation
Sunday, June 15th
15/30
From the meditations of The Pale Metaphysician
Before beginning today’s article I want to take a moment to reflect that it is day 15 of 30 — I’m halfway through this project. When beginning this I had some idea of how hard it would be, and it indeed has been difficult at times, especially on days where it is late or I’m tired or lacking motivation or a million other excuses, but I didn’t consider how easy in other respects this would be, nor how taking time each and every day to really stick with an idea and tease out a concrete logical 500 words would remind me so deeply of who I want to be(come). Sometimes I feel as if I’m just a voice lost on the wind, a thought that can be both en- and discouraging, but then I remember that at least one person is reading this stuff every day and to be internally consistent with my metaphysics of “effect on affect = real” (oversimplification), then I can’t just be noise. So thank you, dear reader, and I hope you enjoy the rest.
Yesterday I wrote on agency and autonomy where I made the point that automation has stripped agency from many processes raising questions of responsibility and value. Additionally, in part two of my articles on speed, I discussed one of capitalism’s fundamental contradictions, the falling rate of profit, where I offhandedly made the point that the increased leisure offered by speedy automation is enjoyed mostly by the bourgeoisie. I even posed the question: “leisure for who?” Today I want to ask a related question based on the premise that machines are increasingly automating the enforcement of certain rules: rules for who?
A system of laws striate our social contract offering it a recognizable hierarchy and immediately legible normative standards, or at least, that’s the idea. This “rule of law” is fundamental to how the modern world works.1 As such, there are not many steps between this idea and one that suggests automating their enforcement — it would be more efficient, more reasonable, more fair. Justice, after all, ought to be independent of the person enforcing it, or on whom it is being enforced, no? So the idea caught on and lots of decisions previously left to messy humans were swiftly automated away leaving the key decisions to important humans and we called this automation “bureaucracy”2 and it has done a pretty damn good job, so we made it more efficient by removing the people filing the paperwork with machines that can file more paperwork faster and left even fewer decisions to even more important people and that has done an even better job.3 Except for when it hasn’t. More and more are realizing that justice isn’t actually all that impartial and that there really are some classes of people who have a different set of rules.4 So again I ask: whose rules are these systems enforcing, and who’s to gain? You can see where I’m going with this.
Automation encodes a system designed by and for a specific people and those who don’t immediately fall into the protected category are by default excluded. This progressive stratification is not necessarily intentionally carried out by those who stand to gain the most — that’d be too easy, not to mention conspiratorial, for often times the methods of and reasoning for this judicial calcification fall out of the logic underpinning the system in the first place, re: import of rule of law. Examining the notion from this angle reveals an almost recursive structure: rules by and/or for the bourgeoisie stratify flowing potentials which encourage further development of rules that more deeply entrench those that came before and so on and so forth, a cybernetic of political deactualization.
An example may help. In yesterday’s article I mentioned electronic banking so let’s run with that. The process of loan approval, called underwriting, is long, arduous, but very predictable as it takes into account a fixed set of variables and results in a binary response: either the person gets the loan or they don’t.5 A lot of banking revolves around hashing out the terms of these loans and determining if the requester ought to be given the money or not and traditionally human bankers carried out this work. This process was prone to bias and prejudice, as evidenced by how difficult/impossible getting a loan was if you were Black or female, so much so that laws were created to make lending fairer. Eventually someone realized that if computers can crunch numbers really well and like binary, then it’d be relatively easy to program one to ingest the variables going into a loan approval (e.g., income, assets and liabilities, credit score, etc.) and output the response; the rentier class loved the idea as it reduced risk and increased profits, and thus automated underwriting systems (AUS) were formed. Ostensibly this helped borrowers as it sped up the process for them and removed much of the guess work. Except, lending discrimination didn’t just go away and the workings of AUS themselves don’t have to be disclosed to borrowers, so new laws came to address those issues, which simultaneously entrenched the practice of using AUS as they weren’t legislating the practice itself but how it was carried out, a subtle yet significant difference.
This all isn’t to say that rules and regulations shouldn’t exist nor their efficient enforcement — there is no out, over, or above from the system we find ourselves within, only through, but that they always appear pregnant with the question of who benefits the most, either explicitly through what they allow or implicitly through what the proscription they create or enforce may validate. “Automate everything” seems great until the glaring realities of such a desire emerge, and to go one step further, one needs only consider who benefits the most from such automation to recognize the contradiction inherent in claims that it will “democratize X” or “make Y fairer.” Go yet another step and you’ll realize how much money there is to make off automating these systems (whoever developed the first AUS made a fortune) which casts further doubt on the whole ordeal. If you’ll allow me to conclude in similar fashion to yesterday, it is ultimately people who develop these tools and are affected by them, no matter how sophisticated the automation becomes, and thus it is people who will dictate their use — for them to be truly fair the recursion must be driven by the People, not those who pretend to greatness with their immense wealth.
Indeed, the exceptions prove the rule here. One must look no further than Trump and co. and their disregard for law to see how deeply embedded this reverence for law is — it is the bedrock of liberal democracy. The Cold War entrenched this belief insofar that any non-democratic nation not just disrupted the proclamation that only those nations with open markets and democracy are good, but that only they ought to exist.
The history of bureaucracy is fascinating and worthy of its own article. Sociologists have been explicating the nature of it since sociology as a field existed — one of it’s founders, Max Weber, dedicated much of his work to investigating it.
I’m skipping a few steps here but you get the idea.
Hundreds of thousands of them took to the streets just yesterday!
The actual outcomes are a bit more subtle insofar that there can be conditional approvals or suspended considerations, but generally the idea is that you are given money or not.