← Invisible Machines · Español

The Score

You’re applying for a loan. An automated system will decide in a second — approve, review, or deny. You won’t be told how it works. Fill in the application, submit, and then try to figure out what really moved the verdict. (A model — figures are illustrative; the pattern is very real.)

Loan application

Annual income$58,000
Years at current job3
Existing debt$12,000
Gap in employment (months)0
Your neighborhood

The decision

— —
Submit your application to get a decision.

What you just ran into

Two applicants with the identical income, job history, and debt can get opposite answers because of their neighborhood — a stand-in for race and class that echoes historic redlining — or a gap in employment that often just means illness or caregiving. The system shows you a verdict and a number, never the real reasons. That’s automated decision-making, and it already gates credit, apartments, jobs, insurance, and benefits. The harm isn’t only bias — it’s opacity: you can’t challenge a reason you’re never given.

You have more rights than the screen admits

  • If you’re denied credit, housing, insurance, or a job based on a report or score, you’re generally owed an adverse-action notice stating the main reasons and the data source (in the US, the Fair Credit Reporting Act / ECOA). Ask for it in writing.
  • You can get your data and dispute errors — in your credit file, and in tenant- and employment-screening reports — and the company must investigate.
  • For decisions made solely by a machine with major effects, some laws (e.g. EU/UK GDPR Art. 22) give you the right to a human review and to contest it. Always ask for a person.
  • A factor like ZIP code standing in for race can be illegal disparate impact under fair-lending and fair-housing law. If a decision feels off, that’s worth raising.

General information, not legal advice — rights vary by country and situation. The point: the verdict is not the whole story, and you’re allowed to make it explain itself.

A teaching model — the weights are illustrative, not any real lender’s. No accounts, no tracking, nothing leaves your device. Free to copy and reuse (CC0).