Australia's Algorithm Law: Who Must Prove the Feed Is Safe?

The Ban That Did Not Ban Very Much

Five million accounts, gone. Australia's algorithm law—the Online Safety Amendment, passed in 2024—set the cutoff at sixteen, and by 10 December 2025, roughly five million accounts belonging to under-16s had been deactivated across platforms. Then came the June 2026 study. Eighty percent of those same children were back online.

That number is worth sitting with. Not because it reveals some shocking flaw in Australian parenting or teenage ingenuity, but because it forces a precise question: what, exactly, had the ban been fighting?

Five million accounts deleted, and the machinery underneath them kept running, untouched, pulling the same young users back through friends' phones, misreported birthdays, borrowed logins. The age gate was real. The algorithm was indifferent.

Here is what that 80 percent figure represents at the policy level: a controlled experiment, accidental and expensive, which demonstrated that the problem was never the account. The problem was the engine.

The recommendation system that learns what makes a particular thirteen-year-old stay for one more minute, then another, then twenty more—that system does not check a birth certificate. It just works. And it worked, loyally, on the children Australia had formally asked it to ignore.

The government's response was not to abandon the age ban. It was to look past it, toward the code. If policing users had failed to stop the draw, perhaps policing the draw itself was the next rung up.

That pivot—from regulating who could enter a platform to regulating how a platform behaves toward everyone inside it—is the whole story of what this algorithm legislation is now attempting. The scale is considerably larger this time.

What Australia's Algorithm Law Actually Requires Platforms to Build

In September 2026, Communications Minister Anika Wells introduced a bill with a mechanically precise demand: every user gets a pop-up, and the pop-up asks a genuine question. Do you want the algorithm, or not?

If the user says no, the platform must switch them to a feed built from nothing but accounts they have manually chosen to follow. No ranked recommendations, no engagement-optimized injections, no "suggested for you." Just the people you picked, in whatever order the platform receives them.

The legal term Wells' department uses is "digital autonomy," but the operational description is simpler: the algorithm goes dark.

Ten platforms are named in the legislation, and the list is worth reading in full because of its range. Facebook, Instagram, TikTok, Snapchat, X, YouTube, Reddit, Twitch, Threads, and Kick. That span matters. It covers the short-form video engines most scrutinized for addictive design alongside the older, slower architectures of Reddit and Twitch, where the curation problem looks quite different. The law does not distinguish between them.

The mechanism of the pop-up is not incidental detail. Regulators have learned that burying an opt-out inside a settings menu twelve taps deep is functionally equivalent to offering no opt-out at all. The mandatory prompt forces the choice to the surface, where it becomes a real decision rather than a theoretical right.

Whether a one-time selection counts, or whether platforms must ask periodically, is one of the questions the bill has not yet answered. What the legislation has answered is the direction of obligation. The burden now falls on the platform to build the switch, surface it clearly, and make it work.

A Duty of Care for Code

Here is the strange part about most digital regulation until now: the burden sat entirely with the person who got hurt. You had to demonstrate damage, trace it back to a platform, survive a legal process designed for an era of physical products. The platforms, meanwhile, operated like architects who could walk away from a collapsing building by pointing at the people inside.

Australia's new algorithmic regulation flips that logic. The "Digital Duty of Care" written into the bill moves the burden of proof onto the platform itself. Companies must now demonstrate, proactively, that their systems are not causing harm.

If they cannot, the fine ceiling sits at 109.2 million Australian dollars. That is double the previous maximum penalty for social media violations. The number is meant to sting rather than be absorbed as a line item in a quarterly report.

The conceptual scaffolding here comes from product liability law, a framework most people encounter through defective kettles or unsafe car parts. Under that tradition, the manufacturer is responsible for what the product does, not merely for what users choose to do with it.

Anika Wells applied the same logic to code when she described the target of the legislation as "predatory algorithms," a phrase that does real legal work. It frames the recommendation engine not as a neutral tool but as a design choice with intent and consequence.

This is the philosophical inversion the law is attempting. Previous regulation asked: were you harmed? The new framework asks the platform: can you prove you are not harming? Shifting that question is, quietly, a considerable legal revolution.

What 109 Million Dollars Is Supposed to Mean

Picture a conference room somewhere in Menlo Park. A compliance officer slides a single page across the table: "Australia fine risk, FY2027." The number at the bottom reads 109.2 million Australian dollars.

Someone at the far end of the table nods slowly, opens a spreadsheet, and begins typing. That is precisely the scenario Canberra is trying to make impossible.

The fine ceiling of 109.2 million AUD is not arbitrary. It is double the previous maximum penalty for social media violations in Australia, a deliberate signal that the old numbers were not landing.

A fine that fits neatly inside a quarterly advertising revenue line is not a deterrent. It is a licensing fee.

A fine that fits neatly inside a quarterly advertising revenue line is not a deterrent. It is a licensing fee.

The teeth behind the number belong to two bodies. The eSafety Commissioner, Julie Inman Grant, gains new powers to reach directly into platform architecture and investigate whether a recommendation system is actually causing harm—not merely whether a company has filed the right paperwork. Algorithmic impact, not just algorithmic policy.

The Office of the Australian Information Commissioner sits alongside her, watching a different door: every age verification system a platform deploys to comply with the law generates personal data, and the OAIC's job is to ensure that data does not become its own problem.

Whether nine figures actually changes behavior in Silicon Valley is the question the law cannot yet answer. For a company the size of Meta, 109 million dollars is real money. It is also, depending on the quarter, roughly two days of net income.

The number is meant to hurt. Whether it is large enough to sting more than it is cheap enough to absorb is an open calculation, and Canberra knows it.

The Censorship Question Nobody Wants to Answer Simply

Angus Taylor, the Australian Opposition Leader, reached for an old word: censorship. It is a word that shuts down conversations rather than opening them, but that does not mean the underlying anxiety is wrong.

The ambiguity here is real, and it runs deeper than political point-scoring. Consider the spectrum. At one end sits pure chronological sorting, a simple timestamp list with no opinion.

At the other end sits a recommendation engine that has read your last three years of behavior and is quietly steering your attention. Between those two poles lies almost everything a modern platform actually does, and the current legislation does not clearly define where on that spectrum an "algorithm" legally begins.

Compare this to the editorial question newspapers faced in the twentieth century. A newspaper chose which stories ran on the front page; nobody called that censorship. But if a regulator had required that front page to be arranged by publication time regardless of importance, editors would have called it interference with editorial judgment.

Platforms now make the same argument, with more sophisticated lawyers. The honest accounting is this: the law raises a question it does not yet answer—namely who gets to decide what a feed "should" look like, and by what standard. That question will almost certainly end up before a court before it ends up settled.

The Open Horizon: What an Off Switch Cannot Settle

Regulators in the United Kingdom and France are watching Canberra closely, treating this algorithm legislation as a live experiment rather than a curiosity. That tells you something about the stakes. But it also tells you that nobody, anywhere, is certain this will work.

The honest list of unknowns is longer than the law itself. Nobody has defined, in precise technical language, where "algorithm" ends and basic functional sorting begins. Nobody knows whether the opt-out prompt will appear once at account setup or recur regularly—and that detail matters enormously for actual behavior change.

Nobody knows how a court will measure whether a platform has fulfilled its "duty of care" in practice. And the impact on smaller platforms like Reddit or Twitch, compared to the giants Meta and TikTok, remains entirely uncharted.

When 80% of banned minors found their way back online anyway, Australia stopped policing users and started policing code. That pivot is the real story behind Australia's algorithm law—and an off switch only changes who controls the dial. Who decides what you are shown, and on whose authority—that question will outlast every law written to answer it.