"It's Just a Productivity Tool" — Why Your AI Is Probably Higher-Risk Than You Think

There is a category error that keeps coming up in our AI vendor audits. A company deploys an AI tool. When asked whether it is a high-risk or high-impact system, the answer is almost always no. It is just a transcription tool. It is just a meeting summariser. It is just a chatbot. It is just helping people work faster. It is not making decisions. It is not high-risk.

Under both the EU AI Act and Australia's proposed Automated Intelligence and Data Act, that answer is increasingly wrong. Not because the tools have changed. Because the regulatory definition of high-impact AI captures much more than most organisations realise.

AIDA, like the EU AI Act, classifies AI systems based on their impact on people, not on whether the deploying organisation considers them risky. The test is not "do we think this is dangerous?" The test is "does this AI system affect decisions about people's rights, opportunities, or access to essential services?" Under that test, a surprising number of "productivity tools" qualify.

The AI That Doesn't Make Decisions but Still Counts

The most persistent misconception is that an AI system only counts as high-impact if it makes decisions autonomously. If a human reviews the AI's output and makes the final call, the thinking goes, the AI is just a tool, and the risk sits with the human.

AIDA does not draw this line. The bill covers AI systems that "make or influence" decisions. Influence is the word most organisations miss. An AI that summarises a customer call, and the summary is read by a human who decides whether to escalate the complaint, has influenced a decision. An AI that scores job applicants, and a recruiter uses that score to decide who to interview, has influenced a hiring decision. An AI that flags transactions as potentially fraudulent, and an analyst reviews the flags, has influenced a financial decision.

In each case, the human made the final call. In each case, the AI shaped the information the human used to make that call. The AI influenced the outcome. Under AIDA's definition, that may be enough to classify the system as high-impact if the decision itself falls within a regulated category.

This matters because the obligations that attach to high-impact AI are substantial: mandatory impact assessments, bias testing, explainability documentation, human oversight requirements, and regulatory notification. If your "productivity tool" is actually influencing decisions that affect people's employment, credit, insurance, healthcare, or education, you may be deploying a regulated AI system without knowing it.

The Categories That Capture the Most Organisations

AIDA's high-impact categories are specific, and several of them will surprise organisations that have not read the bill carefully.

Employment is the category that will capture the most Australian businesses. If AI is used in recruitment, candidate screening, performance evaluation, promotion decisions, or termination recommendations, the system is likely high-impact. This includes AI that screens resumes, scores video interviews, analyses assessment responses, or generates candidate rankings. Most medium and large Australian employers are already using at least one of these tools. Most have not conducted an impact assessment. Most have not tested for bias against Australian protected attributes. Most are deploying what they think of as a recruitment efficiency tool that is actually a regulated high-impact AI system.

Credit and insurance is the category that will capture financial services. Automated underwriting, AI-driven claims assessment, credit scoring, and pricing optimisation all fall within scope. Many Australian insurers and lenders are using AI models for these functions, often from third-party vendors, often without independent validation of the model's fairness or explainability. Under AIDA, the deploying entity — not the vendor — is responsible for compliance.

Healthcare and education each capture their own ecosystems. Diagnostic support AI, treatment recommendation systems, student assessment tools, and automated grading all involve decisions that significantly affect individuals. The AI might be assisting a doctor or a teacher rather than replacing them, but under AIDA's influence test, assistance that shapes the decision counts.

Government services and critical infrastructure round out the high-impact categories. Any AI that affects citizens' access to benefits, services, or rights is captured. Any AI whose failure could disrupt energy, water, transport, or communications is captured. The Robodebt Royal Commission demonstrated what happens when automated systems in government go wrong. AIDA is in part a legislative response to that lesson.

The Gap Between "We Think" and "The Law Says"

When we audit organisations' AI deployments, we consistently find a gap between what the organisation believes its AI is doing and what a regulator would conclude it is doing. The gap exists because the organisation assessed the AI based on its intended use — "helping recruiters work faster" — while a regulator will assess based on the AI's actual effect on individuals — "determining which candidates advance in the hiring process."

This gap is not resolved by the organisation's intention. If the AI influences employment decisions, it is captured by AIDA's employment category regardless of whether the organisation intended it to be a productivity tool. The law classifies based on impact, not intent.

Closing the gap requires an honest audit of what your AI actually does. Not what the vendor's marketing says it does. Not what your internal business case described. What it does when it's running. What data goes in. What outputs come out. Who uses those outputs. What decisions those outputs influence. If the answers place the AI in a regulated category, your organisation has obligations whether you knew about them or not.

The Vendor Classification Trap

Some AI vendors market their products as "low-risk" or "not intended for high-impact use" as a way of deflecting regulatory responsibility. This has no legal effect under AIDA. The deploying entity classifies the system based on how it is used, not based on how the vendor labelled it.

If you buy a "general purpose" AI tool and deploy it to screen job applicants, you have deployed it in a high-impact context. The vendor's disclaimer that the tool is not intended for employment decisions is irrelevant. You used it for employment decisions. You are responsible for the classification and the obligations that follow.

This is the part of AIDA that will catch out the most organisations. They will assume the vendor's risk classification applies to their deployment. It doesn't. The deploying entity owns the classification. If you classify wrong, you carry the consequences.

What to Do Before AIDA Lands

First, audit every AI tool your organisation uses against AIDA's high-impact categories. Not what you think the tool does. What it actually does. If it touches employment, credit, insurance, healthcare, education, government services, or critical infrastructure in any way, assume it may be captured and work backwards from there.

Second, for each tool that falls into a high-impact category, conduct a preliminary impact assessment. What decisions does the AI influence? What is the potential harm if the AI produces a biased or incorrect output? Who is affected? What mitigations are in place? Document everything. When AIDA becomes law, this documentation becomes your compliance trail.

Third, for any tool that you cannot confidently classify, get an independent assessment. The cost of getting the classification wrong under a regime with five per cent of global revenue penalties is far higher than the cost of getting it right now. If your internal team is unsure, bring in someone who isn't.

Fourth, start the conversation with AI vendors now. Ask whether they have conducted bias testing against Australian protected attributes. Ask whether their model's decisions can be explained in plain language. Ask whether they have documented their training data provenance and composition. If they cannot answer these questions, they may not be suitable for high-impact deployment under AIDA. Better to find out now than when the regulator asks.

The organisations that will be ready when AIDA becomes law are the ones that took classification seriously before it was mandatory. The ones that will be caught out are the ones still calling their AI "just a productivity tool" when the regulator calls it something else.

Book a scoping call to discuss an independent AI impact classification audit under the proposed AIDA framework.