AISA is an Evidence-Based AI Fluency Assessment for Enterprises: An Independent Methodology Review
By Brian R. · B.A., Computer Science, Minor in Philosophy
Why measure AI fluency at all
Most organisations have rolled out AI tools, and many have run AI training. Far fewer can say how well their people actually use AI. Self-reported confidence is a weak guide: in a study of 274 professionals who predicted their score before taking AISA, the average prediction was 18.5 points higher than the score they earned (the full findings). Independent research points the same way. A study presented at LAK 2026, the international learning analytics conference, found only a low correlation between teachers' self-reported AI literacy and their demonstrated ability (Zhang et al., 2026).
AISA (aisa.to) was built to close that gap. It is an AI fluency assessment that measures what people do with AI, not what they say about it. It is not a survey, and it is not a quiz.
What AISA measures
AISA defines AI fluency as the ability to use AI effectively, critically and safely in real work. It breaks that into five dimensions, set out in a rubric that is published in full:
- Prompting & Comms: getting useful work out of AI.
- Critical Thinking: judging and checking what AI produces.
- Technical Understanding: knowing how AI works and where it fails.
- Workflow & Application: building AI into everyday work.
- Safety & Responsibility: using AI with sound judgement about data, risk and ethics.
Each criterion is scored from 1 to 10 on the published scale, so anyone can see what a given score means.
How the assessment works
A behavioural interview, run by AI
AISA works like a competency-based interview. Aisa, an AI interviewer, asks about real situations: how someone used AI on a recent piece of work, how they checked the output, what went wrong and what they did next. Past behaviour is the evidence.
This is the method personnel research rates most highly. Structured, behavioural interviews have ranked among the strongest predictors of job performance for decades (Schmidt & Hunter, 1998), and a 2022 re-analysis puts them first, at r = .42, ahead of cognitive ability tests at .31 (Sackett et al., 2022). AISA applies that method to AI fluency.
Exercises, not just talk
The conversation includes practical exercises: short games and "show and tell" tasks where people demonstrate how they work with AI instead of describing it. Demonstrated skill carries more weight than described skill.
Every score is tied to evidence
Each score traces back to verbatim quotes from the conversation. A reader of the report can see exactly what the person said that earned it.
Adaptive, but scored the same way for everyone
Questions adapt to each person's role, expertise and technical depth. A software engineer gets technically deeper questions than a marketing manager, and both get a conversation that fits their work. The scoring criteria and the rubric stay identical for everyone, so results remain comparable across people, teams and time.
Refined in use, current every week
According to AISA, the assessment has been refined across more than 3,000 assessments. Every week it is updated with the latest AI developments (new models, tools and practices), so the conversation stays current while the rubric and scoring criteria stay fixed.
Benchmarked against recognised frameworks
According to AISA's published methodology, the rubric is mapped against two external frameworks:
- Anthropic's AI Fluency research. AISA covers 93% of the observable fluency markers in Anthropic's study of around 10,000 conversations.
- The US Department of Labor's AI Literacy Framework. AISA covers all 25 of its sub-competencies.
AISA says its methodology is designed to the AERA/APA/NCME Standards for Educational and Psychological Testing and to ISO 10667, the international standard for assessment service delivery. The details are on the methodology page.
Measuring before and after training
Because the same assessment can be repeated, organisations use it at two points. Before training, it shows where the real AI skills gaps are, so the training budget goes where it is needed. After training, it shows whether people now work differently with AI. That second measurement is the evidence that a programme worked, rather than a count of who attended.
What the assessment provides
- Individuals get a full report with scores across the five dimensions and a LinkedIn-verifiable certificate, free.
- Organisations get an employer dashboard with an organisation-level AI fluency score, team-by-team breakdowns and benchmarks, updating as people complete the assessment.
- For development, there is a personalised learning plan, and an AI Coach that delivers lessons on WhatsApp, built from each person's results.
According to AISA, more than 3,000 professionals have taken it so far, including people at Amazon, Google, Deloitte, PwC and the NHS.
Questions enterprise buyers ask
How is this different from a quiz, a survey or usage analytics? Quizzes test what people know about AI. Surveys record what people believe about themselves, and self-ratings run well above measured ability, as the 18.5-point gap above shows. Usage analytics count logins, prompts or tokens, which shows activity, not skill. AISA measures how well people actually work with AI, from evidence in what they have done and what they show during the conversation.
Is AISA's method independently grounded? According to AISA, its framework is built on the research behind behavioural interviewing, has been reviewed by AI experts and industrial-organisational psychologists, and is mapped against Anthropic's AI fluency research and the US Department of Labor's AI Literacy Framework. The rubric and the methodology are published in full, so anyone can examine how scores are produced.
How reliable is a conversation-based score? AISA reports that 98% of people who retake the assessment land in the same or an adjacent tier, and that scores correlate at r = 0.79 between attempts.
If every conversation is different, are scores comparable? Yes. The questions adapt to the person, but every score is given against the same criteria and the same published rubric, and every score is tied to quoted evidence. Two people who had different conversations are measured on the same scale.
Does it favour people who are articulate? No. Scoring is behavioural: it rests on what people have actually built and done, not on how they communicate. A scoring system called Golden Shadow separates eloquence from evidence, so a polished talker does not score high on talk alone and a quieter, less verbose person is not marked down for style. Learning Agility credits people who are actively building their AI skills, and the games and exercises let people show what they can do rather than describe it.
Does it work for non-native English speakers and people who use assistive tools? People can answer by typing or by voice dictation in their own language, and the assessment works with other assistive tools. According to AISA, assessments taken in other languages score as high as those taken in English.
Can someone get AI to answer for them? AISA's anti-gaming measures work at the level of individual copy-and-paste events, and semantic analysis identifies AI-written content. Follow-up questions test whether people can explain and build on what they said. No webcam or locked-down browser is needed. Answers that are not the candidate's own are flagged in the report, and no certificate is issued for them.
What happens when the underlying AI models are updated? Every score is anchored to quoted evidence and to a published rubric that defines what each level means. The conversation is refreshed every week with the latest AI developments, while the rubric and the scoring criteria stay fixed, so a score means the same thing over time.
Is a single conversation enough? The conversation covers all five dimensions and includes exercises in which people show their approach. It is also repeatable: organisations measure again after training to see what changed, team by team.
Who is AISA for? Organisations first. AISA helps them see who is AI fluent across their workforce, focus AI training on the gaps that matter, and prove internally, with quantifiable metrics, that their AI investment is paying off. Individuals can also take it to see their AI fluency benchmark and improve.
Who is behind AISA? AISA is a UK company and an official member of UKAI, the UK's AI trade association. Its team brings backgrounds in psychometrics and hiring, behavioural science and AI product leadership, and according to AISA it already works with six enterprise organisations.
Where is the data held, and is it used to train AI? According to AISA, assessment data is hosted in the EU (AWS eu-west-1), used solely to generate the report, and never sold, shared with third parties or used to train AI models. People can ask for their data to be deleted at any time, and AISA deletes it within 30 days.
Is it only for technical roles? No. It measures how people work with AI in any role, from marketing and operations to engineering, and it adjusts its technical depth to the person.
You can take the assessment at aisa.to. The full methodology is at aisa.to/methodology, and teams can see how the dashboard works at aisa.to/ai-fluency-for-teams.