# Assessment Integrity in the Age of Generative AI *Keeping assessment evidence trustworthy when AI can produce the work.* Course ID: ICB-A02 | ICSF-ASMT-L4-A | Ability, level L4 "Assure" ## About this course Generative AI can write essays, solve problems, produce code, summarise readings and, increasingly, act as an agent that completes multi-step tasks. Every issuer of credentials now has to ask a direct question: when someone submits work, what does that work tell us about what they can do? This course gives you practice in answering that question for your own assessments. You will map integrity risks, redesign assessments, plan identity and supervision, draft a policy on permitted AI use, and plan how to explain all of this to earners. ### Who this course is for - Programme and micro-credential leads - Learning designers and teaching staff - Quality, registry and admissions staff - L&D and corporate academy leads - Professional bodies and sector associations - ICoBC Lead Reviewers ### Prerequisite ICB-P02 Assessment Design for Credentials: Valid, Reliable, Fair. You should be comfortable with ideas such as validity, reliability, fairness, constructive alignment and assessment blueprints. ### Learning outcomes After this course you can: 1. Analyse integrity risks of an assessment when generative AI and agents are available. 2. Redesign assessments to produce trustworthy evidence (authentic tasks, process evidence, oral and supervised elements). 3. Write an assessment integrity policy that states permitted AI use. 4. Explain why AI detection tools are not a reliable basis for decisions. ### Framework alignment This course aligns with the ICoBC Credentialing Skills Framework (ICSF), skill ASMT "Assessment and evidence" (VRITE pillar: Rigor), level L4 "Assure", stage A "Ability". - **Skill at this level:** leads assessment strategy for significant programmes; designs workplace-based competency assessment; manages integrity risks including generative AI. - **Stage definition:** practising a capability on realistic tasks, using the knowledge gained, in a controlled setting such as a case, simulation, sandbox or project. ### Workload 8 hours notional learning in a blended cohort. This file covers the self-paced part. Live or workshop elements appear as facilitator notes. Plan roughly 4 to 5 hours for the self-paced material and the remainder for the cohort sessions and your final task. ### How it is assessed You complete a short knowledge check inside this file. The main assessment is a practical task: an integrity review and redesign of an existing assessment from your own context, plus a draft policy. A person assesses it with the rubric in this file. Nothing in this file grades your task. ### Badge Completing the course and meeting the pass rule leads to the badge "Assessment Integrity in the Age of Generative AI (Ability, L4)", alignment code ICSF-ASMT-L4-A. The badge records that you completed practice tasks to a stated standard in a controlled setting. It does not prove that you can do any particular job. ### Outcomes, modules and assessment | Outcome | Module | How it is assessed | |---|---|---| | 1. Analyse integrity risks of an assessment when generative AI and agents are available | A threat model for assessment | Integrity review in the final task; knowledge check | | 2. Redesign assessments to produce trustworthy evidence | Design responses; Identity and supervision | Redesigned assessment in the final task; knowledge check | | 3. Write an assessment integrity policy that states permitted AI use | Policy; Communicating with earners | Draft policy in the final task; knowledge check | | 4. Explain why AI detection tools are not a reliable basis for decisions | Communicating with earners; Policy | Policy rationale in the final task; knowledge check | ### How the practice works Each module has a worked example, a practice activity and a quiz. Two settings run through the course: - **Setting A:** a university, college or training provider. Roles: programme leader, module tutor, registry officer. - **Setting B:** a corporate academy or HR department. Roles: academy lead, HR business partner, line manager, compliance officer. All numbers in examples are illustrative. Keep a working document as you go. Choose one existing assessment from your own context now. You will use it in every practice activity, and it becomes the basis of your final task. > Tip: pick an assessment that matters. A low-stakes quiz teaches little. Choose one that feeds a credential decision. ## A threat model for assessment In this module you learn to look at an assessment the way a security analyst looks at a system. You list what could go wrong, how likely it is, and how much it matters. This addresses learning outcome 1. ### Why a threat model An assessment produces evidence. A decision, such as awarding a badge or certificate, rests on that evidence. If the evidence no longer reflects what the earner can do, the decision loses its basis. The credential then carries less trust for everyone who relies on it. Integrity problems are older than generative AI. Impersonation, collusion, plagiarism and contract cheating all existed before. Generative AI changes three things: - **Cost:** producing plausible work now takes seconds and often costs nothing. - **Scale:** one person can produce many versions, and anyone can do it. - **Quality and fit:** output can match the task wording closely, and it can be adjusted to a target style or level. Agents add a fourth change. An AI agent can carry out a multi-step task: browse a source, run a calculation, fill in a form, write a report and submit it. Tasks that once needed a human at each step may need one at the start only. A threat model asks four questions: 1. What claim does this assessment support? (For example: "this person can analyse a financial statement.") 2. What evidence do we collect to support that claim? 3. How could that evidence be produced without the claimed ability? 4. What would that cost the credential's trust, and who is affected? ### Exposure: what makes an assessment vulnerable Exposure to AI depends on the task, not on the topic. Look at these factors: - **Output type:** text, code, images and structured analysis are easy for generative AI to produce. Live performance and physical handling are harder. - **Task openness:** generic prompts such as "discuss the causes of..." are easy to answer with AI. Tasks tied to local data, personal experience or a specific recent event are harder. - **Setting:** unsupervised, take-home work has high exposure. Supervised, live work has lower exposure. - **Evidence of process:** if only the final product is collected, there is little to check. Drafts, decision logs and discussion give more to examine. - **Stakes:** the higher the stakes, the stronger the incentive to cheat, and the greater the damage. {{1}} The matrix is a starting point. A supervised exam can still be compromised by impersonation. An oral exam can be undermined by a hidden assistant. The matrix shows tendencies, so use it to prompt questions, not to replace analysis. ### Building a risk register A simple risk register for one assessment has these columns: | Column | Question | |---|---| | Claim | What ability does a pass claim? | | Evidence | What artefact or observation supports it? | | Threat | How could the evidence be produced without the ability? | | Likelihood | Low, medium or high, with your reasoning | | Impact | What happens to trust if this occurs? | | Existing control | What already reduces the risk? | | Gap | What remains uncovered? | Write likelihood and impact as short judgements with reasons. A bare "high" is not useful to a reviewer. "High: the prompt is generic and submission is unsupervised" is. **Worked example: Risk register for a data analysis assignment (Setting A)** A programme leader at a college reviews a unit called "Introduction to data analysis". The summative task is a 2,000-word report on a public dataset, submitted online over two weeks. It counts for 60 per cent of the unit grade (illustrative). She builds this register: | Claim | Evidence | Threat | Likelihood | Impact | Existing control | Gap | |---|---|---|---|---|---|---| | Learner can select and justify an analysis method | Final report | AI selects the method and writes the justification | High: dataset is public, prompt is generic | High: unit is a gateway to the diploma | Similarity checker | Similarity checker does not address generated text | | Learner can interpret results in context | Final report | AI interprets output pasted in by the learner | Medium: learner must still run the analysis | Medium | None | No check that learner understood the output | | Learner can work independently | Final report | Collusion or contract cheating | Medium | High | Declaration of originality | Declaration is easy to sign without effect | She notes a general point. The report shows a polished result. It shows nothing about how the learner reached it. That gap is the root of the exposure, and it guides the redesign in Module 2. **Practice activity: Build a risk register for your own assessment** 1. Take the assessment you chose in "About this course". 2. State the claim it supports, in one sentence beginning "A person who passes can...". 3. List the evidence you collect. 4. Identify at least three threats. Include one involving a generative AI agent and one that existed before AI. 5. Fill in a risk register with the columns above. Give a reason for every likelihood and impact rating. 6. Mark the single largest gap and write one sentence on why. Keep the register. It is the core of the "integrity review" in your final task.

Facilitator notes

**Live session (60 minutes).** Run a "red team" exercise. In groups of three, participants swap their assessments and try to find the quickest route to a passing submission using generally available AI tools. Ask groups to record the route, the time taken and the evidence that would reveal it. Debrief with a plenary on which threats were unexpected. Cautions: do not ask participants to run real learner submissions through tools. Use anonymised or invented material. Do not name commercial tools in plenary notes. Remind participants that the goal is to find gaps in design, not to blame learners. Timing: 10 minutes brief, 30 minutes group work, 20 minutes debrief.

### Quiz: A threat model for assessment **Question 1.** Which feature most raises an assessment's exposure to generative AI? [( )] The assessment is worth a high proportion of the final grade. [(X)] The task is generic, produces text only, is unsupervised and collects only the final product. [( )] The assessment is marked by two assessors. [( )] The assessment is held at the end of the term. [[?]] Think about what makes it easy to produce the output without the ability. **** Generic prompts, text output, no supervision and no process evidence together make it easy to generate a plausible submission. High stakes raise the incentive but do not change how easy the task is to complete with AI. Double marking and timing are not exposure factors in themselves. **** **Question 2.** What is the first question in a threat model for an assessment? [( )] Which AI tool will learners use? [( )] Which detection tool should we buy? [(X)] What claim does a pass support? [( )] How many learners will cheat? [[?]] Start with what the assessment is supposed to show. **** A threat model starts from the claim, because every threat is a way of producing evidence for the claim without the ability behind it. Tools and cheating rates are secondary questions. **** **Question 3.** What does an AI agent add to the risk picture compared with a chat-style assistant? [( )] It makes plagiarism checkers more accurate. [(X)] It can carry out several steps of a task, such as researching, calculating and submitting, with little human input. [( )] It removes the need for identity checks. [( )] It only works on paper-based assessments. [[?]] Think about what "agent" means: acting over several steps. **** Agents can complete multi-step tasks, so assessments that rely on a sequence of ordinary steps may no longer need a human at each step. This increases exposure for tasks that were previously safe because they were laborious. **** **Question 4.** In a risk register, which entry for "likelihood" is most useful to a reviewer? [( )] High [( )] Probably fine [(X)] High: the prompt is generic and submission is unsupervised [( )] Not applicable [[?]] A rating without a reason cannot be challenged or revisited. **** A rating with a stated reason can be reviewed and improved. Bare ratings hide the logic and make later decisions harder to defend. **** ## Design responses This module addresses learning outcome 2. You now move from finding risks to redesigning the assessment so that its evidence can be trusted. ### Principles for redesign Do not start by banning tools. Start by asking what evidence would show the ability. Then ask how you could collect that evidence in a way that is hard to produce without the ability. Four principles guide redesign: 1. **Align to the claim.** Evidence must match what the pass claims. If the claim is "can analyse data", the evidence should show analysis happening. 2. **Collect process, not only product.** Drafts, notes, decision logs and tool histories show how work developed. Process evidence is harder to fake and gives assessors more to examine. 3. **Add a live element at critical points.** A short conversation, demonstration or defence lets an assessor probe understanding directly. 4. **Be proportionate.** Not every assessment needs every safeguard. Match the strength of control to the stakes and the exposure. ### Design options Common options fall into five groups: - **Authentic tasks.** Use tasks tied to real or realistic contexts: a local case, a workplace problem, a live client brief. These tasks require choices, trade-offs and judgement. Authentic does not mean immune to AI, but it can raise the quality of evidence and reduce generic answers. - **Process evidence.** Ask for a short planning note, a draft with comments, a log of decisions, version history or a reflective commentary on choices made. - **Oral and interactive elements.** A short viva, a walk-through of the work or a scenario discussion. The assessor asks follow-up questions that depend on the submitted work. - **Supervised elements.** Sit part of the assessment in a controlled setting: a timed task, a practical demonstration, or a supervised analysis using provided data. - **Open AI use with accountability.** Where AI use is permitted, ask learners to document how they used it, what they checked and what they changed. Assess their judgement, verification and ownership of the final product. {{1}} ### Trade-offs Every option has costs. Plan for them. - **Oral elements** take assessor time and can disadvantage anxious or non-native speakers. Offer adjustments and clear question formats. - **Supervised elements** need rooms, invigilators or secure platforms, and can limit authenticity. - **Process evidence** adds workload for learners and assessors. Ask for a small number of well-chosen artefacts. - **Authentic tasks** take design effort and can be unequal if some learners have better access to real contexts. - **Open AI use** needs clear rules and assessor training so that judgements stay consistent. Fairness matters throughout. A redesign that improves integrity but disadvantages some groups is a poor redesign. Check each change against accessibility and reasonable adjustments, using what you learned in ICB-P02. ### Keeping the claim visible After redesign, check alignment again. Write the claim, then list how each part of the new assessment supplies evidence for it. If a part supplies no evidence for the claim, remove it. If part of the claim has no evidence, add something. **Worked example: Redesigning a client report assessment (Setting B)** An academy lead at a mid-sized company runs an internal certificate in "Customer insight". The final assessment is a written client insight report based on a supplied dataset, submitted by email over one week, and marked by the line manager (illustrative). The risk register shows high exposure. The dataset is shared, the prompt is generic, and the report is the only evidence. Managers report that recent submissions look polished but vary little in structure. The academy lead redesigns the assessment in four steps: 1. **Keep the claim:** "A person who passes can turn customer data into a recommendation that a manager could act on." 2. **Make the task authentic:** each participant picks a real, current question from their own team, with manager approval, and uses data they are permitted to access. 3. **Add process evidence:** participants submit a one-page plan before analysis, and a short decision log recording which choices they made and why. 4. **Add a live element:** a 15-minute conversation with a manager and one academy assessor, where the participant walks through the recommendation and answers two prepared questions and two follow-ups. AI use is permitted for drafting and for checking calculations, provided participants record what they used it for and what they verified. The assessor scores the recommendation, the reasoning in the log and the conversation. Each element has a criterion in the rubric. The academy lead estimates the extra assessor time at about 25 minutes per participant (illustrative) and decides it is proportionate to the stakes, since the certificate feeds into promotion discussions. **Practice activity: Redesign your assessment** 1. Return to your risk register from Module 1 and the largest gap you marked. 2. Write the claim again, in one sentence. 3. Choose at least three design options from this module. For each, state what evidence it adds and which threat it reduces. 4. Describe the redesigned assessment in a short specification: task, evidence collected, who assesses, when, with what AI use permitted. 5. List two costs or risks of your redesign, and one measure to reduce each. Include one fairness check. 6. Update the register with the remaining gaps.

Facilitator notes

**Live workshop (90 minutes).** Run a design clinic. In groups of three, one participant presents their redesign in five minutes. The group asks: "What would a determined person do to pass this without the ability?" and "Who might this disadvantage?" Rotate presenters. Provide a template for the one-page assessment specification so that outputs are comparable. Encourage participants to simplify. Over-designed assessments rarely survive contact with workload. Watch for: designs that rely on a single safeguard, oral elements with no scoring criteria, and redesigns that raise assessor workload beyond what the programme can carry.

### Quiz: Design responses **Question 1.** What should drive the choice of redesign options? [( )] Whichever option is cheapest. [(X)] The claim the assessment supports, the exposure, and the stakes. [( )] What other providers have chosen. [( )] The tools learners already use. [[?]] Think about proportion and purpose. **** Redesign starts from the claim and the risks. Options are matched to exposure and stakes. Cost matters but does not decide the design on its own, and copying others or following tools skips the analysis. **** **Question 2.** Why does process evidence strengthen an assessment? [( )] It replaces the need for a final product. [( )] It proves the learner did not use AI. [(X)] It shows how the work developed and gives assessors more to examine. [( )] It removes the need for assessor judgement. [[?]] Consider what an assessor can check beyond the final product. **** Process evidence shows decisions, drafts and reasoning over time. It does not prove that AI was not used, and it still needs assessor judgement. It usually complements the product. **** **Question 3.** A short oral element is added to a written assessment. What must be planned alongside it? [( )] A rule that all learners must pass the oral element with full marks. [( )] A ban on written work. [(X)] Scoring criteria, question formats and adjustments for learners who need them. [( )] A requirement that assessors do not ask follow-up questions. [[?]] Fairness and consistency matter. **** An oral element needs criteria so assessors score consistently, defined question formats, and adjustments so that it does not disadvantage particular groups. Follow-up questions are what make the element useful. **** **Question 4.** In an assessment that permits AI use, what is a sensible thing to assess? [( )] The speed at which the learner produced the work. [( )] Whether the learner can avoid every mention of AI. [(X)] The learner's judgement, verification of output and ownership of the final product. [( )] The length of the AI conversation history. [[?]] If the tool is allowed, which human abilities still matter? **** Where AI use is permitted, the human contribution is judgement, checking and accountability. Speed, avoidance and log length say little about the ability the pass claims. **** ## Identity and supervision This module continues learning outcome 2 and adds planning for who did the work and under what conditions. Redesigned tasks only help if you also know the right person completed them. ### Identity: who is the earner An assessment result is a statement about a particular person. Identity risk appears in several forms: - **Impersonation:** someone else sits the assessment. - **Contract work:** someone else completes the work and the earner submits it. - **Shared access:** credentials or sessions are shared during the assessment. - **Synthetic identity:** a fabricated persona, or an AI-generated voice or video, used in remote settings. Identity checks should be proportionate. For a low-stakes formative task, a login may suffice. For a credential decision with external value, stronger checks are justified. Options include: - Checking identity documents at registration and again at critical assessment points - Using a known contact such as an employer or tutor to confirm the person - Live, on-camera or in-person identity confirmation before a viva or exam - Comparing the work with earlier work by the same person, as a prompt for a conversation, never as proof on its own - Secure assessment platforms that restrict other applications during a task Each option brings privacy and data protection duties. Collect only what you need, tell earners what you collect and why, and keep it only as long as necessary. Check the rules that apply in your jurisdiction, including data protection law. ### Supervision: the conditions of the assessment Supervision means that someone, or something, observes or controls conditions during assessment. It ranges from none to full invigilation. | Level | Description | Typical use | |---|---|---| | Unsupervised | Work done in the earner's own time and place | Formative tasks, reflective work | | Light supervision | Check-ins, supervised start and finish, or progress meetings | Projects, extended tasks | | Controlled | Timed, in a set setting, limited resources | Exams, practical tests | | Observed | Assessor watches the work happen | Workplace-based assessment, practical skills | Higher supervision reduces exposure and raises cost. It can also reduce authenticity. A workplace-based assessment, where a trained assessor observes real work, can be both authentic and controlled, which is why it is attractive for competency claims. ### Remote and hybrid settings Remote supervision brings extra risks: hidden devices, off-screen helpers and screen sharing. It also brings extra concerns: privacy in earners' homes, bandwidth, and fairness for people without a quiet space. Where you use remote supervision: - Say clearly what is monitored and why. - Offer an alternative where monitoring would cause harm or unfair disadvantage. - Do not treat automated flags as proof. A flag should prompt human review. - Record decisions and keep a route to appeal. ### Workplace-based assessment Workplace-based assessment collects evidence from real work. Typical elements are: - Direct observation by a qualified assessor against stated criteria - Work products with context, such as a customer file or a project output - Witness statements from colleagues or managers - Professional discussion linking evidence to the competency standard Integrity risks differ here. Rather than copying, you manage conflicts of interest (a manager who wants a favourable result), inconsistent standards between assessors, and thin evidence. Mitigations include assessor training, sampling by a second assessor and clear evidence rules. {{1}} **Worked example: A hybrid assessment with a supervised checkpoint (Setting A)** A university department runs a short micro-credential in "Project planning for research teams". The assessment is a project plan for a fictional research grant, completed over three weeks. The module tutor and registry officer review the integrity risks (illustrative scenario). Risks identified: the plan can be drafted by AI, a third party could write it, and the registry has no direct way to confirm who submitted it. The cohort is 60 people, studying part-time and across several time zones. They choose a proportionate design: - **Identity:** registration uses the university's standard identity process. At the final checkpoint, earners join a short video call and confirm identity against their student card. - **Process evidence:** earners submit a one-page outline in week one and a revised plan in week two, each with a short comment on what changed and why. - **Checkpoint conversation:** in week three, each earner has a 12-minute conversation with the tutor, who asks about three decisions in the plan and one "what if" scenario. - **AI use:** permitted for brainstorming and language editing. Earners list any use in an appendix. - **Adjustments:** earners who cannot join a video call may take the conversation by phone, or in person on campus. The registry officer records which earners completed the checkpoint, and keeps notes on any concerns for review by a second assessor before decisions are made. No decision relies on a single flag or impression. Tutor time is about 12 minutes of conversation plus 5 minutes of notes per earner, around 17 hours across the cohort (illustrative). The department agrees this is workable because the micro-credential runs twice a year. **Practice activity: Plan identity and supervision** 1. Take your redesigned assessment from Module 2. 2. State the identity risk and the supervision level that fits the stakes. Give your reasons. 3. List the identity checks you will use. For each, name the data you collect and how long you keep it. 4. Describe one adjustment for earners who cannot take part in a standard check or supervision arrangement. 5. State what happens if a concern arises: who reviews, on what evidence, and how earners can respond. 6. Add one line to your risk register for any new risk created by these measures.

Facilitator notes

**Live session (45 minutes).** Use a case discussion. Present a scenario in which an automated remote supervision tool flags an earner for "unusual eye movement". Ask groups what decision process would be fair, what additional evidence they would seek, and how they would communicate with the earner. Aim to draw out that flags are prompts for human review and that earners have a right to explain. Keep the discussion vendor-neutral and avoid naming products. Cautions: do not use real earner data. Be sensitive to participants who have experienced disability-related disadvantage in supervised settings.

### Quiz: Identity and supervision **Question 1.** Which statement about identity checks is most sound? [( )] Every assessment should use the strongest check available. [( )] A login is always enough. [(X)] Checks should be proportionate to the stakes, and only the data needed should be collected. [( )] Identity checks are not a data protection matter. [[?]] Think about proportion and privacy together. **** Stronger checks cost more and collect more personal data. They are justified where the stakes and risks are high. Data protection duties apply to identity data like any other personal data. **** **Question 2.** An automated tool flags an earner during a remotely supervised test. What is the best practice? [( )] Fail the earner automatically. [( )] Ignore flags, since tools are unreliable. [(X)] Treat the flag as a prompt for human review, gather further evidence and let the earner respond. [( )] Publish the flag to the cohort as a warning. [[?]] A flag is a question, not a finding. **** Automated flags can be wrong. A fair process uses them to start a human review, collects more evidence, and gives the earner a chance to respond before any decision. **** **Question 3.** Why is workplace-based assessment attractive for competency claims? [( )] It removes the need for assessor training. [(X)] It collects evidence from real work, which can be both authentic and observed. [( )] It guarantees that no integrity problems occur. [( )] It is always cheaper than exams. [[?]] Think about what is being observed. **** Observed real work provides authentic evidence under controlled conditions. It brings its own risks, such as conflicts of interest and inconsistent standards, so assessor training and sampling remain necessary. **** **Question 4.** What is a sensible response where remote monitoring would disadvantage some earners? [( )] Require everyone to use it anyway. [( )] Remove all supervision. [(X)] Offer a fair alternative, such as an in-person or phone-based option. [( )] Ask earners to prove their disadvantage in writing first. [[?]] Integrity measures must still be fair. **** Integrity measures should not create unfair disadvantage. An alternative route keeps the standard the same while allowing different ways of meeting the conditions. **** ## Policy This module addresses learning outcome 3 and part of learning outcome 4. You write an assessment integrity policy that says clearly what AI use is permitted, and you set out how concerns are handled. ### What a policy is for A policy tells earners, assessors and reviewers what is expected. It makes decisions consistent and defensible. A good integrity policy is short enough to read and specific enough to apply. A weak policy says "AI must be used responsibly". A strong policy says which uses are permitted for which tasks, what must be declared and what happens if rules are broken. ### A model of permitted use A simple approach sets levels of AI use, assigned to each assessment: | Level | Name | Description | |---|---|---| | 0 | No AI | The work must be done without generative AI. Typically used in supervised or live elements. | | 1 | Limited AI | AI may be used for specified purposes only, such as spelling and grammar checks or finding sources, and must be declared. | | 2 | Open AI with accountability | AI may be used widely. Earners must document use and verify output, and are assessed on judgement and ownership. | | 3 | AI as the subject | The task requires AI use and assesses the ability to use it well. | Assign a level to each assessment, and state it in the assessment brief. Different elements of one assessment can have different levels. For example, a written plan at level 2 and a live conversation at level 0. ### Elements of an integrity policy A policy should include: 1. **Purpose and scope.** Which credentials, assessments and people it covers. 2. **Definitions.** What counts as generative AI, an agent, misconduct and permitted assistance. 3. **Permitted use.** The levels above, how each assessment is assigned a level, and where earners find the level. 4. **Declaration.** What earners must declare and in what form. 5. **Assessment design commitments.** The issuer's own duties: identity, supervision, process evidence and oral elements in proportion to stakes. 6. **Handling concerns.** How concerns are raised, who reviews them, what evidence counts, and how earners respond. 7. **Use of detection tools.** A statement that automated detection outputs are not used as the basis for decisions (see below). 8. **Outcomes and appeals.** Possible outcomes, proportionality, and a route to appeal. 9. **Adjustments.** How reasonable adjustments are requested and applied. 10. **Data and privacy.** What data is collected for integrity purposes, why, and for how long. 11. **Review.** Who owns the policy and when it is reviewed. ### Why detection outputs are not a basis for decisions This links to learning outcome 4. AI detection tools estimate whether text was likely machine-generated. They do not establish who wrote it. Reasons to avoid using them as the basis for decisions: - **They give probabilities, not proof.** A score is an estimate with error, and false positives and false negatives both occur. - **They can disadvantage some writers.** Writers working in a second language, or with a plain or formulaic style, may be flagged more often. Check current research before making claims about particular rates.- **They are easy to evade.** Light editing or paraphrasing can change results, so the tools offer little protection against determined misuse. - **They cannot say how AI was used.** Permitted uses, such as editing, can look the same as prohibited ones. - **Results are hard to explain or challenge.** Earners cannot easily contest a score whose method is not transparent. The status for this topic is established: detection tools are not a reliable basis for decisions. A policy can allow a score to prompt a conversation, but a decision needs other evidence, such as a discussion of the work, process evidence and comparison with other work, assessed by a person. ### Regulatory context Regulation affects how assessment systems may be used, especially where AI is part of the system. Under the EU AI Act, certain uses of AI in education and training are classified as high-risk, including some uses connected with assessment. Under the AI Omnibus (provisional agreement of 7 May 2026), Annex III high-risk obligations apply from 2 December 2027 and Annex I product-embedded systems from 2 August 2028. Check final published text. Status: November 2026. For policy writers, the practical point is to record where any AI tool is used in assessment decisions, keep human oversight, and seek legal advice for your own situation. This course does not give legal advice. {{1}} **Worked example: A policy extract for a corporate academy (Setting B)** An academy lead and a compliance officer at a company draft the integrity section for an internal certificate programme in "Supplier risk review" (illustrative scenario). They write this extract: > **Permitted AI use.** Each assessment brief states a level of AI use. For the written risk summary, level 2 applies: participants may use generative AI tools approved by the company, must list each use in the declaration form, and must check all facts and figures against source documents. For the review conversation with the line manager, level 0 applies. > > **Declaration.** Participants state which tools they used, for what purpose, and what they changed or checked. A missing or inaccurate declaration is handled under the concerns process. > > **Concerns.** Anyone may raise a concern. The academy lead appoints a reviewer who was not involved in marking. The reviewer considers the submission, the process evidence and a conversation with the participant. The participant may bring a colleague or representative. > > **Detection tools.** The academy does not use automated AI detection scores as the basis for any decision. A score, where one is available, may prompt a conversation. It is not evidence of misconduct. > > **Outcomes.** Outcomes are proportionate: a resubmission, a reassessment, or, in serious and confirmed cases, withdrawal of the certificate. Participants may appeal to the head of HR. > > **Data.** Declarations and review notes are kept for two years and then deleted (illustrative). The compliance officer adds a note to check data retention against company policy and local law, and to confirm that company-approved AI tools are covered by existing data protection agreements. **Practice activity: Draft the policy** 1. Using the model in this module, write a draft integrity policy of one to two pages for the credential or programme your assessment belongs to. 2. State the AI use level for each element of your redesigned assessment. 3. Write the declaration requirement in plain words. 4. Write the section on detection tools, with a short rationale an earner could understand. 5. Describe the process for handling concerns, naming roles rather than individuals. 6. Add a sentence on adjustments, appeals and data retention, and mark any point you need to check with legal or data protection colleagues.

Facilitator notes

**Live session (60 minutes).** Run a policy peer review. Pairs swap drafts and use a short checklist: Is permitted use clear for each assessment? Can an earner tell what to declare? Is the concerns process fair and specific? Is the position on detection tools explicit? Ask each reviewer to find one sentence that could be read two ways, and propose a clearer version. Cautions: participants may ask for legal advice. Remind them that the course gives practical guidance and that regulatory points need checking with qualified advisers and the final published texts.

### Quiz: Policy **Question 1.** Which policy statement is most useful to an earner? [( )] "Use AI responsibly." [( )] "AI is discouraged." [(X)] "For the written summary you may use AI for drafting and must list each use in the declaration form. For the live conversation, no AI may be used." [( )] "Staff will decide what counts as acceptable." [[?]] Look for specifics: what, where and how to declare. **** The strongest statement names the permitted uses, links them to specific elements and states what to declare. The others leave earners guessing. **** **Question 2.** What is a sound position on AI detection scores in an integrity policy? [( )] A score above a set threshold is sufficient evidence of misconduct. [( )] Scores should be kept secret from earners. [(X)] A score may prompt a conversation but is not the basis for a decision. [( )] Scores are only used for earners who write in a second language. [[?]] Consider reliability and fairness. **** Detection tools produce estimates that can be wrong and may affect some groups more than others. A fair policy uses other evidence reviewed by a person. **** **Question 3.** Which element belongs in an integrity policy? [( )] A promise that no earner will ever be investigated. [(X)] A route for raising concerns, a reviewer not involved in marking, and an appeal route. [( )] A list of commercial products that earners must buy. [( )] A ban on all adjustments to assessment conditions. [[?]] Fair process matters as much as rules. **** A fair policy states how concerns are raised, who reviews them, and how earners can appeal. Policies should be vendor-neutral and should allow reasonable adjustments. **** **Question 4.** What is the best practical response to the EU AI Act timelines in this course's guidance? [( )] Quote the dates as final and give legal advice. [( )] Ignore regulation until it applies. [(X)] Note the stated dates, check the final published text, record where AI is used in assessment decisions and seek qualified advice. [( )] Assume all assessment uses of AI are banned. [[?]] The timeline stated here comes with a "check final text" note. **** The dates come from a provisional agreement and the final text must be checked. Good practice is to record where AI supports decisions, keep human oversight and take qualified advice for your situation. **** ## Communicating with earners This module addresses learning outcome 3 and learning outcome 4 from the earner's side. A policy only works if earners understand it, trust it and know how to act on it. ### Why communication is part of integrity Many integrity problems come from unclear rules. Earners may not know which uses are allowed. Some may be afraid to declare use because they expect to be penalised. Others may assume that everyone else is using AI freely. Clear communication does three things: - It helps honest earners comply. - It builds trust in the credential. - It gives a fair basis for acting when rules are broken. ### What to tell earners, and when | Moment | What to communicate | |---|---| | Before enrolment | The integrity approach, the types of assessment, the general stance on AI | | At the start of the course | The policy, how levels of AI use work, where to find them, support available | | In each assessment brief | The level of AI use, what to declare, the evidence required, the date and format of any live element | | Before a live element | What to expect, how long it lasts, adjustments, what the assessor will ask about | | When a concern arises | What the concern is, what evidence will be looked at, the earner's right to respond, timelines | | After a decision | The outcome, the reasons in plain words, how to appeal | ### Principles for the message - **Say what is permitted, not only what is banned.** Give examples of acceptable and unacceptable use for the actual task. - **Explain the reason.** Earners are more likely to follow a rule they understand. "We ask for a decision log because it shows your reasoning, which is what the badge records." - **Make declaring safe.** If honest declaration of permitted use leads to penalties, earners will not declare. State that permitted use declared properly carries no penalty. - **Be consistent.** The brief, the policy and the practice of assessors must match. - **Be accessible.** Use plain language, offer formats that work with assistive technology, and give a contact for questions. ### Talking about detection tools If you do not use detection tools as the basis for decisions, say so and say why. A short statement can read: > We do not use automated AI detection scores to decide whether work is your own. These tools give estimates that can be wrong. If we have a concern, we look at your drafts and notes and talk with you about the work. This protects earners from fear of false accusations and tells them that process evidence matters. It also explains why you ask for it. If earners tell you they have received a detection result from elsewhere, treat it as one input to a conversation, not as a finding. ### Handling a conversation about a concern A conversation about a concern should be calm, factual and fair: 1. State the concern and the evidence behind it. 2. Ask open questions about how the work was produced. 3. Ask the earner to explain specific parts of the work. 4. Listen, and note what is said. 5. Explain the next steps and timelines. 6. Record the conversation and the reasons for the decision. Avoid accusing language, and avoid relying on impressions such as "this does not sound like you". Point to specific features of the work and specific gaps in understanding. ### Supporting earners Integrity depends on support as well as control. Offer: - Examples of acceptable use for common tasks - Guidance on how to cite or declare AI use - Study skills support, so that earners who struggle are not driven to shortcuts - A named contact for questions, with a promise that asking is safe {{1}} **Worked example: Briefing earners and handling a concern (Setting A)** A college runs a short course in "Community health outreach" with a written case analysis and a ten-minute discussion with the tutor (illustrative). The programme leader prepares an assessment brief for earners. The brief says: - **AI level:** level 2 for the written case analysis. Earners may use AI for brainstorming, structuring and language support. They must record each use, check every fact against course materials and sign the declaration. - **AI level:** level 0 for the discussion. - **Why:** "The discussion lets you show your thinking. It is the same for everyone, and it protects the value of your certificate." - **Detection:** "We do not use AI detection scores to decide anything. If we have a concern, we will talk with you." - **Safe declaration:** "Permitted use that you declare correctly carries no penalty." After submission, the tutor notices that one analysis refers to a policy document the course did not cover, and the earner could not describe it in the discussion. She does not claim the work is AI-generated. She asks the programme leader to appoint a reviewer. The reviewer meets the earner. He states the concern: the analysis cites a document that is not in the course materials and that the earner did not explain. He asks how the earner found it. The earner explains that she used an AI tool to suggest sources and did not check them, and had listed the tool in her declaration. The reviewer finds that the use was permitted and declared, but that verification was missing, which the brief required. The outcome is a resubmission with verified sources, and a note on the file. The reviewer explains the decision in plain words and tells the earner how to appeal. He also tells the programme leader that the brief should include an example of how to verify sources, and the programme leader adds one for the next cohort. **Practice activity: Write the earner-facing message** 1. Write the integrity section of an assessment brief for your redesigned assessment, in no more than 250 words. Include: the AI level for each element, what to declare, why the rules exist, the statement on detection tools and the route to ask questions. 2. Write a short script (five to eight lines) for opening a conversation about a concern, using calm and specific language. 3. List three types of support you will offer earners. 4. Ask a colleague, or read aloud to yourself, and mark any sentence that sounds like an accusation or could confuse an earner. Rewrite it.

Facilitator notes

**Live session (60 minutes).** Run a role-play of a concern conversation in groups of three: tutor, earner and observer. Provide a short scenario card, such as an analysis with an unexplained source, or a polished submission by an earner who struggled in earlier work. The observer uses a checklist: stated the concern clearly, asked open questions, avoided impressions, explained next steps. Rotate roles. In the debrief, collect phrases that worked and phrases to avoid. Cautions: some participants may have experienced accusations themselves. Offer the option to observe rather than play the earner. Keep the scenarios invented.

### Quiz: Communicating with earners **Question 1.** Why should a policy say that properly declared, permitted use carries no penalty? [( )] So that earners use AI more often. [(X)] So that earners feel safe declaring, which gives the issuer honest information. [( )] So that assessors do not need to review declarations. [( )] So that detection tools become unnecessary. [[?]] Think about what discourages honest declaration. **** If declaring leads to penalties, earners will hide use. A clear promise helps honest declaration, which supports review and fair decisions. **** **Question 2.** Which opening is best for a conversation about a concern? [( )] "Your work does not sound like you wrote it." [( )] "The detection score says 92 per cent." [(X)] "Your analysis cites a document that is not in the course materials. Can you tell me how you found it?" [( )] "Please confirm you cheated." [[?]] Be specific, factual and open. **** A specific, factual observation followed by an open question gives the earner a fair chance to explain. Impressions and scores are weak grounds and accusations close the conversation. **** **Question 3.** What is the best reason to tell earners that you do not use detection scores to decide? [( )] It means you can ignore integrity concerns. [( )] It guarantees that no one uses AI improperly. [(X)] It reduces fear of false accusations and explains why you ask for process evidence. [( )] It is required for every credential. [[?]] Consider trust and the reasons for process evidence. **** Saying so reassures earners, sets expectations about how concerns are handled and explains the purpose of drafts, notes and live discussions. It does not mean concerns are ignored, and no universal requirement is claimed. **** **Question 4.** Which support measure best reduces the pressure to take shortcuts? [( )] Longer deadlines for everyone, with no other change. [( )] Removing all assessments. [(X)] Examples of acceptable use, study skills support and a safe route to ask questions. [( )] Publishing the names of earners who ask for help. [[?]] Pressure often comes from uncertainty and lack of support. **** Clear examples, practical support and a safe way to ask questions help earners who are unsure or struggling. Publishing names would discourage help-seeking. **** ## Assessment and badge ### The task You complete an integrity review and redesign of an existing assessment from your own context, plus a draft policy. A person assesses your submission with the rubric below. This file does not grade your task. Choose a real assessment that feeds a credential decision, or a realistic one from your setting. Remove personal data and anything confidential before you submit. ### Evidence to submit For the badge ICSF-ASMT-L4-A, submit three items: 1. **Review.** A risk register for the existing assessment (claim, evidence, threats including AI agents, likelihood and impact with reasons, existing controls, gaps), and a short summary of the main findings (up to 400 words). 2. **Redesigned assessment.** A specification of the redesigned assessment: claim, task, evidence collected, identity and supervision measures, AI use level for each element, assessor roles, fairness and adjustment measures, and a note on costs and trade-offs (up to 800 words). 3. **Draft policy.** An assessment integrity policy of one to two pages that states permitted AI use, declaration, handling of concerns, the position on detection tools with a rationale, outcomes and appeals, data, and a short earner-facing message. ### Rubric Each criterion is rated from 0 to 3. A person assesses the evidence against the descriptors. | Criterion | 0 Not shown | 1 Developing | 2 Meets the standard | 3 Strong | |---|---|---|---|---| | A. Risk analysis (outcome 1) | No usable analysis of risks | Lists generic risks, few reasons, little link to the claim | Risks tied to a stated claim, includes AI agents and non-AI threats, likelihood and impact with reasons, gaps identified | As level 2, with prioritised gaps, consideration of stakes and context, and clear links between findings and design choices | | B. Redesign (outcome 2) | No redesign, or a ban only | Adds one safeguard without a link to risks | Several measures matched to identified risks, evidence aligned to the claim, identity and supervision in proportion to stakes | As level 2, with considered trade-offs, workload estimates, fairness checks and a coherent whole that an assessor could run | | C. Permitted AI use in policy (outcome 3) | Not stated | General statements only | Clear levels or rules per assessment element, a declaration requirement and a concerns process with roles | As level 2, with consistent wording across policy and earner message, adjustments, appeals, data retention and review owner | | D. Detection tools (outcome 4) | Not addressed, or relies on scores as evidence | Mentions limits without reasons | States that scores are not the basis for decisions, gives accurate reasons, names what evidence is used instead | As level 2, with a fair process for handling any score an earner presents, and plain-language explanation for earners | | E. Communication and fairness | Not addressed | Earner message is unclear or accusatory | Earner message is clear, specific, safe for declaration and accessible; adjustments mentioned | As level 2, with tested wording, support measures and alignment across all documents | | F. Accuracy and caution | Contains invented claims | Some unsupported claims | Claims limited to what can be supported; regulatory points flagged for checking | As level 2, with sources named correctly and clear notes on what needs legal or data protection review | ### Pass rule To earn the badge: - Achieve at least 2 on each of criteria A to D, and at least 1 on criteria E and F. - Achieve a total of at least 14 out of 18. - Complete the knowledge check below with at least 8 correct answers out of 10. If you do not meet the standard, the assessor gives feedback and you may resubmit once. The badge records that you completed practice tasks to a stated standard in a controlled setting. It does not prove that you can do any particular job. ### Facilitator and assessor notes

Facilitator notes

**Live element (about 90 minutes).** Hold a final clinic where participants present a two-minute summary of their redesign to a small panel. The panel asks one question about threats and one about fairness. This is formative; the formal assessment uses the written evidence and the rubric. Assessor guidance: moderate a sample of submissions between two assessors. Check that criteria A to D are applied consistently, especially the distinction between level 2 and level 3. Record reasons for each rating. Remind participants not to submit confidential or personal data. Use anonymised material.

### Knowledge check **Question 1.** A programme lead lists the claim, evidence, threats, likelihood, impact, existing controls and gaps for an assessment. Which activity is this? [( )] Writing a declaration form. [(X)] Building a risk register as part of a threat model. [( )] Running a detection scan. [( )] Moderating marks. [[?]] It is the core tool of Module 1. **** A risk register is the structured output of a threat model. It links the claim to threats and gaps so that redesign targets the real weaknesses. **** **Question 2.** Which assessment is likely to have the highest exposure to generative AI agents without extra safeguards? [( )] A live practical observed by an assessor. [( )] A ten-minute oral discussion. [(X)] An unsupervised online report on a public dataset with a generic prompt. [( )] A supervised written exam with no devices. [[?]] Look for generic task, text output and no supervision. **** Generic tasks with text output, public data and no supervision are easiest to complete with AI and agents. Live, oral and supervised elements have lower exposure, though not zero. **** **Question 3.** An academy replaces a take-home report with a plan submitted early, a decision log and a short live conversation. Which principle does this apply? [( )] Ban all tools. [(X)] Collect process evidence and add a live element in proportion to stakes. [( )] Rely on a detection score. [( )] Remove the assessment claim. [[?]] Think about what evidence is added. **** The plan and log give process evidence and the conversation gives a live check. Together they strengthen evidence for the claim without banning tools. **** **Question 4.** What is a sound approach to identity checks for a high-stakes credential? [( )] Check identity only at registration, never again. [( )] Use the most intrusive method available in every case. [(X)] Use proportionate checks at critical points, collect only needed data and offer fair alternatives. [( )] Rely on how an earner's writing sounds. [[?]] Proportion, privacy and fairness. **** Proportionate checks at critical points balance integrity with privacy and fairness. Alternatives prevent unfair disadvantage, and impressions of writing style are not reliable identity evidence. **** **Question 5.** In a remotely supervised test, an automated system flags an earner. What should happen next? [( )] The result is cancelled at once. [( )] The flag is deleted. [(X)] A person reviews the flag, gathers further evidence and gives the earner a chance to respond. [( )] The cohort is informed. [[?]] A flag prompts review, not a decision. **** Automated flags can be wrong. A fair process uses them to begin a human review, gathers other evidence and gives the earner a right to respond. **** **Question 6.** Which policy wording best states permitted AI use? [( )] "AI should be used wisely." [(X)] "For the written plan, AI may be used for drafting and editing; each use must be declared. For the live discussion, no AI is permitted." [( )] "AI is the staff's decision." [( )] "Earners are responsible for knowing the rules." [[?]] Look for element-by-element detail. **** Clear policy names the element, the permitted purposes and the declaration duty. Vague wording leaves earners and assessors without a common standard. **** **Question 7.** Which statement about AI detection tools is accurate? [( )] A high score proves that an earner did not write the work. [( )] Detection tools identify who wrote a text. [(X)] Scores are estimates that can be wrong, so they are not a reliable basis for decisions. [( )] Detection tools are accurate for every group of writers. [[?]] Established position: not a reliable basis. **** Detection outputs are estimates, with errors in both directions. The established position is that they are not a reliable basis for decisions, so other evidence reviewed by a person is needed. **** **Question 8.** An earner says a detection tool elsewhere flagged her essay. What is the best response under a sound policy? [( )] Fail the essay. [( )] Dismiss the earner's concern. [(X)] Treat the result as a prompt for a conversation and review the work, process evidence and her explanation. [( )] Require her to pay for a second tool. [[?]] A score is one input to a conversation. **** A score can start a conversation, but a decision needs other evidence. Reviewing drafts, notes and the earner's explanation is fair and more reliable. **** **Question 9.** Which communication practice best supports honest declaration of AI use? [( )] Penalising all declared use. [( )] Asking earners to guess the rules. [(X)] Stating that permitted, correctly declared use carries no penalty, and giving examples. [( )] Publishing earners' declarations. [[?]] Earners need to feel safe. **** Clear examples and a promise of no penalty for permitted, declared use encourage honest declaration, which supports fair review. **** **Question 10.** The AI Omnibus timelines in this course come from a provisional agreement. What is the right way to use them in a policy briefing? [( )] Present them as final and complete legal advice. [( )] Leave out any mention of regulation. [(X)] State them with the status note, flag that the final published text must be checked and recommend qualified advice. [( )] Assume they apply from today. [[?]] Status: November 2026. **** The dates come from a provisional agreement of 7 May 2026, so the final text must be checked. A careful briefing states the status, flags the check and recommends qualified advice. **** ## Sources and further reading Status: November 2026. The sources below are official texts, framework owners and standards bodies. Titles are given without links so that you can find the current versions through the publishers' own sites. - **ICoBC Credentialing Skills Framework (ICSF)**, International Council on Badges and Credentials: skill ASMT "Assessment and evidence", level L4 "Assure", stage A "Ability". Use the current version published by ICoBC. - **Regulation (EU) 2024/1689 (the EU AI Act)**: the regulation laying down harmonised rules on artificial intelligence. Published in the Official Journal of the European Union. Read the consolidated text for Annex III (high-risk systems, including the education and vocational training area) and Annex I.- **The Digital Omnibus on AI and the AI Omnibus**: proposals and the provisional agreement of 7 May 2026 as summarised in this course (Annex III high-risk obligations from 2 December 2027; Annex I product-embedded systems from 2 August 2028). Check the final published text in the Official Journal.- **Regulation (EU) 2016/679 (General Data Protection Regulation)**: relevant to identity checks, remote supervision and retention of integrity records. Published in the Official Journal of the European Union. - **UNESCO, guidance on generative AI in education and research**: UNESCO publishes guidance for policy makers on generative AI in education. Find the current edition on the UNESCO website.- **Tertiary Education Quality and Standards Agency (TEQSA, Australia)**: publishes guidance and resources on assessment reform in the age of generative AI. Find the current resources on the TEQSA website.- **Quality Assurance Agency for Higher Education (QAA, UK)**: publishes guidance on academic integrity and on generative AI in assessment. Find the current guidance on the QAA website.- **ICB-P02 Assessment Design for Credentials: Valid, Reliable, Fair**: the prerequisite course in the ICoBC course library. This course does not give legal advice. Seek qualified advice on regulatory and data protection questions in your own situation.