Assessment Design for Credentials: Valid, Reliable, Fair
Assessment that produces evidence for the claim, and holds up when someone checks.
Course ID: ICB-P02. Framework alignment: ICoBC Credentialing Skills Framework (ICSF), skill ASMT "Assessment and evidence" (VRITE pillar: Rigor), level L3 "Enable", stage A "Ability".
Who this course is for
- L&D and corporate academy leads
- Programme and micro-credential leads
- Learning designers and teaching staff
- Professional bodies and sector associations
If you design, approve or review assessments that sit behind a credential, this course is for you. It assumes you have completed ICB-F01 Digital Badges and Credentials Essentials.
What you will be able to do
At level L3 "Enable", you design assessments and rubrics for credentials, plan moderation, and design recognition of prior learning routes for a programme. This course is at the "Ability" stage: you practise on realistic tasks in a controlled setting, and you produce a real assessment package for one credential.
Learning outcomes
- Choose assessment methods that produce evidence for each criterion.
- Build analytic rubrics with clear performance descriptors.
- Plan moderation and second marking in proportion to the stakes.
- Check an assessment for bias, accessibility and integrity risks.
Workload and format
The course is a blended cohort course with 12 hours of notional learning. This file covers the self-paced part. Live or workshop elements appear as facilitator notes inside each module.
How it is assessed
You complete a short knowledge check in this file. You also submit an assessment plan, a rubric and a moderation record for one credential. A trained assessor marks your submission with the rubric given in the section "Assessment and badge". The file does not grade your submission.
Badge
On passing you can be awarded the badge "Assessment Design for Credentials: Valid, Reliable, Fair (Ability, L3)", alignment code ICSF-ASMT-L3-A. The badge records that you demonstrated this skill on the assessed task. It does not prove competence in any particular job.
| Outcome | Module | How it is assessed |
|---|---|---|
| 1. Choose assessment methods that produce evidence for each criterion. | What assessment must prove; Matching methods to claims | Knowledge check; assessment plan |
| 2. Build analytic rubrics with clear performance descriptors. | Rubrics that work | Knowledge check; rubric |
| 3. Plan moderation and second marking in proportion to the stakes. | Reliability and moderation | Knowledge check; moderation record |
| 4. Check an assessment for bias, accessibility and integrity risks. | Fairness and accessibility; Integrity by design | Knowledge check; assessment plan |
How to work through the modules
Each module has short units, one graphic, one worked example, one practice activity and a quiz. Keep one credential in mind from the start. The practice activities build the parts of your final package, so the work you do along the way is not wasted.
Pick a credential that is real or realistic, for example a micro-credential for a college short course, or an internal badge for a corporate academy. Choose one with a clear claim, because everything in this course starts from the claim.
Facilitator notes
Run a 45-minute opening session. Ask each participant to name the credential they will use for the whole course and to state its claim in one sentence. Pair participants who work in different settings (for example one from a training provider and one from a corporate academy) so that later moderation exercises bring fresh eyes. Collect the credential choices in a shared list so you can spot weak choices early, for example credentials with no clear claim.What assessment must prove
The claim comes first
Every credential makes a claim. The claim says what the earner can do, know or show, and under what conditions. An assessment exists to produce evidence for that claim. If the evidence cannot support the claim, the credential is weak, however good the learning was.
Start with three questions:
- What exactly does the credential claim?
- What would convince a sceptical third party that the claim is true?
- Who will rely on the credential, and what decisions will they make with it?
The third question sets the stakes. A badge that shows attendance at an internal workshop carries low stakes. A credential that opens a route to a regulated role, a promotion or a funding decision carries higher stakes. The higher the stakes, the stronger the evidence must be.
A claim is a statement you can check. "Understands data protection" is vague. "Applies the organisation's data protection procedure to classify and handle five realistic data requests" can be checked.
Validity, reliability and fairness
Three ideas run through this whole course.
Validity asks whether the assessment measures what the claim says. A written test about giving feedback may be valid for knowledge of feedback models. It is a poor source of evidence that someone gives feedback well in a live conversation.
Reliability asks whether the result would be the same if a different assessor marked it, or if the earner did a comparable task on another day. A reliable assessment gives consistent decisions.
Fairness asks whether every earner has an equal chance to show what they can do. Barriers that have nothing to do with the claim, such as unclear language, inaccessible formats or hidden assumptions, reduce fairness.
You need all three. A reliable but invalid assessment produces consistent results about the wrong thing. A valid but unreliable assessment produces results that depend on who marks. An assessment that is not fair excludes earners for reasons unrelated to the claim.
Constructive alignment: the triangle
The alignment triangle links three corners: the outcome (what the earner should be able to do), the assessment task (what they do to show it) and the criteria (how you judge the evidence). If one corner moves, the others must move with it.
Criteria and evidence
A criterion is one thing you will judge, for example "selects an appropriate analysis method" or "communicates the finding to a non-specialist". Each criterion needs evidence. Evidence is something the earner produces or does that you can observe, record and judge.
A useful test is to read each criterion and ask: "What would I point to?" If you cannot name a piece of evidence, the criterion is a wish. Rewrite it or drop it.
Worked example: From vague outcome to checkable claim
University setting. A programme lead at a university runs a short micro-credential called "Introduction to Research Data Management". The draft outcome reads: "Learners understand research data management."
The lead rewrites it step by step.
- Claim: The earner can prepare a basic data management plan for a small research project.
- Sceptical third party: A research supervisor who will rely on the badge when allocating a student to a project.
- Stakes: Medium. The badge supports a decision but does not replace the supervisor's judgement.
- Criteria (illustrative): identifies the data the project will create; chooses a storage and backup approach with reasons; states who may access the data and why; describes how data will be documented.
- Evidence: a completed data management plan for a supplied project brief.
Corporate academy setting. An HR department runs an internal badge, "Handling Employee Data Requests". The draft outcome reads: "Staff are aware of the procedure."
The HR lead rewrites it.
- Claim: The earner can handle a routine employee data request using the company procedure.
- Sceptical third party: A line manager who will let the earner process requests without supervision.
- Stakes: Medium to high, because errors affect colleagues.
- Criteria (illustrative): identifies the request type; checks the requester's identity; applies the correct deadline; records the decision.
- Evidence: observed or recorded handling of three simulated requests.
In both cases the vague word "understand" or "aware" disappeared. The lead replaced it with a task and criteria that can be checked.
Practice activity: Write and test your claim
Choose your credential for the course. Complete these steps in your working document.
- Write the claim in one sentence, starting with "The earner can".
- Name the sceptical third party who will rely on it.
- Rate the stakes as low, medium or high, and write one sentence to justify the rating.
- List three to five criteria.
- For each criterion, name the evidence you would point to.
- Draw your own alignment triangle on paper and check that all three corners match.
Keep this document. It becomes the first page of your assessment plan.
Facilitator notes
Run a 60-minute workshop. In pairs, participants swap their claims and play the sceptical third party: they ask "How would I know?" until each criterion has a named piece of evidence. Look out for criteria written with verbs such as "understand", "be aware of" or "appreciate". Ask pairs to rewrite them as observable actions. Close with two or three volunteers reading out a before and after version.Check your understanding
The claim is the starting point. An assessment exists to produce evidence for it. Ranking, attendance and opinions may be useful for other purposes but they do not show that the claim is true.
Reliability is about consistency of decisions. Validity asks whether the assessment measures what the claim says, and fairness asks whether every earner has an equal chance to show what they can do.
"Appreciate" describes an inner state. A criterion needs evidence you can see, record and judge, such as "identifies three hazards in a supplied workplace scenario and states a control for each".
Stakes describe what depends on the credential. A credential that supports a hiring or regulated decision needs stronger and better checked evidence than a low-stakes participation badge.
Matching methods to claims
Methods produce different kinds of evidence
No single method suits every claim. Each method produces a particular kind of evidence, and each has strengths and weaknesses. Your job is to match the method to the claim and to the criteria.
Common methods include:
- Written or online tests with selected or constructed answers: good for knowledge and some reasoning, efficient to mark, weak for practical skill.
- Case analysis or scenario tasks: good for applying knowledge to realistic situations.
- Practical demonstration or observation: good for showing a skill in action, more costly to run and to mark consistently.
- Simulation or sandbox task: good for tasks that are risky, rare or expensive to observe in real life.
- Project or portfolio: good for complex, extended work, but needs clear authorship checks and a clear brief.
- Oral questioning or viva: good for probing understanding and checking authorship, but needs structure to stay fair.
- Workplace evidence with a supervisor statement: good for claims about real work, but depends on the quality of the supervisor's judgement.
Match method to the verb in the claim
Look at the verb in each criterion. It points to the method.
| Verb in the criterion | Evidence you need | Method options |
|---|---|---|
| Recall, define, identify | A correct statement or selection | Short test, oral question |
| Explain, justify | Reasoned written or spoken argument | Short written response, viva |
| Apply, calculate, select | A worked answer to a new case | Case task, scenario test |
| Perform, operate, demonstrate | A live or recorded action | Observation, simulation |
| Design, produce, create | An artefact | Project, portfolio |
| Evaluate, advise | A judgement with reasons | Case task, report, viva |
Sampling and coverage
You cannot test everything. You sample. A good sample covers the important criteria, in realistic conditions, often enough to make the result dependable. A single task can be a lucky or unlucky day. For higher-stakes credentials, collect evidence from more than one task or more than one occasion.
Write down which criteria each task covers. A simple coverage grid shows gaps and duplicates. If one criterion is checked in three tasks and another in none, rebalance.
Designing tasks that stay valid with AI tools
Learners now have access to generative AI tools. This affects the choice of method. Ask of each task: "If the learner uses an AI tool, does the evidence still support the claim?"
There are two sound responses, and you can use both in one credential.
Design the task so the claim is still shown. Use tasks that depend on context the tool cannot easily supply: a specific local case, the learner's own workplace data, a live conversation, an observed performance, or a short oral follow-up on a written submission. Ask for the reasoning behind choices, and for reflection on specific decisions made during the task.
Include AI use openly as part of the task. If the real job involves AI tools, assess the use of them. The task can ask the learner to use a tool, to check its output, to correct errors and to explain what they changed and why. The criteria then judge the quality of the checking and the judgement, not only the final text.
Whichever response you choose, tell learners clearly what is allowed. Say which tools, for which steps, and how to declare use. Unclear rules are unfair to learners and hard to apply.
Worked example: Choosing methods for a four-criterion claim
Training provider setting. A college runs a micro-credential in "Customer Complaint Handling for Retail Staff". The claim reads: "The earner can handle a routine customer complaint in line with the provider's service standard."
| Criterion | Method chosen | Why |
|---|---|---|
| Identifies the complaint type and the relevant policy | Short scenario-based test | Efficient check of knowledge applied to cases |
| Responds calmly and clearly in a live conversation | Observed role-play with a trained actor | A verbal performance can only be judged in action |
| Records the complaint accurately in the case form | Completed form from the role-play | Produces an artefact that can be checked |
| Reflects on what went well and what to change | Short oral question after the role-play | Hard to outsource, and probes understanding |
The designer decides that the written reflection would be risky, because a learner could generate it with a tool without having done the role-play well. The oral follow-up, tied to specific moments in the role-play, avoids that problem.
Corporate academy setting. An HR team runs a badge for "Writing a Clear Job Advert". The claim says the earner can draft an advert that follows the organisation's inclusive language guide. The team decides to allow an AI drafting tool openly. The task asks the earner to produce a first draft with the tool, then to mark up at least five changes needed to meet the guide, and to explain each. The criteria judge the quality of the changes and the explanations. The assessment now matches a realistic workplace use.
Practice activity: Choose methods for your criteria
Return to the criteria you wrote in the previous module.
- Create a table with the columns Criterion, Verb, Evidence, Method, AI risk and Response.
- Fill one row per criterion.
- For each row, write one sentence on what a learner could do with an AI tool, and decide whether you will redesign the task or allow and assess the tool openly.
- Check coverage: does each criterion appear in at least one task, and is the effort of assessing in proportion to the stakes?
- Write a short statement for learners (three or four sentences) on what AI use is allowed and how to declare it.
Facilitator notes
Run a 90-minute workshop. Give each pair a different sample claim (for example "operate a piece of equipment safely", "write a short project proposal", "advise a colleague on a policy question"). Ask them to propose two methods and argue for one. Then ask what changes if the learner has access to an AI tool and no one is watching. Encourage the group to compare the cost of the method with the stakes of the credential. Capture the best AI-use statements on a flipchart for reuse.Check your understanding
The verb is "demonstrates", so the evidence must be an action. Tests and essays show knowledge about the task, and a self-rating shows opinion. Observation or simulation shows the performance itself.
A coverage grid shows gaps and duplicates. It helps you spend assessment effort where it is needed and makes sure the sample supports the claim.
Detection tools are not a reliable basis for decisions. A better response is to design the task so that the claim is still shown, for example through observed work and oral follow-up, or to bring the tool use into the task and judge it.
If the claim reflects real work that uses AI tools, the assessment should reflect that too. The criteria then judge how well the learner directs, checks and corrects the tool, which is evidence of the skill that matters.
Rubrics that work
What an analytic rubric is
An analytic rubric breaks performance into separate criteria and describes several levels of quality for each. It lets different assessors judge the same evidence in the same way, and it tells earners what good work looks like before they start.
Compare it with a holistic scale, which gives one overall grade. Holistic scales are quick, but they hide which parts of a performance were weak. For credentials, where decisions need to be explained and checked, an analytic rubric is usually the better tool.
An analytic rubric has four parts:
- Criteria in the rows: what you judge.
- Levels in the columns: how well it was done.
- Descriptors in the cells: words that say what each level looks like.
- Weighting or decision rule: how the cell judgements combine into a result.
Anatomy of a rubric
Writing good descriptors
A descriptor is useful when two assessors can read it, look at the same evidence and reach the same judgement. Follow these rules.
Describe what you can observe. "Records all three required fields in the case form" is observable. "Shows good attention to detail" is not.
Use the same scale of difference across levels. The levels should differ in a clear way: number of errors, completeness, accuracy, independence or quality of reasoning. Do not change several things at once from one level to the next.
Avoid vague quantifiers. Words such as "some", "many", "good" and "adequate" mean different things to different assessors. Use counts or clear thresholds where you can, for example "no more than one minor error".
Avoid negative-only language at the lowest level. A level such as "fails to do X" says what is missing but not what is present. Describe what the evidence does show, so earners can improve.
Keep each cell short. Aim for one or two sentences. Long cells are skipped.
Do not mix criteria. If a cell judges two separate things, assessors cannot decide when one is strong and the other weak. Split the criterion.
Number of levels and the pass rule
Three or four levels are usually enough. More levels invite false precision and lower agreement between assessors. For a credential, the most important line is the one that divides "not yet" from "meets". Write the pass rule before you mark anything, for example "meets on every criterion" or "meets on at least four of five, and no 'not yet' on criteria marked as essential".
Essential criteria are those where failing makes the claim untrue. For a safety-related claim, "applies the isolation procedure correctly" is likely essential. Do not let a high score on other criteria hide a fail on an essential one.
Worked example: Rewriting a weak descriptor
Corporate academy setting. A corporate academy runs a badge in "Running an Effective Team Meeting". The draft rubric row reads:
| Criterion | Not yet | Meets | Exceeds |
|---|---|---|---|
| Meeting management | Poor | Good | Excellent |
The words do not tell the assessor, or the earner, what to look for. The designer rewrites the criterion and splits it.
| Criterion | Not yet | Meets | Exceeds |
|---|---|---|---|
| Sets the purpose | Meeting starts without a stated purpose or agenda. | States the purpose and shares an agenda before or at the start. | States the purpose, shares an agenda and agrees the expected outcome with attendees. |
| Keeps to time | Meeting overruns by more than ten minutes without agreement from attendees. | Meeting ends within ten minutes of the planned time, or attendees agree an extension. | Meeting ends on time and the leader uses time checks to move the discussion on. |
| Records actions | No actions are recorded. | Actions are recorded with an owner for each. | Actions are recorded with an owner and a date, and sent to attendees within one working day. |
The ten-minute threshold is illustrative and should be set by the organisation. Each cell now describes something an observer can see.
University setting. A programme lead writes a rubric for a short report on a data management plan. The row "Quality of writing" mixed grammar, structure and tone. The lead splits it into "Structure follows the supplied template" and "Language is clear for a non-specialist reader", with separate descriptors for each, so that a report with a strong structure and weaker language is judged fairly on both.
Practice activity: Build your rubric
Using your criteria from the earlier modules:
- Build a table with one row per criterion and three or four level columns.
- Write observable descriptors for the "Meets" level first. These define the standard.
- Write the "Not yet" and any higher levels by changing one thing at a time.
- Mark any criteria as essential.
- Write the pass rule in one or two sentences.
- Swap your rubric with a peer. Each of you underlines every word that could mean different things to different assessors, and suggests a replacement.
- Revise your rubric using the feedback.
Facilitator notes
Run a 90-minute workshop. Provide two or three short sample pieces of work for a sample task (for example a short complaint response email, an anonymised form, a short video). Ask participants to mark them individually using a draft rubric, then compare. Disagreements show where descriptors are vague. Revise one rubric row as a whole group. Encourage use of counts and thresholds, and remind the group that thresholds in examples are illustrative and must be set for the context.Check your understanding
An analytic rubric judges criteria separately, so feedback is specific and decisions can be explained. It supports consistency but does not guarantee it, which is why moderation is still needed.
The third descriptor names what can be observed and sets a clear threshold. The others use vague words such as "good", "most" and "adequate", or describe effort rather than evidence.
Essential criteria protect the claim. If an earner does not meet one, high scores elsewhere cannot make up for it, so the pass rule must say so.
The cell judges two things. If an earner writes clearly but uses the wrong structure, an assessor cannot judge fairly. Separate criteria give separate judgements.
Reliability and moderation
What reliability means in practice
Reliability means consistent decisions. Two competent assessors should reach the same decision on the same evidence, and the same assessor should reach the same decision on different days. Perfect agreement is rare. The aim is to make disagreement small, visible and corrected.
Common causes of weak reliability:
- vague descriptors, as seen in the previous module
- assessors who have not seen examples of work at each level
- tiredness or time pressure during marking
- knowing the earner, which can sway judgement either way
- different interpretations of the task brief
- inconsistent support or instructions given to earners
Reliability is not only about marking. It is also about the task, the conditions, the instructions and the evidence you collect.
Moderation: the toolkit
Moderation is the set of activities that check and improve consistency of judgements. Some common tools:
- Assessor briefing and standardisation: assessors review the task, the rubric and sample work at each level before marking.
- Benchmark samples: agreed examples of work at the "Meets" boundary, used as reference points.
- Second marking: a second assessor marks some or all work independently.
- Blind marking: the assessor does not see the earner's name or other identifying information, where this is practical.
- Sampling and review: a moderator reviews a sample of decisions, with extra attention to borderline and fail decisions.
- Calibration meetings: assessors discuss disagreements, agree how to read descriptors and note changes needed to the rubric.
- Appeals and review: a clear route for earners to ask for a decision to be reviewed.
Proportion to the stakes
Moderation takes time and costs money. Match the effort to the stakes of the credential.
| Stakes | Example | Moderation approach (illustrative) |
|---|---|---|
| Low | Internal participation badge, no external use | Clear rubric, assessor briefing, light spot-check |
| Medium | Credential used in internal selection or as a prerequisite | Briefing, benchmark samples, sample second marking, review of all borderline results |
| High | Credential used for regulated work, licensing or funding decisions | Briefing, benchmarks, second marking of all fails and borderline work and a larger sample of passes, a named moderator, a documented appeals process |
The percentages and sample sizes are for each organisation to set and justify.
Whatever you decide, write it down. A moderation plan states who moderates, what is sampled, when, how disagreements are resolved, and how changes are recorded.
Handling disagreement
When two assessors disagree, do not average. Averaging hides the problem. Instead:
- Each assessor states the evidence they relied on and the descriptor they applied.
- They return to the rubric wording and the benchmark samples.
- If they still disagree, a third person, usually the moderator, decides.
- If the disagreement points to unclear wording, record the issue and revise the rubric after the cycle.
Keep a short moderation record. It should list the work sampled, the original decisions, the second decisions, the final decision and any rubric change. This record is evidence that the credential is checked.
Worked example: Proportionate moderation for two credentials
Training provider setting. A training provider runs a medium-stakes micro-credential, "Safe Use of Workshop Equipment", which employers use to allow learners onto a workplace placement. Four assessors observe a practical task.
The programme lead sets this moderation plan (illustrative):
- All assessors attend a 60-minute briefing and mark three sample videos before live marking.
- Every fail and every borderline result receives a second marking by a different assessor.
- A moderator reviews a sample of passes from each assessor.
- Disagreements are discussed and recorded in the moderation record. Rubric wording changes are listed for the next cycle.
- Learners can request a review within an agreed period.
Corporate academy setting. A corporate academy runs a low-stakes badge for completing an internal onboarding project. One assessor per site marks the project against a rubric. The plan is lighter: a short briefing, a shared benchmark example, and a quarterly check by the academy lead of five randomly chosen projects across sites. The lead records any wording problems and updates the rubric once a quarter.
Both plans are proportionate. The training provider spends more effort because employers rely on the badge for safety-related decisions.
Practice activity: Plan and rehearse moderation
- Using your credential's stakes rating, write a one-page moderation plan with the headings: who, what is sampled, when, how disagreements are resolved, how changes are recorded.
- Create or choose a short piece of sample evidence (a one-page work sample or a short recording). Anonymise it.
- With one or two peers, mark the same sample independently using your rubrics.
- Compare decisions criterion by criterion. Note each disagreement and the wording that caused it.
- Start your moderation record: list the sample, the decisions, the final decision and one rubric revision for each disagreement.
You will extend this record in the final assessment.
Facilitator notes
Run a 90-minute live moderation workshop. Prepare two or three anonymised samples of work at different quality levels. Ask groups of three to mark independently, then reveal and compare marks on a flipchart. Focus the discussion on the descriptors, not on the people. Stop each group from averaging and push them to quote evidence. Keep a short log of rubric changes that come out of the session. Remind participants that the moderation record is part of their badge evidence.Check your understanding
Averaging hides the disagreement and hides unclear wording. Discussing evidence against the rubric resolves the issue and shows whether the rubric needs revision.
The effort of moderation should match the stakes. Where decisions affect safety or regulated work, more checks and clearer documentation are needed, particularly for fails and borderline results.
Benchmarks help assessors share an understanding of the standard. Work at the "Meets" boundary is especially useful, because most disagreements occur there.
A moderation record shows that checks took place, what they found and what changed. This supports trust in the credential and helps the next cycle.
Fairness and accessibility
Fairness starts at design
A fair assessment gives every earner an equal chance to show what they can do. Barriers that are not part of the claim should be removed. This is a design task, not a repair job after complaints.
Ask of each task: "Is every demand in this task part of the claim?" If a task about financial reasoning demands fluent idiomatic English, or a task about equipment safety requires a particular cultural reference, the task tests more than the claim. That extra demand is a source of bias.
Common sources of bias
Bias in assessment can arise from:
- Language: idioms, long sentences or unnecessary jargon in tasks and instructions.
- Context: scenarios that assume experiences not all earners have, for example a particular sport, region, family structure or working pattern.
- Format: timed written tests that measure speed when speed is not part of the claim.
- Assessor effects: first impressions, handwriting, accent, name, or the "halo" effect of a strong first criterion.
- Technology: tasks that assume a fast connection, a particular device, or familiarity with a platform.
- Hidden criteria: things assessors reward but the rubric does not mention, such as a polished style.
You will not remove all bias. You can reduce it, check for it and respond to it.
Accessibility and reasonable adjustments
Accessibility means that earners with disabilities, health conditions or other circumstances can take the assessment without unnecessary barriers. Design for access from the start, so that fewer special arrangements are needed later.
Principles to apply:
- Offer information in more than one format where you can, for example text and audio.
- Use plain language in tasks and instructions.
- Allow flexible timing where time is not part of the claim.
- Make digital materials work with assistive technology, such as screen readers.
- Provide clear structure, headings and consistent layout.
- Offer alternative ways to show the same evidence where the claim allows, for example an oral response instead of a written one.
Some adjustments will still be needed for individuals. Have a clear, respectful process for earners to request them, and a clear rule on what cannot change. The test is the claim: an adjustment may change how an earner shows the evidence, but it must not remove a part of the claim. For example, an earner can answer a case question by speaking rather than writing, if writing is not part of the claim. The same earner cannot skip the observed performance if the claim is about performing the task.
Recognition of prior learning
Recognition of prior learning (RPL) is a route by which people can gain a credential by showing evidence of learning and experience they already have, rather than by repeating a course. RPL is a fairness issue: people with relevant experience should not be forced to repeat what they can already do.
A sound RPL route:
- Uses the same claim and criteria as the standard route. The standard does not change.
- Accepts different kinds of evidence, for example work samples, supervisor statements, previous credentials, or a short observed task.
- Applies the same rubric and the same pass rule.
- Includes moderation, in proportion to the stakes.
- Is described clearly, with the evidence required and the steps and timescale.
The route may differ, but the standard is the same. If RPL is easier than the standard route, the credential loses value. If it is much harder, it is unfair.
Worked example: Removing a hidden barrier
University setting. A programme lead reviews a case task for a micro-credential in project planning. The case describes a charity fundraiser involving a "village fete" and uses phrases such as "pull out all the stops". The lead asks of each demand: "Is this part of the claim?" The claim is about planning tasks and risks, not about knowing local customs or idioms. The lead rewrites the case around a neutral event, "a one-day community event with 200 attendees" (illustrative), removes the idioms and shortens the sentences. The lead also adds an audio version of the brief and states that the task can be answered in writing or as a recorded spoken response, since written style is not a criterion.
Corporate academy setting. An HR team reviews a badge assessment for "Giving Constructive Feedback". The role-play is timed and recorded on a webcam. The team notes two barriers: an unreliable connection for staff at remote sites, and anxiety about recorded video for some staff. The claim is about the quality of feedback conversations, not about being recorded on a webcam. The team offers a live observed role-play with an assessor as an alternative to the recording, and states that both routes use the same rubric and the same pass rule.
Practice activity: Review your assessment for bias and access
- List every demand your task places on the earner: language level, time, technology, format, background knowledge, physical or sensory requirements.
- Mark each demand as "part of the claim" or "not part of the claim".
- For each demand that is not part of the claim, write a design change that removes or reduces it.
- Write a short statement describing how earners can request adjustments, and one sentence stating what cannot change.
- Write a short outline of an RPL route: the evidence accepted, who assesses it, and how the same standard is kept.
- Add this review to your assessment plan.
Facilitator notes
Run a 75-minute session. Ask participants to swap tasks and conduct a "barrier walk": read the task as an earner with a specific constraint (for example reading in a second language, using a screen reader, working on a phone with a slow connection, working night shifts with limited time). Collect barriers on a flipchart and group them. Discuss how to distinguish a legitimate demand from an unnecessary one, using the claim as the test. Remind participants that local legislation and standards vary; they should check their own context.Check your understanding
A demand that is not part of the claim, such as unnecessary idioms or speed, adds a barrier unrelated to what is being certified. The claim is the test.
An adjustment may change how evidence is shown, but not the standard. The same rubric and pass rule keep the credential consistent and fair.
RPL changes the route to the evidence, not the standard. If the standard shifts, the credential means different things for different earners.
This is sometimes called the halo effect. Analytic rubrics, blind marking where possible, and moderation reduce its influence.
Integrity by design
Integrity is a design question
Assessment integrity means that the evidence is authentic: it shows what the earner can do, and the earner produced it. Integrity risks include copying, impersonation, collusion, unauthorised help, and unauthorised or undeclared use of tools, including generative AI.
You can respond in two ways: by policing or by design. Policing relies on rules, surveillance and sanctions. Design makes it harder or less useful to cheat, and easier to be honest. Policing has a place, but design is stronger and costs less over time.
Detection tools are not a basis for decisions
Generative AI and assessment integrity remains an ongoing issue for all issuers. Tools that claim to detect AI-generated text are not a reliable basis for decisions about individual earners. They can produce false positives and false negatives, and a decision that rests on them alone is hard to defend and may be unfair. Do not use a detection score as the sole evidence of misconduct.
Status as of November 2026; check before publishing.
Instead, rely on design and on human judgement supported by evidence, such as drafts, oral discussion of the work, comparison with the earner's other work, and observed performance. If you suspect misconduct, follow a fair process: state the concern, give the earner a chance to respond, consider the evidence as a whole, and record the decision.
Design features that support integrity
- Authentic, specific tasks: use local cases, the earner's own context, or recent events, so that a generic answer does not fit.
- Staged work: ask for a plan, a draft and a final version, with brief feedback in between. Staging shows how the work developed.
- Observed or live elements: include a short live component, such as a demonstration or an oral discussion of the submitted work.
- Process evidence: ask for notes, decision logs or version history where this is proportionate and accessible.
- Choice of case within a frame: give different earners different cases or data, so that copying is of little use.
- Clear rules: state what help, sources and tools are allowed, and how to declare them.
- Identity checks in proportion: for higher-stakes credentials, confirm identity at the point of assessment.
- Open AI use where relevant: if AI tools are part of real work, include them in the task and assess the quality of the checking and judgement.
Each feature adds some effort for earners and assessors. Choose features in proportion to the stakes, and make sure they do not create new barriers for people with access needs. A live oral element, for example, needs an alternative for earners who cannot take part in that form.
Putting the risk review together
A short risk review for any assessment can use a simple table: the risk, how likely and serious it is for this credential, the design response, and the residual risk you accept. Write it for your assessment plan. A reviewer should be able to see that you looked for risks in bias, accessibility and integrity together, and chose responses that fit the stakes.
Worked example: An integrity review with AI in mind
University setting. A university runs a micro-credential in "Writing a Policy Brief". The first design asks for a 1,500-word brief on a topic chosen by the earner, submitted online. The programme lead identifies an integrity risk: a generic brief could be produced by a tool with little input from the earner.
The lead redesigns the task:
- Each earner receives one of several short local case descriptions with different data tables (illustrative).
- The earner submits a one-page outline, then a draft, then the final brief, with brief feedback after the outline.
- The earner declares any AI tool use, with a short note on what it was used for and what they changed.
- A ten-minute online discussion of the brief is held with an assessor, who asks the earner to explain two choices in the brief. An audio-only option is offered, and a written Q and A option is available on request, with the same questions.
- The rubric includes a criterion for "justifies the main recommendation using the case data", which a generic brief cannot meet.
The lead does not use an AI detection score in any decision.
Corporate academy setting. An HR team runs a badge for "Preparing a Learning Needs Analysis", which the team knows employees may draft with an AI tool. In real work they will do so too. The team decides to include the tool openly. The task asks each earner to produce a first draft with a tool of their choice, using their own team's anonymised data, then to submit an annotated version showing at least five corrections or improvements and the reasons. The criteria judge the accuracy of the corrections, the use of the team's data and the quality of the reasoning. The assessor asks one follow-up question by phone. The team states in advance what is allowed and how to declare use.
Practice activity: Complete the risk review
- Create a table with the columns Risk, Type (bias, access or integrity), Likelihood and seriousness (low, medium, high), Design response, Residual risk.
- List at least two risks for each type for your assessment. Include at least one risk related to AI tools.
- For each risk, choose a design response. Check that the response does not create a new barrier for earners with access needs.
- Write the rule for earners on allowed help and tools, and how to declare use.
- Write a short fair process for handling a suspected integrity concern that does not rely on a detection score alone.
- Add the table and the rules to your assessment plan.
Facilitator notes
Run a 90-minute session. Ask pairs to "attack" each other's assessments: one plays an earner trying to produce acceptable evidence with minimal real effort, using any realistic means including AI tools, and the other records what would work. Then discuss design responses. Keep the discussion on design and fairness rather than on catching people. Remind participants that detection tools are not a reliable basis for decisions about individuals, and that the integrity process must be fair and proportionate. Encourage them to check their organisation's own policy on AI use.Check your understanding
Detection tools can give false positives and false negatives. A decision should rest on a wider body of evidence and a fair process in which the earner can respond.
Specific, varied cases and a short discussion require the earner to engage with the actual material and to explain their choices, which a generic text does not show.
If the real work uses the tool, valid evidence shows how well the earner uses it. Judging the checking and reasoning gives evidence of the skill that matters.
Structure keeps the element reliable, and an alternative keeps it accessible. A record of questions and responses supports moderation and review.
Assessment and badge
The task
You will design an assessment package for one credential and test it with peers. The assessed task is marked by a trained assessor using the rubric below. This file does not grade your work.
Task: Design, moderate and revise an assessment for a credential
Choose one credential, real or realistic. It can be a micro-credential from a university, college or training provider, or an internal badge from a corporate academy or HR department. The credential must have a claim you can state in one sentence.
Produce the following three pieces of evidence.
1. Assessment plan (about 4 to 6 pages)
- The claim, the sceptical third party and the stakes rating, with a short justification.
- The criteria and the evidence for each, with the alignment checked.
- The methods chosen, a coverage grid and the reasons for each choice.
- How AI tools are handled: either how the task stays valid when learners have tools, or how AI use is included openly, and the rule given to earners.
- A bias, accessibility and integrity risk review, with design responses.
- An outline of how adjustments are requested and an outline of an RPL route.
- The moderation plan, proportionate to the stakes.
2. Analytic rubric
- At least four criteria, three or four levels, observable descriptors and a stated pass rule.
- Essential criteria marked, where they apply.
3. Moderation record
- The anonymised sample of evidence used.
- The independent decisions of at least two markers, including you.
- The comparison, the disagreements, the reasons and the final decisions.
- At least two revisions made to the rubric or task as a result, with before and after wording.
Submission: submit the three items as separate files or as one document with clear headings. Include a short cover note naming the credential and the setting.
Facilitator notes
Allow about 4 hours of the 12 notional hours for the final task, including a live peer moderation session of about 90 minutes. Assign each participant to a moderation group of three. Provide a template for the moderation record. Assessors should be trained on the rubric and should mark a common sample before live marking. Use a second assessor to mark a sample of submissions, in line with the moderation guidance in this course. Provide a resubmission route for participants who do not meet the pass rule, with written feedback by criterion.Rubric for the assessed task
| Criterion | Not yet | Meets | Exceeds |
|---|---|---|---|
| A. Claim and alignment (Outcome 1) Essential | Claim is vague or cannot be checked; criteria do not match the claim or the task. | Claim is stated in checkable terms; criteria, evidence and task are aligned and the stakes rating is justified. | As "Meets", and the plan shows how the alignment was tested, for example by a peer challenge, with a change made as a result. |
| B. Choice of methods (Outcome 1) Essential | Methods are listed without reasons, or do not produce evidence for several criteria. | Each criterion has a method that produces suitable evidence, with a coverage grid and a reason for each choice. | As "Meets", and the plan weighs cost and effort against the stakes, and uses more than one source of evidence where the stakes call for it. |
| C. Rubric structure and descriptors (Outcome 2) Essential | Fewer than four criteria, or descriptors use vague terms such as "good" or "adequate" in most cells; no pass rule. | At least four criteria with observable descriptors, levels that differ in one clear way, a stated pass rule and essential criteria marked. | As "Meets", and descriptors use counts or thresholds where suitable, and have been revised after peer use. |
| D. Moderation plan (Outcome 3) | Plan is missing, or the effort is clearly out of proportion to the stakes without explanation. | Plan states who, what is sampled, when, how disagreements are resolved and how changes are recorded, in proportion to the stakes. | As "Meets", and explains the reasons for the sampling choices and links the plan to the risks identified in the review. |
| E. Moderation record and revision (Outcome 3) | Fewer than two independent decisions, or no disagreements analysed, or no revision made. | Two or more independent decisions, disagreements analysed against descriptors, final decisions recorded and at least two revisions with before and after wording. | As "Meets", and revisions address the root cause of disagreements and are tested on a second sample or explained clearly. |
| F. Bias and accessibility review (Outcome 4) Essential | Review is missing, or lists generic risks not linked to the task. | Demands are checked against the claim, barriers are identified, design responses are given, and the adjustment and RPL routes keep the same standard. | As "Meets", and the review shows evidence of testing with a peer or user perspective, with changes made as a result. |
| G. Integrity and AI (Outcome 4) Essential | Integrity risks are ignored, or the plan relies on a detection tool as the main safeguard. | Integrity risks are identified; design responses fit the stakes; the plan states the rules on AI tools, and the task either stays valid with tools or includes their use openly; a fair process for concerns is outlined. | As "Meets", and the plan shows how the design responses were checked for access barriers and how earners are told the rules. |
Pass rule
To pass, the submission must reach "Meets" or higher on every criterion marked Essential (A, B, C, F and G) and at least "Meets" on one of the other two criteria (D or E). A second assessor reviews every submission that is borderline or at "Not yet" on any criterion. Participants who do not pass receive written feedback by criterion and may resubmit the affected items once.
Evidence submitted for the badge
- Assessment plan
- Rubric
- Moderation record
The badge is "Assessment Design for Credentials: Valid, Reliable, Fair (Ability, L3)", alignment code ICSF-ASMT-L3-A. It records that the earner met the standard on this assessed task. It does not prove competence in any specific job.
Final knowledge check
Answer the questions below before you submit your task. They cover all four learning outcomes. The questions in this section are for your own review and for your assessor to see that you have worked through them.
Your certificate of completion
You passed the final check with %. Type your name as it should appear on the certificate. Your name stays on this device.
Certificate ID . Keep this ID: you need it to upgrade your certificate.
Need proof that others can check? ICoBC can upgrade your certificate to a signed Open Badges 3.0 credential after an identity check and a competence check, for €110 plus VAT.
Upgrade to a verifiable credentialBringing credentials to your organisation? ICoBC offers vendor-neutral consultancy.
Sources and further reading
Status as of November 2026; check before publishing.
The sources below are official texts, standards bodies or framework owners. Check the current edition and the exact title before you rely on them.
- ICoBC Credentialing Skills Framework (ICSF): the framework that defines the skill ASMT "Assessment and evidence", the levels and the stages used in this course. Owner: ICoBC, International Council on Badges and Credentials.
- ICB-F01 Digital Badges and Credentials Essentials: the prerequisite course in the ICoBC Kursbibliothek.
- Open Badges specification: published by 1EdTech Consortium. Read the current version for how achievement criteria and evidence can be described in a badge.
- W3C Verifiable Credentials Data Model: a W3C Recommendation describing how credentials can be expressed and verified digitally.
- Web Content Accessibility Guidelines (WCAG): published by the W3C Web Accessibility Initiative. Use the current version for digital assessment materials.
- ISO/IEC 17024, Conformity assessment: General requirements for bodies operating certification of persons: an international standard of relevance to bodies that certify persons.
- European Qualifications Framework (EQF): published by the European Union. Use it to understand level descriptors if your credential is referenced to a national or European framework.
- Applicable regulator or awarding body guidance: where your credential falls under a regulator, an awarding body or a professional body, follow its published rules on assessment, moderation and appeals.
Time-sensitive topic
Generative AI and assessment integrity is an ongoing issue for all issuers. Detection tools are not a reliable basis for decisions. Status as of November 2026; check before publishing.