# Workplace-Based Competency Assessment *Assessing competency in context, against a level-based framework.* Course ID: ICB-A03 | Framework: ICoBC Credentialing Skills Framework (ICSF) | Skill: ASMT "Assessment and evidence" | Level: L4 "Assure" | Stage: A "Ability" | Pillar: Rigor ## About this course ### Who this course is for This course is for people who lead assessment for credentials that claim to show competency at work: - L&D and corporate academy leads. - Professional bodies and sector associations. - Programme and micro-credential leads. - ICoBC Lead Reviewers. At ICSF level L4 "Assure" you lead assessment strategy for significant programmes, design workplace-based competency assessment and manage integrity risks, including generative AI. This course sits at the "Ability" stage. You practise the capability on realistic tasks in a controlled setting, using the knowledge you already have. You need to have completed ICB-P02 Assessment Design for Credentials: Valid, Reliable, Fair. The course assumes you know the basics of validity, reliability, fairness and rubric design. ### Learning outcomes After this course you can: 1. Design competency assessment in the workplace against a level-based framework such as SFIA. 2. Specify portfolio evidence, attestation and professional discussion. 3. Train and calibrate workplace assessors and attesters. 4. Set validity periods and renewal rules for competency credentials. ### Outcomes, modules and assessment | Outcome | Module | How it is assessed | |---|---|---| | 1. Design competency assessment in the workplace against a level-based framework such as SFIA | Ability and competency | Module quiz; scheme document, section on role, level and assessment design | | 1. Design competency assessment in the workplace against a level-based framework such as SFIA | Evidence from work | Module quiz; scheme document, section on evidence rules | | 2. Specify portfolio evidence, attestation and professional discussion | Evidence from work | Module quiz; scheme document, evidence specification | | 2. Specify portfolio evidence, attestation and professional discussion | Attestation | Module quiz; scheme document, attestation rules | | 2. Specify portfolio evidence, attestation and professional discussion | Professional discussion | Module quiz; scheme document, discussion protocol | | 3. Train and calibrate workplace assessors and attesters | Assessor calibration | Module quiz; scheme document, assessor training and calibration plan | | 4. Set validity periods and renewal rules for competency credentials | Renewal | Module quiz; scheme document, validity and renewal rules | ### Workload and format The course is a blended cohort programme with 10 hours of notional learning. This file covers the self-paced part: reading, worked examples, practice activities and quizzes. Live or workshop elements appear as facilitator notes inside each module. ### How it is assessed You complete a knowledge check with 10 questions. Then you write one **competency assessment scheme** for one role or skill at one level. A trained person assesses the scheme with the rubric in the section "Assessment and badge". The file does not grade your scheme. ### Badge Successful completion leads to the badge **"Workplace-Based Competency Assessment (Ability, L4)"**, alignment code ICSF-ASMT-L4-A. The badge evidence is your scheme document. The badge shows that you practised designing a competency assessment scheme to the standard of the rubric. It does not prove that you can do the job in any workplace. ### Two worked-example contexts Examples run in two settings: - **A corporate academy or HR department**, for example the academy of a logistics company (illustrative). - **A university, college or training provider**, for example a college that places learners in employers (illustrative). All numbers in examples are illustrative. ### A note on frameworks and licences The course uses SFIA as an example of a level-based framework. SFIA 9 is established. SFIA 10 is in consultation, and the SFIA Foundation targets release around Q4 2027. Status as of November 2026; check before publishing. Check the SFIA licence terms before you reproduce descriptors in your own material. This course does not reproduce SFIA descriptors. ## Ability and competency ### From knowledge to ability to competency Three terms need clear use in this course. - **Knowledge** is what a person knows and understands. You can test it with questions. - **Ability** is what a person can do on a realistic task in a controlled setting, such as a case, simulation, sandbox or project. You can observe it, but the setting is prepared. - **Competency** is the sustained capability to perform to a defined level in real work, with real constraints, real people and real consequences. A workplace-based competency assessment looks at competency. It gathers evidence from real work, so it needs rules for what counts, who vouches for it and how long it stays valid. ### Level-based frameworks A level-based framework describes a skill at several levels. Each level describes the autonomy, influence, complexity and knowledge expected. SFIA is a well-known example. It describes professional skills in the digital field across seven levels of responsibility. For your assessment design, a level-based framework gives you three things: 1. A skill with a stable name and definition. 2. A level that says how much responsibility the person carries. 3. A shared language that employers, providers and assessors can use. Your first design decision is the **target**: one role or skill at one level. Fix this before anything else. A scheme that tries to assess "digital skills" in general cannot be assessed with rigour. ### Writing performance statements A framework descriptor is general. Your scheme turns it into **performance statements**: short, observable statements of what a competent person does in your context. A good performance statement: - starts with an active verb; - names the work product or situation; - states the standard or constraint; - can be evidenced from real work. Example for a data analyst at a mid level: "Produces a monthly report from the company data warehouse that the finance team uses without correction, and documents the data sources and checks." Avoid statements that cannot be seen in work, such as "understands data quality". Move those to knowledge checks. Do not copy framework descriptors word for word into your scheme unless the licence allows it. Link to the framework and write your own performance statements. ### Why the workplace The workplace gives you what a simulation cannot: real stakeholders, incomplete information, time pressure and consequences. It also gives you problems: work varies from person to person, supervisors differ, and evidence is messy. A good scheme accepts the variation and controls it with clear rules. **Worked example: Targeting a skill at one level** *Corporate academy.* The academy of a logistics company (illustrative) wants a credential for warehouse planning analysts. The lead picks one skill from the framework, data analysis, and one level, the level where the analyst works with limited supervision and advises others. She writes four performance statements. One reads: "Produces the weekly stock-variance analysis, explains outliers to the warehouse manager and records the corrective action agreed." *College.* A college (illustrative) places second-year software learners with employers for 12 weeks. The programme lead chooses one skill, programming, at the entry practitioner level. One performance statement reads: "Delivers a code change through the team's review process and responds to review comments within the agreed time." In both cases the target is narrow, the statements are observable and the work exists anyway. **Practice activity: Define your target** Choose a role or skill and one level in a framework you know. Write the following in your notes: 1. The role or skill, and the framework and level. 2. Four to six performance statements, each with an active verb, a work product or situation and a standard. 3. One statement you removed because it cannot be seen in work, and where you will assess it instead. Keep this text. It becomes the first section of your scheme document.

Facilitator notes

Run a 45-minute workshop. Participants swap targets in pairs. The partner marks each performance statement as "observable in real work", "needs a simulation" or "knowledge only". Ask pairs to rewrite two weak statements. Collect common weak verbs on a flipchart, for example "understands" and "is aware of". Remind participants not to paste licensed framework text into shared documents.

### Quiz: Ability and competency **Question 1.** Which description fits competency as used in this course? [( )] Passing a written test on the subject. [( )] Completing a simulation with a prepared scenario. [(X)] Performing to a defined level in real work, with real constraints, over time. [( )] Attending all sessions of a training programme. [[?]] Think about the third step in the three-step graphic. **** Competency is sustained performance to a defined level in real work. A written test shows knowledge, a prepared simulation shows ability, and attendance shows neither. **** **Question 2.** What is the first design decision for a workplace-based competency assessment scheme? [( )] Choosing the number of assessors. [(X)] Fixing the target: one role or skill at one level. [( )] Choosing a renewal period. [( )] Selecting a digital badge platform. [[?]] Everything else depends on what you are assessing. **** The target defines what evidence is relevant. Assessors, renewal and platform choices follow from it. **** **Question 3.** Which performance statement is best suited to workplace evidence? [( )] The analyst understands data quality. [( )] The analyst is aware of reporting standards. [(X)] The analyst produces a monthly report that the finance team uses without correction and documents the checks made. [( )] The analyst appreciates the value of accurate data. [[?]] Look for an active verb, a work product and a standard. **** The correct statement is observable, names a work product and states a standard. The others describe states of mind that nobody can see in work. **** **Question 4.** Why should you write your own performance statements instead of copying framework descriptors? [( )] Framework descriptors are always wrong. [( )] Copying makes the scheme longer. [(X)] Descriptors are general and may be licensed, so your scheme needs context-specific statements and a licence check. [( )] Assessors cannot read framework text. [[?]] Think about both context and licence terms. **** Descriptors describe a skill in general terms. Your scheme needs statements that fit your context, and you must check the licence terms before reproducing framework text. **** ## Evidence from work ### What counts as evidence Evidence from work is any record that shows a person performed a task to a standard. It comes in three forms: - **Product**: what the person made or changed, such as a report, a code change, a plan or a recording. - **Testimony**: what others say about the performance, such as a supervisor statement or a customer message. - **Discussion**: what the person says when asked to explain and defend their work. No single form is enough. A product may have been made by someone else. Testimony may be generous. A discussion may be rehearsed. When the three agree, the claim is strong. ### Specifying a portfolio A portfolio is a structured collection of evidence from work. In your scheme, specify the portfolio rules in advance so that every candidate knows what to submit and every assessor knows what to check. A portfolio specification states: 1. **Coverage**: which performance statements must be evidenced, and how many pieces of evidence each needs. 2. **Evidence types**: which products are accepted, and which are not. 3. **Recency**: how old evidence may be. Work from five years ago says little about current competency. 4. **Variety**: whether evidence must come from more than one task, project or time point. 5. **Context record**: what the candidate says about their role, the task, the constraints and what they personally did. 6. **Confidentiality**: how to handle client or employer data, for example by redaction or by assessor visit. 7. **Format and size**: file types, length and storage. Keep the portfolio proportionate. A large portfolio does not mean higher rigour. A small, well-specified portfolio with clear coverage is easier to assess consistently. ### Quality criteria for evidence Assessors judge each piece of evidence against four criteria. They come from assessment practice and fit workplace evidence well: - **Valid**: it relates to the performance statement. - **Authentic**: it is the candidate's own work. - **Current**: it reflects present competency. - **Sufficient**: there is enough of it to show consistent performance, not a one-off. Write these into your scheme as the test for every item. ### Work produced with agents Many workplaces now use AI agents and generative tools to produce drafts, code, analyses and plans. Work produced with agents can be valid evidence. The question is what the candidate did. Two concepts help: - **Authorship**: who made the decisions that shaped the product, and which parts came from the tool. - **Supervision**: how the candidate directed, checked, corrected and took responsibility for the agent's output. Your scheme should ask for a short **agent-use declaration** with each piece of evidence: which tools were used, for what, what the candidate wrote or decided, and what checks were done. Then the discussion tests it. A candidate who supervised well can explain why they accepted, changed or rejected the output. A candidate who only pasted output usually cannot. Decide in advance which skills are assessed on unaided work and which on supervised agent work. For some roles, supervising agents is itself part of the skill, and the performance statements should say so. **Worked example: A portfolio specification with agent declaration** *Corporate academy.* The logistics academy (illustrative) requires analysts to submit three pieces of evidence per performance statement group: two work products from different months, and one supervisor statement. Each product carries a one-page context record and an agent-use declaration. One analyst used a generative tool to draft a variance summary. Her declaration states that she wrote the analysis plan, ran the queries herself, used the tool to draft text, then checked every figure against the source and rewrote two paragraphs. The assessor notes the declaration and plans discussion questions about those checks. *College.* The college (illustrative) asks learners to submit code changes from the employer repository, with the review thread attached. The declaration asks which parts were produced with a coding assistant. One learner reports that the assistant suggested a test structure, which he rejected because it duplicated an existing helper. The review thread supports this. The assessor sees authorship and supervision in the evidence. **Practice activity: Specify the portfolio** Using the target from the previous module, write a portfolio specification of no more than one page. Include: 1. Coverage per performance statement, and the number of evidence items. 2. Accepted evidence types and one type you exclude, with a reason. 3. Recency and variety rules. 4. A confidentiality rule. 5. An agent-use declaration with at least four questions the candidate must answer. Check each rule against the four quality criteria: valid, authentic, current, sufficient.

Facilitator notes

Run a 60-minute workshop. Give each group two sample evidence items (prepared in advance, fictional): one clearly authentic, one with an unclear agent-use history. Groups apply the four criteria and list what extra information they would need. Debrief on what the declaration questions must ask. Keep fictional samples free of real client data.

### Quiz: Evidence from work **Question 1.** Which three forms of evidence does the evidence triangle contain? [( )] Test, certificate and attendance. [(X)] Product, testimony and discussion. [( )] Portfolio, badge and transcript. [( )] Observation, survey and ranking. [[?]] One concerns the work, one concerns others, one concerns the candidate speaking. **** The triangle combines the work itself, what others say about it and what the candidate explains. A claim is strong when all three agree. **** **Question 2.** Which rule in a portfolio specification protects the credential against outdated evidence? [( )] Format and size. [(X)] Recency. [( )] Confidentiality. [( )] Coverage. [[?]] Think about how old a piece of evidence may be. **** A recency rule sets the maximum age of evidence, so that the claim reflects current competency. **** **Question 3.** A candidate submits a report drafted with an AI tool. What should the scheme ask for first? [( )] A refusal to accept the report. [( )] A guarantee that the tool is safe. [(X)] A declaration of which tools were used, what the candidate decided and wrote, and what checks were made. [( )] A longer report. [[?]] Authorship and supervision are the two ideas to test. **** Work produced with agents can be valid evidence. The declaration records authorship and supervision, and the discussion then tests it. **** **Question 4.** Which statement about portfolio size is correct? [( )] More evidence always means higher rigour. [( )] Portfolios should have no limit. [(X)] A small portfolio with clear coverage rules can be assessed more consistently than a large unstructured one. [( )] Only one piece of evidence is ever needed. [[?]] Consistency of judgement matters more than volume. **** Rigour comes from clear rules and consistent judgement. A proportionate portfolio with defined coverage supports both. **** ## Attestation ### What attestation is Attestation is a formal statement by a third party that the candidate performed the work and reached a standard. The attester is usually a line manager, a senior colleague, a client or a placement supervisor. Attestation supplies the testimony corner of the evidence triangle. An attestation is more than a reference letter. A reference says "she is a good colleague". An attestation says "she led the stock-variance analysis in March and April, worked without supervision, and the output met our reporting standard". ### Who can attest Not everyone can attest to everything. Set **attester eligibility rules**: - The attester has directly observed or reviewed the work. - The attester has enough competency in the skill to judge it. At least the target level is a common rule, or a defined role such as a registered professional. - The attester is not in a conflict of interest, for example a close family member, a business partner or someone whose own result depends on the candidate. - The attester accepts responsibility for the statement. Decide what happens when no suitable attester exists, for example for self-employed candidates. Options include a client attester, a peer attester from a professional body, or an extra professional discussion. ### Structured attestation Free-form attestation is hard to compare. Use a **structured attestation form**. It asks the attester to: 1. Identify themselves, their role and their relationship to the candidate. 2. Say how long and in what setting they observed the candidate. 3. For each performance statement, state whether they observed it, with a dated example. 4. Rate the performance against the level descriptors, using a defined scale with anchors. 5. State any limits, for example "I did not observe the planning stage". 6. Declare any conflict of interest. 7. Sign or confirm electronically. Anchors on the scale matter. "Meets the standard" needs an explanation of what meeting the standard looks like. ### Risks and safeguards Attestation has known weaknesses: - **Leniency**: attesters want to help the candidate. - **Halo effect**: a good impression in one area colours all ratings. - **Pressure**: employees may feel unable to refuse. - **Fraud**: forged or invented statements. Safeguards include: requiring specific, dated examples; verifying the attester's identity and role; sampling attestations for follow-up contact; asking the attester to confirm the statement through a separate channel; and training attesters (covered in the module Assessor calibration). Make clear to attesters that false statements may lead to withdrawal of the credential. Attestation of agent-supervised work needs one more question: did the attester see the candidate directing and checking the agent, or only the final product? The form should ask this directly. **Worked example: A structured attestation** *Corporate academy.* The logistics academy (illustrative) asks line managers to attest. A manager must have supervised the analyst for at least three months and hold the target level or above. The form asks for two dated examples per performance statement. One manager writes: "On 12 March the analyst found a pattern of late goods-in entries, traced it to one dock team and proposed a fix that I approved." The reviewer notes the specificity and moves on. A second form says only "excellent worker". The reviewer returns it with a request for examples. *College.* The college (illustrative) asks workplace supervisors to attest, and adds a college tutor visit. The supervisor form includes a question: "Did you see the learner check or correct output from any AI tool? Give an example." A supervisor describes how the learner rejected a suggested query because it ignored a date filter. The tutor, at a visit, finds the same story in the learner's notes. **Practice activity: Design the attestation** Write the attestation rules for your target: 1. Attester eligibility, with at least one conflict-of-interest rule. 2. A fall-back route when no suitable attester exists. 3. A structured form with at least six items, including a scale with anchors for each rating point. 4. Two safeguards against leniency or fraud, and how you will apply them. Then test your form by completing it for a fictional candidate. Note any item that invites vague answers and rewrite it.

Facilitator notes

Hold a 45-minute exchange. Participants swap forms and fill them in for a fictional candidate prepared by the facilitator. Compare completed forms: which items produced specific answers and which produced vague ones? Ask participants to bring one real example (anonymised) of a weak reference they have received. Discuss how a structured form would change it.

### Quiz: Attestation **Question 1.** What distinguishes attestation from a reference letter? [( )] It is longer. [( )] It is written by the candidate. [(X)] It states specific, dated observations of performance against the standard. [( )] It is always signed by a lawyer. [[?]] Think about specificity. **** An attestation names the work, the period and the standard met. A reference letter usually gives a general impression. **** **Question 2.** Which person is least suitable as an attester? [( )] A line manager who supervised the work for six months. [( )] A client who reviewed the deliverable. [(X)] A business partner whose own result depends on the candidate passing. [( )] A senior colleague who reviewed the code. [[?]] Look for a conflict of interest. **** A conflict of interest reduces trust in the statement. The other three have observed or reviewed the work and have no stake in the result. **** **Question 3.** Why do rating scales on attestation forms need anchors? [( )] To make the form look professional. [( )] To shorten the form. [(X)] To give attesters a shared meaning of each rating point and reduce leniency. [( )] To remove the need for dated examples. [[?]] Consider what "meets the standard" means to different people. **** Anchors describe what each rating point looks like in practice. They support consistency across attesters and limit leniency. **** **Question 4.** What additional question helps when the candidate used AI agents in the attested work? [( )] Which vendor supplied the tool? [( )] How much did the tool cost? [(X)] Did the attester see the candidate direct and check the agent, or only the final product? [( )] Was the tool installed on the candidate's own device? [[?]] The concern is supervision. **** Testimony about the final product alone does not show supervision. The attester should say what they saw of the candidate directing, checking and correcting the agent. **** ## Professional discussion ### What a professional discussion is A professional discussion is a structured conversation between an assessor and a candidate about the candidate's real work. It is the discussion corner of the evidence triangle. The assessor asks questions that make the candidate explain decisions, show reasoning and reflect on what they did. It is not an interview about the candidate's character, and it is not a quiz. The discussion has three jobs: 1. **Authenticate**: confirm the candidate did the work. 2. **Deepen**: reach competencies that are hard to see in products, such as judgement, risk handling and working with people. 3. **Fill gaps**: explore performance statements where the portfolio is thin. ### Designing the protocol A written protocol makes discussions comparable. Specify: - **Purpose and scope**: which performance statements the discussion covers. - **Length**: for example 45 to 60 minutes (illustrative), with a fixed range. - **Mode**: in person, by video, recorded or not. State how you handle consent and records. - **Question bank**: a set of core questions asked of everyone, plus prompts for follow-up. - **Structure**: opening, portfolio walk-through, scenario or "what if" questions, close. - **Timing and adjustments**: reasonable adjustments for candidates with additional needs, and language support where the language of work is not the candidate's first language. - **Record**: notes or recording kept against a template. - **Decision rules**: how the assessor turns the discussion into judgements, using the same rubric as for the portfolio. ### Good questions Good questions are open, specific to the candidate's evidence and tied to a performance statement. Useful types: - **Decision**: "At this point in the project you chose option B. What were the alternatives and why did you reject them?" - **Counterfactual**: "What would you have done if the data had arrived a week late?" - **Failure**: "Tell me about something that went wrong and what you changed." - **Transfer**: "How would you handle this in a different setting?" - **Authenticity probe**: "Walk me through how you produced this chart, step by step." Avoid leading questions, double questions and questions that can be answered with yes or no. ### Discussion and agent-supervised work When evidence was produced with agents, the discussion is the strongest test of supervision. Useful probes: - "Which parts did the tool draft and which parts did you write?" - "What did you check, and how? Show me one example where the output was wrong." - "What would you do if the tool gave a plausible but incorrect answer here?" - "What did you decide not to delegate to the tool, and why?" - "Reproduce this step now, without the tool, at a basic level." The last probe has limits. Use it only if the performance statements require unaided capability. If supervising agents is the skill, test the supervision. A candidate who supervised well answers with detail, names specific checks and shows judgement. A candidate who did not supervise gives general answers, cannot explain choices or contradicts the portfolio. ### Fairness and records Keep the discussion fair. Use the same core questions for every candidate. Train assessors to avoid bias from accent, appearance or style. Allow thinking time. Keep a record in a form that allows moderation and appeal. Tell candidates in advance what the discussion covers and how the result is used. **Worked example: A discussion protocol** *Corporate academy.* The logistics academy (illustrative) runs a 50-minute video discussion. The assessor opens with five minutes on context, then asks the candidate to walk through one portfolio item, then asks three scenario questions from the bank, then two agent-use probes, then closes. Notes go on a template with one line per performance statement. When a candidate describes a variance analysis in vague terms, the assessor asks for the first query she ran and why. The detail she gives, including a data-quality filter she added, supports authorship. *College.* The college (illustrative) holds discussions on the employer site where possible, with the workplace supervisor not present. The tutor-assessor asks a learner about a code change that used an assistant. The learner explains the design, names a bug in the suggested code that he found through a failing test and explains how he fixed it. The tutor records the evidence against two performance statements and cross-checks with the review thread. **Practice activity: Write the discussion protocol** Write a protocol of no more than two pages for your target: 1. Purpose, scope, length and mode. 2. A question bank with at least eight questions, covering decision, counterfactual, failure and authenticity types, plus at least two agent-use probes. 3. A structure with timings. 4. Rules for adjustments and for records. 5. Decision rules: how the discussion links to the rubric. Then rewrite any question that is leading, double or closed.

Facilitator notes

Run a 90-minute live practice session in triads: assessor, candidate and observer. The candidate brings a short (fictional or anonymised) work example. The assessor runs 15 minutes of discussion using the group's question bank. The observer notes which questions produced detail and which did not. Rotate roles. Close with a plenary on question quality and on how to handle a candidate who becomes distressed. Do not use real confidential work.

### Quiz: Professional discussion **Question 1.** Which is a core purpose of a professional discussion? [( )] To test the candidate's personality. [(X)] To authenticate the work and explore competencies hard to see in products. [( )] To replace the portfolio. [( )] To teach new content. [[?]] Think about the discussion corner of the triangle. **** The discussion confirms that the candidate did the work and reaches reasoning, judgement and gaps in the portfolio. It supports the portfolio and does not replace it. **** **Question 2.** Which question is best for a professional discussion? [( )] Did you enjoy the project? [( )] Was the report correct, and who read it, and what did they say? [(X)] At this point you chose option B. What were the alternatives and why did you reject them? [( )] You used the tool a lot, didn't you? [[?]] Look for an open question tied to the candidate's own work. **** The correct question is open, specific and tied to a decision in the evidence. The others are closed, double or leading. **** **Question 3.** What makes a discussion protocol support fairness? [( )] Each assessor writes their own questions. [(X)] The same core questions for every candidate, adjustments where needed and a record for moderation. [( )] No notes are taken. [( )] The candidate chooses the assessor. [[?]] Comparability and traceability both matter. **** A shared question core, documented adjustments and a record support comparable decisions and make moderation and appeal possible. **** **Question 4.** When is asking a candidate to reproduce a step without the AI tool appropriate? [( )] Always, for every candidate. [( )] Never. [(X)] Only when the performance statements require the capability to work unaided. [( )] Only when the candidate seems nervous. [[?]] The test must match the skill defined in the target. **** Test what the performance statements say. If supervising agents is the skill, test supervision. If unaided capability is required, test that. **** ## Assessor calibration ### Why calibrate Two assessors can look at the same evidence and reach different decisions. In workplace-based assessment the risk is high: evidence varies, assessors work in different organisations and many are not professional assessors. Calibration reduces the gap. It means bringing assessors to a shared understanding of the standard. Calibration covers workplace assessors, who judge candidates, and attesters, who give testimony. Both need preparation, but at different depth. ### Assessor roles Define roles clearly in the scheme: - **Attester**: gives structured testimony. Light training. - **Workplace assessor**: reviews portfolio evidence and leads discussions. Full training and authorisation. - **Internal quality assurer (moderator)**: samples decisions, checks consistency, supports assessors. - **Lead assessor or panel chair**: resolves disputes and owns the standard. Set **eligibility** for each role: relevant competency at or above the target level, assessment training, and independence from the candidate. ### Training content A workable training programme covers: 1. The framework, the target skill and level, and the performance statements. 2. The rubric and what each rating point looks like. 3. The evidence rules: validity, authenticity, currency, sufficiency. 4. Conducting the discussion: questioning, listening, recording. 5. Handling bias, conflicts of interest and fairness. 6. Handling work produced with agents: reading declarations, probing supervision. 7. Confidentiality, data protection and record keeping. 8. Appeals and escalation. Use practice, not only briefing. Assessors work through anonymised cases, make decisions and compare them. ### Calibration activities Calibration is a cycle, not a one-time event: - **Benchmarking**: assessors judge the same set of sample portfolios and discuss differences. Samples include clear passes, clear fails and borderline cases. - **Exemplars**: a library of annotated evidence showing each rating point. - **Double marking**: for a sample of live candidates, a second assessor judges independently. - **Moderation meetings**: regular sessions to discuss disputed cases and update guidance. - **Agreement checks**: track how often assessors agree, and review cases where they differ. Set an agreement target in your scheme, for example a share of decisions within one rating point (illustrative). - **Authorisation and re-authorisation**: assessors are authorised after calibration and re-checked at fixed intervals. When agreement is low, find the cause: unclear rubric, weak anchors, insufficient training or an assessor who needs support. Fix the cause, not only the individual. ### Calibrating for agent-supervised evidence Add calibration cases where the evidence involves agents. Include one with strong supervision, one with weak supervision and one with an unclear declaration. Ask assessors to decide and to say what extra evidence they would request. Keep these cases current, as tools and practices change. **Worked example: A calibration session** *Corporate academy.* The logistics academy (illustrative) trains eight line managers as attesters and four senior analysts as assessors. In a half-day session the assessors rate three anonymised portfolios independently, then compare. Two assessors differ on a borderline case: one accepted a single supervisor statement as sufficient, the other did not. The group agrees on a rule: at least two dated examples per performance statement group. The rule goes into the guidance, and the exemplar library gains the case. *College.* The college (illustrative) draws assessors from tutors and placement supervisors. Each term, tutors double-mark a sample of portfolios. In one round, agreement on the rating for "reviews own code" falls. The moderator finds that supervisors read "reviews own code" differently when a coding assistant is used. The team adds an anchor example and a calibration case about checking assistant output. **Practice activity: Plan assessor training and calibration** Write an assessor and attester plan for your target, no more than two pages: 1. Roles and eligibility rules. 2. Training outline with at least six topics and the method for each. 3. A calibration cycle: benchmarking, exemplars, double marking, moderation, with a frequency for each. 4. An agreement measure and a target, and what you will do when it is missed. 5. One calibration case on agent-supervised evidence, described in four lines.

Facilitator notes

Run a 2-hour calibration workshop. Prepare three fictional portfolios in advance: one clear pass, one clear fail and one borderline, with at least one involving agent use. Participants judge individually using the group's rubric, then reveal ratings at the same time. Chart the spread on a flipchart. Discuss the largest differences and agree on one clarified anchor. Close by asking participants to estimate how often they would expect agreement in a live scheme and how they would check.

### Quiz: Assessor calibration **Question 1.** What is the aim of calibration? [( )] To make all candidates pass. [(X)] To bring assessors to a shared understanding of the standard so that decisions are consistent. [( )] To replace the rubric. [( )] To reduce the number of assessors. [[?]] Think about two assessors reading the same evidence. **** Calibration aligns judgements against the standard. It does not change the standard or the pass rate. **** **Question 2.** Which activity is a calibration method? [( )] Giving assessors a longer manual. [(X)] Assessors judging the same sample portfolios independently and then discussing differences. [( )] Letting each assessor set their own pass mark. [( )] Removing borderline cases from the sample. [[?]] It involves comparing real judgements. **** Benchmarking on shared samples shows where judgements differ and gives a basis for agreement. Borderline cases are the most useful samples. **** **Question 3.** Agreement between assessors falls on one rating. What should the scheme owner do first? [( )] Replace all assessors. [( )] Remove the rating from the rubric without review. [(X)] Look for the cause, such as unclear anchors or training gaps, and fix it. [( )] Ignore it until the next renewal. [[?]] Find out why before acting. **** Low agreement often comes from unclear anchors or insufficient training. Fix the cause, then re-check agreement. **** **Question 4.** Why add agent-supervised evidence to calibration cases? [( )] Because assessors dislike AI. [( )] To discourage candidates from using tools. [(X)] Because assessors need a shared way to judge authorship and supervision, and practices change. [( )] Because it makes sessions longer. [[?]] Consistency applies to new kinds of evidence too. **** Assessors need practice in reading declarations and judging supervision so that decisions are consistent. The cases need updating as tools and practices change. **** ## Renewal ### Why competency credentials expire Competency changes. Skills fade without use, tools and standards move, and roles change. A credential that never expires keeps saying "competent" about a person whose work may have changed. Validity periods and renewal rules keep the claim honest. A credential is not valid "for ever" and not valid "for a fixed time for everyone". The right period depends on the field. ### Setting the validity period Consider these factors: - **Pace of change** in the field, tools and regulation. - **Risk** if a holder is no longer competent, for example safety, finance or personal data. - **Level**: higher levels often involve judgement that changes more slowly, but the consequences of error may be larger. - **Frequency of practice**: roles that use the skill daily keep it current more easily. - **Burden** on holders and employers. - **Expectations** of regulators, professional bodies and employers where relevant. Pick a period that you can justify in writing, for example two, three or five years (illustrative). Record the rationale in the scheme. Avoid copying a period from another scheme without reason. ### Renewal routes Offer one or more routes and state the evidence for each: - **Re-assessment**: a full or reduced repeat of the original process. - **Evidence of continued practice**: dated work evidence from the validity period, with attestation, assessed against current performance statements. - **Continuing development**: documented learning relevant to changes in the skill, with a limit on how far this alone can renew. - **Bridging assessment**: a focused check on what has changed, for holders who meet recency but not currency. A common design uses **continued practice plus a short discussion** as the default route, and full re-assessment when the holder has not practised or the standard has changed substantially. ### Lapse, suspension and withdrawal Define what happens in each case: - **Expiry**: the credential ends on the stated date. Say if there is a grace period and what the holder may claim during it. - **Lapse and reinstatement**: rules for people who return after a gap. Longer gaps require fuller assessment. - **Suspension**: temporary, pending investigation, for example when concerns arise about integrity. - **Withdrawal**: permanent removal, for example for false evidence or serious misconduct. - **Appeal**: how holders challenge a decision. State how the status shows in the digital credential and in any public register, so that employers can check it. ### When the standard changes Frameworks and scheme standards change. When the framework publishes a new version, decide whether existing credentials stay valid until expiry, and what holders need to do to move to the new version. SFIA 10, for example, is in consultation and targeted for release around Q4 2027. Status as of November 2026; check before publishing. Your scheme should state which framework version it uses and a rule for version change, so holders and employers know where they stand. ### Agent-era renewal If the skill changes because agents take on tasks, renewal evidence should show current practice: how the holder now works with agents, supervises them and keeps authorship. A renewal route that asks only for old-style evidence may leave the credential out of date. Review performance statements at each scheme review. **Worked example: Validity and renewal rules** *Corporate academy.* The logistics academy (illustrative) sets a three-year validity for its analyst credential. Rationale: the data tools change about every two to three years, and errors in stock analysis carry moderate cost. The default renewal route is a portfolio of two recent products with attestation, plus a 30-minute discussion. If the holder has not worked as an analyst for more than 12 months, they must complete a bridging assessment. If the framework version changes, current credentials stay valid until expiry, and renewals use the new version. *College.* The college (illustrative) issues a credential for entry practitioners with a two-year validity, reflecting fast change in tools and the early stage of careers. Renewal requires evidence of 12 months of relevant work in the last 24, one attestation and a short discussion. Holders whose work has moved on to a higher level may apply for the higher-level credential instead of renewing. **Practice activity: Write the renewal rules** Write the validity and renewal section for your scheme, no more than one page: 1. Validity period and a written rationale using at least three of the factors above. 2. Renewal routes and the evidence each requires. 3. Rules for expiry, grace period, lapse and reinstatement. 4. Rules for suspension and withdrawal, and an appeal route. 5. A rule for a framework version change. 6. One sentence on how renewal evidence will reflect current work with agents.

Facilitator notes

Run a 45-minute workshop. Groups receive a fictional role with different risk and change profiles (for example a data-protection officer, a junior web developer and a training coordinator). Each group proposes a validity period with a rationale and a renewal route. Groups present, and others challenge the rationale. Highlight that different periods can all be defensible if justified. Ask groups to name who would be affected by a lapse.

### Quiz: Renewal **Question 1.** Why do competency credentials need a validity period? [( )] To raise revenue. [(X)] To keep the claim of competency current as skills, tools and roles change. [( )] To make renewal harder than the first assessment. [( )] To avoid keeping records. [[?]] Think about what the credential claims about the holder. **** A validity period keeps the credential's claim honest, because competency can fade and the field changes. **** **Question 2.** Which factor should influence the validity period? [( )] The colour of the digital badge. [( )] The number of assessors available. [(X)] The pace of change in the field and the risk if a holder is no longer competent. [( )] The length of the course. [[?]] Look for factors linked to the field and its risks. **** Pace of change and risk are core factors. Others include frequency of practice, burden and regulatory expectations. **** **Question 3.** Which renewal route fits a holder who has worked continuously in the role? [( )] No evidence, because they already hold the credential. [(X)] Dated work evidence from the validity period with attestation and a short discussion. [( )] Only attendance at a webinar. [( )] A self-declaration of competency. [[?]] The route should show current practice. **** Evidence of continued practice, attested and discussed, shows current performance. A webinar or a self-declaration does not. **** **Question 4.** The framework publishes a new version. What should the scheme have in place? [( )] Nothing; holders will find out. [( )] Immediate withdrawal of all existing credentials. [(X)] A stated framework version and a rule for how existing credentials and renewals treat a version change. [( )] A new badge design. [[?]] Holders and employers need to know where they stand. **** The scheme should state the version it uses and what happens at version change, for example that credentials stay valid until expiry and renewals use the new version. **** ## Assessment and badge ### The task: a competency assessment scheme Write a **competency assessment scheme** for one role or skill at one level, in a framework of your choice. It must be usable by another organisation without your help. Aim for 8 to 12 pages (illustrative). Use your outputs from the practice activities, and improve them. Your scheme must contain: 1. **Target and framework**: the role or skill, the framework and level, the framework version and a licence note. Four to eight performance statements. 2. **Assessment design**: how the three evidence forms (product, testimony, discussion) work together, with a map from performance statements to evidence. 3. **Evidence specification**: the portfolio rules, quality criteria and the agent-use declaration. 4. **Attestation rules**: eligibility, structured form and safeguards. 5. **Professional discussion protocol**: structure, question bank, adjustments and records. 6. **Assessor and attester training and calibration plan**: roles, training, calibration cycle and agreement measure. 7. **Validity and renewal rules**: period with rationale, routes, lapse, suspension, withdrawal, appeal and version change. 8. **Integrity and AI section**: how the scheme handles authorship and supervision of work produced with agents, and how it will be reviewed. Do not include real personal data or confidential client material. Do not state that completing the scheme or the course proves competency in a job. ### Rubric A trained assessor rates each criterion on four levels: 1 Not yet, 2 Developing, 3 Meets, 4 Strong. The file does not grade your scheme. | Criterion | 1 Not yet | 2 Developing | 3 Meets | 4 Strong | |---|---|---|---|---| | Target and performance statements (outcome 1) | Target is broad or missing; statements are vague | Target is clear; some statements are not observable | One role or skill at one level; statements are observable and tied to the framework | As 3, and statements are tailored to context with clear standards and a licence note | | Assessment design (outcome 1) | No link between statements and evidence | Partial map; one evidence form dominates | Clear map from each statement to product, testimony and discussion | As 3, and with justified weighting and a plan for gaps and borderline cases | | Portfolio evidence (outcome 2) | Evidence rules missing or unusable | Some rules; quality criteria incomplete | Coverage, recency, variety, confidentiality and quality criteria stated and workable | As 3, and proportionate, tested on a sample and with clear handling of weak evidence | | Attestation (outcome 2) | Informal references only | Form exists but eligibility or safeguards are weak | Eligibility, structured form with anchors and safeguards specified | As 3, and with fall-back routes and checks on attester independence | | Professional discussion (outcome 2) | No protocol | Protocol lacks question bank or decision rules | Protocol with structure, question bank, adjustments, records and links to the rubric | As 3, and questions are tied to evidence types and tested for fairness | | Assessor training and calibration (outcome 3) | No plan | Training listed; no calibration cycle | Roles, eligibility, training content, calibration cycle and agreement measure | As 3, and with response to low agreement and a re-authorisation rule | | Validity and renewal (outcome 4) | No period or rules | Period set without rationale; routes incomplete | Period with rationale, routes, lapse, suspension, withdrawal, appeal and version rule | As 3, and rules tuned to risk and change in the field, with reinstatement paths | | Integrity and work with agents (outcomes 2 and 3) | Not addressed | Mentioned in general terms | Declaration, discussion probes, attester question and assessor calibration for agent-supervised work | As 3, and with a review cycle and a clear position on unaided versus supervised capability | | Clarity and usability | Hard to follow; gaps | Understandable; some gaps | Another organisation could use it; terms are consistent | As 3, and well structured with templates and examples | ### Pass rule To earn the badge **"Workplace-Based Competency Assessment (Ability, L4)"** (alignment code ICSF-ASMT-L4-A): 1. Score at least 7 out of 10 on the knowledge check below (you may retake it). 2. Achieve level 3 "Meets" or higher on every rubric criterion. A single criterion at level 2 may be resubmitted once after feedback. 3. Submit the scheme document as badge evidence. ### Evidence submitted for the badge - The scheme document (PDF or word-processed file). - A one-page cover note naming the framework version, the target and the author, and confirming that no confidential or personal data is included. The assessor may ask for a short conversation about your scheme.

Facilitator notes

Schedule a 2-hour live clinic in the final week. Participants present a two-minute outline of their scheme and receive one piece of peer feedback against the rubric. Assessors review submitted schemes independently, then meet to compare ratings on at least two schemes before releasing results. Provide written feedback against each criterion. Offer resubmission as stated in the pass rule.

### Knowledge check **Question 1.** A lead wants to create a credential for "digital skills". What should she do first? [( )] Choose a badge platform. [( )] Recruit assessors. [(X)] Narrow the target to one role or skill at one level and write observable performance statements. [( )] Set the renewal period. [[?]] Everything else depends on the target. **** A scheme that assesses "digital skills" in general cannot be assessed with rigour. Narrow the target and write observable performance statements first. **** **Question 2.** How does competency differ from ability as used in this course? [( )] Competency is tested by questions and ability by observation. [( )] Ability is sustained and competency is a single task. [(X)] Ability is shown on realistic tasks in a controlled setting; competency is sustained performance to a level in real work. [( )] They mean the same. [[?]] One happens in a prepared setting, the other in real work over time. **** Ability is practising in a controlled setting. Competency is sustained performance in real work. A workplace-based scheme assesses the second. **** **Question 3.** Which portfolio rule best protects against evidence that no longer reflects current competency? [( )] A limit on file size. [(X)] A recency rule stating the maximum age of evidence. [( )] A rule that the candidate writes the context record. [( )] A rule on file format. [[?]] Think about the quality criterion "current". **** A recency rule links directly to the criterion "current". The other rules serve practical or authenticity purposes. **** **Question 4.** A candidate submits analysis produced with an AI agent. Which combination best tests authorship and supervision? [( )] A ban on agents and a signed promise. [( )] A longer portfolio. [(X)] An agent-use declaration, a supervisor statement on what they saw, and discussion probes on checks and decisions. [( )] A multiple-choice test on the tool's features. [[?]] Use all three corners of the evidence triangle. **** The declaration records the candidate's account, the supervisor statement adds testimony and the discussion tests the account. Together they address authorship and supervision. **** **Question 5.** Which attestation is strongest evidence? [( )] "She is a great team member." [( )] "I recommend her without reservation." [(X)] "On 12 March she traced late goods-in entries to one dock team and proposed a fix that I approved." [( )] "He has been here five years." [[?]] Look for dated, specific observation of performance. **** A strong attestation names the date, the task and the outcome. The others give impressions or tenure. **** **Question 6.** Which discussion question best supports authentication of the work? [( )] Did you do this yourself? [( )] Are you proud of this report? [(X)] Walk me through how you produced this chart, step by step, and what you checked. [( )] Everything went well, didn't it? [[?]] An open question that requires detail only the author would have. **** Asking for a step-by-step account with checks produces detail that the author can give and others find hard to invent. The other questions are closed or leading. **** **Question 7.** In a scheme with attesters, what is the purpose of an attester eligibility rule? [( )] To limit the number of candidates. [(X)] To make sure attesters have observed the work, have enough competency to judge it and have no conflict of interest. [( )] To make forms shorter. [( )] To avoid the need for assessors. [[?]] Consider who can credibly vouch for the work. **** Eligibility rules support credible testimony: direct observation, adequate competency and independence. **** **Question 8.** Agreement between workplace assessors is lower than the scheme's target. What is the best response? [( )] Raise the pass mark. [( )] Stop double marking. [(X)] Analyse the cases of disagreement, clarify anchors or training, and re-check agreement. [( )] Ask the candidates to resubmit. [[?]] Find the cause before changing anything. **** Analysing disagreement shows whether anchors, training or individual support need work. Then a re-check shows whether the fix worked. **** **Question 9.** Which element belongs in an assessor calibration cycle? [( )] A single briefing at the start and no further checks. [(X)] Benchmarking on shared samples, exemplars, double marking, moderation and re-authorisation at fixed intervals. [( )] Letting assessors choose which cases to review. [( )] Using only clear pass cases. [[?]] Calibration repeats and uses a range of cases. **** Calibration is a cycle. It uses shared samples including borderline cases, a library of exemplars, double marking, moderation and re-authorisation. **** **Question 10.** Which statement describes a defensible validity period? [( )] Always three years, because most schemes use three years. [( )] Unlimited, because the person passed once. [(X)] A period set with a written rationale based on pace of change, risk, frequency of practice and burden. [( )] The period that makes renewal revenue highest. [[?]] The period needs a justification you can write down. **** A validity period is defensible when the scheme owner can explain it through the field's pace of change, the risk of lapse, practice frequency and burden on holders. **** ## Sources and further reading Status as of November 2026; check before publishing. The following are sources of official texts and standards bodies. Check each for the current version before use. - **SFIA Foundation**: owner of the Skills Framework for the Information Age (SFIA). Consult the SFIA Foundation's official website for SFIA 9, the licence terms and the status of SFIA 10. SFIA 9 is established. SFIA 10 is in consultation, with a release targeted around Q4 2027. Check the licence terms before reproducing descriptors in course material. - **ICoBC Credentialing Skills Framework (ICSF)**: the ICoBC framework that defines the skill ASMT "Assessment and evidence", levels and stages used in this course. Obtain the current version from ICoBC (info@icobc.net). - **ICB-P02 Assessment Design for Credentials: Valid, Reliable, Fair**: the prerequisite course in the ICoBC Kursbibliothek. - **ISO/IEC 17024**: international standard on general requirements for bodies operating certification of persons. Consult the International Organization for Standardization for the current edition. Relevant to assessment, examiner competence and certification validity. - **ISO 21001**: international standard on management systems for educational organisations. Consult the International Organization for Standardization for the current edition. - **Standards for Educational and Psychological Testing**: joint publication of the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education. Relevant to validity, reliability and fairness.