{"id":804,"date":"2026-07-17T23:58:22","date_gmt":"2026-07-18T06:58:22","guid":{"rendered":"https:\/\/blogs.ubc.ca\/educationaltech\/?p=804"},"modified":"2026-07-17T23:58:22","modified_gmt":"2026-07-18T06:58:22","slug":"etec-565t-assignment-2-evaluation-heuristic-and-testing","status":"publish","type":"post","link":"https:\/\/blogs.ubc.ca\/educationaltech\/etec-565t-assignment-2-evaluation-heuristic-and-testing\/","title":{"rendered":"ETEC 565T &#8211; Assignment #2: Evaluation Heuristic and Testing"},"content":{"rendered":"<h1>Using AI to Generate Teaching a Serve Receive Progression to High School Volleyball Players Overview<\/h1>\n<p><span style=\"font-weight: 400;\">As a high school volleyball coach, I spend a lot of time deciding what my team needs to improve on and then designing drills to target those skills. I carefully consider my athletes\u2019 skill level, ensuring the drills are safe, technically sound, and developmentally appropriate. I have increasingly relied on GenAI to help with practice planning, but the challenge is making sure what it creates is appropriate and of high quality. GenAI can be a very useful tool for coaching, but only if the output is carefully critiqued against the same standards I already hold my own coaching to. If the GenAI content is not carefully vetted, I risk wasting precious practice time, disengaging my athletes, or, worse, putting them at risk of injury.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In this assignment, I will create a simple, usable evaluation framework I can follow to judge if the GenAI output is appropriate for my athletes\u2019 needs, focusing on the skill of serve-receive. Serve-receive is one of the most foundational skills in volleyball, since it is the receiving team&#8217;s first opportunity to set up a scoring play.\u00a0<\/span><\/p>\n<h1><b>Procedure<\/b><\/h1>\n<p><b>A task GenAI can help me with<\/b><\/p>\n<p><span style=\"font-weight: 400;\">I will use <\/span><a href=\"http:\/\/claude.ai\"><span style=\"font-weight: 400;\">Claude<\/span><\/a><span style=\"font-weight: 400;\"> to create a serve-receive drill sequence for high school volleyball players that I can use in my own coaching.<\/span><\/p>\n<p><b>Evaluation Framework \/ Rulebook for judging if GenAI did it well\u00a0<\/b><\/p>\n<p><span style=\"font-weight: 400;\">I will use my evaluation framework to judge whether the response I get from Claude is actually useful. The goal is to make sure I don&#8217;t simply enter a prompt, receive a response, and blindly follow the practice plans, but instead critically evaluate whether the response is good enough to fit my context, match my athletes&#8217; skill level, align with my coaching philosophy, and, most importantly, are safe.<\/span><\/p>\n<p><strong>Procedure<\/strong><\/p>\n<p><span style=\"font-weight: 400;\">First, I developed my own evaluation framework, grounded in course scholarship as well as UBC\u2019s institutional guidelines on GenAI use in teaching and learning. This framework was designed to assess if the AI-generated content was technically accurate and age-appropriate, and also if it is aligned with principles of transparency, equity, sustainability, and human oversight. I then tested the framework by having <\/span><a href=\"http:\/\/claude.ai\"><span style=\"font-weight: 400;\">Claude<\/span><\/a><span style=\"font-weight: 400;\"> generate a sample serve-receive progression and evaluated the output against each criterion.\u00a0<\/span><\/p>\n<h1><b>Core Evaluation Criteria &#8211; &#8220;what good output looks like&#8221;<\/b><\/h1>\n<h2><b>1. SAFETY<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">* Is the drill safe for my athletes to perform? Is there a potential for injury or harm?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">* Is the progression from simple to complex (warming up large muscle groups, avoiding overuse activities, preventing collisions)<\/span><\/p>\n<h2><b>2. TECHNICAL ACCURACY &amp; PRACTICALITY<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">* Are the skill mechanics correct for serve-receive?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">* Is this drill sequence something I can realistically run within the time, space, and equipment I have available?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">* Is the practice planning in a logical sequence?<\/span><\/p>\n<h2><b>3. AGE &amp; SKILL APPROPRIATENESS<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">* Will these drills be appropriate for my athletes\u2019 level (not too simple, not too complex)?\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">* Are there suggestions or potential for varying the skill levels that match my athletes\u2019 skills?<\/span><\/p>\n<h2><b>4. COACH OVERSIGHT &amp; VOICE<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">* Is this drill something that aligns with my coaching philosophy?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">* How much do I need to edit the drills to adapt it to my coaching?<\/span><\/p>\n<h2><b>5. SUSTAINABILITY\u00a0<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">* Is this actually saving me time?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">* Are the environmental costs justified?<\/span><\/p>\n<h2><b>6. ETHICS &amp; BIAS CHECK<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">* Are the results free from assumptions that are biased or skewed?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">* Is the language free from assumptions about gender, athleticism, etc.?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">* Can I complete these drills regardless of facilities\/equipment\/coaching staff I have access to?<\/span><\/p>\n<h2><b>EVALUATION<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">Results will be evaluated using the rating of 0-2<\/span><\/p>\n<p><span style=\"font-weight: 400;\">2 = \u201cusable as-is\u201d<\/span><\/p>\n<p><span style=\"font-weight: 400;\">1 = \u201cusable with edits\u201d<\/span><\/p>\n<p><span style=\"font-weight: 400;\">0 = \u201cnot usable\u201d<\/span><\/p>\n<p><span style=\"font-weight: 400;\">* If the safety category is \u201c0\u201d, the drill will not be used since safety overrides everything.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">* I decided to not use Transparency\/Attribution because this would be for my own coaching resource library, and I wouldn&#8217;t be sharing it with others.\u00a0<\/span><\/p>\n<h1><b>Actually test it:\u00a0<\/b><\/h1>\n<p><span style=\"font-weight: 400;\">Prompt for Clauide.ai: \u201cgenerate a serve-receive drill sequence for high school athletes &#8211; design 4 practices that are each 2 hours long\u201d<\/span><\/p>\n<h2><b>RESPONSE FROM CLAUDE.AI:<\/b><\/h2>\n<p><b>PRACTICE 1: FOUNDATIONAL PLATFORM &amp; FOOTWORK<\/b><\/p>\n<p><b>Focus: Individual passing mechanics<\/b><\/p>\n<p><b>0:00-0:20 \u2014 Warm-up: Dynamic stretching, arm circles, light jogging, partner tosses (no contact, just tracking the ball)<\/b><\/p>\n<p><b>0:20-0:45 \u2014 Platform mechanics: Partner passing drill \u2014 stationary, focus on platform angle, hand position, and &#8220;shoulders to target.&#8221; Coach walks the line correcting form.<\/b><\/p>\n<p><b>0:45-1:10 \u2014 Footwork progression: Shuffle-and-pass drill \u2014 players start 3 feet off their spot, shuffle to the ball on a coach toss, pass to target. Repeat with tosses to both sides.<\/b><\/p>\n<p><b>1:10-1:35 \u2014 Serve-receive vs. easy serves: 3-player passing line receives controlled, coach-served balls (underhand or light overhand). Focus purely on platform control, not target accuracy yet.<\/b><\/p>\n<p><b>1:35-1:55 \u2014 Reinforcement circuit: Rotate through 3 stations (wall passing, partner passing, coach toss) in small groups to maximize reps<\/b><\/p>\n<p><b>1:55-2:00 \u2014 Cool down + water: Static stretching, quick recap of key cues (angle, feet, follow-through)<\/b><\/p>\n<p>&nbsp;<\/p>\n<p><b>PRACTICE 2: READING THE SERVE &amp; TARGET PASSING<\/b><\/p>\n<p><b>Focus: Serve recognition and passing accuracy to target<\/b><\/p>\n<p><b>0:00-0:20 \u2014 Warm-up: Same as Practice 1, plus dynamic footwork ladder<\/b><\/p>\n<p><b>0:20-0:45 \u2014 Serve recognition drill: Players pass in pairs; server calls out serve type (float, topspin) before serving so passers learn to anticipate platform adjustments<\/b><\/p>\n<p><b>0:45-1:10 \u2014 3-person passing triangle: Server, passer, target (setter position). Focus on passing to a consistent target zone, not just controlling the ball.<\/b><\/p>\n<p><b>1:10-1:35 \u2014 Live serve-receive reps: Full 6-player serve-receive formation (3 passers) receiving live serves from a rotating group of servers. Coach tracks pass quality (in system \/ out of system).<\/b><\/p>\n<p><b>1:35-1:55 \u2014 Target accuracy competition: Passers earn points for passes landing in a marked target zone; rotate passers every 5 reps<\/b><\/p>\n<p><b>1:55-2:00 \u2014 Cool down: Stretch, discuss common reading errors from the session<\/b><\/p>\n<p>&nbsp;<\/p>\n<p><b>PRACTICE 3: COMMUNICATION &amp; TEAM SYSTEM<\/b><\/p>\n<p><b>Focus: Calling the ball, seams, and rotational responsibility<\/b><\/p>\n<p><b>0:00-0:20 \u2014 Warm-up: Standard warm-up + quick reaction drill (coach points, players shuffle direction)<\/b><\/p>\n<p><b>0:20-0:45 \u2014 Communication drill: 3-passer serve-receive with mandatory verbal calls (&#8220;Mine!&#8221;) on every rep; coach serves into seams between passers to force communication<\/b><\/p>\n<p><b>0:45-1:15 \u2014 Rotational serve-receive: Full 6-rotation serve-receive, cycling through all six positions so players understand passing responsibilities from every spot on the court<\/b><\/p>\n<p><b>1:15-1:45 \u2014 Competitive serve-receive game: Two teams compete for &#8220;points&#8221; based on pass quality (3 pts = perfect pass, 2 = playable, 1 = out of system); server also earns points for aces or forcing bad passes<\/b><\/p>\n<p><b>1:45-1:55 \u2014 Seam-serving challenge: Servers specifically target seams\/gaps between passers; passers practice communicating to cover<\/b><\/p>\n<p><b>1:55-2:00 \u2014 Cool down + reflection: Ask players what they noticed about communication gaps<\/b><\/p>\n<p>&nbsp;<\/p>\n<p><b>PRACTICE 4: GAME-SPEED INTEGRATION<\/b><\/p>\n<p><b>Focus: Serve-receive under live, game-like pressure<\/b><\/p>\n<p><b>0:00-0:20 \u2014 Warm-up: Standard warm-up, light hitting\/passing combo to raise intensity<\/b><\/p>\n<p><b>0:20-0:45 \u2014 Serve-receive-to-attack: Passers receive serve, setter sets, hitter attacks \u2014 full sequence live, focus on pass quality enabling a real offensive play<\/b><\/p>\n<p><b>0:45-1:15 \u2014 Pressure serving: Servers serve at full game intensity\/variety (jump serves, floaters, targeted zones) while passers receive in full rotation<\/b><\/p>\n<p><b>1:15-1:45 \u2014 Scrimmage with serve-receive scoring emphasis: Play 6v6 games where rallies only &#8220;count&#8221; if the first pass is playable, reinforcing the link between serve-receive and winning points<\/b><\/p>\n<p><b>1:45-1:55 \u2014 Situational reps: Simulate specific game scenarios (down 2 points, must-serve situations) to build composure under pressure<\/b><\/p>\n<p><b>1:55-2:00 \u2014 Cool down + season recap: Discuss progression from Practice 1 to now, identify individual\/team growth areas going forward<\/b><\/p>\n<p>&nbsp;<\/p>\n<h2><b>Framework Results<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">See my framework results here: <\/span><a href=\"https:\/\/docs.google.com\/spreadsheets\/d\/1sCPkExYFqiDoN3vds-ozRLcR9dOYHlug6xyspMU90lc\/edit?usp=sharing\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">https:\/\/docs.google.com\/spreadsheets\/d\/1sCPkExYFqiDoN3vds-ozRLcR9dOYHlug6xyspMU90lc\/edit?usp=sharing<\/span><\/a><\/p>\n<h2><b>Reflection<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">Overall, Claude did a fair job of generating serve-receive drills, but the output would need a lot of improvements before I would use it with my team. The practice plans were well-organized and logically sequenced, moving from basic mechanics to game-like pressure over four practice sessions. Some of the drills, however, lacked specificity for what I would actually need. The time chunks seemed arbitrary, and the skill level was way below where my players are at. A few of the coaching principles I follow are not having athletes stand in long lines waiting for their turn and making drills as game-like as possible. I would imagine with a few more prompts, I could get the drills to morph into something that is more aligned with what I am looking for. There were no coaching tips, cues, or benchmarks for what good looks like for the drills. There were also no explanations of how many athletes are needed for the drill, what equipment was needed, the flow of the drills, and no diagrams to help illustrate ball flow.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Comparing Claude\u2019s output to my evaluation framework confirmed what Holmes (2025) said about AI doesn\u2019t save much time, just displaces it. I spent nearly as long reviewing it, adjusting, and fact-checking drills as I likely would have spent designing myself, or finding from a trusted source online. The output also requires edits to match my own coaching voice and coaching philosophy.\u00a0 Bucher (2025) argues that GenAI harms are mundane meaning that the risk of me using AI wasn\u2019t going to be catastrophic, rather it would have a more subtle effect of having generic content that would lower the quality of our team\u2019s practices if I had not questioned the quality of the output.\u00a0<\/span><\/p>\n<p><b>Connection to course readings<\/b><\/p>\n<p><span style=\"font-weight: 400;\">There are many connections that can be made to the course readings. The UBC Guidelines on AI (n.d.) provided the foundation for my evaluation framework, particularly its emphasis on human oversight, equity, and sustainability. Holmes (Royal Irish Academy, 2025) argues that AI doesn&#8217;t actually save teachers time, rather it simply shifts time from one task to another. This held true in my test: even though Claude could generate practice plans quickly, I still had to spend time checking them for accuracy, appropriateness, and safety. Holmes (Royal Irish Academy, 2025) also points out that GenAI output can look credible without actually being accurate. This is why human review is not optional in my framework. It is the only way to confirm the drills are technically sound and consistent with my coaching philosophy. Winterson (2021) discusses how women&#8217;s contributions to technology have often been erased from mainstream narratives about AI. This connects to my Ethics &amp; Bias Check criterion: the gendered labor behind tech innovation is a reminder that human intervention is needed to catch any biased assumptions Claude&#8217;s language might make about who plays volleyball or how bodies move. Bucher (2025) argues that AI&#8217;s harms tend to be mundane rather than dramatic. This reframed my expectations: Claude wasn&#8217;t going to generate a perfect lesson plan, but rather a draft that still needed to be vetted against my own coaching philosophy, expertise, and practicality. Crawford (2021) writes about the difficulty of assessing sustainability in exercises like this. I had no real visibility into how much energy my queries consumed or what went into producing the outputs I received.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In summary, Claude was a useful starting point, not a polished, ready-to-use practice plan, and using the framework was helpful for me to assess the quality of the results.<\/span><\/p>\n<hr \/>\n<p><em><span style=\"font-weight: 400;\">AI Disclaimer: <\/span><a href=\"http:\/\/claude.ai\"><span style=\"font-weight: 400;\">Claude.ai<\/span><\/a><span style=\"font-weight: 400;\"> was used to generate the results for my prompt. Claude was used to revise my work for clarity and grammar. The framework and all final edits are my own.\u00a0<\/span><\/em><\/p>\n<hr \/>\n<p><span style=\"font-weight: 400;\">REFERENCES<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Anthropic. (2025). <\/span><i><span style=\"font-weight: 400;\">Claude<\/span><\/i><span style=\"font-weight: 400;\"> [Large language model]. <\/span><a href=\"https:\/\/claude.ai\"><span style=\"font-weight: 400;\">https:\/\/claude.ai<\/span><\/a><span style=\"font-weight: 400;\">\u00a0\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Bucher, T. (2025). Beyond the hype: Reframing AI through algorithms and culture. <\/span><i><span style=\"font-weight: 400;\">Journal of Communication, 75<\/span><\/i><span style=\"font-weight: 400;\">(1), 81-84.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Crawford, K. (2021). <\/span><i><span style=\"font-weight: 400;\">Atlas of AI: Power, Politics and the Planetary Costs of Artificial Intelligence<\/span><\/i><span style=\"font-weight: 400;\">. Yale University Press.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Royal Irish Academy. (2025, Oct. 2). <\/span><i><span style=\"font-weight: 400;\">The future of AI and education: Keynote: Critical studies of artificial intelligence and education<\/span><\/i><span style=\"font-weight: 400;\"> [Video]. YouTube. <\/span><a href=\"https:\/\/www.youtube.com\/watch?v=aHbM3L3M8KE\"><span style=\"font-weight: 400;\">https:\/\/www.youtube.com\/watch?v=aHbM3L3M8KE<\/span><\/a><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Suchman, L. (2023). The uncontroversial \u2018thingness\u2019 of AI. <\/span><i><span style=\"font-weight: 400;\">Big Data &amp; Society, 10<\/span><\/i><span style=\"font-weight: 400;\">(2) <\/span><a href=\"https:\/\/doi.org\/10.1177\/20539517231206794\"><span style=\"font-weight: 400;\">https:\/\/doi.org\/10.1177\/20539517231206794<\/span><\/a><\/p>\n<p><span style=\"font-weight: 400;\">The University of British Columbia. (n.d.). <\/span><i><span style=\"font-weight: 400;\">Guidelines for all uses of GenAI in teaching &amp; learning<\/span><\/i><span style=\"font-weight: 400;\">. <\/span><a href=\"https:\/\/genai.ubc.ca\/guidance\/teaching-learning-guidelines\/guidelines-for-all-uses-of-genai-in-teaching-learning\/\"><span style=\"font-weight: 400;\">https:\/\/genai.ubc.ca\/guidance\/teaching-learning-guidelines\/guidelines-for-all-uses-of-genai-in-teaching-learning\/<\/span><\/a><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The University of British Columbia. (n.d.). <\/span><i><span style=\"font-weight: 400;\">Teaching &amp; Learning Guidelines.\u00a0<\/span><\/i><a href=\"https:\/\/genai.ubc.ca\/guidance\/teaching-learning-guidelines\/\"><span style=\"font-weight: 400;\">https:\/\/genai.ubc.ca\/guidance\/teaching-learning-guidelines\/<\/span><\/a><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Winterson, J. (2021). <\/span><i><span style=\"font-weight: 400;\">12 Bytes: How Artificial Intelligence will change the way we live and love<\/span><\/i><span style=\"font-weight: 400;\">. Vintage. Chapters 1, 2, and 3.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Using AI to Generate Teaching a Serve Receive Progression to High School Volleyball Players Overview As a high school volleyball coach, I spend a lot of time deciding what my team needs to improve on and then designing drills to target those skills. I carefully consider my athletes\u2019 skill level,&#8230;<\/p>\n<div class=\"more-link-wrapper\"><a class=\"more-link\" href=\"https:\/\/blogs.ubc.ca\/educationaltech\/etec-565t-assignment-2-evaluation-heuristic-and-testing\/\">Continue Reading<span class=\"screen-reader-text\">ETEC 565T &#8211; Assignment #2: Evaluation Heuristic and Testing<\/span><\/a><\/div>\n","protected":false},"author":100070,"featured_media":359,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5,12,17],"tags":[],"class_list":["post-804","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-assignments","category-blog","category-etec565t","entry"],"_links":{"self":[{"href":"https:\/\/blogs.ubc.ca\/educationaltech\/wp-json\/wp\/v2\/posts\/804","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.ubc.ca\/educationaltech\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.ubc.ca\/educationaltech\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.ubc.ca\/educationaltech\/wp-json\/wp\/v2\/users\/100070"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.ubc.ca\/educationaltech\/wp-json\/wp\/v2\/comments?post=804"}],"version-history":[{"count":3,"href":"https:\/\/blogs.ubc.ca\/educationaltech\/wp-json\/wp\/v2\/posts\/804\/revisions"}],"predecessor-version":[{"id":807,"href":"https:\/\/blogs.ubc.ca\/educationaltech\/wp-json\/wp\/v2\/posts\/804\/revisions\/807"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.ubc.ca\/educationaltech\/wp-json\/wp\/v2\/media\/359"}],"wp:attachment":[{"href":"https:\/\/blogs.ubc.ca\/educationaltech\/wp-json\/wp\/v2\/media?parent=804"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.ubc.ca\/educationaltech\/wp-json\/wp\/v2\/categories?post=804"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.ubc.ca\/educationaltech\/wp-json\/wp\/v2\/tags?post=804"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}