Skip to content

ETEC 565T – Assignment #2: Evaluation Heuristic and Testing

Using AI to Generate Teaching a Serve Receive Progression to High School Volleyball Players Overview

As a high school volleyball coach, I spend a lot of time deciding what my team needs to improve on and then designing drills to target those skills. I carefully consider my athletes’ skill level, ensuring the drills are safe, technically sound, and developmentally appropriate. I have increasingly relied on GenAI to help with practice planning, but the challenge is making sure what it creates is appropriate and of high quality. GenAI can be a very useful tool for coaching, but only if the output is carefully critiqued against the same standards I already hold my own coaching to. If the GenAI content is not carefully vetted, I risk wasting precious practice time, disengaging my athletes, or, worse, putting them at risk of injury.

In this assignment, I will create a simple, usable evaluation framework I can follow to judge if the GenAI output is appropriate for my athletes’ needs, focusing on the skill of serve-receive. Serve-receive is one of the most foundational skills in volleyball, since it is the receiving team’s first opportunity to set up a scoring play. 

Procedure

A task GenAI can help me with

I will use Claude to create a serve-receive drill sequence for high school volleyball players that I can use in my own coaching.

Evaluation Framework / Rulebook for judging if GenAI did it well 

I will use my evaluation framework to judge whether the response I get from Claude is actually useful. The goal is to make sure I don’t simply enter a prompt, receive a response, and blindly follow the practice plans, but instead critically evaluate whether the response is good enough to fit my context, match my athletes’ skill level, align with my coaching philosophy, and, most importantly, are safe.

Procedure

First, I developed my own evaluation framework, grounded in course scholarship as well as UBC’s institutional guidelines on GenAI use in teaching and learning. This framework was designed to assess if the AI-generated content was technically accurate and age-appropriate, and also if it is aligned with principles of transparency, equity, sustainability, and human oversight. I then tested the framework by having Claude generate a sample serve-receive progression and evaluated the output against each criterion. 

Core Evaluation Criteria – “what good output looks like”

1. SAFETY

* Is the drill safe for my athletes to perform? Is there a potential for injury or harm?

* Is the progression from simple to complex (warming up large muscle groups, avoiding overuse activities, preventing collisions)

2. TECHNICAL ACCURACY & PRACTICALITY

* Are the skill mechanics correct for serve-receive?

* Is this drill sequence something I can realistically run within the time, space, and equipment I have available?

* Is the practice planning in a logical sequence?

3. AGE & SKILL APPROPRIATENESS

* Will these drills be appropriate for my athletes’ level (not too simple, not too complex)? 

* Are there suggestions or potential for varying the skill levels that match my athletes’ skills?

4. COACH OVERSIGHT & VOICE

* Is this drill something that aligns with my coaching philosophy?

* How much do I need to edit the drills to adapt it to my coaching?

5. SUSTAINABILITY 

* Is this actually saving me time?

* Are the environmental costs justified?

6. ETHICS & BIAS CHECK

* Are the results free from assumptions that are biased or skewed?

* Is the language free from assumptions about gender, athleticism, etc.?

* Can I complete these drills regardless of facilities/equipment/coaching staff I have access to?

EVALUATION

Results will be evaluated using the rating of 0-2

2 = “usable as-is”

1 = “usable with edits”

0 = “not usable”

* If the safety category is “0”, the drill will not be used since safety overrides everything. 

* I decided to not use Transparency/Attribution because this would be for my own coaching resource library, and I wouldn’t be sharing it with others. 

Actually test it: 

Prompt for Clauide.ai: “generate a serve-receive drill sequence for high school athletes – design 4 practices that are each 2 hours long”

RESPONSE FROM CLAUDE.AI:

PRACTICE 1: FOUNDATIONAL PLATFORM & FOOTWORK

Focus: Individual passing mechanics

0:00-0:20 — Warm-up: Dynamic stretching, arm circles, light jogging, partner tosses (no contact, just tracking the ball)

0:20-0:45 — Platform mechanics: Partner passing drill — stationary, focus on platform angle, hand position, and “shoulders to target.” Coach walks the line correcting form.

0:45-1:10 — Footwork progression: Shuffle-and-pass drill — players start 3 feet off their spot, shuffle to the ball on a coach toss, pass to target. Repeat with tosses to both sides.

1:10-1:35 — Serve-receive vs. easy serves: 3-player passing line receives controlled, coach-served balls (underhand or light overhand). Focus purely on platform control, not target accuracy yet.

1:35-1:55 — Reinforcement circuit: Rotate through 3 stations (wall passing, partner passing, coach toss) in small groups to maximize reps

1:55-2:00 — Cool down + water: Static stretching, quick recap of key cues (angle, feet, follow-through)

 

PRACTICE 2: READING THE SERVE & TARGET PASSING

Focus: Serve recognition and passing accuracy to target

0:00-0:20 — Warm-up: Same as Practice 1, plus dynamic footwork ladder

0:20-0:45 — Serve recognition drill: Players pass in pairs; server calls out serve type (float, topspin) before serving so passers learn to anticipate platform adjustments

0:45-1:10 — 3-person passing triangle: Server, passer, target (setter position). Focus on passing to a consistent target zone, not just controlling the ball.

1:10-1:35 — Live serve-receive reps: Full 6-player serve-receive formation (3 passers) receiving live serves from a rotating group of servers. Coach tracks pass quality (in system / out of system).

1:35-1:55 — Target accuracy competition: Passers earn points for passes landing in a marked target zone; rotate passers every 5 reps

1:55-2:00 — Cool down: Stretch, discuss common reading errors from the session

 

PRACTICE 3: COMMUNICATION & TEAM SYSTEM

Focus: Calling the ball, seams, and rotational responsibility

0:00-0:20 — Warm-up: Standard warm-up + quick reaction drill (coach points, players shuffle direction)

0:20-0:45 — Communication drill: 3-passer serve-receive with mandatory verbal calls (“Mine!”) on every rep; coach serves into seams between passers to force communication

0:45-1:15 — Rotational serve-receive: Full 6-rotation serve-receive, cycling through all six positions so players understand passing responsibilities from every spot on the court

1:15-1:45 — Competitive serve-receive game: Two teams compete for “points” based on pass quality (3 pts = perfect pass, 2 = playable, 1 = out of system); server also earns points for aces or forcing bad passes

1:45-1:55 — Seam-serving challenge: Servers specifically target seams/gaps between passers; passers practice communicating to cover

1:55-2:00 — Cool down + reflection: Ask players what they noticed about communication gaps

 

PRACTICE 4: GAME-SPEED INTEGRATION

Focus: Serve-receive under live, game-like pressure

0:00-0:20 — Warm-up: Standard warm-up, light hitting/passing combo to raise intensity

0:20-0:45 — Serve-receive-to-attack: Passers receive serve, setter sets, hitter attacks — full sequence live, focus on pass quality enabling a real offensive play

0:45-1:15 — Pressure serving: Servers serve at full game intensity/variety (jump serves, floaters, targeted zones) while passers receive in full rotation

1:15-1:45 — Scrimmage with serve-receive scoring emphasis: Play 6v6 games where rallies only “count” if the first pass is playable, reinforcing the link between serve-receive and winning points

1:45-1:55 — Situational reps: Simulate specific game scenarios (down 2 points, must-serve situations) to build composure under pressure

1:55-2:00 — Cool down + season recap: Discuss progression from Practice 1 to now, identify individual/team growth areas going forward

 

Framework Results

See my framework results here: https://docs.google.com/spreadsheets/d/1sCPkExYFqiDoN3vds-ozRLcR9dOYHlug6xyspMU90lc/edit?usp=sharing

Reflection

Overall, Claude did a fair job of generating serve-receive drills, but the output would need a lot of improvements before I would use it with my team. The practice plans were well-organized and logically sequenced, moving from basic mechanics to game-like pressure over four practice sessions. Some of the drills, however, lacked specificity for what I would actually need. The time chunks seemed arbitrary, and the skill level was way below where my players are at. A few of the coaching principles I follow are not having athletes stand in long lines waiting for their turn and making drills as game-like as possible. I would imagine with a few more prompts, I could get the drills to morph into something that is more aligned with what I am looking for. There were no coaching tips, cues, or benchmarks for what good looks like for the drills. There were also no explanations of how many athletes are needed for the drill, what equipment was needed, the flow of the drills, and no diagrams to help illustrate ball flow.

Comparing Claude’s output to my evaluation framework confirmed what Holmes (2025) said about AI doesn’t save much time, just displaces it. I spent nearly as long reviewing it, adjusting, and fact-checking drills as I likely would have spent designing myself, or finding from a trusted source online. The output also requires edits to match my own coaching voice and coaching philosophy.  Bucher (2025) argues that GenAI harms are mundane meaning that the risk of me using AI wasn’t going to be catastrophic, rather it would have a more subtle effect of having generic content that would lower the quality of our team’s practices if I had not questioned the quality of the output. 

Connection to course readings

There are many connections that can be made to the course readings. The UBC Guidelines on AI (n.d.) provided the foundation for my evaluation framework, particularly its emphasis on human oversight, equity, and sustainability. Holmes (Royal Irish Academy, 2025) argues that AI doesn’t actually save teachers time, rather it simply shifts time from one task to another. This held true in my test: even though Claude could generate practice plans quickly, I still had to spend time checking them for accuracy, appropriateness, and safety. Holmes (Royal Irish Academy, 2025) also points out that GenAI output can look credible without actually being accurate. This is why human review is not optional in my framework. It is the only way to confirm the drills are technically sound and consistent with my coaching philosophy. Winterson (2021) discusses how women’s contributions to technology have often been erased from mainstream narratives about AI. This connects to my Ethics & Bias Check criterion: the gendered labor behind tech innovation is a reminder that human intervention is needed to catch any biased assumptions Claude’s language might make about who plays volleyball or how bodies move. Bucher (2025) argues that AI’s harms tend to be mundane rather than dramatic. This reframed my expectations: Claude wasn’t going to generate a perfect lesson plan, but rather a draft that still needed to be vetted against my own coaching philosophy, expertise, and practicality. Crawford (2021) writes about the difficulty of assessing sustainability in exercises like this. I had no real visibility into how much energy my queries consumed or what went into producing the outputs I received.

In summary, Claude was a useful starting point, not a polished, ready-to-use practice plan, and using the framework was helpful for me to assess the quality of the results.


AI Disclaimer: Claude.ai was used to generate the results for my prompt. Claude was used to revise my work for clarity and grammar. The framework and all final edits are my own. 


REFERENCES

Anthropic. (2025). Claude [Large language model]. https://claude.ai  

Bucher, T. (2025). Beyond the hype: Reframing AI through algorithms and culture. Journal of Communication, 75(1), 81-84.

Crawford, K. (2021). Atlas of AI: Power, Politics and the Planetary Costs of Artificial Intelligence. Yale University Press. 

Royal Irish Academy. (2025, Oct. 2). The future of AI and education: Keynote: Critical studies of artificial intelligence and education [Video]. YouTube. https://www.youtube.com/watch?v=aHbM3L3M8KE 

Suchman, L. (2023). The uncontroversial ‘thingness’ of AI. Big Data & Society, 10(2) https://doi.org/10.1177/20539517231206794

The University of British Columbia. (n.d.). Guidelines for all uses of GenAI in teaching & learning. https://genai.ubc.ca/guidance/teaching-learning-guidelines/guidelines-for-all-uses-of-genai-in-teaching-learning/ 

The University of British Columbia. (n.d.). Teaching & Learning Guidelines. https://genai.ubc.ca/guidance/teaching-learning-guidelines/ 

Winterson, J. (2021). 12 Bytes: How Artificial Intelligence will change the way we live and love. Vintage. Chapters 1, 2, and 3.

Leave a Reply

Your email address will not be published. Required fields are marked *

Spam prevention powered by Akismet