Here are rubrics for the various graded components of the course.
Two items to highlight:
- Absences count rapidly against the participation rubrics and we reserve the right to significantly reduce your other grades (maybe to failing) if you have frequent absences. (What’s frequent? We’re not committing to that, but we plan not to apply these rules unless you qualify for a “0” in either participation grade. Take a look at the participation sections below for what that means.)
- You can learn more about late submissions in the syllabus.
Table of Contents
- 1. Individual Assignments Rubric (Demo-Based)
- 2. Scrum Reports
- 3. Slack and Other Productive Participation
- 4. Lab participation (attendance)
- 5. Workshop participation (attendance)
- 6. Design/Code Reviews (Peer/TA; 2nd half of each workshop)
- 7. Final Project Presentation
- 8. Final Project Submission
- 9. Intra-Team Peer Evaluations
1 Individual Assignments Rubric (Demo-Based)
Submitted and graded individually.
Before each assignment demo session, the course staff (privately) constructs a set of ~4–6 standard questions for TAs to probe students’ knowledge. TAs will likely use only 1–3 of these with any given student, with leeway to open follow-up questions.
Functionality; we expect that well-crafted solutions will receive a 3/3:
- Meets all requirements of the assignment specification, possibly with minor glitches or problems in tricky cases. (See the “demonstrated learning” rubric below.)
- Meets most requirements of the assignment specification and shows substantial progress toward meeting all requirements.
- Clear effort toward meeting the requirements of the assignment specification.
- No substantive effort toward meeting the requirements of the assignment specification.
Demonstrated learning; we expect that a well-articulated, clear demonstration will earn a 2.5/3, and occasional exceptional demonstrations will earn a 3/3:
- Shares insight into the tools/skills used for the assignment—particularly where difficulties or problems arose in meeting the specification—and how to integrate the tool/skill with the student’s existing knowledge
- Clearly articulates how the tools/skills used for the assignment were used to meet the specification, with no difficulties explaining code or accounting for design decisions.
- Articulates how the tools/skills used for the assignment were used to meet the specification, but has difficulty explaining some sections of code or accounting for some design decisions.
- Is unable to articulate how the tools/skills used for the assignment were used to meet the specification.
2 Scrum Reports
Each item is marked on a 0/1/2 scale:
- Requirement fully met
- Requirement nearly met
- Requirement not met
- On time delivery. (A 0/2 here results in an overall 0 without review of the subsequent items.)
- Includes each required item in the scrum report.
- Demonstrates a bit of meaningful reflection on at least one item in the report. (It’s particularly natural for this to be in “where you might have gotten stuck”.)
Format: Each individual posts their own update, but to the group repo.
Describe (for the project): (1) what you’ve worked on in the previous two weeks, (2) what you’ll be working on in the next two weeks, and (3) where you might have gotten stuck during the last two weeks.
These should be quite short (a sentence for each of the three items above).
3 Slack and Other Productive Participation
Along with Slack, the course staff will consider other contributions. (Students can flag non-Slack contributions to us on Slack if they’d like!)
- Regular, sustained, and outstanding contributions to improving the learning, progress, or success of the class as a whole.
Must meet the requirement for 2.5 and further stand out to the course staff as an outstanding contributor.
- Regular, sustained, and meaningful contributions to improving the learning, progress, or success of the class as a whole.
Necessary (but may not be sufficient) requirement: Course staff citation for at least 6 positive contributions over the term and at multiple times over the term, i.e., not clustered in a single module.
- Significant contributions to improving the learning, progress, or success of the class as a whole during at least two stages of the course.
- Significant contributions to improving the learning, progress, or success of the class as a whole during one stage of the course.
- No significant contributions to improving the learning, progress, or success of the class as a whole.
4 Lab participation (attendance)
Only for first lab of each unit.
2 points for each lab (~6 labs), recorded via attendance taken by 1 TA. 10 minute grace period, after which we simply record everyone else at the hour break as late. (2 points for attending; 1 point for late.)
If a student earns k points out of 2*n possible points, they earn the grade: (k – (n – 2)) / n clipped to the percentage range [0%, 100%].
In other words: You can miss one or be late to two labs with no impact on your mark. However, if you miss (roughly) half the labs, you’ll earn a zero.
Unlike workshops, if you contact in advance and if your team can work with it, you may be able to “make up” your lab participation at the other session’s lab. Please don’t do this except for rare hard conflicts and emergencies, as it will strain our course staff resources!
5 Workshop participation (attendance)
3 points for each workshop (6 workshops), recorded via attendance taken at start. 10 minute grace period, after which we record people as they arrive as late. (3 points for attending; 2 points for late.)
If a student earns k points out of 3*n possible points (should be 18), they earn the grade: (k – (2n – 2)) / n clipped to the percentage range [0%, 100%].
In other words: You can be late to two workshops with no impact on your mark. However, if you miss two of the workshops, you’ll earn (roughly) a zero. If you’re late to all of them, you’ll also earn (roughly) a zero.
We may also deduct a mark if a student failed to complete the unit’s survey or did not include substantive feedback in the survey.
6 Design/Code Reviews (Peer/TA; 2nd half of each workshop)
Your group, a TA, and ~2 other groups will cluster for design/code reviews in the second half of each workshop (except the first workshop!) for a design/code review.
YOUR TEAM is responsible for presenting one design element and one small piece of code (no more than 30 lines of code) that you’d like reviewed. Your TA may also prompt you to show some other elements of your design or pieces of code. (Somewhat like assignment demos, the course staff (privately) chooses a set of other elements of your design/code we may want to review.)
The goal is to show that you are making substantial progress in your project and have reflected on challenges and opportunities in your design/code.
Your grade comes in two parts: a team grade for your design/code review, and an individual grade (accrued over the term) for substantive contributions in others’ design/code reviews.
Requirement fulfillment; we expect that well-crafted designs/reviews will receive a 2/2:
- Shows substantial progress toward meeting a requirement for the project for the current unit or meets an older requirement by addressing a problem found in previous units’ work. The team should ensure they make clear what requirement they’re addressing!
- Clear effort toward meeting some stated requirement, although the team is unable to clearly articulate how they’ve made progress.
- No clear and meaningful effort toward meeting a requirement.
Demonstrated learning; we expect that a well-articulated, clear demonstration will earn a 1.5/2, and some outstanding demonstrations will earn a 2/2:
- As for 1.5, but outstandingly productive.
- Clearly articulates one of the following, connecting it to a project requirement for the unit: a crucial challenge amenable to feedback in the design/code review, a substantial opportunity to improve the team’s code/design that the team may be able to act on for the upcoming unit, or a particularly useful solution to such a crucial challenge that the team solved. (For the first two, the discussion should allow meaningful feedback from the peer reviewers. For the last, it should allow meaningful learning for and perhaps adoption by the peer reviewers.)
- Attempts to show a plausible challenge, opportunity, or useful solution, but is unable to either to communicate it to the group or connect it to the project requirements for the unit (or both).
- No meaningful attempt to prepare a plausible challenge, opportunity, or useful solution.
Responds well to TA questions; we expect that a well-informed team will earn a 2/2:
- Identifies and presents relevant sections of code or design in response to all TA questions. Able to discuss challenges, opportunities, or solutions in those sections.
- Identifies and presents relevant sections of code or design in response to some TA questions.
- Unable to identify or present relevant sections of code or design in response to any TA questions.
The TA will also record whether each person performing the review gave some meaningful and substantive piece of feedback or asked a substantive question. Full credit for this component is available over the term even if you have no substantive contribution in two of the available peer reviews.
(You will generally be doing two peer reviews of two different groups for each unit; so, you have two or more opportunities each workshop to contribute. If there are more groups in your cluster, you have more opportunities, which is balanced by somewhat more time pressure.)
7 Final Project Presentation
Presented and graded as a group.
- Poster-style (with a handful of teams giving an actual presentation).
- Multiple audiences will be performing assessment: peers, industry, staff, audience. Clearly, the course staff will use the rubric. We may want peers to do so as well. We probably want a simpler rubric for audience and industry.When you’re being evaluated with a rubric, you’ll want to give an explicit elevator pitch targeted at the rubric. Otherwise, you’re free to wow your visitors with a shorter elevator pitch and whatever else seems appropriate!
7.1 Final Project Presentation Instructions
We expect the course staff reviewing you can spend at most 8–10 minutes on your poster. During that time, you should illustrate to them how you have completed all the project requirements.
You will not be penalized for going under this time limit, but try to hit all the points, and answer them well. If you go over 10 minutes, we will likely need to cut you off and move to the next poster! You may be lightly penalized for having gone over, but more importantly, you will be penalized for the points you were unable to address.
Have every group member be ready to take a significant role presenting. Course staff may ask questions of any group member and expect articulate, well-justified responses. The team member can hand off the question in a meaningful way to another team member (e.g., “I can tell you this much … however, my team member X focused on that element of our design; X, what can you add?” If you have compelling reasons for one member not to present, contact us NOW!
7.2 Final Project Presentation Structure and Rubric
Note that we expect a solid presentation for a solid project that completed the course requirements but did not go “above and beyond” to earn a 35/40. (Specifically, the hypothetical solid presentation would earn less than full marks on the rubric items about a “super-cool” element and response to questions.)
- Discuss your original goals for the app (minimal + extended), including how those goals evolved over the term and might evolve further in the future, and predict what it would take to complete the app. /4
- Original goals are clearly stated, and it is made clear which goals were achieved, as well as which were changed or removed,and why. What is required to complete the app (which goals remain) is discussed.
- It is unclear to the audience what the goals of the app were, and which ones were completed. It is also unclear what goals they propose for the completion of the app.
- Goals are not addressed in any capacity.
- Demo your application, walking through at least one or two use cases where you can explain what the user wants to do and how they accomplish it. /10
- Demo shows a complete application (which must fulfill the course requirements) fulfilling the needs of at least two compelling use cases.
- Demo shows how the completed work makes significant progress toward the intended application (which must fulfill the course requirements), fulfilling the needs of at least one compelling use case.
- As above, hampered by technical difficulties during the demo but with smooth workarounds (like screenshots prepared in advance).
- Demo illustrates some elements relevant to the intended application (which must plausibly be able to fulfill the course requirements).
- Demo fails to illustrate progress toward the intended application or could not plausibly fulfill the course requirements.
- Explain something super-cool about your design. Note: Obviously, this is entirely subjective, but you should strive to show some element where you went above and beyond, which could be in functionality, research, design, learning, debugging, documentation, or some other area. /6
- Outstanding element shown with a clear and insightful explanation of what the team accomplished. Should be well above and beyond basic course requirements and build on multiple elements of the course.
- Genuinely super-cool element shown with an explanation as to why it is super-cool, and/or the difficulty that had to be overcome to produce it. Must be clearly above and beyond basic course requirements.
- Interesting element that goes beyond basic course requirements shown with a good explanation, although it may not qualify as super-cool.
- Point is not addressed, or it is not clear that there is anything super-cool.
- Bring up one or two interesting problems you ran into and how you solved them. /5
- At least one clearly described issue or problem is discussed, with a story that makes it clear why the issue was significant, what caused the issue, the team’s solution process, their final resolution of the problem, and why that resolution is satisfactory.
- An issue is provided. The explanation addresses most but not all elements of the item above (e.g., leaves the audience without a clear idea of how the problem was resolved or why the resolution is satisfactory).
- No issues are provided, and/or no explanation is provided.
- Walk us through how each of the first five units factored into your design. Feel free to show pieces of your code! /6
- It is clear how appropriate portions of the material from each unit contribute toward the completed work.
- It is clear how appropriate portions of the material from all but one of the units contribute toward the completed work.
- It is not clear how two or more units’ material contributes toward the completed work.
- Respond to our questions effectively, demonstrating clear understanding of your design and of the technologies used this term.Team response as a whole. /5
- Responses demonstrate genuine and outstanding insight into the technologies studied this term and their relationship to the app, in the context of the questions.
- Responses demonstrate a clear understanding of the app in the context of the questions.
- Responses generally demonstrate understanding of the app in the context of the question but with occasionally confused or incomplete elements to the answers.
- Responses generally demonstrate understanding of the app but not a substantive response to the question.
- Responses do not demonstrate understanding of the app.
Team member participation. /4
- Every team member makes substantive contributions to the discussion.
- Multiple team members make substantive contributions to the discussion.
- At most one team member makes substantive contributions to the discussion.
8 Final Project Submission
Submitted and graded as a group.
Each element below is graded on the following rubric:
- Exceptional fulfillment of this rubric element. Goes above and beyond.
- Entirely fulfilled this rubric element.
- With minor concerns, fulfilled this rubric element.
- Showed substantial progress toward fulfilling this rubric element but with significant issues.
- Showed some meaningful progress toward fulfilling this rubric element but with major issues.
- Did not show meaningful progress toward fulfilling this rubric element.
- Code Style: The project is easy to navigate and understand for someone familiar with the technologies but unfamiliar with this particular codebase. It uses appropriate best practices for each language/tool/system employed. (Note: problems on this item may cascade to cause lower marks elsewhere when we cannot find or understand your contributions!)You are welcome and encouraged to provide us a short additional document with links into the project code and other relevant items to help us find our way to supplying all marks in this rubric to you!
- Basic Technology Requirements: The project meaningfully employs each of the five core technologies we learned about in the workshops.(WARNING: One missing technology would be a “major issue”. Multiple would be “no meaningful progress”.)
- Basic Contribution Requirements: The documentation describes at least one area/issue/technology where each team member took a substantial leadership role, and that team member’s contribution is reflected in the project.
- Basic Functionality Requirements: The documentation clearly communicates the problem the project is intended to solve, and the project meaningfully and effectively addresses this problem.Note:
- The “problem” must be consistent with the team’s overall trajectory during the term. We understand that you are likely to “pivot” multiple times during the term, and your final submission may be the result of a pivot. However, it shouldn’t be a completely unrelated direction to where you were coming from!
- “Meaningfully and effectively” does not necessarily mean “completely”. Use your documentation to communicate what aspect of the problem you solved and how!
- Challenges, learning, and future directions: The documentation highlights challenges the group struggled with, how and to what extent they addressed those challenges, what they learned from addressing them, and what steps they would take next to progress further.
- Initiative and additional contributions: The documentation highlights how the team’s contributions go above and beyond simply incorporating each learned technology, and that additional contribution is reflected in the project. This may be in excelling in one or more technologies we did learn, in integrating these into a particularly impactful project, or in integrating with additional technologies the team explored over the term.
9 Intra-Team Peer Evaluations
We will release a peer evaluation survey in which you assess your project team members’ work. One survey will be completed mid-term. (This is required but not graded. Failure to submit will negatively impact participation marks.) The other will be graded and will be completed at the end of the term.
In the survey, you rate yourself and your team members according to whether you met expectations with a brief justification.
A few notes on how we review these evaluations:
- We look first at median ratings on the numerical question below before adjusting our assessment based on open-ended responses and our own knowledge of teams’ work.
- We anticipate that a student who is generally rated as “met expectations” with reasonable justification will get a strong peer assessment grade (in the A to A+ range and perhaps even 100%) and the peer assessment will have no impact on other graded elements of the course.
- We always expect your justifications to actually back up the rating you provide. However, if your rating indicates the team member fell short or went above and beyond, we’ll place additional onus on the explanation to justify that statement.
- The assessments are private in that only the course staff (instructors and TAs but not guest lecturers or any students) will see them. Your team would be wise however to use the midterm peer evaluations as an opportunity to acknowledge the hard work everyone is doing and discuss how you can be more effective as a team.
9.1 Intra-Team Peer Evaluation Survey
We will ask you, for each team member including yourself to respond to these five questions:
- Identify the team member you are rating. (We will request name and @ugrad.cs.ubc.ca login ID.)
- Briefly describe the major roles that this team member took on (which may be technical, leadership, documentation, communication, or other!).
- Rate this team member’s contribution on this scale:
- This team member exceeded expectations, going above and beyond what the team expected of them.
- This team member met expectations, doing a solid job of what the team expected of them.
- This team member fell somewhat short of expectations, but still contributed substantially to the project.
- This team member fell short of expectations, but still contributed somewhat to the project.
- This team member made little or no meaningful contribution to the project.
- Spend a few sentences explaining your previous rating. Spend a few extra sentences if you gave a rating besides “met expectations”.