Assessment Rubrics

Here are rubrics for the various graded components of the course.

Two items to highlight:

  • Absences count rapidly against the participation rubrics and we reserve the right to significantly reduce your other grades (maybe to failing) if you have frequent absences. (What’s frequent? We’re not committing to that, but we plan not to apply these rules unless you qualify for a “0” in either participation grade. Take a look at the participation sections below for what that means.)
  • You can learn more about late submissions in the syllabus.

1 Individual Assignments Rubric (Demo-Based)

Submitted and graded individually.

Before each assignment demo session, the course staff (privately) constructs a set of ~4–6 standard questions for TAs to probe students’ knowledge. TAs will likely use only 1–3 of these with any given student, with leeway to open follow-up questions.

Functionality; we expect that well-crafted solutions will receive a 3/3:

3:
Meets all requirements of the assignment specification, possibly with minor glitches or problems in tricky cases. (See the “demonstrated learning” rubric below.)
2:
Meets most requirements of the assignment specification and shows substantial progress toward meeting all requirements.
1:
Clear effort toward meeting the requirements of the assignment specification.
0:
No substantive effort toward meeting the requirements of the assignment specification.

Demonstrated learning; we expect that a well-articulated, clear demonstration will earn a 2.5/3, and occasional exceptional demonstrations will earn a 3/3:

3:
Shares insight into the tools/skills used for the assignment—particularly where difficulties or problems arose in meeting the specification—and how to integrate the tool/skill with the student’s existing knowledge
2.5:
Clearly articulates how the tools/skills used for the assignment were used to meet the specification, with no difficulties explaining code or accounting for design decisions.
1.5:
Articulates how the tools/skills used for the assignment were used to meet the specification, but has difficulty explaining some sections of code or accounting for some design decisions.
0:
Is unable to articulate how the tools/skills used for the assignment were used to meet the specification.

2 Scrum Reports

Each item is marked on a 0/1/2 scale:

2:
Requirement fully met
1:
Requirement nearly met
0:
Requirement not met

Items:

  • On time delivery. (A 0/2 here results in an overall 0 without review of the subsequent items.)
  • Includes each required item in the scrum report.
  • Demonstrates a bit of meaningful reflection on at least one item in the report. (It’s particularly natural for this to be in “where you might have gotten stuck”.)

Format: Each individual posts their own update, but to the group repo.

Describe (for the project): (1) what you’ve worked on in the previous two weeks, (2) what you’ll be working on in the next two weeks, and (3) where you might have gotten stuck during the last two weeks.

These should be quite short (a sentence for each of the three items above).

3 Slack and Other Productive Participation

Along with Slack, the course staff will consider other contributions. (Students can flag non-Slack contributions to us on Slack if they’d like!)

3:
Regular, sustained, and outstanding contributions to improving the learning, progress, or success of the class as a whole.

Must meet the requirement for 2.5 and further stand out to the course staff as an outstanding contributor.

2.5:
Regular, sustained, and meaningful contributions to improving the learning, progress, or success of the class as a whole.

Necessary (but may not be sufficient) requirement: Course staff citation for at least 6 positive contributions over the term and at multiple times over the term, i.e., not clustered in a single module.

2:
Significant contributions to improving the learning, progress, or success of the class as a whole during at least two stages of the course.
1:
Significant contributions to improving the learning, progress, or success of the class as a whole during one stage of the course.
0:
No significant contributions to improving the learning, progress, or success of the class as a whole.

4 Lab participation (attendance)

Only for first lab of each unit.

2 points for each lab (~6 labs), recorded via attendance taken by 1 TA. 10 minute grace period, after which we simply record everyone else at the hour break as late. (2 points for attending; 1 point for late.)

If a student earns k points out of 2*n possible points, they earn the grade: (k – (n – 2)) / n clipped to the percentage range [0%, 100%].

In other words: You can miss one or be late to two labs with no impact on your mark. However, if you miss (roughly) half the labs, you’ll earn a zero.

Unlike workshops, if you contact in advance and if your team can work with it, you may be able to “make up” your lab participation at the other session’s lab. Please don’t do this except for rare hard conflicts and emergencies, as it will strain our course staff resources!

5 Workshop participation (attendance)

3 points for each workshop (6 workshops), recorded via attendance taken at start. 10 minute grace period, after which we record people as they arrive as late. (3 points for attending; 2 points for late.)

If a student earns k points out of 3*n possible points (should be 18), they earn the grade: (k – (2n – 2)) / n clipped to the percentage range [0%, 100%].

In other words: You can be late to two workshops with no impact on your mark. However, if you miss two of the workshops, you’ll earn (roughly) a zero. If you’re late to all of them, you’ll also earn (roughly) a zero.

We may also deduct a mark if a student failed to complete the unit’s survey or did not include substantive feedback in the survey.

6 Design/Code Reviews (Peer/TA; 2nd half of each workshop)

Your group, a TA, and ~2 other groups will cluster for design/code reviews in the second half of each workshop (except the first workshop!) for a design/code review.

YOUR TEAM is responsible for presenting one design element and one small piece of code (no more than 30 lines of code) that you’d like reviewed. Your TA may also prompt you to show some other elements of your design or pieces of code. (Somewhat like assignment demos, the course staff (privately) chooses a set of other elements of your design/code we may want to review.)

The goal is to show that you are making substantial progress in your project and have reflected on challenges and opportunities in your design/code.

Your grade comes in two parts: a team grade for your design/code review, and an individual grade (accrued over the term) for substantive contributions in others’ design/code reviews.

Requirement fulfillment; we expect that well-crafted designs/reviews will receive a 2/2:

2:
Shows substantial progress toward meeting a requirement for the project for the current unit or meets an older requirement by addressing a problem found in previous units’ work. The team should ensure they make clear what requirement they’re addressing!
1:
Clear effort toward meeting some stated requirement, although the team is unable to clearly articulate how they’ve made progress.
0:
No clear and meaningful effort toward meeting a requirement.

Demonstrated learning; we expect that a well-articulated, clear demonstration will earn a 1.5/2, and some outstanding demonstrations will earn a 2/2:

2:
As for 1.5, but outstandingly productive.
1.5:
Clearly articulates one of the following, connecting it to a project requirement for the unit: a crucial challenge amenable to feedback in the design/code review, a substantial opportunity to improve the team’s code/design that the team may be able to act on for the upcoming unit, or a particularly useful solution to such a crucial challenge that the team solved. (For the first two, the discussion should allow meaningful feedback from the peer reviewers. For the last, it should allow meaningful learning for and perhaps adoption by the peer reviewers.)
0.5:
Attempts to show a plausible challenge, opportunity, or useful solution, but is unable to either to communicate it to the group or connect it to the project requirements for the unit (or both).
0:
No meaningful attempt to prepare a plausible challenge, opportunity, or useful solution.

Responds well to TA questions; we expect that a well-informed team will earn a 2/2:

2:
Identifies and presents relevant sections of code or design in response to all TA questions. Able to discuss challenges, opportunities, or solutions in those sections.
1:
Identifies and presents relevant sections of code or design in response to some TA questions.
0:
Unable to identify or present relevant sections of code or design in response to any TA questions.

The TA will also record whether each person performing the review gave some meaningful and substantive piece of feedback or asked a substantive question. Full credit for this component is available over the term even if you have no substantive contribution in two of the available peer reviews.

(You will generally be doing two peer reviews of two different groups for each unit; so, you have two or more opportunities each workshop to contribute. If there are more groups in your cluster, you have more opportunities, which is balanced by somewhat more time pressure.)

7 Final Project Presentation

Presented and graded as a group.

Notes:

  • Conference-style with a time-slot for each group
  • Multiple audiences will be performing assessment: industry guests and course staff. Clearly, the course staff will use the rubric. We will use a simpler rubric for industry guests.When you’re being evaluated with a rubric, you’ll want to give an explicit elevator pitch targeted at the rubric. Otherwise, you’re free to wow your visitors with a shorter elevator pitch and whatever else seems appropriate!

7.1 Final Project Presentation Instructions

We expect the course staff reviewing you to spend at most 12 minutes on your project. You will have max. 7 minutes to present, and 5 minutes for answering questions. During that time, you should illustrate to them how you have completed all the project requirements.

You will not be penalized for going under this time limit, but try to hit all the points, and answer them well. If you go over 7 minutes, we will likely need to cut you off and move to asking questions! You may be lightly penalized for having gone over, but more importantly, you will be penalized for the points you were unable to address.

Have every group member be ready to take a significant role presenting. Course staff may ask questions of any group member and expect articulate, well-justified responses. The team member can hand off the question in a meaningful way to another team member (e.g., “I can tell you this much … however, my team member X focused on that element of our design; X, what can you add?” If you have compelling reasons for one member not to present, contact us NOW!

7.2 Final Project Presentation Structure and Rubric

Note that we expect a solid presentation for a solid project that completed the course requirements but did not go “above and beyond” to earn a 30/36. (Specifically, the hypothetical solid presentation would earn less than full marks on the rubric items about a “super-cool” element and response to questions.)

Project Demo Rubric

  1. Demo your application, walking through at least one or two use cases where you can explain what the user wants to do and how they accomplish it. /10

10:
Demo shows a complete application (which must fulfill the course requirements) fulfilling the needs of at least two compelling use cases. Demo is smooth, understandable, and engaging.
8:
Demo shows how the completed work makes significant progress toward the intended application (which must fulfill the course requirements), fulfilling the needs of at least one compelling use case. Demo is smooth and efficient.
7:
As above, hampered by technical difficulties during the demo but with smooth workarounds (like screenshots prepared in advance).
4:
Demo illustrates some elements relevant to the intended application (which must plausibly be able to fulfill the course requirements).
0:
Demo fails to illustrate progress toward the intended application or could not plausibly fulfill the course requirements.

  1. Explain something super-cool about your design. Note: Obviously, this is entirely subjective, but you should strive to show some element where you went above and beyond, which could be in functionality, research, design, learning, debugging, documentation, or some other area. Feel free to show code here, as long as it is effective in demonstrating your approach. /8

8:
Outstanding element shown with a clear and insightful explanation of what the team accomplished. Should be well above and beyond basic course requirements and build on multiple elements of the course.
6:
Genuinely super-cool element shown with an explanation as to why it is super-cool, and/or the difficulty that had to be overcome to produce it. Must be clearly above and beyond basic course requirements.
4:
Interesting element that goes beyond basic course requirements shown with a good explanation, although it may not qualify as super-cool.
2:
Super-cool element is super-minimal. It is perhaps interesting, but does not go beyond the course requirements, or explanation shows a lack of understanding.
0:
Point is not addressed, or it is not clear that there is anything super-cool.

  1. Bring up one or two interesting challenges you ran into and how you solved them. These may be technical, design, organizational, or other challenges, but should clearly connect to the success of your app. /8

8:
At least one clearly described challenge is discussed, with a story that makes it clear why the challenge was significant, what caused the challenge, the team’s solution process, their resolution of the challenge, why that resolution is satisfactory, and what will come next (if anything) in continuing to address the challenge. ​Audience gains insight into what it took to build the app, and the types of challenges that were overcome.
6:
At least one clearly described challenge is discussed, with a story that makes it clear why the challenge was significant, what caused the challenge, the team’s solution process, their resolution of the challenge, and why that resolution is satisfactory.
4:
A challenge is provided. The explanation addresses most but not all elements of the item above (e.g., leaves the audience without a clear idea of how the challenge was resolved or why the resolution is satisfactory).
2:
A superficial challenge is provided that was not significant in building the app. Few or none of the elements from level-8 are addressed.
0:
No challenges are provided, and/or no explanation is provided.

  1. Respond to our questions effectively, demonstrating clear understanding of your design and of the technologies used this term. Team response as a whole. /5

5:
Responses demonstrate genuine and outstanding insight into the technologies studied this term and their relationship to the app, in the context of the questions.
4:
Responses demonstrate a clear understanding of the app in the context of the questions.
3:
Responses generally demonstrate understanding of the app in the context of the question but with occasionally confused or incomplete elements to the answers.
2:
Responses generally demonstrate understanding of the app but not a substantive response to the question.
0:
Responses do not demonstrate understanding of the app.

  1. Team member participation. /5

5:
Every team member makes substantive contributions to the discussion.
3:
Multiple team members make substantive contributions to the discussion.
0:
At most one team member makes substantive contributions to the discussion.

Total: /36

8 Final Project Submission

Submitted and graded as a group.

Final Project Report

To be included in your project README.

Please document the items in the order below, starting at the top of your README. If you wish to include further materials (ex. Prototypes), put them below the items listed here.

  1. Top of README – clear description of app (can be like an elevator pitch!) [2-3 sentences]
  2. Clear statement of goals (minimal, standard, stretch) and some indication of what was completed [It is enough to simply put a green check icon or red ‘x’ (or yellow ‘!’) next to each goal.]
  3. Description on how tech from Units 1-5 are used in the project. See rubric item #2 for a sense of what we’re looking for in this description. [2-3 sentences each]
  4. Description of ‘Above and Beyond’ functionality. Please give a clear description and in-depth explanation of how you went above and beyond the requirements of the course. This will help us awards marks for rubric item #4.
  5. Description of Next Steps. What would you do next to further improve the app, or add additional relevant functionality? You may want to reference your in-progress or incomplete goals in this section. [2-3 sentences]
  6. List of contributions. Highlight areas where each team member contributed significantly. [2-3 sentences per team member]

 

Final Project Rubric

For evaluating the code, the documentation, and the app itself.

  1. Code cleanliness [ /4]

4:
Any remaining comments are tidied up. Code structure is logical and indenting is consistent. Code is production-ready.
2:
Some code is commented out, or project structure is illogical, or code is messy in general.
0:
Large swaths of code are commented out in the production branch. Node modules or other dependencies are NOT in the .gitignore. (and are pushed to the repo)

  1. Utilizes tech from Unit 1-5 [ /25]
    [5 pts per Unit]
    [Note: If 2 or more “main” technologies from the class are not used, you will earn a 0/25 for this section. Ex. Not using MongoDB, and site is not deployed.]

5:
Usage of tech includes best practices. Code is clean and clear. Description of usage explains in-depth how the technology has made the app better. Possibly a mention of how it compares to other similar tech. Documentation demonstrates a solid understanding of the tech learned throughout the term, and its purpose in creating a production-level full-stack web application.
2.5:
Usage of tech is good, but simplistic and some minor mistakes, or lack of “best practices”. Ex. Not using state to manage UI data, or no error handling in APIs.
1:
Tech is barely used, or used extremely poorly.
0:
No meaningful effort to use tech.

  1. Polished Usability [ /6]

6:
App has no, or extremely minimal bugs. Any unfinished functionality is wrapped up (ex. a “coming soon” page). Site is usable, with no major UX flaws (ex. red text on green background.)
4:
App has a few bugs that we run into as we try to use it normally. It detracts a little, but overall is still a well polished application.
2:
App is buggy, and it is easy to find ways to break it (ex. by entering an invalid date), or the user interface is confusing, and difficult to understand.
0:
App is buggy or ill-designed, and it is difficult or impossible to use the app for its intended purpose. It is easy to find ways to break it (ex. by entering an invalid date)

  1. Above and Beyond Functionality [ /9]
    [Be sure to include this in your Documentation.]

9:
This project is a show-stopper and goes beyond the complexity of other projects. Would have expected this to be produced by a small start-up. We expect few or no projects to earn this rubric item in any given year. It’s here for truly exceptional work communicated effectively to the grading team.
6:
This project goes well above and beyond the requirements, including multiple points from level-3 of this rubric, or one particularly complex piece of technology.
3:
Project goes beyond the basic requirements by incorporating one to a few “extra” requirements. Some examples could be: Fully responsive, fully accessible, uses external APIs, implements a complex algorithm, utilizes ML/AI, did research for UX, supports multiple languages and/or timezones, uses location services, integrates with social media.
0:
Project is very similar or only slightly more advanced than the posting messages assignments.

  1. Description of Next Steps [ /3]

3:
Documentation clearly describes specific, relevant goals that would continue to improve upon the functionality or usability of the app. It is clear how this would be incorporated into the existing app.
1.5:
Documentation describes some further goals, but they do not seem concrete, or particularly relevant.
0:
No future goals stated.

  1. List of contributions [ /3]

3:
It is very clear which team member worked on which parts of the application. The documentation describes at least one area/issue/technology where each team member took a substantial leadership role, and that team member’s contribution is reflected in the project.
1.5:
Some contributions are listed, but they are not very substantial, or it is unclear who worked on some major functional aspects of the project.
0:
Not listed.

Total:  /50

 

9 Intra-Team Peer Evaluations

We will release a peer evaluation survey in which you assess your project team members’ work. One survey will be completed mid-term. (This is required but not graded. Failure to submit will negatively impact participation marks.) The other will be graded and will be completed at the end of the term.

In the survey, you rate yourself and your team members according to whether you met expectations with a brief justification.

A few notes on how we review these evaluations:

  • We look first at median ratings on the numerical question below before adjusting our assessment based on open-ended responses and our own knowledge of teams’ work.
  • We anticipate that a student who is generally rated as “met expectations” with reasonable justification will get a strong peer assessment grade (in the A to A+ range and perhaps even 100%) and the peer assessment will have no impact on other graded elements of the course.
  • We always expect your justifications to actually back up the rating you provide. However, if your rating indicates the team member fell short or went above and beyond, we’ll place additional onus on the explanation to justify that statement.
  • The assessments are private in that only the course staff (instructors and TAs but not guest lecturers or any students) will see them. Your team would be wise however to use the midterm peer evaluations as an opportunity to acknowledge the hard work everyone is doing and discuss how you can be more effective as a team.

9.1 Intra-Team Peer Evaluation Survey

We will ask you, for each team member including yourself to respond to these five questions:

  1. Identify the team member you are rating. (We will request name and @ugrad.cs.ubc.ca login ID.)
  2. Briefly describe the major roles that this team member took on (which may be technical, leadership, documentation, communication, or other!).
  3. Rate this team member’s contribution on this scale:
    • This team member exceeded expectations, going above and beyond what the team expected of them.
    • This team member met expectations, doing a solid job of what the team expected of them.
    • This team member fell somewhat short of expectations, but still contributed substantially to the project.
    • This team member fell short of expectations, but still contributed somewhat to the project.
    • This team member made little or no meaningful contribution to the project.
  4. Spend a few sentences explaining your previous rating. Spend a few extra sentences if you gave a rating besides “met expectations”.