Computer Adaptive Assessment –

Here is my venture pitch for a real company (Castle Rock Research Corp) and one of their most recent projects.  This is the “what I would do if i was running the company” proposals for the Computer Adaptive Assessment system that was recently developed and piloted in Alberta.  The executive summary is a MS Publisher document converted to PDF and the Pitch is a presentation in Google Docs.  It was to have a link to a 10 minute video pitch but I have been away from home for the past week and much to my frustration unable to get my tablet to do what I need it to do.  I’ll be home on the 25th so I will add the video to the presentation and to this post at that time.

Executive Summary

Venture Pitch Presentation

 Venture Pitch Video Part 1

Venture Pitch Video Part 2

16 comments


1 Sarah Wood { 11.26.08 at 3:18 pm }

Bryan,

Your venture looks exciting and your executive summary laid the foundation very well. I didn’t fully understand what was done with these “assessments” and perhaps that is my hesitation. I do know that “testing” is a growing market and I am happy to see an Alberta company on board.

Thanks,
Sarah


2 Carolann Fraenkel { 11.26.08 at 6:18 pm }

Hi Bryan,

Your slide show had a nice presentation. (there were some formatting things that were distracting though) The exectutive summary text is difficult to read over the background, and the summary does not give me an idea of what the product is and how it addresses the problem. You never said what the 3 million dollars was going to fund other than expansion. Is this product development? sales and marketing? Distribution? What is the sales/marketing plan, it is not explicitly mentioned.

Carolann


3 Joe Dobson { 11.26.08 at 10:22 pm }

Hi Brian,

This is the first that I’ve heard of Castle Rock. Computer adaptive testing is a real dynamic field and there is a lot of potential as it’s the only player on the ground, but you need to explain clearly what it means (we use accuplacer here and its remarkable how reliable it is). In the pitch you argue that it would replace current ineffective tools – why are they ineffective? It would be useful to explain further. When you discuss the risks, it would be stronger to rebut them and turn them into strengths somehow. As Carolann mentions, it would be useful to identify roughly how the $3m would be used, and specifically which subject areas are applicable.

With some fine-tuning and added detail this could make a lot of sense to EVAs.

Cheers,
Joe


4 Bryan Funk { 11.27.08 at 12:32 am }

Thanks for the feedback. Now that I am home to my computer I will post my Venture Pitch Video and I think it addresses some of the questions as I took time to elaborate a little more that I did in the slideshow. If you would take the time to review it that would be great. I understand that you have 9 others to evaluate so I’ll understand if you don;t get that chance.

There are already some things I could add after reading everyone’s comments so far.

Thanks
Bryan


5 Bryan Funk { 11.27.08 at 12:33 am }

I’ll re-post it as Venture Presentation Video included when I have it up and running.

Bryan


6 Ellen Wu { 11.27.08 at 11:25 pm }

Hi Bryan,

There is a trend of making all provincial exams computerized, as I heard from the science teachers in my school. So, there is definitely a need for an assessment tool like the one you have outlined. What about competition from other software companies? I know that when I took the TOEFL test 8 years ago, the computerized test already did what you’re proposing. The program picked the next question according to the answer of the previous question. So it seems like there’s already similar software on the market. This may be of consideration.

Ellen


7 Bryan Funk { 11.28.08 at 12:16 am }

You are correct, there has been software designed and used successfully for adaptive testing and continues to be used. The unique selling point of CAA is that not only is it on the computer but it is online so students/teachers can access the assessments anywhere they have internet access and that the assessents are created based on the specific learning outcomes of the provincial/state curriculum. When an assessement is scored the specific learning outcome that was achieved or was not achieved is listed underneath the question and gives immediate feedback to teh teacher and student as to the outcomes that they need to work on and which they have mastered.

Bryan


8 Joey Dabell { 11.28.08 at 9:59 am }

Good summary and pitch, and interesting venture! I like the idea of adaptive assessment. Some of our instructors have requisitioned some small custom programs from us that do targeted adaptive assessment. I think yours is an important project. My main concern was (and I think already expressed by others) that while asking for $3mil. funding on the basis of the Alberta costs may be completely valid, as an EVA I would like to see some breakdown and stronger evidence to support this. Would a bit of a comparison between enrolment numbers, or number of schools the original Alberta case and your proposed BC/Ontario pitch maybe provide some scope and help to support the request? I also wondered if giving a brief example scenario from an existing adaptive test in your pitch might help clarify for people who hadn’t heard about it before. Well done!


9 D.B. { 11.29.08 at 12:06 am }

Bryan: I am unable to view the videos for the presentation since YouTube says that they are ‘private videos’. 🙁


10 nancy castonguay { 11.29.08 at 9:28 am }

Bryan, I am not able to access your vids. When I click on the link I get prompter to enter a user/pass.


11 Melissa Anders { 11.29.08 at 5:35 pm }

Hi Bryan,
As others have mentioned, I your videos are coming up as private and so I can’t see them, and for some reason I am having a problem opening your executive summary. I wasn’t completely able to follow the presentation without any other explanation so I was a bit confused about the details. In terms of the 3 million investment needed, how did you get to this number and where will the money be used?

Hopefully I’ll be able to comment more once I am able to see the video.

Melissa


12 Bryan Funk { 11.29.08 at 9:58 pm }

Sorry Everyone. I forgot to make it public. I think I have rectified the problem if not let me know ASAP and I’ll get it up and running.

Bryan


13 nancy castonguay { 11.29.08 at 9:58 pm }

This is a fantastic idea for formative assessment… I’m just not sure how it would work in the case of summative assessment? In any case, it seems that your company has a developed a tool highly in demand because exam-type summative assessment tools appear to be problematic in general in that they often do not seem to reflect other forms of evaluation for the same student for example. Looks like some in-depth research was done with respect to both targeted market and funding relevant to this market. The idea of adaptive assessment, in itself is innovative and in my opinion, probably one of the best directions to take for assessment technology. For online and face-to-face, this is a very feasible proposition, if it can increase student achievement. Testing is controversial, especially for online courses. I am just wondering if there are safety features that would in fact make the testing less at risk of academic dishonesty? Your course of action appears to have been carefully drawn and shows that you have thought of all the major hurdles that could impede the project’s long term goals. I agree that penetrating this market at the earliest stage of the technology being available is key here. I also wonder if a partnership, say with a company like Smart Technologies, might not be more advantageous given that they already have a strong brand and similar technology available? Your overall presentation was very clean-cut and to the point. I wasn’t able to view the videos, unfortunately due the fact that they are private. Overall, I think this has real potential but it’s difficult to envision the product without more information as to how the tool works? Is this similar to LXR, where a common databank is shared on a common topic? Are the adaptations generated by the instructors? Is this assessment tool aimed at reading and math skills only? With more information on the product itself, I think I would definitely approve funding.


14 Bryan Funk { 11.29.08 at 10:14 pm }

Thanks Nancy. I think you may be able to view the video now as I have made it public. The video I hope addresses some of your comments.

On the topic of the video you will notice the lighting is a little dark and I am speaking relatively quietly. The last week before the assignment was due I was staying out of town with my 6 month old (who was born 15 weeks early) as he had a barrage of medical appointments scheduled over 6 days. I did my taping when he was sleeping in the bed beside my desk of course in the dimly lit hotel room. I had also just had Lasik eye surgery done 30 hours prior so you’ll notice that I had my laptop on my lap so that I would be able to read the notes I had prepared and I’m doing a little squinting. (On those two items everything turned out excellent, my son is doing fantastic and I have perfect vision for the first time since I was 6 years old…WOW)

As far as the content in the video, it is as complete as I had planned and I am not making an excuse for anything that may be missing in my pitch.
My only concern is with the quality of the video image and the lack of eye contact that is clearly evident despite my best efforts to look up towards the light.

Please feel free to evaluate the content of the pitch and your feedback is greatly anticipated.

Bryan


15 D.B. { 11.29.08 at 10:52 pm }

Now that I’ve viewed the video and seen it in conjunction with your executive pitch and your slide show, I can see how well thought out the project is.

[My hat is off to you for doing a video presentation under such a challenging circumstance too! Wow! {congrats on the good news for you and your son as well!} ]

Your Executive Summary is good and summarizes the key points. I would suggest using a different background colour/pattern though since the background distracts a bit from the structure and organization of the document.

Your Market Analysis and points are also well organized and you provide a systematic and thorough analysis of the opportunities (especially south of the border with the No Child Left Behind Act). The content of your presentation and the slide show do make for a compelling argument on how to make some money from this exercise and also advance the cause of student education at the same time.

Would I as an EVA support funding this project? Sure, but not unconditionally. I would want to see more information in ways of samples of how Adaptive Testing works and case-studies/previews of how Adaptive Testing is used in helping kids achieve better results.


16 fetya { 11.30.08 at 2:36 am }

Hi Bryan,

I like how you’ve suggested the next plan of action for Alberta CAA, which has already been piloted for three years.

I personally would’ve liked the list of potential clients/buyers to be unveiled earlier as I was a little confused as to how this product would be used until it was finally detailed.

I particularly appreciated the discussed distribution model and the table outlining the competition in the presentation. It would have been really efficient if the slideshow was coordinated with your video presentation or if there were other visual aids throughout the video. You might have been able to focus more on that if you had easier circumstances throughout the week (btw, congratulations on both victories).

Another issue that has been mentioned above is the emphasis on Risks would no clear subsequent reassurances.

Overall I think the product is great and making it accessible could be a profitable venture.

To be realistic though, as a “busy” EVA (role-playing here!), I don’t feel 12 minutes of undivided attention (video VS presentation) gave me the information I need to support this $3M investment.

You must log in to post a comment.