ID+Project+3

Design Report 3 flat =**Project title**= =Mastering Math Mayhem = = = =Revisions since Report 2= The group developed the //Mastering Math Mayhem// course site at http://masteringmathmayhem.wikispaces.com/ using the objectives and instructional materials described in Report 2. The purpose was to create a working site which could be used, assessed, and modified according to student feedback. This site will remain a working site to be used by current and future students of the MAT 116 course.

=Goals, objectives and task analysis, and Report 2= There was little modification from Report 2 to the development and implementation of the //Mastering Math Mayhem// course. It was decided that since participation in its use was voluntary that it should not be hosted on the school's learning management system (LMS). Therefore, it would be difficult to automatically score the pre- or post-assessments. Downloadable pre- and post-assessments were created and the students were directed to submit them to their instructor.

=Plan for one-to-one formative evaluation= Learners- The student responding to the individual formative evaluation has been out of high school for almost 20 years. She says she has math anxiety and has never taken an algebra course. She is a visual learner and prefers to have someone explain math to her or watch videos. Algebra is one of the last courses she needs to complete her program in Elementary Education. She is a lead cashier at a retail merchandiser and is interested in earning a degree to change jobs. She is interested in a career-track occupation.

Materials- The students were given the opportunity to sign up and use the //Mastering Math Mayhem// website. Twelve students accessed the site, and one student (described above) volunteered to take the survey shown in following section. Survey Monkey was used to collect the data and the survey can be found [|here].

Procedures- Students were sent an individual email stating the purpose of the fraction wiki and asked if they were interested in visiting the website. The interested students replied and were given the website address. According to Morrison, Ross, Kalman and Kemp (2011, p.354), the purpose of a one-to-one formative evaluation is to collect "descriptive information pertaining to the clarity, impact, and feasibility of initial versions of the instruction." A ten-question survey was created following the model presented on page 375 (Morrison et al, 2011) to ascertain the students' initial impression of the fraction wiki. This survey was created to inquire about the ease of site navigation, thoughts about the overall site, thoughts about the instructional materials, difficulties encountered, and their feelings about the practice activities and resources. We also asked about any changes they would recommend regarding the website and instructional materials. The questions were carefully constructed to gather a complete picture of the student's perspective of the wiki. The students were sent the survey link requesting their feedback.

=Results of one-to-one formative evaluation= Once the initial development was complete the survey was completed by a single student. The results were: 1. Could you follow the order of activities on each page? >> **Yes** 2. Was the website easy to navigate? >> **Yes** 3. What did you think of the website? >> **It was very informative and easy to use** 4. What did you think about the instructional resources which includes the videos, practice options, and printable resources? >> **I liked how you could print out activites [sic]** 5. Was there anything difficult to understand? If so, which module or resource did you find difficult and why? >> **Just getting use to navigating around it** 6. Could you relate to the presented material? >> **yes it was all that we are doing now to what was coming** 7. How did you feel about the practice activities and feedback from the tutorials? >> **I find it very helpful, this gives you a great idea on what you need to know.** 8. How did you feel about the resources available on the website (such as videos and examples)? >> **They will be helpful with the upcoming lessons** 9. Are there any instructional changes you would recommend? >> **no** 10. Are there any website changes you would recommend? >> **no**

Outcome:
Based on the response to question #5, a few sentences concerning how to navigate through the course were added to the Home page. =Materials and assessments for small-group evaluation= During the first week more than fifteen individuals students visited the //Mastering Math Mayhem// website located here. Students were asked to answer a pre- and post-assessment that can be found below: Upon completion of the assessments, students were instructed to send their answers to the instructor. Survey Monkey was used to collect data from the students, similar to the one-to-one evaluation, which can be found here.

=Characteristics of small-group learners= The students responding to the fraction website reported that they were out of high school a minimum of 10 years; most of them have been out of school longer. They reported having math anxiety and a fear of taking/passing the course. Their learning preference was using visual materials. The students are in their second year of varying associate programs with most of them needing Algebra as their final course before graduation. Several of the students are repeating this course and expressed their fear of failure the second time around. Overall, most of the students reported that they are comfortable using online resources and are able to navigate websites.

=Instruments for small-group evaluation= According to Morrison, Ross, Kalman, and Kemp, the first step in conducting an evaluation is to determine its purpose (2011). The goal of this small-group evaluation was to gauge "observational, attitudinal, and performance data" (Morrison et al, 2011, p.354). Students were asked to take the same survey about the fraction wiki using Survey Monkey that was administered during the one-to-one formative evaluation. Since our website is in the beginning stages, we decided to inquire about the look of the website, ease of use, instructional materials, and feelings about the site and resources. A web survey was the most practical way to get feedback about our website from our target audience of online learners scattered throughout the country. Students were also directed to complete a pre- and post-assessment found on the front page of the wiki to be submitted to the instructor in an effort to measure any performance changes.

=Procedures for small-group evaluation= The small group evaluation procedure was similar to the individual evaluation. A note was posted to the class providing information about the purpose of the website. Included in the post was the link to the wiki as well as the survey link. Students were also given the opportunity to post responses or comments to this thread. It was also noted that the survey should only be taken one time. Effective online instruction is dependent on a well designed course and it is necessary to continually perform quality assessments and make needed adjustments (Ruhe & Zumbo, 2009). The small-group evaluation is one of the first steps in assessing the quality of the course.

=Summary of small-group evaluation= Three students responded to the survey as follows: 1. Please answer yes or no to the following questions. 2. What did you think of the website? 3. What did you think about the instructional resources which includes the videos, practice options, and printable resources? 4. Was there anything difficult to understand? If so, which module or resource did you find difficult and why? 5. Could you relate to the presented material? 6. How did you feel about the practice activities and feedback from the tutorials? 7. How did you feel about the resources available on the website (such as videos and examples)? 8. Are there any instructional changes you would recommend? 9. Are there any website changes you would recommend?
 * Could you follow the order of activities on each page?
 * **Yes**
 * **Yes**
 * **Yes**
 * Was the website easy to navigate?
 * **Yes**
 * **Yes**
 * **Yes**
 * **It is a really good site that should of been offered before taking the actual algebra class.**
 * **Very very helpful ! Will definately use it again for future reference .**
 * **Very easy to navigate. Clearly shown how to work fraction problems on initial page after clicking on a link. Great for a quick refresher on a process involving fractions.**
 * **They are a great asset.**
 * **The printable resources are awesome for extra practice . The videos are great, very effective and easy to understand**
 * **I think they are more than adequate to teach even someone who does not have a strong math background**
 * **No**
 * **No, every module was easy to comprehend and very well put together**
 * **No. Everything was clear and concise.**
 * **Yes**
 * **Absolutely**
 * **Yes. Can not tell the difference between being in a physical lecture hall as opposed to an online classroom.**
 * **Positive and understandable,**
 * **The practice activities were very effective and gave great practice**
 * **I feel they both have been designed to really help one excel at solving problems involving fractions.**
 * **They are a great asset.**
 * **The examples were very easy to understand and the videos were very helpful . Practice problems were great also**
 * **I like the fact that there are a few resources to choose from as opposed to one. I think repetition helps with learning math concepts. Multiple resources allows one to practice solvings problems from different perspectives.**
 * **No**
 * **No way**
 * **No. I think this is a great tool. I think utilizing this website on weeks without discussion questions will allow one to work on this site without interfering with one's regular study schedule.**
 * **No**
 * **No wAy . Very effective and helpful !**
 * **No. I think it is fine as is.**

All of the students that replied to the group survey reported positive experiences with the fractions wiki. There was no mention of navigation issues, indicating that the changes made between the one-to-one evaluation and the group evaluation rectified any unclear directions. Students found the site effective, helpful, easy to navigate and a resource they could return to in the future when necessary for help with fractions.

The survey results helped us collect observational and attitudinal data. No students opted to complete the pre- and post-assessments as directed. Due to this lack of participation, increased performance cannot be shown.

=Discussion of small group data= According to the small-group evaluation outcome, the fraction wiki has been constructed in a user-friendly manner. Students find value in the material and the manner in which it is presented. The different types of practice problems were useful to all students that completed the survey.

=Revisions for instruction and assessment= Performance was not measured in the one-to-one or group evaluations for this report. An effort was made to encourage students to participate in a pre- and post-assessment but students opted out. In order to increase participation in this aspect for the future, an interactive pre- and post-assessment will be implemented that will be automatically graded on the website. This will store the information for performance data and allow students to receive immediate summative feedback. Comparison of pre- and post-assessment scores will allow students and instructors to easily see any changes in performance.

=Relevant current references=

Morrison, G. R., Ross, S. M., Kalman, H. K., & Kemp, J. E. (2011). //Designing Effective Instruction //, 6th ed. Hoboken, NJ: Wiley.

Ruhe, V. & Zumbo, B. D. (2009). //Evaluation in Distance Eduction and E-Learning//, New York, NY: The Guilford Press.