Use of iPads in the Classroom

PART I:  EVALUATION PLAN  

 

Prior evaluations and their limitations

Prior evaluations have been successful in identifying math anxiety and implementing programs to help raise test scores; however, very little research has been done to evaluate the use of new technology such as iPads to aid in the teaching process. Many tools are used to decrease math anxiety and increase test scores in elementary students; however, with new technologies becoming less expensive classrooms could include these computer-based programs to breathe new life into the learning experience. First, we must study and evaluate their usefulness as a learning tool before we override those activities that have a proven track record for increasing interest in learning activities.

In the article, “New Study Finds iPads in the Classroom Boost Test Scores”, the author Courtney Subramanian reports that in Auburn, Maine 266 kindergarteners were instructed using the iPad and those who used the device scored higher on literacy tests and were more enthused about learning (2012). Phil Schiller, Apples SVP of Worldwide Marketing states, “Education is deep in Apple’s DNA and iPad may be our most exciting and education [al] product yet” (Subramanian, 2012). Many schools are starting to use iPad technology in the classroom to improve reading skills; however, there is little evidence to support iPads success in improving elementary student’s math skills thereby reducing math anxiety and consequently improving test scores.

Another study of the use of iPads for mastery of geometry in a 7^th grade classroom proved successful. Most students had a positive reaction to the implementation of the iPads based on an open-ended questionnaire and post-test evaluation. Student test scores in geometry improved, students begin to engage more in the learning process, and the iPads increased student motivation.  However; this study had many limitations. There were only 7 sample students who attended a Catholic school and all came from upper to middle class backgrounds. Geometry was the only subject assessed in the study and there was no research done to see if using the iPad decreased math anxiety (Fuchs, 2013).

The study of math anxiety in children is a neglected area. There are many assessments for math anxiety for adolescents and adults; however, evidence is mixed regarding anxiety in younger children so more research is needed to clarify levels of math anxiety in this age group (Jameson, 2013, p.392).   This could be due to the lack of age appropriate measurement tools. Traditional tests such as the 22-item Mathematics Anxiety Scale for Children (MASC), the Mathematics Anxiety Survey (MAXS), and the Suinn Mathematics Anxiety Rating Scale (MARS-E) all use Likert-type numeric responses.

These responses may pose a problem for elementary age children because children often have difficulty representing symbols that have little in common with items they represent such as being asked to gage feelings by circling a number to assess anxiety  (Jameson, 2013, p.392). In the new study conducted by Youngstown State University a pilot test of a Children’s Anxiety in Math Scale (CAMS) using  facial images was used that included 5 facial expressions that ranged from not at all anxious to very anxious was given to 438 children and had promising results. However, the study included 75% White, 14% African Americans, 5% biracial, and 1% Hispanic children so a more diverse population of “at risk” children such as those in minority groups or low socioeconomic status need to be assessed.

Math anxiety is often a product of environment and many parents of “at risk” students also have noted math anxieties that add to the child’s own dislike of the subject matter. Eugene Geist reports in his article “The Anti-Anxiety Curriculum: Combating Math Anxiety in the Classroom”, that, “Children from low socioeconomic backgrounds often have parents with less educational background and who often have negative attitudes towards mathematics themselves” (2010,p.24).  This leaves minorities at an increased risk of developing math anxiety and this can lead to a drop in performance. In future research more tests need to be done with parent and child to see the validity of this connection. Then we can focus on ending math anxiety by focusing on the actual source of the problem.

An overview of the proposed program

Before beginning The Beecher iPad Project which is an IRB Standard Study I will gain approval from the school administrative staff and consent from the parents and student. I will also make sure that confidentiality of the students is protected during the study.  I will adhere to all guidelines, policies and procedures of the University of Michigan-Flint and will follow the protocol at every step of the research process. I will also respect the autonomy of the subjects and place the principles of ethics such as beneficence and justice at the forefront of all processes and decisions made during the study.

The purpose of the study will be to evaluate the effectiveness of using technology in the classroom versus traditional methods for reducing math anxiety and increasing math test scores.  This program is a quasi-experimental research study that will explore the use of Apple iPad technology in the classroom in reducing math anxiety and improving math test scores in “at risk” elementary student from  low socioeconomic backgrounds. In this study the participants will include 150 4^th grade students in the Beecher Community School District, 75 students from Tucker will be the program group and 75 students from Dailey will be the control group. The students chosen from both elementary schools are from similar backgrounds, are similar in socioeconomic status, race, age, gender and cultural background.

I have hypothesized that using the iPad as an in classroom intervention tool could decrease math anxiety and improve MEAP test scores in student who are of from “at risk” populations and low socioeconomic backgrounds.

I will assess all 150 students’ math anxiety level before the beginning of the program with the 16-item Children’s Anxiety in Math Scale (CAMS) to get a base-line for the study. I will also review prior year MEAP test results for all students involved in study. I will then administer a short practice version of the MEAP test to all 150 students. Interviews will be held with the teachers in each classroom to get additional information on students that may be confounding factors that could skew results.

Dailey Elementary students will receive a packet of take home materials including the Michigan MEAP Coach Preparation Handbook to aid in learning math skills and testing tips.  Parents of Dailey elementary students will also sign an agreement stating they will keep a journal of the child’s progress and time spent working on take home materials.

Tucker students will each receive an Apple iPad which will be provided through a research grant from the U.S. Department of Educations, Institute of Education Sciences. Teachers will incorporate using basic arithmetic apps suitable for a 4^th grade curriculum, MEAP preparation and drilling apps into daily instruction.

The study will take 12 weeks to complete after the initial implementation of the interventions. At the 6 week mark there will be a process evaluation to assess barriers, staff needed, problems, satisfaction and number of people recruited.  At the conclusion of the study there will be another CAMS assessment given to all students and another short practice version of the MEAP test administered with different test questions. There will also be interviews conducted with all the students who completed the study using a qualitative method to evaluate intervention. Results will be compared between the two groups. I will then follow up with an outcome process evaluation by interviewing parents and staff to see how effective they felt the program was in achieving the target outcome and to measure outcomes such as how this program motivated the children to learn the subject matter, if they feel the iPad is a helpful learning tool for the classroom, rate training difficultly for students and teachers for iPad use, parent involvement, and how might the intervention be changed in the future.

The expected outcome will be a 60% decrease in math anxiety and 30% higher practice MEAP test scores in the program group; however, the control is also expected to have a slight decrease in math anxiety and increase in practice MEAP test scores but it will be expected the higher success rate will be found in the program group. I believe the program group will also show more motivation to learn math material using the iPad than the control group who received the take home math packet.

The purpose of this study will help to evaluate the effects of traditional interventions vs. new technological interventions and their effectiveness at eliminating math and testing anxiety to help to raise MEAP scores in Beecher Community School District where the percentage of 4^th grade students who score Level 4 (Not Proficient) is 80%. At Dailey Elementary the students Mathematics scores of the MEAP test for the 2012-2013 were as follows: Proficient (Level 2) is < 10%, Partially Proficient (Level 3) 12.5%, and Not Proficient (Level 4) 78.6%. At Tucker Elementary the students Mathematics scores of the MEAP test for the 2012-2013 were as follows: Proficient (Level 2) is 10%, Partially Proficient (Level 3) <10%, and Not Proficient (Level 4) 83.3%. The Michigan state average for the MEAP test Mathematics section for the state of Michigan for white 4^th grade students tested in the 2012-2013 were as follows: Proficient (Level 2) is 43.3%, Partially Proficient (Level 3) 16.0%, and Not Proficient (Level 4) 31.0%. The state average for the MEAP test Mathematics section for the state of Michigan for African American 4th grade students tested in the 2012-2013 were as follows: Proficient (Level 2) is 18.4%, Partially Proficient (Level 3) 13.4%, and Not Proficient (Level 4) 66.5%. As seen from this data the Beecher Community School district is far behind the state average for both White and African American students. The addition of Apple iPads in the classroom could have short term benefits such as help to end math anxiety, increase motivation to learn math skills, increase test scores. Long-term benefits and public return for these technological implementations could be tremendous.

Explanation of why the proposed program is unique 

This program is unique because very little data is available on the effectiveness of using iPads as a learning tool in the classroom. Also, the data that is available does not focus on the “at risk” populations, minorities and those students who present with low socioeconomic status. Using the iPad to help with MEAP preparation has not been tested in the state of Michigan. If this study shows success with the use of iPads it could led to more schools implementing their use and to the development of apps that will mirror material from The Michigan MEAP Coach Preparation Handbooks for all testing ages. This will not only be an eco-friendly addition to the curriculum it will also help to motivate and excite children to learn, decrease math anxiety, improve test scores and teach them how to utilize new technology.

Logic model

The need for this program is a pilot study to promote the use of classroom technology in low income communities and among minority students since research has found these tools can improve test scores in other population settings. The objective is to decrease math anxiety by 60% in minority children and children with low socioeconomic status in program group and increase MEAP test scores from 80% percent not proficient to 50% not proficient in program group in the next testing cycle for the Beecher Community School District. Inputs will include research grant from the Institute of Education Sciences in the amount of 60,000 USD, technology and equipment, knowledge, parents, school board, teachers, students, study coordinator, local government, and State of Michigan.  Technical efficiency: cost per client served is 60,000/150 = $400.00 per child who receives intervention. The cost benefit analysis: The cost of the program is 60,000. Some potential societal benefits are more children committed to school, high school achievement, more high school graduates, more employed with high paying jobs, more college graduates, less arrests, and less violent, drug & property crimes in these minority populations. The public return on investment could be immense.

Cost effectiveness: cost per unit of outcome, cost per child who decreased math anxiety and improved math scores 60,000/70 = 857.14 per child who decreased math anxiety and improved math scores.  The objective is consistent with the outcome. Activities will include the following: provide meetings with all 150 students parents before program – IO to get signed consent from students/parents and offer a Q/A session; providing 75 students and 6 teachers  with iPads to use in the classroom – IO to provide student and teachers with technology used for intervention in program group; 75 students with learning packets – IO to provide student with take home materials for MEAP study to compare intervention with traditional forms of study; meet with staff – IO to provide program overview, goals and objectives; provide students with in classroom instruction on how to operate the device by teachers – IO to ensure students understand how to use the device so they are not hindered from the intended outcome; provide one training session for teachers and staff on how to use the iPad & arithmetic apps –IO to make sure teachers and staff understand how to use the iPad device before it is incorporated into the curriculum; provide short practice MEAP test before and after study – IO to provide baseline of student performance and to measure success of intervention after 12 week study; have student complete the 16-item CAMS to measure anxiety before and after the study- IO to provide baseline of math anxiety and to measure success of intervention on math anxiety at completion of study; monitoring of student progress – IO to make sure the program is on course with outline; weekly goal setting from staff- IO to provide a more structured program; provide follow up meeting with parent and staff to interview– IO to evaluate and review program effectiveness.

Outputs are as follows: The classroom curriculum in the program group was successfully implemented and ran the 12 week duration; in the control group learning materials were sent home to all 75 participating students; journals were kept for 68 students in the control group; teachers were trained to use device; students in program group were trained to use device; meetings were held for progress reports, 5 students in program group received extra training after school hours to help learn how to use apps; two meetings were held with parents of both the control and the program group at the beginning of the research study and at the conclusion to gain consent from parents/students, hold interviews; and answer questions; the school was given regular updates and progress reports; the principals at both elementary school were given weekly updates about program.

I believe the program contributed to the outcome and the program covaried with the observed outcomes. Even though some students from both schools may have had contact with each other outside of the classroom the students were not allowed to take the devices home. There may have been exposure in the community to the take home materials by those in the program group; however, this is unlikely. No other in school programs or campaigns were running at the time of study. The rival hypothesis is that children in both the control and the program group may have been exposed to a new television program on PBS in the environment that focused on learning math and reading skills with the target age group of Pre-K through 5^th grade. Parents in both group may have been motivated to spend more time with children doing homework because of the study which could have contributed to positive results. Also, changes in teaching techniques in the classroom could have contributed to increase in test scores and decrease in math anxiety among both control group and program group.

Initial outcomes included: Students in the program group were more motivated to learn math skills in the classroom. Students in program group gained a better understanding of basic arithmetic. Parents of students in the program group reported increased motivation in students to attend school. Students in program group had decreased math anxiety by 70% when given the 16-item Children’s Anxiety in Math Scale (CAMS) at the end of the study which exceed expectations by 10%. Students in the program group also had a 40% increase in the short practice version of the MEAP test at the end of the study. This test included a different set of related math questions than the initial test. The test questions were also obtained from the Michigan Department of Education’s MEAP test bank. Students in control group only had a 10% reduction in math anxiety when given the CAMS at the end of the study and students in control group only showed an 8% increase in short practice version of MEAP test at the end of study. Parents in the control group also reported a decrease interest in take home learning packets after approximately week 3 of initial startup of study which could have contributed to the poor outcome.

Intermediate outcomes included: Students at Tucker continued to show increased interest in learning math. 20% of students that were in the program group for 4^th grade math anxiety and technology study scored Proficient (Level 2) on the following years MEAP test in Mathematics. 40% of students in the program group scored Partially Proficient (Level 3) on the following years MEAP test and only 40% scored Not Proficient (Level 4). This outcome exceeded expectations by 10%. This evidence shows that the students continued to benefit from the intervention and continued to be enthusiastic about learning math in the classroom after the study.

In the control group 6% of the students scored Proficient (Level 2) on the following years MEAP test in Mathematics and 15% of students in the control group scored Partially Proficient (Level 3) on the following years MEAP test in Mathematics and 79% scored at the Not Proficient (Level 4). The control group showed little improvement from the prior 2012-2013 test scores. Students in the program group also showed marked improvement in other subjects as time went by. Initial and Intermediate outcomes for those in the program group are as follows: better performance in math; reduce test anxiety; overall improved academic achievement; increased motivation to attend school, better relationships with teachers and increase self-esteem.

 

Population and sample

The population for this study was from the Beecher Community School district which is an undereducated community that has a low socioeconomic background. In this community 84.9% of residents are African American. The median income was $23,234. Approximately 21.5% of families have less than 10,000 in annual income, 43.0% of families receive SNAP benefits, families whose income was below poverty level in 2010 with children under the age of 18 was 52.1%. Families below poverty with females as householders (no husband present) with children under 18 was 62.9% (US Census Bureau, 2011).

The sampling strategy used was quota sampling. The students selected for the study from Tucker and Dailey Elementary were similar in age, race, cultural and economic backgrounds. The classroom sizes were the same and the number of students in the initial study for each group was 75 for a total of 150 participants. The average race of the 4^th graders was African American at 85%. White students made up 7%, 5% Hispanic, and 3% other races. I believe this sample distribution mirrors the general population in the Beecher community.

Of the 150 4th grade students who were involved in study. 73 students successfully completed the intervention in the program group while 2 students dropped out of program. One of the two students moved from the area and the other had to leave school because of illness. In the control group 43 students successful completed 100% of the MEAP Coach workbook, 11 students completed 75% of the workbook; 14 student approximately 50% of the workbook and 7 workbooks were unaccounted for at the completion of the study.

Process evaluation design and methods

The process evaluation will be conducted at different points in the program to assess if the intervention was implemented as intended and what problems exist. I will be using the qualitative design method. I will keep the “basics” of face-to-face interviewing a priority such as cautious flexibility, listening and observing, asking for clarifications, and making sure participants are informed before asking open-ended questions in interviews conducted with parents, and staff (McDavid et al., 2013, p.211). The transcripts will be analyzed by a trained researcher and themes will be developed based on these transcripts to construct an overall “set” of findings, conclusions, and recommendations. These open-ended questions will include assessments of participant satisfaction from both parents and teachers, parent satisfaction with program, staff satisfaction, and training issues/suggestions and how to improve the program in the future. Issues such as how effective they felt the program was implemented, if they feel the iPad is a helpful learning tool for the classroom, rate training difficultly for students and teachers for iPad use, was there adequate parent involvement, and how might the intervention be changed in the future will be addressed during the process evaluation.

 

Outcome evaluation design and methods

The outcome evaluation will be conducted after the completion of the research study and will be done by conducting interviews with staff, students, policy makers and school board. I will be using the qualitative design method. These open-ended questions will clarify if the program was a success, if changes in policy where made and what was accomplished by conducting the research. The transcripts will then be analyzed by a trained researcher and themes will be developed based on these transcripts to a construct an overall “set” of findings, conclusions, and recommendations.

I will also hold another outcome evaluation using a quantitative design. That will include a survey with close-ended questions given to parents and teachers to gage their overall satisfaction with the research study and the intervention. These measureable responses will then give us a percentage of those who approved of the overall effectiveness of the program. These outcome measures will be conducted by different researchers to increase inter-rater reliability. Using two different outcome survey methods will also increase inter-method reliability and the validity of the study.

Economic analysis

The budget for the study will be $60,000. This money comes from a research grant from The Institute of Education Sciences. Is will be used to buy the Apple iPad for 75 students and 3 teachers and 3 teachers aids cost of the 81 Apple devices will cost 30,000. The learning packets and journals for the control group will cost 5,000. The remaining cost will go to pay researcher coordinators, training, and other administrative needs as they occur during the study and to evaluate outcomes. Technical efficiency: cost per client served is 60,000/150 = $400.00 per child who receives intervention. Cost effectiveness: cost per unit of outcome, cost per child who decreased math anxiety and improved math scores 60,000/70 = 857.14 per child who decreased math anxiety and improved math scores.

Benefits for stakeholders

Children in the Beecher Community have some of the lowest proficiency rates in the state of Michigan and in is imperative to the overall survival of the community to needs to combat this problem. Recent studies have found that the use of iPads in the classroom can boost test scores and increase students desire to learn and participate in the classroom. We must invest in the children to give them the skills they need to become productive members of society. Potential societal benefits include: more children committed to school; decreased math anxiety; increased school achievement; more high school graduates; more employed in the community with higher paying jobs; more college graduates; less arrests; and less violent; drug & property crimes in these minority populations. The public return on investment could be immense.

 

 

 

 

 

 

 

                                                                           PART II: PROGRAM MATERIALS

Figure 1.  Logic model, including implementation and outcome objectives

Process evaluation tools

This is the 16-item Children’s Anxiety in Math Scale (CAMS) survey that was created by experts especially for younger children which includes questions about math performance and anxiety. Instead of traditional tests which ask children to represent emotional with numeric symbols this test allows children to respond by using a facial image scale (Jameson, 2013, p.392-393). The facial image scale was initially used for the assessment of dental anxiety; however results have proven successful when used in conjunction with the CAMS (Jameson, 2013, p.393). This survey will be given at the beginning of the program and at the conclusion to evaluate the success of the intervention.

The following is a list of the survey questions that students in both the program and control group will answer from the CAMS:

1. When I solve math problems, I feel:
2. When I think about doing math, I feel:
3. When I am working on math problems that are difficult and make me think hard, I feel:
4. Compared to other school subjects, math makes me feel:
5. When I solve math puzzles, I feel:
6. When I have a hard math question, I feel:
7. When the teacher calls on me to answer a math problem, I feel:
8. When the teacher is showing the class how to solve a math problem, I feel:
9. If I have to add up numbers on the blackboard in front of the class, I feel:
10. When I make a mistake in math, I feel:
11. Thinking about working on math in class makes me feel:
12. Working on math at home makes me feel:
13. When the teacher gives the class a math problem I don’t understand, I feel:
14. When my teacher says that he or she is going to give me a math problem on the blackboard, I feel:
15. When I know that my class will be working on math at school, I feel:
16. When I know that I am going to have a math test, I feel:

Process evaluation tool

 

 

 

MEAP Practice Exam based on the Michigan Grade Level Content Expectation (GLCE) for 4^th Grade

 

The 4th grade students in both the control group and the program group were given a short 40 question practice version of the MEAP test at the beginning of the study and at the conclusion of the study. This was done to evaluate the success of the intervention for both the control and the program group. Using the two MEAP practice test, the CAMS anxiety survey, parent survey and staff survey will increase the validity of the program. The questions for both exams were similar in subject matter but used different test questions selected from the Michigan Department of Education test bank and included arithmetic items that were expected to be mastered at the student’s present 4^th grade level.

All students were given 2:00 hours to complete the exam and calculators were not allowed. There was a school teacher present to monitor the students during the exam. The students were instructed to read directions and the test administrator also went over the directions. The students were then asked to answer multiple-choice mathematics questions.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Process Evaluation Survey for Stakeholders – Staff – Program Group

 

1. How many staff members were needed to initiate school based iPad program?
2. How many 4^th grade students did we actually recruit?
3. What barriers did we encounter when recruiting the student to the program?
4. Were parents and community receptive to the program, did you encounter problems?
5. Have there been barriers to keeping these “at risk” children in the intervention?
6. So far has it proved difficult to gain the cooperation of the program group parents?
7. What could help to gain the cooperation of the parents? Do you feel an incentive would help?
8. Is parent involvement sufficient for both the control and program group?
9. Do you feel that there are any outside influences that could contribute to the success or failure of the program?
10. Do you think the program students received enough training to get the full utilization of the iPad device?
11. Was staff training sufficient in the use of the iPad?
12. Do you feel that iPad apps are equivalent to the required classroom curriculum and learning materials?
13. Do you feel there were any training issues?
14. Do you feel the Beecher project is well organized?
15. What resources, if any, need to be added to the program to improve execution?
16.  Do you feel the administrative needs are being met and handled appropriately for the Beecher Community program to run in an efficient and effective manner?  Please explain your answer.
17. Do you feel that children are engaged in the activities of the intervention?
18.  Do you feel there are children in the program who may need further assistance?
19. Have you at any time during the implementation of the program felt the intervention was having a negative effect on the students?
20. How satisfied are you with the program?
21.  How satisfied do you think parents are with the program and are you receiving positive/negative feedback about the intervention program?
22. Do you feel that resources for the program are being used in an efficient manner?
23. Do you have any further question or concerns regarding the research study?
24.  Do you have any suggestions for future improvement?

Process Evaluation Survey for Stakeholders – Staff – Control Group

 

1. How many staff members were needed to initiate the school based iPad program?
2. How many 4^th grade students did we actually recruit?
3. What barriers did we encounter when recruiting the student to the program?
4. Were parents and community receptive to the program, did you encounter problems?
5. Have there been barriers to keeping these “at risk” children in the intervention?

6. So far has it proved difficult to gain the cooperation of the control group parents to help with the take home learning materials and daily journal entries? What could help to gain the cooperation of the parents? Do you feel an incentive would help?
7. Is parent involvement sufficient in the control group?
8. Do you feel that there are any outside influences that could contribute to the success or failure of the program?
9. Was staff training sufficient?
10. Do you feel there were any training issues?
11. Do you feel the Beecher project is well organized?
12. What resources, if any, need to be added to the program to improve execution?
13.  Do you feel the administrative needs are being met and handled appropriately for the Beecher Community program to run in an efficient and effective manner?  Please explain your answer.
14. Do you feel that children are engaged in the activities of the intervention?
15.  Do you feel there are children in the program who may need further assistance?
16. Have you at any time during the implementation of the program felt the intervention was having a negative effect on the students?
17. How satisfied are you with the program?
18.  How satisfied do you think parents are with the program and are you receiving positive/negative feedback about the intervention program?
19. Do you feel that resources for the program are being used in an efficient manner?
20. Do you have any further question or concerns regarding the research study?
21.  Do you have any suggestions for future improvement?

 

Process Evaluation Survey for Stakeholders – Parents – Control Group

1. Do you feel this program has the potential to benefit the students of Beecher Community?
2. As a taxpayer do you feel this is a good use of your tax dollars?
3. Has your child expressed more interest in learning math since the beginning of the research study?
4. Do you feel that the program implementation process has disrupted your child’s routine in a negative manner?

5. Do you feel your child may need further assistance during this study?
6. Do you feel that the MEAP take home material is easy to understand?
7. Do you find the use of the home journal to be too time consuming?
8. Do you have suggestions that may help us with gathering information besides using a journal?
9. Do you have any suggestions regarding the future improvement of the program?
10. How satisfied are you with the program so far?
11. Do you have any further concerns?

 

 

Process Evaluation Survey for Stakeholders – Parents – Program Group

1. Do you feel this program has the potential to benefit the students of Beecher Community?
2. As a taxpayer do you feel this is a good use of your tax dollars?
3. Has your child expressed more interest in learning math since the beginning of the research study?
4. Do you feel that the program implementation process has disrupted your child’s routine in a negative manner?

5. Do you feel your child may need further assistance during this study?
6. Do you feel that the use of the iPad in the classroom could enhance your child’s learning experience?
7. Do you think all children in the Beecher Community should be offered the opportunity to benefit from in class technologies such as the iPad?
8. Has your child expressed having any issues with the use of the iPad device?
9. Do you think that this type of learning style will benefit your child in the future?
10. Do you have any suggestions regarding the future improvement of the program?
11. How satisfied are you with the program so far?
12. Do you have any further concerns?

 

 

 

 

Outcome Evaluation Survey for Stakeholders – Students – Program and Control Group

This is the 16-item Children’s Anxiety in Math Scale (CAMS) survey that was created by experts especially for younger children which includes questions about math performance and anxiety. Instead of traditional tests which ask children to represent emotional with numeric symbols this test allows children to respond by using a facial image scale (Jameson, 2013, p.392-393). The facial image scale was initially used for the assessment of dental anxiety; however results have proven successful when used in conjunction with the CAMS (Jameson, 2013, p.393). This survey will be given at the beginning of the program and at the conclusion to evaluate the success of the intervention.

The following is a list of the survey questions that students in both the program and control group with answer from the CAMS:

1. When I solve math problems, I feel:
2. When I think about doing math, I feel:
3. When I am working on math problems that are difficult and make me think hard, I feel:
4. Compared to other school subjects, math makes me feel:
5. When I solve math puzzles, I feel:
6. When I have a hard math question, I feel:
7. When the teacher calls on me to answer a math problem, I feel:
8. When the teacher is showing the class how to solve a math problem, I feel:
9. If I have to add up numbers on the blackboard in front of the class, I feel:
10. When I make a mistake in math, I feel:
11. Thinking about working on math in class makes me feel:
12. Working on math at home makes me feel:
13. When the teacher gives the class a math problem I don’t understand, I feel:
14. When my teacher says that he or she is going to give me a math problem on the blackboard, I feel:
15. When I know that my class will be working on math at school, I feel:
16. When I know that I am going to have a math test, I feel:

Outcome Evaluation Survey for Stakeholders – Students – Program and Control Group

 

 

MEAP Practice Exam based on the Michigan Grade Level Content Expectation (GLCE) for 4^th Grade

 

The 4th grade students in both the control group and the program group were given a short 40 question practice version of the MEAP test at the beginning of the study and at the conclusion of the study. This was done to evaluate the success of the intervention for both the control and the program group. Using the two MEAP practice test, the CAMS anxiety survey, parent survey and staff survey will increase the validity of the program. The questions for both exams were similar in subject matter but used different test questions selected from the Michigan Department of Education test bank and included arithmetic items that were expected to be mastered at the student’s present 4^th grade level.

All students were given 1:45 minutes to complete the exam and calculators were not allowed. There was a school teacher present to monitor the students during the exam. The students were instructed to read directions and the test administrator also went over the directions. The students were then asked to answer multiple-choice mathematics questions.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Outcome Evaluation Survey for Stakeholders – Staff – Program Group

 

1. Did the program achieve the outcome you expected?
2. Do you feel that the program was successful?
3. Do you feel that the intervention had a positive impact on student performance?

4. Was there an immediate benefit from the program implementation?
5. Do you think this intervention will have a lasting impact on the 4^th grade students at Tucker Elementary?
6. After completing the 12 week intervention do you think the iPad is a valuable learning tool for students?
7. Do you think this device is an inconvenience in the classroom compared to traditional teaching?
8. Do you think this intervention disrupted the students learning process?
9. Do you think the iPad will continue to be used as a learning tool in the classroom?
10. Do you think the iPad would be a beneficial learning tool for other school subjects besides math?
11. Do you think an alternate approach to helping students in the Beecher community would be more beneficial and more cost effective than the use of iPad technology?
12. What problems did you encounter during the program?
13. How well do you think the administrative duties were carried out during the research study?
14. How satisfied are you with the outcome of the program?
15. Do you have any suggestions on how to conduct this intervention in the future?
16.  Do you have any other concerns you would like to discuss about the program?

Outcome Evaluation Survey for Stakeholders – Staff – Control Group

1. Did the program achieve the outcome you expected?
2. Do you feel that the intervention had a positive impact on student performance?
3. What problems did you encounter during the 12 week program?
4. Was there an immediate benefit from the program implementation?
5. Do you think this intervention will have a lasting impact on the 4^th grade students at Dailey Elementary?
6. After completing the 12 week intervention do you think the take home learning packet helped students and is a valuable learning tool?
7. Did you have trouble with parents completing take home journals?
8. Did the program incentive help with compliance?
9. Did children seem motivated to complete the take home study material?
10. Do you think this intervention disrupted the students learning process?
11. Do you believe material added to much time to the student’s normal homework routine?
12. Do you think an alternate approach to helping students at Dailey Elementary improve math scores would be more beneficial?
13. How do you feel about using iPad technology as a learning tool in the classroom?
14. How well do you think the administrative duties were carried out during the research study?
15. How satisfied are you with the outcome of the program?
16. Do you have any suggestions on how we can conduct this intervention in the future?
17.  Do you have any other concerns you would like to discuss about the program?

 

Outcome Evaluation Survey for Stakeholders – Parents – Control Group

1. Did the program achieve the outcome you expected with you child?
2. Do you feel that during the intervention there was a positive impact on your child’s performance?
3. What problems did you encounter during the 12 week program?
4. Do you notice an immediate benefit from the program implementation?
5. Do you think this intervention will have a lasting impact on your child?
6. Do you think the take home learning packet helped your child and is a valuable learning tool?
7. Did you have trouble completing take home journals?
8. Did your child seem motivated to complete the take home study material?
9. Do you think this intervention disrupted your child’s schedule?
10. Do you believe material added to much time to your child’s normal homework routine?
11. Do you think there is another approach that should be taken at Dailey Elementary to improve math scores that would be more beneficial?
12. What do you think about using iPads in the classroom as a learning tool?
13. How satisfied are you with the outcome of the program?
14. Do you have any suggestions on how we can better deliver this intervention in the future?
15.  Do you have any other concerns you would like to discuss about the program?

 

 

 

Outcome Evaluation Survey for Stakeholders – Parents – Program Group

 

1. Did the program achieve the outcome you expected with you child?
2. Do you feel that during the intervention there was a positive impact on your child’s performance?
3. What problems did you encounter during the 12 week program?
4. Do you notice an immediate benefit from the program implementation?
5. Do you think this intervention will have a lasting impact on your child?
6. Do you think your child had trouble learning how to use the iPad?
7. Was your child familiar with this technology before the intervention?
8. Do you think using the iPad was beneficial to your child?
9. What do you think about using iPads in the classroom as a learning tool?

10. Did your child seem motivated to complete his/her math homework or other school work?
11. Do you think this intervention disrupted your child’s normal routine in a negative manner?
12. Do you think there is another approach that should be taken at Tucker Elementary to improve math scores that would be more beneficial and more cost effective?
13. How satisfied are you with the outcome of the program?
14. Do you have any suggestions on how we can better deliver this intervention in the future?
15.  Do you have any other concerns you would like to discuss about the program?

 

 

 

 

 

 

 

 

Budget form, cost per subject, cost-effectiveness ratio

 

Enter here.   (1 page; Pages will vary depending on the length of the budget form.)  Discuss the budget in Part

Informed Consent Form for Research Study

 

November 9, 2013

CFPH Program

1700 SPH I

1415 Washington Heights

 Ann Arbor, MI 48109

Dear Parent:

Researcher Johnette Kinney, Ph.D. of the University of Michigan-Flint, Department of Public Health, invite your child to participate in a research study entitled The Beecher iPad Project.  You and your child are being contacted because we would like to include your child in a 4^th grade intervention for “at risk” students who attend school in the Beecher Community School District.

We would like to talk with your child ———————-.  The purpose of this study is to evaluate the use of iPad technology in the classroom to enhance learning of basic arithmetic, decrease math anxiety and improve MEAP test scores.  We plan to ask all 4^th grade students in the Beecher School District to participate in our research.  This study is being funded by the U.S. Department of Educations, Institute of Education Sciences.

If you agree, your child will talk to an interviewer about topics such as math anxiety, math performance, struggles in math, and MEAP preparedness.  An interviewer will come to your home to conduct the interview at a time convenient for you and your child.  The interview is expected to take about 45 minutes to complete. We would like to audiotape the interview, but taping is not required for your child to be part of the study. Your child will also be asked to complete a short practice MEAP test and a 16-item Children’s Anxiety in Math Scale (CAMS) prior to the start of the intervention.

While your child may not directly benefit from participating in our study, we hope that this study will contribute to the improvement of math proficiency and overall school performance within the Beecher Community School District.

We understand participating in The Beecher iPad Project may be difficult for your child.  The staff has been trained to work with children and will stop the intervention at anytime if there are concerns.

Your child will be paid $300 for completing the full 12 week program.  If your child decides not to finish the program, your child will be paid $50.

We plan to publish the results of this study, but will not include any information that would identify you, your child or family members.  To keep this information safe, the audiotape of your child’s interview will be placed in a locked file cabinet until a written word-for-word copy of the discussion has been created.  As soon as this process is complete, the tapes will be destroyed.  The researchers will enter study data on a computer that is password-protected.  To protect confidentiality, your child’s real name and the names of any family members will not be used in the written copy of the discussion.  The researchers plan to keep this study data indefinitely for future research.

There are some reasons why people other than the researchers may need to see information your child provided as part of the study.  This includes organizations responsible for making sure that the research is done safely and properly, including the University of Michigan, government offices, or the study sponsor.  Also, if your child tells us something in the interview that makes us believe that your child or others have been or may be physically harmed, we may report that information to the appropriate agencies.

An interviewer will call you to make an appointment to interview your child in approximately 1 week.  We hope that you will be willing allow your child to participant in this research study.

If you have questions about this research study, you can contact Researcher Johnette Kinney., University of Michigan Flint, Department of Public Health,1700 SPH,  1415 Washington Heights, Flint, MI 48502, (810) phone number,  or [email protected].

If you have questions about your rights as a research participant, please contact the UM Flint Institutional Review Board, 303 E Kearsley, 4204 William S White Bldg, Flint, MI 48502-1950, (810) 762-3384, [email protected].

Sincerely,

Johnette Kinney, Ph.D.

University of Michigan Flint

Department of Public Health

1700 SPH,

1415 Washington Heights

Flint, MI 48502

Parental Permission

By signing this document, you are agreeing to allow your child, ­­­­­­­­­­­­­­­­­­­_______________, to be part of the study entitled The Beecher iPad Project.  Your child’s participation in this study is completely voluntary.  If you allow your child to be part of the study, you may change your mind and withdraw your approval at any time.  Your child may choose not to be part of the study, even if you agree, and may refuse to answer an interview question or stop participating at any time.

You will be given a copy of this document for your records and one copy will be kept with the study records.  Be sure that the questions you have asked about the study have been answered and that you understand what your child will be asked to do.  You may contact the researcher if you think of a question later.

I give my permission for my child to participate in this study.

 

_____________________________________  ____________________

Signature       Date

 

I give my permission for the interview with my child to be audiotaped.

 

_____________________________________  ____________________

Signature       Date

 

References

“American FactFinder”. (2010). United States Census Bureau. Retrieved from http://factfinder2.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=DEC_10_DP_DPDP1

Buchanan, H., & Niven, N. (2002). Validation of a facial image scale to assess child dental anxiety: Assessing child dental anxiety. International Journal of Paediatric Dentistry, 12(1), 47-52.

Fuchs, L. (2013). Using iPad apps to motivate seventh grade students in the math classroom. (Order No. 1537114, Caldwell College). ProQuest Dissertations and Theses, , 37. Retrieved from http://search.proquest.com/docview/1356837570?accountid=14584. (1356837570).g

Geist, E. (2010). The anti-anxiety curriculum: Combating math anxiety in the classroom. Journal of Instructional Psychology, 37(1), 24-31.

Jameson, M. M. (2013). The development and validation of the Children’s anxiety in math scale. Journal of Psychoeducational Assessment, 31(4), 391-395.

McDavid, James C., Irene Huse, Laura R. L. Hawthorn, and James C. McDavid. Program Evaluation and Performance Measurement: An Introduction to Practice. Los Angeles: SAGE, 2013. Print.

Subramanian, C. (2012, February 22). New study finds iPads in the classroom boost test scores. Time Magazine. Retrieved from https://ed.stanford.edu/in-the-media/new-study-finds-ipads-classroom-boost-test-scores