Does Study Abroad Grading Motivate Students?
Stevan Trooboff
William Cressey
Sue Monty
This article was published in Frontiers: The Interdisciplinary Journal of Study Abroad, Volume 10 (Fall 2004) pp. 201-217.
Council on International Educational Exchange
This paper reports the results of an investigation of the relationship between study abroad grade reporting policies and students’ motivation and achievement. While our research provides partial support for the “conventional wisdom” – that a high impact grade reporting policy serves to motivate students to do their best work – the data taken as a whole suggest that the situation is considerably more complex than commonly thought. The authors believe the results of this study do not support policy decisions that favor a high impact grade reporting policy. Other strategies for motivating students are also discussed.
Introduction
Academic officers at US colleges and universities face a number of important decisions regarding how credits and grades earned overseas as part of a study abroad program should be incorporated into students’ home academic records. These include:
- Should credits earned be considered home institutional credit or transfer credit?
- How should institutions convert a foreign credit scheme or grading scale into a US equivalent?
- Should grades earned abroad be posted on transcripts?
- Should grades earned abroad be included in the cumulative Grade Point Average (GPA)?
- Should study abroad courses be deemed to have all been taken pass-fail, which frequently means that a C is required as the minimum passing grade?
Institutions have reached different conclusions concerning all of these issues. They have frequently based their decisions on a number of assumptions about the context of study abroad and the impact of one decision or another on student behavior. The purpose of this article is to take a closer look at the way grades are handled at home institutions and student motivation. It presents the results of a research study conducted at CIEE Study Centers during the Fall 2003 semester. We also offer a few suggestions as to what our findings suggest related to institutional policy.
The conventional wisdom in study abroad circles is that how grades are counted at home influences how diligently students apply themselves to their academic work while abroad. The more the grades count, the more the student pays attention and cares.
Concomitantly, since motivation and performance are presumably linked, a second underlying assumption is that students whose grades are counted at home achieve better results abroad or, at least, receive better grades. It should be noted that these assumptions presuppose a broader but often unstated hypothesis; namely, that grades in general serve to motivate students to do their best work. This latter issue is explored somewhat in previous studies discussed below. The presumption of a positive motivational value in counting grades while studying abroad has often been used to advocate posting all study abroad grades on home transcripts and including them in the student’s cumulative GPA.
Usually only anecdotal evidence is adduced in support of these assumptions. However, there is empirical evidence. One recent study that explored the relationship between grading policies and success abroad through research was conducted by Mary Merva of John Cabot University in Rome, Italy. She studied the semester GPAs of approximately 400 students during the academic year 2000-2001 and compared the mean semester GPAs for students whose grades “count” to those of students whose grades “do not count.” Merva’s results from that particular study are that “students whose grades are averaged into their cumulative GPA are estimated to have an increase in the mean semester GPA of .36 points (11.4%) above the average.” She concludes that “whether or not grades count does have an affect on students’ effort with a particularly deleterious affect on the overall academic gains from study-abroad programs.” In sum, she believes that her research supports the conventional wisdom.
While we don’t argue with Merva’s results, the sample she used and the setting in which the research was conducted is somewhat limited. John Cabot is a small school, populated about half with Americans studying abroad, teaches largely in English, and offers an experience designed to closely parallel home experience while incorporating the benefits of being in a different culture and learning or perfecting a new language. Study abroad programs, however, are far more diverse in their composition and settings and it was our belief that a wider sample of programs and settings would be likely to elicit different results more consistent with our own experience in the field.
CIEE has been a leader in the field of international educational exchange since 1947. Several of our Study Centers have been in operation for almost 40 years and we have been one of the pioneers in developing empirical data on the study abroad experience. Today, we serve more than 1000 colleges and universities, more than 3000 students annually, and operate more than 50 study abroad programs in 25 countries. Grades, grading, and the impact of grade-recording policies are of keen interest to us and to our partners and professional colleagues. Accordingly, in the fall of 2003, we set out to explore the relationships between motivation, grades received abroad, and the grade- recording policies of sending institutions.
Our goal in conducting this research is twofold: (a) to contribute to this discussion by exploring relationships of various sorts between grading and motivation (more fully described below), and (b) to broaden the discussion of the central question under investigation. Since study abroad is about a good deal more than grades and academic achievement, we sought to widen the scope of the investigation. We considered an assessment of motivation that includes an evaluation of the degree to which the student is engaged with the host community and is making an effort to derive maximum benefit from the intercultural dimensions of the program, even though these efforts are rarely related to grades received. Simply said, we believe that there is more to study abroad than grades and courses. Cultural integration and learning are an important and unique part of the study abroad experience and we felt a need to explore student motivation in these regards, while also tracking the impact of grades related to transcript practices.
Motivation
The hypothesis entailed in the conventional wisdom is that “how grades are counted at home influences how diligently students apply themselves to their academic work while abroad.” In other words, students whose grades have more impact on their official academic records are more highly motivated to do well academically abroad. Merva and the previous studies she cites provide arguments to support the assumption that “the grade – effort relationship should be positive” (149). While positively relating grading practices and grades, these studies almost always leap over the issue of student motivation and are largely silent on the issue of students’ efforts to make the most culturally of the study abroad experience. Since this cultural learning is a key part of the rationale for studying abroad, it is important in our view to look at grades and transcript policies in the context of motivation, not as an absolute in and of themselves.
Motivation is a complex construct and there are different kinds of motivation. In an interesting study of the relationship between goals and motivation, on the one hand, and academic learning, Antonio Valle and others distinguish learning goals, performance goals, and social reinforcement goals. Aspects of their discussion are directly relevant to the study abroad context: “Students with learning goals are interested in acquiring new skills and improving their knowledge even if they make mistakes; students with performance goals are … interested in obtaining positive evaluations of ability and … prefer to obtain a positive evaluation of a fairly simple task rather than run the risk of receiving a negative evaluation of a more challenging and meaningful task.” (72).
A similar distinction is drawn in a study of the impact of testing on students’ motivation for learning conducted by Wynne Harlen and Ruth Deakin Crick . The purpose of their study was “to provide evidence in relation to claims that, on the one hand, testing raises standards and, on the other, that testing … has a negative impact on motivation for learning.” (169). Harlen and Crick distinguish intrinsic and extrinsic motivation. Intrinsic motivation is associated with a number of positive outcomes: “learners find interest and satisfaction in what they learn, … recognize their own role in learning and so take responsibility for it, … [and achieve] levels of engagement that lead to development of conceptual understanding and higher level thinking skills.” (175). Learners driven largely by extrinsic motivation, on the other hand, “engage in learning because it is a means to an end, … [and] learning may stop … in the absence of … external incentives, [and] what is learned is closely targeted at behavior that is rewarded.” (175).
We will return to these issues as we discuss the implications of our research. The key points are that there are a variety of ways of thinking about motivation and that grading can have a positive or negative impact on various types of motivation depending on the individual involved. Therefore, to suggest that transcript policies lead to better grades, and that this demonstrates that students are “positively motivated” by such practices is, in our view, an oversimplification. We explore some of these issues in our discussion of the research outcomes and policy recommendations.
Objectives of the Current Study
In an effort to investigate the possible impact of alternative transcripting policies, we investigated the statistical relationships among four variables in a population of 551 study abroad participants from the Fall semester of 2003:
- The student’s cumulative GPA prior to going abroad;
- The transcript policy of the student’s home school;
- An assessment of the student’s motivation and involvement with the program content; and
- The GPA received by the student on the study abroad program
We tested the following hypotheses:
H1: Grades obtained by a particular student abroad correlate positively with the grades obtained by that student prior to the study abroad experience. This seems a natural hypothesis—good students are good students regardless of the setting and regardless of the transcripting policies of their home schools.
H2: Motivation of students while abroad is greater if grades are counted in the GPA and/or posted on the home transcript. We debated the merits of stating this hypothesis positively or negatively, and, in the end, opted to state it this way because it is consistent with what we have been calling the “conventional wisdom.”
H3: Grades earned studying abroad are higher if grades are counted in the GPA and/or posted on the home transcript. This hypothesis, as well, could be stated positively or negatively. Our formulation is consistent with the “conventional wisdom.”
In addition, a continuation of this study beyond the time limits imposed by publication deadlines for this article will investigate the relationship between participation in study abroad and achievement after students return home. We know from experience that overseas study often has a strong impact on students’ views of the world and themselves. We believe that one of the positive impacts of study abroad is that students become more serious academic students as a result of the study abroad experience. Our final hypothesis is that grades of students who study abroad show an improvement over their historical record once they return to their home campuses. Our research therefore will attempt to track students beyond the time frame of this article and see what happens to their grades when they return. However, this article focuses solely on the first three hypotheses.
Methodology
Sample
During the (northern hemisphere) Fall semester of 2003, more than 1000 students participated on CIEE study programs. For purposes of this research, we extracted a sample of 551 students at 32 program sites. These sites were chosen for a variety of reasons. First, they represent a broad spectrum of program types rather than a single type. Next, all of the programs are relatively small (none larger than 35 students). This means that the Resident Directors on these programs will know their students well and have a good sense of their level of motivation. In addition, these sites represent a broad spectrum of sending institutions. All together, there are 136 sending institutions represented in our sample, with an average of 4 students per institution—a high of 50 and low of 1. No single institution accounts for the majority of students which might distort the results.
Data Collection
CIEE maintains a comprehensive database for student applications, enrollments, and grading. Therefore, we have on hand the cumulative GPAs (prior to study abroad) as reported for most students attending our programs. Grade data is taken from student transcripts submitted prior to the program and from grade reports at the program sites.
Institutional practices regarding how grades are recorded on home transcripts and whether or not study abroad grades count in student GPAs were verified by a questionnaire sent to each sending institution. While there were varying practices from institution to institution and some differences within an institution from program to program, overall we found it most meaningful to break transcript practices into three groups. The first group records grades on the home transcript and counts them in student GPAs (henceforth “Grades and GPA”). The second group records grades on transcripts but does not count them in the GPAs (henceforth “Grades Only”). The third group does not record study abroad grades on the home transcript and does not count them in the GPA (henceforth “Credit Only”). Table I provides a breakdown of these groups by student count.
Table I: Number of students subjected to each of the three home campus policies studied.
Policy |
# of Students |
GRADES & GPA |
263 |
GRADES ONLY |
116 |
CREDIT ONLY |
116 |
Along with grades, we sought to measure student motivation. To achieve this, we asked the resident director at each site to evaluate each student’s motivation at one of four levels. In our view, study abroad (and in fact education in general) is about a good deal more than grades. One of the primary goals of study abroad is to provide exposure to and involvement in a local culture. Therefore, one of the important measures of the effectiveness of study abroad is whether or not students are motivated to make the most of the cultural integration opportunities abroad as well as putting effort forward in their academic work. To go abroad and not get involved in the local culture, even with good grades, misses the point of the experience. And, to get involved in the culture and make the most of these learning opportunities while flunking every course or doing poorly, is equally problematic. A balance of student motivation in these two dimensions of the successful overseas experience—cultural involvement and academic performance— is essential. Therefore the levels as we defined them make reference both to academic effort and attempts to be engaged in the local culture.
The four levels are defined below.
- HM – Highly Motivated - These are students who clearly are making the most of their study abroad experience. They are actively engaged in the academic program, doing everything possible to embrace the local culture, and overall have shown enthusiasm and interest in maximizing their study abroad experience. These are students who one would characterize as outstanding in terms of overall motivation.
- FA – Fully Acceptable – These are students whose motivation is certainly adequate. They seem to be reasonably invested in academic and cultural opportunities that the program offers. They participate in cultural activities most of the time and seem to be trying to gain as much as possible from the experience. However, they do not demonstrate the outstanding characteristics of the highly motivated.
- LM – Limited Motivation – These students are attending classes and doing most of what is required in terms of academic and cultural assimilation. They are less likely to break out of the American student shell. They clearly are not making the most of the many opportunities offered by the program. While their work and motivation is not unacceptable, they are not particularly motivated and will do what they have to do to pass, no more.
- UN – Unmotivated – These students are not motivated to gain from the study abroad experience. They are uninterested in engaging in the myriad of academic and cultural opportunities available. They might work hard enough to pass their courses, but that’s all. Overall, they are treating the experience more as a holiday than an educational experience.
The students in our sample were evaluated on this dimension as shown in Table II:
Table II: Number of Students receiving each of the motivation ratings
We are aware of the subjectivity of this measure of motivation. However, as indicated earlier, motivation is complex, and to our knowledge there is no single measurement technique that has been shown to be completely reliable. We believe that our resident directors are in as good a position as anyone to make relevant judgments on motivation and that their ratings are likely to be reliable for our purposes. They are experienced in working with students abroad and certainly have a good feel for which students are really invested in the experience of study abroad, both academically and culturally, and which are not.
Data Organization
Three of the four variables investigated are organized in a straightforward way as linear progressions from a “low” end to a “high” end. The two grade point averages (prior to participation and during participation) range from a low number which is “bad” (theoretically zero but in practice 2.32 for the prior GPAs and 1.5 for the study abroad GPAs) to a high (“good”) of 4.0 for the prior GPAs and 4.3 for some study abroad programs that award A+ grades. We organized each of the sets of grade point averages into quintiles as shown in Table III:
Table III: Grade Point Averages organized into quintiles.
Quintile |
Range for Prior GPAs |
Range for Study Abroad GPAs |
5 |
3.435 - 4.00 |
3.765 - 4.33 |
4 |
2.865 - 3.43 |
3.195 - 3.76 |
3 |
2.295 - 2.86 |
2.625 - 3.19 |
2 |
1.725 - 2.29 |
2.055 - 2.62 |
1 |
1.155 - 1.72 |
1.484 - 2.05 |
The four motivation ratings also range from Highly Motivated, which is “good” to Unmotivated, which is “bad.” We numbered the ratings from 4 (highest) to 1 (lowest).
The final variable—home school transcripting policies—could be viewed as two independent variables: (i.) whether or not study abroad grades are recorded on the home school transcript, and (ii.) whether or not study abroad grades are factored into a student’s cumulative GPA. The policy could also be viewed as a single variable, organized as a progression from “least impact” (grades are not recorded and are not averaged into cumulative GPAs) to “most impact” (grades are recorded and averaged into cumulative GPAs), with the practice of recording the grades but not averaging them into the GPA being a “middle value” in the range. In our investigation, we explored both possible groupings and ran the statistical tests both ways.
Analytic Tools
All data was tabulated and various statistical tests were performed on the data to measure correlations and similar statistical relationships. We examined results in two ways—first adhering to commonly accepted standards for significance (i.e., requiring at least a 95% chance that there is a correlation, based on a Pearson Chi-square), and less formally examining apparent relationships which, although not statistically significant, seemed interesting to us.
Thus, we did not completely discount relationships simply because they did not meet common tests of statistical significance. In our view statistical significance and practical applicability do not always coincide. As practitioners, our goals were to review and investigate the data in as many ways as possible to try and understand exactly what it means rather than adhering to mathematical models that, in spite of widespread use, are not always consistently understood. For this reason, we have reported some findings in spite of the fact that they fall short of the 95% “relevance” threshold. In the same way, we have tried to take the widest possible view of the information we assembled, sometimes looking at groupings other than those originally set for some of our variables.
Over all, we believe our methodology is consistent with good social science practice but we recognize that we have taken liberties in the spirit of exploration of outcomes that might not be to the liking of some.
The primary analytic tools used to explore relationships among the variables were Analysis of Variation (ANOVA) and Pearson’s Correlation Matrix, which is used to find a correlation between at least two continuous variables. The value for a Pearson's can fall between 0.00 (no correlation) and 1.00 (perfect correlation). Other factors such as group size will determine if the correlation is significant. Generally, correlations above 0.80 are considered quite high.
Findings
H1: Grades obtained by a particular student abroad correlate positively with the grades obtained by that student prior to the study abroad experience.
The data we analyzed provides strong support for our first hypothesis. Table IV displays a simplified version of the results generated by our statistical analysis. The pre-participation quintiles are the columns; the study abroad quintiles are the rows.
Table IV: Study Abroad GPA Quintiles: Prior (Columns) Compared to During (Rows)
Percentages
| |
3-Middle |
4 |
5-Highest |
Total |
N |
Lowest-1 |
|
|
|
|
|
2 |
|
|
|
|
|
3 |
|
|
|
|
|
4 |
|
|
|
|
|
Highest-5 |
|
|
|
|
|
Total |
|
|
|
|
|
N |
|
|
|
|
|
P=0.0000
The shaded values are higher than the corresponding percentages for the population taken as a whole. There is a linear progression from upper left (lower GPAs both prior and during study abroad) towards lower right (higher GPAs both prior and during study abroad). For each table, we give a probability vale (P), which indicates the likelihood that the results are statistically significant. Higher P values indicate greater likelihood of significance. In the field of statistics, values higher than 95% are considered reliable. When evaluating the table’s probability of being statistically significant, this table achieved the highest score of any in our study as its significance level is greater than 99.9%. This indicates there is very little likelihood that chance caused the pattern we see in this data table.
The table is a clear confirmation of our first hypothesis. It also shows that more students scored in the second quintile (B grades, roughly) abroad than at home while significantly fewer students scored in the top quintile abroad. (This in spite of the fact that overall there is slight increase in average GPA from at home (3.34) to abroad (3.45). We return to this finding in our conclusions.
It should come as no surprise to anyone that good students are good students, whether at home or abroad, and that they are just as likely to do well while abroad as at home. All CIEE Study Centers have a GPA admission requirement at the upper levels of the grading spectrum. In general, students on our programs are from quality institutions, are generally serious about their education and have performed near a 3.0 GPA (or higher) prior to attending a program. Thus, while the students at the top of the range outperform those lower in GPA, the reality is that all these students are good students capable of quality academic work. This holds irrespective of the transcript practice of the sending institution. In other words, whether or not the grades earned while studying abroad count, the stronger student performs better.
H2: Motivation of students while abroad is greater if grades are counted in the GPA and/or posted on the home transcript.
This hypothesis turned out to be only half true. Examining the data revealed an unexpected trend in that the largest group (48%) to score the highest for motivation was not from either end of the spectrum of Credit Only or Grades & GPA but rather fell in the middle Grades Only as shown in table V below. Surprisingly the most rigid policy which transcribes the grades and factors them into the GPA yielded the lowest number of highly motivated students (35%). Another anomaly worthy of further investigation is the apparent split in the motive ratings of the students in the Credit Only group, where a larger than expected number of student (23%) were ranked as having low motivation. More students are less motivated in this group than in either of the others. The data was grouped by gender to see if that might be a confounding variable or account for the odd distribution in the Credit Only group and nothing was significant. Gender is not a confounding variable.
Table V: Home Institution Policy (Columns) Compared to Motivation Ratings (Rows)
Percentages
| |
Credit Only |
Grades Only |
Grades & GPA |
Total |
N |
Unmotivated |
4.31 |
0.00 |
4.55 |
3.43 |
17 |
Less Motivated |
23.28 |
12.93 |
17.05 |
17.54 |
87 |
Fully Acceptable |
35.34 |
38.79 |
43.18 |
40.32 |
200 |
Highly Motivated |
37.07 |
48.28 |
35.23 |
38.71 |
191 |
Total |
100 |
100 |
100 |
100 |
|
N |
116 |
116 |
264 |
|
495 |
P=0.0300
The highlighted cells are those with the highest percentage for that column.
There is an interesting and counter-intuitive trend in the distribution—students whose home schools record grades but do not average them into the cumulative GPA have the highest motivation ratings. Just over 87% of the Grades Only group score in the top two motivation categories, while 78% of the Grades & GPA group and 72% of the Credit Only group score in these top two categories.
Interpretation of these statistics is by no means straightforward. Viewing policy differences as a single variable ranging from most impact to least impact, we would expect the Grades & GPA group to rank higher than the Grades Only group.
But it should be remembered that our motivation ratings are based on assessment of effort both in academic work and in intercultural involvement. Perhaps adding the burden of GPA implications causes students to concentrate so hard on grades that they are unable to participate fully in the cultural opportunities, and therefore are seen as less motivated by resident directors.
To probe this possibility a bit further, we examined the students who came from Grades & GPA schools and who received the two lowest motivation ratings. We have final study abroad grades for 45 of these students. Their average Study Abroad GPA was 3.36, which is lower than the average for the entire population.
Another possible conclusion is that it may be more accurate to view the policy question to entail two separate policy decisions—whether or not to record grades and whether or not to count grades in the GPA. We therefore ran the tests again comparing the motivation ratings separately with each of two independent variables. From the distribution displayed in our first test, we can suspect that of the two new variables, whether or not grades are recorded might turn out to be more significant than whether or not the grades are factored into the GPA. Our results as shown in Tables VI and VII suggest that this is true.
Table VI: Motivation by whether or not grades are recorded.
Percentages
| |
Grades Not Recorded |
Grades Recorded |
Total |
N |
Unmotivated |
|
|
|
|
Less Motivated |
|
|
|
|
Fully Acceptable |
|
|
|
|
Highly Motivated |
|
|
|
|
Total |
|
|
|
|
N |
|
|
|
|
P=0.2400
Table VII: Motivation by whether or not grades are included in GPA
Percentages
| |
|
|
|
N |
Unmotivated |
|
|
|
17 |
Less Motivated |
|
|
|
87 |
Fully Acceptable |
|
|
|
200 |
Highly Motivated |
|
|
|
191 |
Total |
|
|
|
|
N |
|
|
|
495 |
P=0.1500
The shaded cells contain percentages that are higher than the corresponding percentages for the population taken as a whole. In Table VI the higher-than-total cells fall where we would expect them to fall, whereas in Table VII, the distribution seems random. Unfortunately, neither of these sets of results returns a chi-square value that indicates that the figures are significant.
Yet another possibility is that the apparent anomaly between Grades Only percentages and Grades & GPA percentages could be the result of other factors. For example, over 50% of the Grades Only students come from schools included in US News and World Report’s top colleges and universities, while only 11% of the students in the Grades & GPA group come from similarly ranked schools. It could be that students from so-called “top colleges” display characteristics to resident directors that make them seem more motivated.
However, when we excluded the “top colleges” students from the sample and ran the correlation between home school policy and motivation, the results were similar to those obtained for the entire sample (Table V). Even without the “top colleges” students, the Grades Only policy has 24% more students in the top motivation bracket than the Grades & GPA policy.
We are left with the conclusion that the apparently higher motivation of students in the Grades Only category (as compared to the Grades & GPA category) is an unexplained attribute of this particular population. If we ran the experiment again with a new population of students and came up with the same results, this would strengthen our supposition that this correlation is real. For many study abroad professionals, this aspect of our results will seem anomalous, but the fact remains that in our data it is shown to be statistically significant.
For us, the bottom line is that the common assumption—that including study abroad grades into students’ cumulative GPAs leads to higher levels of motivation—is simply not very well supported by this empirical evidence.
It is important to point out that a significant majority (78%) of students in the sample scored in the top two motivation categories using our rating scheme. This does not indicate that resident directors were overly generous but rather that in general we are working with a motivated group of students. The ratings represent slight degrees of difference which, while statistically significant, are nonetheless fairly close together.
H3: Grades earned studying abroad are higher if grades are counted in the GPA and/or posted on the home transcript.
In the case of this hypothesis, as with H2, there is some support but it is far from conclusive. Table VIII shows the percentages of students scoring in each of the GPA quintiles (rows) compared to the three policies that we are investigating.
Table VIII. GPA Earned Abroad Compared to Home School Grading Policy
Percentages
Quintiles |
Credit Only |
Grades Recorded |
Grades & GPA |
Total |
N |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
P=0.0910
Shaded cells contain the highest percentages for each column. The policy of Grades Only precipitates the largest percentage of students in the highest grade quintile (42.5%) while Credit Only has about 27% and Grades & GPA has only 24%. This finding is compatible with the previous finding in that they both suggest that a Grades Only policy is the most conducive to a fuller study abroad experience and to higher grades as well.
When the mean GPAs for the study abroad semester were compared by policy group, the Grades Only group scored 0.12 higher than the Credit Only students and 0.13 higher than the Grades & GPA students.
Table IX. Mean Home and Study Abroad GPAs by Policy
| |
Credit Only |
Grades Only |
Grades & GPA |
|
|
|
|
|
|
|
|
It could certainly be argued that the proper basis for comparison among policy groups is not the absolute GPA obtained abroad, but rather the change in GPA, if any. To make this measurement, we divided the population into three groups: those whose grades went down one quintile or more, those who stayed in the same quintile, and those who improved by at least a quintile. Table X shows the result of this comparison.
Table X: GPA Change by Policy
Percentages
Table X gives us the expected linear progression from upper left to lower right, at least for the Credit Only and Grades Only policies, and the chi square test returns a value showing that the distribution is significant. The data therefore suggests that whether or not grades are recorded is important but whether or not they are included in the GPA is not. When we ran similar tests for each of the policies taken as an independent variable, we did get linear results for Grades Recorded versus Credit Only but not for GPA versus No GPA.
In order to enable a comparison of our results with those reported in Merva, we also ran tests using the actual semester GPA scores of the students in each Policy group to obtain mean scores. The left column includes both the Credit Only group and the Grades Only group; the right column includes only the Grades & GPA group. This is the way Merva grouped the policies. As can be seen in the Table XI, when grades don’t count towards their cumulative GPA the mean semester GPA is higher (3.49), than the mean semester GPA of those students whose grades will count in their cumulative GPA(3.42). This is a difference of -0.07, and shows a small negative correlation between counting study abroad grades in cumulative GPA’s and grade results. This result is in opposition to the result obtained by Merva.
Table XI. Mean GPAs compared by whether or not study abroad grades are included in cumulative GPAs at home.
Credit Only
Grades Only |
|
N of cases: 216
|
241
|
|
|
|
|
This entire set of tests related to our third hypothesis points to the superior performance of the Grades Only group as compared to the other two groups. Once again the relationship between the Grades Only group and the Grades & GPA group is counter-intuitive and runs contrary to the “conventional wisdom, which holds that the more impact grades have, the more they motivate students to do their best work.
It is an intriguing question whether the results for H2 and H3 are two separate results or simply two consequences of the same phenomenon. To test this, we ran correlations between MOTIVATION and STUDY ABROAD GPA and between MOTIVATION and GPA CHANGE. Neither test produced convincing correlations. The results for hypothesis two and those for hypothesis three seem to be unrelated to each other.
Conclusions
Our research leads us to conclude the following:
1. This evidence suggests that grade recording policies can affect the overall motivation of some students. Specifically, recording students’ grades on home transcripts can lead to higher motivation. However, Including grades in a student’s GPA does not seem to produce the desired result, and may even be counter-productive.
2. The way the three policies appear to line up is unexplained at this point.
3. The most clear-cut correlation in our findings is between a student’s cumulative GPA coming in to study abroad and the GPA he or she achieves while abroad. This comparison was shown by standard statistical methodologies to be better than 99% reliable.
Policy Implications
We do not believe the evidence supporting the motivational value of grade recording policies is strong enough so that it should be a primary influence in policy-making in this regard at US colleges and universities. Some aspects of our analysis, our review of the previous literature, and our experience with study abroad in general suggests that better overall study abroad results can be obtained by giving students the freedom to experiment, move outside the traditional academic box, and take risks. This can best be achieved by relieving them from concern about what might happen to their academic records as a result of studying abroad.
Problems with Study Abroad Grades
Study abroad grades come in many varieties. Some US letter grades are the result of applying conversion charts to foreign grading systems. Other host institution grades, while looking very much like US letter grades, are not necessarily being awarded in distributions that are similar to ours or in contexts where the same letter grade has the same meaning in both cultures.
Our research indicates that grades improve slightly but that the top grades are less frequently awarded abroad than at home. One of the authors of this study routinely addresses upwards of fifty study abroad grade appeals per year. Many of these appeals come from students who have received B grades but feel their work deserves A grades. Many of them are concerned about what a B or two will do to their honor status at home and/or to their possibility of gaining admission to selective graduate schools, law schools, or medical schools. In many cases it is probably true that a similar performance at home may well have earned a particular student an A rather than a B.
There are many issues related to study abroad grades and grading policies in the overseas context and therefore some reason to consider whether the small and dubious gain often attributed to “counting grades” really is worth it. If by “counting grades,” we mean including them in the student’s GPA, our study would suggest that that the answer to this question is that no, it is not. There is no gain in counting them in the GPA, only in recording them.
Our Goals and How Best to Achieve Them
We should ask ourselves which approach to maximizing student motivation is more productive to the overall goals of study abroad—a grades-and-GPA policy that motivates students to be cautious in their selection of courses or one that encourages them to take the risk possibly entailed in, for example, taking a regular university course taught in a foreign language. We did not address such issues in this study, but they may be an interesting theme for future research.
In addition, we should remember the distinctions drawn in some of the prior studies we reviewed, particularly the difference between learning goals and social reinforcement goals. Clearly the kind of motivation produced by a high impact grading policy is directed to the latter goals, whereas the former are much more likely to result in a rewarding and meaningful intellectual experience overseas. One claim made in the Harlen and Crick study seems particularly relevant to this point. “People who commit themselves to a goal will direct themselves towards [productive] actions.” (175) What we ought to be doing is finding ways to encourage students to become committed to meaningful goals such as integrating into the host society, achieving a deeper understanding of another culture and therefore a new understanding of their own culture.
There are a number of interesting student development techniques that would certainly lead more productively in this direction than a high impact grading policy. But that is a subject for another paper.
We recognize, of course, that for a small but very visible cohort of students, study abroad is not the serious enterprise we would like it to be. We are mindful of the fact that the reputation of study abroad can be damaged in faculty members’ minds by the knowledge that some students are not taking this opportunity seriously. We are aware that in some cases students are not putting forth their best efforts. The push towards high impact grading policies is no doubt due to a desire to counteract these tendencies. However, we do not believe this is the majority of students and many of these students can be screened out in the admissions process. To subject all students to policies directed at this cohort is problematic.
One additional point raised in the Harlen and Crick study seems to point towards a better way to address this issue. They distinguish between “summative assessment” and “formative assessment” and point out that formative assessment (testing that helps students develop better learning strategies) “can significantly raise standards of attainment,” whereas summative assessment (final examinations, standardized tests, etc.) “has a negative impact on motivation for learning.” (170). Their summative testing and formative testing can, for the purposes of study abroad, be equated to final examinations and continuous assessment, respectively. All too often in a study abroad context, students receive little or no feedback on their work until the very end, when, of course, it is too late to help with the final outcome. A better way to motivate students and keep them on task, we believe, would be to incorporate more feedback and continuous assessment into the coursework they take overseas, however difficult this may be due to cultural differences.
To sum up, there may be some motivational value to recording study abroad grades on home school transcripts, however including study abroad grades in cumulative GPAs does not seem to achieve desirable results and has other negative consequences. Most important, there are other strategies for generating more meaningful student motivation. These strategies can be built into participant selection, orientation, program design, program execution, and student re-entry and evaluation, which can significantly reduce the importance of grading policies while at the same time increasing the overall value of the experience.