The Effect of Self-Selection on Student Satisfaction and Performance in On-Line Classes at the Huizenga School of Business and Entrepreneurship

Pan Yatrakis, Nova Southeastern University
Helen Simon, Nova Southeastern University

Abstract
This paper examines student performance, satisfaction and retention of information in online classes as a function of student choice as to the format of instruction. Student outcomes are studied for two groups enrolled in on-line classes: those who were allowed to choose between an on-line and a ground-based format and who chose the on-line format voluntarily; and those who were obliged to take classes in the on-line format without being afforded the opportunity to choose.

Publication Reference:
Yatrakis, Pan & Simon, Helen (May 2002). The Effect of Self-Selection on Student Satisfaction and Performance in On-Line Classes. North American Regional Distance Education Conference, International Council for Open and Distance Education, Calgary, Canada.

Introduction

The market for courses provided over the Internet has experienced the fastest growth among all segments of the education industry. It has been estimated that college revenues from Internet and Intranet courses will reach $15 billion worldwide this year, up from only $4 billion four years ago (Jones 2000), and that web-based instruction will account for 46% of the education and training market by 2003 (Mottl 2000). Nevertheless, many colleges and universities which provide courses over the Internet continue to offer the same courses in the traditional, ground-based format, as well. This practice permits students to select between the two formats according to their perception of the advantages or drawbacks that each provides in their own particular situation. For example, a student who is a self-starter and who values flexibility and convenience may choose to take a course in the on-line format, while a less self-directed student who values guidance and the supportive environment of a traditional classroom may select the ground-based format.

On the other hand, some institutions offering on-line courses do not afford students the opportunity to choose. Several, such as Jones International University and Cardean University, offer all their courses in the on-line format only. Others, such as the University of Phoenix enroll students in separate ground-based and on-line tracks and compel them to take all their courses in the specific track which they have chosen. Still others, such as the authors’ school, the Wayne Huizenga Graduate School of Business and Entrepreneurship at Nova Southeastern University, provide most courses in both formats but offer some (e.g., five MBA prerequisite courses and an information technology course) only on line.

The purpose of the authors’ research is to determine whether there exists a significant difference in on-line students’ learning outcomes, dependent on whether they enrolled in their on-line courses by choice or because no other alternative was available to them

Previous Research

There is a wealth of literature on student outcomes in on-line courses, but its main theme is comparisons between on-line and ground-based formats. Most researchers have found no significant difference in outcomes between the two formats (Dominguez and Ridley 1999; Ryan 2000; Gagne and Shepherd 2001; Cooper 2001; Shea et al. 2001). Some, however, have cited student concerns about instructional quality in on-line courses (Bloom 1998; Terry 2000), while others consider virtual courses an "inferior technology", particularly in the teaching of complex material (Farrington, 1999; Brown and Liedholm, 2002).

Other researchers have focused on approaches and methods which may improve outcomes in the on-line environment. For example, Haythornthwaite et al. (2000) found that a ground-based "boot camp" preceding on-line courses can help build a sense of community among distance learning students and enhance their satisfaction and learning outcomes. Doran (2001) found that small group collaborative activity improved outcomes in on-line courses. A similar finding was made by one of the authors of the present study, who compared the grades and course evaluations of two student groups who had taken his on-line MBA-level Managerial Economics class at two different times. He found that the second group, who were given assignments on which they had to collaborate (via chat rooms, bulletin boards, conference calls, etc.) showed significantly higher levels of satisfaction and information retention than the first group, who were allowed to work individually (Yatrakis 2001).

To date, however, there have been no studies of the consequences of allowing students to choose between taking courses on line or in the traditional, ground based format, despite numerous indications that such consequences may, in fact, exist. For example, it is widely recognized that on-line courses appeal to students who value flexibility (Cooper 2001), but that they also require discipline and extra work beyond that expected in ground-based courses (Brown and Liedholm 2002). The learning styles of on-line MBA students and their preferred methods of course delivery have also been identified (Barnes, Preziosi and Gooden 2002). It would stand to reason that, ceteris paribus, students who feel most comfortable with on-line course delivery would be the most likely to enroll in courses taught in the on-line format. This process of self-selection may indeed contribute to the findings of no significant difference in outcomes between students taking courses in either format. What happens, however, when a student is not given that choice, but is compelled to take a course in the on-line format whether or not he or she fits the profile of a potentially successful on-line student? Students themselves seem to recognize that the availability of alternatives can be advantageous. In the Cooper (2001) study, 131 students (94 from traditional and 37 from on-line classes) were asked whether a computer applications class should be offered only on line, only on the ground, or in both formats; 100% responded that it should be offered in both formats. It is the purpose of the present study to examine this issue in the context of the authors’ own institution, and specifically in its Master of Business Administration (MBA) program, where some courses are offered only in an on-line format, while others are conducted both on line and in traditional, ground-based classrooms.

The Huizenga School MBA Program

The Huizenga Graduate School of Business and Entrepreneurship is part of Nova Southeastern University, the 12th largest private university in the United States, with campuses in Ft. Lauderdale, Florida and programs in 22 states and eight countries outside the U.S. Enrollment at the Huizenga School totals about 2500 students, of whom approximately 1600 are MBA candidates, with the rest enrolled in other master’s programs and a doctoral program. The MBA program offers 46 courses, including prerequisites, core courses, specialty courses, and a capstone course. Of these, all five prerequisite courses are available only on line, as is the information technology course. All other courses are taught in both ground-based and on-line formats, except for the capstone course, which is ground-based only. Registration for the on-line classes is capped at 15 students, about two-thirds the size of ground-based MBA classes. The online classes make use of the usual support facilities, including bulletin boards, chat rooms, assignment managers, grade managers and the like. At the instructor’s discretion, streaming video and/or voice chat sessions may be used. Information technology personnel are available on weekdays from 8:30 AM to 10 PM, on Saturdays from 8:30 AM to 8 PM and on Sundays from noon to 6 PM to solve problems and provide real-time assistance to instructors and students.

At the end of the course, students are required to complete a uniform evaluation. This evaluation rates the course, the instructor, and the school’s infrastructure and support facilities. Students are asked to assign a value of one through five to statements in the evaluation, according to the following scale:

1 – Strongly Disagree
2 – Disagree
3 – Neutral
4 – Agree
5 – Strongly Agree

Two among these statements are relevant to the issues examined in the present study:
1) I was very satisfied with the amount of time spent interacting with my classmates;
and
2) The course work and discussions resulted in high retention of information.

Although Statement 1 does not rate overall satisfaction with the on-line course, it is the only statement in the evaluation that measures student satisfaction of any kind, and is therefore used as a proxy for overall student satisfaction. It should also be noted that the second question measures student perceptions of information retention. However, as Ryan (2000) observed, comparisons of student perceptions are a good indicator of the relative effectiveness of different delivery formats; they may also be useful in comparing the effects of student choice or lack thereof. In any event, the present study also uses a third metric, the actual grades achieved by students enrolled in on-line classes for which there was no ground-based alternative, as well as those achieved by students in classes where ground-based alternatives were available.

Methodology

Student responses to the two course evaluation statements cited above were collected and tabulated from a sample of 397 students who chose to enroll in on-line versions of courses offered in both on-line and ground-based formats (the "Choice" group); and a sample of 433 students enrolled in courses available only on line (the "No Choice" group). Not all students answered both questions: the response rate to the first question (satisfaction) was 409 out of a possible 433 (94.5%) for the "No Choice" group, and 305 out of 397 (76.8%) for the "Choice" group. On the second question (retention of information), 429 of the "No Choice" students responded (99.1%), compared to 315 (79.3%) of the "Choice" group.

Student grades for these courses were also tabulated for the same two student groups. Two-Factor Analysis of Variance (ANOVA) Without Replication was used to determine whether significant differences existed between the two student groups in their responses to the two questions, and in their course grades.

Results

Tables 1 through 3, below, present the results of the ANOVAs. Figures 1 through 3 are graphic representations of the student grade and evaluation data.

Figure 1 – Student Performance

Table 1 – Student Performance

Figure 2 – Student Satisfaction

Table 2 – Student Satisfaction

Figure 3 – Student Retention

Table 3 – Student Retention

As can be seen from the data, students who enrolled in on-line courses after being given a choice between the on-line and ground-based formats (the "Choice" group) rated their courses higher on satisfaction and retention of information that did students in the "No Choice" group. The P-values for the two ANOVAs indicate that the null hypothesis of no difference in responses between the two groups is rejected at the 9% confidence level for perceived retention of information, and at the 11% confidence level for satisfaction. Though significant at levels somewhat higher than the conventional 5%, the results nevertheless suggest that considerable differences exist between the two student groups in their responses to these two questions. However, the ANOVA of student grades conclusively supports the null hypothesis of no difference between the two groups. Indeed, the median grades for both groups are the same (A- or 3.7), as is their mean grade average (3.3). The only discernible difference seems to be at both tails of the grade distribution, which are fatter for the "No Choice" group.

Conclusions

The results of this study suggest that students taking courses in an on-line format by their own choice achieve higher rates of satisfaction and perceived retention of information than do students in courses where no such choice is provided. This finding is consistent with those of earlier studies, which identified different types of learning styles and characteristics of successful on-line learners, and suggests that students enrolled in such courses by choice probably possess the attributes likely to make on-line learning a satisfactory and constructive experience. The results seem to indicate that voluntary on-line courses apparently attract a higher percentage of such students than do the mandatory ones, and therefore produce higher levels of satisfaction and perceived information retention as measured by the students’ course evaluations.

The findings of this study with respect to student grades do not seem to correlate with students’ perceptions. It might reasonably be supposed that students reporting higher levels of satisfaction and perceived retention of information should also perform better as measured by course grades. In fact, this was not the case. These seemingly conflicting results suggest that students feel a greater degree of satisfaction when allowed to self-select for on-line courses, and that this may carry over into their perception of retained information. But in terms of actual learning, to the degree that this is measured by grades, being afforded a choice seems to make little difference in outcomes.

The results with respect to grades do suggest that the two groups of students are basically similar in their learning abilities, and that the instructors’ grading is reasonably uniform and consistent across the on-line courses of the Huizenga School’s MBA program. Moreover, this seems to be true regardless of whether or not students are afforded the opportunity to select the format of course delivery.

The implications of this study’s findings for college administrators are two-pronged. On the one hand, student choice increases student satisfaction and perception of information retention. On the other hand, choice apparently makes no difference to grade outcomes. Accordingly, the results can be used to support choice as satisfying the preferences of the learning institutions’ customers, or to deflect requests to implement such choice on the grounds that it makes no difference in learning outcomes. In practical terms, the provision of choice may come down to financial considerations, and tradeoffs between these and student satisfaction. It may be more costly to provide courses on both the on-line and ground-based tracks, but the additional costs should be weighed against the generally positive learning experience which choice provides.

It should be noted that this study is far from definitive, and that follow-on work is needed to corroborate its findings in a broader context. There is surely some random noise in the data: by definition, the courses taken by the two student groups are not the same; the instructors are not the same; and it is possible that the characteristics of the students themselves may vary across the two groups. These potential differences are mitigated by the application of uniform admission standards, but they may not be entirely absent. For example, it is possible that students enrolling voluntarily in on-line courses may have higher GMAT scores or may be more motivated or more committed than their peers. Still, any substantial difference between the two student groups Effect of Self Selection with Online Classes could be expected to show up in their relative performance, and no such difference is apparent with respect to their grades.

A related area which should be investigated is the performance of students in conventional ground-based classes when allowed to self-select and when afforded no such choice: a study, in other words, analogous to the present analysis except in its focus on groundbased rather than on-line students. Such research could corroborate, and would certainly complement, the findings of the present study with respect to student choice.

Finally, the aforementioned differences between the two groups in the tails of the grade distribution is curious and warrants further investigation. A comparison indicates that the "Choice" group displays a tighter distribution of grades about the mean, while the "No Choice" group shows comparatively fatter tails at both extremes. The proportion of "No Choice" students receiving high grades (A and A-) exceeds that of "Choice" students, as does the proportion receiving low grades (C+, C and F). Conversely, the proportion of "Choice" students with grades of B-, B and B+ is higher than that of the "No Choice" students, while median and mean grades for the two groups are basically the same. Further research on these issues might expand the analysis to other institutions of higher education, utilize broader measures of student learning, and perhaps focus on specific characteristics of the two groups which might contribute to the observed differences in their behavior.

References

Barnes, F. Barry, Robert C. Preziosi, and Doreen J. Gooden, 2001. An Examination of the Learning Styles of Online MBA Students and Their Preferred Delivery Methods. Paper presented at The Institute of Behavior and Applied Management, San Diego, CA.

Bloom, David F., 1998. Digital Diploma Mills, Part III. The Bloom Is Off the Rose, Monograph, November. Retrieved May 15, 2002 from the University of Illinois, Web site: http://www.vpaa.uillinois.edu/tid/resources/noble.html.

Brown, Byron W., and Carl Liedholm, 2002. Can Web Courses Replicate the Classroom in Principles of Microeconomics? American Economic Review, May.

Cooper, Linda W., 2001. A Comparison of Online and Traditional Computer Applications Classes. THE Journal Technological Horizons in Education, March.

Dominguez, Paula Szulc and Dennis Ridley, 1999. Reassessing the Assessment of Distance Education Courses. THE Journal Technological Horizons in Education, September.

Doran, Cheryl L., 2001. The Effective Use of Learning Groups in Online Education. New Horizons in Adult Education, Summer.

Farrington, Gregory C., 1999. The New Technologies and the Future of Residential Undergraduate Education.

Gagne, Margaret, and Morgan Shepherd, 2001. Distance Learning in Accounting. THE Journal Technological Horizons in Education. April.

Haythornthwaite, Caroline, Michelle M. Kazmer and Jennifer Robins, 2000. Community Development Among Distance Learners: Temporal and Technological Dimensions. Journal of Computer-Mediated Communication, September.

Jones, Del, 2000. Will Business Schools Go Out of Business? E-Learning, Corporate Academies Change the Rules. USA Today, May 23.

Mottl, Judith N., 2000. Learn At A Distance. Information Week, p.767.

Ryan, Richard C., 2000. Student Assessment Comparison of Lecture and Online Construction Equipment and Methods Classes. THE Journal Technological Horizons in Education, January.

Shea, Peter, Eric Fredericksen, Alexandra Pickett, William Pelz and Karen Swan, 2001. Measures of Learning Effectiveness in the SUNY Learning Network in Bourne, John, and Janet C. Moore, eds., Online Education, Volume 2, Learning Effectiveness, Faculty Satisfaction, and Cost Effectiveness.

Terry, Neil, 2000. MBA Student Perceptions About the Effectiveness of Internet Instruction. Business Education Forum, April.

Yatrakis, Pan G., 2001. The Effect of Self-Selection on Student Performance in On-Line Courses. Paper presented at International Council for Open and Distance Education (ICDE) Conference, Dusseldorf, Germany, April.

About the Authors

Dr. Pan G. Yatrakis is a Professor of Finance at the NSU Wayne Huizenga Graduate School of Business and Entrepreneurship. He has been with NSU SB&E as an adjunct professor since 1993, and as a full-time faculty member since 1996. Dr. Yatrakis teaches Economics, Finance, and International Business at both the master's and Doctoral levels. He chairs several doctoral dissertation committees, and his own research interests are in the areas of inflation, growth and productivity; and economic and financial systems of transforming economies. He has had numerous publications and presentations at academic conferences. Dr. Yatrakis was Chief Economist of the Xerox Corporation from 1973 to 1982, and Senior Partner of The Center for Economic Analysis between 1982 and 1990. He has taught at New York University’s Stern School of Business Administration, Seton Hall University, Sacred Heart University, and the University of Bridgeport. Dr. Yatrakis is active in the field of economic consulting, primarily in forensic economics and litigation support. He has served as an expert witness in numerous cases involving economic damages. Dr. Yatrakis received his B.A., M.B.A., and Master of International Affairs Degrees from Columbia University. His Ph.D. is from the New York University Stern School of Business Administration, in International Economics and International Business.

Helen K. Simon, CFP is President of Personal Business Management Services, LLC, located in Fort Lauderdale, Florida. Helen has resided in South Florida since 1974 and has nearly 20 years of professional experience in the financial services and investment field. She serves on the adjunct faculty of Florida International University, Florida Gulf Coast University and The Huizenga Graduate School of Business at Nova Southeastern University where she teaches classes in financial management, personal finance and financial planning. She received a BBA, Magna Cum Laude, from Florida Metropolitan University, the CFP designation from The College for Financial Planning, an MBA from Nova Southeastern University and is currently ABD in pursuing a Doctorate of Business Administration (DBA) with a concentration in Finance.