Assessing Student Needs in Web-Based Distance Learning
Editor’s Note: Needs assessment is the first step in the design of instruction. It provides key information to select appropriate technology and instructional strategies for online learning. This paper provides direction for evaluating student needs in web-based distance education courses. This paper outlines and describes the student needs assessment process in five essential areas: computer skills, learning styles, available resources, learning outcomes, and prior learning experiences. The instructor is committed to adapting and modifying instructional strategies to match the needs of the group and individuals within the group.
Pamela A. Dupin-Bryant and Barbara A. DuCharme-Hansen
Over the past decade, distance education programs have developed at an extraordinary rate. Web-based distance education has emerged in higher education as a means for providing a variety of educational opportunities to a diverse community of individuals. As the number of participants continues to increase, so to does the importance of providing effective instruction that focuses on the needs of learners. Successful distance education is believed to revolve around a learner-centered system of instruction designed to meet the needs of individual learners. The first step in developing a learner-centered system of web-based instruction is to determine the needs of students. Assessing student needs provides instructors with information necessary to select appropriate technology and instructional strategies to develop an online learning environment that is appropriate, responsive, and beneficial for both the learners and the instructor. The goal of this paper is to provide direction for evaluating student needs in web-based distance education courses by identifying necessary assessment areas and outlining a process for assessing student needs.
Keywords: student needs assessment; web-based distance education; online education; needs assessment process; learner-centered education; distance learning; electronic learning; e-learning; computer skills; learning styles; available resources; learner’s desired outcomes; prior learning experiences
Over the past decade, distance education programs have developed at an extraordinary rate. Web-based distance education has emerged in higher education as a means for providing a variety of educational opportunities to a diverse community of individuals. As the number of participants continues to increase, so to does the importance of providing effective instruction that focuses on the needs of learners. Successful distance education is believed to revolve around a learner-centered system of instruction designed to meet the needs of individual learners (American Council on Education, 1996; Dillon & Walsh, 1992; Granger & Bowman, 2003). Many believe that technology nurtures educational customization in which the unique learning needs of individuals can be fulfilled (Farmer, 1997; Gardner, 2000; Granger & Bowman, 2003). Yet, how can instructors refine their teaching approach to meet the unique needs of students if they unaware of their students’ needs?
The first step in developing a learner-centered system of web-based instruction is to determine the needs of students. Assessing student needs provides instructors with information necessary to select appropriate technology and instructional strategies to develop an online learning environment that is appropriate, responsive, and beneficial for both the learners and the instructor. The goal of this paper is to provide direction for evaluating student needs in web-based distance education courses by identifying necessary assessment areas and outlining a process for assessing student needs.
What is Student Needs assessment?
Student needs assessment for web-based instruction includes the collection, synthesis, and interpretation of data about learners that can assist the instructor in matching student needs with the demands of the online learning environment (DuCharme-Hansen & Dupin-Bryant, 2005). The main purpose of student needs assessment is to give the instructor the tools and information necessary to make solid decisions about how to best facilitate the educational experience from start to finish. This information will assist in setting learning objectives, selecting appropriate technology, deciding on curriculum content, and determining strategies for effective learning. The overall importance of web-based student needs assessments is to establish, facilitate, and maintain an environment that is learner focused.
So what areas need to be assessed? Unfortunately there is no definitive answer to this question. To determine assessment areas, instructors must ask themselves “What do I want to know about my students that will help me determine what to teach, how to teach, what technology to employ, and where to start the educational experience?” While there can be numerous assessment areas, a number of areas are essential when planning web-based learning, including:
- Computer skills
- Learning styles
- Available resources
- Learner’s desired outcomes
- Prior learning experiences
By looking at these five areas at varying depths and degrees, instructors will gain a holistic view of the group and the individual learners who compose that group. The following sections describe the importance of each assessment area and introduce strategies for implementing the assessment results into the web-based education environment.
To succeed in web-based courses students must have adequate computer skills. These computer skills include basic computer operation, file management, web browsing, and email operation. Computer needs must be identified so that students may be provided with options for improving their skills prior to or during the course. “Researchers have indicated that early intervention to compensate for limited technical skills is important” (Hannafin, Hill, Oliver, Glazer, & Sharma, 2003, p. 247). There are a number of methods for assessing students’ computer needs. Introductory email observations, focus groups, student interviews, and survey instruments are available to assess computer skills. A variety of instruments related to specific platforms and software can be found on the Internet.
Once computer skills have been assessed, the instructor will need to determine the range of computer skills and the collective group needs. Although computer skill development is now being taught in kindergarten classrooms, instructors can not assume everyone is computer literate. “Until the use of technology is as innate as listening, reading, and writing, we cannot assume students are ‘tech-ready’ ” (Duncan & Wallace, 2002, p. 29). Therefore, based on the computer skills needs assessment, instructors may decide to provide a general technology review at the beginning of the semester. This review could include computer concepts, computer terminology, or simply the computer skills needed to succeed in the specific course.
In adding a technology review to a web-based course, the instructor may fear that precious time is taken away from content time, but this does not have to be the case. Computer skill development does not need to detract from curriculum. By tying skill development to lesson content, both skill and content can be addressed. Computer skill development for students is highly linked to the demands of the course. If instructors expect students to submit assignments via email or participate in communication activities via synchronous or asynchronous technologies, those skills should be addressed at the onset of a course. The key is to incorporate computer skill development activities as soon as possible, thus paving the way for student success throughout the remainder of the course.
After conducting the computer skills needs assessment, an instructor may find that there is an individual or a very limited group of learners who lack basic and necessary computer skills. Instead of requiring technology reviews for the entire group, the instructor should incorporate individual assignments for student who need additional computer help. Common tools for helping students are supplemental online tutorials, a CD-ROM of basic computer operation, and video short courses. Please note, if an instructor is going to require different or additional assignments for individuals but not all students, this needs to be identified in the syllabus. For example, an instructor might state: “based on assessment results and the desire to create an environment that is focused on individual student success and learning, the instructor may ask individuals within the class to complete additional activities.”
Assessing student learning styles can be a valuable tool in planning course activities that complement student learning needs. Anderson (2004) suggests that “developing quality education systems requires that educators have a deep understanding of how individuals and groups of students learn” (p. 239). With an understanding of student’s learning styles, SEQ CHAPTER \h \r 1 SEQ CHAPTER \h \r 1 an instructor can select teaching methods to match and in turn create an environment that will support student success (Bonham, 1989).
Although “learning style characteristics do not typically predict whether the student will succeed or fail in a distance environment” (Hannafin, et al., 2003, p. 249), assessing learning styles can help instructors recognize that each student learns in a variety of ways. Consequently the learning style assessments can help instructors integrate an assortment of activities that match various learning styles. Although there is no all encompassing, agreed-upon definition of learning style SEQ CHAPTER \h \r 1several frameworks have been proposed to organize the various approaches to assessing learning styles. James and Gardner’s model of learning styles is defined “as three distinct but interconnected dimensions” (1995, p. 20) including: (a) perceptual dimension mode (physiological or sensory), (b) cognitive mode (mental or information processing), and (c) affective mode (emotional or personality characteristics). A variety of learning-style instruments are available to help instructors assess students in these areas. James and Blank (1993) compiled an excellent summary that instructors should review before selecting an instrument. Their summary provides a general overview of each instrument including validity, reliability, and instrument cost. After selecting and administering an instrument, instructors can use the results to decide on teaching methods to match the individual needs of learners. In addition, many instruments provide practical guidelines for implementing methods, activities, and strategies based on the instrument’s unique results.
Instructors may select from a variety of learning style instruments to help generate learning initiatives that match learner needs. However, instructors should be careful not to pigeonhole students based on the results of an instrument. All assessment results should be used to enhance learning and it is important to remember that all students can benefit from participating in learning activities that match various learning styles. Dillon and Greene (2003) suggest that “our most important task as educators is indeed to help learners build a repertoire of approaches to learning so that they can learn under the variety of circumstances that life will surely bring” (p. 239).
As obvious as it may sound, students must have access to the Internet and must have appropriate computer hardware and software to successfully complete a web-based course. Yet, since web-based learning is a new experience for most students, these access requirements may be unclear to many prospective students. Perkins (1991) suggests that resources are an important component of all learning environments. Therefore, instructors should undertake a student needs assessment related to available resources prior to the course. Once technological deficiencies are identified, students can be notified and assisted in obtaining necessary resources. Assisting students in this area will help break down barriers to learning and will diffuse a potential learning environment rife with frustration. Data can be easily collected with a survey including questions related to availability of hardware and software resources.
In an ideal educational setting this type of student assessment and subsequent assistance would be conducted and provided by administrative services (e.g. continuing education department or online enrollment specialists). However, web-based instructors should not assume this is the case. Instructors should seek to be proactive in identifying student resource deficiencies and consequently provide students with necessary assistance.
Simplicity plays a crucial role in effective web-based education. Instructors should therefore try to keep resource requirements to a minimum. Requiring the use of an obscure piece of software or hardware, for example, could exclude students from participation or hinder their success. Keeping resources streamlined will help eliminate barriers to participation, will simplify distance learning processes, and will help decrease some of the issues that may disrupt the flow of a course and of student learning.
A common assessment result for many students might simply be the lack of appropriate software. With freeware and shareware programs available for educational purposes, software can be easily downloaded and updated for the student. If the assessment proves that students are unable to participate due to hardware-related issues, students should be notified immediately of their resource limitations and what hardware they will need to successfully complete the course.
Learner’s Desired Outcomes
Understanding what a learner desires as an educational outcome gives the instructor the needed information to ensure congruency between course goals and student goals. Instructors should seek to find out why learners have elected to participate, what they want to learn from the course, and how willing they are to be part of the learning process. Of all the student assessment areas, understanding learners’ desired outcomes is pivotal to student success and learning. When privy to this information, the instructor can match course objectives with those of the students. Within the constraints of higher education settings, it is understandable that specific objectives must be taught; however, these mandated course outcomes can match the desired learning outcomes of the learners. Web-based courses can incorporate activities that will help students achieve both institutional mandated outcomes and the general learning outcomes identified by the majority of the students in the course. This inclusion gives the learner responsibility and allows the learner to help set course objectives, which can lead to deep learning.
A clear articulation of student learning goals can help the instructor select appropriate instructional strategies to achieve these mutually benefiting goals (Wiggins & McTighe, 1998). For each student in a web-based course, the instructor can focus on adapting activities to match specific needs. For example, if a student in a research methods course indicates she would like to learn how to analyze her company’s research and development data – she may be encouraged to incorporate this into her final project. The student could meet the same learning objective as her classmates (learning to read, analyze, and critique research methods) but she could do so by analyzing her company’s work. Knowing what students desire to learn from the course, will help the instructor craft a customized and relevant learning experience for each student.
Prior Learning Experiences
Prior experience and success in the use of computer-based applications is important to success in technologically mediated learning environments (Hannafin, et al., 2003). Assessing a student’s prior learning experiences with web-based distance education will result in the collection of valuable information. If the students are novices to the web-based process, demands, and technology requirements, instructors can begin the course with activities that address preliminary issues in order to build a foundation for success.
With information about prior learning experiences, instructors may opt to use learning activities that require students to use all technology avenues within the first two weeks of the course, thusly building skills at the onset of a course. Instructors can also incorporate activities such as support networks (i.e. pairing learners who have had prior experiences with novice web learners). Collecting data on prior experiences can range from a tally response to a short written student self-report so that the instructor can have an understanding of the depth and extent of experiences. A data review may reveal a link with computer skills or available resources. Such areas could be jointly addressed through a variety of activities that give novice learners experience with the learning environment, build computer skills, and expand resources at the same time.
The necessity for student assessments is easily justified and commonly supported as an important endeavor. Unfortunately, understanding the importance does not guarantee the success of the venture. Implementing a student needs assessment may be disastrous without the assistance of a well-organized plan. When conducting needs assessment, planning is central. Galbraith, Sisco, and Guglielmino (1997) identified a sequence of events for successful needs and interest assessment at the program planning level. A similar sequence of events should be conducted for web-based courses (DuCharme-Hansen & Dupin-Bryant, 2005):
- Define the purpose
- Choose the assessment methods
- Develop a timeline for data collection
- Conduct the student needs assessment
- Analyze the data
- Match student needs with the learning environment
Define the Purpose
The first step in conducting a needs assessment is to identify the purpose. Five common needs assessment areas were identified in the previous section including computer skills, learning styles, available resources, learner’s desired outcomes, and prior learning experiences. While comprehensive in nature, these are not the only areas an instructor may decide to assess. As long as an instructor can justify the purposefulness of an assessment, there is a high probability that the assessment will benefit the instructor, the learning environment, and in turn the learner.
Choose the Assessment Methods
Once the instructor has defined the reasons for the student needs assessment, the next step is to select the methods for data collection. Qualitative methods play “a discovery role, while quantitative research plays a confirmatory role” (Gall, Borg, & Gall, 1996, p. 29). Therefore, qualitative methods may be more useful to an instructor who has little information about students and seeks to discover themes and relationships. Quantitative methods might help an instructor who wishes to validate those themes and relationships for the entire class. Whether conducting a needs assessment with qualitative methodology, quantitative methodology, or a combination of both, it is imperative that the methods result in valid and reliable data.
Develop a Timeline
After the purpose is identified and the method is selected, the instructor can develop a timeline for data collection. The timeline should take into account the time it will take to distribute the assessment, the time it will take students to complete the assessment, the time for receipt of the assessment, and finally, the time to analyze the data and match it to the course methods. For example, when conducting a computer skills assessment (with the idea that if the students lack a skill they will need prerequisite training) the instructor will need to conduct the assessment with ample cushion time prior to the course. In higher education, however, a professor typically does not know the individual students who are taking the course until the day the course begins. In this common case, a student needs assessment can still be conducted. The importance of a written timeline becomes even more critical since it will help streamline the time it takes to conduct an assessment at the beginning of the course when time is of the essence.
Conduct the Needs Assessment
Conducting the needs assessment can be done in a variety of ways. The instructor may combine various assessment areas into a general assessment or may simply choose to conduct each area as an individual assessment. Common collection methods for both general and individual assessments include telephone, email, and web-based surveys. Interviews can be conducted via the telephone although this is time-consuming and costly. Questionnaires, standardized tests, and informal questioning via email and web-based methods are inexpensive and can provide immediate feedback. They can also provide data for assessing computer skills and available resources. That is, if students respond to an email survey they have a necessary computer skill and technology resource.
Analyze the Data
Many methods exist for analyzing needs assessment data. If an instructor selected a quantitative method, instruments and standardized tests usually come with a scoring procedure and analysis guide. Qualitative data can be interpreted systematically by first identifying major themes and then categorizing individual responses according to the major theme. Most instructors intuitively analyze qualitative data in a quasi-structured fashion. Instructors should follow their intuition when analyzing the data and should note both specific group needs and individual needs in their analysis.
Match Student Needs with the Learning Environment
The final and most important step in the needs assessment process is to match student needs with course strategies, methods, and activities that will facilitate learning in the web-based environment. The instructor has two main tasks related to matching student needs. The first is to create a plan that meets group needs and the second is to verify that the plan embraces the individual student needs within the web-based distance education environment.
The concept of matching the learning environment and educational methods with individual learner needs is based primarily on progressive and humanistic educational theories. These philosophies highlight the unique nature of human beings. Lindeman (1961) stated “if we take for granted that human nature is varied, changing and fluid, we will know that life’s meanings are conditioned by the individual” (p. 8). In the same vein, due to the flexibility offered by web-based distance education environments, it is easy to see how this environment attracts and supports unique and diversified learners. Therefore, in an educational environment that has more than one person, the environment is laced with subject matter differences, situational differences, individual learner differences as well as individualistic goals and purposes for learning. The value of matching the learning environment and educational methods with individual learner needs is that if educators believe learners have differing personalities and backgrounds and as such differing needs, then there is no one best way to teach or to learn. Rather, the best way of teaching and learning is what is best for each individual learner at that moment in time.
Focusing on the uniqueness of each student becomes paramount to effective web-based education. By obtaining information about students through the needs assessment process, instructors can focus on individual learners and become facilitators of learning, rather than using the “easy method of giving the same dose to more individuals” (Lindeman, 1961, p. 4). Successful web-based instructors seek to prescribe for individual student needs within a group setting while maintaining the depth of content learning that embraces the diversity of the individuals in the group. Fundamental to the assessment-match process is its cyclical nature. Assessment-match should be conducted before the web-based experience as well as during the experience. Assessment gives the instructor the tools and information necessary to make solid decisions about how best to facilitate the educational experience from start to finish.
Web-based education requires an upfront investment from both the instructor and the learner. The instructor needs comprehensive and accurate student needs assessment information in order to prepare, facilitate, and adapt learning strategies as online education unfolds. Without this vital information, distance education becomes a game of trial and error without the probability of high success. This paper outlined and described the student needs assessment process as well as five essential areas in which to conduct student needs assessments. Each piece independently is important, but not as important as the information gained when all the pieces are interlocked to reveal the true needs of the individuals as well as the group. Knowing student needs is just the first step. The second step requires the instructor to be committed to adapting and modifying instructional strategies to match the needs of both the group and the individuals in the group.
American Council on Education (1996). Guiding principles for distance learning in a learning society. Washington, DC.
Anderson, T. (2004). A second look at learning sciences, classrooms, and technology: Issues of implementation: Making it work in the real world. In T. M. Duffy & J. R. Kirkley (Eds.), Learner-centered theory and practice in distance education (pp. 235-249). Mahwah, NJ: Lawrence Erlbaum Associates.
Bonham, A. L. (1989). Using learning style information, too. In E. Hayes (Ed.), Effective teaching styles. New directions for continuing education, No. 43. San Francisco: Jossey-Bass.
Dillon, C., & Greene, B. (2003). Learner differences in distance learning: Finding differences that matter. In M. G. Moore & W. G. Anderson (Eds.), Handbook of distance education (pp. 235-244). Mahwah, NJ: Lawrence Erlbaum Associates.
Dillon, C. L., & Walsh, S. M. (1992). Faculty: The neglected resource in distance education. The American Journal of Distance Education, 6(3), 5-21.
DuCharme-Hansen, B. A., & Dupin-Bryant, P. A. (2005). Web-based distance education for adults. Malabar, FL: Krieger.
Duncan, J., & Wallace, M. K. (2002). Assessing online technology: Edutainment or desktop-rubbishing. The Delta Pi Epsilon Journal, 44(1), 25-38.
Farmer, J. (1997). Using technology. In J. Gaff, J. Ratcliff, & Associates (Eds.), Handbook of the undergraduate curriculum (pp. 478-492). San Francisco: Jossey Bass.
Galbraith, M. W., Sisco, B. R., & Guglielmino, L. M. (1997). Administering successful programs for adults: Promoting excellence in adult, community, and continuing education.Malabar, FL: Krieger.
Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Educational research: An introduction(6th ed.). White Plains, NY: Longman.Gardner, H. (2000). The disciplined mind. New York: Penguin.
Granger, D., & Bowman, M. (2003). Constructing knowledge at a distance: The learner in context. In M. G. Moore & W. G. Anderson (Eds.), Handbook of distance education (pp. 169-180). Mahwah, NJ: Lawrence Erlbaum Associates.
Hannafin, M., Hill, J. R., Oliver, K., Glazer, E., & Sharma, P. (2003). Cognitive and learning factors in web-based distance learning environments.
In M. G. Moore & W. G. Anderson (Eds.), Handbook of distance education (pp. 245-260). Mahwah, NJ: Lawrence Erlbaum Associates.
James, W. B., & Blank, W. E. (1993). Review and critique of available learning style instruments for adults. In D. D. Flannery (Ed.), Applying cognitive learning theory to adult learning. New directions for adult and continuing education, No. 59. San Francisco: Jossey-Bass.
SEQ CHAPTER \h \r 1James, W. B., & Gardner, D. L. (1995). Learning styles: Implications for distance learning. In M. H. Rossman & M. E. Rossman (Eds.), Facilitating distance education. New directions for adult and continuing education, No. 67. San Francisco: Jossey-Bass.
Lindeman, E. C. (1961). The meaning of adult education. Norman, OK: Oklahoma Research Center.
Perkins, D. N. (1991). Technology meets constructivism: Do they make a marriage? Educational Technology, 31(5), 18-23.Wiggins, G., & McTighe, J. (1998). Understanding by design. Alexandria, VA: Association for Supervision and Curriculum Development.
About the Authors
Pam Dupin-Bryant is an Assistant Professor of Business Information Systems/Extension at Utah State University Tooele. She received a B.S. degree in Business Administration/Marketing (1993) and an M.S. degree in Business Information Systems (1994) at Utah State University. She completed a Ph.D. in Adult Learning and Technology (2000) from the University of Wyoming. Her research focuses on facilitating learner-centered distance education environments and student retention in distance education courses. She recently co-authored a textbook titled Web-Based Distance Education for Adults (Krieger, 2004). She currently serves as the co-editor of the Mountain Plains Adult Education Association Journal of Adult Education.
Pamela A. Dupin-Bryant, Ph.D.
Utah State University Tooele
1021 West Vine, Tooele, UT 84074
Phone: (435) 882-6611
Fax: (435) 882-7916
Barbara DuCharme-Hansen, a counselor at Madison Area Technical College in Wisconsin, earned a B.S. degree in psychology (1992) and an M.S. degree in counseling (1994) both from the University of Wisconsin – Whitewater and has completed her Ph.D. degree (2000) in adult learning and technology from the University of Wyoming. DuCharme-Hansen has focused her research and publications on the needs of adult learners in the distance education arena. She recently co-authored a textbook titled Web-Based Distance Education for Adults (Krieger, 2004).
Barbara A. DuCharme-Hansen, Ph.D.
Madison Area Technical College|
3550 Anderson Street, Madison, Wisconsin 53703
Phone: (608) 243-4279