CHAPTER 4. DATA COLLECTION AND ANALYSIS
Statement of the Problem
Interaction has long been identified as a key element to successful online learning programs (Beldarrain, 2006; Moore, 1993). While not the sole indicator of high-quality and effective online education programs, there is significant evidence to suggest that meaningful interaction with other students and the instructor is integral to the development of thriving learning environments (R. E. Brown, 2001; Garrison & Cleveland-Innes, 2005; H. C. Greene, 2005; Joyce Lee et al., 2006; Swan, 2002). Insufficient or ineffective interaction may lead to student isolation, while exorbitant levels may lead to overload or frustration (Berge, 1999; Willging & Johnson, 2004). Yet, little empirical evidence currently exists as to the value that learners place upon the various types of interactions in a self-paced learning environment.
Purpose of the Study
The purpose of this study was to expand upon previous research advocating for the purposeful design of interaction within the online learning experience (C.-W. Chang, 2006; Hirumi, 2002). This study builds upon existent knowledge of online education by specifically examining the composition of the online learning experience of adult learners in a hybrid self-paced learning environment that includes numerous interactive components and instructor facilitation designed to foster an engaging and meaningful learning experience. The goal of this study was therefore to extend the bounds of previous research into interaction within the online learning experience by examining what forms of interaction learners in a self-paced online course value most as well as what impact they perceive interaction to have on their overall learning experience.
Research Questions
This exploratory study sought to examine the experiences and preferences of adult learners concerning the various interactions that they encounter in a self-paced online course. The following four primary research questions guided data collection and analysis efforts:
- What forms of interaction do adult learners engage in most in self-paced online courses?
- What forms of interaction do adult learners value most in self-paced online courses?
- What forms of interaction do adult learners identify as equivalent in self-paced online courses?
- What impact do adult learners perceive interaction to have on their self-paced online learning experience?
Data Collection and Analysis
Interview questions aimed to explore the recent experiences of participants as they completed the course, noting their preferences for various interactive components of the self-paced learning experience. Questions sought both quantitative and qualitative descriptors from participants and are described in detail in successive portions of this chapter.
This study examined the dynamics of interaction within a self-paced online learning environment utilizing both rich media and a mix of traditional and emerging asynchronous computer-mediated communication tools to determine what forms of interaction learners in a self-paced online course value most as well as what affect such interactions have on their overall learning experience. Four primary research questions guided this research, analyzed below.
Analysis of Research Question 1
Research Question 1: What forms of interaction do adult learners engage in most in self-paced online courses?
In order to gain a sense of the forms of interaction that learners engaged in most in a self-paced online course, participants were asked to first of all reflect on their overall self-paced learning experience and share what were the best as well as weakest aspects of the course. Successive questions inquired as to their preferences for various components of the course as well as the frequency in which they utilized such features.
Participants expressed overwhelming pleasure with the various course activities and instructional content that were utilized. Such instructional content included a mix of both instructor-developed online multimedia presentations including audio and static slides as well as supplementary video presentations linked to from outside of the online course room. Dan noted that the best parts of the course for him were, “Hands down all the videos and the links and the stuff of that sort. The resources that were provided.” Sue concurred, further noting,
I think the presentation vides and the instruction videos that you included from YouTube. Those were just, it was like the presentation videos gave me an idea of where I was going because this time I was totally in the dark but the instructional videos from YouTube were like, ‘Oh my, I can do this, this is really simple.’ I was trying to complicate it.
Others identified other more overarching characteristics of the course as being what they found to be the best parts of their learning experience. John selected the self-paced format in general along with the immediate application of the concepts to his real-life ministry context as being the strongest aspects of the course, stating, “I liked being able to go through it and if I needed to take more time on something or something ‘Oh, I caught that’ and I could zip through it. Being able to do that was good.” John went on to say, “Really, the application assignments. Learning about blogs but then having to go and make your own blog and the same with the podcasting and those things, that was good.” Brenda fully agreed, stating, “Probably being able to immediately put into practice the things that we just learned.” Recurring among the responses from participants regarding the strongest aspects of the course experiencing included the various rich media instructional components and the practical application assignments that were intentionally designed into each unit providing opportunities for learners to immediately put into practice skills and new knowledge acquire.
In terms of areas of the course identified as weakest, sentiment was mixed. Robert highlighted the textbook readings as weakest, noting “It seemed like the online readings were a little more relevant or whatever but the textbook was a little bit on the dry side.” Christine felt that the course discussion board and the requirement of having to post 1-2 paragraph “I Learned” reflection statement describing at least three key concepts, ideas, or practices from the unit presentations, reading, and study activities that can be applied to her unique children's ministry setting was one of the weaker aspects of the course, stating, “It was good but…probably was the weaker part of the whole course.”
Karen made an interesting observation, stating, “Well for me, obviously, the weakest aspect was that I was the only student.” She happened to be enrolled in a section of the course in which she was the online student. When asked to describe how she thought that impacted her learning, she responded as follows:
I really don't know if I would have learned more with other students or not. I feel like I learned the material. So I don't feel like I was hurt in that way that I learned the material. But hearing other people's feedback and being able to go back and forth and say, "I'm not sure I'm doing this right", you know, that's helpful. But I didn't feel like I couldn't come to you if I didn't understand something. But I think that's part of the learning experience is the feeding back from each other and I didn't have that. Actually in two of my courses I don't have that.”
In her response, Karen alluded to the course community that consisted of a learning space that encompassed all the current sections of the course and involved a community blog and individual blogs for each learner and instructor. Although she happened to be the only student in a particular section of the course, she had access to the course blog as well as the blogs of all learners currently taking the course where she did have opportunity to interact outside the “walls” of her specific course section. Karen noted the value of this component of the course especially in the case of students who may happen to be in a small section of the course. Therefore, the course size need not necessarily have a direct impact on the quality or frequency of the interpersonal interaction. Yet, Karen mentioned that in other courses she has taken where she in fact isn’t the only student, the feedback from other learners was lacking in those courses as well. She provided more insights into her unique learning experience as the interview continued.
Participants were next asked to identify what they perceived to be the most beneficial activities of the course. The unanimous consensus among participants was that the hands-on activities involving practical application of new skills and knowledge were most beneficial. Sue defined these assignments collectively as, “The total package of practical application.” Karen further elaborated on the necessity for these assignments, stating,
If you had not required me to go and subscribe to blogs and podcasts, I probably wouldn’t have done it. Because, you know, you’re busy, you’re working and I probably just wouldn’t have done it and would never have realized the value.
The pertinence of such activities may have been further exacerbated by the technical nature of the course in which the objectives involved learners demonstrating proficiency in a host of advanced Web communication skills. Yet, educational best-practices commonly reiterate the value of application and reflection activities no matter the discipline or subject matter being discussed (A. K. Ellis, 2001; Jonassen, 1991; Vrasidas, 2000).
All participants confirmed that they did in fact make use of the course community, CMUOnet. This was anticipated as participation in CMUOnet was mandated by the course design, as many of the required course activities involved learners utilizing the various tools of CMUOnet.
When provided the opportunity to elaborate upon the their experiences with CMUOnet, participants shared overwhelming positive responses of the availability of the tool regardless of whether or not they made active use of the full suite of communication features. Robert mentioned, “I enjoyed and appreciated the opportunity to use it. I just didn't find that much need to use it at this time.” Pam stated,
I would say it was good to have. I enjoyed reading (the postings). I wouldn't say that it was critical to have that in order to learn what I learned. It was good to know that I wasn't the only one struggling.
Amy enjoyed the networking opportunities, stating, “It's kinda like you read it and then find out what some others, not just their blog but also find out what they've been doing.”
John echoed Amy’s comments, stating, “I know we had to post assignments and stuff on there but I went on and read some of the other learners the stuff that they had put on there and checked out some of the things that they had linked to.” Brenda mentioned yet another facet of the community, noting,
I guess to me it was that nice safe place to start. Whatever you did just took place between you and the other students. It was a nice safe place to try things before you went out and did it online live.
Karen found the course community to be much more than simply a helpful add-on to the course, explaining, “It was helpful to me because I didn't have any other students. So when I really wasn't sure, am I doing it right, I kind of went through and said, ‘OK, they're having the same issues.’” In the instance of students who are in a very small course section, the course community does in fact serve as a venue for interaction and social networking to take place beyond the restricted course environment consisting of the participants in a single course section.
One primary component of CMUOnet was the course community and blog where all members of the course community could post and comment. All participants reported making use of the course blog. Again, this was expected as the course blog was an obligatory component of the course requirements.
Experiences expressed by participants concerning the course blog were very similar to that of the course community in general. Some noted just using the course blog as required by the various course assignments while others went beyond what was required to use the course blog as a way of networking with others in the course. Participants’ comparisons of the discussion board to the course blog are provided in Analysis of Research Question 3 later in this chapter.
When asked whether the activity of bookmarking relevant links added to the course experience, participants noted unanimously that the requirement of engaging in the activity of social bookmarking to share resources relevant to the course subject matter with others was of great value. The course utilized social bookmarking activities in several different ways allowing learners to not only easily access shared resources from the instructor but also to tag and share resources they found throughout the course as well as their assignment samples that they wished to share with their peers. While a fabulous component in theory, not all participants were as enthusiastic about its utility. Pam noted, “I did it because I had to do it but with my time constraint I did not have a chance to glean anything from it… I just haven't had time.”
It was VERY important to me because it was so easy to tag things. Once I figured out how to get them under a certain topic, I found myself going back day to day using it for this or for that. Yes, it's definitely something that I used then (during the course) but I will definitely use all the time.
Karen echoed Sue’s comments, stating, “Everything I found and everything I'm still finding, even if I don't have time to deal with it, well I just bookmark it.” Christine was equally enthusiastic in regards to the value of the social bookmarking activities, pointing out,
It saved so much time when you're actually looking for Web sites and that sorta thing and even like looking at other people's bookmarks and that kinda thing. So it's really a great resource to have. I really, really enjoyed that.
After reflecting on their overall learning experience, participants were asked to reflect upon the nature of the self-paced learning approach. In particular, the fact that they were able to proceed through the course materials and activities at their own pace. Participants were asked to share what they felt were the best parts of the experience as well as what were the hardest or most difficult. These reflections were especially illuminating as they provided greater insight into the forms of interaction in the self-paced learning environment that learners engaged in most found to be most beneficial.
Participants unanimously mentioned the flexibility afforded by the self-paced format. Pam praised the self-paced approach, noting, “It leant itself to my schedule which is challenging at best. As opposed to having to actually go to the classroom on a specific day at a specific time.” Dan referenced the fact that new course sections begin monthly, as he commented,
I think the best part is I get to pick kind of the pace. I know that's even how I picked even when I'm going to go through the course. I look at my calendar and say I've got a month that I can use morning times to work on the assignments. So that was kind of nice and obviously online I'm a huge fan of that just cause then I can plug in when I need to and go as fast as I want, get a couple extras done, so that's always very good for me.
John also noted the benefits of the self-paced online format, mentioning,
I personally enjoyed the self-paced. I understand that it's probably not for everybody but I enjoy that because sometimes it's hard to block the same time every week without things coming up. So, being able to go home, put the kids to bed, and then work for an hour here or an hour on my day off, I enjoyed being able to do that.”
Sue exclaimed,
I love it that if I want to set down at midnight and work on something that I can. I like that I can move ahead. I like that if I'm feeling like I'm falling behind that there's a chance to move ahead but it's ok if I'm behind today because probably next week I'll be where I need to be. So, it has definitely taken that pressure off. I like to have the convenience of choosing when I'm going to sit down because that's the only way that I think I would be able to use it because there's too many things going on.
When asked to mention what perhaps were the hardest or most difficult aspects of the self-paced approach, participants noted self-discipline as well as simply finding the available time as chief oppositional factors. Sue stated, “The hardest and most difficult was finding the time.” Christine pointed out the importance of being self-disciplined in a self-paced course, noting,
Probably just lack of self-discipline. If you're not a self-disciplined person then it's not a great option but like I said there's still plenty of time allotted for it so you should be able to get it down in time.
Along those same lines, John said, “Probably the hardest thing is at times just getting started and getting going.”
When participants were asked if they found the self-paced nature of the course to be either a help or a hindrance, the unanimous consensus was that the self-paced approach was definitely a help and that if given the opportunity to take the course over in either a self-paced or instructor paced format, self-paced would be the preferred format of choice.
Karen elaborated on her choice, noting,
For me personally it was a help. It allows people that are in, that are not just students, that are people working full-time, have families, it allows them to be able to go to school and get the training that they want.
Pam referred to her individual experience in the self-paced course,
For a lot of the unit presentations, there were a lot of those that I actually listened to at ten-eleven o'clock at night because that's when I finally got everyone in bed and could pay attention to what I was listening to.
Christine likened the self-paced format to that of online learning in general, stating, “Like I said, just the flexibility of it and even kinda also you can take it anywhere. Like I can do it while I'm at work or while I'm at home or whatever so it's great.”
To begin a series of questions regarding interactions with the instructor, participants were asked to rate how involved the instructor was in the overall learning experience. Robert noted, “I felt you were very involved. Anytime I didn't quite get something you were there and you tested out everything that I posted.” Pam concurred, noting, “There was not a question that didn't get answered in a quick, honestly quicker than I had even anticipated.” Sue spoke of the level of instructor presence, commenting,
I actually felt like I was in a classroom with you… I felt like you took a personal interest in me, whether you did or not, it doesn't matter. The point is that when you would make just the little comment, like on the grades in the grading area where you posted your little comment. I actually felt like I had turned in a paper and you had written that.
Participants were asked to continue discussing their experiences interacting with the instructor in more informal ways throughout their learning experience. When asked if they had communicated with the instructor outside of the course room, such as via e-mail, phone, or CMUOnet, the majority confirmed that they had communicated with the instructor either via e-mail or the blogging capabilities of CMUOnet. In a follow-up question inquiring as to how such communications varied from those in the course room environment, participants commonly noted that such interactions were more personal and informal.
A variety of emerging communication technologies were incorporated into the course, including such approaches as blogging, podcasting, and social bookmarking. The majority of the participants confirmed that they did in fact take read and enjoy the instructor’s personal blog, as depicted in Table 3.
Another very popular element of the course experience was the instructor’s collection of shared resources. This collection included bookmarked resources via del.icio.us, photos on Flickr, and videos on YouTube. All participants reported making use of these external resources compiled and shared by the instructor.
These resources served as examples of how the skills being learned in the course could be applied in a practical ministry setting. The utility of such resources may certainly have contributed to the reported popularity of this course component. Sue mentioned in regards to the instructor’s shared resources, “It helped me to see that what I was learning, how I would be able to use and it was practical.”
When asked if they had ever tried contacting another expert apart from the course instructor or their fellow learners, the majority of the participants noted that they did not take this step. Rather, they predominantly felt content with the instructor’s expertise and responsiveness.
When asked to rate how involved other learners were in the online discussion in Blackboard, responses were mixed. Some felt that ample discussion took place while others mentioned that the discussions lacked much extra participation other than what was required. Most noted that they did in fact read the “I Learned” statements posted by other learners after posting their own statement. A few indicated that they did periodically read the statements of others prior to posting their own in order to get an idea for the quality and substance of the postings of others. No one chose to respond to the “I Learned” statements of others but rather perceived the reflection requirement as simply a place for learners to post their reflections for the instructor to read and for a grade. It is worth noting that learners were not required to respond to each others’ reflection statements but it was hoped that such interpersonal interactions would blossom naturally. This unfortunately was not the case.
Participants were asked to also reflect upon the various informal interactions that they engaged in with other learners throughout the duration of the course. Responses were mixed concerning whether or not participants took advantage of various opportunities to connect with other learners outside of the formalized online learning environment as evidenced by Table 4.
Amy was among those who were not aware of the full potential of the course community for facilitating communications outside of the specific course section that they were enrolled in, noting, “No, because I didn't even think we anybody was doing it. I never thought of doing that and I didn't think that anybody else doing that.” Some took advantage of the opportunity to e-mail or comment on the blogs of others, whereas others simple couldn’t afford to take the time for such additional communications.
Many did read the blogs of their fellow learners in order to see not only examples of the various kinds of assignments being accomplished but also as a way of connecting in a more social way with peers. Table 5 displays the frequency of participants who reported reading the blogs of other learners.
In addition to being provided access to read the personal blogs of other learners, learners were afforded the opportunity to comment on any of the individual postings. Table 6 displays the number of participants who reported commenting on the blogs of other participants in the course.
While many did in fact read the blog postings of others, they were not as inclined to post comments to peers. This resulted in the blog communications being primarily a one-way means of communication rather than reaching a truly engaging level of interaction.
Participants were asked if they recalled anyone commenting on any of their blog postings. As the instructor for the most part made an effort to comment on at least one posting of each student, participants responded primarily that they did receive a comment from the instructor but many didn’t receive comments from anyone else.
For the most part, participants reported containing their interactions with other CMUO learners to those enrolled in their specific course section. However, the participant in particular who was alone in August section reported that she did take advantage of the course learning community to see what other learners taking the course where doing, etc. Table 7 displays the frequency in which participants reported communication with learners in other sections of the course.
Those who did not attempt to communicate with learners outside the course section were asked to elaborate as to why they chose to not do so. A lack of time was the most commonly cited mitigating factor inhibiting participation.
Along a similar line of questioning, response was mixed when inquiring as to whether participants took the initiative of subscribe to blogs or access resources of individuals outside of the course to help understand concepts covered within the course. Table 8 shows the number of participants who chose to utilize outside resources to better grasp course concepts.
Sue clarified the reason why she did in fact make use of available outside resources, noted that, “For me it wasn't so much to better grasp what was in the course but to again build on what was in the course...to see what else was out there.”
In terms of Research Question 1, the analysis of responses from participants leads to the conclusion that participants engaged most frequently in interactions involving either the course content or course instructor, on both formal and informal levels. Such interactions support the notion of the influence of the “network” and “collective” proposed by Anderson and Dron (2007). Such emerging influences extend beyond the confines of the defined learning space and can potentially serve as an option for learners and designers alike to leverage within the self-paced learning environment. The Interaction Matrix displayed as Figure 10, has been developed as one potential model for considering the incorporation of the holistic interaction sources available to date. The Interaction Matrix will be provided and described in further detail in chapter 5.
Analysis of Research Question 2
Research Question 2: What forms of interaction do adult learners value most in self-paced online courses?
Participants were asked to rate on a scale of 1-5 with 5 being most important and 1 being least important 30 different elements of the course. These elements varied from specific technological tools employed to pedagogical components of the course experience. Of the 30 course elements that participants rated, Tables 9-11 depict how collectively the respondents ranked each of the thirty course elements they were presented with. Rankings were computed by multiplying the number of responses at each rating level by the corresponding rating numerical value and then totaling. Relative frequencies and rankings are displayed in three sections of importance: highest, intermediate, and lowest. Table 9 displays the course elements ranked by participants as having highest importance.
Of the top ten elements ranked highest by participants, they all involved interactions either with the content or instructor. Table 10 presents course elements ranked by participants as being of intermediate importance. Again, the course content and instructor interaction items were ranked among some of the highest elements.
Table 11 contains the course elements ranked by participants as being of least importance. While participants ranked the majority of course elements as being at least moderately important, they consistently ranked elements involving interactions with other learners as lowest in comparison to all the possible choices. The narrative responses by participants to the remaining interview questions served to validate this trend, as participants indicate their preferences for quality content and instructor interactions and the necessity for such components in preferred self-paced online learning experiences.
Participants overwhelmingly rated interactions with the instructor and quality course content as the overall most important aspects of a self-paced course, with interaction with other learners trailing behind. Such sentiment directly supports previous research emphasizing the importance of the roles of the instructor and content in the overall quality online learning experience (Abdulla, 2006; Heinemann, 2003; Perry & Edwards, 2005; Strachota, 2003; Su, 2006; Yang & Cornelious, 2005). Simultaneously, these findings refute the notion purported by some that collaboration with fellow learners is either preeminent or can potentially compensate for well-designed content and active instructor involvement (Anderson et al., 2005; Rovai & Barnum, 2003; Russo & Benson, 2005; Tu & McIsaac, 2002). Participants noted that while they enjoyed the interactions with other learners and often wished that there was more interaction among learners taking place in the online learning environment, they conceded that in the self-paced environment that such interactions are often more challenged. The qualitative rankings applied to the aforementioned course elements correspond with the qualitative responses that participants shared concerning their values and importance of various course elements.
When asked what types of interactions with the instructor are most beneficial, common responses included communications either via e-mail or blog. Robert preferred e-mail, stating, “From a personal preference standpoint, I prefer to e-mail back and forth just 'cause its quick, it's easy, you don't have to do anything special to get them.” Christine further stated, “It just provides enough room, or as much room as you need to explain something and then also I'm checking it regularly.” Amy agreed, remaking,
Definitely your personal e-mail. That one is the most important. I think also your blog is also very important. Your presentations are important because otherwise we would not have comprehensive learning starting from that. But I got that out of your blog and definitely your e-mails.
Karen didn’t select one communication tool in particular, but rather opted for a mode of interaction, stating,
I think that personal feedback, to me, is most important because it makes you feel like you just didn't go to a library and check out a book, but that actually you are learning and have somebody that you're responding to. So, more so than just a presentation that you see. Knowing that there is that personal contact, that personal guidance if you need it.
Karen’s comments in conjunction with all the others reiterate the value that participants place in the connection with the instructor throughout the self-paced learning experience.
When asked to comment upon the value of the instructor’s postings in the course environment to the overall learning experience, participants again found such contributions by the instructor as beneficial. John stated,
Yeah, I found 'em good as you answered other people's questions. If I had the same question or was wondering about something. Or a lot of times you'd post things like resources and just hey, I found this, check this out. I thought they were all good.
Brenda agreed and noted, “Yeah. When you put them all together, yeah, they are very valuable. Because even there were times when you would just post your general announcements and things. I found that really helpful.” So, despite the apparent non-use of elements of the formal course environment in Blackboard, participants in general appreciated the instructor’s contributions in that communication venue.
As instructor feedback was rated as such an important aspect of the self-paced learning experience, similar questions were asked concerning the importance of receiving feedback from other learners. Responses were much more subdued, noting that while feedback from other learners is certainly valued, its perhaps is not a requirement for a quality self-paced online course. Amy explains regarding feedback from other learners,
I can do without it but then it it's kinda nice to know that we have the same problem and then when we have something and somebody wrote back and said ok, you're doing great or I had that problem too. So just very encouraging.
I think it's pretty important. I didn't have a lot of feedback as far as the blog and that sorta thing or discussion board, but I think it's pretty important just reading some other people's feedback on other people's blogs. I thought it was good and they were kinda being honest about their responses and oh this is what you had to do or I think it would be better if you did this.
I don't think it's really valuable to the outcome of your class. Only because I've done six (courses) and I've really not had much of that. But, I've had to admit that there are times when it would be nice.
As a final question regarding formal interactions in the online course room, participants were asked to reflect upon the value of various announcements posted by the instructor at different intervals within the course in Blackboard. Some found these to be of great value, such as Brenda who noted, “I like the announcement area. It's just kinda that initial ‘welcome back’ after you've been logged out and you pop back on. I like it.” Others reiterated the value over the type of communication rather than the mode of technology used to transmit, such as Karen who stressed, “I'd rather have feedback than announcements.” The value of meaningful communications among the instructor and learners can not be overstated as evidenced by consistent calls from participants for feedback from the instructor.
When asked what value the instructor’s blog had on their overall learning experience, participants spoke positively of the blog as not only a means for communication but also a resources for students to refer back to even after the course draws to a close. Mary mentioned,
I think it was an extra nice thing to have, but I also think I remember some of the stuff that was there was actually helpful. I remember at one point there was something I was struggling with and I think I'd e-mailed you and then I went on your blog and funny the answer was there and yet I hadn't even, you hadn't even received the question yet.
The blog really added something to what we're doing…Plus then you have the benefit, like if I start a class in October and I'm going to the instructor's blog, I have the benefit of seeing what past students have dealt with and how the instructor has interacted with them and that gives you a feel for what lies ahead.
Participants were asked to discuss how valuable the interactions with other learners in the course room were to the overall learning experience. Responses predominantly downplayed such interaction within Blackboard. Dan noted that,
It was very, very low” to begin with. Robert elaborated by saying,
Since there wasn't a lot of interaction there I can't really speak to that in depth. But I felt like as far as from a giving standpoint where I could share some of my knowledge and expertise, it was nice to be able to help some other people out.
I think for me it is not really a big thing because I would say about 95% of my learning is or what I receive is from the instructor - the presentations and everything else. So, I wouldn't not say that it's really, really important if I could not have those things - the Blackboard discussion.
Christine disagreed, noting, “I think it's pretty important. It's, like I said, a great benefit to be able to see what other people are learning and how they're learning and that sorta thing and kinda reflect on that.”
When asked to compare how interactions within the course blog or individual blogs compared to communication within the discussion board, participants did note a difference. Sue mentioned that, “A lot of times it would shed more light on whatever a subject might have been at the time.” This may be in part due to the fact that learners were by and large more engaged in the course blog and used their personal blogs more than the discussion board.
Participants noted that the discussion board was mandated whereas the blog was generally a more optional form of communication. Pam made an interesting analogy in comparison, stating,
If there was any variance, I would say the course blog was a little more if you want to describe it, the course blog had more of a ‘friends living room’ feel to it as opposed to the discussion board which was more of ‘meeting at the library.’
Sue expressed the same sentiment, stating, “The discussion board to me, and maybe it was just the way I read into it but to me the discussion board was more of a formal area as far as posting what I learned whereas in the blog is was more like, ‘Wow, did you see?’ like you were actually having a conversation with the person because it was more relaxed.” In general, the terms requirement and formal were often used by participants when describing the discussion board, whereas the terms conversation, relaxed, in-depth, and open were used by participants to describe the discussion board. Responses from participants suggest that the pedagogical value that traditionally has been attributed to asynchronous threaded discussion took place in the more learner-driven blogging context.
Participants were provided opportunity to discuss their various interactions with the various types of content and activities in the course. First of all, participants were asked to describe which aspects of the course content or activities were most helpful. Similarly to the ranking questions asked earlier, participants predominantly enjoyed the instructor-developed instructional multimedia presentations as well as the supplementary external media that was incorporated into the course. Robert noted the utility of the instructor presentations, admitting, “I really liked the slide presentations and was really wanting a way to save those locally so that I could go back and listen to 'em again in the future.” Mary pointed to the practical applications assignments, claiming,
I could have done all the reading…and I could have watched all the presentations in the world. But until I actually went through it and messed it up and went through it again and did everything again to actually get it, I would have never got it. So definitely doing it.
I think it comes down to the practical application things. You can read all you want to read to read. You can answer those questions on what the textbook says. But until you actually do it a lot of times that is when it like hits. I think it makes sense.
It was therefore no surprise that when asked to discuss how valuable the instructional presentations were to the overall learning experience, the response from participants was overwhelmingly positive. Robert stated, “Those were very valuable. I felt like they brought more stuff to the course than any of the other instructional materials did.” Mary concurred,
You're going step by step and actually that's great because when you actually get to the place where you're supposed to be doing this you already have a mental picture of what I'm supposed to be seeing next, what I'm supposed to be doing next. So I think that they're very, very important as well.
Another component of the course instructional content was unit introductions in the form of podcast episodes. These short audio recordings offered a brief introduction to the unit and outlined the various activities that learners would be engaging in. While participants seemed to agree that these unit audio introductions were nice to have, they were not deemed as important the other more formal and in-depth unit instructional presentations. Dan responded, “I thought it was more of just an exercise actually I thought that's the reason you did it.” Karen stated, “I thought they were helpful. They went over the objectives of what was going to happen and I had a perspective in mind, so I knew what I was looking for.”
Yet another component of the course content included required textbook readings. Participants shared the full spectrum of responses regarding the textbook readings with no apparent trend distinguishable. Robert referred to the textbook readings as, “a little bit dry” while Dan’s comment was, “Excellent, great book.” Pam reflected by saying, “You covered I'd say 98% of it in the course presentations within the Wikipedia readings you had us do. So again, it was a little redundant but good.” The single recurring sentiment concerning the course textbook was that it could possibly be replaced by many of the other online resources that were incorporated into the course.
Participants were asked to then discuss how valuable the practical application assignments were to their overall learning experience. Again, consistent with previous responses, participants unanimously viewed these activities as integral to the success of the course and in fact were identified as potentially the most important component of the course. Sue stated, “For me that was the most valuable of all.” Karen further elaborated in terms of the practical application assignments by stating,
The course would have been wasted for me without them, because I wouldn't have done em'. I thought it was a key because I would have read everything and said, ‘Oh this is great. I can use this’ and it would have gone on the back-burner and the day to day demands. But because I had to do, I was gonna make it applicable to me.
The practical application assignments certainly have been shown to be a crucial component to this course. It remains to be seen if this is the case in other self-paced courses or simply in this particular instance.
In concluding questions pertaining to participants’ experiences with various interactive components of the course, participants were asked to reflect upon their experiences interacting on a more informal level with various types of content. When asked about whether they found other external resources such as text, videos, tutorials, or other helpful Web sites during the course, most confirmed that they did in fact come across other helpful content. Mary described her experience, sharing,
Yeah, I did. A few things I saved in my bookmarks. A couple times I was really struggling as you know and I would set aside time late in the night to do it and then I would get stuck. So, of course I couldn't reach anybody to help me at that point. So I would start looking up some stuff to try to help me to figure it out past that point so I could move on.
Others pointed to simply the fact of such external resources being accessible after the course ends and that they served to be a just-in-time learning aid. In general, participants found such external resources to be a valuable contribution to their overall learning experience.
When participants were asked whether they preferred the external resources to the instructional presentations and various content resources in the course, participants still overwhelmingly preferred the course-specific content. Robert noted,
The step-by-step, it seemed like you gave step-by-step instructions to several of the things and you were really able to tie some of those things into what you could use them in the children's ministry whereas the other stuff was just generic ‘ok this is blogging, this is RSS’ and things like that.
Karen agreed, stating, “My preference would be the instructional presentations specific to children's ministry. Because I could always find those on my own, but I couldn't find the specific to children's ministry.”
After having been given the opportunity to reflect on the various individual interactive elements of the self-paced online course, participants were finally asked to reflect upon their experiences in sharing their preferences for interaction. First and foremost, participants were asked to identify which of the different types of interaction that they believe to be most important to the overall success of a self-paced online course. Participants overwhelmingly selected either interaction with the content or interaction with the instructor as most important to the overall success of self-paced courses.
When asked why they made their particular selections, participants shared candidly their rationale. Robert said,
Probably the interaction with the instructor and again just because the instructor is the expert and when I'm going to a course or studying I want to talk to experts so that's probably the most important interaction that I could get.
I think the quality presentations with examples because it allows different levels of learning. You could either take it for what it was at face value and do what you needed to do for the basics or you could go deeper and try and figure out ‘Oh my goodness, how much deeper can we go and what else can this be used for?’ The various applications that you can have. So for me, the presentations with examples were the most important.
Because it's a self-paced course, I would say the interaction of the content. Because if it's self-paced and it was just reading and not applying you could just read through all that in no time at all and never apply it. But that's the danger of a self-paced course. You're just working towards the end goal and then you go to apply it at some future date, which you never will.
Brenda summarized by saying, “To be honest with you, what I saw to start with what was the most valuable was the instructional presentations. Whether you made an actual connection with your instructor, you felt like you did.”
The analysis of responses from participants regarding Research Question 2 leads to the conclusion that participants value quality interaction with the instructor and course content as being most important to the overall success of a self-paced online course. Through not only ranking of importance but also in rich narrative dialogue, participants expressed their preferences for quality interaction with the instructor and course content over interaction with other learners. Data presented in Tables 9-11 displayed the preferred importance participants placed upon the various elements of a self-paced online course. Subsequent interview results reported throughout the remainder of chapter 4 support the quantitative rankings that participants assigned to the various course components and forms of interaction. Adult learners reported that while a balance of all interaction forms are most desirable, they value most the interactions with the instructor and content. Participants concurred that well-designed application exercises fostering meaningful interactions with the course content were crucial to the success of their self-paced online learning experience.
Analysis of Research Question 3
Research Question 3: What forms of interaction do adult learners identify as equivalent in self-paced online courses?
Participants were asked to compare the course blog to the Blackboard discussion board. They were asked to share which communication tool they preferred as well as to whether they felt both were necessary or if one could replace the other. The design of the course involved a few redundant communication tools, of which the discussion board and course blog were among those who did overlap in many ways. The course blog, while accessible as a standalone communication tool as part of the learning landscape CMUOnet, was also integrated into the structured learning environment in Blackboard and was accessible from the course navigation menu as depicted in Figure 6.
Robert admitted that he was very experienced with asynchronous threaded discussion and compared the discussion board and course blog as follows:
The discussion board has the advantage of being easy to post things to and you've got definite threads within it. I can see for a user who is unfamiliar with blogs and kinda doesn't really know how to do the things there I could see those users really liking the discussion board. In fact, it was a whole lot easier for me before I started this course to post things to the discussion board than it was to the blog. By the time I reached the end of the course, I preferred to use the blog.
When probed as to why the change in preference by the end of the course Robert referenced the distinction and yet similarities of the individual versus course blogs and stated, “It just seemed like it was the easiest place to post things and links to. I could something either on my personal side or switch over very quickly to the course side and kinda do 'em both there.”
Pam noted the similarities among the tools and didn’t recognize many notable differences, stating,
Really, they're basically the same thing. I had been familiar with course discussion board because of my previous courses so honestly, just in all fairness, I think I probably preferred that one over the other just because I was used to it. Not, necessarily that one was better than the other. For the most part, the information was the same. I don't know that I would have both. I would kinda do a one or the other.
Sue found the discussion board to be more of an assignment submission tool rather than communication mechanism. She mentioned,
As far as the discussion board, I felt for me that it was just a place to put an ‘I learned’ statement, not to actually get involved or interact. But, and I know it seems ironic, but I felt more comfortable with the blog because to me it felt like it would be more one-on-one even though several people could be reading it.
I like them both because they are both very different. When you're going on the discussion board, it's very much targeted to this certain chapter. But when you go on the course blog, it's a little more in general and I think you kinda feel like the course blog is more like between students and the discussion board is more between the teacher and the students.
When asked whether they felt as though they had the support of the instructor and other learners throughout the learning process, participants unanimously concurred that they did in fact feel supported. When asked what made them feel that way, participants frequently noted responsiveness to questions as well as a sense of camaraderie among participants as key factors. Pam boiled it down to simply, “Quick responses to any e-mails or questions that I had, both from you and from other students.”
Next, participants were asked whether they felt that the different types of interaction discussed are in any way equal. Consistent with previous responses, many viewed interaction with content and the instructor as either equal or very close to being equal but felt that interaction with other learners was inferior in some respects. Robert stated, “Probably the interaction with instructor and the course content is actually probably pretty equivalent.” When asked why he made this claim, Robert responded, “Again, it goes back to the expertise of the interaction. Getting as much information into my hands as possible.” Amy explained her position as follows:
I think the instructor one and the general presentations like the one you did that those are probably equal although they're different way because without the general instructions if it's a self-paced and you said ask me any questions and I can help you, it would not be useful because I would not know how to ask you questions. So, I mean if there's a course in such as an expert would say ok I'm here and I will help you with whatever questions in advance. Well then that course would not be good for me 'cause I would not how to ask but so those are two equal of importance.
Along a similar line of questioning, participants were asked if one type of interaction is more important than others and whether that type of interaction could be replaced by an increased level of the others. Dan’s response contradicted others, noting,
Out of the content, instructor, others, if any, the instructor part could be limited down, I know that sounds very strange, but from the standpoint of increasing possibly some of the content or others. In an online course, I don't view the instructor as the main, I don't mean any disrespect, but the instructor is really just more on the facilitator side and as long as I've got comment and sources coming in I'm able to do what I need to do without the instructor.
Robert noted the necessity of both the instructor and content interaction, stating, “I really don't...I can't say that instructor interaction or course interaction is either one greater than the other, they're just different. No, I don't believe that either one of those could be replaced by increasing anything else.”
Participants were next asked if interactions with the instructor were to be either diminished or eliminated from the course if an increase in other types of interaction would fill that void. While participants stated earlier the importance of the instructor, several conceded that it would be possible to in a way diminish the instructor interaction to an extent if learners were more engaged and if the instructional content was of superior quality. Yet, many were hesitant to agree to any degradation of the instructor interaction in exchange for increasing others.
Next, participants were asked to identify whether they believed interactions with other learners could be diminished or eliminated from the course and adequately compensated for by increasing other types of interaction. Of all three types of interaction, the interaction with other learners was by far the most dispensable. This may in part be due to the fact that for this particular course, it was the weakest of the three types of interaction utilized. Robert supported his position, stating, “Again, I didn't, as far as this course, I didn't do a whole lot of interaction with the students so it's not gonna really make a difference in what I took away from the course.” Dan agreed, noting,
I definitely think content could fill that void. Like I've stated before, there really wasn't a lot of interaction with other students. That's why I don't think it wouldn't make from a student's standpoint on my side it doesn't really make a difference even how large the course is. How large the class is irrelevant to me.
I found that the students and other students in the course that it was just more of like encouragement and support and so I think that was pretty important. The course podcast and the course instructional videos they can't really provide that. So I thought that the students like the feedback from the students and that sorta thing was very important. So I don't know, it again is like they all have their own place and it's not they're all very equally important so it's hard to.
Finally, when asked if content could be diminished or eliminated from the course by being compensated for by increasing other interaction types, participants unanimously denounced such a notion, claiming that the content of self-paced course is truly indispensable. Participants mentioned both formal and informal content sources as very important, noting that the optimal self-paced learning experience involves a balance of all available interaction sources.
In summary, in terms of Research Question 3, the analysis of responses from participants leads to the conclusion that adult learners identified interaction with the instructor and content as very nearly equivalent in a self-paced online course. Participants pointed out that quality interaction with content is indispensable in the self-paced learning environment and can not in any way be replaced, while interaction with the instructor could potentially be diminished to an extent and compensated for through increased quality interactions with content or learners. Participants further noted that while interaction with other learners is desirable within the self-paced learning environment, the self-paced nature of the course makes such interactions challenging and therefore participants were willing to forgo such interpersonal interactions deemed by some as more tangential in exchange for the flexibility afforded by the self-paced learning approach.
Analysis of Research Question 4
Research Question 4: What impact do adult learners perceive interaction to have on their self-paced online learning experience?
To catch a glimpse of learners’ perceptions of the impact that interaction has on the self-paced online learning experience, participants were asked if they ever felt isolated from the rest of their classmates or all alone. Robert mentioned that to an extent he did feel a bit isolated, stating, “From the rest of the class, I kinda felt like I was sorta on my own. On the other hand, from you as the instructor, no I felt like you were there the whole time.” Pam’s experience however was different, responding in regards to whether she ever felt isolated, “Not unless I intentionally tried to do it, no.” John made an interesting statement, “No. It was one of those things where I didn't always need the contact but in the back of your mind, you knew it was there. You knew that you could fall back on that, and that was good.”
Questions concerning the nature of the self-paced approach to the course were concluded by providing participants with the theoretical choice between taking this course again in its self-paced format as compared to a more instructor-paced format, in which certain deadlines had to be met completed by a specific dates. As might be expected, participants unanimously chose the self-paced format.
When asked why the choice for the self-paced format, participants predominantly mentioned the flexibility factor. Pam defined this aspect as, “The level of grace afforded for completion of the work” whereas John described it as, “You can tackle it as time permits.” Karen elaborated on the value of the self-paced approach for her, noting,
My time is so limited and so I like the self-paced because I can still do it without being totally stressed out. I like the fact that, with the self-paced, I wasn't just turning in to you an assignment to meet the deadline, which I would have done. But because it was self-paced, I made it applicable to my situation and I'm actually using what I did.
Participants expressed consistent positive regard for the self-paced approach of the course largely due to the flexibility that such a format affords. Questions shifted to query the experiences of participants regarding the different types of interactions they engaged in and their preferences for each.
Participants concurred that feedback from the instructor was very important to the overall course experience. This sentiment may have been elevated by the fact that this particular course was very technical in nature, requiring technical assistance from the instructor at various intervals. Participants noted that their experiences interacting with the instructor within Blackboard was limited, as primary interactions took place with the instructor’s blog, course blog, or in responses to questions submitted via e-mail.
Next, participants were asked to share their preferred method for having the instructor interact with them. E-mail was overwhelmingly the most popular choice with the instructor’s blog also receiving a substantial positive response. It appears that familiarity is one of the primary reasons why e-mail was preferred, as it is the most ubiquitous online communication technology for this population of adult learners. Pam points this out, noting, “Now that I've completed the course and understand the blog, probably the blog. Prior to this course, it would've been e-mail.” Brenda also chose the blog and explains why her preference is not the discussion board,
I think the blog is a nice place to start. The discussion board is kinda cold, to be honest with you. I guess you don't honestly see a lot of reaction from the instructor where actually the blog would be a neat way to get more interaction.
Conversely, participants were then asked to indicate how they prefer to interact with other learners. Blogging edged out e-mail for pre-eminence among learners. Brenda summarizes the responses received, stating,
Blogging honestly yeah would be my first choice because like I said the discussion board it just kinda feels cold. You are there to answer the questions required by the class, where the blogging gives you the freedom to say, "Ok I took this away from that class. Did anybody else get that?"
For many participants, the blog had a user-friendly appeal to it and once participants grasped the concept of subscription via RSS, they found consumption of postings and responses much easier and less time consuming that the course discussion board. Also, the blog software powering CMUOnet includes a feature whereby new comments to a blog posting are automatically delivered via e-mail to the author of the original posting. The convenience factor was certainly a notable reason why the blogging was valued higher than the discussion board for informal interactions among learners.
Next, participants were asked to consider the size of a self-paced online course in relation to the quality of the overall learning experience. Participants were asked to discuss how important in their opinion the size of the class is to the overall learning experience and whether a self-paced course could be too large or too small. Responses included an interesting mix of perspectives. Dan noted, “No. For me as a learner I don't think it makes much of a difference to me. I have no idea how many people have plugged in.” Karen, who had the unique perspective of being the only student in a section of the course, didn’t feel that being the only student negatively imparted her learning experience. On the contrary, she mentioned in regards to class size,
I think it can be too large. I think it can be too large because of the larger the class is the greater demand on the instructor and the less personal attention you can get from instructor...So I think it can be too large. Do I feel like I was hurt by being the only student? I really don't.
As the interviews drew to a close, participants were asked to consider the impact that interaction has on the quality of the online learning experience. Participants in general identified a correlation between the quality of the interaction in a self-paced online course and the overall quality of the learning experience. Robert explained,
I think the quality of the interactions directly affects the quality of the overall learning experience. If you halfway answer a question or the instructor halfway answers a question then the student is not going to get near as much out of course but it seemed like you went over and above to make sure that we understood what we were doing.
I would say it definitely impacts the quality because if you didn't interact at all then the quality of learning experience would have been just baseline. And the more I interacted with your blog, the course blog, and maybe following rabbit trails to other Web sites it gave me more information that I wouldn't have gotten had I not interacted in that way.
Dan was a bit less supportive of the importance of interaction, but affirmed a utilitarian perspective, stating, “Again, I think it's just one of those that I think while you're doing the course, as long as I can get answers when I need answers I think that was good.”
In analyzing the responses from participants regarding Research Question 4 it is recognized that interaction invariably may have a direct impact on the self-paced online learning experience. Participants reported their experiences regarding the various types of interactions they engaged in throughout the duration of a self-paced online course and pointed to the important part that the interactions with the instructor and course content played in their overall online learning experience. While interaction with other learners was not the most influential component of the particular course studied, the logical assumption may be drawn that in other courses where interpersonal discussion with other learners is in fact maximized, such interactions could also be equally important. Further study will be necessary to explore whether such hypotheses are in fact confirmed. More recommendations regarding future studies will be shared in chapter 5.
Summary
This study explored the dynamics of interaction within a self-paced online learning environment utilizing both rich media and a mix of traditional and emerging asynchronous computer-mediated communication tools to determine what forms of interaction learners in a self-paced online course value most as well as gain an understanding of their perceptions of the impact that such interactions have on their overall learning experience. The results of this study to be further discussed in chapter 5 provide guidelines for instructional designers developing instructional strategies for online environments when neither the instructor nor course requirements impose pace upon the learners.
Posted by Jason on February 3, 2008
Tags: Chapter 4
Comments on specific paragraphs:
Click the
icon to the right of a paragraph
Comments on the page as a whole:
Click the
icon to the right of the page title (works the same as paragraphs)