This is a draft of a paper which has been accepted for publication in International Journal of Continuing Engineering Education and Lifelong Learning, Vol. 10, Nos. 1&endash;4, 2000.

Issues and innovations in very large scale on-line distance learning:
A case study

Dr Gary R. Alexander

Director, Electronic Media in Education Research Group, Technology Faculty, The Open University, Milton Keynes MK7 6AA, UK

E-mail: g.r.alexander@open.ac.uk, Web site: http://sustainability.open.ac.uk/gary/

Abstract: A new on-line course catering for 900 students in its pilot year and about 13,000 in its full presentation is described. It covered beginning use of computing, collaborative work on line, finding information on the Web, creating Web pages, and skills of learning to learn. Issues discussed are the best learning approach to use, ways of coping with the wide variety of student backgrounds, experience and software, and the nature of the on-line environment needed.

Key Words: Distance learning, on-line learning, large scale, teaching on the Web, computer conferences, collaborative learning, generic teaching

Biographical notes: Dr. Gary Alexander received the BSEE degree from Columbia University in 1963 and his PhD in non-linear circuit theory from Purdue University in 1971. Since then he has been employed in the Technology Faculty at the Open University in the UK, in what is now called the Department of Telematics. He has been involved in the development of innovative methods of distance learning since the early years of the Open University. He has an international reputation as a pioneer in the use of computer communications and collaborative learning techniques in distance learning through his research publications and through participation in such international research projects as the EC’s DELTA project, JANUS project, Café Mondial, and others. Since July 1996 he has been Director of the Electronic Media in Education Research Group. Recently he has been the author and architect of the first module of the on-line course, T171 You, your computer and the Net, described in this paper.


Introduction

The UK Open University, (see footnote) in common with many other higher education institutions and corporate training organisations, has been facing a growing demand for courses providing:

Most of the prospective students looking for these skills are mature adults who have not pursued formal study for many years, thus adding the need for:

To meet this need, the Open University Faculty of Technology decided to produce an ICT foundation course to be taught entirely on-line as part of a complete re-vamp of its first-year course offerings. The course is called T171 You, your computer and the Net, Learning and living in the information age and is made up of three modules:

This paper concentrates on the first of these modules, which covers the skills listed above, although some of the issues to be discussed apply to the whole course.

The course was expected to be extremely popular and has, in fact, proved to be one of the most popular Open University courses of all time. It was first presented in a pilot version in 1999 with student numbers limited to 900 (a very large pilot by many standards) to try out the technological innovations. For its first full presentation, student numbers had to be limited to 9000 (because of difficulties in finding and training tutors) for an initial presentation starting in February, 2000 and a second presentation scheduled to start in May 2000 has already attracted 3500 students. With this prospect, the course had to be designed on a very large scale.

The Open University has considerable experience in the use of computing, and especially computer conferencing, in its courses [1]. By the end of 2000 there will be over 100,000 students using its conference system (based around the FirstClass® software) as part of their courses. However, most of these students will also have access to some face-to-face tuition and will receive most of their learning materials in print through the post. The computer conferences are largely an add-on to the more traditional OU style of distance learning. We intended T171 to be much more fully based around on-line learning.

The OU also has experience in small scale courses taught entirely on-line [2 ,3]. However, T171 was the first large scale Open University course in which:

Although we were able to build upon our earlier experience, many new issues arose through the planned new use of media. We were determined that we would not simply transplant our traditional distance learning techniques to the Web, but would instead develop an approach tailored to its strengths. Too many institutions seem to think that on-line teaching means simply putting lecture notes (or maybe even lectures!) on the Web and adding e-mail.

The principal issues we faced as we designed the course were:

These issues are likely to be common to any large-scale provider of distance education whether in a conventional higher education institute or in corporate training.

 

Educational Framework

The historical models of learning in higher education and distance learning are based around largely individualistic activities, at least in their formal structure. The conventional university model is for students to attend lectures, read textbooks and as a result to produce essays or other assignments and take examinations. Of course, the Oxbridge seminar model is an important exception to this, and in practice, students often study together informally.

In traditional Open University courses, and in other distance learning organisations, the focus is on producing written materials, or broadcast or distributed audio-visual materials that form the basis of the students' learning experience. Students are expected to digest these materials and confirm their mastery of them through written assignments and examinations.

In recent years, collaborative models of education have been taken much more seriously, and have come to be seen as having important pedagogical advantages.[ 5, 6 ] It is even seen as worthy of mention in a British government briefing document on a major new initiative to upgrade training for industry, where it is seen as having the additional advantage of reducing staff input [7].

The educational basis for collaborative learning is rooted in constructivist models of the way people think and learn.

  • "From the constructivist perspective, concepts are not inherent in things but have to be individually built up by reflective abstraction."[8]
  • Learning is not a matter of passively accreting concepts, but of constructing a conceptual edifice in which what is learned is integrated and linked to one's entire life experience. Understanding is not the same as remembering. For 'deep learning', where what is learned can be applied to new contexts and generalised, we are concerned with the higher levels of Bloom's cognitive hierarchy [9].

    In practical terms, this means that students have to manipulate concepts they are learning, turn them inside out, and look at their connections to other concepts. The job of the educator or instructional designer then is not simply to create materials in which concepts are clearly explained, but to create learning situations in which students find themselves actively engaging with the concepts they are learning. A particularly powerful situation is one in which one discusses what one has learnt with peers.

  • "In order to describe verbally what we are perceiving, doing, or thinking, we have to distinguish and characterise the items and relations we are using. This often focuses attention on features of our construction that had remained unnoticed, and it is not at all uncommon that one of these features, when put into words, leads us to realise that some conclusion we had drawn from the situation is not tenable."[10]
  • and

  • "Putting students into groups...compels them to explain their thoughts to one another and this has several advantages: on the one hand, verbalisation requires reflection (upon one's own thoughts as well as upon what the others are saying) and, on the other, students tend to listen more openly and with more interest to their fellow students than to the teacher."[11]
  • This approach to learning has led to the twin pillars in the instructional design of Module 1 of T171.

    One final theoretical point needs to be made: To obtain the benefits of collaborative learning it is not sufficient to put people into groups. Groups do not automatically work well. People may get offended by comments made to them, or feel let down if some members are perceived as not pulling their weight. It is necessary to foster a group ethos, including skills of working together and of giving and receiving comments and criticisms. Furthermore, our understanding is that learning is most effective in a social situation where learners feels appreciated, supported and accepted with all their strengths and weaknesses. [12] We have thus been concerned to develop on-line educational communities that foster these principles.

     

    Issue 1: The learning approach

    We devised a course structure that would enable us to embody the principles described in the last section:

    To provide collaborative working in a course with thousands of students, we had to provide activities at several different scales:

     

    Issue 2: Handling student variety

    The ‘preparatory period’

    There were several aspects of student variety we had to consider. Firstly there was the range of experience in using computers and in previous learning. We handled that by specifying a ‘preparatory period’, before the course started. During that period, students with the least background could begin to catch up with their more experienced fellows. For this initial period, we could not assume students knew even the most basic uses of a computer so we could not present material on-line.

    For the preparatory period we sent students a set of materials to help them get started:

     

    A generic approach to teaching software

    Another major aspect of student variety was that we expected students to have a variety of different browsers, word processors, spreadsheets, etc. installed on their PCs that they would want to learn to use in the course.

    In earlier courses we had specified a particular, low-cost ‘Works’ package for all students which incorporated a word processor, spreadsheet, etc. so that we could give students specific instructions on how to use it and so that they would easily be able to exchange files with one another. We encountered considerable resistance to this package from students who already had other software and were used to it.

    In this course we decided to try to handle the variety students brought with them by teaching a ‘generic’ approach to learning software. We would teach them the general principles of learning new software, combined with exercises taking them through the basic features of any word processor, spreadsheet, graphics software, etc. We saw this as having many advantages to the student. It would enable them to learn to use new versions of their software or entirely new software without needing to take training courses whenever something new appeared.

    The starting point for this was a document describing general strategies for learning software, with hints about toolbars, menus, help files, ways of exploring new software, ways of getting personal support. Backing this up was a set of on-line computer conferences geared to providing help with each type of software. We encouraged students to give and get help from each other through this conference but specialist tutors also staffed it.

    As an example of this approach, we started the section on spreadsheets with a short document called ‘What is a spreadsheet?’ explaining what spreadsheets were for and giving a few basic concepts. This was followed by three practical exercises:

    All of these exercises describe generic features of a spreadsheet. When combined with the support conferences, they were sufficient for students to learn the basics of spreadsheets and also, to develop their ability to learn new software on their own.

    The major exception to this generic approach to learning software was that we required all students to use our computer conferencing and email system based around FirstClass® within the course. That was because there are no common Web-based computer conference systems in general use that were suitable for our needs. The alternative would have been to rely on e-mail and mailing lists, or to set up private newsgroups. The latter do not offer the richness and control we needed. We needed very large numbers of conferences, some of which are restricted to small groups and some to much larger groups.

     

    Issue 3 - The on-line environment

     

    Providing a rich experience

    One of our goals in developing the course materials and the on-line environment was to provide a rich experience for the students. We wanted to offer a variety of materials of different types, partly to satisfy students with different preferred study modes, and also to make best use of the on-line media.

     

    Integrating the components

     

    Given all the different components described above, the challenge was to create a design which

    Figure 5 shows the basic design we came up with for the main part of the Website. There is an overall navigation bar to the left that links to the major activities students will encounter. A local table of contents at the top of the page allows for navigation around the materials related to the current section. Below this appeared the major types of content: activities, documents to read, audio-visual materials, self-diagnostic tests.




    Figure 5 - The basic design for the main part of the Web site.

    We offered the students three ways of accessing the FirstClass conferences:

    We developed a browser interface for FirstClass for the course that was consistent with the look and feel of the rest of the course Web site. It is shown in Figure 6. At this level, it is showing the main T171 conferences:

    Not shown in that view are the student’s mailbox, their tutorial group conference, or a place for their personal home page.



    Figure 6 - The design of the browser interface for the T171 FirstClass conferences

    A member of the OU’s design studio produced the design of the Website, as we wanted a site that embodied professional design standards. At the same time, the authors wanted to be able to write using an HTML editor. They wanted to have the functionality of HTML as they wrote, so that they could incorporate internal and external hyperlinks and feedback devices. They also thought it would encourage them to write in a style suited to the Web (i.e. short punchy sections with a lot of structure rather than long, verbose descriptive passages). We thus needed to devise a way to transform the author’s more basic HTML into the designer’s styles, without too much hand coding.

    The solution was to use a rendering engine called Frontier to strip out the authors’ original HTML tags and replace them with the designers’ tags. The course editor, one of the development team, was responsible for this process, as well as for editing the materials for style and consistency.

     

    Evaluation

    We were concerned to learn as much as possible about the successes and difficulties of the course, and so set up a systematic evaluation of the pilot presentation. We gathered information about it from several sources:

    As a result of the feedback received, some changes were made to module 1 after the pilot presentation. We simplified the structure, reduced the workload, and increased the preparatory time. Then students were surveyed at the end of Module 1 in the first year of full presentation.

    Some of the overall results of the feedback were that:

  •  There were many positive comments and satisfied students, with compliments about the quality of the course and its Web site.
  • Many students who were beginners to computing found the course overloaded, especially at the beginning, but those who persevered were pleased with the results. The drop-out rate was much higher than we had hoped, but not unprecedented for the first year of a new course.
  • There was a mixed reaction to the groupwork. Where the groups worked well, students were very enthusiastic. Some did not work well because students dropped out or didn’t participate actively. In part this was a knock-on effect of the overload which some found.
  • Software difficulties plagued a minority of students and tutors.
  • More specifically,

    1. Of those students who started the pilot and completed payment only 43% completed and passed the course. This is quite low for a level 1 course. With the changes made after the pilot year, it appears that this rate will be substantially higher in the first full year.
    2. Nonetheless, 80% of the students responding to the IET survey thought that the course achieved its stated aims and objectives very well or fairly well and 91% responded very well or fairly well when asked if the course was value for money.
    3. 71% found the on-line tutor group conferences very or fairly helpful and 57% found the on-line support conferences very or fairly helpful.
    4. In the survey taken at the end of module 1 of the first full year of presentation 88% said very or quite a bit when asked whether module 1 was enjoyable.
    5. When asked about their attitudes towards the groupwork components, students said:

      They made me feel part of the group - 30%

      I found them difficult but rewarding - 40%

      I don’t like working in a group - 13%

      I didn’t see the point in them - 5%

      None of these - 12%

    6. 22% of the students surveyed said they would like to see more of the groupwork, but 29% said they would like to see less.
    7. 94% of the students said they found the web site very or quite easy to navigate. As for the technical aspects which worked the best, the web site with 61% and FirstClass with 62% were the highest ranked (multiple answers allowed).

     

    Conclusions

     

    Our overall conclusion is that we have successfully produced a very innovative on-line course at a very large scale. It was generally appreciated by the students but with a certain degree of technical difficulties and difficulties for beginners in the pilot year of presentation.

    Issue 1 - We consider that the activity-based and collaborative approach that we used was vital to the success of the course, although not fully appreciated by all the students. This was a mainstream course for the Open University, rather than a research experiment, so even with the evaluation we did we cannot draw fully scientific conclusions. We do not know of any comparable courses at this scale, taught at a distance, much less any which do not use a collaborative activity-based approach to compare with this course. However, it is clear from the results of the students' assessment that most produced good work and a lot produced excellent work, without any face-to-face support. From the evaluation we are convinced that this would not have been possible without the culture of collaboration and mutual support we so carefully nurtured.

    This is reinforced by the experience of the minority of tutorial groups where the group support did not work as well as it should have. Here there were substantial complaints from the affected students. We draw the conclusion that it is not sufficient to put learning materials on the web, even if they are of very high quality, and hope that students will cope. Collaboration and support make a very large difference.

    The cost of this to the Open University was very high. There was much greater effort required in development work, in support for students and in training and secondary support for the on-line staff than we had anticipated.

    Issue 2 - Our approach to handling the variety of students by providing extensive on-line support and by taking a generic approach to learning software was generally successful. However, in the pilot year there was a significant group that included a lot of absolute beginners who did not cope well with the course. The changes in the first full presentation year helped with this considerably, with its simplified structure and especially with its attention to preparatory work. It appears that for absolute beginners to computing, most can learn successfully from an on-line course with sufficient support. However, there is also a significant minority who would probably be better off learning from a more conventional face-to-face course.

    Issue 3 - Our rich, complex on-line environment with its interactive elements and extensive on-line conference system worked well and was appreciated by the students. However, it required a lot more effort to develop and maintain than we anticipated. With hindsight, a somewhat simpler design would have been preferable.

    Also, the problems of large scale sometimes worked against us, because a small minority of several thousand can still be several hundred students having problems. For example, we included four short audio-visual lectures presented through the Web site using streaming media. A small minority faced problems with the installation of the required plug-ins and they took up inordinate amounts of staff time before their difficulties were resolved.


    Footnote

    *The British Open University is one of the earliest distance learning universities, pioneering techniques of supported open learning since the early 1970s using specially written texts, TV and radio programmes as learning materials. It currently has around 200,000 students and is rapidly moving into on-line learning.


    References

    [1] Alexander, G. (1997) "The Experience of the
              Open University: Pointers Towards a Global
              Network for Education and Research",
              Proceedings of the Twenty-first International
              Conference on the Unity of the Sciences
    ,
              Washington, D.C., November 24-30, 1997

    [2] Alexander, G. (1993) "XT001 Renewable Energy
              Technology" presented at Hypermedia in Vaasa '93,
              Vaasa, Finland, May, 1993.

    [3] Alexander, G.(1995) "Enhancing Quality in Distance
              Learning through Collaborative and Resource-based
              Learning", One World Many Voices, 17th World
              Conference for Distance Education
    , ICDE, June 1995.

    [4] Tien, J.M. (1999) 'A para/professional approach to
              restructuring engineering education', Int. J. Continuing
              Engineering Education and Life-long Learning
    , Vol. 9,
              No. 2, pp. 117-127.

    [5] Davidson, N and Worsham T (Eds) (1992)
              Enhancing thinking through cooperative learning
    ,
              New York, Teachers College Press.

    [6] Kaye, A (Ed) (1992) Collaborative Learning through
             Computer Conferencing: the Najaden Papers
    ,
             New York, Springer-Verlag.

    [7] McNair, S. (1997) Higher Education, Communication
              Technologies and the University for Industry
    , DfEE,
              London.

    [8] von Glasersfeld, E. (1995) Radical Constructivism,
              A Way of Knowing and Learning
    , Studies in
              Mathematics Education Series, no. 6, London,
              The Falmer Press, p.184.

    [9] Bloom, B.S., (1956)Taxonomy of educational
              objectives: the classification of educational goals
    ,
              London, Longmans, p. 207.

    [10] von Glasersfeld (1995), op. cit., p. 188.

    [11] ibid p. 190.

    [12] see examples and discussions in Goleman,
                D (1996) Emotional Intelligence, Bloomsbury.

    [13] Northedge, A., Thomas, J. Lane, A,
                and Peasgood, A. (1997),
                The Sciences Good Study Guide
    , Open University.

    [14] Collins, K., Ashby, A. Courses Survey, Student
                Statistics Team, Institute of Educational Technology,
                16 Feb. 2000.