Hi all
I have completed my report and it's ready for you to view. Please feel free to read and add your comments. Here is the link:
http://milyas.pbworks.com/w/page/34132631/Elearning-Evaluation
Enjoy your holiday!
Best Wishes from
Mareena
Wednesday, December 22, 2010
Thursday, November 18, 2010
Analysing data
I want to call it a success! 87 students took part in my need analysis survey for the Wimba project (Normally they run away when they hear the word ‘survey’). As I was planning to offer online support to our remote campus students, I wanted to know if they have computers at home with Internet access. The survey result looks positive and 98 % of them had computers at home with internet connection. I also wanted to know how confident they were using the computer and 96% of them showed varying levels of confident in using computers (just ok-45.2%, confident-35.7%, extremely confident-15.5%) and only 3.6% mentioned that they were extremely nervous about using computers. Majority of them (98%) said that they would like to get help via the internet to improve their English. Almost equal number of students indicated that they need help with all areas of English such as reading, writing, speaking, vocabulary and grammar. I offered writing workshops to different classes and small groups through Wimba and got the students to do a short poll after the sessions. I will come back with the poll result soon.
Wednesday, November 10, 2010
Where were you Aucklanders?
You missed a good lunch with our special guest Bronwyn on Tuesday 9. The 'faceless' people that I've had contact with all these days became 'real'. Katie, Misha, Dana... are we the only Aucklanders on the course? Bad luck people, you not only missed the food but 1:1 face to face feedback about the evaluation plan... It was quite useful Bronwyn. Thanks for your time.Lovey meeting you Katie and Misha. Good luck with your reports!
Cheers
Mareena
Cheers
Mareena
Tuesday, September 28, 2010
Sha-E-fest 2010
Just returned from sha-E-fest 2010 conference at Waikato Institute of Technology and learned a lot. I was particularly interested in Sarah Stewart’s workshop ‘The best things in life are free’. She mentioned that Bronwyn Hegarty was the co presenter of the workshop but couldn’t attend the conference for some reasons. Genuinely speaking, that was a wonderful collection of free programmes put together for tecno newbies. I need to create accounts and play around with those programmes to get the benefit of it. Thanks for the effort Bronwyn. I enjoyed it.
Tuesday, September 21, 2010
Need analysis
I have developed a need analysis survey using ‘surveymonkey’ and posted the link on my project page for people to access so that I can have some feedback before doing the actual survey. Here is the link if you haven’t visited my project page. Click here to take survey
You can see the overview of my project here
You can see the overview of my project here
Sunday, September 19, 2010
Catching up
I had gone away on bereavement leave for two weeks and just got back to work today. Unfortunately, I did not have internet facilities to keep up with the discussions/ readings. I had a hectic time at the funeral house and hardly had any sleep. My return journey was a nightmare with 15 hrs travel and 10 hrs transit! At the Auckland airport I waited extra two hours as I lost my baggages… I arrived yesterday and had to come to work today though still feeling jetlag…
I don’t know where to start now. I’m far behind the course. I wanted to start my need analysis this week but the ESOL students are on term break and won’t be back till October.
I’m attending the e-fest conference in Hamilton next week on the 27/28 September. Anyone else from our course taking part?
I don’t know where to start now. I’m far behind the course. I wanted to start my need analysis this week but the ESOL students are on term break and won’t be back till October.
I’m attending the e-fest conference in Hamilton next week on the 27/28 September. Anyone else from our course taking part?
Wednesday, August 18, 2010
OIL Module
Module Six
For the second evaluation pair activity, Veronique and I decided to look at Module six from OIL Online Information Literacy. We communicated via email and shared our views.
Module six focuses on specific skill ‘Searching for Information’ in a step-by-step format. The aims of lessons are outlined at the very beginning of the module and they are directly related to the learning outcomes. This is a well constructed module with an easy to use interface and is elegantly simple.
The site allows optimum readability with sufficient white space. There aren’t any long scrolling pages. The format is uncluttered and colour and fonts are used effectively and consistently. The writing style is clear and easy to understand. At the end of each session there are interactive quiz to complete that provide instant feedback. This sequence is repeated for all sessions.
Lessons are organized in non threatening way with small chunk of information followed by graphic demos and animations which make the comprehension easier. Specific terminologies appear in purple when place the mouse over the word/phrases it gives the definition. In addition to that there is also a glossary page at the end of the lesson.
The navigation menu appears on the left hand side on each page offering a choice for the learners to complete the lessons at his/her discretion without following a linear order. There are a graphical navigation buttons at the end of each page which allow the learners to go back and forth from the lessons. There is also handy print option available for selected pages of whole module.
I tried to do some exercises but instructions are not very clear. For example, the help but was not active Truncation exercise. It says, ‘check the ‘Help’ for each search tool to find which symbol to use’. It’s not clear what this is actually referring to.
Some demos took a long time to download so I lost interest and navigated away from the page. The reason could be because I was using a dial up connection. Veronique did not experience any problem as she was on broadband wonder if they picked up this problem during the evaluation. If we are planning to make the course accessible to a wide range of learners then we need to avoid slow downloads.
Veronique also brought a point about timing to work through an entire module. Again it would take longer for dial up users than broadband users. She quoted Alex on the last elluminate session, saying they develop modules of learning that take no longer than 20 minutes to work through. I think it is something we should consider in elearning/teaching.
Lacks of visual and audio technologies are obvious on the site. Highlighted terminologies with definition would have been much better if sound is attached to them. The option of listening to instructions could have been appealing to auditory learners. Catering for a variety of learning styles would help the learners to engage in the lessons. However, I must agree that the site has a thoughtful design, with very useful resources. Incorporating different media types such as sound is something to consider in the future.
Veronique mentioned that the Module 6 is often used by the Otago Poly staff as a preparation exercise before going to the library for a hands-on with library staff. Then they have the online module to refer back to for reference. One concern is that some sections are very specific to Otago University library services. The searching skills gained in this module should easily transfer to searching any library, but for novice learners it could be quite confusing. So it may be worthwhile customizing these with examples from your own library. All the modules can be edited and republished (guidelines are provided in the Springboard module). This requires a bit of technical know-how to accomplish.
For the second evaluation pair activity, Veronique and I decided to look at Module six from OIL Online Information Literacy. We communicated via email and shared our views.
Module six focuses on specific skill ‘Searching for Information’ in a step-by-step format. The aims of lessons are outlined at the very beginning of the module and they are directly related to the learning outcomes. This is a well constructed module with an easy to use interface and is elegantly simple.
The site allows optimum readability with sufficient white space. There aren’t any long scrolling pages. The format is uncluttered and colour and fonts are used effectively and consistently. The writing style is clear and easy to understand. At the end of each session there are interactive quiz to complete that provide instant feedback. This sequence is repeated for all sessions.
Lessons are organized in non threatening way with small chunk of information followed by graphic demos and animations which make the comprehension easier. Specific terminologies appear in purple when place the mouse over the word/phrases it gives the definition. In addition to that there is also a glossary page at the end of the lesson.
The navigation menu appears on the left hand side on each page offering a choice for the learners to complete the lessons at his/her discretion without following a linear order. There are a graphical navigation buttons at the end of each page which allow the learners to go back and forth from the lessons. There is also handy print option available for selected pages of whole module.
I tried to do some exercises but instructions are not very clear. For example, the help but was not active Truncation exercise. It says, ‘check the ‘Help’ for each search tool to find which symbol to use’. It’s not clear what this is actually referring to.
Some demos took a long time to download so I lost interest and navigated away from the page. The reason could be because I was using a dial up connection. Veronique did not experience any problem as she was on broadband wonder if they picked up this problem during the evaluation. If we are planning to make the course accessible to a wide range of learners then we need to avoid slow downloads.
Veronique also brought a point about timing to work through an entire module. Again it would take longer for dial up users than broadband users. She quoted Alex on the last elluminate session, saying they develop modules of learning that take no longer than 20 minutes to work through. I think it is something we should consider in elearning/teaching.
Lacks of visual and audio technologies are obvious on the site. Highlighted terminologies with definition would have been much better if sound is attached to them. The option of listening to instructions could have been appealing to auditory learners. Catering for a variety of learning styles would help the learners to engage in the lessons. However, I must agree that the site has a thoughtful design, with very useful resources. Incorporating different media types such as sound is something to consider in the future.
Veronique mentioned that the Module 6 is often used by the Otago Poly staff as a preparation exercise before going to the library for a hands-on with library staff. Then they have the online module to refer back to for reference. One concern is that some sections are very specific to Otago University library services. The searching skills gained in this module should easily transfer to searching any library, but for novice learners it could be quite confusing. So it may be worthwhile customizing these with examples from your own library. All the modules can be edited and republished (guidelines are provided in the Springboard module). This requires a bit of technical know-how to accomplish.
What & Why of Evaluation
What is evaluation and why evaluate?
When I go to a shopping mall to buy clothes I don’t buy them at once. I walk into a few shops, choose a few items that are appealing, and check the material, design, colour, and size; then walk into to the fitting room with a couple of chosen items, try them on one by one standing in front of the mirror; sometimes get my hubby’s or daughter’s feedback before deciding to buy… Is it part of evaluation where I measure the quality in terms of price…? I wonder!
I browsed the web and landed on wiki which states, citing the American Evaluation Association’s definition as ‘assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness’.
Why evaluate?
Evaluation for me is to learn, learn what mistake have I made and how to rectify them to see the best outcome. Evaluation can be carried out at the beginning of a project to see the needs (need analysis) or during the process to see how the project is going and what are the shortcomings and at the end of the project to see if the needs were met and if not why.
Thursday, August 12, 2010
Case study
I hooked up with Veronique to do the case study exercise and looked at the Engineering Mechanics case study. Veronique has already published the response but I’m not sure if I am supposed to post to post a copy. Anyway, to be on the safer side here is the copy of the response.
What are the key issues?
Is eLearning appropriate for these students? Do they have the required abilities and skills? What support and training would students need to access and use eLearning resources.
He has limited time for developing interactive online resources but has found an online resource that looks useful. Does this resource provide knowledge and/or skills that will be useful for the job?
How can the problem be solved? What strategies can be used?
Need to assess students’ knowledge and use of computers; a questionnaire would be a good tool for this.
It would be useful to trial the eLearning resource with students; this could be done by observing the students using the resource and providing a follow-up interview or questionnaire.
Get feedback on the usefulness of the eLearning resource from teaching staff and from industry representatives; discussion group, individual interview or questionnaire.
How can the evaluation process assist in finding a solution? What type of evaluation is appropriate in this situation?
A well planned and carefully designed formative evaluation should provide information relevant to the specific issues being evaluated. In this example, the evaluation is being used to inform recommendations about the usefulness of eLearning in the course and the effectiveness of the eLearning resource in question.
The evaluation process involves clearly identifying the rationale, purpose and context of an evaluation. This information will then influence the type of evaluation and the choice of methods and instruments that will be used to collect data. Analysis of the data will provide information that can be used to make recommendations. A formal report detailing the evaluation process used and the final recommendations can then be prepared.
The type of evaluation used would be a ‘needs analysis’ to determine whether eLearning is appropriate for teaching and learning in this specific course. A thorough evaluation would help to inform teaching staff of the usefulness and effectiveness of eLearning as well as identifying what additional support students and staff might need, and ensure time and effort is not wasted on implementing and developing resources that may not be useful
What are the key issues?
Is eLearning appropriate for these students? Do they have the required abilities and skills? What support and training would students need to access and use eLearning resources.
He has limited time for developing interactive online resources but has found an online resource that looks useful. Does this resource provide knowledge and/or skills that will be useful for the job?
How can the problem be solved? What strategies can be used?
Need to assess students’ knowledge and use of computers; a questionnaire would be a good tool for this.
It would be useful to trial the eLearning resource with students; this could be done by observing the students using the resource and providing a follow-up interview or questionnaire.
Get feedback on the usefulness of the eLearning resource from teaching staff and from industry representatives; discussion group, individual interview or questionnaire.
How can the evaluation process assist in finding a solution? What type of evaluation is appropriate in this situation?
A well planned and carefully designed formative evaluation should provide information relevant to the specific issues being evaluated. In this example, the evaluation is being used to inform recommendations about the usefulness of eLearning in the course and the effectiveness of the eLearning resource in question.
The evaluation process involves clearly identifying the rationale, purpose and context of an evaluation. This information will then influence the type of evaluation and the choice of methods and instruments that will be used to collect data. Analysis of the data will provide information that can be used to make recommendations. A formal report detailing the evaluation process used and the final recommendations can then be prepared.
The type of evaluation used would be a ‘needs analysis’ to determine whether eLearning is appropriate for teaching and learning in this specific course. A thorough evaluation would help to inform teaching staff of the usefulness and effectiveness of eLearning as well as identifying what additional support students and staff might need, and ensure time and effort is not wasted on implementing and developing resources that may not be useful
Monday, August 9, 2010
My thoughts on evaluation
Evaluation is part of my teaching career. I used to teach ESOL at a college and the first thing I would do with our new students is the diagnostic test to see where they are at. The diagnostic test is then used to make decision about the course, content, skills and resources.
Before we get to the end of the course we would do formative assessment in the form of homework, group work, discussions and teacher observations etc to see how well the students understand the subject and make changes if necessary. Summative assessment comes in the form of NCEA exam where they get the final result as achieved or not achieved.
We also do teacher evaluation by distributing questionnaires to the students to find out how effective the teacher had been and this information is then used to select the ‘The best teacher of the Year’ award .
I also used a reflective journal as part of self evaluation. For me evaluation involves self reflection, student feedback and peer feedback that can be used in decision making.
Before we get to the end of the course we would do formative assessment in the form of homework, group work, discussions and teacher observations etc to see how well the students understand the subject and make changes if necessary. Summative assessment comes in the form of NCEA exam where they get the final result as achieved or not achieved.
We also do teacher evaluation by distributing questionnaires to the students to find out how effective the teacher had been and this information is then used to select the ‘The best teacher of the Year’ award .
I also used a reflective journal as part of self evaluation. For me evaluation involves self reflection, student feedback and peer feedback that can be used in decision making.
Sunday, August 8, 2010
Another late starter
I am another late starter. I was on holiday on the first week of the course and still did not have a good look at the course as I had been so busy at work since the semester started. I’ve just found a partner to work on the case studies (Thanks Veronique) and this is my first blog entry.
Subscribe to:
Posts (Atom)