EDUCATIONAL PORTFOLIO

Papers and Conferences

2010-11-08T13:24:00.000-08:00

Nelson Dordelly Rosales

Education

Universidad de Ottawa, Laws Doctoral Candidate, fourth year
Universidad de Ottawa, Master in Laws, 2007
University of Saskatchewan, Master of Educ. Communication & Technology, 2010
Universidad Católica Andrés Bello, Abogado, J.D. (2000-2005)
Universidad Católica Andrés Bello, Licenciado Educacion (1999-2004)

Associations:

Law Society of Upper Canada
The Osgoode Society for Canadian Legal History
Canadian Studies of Law
Inter-American Bar Association
Caracas Bar Association, 2006
Catholic University Graduates – Venezuela

Conferences - Presentation of Papers

by Nelson Dordelly-Rosales

World Congress of Constitutional Law: Constitutions and Principles

Web site: http://www.juridicas.unam.mx/wccl/en/default.htm

http://www.juridicas.unam.mx/wccl/en/g3.htm

Conference on Jurisprudence and Legal Theory

Cambridge University Interdisciplinary Graduate Conference: “Diverse Engagement: drawing in the Margins.”

University of Cambridge

Web site: http://www.igc2010.co.uk./programme.htm

Educational Portfolios in Higher Education

Representing the University of Saskatchewan

XI Encuentro internacional Virtual Educa

Santo Domingo, Dominican Republic

Web site: http://www.virtualeduca.info/fveducasd/

My presentation:

http://www.virtualeduca.info/fveducasd/index.php?option=com_content&view=article&id=320

Conference on Environmental Toxicity: Canada

Location: University of Ottawa

Web site: http://www.gsaed.ca/conference2010/

Award for the best presentation paper

http://www.gsaed.ca/conference2010/

Conference on Democracy and Constitution: Venezuela.

Paper presented in the Faculty of Social Sciences, University of Ottawa:
http://www.socialsciences.uottawa.ca/eng/documents/conference_latinamerique.pdf

http://www.facebook.com/album.php?aid=2284639&id=121512340&l=f1ebc306c4

BLOGS

Deontologia Juridica
http://www.deontologiajuridica.blogspot.com/
www.deontologiajuridicaucab.blogspot.com

Educational Portfolio
www.researchphilosophy.blogspot.com

Curriculum Planning
http://www.curriculumplanning.blogspot.com/

WELCOME TO MORE TENNIS
http://moretennis.blogspot.com/

http://www.youtube.com/watch?v=FldqCv2KYYY

Teaching in an online environment
http://teachinginanonlineenvironment.blogspot.com/

INSTRUCTIONAL DESIGN & EVALUATION
http://instructionaldesign874.blogspot.com/

https://eci834module5.wikispaces.com/

TU VOZ UNIVERSITARIA UCAB
http://tuvozuniversitaria.blogspot.com/

Margarita Island
http://www.venezuelanmargaritaisland.blogspot.com/

Asesoria Juridica Gratuita
http://asesoriajuridica.blogspot.com/

Research Philosophy
http://researchphilosophy.blogspot.com/

Curriculum Planning
http://curriculumplanning.blogspot.com/

Professor Igor barreto - Latin American literature
http://igorbarreto.blogspot.com/

Thomas Mann - German Literature
http://necesidadesexpresivas.blogspot.com/

Edward Lear - English Literature, UK
http://edwardlear.blogspot.com/

Margarita Island
http://venezuelanmargarita.blogspot.com/

Constitutional Interpretation
http://papelesdetrabajo.blogspot.com/

Professor Ronald Dworkin and Justice Aharon Barack
http://dworkinintegrity.blogspot.com/

CURRICULUM TRENDS: HOLISTIC & INTEGRATED VIEW

2010-02-25T07:40:00.001-08:00

Final Paper Course 802 Nelson

Conflicting Curriculum Conceptions

2010-02-25T07:02:00.000-08:00

BIBLIOGRAPHY

Bobbitt, F.(1928). How to Make a Curriculum, Boston: Houghton Mifflin

Bloom, B. S. and Krathwohl, D.R. (1956). Taxonomy of Educational Objectives: The Classification of Educational Goals, Handbook I: Cognitive Domain. New York: Longmans, Green.

Bruer, J. T. (1993). Schools for thought: A science of learning in the classroom. Cambridge, MA: The MIT Press.

Bruner, J. (1996). The Culture of Education, Cambridge, Mass.: Harvard University Press.

Bruce, F. and Dodson, E. (2001) Technology and the Human Spirit: Confronting the New Millenium. [Electronic Version]. Association of Humanistic Psychology. Retrieved Jan. 18th, 2006 from http://www.ahpweb.org/involve/centers/technologyandspirit.html

Conway, Judith (1997) Educational Technology's Effect on Models of Instruction [Electronic Version]. Retrieved Jan. 20th, 2006 from http://udel.edu/~jconway/EDST666.htm

Dewey, J. (1938). Experience and Education, New York: Macmillan.

Driscoll, M.(1994). Psychology of Learning for Instruction. MA: Allyn and Bacon.

Eisner, E. W. (1994). Cognition and curriculum reconsidered, 2e, New York: Teachers College Press.

Eisner, E. W. (1985). The Art of Educational Evaluation, Lewes: Falmer Press.

Eisner E. W. and Vallance E. (1974). Conflicting Conceptions of Curriculum, Berkeley, CA: McCutchan.

Eisner, E. W. (1979). The educational imagination. New York: Macmillan.

Freire, P. (1972). Pedagogy of the Oppressed. Harmondsworth: Penguin.

Gagne, R. and Briggs, L. (1979). Principles of Instructional Design. New York: Holt, Rinehart and Winston.

Gardner, Howard (1983) Frames of Mind: The Theory of Multiple Intelligences, New York: Basic Books.

Gardner, Howard (1999) Intelligence Reframed. Multiple intelligences for the 21st century, New York: Basic Books.

Gendron, B. (1977) Technology and the Human Condition, NY: St.Martin. p. 23

Ghaoui, C. (2004). E-Education Applications: Human Factors and Innovative Approaches. Hershey PA: Idea Group Inc.

Henderson, James G. and Kesson, Kathleen R. (2004). Curriculum Wisdom: Educational Decisions in Democratic Societies. OH: Merrill/Prentice Hall.

Hewit, T. W. (2006). Understanding and Shaping Curriculum: What We Teach and Why. Thousands Oaks, CA: Sage Publications.

Hurd N. S. and Stein R. F. (2004). Building and Sustaining Learning Communities. The Syracuse University Experience. MA: Anker Publishing Company, Inc.

Magoulas D. G. and Chen, S.Y. (2006). Advances in Web-Based Education: Personalized Learning Environments. (London: Idea Group Inc.)

Maslow, A. (1954). Motivation and Personality. New York: Harper.

McNeil, J. D. (1985). Curriculum: A comprehensive introduction. Boston, MA: Little-Brown.

McNeil, J. D. (2006). Contemporary Curriculum: In Thought ad Action, 6th Edition. Boston, MA: Little-Brown.

Motschnig-Pitrik, R. and Holzinger, A. Student-Centered Teaching Meets New Media:
Concept and Case Study Educational Technology & Society 5 (4) 2002 at 3. [Electronic Version]. Retrieved Jan 18,th 2007, from http://ifets.ieee.org/periodical/vol_4_2002/renate.html

Panitz, T. (2001). Learning Together: Keeping Teachers and Students Actively Involved by Writing Across the Curriculum A Source Book of Ideas and Writing Assignments. Stillwater, Oklahoma: New Forums Press.

Posner, G. J. (2004). Analyzing the Curriculum. 3d edit. NY: McGraw Hill.

Roberts, Tim S. (2004) Online Collaborative Learning: Theory and Practice (London: Idea Group Inc.)

Rogers, C.R. (1983). Freedom to learn for the 80s. Columbus, OH: Charles Merrill.

Rosales-Dordelly, C.L. and Short, E.C. (1985). Curriculum Professors’ Specialized Knowledge. Lanham, MD: University Press of America.

Redmond A. (2004). The Education Living Learning Community in Hurd. S.N. and Stein, R.F. Building and Sustaining Learning Communities (p.190)

Sagor, R. (1993). At-Risk Students: Reaching and Teaching Them. Swampscott, Massachusetts: Watersun Publishing Company, Inc.

Schwab, J. (1978). Education and the Structure of the Disciplines. In J. Westbury & N. Wilkof (Eds.), Science, curriculum, and liberal education. Chicago: University of Chicago Press.

Schwab, J. (1971). The Practical: Arts of Eclectic. School Review, 79, 493-542.

Short Edmund C. (1991). Forms of Curriculum Inquiry. New York: State University of New York.

Schwier, R., and Misanchuk, E. N. (1993). Interactive Multimedia Instruction. CA: Englewood Cliffs.

Taba, H. (1962). Curriculum Development: Theory and practice. New York: Harcourt Brace and World.

Tim S. R. (Ed.). (2004). Online Collaborative Learning: Theory and Practice. London: Idea Group Inc.

Tyler, R. W. (1949). Basic Principles of Curriculum and Instruction. Chicago: University of Chicago Press.

Wasson, B., Ludvigsen, S., and Hoppe, U. (Ed.). (2003). Designing for Change in Networked Learning Environments, Proceedings of the International Conference on Computer Support for Collaborative Learning. Computer –Supported Collaborative Learning. Netherlands: Kugler Academic Publishers.

White, J.A. and Purdom, D. A. (n.d.) Viewing Modern Instructional Technology through Conceptions of Curriculum [Electronic Version]. Retrieved Jan 18,th 2007, from http://gozips.uakron.edu/~jsavery/emerging/it-and-curriculum.htm

Zuga, Karen F. (n.d). Relating Technology Education Goals to Curriculum Planning, [Electronic Version] Journal of Technology. Retrieved Jan.18th, 2007 from http://scholar.lib.vt.edu/ejournals/JTE/jte-v1n1/zuga.jte-v1n1.html

My Portfolio Class Jan 30th

2010-01-30T08:02:00.000-08:00

I got Voice –In my MEd I am learning to be a leader and feeling free to ‘speaking my truth.’
Vision and Passion: my view of the future is that Venezuela will make progress, and I will be injecting enthusiasm and energy to transform the vision of education. University education, particularly the program communication and technology at UCV is rigid, follows a traditional pattern, offering a couple of courses via online. I have a simple vision:

To teach people that Life is a Journey and that this Journey requires flexibility, freedom and commitment to research to pursue happiness. Given the constant progress of communications and technology, the challenge is:

To seek the integral formation of the student so that he and she may develop positively and creatively in his or her environment, being committed to find new ways for his/her own growth, as well as that of his/her society.
To help students to apply technology skills, as well as teaching in an online environment, research methods and abilities to value an integrated and a coherent curriculum. The final purpose is to encourage them to keep learning on their own.
The use of different teaching techniques motivate students to learn better the material presented to them in class, besides providing them with enough tools to continue developing their learning skills on their own.

My Preparation: Courses taken M.Ed Communications & Technology in U Sask. Canada:
EDCMM 802.6 Historical & Theoretical Foundations of Educational Technology (Fall 2006/Winter 2007) 6 cr.
ECMM 804.3/ Distance Education Theory and Practice (Online option) (CRN: ECMM 804.3-82701) 2008 Dr Morrison
EDCMM 873.3 Instructional Design: Designing Materials for Individualized Instruction (Fall 2007) 3 credit
EDCMM 874.3 Advanced Principles of Instructional Design (Winter 2008) 3 U Dr Wilson
EC&I 834.3 Teaching in an Online Environment,University of Regina(Spring/Summer 2008
ERES 800 Ed Research Dr Noonan –Spring/Summer 2009
ECUR 805 Ed Evaluation Winter 2009 Dr Jay Wilson
ECUR 960 Ethics
ECUR 990 Curriculum Seminar Fall/Winter 2009-2010 Dr Janet McVittie
ECUR 991 Curriculum Portfolio Winter 2010 Dr Tim Molnar

My values are: freedom, honesty, balance, trust, determination, confidence, being healthy, contribution, family… Personal integrity: creating trust and being a good listener. Intellectual stimulation (researcher)

Confidence Principle & Personal Charisma: persuasiveness and sound self-esteem (no arrogance). “A confident persuader creates confidence in the persuaded.”
The Route Forwards: inspirational motivation, finding the way forwards, feeling progress, collaboration…

Seeking to Transform: seeking change, to transform the organization. My focus will be on transforming traditional rigid view of teaching into a modern technology & freedom perspective, combining face to face with an online environment in my University in Venezuela: helping to restructure the graduate program of education (a specialization "Tecnologias de la Informacion Comunicacion") by improving program conditions e.g., "Communication & Technology for Freedom" emphasizing in three components: Research Methodology using Data Bases online, Curriculum Portfolio, teaching in an online environment. This program will be good not only for educators, teachers, faculty but also for lawyers, politicians, etc. (interdisciplinary). Different people often seek updating knowledge in Communication & Technology. This program can be transformed into a more freedom and happiness to the students. To that end, I will start today with a Survey questionnaire to participants and stakeholders to find out specific needs. Please click: https://survey.usask.ca/survey.php?sid=18846&preview_survey=1

For more information about the current program "Specialization in Communications & Technology" UCV, see: http://www.ucv.ve/?id=1684 http://www.ucv.ve/

Nelson

Transformational Leadership

2010-01-29T22:36:00.000-08:00

Thoughts

“If you do not know where you are going, every road will get you nowhere.”
~ Henry Kissinger ~

“A single lie destroys a whole reputation for integrity.”
~ Baltasar Gracian ~

“As we look ahead into the next century, leaders will be those who empower others.”
~ Bill Gates ~

“Leadership is the challenge to be more than average.”
~ Jim Rohn ~

James S. Pounder (2006) in "Transformational Classroom Leadership
The Fourth Wave of Teacher Leadership?" emphasizes in the following "waves" of teacher leadership:
- 1. confined teacher leadership within the formal organizational hierarchy
and merely placed the concept close to the teaching function
- 2. placed more emphasis on the instructional dimension of
the teaching function but still vested teacher leadership in formally created
organizational positions such as team leader and curriculum developer
- 3. teacher leadership is a process rather than a positional concept
- 4. it could include transformational classroom leadership as one
of the defining qualities of a teacher leader and could embrace both school and
university contexts.
The author notes that teacher leaders tend to possess many of the characteristics of
transformational leaders.

Huber and West (2002) delineate the following stages:
1. The personality or trait theory of leadership: focusing on great men and women leaders in history, for example: Gandhi, Mandela, Churchill, Thatcher….
2. Examining what good leaders actually do
3. Attention to the context in which leadership is exercised: Task-related and people-centred behaviours.
4. Linking the culture of the organization to the leader.

Key characteristics of Instructional & Transformational Leadership that I as teacher-scholar experienced during my M.Ed experience:

I got Voice –means feeling free to ‘speaking my truth.’
Vision and Passion: a view of the future, injecting enthusiasm and energy
I have a simple vision: To teach people that Life is Happiness
My values are: freedom, honesty, balance, trust, determination, confidence, being healthy, contribution, family…
Personal integrity: creating trust and being a good listener. Intellectual stimulation (researcher)
Confidence Principle & Personal Charisma: persuasiveness and sound self-esteem (no arrogance). “A confident persuader creates confidence in the persuaded.”
The Route Forwards: inspirational motivation, finding the way forwards, feeling progress…
Seeking to Transform: seeking change, to transform the organization, the followers are the product of the transformation… for example, my focus at the University in Venezuela: helping to restructure the graduate program of education(a specialization "Tecnologias de la Informacion Comunicacion") by improving program conditions e.g., "Communication & Technology for Freedom". This program will be good not only for educators, teachers, faculty but for lawyers, politicians, etc. as well. Different people often seek updating knowledge in Communication & Technology (see: http://www.ucv.ve/?id=1684)
http://www.ucv.ve/
Transformational Leadership Sources: http://changingminds.org/disciplines/leadership/styles/transformational_leadership.htm
http://www.waynelee.com/become-a-great-leader.htm

Hallinger’s (2003) re: instructional leadership proposes three dimensions:
1. defining the school’s mission,
2. managing the instructional program,
3. promoting a positive school-learning climate.

Marks and Printy (2003) the term “instructional leadership” should replace the hierarchical and procedural notion with the concept of “shared” instructional leadership. In this model, the principal is the “leader of instructional leaders” not the person who is independently responsible for leadership initiatives within the school…

Dimmock (1995) asserts that instructional leadership is too prescriptive and relies on a top down process of management. A better strategy would be a bottom-up approach. The proposed “backward mapping” would begin with student outcomes and then progress up through the following: learning styles and processes; teaching strategies; school organization and structure; and leadership, management, resources and culture/climate.
The challenge for many principals is to work with the various educational stakeholders to maintain some sense of balance…

Adams and Kirst (2004)state, “Policy makers, educational leaders, practitioners, and parents also continued to seek better student performance and accountability through management practices, professional standards, teacher commitment, democratic processes, and parent choice” (p. 466).

New terms began to emerge in literature such as: shared leadership, teacher leadership, distributed leadership and transformational leadership. Burns’ latest book entitled Transforming Leadership: A New Pursuit of Happiness (2003) offers an expansion of his earlier book.

Bernard M. Bass & Bruce J. Avolio (1998) "Multifactor Leadership Questionnaire" (MLQ). Components of transformational leadership:
1. Charismatic Leadership, or Idealized Influence. Transformational leaders are role models; they are respected and admired by their followers. Followers identify with leaders and they want to emulate them. Leaders have a clear vision and sense of purpose and they are willing to take risks.
2. Inspirational Motivation. Transformational leaders behave in ways that motivate others, generate enthusiasm and challenge people. These leaders clearly communicate expectations and they demonstrate a commitment to goals and a shared vision.
3. Intellectual Stimulation. Transformational leaders actively solicit new ideas and new ways of doing things. They stimulate others to be creative and they never publicly correct or criticize others.
4. Individualized Consideration. Transformational leaders pay attention to the needs and tthe potential for developing others. These leaders establish a supportive climate where individual differences are respected. Interactions with followers are encouraged and the leaders are aware of individual concerns (Bass, 1998)
Source:
http://www.umanitoba.ca/publications/cjeap/articles/stewart.html

E-Portfolio Reading

2010-01-25T18:40:00.000-08:00

Class Ottawa-USask, Jan 30 -

Topics:
Topic 1-Jan 10th
What are your plans for developing your final portfolio, presentation or performance?
Describe your initial ideas, what you may focus on as meaningful from your M.Ed experience
and perhaps a hint of why.
1 To create and maintain a learning environment that encourages and supports the
growth of the whole student.
2 To demonstrate a professional level of knowledge about the curriculum and the skills
and judgment required to apply this knowledge effectively.
3 To demonstrate and support a repertoire of instructional strategies and methods that
are applied in teaching activities.
4 To carry out professional responsibilities for student assessment and evaluation.
5 To reflect upon the goals and experience of professional practice, and adapt oneʼs
teaching accordingly.
6 To work with colleagues in mutually supportive ways and develop effective
professional relationships with members of the educational community.
7 To conduct all professional relationships in ways that are consistent with principles of equity, fairness and respect for others.
ECUR 801.6 - Principles and Practices of Curriculum Construction
ERES 800.3 - Research Methods: Introductory Level
ECUR 990.0 - Seminar in Curriculum Research
ECUR 992.6 - Project or
ECUR 991.3 – Portfolio
ECUR 994 - Thesis
ECUR 805.3 - Trends and Issues in Educational Research and Development
ECUR 809.3 - Models and Methods for the Evaluation of Educational Programs
ECUR 810.3 - Design and Practice of the Evaluation of Educational Programs
ECUR 819.3 Trends and Issues in Mathematics Education
ECUR 820.3 Introduction to Graduate Studies in Science Education
ECUR 830.3 Research in Teaching and Learning
ECUR 832.3 Practicum in Professional Development
ECUR 843.3 Reading Process and Practice
ECUR 870.3 Literacy Education and Curriculum
ECUR 872.3 Trends and Issues in the Study of Writing
ECUR 888.3 Trends and Issues in Social Studies
ECUR 898.3 or ECUR 899.6 Individual Reading Courses in Curriculum
ERES 845.3 Qualitative Research
ERES 840.3 Quantitative Research
_________
Required courses for the MEd degree in Educational Technology and Communication:
ECMM 802.6 - Historical and Theoretical Foundations of Educational Technology
ERES 800.3 - Research Methods: Introductory Level
ECUR 990.0 - Seminar in Curriculum Research
Elective courses for the MEd degree in Educational Technology and Communication:
ECMM 803.3 - Principles and Practices of Designing Multimedia Resources
ECMM 804.3 - Distance Education
ECMM 873.3 - Principles and Practices of Instructional Design
ECMM 874.3 - Advanced Approaches to Instructional Design
ECMM 876.3 - Organization and Administration in Educational Technology
ECMM 877.3 - Advanced Video Production in Education
ECMM 879.6 - Television in Education
University of Saskatchewan: Department of Curriculum
Studies - ECUR 991.3
Scholarship in Teaching
Course Description
Students will demonstrate their scholarship in teaching through developing a collection of academic and professional work. The work may be represented through a variety of media, such as portfolios (including electronic portfolios), multi-media presentations, and professional development initiatives.
This course will be a culmination of students’ research into professional knowledge and identity undertaken throughout the MEd degree within a transformative and reflective learning culture.
Course Objectives
• To analyze, synthesize and evaluate teaching scholarship through growth in
understanding educational theory.
• To demonstrate academic and professional competence in their chosen field of
curriculum studies or educational communication and technology.
• To reflect on teaching and learning philosophy.
• To demonstrate awareness of the ethical nature and practice of professional work and research.
Teaching methodologies
Since students in this course are likely to be studying part-time, a variety of distance approaches (online discussion boards, video/podcast) as well as face-to-face meetings, will be used.
Academic Honesty
Students are expected to follow the University of Saskatchewan’s guidelines on academic honesty detailed at http://www.usask.ca/honesty/pdf/dishonesty_info_sheet.pdf
Honesty and integrity are expected in class participation, examinations, assignments, and other academic work. Perform your own work unless specifically instructed otherwise. Follow examination rules. Discuss with your professor if you are using the same material for assignments in two different courses. Use the same standard of honesty with fellow students, lab instructors, teaching assistants, sessional instructors and administrative staff as you do with faculty.
Course Activities and Topics
As they build their portfolios, students may:
• Identify major trends in curriculum theory, and major trends in their own specific
areas of interest, then identify how their understanding of curriculum has evolved
through participation in their MEd courses.
• Decide on the final form of their Scholarship in Teaching portfolio and presentation.
• Reflect on teaching practice. Through examples from their own and others’
experiences, students will describe ways in which they can influence practice in
different educational settings, as well as identify ways in which they can best
represent their skills, challenges and changes.
• Take a problem-solving approach to evaluating practice and to curriculum
implementation.
• Reflect on issues of diversity, including Aboriginal issues, teaching and learning in multicultural contexts, and working with the full range of student needs.
• Represent learning from their academic and professional growth during the MEd
program.
• Make a presentation to the class or to another academic/professional group.
• Carry out a final interview with a faculty member, based on their portfolio.
• Construct an annotated personal curriculum vitae that demonstrates leadership in
curriculum or educational technology.
Evaluation:
• Online discussions and/or in-class presentations: 30%
• Final interview/oral examination 25%
• Portfolio 45%
Student work will be evaluated using the grading system from the College of Graduate Studies
and Research. It is available at http://www.usask.ca/calendar/gradstudies/additional/grading/

Resources and References
Resources on portfolios and e-portfolios
Anderson, R. S., & DeMeulle, L. (1998). Portfolio use in twenty-four teacher education
programs. Teacher Education Quarterly, 25(1), 23.
Bartell, C. A., Kaye, C., & Morin, J. A. (1998). Teaching portfolios and teacher education.
Teacher Education Quarterly, 25(1), 5.
Barton, J., & Collins, A. (1993). Portfolios in teacher education. Journal Of Teacher Education, vol, 44(3), 200-210.
Berliner, D. (1986). In pursuit of the expert pedagogue. Educational Researcher, 15(7), 5-13.
Biggs, J. (2003). Teaching for quality learning at university: What the student does (2nd ed.).
London: SRHE and Open University Press.
Biggs, J., & Tang, C. (1998). Assessment by portfolio: Constructing learning and designing teaching. In P. Stimpson & P. Morris (Eds.), Curriculum and assessment for Hong Kong: Two components, one system (pp. 443-462). Hong Kong: Open University of
Hing Kong Press.
Borko, H., Michalec, P., Timmons, M., & Siddle, J. (1997). Student teaching portfolios: A tool for promoting reflective practice. Journal Of Teacher Education, vol, 48(5), 345-357.
Cambridge, B. (2001). Electronic portfolios as knowledge builders. In A. Cambridge (Ed.),
Electronic portfolios: Emerging practices in student, faculty and institutional learning. New York: AAHE Publications.
Grant, G. E., & Huebner, T. A. (1998). The portfolio question: A powerful synthesis of the personal and professional. Teacher Education Quarterly, 25(1), 33-43.
Heath, M. (2003). Telling it like it is: Electronic portfolios for authentic professional development. Library Media Connection, 21(6), 38.
Hopper, T., & Sanford, K. (2004). Representing multiple perspectives of self-as-teacher: Integrated teacher education course and self-study. Teacher Education Quarterly, 31(2), 57-74.
Hopper, T., Sanford, K., Lauzon, L., Yeo, M., & Stogre, T. (2003). Enhancing professional development through integrated campus/field-based teacher education: Rekindling the passion for teaching. Paper presented at the CATE, CSSE, Halifax.
Lyons, N. (Ed.). (1998). With portfolio in hand: Validating the new teacher professionalism. New York: Teachers College Press.
Munby, H., Russell, T., & Martin, A. (2001). Teachers' knowledge and how it develops. In V. Richardson (Ed.), Handbook of research on teaching (4th ed., pp. 877-904).Washington, DC: American Educational Research Association.
Young, J. R. (2002). "e-portfolios" could give students a new sense of their accomplishments.
Chronicle of Higher Education, 48(26), 31-32.
Resources on educational issues
Aoki, T. T. (2005.). In Pinar W., Irwin R. L., (Eds.), Curriculum in a new key :The collected works of ted T. aoki. Mahwah, N.J.: Lawrence Erlbaum Associates Publishers,.
Ayers, W., Miller, J., & Miller, J. (1998). A light in dark times: Maxine Greene and the unfinished conversation. New York: Teachers College Press.
Ayers, W., & Schubert, W. H. (c1992.). Teacher lore :Learning from our own experience. New York ;: Longman.
Battiste, M. A. (2000.). Reclaiming indigenous voice and vision. Vancouver: UBC Press.
Battiste, M. A., & Barman, J.,. (c1995.). First Nations education in Canada: The circle unfolds.Vancouver: UBC Press.
Clandinin, D. J., Huber, J., Huber, M., Murphy, M.S., Pearce, M., Murray-Orr, A., & Steeves, P.(2006). Composing diverse identities: Narrative inquiries into the interwoven lives of children and teachers. London: Routledge.
Clandinin, D. J., & Connelly, F. M. (2000). Narrative inquiry: Experience and story in qualitative research. San Francisco, Calif.: Jossey-Bass.
Connelly, F. M., & Clandinin, D. J. (1988). Teachers as curriculum planners: Narratives of experience. New York: Teachers College Press.
Davis, B., & Sumara, D. J. (2006). Complexity and education : Inquiries into learning, teaching, and research. Mahwah, NJ: Lawrence Erlbaum Associates.
Delpit, L. D. (1995). Other people's children : Cultural conflict in the classroom. New York: New Press : Distributed by W.W. Norton.
Dewey, J. (1938). Experience and education. New York: Simon and Schuster Inc.
Dewey, J. (1966). Democracy and education : An introduction to the philosophy of education. New York: Free Press.
Fine, M. (c1992.). Disruptive voices :The possibilities of feminist research. Ann Arbor: University of Michigan Press.
hooks, B. (1994). Teaching to transgress: Education as the practice of freedom. New York: Routledge.
Huber, J., Murphy, M., & Clandinin, D. (2003). Creating communities of cultural imagination: Negotiating a curriculum of diversity. Curriculum Inquiry, 33(4), 343-362.
Huber, J. C.,Jean. (2002). Ethical Dilemmas in Relational Narrative Inquiry with Children.
Liston, D. P. (1991.). In Zeichner K. M. (Ed.), Teacher education and the social conditions of schooling. New York: Routledge.
Mirochnik, E., & Sherman, D. C.,. (c2002.). Passion and pedagogy :Relation, creation, and transformation in teaching. New York: P. Lang.
Paley, V. G. (1992). You can't say you can't play. Cambridge, Mass.: Harvard University Press.
Pinar, W. (c1998.). Curriculum:Toward new identities. New York: Garland Publ.
Reason, P., & Bradbury, H. (2001.). Handbook of action research :Participative inquiry and practice. London ;: SAGE.
Richardson, L. (1997). Fields of play: Constructing an academic life. New Brunswick, N.J.: Rutgers University Press.
Schwab, J. J. (1978.). The practical: Translation into curriculum. In I. Westbury, & N. J. Wilkof (Eds.), Science, curriculum, and liberal education: Selected essays [Science, curriculum, and liberal education : Selected essays] (pp. 365-383). Chicago: University of Chicago Press.
Sleeter, C. E. (2005). Un-standardizing curriculum : Multicultural teaching in the standardsbased classroom. New York: Teacher College Press.
Sleeter, C. E., & McLaren, P. (1995). Multicultural education, critical pedagogy, and the politics of difference. Albany: State University of New York Press.
St. Denis, D. V. Literature review on racism and the effects on aboriginal education. Ottawa, Ontario November 15/2002.:
Van Manen, M. (1990). Researching lived experience : Human science for an action sensitive pedagogy. London, Ont.: Althouse Press.
Woyshner, C. A., & Gelfond, H. S. (1998). Minding women: Reshaping the educational realm.
Cambridge, Mass.: Harvard Educational Review.

______________________________________
ECUR 991 eportfolio-1

2010-01-09T11:35:00.001-08:00

Educational Portfolios

ECUR 991 (2009-2010)

ECUR 991

MINI MANUAL Mahara by Constantinescu Nicolaie

Mini Manual Mahara En
______________________________________
ECUR 991 eportfolio-1

Paper ECUR 809

2009-12-06T18:27:00.000-08:00

Program Evaluation Final

ECUR 990

2009-12-06T18:26:00.000-08:00

ECUR 990 Nov.

SURVEY QUESTIONNAIRE RESULTS 809

Survey Results: Data Analysis

Assignment # 5 Final Survey Questionaire

2009-11-17T21:58:00.001-08:00

Survey Questionnaire

To take this survey please click here:
https://survey.usask.ca/survey.php?sid=17783

Please read below explanation of previous work done: plan and preliminary versions of this survey questionnaire.

Three Preliminary Versions of Survey Questionnaire

2009-11-06T08:15:00.001-08:00

Assignment # 5- ECUR 809
The focus of this assignment is on clients’ level of satisfaction and their suggestions for the improvement of the Spanish Intermediate Program (SIP) offered by the Community Center in the City of Ottawa.
Overview:
This assignment aims the improvement of SIP in light of the clients' suggestions and responses to the survey questionnaire in The Community Center in the City of Ottawa. Clients in this particular case refers to students or participants of the Program “Spanish Intermediate and Conversation” (this program is complemented with Tennis Teaching) offered in the Community Center in Ottawa. According to Jody L. Fitzpatrick, James R. Sanders and Blaine R. Worthen (2004 p.100-111), there are standards and criteria to judge programs which may have a marked effect in improving the efforts of teachers and the whole community to improve programs: the purpose of the program (s), its rationale and objectives, its content, process and implementation (instructional technology and activities or strategies), and evaluation, form overall judgments about its effectiveness. Below each one is briefly explained:

1. Purpose and objectives: Community Center Programs are offered by season for all members of the community. Their rationale states that the community believes that the programs enhance growth and quality of life of the clients and their academic achievement; the teachers have the primary responsibility for their growth. Rationale of the City of Ottawa and the Community Center believe that the teachers’ performance enhances instruction, clients’ satisfaction and achievement.
Objectives or specific statements of what the project sets out to accomplish:
each teacher shall develop a program for their students in his/her area of expertise and under general guidelines provided by Ottawa City.
Specific objectives are outputs to be achieved; these are immediate or specific, concrete results (direct products of project activities). Each program is to be reviewed by the teacher and students to make changes or improvements, according to own immediate needs, for example: enhancing student achievement, performance and satisfaction, a more productive use of time, to increase professional and personal interactions and discussions, greater sharing of responsibility and leadership, increasing knowledge, involvement, and continuous learning.
2. Content: specific tasks to complete the course content in the classroom; content included in the Program kit or educational package created and published by the City of Ottawa and the Community Center.
3. Process and Implementation: include strategies for goal achievement, activities or actions taken to achieve the program: registering to the course, attending lectures or lessons, seminars and workshops, reading or doing research, peer coaching, mentorship, creation of personal portfolios, videos and recordings, mentorship, understanding Lesson planning, workshops, and meetings, discussing contents, organizing groups and setting ‘drills’, examples, exercises, models, instructional media, training opportunities; using equipment and educational materials and supplies, etc; reflective journal, developing collaboration and learning group, speaking the language (Spanish) in a real setting (playing tennis) or other initiatives that enhance instruction and student achievement.
4. Satisfaction relates to effectiveness: Impacts of SIP on students’ satisfaction shows in some way the effectiveness of the Program. Also, there must be long-term outcomes such as increasing knowledge, involvement, and ownership, continuous and permanent learning, increasing progressive knowledge, involvement, and ownership

Plan of Evaluation:
Who should be involved?
Engage Stakeholders:Teachers, administrators, supervisors, coordinators, volunteers and students; however, the focus of this work will be on the students or participants of the Community Center, particularly of Spanish Intermediate and Conversation (SIP).
How might they be engaged?
Students will be invited in meetings, email, survey-questionnaires.
Focus of the Evaluation:
What are you going to evaluate?
Description of Community Center: Programming (see below logic model 1)
Clients’ satisfaction – students’ reactions (see below logic model 2)
What is the purpose of the evaluation?
The purposes of this evaluation are to evaluate the level of satisfaction about (a) the organization, design and implementation (teaching) of the program; (b) their suggestions and recommendations to better achieve the goals of the program, namely, the enhancement of student learning and achievement.
Who will use the evaluation? How will they use the information?
- Teachers, administrators, supervisors, coordinators, volunteers and students.
- To assess the level of satisfaction with the organization, design and teaching of the program and to propose improvements or changes that can help teachers and students to meet the goals.
What questions will the evaluation seek to answer?
General questions:
Do Community Program (CP) helps participants or clients in their personal and professional growth and satisfaction? Is the Program meeting the goals set out by the Center? What are the reactions of students regarding the program? Are they satisfied with their achievement of goals and performance?
Specific questions:
Do objectives, content and activities match properly? How the teacher implement it?
Do Programs encourage students to develop personally and professionally?
Is the Program being used in the way that it was intended?
How is the Program perceived by students? Are they satisfied with their performance in the programs?
What information do you need to answer the questions?
Indicators – How will I know it? Level of satisfaction of participants.
When is the evaluation needed? At the beginning, in the middle and at the end of the program (s).
What evaluation design will you use? Takes in consideration the Consumer-Oriented Evaluation Approach but the focus is on the Goal-Oriented Approach.
Assessment and evaluation are best addressed from the viewpoint of the students’ reactions to 1) teachers & teaching, 2) class-assignments and 3) assignments-materials.
Collect the information
What sources of information will you use?
Existing information:
Web site – Programs – Written materials provided by the Community Center – Teachers materials – samples of students’ work and/or experiences (videos, photos, etc).
People:
Teachers, administrators and the focus will be on the students' satisfaction.
What data collection method(s) will you use?
E-mail survey questionnaire to students - and teachers - (a larger sample).
Questionnaire- Interview (a small sample of four students).
________
Bibliography:
Jody L. Fitzpatrick, James R. Sanders and Blaine R. Worthen, Program Evaluation: AlternativeApproaches and Practical Guidelines (Boston: Allyn and Bacon, 2004) p.100
City of Ottawa: www.ottawa.ca The Glebe Community Center
Plan of evaluation: http://learningstore.uwex.edu/pdf/G3658-02.pdf

Versions of Survey Questionnaire: Different versions were developed during the process of designing and testing of the survey. Each one included a sample checklist, a variety of question types such as scale rating, short answer, and open-ended; the final version is now in the web site. Two preliminary versions were designed and evaluated together with four students. They provided suggestions regarding the following issues: clarity of questions, wording, style, and importance. A third version was developed and posted in the web site of the University of Saskatchewan. The students answered the questionnaire but their concern was on correcting some discrepancies between items and the characteristics of the actual program. They also made suggestions to improve clarity, wording and style. The fourth version is the final one, which is now on the web site of the University of Saskatchewan. Below is the preliminary versions as they were presented to the students for their evaluation:
A. – Version One or Preliminary version
Short Answer: yes or not
1. Does the content cover a significant portion of the program competencies?
2. Is the content up-to-date
3. Is the course level appropriate for most students?
4. Are objectives, competencies, or tasks stated in the student materials?
5. Are tests included in the materials?
6. Are performance checklists included?
7. Is a student’s guide included that offers how to manage and perform the course theory and practice?
8. Is the material presented in a logical sequence?
Scale rating: Quality and satisfaction Judgments. Use +, 0, - to rate or degree of the quality or your satisfaction with specific aspects of the course:
Quality and satisfaction of objectives, competencies, and/or tasks_____
Degree or match between learning activities and objectives______
Quality of test tests and degree of match with objectives________
Quality and satisfaction with of performance checklists and degree of match with objectives________
Quality and satisfaction of directions for how students are to proceed through the materials_______
Quality of visuals, videos, games, experiences, practices_______
Overall design of the learning activities for individualized instruction_____
Quality and satisfaction on safety practices_____
Satisfaction with degree of freedom from bias with respect to sex, race, origin, age, religion, etc,?________
Quality and satisfaction of content list or the course content-map and competencies covered by the course_________
Short answer: brief comment
Does the course have basic elements, such as those listed below? Please mark with an “x” and make a comment if necessary:
a) Clearly stated outcomes objectives____
b) Sufficient directions_____
c) Other materials required____
d) Prerequisite knowledge based and existing programs___
e) Fit with knowledge base and existing programs___
Process information: what is the nature and frequency of interactions among students or clients relevant others? ________________________________
Have these interactions being evaluated?____________________________
Open ended questions: Please explain or illustrate
- Is evaluation an integral part of (a) the development and (b) the implementation of the program?
- Is there evidence of effectiveness available regarding the course?

B. Second Version: the modified version based on the testing of the survey with four individuals:
Short answer: Yes or not
1. Is the program content of Intermediate Spanish up-to-date?
2. Is the program level appropriate for most students?
3. Are objectives, competencies, or tasks satisfactory stated?
4. Is the program presented in a logical sequence?
5. Are you satisfied with the program have basic elements, such as those listed below?
Scale rating: Choice decision-making. Please write on the spaces below: Very Good (VG), Good (G) or Bad (B), and make a comment if necessary:
a) Outcomes, objectives, competencies or tasks____
b) Directions or instructions for how students are to proceed through the materials___
c) Materials ____
d) Prerequisite knowledge based ___
e) Performance checklists____
f) Student’s guide_____
g) Fit with knowledge base and program___
h) Tests
Scale rating: Judgments. Use +, 0, - to rate or degree of the quality or your satisfaction with specific aspects of the course:
Degree or match between learning activities and objectives______
Quality of test tests and degree of match with objectives________
Quality and satisfaction with of performance checklists and degree of match with objectives________
Quality of visuals, videos, games, experiences, practices_______
Overall design of the learning activities for individualized instruction_____
Quality and satisfaction on safety practices_____
Satisfaction with degree of freedom from bias with respect to sex, race, origin, age, religion, etc,?________
Quality and satisfaction of content list or the course content-map and competencies covered by the program_________
Open ended question:
Please feel free to make any suggestions, comments that can help us to improve our Program on Spanish Intermediate (complemented with Tennis instruction) for the next Spring/Summer:
________________________________________________________________

Third Version of the survey questionnaire: It was posted in the web site and the students evaluated it. The students made different corrections and suggestions to improve it. Based on their comments, most items were re-written and the questionnaire was re-organized and redesigned.

Preview of Survey_ #17783_ ..

Please see below Planning Program Evaluation Steps that served as the basis for the whole work.

2009-10-30T19:56:00.000-07:00

21947403 Evaluation Steps

Logic Models ECUR 809 Assignment # 4

2009-10-14T07:33:00.001-07:00

(A)Logic Model 1 - Flow Chart: General
Worksheet Flowchart 1

Logic Model 2 - Flow Chart: More Specific

21010006-WorksheetFlowchart-ECUR-809

B. Description of the Logic Model- Assignment # 4:
- scope of logic model (how much they cover);
- the number of levels included;
- the description of levels included;
- the direction of information flow
- the amount of text;
- the visual layout.
Each of these variables is described in turn, below.
Scope of Logic Model: The flow chart is a logic model designed for evaluation purposes of the whole programs at the Community Center. It begins with vision, mission, values, motivations, expectations, etc and at the end of the day, the purpose of evaluation is to find out if the programs are making the difference. The community center offers complex, multi-component programs that may require the development of separate logic models for each program component or activity. Thus, I designed one general and a more specific one that can help me to promote the need of evaluating ‘clients satisfaction’ in the community programs (which include Spanish & Tennis Teaching). Assessment and evaluation are best addressed from the viewpoint of the students’ reactions to 1) teachers &teaching, 2) class-assignments and 3) assignments-materials.
Number of Levels: The first flow chart logic model includes several ‘levels’ (goals, population of interest, long and short term objectives and indicators). The second one, includes strategies, activities, process indicators.
Description of Levels: There is no standard set of terminology for our logic models. So, the first one includes general terms and the second one applies more specifics. The discussion will begin with stakeholders:Who should be involved or engaged? Teachers, administrators, supervisors, coordinators, volunteers and students;the focus of my work will be on the students or participants of the Community Center Programs. How might they be engaged? Students will be invited in staff meetings, email, survey-questionnaires.
Direction of Information Flow: Both flow charts-logic models flows from moving from left to right starting with objectives and focus of the Evaluation: What am I going to evaluate?
The Community Center Programming (logic model 1) and the clients’ satisfaction – students’ reactions and satisfaction (logic model 2).
What is the purpose of the evaluation?
The purposes of this evaluation are to evaluate the extent to which (a) the organization and programs help the members of the Community in their personal and professional growth; (b) the participants or students are meeting the goals of the programs, namely the enhancement of student satisfaction and achievement.
Amount of Text: It is well known that the amount of text included in a logic model can vary greatly between logic models. It can be sparse and in point form, or highly detailed. As a matter of preference and the function, my logic models include the information needed for our purposes of presenting the most important issues:
Who will use the evaluation? How will they use the information?
- Teachers, administrators, supervisors, coordinators, volunteers and students.
- To assess the effectiveness of the programs and make changes and improvements to help teachers and students to meet the goals.
-To improve students achievement and satisfaction.
What questions will the evaluation seek to answer?
General questions:
Do Community Programs (CP) help participants or clients in their personal and professional growth and satisfaction? Are Programs meeting the goals set out by the Center? What are the reactions of students regarding those programs? Are they satisfied with their achievement of goals and performance?
Specific questions:
Does the community offer varied programs? Do teachers have adequate resources to implement them? Do they see growth in their students as a result of their CP?
Do Programs encourage students to develop personally and professionally?
Are Programs being used in the way that they are intended?
How are Programs perceived by students? Are they satisfied with their performance in those programs? What are the benefits to students?
Visual Layout: As we know there are many ways to approach visuals and overall layout. This is a highly subjective issue, but an important one as good visual design can greatly enhance the understandability of a logic model. In these cases I tried to avoid confusion and focus on the following questions:
What information do I need to perform evaluation? or to answer the questions?
Indicators – How will I know it? Level of satisfaction of participants.
When is the evaluation needed? At the beginning, in the middle and at the end of the program (s).
What evaluation design will you use? Consumer-Oriented Evaluation Approach.
Assessment and evaluation are best addressed from the viewpoint of the students’ reactions to 1) teachers & teaching, 2) class-assignments and 3) assignments-materials.
Collect the information:
What sources of information will you use?
Existing information:
Web site – Programs – Written materials provided by the Community Center – Teachers materials – samples of students’ work and/or experiences (videos, photos, etc).
People:
Teachers, administrators and the focus will be on students satisfaction.
What data collection method(s) will you use?
E-mail survey questionnaire to students - and teachers - (a larger sample).
Questionnaire- Interview (a small sample of four students).

About Assignment # 5- ECUR 809: The focus for my assignment will be on students’ testimonials of their experience with CP (a sample of four students).My focus will be on Programs: "Spanish Intermediate and Tennis Instruction"

Source: "Logic Models" Online:
http://www.thcu.ca/infoandresources/publications/logicmodel.wkbk.v6.1.full.aug27.pdf

ECUR 809 Assignment # 3

2009-10-13T12:50:00.000-07:00

ECUR 809 Assignment # 3: Evaluation of Organization -Performing an Evaluation Assessment.
Determining the feasibility and direction of my evaluation:
I have selected a community center as organization to use as a model for the rest of the course its programs (Adult General Interest programs such as Intermediate Spanish): http://www.gnag.ca/index.php
http://www.gnag.ca/index.php?page=154 I live close by; so, I can access individuals for input in my work. I chose the City of Ottawa: http://www.city.ottawa.on.ca/ specifically a neighborhood as organization because during the Spring/Summer I taught Spanish (complemented with tennis lessons)(www.moretennis.blogspot.com) as part of the "Ultra Play" program: http://www.ottawatennis.com/detail.php?news_id=294
Kindly please see below overview of my chosen organization:
Organization: A Community Center in the City of Ottawa, ON Canada
Program: "Adult General Interest - Spanish: Intermediate/conversational"
Model of Evaluation Assessment:student-centered evaluation Assessment.
According to the Student Evaluation: A Teacher Handbook(Saskatchewan Education, 1991) student evaluation should focus on the collection and interpretation of data which would indicate student progress. This, in combination with teacher self-evaluation and program evaluation, provides a full evaluation. Chapter one states that, "Assessment and evaluation are best addressed from the viewpoint of selecting what appears most valid in allowing students to show what they have learned." In general, the main phases are the following: preparation, assessment, evaluation (formative, diagnostic, and summative) and reflection. Below each one is briefly described:
Preparation: what is to be evaluated, the type of evaluation (formative, summative, or diagnostic) to be used, the criteria against which student learning outcomes will be judged, and the most appropriate assessment strategies with which to gather information on student progress. Decisions made during this phase form the basis for planning during the remaining phases.In the Spanish Intermediate and Conversation Program the criteria and strategies are guided by an instructor (graduate student) from the University of Ottawa.
Assessment: identify information-gathering strategies, construct or select instruments, administer them to the student, and collect the information on student learning progress. The identification and elimination of bias (such as gender and culture bias) from the assessment strategies and instruments, and the determination of where, when, and how assessments will be conducted are important considerations. Performing an evaluation assessment process of the Program "Adults General Interest," Spanish Intermediate and Conversation Program in the Community Center, City of Ottawa, requires an appropriate approach. The Stake's "responsive" approach seems to be an an adequate way to reporting the "success and failure" of that program. Stake (1975, p.19)recommended the "clock" model to reflect the prominent recurring events in a responsive evaluation: talk with clients, program staff, audiences; identify program scope; overview program activities; discover purposes, concerns; conceptualize issues, problems; identify data needs re issues; select observers, judges, instruments, if any; observe designated antecedents, transactions and outcomes; thematize: prepare portrayals, case studies; validate, confirm, attempt to dis confirm; winnow, for audience use; and assemble formal reports, if any. In this sense, the Stake's model helps in reporting evaluation assessment of Intermediate & Conversation Spanish Program, in which not only questionnaires but also specific tests and sample work portfolios were assessed. See example of past questionnaires.
Evaluation: the information gathered during the assessment phase is used to make judgments about student progress. Based on the judgments (evaluations), decisions about student learning programs are made and reported to students, parents, and appropriate school personnel.
Reflection: allows pondering the successes and shortfalls of the previous phases. Specifically, evaluate the utility and appropriateness of the assessment strategies used, and make decisions concerning improvements or modifications to subsequent teaching and assessment. Instruments contain questions that encourage reflection on student assessment,teachers'planning, and on the structure of the curriculum. In the Intermediate & Conversational Spanish, successes of the program of Intermediate and Conversational Spanish, we can mention the following: excellent Audio CD cassettes; and an exciting vacation with a great learning opportunity, offered in combination with similar programs to study Spanish complemented with other programs, such as sports tennis and golf, games and with latin dance programs.
Until now no failures have been reported. To the contrary, students are looking for more "living spanish" programs.

Sources: Program Evaluation, Particularly Responsive Evaluation (Occasional Paper No. 5, p.19) by R.E. Stake, 1975b, Kalamazoo, MI: Western Michigan University Evaluation Center, Adapted by permission. Cited in pag. 138 in Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation: Alternative approaches and practical guidelines. White Plains, NY: Longman.
Online: www.wmich.edu/evalctr/pubs/ops/ops05.pdf
"Student Evaluation: A Teacher Handbook" Retrieved September 24th, 2009 Online:
http://www.saskschools.ca/~ischool/Drafting10/curr/part25.htm
Flowcharts:
http://www.scribd.com/doc/21010132
http://www.scribd.com/doc/21010006

ECUR 809 Assignment # 2 - Model

2009-09-26T07:38:00.000-07:00

Assignment # 2 Model or approach to evaluate the ECS Programming for Children with Severe Disabilities

Summary: Children with severe/profound disabilities are eligible for Program Unit Funding from Alberta Education. According to the Medicine Hat Catholic organization, the “ECS Programming for Children with Severe Disabilities” evaluates and selects eligible children and then it offers educational programs that must meet the individual child’s needs. The educational programming combines center-based programs and in-home programs. The teacher develops an Individual Program Plan with goals and objectives reflective of the child’s needs. The center-based programming takes place in settings such as preschools, kindergartens and day cares.

What approach is appropriate to evaluate this program? In order to effectively evaluate ECS programming, I suggest using qualitative methods to conduct a “naturalistic” evaluation model in light of Emil J. Posavac and Raymond G. Carey’s theory (2003), combined with the “participant-oriented evaluation” approach, wisely described by Jody L. Fitzpatrick, James R. Sanders and Blaine R. Worthen (2004).

In the naturalistic evaluation model, “the evaluator becomes the data gathering instrument, not surveys or records. By personally observing all phases of the program and holding detailed conversation with stakeholders, the evaluator seeks to gain a rich understanding of the program, its clients, and the social environment” (Posavac and Carey, 2004, p.28). In other words, personal observations and detailed reports are necessary to explain information about the home visits, which should be carefully planned and documented. Also this model is useful in explaining the child’s instruction in a classroom setting at a center or school. The steps in preparing to conduct an evaluation comprise: identifying the program and its stakeholders, becoming familiar with information needs, the planning evaluation and evaluating the evaluation itself.

In the participant-oriented evaluation approach, evaluators should not be distracted from what was really happening to the participant in the program by focusing only in “stating and classifying objectives, designing an elaborate evaluation system, developing technically defensible objective instrumentation, and preparing long detailed technical reports” (Fitzpatrick, Sanders and Worthen, 2004, p. 130).The participant-oriented evaluation stresses first hand experience with program activities and settings and involvement of program participants in evaluation. This approach is “aimed at observing and identifying all (or as many as possible) of the concerns, issues, and consequences integral to the human services enterprise” (p.131). Evaluators need to avoid focusing only on the results or on isolated comments or numbers, charts, figures, and tables, missing important individual facts.

In short, a “naturalist & participant-oriented evaluation” combined approach will provide the plurality of judgments and criteria or methods of inquiry that will help the evaluators portray the different values and needs of individuals and groups served by the educational programming. This model requires active involvement of participants. By involving participants in determining the criteria and boundaries of the evaluation, evaluators serve an important educative function by creating “better-informed” program participants.

Sources:

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation: Alternative approaches and practical guidelines. White Plains, NY: Longman.

Posovac, E., & Carey, R. (2003). Program Evaluation:Methods and Case Studies. (6th edition). New Jersey: Prentice Hall

Medicine Hat Catholic, Separate Regional Division # 20.
Retrieved Sept 13th, 2009 from
http://www.mhcbe.ab.ca/cec/

A Revised Version of Assignment # 1

2009-09-23T07:56:00.000-07:00

Assignment #1 A completed evaluation Case: Client Satisfaction Survey Analysis, Quebec Regional Office, Human Resources Development (HRDC) in Quebec, Canada, 2001.

The study is an analysis of requirements and proposal for a client satisfaction measurement program. Circum Network Inc. is listed as the author of this study. It was prepared for Evaluation Services Information and Strategic Planning Directorate Quebec Regional Office Human Resources Development.

Model or process used in the evaluation: An improvement-focused model (Posovac, 2003, p.29). While providing the essential methodological foundations, “the self-directed” training document takes a pragmatic and “integrated” approach to conducting client satisfaction surveys. It includes devices such as decision trees and checklists. The project was carried out in three phases. First, the team developed the standardized questionnaires. Then the researchers developed an operational framework for the client satisfaction measurement program. Finally, they developed the analytical support tools and mechanisms.According to the report, there is a consensus within the HRDC-Quebec Region that systematic, rigorous measurement of client satisfaction with the products and services offered by the Region is essential to building the fifth pillar of the region’s vision: delivering services of the highest quality. There is also a consensus that the primary responsibility for improving the region’s services lies with the HRCCs and other operational centres, because it is they that control the daily delivery of services.

Strengths: (1) The goals were to plan a client satisfaction measurement program and an analysis of requirements and proposal for a client satisfaction measurement program; the team produced a self-directed training document on the implementation of client surveys. (2) The report presents clearly the development of standardized questionnaires, operational and implementation framework and pre-testing. (3) The report is a Guide for human resources development (employees who do not necessarily have the knowledge required to conduct formal, systematic surveys). The tools offered in the guide are to be used for measuring, sampling, collecting data, analyzing data, interpreting results, and implementing recommendations. The results included standardized questionnaires for various types of clients and various services’ conditions. (4) Researchers/evaluators helped programs’ staff to learn how to discover discrepancies between program objectives and the needs of the target population, between program implementation and program plans, between expectations of the target population and the services actually delivered, or between outcomes achieved and outcomes projected (Posovac, 2003, p.29).
In this sense, I think in this case, it was used an approach similar to Stake's countenance model explained by Jay Wilson (2009) because there was a need for formalized evaluation. It was not just anecdotal but descriptive data was necessary. It included description and judgment, intents, and observations, which were compared to standards then a judgment was made. In short, there was a "mix" or mixture of parts pieces (quantitative/qualitative elements) and, as a result, it was an “artistic” evaluation (creative thinking in the minds of evaluators).
Regarding possible weaknesses, I see a couple of things that could be considered in future program evaluations: (1) the program does not compare or discusses all sides of program evaluation both positive and negative. Deliberation of pros and cons are not evident in the survey evaluation of the program. It would be useful if this discussion or deliberative process can take place. (2) It does not describe participant’s “reactions” to and “learning” from the innovative program evaluation, as well as “behavior” changes in real job performance, and other potpourri “results.” A round table or discussion of these issues could help to enlightening program evaluation.

Sources:
Circum Network Inc. (2001) An integrated approach to conducting client satisfaction surveys analysis of requirements and proposal for a client satisfaction measurement program. Prepared for Evaluation Services’ Information and Strategic Planning Directorate Quebec Regional Office Human Resources Development Canada.
Retrieved September 7, 2009 from:
http://www.circum.com/textes/program_hrdc_quebec_2001.pdf

David Crawford (2009) Evaluation exploration. Retrieved September 4, 2009, from http://www.ag.ohio-state.edu/~brick/evalexpl.htm

Miller, R., & Butler, J. (2008) Using an adversary hearing to evaluate the effectiveness of a military program. The Qualitative Report, 13(1), 12-25. Retrieved September 5, 2009 from http://www.nova.edu/ssss/QR/QR13-1/miller.pdf

Posovac, E., & Carey, R. (2003). Program Evaluation: Methods and Case Studies. (6th edition). New Jersey: Prentice Hall.

ECUR 809 Assignment #1 (First Version)

2009-09-09T19:53:00.000-07:00

ECUR 809.3-83551 Assignment #1 Case: Client Satisfaction Survey Analysis, Quebec Regional Office, Human Resources Development (HRDC) in Quebec, Canada, 2001.

Summary: Circum Network Inc. is listed as the author of this study. The program evaluation was prepared and developed for Evaluation Services Information and Strategic Planning Directorate Quebec Regional Office Human Resources Development. It is an analysis of requirements and a proposal for a client satisfaction measurement program. It is a guide for human resources development (employees who do not necessarily have the knowledge required to conduct formal, systematic surveys), that is, a self-directed training document on the implementation of client surveys. The tools offered in the guide are to be used for measuring, sampling, collecting data, analyzing data, interpreting results, and implementing recommendations. The document presents clearly the development of standardized questionnaires for various types of clients and various services’ conditions, operational and implementation framework and pre-testing.

The model or process used in the evaluation is the improvement-focused model.
The purpose is to help programs’ staff to learn how to discover discrepancies between program objectives and the needs of the target population, between program implementation and program plans, between expectations of the target population and the services actually delivered, and between outcomes achieved and outcomes projected, among others (Posovac, 2003, p.29). While providing the essential methodological foundations, “the self-directed” training document takes a pragmatic and “integrated” approach to conducting client satisfaction surveys. It includes devices such as decision trees and checklists. The project was carried out in three phases comprising the development of (1) the standardized questionnaires, (2) the operational framework for the client satisfaction measurement program, and (3) the analytical support tools and mechanisms. According to the report, there is a consensus within the HRDC-Quebec Region that systematic, rigorous measurement of client satisfaction with the products and services offered by the Region is essential to building the fifth pillar of the region’s vision: delivering services of the highest quality. There is also a consensus that the primary responsibility for improving the region’s services lies with the HRCCs and other operational centers, because it is they that control the daily delivery of services.

I see a couple of things that could be considered in future program evaluations: (1) the program does not compare or discusses all sides of program evaluation both, positive and negative. Deliberation of pros and cons are not evident in the survey evaluation of the program. It would be useful if this discussion or deliberative process can take place. (2) It does not describe participant’s “reactions” to and “learning” from the innovative program evaluation, as well as “behavior” changes in real job performance, and other potpourri “results.” A round table or discussion of these issues could help to improve the process of program evaluation. Nelson

Sources:
Circum Network Inc. (2001) An integrated approach to conducting client satisfaction surveys analysis of requirements and proposal for a client satisfaction measurement program. Prepared for Evaluation Services' Information and Strategic Planning Directorate Quebec Regional Office Human Resources Development Canada. Retrieved September 7, 2009 from:
http://www.circum.com/textes/program_hrdc_quebec_2001.pdf

David Crawford (2009) Evaluation exploration. Retrieved September 4, 2009, from http://www.ag.ohio-state.edu/~brick/evalexpl.htm

Miller, R., & Butler, J. (2008) Using an adversary hearing to evaluate the effectiveness of a military program. The Qualitative Report, 13(1), 12-25. Retrieved September 5, 2009 from http://www.nova.edu/ssss/QR/QR13-1/miller.pdf

Posovac, E., & Carey, R. (2003). Program Evaluation: Methods and Case Studies. (6th edition). New Jersey: Prentice Hall.

Please see above a more complete version of my Assignment # 1 (after Dr. Wilson's comments, Sept 25th, 2009)

What is Program Evaluation?

2009-09-02T09:08:00.000-07:00

First Session:
To define and understand “What is program evaluation?”
To understand the historical foundations of program evaluation.
To identify and develop appropriate evaluation assessment techniques used in educational and other program settings.

"Program evaluation is the systematic collection of information for use to improve effectiveness and make decisions with regard to what those programs are doing and affecting." University of Minnesota http://www.evaluation.umn.edu/

"Essentially you are trying to answer the question, "Does the program do what it says it does?". Because evaluation is on-going your evaluation may steer your client in a particular direction and it will also be used to inform the next evaluation" (Jay Wilson, 2009)

I found the following useful links:
http://www.epa.gov/evaluate/whatis.htm
In its broadest definition, Program Evaluation is a systematic way to learn from past experience by assessing how well a program is working.
- Program evaluation is almost always retrospective, i.e., examining and learning from experience, though it may include prospective elements. For example, an analytical study that makes use of data on past performance to estimate future results would be an evaluation, but one done prospectively to estimate the effectiveness of a new environmental program based on assumptions about its design and/or operation would not be.
- An evaluation can be systematic without being elaborate or expensive. It’s possible to keep it simple and affordable

http://www.epa.gov/evaluate/whatis.pdf

http://www.ocde.k12.ca.us/downloads/assessment/WHAT_IS_Program_Evaluation.pdf

http://www.en.wikipedia.org/wiki/Program_evaluation
Program evaluation is a systematic method for collecting, analyzing, and using information to answer basic questions about projects, policies and programs. Program evaluation is used in the public and private sector and is taught in numerous universities.. Program evaluations can involve quantitative methods of social research or qualitative methods or both. People who do program evaluation come from many different backgrounds: sociology, psychology, economics, social work.

http://gsociology.icaap.org/methods/evaluationbeginnersguide.pdf

http://non-profit-governance.suite101.com/article.cfm/board_member_selfassessment
umass.edu/oapa/oapa/publications/online_handbooks/program_based.pdf
http://www.macleans.ca/education/universities/article.jsp?content=20070323_155000_816
http://www.medicine.usask.ca/pt/general-information/school-of-physical-therapy-operations-manual-1/Program%20Evaluation.pdf/view
http://www.evaluationcanada.ca/site.cgi?s=4&ss=2&_lang=an
http://www.uwex.edu/ces/pdande/evaluation/index.html

Requested Readings:
http://www.managementhelp.org/evaluatn/fnl_eval.htm
http://pathwayscourses.samhsa.gov/eval101/eval101_toc.htm
http://jan.ucc.nau.edu/edtech/etc667/proposal/evaluation/summative_vs._formative.htm
http://delicious.com/wi11y0/809
http://www.evaluationcanada.ca/site.cgi?s=1
http://www.eval.org/

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R.(2004). Program evaluation: Alternative approaches and practical guidelines. White Plains, NY: Longman.
Owen, J. M., & Rogers, P. J. (1999). Program evaluation: Forms and approaches. Thousand Oaks, CA: Sage.
Posovac, E., & Carey, R. (2003). Program Evaluation â€“ Methods and Case Studies. (6th edition). New Jersey: Prentice Hall

First assignment: a description of how to do program evaluation in Canada
http://www.spcottawa.on.ca/CBRNO_website/How2program_evaluation.htm#Client
http://www.mhcbe.ab.ca/cec/specialeducation-studentservices/ECSPROGRAMMINGFORCHILDRENWITHSEVEREDISABILITIES.pdf

PENDING ASSIGNMENTS
Assignment # 5 Design and test a short survey. Include a variety of question types such as scale rating, short answer, and open-ended. Original version and the modified version based on the testing of the survey with four individuals. Deadline: November 20th

Major Assignment: Project
Evaluation Plan (Proposal) Dec. 11 - 50 marks
A Proposed Evaluation of the “Spanish Intermediate & Conversational and Ultra Play - Tennis After School” Programs in a Community Center: A Case Study of the City of Ottawa: Parks & Recreation Master Plan Experience
By Nelson Dordelly-Rosales
The purpose of this paper will be to design an evaluation plan after completion the “Ultra Play - After School” Program in Sandy Hill Community Center: A Case Study of the City of Ottawa – Parks & Recreation Experience. The plan will be a theoretical paper that outlines the program and the goals or objectives to be evaluated. It will demonstrate my ability to analyze a program, integrate the different tools and theories addressed recently into an evaluation plan, determine a suitable evaluation plan and create the instruments I would use to conduct the analysis. Essentially the purpose of this evaluation plan is to convince Mr. Martin Travis, Coordinator of Parks and Recreation that I should be “the evaluator for the evaluation.” Hence, I want to convince him that I am the “best” to perform the evaluation of the above mentioned program, and that I have the best team to help me on this matter.
Through case study, this paper will lend insight to ways through a program-based evaluation, or a logic “improvement-focused” model (Posavac, et. al, 2003) can facilitate a “holistic” approach to the evaluation of ‘Ultra Play-After School program.’ So, an important piece of this evaluation plan is to describe, or elaborate upon, different reasons for selecting this particular model and approach. While the term “program” is used, I find a logic model equally useful for describing group work, team work, community-based collaborative and other complex organizational processes as I seek to promote results-based performance. For presentation of paper, I will use a case study format that includes following components:
• Abstract – a brief summary of the major points of the study as well as a short list of key words.
• Introduction
Prior evaluations or policies in a Community in Ottawa:
http://www.gnag.ca/index.php
http://www.gnag.ca/index.php?page=67&id=20

In comparison to:
http://www.topeka.org/parksrec/garfield.shtml

http://www.uml.edu/centers/CFWC/Community_Tips/Program_Evaluation/Program_Evaluation.html

http://www.skaneatelescommunitycenter.com/index.php?option=com_facileforms&Itemid=124

A case in Florida Miami Beach:
http://www.shakealegmiami.org/site/c.kkLUJbMQKpH/b.2521629/k.BF03/Home.htm

Program or outcome evaluation assesses the extent to which planned activities produce the desired outcome among a target population. Evaluation is considered and set up when the project is designed. Program or outcome evaluation assesses the extent to which planned activities produce the desired outcome among a target population. The PE model is led by the Planning and Evaluation Committee consisting of key representatives from the collaborating organizations Planning, Development and Communications is the staff support unit charged with assisting the organization in its efforts to improve its ability to become a self-correcting organization through planning, monitoring and evaluation efforts.
http://www.ccpfc.org/rd/eval_center.cfm

Importance: http://www.cdc.gov/eval/framework.htm

Examples:
www.phoenix.gov/ARTS/eval0405.pdf

http://www.afterschoolflorida.hhp.ufl.edu/evaluation_links.html

Models of Evaluation:
http://www.edtech.vt.edu/edtech/id/eval/eval_models.html

o Defining and addressing the need to change program evaluation methods that rely heavily on data gathering by postal mail to online instruments.
o What do you propose to do?
o What is my plan?
o What are my objectives: INPUT
o Involving stakeholders
• Model & Method: OUTPUT
o Description - How to do it?
o Evaluation Matrix
o Reasons for selecting particular foci and approaches
o What are the challenges and or roadblocks?
o What are the assessment instruments to be used?
o Measurement considerations & Data collection
• Evaluation: OUTCOME
o How well can we meet my objectives?
o Why I and my team are the best to perform the evaluation
o Emerging trends
• IMPACT:Conclusion, Summary and Recommendations
o Summarize what I learned from this experience.
o What would I recommend to others who would like to replicate my efforts?
o What would they need to be prepared for?
o What needs to be improved?
o Strengths and weaknesses
o Other issues

Schedule Fall-Winter 2009-2010

2009-08-29T14:26:00.000-07:00

Fall Dr. Jay Wilson
ECUR 809.3-83551 Models & Methods for Evaluation of Educational Programs.
Calendar: 9 am 250 pm

September 5 - 26
October 17
November 21

ECUR 990- 82712 Curriculum Research Dr Janet McVittie --- Education Building 10 Sep 03, 2009 - Dec 04, 2009 Seminar
http://www.usask.ca/education/people/mcvittiej.htm
http://www.usask.ca/education/coursework/mcvittiej/edcur322.html
http://www.usask.ca/education/coursework/mcvittiej/resources/index.html

Elluminate sessions: September 19 - October 3
November 7 and 21 -

January 16, February 27
March 13 - March 27

GRS 960-86150 Ethics & Integrity Dr. Diane J. Martz
http://www.spheru.ca/spheru-1/research-tem/dr.-diane-martz/dr.-diane-martz/?searchterm=Martz

Blackboard 2 hrs take a test until you get CR Sep 03, 2009 - Dec 04, 2009 Supervised Self Instruction
Research by Dr Martz: http://www.spheru.ca/research-projects/rural-youth-risk-behaviours-and-healthy...
Winter
ECUR 991-27193 Scholarship in Teaching - Portfolios Dr Timothy. Molnar http://www.usask.ca/education/people/molnart.htm
Class 9:00 am - 11:50 am S Education Building 3133
Jan 04, 2010 - Apr 08, 2010 Seminar
Saturday
9:00-11:50pm

Articles Reviews by Nelson Dordelly-Rosales

2009-08-26T13:56:00.000-07:00

Article Review # 1
Bringing the background to the foreground: what do classroom environment that support authentic discussions look like?

References:

Hadjioannou, X. (2007). Bringing the Background to the Foreground: What do Classroom
Environment that Support Authentic Discussions Look Like? American Educational
Research Journal; 44 (2), 370-399

Mapiasse, S. (2007). Influence of the democratic climate of classrooms on student civic learning in North Sulawesi, Indonesia [Electronic version]. International Education Journal, 8 (2), 393-407.
Overview
Hadjionannou (2007) focused on authentic or dialogic discussions in the classroom. Authentic discussions are a classroom-based speech genre in which participants commonly explore issues of interest by articulating ideas and opinions. A case study research is done to shed light on “authentic discussions” using different qualitative approaches such as recorded class sessions, interviews, and field notes. The researcher identified seven elements that appeared to be related to the student’s involvement in classroom activities and to the social relationships among community members. Those elements were physical environment, curricular demands and enacted curriculum, teacher beliefs, student beliefs about discussions, relationships among members, classroom procedures, and norms of classroom participation.
Problem/Issue and the Importance/Significance
Hadjionannou (2007) aimed to answer the question what do classroom environments that support authentic discussions look like? The study examines the features of the environment of a fifth grade classroom community. The author reported part of a wider qualitative research that sought (a) to examine the issue of interpersonal relationships within the classroom, specifically to analyze the texture of talk in the authentic discussions of the community under study, (b) to explore participant perspectives, and (c) to evaluate the classroom environment.
Research Question
What are the features of the classroom environment of this discourse community that frequently used authentic discussions?
Sample and sample selection process
The community under study was a fifth-grade class of 24 students and their teacher. The fifth-grade classroom community under study was part of Grassroots Elementary School (pseudonym), a quintessentially middle-class school in a midsize town in Florida.
Data Collection Method/ Data Analysis Method
Data collection included observation, participant interviews, and audio and video recordings of class sessions during a five month period on an almost daily basis. The process of identifying the major elements of the classroom environment was a generative one, and it began with the initial coding of the field notes and the interview transcripts. Each individual was interviewed four times, using a flexible interview protocol: the author audio and video-recorded four book talk sessions, which were transcribed verbatim and analyzed through discourse-analysis. The goal was to use a database with highly contextualized descriptors to systematically illustrate the content of the data. The findings of the discourse analysis were used primarily for describing the texture of talk in authentic discussions, but also capturing the elements of the classroom environment in action.
Trustworthiness/Validity Considerations
In addressing credibility, Hadjionannou (2007) presented a detailed picture of the phenomenon under scrutiny. The investigator provided sufficient detail of the context of the ﬁeld-work, identified the elements that seemed to shape the environment of the classroom community under study, and described “how those elements functioned as repeatable threads woven to create the fabric of the classroom’s social life.” (p. 374). The researcher suggested that reproducing the environment described in this study in another classroom would be impossible because “the environments in communities are in constant flux, and they are shaped by the personalities and the agendas of community members as well as by the unique circumstances of each community” (p.396). However, through dialogic discussions in the classroom and cultivating amiable relations between the students, teachers can provide opportunities for student self-expression, lively interactions, and substantive collaboration in any classroom.
Ethical Issues
Confidentiality and anonymity was guaranteed to participants of the study. In the ethics literature, confidentiality is commonly viewed as akin to the principle of privacy. In this study the researcher used a pseudonym to identify the institution object of study or elementary school.
Reflection - Questions
Were the studies of value? Why or why not? Mappiasse (2007) examined the influence of the democratic climate of classrooms on student civic learning in North Sulawesi, Indonesia, and analyzed seven dimensions that support democratic climate in the classroom: active participation, avoidance of textbook dominated instruction, reflective thinking, student decision-making and problem-solving choices, controversial issues, recognition of human dignity, and relevance. Hadjionannou (2007) explored the environment of the fifth-grade classroom community in Florida and analyzed the elements that support authentic discussions. Those elements were physical environment, curricular demands and enacted curriculum, teacher beliefs, student beliefs about discussions, relationships among members, classroom procedures, and norms of classroom participation. Both studies identified the elements that seem to shape the environment of the classroom and described how those elements functioned as repeatable threads woven to create the fabric of authentic discussions and a democratic climate in the classroom’s social life. The two studies provide insights of great value for teaching and learning.
What were the strengths of the two studies? The results indicated that the democratic climate and authentic discussions have significant effects on student engagement, student knowledge and interpretation skill. Mappiasse (2007) centered in the advantages of democratic environment in the classroom; Hadjionannou (2007 emphasized in the importance of interpersonal and social interaction among students and teachers. The classroom environment is extremely important to effective teaching and learning. These studies described in great detail different indicators of a good classroom social environment; they also are good examples of qualitative research.
What were the limitations of the two studies? Subjects in each study were students of only one institution. Therefore, the results were limited in their applicability to other institutions. Similar research studies should be repeated in other institutions and in different subjects to determine whether those aspects of the classroom environment that appeared essential to effective teaching are similar to those obtained in these studies.
How would you have changed the two studies to improve the quality of the research? For the first study, I would enlarge the sample size and I would add a questionnaire for data collection. For the second study, I would add participant interviews and audio and video recording of class sessions. In addition, for the first study, it would also be necessary to explain instrument validation and the reliability of items as the second study did.
How would you incorporate the findings of the two studies into your classroom? I would like to develop a similar qualitative research study selecting a convenience sample of schools in Venezuela. In research and teaching, I would incorporate the democratic environment using meaningful classroom activities. I would work toward knowing my students and use this knowledge to create positive, trusting, and respectful relationships with them.
It is important to make students engage in authentic dialogue or discussion and learning activities, especially in civic education classrooms that involve law and education. We should provide the students with opportunities to obtain deeper understanding of the civic values and enable them to implement democratic values critically and responsibly in their social interactions; that is, to engage individuals and groups in developing a clear statement of belief about what strong democracy would look like.

Article Review # 2 by Nelson Dordelly-Rosales
Investigating Self-Regulation and Motivation:
Historical Background, Methodological Developments, and Future Prospects

References:

Zimmerman, B. (2008). Investigating Self-regulation and Motivation:
Historical Background, Methodological Developments, and Future Prospects
American Educational Research Journal; 45 (1), 166-183

Zimmerman, B. (2002). Becoming a Self-regulated Learner: an Overview.
Retrieved June 2, 2009 from
http://findarticles.com/p/articles/mi_m0NQM/is_2_41/ai_90190493/

Overview
Zimmerman (2008) assessed students’ self regulated learning (SRL) online. The focus was on processes and motivational feelings or beliefs regarding learning in authentic contexts using computer ‘traces’ (or gStudy software), think-aloud protocols, diaries of studying, direct observation, and microanalyses measures. The results revealed that students in high SRL online classes were more engaged in their writing than students in low-SRL classes, and that students in the training group reported significantly greater increases in time management skill and self-reflection on their learning than those in the control group. Students in the self-regulation training condition also displayed increases in several measures of motivation. Their willingness to exert effort, their task interest, their learning-goal orientation, and their perceptions of self-efficacy all increased after training and their feelings of helplessness declined significantly. Students in “the self-regulation training group displayed significantly greater gains in math achievement than students in the control group” (p.175).
Problem/Issue and the Importance/Significance
The study defined the issue of innovative environment and how it impacts the students’ use of self regulatory processes during the course of learning. The study is significant because it enlightens the motivation and self-regulation process. One of the lessons for instructors and learners was that self-regulation strategy measure can predict students’ academic grades and their teachers’ ratings of their proactive efforts to learn in class.
Research Questions
The first question concerned the innovative software program (called “gStudy environment”), that is, how traces measure SRL as compared to self-reported measures. The researcher assessed changes in self-regulation during learning. The second question dealt with students’ levels of SRL in personally managed contexts, such as at home or in the library. The idea was to find out if students’ levels of SRL were linked to improvements in the students’ overall academic achievement. The third question involved whether teachers can modify their classrooms to foster increases in self-regulated learning. The fourth question concerned the role of students’ motivational feelings and beliefs in initiating and sustaining changes in their SRL.
Sample and sample selection process
Teachers were randomly assigned to either an experimental or a control group. Nine teachers were trained to convey the underlying cyclical model and to develop homework exercises, quizzes, and a final examination in arithmetic skill. The control group of eight teachers gave the same homework assignments and tests but received no self-regulation training. The students in both experimental conditions kept diary accounts of SRL events.
Data Collection/Analysis Methods
The author used innovative qualitative as well as quantitative methods that included teacher and student data collection and different analysis methods (observation forms, portfolio assessments, interviews, and questionnaires) to measure SRL. Teachers in the SRL training condition gave students a copy of the cyclical model of self-regulation along with a picture of a ‘learning expert’, who recommended self-regulatory practices that the teacher modeled for them. Students were given daily feedback and were encouraged to set challenging goals and choose a specific strategy for themselves. Students in the experimental group were given points on the basis of their homework answers. The students were assessed in their interests, attitudes, and self-related cognition before and after a five week training program. The students’ calibration of the accuracy of their achievement was significantly correlated with their actual posttest score.
Instructional and ethical issues
Technology is a tool that can change the nature of ESL. However, the role of the teacher and instructor is critical in providing guidance and support to self-regulated academic learning.
Reflection - Questions
Were the studies of value? Why or why not? In previous study, Becoming a Self-Regulated Learner, Zimmerman (2002) showed that self-regulation learning (ESL) is not a mental ability or an academic performance skill; rather it is the self-directive process by which learners transform their mental abilities into academic skills. The author identified how a student’s use of specific learning processes, level of self-awareness, and motivational beliefs combine to produce self-regulated learners. In the most recent study, Zimmerman (2007) showed that when compared to control students, SRL trained students displayed significant increases in homework effectiveness, time management skills, a broad array of self-reflection measures, and math performance skill (in fact, the self-regulation training group passed an entrance exam for admittance to a higher level school, which was an increase of 50% compared to past cohort). Both studies were of value
What were the strengths of the two studies? Zimmerman (2000) showed that self-regulated, independent learners take responsibility for what they learn and he analyzed how far they can go with this knowledge. In the second study Zimmerman (2007) showed that (1) “gStudy environment” can provide students with many more ways to self-regulate their learning than provided by traditional instructional software, (2) the “think-aloud methodology” is an effective way to assess students’ self-regulatory processes online, (3) training in self-regulation learning and time-management skills can be implemented by teachers as part of their classroom assignments and strategic planning, and (4) the “micro-analytic methodology” (used to improve athletic skills) for assessing SRL processes and sources of motivation (goal setting and strategic planning, self-reflection, predictive sources of motivation) improve self-regulation. The results showed that the experimental group reported significantly greater increases in time management skill and self-reflection on their learning, homework effectiveness, time management skills, a broad array of self-reflection measures, and math performance skill, than the control group.
What were the limitations of the two studies? In general, there are still raising new questions for future research: more research is needed regarding the accuracy of students’ reports of using self-regulatory processes. In trying to answer the global question: How do students become masters of their own learning processes? Zimmerman (2007 says that “there was not a standardized measure of students’ writing achievement, and this limitation precluded determination of the effects of students’ SRL on their writing competence” (176). Students in the high and low-SRL classes did not display significant differences in measures of motivation (beliefs, values etc), which is attributed to the ineffectiveness of the measures.
How would you have changed the two studies to improve the quality and usefulness of the research? I would follow Zimmerman’s research approach (2007) and would take his suggestions. There is a need to (a) extend the use of the four ways to assess the effectiveness of academic interventions designed to motivate recalcitrant students to engage on SRL, (b) extend a micro analytic methodology to learning academic tasks over longer periods of time when students’ motivation is expected to wane, (c) apply additional measures of motivation and feelings, such as anxiety and goal orientation, (d) extend the “think-aloud methodology” to see if planning and motivation will emerge as significant predictors of students’ mental models study.
How would I incorporate the findings of the two studies into your classroom? I would provide innovative environment (gStudy software, think-aloud protocols, diaries of studying, direct observation, microanalyses measures) so that students become masters of their own learning process: SRL a “proactive processes that students use to acquire academic skill, such as setting goals, selecting and deploying strategies, and self-monitoring one’s effectiveness” (166).

Article Review # 3 by Nelson Dordelly-Rosales

Students’ Perceptions of Characteristics of Effective College Teachers: A Validity Study of a Teaching Evaluation Form Using a Mixed-Methods Analysis
By Anthony J. Onwuegbuzie; Ann E Witcher; Kathleen M T Collin; Janet D Filer; e al

Reference(s):

Onwuegbuzie, A.J., Witcher, A. E., Collin, K.M., Filer, J.D., et al. (2007). Students’ Perceptions of Characteristics of Effective College Teachers: Validity Study of a Teaching Evaluation Form Using a Mixed-Methods Analysis. American Educational Research Journal; 44 (1), 113-160

Suwandee, A. (1995). Students' perceptions of university instructors' effective teaching characteristics [Electronic version]. Studies in Language and Language Teaching Journal,5, 6-22

Overview
Onwuegbuzie, et al. (2007) assessed the content-related validity and construct-related validity of the Teaching Evaluation Form (TEF). Using sequential-mixed methods analysis lead the researchers to the development of a more complete form, the CARE-RESPECTED Model of Teaching Evaluation (CRMTE), which includes three of the least represented themes of the TEF: student-centered, enthusiast, and ethical. The words consistency, fair evaluator, and respectful describe the item ethical. The CRMTE is a useful data-driven test that will benefit all stakeholders –college administrators, teachers, and, above all, students.
Problem/Issue and the Importance/Significance
The problem was students’ perceptions of characteristics of effective college teachers: a validity study of a teaching evaluation form using a mixed-methods analysis. To Onwuegbuzie, et al. (2007) “the TEFs (a) are developed atheoretically and (b) omit what students deem to be the most important characteristics of effective college teachers” (p.151). In an era in which information gleaned from TEFs is used to make decisions about faculty, this potential threat to validity is disturbing and warrants further research.
Research Questions
What themes reflect effective college teachers as identified by students? What students’ attributes affect perceptions of effective college teachers? What is the content-related validity and construct-related validity pertaining to a TEF?
Sample and sample selection process
Participants were 912 undergraduate and graduate students (out of 8,555 students enrolled) from various academic majors enrolled at a public university in a mid-southern state of the United States. The sample represented 10.7% of the total population and reflected 68 degree programs offered by the university.
Data Collection and Analysis Methods
This study used a multistage mixed-methods analysis to collect and to assess the content-related validity and construct-related validity of TEF. The researchers approached instructors/professors before the study began to solicit participation of their students and thus maximize participation rate. The researchers collected qualitative data (e.g., respondents’ perceptions of the questionnaire), and quantitative data (e.g., response rate information, missing data information) before the study began (plot phase) and used member checking techniques to assess the appropriateness of the questionnaire and the adequacy of the time allotted to complete it, after the major data collection phases. A sequential mixed-methods analysis (SMMA) was undertaken to analyze students’ responses. The process included: data reduction, data display, data transformation, data correlation, data consolidation, data comparison and data integration. This analysis, incorporating both inductive and deductive reasoning, employed qualitative and quantitative data-analytic techniques.
Limitations/Delimitations/Assumptions
Because the sample represented students at a single university whose perspectives about effective teachers were gathered at a single point in time, the extent to which the present findings are generalizable to students from other institutions is not clear.
Trustworthiness/Validity Considerations
The focus of the study was on population validity, ecological validity, temporal validity and adequate external validity. The findings cast some serious doubt on the content-related validity and construct-related validity of TEF scores (e.g., endorsement of most themes varied by student attribute: gender, age). The validity of responses might have been affected by the fact that “the students’ perceptions were assessed via a relatively brief-self-report instrument” (p.144).
Reflection - Questions
Were the studies of value? Why or why not? Both studies were of great value. In the study by Suwandee (1995) data was obtained from 505 university students in the Faculty of Science. The results indicated that students considered an effective teacher as one who has a good knowledge of his/her subject and applies pedagogical skills, making difficult topics easy to understand and explains clearly; his/her personality is generous, willing to help students in and out of the classroom; and his/her research-teaching background shows a well-prepared instructor for class. Onwuegbuzie and others (2007) identified characteristics that students considered effective college teaching comprising four metha-themes, which were the following: communicator, advocate, responsible, and empowering, and nine themes comprising the following descriptors: responsive, professional, expert, connector, transmitter, director, enthusiast, student centered, and ethical. The researchers developed the CARE-RESPECTED Model of Teaching Evaluation (CRMTE) that emerged from the study, which included the last three descriptors, which were not represented in the TEF. These two studies have added to the current yet scant body of literature regarding the score validity of TEFs.
What were the strengths of the two studies? The studies examined students’ perceptions of characteristics of effective college teachers and the factors that are associated with their perceptions. The researchers used mixed methods for the rationale of optimizing participant enrichment, instrument fidelity and significance enhancement. Findings included a more complete test and the identification of prevalent characteristics, themes and metha-themes for faculty training.
What were the limitations of the two studies? Subjects in both studies were students of only one university. The results are, therefore, limited in their applicability to other institutions. For this reason, similar research studies should be repeated for students in other universities to determine whether their perceptions of effective teaching are similar to those obtained in those two studies. None of these studies found any relationship between GPA and students' perceptions of teaching characteristics. Further research studies should be carried out to determine if there is any relationship between both variables.
How would you have changed the two studies to improve the quality and usefulness of the research? The two studies illustrated how to use a multistage mixed-methods analysis to assess the validity of the teaching evaluation forms. Future research studies should be carried out using a multistage mixed-methods analysis and involve instructors as subjects to determine their perceptions of valued teaching characteristics. Conducting a study using both students and instructors in an educational institution as subjects would improve validity of results. The results obtained for each group can then be compared to determine whether any congruency or discrepancy is observable between students' and instructors' perceptions of effective teaching.
How would you incorporate the findings of the two studies into your institution? We should promote the highest academic standards in our teaching, our scholarship, and the connections between them. Specifically, I should be able to apply the characteristics of teaching that emerged from those studies. I would attempt to do similar research in my home institution. In Venezuela the current TEFs forms do not represent all characteristics that students consider to reflect effective college teaching. Findings regarding the characteristics of effective teaching can be inputs for faculty training. We should provide teaching support and conduct training for faculty, teaching assistants and librarians.

Article Review # 4 by Nelson Dordelly-Rosales
Can Teacher Education Make a Difference?
By Niels Brouwer and Fred Korthagen

References:

Brower, N., & Korthagen, F. (2005). Can Teacher Education Make a Difference?
American Educational Research Journal, 42 (1), 153-224

Crocker, R., & Dibbon, D. (2008). Teacher Education in Canada. Retrieved May 24, 2009
from www.saee.ca/pdfs/Teacher_Education_in_Canada.pdf

Problem/Issue and the Importance/Significance
Brower and Korthagen (2005) examined the graduates’ teaching competence originated from their pre-service programs, as observed in one university teacher education institution that aimed deliberately at integrating practice and theory. This longitudinal study of over a period of 4.5 years aimed to examine the impact of specific characteristics of the teacher education programs in the United States involving the integration of practical experience and theoretical study. The research model included eight variables: curriculum program conditions, non-curricular program conditions organization and content of activities during student teaching, organization and content of activities during college-based seminars, learning effects during pre-service programs, schools context factors during beginning teachers’ entry into the profession, beginning teachers’ experiences and options, learning effects during the first in-service years and personal background variables. The researchers demonstrated that occupational socialization in schools has a considerable influence on the development of graduates’ in-service competence (educating “innovative teachers”). They discussed specific ways in which pre-service teacher education can influence beginning teachers’ professional performance and competence development.
Research Questions
How does teaching competence develop over time? What are the relative influences of teacher education programs and occupational socialization in schools on the development of teaching competence? Which program characteristics are related to competence development? Does the program require beginning teachers to display, in real life situations, the competence that their pre-service programs aimed to foster?
Sample and sample selection process
The whole sample included “357 students, 128 cooperating teachers and 34 university supervisors from 24 graduate teacher education programs. On average, the beginning teachers in the sub sample had more teaching experience, ranging between 12 and 30 months after graduation, than those in the whole sub-sample, which ranged between 11 and 22 months after graduation”(Brower, & Korthagen, 2005, p.155). The reason is that the observations of and interviews with the beginning teachers in the sub-sample were based in part on their questionnaire responses. In order to ensure that sub- samples were representative as possible, the researchers applied several criteria, for example, the largest possible number of school subjects. From the total number of 31 university supervisors, those with the most professional experience were selected.
Data Collection Method/ Data Analysis Method
Quantitative survey data as well as in-depth qualitative data were collected using quantitative and qualitative methods: a longitudinal survey, interviews, observations, a written questionnaire (closed items), and classroom artifacts (program documents). The first step was to determine which activities were carried out in each program, in which order, and at which moments. Then all of the information was schematized. In the questionnaire, repeated measures were used to describe how the programs were implemented, to trace how the students experienced them, and to record their self-evaluations of their progress on the criterion variable. After graduating and finding work, the beginning teachers answered specific questions. After the programs had ended, the graduates completed one additional questionnaire (a few factual questions for those graduates who had not found work as beginning teachers). The University supervisors completed a questionnaire after completion of the entire program. Findings were reported from three epistemological perspectives: the ecological (collaboration and contextual conditions), the genetic (beginning teachers’ experiences) and the activity perspective (respondents’ actions in classroom and schools).
Reflection - Questions
Were the studies of value? Why? The study on teacher education in Canada by Crocker and Dibbon (2008) examined program structures, content emphasis and usefulness, perceptions of teaching knowledge and skill, the practicum experience, and the transition into the teaching profession. Among the important findings, the researchers found that (1) teacher education programs across Canada differ markedly in structure and duration, and (2) there were significant variations among the respondent groups’ perceptions of program content, emphasis, and quality. Relatively few (about 13%) graduates gave overall “excellent” ratings to their teacher education programs, while about half gave “good” ratings. To the researchers, those areas of content, knowledge and skill are highly valued in the field but are not being emphasized as strongly in teacher education programs as they might be. Brower and Korthagen (2005) in the United States analyzed the structure of teacher education programs. They found that those programs may be counterproductive to student teacher learning, and consequently, teacher educators may not display the best examples of good teaching. They also found that during and immediately after their pre-service programs, teachers experience a distinct attitude shift that entails an adjustment to teaching practices existing in schools. The authors showed that “integrative” theory-practice approaches in teacher education, in which student teachers’ practical experiences are closely linked to theoretical input, strengthen graduates’ innovative teaching competence.
What were the strengths of the two studies? Both studies are good longitudinal research that highlight the importance of integrating theory and practice in pre-service teacher education programs, and support the need for educating innovative teachers. Important suggestions for the design of teacher education programs and the conduct of teacher education research are drawn from both studies, for example, finding better ways to support and mentor novice teachers, developing stronger models of collaboration between the teachers and the institutions they serve, and developing a common vision for teacher education which articulates core content and competencies. Teacher education research should take a more longitudinal comparative approach.
What were the limitations of the two studies? Though large scale, longitudinal surveys may offer some advantages in terms of reducing validity threats, the literature suggests that researchers should prepare to deal with problems related with the longevity of longitudinal surveys. Some of the limitations were: resource restrictions, sample size, the absence of comparative information from other similar studies.
How would you have changed the two studies to improve the quality and usefulness of the research? I would take into account suggestions provided by the researchers: (1) refining the selection of respondents and measurement of criterion variables, (2) intensifying qualitative data collection during pre-service programs and carrying out repeated measurements and observations at increasing numbers of standardized moments after graduation, (3) developing a drop out study that can produce clues about differences between graduates who did and did not seek and find work as teachers that were associated with variables other than gender, number of applications, or progress during the pre-service program.
How would you incorporate the findings of the two studies into your classroom? Repeated cross-sectional studies or more longitudinal studies would be of great value in examining trends in teacher education. I would like to be engaged in longitudinal research, particularly the cohort study. We should focus into the ways in which prospective teachers learn from practice and develop competence and positive attitudes. The goal should be to equip teachers for entry into the teaching profession encompassing problem-based learning, authentic contexts and materials.
Reflective Summary of 4 articles
In the first article, Hadjionannou (2007) argued that the best way to understand the educational phenomenon is to view it in its context. To that end, she used different qualitative approaches such as recorded class sessions, interviews, and field notes. As a result she found important elements that appeared to be related to the students’ involvement in dialogic discussions and the social relationships in the classroom. She provided descriptions of classroom environment that supports authentic discussions. In the second article, Zimmerman (2008) used qualitative approaches such as portfolio assessments, direct participant observation and survey questionnaires. The focus was on the development of online measures of self-regulatory learning (SRL) processes and motivational feelings using innovative methods such as computer traces (gStudy software), think-aloud protocols, diaries of studying, direct observation, and microanalysis measures. This study adopted an inductive approach to its reasoning; observations were made from data collected through survey questionnaires, and then sought to work towards a theoretical integration of what it had found. Therefore, the study moved from the data to a theory and vice versa. The focus was on the uniqueness of the students in the self-regulation training group. The results revealed that students in the training group reported the greater increase in time management skill and self reflection on their learning than those in the control group.
In sum, both qualitative studies tended to be oriented toward individuals and case studies. They allowed for a richer analysis of subjects and for information to be gathered that would otherwise be entirely missed by a quantitative approach. The qualitative research focused on collecting, analyzing, and interpreting data by observing what people did and said, involving a continual interplay between theory and analysis. In analyzing qualitative data, the researchers discovered patterns such as changes over time or possible causal links between variables. The findings were a personal construction of how researchers viewed events and their job was to persuade us that their interpretation was valid. From the phenomenological point of view, the authors held that the subjects’ perceptions define reality.
In the last articles the authors applied quantitative research methods. Onwuegbuzie et al. (2007) developed a validity study of a teaching evaluation form (TEF). The researchers assessed 912 College students’ perceptions through a survey questionnaire. As a result, the researchers identified a list of characteristics that students considered descriptors of effective college teaching, three of which were not represented in the TEF. The researchers were able to develop a new and more complete form called the CARE RESPECTED Model of Teaching Evaluation (CRMTE). Brouwer and Korthagen (2005) developed a longitudinal study of a period of 4.5 years using questionnaires, interviews, observations and analysis of classroom artifacts to find out if occupational socialization in schools has a considerable influence on the development of graduate teachers’ in-service competence. The researchers quantified the variables of interest and examined the relationships between the variables mathematically through statistical analysis. The researchers showed that “integrative” theory-practice approaches in teacher education strengthen graduates’ innovative teaching competence. Quantitative research methods in both studies, simply put, were about numbers, objective hard data, quantitative and statistically valid results. Tools were used to minimize any bias in collecting information. The studies involved gathering data that is absolute such as numerical data, testing hypotheses, promoting its supposed neutrality.
In conclusion, researchers showed how to work collaboratively across qualitative and quantitative research paradigms. Mixed research rests on rich and varied approaches, which come from multiple disciplines to address different research topics.

Comparison between Historical Research and Evaluation Research

2009-06-15T15:59:00.001-07:00

Reference:
Borg, W.R., and Gall, M.D., (1999). Educational Research: An Introduction (6th ed.). Toronto, ON: Allyn & Bacon.
Chapters 16 and 17

Summary by Nelson Dordelly-Rosales, June 20th, 2009

Historical Research: What is it?

• Historical research is the systematic search for facts relating to questions about the past, and the interpretation of these facts. By studying the past, the historian hopes to achieve a better understanding of present institutions, practices and issues in education.
• There is no single, definable method of historical inquiry (Edson, 1988)

What does historical research mean from the qualitative and quantitative perspectives?

• From the qualitative perspective, historical research means historical inquiry. It proposes to learn from past discoveries and mistakes, and provides a moral framework for understanding the present and predicting future trends.
• From the quantitative perspective, historical research is the systematic collection and objective evaluation of data related to past occurrences in order to test hypotheses concerning causes, effects, or trends of these events that may help to explain present events and anticipate future events.

How to conduct a historical research?

– Definition of a problem: topic (s) or questions to be investigated
– Formulation of questions to be answered, hypotheses to be tested or topics to be investigated.
– Systematic collection and analysis of historical data
– Summary and evaluation of data and the historical sources
– Interpretation: present the pertinent facts within an interpretive framework
– Production of a synthesis of findings or confirmation/disconfirmation of hypotheses or questions (Borg & Gall, 1999, p. 811-831)

What are the types of historical sources?

• Preliminary sources: published aids for identifying the secondary source literature in history. An important requirement is to list key descriptors for one’s problem or topic, e.g., bibliographies and reference works.
• Primary: those documents in which the individual describing the event was present when it occurred, e.g., diaries, manuscripts.
• Secondary: documents in which the individual describing the event was not present but obtained a description from someone else, who may or may not have directly observed the event, e.g., historian’s interpretations (Borg & Gall, 1999, p.815-817).

How to record information from historical sources?

• Examining availability and deciding what information to record from:
- Documents: diaries, memoirs, legal records, court testimony, newspapers, periodicals, business records, notebooks, yearbooks, diplomas, committee reports, memos, institutional files, textbooks, tests.
- Quantitative records: census records, school budgets, school attendance records, test
scores.
- Oral history: i.e., records and interviews.
- Relics: an object whose physical or visual properties provide information about the
past.
• Summarizing quantitative data (Borg & Gall, 1999, p. 818-819)

How to evaluate the worth and meaning of historical sources?

• External criticism: evaluation of the nature of the source, e.g., Is it genuine? Is it the original copy? Who wrote it? Under what conditions?
• Internal criticism: the evaluation of the information contained in the source, e.g., is it probable that people would act in the way described by the author? Do the budget figures mentioned by the writer seem reasonable? (Borg & Gall, 1999, p. 821-823).

How to interpret historical research?

• Use of concepts to interpret historical information:
- Concepts are indispensable for organizing the phenomena that occurred in the past.
- Group together those persons, events, or objects that share a common set of attributes.
- Place limits on the interpretation of the past.
• Being aware of bias, values, and personal interests allow researchers to interpret or “reconstruct” certain aspects of past events, but not others. Also, it allows interpreting past events using concepts and perspectives that originated in more recent cases.

What is the role of the historical researcher?

• Historians cannot ‘prove’ that one event in the past caused another, but they can be aware of, and make explicit, the assumptions that underlie the act of ascribing causality to sequences of historical events (Borg & Gall, 1999, p. 831).
• Generalizing from historical evidence means looking for consistency across subjects or an individual in different circumstances (Borg & Gall, 1999, p. 834).
• Causal inference in historical research is the process of reaching the conclusion that one set of events brought about, directly or indirectly, a subsequent set of events (Borg & Gall, 1999, p. 836).

What is Evaluation Research?

• Educational evaluation: is the process of making judgments about the merit, value, or worth of educational programs (Borg & Gall, 1999, p. 781).
• Evaluation Research: is usually initiated by someone’s need for a decision to be made concerning policy, management, or political strategy. The purpose is to collect data that will facilitate decision-making (Borg & Gall, 1999, p. 782).
• Educational Research: is usually initiated by a hypothesis about the relationship between two or more variables. The research is conducted in order to reach a conclusion about the hypothesis - to accept or reject it (Borg & Gall, 1999, p. 783).

How to conduct an ‘Evaluation Study’?

• Clarifying reasons for doing the evaluation
• Identifying the stakeholders
• Deciding what is to be evaluated
- Program goals
- Resources and procedures
- Program management
- Identifying evaluation questions
- Developing an evaluation design and timeline
- Collecting and analyzing evaluation data
- Reporting the evaluation results (Borg & Gall, 1999, p.744-753).

What are the criteria of a good evaluation study?

• Utility: an evaluation has utility if it is informative, timely, and useful to the affected persons.
• Feasibility: the evaluation design is appropriate to the setting in which the study is to be conducted and that the design is cost-effective.
• Propriety: if the rights of persons affected by the evaluation are protected.
• Accuracy: extent to which an evaluation study has produced valid, reliable, and comprehensible information about the entity being evaluated (Borg & Gall, 1999, p.755).

What is involved in ‘quantitatively oriented evaluation’ models?

• Evaluation of the individual.
• Objectives-based evaluation for determining the merits of a curriculum or an educational program.
• Needs assessment.
• Formative and summative evaluation.

(Borg & Gall, 1999, 758-767).

Evaluation of the individual
• This type of research involves the assessment of students’ individual differences in intelligence and school achievement.
• It also involves evaluation of teachers, administrators, and other school personnel.
• Like assessment of students, personnel evaluation focuses on measurement of individual differences, and judgments are made by comparing the individual with a set of norms or criterion (Borg & Gall, 1999, p.759)

Objectives-based evaluation: Four Models

• Discrepancy evaluation between the objectives of a program and students’ actual achievement of the objectives (Provus, 1971).
• Cost-benefit evaluation to determine the relationship between the costs of a program and the objectives that it has achieved. Comparisons are made to determine which promotes the greatest benefits for each unit of resource expenditure (Levin, 1983).
• Behavioral objectives to measure the learner’s achievemen(Tyler,1960)
• Goal-free evaluation to discover the actual effects of the program in operation that may differ from the program developers’ stated goals (Scriven, 1973).

Needs assessment

• This type of research aims to determine a discrepancy between an existing set of conditions and a desired set of conditions.
• Educational needs can be assessed systematically using quantitative research methods.
• Personal values and standards are important determinants of needs, and they should be assessed to round out one’s understanding of needs among the groups being studied.
• Needs assessment data are usually reported as group trends (Borg & Gall, 1999, p. 763)

Formative and summative evaluation

• The function of formative evaluation is to collect data about educational products while they are still being developed. The evaluative data can be used by developers to design and modify the product (Borg & Gall, 1999).

• The summative function of evaluation occurs after the product has been fully developed. It is conducted to determine how worthwhile the final product is, especially in comparison with other competing products. Summative data are useful to educators who must make purchase or adoption decisions (Borg & Gall, 1999).

Evaluation to guide program management

• It includes context evaluation, input evaluation, process evaluation, and product evaluation (CIPP). The CIPP model shows how evaluation could contribute to the decision-making process in program management (Stufflebeam and others 1971).
• Context evaluation involves identification of problems and needs in a specific setting.
• Input evaluation concerns judgments about the resources and strategies needed to accomplish program goals and objectives.
• Process evaluation involves the collection of evaluative data once the program has been designed and put into operation.
• Product evaluation aims to determine the extent to which the goals of the program have been achieved.

What does a ‘qualitatively oriented evaluation’ model mean?

• The worth of an educational program or product depends heavily on the values and perspectives of those doing the judging.
• For example the three following models:
- Responsive evaluation (Stake, 1967)
- Adversary evaluation (positive and negative judgments about the program) (Wolf, 1975)
- Expertise-based evaluation (Eisner, 1979)

Responsive evaluation

• Focuses on the concerns, issues and values affecting the stakeholders or persons involved in the program (Stake, 1967)
• Egon, Guba & Yvonna (1989) identified four major phases that occur in evaluation:
- Initiation and organization: negotiation between the evaluator and the client.
- Identifying the concern’s issues and values of the stakeholders using questionnaires
and interviews.
- Collection of descriptive evaluation using observations, tests, interviews, etc
- Preparing reports of results and recommendations.

Adversary evaluation

• Adversary evaluation relates in certain respects to responsive evaluation (positive and negative judgments about the program) (Wolf, 1975). It uses a wide array of data.
• Four major stages:
- Generating a broad range of issues, the evaluation team surveys various groups involved in the program (users, managers, funding agencies, etc).
- Reducing the list of issues to a manageable number.
- Forming two opposite evaluation teams (the adversaries) and provides them an opportunity to prepare arguments in favor of or in opposition to the program on each issue.
- Conducting pre-hearing sessions and a formal hearing in which the adversarial teams present their arguments and evidence before the program’s decision makers (p.774).

Expertise-based evaluation

• Expertise-based evaluation or educational connoisseurship and criticism or judgment about the worth of a program made by experts (Eisner, 1979)
• One aspect of connoisseurship is the process of appreciating (in the sense of becoming aware of) the qualities of an educational program and their meaning. This expertise is similar to that of an art critic who has special appreciation of an art work because of intensive study of related art works and of art theory.
• The other aspect of the method is criticism, which is the process of describing and evaluating that which has been appreciated. The validity of educational criticism depends heavily on the expertise of the evaluator.

Differences between Historical and Evaluation Research

• Historical research aims to assess the worth and meaning of historical sources, documents, records, relics, oral history, etc. The search is for facts relating to questions about the past, the interpretation of these facts and its significance for the present.
• Evaluation research aims to assess the merit, value, or worth of educational programs and materials of any level of schooling. It facilitates decision-making concerning policy, management, or political strategy to improve educational matters.

Conclusion
• Each type of research addresses different types of questions, and each one is necessary for advancing the field of education. The decision to undertake one of these types of research will depend primarily on the interests of the study. However, both, historical and evaluation research draw to varying degrees on the qualitative and quantitative traditions of research.
• In quantitative evaluation research, objectives provide the criteria for judging the merits of the product, e.g., publication and cost, physical properties, content, instructional properties, etc. In qualitative research, the worth of an educational program or product depends heavily on the values and perspectives of researchers.
• In historical research the historian discovers objective data but also can interpret and critique, making personal observations on the worth & value of findings.

2009-06-04T14:01:00.000-07:00

Summary #1: Statistical Techniques (for processing and analysis of data)by Nelson Dordelly-Rosales
References:
Gall, M. D., Gall, J. P. & Borg, W. R. (1999). Educational research: An introduction (6th ed.). Toronto, ON: Allyn & Bacon.
Taylor, J. K., and Cihon, C. (2007). Statistical techniques for data analysis. (2nd ed.). New York: Chapman & Hall/CRC.
Key terms and definitions
Research Design: refers to all procedures selected by a researcher for studying a particular set of questions or hypotheses. In this process the researcher creates an empirical test to support or refute a hypothesis. The process of designing a research study has several steps: conclusions from previous studies, rationale or theory, questions and hypotheses (or suggested explanation or a reasoned proposal predicting a cause), design, gathering the data, summarizing the data and determining the statistical significance of the results, conclusions and beginning of next study.
Quantitative research: using statistical methods typically begins with the collection of data based on a theory or hypothesis, followed by the application of descriptive or inferential statistical methods. Descriptive statistics: also called summary statistics are used to “describe” the data we have collected on a research sample. The main descriptive statistics are: the mean, median, and standard deviation; they are used to indicate the average score and the variability of scores for the sample. Inferential statistics: are used to make inferences from sample statistics to the population parameters. It includes: sampling distributions and confidence intervals, one and two sample topics (comparison of means, ratio of variances), propagation of error in a derived or calculated value, regression analysis, testing hypotheses, drawing inferences.
Types of Quantitative Research Designs: Descriptive studies and studies aimed at discovering causal relationships (causal-comparison, correlation, or experiment). Causal-Comparison: refers to causal or functional relationships between variables (the way in which variables influence or affect each other). The causal-comparative method is aimed at the discovery of possible causes for the phenomenon being studied by comparing subjects in whom a characteristic is present with similar subjects in whom it is absent or present to a lesser degree. Experimental research design: is ideally suited to establish causal relationships if proper controls are used. The key feature of experimental research is that a treatment variable is manipulated. Correlational studies: include all research projects in which an attempt is made to discover or clarify relationships through the use of correlation coefficients. It tells the researcher the magnitude of the relationship between two variables.
Response to questions
The chapter aims to describe and explain the main statistical techniques for processing and analysis of data. It helps the reader to become familiar not only with the language, principles, reasoning and methodologies of statistical techniques of quantitative (research that is rooted in the positivistic approach to scientific inquiry) but also of qualitative (observation, ethnographic interview, survey) research methods
The specific topic of the chapter is the description of three main types of statistical techniques are descriptive inferential, and tests statistics. The chapter also deals with measurements in educational research usually expressed in different types of scores.
The overall purpose is to help the researchers in understanding four kinds of information about statistical tools: (1) what they should know about statistics and what statistical tools are available? (2) Under what conditions each tool is used? (3) What the statistical results mean? And (4) how the statistical calculations are made?
In general, the author is saying that we need to analyze research results effectively. The authors suggest that we make maximum use of data collected and apply appropriate statistical techniques when analyzing our research data.
This information is interesting because statistical techniques are used to a) describe educational phenomena; c) make inferences from samples to populations, d) identify psychometric properties of tests, e) apply mathematical procedures involved in the use of statistical formulas: measures of central tendency, of variability, correlation, tests, etc. A sound research plan is the one that specifies the statistical tools to be used in the data analysis. Statistical tools should be decided upon before data have been collected because different tools may require that the data be collected in different forms.
Closing summary
As researchers we should know that a statistical research project aims to investigate causality, and in particular to draw a conclusion on the effect of independent variables (predictors) on dependent variables (responses); and that there are two types of causal statistical studies: experimental and observational. An experimental study involves taking measurements, manipulating the system, and then taking additional measurements using the same procedure to determine if the manipulation has modified the values of the measurements. In observational study we just gather data and investigate correlations between predictors and response. Basic steps of an experiment are: planning, design, summary (descriptive statistics), reaching consensus (inferential statistics), and documenting-presenting results. We should be careful in choosing the right statistical tools to be used in the data analysis (see visual graphic) because occasionally a statistical tool is used when the data to be analyzed do not meet the conditions required for the tool in question. After appropriate statistical tools have been selected and applied to the research data, the next step is to interpret the results. Interpretation must be done with care. Fortunately today, as researchers, we have access to the techniques and technology we need to analyze statistical data. Computers can help with data analysis techniques that were once beyond the calculation reach of even professional statisticians. All we need is practical guidance on how to use them. For example, measurement analysis can be performed through the MINITAB Statistical Software, which improves presentation of the results.
Reflection
The purpose, structure, and general principles of educational research methodology, quantitative and qualitative measurement analysis, and most importantly, the statistical techniques are valuable to everyone who produces, uses, or evaluates data. Descriptive statistics help us to summarize the data we have collected on a research sample, and inferential statistics techniques are important in educational research because they allow us to generalize from a sample or samples, to reach conclusions about large populations. We must be aware of misuses and abuses of statistics in research. When the variables are the values of experimental measurements they have uncertainties due to measurement limitations (e.g. instrument precision) which propagate to the combination of variables in the function. Statistical techniques can help us to examine propagation of error if we are not careful. Statistical techniques are useful tools for collecting, classifying and using statistics in research, methods of using numerical facts systematically collected. They are tools for designing research, processing and analyzing data and drawing inferences or conclusions.
A few years ago, I had an excellent research experience using statistics in a survey. I worked in a survey of teaching methods of History teachers at the Ministry of Education in Venezuela. The survey system included the most commonly used survey descriptive statistics, including: percents, medians, means, and standard deviations. The results were presented in tables and we interpreted and drew some conclusions. We established significant differences between data points. Statistical Software was used to improve quality in presenting the final results. We interpreted some similarities and some differences between the teachers of public and private schools. We found that about 60% involved used of the traditional “lecture” method. As a result, the Ministry of Education developed training workshops on a variety of teaching methods.

Summary # 2: Collecting Research Data
References:
Gall, M. D., Gall, J. P. & Borg, W. R. (1999). Educational research: An introduction (6th ed.). Toronto, ON: Allyn & Bacon.
Wallen, E., and Fraenkel, J.C. (2007). Educational Research: A Guide to the Process. (2nd ed.). London: Lawrence Erlbaum Associates.

Key terms and definitions
Data-Collection Tools: questionnaires, interviews, and observations aimed at gathering similar kinds of data, are the most common instruments for data collection in survey research. Other techniques for collecting survey information are tests, self report measures, and examination of records.
Survey research is a distinctive research methodology for systematic data collection. Surveys often are used to simply to collect information, such as the percentage of respondents who hold or do not hold a certain opinion. Surveys can also be used to explore relationships between different variables. The cross-sectional survey: standard information is collected at one point in time from a sample drawn from a predetermined population. When information is collected from the entire population, the survey is called a census. The longitudinal survey: in the longitudinal survey, data are collected from respondents at different points in time in order to study changes or explore time-ordered associations. Three longitudinal designs are commonly employed in survey research: trend studies, cohort studies, and panel studies. Trend studies: in this design a given general population is sampled at each data-collection point. The same individuals are not surveyed, but each sample represents the same population. For example, each year we survey teachers of History and would compare from year to year. Cohort studies: in this design a specific population is followed over a period of time. Panel studies: in this design the researcher selects a sample at the outset of the study and then at each subsequent data-collection point the same individuals are surveyed.
Survey Interview: involves the collection of data through direct verbal interaction between individuals. It permits direct follow-up (in person, telephone, computer, and recording); obtain more data using self-check test, and greater clarity than questionnaires.
Collecting observational data: three types of observational variables may be distinguished: descriptive, inferential and evaluative.

Content Synthesis
The aim of this chapter is to explain different techniques for collecting research data or information. Some of these methods depend on the methodology and the theoretical assumptions used in the research. There is a tendency for researchers in the functionalist, positivist or ‘scientific’ paradigm to collect hard objective numbers by observation, experimentation, and extraction from published sources, questionnaires and structured interviews. They emphasise quantitative techniques over qualitative methods. Law and humanistic researchers in the interpretative and radical humanist paradigms use qualitative methods. However, matching methodologies and methods is the current tendency in educational research. Mixed method research paradigm and triangulation studies are ways to make research studies more robust and rigorous by verifying results through different methods, thus ensuring that the results are not a function of the research method.
The specific topic of the chapter is data-collection tools in surveys to obtain standardized information from all subjects in the sample. The focus is on survey research, a distinctive research methodology for systematic data collection. Information to be collected is assumed to be quantifiable. The chapter helps graduate students in education learn steps needed to carry out the collection of data process. The overall purpose is the improvement of educational research through an appropriate collection of data.
What are the authors saying? The chapter provides an explanation of the techniques for preparing and using tools of survey research, considering the various types of knowledge that can be generated by analysis of survey data. Collecting research data properly is worth doing. Survey research leads to new knowledge and this knowledge contributes to improve education in different ways.
By selecting and using gathering techniques and survey research in an appropriate way, we can avoid mistakes sometimes made by researchers. Cautions include several threats to the validity of the instrumentation process. For example, an extraneous event may cause the respondents to answer differently. It also warns that for our theses we need to obtain university IRB approval for the collection of data from human subjects.

Closing summary
In general, research data may be categorised as primary and secondary data. Primary data are data generated by the researcher using data gathering techniques (questionnaires, interviews, etc). Secondary data are those that have been generated by others and are included in data-sets, case materials, computer or manual databases or published by various private (e.g. Annual Reports of companies), public organisations or government departments (official statistics by the Statistical Office) and International Organisations such the International Monetary Fund and the World Bank and the United Nations, among others. The chapter mainly focuses on what is survey research, what are the data collection tools and what are the types of research survey. It describes the cross-sectional survey and the longitudinal survey. It explains three ways for collecting research data through longitudinal survey, specifically trend studies, cohort studies and panel studies. It also provides excellent examples to illustrate the major characteristics of each type of research survey and describes advantages and disadvantages of each one.
Reflection
The major purpose of surveys is to describe certain characteristics or variables of a population. Some characteristics are: 1) information is collected from a group of people (rather than from every member of the population) in order to describe some aspects (such as abilities, opinions, attitudes, beliefs, and/or knowledge) of the population of which that group is a part.
2) The main way in which the information is collected is through asking questions through questionnaires or/and interviews. The answers by respondents constitute the data of the study. Among major advantages of survey research are reduced cost and that information collected can be of various types. Among major disadvantages are biases inherent in the data collection process and possible security or confidentiality concerns. I was able to participate in 2001 in a longitudinal survey (cohort studies survey) at Catholic University in Venezuela. In this design, the College sampled the graduating class throughout a couple of years using questionnaires. I realized that there are unique problems and pressures that affect longitudinal studies because of the extended period of time over which data is collected in comparison to cross-sectional studies. One danger is that the issues studied, and the measures and theories used, may become obsolete over the course of the study. Also, the survey was too long and a number of participants left the last questions without response. Reading Borg & Gall (1999) made me reflect on the importance of carefully planning research surveys (and short-term uses for the data should be planned ahead). Indeed, the success is dependent on clearly defining long term goals, specific variables, limitations and delimitations in the generalizability of findings. In order to guard against obsolesced longitudinal research should be theoretically broad-minded and mixed. It is important a carefully selected sample of respondents and selecting a large enough sample size. In order to make legitimate conclusions about the specified population, sampling must be representative and valid statistical assumptions must be present.

Summary # 3: Collecting Research Data with Questionnaires
References:
Gall, M. D., Gall, J. P. & Borg, W. R. (1999). Educational research: An introduction (6th ed.). Toronto, ON: Allyn & Bacon.
Fleming, C. M., and Bowden, M. (2009) “Web-based surveys as an alternative to traditional mail methods” Journal of Environmental Management, 90, 1, pp. 284-292

Key terms and definitions
Questionnaire: can be defined as a set of questions to which participants record their answers, usually within largely closely defined alternatives (Fleming and Bowden, 2009). There are mainly three types: postal or mail questionnaire, online questionnaires and personally administered questionnaires.
Mail Questionnaires: the questionnaires are sent (using the post office) to the sample participants, usually with a pre-paid self-addressed envelope to encourage response. Some advantages are low cost, and anonymity; the respondents can give more thought to the questions; researcher bias is less as compared to administered questionnaires. Some disadvantages are possible misinterpretation of questions, possible problems with language, and lower level response rate (usually is small which requires a second or even a third mailing)
Hand-delivering or personal administered questionnaires: the researchers personally administer the questionnaire to the participants, usually at the participants’ workplace, residence or any other adequate location. Some advantages are faster response as compared to the mail questionnaires, the research can clarify questions to the participant, the researcher can motivate honest answers by emphasising the participants’ contribution, and personal persuasion increases response rate. A possible disadvantage is that the researcher may introduce his/her personal bias and the responses may vary as compared to mail questionnaires.
Online questionnaires: Using online questionnaires enables the researcher to collect large volumes of data quickly and at low cost; direct access to research populations; it is possible to make them friendly and attractive, thus encouraging higher response rates and data entry errors are often low. Some disadvantages are sample bias, technology knowledge of respondents, anonymity, privacy and confidentiality (Fleming and Bowden, 2009).
Content synthesis
The chapter aims to help the reader to understand the main steps in conducting questionnaire surveys, the rules that researchers should follow to guarantee high-quality surveys and the importance of careful planning and sound methodology.
The steps in conducting a questionnaire survey are the following: (1) defining objectives, (2) selecting a sample, (3) writing items, (4) constructing the questionnaire, (5) pretesting, (6) preparing a letter of transmittal, (7) sending out questionnaire and follow-ups, and (8) analysis of the results and preparing the research report. Surveyors should try to make questionnaires attractive, and easy to complete. Also, they should number the questionnaire items and pages; put name and address of person to whom form should be returned; include brief, clear instructions; use examples before any items, etc.
The overall purpose of the chapter is to guide, especially to graduate students of education, and teachers, in how to apply the necessary tools, procedures and techniques for effectively designing and conducting a survey questionnaire in educational research.
The authors are saying that given the objectives of a survey, we as graduate students should know the rules related to questionnaire format and how to write both closed-form and open-ended questionnaire items to measure them.
Information provided is interesting because questionnaires are useful instruments to obtaining access to organisations and, more specifically, to obtain evidence of consensus among the respondents on different issues. With careful planning and sound methodology, the mail questionnaire can be a very valuable research tool in education.
Closing summary
Chapter 8 provides a clear explanation of the steps in conducting a questionnaire survey and the set of rules that researchers should apply when conducting it. Among the rules that we should apply when conducting a questionnaire survey are: to define the problem clearly, list objectives, construct neat items, make the questionnaire attractive, put name and address of person to whom form should be returned. Regarding the form, the questionnaire as such should include brief, clear instructions, use examples before any items, organize the questions in logical sequence and it should be easy to complete. In relation to the organization of content, the questionnaire when moving to a new topic, it should include a transitional sentence to help respondents switch their trains of thought, begin with a few interesting and nonthreatening items, do not put important items at the end and put threatening or difficult questions near the end; and items should meaningful to the respondents. Finally, if there is attitude measurements you should investigate respondents’ familiarity (prove the questionnaire previously with a small sample), and watch out anonymity because non-respondent individuals cannot be identified (but all depends if anonymity is necessary to achieve the specific goals). The authors recommend pre-testing the questionnaire, which requires doing the following: select a sample of individuals from a population similar to our subjects and ask them to repeat their understanding of the meaning of the question in their own words to make sure items are clearly stated; apply the questionnaire to a sample to check the % of replies. Read the subjects’ comments and make necessary changes to improve it; make a brief analysis of the pre-test results. Make necessary changes (adding questions, correcting words, etc). Prepare a letter of transmittal; the authors say that it is important to pre-contact a sample to assure cooperation. Letter must be brief, precise, explain good reasons, assure privacy and confidentiality, if possible, and associate it with some professional institution or organization (authority symbol).
Reflection
The questionnaire can be a very valuable research tool in education. It is a data collection tool in which written questions are presented that are to be answered by a selected sample of respondents. Collecting research data with questionnaires requires careful planning and sound methodology. The authors describe in detail all the steps that must be taken to carry out a successful questionnaire survey.
The key in carrying out a satisfactory questionnaire study is to begin by clearly defining the research problem and list specific objectives or hypotheses. That is the researcher needs to have a clear understanding of what s/he hopes to obtain from the results. Otherwise, it will be very difficult to make right decisions regarding selection of a sample, construction of the questionnaire, and methods for analyzing the data. Identifying the target population and selecting a sample is also a key to guarantee success in conducting a questionnaire survey.
The researcher also must be very careful in designing or constructing items. The qualities of a good questionnaire survey are the following: Clarity, short items, avoid items that include two separate ideas in the same item, do not use technical terms, jargon or confuse words, ask general questions first and then specific questions and it is important to avoid biased or leading questions (the subject is eager to please), and avoid questions that may be psychologically threatening (low moral). The authors suggest sending questionnaires and follow-ups by using special delivery mail. The questionnaires must be neat, and carefully planned.
Below, there are two diagrams that synthesize statistical techniques and a glossary that can help in understanding some of the terms that are useful in conducting research.
Reference:
Vern Lindberg (2000) “Uncertainties and Error Propagation” http://www.rit.edu/cos/uphysics/uncertainties/Uncertaintiespart1.html#range

2009-05-23T12:34:00.000-07:00

About Surveys
Surveys provide the chance to express participants’ ideas and we can get precise answers to our questions. I think the secret of an excellent survey is the structure of the questions and having very clear the goals of the study. One of the best surveys I have reviewed is the following:
Rosales-Dordelly, C.L. and Short, Edmund C. (1985) Curriculum Professors’ Specialized Knowledge. Lanham, MD: University Press of America. This book reports a survey among professors between Canada and the US.

# 1 Validity, Reliability, Trustworthiness
A. Summary by Nelson Dordelly-Rosales: McMillan (2007) summarizes and provides suggestions to monitor the threats that put at risk internal validity in randomized field trial studies or control trials or, on quasi-experiments in which there is “equating” of pre-test differences: unit of randomization and local history (equivalence of the groups), intervention-treatment (fidelity, consistency with theory), differential attrition (mortality-tracking participants), testing (instrumentation variations and procedures), subject effects (selection-maturation interaction), diffusion of intervention (treatment, asking appropriate questions), experimenter effects (checking values, biases, needs) and novelty effects (changes to normal routines). It is the responsibility of researchers “to include design features that will lessen the probability that the threat is plausible” (p.5)
B. Experience: As graduate student, my recent research activity is in the area of interpretation/construction (my theses). But, I recently reviewed an excellent research by Chauncey Monte-Sano (2005) which resembles another study done at the Ministry of Education in Venezuela on comparative studies. The researchers monitored and supervised the threats to validity and trustworthiness through intervention fidelity (pre- and post-test essays, interviews, observations, teacher feedback, assignments, and readings; analysis of students’ progress within each classroom and between both classrooms, assessing any changed observed in the students’ work).
C. Suggestions: I think that in studies in which there are pre-test differences, it is necessary to include design features to monitor all plausible threats. Particularly, I would suggest increasing the number of homogeneous comparative groups.

# 2 Different types of sampling methods
Summary: Cui, Wei Wei (2003)aims to help the readers to understand what is sampling (a technique of selecting a representative part of a population for the purpose of drawing conclusions of the whole population), the different types of sampling (probabilistic, non-probabilistic, simple, systematic, stratified, cluster, purposeful), potential sources of error (sampling error, non-coverage error, non-response error, and measurement error) and how to reduce error in mail surveys and interviews (avoid unrepresentative number, enlarging the sample size; avoid bias of interviewer or survey researcher in favouring the selection of units that have specific characteristics; improving survey return rates, etc).
The value for me as an educator and as a consumer of educational research: I should use appropriate sampling methods and an adequate response rate for a representative sample. However, I should also evaluate different factors that may affect the quality of data from a research study, for example, procedures, questions asked, validity of questionnaire, among others.

# 3 Two concepts regarding data analysis
a. Helberg (1996) warns the researchers on the paradox that statistics can produce dissimilar or contradictory results. The author provides suggestions about how to cope with sources of bias, errors in methodology, and misinterpretation of results. To that end, he explains how to assuring representative sampling and valid statistical assumptions; recommends using methods available for taking measurement error into account in some statistical models and applying more precision and accuracy in interpretation of results.
b. Oliver-Hoyo snd Dee Dee (2006) reviewed data collection through three qualitative methods to study qualitative variables (meaning constructed by individuals): surveys, journal responses and field notes. The authors are persuasive in that relying on more than two methods is invaluable to avoid gross errors when drawing conclusions in surveys.
c. How the articles’ information could be of value to you as an educator and consumer: From these readings, I learned that multiple methods of data collection and analysis (quantitative and qualitative) help us to develop a more complete view of the problem and the solution. So, in the example provided regarding accountability, I think that the right approach is integrating different “assessment strategies” so that educators can take advantage of all the information.

2009-05-23T12:22:00.000-07:00

Article Review by Nelson Dordelly-Rosales:
1. Krauss, S., (2005) Research Paradigms and Meaning Making: A Primer.
The Qualitative Report, 10 (4), 758-770.
The paper Research Paradigms and Meaning Making: A Primer provides an introduction to some of the basic issues in attempting to work with both quantitative and qualitative research methods. It explains how qualitative data analysis can be used to organize and categorize different levels and forms of meaning. It argues that the heart of the quantitative vs. qualitative “debate” is philosophical, not methodological, and it offers an overview of the epistemological differences of quantitative and qualitative research methodologies. The article introduces the notion of meaning making in social sciences research and how it actually occurs through qualitative data analysis. It defines meaning as “the underlying motivation behind thoughts, actions and even the interpretation and application of knowledge” (Krauss, 2005, p. 763). The task of constructing meaning through qualitative data analysis is explained through a variety of perspectives and approaches.
Problem/Issue and the Importance/Significance
The focus is on the task of constructing meaning through qualitative data analysis. This paper is significant because it examines the concept of the philosophical realist paradigm and introduces the notion of “meaning making” in research methods and how meaning is generated from qualitative data analysis specifically. To that end, it explains epistemological differences between quantitative and qualitative research that allows us to understand phenomena and to get more realistic results. Some examples are also provided.
Research Question(s)
What will be the epistemological similarities and differences between quantitative and qualitative research paradigms? How can the realist philosophical paradigm accommodate both, quantitative and qualitative research paradigms? How meaning can be constructed and organized using a qualitative data analysis approach?
Sample and sample selection process
Qualitative data selection in qualitative research is intuitive to discover (not measure) potentially important insights. The author explains the need to make use of multiple research methods to optimize the data selection process, to increase both the breadth and width of data selection.
Data Collection Methods
Krauss (2005) found that qualitative data collection meaning is constructed on “a variety of levels of daily life through the exchange of ideas, interaction, and agreement between the researcher and the participants” (p. 764). The author supports his point of view through some examples from the literature, within social sciences, about how meaning can be constructed and organized using a qualitative data analysis approach (interpretivism). The author also cites a multi-year religiosity initiative as a case in which he was involved in conducting both qualitative and quantitative research to assess religiosity in the lives of young people.
Data Analysis Method
Krauss (2005) argues that qualitative data analyses in qualitative research are guided by a reflective paradigm in an attempt to acquire social knowledge. In this sense, according to the author, meaning is constructed in a variety of ways, that is, “through construction, the researcher is not a blank slate; rather s/he is an active participant in the process” (Krauss, 2005, p. 767). This means that, epistemologically, the researcher is engaged in the setting, participating in the act of ‘being with’ the respondents in their lives to generate meaning of them” (p. 769). In addition, developing themes and storylines featuring the words and experiences of participants themselves adds richness to the findings.
Limitations/Delimitations/Assumptions
Krauss (2005) explains that the realist paradigm has the unique goal of facilitating the meaning-making process, which is an important learning facilitator that has the power to encourage transformative learning. The realist philosophical paradigm attempts to accommodate quantitative and qualitative research methods. In the area of religion for example: “the result of the process was a major study that tapped into the richness of individual religious experience, along with a broader understanding of religious behaviors and knowledge levels across large groups of young people” (p.758). As a whole, the realist paradigm has less limitations than each one separated.
Trustworthiness/Validity Considerations
According to the author, realist researchers reject the framework of validity that is commonly accepted in more quantitative research in the social sciences. Nevertheless, realist research inherently assumes that there is some reality that can be observed with greater or less accuracy or validity. In this sense, “rigor in qualitative data analysis is a necessary element for maximizing the potential for generating meaning” (Krauss, 2005, p.765). This rigor provides trustworthiness to the results.
Ethical Issues
Qualitative researchers can operate under different epistemological assumptions from quantitative researchers. Ethical issues can sometimes result in confusion and uncertainty among researchers. In qualitative research, as well as in quantitative research, researchers are expected to employ high standards of academic rigor, and to behave with honesty and integrity. Ethics can emerge from value conflicts. I think that being a ‘purist’ researcher looking only at one small portion of a reality that cannot be split or unitized “without losing the importance of the whole phenomenon brings an ethical issue to the research process” (Krauss 2005, p. 767).
Reflective Assessment
The concept of meaning making in research methods and how meaning is generated from qualitative data analysis are the most important contributions of this paper. This article discusses the philosophical differences between quantitative and qualitative research. Quantitative research is positivist, objective and scientific that can be accomplished by statistical software packages commonly used for quantitative data (descriptive data). Qualitative researchers operate under naturalist, constructivist, eclectic and subjective assumptions (researcher interpretation). Qualitative research is a highly intuitive activity that contributes greatly to the construction of meaning. As researchers, we should focus on “the significance of different levels of meaning such as worldviews or philosophies of life, and the importance of meaning as a critical element to human existence and learning” (Krauss, 2005, p.767). I think that the author makes a good point regarding the need to make use of multiple research methods to optimize the data collection and the analysis process; that is, a mixed approach increases both the breadth and width of data collection and data analysis. The author provides an excellent overview of the basic issues in attempting to work with both quantitative and qualitative research methods toward the goal of generating meaning. Using both methods together contributes to a better understanding of phenomenon. I think that, it is important to make use of multiple research methods because it means broader understanding of behaviors and knowledge levels across large groups of people. Different philosophical assumptions or theoretical paradigms about the nature of reality are essential to understanding the overall perspective from which the study is designed and carried out. Within this holistic approach, critical realism framework, both qualitative and quantitative methodologies together are appropriate toward the goal of generating meaning and understanding thinking, behavior and worldwide formation. Indeed, the heart of the quantitative-qualitative “debate” is philosophical, not methodological.

Article Review
2. Gephart, R. (1999, 11 14). Paradigms and Research Methods. Retrieved 05 21, 2009, from Academy of Management, Research Methods Division: http://division.aomonline.org/rm/1999_RMD_Forum_Paradigms_and_Research_Methods.htm
Gephart (1999) explains three prevailing paradigms or views of the world which are currently shaping research: Positivism, Post-positivism or Interpretivism and Critical Postmodernism. The author introduces the concept of each paradigm, describes the key features of each worldview or “forms of scholarship,” the nature of knowledge pursued, and the different means by which knowledge is produced and assessed within each paradigm or worldview. Very synthetically, the three strands of thinking and researching are different in the following way: Positivism assumes an objective world which scientific methods (for example, experimental and survey) can more or less readily represent and measure statistically, seeking to predict and explain causal relations among key variables. Post positivists or interpretivists assert that these methods “impose a view of the world on subjects;” their concern is the interplay between objective and subjective meanings. Critical postmodernists, on the other hand, argue that these imposed views or measures implicitly “support forms of scientific knowledge that explicitly reproduce capitalist structures and associated hierarchies of inequality” (p.5). In this sense, the goal is “social transformation involving the displacement of existing structures of domination” (p. 6). The author concludes saying that these paradigms or theories are somewhat “separate but not greatly distant from one another” (p.7). In general the goal of research is adequate reflection of peoples’ experience in making inquiries from one or more theoretical frameworks.
Problem/Issue and the Importance/Significance
The need to help readers understand some of the basic assumptions underlying forms of research present in the field: Positivism, Post-positivism or Interpretivism and Critical Postmodernism. The author attempts to make us understand the key features, the usefulness of each paradigm, and how the three paradigms can be interwoven into research.
Research Question(s)
What will be the epistemological similarities and differences between positivism, interpretivism, and critical postmodernism? What are the main assumptions, key ideas, theories, figures, goals, theories, and criteria, unit of analysis and research methods of each paradigm?
Sample and Sample Selection Process
Sample and sample selection varies according to specific paradigm. Positivism uses quantitative criteria; interpretivism and critical postmodernism apply qualitative criteria and therefore, are more intuitive and flexible; however, rigor and specific principles are essential for good research.
Data Collection Methods
In this article, the author uses grounded theory development and suggests the mixture of quantitative and qualitative paradigms. The author explains, however, that data collection and the research methods, goals, criteria and unit of analysis of each paradigm are different: (a) Positivist research uses experiments; questionnaires; secondary data analysis; quantitatively coded documents, Likert scaling, and structural equation modeling, Qualitative uses grounded theory testing, among others. (b) Interpretivism uses: ethnography, participant observation, interviews, conversational analysis, and grounded theory development; case studies, conversational and textual analysis, and expansion analysis. (c) Critical Postmodernism uses: field research, historical analysis, dialectical analysis.
Data Analysis Method
The author explains that the positivist research unit of analysis is the variable. The unit of analysis of interpretivism is the meaning. Critical theory-Postmodernism (PM) uses as unit of analysis: deconstruction, textual analysis.
Limitations/Delimitations/Assumptions
Among the limitations and delimitations of each paradigm are the following: Positivist research assumes as its goals uncovered truth and facts as quantitatively specified relations among variables. Interpretivism/Constructivism is a related approach, which is based on analysis and look for persuasion using ‘sensitizing’ concepts. Its goals are: describe meanings, understand members' definitions of the situation, and examine how objective realities are produced. Critical theory-Postmodernism (PM) aims to investigate/uncover hidden interests, enable more informed consciousness, displace ideology with scientific insights, and change.
Trustworthiness/Validity Considerations
Criteria of validity vary according to each paradigm; the author explains that positivist research, in place of prediction uses explanation, rigor, internal and external validity and reliability. Interpretivism uses the trustworthiness and authenticity. Critical theory-Postmodernism (PM) uses theoretical consistency, historical insights, transcendent interpretations, basis for action, change potential and mobilization. Its unit of analysis are: contradictions, ‘incidents of exploitation’ and the sign. In general, realist researchers reject the framework of validity that is commonly accepted in just one method of research.
Ethical Issues
Currently there is a reexamination of ethical standards to protect better the rights of research participants within each paradigm. Among principles are: voluntary participation, informed consent, confidentiality, anonymity, and prevention of risk of harm.
Reflective Assessment
Gephart (2005) provides an excellent overview of the three important and prevailing paradigms, views of the world, or philosophies of research, namely, Positivism, Post positivism or Interpretivism, and Critical Postmodernism. I think that we might consider that each school is eclectic in the sense that they choose the best from all sources. I also think that the three strands are different but they can be interwoven, integrated or mixed into the research with the purpose of improving the process of inquiry and its results.
I regard the mixed paradigm as a more reasonable strategy for research. Why? Because I think that by understanding the main differences, and weakness of each of the current leading theories, and by demonstrating that each one, separately, provides just one view of the world, the promise of impartiality of each one might be criticized as illusory. An integrated perspective is a strategic open-mindedness approach, which can tolerate all different theories and points of view coming from all different directions that helps us better understand research problems and their solutions. Positivism provides a partial view of reality (its inductive view of quantitative data). Interpretivism and critical studies are also a partial view of life; each one is subjective (a deductive view of qualitative data). Given the progressive changes evidenced in contemporary society, it is imperative a holistic understanding of phenomena (and view each problem from different angles or from an interdisciplinary view). In this sense, “toleration of others requires broken confidence in the finality of our own truth” (Tuck, 1988, p.21). So, I think is important synthesizing into a unity all of the best components in the various philosophical theories. My idea of being integrative is to retain the scientific commitment, but also to combine it with the assumptions of qualitative research.

Article Review
3. Mackenzie, N., & Sally, K. (2006). Research Dilemmas: Paradigms, methods and methodology. Retrieved 05 21,2009, from Issues in Educational Research: http://www.iier.org.au/iier16/mackenzie.html
Mckenzie and Knipe (2006) criticize the perceived dichotomy between qualitative and quantitative research methods in research textbooks and journal articles: “considerable literature supports the use of mixed methods” (p.1). The authors begin with a definition of leading paradigms in educational research, which are the following: positivist, post-positivist, interpretive/constructivist paradigm, transformative, and pragmatic paradigm. They discuss the language commonly associated with each major research paradigm. The focus is in the basic issues in attempting to work with mixed research methods and how “the research paradigm and methodology work together to form a research study” (p.1). To that end, they clarify the difference between methodology and method: “The most common definitions suggest that methodology is the overall approach to research linked to the paradigm or theoretical framework while the method refers to systematic modes, procedures or tools used for collection and analysis of data” (p.4). The authors also clarify the difference between paradigms, methodologies and the traditional “dichotomy” of quantitative and qualitative research methods and data collection tools. They conclude with a discussion and explanation of how to combine paradigms and methods.
Problem/Issue and the Importance/Significance
The paper examines the features of each paradigm and their main argument is to “demystify” the role of paradigms in research. It questions the “dichotomy” quantitative-qualitative as a way to teach research methodology. Research texts and university courses “can create confusion to undergraduate, graduate and early career researchers” (McKenzie and Knipe, 2006, p.2). It suggests teaching a combination of both, quantitative and qualitative methods of research, making use of the most valuable features of each. That is, “research methods in research texts and university courses should include mixed methods and should address the perceived dichotomy between qualitative and quantitative research methodology” (p.1).
Research Questions
Why qualitative and quantitative methods should be combined? How the research paradigm and methodology work together to form a research study? Is there a difference between methodology and methods? How to match paradigms and methods?
Sample/Sample selection process and Data Collection/Analysis Method
According McKenzie and Knipe (2006) each research paradigm, framework, or methodology applies different sample/sample selection process and data collection/analysis methods. Each overall framework or methodology of research is consistent with the definition of each paradigm and hold unique features, which are specific to their particular approach. For example, the positivist and postpositivist paradigm usually applies experimental, quasi-experimental, and correlational, among others. The interpretivist/ constructivist paradigm applies naturalistic, phenomenological, hermeneutic, interpretivist, ethnographic, multiple participant meanings, and social and historical construction. The transformative paradigm applies critical theory, neo-Marxist, feminist, critical race theory, Freirean, participatory or emancipatory, among others. The pragmatic paradigm applies among others, consequences of actions, problem-centered, pluralistic and political. In a mixed research paradigm the decision-making process does not necessarily follow a linear path, the process is more realistically cyclical, and the researcher can mix methods or can make changes as the research progresses.
Limitations/Delimitations/Assumptions
In this article, the authors assume that educational research should be taught as a mixed paradigm. They argue that a combination of both, paradigm and methods and tools is the right way to teach research methodology. Mixed method is itself a statement of what could be, rather than a groundbreaking notion, especially in the instance of educational research.
Trustworthiness/Validity Considerations
The rejection of reliability and validity in qualitative research has resulted in a shift for “ensuring rigor” from the researcher’s actions during the course of the research. Each researcher's theoretical orientation has implications for every decision made in the research process. This obviously has implications for trustworthiness/validity considerations. The emphasis on strategies that are implemented during the research process has been replaced by strategies for evaluating trustworthiness and utility, which are implemented once a study is completed.
Ethical Issues
According to the authors, many writers fail to adequately define research terminology and sometimes use terminology in a way that is not compatible in its intent, omitting significant concepts and leaving the reader with only part of the picture. In this sense, confusion can be created when authors use different terms with different meanings to discuss paradigms; for instance, methodology and methods are usually used interchangeably but they have different meanings. The authors conclude stating that mixed method, like all research approaches, needs to be viewed through a critical lens while at the same time recognizing as valid its contribution to the field of research.
Reflective Assessment
From philosophical perspective, showing how much reflection it takes to start up an investigation, the article discusses different types of research and the language associated with them. The terms qualitative and quantitative refer to the data collection methods, analysis and reporting modes, instead of the theoretical approach to the research, which is the methodology (the overall approach to research linked to the paradigm or theoretical framework, while the method refers to systematic modes, procedures or tools used for collection and analysis of data).
This article applies almost directly to our situation as young researchers and is very useful for us to distinguish the type of methodology to apply in our own research. It explains the strengths of each leading methodology or theory of research, which are the following: positivism and post-positivism, interpretivism/constructionism, transformative, pragmatism, and a mix-methods approach to research, which are excellent theoretical frameworks. In a mixed research paradigm the decision-making process does not necessarily follow a linear path, the process is more realistically cyclical, and the researcher can mix methods or can make changes as the research progresses. This view applies to constitutional interpretation. This past year I proposed the mix paradigm in constitutional interpretation in the Annual Graduate Conference in King’s College, University of London:
http://www.iglrc.com/2008website/sessions.html
I'm delighted to say that my paper on “Eclecticism in Constitutional Interpretation” was selected for publication in the book Law and Outsiders Norms, Processes and 'Othering' in the 21st Century, edited by Cian C. Murphy and Penny Green, (Oxford: Hart Publishing, 2009)
http://www.hartpub.co.uk/books/search.asp?s=Legal+Theory&st=0&cp=6

4. Article Review
Monte-Sano, Chauncey (2008) Qualities of Historical Writing Instruction: A Comparative Case Study of Two Teachers’ Practices. American Educational Research Journal, 45 (4), p. 1045-1079.
The author analyzed qualitative and quantitative differences of the practices of two high school teachers of U.S. History, in real-world History and writing instruction over time (seven months). The analysis included forty two students’ performances that resulted from these practices rather than researcher interventions. The author demonstrated that both, teachers and students need training in the work of “writing evidence-based historical essays that involves sifting through evidence and constructing an interpretation in writing” (Monte-Sano, 2008, p. 1046). The results show that the teacher that supported students’ development in writing evidence-based historical essays was more successful in improving students’ growth than the other. The author aims to help high school teachers of U.S. history become more acquainted of different qualities of instruction to help students to learn how to read, write and think historically. She explains that there are different qualities of instruction that support students’ growth in writing evidence-based historical essays. These qualities are the following: approaching history as evidence-based interpretation; reading historical texts and considering them as interpretations; supporting reading comprehension and historical thinking; asking students to develop interpretations and support them with evidence; and using direct instruction, guided practice, independent practice, and feedback to teach evidence-based writing. According to the author, “the act of writing alone is not sufficient for growth in evidence-based historical writing” (Monte-Sano, 2008, p.1045).
Problem/Issue and the Importance/Significance
Students and teachers tend to have difficulties integrating documentary evidence into written accounts of past events. The author proved that the above mentioned qualities of teaching would foster growth in evidence-based historical writing. This study is important because history educators still know little about the relationships between teaching and learning with regard to evidence-based writing and reasoning (Monte-Sano, 2008).
Research Question(s)
How do teachers prepare students to write evidence-based historical essays? What messages about history, evidence, and writing do teachers’ practices convey? What opportunities to think and write historically do these teachers provide? How do teachers think about their subject matter, students, and pedagogy? In what ways do teachers’ practices coincide with improvements in students’ evidence-based historical writing?
Sample and sample selection process
Two teachers were selected in two urban high schools in Northern California. One class period per teacher – selection was based on class size in U.S history course. A total of 42 students from these classes participated in pre and post assessments of their historical learning. Over 7 months the researcher identified patterns of growth (or lack thereof).
Data Collection Methods
Data were collected from four sources: interviews, observations, feedback, and classroom artifacts (assignments and materials). Interview questions asked teachers their view of students’ progress and needs, and the reasoning behind their instructional decisions. Observations focused on what students did during class, how the teacher represented history and what opportunities there were to learn evidence-based reasoning, argumentation and writing. Field notes and data summary charts where completed during and after every observation. Feedback included teachers’ oral assessment on homework and essays.
Data Analysis Method
The author used mixed methods in an embedded multiple-case design that included teacher and student data analysis. Regarding teacher data, she organized field notes and interview data chronologically, transcribed them into codes, and used memos to track key ideas, to highlight illustrative excerpts of class, and to note what to look for in future observations. Data showed the amount of time that each teacher devoted to a particular topic, the agreement in the number of assignments and the number of readings per topics and key components of assignments. With respect to student data, the author measured, through pre and post test instruments, how students composed arguments that recognize historical perspectives from multiple documents.
Limitations/Delimitations/Assumptions
The reduced number of teachers and students was one of the limitations. The researcher had to create a matrix of questions and possible answers and to ensure that both instruments were appropriate (in terms of age of participants). Each instrument presented several points of agreement between sources and so allowed for multiple responses to the questions. Each one asked a why question that prompts students to make a supporting argument explaining why an action was taken in the past.
Trustworthiness/Validity Considerations
The author created specific instruments to study historical reasoning and writing in history. In terms of content validity, the pre- and post-test instruments were consistent with the following variables: the notions of historical reasoning as analysis of evidence, the use of evidence to construct interpretations of the past, and communication of arguments in writing. According to the author, the strength of these instruments lied in their ecological validity (Monte-Sano, 2008, p. 1051). Even so, the author noticed that contextual changes over the course of multiple administrations of the tests can influence results. For example, the constraints of working in the classrooms led to certain agreement on the essay topics.
Ethical Issues
Comparing two teachers with different approaches (one teacher worked in groups, the other worked with lectures in which students listened to lectures and worked independently), “was not entirely fair” (Monte-Sano, 2008, p.1079); however, the author explains that comparison is instructive when considering how to develop students’ historical thinking and writing.
Reflective Assessment
The report is a comparative case study of teaching and it uses student performance as a backdrop for claims of teaching effectiveness. The target was to examine two teachers’ practices with regard to the learning outcome of writing evidence-based essays. The strength of the body of the article lies in four main aspects, (1) the notions of historical reasoning as analysis of evidence; use of evidence to construct interpretations of the past, and communication of arguments in writing, (2) the list of qualities of instruction that support students’ growth in writing evidence-based historical essays, (3) the list of questions that teachers of history can ask of research, and (4) the use of multiple research methods to optimize the data collection and the analysis process. The results show the usefulness of qualitative and quantitative comparisons of students’ work to determine how each class improves in writing evidence-based history essays.
Traditionally, teachers and students tend to view history as established fact (literal meaning of documents), not analysis or interpretation. Monte-Sano shows that there are creative ways that teachers implement approaches to history writing. This entails synthesizing and organizing information to suit the writer’s purposes; problem-based writing tasks, encouraging historical thinking, and transformation of knowledge already in the mind.
This article is an excellent example of how to work with both quantitative and qualitative research methods toward the goal of generating a new way to teach and learn History writing. To explore further, I read other articles and books, which expand the basic theme and I came to the conclusion that teachers that embark on such a study of History must feel the passion of teaching and be prepared to devote time and energy to the endeavor. In my future endeavor, I will be teaching Law History in Venezuela through Court cases, using high Court precedents. I think it affords creative alternatives such as reading historical texts and considering them as interpretations; asking students to develop interpretations and support them with evidence, etc.

Quantitative and Qualitative Research by Nelson Dordelly-Rosales

2009-05-06T16:57:00.000-07:00

Reference # 1 Krauss, Steven Eric (2005) Research Paradigms and Meaning Making: A Primer. The Qualitative Report, 10 (4) December, p.758-770.

In this article Krauss offers an overview of the epistemological differences of quantitative and qualitative research methodologies and proposes the realist philosophical paradigm. The realist paradigm was discussed as a “middle ground” between the poles of positivism and constructivism. Within a critical realism framework, both qualitative and quantitative methodologies are seen as appropriate. Krauss used mixed methods (and illustrated through examples) toward the goal of generating meaning. It introduces the notion of ‘meaning making’ in research methods, within the social sciences, and looks at how it actually occurs through qualitative data analysis. Krauss highlights the task of constructing meaning through qualitative data analysis was described citing a variety of perspectives and approaches. Overall, the article provides an introduction to some of the basic issues in attempting to work with both quantitative and qualitative research methods, and explains how qualitative data analysis can be used to organize and categorize different levels and forms of meaning.

Reference # 2 R. Burke Johnson and Anthony J. Onwuegbuzie (2004)Mixed Methods Research: A Research Paradigm Whose Time Has Come, Educational Researcher, Vol. 33, No. 7, pp. 14–26

The authors provide arguments against the polarization between quantitative and qualitative research, which is a “fallacious” dichotomy. This 'paradigm wars' “is not meaningful or productive for education research.” It distorts the conception of education and has serious implications for the quality of present educational research practice. The “subjective” vs. the “objective” is a wrong question. Consequently, the authors propose an “integrated approach to education research inquiry.” How we can integrate both paradigms? The researchers should focus on the construction of “good research questions and conducting of good research.” The questions asked should determine “the modes of inquiry that are used to answer them.”

I certainly believe the authors make a good enough case to suggest integrated methods. Indeed, phenomena are quantitative and qualitative at the same time; researchers follow similar interpretation processes for all educational research. I think integration is the authentic paradigm. We may accomplish “the integration of different modes of inquiry by collaborating with researchers with expertise.”

Comparison/contrast: there are similarities as well as differences between article 1 and article 2. Both agree that there is a third option to make good research. Either the “purists” or the ‘fallacious dichotomy” does not help in guiding productive research. Both articles argue that educational research practice should not focus in just one paradigm but in the mix or integration of both paradigms. As to the differences, the first article focuses on the commonalities and proposes the mix of methods: the second focuses on the strengths of both paradigms and suggests the integration. I think the second one complements the first article. Thanks!

Educational Research - Analysis of Research Journals by Nelson Dordelly-Rosales

2009-03-05T17:08:00.001-08:00

The idea is to promote educational research and its practices..
What is educational research? It refers to research conducted to investigate behavioral, social and strategic patterns in students, teachers and other participants in schools and other educational institutions.

Suggested Textbook:
Gall, M. D., Gall, J. P. & Borg, W. R. (2007). Educational research: An introduction (8th ed.) Toronto, ON: Allyn & Bacon.

Research Journals:
In Education there is particular value in journals published by the American Educational Research Association including, but not limited to: American Educational Research Journal, Review of Educational Research, and Educational Research. Examples of Canadian educational research journals include: The Canadian Journal of Education, The Alberta Journal of Educational Research, Curriculum Inquiry, and The Canadian Journal of School Psychology.

Example of educational research:

Book Review by Nelson Dordelly-Rosales
Katheleen M. Iverson, E-Learning Games Interactive Learning Strategies for Digital Delivery (NJ: Pearson Prentice Hall, 2005)

This book is about (main discussion)

• Classes of interaction: learner-interface interaction, learner-content interaction, learner-facilitator interaction, learner-learner interactions.

• Constructivist E-learning design steps: (1) identify course goals and objectives, (2) assess learner pre-knowledge and characteristics (use the appropriate language, consider learner preparation, adjust course pace, provide additional support, assess pre-training environment and learner motivation, assess available technology, consider learner’s capability of working in virtual teams or groups), (3) build motivational elements, (4) select a grounded instructional strategy (Gagne’s nine events of instruction), (5) define events, (6) select appropriate technological delivery tools (asynchronous delivery, synchronous delivery, delivery media) and interactive approach(es).

• Use of e-learning “session openers” to make a positive first impression and set course expectations, and to facilitate confidence in using new technology. Examples of icebreakers: the use of the personal blog, talk about each learner’s particular area of expertise or about favourite picture, sports, songs, movies, etc.

• Use of “scenario-based” e-learning that consists of highly engaging, authentic learning environment that allows trainees to solve authentic, work-based problems collaboratively anytime, anywhere. It involves key role play, including case studies, problem-based learning and goal-based scenarios; i.e., our course 874

• Use of “peer learning” support: belonging to a network, or community of learners is vital in a virtual environment. Opportunities for connection must be embedded in the course design to overcome the feelings of loneliness, i.e., working in pairs.

• Use of “content review and practice” to engage learners in higher order thinking tasks or in doing things and thinking about what they are doing, such as analysis, synthesis and evaluation, interpretation, problem solving, enhancing affective area, i.e., multimedia scrapbook, virtual field trip, webquests, and blog.

• Use of “group discussions” to explore issues and topics relating to the course content, express opinions, draw upon prior knowledge and construct a new one, i.e., jigsaw (online chat, e-mail, board), the projector and screen, the fishbowl, etc

• Use of “idea generation” or brainstorming to quickly develop and communicate new ideas for problem development, process revision, and problem resolution; i.e., the tope ten lists, defining excellence as it relates to the topic under study, etc.
• Use of “closers” which is a bit of ceremony at the end that allow learners to revisit the course, record their ideas, and provide a link to the workplace; i.e., websites or webpages with guest book, E-mail check up, virtual reunion, etc.
____________________________________________________________________
The author argues that:

• Until recently, most interaction in web-based training environments was technologically driven. Intelligent tutors, video, audio, and animated graphics were the accepted vehicles for adding interest and excitement to otherwise bland and boring script-based training. Although these advances are valuable, they come with a price in both development time and dollars.

• E-Learning Games contains ideas and practices that will add excitement to courseware without considerable expenditure of resources. Relying primarily on low-tech vehicles such as synchronous and asynchronous chat, e-mail, and instant messaging, the activities described in this textbook can be implemented in web-based training and educational courses alike.
______________________________________________________________________
The author makes the following statements or sites the following references on support of his/her argument (provide 2-3 quotes):

• What exactly is interaction in e-learning? Interaction is an interplay and exchange in which individuals and groups influence each other. Thus, “interaction is when there are reciprocal events requiring two objects and two actions.” (G. Moore, “Three Types of Interaction,” The American Journal of Distance Education 3 (1989):6)

• Our role as instructional designer is to move from merely sequencing material to creating highly interactive online environments in which constructivist learning may occur by creating rich contexts, authentic tasks, collaboration and abundance of tools to enhance communication and access to real world examples and problem solving, and mentoring relationships to guide learning. (T. Duffy & D. Jonassen, Constructives and Technology of Instruction: A Conversation (Hilldale: NJ: Lawrence Erlbaum Associates, 1996) p. 67
______________________________________________________________________
The author concludes that:

• It is much more effective to place learners in groups where they receive guidance on how to use web resources to explore the topic, discuss their findings with others, work together to locate answers, create their own model of motivation, and receive feedback and further guidance from facilitator. “Building ties to highly connected, central other is more efficient than links to peripheral others who are not well connected” (Iverson, 2005, p. 187)

• The author includes a long list of software resources facilitate the delivery of some activities included in the book for virtual greeting cards, webloghosting desktop collaboration, MOOs, visual diagramming, digital photo album, storyboarding, multimedia scrapbooks, virtual field trips, guest books, virtual meetings, and miscellaneous free software trials” (Iverson, 2005, p. 175-178).

• The following strategies are useful in e-learning for digital delivery (1) use e-learning design checklist (in p.179-180) (2) use a checklist to adapt and create e-learning games that fit the needs of learners (model in p. 181-183) and (3) use a variety of examples of learning activities (such as the ones that are provided in the book and in addendum D (pages185-188).

Article Review by Nelson Dordelly-Rosales

Janice Redish and Dana Chisnell, (2004) “Designing Web Sites for Older Adults:
A Review of Recent Research,” AARP, Washington D.C. 67 pages. Online:
http://search.yahoo.com/search?p=book+designing+web+sites+instructional+design&fr=yfp-t-501&toggle=1&cop=mss&ei=UTF-8&fp_ip=CA&vc=
_________________________________

This article is about (main discussion)
• Review recent, relevant research about Web site design and older adults or users. From the research reviewed in this article, the authors developed a set of heuristics to use in person-based, task-based reviews of 50 sites that older adult users are likely to go to.
• It concentrates on research from the disciplines of interaction and navigation, information, architecture, presentation or visual design, and information design. Article includes three sections: firstly, it discusses issues such as who is an “older adult”, what factors besides age must be considered? How these factors been used in research studies? What must be keeping in mind about older adults? Secondly, it deals with “Interaction Design: Designing the way users work with the site.” Thirdly focuses on “Information Architecture: Organizing the content” on Visual Design: Designing the pages, Information Design: Writing and formatting the content, and finally, fifth, it explains how “Conducting Research and Usability Studies with Older Adults.”
• The authors conducted this literature review to (a) better understand the “older adult” audience, (b) identify common usability and design issues specific to older Web users, (c) provide guidance to designers and developers of any Web site or Web-based application who have older adults in their audiences, (d) add information about –e-commerce Web sites and Web transactions to AARP’s Older Wiser Wired (OWW) Web site (www.aarp.org/olderwiserwired)
_________________________________________________________________
The authors argue that:

• Adults are more diverse than younger people are. Within this group, older adults have different experiences and different needs, habits, thoughts, and beliefs. Because of this diversity, it is extremely difficult to generalize performance, behaviours, and preferences to the million of people in a state. Some older adults take technology for granted, but for others using the Web is new territory. People in their 50s and 60s are more likely to have used computers at work. But many older adults – even those who are middle aged – are learning to use computers and the Web on their own.
• The authors propose a new tool that could be used by Web design teams to help them make decisions about where their users fall along these dimensions and thus how best to serve their audiences. The authors’ approach looks at the four factors: (a) age: including chronological age, but taking into account life experiences (b) ability: cognitive and physical (c) aptitude: expertise with the technology (d) attitude: confidence levels and emotional state of mind.
• The implications of those attributes are: those attributes can be used to judge the need for support and training and the level of complexity of features and functions that different users can be expected to handle. That is, increased age is likely to require less complexity, but increased aptitude allows for more complexity. Higher ability (that is, physical and mental fitness) allows for more complexity, and higher ability is likely to also correlate with lower age.
• “User experience” seems to include these qualities: • clear understanding by the site designers and content providers of who the users are (including demographics, domain knowledge, technical expertise, and frame of mind) and why they come to the Web site (tasks, triggers, and motivations) • plain and immediate communication of the purpose and scope of the Web site (as shown through the visual design, information architecture, and interaction design) • compelling, usable, desirable, useful, and possibly delightful content (including tone, style, and depth of content)
______________________________________________________________________
The authors make the following statements or sites the following references on support of their argument (2-3 quotes):
• It takes many roles to design a web site for older adults: DUX, a conference organized by a convergence of professional organizations, suggests that all of these roles (and probably more) contribute to designing the user experience: Authors suggest to view the following site: www.dux2005.org
• The authors suggest viewing Interaction Design Group at http://interactiondesigners.com. Interaction design is “defining the complex dialogues that occur between people and interactive devices of many types— from computers to mobile communications devices to appliances.” Humans and technology act on each other. In the case of Web sites, interaction design determines how a Web site behaves. This behaviour manifests as navigation elements: scrolling, links, buttons, and other widgets, along with how they are placed on a page, what their relationships are to each other on the page, and how easily users can recognize the elements and what the elements will do for them.
• Older participants were very likely to include widgets that were obviously clickable and visually looked like buttons (Chadwick-Dias, Ann with Michelle McNulty and Tom Tullis. “Web usability and age: Howdesign changes can improve performance.” Conference paper, ACM SIGCAPH Computersand the Physically Handicapped, Proceedings of the 2003 conference on universal usability, Issue 73-74).
• The authors quoted 57 references. Among them: Bailey, Koyani, et al. (Bailey, Bob with Sanjay Koyani, Michael Ahmadi, Marcia Changkit, and Kim Harley (NCI). “Older Users and the Web.” Article, Usability University July 2004; jointly sponsored by GSA, HHS and AARP) that found that older users tended to get lost on Web sites much more quickly than younger users “because they were penalized much more by poor labels and headers than were the younger users” and seemed less able to recover from these types of selection mistakes. Because their research shows that Web users skim or scan pages and are attracted to visual elements such as links, Theofanos and Redish suggest using highly descriptive link labels, ensuring that a link will be understandable and useful on its own. They also suggest starting links with relevant keywords and avoiding multiple links that start with the same words. This should help all types of users, not only those who use screen readers or talking versions of Web sites. Theofanos, Mary and Janice Redish. “Guidelines for accessible and usable websites: Observing users who work with screen readers.” Article, Interactions, X (6), November- December 2003, pp 38-51. ACM, the Association for Computing Machinery.
______________________________________________________________
The author concludes that:

Further research is needed to assess the relative importance of the different dimensions in designing Web sites. Older adults exhibit different usage behaviours. Realize that many older adults have cognitive and other medical limitations.
________________________________________________________________
PROGRAM EVALUATION
Modules:
Module 1 September 5 in Room 2001 at the College of Education. It will focus on the basics of program evaluation.

Module 2 September 26 in Room 2001 at the College of Education. The specific techniques involved in conducting an evaluation. Pre-planning, logic models and resources will be discussed.

Module 3 October 17 in Room 2001 at the College of Education. This module will focus data collection and analysis. The understanding and application of focus groups and online survey techniques will also be addressed.

Module 4 on November 21 in Room 2001 at the College of Education and deal with ethics in evaluation and review of the final project in the course
Assignments
One: Choose a completed evaluation; any kind, your choice. Explain the model or process used in the evaluation and identify in your mind the strengths and weaknesses of the evaluation and the approach that was taken. The finished piece should be 500 words in length. You will share your ideas on your blog.
Due Sept 12 Value - 10 marks

Two: a simulated program case study. You will choose a model or approach that you feel is appropriate to evaluate this program and explain why you think it would work. This will be a one-page document that you will post on your blog.
Due Sept 19 Value – 10 Marks

Three: Using your test organization or program you will perform an evaluation assessment. This step is used to determine the feasibility and direction of your evaluation. You will post your assessment on your blog.
Due October 15? Value - 10 marks

Four: Objectives: to become familiar with logic models as a method for understanding the workings of an organization. You will map out and get a thorough overview of your chosen organization or program you need to create a logic model. It can be in the form of a flow chart or any of the other models we have reviewed in the course. The assignment will consist of a logic model (generally a single page) and a description of the model. This will also be posted on your blog. It is due October 15. Value - 10 marks

Assignment Five: You will design and test a short survey. Include a variety of question types such as scale rating, short answer, and open-ended. You will submit the original version and the modified version based on the testing of the survey with four individuals. You will post your information on your blog.

This assignment is due on November 20. Value - 10 marks

Major assignment: Evaluation Plan

Objective: To demonstrate the ability to integrate the different tools and theories addressed in the class into an evaluation plan.

You will design an evaluation plan for the organization or program of your choice. Your final assignment will be a culmination of all we have done in the course. The plan will be a theoretical paper that outlines the program to be evaluated and the goals or objectives to be evaluated. It will demonstrate your ability to analyze a program, determine a suitable evaluation plan and create the instruments you would use to conduct the analysis. Essentially the purpose of an evaluation plan is to convince someone that you should be the evaluator for the evaluation. Hence, you want to convince an agency/institution/individual that you have the “best” team to perform the evaluation. So, an important piece of the evaluation plan is for you to describe, or elaborate upon, your reasons for selecting particular foci and approaches. We will address the specifics of this plan later in the course.

Due December 11, 2009. Value - 50 marks

Introduction to Program Evaluation

Course Description:
This course examines current models for the evaluation of educational programs. The emphasis is on exploring the range of options that is available to the program evaluator and on developing an awareness of the strengths and limitations of the models and techniques. Problems in carrying out educational evaluations are also studied: examples of such problems are the utilization of evaluation results and the ethics of evaluation. The course will use the Blackboard learning management system. You can access the course material by logging into http://webct6.usask.ca. Students will be required to create and maintain a blog to share their experiences and assignments with the others in the class (We will review suitable blog choices on the first class day).

Class Times, Appointments and Office Hours

This course will be taught in modules. If you are unable to attend any of the module you will be able to join via the Internet using a program called Elluminate. Please contact the instructor for details.

The first module will be held on September 5 in Room 2001 at the College of Education. It will focus on the basics of program evaluation.

The second module will be held on September 26 in Room 2001 at the College of Education. The specific techniques involved in conducting an evaluation. Pre-planning, logic models and resources will be discussed.

The third module will be held on October 17 in Room 2001 at the College of Education. This module will focus data collection and analysis. The understanding and application of focus groups and online survey techniques will also be addressed.

The fourth and final module will be held on November 21 in Room 2001 at the College of Education and deal with ethics in evaluation and review of the final project in the course.

I will be available to see you at any time by appointment. I will always be available to you through e-mail without an appointment.

Text: The course will not have a required textbook. If you wish to supplement the resources I have offered you in the course Owen and Roger’s book or McDavid and Hawthorne’s text would be useful additions to your professional library.

http://www.amazon.ca/Program-Evaluation-Approaches-John-Owen/dp/076196178X

Program Evaluation and Performance Measurement: An Introduction to Practice (McDavid and Hawthorne, 2005

Course Objectives
• To define and understand “What is program evaluation?”

• To understand the historical foundations of program evaluation.

• To identify and develop appropriate evaluation assessment techniques used in educational and other program settings.

• To understand appropriate data gathering techniques for evaluation purposes.

• To demonstrate the ability to create data gathering instruments.

• To understand the process and procedures involved in data analysis.

• To understand the unique roles and responsibilities of the various members of an evaluation team.

• To become aware of the ethical responsibilities of evaluators and the political implications of evaluations.

• To prepare for learning in a variety of authentic situations

http://www.schoolofed.nova.edu/arc/research_courses/sylpep.pdf
http://www.epa.gov/evaluate/whatis.htm
www.epa.gov/evaluate/whatis.pdf
http://cde.athabascau.ca/syllabi/mdde617.php
www.gsociology.icaap.org/methods/evaluationbeginnersguide.pdf
www.ocde.k12.ca.us/downloads/assessment/WHAT_IS_Program_Evaluation.pdf
www.en.wikipedia.org/wiki/Program_evaluation

Assignments

Assignment One: Choose a completed evaluation; any kind, your choice. Explain the model or process used in the evaluation and identify in your mind the strengths and weaknesses of the evaluation and the approach that was taken. The finished piece should be 500 words in length. You will share your ideas on your blog.
Due Sept 12.
Value - 10 marks

Assignment Two: I will e-mail you a simulated program case study. You will choose a model or approach that you feel is appropriate to evaluate this program and explain why you think it would work. This will be a one-page document that you will post on your blog.
Due Sept 19.

Value – 10 Marks

Assignment Three:
Using your test organization or program you will perform an evaluation assessment. This step is used to determine the feasibility and direction of your evaluation. You will post your assessment on your blog.
Due October 15.
Value - 10 marks

Assignment Four:
Objectives: to become familiar with logic models as a method for understanding the workings of an organization.
You will map out and get a thorough overview of your chosen organization or program you need to create a logic model. It can be in the form of a flow chart or any of the other models we have reviewed in the course. The assignment will consist of a logic model (generally a single page) and a description of the model. This will also be posted on your blog. It is due October 15. Value - 10 marks

Assignment Five:
You will design and test a short survey. Include a variety of question types such as scale rating, short answer, and open-ended. You will submit the original version and the modified version based on the testing of the survey with four individuals. You will post your information on your blog.

This assignment is due on November 20. Value - 10 marks

Major assignment: Evaluation Plan

Objective: To demonstrate the ability to integrate the different tools and theories addressed in the class into an evaluation plan.

You will design an evaluation plan for the organization or program of their choice. Your final assignment will be a culmination of all we have done in the course. The plan will be a theoretical paper that outlines the program to be evaluated and the goals or objectives to be evaluated. It will demonstrate your ability to analyze a program, determine a suitable evaluation plan and create the instruments you would use to conduct the analysis. Essentially the purpose of an evaluation plan is to convince someone that you should be the evaluator for the evaluation. Hence, you want to convince an agency/institution/individual that you have the “best” team to perform the evaluation. So, an important piece of the evaluation plan is for you to describe, or elaborate upon, your reasons for selecting particular foci and approaches. We will address the specifics of this plan later in the course.

Due December 11, 2009. Value - 50 marks

Module 1 What is evaluation?
What is Program Evaluation?
Please review the following material before we meet on Sept 5. They will give you a grounding in the concepts behind program evaluation.
http://www.managementhelp.org/evaluatn/fnl_eval.htm
http://pathwayscourses.samhsa.gov/eval101/eval101_toc.htm
This module is intended to introduce you to the concepts of Program Evaluation.It is not program or content specific. It does not matter what area you are most knowledgeable PE is a tool that you can apply to generate a better understanding of what is happening. The program evaluation you choose may based on your personal approach to a situation or the situation itself may point to a particular method. There are a number of approaches that will fit any given setting. Most program evaluations are short term. They are a snapshot of what is happening at a particular point in time. Longitudinal evaluations are difficult to conduct as they are more time consuming and costly. Essentially you are trying to answer the question, "Does the program do what it says it does?". Because evaluation is on-going your evaluation may steer your client in a particular direction and it will also be used to inform the next evaluation.
PE is essentially research into an organization program or process.
As you will learn when we study logic modelling four aspects of evaluation may include:
1. Input
2. Output
3. Outcome
4. Impact

The Canadian government views evaluation as:
1. Planning
2. Evaluating
3. Reccomending

You will develop your own approach to evaluation. It may be based on an existing model a combination of different factors that suit they type of evaluator you are and the situation you are involved in. The following section introduces you to some of the formatlized appraoches to evaluation.
Major theoretical concepts behind Program Evaluation
Evaluations can be formative, intended to provide feedback on the modification of an on-going program or summative, designed to determine if a process or program was effective not necessarily to change it. Here is a comparison of the two approaches.
http://jan.ucc.nau.edu/edtech/etc667/proposal/evaluation/summative_vs._formative.htm

Many modules have been developed by those who have studied PE over the years. A quick overview of the major models and the theorists who developed them is presented in this pdf document by Michael Scriven, one of the leading academics in the area of program evaluation. It is important to understand that a variety of models exist and the program evaluation has evolved in much the same way that research models in general have changed. Some of the more well-known models are the CIPP, Discrepancy, Adversary, goal-free, transactional. Here is an overview of the history and the major theoretical models in program evaluation.
Resources
809 Delicious account: http://delicious.com/wi11y0/809
Canadian Evaluation Society
http://www.evaluationcanada.ca/site.cgi?s=1
American evaluation association
http://www.eval.org/
Helpful textbooks
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R.(2004). Program evaluation: Alternative approaches and practical guidelines. White Plains, NY: Longman.
Owen, J. M., & Rogers, P. J. (1999). Program evaluation: Forms and approaches. Thousand Oaks, CA: Sage.
Posovac, E., & Carey, R. (2003). Program Evaluation â€“ Methods and Case Studies. (6th edition). New Jersey: Prentice Hall
ISBN #: 0130409669
Evaluation cookbook
http://www.icbl.hw.ac.uk/ltdi/cookbook/
Assignments for this module
1. Choose a completed evaluation; any kind, your choice. Determine the model used and identify in your mind the strengths and weaknesses of the evaluation and the approach that was taken. The finished piece should be 500 words in length. You will share your ideas on your blog. Due Sept 12
2. I will e-mail you a simulated program case study. You will choose a model or approach that you feel is appropriate to evaluate this program and explain why you think it would work. This will be a one page document that you will post on your blog. Due Sept 19

Module 2 - The process of evaluation

Before you conduct an evaluation you need to have as complete of an understanding of the focus of your evaluation as possible. You need to learn all that you can about the program, the purpose and the people that you will be working with. This means generating a thorough understanding of the organization that is connected to your evaluation. A good place to start is with any previous evaluations. This information will let you know how the organization has dealt with evaluations in the past and may help you determine if there is a willingness to put into practice the results of a study.
The following resources is a systematic look at the steps that are involved in an evaluation.
http://www.uwex.edu/ces/pdande/evaluation/index.html
Designing Evaluations : http://www.wmich.edu/evalctr/jc/DesigningEval.htm
Pdf version from the University of Wisconsin
Here is the checklist to get you through the process as a Word file.
A next step is to design a flow chart or a model of the organization you are working with that shows how the organization operates and how what you are evaluating fits into the big picture. This is done to cast a wide net to see where you will look to for input as well as to determine who will be effected by the outcomes of your evaluation. This can be done with a flow chart or what is known as a logic model. Logic models give a thorough breakdown of an organization.

Follow this link to learn about logic models
http://www.tbs-sct.gc.ca/eval/tools_outils/RBM_GAR_cour/Bas/module_02/module_0201_e.asp
http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

Here is a helpful checklist for preparing to begin your evaluation
http://www.managementhelp.org/evaluatn/chklist.htm

Working with your clients
It is important for those you are working with to understand what you will and will not do. They must also understand what is needed from them and their organization. This is where the art and science meet. You will need to carefully judge the political climate and the willingness of the organization to actually change. The case may be that the higher-ups in an organization are implementing an evaluation without the support of the members of the organization. You may be seen as a threat and it may make sense for you to spend time working on the relationship component of the evaluation.

Assignments for this module
Case study
You will need to select an organization or program to use as a model for the rest of the course. It can be an educational program, a government program, or a particular organzation that has a specific mandate. It may be beneficial to choose an organization in your local community so that you can access individuals for input in your school work. Once you have decided on who you would like to use please e-mail me your choice and why you chose the program or organization.

Assignment #3
Using your test organization or program you will perform an evaluation assessment. This step is used to determine the feasability and direction of your evaluation. You will posted your assessment on your blog. Due October.

Assignment #4
To map out and get a thorough overview of your chosen organization or program you need to create a logic model. It can be in the form of a flow chart or any of the other models we have reviewed in the course. This will also be posted on your blog. It is due October

Module 3 - Gathering and evaluating data
You will now be confident that you can proceed with the evaluation based on the results of your evaluation assessment. At this point you will need to create a set of instruments to generate data that will answer your questions about the chosen program or organization.

Designing your evaluation
Once you have done the preliminary work with the client and the focus of the evaluation you need to develop the measures and instruments that you will use to answer your questions. This means choosing the format, type and then testing the instruments to ensure that they will work properly. Here is an overview of some of the different options you have for gathering data. You may want to begin by looking at any information that has already been gathered by an organization. This may be survey data, graduation rates, or financial records. You will likely create a survey of the major stakeholders or interview them individually or in a focus group. This file will give you a good grounding in designing surveys and working with focus groups.
Creating surveys
A survey is a common way to generate data from stakeholders, employees and clients connected to a program or policy. Having clear well-written questions presented in a variety of formats will go along way to generating a reliable means to generate data. You can use existing surveys and modify them to work with the specifics of your particular evaluation. Here is a sample survey for you to review.Traditionally this has been done using a paper form. This has worked well but there is now the option of using the Internet. Using the Internet allows for data to be in a format that can be more easily collected and analyzed. The U of S has an online survey tool available for you to use. It can be accessed at http://www.usask.ca/its/services/websurvey_tool/

Here are two other useful resources for creating surveys.
Creating a paper survey
Getting better results from online surveys
Focus groups
FG allow you to meet with many people at once to discuss and collectively generate data. Focus groups where you gather people together to discuss issues are also useful. They will allow you to get homogenous or mixed groups to share and feed off one another.
Here is an example of a document used to organize and conduct a focus group.
Validity of your instruments
Having confidence in your data gathering instruments is very important. You cannot have any useful results if they have been based on flawed data. This is why evaluators will often use instruments that have been used and tested by others. If possible taking an existing survey and modifying it slightly to fit your client's needs will give you peace of mind and will be a better judge of what you are trying to measure. If you are designing a survey from scratch you need to make sure that what you are asking and how you are asking it is correct. This means sharing your instrument with others in the know or experts in measurement. Pilot testing and useability testing your survey with a group similar to the one you will be surveying is also very important.
Once you are confident that your instruments are valid and reliable then you can gather your data.

Data analysis
Results must be shared for your evaluation to of use to anyone. You should make recommendations to those who offer the program. This cannot be done without a careful analysis of the data that you have collected. Once you have gathered enough data you will have to compile and compare the results with the original objectives. This link gives you some insights into the process of data analysis http://www.uwex.edu/ces/tobaccoeval/resources/surveynotes28aug2001.html#defs
http://hsc.uwe.ac.uk/dataanalysis/qualTextDataEx.asp
(qualitative analysis)
Here is an example of an interview transcript that has been analyzed. Read through it and then test your own skills with what the researcher discovered. http://hsc.uwe.ac.uk/dataanalysis/qualTextDataEx.asp

From the same website here is a look at quantitative data analysis. http://hsc.uwe.ac.uk/dataanalysis/quantWhat.asp
Don't be scared, you will not have to become an expert in this type of analysis (At least not for this class).

Assignment #5
You will design and test a short survey. I have included an example for you to use as a guide. Include a variety of question types such as scale rating, short answer, and open-ended. You will submit the original version and the modified version based on the testing of the survey with a group of 4 different individuals. You will post your information on your blog. This assignment is due on November 16, 2009.

Module 4 - Ethics of evaluation
It is important that as an evaluator you need to be objective. Your primary purpose is to serve the needs of your client. That being said you must design and conduct your evalaution with the needs and protection of all those impacted by your results. There is often fear associated with the evaluation of one's performance. This is especially true when the evaluator is someone who is coming from the outside and does not have the chance to have a longitudinal look at programs or organizations.
Program evaluation standards are put forth by the American Evaluation Society to guide evaluators in their conduct.

This powerpoint presentation looks at the guiding principles of evaluation.
Sharing the Evaluation Results ( I found this and modified it for our class). http://www.busreslab.com/ESATsharingresults.htm

It is critical to share results in a timely manner for at least two reasons:

1. Everyone must know where the organization as a whole and their individual areas stand if you are going to fully leverage the creativity and efforts of the employee base.

2. Participants need to know that the time they spent in completing the survey was worthwhile.

Each organization has its own information-sharing culture in place. While in some cases, particularly if the survey showed communication to be a problem, the process will need some adjustment, we recognize that each organization will have an approach to information dissemination that it typically leverages. As such, modifications to our recommended approach may be in order to account for an organization's information-sharing culture.

The Basic Principles of Sharing Survey Results

1. Be honest. An organization must be willing to share both its strengths and its areas in need of improvement. Employees will see through attempts to hide or "spin" information.

2. Be timely. The sooner you release results, the sooner the organization can begin to move toward positive change.

3. Share appropriate information at each level. Senior management will need encapsulated results and access to detailed results for the organization as a whole and differences between divisions/departments. Division managers will need to know how their division compares to the organization as a whole and how departments in the division compare to each other. Department managers will need to know how their results compare to the organization as a whole and to the division to which they belong.

4. Don't embarrass people in front of their peers. Teamwork and morale can be harmed if, for example. Rather than pointing out low-scoring departments to all department managers, let all department managers know how they fared compared to other departments via one-on-one meetings.

5. Discuss what happens next. After the results have bee presented, let the audience know what steps will be taken to improve those items in need of improvement.

6. Respect confidentiality. Don't present information that would make people feel that their responses are not confidential. For example, it would not be appropriate for anyone in the organization to have access to comments for a small department, since some people may be able to accurately guess who made what comment. Your research supplier should assist in this by not providing information that could breach, or could be perceived to breach, confidentiality.

Process Considerations

Have a plan in place to disseminate information before the survey has been completed.

1. The CEO/president should be briefed by the internal project manager and/or the research supplier.
2. The CEO/president should share the results with division managers. Overall results should be shared in a group setting. Individual results should be shared in one-on-one meetings.
3. Key findings and implications should be highlighted in each presentation. Detailed results also should be presented. However, take care to avoid drowning people in information. This can be done by relying more heavily on graphics than on detailed tables to communicate.
4. Give employees an overview of overall results through the best means possible. For some organizations, this will be in a group setting. For others, it will be via email, Intranet, or newsletter. Consider using multiple methods.
5. Department managers should share departmental results with employees in a group meeting. It may be helpful to have an HR manager assist in the presentation. If HR managers will be part of this process, planning ahead will help the meetings to proceed smoothly and take place in a timely manner.

In all communications, make sure the communication is "two way." Questions should be encouraged.

Assignment
The final assignment in this class will be a proposed evaluation of the program of your choosing. Consult the sylabus and other material I have shared with you for details.

Here are some sample proposals.
Proposal 1
Proposal 2

These are examples of requests for proposals.
This is when organizations solicit input for evaluations.

Request for proposal 1
RFP 2
RFP 3
Evaluation reports
Sask Aboriginal Literacy Report pdf
Sask Literacy Report pdf

Saskhealth evaluations http://docs.google.com/gview?a=v&q=cache:V5tQS5tiZQgJ:www.health.gov.sk.ca/hen-newsletter-072006+evaluation+proposal+government+saskatchewan&hl=en

http://www.nrcan.gc.ca/evaluation/reprap/2006/e06003-eng.php
________________________________________________________________

What is your Philosophical Paradigm in Educational Research? Reflections by Nelson Dordelly-Rosales

2009-02-17T11:13:00.000-08:00

Deciding what is your paradigm is an effort depending of the type of research.

Let's review main concepts. You will find in this site a list of articles that will help to discuss the problem. Below I transcribe the answer that Wiki provides regarding the question:
What are the major differences between quantitative and
qualitative research?
Retrieved May 7th, 2009 from:
http://wiki.answers.com/Q/What_are_the_major_differences_between_quantitative_and_qualitative_research

“QUALITATIVE: Concerned with meaning, rather than with measurement. The emphasis is on subjective understanding, communication, and empathy, rather than on prediction and control, and it is a tenet that there is no separate, unique, ‘real’ world. Qualitative methods vary, and are generally based on empirical research, and all qualitative researchers are positioned subjects. As such, the rigour of their research depends not only on the suitability of the methodology, the use of multiple methods, and the inclusion of verbatim quotations, but also on its credibility and transferability. For a checklist for evaluating qualitative research, see Baxter and Eyles, Transactions of the Institute of British Geographers 22. There is some discussion over the extent to which the researcher should intervene, and much awareness of the way in which any research process will affect the subjects of the investigation. See also L. Kong, Area 30.”(http://wiki.answers.com)

“QUANTITATIVE: A quantitative attribute is one that exists in a range of magnitudes, and can therefore be measured. Measurements of any particular quantitative property are expressed as a specific quantity, referred to as a unit, multiplied by a number. Examples of physical quantities are distance, mass, and time. Many attributes in the social sciences, including abilities and personality traits, are also studied as quantitative properties and principles.”(http://wiki.answers.com)

References
• Michell, J. (1993). The origins of the representational theory of measurement: Helmholtz, Hölder, and Russell. Studies in History and Philosophy of Science, Vol. 24 No. 2, 185-206.
• Nagel, E. (1932). Measurement. Erkenntnis, 2, 313-33, reprinted in A. Danato and S. Morgenbesser (Eds.), Philosophy of Sciences (pp. 121-140). New York: New American Library.
What are the major differences between quantitative and
qualitative research?
Retrieved May 7th, 2009 from http://wiki.answers.com

How important is a portfolio? By Nelson Dordelly-Rosales

2009-02-17T10:52:00.000-08:00

An e-portfolio on educational research provides us with the opportunity to share our views about this important field. The challenge to shift to new strategies to assess academic success entails many decisions and changes. This portfolio aims to select, collect, reflect and connect links and publications providing the opportunity for feedback. The focus is on introduction to educational research using an e-portfolio. Through this activity I show that designing an e-portfolio is a matter of assessment and decision making process: Why e-portfolio is a decision-making process to assessing academic success? How do I make e-portfolio decision-making an ongoing and consistent process to assess academic success?According to George Lorenzo and John Ittelson (2005) portfolios can help show the student’s talents; that is, students will be able to display their best products by creating their portfolios. In addition, “positive effects on students’ learning have occurred through the use of portfolio assessment” (Hewett, 2008 p. 3202). E-portfolios “have opened the doors to many opportunities for the person who has a professionally organized display of their finest works” (Hewett, 2008 p. 3200).

How to design an e-portfolio?

1. An Overview of E-Portfolios
- What is a portfolio and why use a portfolio?
- How to develop an e-portfolio: a multidimensional process
- Key Steps in Portfolio Design.
- E-Portfolios in Higher Education

2. E-portfolio as a different way to assess academic success
- A multi-faceted process
- Candidate-centered
- Development of effective e-portfolio:
a) Collection and Selection
b) Reflection and Direction
c) Connection and Publication

3. E-portfolio as a decision-making process to assess academic success: How do I make e-portfolio decision-making an ongoing and consistent process to assess academic success?

Decision 1 Audience: Who should be involved?
Decision 2 Outcomes: What are the purposes?
Decision 3 Contents: What do I need to know?
Decision 4 Strategies: How and when to accomplish the assessment?
Decision 5 Evaluation: Is it an ongoing improvement process?
Continuous process of evaluation and future use

Conclusion and suggestions for future research

Bibliography

------------------------------------------------------------------------------------

Designing Courses and Teaching on the Web by Nelson Dordelly-Rosales

2009-02-17T10:34:00.000-08:00

E- Portfolios in Higher Education: Different higher education institutions use e-portfolios in a variety of careers including art, architecture, medicine, education, journalism, among others. According to Stephenie M. Hewett, the e-portfolios are used “to display a person’s skills and talents” (2008, p. 3200). These institutions have shown and continue to show a strong interest in the theme of E-Portfolios, which is a “different way to assess academic success” (Stephenie M. Hewett, 2008). However, what does it mean ‘a different way’? There is a great need for expertise and experience in this area.

Book Review by Nelson Dordelly-Rosales

Mercedes Fisher, Designing Courses and Teaching on the Web: A “How-To” Guide to Proven, Innovative Strategies (Lanham: Scarecrow Education, 2003)
________________________________________________________________________

This book is about (main discussion) Themes or topics

• The Constructivist Approach to Online Learning with Technology, Effective Planning and Design for Online Teaching, Foundation for Instructional Screen Design, Communication and Community Create Online Success and Evaluating the Student.

• This book focuses on pedagogical strategies, guidelines, rubrics, and plans for implementation, and explains the benefits and liabilities of interactive learning, online collaboration, knowledge sharing, project-based learning, future-thinking faculty, and technology supported curriculum, and program innovation.

• This book provides information, examples and check lists of online course maintenance, technology resources, tools, online tutorials, journals, instructor’s websites and “real-time chat activity.”
________________________________________________________________________
The author argues that:

• The framework presented in the book, provides an excellent scaffold for instructors to base their web-based instruction on.

• Successful online courses must become systematic, even organic.

• Soon, education will no longer be defined by static guidelines but rather by growing, changing, and evolving sets of opportunities, technologies, projects and people. Connecting others that might not otherwise be connected can provide important, new ideas to the group.
________________________________________________________________________
The author makes the following statements or cites the following references on support of his/her argument (provide 2-3 quotes):

• The challenge for teachers is to use the technology in new and unique ways instead of just trying to recreate traditional lessons. “’Shovelware’ is widespread in today’s online courses. Instead of making a pedagogical change in delivery, a teacher simply “shovels” current content into new container (A.B. Fraser, 1999, b8). Like, Arthur C. Clarke, we agree that information is not knowledge, knowledge is not wisdom, and wisdom is not foresight. Each grows out of the other and people need them all.” p.xvi

• Socrates noted that a teacher is only a midwife to students, who must carry out the labor of learning themselves. There is no learning unless the student is the worker. The student learns by listening, by writing, by arguing, by imagining, by building, by drawing, by experiencing. ( Fisher, 2003 p.13

• Learning is a social process that occurs through interpersonal interaction within a cooperative context. Individuals, working together, construct shared understanding and knowledge (David Johnson, Roger Johnson, and Karl Smith, 2002)
________________________________________________________________________
The author concludes that:

• There are five major challenges of proactive application of online teaching and learning: (1) “selectivity,”(depending on time and money), (2) “malleability,” creativity the author explains that ‘morphing’ method describes computer user’s ability to take something and change it into something else, i.e., Forrest Gump wherein the fictional character interacted with 3 dead presidents…as educators and course designers, we have always rested in the notion of fact, verifiability, and reality, (3)“vulnerability,” the author says that the vulnerability that technology has created is both liberating and threatening in the sense that it expands students ‘capabilities as well as increases their dependence on technology. Also from a knowledge standpoint, students are becoming increasingly dependent on those people who are proficient with technology (4) “exclusivity,” that is, some people know about technology and others don’t know, (5) and “superficiality” the academic and social challenge for instructors is to navigate through the “clutter” and accumulate a true body of knowledge. (Fisher, 2003, p. 167-169)

• Regardless of the teacher experience, s/he can use the information, instructional strategies, principles, activities, and processes outlined in this book to make decisions about pedagogical issues and to become a better course designer and facilitator. The teacher can create online courses with the image s/he wants and the impact s/he needs. (Fisher, 2003, p. 176).
________________________________________________________________________
The author feels that the challenge for instructors and institutions is how to create educational experiences for students that enhance learning and take advantage of the inherent capacity of the World Wide Web. A unique aspect of quality online courses is how they rely heavily on effective collaboration to create a meaningful learning environment. (Fisher, 2003, p.1-224).
________________________________________________________________________
Are any references given (footnotes or bibliography)? What is the size of the reference section? Are the references recent, important? How are the references used: for support, rebuttal, etc.?

One hundred forty six references are provided, dating from 1947 to 2002.