185: Pros & Cons of Various Assessments

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 5.02.10 PM

 

Presentations
  • Assess content knowledge and skills
  • Pros
    • Authentic audiences can create authentic contexts for assessments
    • Allows for integration of complex skills
  • Cons
    • Difficult to setup, especially with large classes
Written Products
  • Assess content knowledge and skills
  • Pros
    • Students work over extended periods of time on revisions
    • Allows for personal craftsmanship, expression, and pride
  • Cons
    • Difficult to assess individual contributions in a group project
    • Not all learning outcomes are readily assessed by this product
Tests
  • Assess content knowledge
  • Pros
    • Allows for standardized administration to all students
    • Good individual assessments
  • Cons
    • Difficult to assess skills
Self Reports
  • Assess habits of mind
  • Pros
    • Assess student attitudes, reflections and thinking processes
    • Students can realize impacts of project work
  • Cons
    • Hard to develop reliable assessment criteria

 

3-sowhat
Knowing the purposes, advantages and disadvantages of various assessments can help teachers select the right assessment to assess the right concept, habit, or skill.  Applying this type of analysis to a wider collection of assessments can make teachers more effective and efficient at selecting the right assessments for projects.

 

4-nowwhat
 
Preparation Steps
  • Evaluate bundle of upcoming standards.  Unpack standards and write set of academic long term and supporting learning targets.
  • Determine which habits of mind and workplace competencies students need to successfully learn content and develop products and uses these selected skills to write character  learning targets.
  • Use purposes, pros and cons of learning assessments to select the right type of assessment for each academic and character  learning target.
  • Create rubric that assesses key academic and character learning targets.
  • Gather exemplars that go with rubric criteria
Early Implementation Steps
  • Implement assessments throughout the product to assess and guide student understanding and skills as they develop throughout the project
  • Set aside project checkpoints to meet with students in formal conferences to generate timely reflection and feedback on major works in progress.  Set aside time for students to use feedback to improve products.  For more details on what this could look like, see this article: Writing workshop
Advanced Implementation Steps
  • Use analysis of student work to improve assessments (especially rubrics).
  • Use Assessments Data base to discover and implement new assessment strategies other than traditional quizzes and tests
  • Analyze purposes, pros and cons of favorite assessments other than the ones listed in this article.

 

5-relatedstuff

184: Plan the Assessment

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 4.29.05 PM

 

 
  1. Align Products with Outcomes
    • Identify culminating products for a project that provide opportunities to apply skills and understandings from targeted standards
    • Use multiple products and multiple checkpoints to provide formative feedback that students use to improve their understandings and products
    • Use artifacts – evidence of student thinking – to assess targeted habits of mind and skills
    • Unpack standards and use unpacked standards to write product rubrics
    • Questions to answer with assessments
      • How well do students know the content?
      • What are students’ skill levels?
      • How well do students apply knowledge while preparing products?
    • Work backwards to make sure collection of assessments assesses all academic and character learning targets
    • Plan out culminating products –
      • these should involve transfer of knowledge and enduring understandings
      • involve students in peer and self assessments of culminating products
    • Plan out multiple products due at checkpoints so that students can get early and timely feedback
      • examples: outlines, plans, blueprints, storyboards, report drafts, websites, final drafts, etc.
    • Plan out artifacts that students will use to display their thinking as it evolves
      • examples: notes, journal entries, emails, interviews, reflections, etc
  2. Know What to Assess
    • Targeted standards, habits of mind, and 21st century skills can be further unpacked and rewritten as a set of academic and character learning targets.
    • Learning targets can be long term (takes several days) and supporting (one day focus)
    • Learning targets align with standards and other learning goals, they are specific and written in student friendly language.
    • Learning targets can be used to convey learning goals to students.
  3. Use Rubrics
    • Use rubrics to make project expectations explicit and transparent
    • Use models to help convey meanings of rubric criteria. See this article: Models, critiques & descriptive feedback
    • Analytic rubrics divide up criteria into several categories
    • Holistic rubrics combine all rubric criteria into one score
    • Design rubrics based on reflections and analyses of student work
    • Focus criteria on central features of performance, not the features that are necessarily the easiest to see and count
    • Use specific descriptive language to distinguish between criteria at different levels
    • Use student friendly language that students can use to self-assess and peer-asses products
    • When descriptive language fails to convey expectations clearly, use examples.
    • 3 common features of rubrics: elements, scales and criteria.
      • Elements are the names of the categories that group rubric criteria
      • Scales are the words that describe levels of performance.  Example: emerging, proficient, advanced
      • Criteria – rubrics descriptors of specific expectations
        • observable and/or measurable
        • focus criteria on critical understandings and skills
        • if possible use student work to develop criteria
        • use Bloom’s Taxonomy to write criteria at different levels
        • link scoring criteria to specific standards
        • use student-friendly language
        • maintain a high standard for exemplary work
        • judge the product, not the process to create the product – avoid having to base grades on guesses

 

3-sowhat
Standards-based project-based learning uses a collection of formative and summative assessments to check if students are learning knowledge and skills related to standards, habits of mind, and workplace competencies.  Teachers need to design a balanced portfolio of assessments that will generate feedback for all targeted knowledge and skills.  Students need timely feedback to clear up misconceptions, develop understandings and skills and improve products.

 

4-nowwhat
Preparation Steps
  • Evaluate bundle of upcoming standards.  Unpack standards and write set of academic long term and supporting learning targets.
  • Determine which habits of mind and workplace competencies students need to successfully learn content and develop products and uses these selected skills to write character  learning targets.
  • Create a portfolio of assessments that can assess all academic and character learning targets.
  • Create rubric that assesses key academic and character learning targets.
  • Gather exemplars that go with rubric criteria
Early Implementation Steps
  • Implement assessments throughout the product to assess and guide student understanding and skills as they develop throughout the project
  • Set aside project checkpoints to meet with students in formal conferences to generate timely reflection and feedback on major works in progress.  Set aside time for students to use feedback to improve products.  For more details on what this could look like, see this article: Writing workshop
Advanced Implementation Steps
  • Use analysis of student work to improve assessments (especially rubrics).
  • Use Assessments Data base to discover and implement new assessment strategies other than traditional quizzes and tests

 

5-relatedstuff

183: Crafting the Driving Question

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 8.33.40 AM

 

Driving questions are provocative
  • engages student throughout the duration of a project
  • example: Do music videos paint an accurate picture of America?
Driving questions are open-ended
  • no easy answers
  • requires higher level thinking
  • require students to integrate, synthesize and critically evaluate info
Driving Questions go to the heart of a discipline or topic
  • can focus on central controversies of the discipline
  • can require discipline-specific processes to resolve
  • example: How safe is our water?
    • biology / chemistry / physiology project
Driving questions are challenging
  • opportunities to try out unfamiliar behaviors
  • confront difficult issues
  • example: When are people justified in revolting against an established government?
    • project on conflict and revolution in Latin America
Driving questions can arise from real-world dilemmas that students find interesting
  • example: How could we build a new community center using only materials that are native to the state?
    • project on physical and chemical properties
Driving questions are consistent with curricular standards
  • lead students to master knowledge and skills in project standards
Refining the driving question
  • Broaden the question:
    • Ex: Was Truman’s decision to drop the bomb justified? -> Can the use of nuclear weapons be justified?
  • Add predictions to question:
    • Ex: How have robotics and automation changed our society in the past century? -> How might robotics and automation change our town and its businesses in the next century?
  • Add expert guidelines and opportunities to examine central themes:
    • Ex: What happened to the ancestral Pueblo people? Create an exhibit using words and pictures. -> Why do civilizations such as the ancestral Pueblo, Inca, or Aztec civilizations disappear?  Put together a presentation suitable for an archeology convention that supports your case.
  • Go local
    • Ex: What is global warming? -> Should we be worried about global warming in our town
  • Add investigations of change
    • Ex: What have been the the most popular novels among teenagers in the last 30 years? -> How has reading changed for teenagers over the last 30 years?
  • Include a problem that needs solving
    • Ex: What is radiation fog and how can it be dangerous? -> How can we reduce traffic accidents associated with radiation fog?
For more discussion on driving questions, see this article: Essential questions.

 

3-sowhat
Driving questions motivate students to apply their new knowledge and skills to tackle a difficult problem.  They are lighthouses that inspire and unite activities and products in a project.  A good driving question can be used as a pre-assessment, formative assessment and summative assessment.

 

4-nowwhat
Preparation Steps
  • Analyze upcoming bundle of standards
  • Identify authentic and interesting themes that go with bundle of standards
  • Craft a driving question that highlights important issue(s) in the chosen project theme
  • Examine driving question using revising methods (see list above) and see if any of these can be used to improve the question
Early Implementation Steps
  • Display the driving question (online in project hub and physical in readily visible part of classroom)
  • Use driving question to trigger knows and need-to-knows
  • Use driving question to frame products and scaffolding activities
Advanced Implementation Steps
  • Ask for feedback on driving question and use that feedback to refine the question
  • Teach students how to create related questions from driving question and use these as the center for their projects
  • Use driving question as pre-assessment and priming the brain activity near the start of a project
  • Maintain a concept map wall that goes with driving question that is updated throughout the project

 

5-relatedstuff

175: Assessing Learning Activities (1 of 2)

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 9.04.05 PM

 

  1. RSQC2
    • Description
      • Students respond briefly to the prompts: recall, summarize, questions, connect, and comment.
    • Purpose
      • Teachers can compare detailed info on students’ recall and understanding of previous classes with their own
      • Informs teachers of questions and comments that need to be addressed in a timely manner
      • Provides students with a framework for recalling, summarizing and processing information.
    • Step-by-step procedure
      • Recall:
        • At the beginning of class, ask students to make a list of words/phrases of important and useful information from the previous class. (1-2 min)
        • Ask students to circle top 3 to 5 points and rank them in order from most to least important. (1-2 min)
      • Summarize:
        • Ask students to write a summary sentences that covers as many of the important topics as possible (1-2 min)
      • Question:
        • Asks students to write 1-2 unanswered questions from last class (1-2 min)
      • Connect
        • Ask students to write a sentence or two to connect some of the important topics from last class to overarching course goals (1-2 min)
      • Comment
        • Write an evaluative statement about the class.  Can offer sentence stems such as:
          • One thing I enjoyed in the last class was _________
          • What I found most/least useful was __________
          • During most of the class, I felt _____________
      • Collect the RSQC2 feedback and let students know what kind of feedback they will receive and when they get it.
    • Analysis Steps
      • Analyze one type of response – don’t do all 5 letter responses.
      • Compare their responses to your responses to the same prompts.
        • Note additions, commissions and errors.
        • Assess degree of fit – how close their key points match yours
        • Look for patterns in students’ responses
      • Share key positive things you noticed while analyzing their assessments.
      • Share tips for overcoming some of the difficulties noticed while analyzing their assessments.
    • Extension activities
      • Encourage students to share their results with each other.
      • Gather responses to create a class Recall list.
      • When students are familiar with the strategy, let them facilitate it.
      • Can assign a committee to gather and analyze these results prior to class time.
      • Use at the end of long classes.
      • Let students work in pairs or small groups to produce responses.
    • Pros 
      • Immediate feedback on what students recall, value, question, etc.
      • Highly structured way to recall, summarize and evaluate course materials and share responses with others
      • Connections step builds bridges between prior knowledge and new material
      • Using repetitively will help students review, reorganize, reconsider and integrate major points of the course
    • Cons
      • Time consuming, especially on first attempts
      • If overused or used poorly, can become a mindless activity
      • Too much data to analyze at one sitting
      • May frustrate students to not get feedback on all points
    • Caveats
      • Let students know what parts of the data you will focus on during your analysis
      • May need to model responses during first times
      • Allow for more time on first attempts
      • Don’t ask for questions or comments unless you are ready to respond to them in a thoughtful manner
      • Use time limits to encourages students to work quickly
  2. Group-Work Evaluations
    • Description
      • Students provide feedback on group learning experiences
    • Purpose
      • Teachers can learn what’s working (and not) in learning teams
      • Teachers can learn if they need to intervene on situations that are hindering learning
    • Step-by-step procedure
      • Decide what you want teachers and students to learn about group work.  Design questions around these.
      • Select top 4-5 questions to design a Group Work Evaluation Form for gathering feedback
      • Before handing out forms, explain the purpose and process to students.
    • Analysis Steps
      • Tally responses to questions.  Note: if this data is gathered in Google forms, tallies can be done quickly in Google spreadsheets.
      • Identify outlier teams who are doing very poorly and very well.
      • Intervene on struggling teams.
      • Try to learn from very successful teams and share their strategies with the class.
    • Extension activities
      • Have students come up with group evaluation questions.
      • have students brainstorm solutions to concerns raised during the assessment.
    • Pros 
      • Detect conflicts before they permanently damage team morale.
      • Students and teachers can find out what’s working well and profit from that info.
      • Students learn about common advantages and disadvantages of group work.
    • Cons
      • May raise concerns that students do not know how to resolve.
      • Students who hate working in groups may use this process to sabotage their teams
      • Some students may not like focus on process over outcomes
    • Caveats
      • When things are not going well, there is social pressure to lie.  Make it clear that you expect honest answers, especially if it’s hard to do so.
  3. Reading Rating Sheets
    • Description
      • Students rate their reading assignments
    • Purpose
      • Teachers learn how motivated students are by assigned readings
      • This data can help teachers think about how to frame these readings in lessons and how to rethink the course reading list
    • Step-by-step procedure
      • Determine your reasoning for student rating the texts.  Reasons could include:
        • to make deacons about future syllabi
        • call attention to specific parts of text
      • Write a few questions (no more than 4-5) that gather data you want to learn or serve your intended aims.
        • most – yes/no format
        • 1-2 short answer format (reasoning and explanations)
        • include at least one question that asks students how thoroughly they read the text they are rating
      • Try answering assessment yourself.  Revise as needed.
      • Create a simple Reading Response Sheet.  Ask students to complete it as soon as they finish a reading or give time in class to do it.
    • Analysis Steps
      • Tally responses for the yes/no questions for the students who read the text.  (Note:  If students submit their responses via Google form, theses tallies an be done quickly in Google spreadsheet)
      • Look for patterns in the comments.
      • Focus most on data and comments that rate the learning value of the reading.
      • Share interesting results and patterns with students.
    • Extension activities
      • Use same basic format to assess other learning aides: lectures, videos, etc.
      • Let students tally the results
      • Have students discuss their responses in small groups and brainstorm ways to get more out of the readings
    • Pros 
      • Draws attention to readings as learning aides
      • Recognizes that students can evaluate the usefulness of texts
      • Feedback on how well students read the assignments
    • Cons
      • Students who dislike reading may rate all texts low.
      • If most students don’t read texts deeply, data will be useless.
    • Caveats
      • Include question that separates feedback of students who read the text and didn’t
      • Don’t suggest you will change reading assignments unless you intend to do so

 

3-sowhat
The RSQ2C assessments is a strategy that students can use to process learning activities and that teachers can use to answer timely questions and to learn how much students are gathering from learning activities.

 

The Group-Work Evaluations can show students and teachers what is working well and not well in cooperative learning activities.  The data can also by used to facilitate timely meetings to resolve conflicts that are hindering teams from learning.

 

The Reading Rating Sheets can help teachers learn what students are gathering from texts and whether or not they think the texts are helpful.  This data can inform how readings are framed in lessons and can impact the future design of syllabi.

 

4-nowwhat
Preparation Steps
  • Decide what learning data you would like to learn about students and what you will do with that data.
  • Select the strategy that will help you learn the things you want.
  • Prepare resources that go with that strategy.  See above for details.
Early Implementation Steps
  • Model how to complete the assessment.  State the purpose of the assessment.
  • Administer the assessment.
  • Analyze the assessment data.  Share key learnings with the students.  Describe how these will affect future teaching and learning.
Advanced Implementation Steps
  • Try out some of the extension activities suggested above.
  • Use student feedback gathered from these assessments to modify lesson plans and course design.

 

5-relatedstuff

173: Assessing Metacognition (3 of 4)

  1. Classroom Opinion Polls
    • Description
      • Students complete anonymous polls of their opinions
    • Purpose
      • Discover student opinions related to course-related issues
      • Helps students become aware of opinions that can support or hinder learning
      • Teachers can get advance information that can help him or her better frame course-related issues
    • Step-by-Step Procedure
      • Preview upcoming materials and search for issues that may have related opinions that can affect learning.
      • Choose 1-2 issues to convert to Classroom Opinion Poll questions.
      • Create a polling electronic form or paper form.  Tools for electronic forms: Socrative, Nearpod, PollEveryhwere, Mentimeter
      • Administer the assessment.  Remind students not to put their names on the paper forms if you use these.
    • Analysis Steps
      • Count and tally responses.
      • Electronic tools (see above) will count and tally responses automatically.
    • Extension Ides
      • Ask students to justify their responses.
      • Have students write rebuttals for opinions different than their own.
      • Let students come up with polling questions.
    • Pros
      • Low floor task that is accessible to all
      • Showing poll results helps students acknowledge the diversity of opinions in the class
      • Teachers gain info that can help refine/reframe future lessons
    • Cons
      • May find that students are opposed to using evidence to justify opinions
      • Students may find it difficult to commit to a response
    • Caveats
      • Be prepared to discuss issues related to opinion polls
      • Gather assessment data anonymously
      • Model respectful consideration of varied opinions
      • May opt to summarize data in private in order to hide reactions to unexpected surprises in the data
  2. Double Entry Journals
    • Description
      • Students take notes in 2 columns.  On one column that write key ideas, assertions and arguments learned during activity.  In the other column they explain the personal significance of those ideas and ask questions of the ideas.
    • Purpose
      • Detailed feedback on how students read, analyze and respond to assigned texts
      • Teachers learn what students value in the text
      • Students learn about how course issues relates to issues in their personal lives
      • Students become more aware of how they read and respond to texts
    • Step-by-Step Procedure
      • Select important parts of text to be annotated.  Should be short, challenging and provocative.
      • Show students how to setup notebook page for the strategy.
      • Model how to use the strategy by getting the annotations started on both columns while thinking aloud.
        • on left column – copy key quotes and passages from the text
        • on right column – explain why they chose each excerpt – could include agreements, disagreements, questions etc
        • encourage 2 columns to interact like a dialogue
    • Analysis Steps
      • Compare students’ left column to key points in the text
      • Try to categorize the types of responses that appear in the right column
    • Extension Ides
      • Let students do a double entry journal on a lecture – provide recording of the lecture
      • May offer 2-3 excerpts for them to interpret and have them add a couple more items in the right column
      • Use as first step in a bigger writing assignment – write an essay based on double entry journal
      • After they’ve done this several times, get students to look over several double entry journals and notice patterns in the types of excerpts they choose to include
    • Pros
      • Student practice careful reading and responding to texts
      • Encourages students to look for personal meanings in texts
    • Cons
      • Students may try to write for the teacher instead of candidly write for themselves
      • Students may struggle with this strategy at first
    • Caveats
      • Importance to distinguish between lack of skill with this note-taking strategy and lack of awareness
      • Students may need a lot of coaching before students start genuinely sharing their opinions
  3. Profiles of Admirable Individuals
    • Description
      • Students write a brief focused profile of an individual in a field related to the course whose values, skills and/or actions they admire
    • Purpose
      • Teachers learn what students value
      • Students acknowledge what they value in professions related to the course
    • Step-by-Step Procedure
      • Try out strategy yourself – write a profile for an admirable individual for a course.
      • Decide what criteria you will use to assess the profiles.
      • Develop clear instructions for profiles.  Be sure to include length limits and to define what populations students can draw their people from
    • Analysis Steps
      • Notice the characteristics and values that appear repeatedly in students’ profiles.  Tally these.
    • Extension Ides
      • Provide a list of people who students can write profiles for in case they can’t come up with a person.
      • Have students rank the characteristics in their profiles in order from most to least important.
      • Have students write parallel profiles of people who are not admirable and explain their negative appraisals.
      • Have students work in teams to read each other’s appraisals and come up with a composite list of admirable characteristics.
    • Pros
      • Requires students to consider their own values.
      • Teachers can learn about role models who have influenced students.
    • Cons
      • May make some students uncomfortable.
      • May need to do research to identify people for profiles.
      • Specifying admirable characteristics of individuals can be challenging.
    • Caveats
      • Keep assessments anonymous
      • Model how to write good profiles that elaborate on individual’s admirable qualities
      • If students chose someone you’ve never heard of, remembers that identifying the admirable qualities is more important than knowing the role model

 

3-sowhat
All 3 of these strategies have students notice and share personal connections they are making with course.  Making personal connections with course materials is one powerful way to motivate students and to help them remember new content.

 

Classroom opinion polls can help teachers identify student opinions that may help or hinder learning in the course.  The double entry journal can help teachers see what students value in assigned texts and why they value these.  The profiles of admirable individuals can help teachers see what positive characteristics students see and value in professionals who relate to the course.

 

4-nowwhat
Preparation Steps
  • Identify what you would like to learn about students’ beliefs, ideas, etc.
  • Identify the strategy that would most readily access the information you’d like to learn about students.
  • Create resources that go with the selected strategy.  See above for details.
Early Implementation Steps
  • Describe how to complete the assessment.  Provide early coaching in the from of modeling if needed.
  • Give students time to complete the assessment inside or outside of class.
  • Analyze the assessment to look for interesting patterns and comparisons.  Compare these with course character learning targets
  • Share insights from analyzing the assessment with the class.  Facilitate discussions related to these insights.
  • Describe to students how these insights will impact future teaching and learning in the course.
Advanced Implementation Steps
  • Try out some of the extension activities.  See above.
  • Use knowledge of students to reframe upcoming projects and scaffolding activities to more closely engage with students’ interests and values.

 

5-relatedstuff

172: Assessing Metacognition (2 of 4)

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 9.33.21 PM

 

  1. Goal Ranking and Matching
    • Description
      • Students list a few learning goals they have for the course and rank their importance and difficulty.
    • Purpose
      • Measures degree of fit between student and teacher goals
      • Can be used to create shared class goals
      • Students identify and clarify their learning goals
      • Students connect their learning goals with course goals
    • Step-by-Step Procedure
      • Clarify instructional goals for the course.  This can be down by writing long term character learning targets for the course.
      • Assess negotiability and flexibility of character learning targets – are you willing to change these in response to knowing students’ goals?  If not, don’t use use this assessment.
      • Ask students to list 3 to 5 specific goals they have for the course.  Eliminate common answers such as: to do well in the course, to complete course requirements.
      • Direct students to rank goals from most to least important.  May need to model this step.
      • (Time allowing) Direct students to rank goals from most to least difficult.
      • Collect the responses and tells students what data will be used for and when you will discuss related analyses.
    • Analysis Steps
      • Categorize responses with similar goals.  What goals are most common?
      • Can tally up the number of votes that go with each type of goal.
      • Decide whether or not to incorporate student goals into course goals.
      • Compare student goals to course goals.
    • Extension Ides
      • Ask students to rank goals by amount of time needed to reach these.
      • Have students complete the assessment in teams
      • Do follow-up assessment mid-course and end-of-course.  Look for changes in student goals.  More specific? More realistic?
      • Can have students plot learning goals on an importance difficulty matrix so they can label goals as: targeted, high value, strategic and luxurious
    • Pros
      • Students become aware of how their learning goals (or lack of) connect to the course
      • Assess overlap in teacher and student goals
      • Nice conversation starter for course goals and aims
    • Cons
      • Students may struggle to articulate their own goals
      • Students are not used to critiquing course goals
      • Students may get discouraged by mismatches in student and teacher goals
    • Caveats
      • Being aware of goals takes practice
      • Don’t ask if you don’t want to know
      • Don’t hesitate to compare/contrast teacher and student goals (be transparent)
      • Make an effort to respond to student goals in some way
  2. Self-Assessment of Ways of Learning
    • Description
      • Students compare themselves to profiles to identify their preferred ways of learning
    • Purpose
      • Assess students’ preferring learning styles and modes
      • Teachers can use approaches that match students’ learning styles
      • Students realize their learning styles and can make choices that make learning more efficient for them
    • Step-by-Step Procedure
      • Select a framework for describing learning styles and preferences.
      • Determine main categories within selected framework.
      • Create profiles of learners that fit different types in the framework.
      • Design prompts that will have students identify their affinity for one (or more) of the learning profiles and explain their affinity
      • Create a 1 page handout with assessment questions and related profiles.
      • Trial the form.
      • Administer the assessment.
    • Analysis Steps
      • Tally up responses for each type of profile.
      • Show data summary to the class and lead discussion on implications of the data summary on future instruction and learning.
    • Extension Ides
      • Organize class debate teams by grouping together students with similar learning profiles.
      • Asks students to identify pros and cons of their preferred learning type
      • Ask students how someone in their style would approach a learning task
      • Organize study groups that mix up students with different learning styles
    • Pros
      • Students learn to acknowledge multiple learning styles
      • Students become more aware of preferred learning style
      • Students may experiment with different learning styles
      • Teacher can gain quick insight into learning styles without using complicated inventories
    • Cons
      • Students with multiple styles may oversimplify their preferences by having to choose one
    • Caveats
      • Treat learning styles as tendencies, not absolute prerequisites.
      • Learning styles can be context dependent
      • Allow students to make relative choices to describe their preferences

 

3-sowhat
These assessments can help teachers gather information on students’ learning goals and learning styles.  Knowing either or both of these can help teachers refine their course goals to better connect with students.  Knowing the predominant learning styles in a class can help teachers select activities and assessments that favor those learning styles.  Knowing students’ learning goals can help teachers engage students by alerting them to activities and goals that overlap with their goals.

 

4-nowwhat
Preparation Steps
  • Decide what information you would like to gather about students’ learning.  Decide how you will use that information.
  • Select the strategy that will harvest the information you want most.
  • Create a handout that goes with the selected strategy.  See above for details.
  • Share with students how you will use the information gathered by the assessment.
Early Implementation Steps
  • Administer the assessment.
  • Analyze the assessment – look for interesting patterns and comparisons.
  • Share results with the class and their impact on future instruction and learning.
Advanced Implementation Steps
  • Implement some of the extension ideas described above.
  • Use student data to refine overall course goals, lesson formats, and project products.
5-relatedstuff

171: Assessing Metacognition (1 of 4)

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 9.41.20 PM

 

  1. Focused Autobiographical Sketches
    • Description
      • Students write a 1 to 2 page autobiographical sketch about a past successful learning experience that may be relevant to a current course
    • Purpose
      • Can help teachers set realistic executives and objectives based on knowledge of students
      • Provides starting line info on how to assess learning
      • Build feelings of self efficacy associated with a course
    • Step-by-Step Procedure
      • Find an element of learning experiences to focus on that relates to course goals and objectives.
      • Can limit still further by limiting the life areas (personal, work, academic) and time periods autobiographical sketches can be drawn from.
      • Determine criteria that will be used to assess the sketches.  Make sure prompt will direct students to create sketches that relate to assessment criteria.
      • Design a prompt that ties to focus & assessment criteria of assignment.
      • Assign prompt to students.
    • Analysis Steps
      • Scan responses and read for stories that relate to course goals.  Try to notice stories and lessons that can be shared to give students advice on how to succeed in the course.
    • Extension Ides
      • Ask students to explain why they deemed their experiences to be successful.
      • After trust is established, ask students to write a Focused Autobiographic Sketch about lessons learned from personal failures.
      • Ask students to focus on story from the point of view of someone else involved.
    • Pros
      • Focused prompts give teachers info on students that’s more relevant to the course than general background statements.
      • Give info on range of past learning experiences and self awareness of students.
    • Cons
      • No simple guidelines for assessing the quality of this assessment.
      • Reading and analyzing sketches takes a lot of time.
    • Caveats
      • Students may need coaching on writing self-reflective prose before this assessment can yield good information.
      • Some students may balk at sharing their stories, even if done anonymously.
  2. Interest / Knowledge / Skills Checklists
    • Description
      • Students respond to checklists to communicate their interests and levels of related skills
    • Purpose
      • Inform teachers of students’ interest in course-related topics and students’ levels of related skills
      • Teachers can use data above to adjust syllabi
    • Step-by-Step Procedure
      • Divide a paper into two columns.  On one column list course topics.  On other column list related skills.
      • Come up with a simple form that will help you code students’ answers.  Examples:  use checklist with Likert scale response per item
      • Let students know why you are gathering data on their interests and skills.  Can keep surveys anonymous.
    • Analysis Steps
      • Can generate bar graphs that show average Likert responses per item.
      • If the surveys are submitted via Google form these bar charts can be created in Google spreadsheets
      • Share interesting data trends and features with students
    • Extension Ides
      • Ask students to explain why they are interested (not interested in) top (bottom) 3 topics
      • Use a graphic display to show overlap between student interests and course goals
    • Pros
      • Gives teachers access to data that helps with course planning
      • Teacher can compare students’ self perceived competencies with his or her own observations
      • Having students be explicit about their interests and skills and how they relate to the course will help them be more self-aware
    • Cons
      • Sizable front end investment to create checklists and analysis tools
      • Results may show that students’ interests do not align well with current course goals
    • Caveats
      • Students may lack interest in topics due to ignorance.  They may develop an interest when they learn more.
      • Assessments may be more a measure of self-confidence than prior learning.

 

3-sowhat
Getting to know students is a key step in building positive relationships with students that can promote deeper learning.  The strategies above can help teachers harvest information on students’ habits, motivations, interests and skills.  Teachers who use this data to improve their course design are more likely to create and facilitate engaging projects for students.

 

4-nowwhat
Preparation Steps
  • Determine what you would like to learn about your students as learners – what information could help you design better projects and interventions?
  • Decide which format (Focused Autobiographical Sketch or Interest/Knowledge/Skills Checklists) will more readily yield the information you want.
  • Design prompts or forms that go with the selected strategy.
  • Decide how you will analyze the data gathered by the selected strategy.
Early Implementation Steps
  • Explain why you want students to complete the selected assessment.  Explain how it will inform your teaching and course design.
  • Give students time to complete the assessment – in class or out of class.
  • Analyze patterns and trends in students’ responses.
  • Share interesting stories, patterns and trends with students and explain how these connect to course goals.
Advanced Implementation Steps
  • Let students complete assessment at the start and beginning of the course to see if habits, skills, and/or interests change.
  • Try out one of the extension ideas described above.

 

5-relatedstuff

170: Assessing Academic Behaviors & Skills (2 of 2)

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 9.50.00 PM

 

  1. Process Analysis
    • Description
      • Students record the steps they take to carry out an assignment and comment on any conclusions they draw about their approaches to the task.
    • Purpose
      • Gathers explicit detailed information on the steps students take to complete an assignment
      • Can help students pinpoint issues in how they work and help them improve on these
      • Teachers can learn what steps in assignments are giving students the most trouble
      • Good for noticing and improving habits that relate to completing routine assignments
    • Step-by-Step Procedure
      • Select an assignment that meets these criteria
        • complex enough to warrant recording steps in a Process Analysis
        • you are genuinely interested in how students work through it
        • focusing on this activity will teach students lessons that can apply in future similar assignments
      • Describe what students will record:
        • steps to complete an assignment &
        • time spent on each step
      • Give students access to example Process Analysis records to show them how to record steps.
      • Assign a task to be logged with the Process Analysis and direct students to complete the task along with the associated Process Analysis
      • Collect Process Analysis along with assignment.
    • Analysis Steps
      • Read assignments first and grade without looking at Process Analyses.
      • In Process Analyses
        • assess overall work schemes for clarity and explicitness
        • the number of steps taken
        • effectiveness of each step
        • similarities & differences among students’ analyses in number, content, and order of steps
        • identify patterns between process steps and levels of performance on assignment
    • Extension Ideas
      • To make them easier to analyze, divide class into groups and stagger when each group completes Process Analyses
      • Have students use Process Analyses to compare notes on how they worked
      • Direct students to focus on only one stage of the process in their discussions (beginning, middle or end)
      • Ask students to focus on one step they would like to add or improve in their next assignment.
    • Pros
      • Focus on process is transferable to other assignments
      • Focus on steps allows students to break down, add, improve specific steps in order to improve overall process
      • Teachers can see if students are implementing steps they are taught to take to complete assignments such as term papers
      • Can uncover productive strategies that can be shared among students
    • Cons
      • Students may resist or struggle to complete the Process Analysis
      • Teachers may find that quality of processes do not always correlate with quality of end products
      • Time consuming to analyze these.  (Note – if these are submitted via Google form, can use pivot tables, sorting, and conditional formatting to speed through analyses of these)
    • Caveats
      • Don’t make process to laborious.  Ask for an outline of steps taken, not a narrative.
      • May need to model how to document steps for students who aren’t used to this.
      • Student who could benefit most from this analysis may be the most unwilling to do it.
  2. Diagnostic Learning Logs
    • Description
      • Students maintain a log for assignments and learning activities.  After learning activities they make lists of key points learned and things that were unclear.  For assignments, they lists problems encountered or errors made and successful responses.
    • Purpose
      • Students identify their own strengths and weaknesses as learners
      • Students can diagnose and suggest and test remedies for their learning difficulties
      • Students develop skills needed to become independent learners
      • Teachers learn more about students strengths and weaknesses that can improve their coaching of students through difficulties
    • Step-by-Step Procedure
      • Explain to students that they will logging information about their academic successes and struggles and the benefits of being aware of these.  The main benefit is that the greater awareness can also lead to better ideas to improve upon processes used to overcome struggles.
      • Provide them with a simple format.  See below:
        • After a learning session:
          • List the main points you learn from the session.  Give examples where possible.
          • List points that are unclear to you.  Give examples where possible.
          • Write a few questions that relate to the unclear points.
        • Homework assignment:
          • Briefly describe the assignment.  What is it about?
          • Give 2 examples of successful responses. What made them successful?
          • Give 1-2 examples of less successful responses.  What errors or ommissions did you make?
          • The next time you confront a similar situation what could you do to increase your learning?
    • Analysis Steps
      • Can categorize assessment using
        • Don’t identify successful and unsuccessful responses
        • Identifies successes and struggles but does not diagnose them.
        • Identifies and diagnoses, but doesn’t offer solutions
        • Identifies, diagnoses and offers solutions
      • Record types of questions students are raising and types of problems they identify
      • What is range of responses students use to describe their own learning?
    • Extension Ideas
      • To make them easier to analyze, divide class into groups and stagger when each group completes the logs
      • Ask students to focus logs on the learning of specific key topics
    • Pros
      • Develop more independent, active, self-reflective learners
      • Protocol can be transferred to any academic situation
      • Provides teachers with assessable, classifiable data on students’ metacognition
      • Can provide insights that improve student learning
    • Cons
      • Requires a lot of time and effort from teachers and students
      • Attention to failures may demoralize students.  Can counteract this by teaching students about the power of struggle
    • Caveats
      • Students may need coaching to complete this well
      • Encourage students to capitalize on their strengths as well as improve upon their weaknesses

 

3-sowhat
Both the Process Analysis and the Diagnostic Learning Log make students more aware of the steps they are taking to learn.  The Process Analysis alerts students to the steps they take to complete assignments.  Noticing these steps can help students identify strengths to reinforce and gaps to fix in order to improve overall learning processes.  The Diagnostic Learning Log makes students aware of what they understand and what they don’t and encourages them to take steps to work through their academic difficulties.

 

4-nowwhat
Preparation Steps
  • Determine if there are upcoming tasks or activities that could benefit from the use of a Process Analysis or Diagnostic Learning Tool
  • Design the form that will gather the data related to the selected strategy.  See above for examples.
  • Create sample completed forms that will be used to model the selected strategy
Early Implementation Steps
  • Talk through a sample completed form in order to show students how to complete it and how to think through responses in the form.
  • Explain the purpose of gathering and analyzing the data in the strategy.
  • Analyze the data in order to identify interesting patterns and strategies.  (Note:  Gathering data through Google forms can facilitate this data analysis due to functions such as sorting, conditional formatting, averaging, and pivot tabling).
  • Share key insights that go with the analysis of the data gathered by the selective strategy.
  • Give students time in class to compare their observations with other students and to discuss possible strategies to try in the future.
Advanced Implementation Steps
  • Have students do the strategy more than once in order to see if their learning processes are improving
  • Have students set a reinforcement and an improvement goal – reinforce a strength and improve upon a weakness.
  • Try one of the extensions described above.
  • Create character learning targets that describe key learning processes.  Let students use analyses to gather evidence of progress towards these learning targets.

 

5-relatedstuff

169: Assessing Academic Behaviors & Skills

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 9.57.55 PM

 

  1. Productive Study-Time Logs
    • Description
      • Students keep simple records of how long, what time and how well they study at different times of the day.
    • Purpose
      • Students learn:
        • how much they study for a specific course
        • how well they use their study time
        • when they are most productive
        • how much time they are studying relative to their peers
      • Students can adjust choices – such as scheduling study times to occur at more productive times of day.
    • Step-by-Step Procedure
      • Decide what you most want to know and want your students to notice about their study habits.
      • Make a simple log sheet that helps students gather study data related to the things you want yourself and your students to learn.
      • Decide frequency and duration for logging entries.
      • Create sample completed sheets.
      • Share log sheets with students.  Use sample completed logs to explain how to fill out the sheets.  Describe what to include and exclude from the log sheets.
      • Tell students the purpose of the sheets and when you will share a summary of the log sheet data.
    • Analysis Steps
      • Calculate averages for quantitative data in the log sheets.
      • Look for trends that describe:
        • when most studying occurs
        • when most productive studying occurs
        • where most productive studying occurs
      • Note:  If you gather this data in Google forms, the data will pool into Google spreadsheets where pivot tables can be used to identify the trends above.
    • Extension Ides
      • Share data trends extracted from the log sheets and use them to start discussions in which students share ideas and strategies for improving study habits.
      • In statistics, the analysis of the study logs can be done as a course assignment.
    • Pros
      • Student learn more about variations in their study habits and can use these to make better decisions.
      • Summarizing log sheet data can give teachers a measure of student’s time commitment to the course during out of class time.
      • Peer comparisons of log sheet data will help students realize if they are studying too little and make better choices.
      • Teachers can use data to make adjustments to student’s work loads.
    • Cons
      • Some students may forget to complete the form as they go.
      • Students logging many hours but still doing poorly are likely to get discouraged by this assessment.
      • Data summarization can be time consuming (not if you use Google forms and pivot tables)
    • Caveats
      • Students may exaggerate work loads if they think you’re going to make adjustments in response to the data summaries.
      • Finding out summary of study time may be an unwelcome surprise.
  2. Punctuated lectures
    • Description
      • Teacher periodically pauses in a lesson and gives students time to reflect on how their most recent behavior helped or hindered their learning and to jot down insights in the form of anonymous notes to the teacher.
      • Sounds similar to Writing Breaks.
    • Purpose
      • Focuses reflections on how or whether (or not) students are processing information in activities.
      • Encourage students to become self monitoring learners.
    • Step-by-Step Procedure
      • Choose a lecture that can be chunked into smaller segments (each segment no longer than 10-20 min).   Decide on times when you will punctuate the lecture and give students time to reflect and write.
      • At time of punctuation, explain the purpose of the assessment (to reflect on one’s current learnings).
      • Give them reflection time and writing time.  Can also give them prompts such as
        • How fully where you concentrating during the lecture? Did you get distracted? If so, how did you redirect your attention?
        • What were you doing to record the information in the lecture?  How successful were you?
        • What were you doing to make connections between this new information and what you already know?
        • What did you expect to come next in the lecture and why?
      • Collect their feedback.
    • Analysis Steps
      • Observe how specific and how aware students are of their habits.
      • Share some of their successful redirection and recording strategies with the rest of the class.
    • Extension Ides
      • Ask students to save several of their reflections and analyze them for patterns and changes over time.
      • Share especially astute self analyses with the class as examples of keen self awareness.
      • Students can develop “processing plans” that describe:
        • how they will redirect their attention,
        • how they will record their learnings and questions, and
        • how they will connect old and new information.
    • Pros
      • Shows how students learn.
      • Promotes active listening and self reflection.
      • Focuses attention on self-monitoring – important part of metacognition.
    • Cons
      • Students may struggle to be aware of what they were doing and get frustrated.
      • Many students and teachers have not built up a shared vocabulary to describe learning processes.  Building up this language takes time.
    • Caveats
      • Don’t expect immediate results from this strategy.  Being self aware is hard.
      • May encounter resistance to the idea that the methods for information processing can be consciously changed by those who believe the brain is a black box.

 

3-sowhat
Productive Study Time Logs can help teachers and students learn more about students’ study habits outside of class time.  In a PBL setting, gathering this data can set up frank conversations among team members that have large disparities in the amount of time each member is contributing towards products.  Analyzing this data can also help teachers better understand how students are managing the work loads in their classes.

 

The Punctuated lectures strategy can help students become more aware of the processes students are using to focus their attention and process information during workshops.  It can also get students to try out new strategies that may improve the way they learn during workshops.

 

4-nowwhat
Preparation Steps
  • Decide what types of things you would like students to learn about their learning processes during and outside of class time.
  • Select the strategy that will help students gather the most data and observations about the processes that are most important to you.
  • Write character learning targets that relate to the specific study habits or information processing habits you would like students to practice and assess.
Early Implementation Steps
  • Create models for the selected strategy.  Use these to demonstrate to the students how to record their observations.
  • Explain what teachers and students can learn from investigating the study data.
  • Give students time to practice the strategy.  Provide them with guide sheets to gather their observations.
  • Teachers and peers analyze the data to note interesting comparisons and trends.  Use these insights to identify possible new strategies to try out to improve learning.
Advanced Implementation Steps
  • Use Google forms to consolidate productive study time logs data into a spreadsheet that can be pivot tabled to analyze the trends.
  • Set up discussion protocols that will help peers provide each other with constructive study feedback based on their observations and brainstorm adjustments.
  • Do the strategy at different times in the course that are well separated and compare the data from both times to notice what changes and stayed the same.

 

5-relatedstuff

168: Assessing Skills in Application & Performance (2 of 2)

2-what

 

Screen Shot 2016-05-08 at 10.08.50 PM

 

  1. Human Tableau or Class Modeling
    • Description
      • Students create “living scenes” to model what they know
    • Purpose
      • Students demonstrate what they know by performing it
      • Good for kinesthetic learners
    • Step-by-Step Procedure
      • Select process or image that is important to content
      • Ask yourself what could you and your students learn from reenacting the process or image that couldn’t be learned from pen and pencil assessment
      • Determine how much time students need to plan the model
      • Determine how you will assess performances
      • If you are convinced the activity is still worthwhile, prepare a direction sheet with
        • purpose of assignment
        • procedure
        • points students should address in the Human Tableau’s or Class Models
        • topic and related sub-topics to highlight in the model
        • checklist of key grading criteria
      • Divide class into teams and hand out direction sheet.  Answer questions about assignment.
    • Analysis Steps
      • Use checklist of key features to analyze the reenactments – can note what’s missing and present and quality of each item
      • Consider having performances videotaped
      • Give groups feedback on what if anything they left out and what was especially instructive and clever about their model
    • Extension Ides
      • Have students write out scripts for their models
      • Have students write about their experiences in creating, revising and learning from the Human Tableau or Class Model
    • Pros
      • Students explain, negotiate, clarify and plan while converting what they have learned into a visual or performance team product
      • Can be very entertaining and motivating to learn new material
      • Students get to know each other better while creating it
      • Kinesthetic learners may shine
    • Cons
      • Some students may not take it seriously or refuse to participate
      • No anonymity in responses
      • Time-consuming and laborious
    • Caveats
      • May need to downplay dramatic parts of modeling and emphasize the didactic (instructive) parts of the assessment
      • Students may need a lot of directions if they are not used to collaborating on products like these
  2. Paper or Project Prospectus
    • Description
      • Brief, structured first draft plan for a paper or project
    • Purpose
      • Assess student understanding of assignment logistics and their planning skills
      • Students receive early feedback on their plan prior to devoting a lot of time to it
    • Step-by-Step Procedure
      • Create an assignment sheet that helps students prepare prospectus.
        • For a project:
          • brief project description
          • project site and setting
          • major questions you hope to answer
          • products/results
          • resources needed
          • calendar of component tasks
          • biggest concern about the project
        • For a paper:
          • proposed title
          • purpose
          • major questions you hope to answer
          • work calendar
          • proposed table of contents
          • help / resources needed
          • biggest concern about the paper
        • strive for 1 page in length
      • Create ranked lists of key elements students need to succeed (most to least important) and key elements students will most likely struggle with (most to least difficult)
      • Revise your assignment sheet to reflect your priorities determined from your ranked lists – include and list more important items first
      • Make sure to add questions that will have students unpack elements that you think students will find challenging
      • Hand out prospectus assignment sheets.  Tell students not to commit a lot of work to project until after they get feedback on the prospectus sheets.
    • Analysis Steps
      • Skim responses and circle or check of things that catch your attention.
      • Note the following patterns in their responses
        • which questions received the clearest responses?
        • which question received the muddiest responses?
        • what questions or confusions came up repeatedly?
        • are there groups of students worked on similar projects and may therefore benefit from discussing and comparing their plans?
        • do you need to rethink criteria for evaluating?
        • to what degree are prospectus items related to course goals?
      • Note general suggestions you can make to the class as a whole as a result of what you learned from investigating the questions above
    • Extension Ideas
      • Have students meet in teams to present, discuss and review each other’s prospectuses before turning them in
      • Have students write prospectuses for a “dream project” that relates to the course but there won’t be time to complete
    • Pros
      • Previews students’ ideas and forewarns of their challenges and questions
      • Provides time for timely feedback on large projects
      • Feedback early in the project prevents students from missing the mark
      • Thoughtful planning requirement improves final products
    • Cons
      • Students may aim to please instructors more than learn for themselves
      • Teachers may be overly critical early in the project process
      • Takes a lot of time to plan and assess
      • Some students may need a lot of coaching to complete prospectuses
    • Caveats
      • Reserve prospectuses for major products only
      • If you want students to be creative, don’t specify every part of the assignment
      • Sell prospectus as a plan and living document that can be changed
      • A poor first draft of prospectus may not indicate bad ideas, but more lack of experience of students with planning projects
3-sowhat
The Human Tableau or Class Model can get students to review and communicate complex images or processes through teamwork and dynamic reenactments of the material.  These displays can help students think deeply about processes and create memorable displays to remember them.

 

The Project or Paper Prospectuses can help students plan early drafts of large projects and get early feedback on their ideas.  Getting feedback early in the planning process can help build student confidence in their project plan, help them better organize their plans, and help them create better products.

 

4-nowwhat
Preparation Steps
  • Decide whether there are processes, images or products in upcoming processes that could benefit from either the Human Tableau/Class Model or Project Prospectus strategies.
  • Gather / create models and guidelines for selected assessments.
Early Implementation Steps
  • Describe purpose and instructions for assessment and how students will get feedback on assessment.
  • Give students coached work time in class to start assignment.
  • Provide specific content feedback (warm and cool) on the assessment.
Advanced Implementation Steps
  • Record or collect good student samples of assessments to use as models for future implementations of these assessments.
  • Use model and critique lessons to co-develop criteria for these assessments.  See this article for ideas on how to do this.
  • Use one of the extensions ideas described above.

 

5-relatedstuff