175: Assessing Learning Activities (1 of 2)

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 9.04.05 PM

 

  1. RSQC2
    • Description
      • Students respond briefly to the prompts: recall, summarize, questions, connect, and comment.
    • Purpose
      • Teachers can compare detailed info on students’ recall and understanding of previous classes with their own
      • Informs teachers of questions and comments that need to be addressed in a timely manner
      • Provides students with a framework for recalling, summarizing and processing information.
    • Step-by-step procedure
      • Recall:
        • At the beginning of class, ask students to make a list of words/phrases of important and useful information from the previous class. (1-2 min)
        • Ask students to circle top 3 to 5 points and rank them in order from most to least important. (1-2 min)
      • Summarize:
        • Ask students to write a summary sentences that covers as many of the important topics as possible (1-2 min)
      • Question:
        • Asks students to write 1-2 unanswered questions from last class (1-2 min)
      • Connect
        • Ask students to write a sentence or two to connect some of the important topics from last class to overarching course goals (1-2 min)
      • Comment
        • Write an evaluative statement about the class.  Can offer sentence stems such as:
          • One thing I enjoyed in the last class was _________
          • What I found most/least useful was __________
          • During most of the class, I felt _____________
      • Collect the RSQC2 feedback and let students know what kind of feedback they will receive and when they get it.
    • Analysis Steps
      • Analyze one type of response – don’t do all 5 letter responses.
      • Compare their responses to your responses to the same prompts.
        • Note additions, commissions and errors.
        • Assess degree of fit – how close their key points match yours
        • Look for patterns in students’ responses
      • Share key positive things you noticed while analyzing their assessments.
      • Share tips for overcoming some of the difficulties noticed while analyzing their assessments.
    • Extension activities
      • Encourage students to share their results with each other.
      • Gather responses to create a class Recall list.
      • When students are familiar with the strategy, let them facilitate it.
      • Can assign a committee to gather and analyze these results prior to class time.
      • Use at the end of long classes.
      • Let students work in pairs or small groups to produce responses.
    • Pros 
      • Immediate feedback on what students recall, value, question, etc.
      • Highly structured way to recall, summarize and evaluate course materials and share responses with others
      • Connections step builds bridges between prior knowledge and new material
      • Using repetitively will help students review, reorganize, reconsider and integrate major points of the course
    • Cons
      • Time consuming, especially on first attempts
      • If overused or used poorly, can become a mindless activity
      • Too much data to analyze at one sitting
      • May frustrate students to not get feedback on all points
    • Caveats
      • Let students know what parts of the data you will focus on during your analysis
      • May need to model responses during first times
      • Allow for more time on first attempts
      • Don’t ask for questions or comments unless you are ready to respond to them in a thoughtful manner
      • Use time limits to encourages students to work quickly
  2. Group-Work Evaluations
    • Description
      • Students provide feedback on group learning experiences
    • Purpose
      • Teachers can learn what’s working (and not) in learning teams
      • Teachers can learn if they need to intervene on situations that are hindering learning
    • Step-by-step procedure
      • Decide what you want teachers and students to learn about group work.  Design questions around these.
      • Select top 4-5 questions to design a Group Work Evaluation Form for gathering feedback
      • Before handing out forms, explain the purpose and process to students.
    • Analysis Steps
      • Tally responses to questions.  Note: if this data is gathered in Google forms, tallies can be done quickly in Google spreadsheets.
      • Identify outlier teams who are doing very poorly and very well.
      • Intervene on struggling teams.
      • Try to learn from very successful teams and share their strategies with the class.
    • Extension activities
      • Have students come up with group evaluation questions.
      • have students brainstorm solutions to concerns raised during the assessment.
    • Pros 
      • Detect conflicts before they permanently damage team morale.
      • Students and teachers can find out what’s working well and profit from that info.
      • Students learn about common advantages and disadvantages of group work.
    • Cons
      • May raise concerns that students do not know how to resolve.
      • Students who hate working in groups may use this process to sabotage their teams
      • Some students may not like focus on process over outcomes
    • Caveats
      • When things are not going well, there is social pressure to lie.  Make it clear that you expect honest answers, especially if it’s hard to do so.
  3. Reading Rating Sheets
    • Description
      • Students rate their reading assignments
    • Purpose
      • Teachers learn how motivated students are by assigned readings
      • This data can help teachers think about how to frame these readings in lessons and how to rethink the course reading list
    • Step-by-step procedure
      • Determine your reasoning for student rating the texts.  Reasons could include:
        • to make deacons about future syllabi
        • call attention to specific parts of text
      • Write a few questions (no more than 4-5) that gather data you want to learn or serve your intended aims.
        • most – yes/no format
        • 1-2 short answer format (reasoning and explanations)
        • include at least one question that asks students how thoroughly they read the text they are rating
      • Try answering assessment yourself.  Revise as needed.
      • Create a simple Reading Response Sheet.  Ask students to complete it as soon as they finish a reading or give time in class to do it.
    • Analysis Steps
      • Tally responses for the yes/no questions for the students who read the text.  (Note:  If students submit their responses via Google form, theses tallies an be done quickly in Google spreadsheet)
      • Look for patterns in the comments.
      • Focus most on data and comments that rate the learning value of the reading.
      • Share interesting results and patterns with students.
    • Extension activities
      • Use same basic format to assess other learning aides: lectures, videos, etc.
      • Let students tally the results
      • Have students discuss their responses in small groups and brainstorm ways to get more out of the readings
    • Pros 
      • Draws attention to readings as learning aides
      • Recognizes that students can evaluate the usefulness of texts
      • Feedback on how well students read the assignments
    • Cons
      • Students who dislike reading may rate all texts low.
      • If most students don’t read texts deeply, data will be useless.
    • Caveats
      • Include question that separates feedback of students who read the text and didn’t
      • Don’t suggest you will change reading assignments unless you intend to do so

 

3-sowhat
The RSQ2C assessments is a strategy that students can use to process learning activities and that teachers can use to answer timely questions and to learn how much students are gathering from learning activities.

 

The Group-Work Evaluations can show students and teachers what is working well and not well in cooperative learning activities.  The data can also by used to facilitate timely meetings to resolve conflicts that are hindering teams from learning.

 

The Reading Rating Sheets can help teachers learn what students are gathering from texts and whether or not they think the texts are helpful.  This data can inform how readings are framed in lessons and can impact the future design of syllabi.

 

4-nowwhat
Preparation Steps
  • Decide what learning data you would like to learn about students and what you will do with that data.
  • Select the strategy that will help you learn the things you want.
  • Prepare resources that go with that strategy.  See above for details.
Early Implementation Steps
  • Model how to complete the assessment.  State the purpose of the assessment.
  • Administer the assessment.
  • Analyze the assessment data.  Share key learnings with the students.  Describe how these will affect future teaching and learning.
Advanced Implementation Steps
  • Try out some of the extension activities suggested above.
  • Use student feedback gathered from these assessments to modify lesson plans and course design.

 

5-relatedstuff

174: Assessing Metacognition (4 of 4)

2-what

 

Screen Shot 2016-05-08 at 9.12.14 PM

 

  1. Everyday Ethical Dilemmas
    • Description
      • Students respond briefly and anonymously to abbreviated case studies with moral dilemmas that relate to the course
    • Purpose
      • Students identify and clarify their own values and how these relate to moral dilemmas of the course
      • Students learn how values impact decision making
    • Step-by-Step Procedure
      • Decide on one specific ethical issue to focus on.
      • Locate a short case that illustrates the essential ethical dilemma in a few lines.
      • Write 2-3 questions that require students to take a position in the case and explain their reasoning for their decision.
      • Ask students to write short anonymous responses.
      • Allow time in class or outside class to complete the assessment.
    • Analysis Steps
      • Separate papers based on the position they took and tally responses.
      • Within each pile of similar responses, look for patterns in their rationales for their responses.
      • Share interesting insights and patterns with students and discuss implications of these for the course.
    • Extension Ides
      • After students have submitted responses, have students discuss the issues in small groups.  Then have them complete the assessment again to see if decisions and rationales changed.
      • Classify responses according to course frameworks or framework that describe ethical development.  See Perry’s schemes of intellectual and ethical development for ideas.
      • Ask students to answer cases from 2 different viewpoints.
      • Assign students task of creating Everyday Ethical Dilemmas for the class to analyze and discuss.
    • Pros
      • Students practice thinking through ethical dilemmas and get feedback on these processes.
      • Gathers data that helps teachers develop students’ ethical reasoning skills.
    • Cons
      • Students who are fixed in their opinions may see this exercise as a waste of time.
      • Teachers may lose respect for students based on their opinions.
    • Caveats
      • Start with minor dilemmas and work your way up.  This gradual buildup can build up trust, confidence and skill.
  2. Course-Related Self-Confidence Surveys
    • Description
      • Students answer questions that assess their self-confidence in relationship to course specific skills and activities.
    • Purpose
      • Knowledge of skills that students feel confidence with (and not) can help teachers design learning experiences that better support students
      • Can help students set up positive feedback loops – i.e. they can focus on skills they need which will build competence and confidence
    • Step-by-Step Procedure
      • Focus on skills or abilities that are important to success in a course
      • Make up questions to assess students’ competence or confidence in these specific skills
      • Create a simple survey form for gathering the data
      • Gather responses anonymously during class time
    • Analysis Steps
      • Tally responses to questions.  If this is done in Google forms, this tallying can be done quickly in Google spreadsheets.
    • Extension Ides
      • Have students discuss the skills in teams and brainstorm methods to build confidence or competence in each skill.
      • Ask follow-up questions to identify what classroom variables most influence confidence and how these can be changed to promote confidence
    • Pros
      • Can identify what students are confident and anxious about in terms of course-specific skills
      • Can offer relief to students to see that others struggle in areas where they struggle
      • Teaches students that self-confidence helps with learning
    • Cons
      • Some students are overconfident about their skills and therefore are harder to teach
      • Discovery of low self esteem may hurt teacher and student morale
    • Caveats
      • Teach students to associate confidence with performance so that their confidence can increase as they start to demonstrate more skill
      • Be aware that some serious problems with self esteem won’t be remedied by the course
3-sowhat
The Everyday Ethical Dilemmas assessments can reveal how students reason through ethical dilemmas that relate to the course.  Making students aware of these controversial cases can help students see the value in content-specific information and skills that can be applied to analyze and evaluate the case.

 

The Course-Related Self Confidence Surveys can help students and teachers be aware of what skills student’s feel are strengths and what skills they may struggle with.  Knowing this profile for the students in the course can help teachers reframe projects and project scaffolding to leverage student strengths and to improve upon student gaps.

 

4-nowwhat
Preparation Steps
  • Decide what you like to learn about students’ skills.
  • Decide what strategy will yield the knowledge about students’ skills you want.
  • Develop resources related the selected strategy.  See above for details.
Early Implementation Steps
  • .Explain the purpose for the assessment.
  • Give students time in class to complete the assessment.
  • Analyze the assessment.
  • Share key results from analyzing the assessment with students and facilitate a related discussion with students.
  • Describe how key results will impact future teaching and learning in the course.
Advanced Implementation Steps
  • Try out the enrichment activities described above.
  • Use knowledge gained from the assessments to refine related project scaffolding so that it better meets students’ needs.
5-relatedstuff

173: Assessing Metacognition (3 of 4)

  1. Classroom Opinion Polls
    • Description
      • Students complete anonymous polls of their opinions
    • Purpose
      • Discover student opinions related to course-related issues
      • Helps students become aware of opinions that can support or hinder learning
      • Teachers can get advance information that can help him or her better frame course-related issues
    • Step-by-Step Procedure
      • Preview upcoming materials and search for issues that may have related opinions that can affect learning.
      • Choose 1-2 issues to convert to Classroom Opinion Poll questions.
      • Create a polling electronic form or paper form.  Tools for electronic forms: Socrative, Nearpod, PollEveryhwere, Mentimeter
      • Administer the assessment.  Remind students not to put their names on the paper forms if you use these.
    • Analysis Steps
      • Count and tally responses.
      • Electronic tools (see above) will count and tally responses automatically.
    • Extension Ides
      • Ask students to justify their responses.
      • Have students write rebuttals for opinions different than their own.
      • Let students come up with polling questions.
    • Pros
      • Low floor task that is accessible to all
      • Showing poll results helps students acknowledge the diversity of opinions in the class
      • Teachers gain info that can help refine/reframe future lessons
    • Cons
      • May find that students are opposed to using evidence to justify opinions
      • Students may find it difficult to commit to a response
    • Caveats
      • Be prepared to discuss issues related to opinion polls
      • Gather assessment data anonymously
      • Model respectful consideration of varied opinions
      • May opt to summarize data in private in order to hide reactions to unexpected surprises in the data
  2. Double Entry Journals
    • Description
      • Students take notes in 2 columns.  On one column that write key ideas, assertions and arguments learned during activity.  In the other column they explain the personal significance of those ideas and ask questions of the ideas.
    • Purpose
      • Detailed feedback on how students read, analyze and respond to assigned texts
      • Teachers learn what students value in the text
      • Students learn about how course issues relates to issues in their personal lives
      • Students become more aware of how they read and respond to texts
    • Step-by-Step Procedure
      • Select important parts of text to be annotated.  Should be short, challenging and provocative.
      • Show students how to setup notebook page for the strategy.
      • Model how to use the strategy by getting the annotations started on both columns while thinking aloud.
        • on left column – copy key quotes and passages from the text
        • on right column – explain why they chose each excerpt – could include agreements, disagreements, questions etc
        • encourage 2 columns to interact like a dialogue
    • Analysis Steps
      • Compare students’ left column to key points in the text
      • Try to categorize the types of responses that appear in the right column
    • Extension Ides
      • Let students do a double entry journal on a lecture – provide recording of the lecture
      • May offer 2-3 excerpts for them to interpret and have them add a couple more items in the right column
      • Use as first step in a bigger writing assignment – write an essay based on double entry journal
      • After they’ve done this several times, get students to look over several double entry journals and notice patterns in the types of excerpts they choose to include
    • Pros
      • Student practice careful reading and responding to texts
      • Encourages students to look for personal meanings in texts
    • Cons
      • Students may try to write for the teacher instead of candidly write for themselves
      • Students may struggle with this strategy at first
    • Caveats
      • Importance to distinguish between lack of skill with this note-taking strategy and lack of awareness
      • Students may need a lot of coaching before students start genuinely sharing their opinions
  3. Profiles of Admirable Individuals
    • Description
      • Students write a brief focused profile of an individual in a field related to the course whose values, skills and/or actions they admire
    • Purpose
      • Teachers learn what students value
      • Students acknowledge what they value in professions related to the course
    • Step-by-Step Procedure
      • Try out strategy yourself – write a profile for an admirable individual for a course.
      • Decide what criteria you will use to assess the profiles.
      • Develop clear instructions for profiles.  Be sure to include length limits and to define what populations students can draw their people from
    • Analysis Steps
      • Notice the characteristics and values that appear repeatedly in students’ profiles.  Tally these.
    • Extension Ides
      • Provide a list of people who students can write profiles for in case they can’t come up with a person.
      • Have students rank the characteristics in their profiles in order from most to least important.
      • Have students write parallel profiles of people who are not admirable and explain their negative appraisals.
      • Have students work in teams to read each other’s appraisals and come up with a composite list of admirable characteristics.
    • Pros
      • Requires students to consider their own values.
      • Teachers can learn about role models who have influenced students.
    • Cons
      • May make some students uncomfortable.
      • May need to do research to identify people for profiles.
      • Specifying admirable characteristics of individuals can be challenging.
    • Caveats
      • Keep assessments anonymous
      • Model how to write good profiles that elaborate on individual’s admirable qualities
      • If students chose someone you’ve never heard of, remembers that identifying the admirable qualities is more important than knowing the role model

 

3-sowhat
All 3 of these strategies have students notice and share personal connections they are making with course.  Making personal connections with course materials is one powerful way to motivate students and to help them remember new content.

 

Classroom opinion polls can help teachers identify student opinions that may help or hinder learning in the course.  The double entry journal can help teachers see what students value in assigned texts and why they value these.  The profiles of admirable individuals can help teachers see what positive characteristics students see and value in professionals who relate to the course.

 

4-nowwhat
Preparation Steps
  • Identify what you would like to learn about students’ beliefs, ideas, etc.
  • Identify the strategy that would most readily access the information you’d like to learn about students.
  • Create resources that go with the selected strategy.  See above for details.
Early Implementation Steps
  • Describe how to complete the assessment.  Provide early coaching in the from of modeling if needed.
  • Give students time to complete the assessment inside or outside of class.
  • Analyze the assessment to look for interesting patterns and comparisons.  Compare these with course character learning targets
  • Share insights from analyzing the assessment with the class.  Facilitate discussions related to these insights.
  • Describe to students how these insights will impact future teaching and learning in the course.
Advanced Implementation Steps
  • Try out some of the extension activities.  See above.
  • Use knowledge of students to reframe upcoming projects and scaffolding activities to more closely engage with students’ interests and values.

 

5-relatedstuff

172: Assessing Metacognition (2 of 4)

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 9.33.21 PM

 

  1. Goal Ranking and Matching
    • Description
      • Students list a few learning goals they have for the course and rank their importance and difficulty.
    • Purpose
      • Measures degree of fit between student and teacher goals
      • Can be used to create shared class goals
      • Students identify and clarify their learning goals
      • Students connect their learning goals with course goals
    • Step-by-Step Procedure
      • Clarify instructional goals for the course.  This can be down by writing long term character learning targets for the course.
      • Assess negotiability and flexibility of character learning targets – are you willing to change these in response to knowing students’ goals?  If not, don’t use use this assessment.
      • Ask students to list 3 to 5 specific goals they have for the course.  Eliminate common answers such as: to do well in the course, to complete course requirements.
      • Direct students to rank goals from most to least important.  May need to model this step.
      • (Time allowing) Direct students to rank goals from most to least difficult.
      • Collect the responses and tells students what data will be used for and when you will discuss related analyses.
    • Analysis Steps
      • Categorize responses with similar goals.  What goals are most common?
      • Can tally up the number of votes that go with each type of goal.
      • Decide whether or not to incorporate student goals into course goals.
      • Compare student goals to course goals.
    • Extension Ides
      • Ask students to rank goals by amount of time needed to reach these.
      • Have students complete the assessment in teams
      • Do follow-up assessment mid-course and end-of-course.  Look for changes in student goals.  More specific? More realistic?
      • Can have students plot learning goals on an importance difficulty matrix so they can label goals as: targeted, high value, strategic and luxurious
    • Pros
      • Students become aware of how their learning goals (or lack of) connect to the course
      • Assess overlap in teacher and student goals
      • Nice conversation starter for course goals and aims
    • Cons
      • Students may struggle to articulate their own goals
      • Students are not used to critiquing course goals
      • Students may get discouraged by mismatches in student and teacher goals
    • Caveats
      • Being aware of goals takes practice
      • Don’t ask if you don’t want to know
      • Don’t hesitate to compare/contrast teacher and student goals (be transparent)
      • Make an effort to respond to student goals in some way
  2. Self-Assessment of Ways of Learning
    • Description
      • Students compare themselves to profiles to identify their preferred ways of learning
    • Purpose
      • Assess students’ preferring learning styles and modes
      • Teachers can use approaches that match students’ learning styles
      • Students realize their learning styles and can make choices that make learning more efficient for them
    • Step-by-Step Procedure
      • Select a framework for describing learning styles and preferences.
      • Determine main categories within selected framework.
      • Create profiles of learners that fit different types in the framework.
      • Design prompts that will have students identify their affinity for one (or more) of the learning profiles and explain their affinity
      • Create a 1 page handout with assessment questions and related profiles.
      • Trial the form.
      • Administer the assessment.
    • Analysis Steps
      • Tally up responses for each type of profile.
      • Show data summary to the class and lead discussion on implications of the data summary on future instruction and learning.
    • Extension Ides
      • Organize class debate teams by grouping together students with similar learning profiles.
      • Asks students to identify pros and cons of their preferred learning type
      • Ask students how someone in their style would approach a learning task
      • Organize study groups that mix up students with different learning styles
    • Pros
      • Students learn to acknowledge multiple learning styles
      • Students become more aware of preferred learning style
      • Students may experiment with different learning styles
      • Teacher can gain quick insight into learning styles without using complicated inventories
    • Cons
      • Students with multiple styles may oversimplify their preferences by having to choose one
    • Caveats
      • Treat learning styles as tendencies, not absolute prerequisites.
      • Learning styles can be context dependent
      • Allow students to make relative choices to describe their preferences

 

3-sowhat
These assessments can help teachers gather information on students’ learning goals and learning styles.  Knowing either or both of these can help teachers refine their course goals to better connect with students.  Knowing the predominant learning styles in a class can help teachers select activities and assessments that favor those learning styles.  Knowing students’ learning goals can help teachers engage students by alerting them to activities and goals that overlap with their goals.

 

4-nowwhat
Preparation Steps
  • Decide what information you would like to gather about students’ learning.  Decide how you will use that information.
  • Select the strategy that will harvest the information you want most.
  • Create a handout that goes with the selected strategy.  See above for details.
  • Share with students how you will use the information gathered by the assessment.
Early Implementation Steps
  • Administer the assessment.
  • Analyze the assessment – look for interesting patterns and comparisons.
  • Share results with the class and their impact on future instruction and learning.
Advanced Implementation Steps
  • Implement some of the extension ideas described above.
  • Use student data to refine overall course goals, lesson formats, and project products.
5-relatedstuff

171: Assessing Metacognition (1 of 4)

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 9.41.20 PM

 

  1. Focused Autobiographical Sketches
    • Description
      • Students write a 1 to 2 page autobiographical sketch about a past successful learning experience that may be relevant to a current course
    • Purpose
      • Can help teachers set realistic executives and objectives based on knowledge of students
      • Provides starting line info on how to assess learning
      • Build feelings of self efficacy associated with a course
    • Step-by-Step Procedure
      • Find an element of learning experiences to focus on that relates to course goals and objectives.
      • Can limit still further by limiting the life areas (personal, work, academic) and time periods autobiographical sketches can be drawn from.
      • Determine criteria that will be used to assess the sketches.  Make sure prompt will direct students to create sketches that relate to assessment criteria.
      • Design a prompt that ties to focus & assessment criteria of assignment.
      • Assign prompt to students.
    • Analysis Steps
      • Scan responses and read for stories that relate to course goals.  Try to notice stories and lessons that can be shared to give students advice on how to succeed in the course.
    • Extension Ides
      • Ask students to explain why they deemed their experiences to be successful.
      • After trust is established, ask students to write a Focused Autobiographic Sketch about lessons learned from personal failures.
      • Ask students to focus on story from the point of view of someone else involved.
    • Pros
      • Focused prompts give teachers info on students that’s more relevant to the course than general background statements.
      • Give info on range of past learning experiences and self awareness of students.
    • Cons
      • No simple guidelines for assessing the quality of this assessment.
      • Reading and analyzing sketches takes a lot of time.
    • Caveats
      • Students may need coaching on writing self-reflective prose before this assessment can yield good information.
      • Some students may balk at sharing their stories, even if done anonymously.
  2. Interest / Knowledge / Skills Checklists
    • Description
      • Students respond to checklists to communicate their interests and levels of related skills
    • Purpose
      • Inform teachers of students’ interest in course-related topics and students’ levels of related skills
      • Teachers can use data above to adjust syllabi
    • Step-by-Step Procedure
      • Divide a paper into two columns.  On one column list course topics.  On other column list related skills.
      • Come up with a simple form that will help you code students’ answers.  Examples:  use checklist with Likert scale response per item
      • Let students know why you are gathering data on their interests and skills.  Can keep surveys anonymous.
    • Analysis Steps
      • Can generate bar graphs that show average Likert responses per item.
      • If the surveys are submitted via Google form these bar charts can be created in Google spreadsheets
      • Share interesting data trends and features with students
    • Extension Ides
      • Ask students to explain why they are interested (not interested in) top (bottom) 3 topics
      • Use a graphic display to show overlap between student interests and course goals
    • Pros
      • Gives teachers access to data that helps with course planning
      • Teacher can compare students’ self perceived competencies with his or her own observations
      • Having students be explicit about their interests and skills and how they relate to the course will help them be more self-aware
    • Cons
      • Sizable front end investment to create checklists and analysis tools
      • Results may show that students’ interests do not align well with current course goals
    • Caveats
      • Students may lack interest in topics due to ignorance.  They may develop an interest when they learn more.
      • Assessments may be more a measure of self-confidence than prior learning.

 

3-sowhat
Getting to know students is a key step in building positive relationships with students that can promote deeper learning.  The strategies above can help teachers harvest information on students’ habits, motivations, interests and skills.  Teachers who use this data to improve their course design are more likely to create and facilitate engaging projects for students.

 

4-nowwhat
Preparation Steps
  • Determine what you would like to learn about your students as learners – what information could help you design better projects and interventions?
  • Decide which format (Focused Autobiographical Sketch or Interest/Knowledge/Skills Checklists) will more readily yield the information you want.
  • Design prompts or forms that go with the selected strategy.
  • Decide how you will analyze the data gathered by the selected strategy.
Early Implementation Steps
  • Explain why you want students to complete the selected assessment.  Explain how it will inform your teaching and course design.
  • Give students time to complete the assessment – in class or out of class.
  • Analyze patterns and trends in students’ responses.
  • Share interesting stories, patterns and trends with students and explain how these connect to course goals.
Advanced Implementation Steps
  • Let students complete assessment at the start and beginning of the course to see if habits, skills, and/or interests change.
  • Try out one of the extension ideas described above.

 

5-relatedstuff

170: Assessing Academic Behaviors & Skills (2 of 2)

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 9.50.00 PM

 

  1. Process Analysis
    • Description
      • Students record the steps they take to carry out an assignment and comment on any conclusions they draw about their approaches to the task.
    • Purpose
      • Gathers explicit detailed information on the steps students take to complete an assignment
      • Can help students pinpoint issues in how they work and help them improve on these
      • Teachers can learn what steps in assignments are giving students the most trouble
      • Good for noticing and improving habits that relate to completing routine assignments
    • Step-by-Step Procedure
      • Select an assignment that meets these criteria
        • complex enough to warrant recording steps in a Process Analysis
        • you are genuinely interested in how students work through it
        • focusing on this activity will teach students lessons that can apply in future similar assignments
      • Describe what students will record:
        • steps to complete an assignment &
        • time spent on each step
      • Give students access to example Process Analysis records to show them how to record steps.
      • Assign a task to be logged with the Process Analysis and direct students to complete the task along with the associated Process Analysis
      • Collect Process Analysis along with assignment.
    • Analysis Steps
      • Read assignments first and grade without looking at Process Analyses.
      • In Process Analyses
        • assess overall work schemes for clarity and explicitness
        • the number of steps taken
        • effectiveness of each step
        • similarities & differences among students’ analyses in number, content, and order of steps
        • identify patterns between process steps and levels of performance on assignment
    • Extension Ideas
      • To make them easier to analyze, divide class into groups and stagger when each group completes Process Analyses
      • Have students use Process Analyses to compare notes on how they worked
      • Direct students to focus on only one stage of the process in their discussions (beginning, middle or end)
      • Ask students to focus on one step they would like to add or improve in their next assignment.
    • Pros
      • Focus on process is transferable to other assignments
      • Focus on steps allows students to break down, add, improve specific steps in order to improve overall process
      • Teachers can see if students are implementing steps they are taught to take to complete assignments such as term papers
      • Can uncover productive strategies that can be shared among students
    • Cons
      • Students may resist or struggle to complete the Process Analysis
      • Teachers may find that quality of processes do not always correlate with quality of end products
      • Time consuming to analyze these.  (Note – if these are submitted via Google form, can use pivot tables, sorting, and conditional formatting to speed through analyses of these)
    • Caveats
      • Don’t make process to laborious.  Ask for an outline of steps taken, not a narrative.
      • May need to model how to document steps for students who aren’t used to this.
      • Student who could benefit most from this analysis may be the most unwilling to do it.
  2. Diagnostic Learning Logs
    • Description
      • Students maintain a log for assignments and learning activities.  After learning activities they make lists of key points learned and things that were unclear.  For assignments, they lists problems encountered or errors made and successful responses.
    • Purpose
      • Students identify their own strengths and weaknesses as learners
      • Students can diagnose and suggest and test remedies for their learning difficulties
      • Students develop skills needed to become independent learners
      • Teachers learn more about students strengths and weaknesses that can improve their coaching of students through difficulties
    • Step-by-Step Procedure
      • Explain to students that they will logging information about their academic successes and struggles and the benefits of being aware of these.  The main benefit is that the greater awareness can also lead to better ideas to improve upon processes used to overcome struggles.
      • Provide them with a simple format.  See below:
        • After a learning session:
          • List the main points you learn from the session.  Give examples where possible.
          • List points that are unclear to you.  Give examples where possible.
          • Write a few questions that relate to the unclear points.
        • Homework assignment:
          • Briefly describe the assignment.  What is it about?
          • Give 2 examples of successful responses. What made them successful?
          • Give 1-2 examples of less successful responses.  What errors or ommissions did you make?
          • The next time you confront a similar situation what could you do to increase your learning?
    • Analysis Steps
      • Can categorize assessment using
        • Don’t identify successful and unsuccessful responses
        • Identifies successes and struggles but does not diagnose them.
        • Identifies and diagnoses, but doesn’t offer solutions
        • Identifies, diagnoses and offers solutions
      • Record types of questions students are raising and types of problems they identify
      • What is range of responses students use to describe their own learning?
    • Extension Ideas
      • To make them easier to analyze, divide class into groups and stagger when each group completes the logs
      • Ask students to focus logs on the learning of specific key topics
    • Pros
      • Develop more independent, active, self-reflective learners
      • Protocol can be transferred to any academic situation
      • Provides teachers with assessable, classifiable data on students’ metacognition
      • Can provide insights that improve student learning
    • Cons
      • Requires a lot of time and effort from teachers and students
      • Attention to failures may demoralize students.  Can counteract this by teaching students about the power of struggle
    • Caveats
      • Students may need coaching to complete this well
      • Encourage students to capitalize on their strengths as well as improve upon their weaknesses

 

3-sowhat
Both the Process Analysis and the Diagnostic Learning Log make students more aware of the steps they are taking to learn.  The Process Analysis alerts students to the steps they take to complete assignments.  Noticing these steps can help students identify strengths to reinforce and gaps to fix in order to improve overall learning processes.  The Diagnostic Learning Log makes students aware of what they understand and what they don’t and encourages them to take steps to work through their academic difficulties.

 

4-nowwhat
Preparation Steps
  • Determine if there are upcoming tasks or activities that could benefit from the use of a Process Analysis or Diagnostic Learning Tool
  • Design the form that will gather the data related to the selected strategy.  See above for examples.
  • Create sample completed forms that will be used to model the selected strategy
Early Implementation Steps
  • Talk through a sample completed form in order to show students how to complete it and how to think through responses in the form.
  • Explain the purpose of gathering and analyzing the data in the strategy.
  • Analyze the data in order to identify interesting patterns and strategies.  (Note:  Gathering data through Google forms can facilitate this data analysis due to functions such as sorting, conditional formatting, averaging, and pivot tabling).
  • Share key insights that go with the analysis of the data gathered by the selective strategy.
  • Give students time in class to compare their observations with other students and to discuss possible strategies to try in the future.
Advanced Implementation Steps
  • Have students do the strategy more than once in order to see if their learning processes are improving
  • Have students set a reinforcement and an improvement goal – reinforce a strength and improve upon a weakness.
  • Try one of the extensions described above.
  • Create character learning targets that describe key learning processes.  Let students use analyses to gather evidence of progress towards these learning targets.

 

5-relatedstuff

169: Assessing Academic Behaviors & Skills

1-sources

 

2-what

 

Screen Shot 2016-05-08 at 9.57.55 PM

 

  1. Productive Study-Time Logs
    • Description
      • Students keep simple records of how long, what time and how well they study at different times of the day.
    • Purpose
      • Students learn:
        • how much they study for a specific course
        • how well they use their study time
        • when they are most productive
        • how much time they are studying relative to their peers
      • Students can adjust choices – such as scheduling study times to occur at more productive times of day.
    • Step-by-Step Procedure
      • Decide what you most want to know and want your students to notice about their study habits.
      • Make a simple log sheet that helps students gather study data related to the things you want yourself and your students to learn.
      • Decide frequency and duration for logging entries.
      • Create sample completed sheets.
      • Share log sheets with students.  Use sample completed logs to explain how to fill out the sheets.  Describe what to include and exclude from the log sheets.
      • Tell students the purpose of the sheets and when you will share a summary of the log sheet data.
    • Analysis Steps
      • Calculate averages for quantitative data in the log sheets.
      • Look for trends that describe:
        • when most studying occurs
        • when most productive studying occurs
        • where most productive studying occurs
      • Note:  If you gather this data in Google forms, the data will pool into Google spreadsheets where pivot tables can be used to identify the trends above.
    • Extension Ides
      • Share data trends extracted from the log sheets and use them to start discussions in which students share ideas and strategies for improving study habits.
      • In statistics, the analysis of the study logs can be done as a course assignment.
    • Pros
      • Student learn more about variations in their study habits and can use these to make better decisions.
      • Summarizing log sheet data can give teachers a measure of student’s time commitment to the course during out of class time.
      • Peer comparisons of log sheet data will help students realize if they are studying too little and make better choices.
      • Teachers can use data to make adjustments to student’s work loads.
    • Cons
      • Some students may forget to complete the form as they go.
      • Students logging many hours but still doing poorly are likely to get discouraged by this assessment.
      • Data summarization can be time consuming (not if you use Google forms and pivot tables)
    • Caveats
      • Students may exaggerate work loads if they think you’re going to make adjustments in response to the data summaries.
      • Finding out summary of study time may be an unwelcome surprise.
  2. Punctuated lectures
    • Description
      • Teacher periodically pauses in a lesson and gives students time to reflect on how their most recent behavior helped or hindered their learning and to jot down insights in the form of anonymous notes to the teacher.
      • Sounds similar to Writing Breaks.
    • Purpose
      • Focuses reflections on how or whether (or not) students are processing information in activities.
      • Encourage students to become self monitoring learners.
    • Step-by-Step Procedure
      • Choose a lecture that can be chunked into smaller segments (each segment no longer than 10-20 min).   Decide on times when you will punctuate the lecture and give students time to reflect and write.
      • At time of punctuation, explain the purpose of the assessment (to reflect on one’s current learnings).
      • Give them reflection time and writing time.  Can also give them prompts such as
        • How fully where you concentrating during the lecture? Did you get distracted? If so, how did you redirect your attention?
        • What were you doing to record the information in the lecture?  How successful were you?
        • What were you doing to make connections between this new information and what you already know?
        • What did you expect to come next in the lecture and why?
      • Collect their feedback.
    • Analysis Steps
      • Observe how specific and how aware students are of their habits.
      • Share some of their successful redirection and recording strategies with the rest of the class.
    • Extension Ides
      • Ask students to save several of their reflections and analyze them for patterns and changes over time.
      • Share especially astute self analyses with the class as examples of keen self awareness.
      • Students can develop “processing plans” that describe:
        • how they will redirect their attention,
        • how they will record their learnings and questions, and
        • how they will connect old and new information.
    • Pros
      • Shows how students learn.
      • Promotes active listening and self reflection.
      • Focuses attention on self-monitoring – important part of metacognition.
    • Cons
      • Students may struggle to be aware of what they were doing and get frustrated.
      • Many students and teachers have not built up a shared vocabulary to describe learning processes.  Building up this language takes time.
    • Caveats
      • Don’t expect immediate results from this strategy.  Being self aware is hard.
      • May encounter resistance to the idea that the methods for information processing can be consciously changed by those who believe the brain is a black box.

 

3-sowhat
Productive Study Time Logs can help teachers and students learn more about students’ study habits outside of class time.  In a PBL setting, gathering this data can set up frank conversations among team members that have large disparities in the amount of time each member is contributing towards products.  Analyzing this data can also help teachers better understand how students are managing the work loads in their classes.

 

The Punctuated lectures strategy can help students become more aware of the processes students are using to focus their attention and process information during workshops.  It can also get students to try out new strategies that may improve the way they learn during workshops.

 

4-nowwhat
Preparation Steps
  • Decide what types of things you would like students to learn about their learning processes during and outside of class time.
  • Select the strategy that will help students gather the most data and observations about the processes that are most important to you.
  • Write character learning targets that relate to the specific study habits or information processing habits you would like students to practice and assess.
Early Implementation Steps
  • Create models for the selected strategy.  Use these to demonstrate to the students how to record their observations.
  • Explain what teachers and students can learn from investigating the study data.
  • Give students time to practice the strategy.  Provide them with guide sheets to gather their observations.
  • Teachers and peers analyze the data to note interesting comparisons and trends.  Use these insights to identify possible new strategies to try out to improve learning.
Advanced Implementation Steps
  • Use Google forms to consolidate productive study time logs data into a spreadsheet that can be pivot tabled to analyze the trends.
  • Set up discussion protocols that will help peers provide each other with constructive study feedback based on their observations and brainstorm adjustments.
  • Do the strategy at different times in the course that are well separated and compare the data from both times to notice what changes and stayed the same.

 

5-relatedstuff

168: Assessing Skills in Application & Performance (2 of 2)

2-what

 

Screen Shot 2016-05-08 at 10.08.50 PM

 

  1. Human Tableau or Class Modeling
    • Description
      • Students create “living scenes” to model what they know
    • Purpose
      • Students demonstrate what they know by performing it
      • Good for kinesthetic learners
    • Step-by-Step Procedure
      • Select process or image that is important to content
      • Ask yourself what could you and your students learn from reenacting the process or image that couldn’t be learned from pen and pencil assessment
      • Determine how much time students need to plan the model
      • Determine how you will assess performances
      • If you are convinced the activity is still worthwhile, prepare a direction sheet with
        • purpose of assignment
        • procedure
        • points students should address in the Human Tableau’s or Class Models
        • topic and related sub-topics to highlight in the model
        • checklist of key grading criteria
      • Divide class into teams and hand out direction sheet.  Answer questions about assignment.
    • Analysis Steps
      • Use checklist of key features to analyze the reenactments – can note what’s missing and present and quality of each item
      • Consider having performances videotaped
      • Give groups feedback on what if anything they left out and what was especially instructive and clever about their model
    • Extension Ides
      • Have students write out scripts for their models
      • Have students write about their experiences in creating, revising and learning from the Human Tableau or Class Model
    • Pros
      • Students explain, negotiate, clarify and plan while converting what they have learned into a visual or performance team product
      • Can be very entertaining and motivating to learn new material
      • Students get to know each other better while creating it
      • Kinesthetic learners may shine
    • Cons
      • Some students may not take it seriously or refuse to participate
      • No anonymity in responses
      • Time-consuming and laborious
    • Caveats
      • May need to downplay dramatic parts of modeling and emphasize the didactic (instructive) parts of the assessment
      • Students may need a lot of directions if they are not used to collaborating on products like these
  2. Paper or Project Prospectus
    • Description
      • Brief, structured first draft plan for a paper or project
    • Purpose
      • Assess student understanding of assignment logistics and their planning skills
      • Students receive early feedback on their plan prior to devoting a lot of time to it
    • Step-by-Step Procedure
      • Create an assignment sheet that helps students prepare prospectus.
        • For a project:
          • brief project description
          • project site and setting
          • major questions you hope to answer
          • products/results
          • resources needed
          • calendar of component tasks
          • biggest concern about the project
        • For a paper:
          • proposed title
          • purpose
          • major questions you hope to answer
          • work calendar
          • proposed table of contents
          • help / resources needed
          • biggest concern about the paper
        • strive for 1 page in length
      • Create ranked lists of key elements students need to succeed (most to least important) and key elements students will most likely struggle with (most to least difficult)
      • Revise your assignment sheet to reflect your priorities determined from your ranked lists – include and list more important items first
      • Make sure to add questions that will have students unpack elements that you think students will find challenging
      • Hand out prospectus assignment sheets.  Tell students not to commit a lot of work to project until after they get feedback on the prospectus sheets.
    • Analysis Steps
      • Skim responses and circle or check of things that catch your attention.
      • Note the following patterns in their responses
        • which questions received the clearest responses?
        • which question received the muddiest responses?
        • what questions or confusions came up repeatedly?
        • are there groups of students worked on similar projects and may therefore benefit from discussing and comparing their plans?
        • do you need to rethink criteria for evaluating?
        • to what degree are prospectus items related to course goals?
      • Note general suggestions you can make to the class as a whole as a result of what you learned from investigating the questions above
    • Extension Ideas
      • Have students meet in teams to present, discuss and review each other’s prospectuses before turning them in
      • Have students write prospectuses for a “dream project” that relates to the course but there won’t be time to complete
    • Pros
      • Previews students’ ideas and forewarns of their challenges and questions
      • Provides time for timely feedback on large projects
      • Feedback early in the project prevents students from missing the mark
      • Thoughtful planning requirement improves final products
    • Cons
      • Students may aim to please instructors more than learn for themselves
      • Teachers may be overly critical early in the project process
      • Takes a lot of time to plan and assess
      • Some students may need a lot of coaching to complete prospectuses
    • Caveats
      • Reserve prospectuses for major products only
      • If you want students to be creative, don’t specify every part of the assignment
      • Sell prospectus as a plan and living document that can be changed
      • A poor first draft of prospectus may not indicate bad ideas, but more lack of experience of students with planning projects
3-sowhat
The Human Tableau or Class Model can get students to review and communicate complex images or processes through teamwork and dynamic reenactments of the material.  These displays can help students think deeply about processes and create memorable displays to remember them.

 

The Project or Paper Prospectuses can help students plan early drafts of large projects and get early feedback on their ideas.  Getting feedback early in the planning process can help build student confidence in their project plan, help them better organize their plans, and help them create better products.

 

4-nowwhat
Preparation Steps
  • Decide whether there are processes, images or products in upcoming processes that could benefit from either the Human Tableau/Class Model or Project Prospectus strategies.
  • Gather / create models and guidelines for selected assessments.
Early Implementation Steps
  • Describe purpose and instructions for assessment and how students will get feedback on assessment.
  • Give students coached work time in class to start assignment.
  • Provide specific content feedback (warm and cool) on the assessment.
Advanced Implementation Steps
  • Record or collect good student samples of assessments to use as models for future implementations of these assessments.
  • Use model and critique lessons to co-develop criteria for these assessments.  See this article for ideas on how to do this.
  • Use one of the extensions ideas described above.

 

5-relatedstuff

167: Assessing Skill in Application & Performance (1 of 2)

2-what
Screen Shot 2016-05-08 at 10.30.46 PM
  1. Directed Paraphrasing
    • Description
      • Students paraphrase a part of a lesson for a specific audience
    • Purpose
      • Assess student ability to explain concepts in their own words
      • Assess how well students have internalized content
    • Step-by-Step Procedure
      • Select an important concept, topic, theory, etc that has been covered in class and that has outside classroom implications.
      • Determine a hypothetical realistic audience for a summary of the topic.
      • Try out directed paraphrasing for selected topic and audience.
      • Assign directed paraphrasing.  Be sure to explain;
        • the topic
        • the audience
        • purpose of the summary
        • length and time limits
    • Analysis Steps
      • Divide up responses into 4 piles: confused, minimal, adequate, excellent.
      • Determine common characteristics of the 4 piles in regards to
        • accuracy of the paraphrase
        • suitability for intended audience
        • effectiveness in fulfilling its purpose
      • Circle the clearest and muddiest points in each paraphrase
    • Extension Ideas
      • Direct students to create directed paraphrases on the same topic for 2 different audiences with associated different purposes.
      • Ask students to keep a journal of directed paraphrases to summarize important topics in the course
      • Jigsaw readings and have students develop paraphrases to summarize these and then share them with students who analyzed different texts.
      • Get an appropriate expert that represents the audience to give feedback on the directed paraphrases.
      • Provide handouts with successful examples.
      • Give students warm and cool feedback on their responses.
    • Pros
      • Builds up students ability to comprehend and communicate content
      • Teachers can assess quickly if students understand content and make adjustments to future lessons
      • Emphasizes the relevance of specific concepts
    • Cons
      • Without strict length limits, these can be time consuming to create
      • Hard to establish good qualitative criteria for these
      • Paraphrasing skills can not improve without focused descriptive feedback.
    • Caveats
      • Select appropriate audiences in order to enhance the relevance of the assignment
      • First attempts may not include adjustments for selected audience
      • Use more than once so that teachers and students can learn from the process
  2. Applications Cards
    • Description
      • On an index card or small slip of paper, students provide a real world application for a selected topic
    • Purpose
      • Tie new knowledge (selected topic) with prior knowledge to create possible applications
      • Assess how well students understand and can apply content
    • Step-by-Step Procedure
      • Identify an important topic just studied in a course that has real world applications
      • Decide how many examples (1 to 3) and how much time (3-5 min) you will give students for this task
      • Assign task and hand out cards.  Emphasize that you are asking for “fresh” applications – not the ones mentioned in class.
      • Collect cards and let students know when they will receive feedback
    • Analysis Steps
      • Divide responses into 4 piles – great, acceptable, marginal, not acceptable
      • Find common characteristics of responses in each pile
      • Share 3 to 5 good examples and 1-2 marginal examples with the class.  Try to pick a selection that is varied.  Explain why good examples are accurate and why marginal examples are implausible.
    • Extension Ideas
      • Let students work in small groups to come up with application ideas
      • Encourage students to keep an applications journal – 2 min at end of each class brainstorming possible real world application of concepts covered that day
    • Pros
      • Simple way to gets students thinking about possible uses of what they are learning
      • Tying prior knowledge to content to create new applications creates memorable associations
      • Possibility of real world applications can get students more engaged in the course
      • Students can benefit from hearing about the best examples of applications (even more so than textbook examples)
      • Teachers get access to a new bank of applications to use in class examples
    • Cons
      • Can shift focus of class to more concrete level than teacher intends
      • Students not interested in applications may not see the point in this
      • Not all fields have easily definable real world applications
    • Caveats
      • Students who suggest bad examples may learn misconceptions from these.  Provide feedback to correct these misconceptions.
      • Engagement in examples may eat up more class time than intended
  3. Student-Generated Test Questions
    • Description
      • Students create model test questions and the correct responses to these
    • Purpose
      • Teachers can see what students think is the most important and most fair content to assess
      • Teachers can see how well students can answer their own questions
      • Teachers can adjust students’ expectations of the course if these prove to be unrealistic
    • Step-by-Step Procedure
      • Analyze an upcoming test to determine the types of problems it will include.
      • Write out specs for questions that will direct students to create test questions that align to the types of problems that will actually appear on the test
      • Decide how many questions students will generate (1 to 2 is plenty)
      • Explain what you want students to do, the purpose for it, when they will get feedback and how doing this will improve their performance on the upcoming test
    • Analysis Steps
      • Note the following for each:
        • topic
        • thinking level
        • clarity
        • difficulty
      • Look for helpful patterns in the categories above – what’s included? what’s missing
      • Share observations with the class – especially if the difficulty and thinking levels of their questions is lower than the expectations of the source
    • Extension Ides
      • Let students come up with questions in pairs or small groups.
      • May want to assign specific topics to specific groups in large classes – can do this alphabetically
      • Offer handout of student generated questions for test review and tips for how to prepare for the upcoming test
      • If the students are pre-service teachers – give them more specific feedback on their question design – especially on the thinking levels they used.
    • Pros
      • Students learn what they understand and not
      • Predicting test questions is a form of test preparation
      • Can avoid ugly surprises due to mismatches in students’ and teacher’s expectations of the course
    • Cons
      • Student who lack experience in creating questions may need to have this modeled.  May need trigger words or sentence stems to create good questions.
      • Students may try to get teacher to put easy questions on the test.
      • Some students may be disappointed if their questions are not included on the review or test.
    • Caveats
      • Do NOT promise to include student test questions on the actual test.
      • Unless students understand the purpose of the assignment, may view this as a thinly veiled attempt to get students to do teacher work.
      • May need to offer some grading credit to the assignment if creating it is time consuming.

 

3-sowhat
Applications cards and directed paraphrasing are assessments that build the connections between content and real world applications and audiences.  Creating these connections may get students to use their prior knowledge and content knowledge in ways that create memorable associations.  Tying content to real audiences and real products can get students more engaged in the content.

 

The student-generated test questions can help students start preparing for high stakes tests.  Analyzing students responses can help teachers become more aware of how his or her expectations match (or don’t match) the student’s expectations. Knowing this in time can help teachers make adjustments to lessons that help those expectations converge in time for the exams.

 

4-nowwhat
Preparation Steps
  • Analyze how learning targets connect with real products and real audiences.  Determine whether or not these relationships are engaging and important.
  • Select a strategy that emphasizes the type of connection between content and the real world (via audience or via product) that is the most engaging and helpful.
  • Create a model of the selected strategy.
Early Implementation Steps
  • Explain the expectations, purpose and criteria for the selected assessment.
  • Walk students through a sample assessment and describe the types of thinking that went into creating the model.
  • Assign the selected assessment.
  • Analyze the selected assessment using some of the analysis procedures described above.
  • Share key findings of assessment analysis with the class.
Advanced Implementation Steps
  • Expand the scope of the assessment using some of the extension suggestions describe above.
  • Expand the assessment into a Quick Writes or Writing to Learn activity.
5-relatedstuff

166: Assessing Problem Solving Skills (2 of 2)

 

2-what

 

Screen Shot 2016-05-08 at 10.39.02 PM

 

  1. Documented Problem Solutions
    • Description
      • Students document all the steps they applied to solve a problem.
    • Purpose
      • Assess how students solve problems
      • Assess how well students can communicate how they solve problems
      • Development student awareness and control of problem solving routines
    • Step-by-Step Procedure
      • Select 1 – 3 representative problems for students to solve.  If you select 3, you can pick:
        • one problem ALL students can do
        • one problem MOST can do
        • one problem that will challenge NEARLY ALL students
      • Document problem solutions for each of the problems.  Be sure to follow all the expectations you intend to set for students.
      • Give students time in class or assign problems for homework.
      • Explain that documenting the steps and their rationale is more important than getting the problem right.
    • Analysis Steps
      • Analyze steps in correct solutions and in solutions that had correct steps but arrived at the wrong answer
      • Try to diagnose the types of errors and wrong turns students took when arriving at the wrong answer
      • Distill what was learned to 3 to 4 key observations
      • Share observations with the class
    • Extension Ides
      • Assign a low and medium level problem as a diagnostic pre-assessment so you know how to start the next problem solving lesson
      • Create heterogenous teams and ask students to explain their solutions to each other.  Document mistakes and how students learned from these.
      • Ask students with strong solutions to present their solutions to the class.
      • Make this a regular part of homework. Example:  students document at least one problem in their homework set this way
    • Pros
      • Makes problem solving thinking processes visible
      • Makes students aware of a range of problem solving approaches
      • Promote content-specific metacognitive skills
    • Cons
      • Students may struggle at first to explicitly comment on their problem solving steps
      • Some mistakes are hard to explain
      • When students are solving problems at many varied levels, general feedback may not be helpful
    • Caveats
      • Model the skill a lot – especially in the beginning
      • Don’t expect very good solutions on the first couple tries
      • May need to give credit for responses because they are time consuming
      • Try not to get overwhelmed by grading but try to develop feedback insights that feed students at all problem solving levels
  2. Audio- and Videotaped Protocols
    • Description
      • Students create a video or audio recording of them solving a problem
    • Purpose
      • Assess in detail how well students solve problems
      • Assess how well students are able to communicate problem solving processes
    • Step-by-Step Procedure
      • Select or create a problem that is worth talking through: involves multiple steps and students can be expected to solve it
      • Decide whether to video or audiotape the solution – both can be done fairly easily with most cell phones.  Educreations is also a good tool for this.
      • Decide in advance what you will look for in responses and how you will assess them
      • Draw up a problem sheet with the problem and criteria for taping the solution.  Be sure to include time limits and what they are supposed to learn from the exercise
      • Make students aware of the types of feedback they can expect to get from their recordings.
    • Analysis Steps
      • Create a model solution.
      • Create a checklist that describes model solution.
      • Use the checklist to notice features that student solutions have right, incorrect, or are missing
    • Extension Ides
      • Ask students to work in teams to create individual recordings.
      • Provide students with a checklist for providing feedback on the videos and let students use it to give their teammates feedback.
    • Pros
      • Recordings are accessible and can be viewed multiple times.
      • Recordings can be stopped and rewound to focus in on key steps.
      • Students and teachers work closely together on problem solving processes.
      • Teachers gain insights on student’s problem solving points and can identify stuck points to address in class.
      • Students become more aware of their problem solving routines which gives them better control over them.
    • Cons
      • Takes time to create and assess.
      • Students can’t be anonymous.
      • Solutions will be diverse – hard to compare to other solutions.
      • Some students who are good problem solvers are not good at communicating their thinking.
    • Caveats
      • Time associated with this can’t be justified unless the skill being practiced is critical to the course or future employment
      • Students will expect a grade on this because of the time expended.
      • Teachers need to be open-minded to various valid solutions to problems.

 

3-sowhat
The solution documentation strategies describes in this article can be used to make students’ problem solving processes more visible.  Students can learn more about how they solve problems which can help change their processes as they learn new things.  Although these artifacts take longer to assess, teachers can learn a lot about students’ problem solving skills by evaluating these artifacts.

 

4-nowwhat
Preparation Steps
  • Analyze problem solving skills in upcoming projects.
  • Decide whether any of the problem solving skills are critical and subtle enough to be looked at closely.
  • If so, decide which of the strategies listed above you would like students to use to document their solutions.
  • Create a model solutions of the type your students are about to create.
Early Implementation Steps
  • Show students a model solution that demonstrates what they are about to create.
  • Emphasize the key features of the model solution and how these were created.
  • Give students time in class or outside of class to document solutions.
  • Analyze solutions using one of the approaches described above.
  • Share feedback with students.
Advanced Implementation Steps
  • Teach students how to use checklists to find common flaws in their documented solutions.
  • Have students reflect on the lesson they are learning as they correct flaws in their solutions.
  • If you repeat the strategy multiple times, have students reflect on how their problem solving routines are changing over time.

 

5-relatedstuff