67: HCD Implementation Phase

1-sources

 

2-what
Key Terms:
  • Rapid prototype:
    • tests pieces of solution
    • low fidelity
    • not market ready
  • Live prototype:
    • tests how well solution resonates with the market
    • moderate fidelity
    • tests multiple parts of solution
    • appears to be market ready
  • Pilot:
    • version of solution that is holistically feasible and viable in the market place
    • high fidelity
    • tests whole idea and whole systems
    • actually market ready
  • Bootstrapping
    • no outside partners for funding
    • for very lucrative ideas
    • PROS: lots of control, can change quickly, not reliant on partner preferences
    • CONS: costly, high risk, large staff, slow growth, compete with companies who could be collaborating partners
  • Franchising
    • selling or licensing product to funding partners
    • good when other entrepreneurs like your idea
    • PROS: moderate control, less costly, connections to supply chain
    • CONS:  difficult to maintain quality control, relies on will and preferences of outside partners
  • Integration
    • combine forces with external organizations
    • good when solution complements an existing service or product or when partnering organization has the resources to scale up solution faster
    • PROS: high impact, cheap, connected to supply chains, quick scale up
    • CONS: difficult to maintain quality control, loss of control, reliant on will and preferences of partners
 
STEPS in IMPLEMENTATION PHASE:
  1. Understand your target:
    • what does solution mean to clients and those involved in its implementation?
    • what’s the capacity of the implementation group?
    • plot solutions in an Innovation 2×2 chart to identify incremental, evolutionary, and revolutionary ideas
    • innov22
    • clarify user group (new or existing) for each solution
    • identify and test solutions that fit gaps in the Innovation 2×2 chart
  2. Create an action plan:
    • plan how design will make it to the market – identify key processes and partners
    • make a roadmap – calendar that shows key milestones and interactions with key stakeholders
    • staff project – assemble a team with specific skills or access to funding needed to implement products
    • build partnerships – identify key partners and build relationships with these
    • develop funding short-term and long-term funding strategies – short-term strategy is for product launch until sufficient penetration to market that it can get access to sustainable sources of revenue
    • create a pitch
      • explain how product works, why it counts, who benefits
      • modify pitch for different listeners
  3. Launch your Solution:
    • test idea in the real market place
    • run a life prototype, stress test for market conditions
    • define what to test.  testable items include:
      • pricing – how will it vary? how does it compare to competitor products?
      • payment options – upfront? installments? subscriptions?
      • incentives – how to pay employees? commissions?
      • customer retention – which customers are most important? how to retain these?
      • customer experience – are customers interested in product? does interest linger?
    • go to pilot
      • test ideas AND systems
      • done after testing and refining a few live prototypes
      • idea has proven to be feasible, desirable, viable, and scalable
  4. Keep getting Feedback & Iterating:
    • keep getting feedback:
      • measure and evaluate work
      • dedicate a team to gathering feedback
    • include key stakeholders:
      • convene many stakeholders to get lots of feedback
      • document feedback
    • keep iterating
      • improve solutions using feedback
      • tweak things such as:
        • communications strategy
        • distribution plans
  5. Scale Towards Impact
    • define success
      • determine what success looks like over different time periods
    • sustainable revisions
      • assess new strategies
        • total costs?
        • reliability of funding?
        • what relationships needs to be built?
        • how much to sell to stay viable?
        • how to retain customers?
        • launch new products over time?
      • scaling options – add locations? add products?
    • measure & evaluate:
      • identify measured of success and how to measure them
3-sowhat
Practicing the Implementation phase teaches students and teachers how to implement REAL solutions that impact people outside school.  This phase can be used to scaffold the most authentic projects in which students design solutions that will be actually used by (and perhaps sold to) people outside school.

 

4-nowwhat
Preparation Steps
  • HCD Implementation Steps applied to Solving Student Learning Design Challenge
    • Recruit an implementation team (small) who has the skills needed to help you test and implement your live solutions
    • Decide which of the implementation steps above are practical to trial in a school setting
    • Decide which solutions you will take through live trials
  • Scaffolding HCD Implementation Steps for Students
    • Experience HCD process prior to facilitating it to learn how to better scaffold it
    • Let students in design teams select a worthy design challenge (or assign one)
    • Guide them through a student friendly, time affective version of the Inspiration phase
    • Guide them through a student friendly, time affective version of the Ideation: synthesis phase
    • Guide them through a student friendly, time effective version of the Ideation:prototyping phase
    • Develop visuals and assign readings that teach students how to go through key steps in a student friendly, time effective version of the Implementation phase.
    • Build relationships with community partners who can assist students with testing and implementing their solutions in authentic settings
Early Implementation Steps
  • HCD Implementation Steps applied to solve student learning design challenge
    • Plot solutions in Innovation 2×2 chart.  See above
    • Use Innovation 2×2 analysis to get inspiration for new solutions to iteration if needed
    • Create an action plan.  See above.
    • Launch solution.  see above.
    • Gather feedback and continue to iterate.  See above.
    • Determine impact scale of solution and take steps to bring it to that scale.  See above.
  • Scaffolding HCD Implementation Steps for Students
    • Facilitate student design teams through a student friendly, time-effective version of the steps listed above
Advanced Implementation Steps
  • HCD Implementation Steps applied to solving a student learning design challenge
    • Reflect on the entire HCD process
    • Decide which steps in the process would be appropriate for scaffolding projects of varying levels of authenticity.   Start to implement these phases and related scaffolds into student projects.
  • Scaffolding HCD Implementation Steps for Students
    • Let students reflect on how Implementation Steps can be used to implement and test solutions in other courses and in their own lives
    • Have students reflect on how the HCD process changed their view of themselves as learners and as potential entrepreneurs
    • Build relationships with actual companies who would like to implement real student solutions so that all HCD Phases can be implemented to a high degree of fidelity.  For 2 tools to building relationships with partner, see this article.

 

5-relatedstuff

66: HCD Ideation: Prototyping Phase

1-sources

Class 4 Readings in “Design Kit_The Course for Human-Centered Design.” Dropbox. Web. 14 Mar. 2016.

2-what

 

Screen Shot 2016-05-09 at 9.34.57 PM

 

Key Terms:
  • Ideas:
    • answers to the How Might We … ? questions
    • have potential for success, innovative
  • Concepts:
    • polished, concrete ideas
  • Prototypes:
    • tangible ideas
    • rough, just accurate enough to get useful feedback
  • Experience Maps:
    • Visual storyboard that describes beginning, middle and end of user experience of prototype
PROTOTYPING PHASES
  1. Generate Ideas:
    • Choose appropriate space – large enough to move in & post many ideas
    • Gather tools for displaying ideas
    • Recruit large diverse team
    • Keep pacing high energy – no more than 1 hr total, 15-20 min per question
    • Select facilitator
    • Introduce RULES:
      • Defer judgement
      • Encourage wild ideas
      • Build on other ideas
      • Stay on topic
      • One conversation at a time
      • Be visual.  Sketch ideas.
      • Go for quantity
    • Equip everyone to participate
    • Attend to each question, one at a time
  2. Select Promising Ideas:
    • Cluster similar ideas
    • Everyone votes for Top 2.  Vote directly on top of ideas
    • Tally and discuss results.
    • Decide which ideas to develop.
  3. Determine what to Prototype:
    • Break down beginning, middle and end of user experience
    • Create Experience Maps
    • Identify key questions
    • Create order of operations
      • Prioritize questions and prototypes used to investigate these
  4. Make Prototypes:
    • Create rough 3D models of concepts
    • Use digital tools to build a mock-up
    • Role play user experience
    • Create diagram that maps out key processes and services
    • Create stories – newspaper articles, job descriptions that relate key features of concepts
    • Create fake advertisements that highlight key features, tweak to different audiences
  5. Test & Get Feedback:
    • Consider setting – informal or close to actual setting?
    • Define what to test
    • Define feedback activities – ensure that these are best suited to things being tested
    • Invite honesty & openness – communicate that idea is rough and can change in response to feedback
    • Stay neutral – do not be defensive
    • Adapt on the fly – change as you go if possible
    • Provide multiple prototypes if possible
    • Find space and time to discuss initial impressions of feedback
    • Captures ideas & iterations – photo-journal key processes & prototypes
    • Share impressions – compare notes with team and document findings
  6. Integrate feedback & Iterate:
    • Share findings on Post-Its
    • Cluster similar findings
    • Categorize findings by: concerns? pros? suggestions?
    • Evaluation relationship of findings to original intent of product
    • Prioritize feedback – which is most important? what to respond to?
    • Evolve prototypes – makes changes that eliminate barriers and respond to key suggestions
  7. Repeat Steps above Iteratively

 

3-sowhat
This phase enables teachers and students to convert ideas into tangible solutions and then test pieces of these solutions in quick tests that allow designers to quickly learn and iterate from their mistakes.  Going through this process can teach learners how to use iterations to learn and revise and how to learn from their mistakes.

 

4-nowwhat
Preparation Steps
  • HCD Ideation: Prototyping Steps applied to Solving Student Learning Design Challenge
    • Recruit a design team and select a problem that relates to student learning and complete the Inspiration phase
    • Recruit a design team that will help you identify document key insights that you gathered during the Ideation: Synthesis phase
    • Recruit a diverse larger brainstorming team.
  • Scaffolding HCD Ideation: Prototyping Steps for Students
    • Experience HCD process prior to facilitating it to learn how to better scaffold it
    • Let students in design teams select a worthy design challenge (or assign one)
    • Guide them through a student friendly, time affective version of the Inspiration phase
    • Guide them through a student friendly, time affective version of the Ideation: synthesis phase
    • Develop visuals and assign readings that teach students how to go through key steps in a student friendly, time effective version of the Ideation: Prototyping phase.
    • Secure A LOT of Post-Its, secure materials for making 3D prototypes, or teach them how to use Post-It and CAD software.
Early Implementation Steps
  • HCD Ideation: Prototyping Steps applied to solve student learning design challenge
    • Meet with brainstorming team in a space with lots of Post Its and wall space.
    • Generate ideas while following the brainstorming rules.  See above.
    • Select the most promising ideas. See above.
    • Investigate the user experience and decide what to prototype.  See above.
    • Create prototypes of various forms.  See above.
    • Test prototypes and get feedback.  See above.
    • Integrate feedback into revised prototypes and iterate See above.
    • Develop, test, and refine prototypes several times.  See above.
  • Scaffolding HCD Ideation: Prototyping Steps for Students
    • Facilitate student design teams through a student friendly, time-effective version of the steps listed above
Advanced Implementation Steps
  • HCD Ideation: Prototyping Steps applied to solving a student learning design challenge
    • After enough has been learned from prototyping, move on the Implementation phase.
  • Scaffolding HCD Ideation: Prototypin Steps for Students
    • Let students reflect on how Inspiration: Prototyping Steps can be used to develop better insights to problems in other courses and in their own lives
    • Facilitate students through the Implementation phase

 

5-relatedstuff

65: HCD Ideation: Synthesis Phase

1-sources

Class 3 Readings in “Design Kit_The Course for Human-Centered Design.” Dropbox. Web. 14 Mar. 2016.

 
2-what
Screen Shot 2016-05-09 at 9.39.46 PM
Key Terms:
  • Learnings:
    • highlights from interviews, observations, and anecdotal notes from Inspiration Phase
    • communicated in full sentences to capture whole stories
  • Themes:
    • categories that organize learnings
    • capture similarities in multiple data sources
    • headlines for clusters of similar learnings
  • Insights:
    • succinct expressions of what was learned
    • offer both new and expected perspectives
  • How Might We …?:
    • generative questions based on insights that trigger brainstorming
  • Ideas:
    • brainstormed items that answer How Might We … ? questions
    • during brainstorming these ideas are not judged
    • can be represented by quick sketches
 
HCD Ideation Phase: Synthesis Steps
  1. Capture Your Learnings:
    • Set up a download space: space with lots of room for displaying Post-Its
    • Download learnings: 
      • document key findings on Post-Its
      • print out key photos
      • what to capture?
        • personal details – who? age? location? profession?
        • interesting stories
        • motivations
        • barriers, frustrations
        • interesting interactions with people and the environment
        • lingering questions
    • Share inspiring stories:
      • tell compelling stories
      • be descriptive and specific
      • share the who, what, when, where, why, how
      • identify resonant stores
      • Tips:
        • actively listen – compare / contrast stories, identify recurring themes
        • capture information in small pieces – quotes, succinct stories
        • display notes – display and organize Post-Its on wall by data source details
  2. Search for meaning: identify themes that can inform design
    • Don’t feel shy about retelling key stories
    • Cluster related information:
      • each picks top 3
      • cluster Post-Its in categories such as: consistent problems, shared significance, surprise elements
    • Find themes
      • name clusters
      • rearrange until all themes have been identified
    • Turn themes into insight statements
      • each theme can have multiple insights
      • express as complete sentences
    • Revisit your challenge
      • revisit initial design challenge using insights
      • narrow insights using design challenge to top 3 – 5
  3. Create “How Might We … ?” Questions (generative questions)
    • multiple questions per insight
    • use plain simple language
    • aim for proper scoping
      • too narrow – hinders creativity
      • too broad – not actionable
    • suggests starting points
    • suggests key logistics
    • select Top 3
      • trust gut
      • select most generative questions, not necessarily the easiest ones

 

3-sowhat
The Human Centered Design (HCD) Process is an approach that can be used to develop projects that engage students AND teach content & 21st Century skills. The Ideation: Synthesis phase converts research into meaningful actionable insights.  It converts learnings into design opportunities.  These are skills that students and teachers can develop in order to learn how to better interpret and leverage data.

 

4-nowwhat
Preparation Steps
  • HCD Inspiration Steps applied to Solving Student Learning Design Challenge
    • Select a design problem that relates to student learning and complete the Inspiration phase
    • Recruit a design team that will help you document and identify key insights that you gathered during the Ideation: Synthesis phase
  • Scaffolding HCD Inspiration Steps for Students
    • Experience HCD process prior to facilitating it to learn how to better scaffold it
    • Let students in design teams select a worthy design challenge (or assign one)
    • Guide them through a student friendly, time effective version of the  Inspiration phase
    • Develop visuals and assign readings that teach students key steps in student-friendly version of the Ideation: Prototyping phase
    • Secure A LOT of Post-Its or teach them how to use a post-it app
Early Implementation Steps
  • HCD Ideation: Synthesis Steps applied to solve student learning design challenge
    • Meet with design team in a space with lots of Post Its and wall space.
    • Record these learnings on Post-Its.  See above for details.
    • Work with data team to identify recurring themes. See above for details.
    • Work with data team to identify key insights. See above for details.
    • Work with data team to formulate and prioritize “How Might We … “ questions. See above for details.
  • Scaffolding HCD Inspiration Steps for Students
    • Facilitate student design teams through a student friendly version of the steps listed above
Advanced Implementation Steps
  • HCD Ideation: Synthesis Steps applied to solving a student learning design challenge
    • Gather more research on lingering questions
    • Repeat early implementation steps again
    • Then move unto to Prototyping phase
  • Scaffolding HCD Inspiration Steps for Students
    • Let students reflect on how HCD_ Inspiration Steps can be used to develop better insights to problems in other courses and in their own lives
    • Facilitate students through Prototyping phase
5-relatedstuff

57: From Framework to Protocol

1-sources
This link will only work if you’re logged into Echo.  Sorry, non-New Tech readers.  If you have more questions, you can tweet Jim May at @jimamay.

 

2-what
Why have a focus area for change:
  • we get better SPECIFICALLY, not GENERALLY
Finding a Focus Criteria:
  • related to STUDENT LEARNING
  • emerges from multiple sources of data
  • politically feasible
  • advances school’s mission & purpose
 
The Framework can be used to classify types of focus areas:
framework

 

Caveats:
  • If focus is not shared enough – don’t know where to look to see if it’s working
  • Hope is not a strategy
  • Principal can not be the only source of feedback (need to leverage peer feedback)
Guiding Questions
  • What are the unique challenges that relate to focus area? Evidence of these?
  • What are the unique strengths we already possess relating to focus area? Evidence of these?
  • What do possible strategies and outcomes look like?
  • How will we observe and measure outcomes related to focus area?
From focus to strategy:  Sources:
  • experts in the field,
  • reading literature,
  • disciplined experimentation of new strategies
 
Strategies (Goor characteristics):
  • Can have multiple activities that share one focus area
  • Addresses both technical and adaptive elements
  • Increases learning of stakeholders related to focus area
Driver diagrams: 
  • displays primary drivers (red) and related factors (blue) that connect to focus area (green);
  • can be used to brainstorm strategies (lavender) that relate to focus areas
driver

 

Testing and Learning from Strategies:
  • Use student work as primary source of evidence
  • Analyze student work using specific question(s) related to focus area
  • Examine the learning task that led to the student work
  • Keep time between analysis and action very short
  • Try to link data directly to improving practice
  • Use data analysis to develop a deeper shared understanding of focus area
 
Designing & Interpreting Data Inquiries:
  • data related to specific questions tied to focus area
  • shared knowledge base that can be used to make analytical decisions about data
  • create artifacts that anchor sense making
  • impose structure on data to make meaning of it

 

3-sowhat
School change is hard, but vital.   Developing and implementing processes for setting, implementing, learning from, and revising school change goals is vital to maintaining and improving the quality of PBL schools.  Having a specific focus related to student learning that is backed by data derived from student work helps schools select focus areas that serve their primary clients, the students, and their primary mission, learning.

 

4-nowwhat
Preparation Steps
  • Brainstorm possible focus areas using criteria above.
  • If needed, gather more data related to potential focus areas.
  • Analyze school structures that relate to focus area (drivers).
  • Research & develop potential strategies for improving and learning more about focus areas.
  • Identify evidence that can be used to assess potential strategies.
  • Develop a data inventory that describes all the data sources that will be used to investigate focus area, when these will be gathered, how to access these, and ideas for how to interpret these.
Early Implementation Steps
  • Implement strategies.
  • Use pre-planned evidence gathering strategy, to collect student work that can be interpreted to learn more about focus area.
  • Create data summaries that make sense of data and provoke interesting conversations.
  • Have regular data meetings focused on making sense and developing key questions from data related to focus area.
Advanced Implementation Steps
  • Involve students and alumni in key committees that select focus areas and that interpret and learn from data.
  • Use student panels to gain more qualitative data that relates to other data sets collected to investigate focus area.
5-relatedstuff

55: PBL Trade-Offs

1-sources

 

2-what

 

Screen Shot 2016-05-09 at 10.27.04 PM

 

Big ideas/transfer VERSUS specific knowledge and skills:
  • Err on the side of big ideas -> knowledge may be too abstract to be useful
  • Err on the side of specific -> may learn skills but not why / when to use them
  • Aim for a balance between discrete information and meaningful conceptual frameworks
Complex realistic messy performances VERSUS efficient and sound tests:
  • Err on side of messiness -> may lead to confusion, take too much time
  • Err on side of efficiency -> tests lack validity, learning may end up being inert and useless in real contexts
  • Mitigate some of the messiness of performances with models and rubrics
  • Use tests for evaluating discrete enabling skills
Teacher control VERSUS learner control of the work
  • Err on side of teacher control -> students lack voice and choice in their work, may lose engagement
  • Err on side of student control -> students lack knowledge to design frameworks that tie to key learning goals
  • Let teacher control substantive holistic curriculum choices
  • Involve students in choices about product formats and day-to-day (during day) flow
Direct teach VERSUS constructivist approaches
  • Err on side of constructivism -> may take too long, may skip details in discrete skills
  • Err on side of direct teach -> students don’t make meaningful connections among concepts and skills, can not grasp big ideas
  • Use more direct approaches for discrete straightforward knowledge
  • Use more constructivist approaches for deep subtle concepts
Depth VERSUS breadth of knowledge
  • Err on side of depth -> may not get to enough ideas in course
  • Err on side of breadth -> knowledge learned at level that is too superficial for transfer
  • Develop a yearlong plan that prioritizes important ideas
Uniform VERSUS personalized work and expectations
  • Err on side of uniformity -> lack of differentiation may mean that not all students get opportunities to learn material
  • Err on side of personalization -> unsustainable, not all student learn goals
  • Use rubric for final product and allow choices that do not change the rubric at all (or very much)
  • Cluster student needs and plan remediations and enrichment activities around these
  • Use varied methods to teach the same content
Effective VERSUS merely engaging
  • Err on side of effective -> student learn skills without sense of ownership
  • Err on side of engagement -> students may have fun without learning anything
  • Design activities that are both effective and engaging
  • Make sure all hands on activities are aligned to learning goals
  • Make sure drier lessons are explicitly connected to more engaging work
Complex VERSUS simplistic
  • Err on side of complexity -> leads to confusion and lack of transfer
  • Err on side of simplicity -> big ideas lose meaning, accuracy & transfer
  • Aim to craft essential (Driving) questions that are both age appropriate and meaningful
Well crafted plan VERSUS flexibility & openness
  • Err on side of rigid planning -> may miss out on good learning opportunities, may not make necessary adjustments to students’ needs
  • Err on side of openness -> plan may lack clarity & focus, learning may feel inefficient, disconnected and untied to a larger purpose
  • Develop a well crafted plan and use formative feedback to refine it
Great individual unit VERSUS larger goals 
  • Err on side of single unit -> may neglect other important school goals
  • Err on side of larger goals -> may develop a unit that is too diluted to be effective
  • Design units that are aligned to school and curriculum goals
  • Use textbooks and tools in the service of well designed learning plans

 

3-sowhat
Project design is hard.  There are many balances to strike while designing and implementing effective and engaging projects.  Any PBL educator that’s designed and implemented several projects has probably faced at least 1 or more of the trade-offs above.  Diagnosing which trade-offs keep plaguing one’s design can give hints as to what types of design solutions teachers need to research in order to create more balance in their projects.

 

4-nowwhat
Preparation Steps
  • If a project went well, try to notice how one managed the trade-offs above in order to extract general tips.  Apply these tips to continue to design successful projects.
  • If a project went poorly, try to identify which of the trade-offs above broke down.  Research and brainstorm how to better manage that trade-off and apply those tips to future project designs.
Early Implementation Steps
  • Be aware of how trade-offs (above) are impacting project implementation and note how you manage these.
  • Survey students using some the trade-offs above to check if the right balance is being struck.
Advanced Implementation Steps
  • Use diagnoses of one’s past successes and failures, to identify which tradeoffs one manages poorly.  Develop checklists associated with those trade-offs that help guide one to a better balance in future projects.
  • Ask students for advice in your troublesome trade-offs on how you might strike a better balance.
5-relatedstuff

52: What the Hack?

1-sources
Note: This link will only work if you’re logged into Echo.  Sorry, non-New Tech readers.  If you want to contact the presenters, you can try tweeting them at  @leeafleming and @mpacheco11.

 

2-what

 

Screen Shot 2016-05-09 at 10.41.51 PM

 

Challenges and Pitfalls of Change Efforts:
  • solutionitis – inflamed, too many solutions
  • jumping to solution implementation before unpacking it, the quick fix
  • overplanning, underdiagnosis
  • can lead to change fatigue and risk aversion
  • confirmation bias – looking only for the YES in the feedback loop and not the NO’s
Hackers
  • know systems deeply
  • try several things – persistence
  • innovate
  • set up low stake tests to try out new ideas
Hacket Tips:
  1. Chunk it down
  2. Think outside the box
  3. Use small tests
  4. Shift your mindset – learn, not plan
Recommended Reads:

3-sowhatTeachers are constantly problem solving.  They are also role players in broader solutions that affect the whole school.  A hacker’s approach to problem solving could help stakeholders solve problems in innovative and tested ways.

4-nowwhat
Preparation Steps
  • Find a problem connected to student learning that is worth solving
  • Research and brainstorm solutions related to the problem
  • Research and brainstorm ways to gather data that test various solutions
Early Implementation Steps
  • Trial solutions and gather data on their effectiveness from students and other methods
Advanced Implementation Steps
5-relatedstuff

51: Amping Up the Authenticity

1-sources

 

Note: This link will only work if you’re logged into Echo.  Sorry, non-New Tech readers.  If you’d like to get access to more related information, check out Kevin Gant’s blog at https://intrepidedblog.wordpress.com/

 

2-what

 

Screen Shot 2016-05-09 at 10.46.06 PM

 

Evolution of projects from towards more authenticity:  scenario based -> project world = real world

 

Characteristics & Challenges of Most Authentic Projects:
  • Project work = work of the world
  • Student products are used by people outside school
  • Students have more choice to define products
  • Not all standards can appear in all products
  • Can give all students opportunities to gain mastery of all learning targets in project scaffolding
  • Involves a provocative driving question that acts as a call to action for students to solve a real problem
  • Driving question can be resolved  by a wide variety of end products
  • Content authority ceded to experts outside the classroom
  • Time frame can get mushy for some products due to interactions with real clients running on schedules different from the school schedule

 

Related Useful tools:
  • Nepris 
    • Web-based tool that connects experts to classroom, teachers can post requests and repris matches experts to request
    • Heard that wait time is about 3 weeks
  • IGNITE by DiscoverSTEAM
    • Online platform that sets up secure communication platform between companies that want to interact with students and schools
    • Companies present projects to students that they want solved; if students develop solutions the companies can use, they can pay students
    • Teachers can work with companies to develop rubrics aligned to standards
    • All communication between students and companies is recorded and reviewed for quality and research purposes
    • IGNITE is FREE to schools because companies pay for it as a way to recruit American talent and to market their companies
    • Companies prefer to have a yearlong relationship with students – my idea to extend the interaction time with companies is to relay the company project through different courses;  project could start in ELA for initial research and writing processes, then move to a Science course for designing data studies and gathering data, then move to a Math class to analyze the data, then move to a Social Studies class or Audio Visual class to finalize conclusions and create products, etc
    • NOTE: This company is looking for PBL teachers to pilot their tool.  Contact them if you’re interested
3-sowhat

The more time teachers and students spend doing PBL projects, the more difficult it gets to make projects feel fresh and meaningful.   Facilitating real world projects is one way for advanced PBL teachers and advanced PBL students to frame learning within more relevant contexts and challenges.  Authentic problems can also give students exposure to careers they have not yet considered and give them a better idea of how real experts solve problems.

4-nowwhat
Preparation Steps
  • Extend your network – contact companies, university groups, and local groups of experts who do things that relate to your course
  • Build a database that contains potential contacts – see above
  • Contact them early – in the summer if possible – to see if there are problems they’d like students to solve or explore that could align to standards
  • Check if school has partnered with companies like Nepris or DiscoverSTEAM – if the partnership is there, learn how to use those tools and start using them expand your network
  • Plan Year at a Glance (Scope and Sequence)
  • Contact experts early to see if they can interact with students at the appropriate times in the year (or adjust your scopes & sequence as needed)
  • Develop templates (email and phone call) for students to contact experts
  • Develop tools that students can use to document their unique projects and their ties to key content and problem solving processes that are being uncovered in the classroom
  • Design scaffolding that supports real world project work and aligns to standards
  • Research and commit to a design process (or similar problem solving processes) that will organize the project and that students will use to solve project problems
Early Implementation Steps
  • Work with experts to support students with content knowledge and with formative feedback they can use to improve their understandings and products
  • Organize project around a common design process (or similar problem solving processes) that students will apply to develop different products of their choice
  • Implement class-wide scaffolding that ensures that all students have opportunities to learn targeted content
  • Be flexible with final product completion and final presentation dates as these may vary depending on their solutions and their clients
  • If possible – set a common proposal presentation date to review and give critical friends feedback on proposed solutions
Advanced Implementation Steps
  • Develop sustaining relationships with experts that enable different cohorts of students to gain exposure to real world problems that have better and better project design (if the opportunity arises to interact with same experts on similar projects form year to year)
  • Arrange for field trips that allow students to get close hand  exposure to experts and to gain experiences that can help them gather useful data related to their projects
  • For experts that like prolonged relationships with students, collaborate with other courses to extend project over time (see relay idea above)
5-relatedstuff

50: Building Data Summaries

1-sources

 

2-what

 

Screen Shot 2016-05-09 at 10.49.47 PM

 

Tips for Data Summaries & Data Discussions:
  1. Decide on educational questions
  2. Simpler is better (data summary)
  3. Start conversation with an interesting comparison
  4. Facilitate constructive conversation
 
Tips on Creating Data Displays
  1. Decide on questions that guide graph designs – identify independent and dependent variables in the questions
  2. Choose graph type that best displays the relationship between the independent and dependent variables
    • bar charts – independent variable is a category
    • scatter plots – when independent and dependent variables are both quantitative
    • pie charts – when dependent variable measures percentage or fraction of a whole
  3. Select graph type that address the question clearly and effectively
  4. Reorganize graphs or tables to draw attention to critical comparisons
    • despite measurement error, group comparison can still be useful to highlight interesting features in overall performance
  5. Label graphs to make independent and dependent variables clear
  6. Where possible – use district software to create appropriate data displays
Components of Good Displays
  • explicit informative title that points toward critical element in the chart
  • clear axis labels
  • information pertinent to key questions are the most prominent features in the graphs
  • keep graphs free of extraneous detail
  • communicate what groups are being compared in graphs and data sources in graphs
Leading effective data discussions: important to give time for stakeholders to process and make own sense of data and ask own questions
  1. Give time for stakeholders to puzzle over the data
  2. Pair share what they noticed from studying the data
  3. Encourage stake holders to value questions over conclusions.  Brainstorm provocative questions that go with the data.
  4. Identify most important questions
  5. Brainstorm data needed to answer the key questions

 

3-sowhat
Knowing how to make clear and clean data displays is a necessary preparation step to having good data conversations that are based on actual data trends and comparisons.   Letting data stimulate the development of important questions is a hopeful, learning exercise that invites stakeholders to approach data as learners rather than as evaluators.

 

4-nowwhat
Preparation Steps
  • Decide on important educational questions
  • Identify key independent and dependent variables in the questions
  • Create data displays that effectively display relationships between independent and dependent variables
  • Make sure graphs have titles and axis labels that clearly indicate the data sources and data variables in the graph
  • Reorganize data to make interesting comparisons between different student populations
Early Implementation Steps
  • Facilitate data discusion
    • give stakeholders individual time to investigate and brainstorm questions from the data
    • give stakeholders time to share questions with a partner and prioritize questions
    • prioritize questions as a large group
    • brainstorm ways to address key questions
Advanced Implementation Steps
  • Use this process to develop and track student whole group goals that can be measured by data
  • Survey students to check that this process is helping them to feel like more empowered stakeholders in their own educations
  • Encourage students to gather more data (qualitative and quantitative) that addresses key questions

 

5-relatedstuff

49: Building Assessment Literacy

1-sources

 

2-what

 

Screen Shot 2016-05-09 at 10.52.39 PM

 

Principles for Interpreting Assessment Results:
  • Sampling Principle of Testing – tests are not direct measures of mastery; conclusions about mastery in a domain are based on inferences drawn from a smaller sample
  • Discrimination – assessment items that discriminate, separate students who know from students who don’t; use these items to reveal differences that actually exist
  • Measurement error – many sources – examples: differences in questions, different student moods, etc.
  • Reliability – conditions that give consistent results
  • Score inflations – scores inflated by cheating, luck, etc ..
Accounting for Sampling & Measurement Errors:
  • Measures and attach error bands to test scores
  • Keep test scores in perspective – tests have limitations, they don’t assess all important skills, course grades can not be compared across schools like standardized tests
Different Ways of Reporting Performance:
  • Beware of raw scores – not normalized for difference of questions between tests
  • Norm referenced tests – PRO: results are normalized by results of all testers, CON – improvements are easy near bottom and harder near the top
  • Criterion referenced tests – measure mastery of specific skills, PRO – gives specific information CON – vulnerable to sampling errors due to small number of questions per skill
  • Standards-Referenced tests – tests based on content / performance standards.  PRO – gives specific information relating to standards CON – vulnerable to sampling errors due to small number of questions per skill
Trade-Off
  • Reliability vs detail – more detail tends to smaller sample sizes which tend to be less reliable
    • it’s risky to draw conclusions from small sample sizes – e.g. standards assigned to only 1 – 2 questions on a test
How to Measure Improvement
  • Cohort-to-cohort – compares same grade, different students – CON – susceptible to demographic biases
  • Value-added, longitudinal – compares same students are different times – CON – susceptible to a lot of error PRO – works well with cumulative subjects
Strategies for Interpreting Data
  1. Look beyond 1 year with cohort-to-cohort or value-added data
  2. Compare results to relevant district and state data
  3. Compare results to most recent assessments on similar standards
3-sowhat
Knowing sources of data error can teach one to use data selection, summarizing and interpretation strategies that minimize impacts of data error.  Teachers planning to investigate data with students need to know sources of data error in order to teach them to students and to teach students how to make careful interpretations of data.  Using the strategies for data above can help highlight important data features by using comparisons to relevant reference groups.

 

4-nowwhat
Preparation Steps
  • Decide what data sources will be summarized.
  • Select methods for summarizing data that compare data to relevant reference groups – these can include comparisons to other schools, other districts, state scores and to the same students at another time in the school year
  • Script questions stakeholders (students) will use to investigate the data.  For more details, see this article.
  • Create data displays that provoke data discussion.  See this article for more details.
Early Implementation Steps
  • Facilitate data meeting with students that has them use reflection prompts to draw conclusions about data displays as individuals, in pairs, and as a whole group
  • Brainstorm with students how to further explore key questions and next steps for improving key trends
Advanced Implementation Steps
  • Gather feedback from students aimed at improving data conversations and use these to fine tune future discussions
  • Have students brainstorm data goals that can be measured with specific upcoming assessments
  • Have students brainstorm methods for gathering more reliable data related to group academic goals
5-relatedstuff

48: Building Data Organizations

1-sourcesChapter 1 in Boudett, Kathryn Parker., Elizabeth A. City, and Richard J. Murnane. Data Wise: A Step-by-step Guide to Using Assessment Results to Improve Teaching and Learning. Cambridge, MA: Harvard Education, 2005. Print.

2-what

 

Screen Shot 2016-05-09 at 10.56.50 PM

 

This book was intended for school administrators.  I think it can also apply to teachers.  Here I summarize parts that are generalizable to all settings so that both teachers and administrators can benefit from the book.
Two Question to Ask:
  1. Are you satisfied with the way you capture info garnered from assessments?
  2. Are you satisfied with how you gather information from multiple data sources for your students?
Building Data Organization Steps:
  1. Build a data inventory: a chart or spreadsheet that organizes and lists all data sources that will be interpreted to improve student learning.  Columns on data inventory sheet could include:
    • data source name (hyperlinked)
    • content area
    • collection date(s)
    • students assessed
    • accessibility – how to get to the data
    • current data use
    • possible more effective use
  2. Build a Data Team(s)
    • identify well-connected stake holders who would be interested in investigating data to improve student learning
    • types of data teams:
      • organizing data – small team
      • interpreting data – big, collaborative effort – the more who are involved the larger number of stakeholders have ownership in school improvement
  3. Create a Schedule that Allows for Regular Collaborative Work
  4. Plan for Productive Meetings: 
    • establish norms – example – no blame, approach data as a learner
    • use protocols to structure conversations
    • adopt an improvement process: small groups analyze data charts and discuss what they notice, share key findings on chart paper, use structured protocol for formulating questions about why data looks like that, whole group – establish consensus on most important questions to address
    • plan agendas for meetings
3-sowhat
Organizing data teams to summarize data and organizing data meetings to interpret data and formula key questions from the data is a process that can improve student learning at the staff level and at the teacher student level.

 

Compiling a data inventory and writing out current and possible uses for data is an exercise that can make teachers and administrators more aware of how they are currently leveraging data and what they can possibly do to make more use of the data.

Teachers generate A LOT of data when they comply with school-wide grading expectations. Teachers can make more use of this data by creating compelling, summarizing data displays and using these displays to facilitate small and whole group discussions with students on data inferences, possible emergent questions, and possible followup experiments to further investigate or improve the data.  This process can make students feel like more valued stakeholders in the education process and could help students set and achieve individual and community level academic goals.

 

4-nowwhat
Preparation Steps
  • Create a data inventory of past and upcoming data sources that are worth interpreting
  • Research and/or brainstorm protocols for discussing data
  • Create data displays that are compelling.  These could
    • NOT reveal confidential individual student data
    • show how learning of student group(s) evolve over time
    • mastery levels of student group(s) tied to key big ideas
  • Develop an agenda that has time for students to interpret data as individuals, in pairs, and as a whole group
Early Implementation Steps
  • Facilitate a project reflection (or semester reflection) that features the data discussion agenda developed above
  • Record key findings of students
  • Work with students to achieve consensus on key questions and methods to investigate these and possible next steps and methods to verify that these are working
Advanced Implementation Steps
  • Form a data organization group that includes students after running several data meetings.  This committee can advise teacher on how to summarize data that makes it more easy to identify compelling questions and trends
  • Survey students to check about whether or not data meetings are making them feel like more valued contributors to the learning community
5-relatedstuff