Data & Dragons

I love games.  I played board games as a kid; monopoly, career, mousetrap, and more.  When I first got to college, I was introduced to Dungeons and Dragons.  I was so intrigued by the notion of creating interactive fiction on-the-fly using just dice and our imaginations.  I still am.

Fast forward to today, and for those of you not up-to-date on gaming, D&D is bigger than ever.  It was the backdrop for the popular Sci-Fi thriller “Stranger Things,” and draws hundreds of thousands of viewers every Thursday night on Twitch TV’s “Critical Role.”

For those of you unfamiliar with D&D, it works by having participants roleplay characters with a set of predefined attributes.  To start, each participant creates a character by rolling dice to create values for those attributes: strength, intelligence, agility, charisma, etc.  Your character and the characters of the other participants form a party that explores a world created by a Dungeon Master (a.k.a., the DM).  The DM creates a story narrative over maps of the world you are exploring and uses numeric tables (lots of them) to help determine what the party encounters during the exploration; everything from dragons to treasures.  Dice also determine the effect your character and party decisions have on that world.

Although each participant contributes to the game through their character, much of what happens is determined by dice and the consequences they draw from the DM’s tables.  I can’t think of a circumstance that is more data driven.

Data Driven Instruction

For educators, the construct, “Data Driven,” is now suffering a midlife crisis.  Most educational terms do.  It came into the world with such promise, but has been corrupted in the mire of “slow-but-steady” institutions.  Nowadays, everyone wants to claim they are using data driven instruction, but very few want to be data driven.

via GIPHY

I get it.  Being data driven, for real, is time consuming and requires training that few educators have enough of.  That said, research is abundantly clear, that when, and only when educators are data driven, their world can change for the better.

While the term “data driven instruction” means many things to many people, I propose to provide the most useful perspective I can think of, my own.

• Determine Need – So often, educators think they know the need before the first piece of evidence is entered.  There is an old joke about playing 20 questions with a group of teachers.  The first question is usually, “is it a refrigerator?”  Don’t sneer, try it sometime.  There are many ways of determining need, but all of them require the aggregation of opinions from various stakeholders.  This is your first data source.
• Identify Target – Now that you have collected data from your stakeholders, use those data to prioritize targets for intervention.  I have worked with schools that got this far and still wanted to focus on non-problems.  Typically, this means pet peeves, stuff experienced by only a few stakeholders.  Obviously, if you want to be data driven, here is your first opportunity.
• Select Intervention – For too many educators this is a lot like choosing toilet tissue.  They all seem to serve the same function, so pick the cheapest one.  Ouch.  There is also a buzz around choosing “evidence-based” interventions.  Just so you know, there are so few interventions with valid evidence to support them, and all too often, some common sense will actually work better.  Be picky, and make sure that the intervention is functionally related to the target.  This might mean purchasing a packaged program, but it might be better served by applying a pedagogical principle or creatively using some duct tape and WD-40.
• Negotiate Goals – This gets a little more complicated, but without goals, there is no point to this exercise.  So many efforts at school reform have failed simply because there were no measurable goals attached.  Negotiate goals, understanding that they might need readjustment later, and make them measurable.  And this is not negotiable.
• Establish Methods – For educational researchers to establish the validity of an intervention, they understand that some methods of evaluation are better than others.  The best methods use highly valid measurement in controlled conditions that limit confounding.  The few educational programs that have been validated under these conditions are too often implemented in schools with a casual disregard for that fact.  Because of funding, personnel, or training limitations, truly evidence-based interventions are diluted before they get to students in their natural habitat.  Figure out how the intervention will start and stay at full dosage for every targeted student, and how you will know this to be true.  This is your method.
• Common Understanding – I could have called this step, “summarize progress,” but that’s only part of what needs to happen here.  Data can be a chore to collect and manage.  Someone must gather, organize, tally, and summarize whatever data are part of your methods.  One more thing must happen, however, and that is whatever is created to communicate the state of the intervention and its consequences must contribute to a common understanding of the data for all the decision makers and data consumers.  Make graphs.  This is your second data source.  Find someone who is overly obsessed with detail to manage it. Once you have collected and summarized the data, you will have to communicate those data to pertinent stakeholders (e.g. teachers, parent leaders, district support personnel and often even students). Communicate this information with the goal of creating a common understanding of the state of the intervention and its consequences. In other words, the relevant stakeholders should, for the most part, have a common answer to the questions, “How is the intervention actually going?” and “What did it actually accomplish?” Plotting data on a line graph is a great way to communicate these data and help stakeholders arrive at a common understanding.    
• Adjust Intervention –Admiral Farragut proudly proclaimed at the Battle of Mobile Bay, “Damn the torpedoes,” but that was a desperate act with deadly consequences.  I don’t think educational interventions should suffer casualties.  When data have been summarized for common consumption, you now have your second chance at being data driven.  As long as goals are reasonable and progress is being made, keep going, but if not, rethink and renegotiate.

Summary

Data driven instruction might sound like a fading meme, but its promise has never wavered.  I challenge you to think about a failed effort at your school and evaluate it based on the steps above.  If being truly data driven seems like too much, it probably is right now, but that shouldn’t stop you.  Pick something small and give it a go.  It might be for just a single classroom, but once you get data accurately driving practice, you will never go back to old ways.  It’s too much fun watching valid and valuable data change and know, really know, that your energy and initiative were the cause.