Generating Data on What Customers Really Want 05-08

Generating Data on What Customers Really Want

At a fundamental level, the decisions managers make about revenue and profits fall into two categories—those related to growth and those related to cost reduction. Both types are meant to increase margins. But how data are used in each decision-making process is completely different.

Cost reduction data are precise. Firms know their cost structure very well and can compute with a reasonable level of certainty the savings each alternative being considered will generate. Managers use cost-related data as an objective input for decision making, since they consider it both reliable and reasonably predictable.

Managers do not treat growth-related data in the same way, and for good reason. Growth alternatives usually involve either launching new products or entering new markets, and these are activities where uncertainty is high. 

Here data are used mainly as a tool for persuasion among managers. A stereotypical example would go more or less like this: Managers gather the data that reinforce their own point view on what they believe is the right business decision. During a meeting they all explain their various rationales and present their various data points. Usually, they end up reaching a consensus that they all consider makes sense. Then they leave the room still thinking that their own alternative was best but that in life you have to make tradeoffs.

Most growth-related decisions in management are made this way because managers do not have data that reliably predicts how new customers will react to their offerings or how any customer new or old will react to innovative offerings. If only Coca Cola would have predictably known that its customers would not embrace New Coke! But its executives lacked a way to generate predictable information on that score or tell them that the data they’d generated from their taste tests would ultimately not be relevant.

Disruptive innovation practitioners have just such a tool for reliably predicting customers’ behavior. It’s a methodology that uncovers what in disruptive innovation parlance is called a person’s “job to be done.” Briefly, the idea is this: Consumers don’t go to the store to buy products. They go to the store to buy something that will enable them to get some important job done in their lives. 

The classic example, attributed to HBS professor Ted Levitt, is that people don’t want to buy a quarter-inch drill; they want something that will make a quarter-inch hole. Making a quarter-inch hole is the job to be done. The product that does that job most reliably, easily, conveniently, and less expensively is the tool they will be most likely to purchase for that job.

Work out your customer’s jobs-to-be-done, disruptive innovation practitioners have found, and you will generate data that more reliably predicts what a customer will buy and why. How do you do that? Traditionally, corporate innovators are told to conduct ethnographic studies, starting with no preconceptions, and to observe customers’ behavior and frustrations with the same open mind that start-ups employ. Human nature being what it is, that’s a hard thing to do.

Here, instead, is a simplified version of a methodology for identifying a customers’ job-to-be-done that starts with information about your own product. Since product information is not part of a job-to-be-done, the information about your product will drop out of the process in step 6, so that it doesn’t distort your results. But in this way, you can make your way to a new insight by starting in familiar territory. (And you’ll not keep the people you’re interviewing wondering what all your questions are about during the entire interview process.)

Step 1. Prepare a list of the key characteristics of your offering. 

List at least 10 of them. Your product or service may be faster than your competitor’s. Or cheaper. Or have a better screen resolution. Or have leather seats. Or a battery that lasts for many days. Or connect to the internet and let you play with other people online. Let’s say you sell cars. Your list might include characteristics such as speed, gas consumption, how little it pollutes the environment, number of doors, colors, type of seats, cup holders and amenities inside the car for the driver and the passengers.

Step 2. Interview at least 10 consumers and 10 nonconsumers about the various features connected to your offering. 

Nonconsumers are people who are not buying either your offering or your competitors’. So in this case you would be interviewing both people who drive cars and people who could drive cars but chose not to. The interviews can be anonymous, but you need to record the entire conversation. For each of the characteristics listed above ask three questions.

 First, “Where are you when you are using this feature?”

 Second, “When you use this feature, what are you really trying to do?” 

And third, “If this feature were not available, what would you be using instead?”

 Now, here’s an important part: for consumers, you need to ask the second two questions without reference to your product. So, to return to the car, let’s say you asked: 

“Why do you sit in the passenger’s seat of your own car?” and the answer was: “I am a salesman, and in between meetings I work in the car.” 

Then you ask: “When you do that what are you trying to accomplish?” He answers: “I try to have an environment that mimics my office space so I can concentrate and work comfortably for a while.” 

Then the third question: “If you could not do that what would you do instead?” He replies: “In the car I use the cup holder for my coffee, and in the passenger’s seat I can work with my laptop and recharge my phone. If I could not do that I would go to a cafeteria, but it is difficult to concentrate there. It’s noisy and I waste time locating one.”

Step 3. Transcribe the recordings.

 It is important that you do not miss anything. The transcript must end up looking like an interview, faithfully recording exactly what was said, complete with pauses. Being systematic about when the data stop is important for the statistical analysis used in step 5.

Step 4. Codify the transcripts by tracking all the meaningful nouns, verbs, adjectives, and adverbs. 

To continue our example we would extract from the sentence: “I try to have an environment that mimics my office space” will result in the following codes: “try,” “environment,” “mimics,” “office,” and “space.” 

With this you create a table in which you count the number of instances of each word (so if in the entire first transcript the word office is only repeated once you would have a 1 in the first row). Complete the table until all the sentences from all the transcripts have been codified. There are software tools available to do that more easily.

Step 5. Group the codes. Using the statistical technique of cluster analysis, group the codes based on their proximity. 

That is, it measures how many times each word appears close to one another (that’s why the pauses matter). Let the software that you choose determine the optimal number of clusters. The end result is that all your codes will now appear in groups.

Step 6. Remove descriptive data so you have only prescriptive data left. 

To do this, you remove all the groups that contain information about your product, in this case, all the groups that contain the word “car.” The end result is a series of groups that each contains a number of different code words. Within each group, these codes refer to the customer’s way of thinking and the portions of the context that the customer considers relevant to deciding on which product alternative to buy.

 If you compare how product performs in relation to the concerns expressed in each group, your next product improvement will become compellingly clear, not only to you, but also to your colleagues. In this example, it would become easy to make the case for focusing an innovation effort on helping salespeople become more productive and work more comfortably in their cars. 

Better electrical outlets, perhaps, so people can charge more than one item at a time? Storage for computers or samples? Something that mimics a desk more effectively? 

The point here is this is a fruitful avenue for further attention since most car models sell fewer than 100,000 units per year, but millions of people work in their automobiles.

During the 1950s Edward Deming and others developed such tools as statistical process control charts and total quality management techniques that have made the cost-reduction data we use today predictable and reliable. Before that, though, data about cost reduction was as unreliable as growth related data are today. 

Now the first tools are starting to emerge that add predictability and reliability to growth related decisions. Jobs-to-be-done is one such tool. Once managers learn how to compute a job-to-be-done by themselves, their growth related decisions will become much more objective and less opinion based.

View at the original source

Marketing Can No Longer Rely on the Funnel 05-08

Marketing Can No Longer Rely on the Funnel

One of the central concepts of marketing and sales is the funnel — through which companies are supposed to systematically move prospects from awareness through consideration to purchase.

But consumers are now more informed, connected, and empowered than ever. Does the funnel still work in a digital, social, mobile age?

We asked some of the leading marketers in the world — from companies like Google, Intuit, Sephora, SAP, Twitter, and Visa — to assess the relevance of the marketing funnel.  What we found says as much about the future of business as it does about the future of marketing.

According to these marketers, the primary problem with the funnel is that the buying process is no longer linear. Prospects don’t just enter at the top of the funnel; instead, they come in at any stage. Furthermore, they often jump stages, stay in a stage indefinitely, or move back and forth between them.

For example, consider items that come recommended on an e-commerce site. With a click you can add them to your cart, moving straight from awareness through consideration to purchase in only a few seconds. The same holds true on items discovered in a Tweet, Facebook post, or Pinterest board.

In both B2B and B2C businesses, customers are doing their own research both online and with their colleagues and friends. Prospects are walking themselves through the funnel, then walking in the door ready to buy.

As an example, Julie Bornstein, CMO at Sephora, has seen social media change how people buy beauty products. Recommendations from friends have always been important, but now these recommendations spread “quicker, faster, and further” at every stage in the funnel.

 The decision on what to buy increasingly comes from advocates who share their experience in a way that pulls in new customers and informs their purchase decision. Sephora’s response has been to bring all the stages of the funnel together into a single place, creating its own online community where people can ask questions of experts and each other about brands, products, and techniques.

One popular alternative to the funnel is the Customer Decision Journey popularized by McKinsey. 

A key advantage of this model is that it’s circular, rather than linear. Prospects don’t come in the top and out the bottom, but move through an ongoing set of touchpoints before, during, and after a purchase.

The Customer Decision Journey is an improvement over the traditional funnel, but some marketers see it as incomplete. The problem is in the name itself. Brands may put the decision at the center of the journey, but customers don’t. Jonathan Becher, CMO at SAP, believes that for customers, “the pivot is the experience, not the purchase.” The Customer Decision Journey might be circular, but if the focus is still on the transaction, it is just a funnel eating its own tail.

One of the most critical weaknesses of the Customer Decision Journey is the connection between purchase and advocacy. 

Almost every marketer we spoke to described how social media has disconnected advocacy from purchase. “You no longer have to be a customer to be an advocate. The new social currency is sharing what’s cool in the moment,” says Joel Lunenfeld, VP of Global Brand Marketing at Twitter.

In today’s marketing landscape, people can experience a brand in many ways other than purchase and usage of a product. These include live events, content marketing, social media, and word-of-mouth. Consider all the members of the Nike+ running community who don’t own Nike products or the half million fans of Tesla’s Facebook page who don’t own a Tesla. Or consider companies where employees use their own devices or download their own software until IT purchases the enterprise version for the entire company. In today’s digital age, advocates aren’t necessarily customers. Marketers who think that advocacy comes after purchase are missing the new world of social influence.

Antonio Lucio, Chief Brand Officer at Visa, believes the solution is to shift the focus from the transaction to the relationship.  After exploring the Customer Decision Journey, his team developed what they call a Customer Engagement Journey.  In this model, transactions occur in the context of the relationship rather relationships in the context of the transaction.

As an example, consider a real world journey of a family’s trip from the U.S. to Mexico. Visa has mapped out the entire experience, from where the family gets ideas on where to go (TripAdvisor), to how they gather input from friends (Facebook), to how they pay for their cab (cash from an ATM) or hotel (credit card), to how they share photos of their trip with friends back home (Instagram). Only a few of these situations are opportunities for transactions, but they are all opportunities for relationship. “When you change from decision to engagement,” Antonio says, “you change the entire model.”

Market trends suggest the mismatch will only widen between customers’ actual experiences and the models of the funnel or Customer Decision Journey.  One key trend is the integration of marketing into the product itself.  The funnel presumes that marketing is separate from the product.  But for digital products like games, entertainment, and software-as-a-service, the marketing is built right into the product.  Examples include the iTunes store and Sales force’s App Exchange.

Caroline Donahue, CMO at Intuit, oversees numerous web-based products for which “the product and the marketing become one thing.”  The funnel changes because “with cross-sell and up-sell, you move from awareness to action instantaneously.” Instead of a Customer Decision Journey, her approach might best be described as a User Experience Journey into which opportunities for transactions are thoughtfully embedded.

Google shares a similar view, taking the fusion of product and marketing one step further. Arjan Dijk, the company’s Vice President for Global Small Business Marketing, believes products should be designed to market themselves. For Google, the question is not “how can we market this product?” but “which products deserve marketing?” Marketing isn’t about “pushing people’s thoughts and actions. It’s about amplification, helping what’s already happening grow faster.”

So where do we go from here?  The funnel and Customer Decision Journey aren’t going away. 

 They are useful models, and will continue to be helpful in certain contexts.  But marketing today requires a new mental map to navigate a changing landscape. We need a model that informs marketers how to enable and empower, not just persuade and promote.  There are a variety of alternatives including journey, orbit, relationship, and experience.

Whatever model you choose, what’s most important is that it addresses: first, the multi-dimensional nature of social influence; second, non-linear paths to purchase; third, the role of advocates who aren’t customers; and fourth, the shift to ongoing relationships beyond individual transactions.

View at the original source

How Do You Scale Excellence? 05-07

 How Do You Scale Excellence?

Two Stanford professors discuss their new book, Scaling Up Excellence, which reveals how the best leaders and teams develop, spread, and instill the right growth mindset in their organization.

Eight years ago, over dinner and a bottle of wine,Hayagreeva (Huggy) Rao and Robert Sutton realized they needed better answers for the students of a Stanford management education program, Customer-Focused Innovation, they were running. The business executives appreciated what the pair had to say about reducing bureaucracy in an organization and enabling creativity, but invariably asked, “How do we scale this?”

So, the longtime collaborators set out to find the answer. For more than seven years, they interviewed business leaders, reviewed research, and studied and conducted case studies about the mindset and strategies companies can use to spread excellence within an organization. The result is their book, Scaling Up Excellence: Getting to More Without Settling for Less, which will be published in February.

We recently sat down with the pair to discuss key ideas from the book. Excerpts:

You begin your book by saying that companies that want to spread excellence have the “problem of more.” What exactly do you mean by that?

Rao: To put it simply, the problem of “more” is that organizations have pockets of goodness, and what you want is more of the goodness. At the same time that you’re adding good — by adding pockets or expanding them — you want to make sure you get rid of the bad. So, the problem of “more” is the problem — the challenge — of proliferating goodness.

Sutton: And the word “problem” is important because when things get bigger or you spread them farther, not everything that happens is good. There's this notion of growth and progress in America, that everything gets better as it gets bigger. But it’s a messy process. There will be things that annoy you. And the more you want to scale, probably, the more you have to suffer personally, which is not something I think leaders want to hear.

So how do leaders actually get through that?

Rao: You can have the illusion of drawing up nice little organizational charts and figuring out what growth looks like, but they’re all things on paper. In reality you need both story and structure. What I mean by story is lofty, inspirational messaging about excellence. But you also need the structure — the plumbing, if you will — the unglamorous parts of scaling. If you don’t have both of those, you’re never going to get anywhere.

Sutton: There’s always this challenge, as you get bigger, about structure. If people are telling the same story, you can have less bureaucracy, less micromanagement. But you do need some authority. And people like Twitter’s Chris Fry and Steve Greene, who grew Salesforce from 40 to 600 people, and venture capitalist Ben Horowitz — all of them make this argument that you should have a little less structure than you think you need.

 Then, wait for things to break a little bit as a sign that you should add just a little bit more. Greene called this light structure “running a little bit hot.” If you’re a little bit too heavy, it feels like you’re walking in muck. And, if you’re way too light, things fall apart. So, the ideal condition for scaling is that little things should be breaking all the time, but not the big things.

You make the point that if you’re adding new processes or people, naturally you will need to find ways to subtract — or stop doing — other things. How do you do that?

Rao: That’s what we refer to as “cognitive load.” If the load is too great, there’s a coordination circus. If I’ve got to run an idea by a bunch of people in order to do some single thing, I’m going to give up because it’s too much effort. When Bob and I teach executives, sometimes we say, “Hey, how about having a rule that says ‘If you want to have a meeting, kill an existing meeting.’” Now, that sounds quite obvious, but they say, “Really?” They haven’t even questioned the idea that meetings are to be added, that calendars are to be monopolized. It’s no wonder in a large company you feel like a victim.

Sutton: Now, that’s not true of all companies. Take Apple and Wal-Mart — those are two companies where they have a culture of small teams, so they have fewer meetings and people actually are doing the work most of the time. A lot of this depends on leadership and focus. The question is: “Where is your focus?”

Is there any common thread among organizations that are successful at spreading excellence?

Rao: They tend to “connect and cascade,” as we call it. You connect people so that you cascade the right behaviors throughout the organization. The real problem of scaling excellence is ignorance. What is an excellent organization? One that doesn't repeat the same mistakes. And when do you repeat mistakes? When the connections inside organizations are weak or atrophied. If people aren't connecting, your ignorance multiplies.

Sutton: And I think the important part for senior leaders is to find a way that people can share information in reasonably efficient ways. We talk about many ways in the book, but I really like the example of Salesforce under Steve Greene and Chris Fry — they had a policy where engineers were free to change jobs within the company. So, every four months they’d have an internal job fair, like a bazaar with booths, where people would walk around and learn about what other teams were working on. The policy had many advantages, but one was a higher collective understanding of how the piece of code they’re working on fit into the overall whole.

What’s are the implications of the size of your organization on a scaling effort — going from, say, two to 20 people versus 2,000 to 10,000?

Rao: Going from two to 20, you're probably all still sitting in the same room. It’s just a bigger room.

Sutton: Then there’s a next level, 150 to 200, where you can no longer recognize all the names and faces, and up from there. The thing that companies that were successful at scaling all have in common — regardless of size — is that they were thinking of scaling as spreading a mindset, not a footprint. One person who demonstrates this well is John Lilly, the former CEO of Mozilla, who grew the company from 12 to about 500 people. 

John told us, “When we were 40 or 50 people, I was always changing my mind, but as the company got bigger I had to stifle myself.” He had to be sure he said the same thing over and over so that people wouldn’t get conflicting messages. This is true all the way up to truly huge companies. As Huggy says, if you've all got the same poetry in your head — these mantras like “The Customer is Boss” — they actually drive a bunch of decisions. It lets everybody be on the same page and know where they’re heading.

Rao: The other thing I want to add is when you become larger, what’s unsaid also increases in an organization. The phrase we like to use is: “Smart people inside large organizations become dumb.” They become mute. And so the real problem for a large company is to figure out what’s not being said. Because if you only make decisions on the basis of what's being said, you can go off track pretty quickly.

So how do you build the right mix of people for positive growth?

Rao: This is a big, big thing. Scaling doesn’t just require stars. It requires Sherpas as well — the people who get you to the summit each day. Now, in order to make sure that the Sherpas are taken care of, you should hire managers who are prone to feeling guilty. This idea is based on actual research done by a PhD student and Frank Flynn, a colleague here at Stanford.

I think about the U.S. army general in the Korean War that we talk about in the book, Matthew Ridgway. He says, “The hard decisions are not the ones you make in the heat of battle.” A lot of people can do that. 

The hard part is actually sitting in a meeting and speaking your mind about a bad idea that’s going to put thousands of lives in jeopardy — and convincing the decision makers that it’s a stupid idea. The kinds of people who are going to do that are people who put the interests of others above their own.

Sutton: It’s interesting, the people who are really good at getting things done, they’re not just optimists. In fact, research shows they have high positive and high negative affect, which means they’re really optimistic and confident things will turn out in the end, but they’re really, really worried about every little detail and how it’s going to screw things up.

The two things I would add are that you should make sure to have as many women as possible, because the more men you have in a group, the dumber it gets, controlling for their IQ. There’s actually very good evidence of that. And, that you want people with a sense of accountability, who feel like “I own the place, and the place owns me.” They will push themselves and each other, feel obligated to teach and to learn.

You mention that some mistakes are part of any scaling effort. But what are the kinds of mistakes that can cause failure?

Sutton: When we looked at cases where scaling failed, they seemed to have the trifecta of illusion, incompetence, and impatience; this idea of “We’re going to do it all at once, we don’t have time to slow down and do it right. But we’re so great, we can do it.” 

You can see it, for example, in what the Obama administration did with their healthcare rollout. Apparently, the guy in charge was some career bureaucrat who didn’t really understand how to do it but was a good politician, so he was competent at the wrong thing. But that creates a scenario where you’re turning other people incompetent.

So, how do you start a successful scaling effort?

Rao: One thing to keep in mind is that scaling doesn't mean “Waiting for Godot” — you know, wait for the new boss, wait for a new opportunity, wait for new technology. In reality, you better do something. One good way to develop a plan is to do a pre-mortem: Take a team of people, and get half of them to imagine the plan has been put in motion and failed terribly. 

Then write a story of why that happened. And get the other half to imagine that it succeeded, and write a story of how that happened. The advantage of this is you get more of the unsaid said. You can actually make sure those small details that bite you don’t get in the way. Often, scaling doesn't work because the mistakes you make early on aren’t caught until it's too late.

Sutton: We talked to a top executive who turned around the largest company in Australia, who had the top 100 or so folks write him a 2-page memo about what they should do to turn around the organization. And he said, “I just talked to each person for an hour, and took the best ideas.” 

That said, we also found a single person who wanted to spread an idea, and so began by redecorating her cubicle and starting small, informal training sessions for colleagues. The bottom line is: In every case of successful scaling, you start where you are with what you have.

View at the original source

Is Blended Learning the Magic Bullet for Struggling Students? 05-03

Is Blended Learning the Magic Bullet for Struggling Students?

The title is facetious, but I'm afraid the sentiment can be found throughout the US now. Blended learning refers to using a mix of face-to-face instructional methods with computer-mediated learning, such as a mix of teacher instruction and computer learning programs, or the "flipped" classroom. The primacy of technology in our everyday lives, combined with the rise of charter schools that use "individual learning modules" (i.e. children sitting at computers working on reading or math skills at different rates) has created a sentiment in some circles that if we just had KhanAcademy/Gigi math/RAZ-Kids etc in our classrooms (programs that target reading or math instruction to a student's particular area of need and let him or her practice independently) we would see a meteoric rise in skills.

The problem is that this sentiment just isn't true. The New York Times published a review of a NAEP study that shows that not only is the case for blended learning questionable, but we see the same gap in instructional methods for low-income students vs. other students that we do in other areas of schooling. Low-income students were more likely to use computers for basic drill activities, vs. more cognitively rich activities that other students engaged in. Is blended learning just a new way to reinforce an old status quo?

Don't get me wrong--I love computers, I love technology, and I love using both in the classroom. In my school we use animoto to create slideshows, ALEKS to help remediate/reinforce/extend math skills for students at my school, and tried out Tynker to help students learn basic coding skills, and Google Docs to collaborate around writing and data collection.

My beef with blended learning is the idea that it's an easy, people-proof (teacher-proof really) way to improve student learning. I don't know who actually works with students who thinks we can plunk them in front of computers and have them magically overcome all obstacles. We still need collaboration and instruction, especially to help students with something they are struggling with. And we need to rethink how technology can help engage and illuminate, as opposed to just provide practice. 

Dan Meyer, a doctoral student at Stanford University, shares his frustration with the low-level tasks often presented on computer learning programs inWhat Silicon Valley Gets Wrong About Math Education Again and Again. Contrast that with the way technology can engage and provide deeper levels of thinking, such these 3-act problems (also by Meyers).

I hope we use technology more and more in the future. Education could be enhanced by a real change-up in the way we deliver instruction and experience learning, and technology can be part of that. But I'm skeptical that individualized cubicles with students clicking answers to questions on their own is really the way we're going to revolutionize the learning process. Those programs have their part to play (practice is important!) but they're far from the whole story.

View at the original source

Please also read Shyam's latest article on student learning

How to lead the student from the wrong answer to the correct one? 05-11

The Best Way To Remember Something? Take Notes By hand.05-03

The Best Way To Remember Something? Take Notes By  hand. 

PEOPLE WHO WRITE NOTES BY HAND REMEMBER THE CONTENT OF MEETINGS AND LECTURES BETTER THAN THOSE WHO USE LAPTOPS, NEW RESEARCH SHOWS.

Headed into an important meeting? Grab a pen. Taking notes longhand will help you remember information better than typing them out, according to new research from a pair of psychologists from Princeton University and UCLA. 

The study, published in the journal Psychological Science, compared how well more than 300 students retained information after taking notes on 15-minute TED Talks either by hand or with a laptop. Across three different experiments, the researchers found that taking notes with a laptop can be detrimental to learning. Both groups performed about the same when recalling facts from the lectures half an hour later, but longhand note-takers were much better at recalling concepts.

Handwritten note via Shutterstock

These results of only a few hundred students paid to watch lectures in the lab might not exactly translate in the real world, but they do suggest laptops might not be great for retaining information. The researchers postulate that the effect might stem from the fact that while typing, it's easy to write down verbatim what the speaker is saying, without really thinking about it. Taking notes by hand requires listening to the information being said, processing it and then summarizing it in your own words. 

The students who took notes on laptops tended to write more words than those who wrote by hand, but when given the chance to study their notes afterward, all that extra content didn't help much--students who wrote their notes longhand performed better on a test a week later, both on questions of conceptual understanding and the factual content.

"It may be that longhand note takers engage in more processing than laptop note takers, thus selecting more important information to include in their notes, which enables them to study this content more efficiently," the researchers write.

View at the original source

How Do We Keep Students Engaged in Learning Science? 05-03

How Do We Keep Students Engaged in Learning Science?STEM

Children come to school with curiosity and questions about their world. As educators, we need to keep children curious and excited throughout their schooling—encouraging them to explore their environment, ask questions, make predictions about what they think will happen and why, and test those predictions. Toward this end, states have collaborated to develop a completely new set of K-12 science education standards, called the Next Generation Science Standards (NGSS).

While the NGSS expressly do not dictate to teachers exactly what or how they have to teach or how to assess students’ progress, they provide an opportunity for us to revamp learning and assessment to foster students’ natural curiosity. Prior generations of standards separated what specific content students should know from skills they should be able to perform, leaving the integration of these implicit. In contrast, the NGSS emphasize using scientific practices to develop and apply scientific ideas and paying closer attention to how we build understanding within and across the disciplines of science over time. Only by involving learners in doing science can we connect to their intrinsic interests in how the world works.

With respect to assessment, perfect examples of how to measure this kind of highly contextualized, knowledge-in-use learning don’t exist yet. Most current tests emphasize factual knowledge or require that students participate in “inquiry” without deeply incorporating disciplinary core ideas or requiring students to connect relevant ideas across the science disciplines. We need new models for science assessment that promote engagement in the disciplinary practices of science, and this is what my team at SRI Education is aiming to do.

We are excited to participate in one of the first projects to take on developing the assessments that measure three-dimensional learning. SRI’s Center for Technology in Learning, together with the Learning Sciences Research Institute at the University of Illinois at Chicago, CREATE for STEM Institute at Michigan State University, and the Concord Consortium, were recently awarded a $2.9 million grant from the National Science Foundation to develop a new system of classroom assessments that teachers can use formatively to make instructional decisions.

We are focusing on middle school physical science as a starting point and as a proof of concept to design an approach and a model of what three-dimensional assessments can look like for the NGSS. Our work is a co-design effort: we have included experts in assessment design, science, science education and statistics (to see that we have valid and reliable information about students). We have also notably included teachers as co-designers to help ground what we do and make sure it will work for their students.

In contrast to benchmark tests given maybe quarterly, we are talking about developing assessments and practices that are more an ongoing part of the learning process. We hope our assessments give teachers the tools and resources they need to adapt and adjust their instruction. Thus, tasks will be technology-based to provide teachers more rapid feedback about their students’ thinking. 

We are aiming to include visualizations that, for example, allow students to construct models of interactions among atoms to explain how mass is conserved in a chemical reaction. We also will be thinking about how we can create tasks that connect to and build from students’ cultural and linguistic backgrounds. In doing so, our aim is that these assessments will provide the context for rich dialogue and conversation among teachers and students and that makes science relevant for students.

More soon as we continue on our venture in assessment design!

View at the original source

Please also read Shyam's latest article

How to lead the student from the wrong answer to the correct one? 05-11

Innovation or adaption? 05-03

Innovation or adaption?

I attended a corporate function recently uEntrender the theme of 'Innovation'. It was attended by innovation professionals, corporate execs and a range of participants from other innovation-related sectors.

The discussions were very interesting but I was struck by one single fact: the ambiguous and generic use of the term 'innovation' - and the ramifications of this ambiguity on the corporation.

I am troubled by this for a range of reasons.

Ever since Hamel and Prahalad made 'innovation' the catch-cry of the times for corporations a few decades ago, corporations have interpreted 'innovation' to mean 'reinventing themselves'. That's OK when it's relevant but it doesn't apply to everyone.

The concept of innovation generally refers to the 'creation of something that didn't exist previously'. This is most evident, for example in the sciences and in some technological sectors. The bionic eye and ear initiatives, wireless technologies, solar power, and so on are excellent examples of creating something new.

In the corporate context this is rare however. Corporate interpretation of 'innovation' generally means 'improvement' or 'differentiation'. Both are legitimate but neither necessarily (and rarely) mean 'creating something new.'

Of course some companies need to innovate in their products and services area, but often it's a matter of incorporating features that competitors already offer, rather than of 'creation'.

Nearly everything in the corporate space already exists somewhere. Therefore for a corporation to improve or differentiate, it is often a matter of knowing what exists and being able to apply it efficiently and effectively for its own purposes. It does not therefore mean that the corporation must create something new.

This is particularly relevant because:

1. Creation is different to applying an existing technology.
2. Creation implies cost, time and effort.
3. Creation carries financial, time, effort and reputational and corporate risk.
4. Many managers and directors don't know the difference between the two concepts.

Many of the speakers at the recent session implied that the difficulties with adopting innovation is of embedding innovation as a cultural attribute. Some of corporate speakers therefore appeared to be trying to make everyone in their organisation 'innovative'. Firstly, not everyoneneeds to be innovative. Secondly, not everyone can be innovative (even if you adopt the principles of Kaizan). Thirdly, this is all about effective change and culture management.

Ultimately innovation, regardless of one's interpretation, should never be conducted without a robust and defensible business case that demonstrates the value such innovation would deliver toward the enhancement of the host's organisational objectives.

View at the original source