Experiments – Running Training Like A Business 2.0

Failure: It is all in the definition

The new business context requires L&D organizations to experiment, innovate and transform. However, this requires trying out new ways of structuring the learning organization, designing learning products, delivering learning and partnering with business. Often, L&D leaders shy away from introducing these big, bold new ideas. The key obstacle is that many mature organizations treat failure as a weakness. However, successful startups have adopted a different point of view. For them, failure is nothing more than an evidence based correction to an assumption- better to know sooner, rather than later.

Too often, L&D’s limited resources are misspent. As Peter Drucker has been quoted as saying, “There is nothing so useless as doing efficiently that which should not be done at all.” It is from this perspective that failure becomes a virtue. None of us likes to fail. Nevertheless, in the context of operating successfully in today’s business environment, not only is failure acceptable… it is required.

“Fail often. So, you can succeed sooner.”

– Tom Kelley, Ideo partner

Let us begin by acknowledging that experiments can be difficult in a large company. Unlike startups there is often little ability to take a let’s-try-some- things-and-see-what-sticks attitude. This can challenge any organization tasked with innovating within an established company. In times to come, we are hopeful that this limiting factor will lessen as more companies, of all sizes, recognize the need for increased innovation in all aspects of the business.

However, for now, L&D organizations in mature companies may continue to relate to failure with fear. Having worked so hard to build credibility, L&D leaders might consider failure as a step backwards. I strongly recommend that L&D leaders change their perspective on failure. By properly recognizing failures, L&D’s successes become even more powerful in generating executive, manager, and learner buy-in. Two keys to this are turning failures into learnings and being transparent.

When Viewed in the Right Light…

Steve Blank is a retired serial entrepreneur now teaching entrepreneurship at UC Berkeley, Stanford, and Columbia. He has studied what he calls the stages of startup failure.

Stage 1: Shock and Surprise

Stage 2: Denial

Stage 3: Anger and Blame

Stage 4: Depression

Stage 5: Acceptance

Stage 6: Insight and Change

While failure for L&D is rarely as final as it is for startups, the stages remain relevant, and both startups and L&D must push to achieve stage 6 as quickly as possible. What others refer to as a ‘failure’ is better described as a ‘major lesson.’ This lesson is one that can only be learned by trying. L&D must turn its failures into learnings. This is the only way by which the odds of delivering unmistakable value next time improve. According to Nick Casado, a computer networking star who sold his startup for over a billion dollars, the true skill is “to learn to embrace failure — not only embrace failure, get good at it, and by that, I mean get back up, apply what you’ve learned, and hit reset.”

Show Your Scars

The final principle of handling failures well is to be transparent. This transparency extends to the L&D team, stakeholders, and even other learning professionals. For startups, this sharing of ‘major lessons’ has become part of the culture. CB Insights, a research firm “that helps corporations guess less and win more” has amassed and shared over 200 startups’ “post mortems” in the hope that startups in the future will make new mistakes, not repeat old ones.

L&D conferences are full of best practice case studies and chest thumping showcases. L&D leaders must participate in these conferences, not just to listen to others’ successes, but also to fearlessly acknowledge their own failures. How about seminars on “10 ways I screwed up our roll out” or “3 easy ways for upsetting your executive sponsor”.

Go Ahead and Try

Experimentation is a fundamental activity for startups. Startups are leaping into a great unknown. Does anybody want to use my product? Will anybody pay for my product? How are we going to build this product? Startups are a pile of assumptions.

The goal of successful startups is to, as quickly as possible, prove or disprove assumptions and turn them into facts. They do this through experiments. In addition to the value of turning assumptions into facts, experiments are also key to effectively managing resources. They can help minimize the risk of investing too many resources into a product whose need or value has not been validated.

Experiments are simply a formal process for data collection. Why formal? Without the formality, experiments often produce less data, the wrong data, or even worse, no actionable data! When thinking about an experiment, startups must be able to clearly answer the following questions:

What we believe (our assumption).

What we will do to verify our assumption (our actions).

What we will measure (our metrics).

What our measurement results need to be if we are right (our expectations).

Nothing is perfect and innovation only comes from new experiments. With so much newness occurring every day, if L&D is not allowed to conduct some experiments of its own it will be forever behind the needs of its customers. Experiments, by definition, have an unknown outcome. Therefore, while L&D can’t know the outcome, it must know the parameters of the experiment and be able to work with the business to set the proper expectations. These expectations must be understood to get the most out of every experiment.

With a mutual understanding of the goals, structure, anticipated return and resources requirements, an experiment’s business sponsor can make a reasoned decision regarding participation in the experiment. Setting expectations is about knowing the risk and understanding that the potential reward is critical for experiments to be accepted in organizations unfamiliar with risk-taking.

For L&D, experiments can be used on any number of aspects of their organization. Content is the obvious one. I also believe that those organizations that adopt this approach in areas like process and people will see great return.

I recognize that a scientific approach can only take you so far. There are many factors that could influence the results of an experiment and there is not enough time or resources to prove everything. However, experimental results filtered through the experience and knowledge of the L&D organization can greatly increase the confidence level in any decision.

Project Dragonfly

Last fall a client asked me to write a quick brief on the book Range. With permission from the client, I am sharing it here. You can read it below and download it here Dragonfly Concept Design

Enjoy. – j.

named for an insect whose eye contains thousands of “eyes”, giving it a wide range of perspectives from which its brain constructs its reality and actions.

Project Genesis

Sparked by the release of David Epstein’s book Range, the question that launched the quest was crafted as follows:

“Is there a way to predict employee high performance, and therefore company high performance, by examining the degree to which their path to performance (P2P) is in alignment with the principles outlined in Range?”

Contributing questions include:

What performance does business need from its employees?
What is the traditional/current/accepted P2P for employees?
What are the principles that define the Range P2P?
How can we objectively measure the alignment of a company’s current P2P?
What are the barriers to organizations adopting a “Range-driven” approach to talent management?

What performance does business need from its employees?

Today’s business environment has been termed and widely accepted as VUCA (volatile, uncertain, complex, ambiguous.) This environment has placed a premium on organizations that are:

Agile – able to quickly deploy and redeploy human capital to emerging needs and opportunities.
Experimental – have a culture and resources able to conceive, test and iterate of new hypotheses.
Innovative – ability to generate and execute new ideas in order to capture business opportunities.
Data powered – human capital is augmented by data for management (ie. performance tracking, succession planning) and the delivery of the above capabilities.

Today’s business environment is as being what Robin Hogarth calls a “wicked domain.” Wicked domains are defined as having problems that are not readily computable and have feedback loops that are long and may not provide accurate feedback. The answers to wicked problems are unknown at the outset and need to be created. Examples of wicked domains include improvisational jazz, and cancer research.

Net: Today’s business environment requires a P2P that makes human capital “wicked smart” (Boston joke)

What is the traditional/current/accepted P2P for employees?

Today’s development paths generally follow two paths depending n the employee. Designated high potential employees are often provided a wide ranging development path early on in their careers. Business units rotations, projects and even mentorship expose this employee to all aspects of the company as a means of preparing them for future leadership positions. This cohort is typically extremely small relative to the employee base.

The second path serves the typical employee. On this path employes, who have been hired for a specific domain experience are funneled into deeper knowledge of that domain. Sales people receive sales and product training, an operations employee may receive task management and process training wile operators might attend equipment and safety courses. These trainings, limited to a single domain are associated with the assumption of a kind domain and problems.

Kind problems are domain-constrained with tight and accurate feedback loops. Unlike the wicked variety, answers to kind problems are known and simply need to be found. Kind doesn’t mean easy, a sport can be kind because you quickly know if you executed the correct stroke. Examples of kind domains include classical music, hernia surgery and chess. Kind domains are are often the best targets for automation.

Range principles, where existing, are often found in the initial hiring process. In 1991 David Guest introduced the concept of t-shaped skills. The vertical bar of the T refers to expert knowledge and experience in a particular area, while the top of the T refers to an ability to collaborate with experts in other disciplines and a willingness to use the knowledge gained from this collaboration. This concept was further popularised by Tim Brown, CEO of design firm IDEO. While the concept was seen to have value and gained momentum with HR the concept of developing t-shaped employees never took hold. The priority for most recruiting and promotions involves an emphasis on the vertical domain of the individual.

T-shaped employees provide companies with increased talent agility and mobility. Having an agile workforce can spell the difference between being an industry leader or falling behind. PwC reports that when businesses have development programs that increase agility, 86 percent respond rapidly to changes in the business environment. Without these kinds of programs, only about half do. A Forbes article from earlier this year stated that talent mobility enables organizations to rapidly adapt to changing environments, with the ability to deploy and move key skills across projects, across the business and across borders when needed. Mobility provides avenues for staff to progress and evolve within an organization, and can lead to 30% better processes and 23% more productivity.

Epstein captures it this way. “Facing uncertain environments and wicked problems, breadth of experience is invaluable. Facing kind problems, narrow specialization can be remarkably efficient. The problem is that we often expect the hyperspecialist, because of their expertise in a narrow area, to magically be able to extend their skills to wicked problems. The results can be disastrous.”

Net: Today’s P2P does not fit business to a “T” (I am on a roll)

What are the principles that define the Range P2P?

First it is important to acknowledge, as Epstein does multiple times in the book, that while Range-enabled generalists are critical to business success the value of specialists is not diminished. A multitude of examples are provided showing how generalists draw on the deep expertise of specialists in order to achieve the results delivered. Distilling the book’s insights and translating them to corporate talent management creates three levels of guidance. The first contains the characteristics of a Rangey(?) path-to-performance (rP2P). The second, is guidance on keys to rangey teams. The final level is focused on the organizations itself.

rP2P

The goal of the rP2P is to build polymaths. Polymaths differ from T-shaped employees in that the emphasis is on the horizontal dimension. A polymath’s breadth is greater than T-shaped human capital while their vertical depth may be less than traditional T-shapes. The polymath’s superpower comes from a range of transferable thinking skills (conceptual, computational, lateral, and ambidextrous for example) that allow for innovative problem solving across multiple domains. It also includes more tangible skills such as communication, collaboration and anticipatory competencies that drive higher value solutions. The final element of a range-y employee is attitudinal with value-adding polymaths displaying active open mindedness and scientific curiosity.

The first two principles associated with a Range-aligned P2P focus on the structure and focus of the developmental path. The rP2P contains:

A sampling period – This is characterized by what experts often call “unstructured play.” This feature of the developmental path allows for individuals to experiment in domains other than their own. In the corporate environment this may be a rotational program or project-based. The unstructured element forces participants to improvise first before learning existing rules. This is analogous to the way in which humans learn language. We learn the sounds first before we learn the rules of grammar. This element should be designed to provide the participant with experiences that allow them to better understand alternative domains giving them knowledge of resources that may be valuable when facing later challenges as well as exposure and personal knowledge of their interest and proclivity for other areas.
Mechanisms for improving “match quality” – The rP2P should include opportunities for individuals and organizations to re-deploy individuals to domains better suited to their skills, interests, and proclivities. Short-term assignments, internal internships and project participation can be used to serve this purpose. Up or out development paths serve neither the individual or organization.

The final two principles associated with a Range-aligned P2P focus on the content and presentation of that content on the developmental path. The rP2P contains:

Flexible content – Flexible content is content that is both sticky (retained over the long-term) and broadly applicable. Retention of rP2P content is driven by two key factors. The first is the use of testing. rP2P test questions are connection making in nature versus procedural. Testing should focus on cross domain application, pattern recognition, categorization and decision making. Procedural questions such as “what are the four steps in handling a customer objection?” are minimized in far of questions such as, “what other uses for the customer objection handling process are there?” Stickiness is also enhanced by the use of spacing. Delaying the testing process forces the knowledge to be placed into long-term memory. End of class assessments are less indicative of future performance than follow up assessments given at a later date.
Difficult learning experiences – Learning should include what Nate Kornell calls “desirable difficulties.” These productive difficulties include creating a generational effect, where participants produce their own answers exclusive of guidance. Learners benefit greatly from this self-reliant process even if the answer generated is incorrect. Instructors should also be creating environments in which learners struggle. This may include problems beyond learners’ capabilities, mixing multiple areas of new knowledge together to prohibit “block” learning (aka memorization) and assessments in which few learners achieve a passing score. While this often results in lower instructor/session ratings from participants it has been shown to have significant long-term benefits in retention and performance.

Rangey Groups

While the book focuses primarily on the individual, Epstein highlights two characteristics of high performing groups.

Diverse – High performing groups included a wide variety of participants. This includes range in:
- Geography/World view
- Demographic
- Experience in domain from novice to expert
- Domain, but still polymaths not a collection of specialists
Porous boundaries – Groups that performed well were also not walled off from the rest of the organization. Groups frequently showed improved value when they were able to reach out to specialists across the organization and even outside the organization.

Rangey Organizations

Example such as 3M are cited in the book as examples of an organization that supports its range-y individuals and teams. From creating an internal award for innovation to the ways it allows individuals to follow their passions (increased match quality) 3M regularly produces significant innovations across a wide range of domains. Epstein touches on a few organizational keys.

Culture – 3M’s internal award and its talent management approach are operational examples of a culture that sees the value of supporting its rangey employees. By celebrating, facilitating and empowering range, 3M has created a culture that makes it values more than just an annual statement cliche.
Long-term focus – Because range often does not show immediate results the organizations that embrace it must have a longer view. Think of the innovations that emerged from Amazon and the newly approved LTSE (long-term stock exchange) that clearly states that “Companies that operate with a long-term mindset tend to outperform their peers over time. But going public can pressure even the most visionary founder into a short-term mindset.”
Risk tolerant – Creating new solutions that work often means finding many more that don’t. Acceptance, even encouragement of failure, and the adoption of an experimental scientific mindset are cornerstones of organizations that deliver higher performance over the long-term.

Net: The principles of Range have implications for a number of areas in talent management including; candidate selection, onboarding, development, succession planning and leadership. In order to drive success in today’s wicked environment organizations, and talent management functions that support them, must become an integrated farm (egg-to-soup) for free-range talent. (the roll continues)

How can we objectively measure the alignment of a company’s current P2P?

Standard measures for HR practices are in limited supply and often not publicly available. The table below captures some initial thoughts regarding potential metrics/proxies for the various elements of range. Primary research in collaboration with one or more of the partner listed at the end of this section and/or analyst-like interviews focused on HR leaders may also prove useful in the development and testing of a quantitative range “score”.

	Dimension	Metric/Area of Inquiry
rP2P	Sampling	Onboarding process Use of project assignments Cross domain rotations
	Match Quality	Internal lateral transfers versus upward promotions Use of project assignments Cross domain rotations Former employees now working in another domain Employee satisfaction Employee retention Employee engagement Employee development plans
	Flexible Content	Learning experience (LXP) design LXP satisfaction scores Cross functional applicability of LXP (multi-audience) Layoffs
	Difficult Learning XP	Learning experience (LXP) design LXP assessment timing LXP pre/post assessments Instructor ratings Post-LXP performance reviews
rGroup	Diversity	Employee census Recruitment procedures Job requirements (narrow/broad)
	Porous Boundaries	Resource sharing policies Use of outside experts/consultants
rOrg	Culture	Employee survey Leadership characteristics
	Long-term Focus	Strategic plan R&D spend

What are the barriers to organizations adopting a “Range-driven” approach to talent management?

While the value of rangey practices are widely documented, adoption of the proven practices face a number of challenges. Some of these challenges are documented below. This list is not meant to be all-inclusive as it excludes any number of organizational structure, compensation and process barriers.

Slow thinking requires a longer payback period – As is the case in spacing of testing where immediate results may be poor while results produced further out exceed the current P2P so it is with much of the range principles. Organizations and other stakeholders (HR, Management) may find it difficult to “stay the course” without near-term ROI.
Spacing assessments is less rewarding to learners – Individuals often do not receive the immediate gratification of progress and success when participating in a rP2P. Without buy-in from individuals to the range approach and the safety of knowing that their immediate performance will not be seen as a negative by the broader organization individuals may not actively engage in the path.
The result of increased match quality (job switching) can make individuals feel like they are falling behind – Job switching, often the result of seeking improved match quality may leave individuals feeling behind their peers. The concept of sunk cost, time and energy committed to a pre-switch domain, may make individuals and organizations reluctant to follow through with match optimization.
Misaligned metrics (learner satisfaction, post-assessment scoring) reward non-rangey principles – Current P2P metrics are short-term oriented and in many cases run contrary to effective implementation of range principles. Changes to how rP2P facilitators (recruitment, development, management) are necessary in order to properly measure range-y progress.
Lack of obvious linkage to near-term business results – Rangey performance often delivers value in unexpected areas/was. Measurement and management of impact on wicked problems must be different than on kind ones. New product development (connected/wicked) should have a set of metrics distinct from new store openings (procedural/kind).
Short-term perception of poor performance – Public company cadence (quarterly) may inhibit the longer term investment in a rP2P.

Net: Lots of fences between the herd of sheep and the range. “Just let me know if you wanna go to that home out on the range. They got a lot of nice girls.” – ZZ Top

Final Thoughts

After reviewing the publicly available research used for the compilation of this document I believe the following to be true:

“Wicked” is an accurate description for an ever increasing portion of the business environments and the “kind” portion will increasing face the pressures of automation and commoditization.
The companies that win in a wicked world win exhibit a significantly higher degree of “range”.
Adoption of range positive dimensions require significant change for both the individual seeking to increase their personal range and companies seeking to provide a range-enabling organization.
The current structure for talent development and management is designed for the creation of specialists and ill-prepared for a shift to polymath focused human capital.
With additional work there exists an opportunity to capture or create qualitative and quantitative metrics for assessing range on both the individual and organization levels.

DOWNLOAD BRIEF: Dragonfly Concept Design

Open Source

I just re-read Walter Isaacson’s “The Innovators”, a wonderful history of the people and events that made the digital age possible. In it he describes the creation of the software industry and the early formation of two camps. One thought that software should be free, describing the hobbyists that openly shared code between another. He even noted that that Wozniack’s early schematics for what would become the core of the Apple 1 computer were given away free by Woz at community meetings. The second camp, characterized best by Bill Gates, saw the protection of intellectual property as key to supporting future innovation.

For the open community, allowing innovations to be shared allowed for rapid adoption and accelerated advancement. Innovators didn’t need to recreate a solution that had already been found and could instead focus on building upon it. Also, by adopting an open approach an entire community became an extension to the development team. The opposing camp felt that without compensation innovation could not be supported. I see both sides and as a former interim-CEO for a music technology company, an industry where rights are front and center, feel strongly that the creator has the right to choose, and that choice must be respected.

As I came to complete the book Running Training Like a Startup I realized that I now had to make this choice. I could publish the book, thereby monetizing my efforts or I could give it away. When I work with startups I often refer back to a presentation on startup success that I saw during the first dotcom run that simply stated CFIMITYM. Cash flow is more important than your mother. Without cash flow, a startups life is on the clock. Many startups, even those so-called “unicorns” that are now going public, often chose another path. Monetization slows growth and adoption. “We can always monetize once we have a million users,” they say. And so, the freemium model, where there is a free tier of services available, was born. Once a user has seen the value of the product, they can upgrade to a paid tier later. When I was at Forum we called this “earning the right” in our sales training.

I believe strongly in the value that Running Training Like a Startup can contribute to the industry. I think that upskilling is the number one challenges facing the world. I believe that more minds, not a select few, will accelerate our industry’s ability to overcome this challenge. I also know that at the pace of business today reinventing the wheel will not cut it. For this reason, I have decided to open source the book. Feel free to download it, share it, discuss it, build on and improve its concepts with your own. I will be doing the same. This blog will continue to document my evolving thoughts on the concepts presented in v1 and I am committed to sharing my learnings with our community. All I ask is that you do the same.

Open source book here