Setting strategy, and sticking to it, can be an arduous climb. Done successfully, it's remarkably satisfying.

The key is having the right tools and knowing how to get the most out of them. In the same vein as rope, ladders, pitons, and gloves are indispensable climbers' tools, logic models are an important tool for both planning and evaluation in the non-profit and social service sectors.

Over the years we have seen examples of strong models being implemented in the strategy and evaluation efforts of different organizations. When they work, program outcomes are enhanced. But we have also seen our share of situations where logic models were treated as some kind of add-on activity to ‘regular’ program or project management. The logic models in these cases were mainly used to construct a graphical depiction of a team’s wishful thinking. In such cases the models were not been anything more than just a piece of presentation material with no impact on strategy.

Starting here, and in four future posts, we’ll lay out some tips that can augment the success of your logic modeling. Should you choose to use some of the excellent material available on the web that can guide you in using the building blocks of logic models (such as 10 Great Resources for Creating a Theory of Change from the Philanthropy411 Blog), our hope is that these tips will enhance the process.

Tip 1. In designing your logic model, remember that it’s a process and it should involve people.

We’ve seen too many examples of attractively designed models that, in the end, pursue goals that were neither necessary nor achievable. Often was the case that the designers focused on the diagram itself, and the different techniques and formulas to draw and construct logic models, instead of looking at the process around building the logic model.

To leverage the strengths that logic models can provide, it is crucial to predicate the design on the input of stakeholders with field-expertise who you involve in uncovering the root causes to problems you seek to solve. In a pluralistic fashion, the stakeholders should help you depict the landscape for the initiative (more on this in Tip 2) in such a way that a program team can target relevant, valid causes to a problem.

Also, include measurement and evaluation in the people-process right away. Involve key people who are inextricably linked to the success of the project you are modeling. For instance, if you have a project to build the capacity of teachers, invite teachers into the logic modeling process to discuss inputs and indicators.

Because designing a logic model is a process, never treat it as a “one and done.” Don’t be trapped by false pressures to develop the best, perfect model in one fell swoop. Because good logic models are derived from good processes, take the time to draw several, to think through different scenarios with your stakeholders, and assess the potential impacts of the different versions. (More on this in the forthcoming tips.)

Next tip: Depicting the landscape to strengthen your logic model

Guidestar has just released the results of a survey entitled "The Effect of the Economy on the Nonprofit Sector: A June 2010 Survey.”

Chuck McLean and Carol Brouwer wrote the report, the purposes of which were “to explore how charitable organizations fared during the first five months of 2010 and to try to gauge the effect of the downturn in the economy on the American nonprofit sector.”

Among the findings, some issues stand out among public charities and grantmaker subsectors.

On the Charity Side

Public charities were asked if total contributions to their organization increased, decreased, or stayed about the same between January 1, 2010, and May 31, 2010, compared to the same period a year earlier. Thirty percent experienced an increase in contributions, while 28 percent remained the same. Forty percent reported a net decrease. McLean and Brouwer then asked, “What factors caused total contributions to decrease?” Forty percent said private foundation grants were smaller and 22 percent said that private foundation grants were discontinued. This in the face of 63 percent of the respondents stating that demand for their organization’s services increased!

Takeaway: charitable organizations faced smaller or discontinued grants from foundations in a climate where demand for their services is up.

On the Grantmaking Side

Now let’s look for a moment at the grantmaking side of the house. Eleven hundred of the 6,508 respondents indicated their organization awards grants, with 580 identifying themselves as private foundation/grantmakers (thus, in the mix of grantmakers we probably have a mix of charitable re-granting organizations in addition to private foundations). Among these 1,100 organizations indicating they give grants, 68 percent indicated that their giving remained the same or increased in the five-month period of January-May 2010 compared to last year. Further, 65 percent indicated that they made no major changes in their grantmaking. A much smaller proportions reporting they either cut back on programs (12 percent) or cut back on payments (8 percent).

Takeaway: Grantmaking, from the perch of the grantmaker, has remained the same. Few grantmakers have cut back on programs or payments. This is interesting in light of the above finding that charities report significant declines in grant funding.

What are the implications of this? There are several, I’m sure. For the purposes of this post I’ll constrain my thoughts to issues of impact and evaluation.

What Could Be Going On?

If I’m leading a charity, I might think the following to myself. “If my grant funding is down, yet trends suggest that grant givers have not changed their overall behavior, who is getting funded, and why?”

Since I’m a strategy and impact guy, my mind tends to turn to thoughts of monitoring and evaluation. It may be the case that funders are paying more attention to charities that understand how to evaluate their day-to-day work in ways that demonstrate true impacts.

We see from the Guidestar report that demand for services is up among charities. This implies that organizations are going to have to be very savvy at bringing their services to scale while not diluting impact. This is challenging if an organization doesn’t know its place in the charity ecosystem, doesn’t have clear outcomes delineated that should result from their services, and lacks a systematic way to track successes and failures. This could explain some of the variance in why foundations and grantmakers don’t report a general dilution in their funding. Perhaps there’s been a shift in giving to organizations they trust to create promised, sought-after impacts.

It’s also possible that funding is shifting within intact programs from charities that are working on the ground to those that are further from the “shop floor” – such as charity service providers, think-tanks, and other capacity building organizations.

What else could be at play here?

A little over four years ago we started to ask people what some of the root causes were to better organizational effectiveness in the foundation and nonprofit sectors within the United States. We talked with personnel at large and small nonprofits, and program staff from small family foundations to large private ones . We also consulted the literature on organizational effectiveness in the nonprofit world. Over time we captured what we heard into a poster size (3 feet x 2 feet) root cause “map” with over 65 boxes containing discrete assertions about the challenges foundations and nonprofits face in attaining organizational effectiveness. The diagram also contains scores of arrows that conceptually connect the assertions.

Early versions of the map were shown to our initial interview respondents from the foundation and nonprofit world (and to just about anyone who would look at it!). Over time we confirmed, disconfirmed, and refuted many issues, ever refining the map. It’s not perfect, but it does reveal an honesty and candor that underscores the passion of the people working in the third sector.

Because the concepts are ever evolving, our aim is to make transparent what we've learned and start a conversation on

• where the most viable opportunities for improvement are, and
• what we should address as a field.

In a series of posts we’ll report on what we’ve learned and ask for your reaction. Part One is about foundations. In Part Two we'll look at nonprofit organizations. We'll also publish the map in an upcoming post so anyone can have a copy of it and see the ecology of effectiveness, as we've captured it, for the sector.

Part One: Foundation Effectiveness

We began this thought experiment with the following assertion:

“We, as a social change community, not as effective as we could be.”

We then asked stakeholders in the field why that’s the case, particularly in foundations.

Four major clusters of root causes evolved from the above assertion. One cluster of reasons focused on the way in which the input from consultants, on which many foundations rely, doesn’t always translate into better overall effectiveness. Another cluster focused on how proposal vetting is not as effective as it could be. A third looked at how program evaluation is not leveraged in the best ways possible. And the fourth cluster, which I’ll discuss more in depth below, notes that foundations are not as strategic as they could be.

Why Aren’t Foundations As Strategic As They Could Be?

Three major areas developed in our analysis as to why foundations are not as strategic as they could be.

The first is no surprise. It's hard to stick to a strategy.

• With the passage of time, leadership and staff attain new knowledge that influences how they view the future.
• The stakes for not sticking to a strategy are not readily evident in the day to day work of foundation staff.
• It's hard to make strategy visible and operational in every step of an organization's day.
• It's hard to deliberately re-focus a strategy.
• Board members may have personal interests that can influence grantmaking.
• The needs of a community change over time.

Another second reason why foundations are not as strategic as they could be is that program portfolios can become populated with projects that may not be mission-related. This can occur when foundations have vague, unattainable or unrealistic goals. There are several reasons why foundations would hold such goals:

• It can be more appealing for a foundation to make small contributions to a big problem rather than completely solving a delimited one
• Foundations and grantees are not necessarily centered on a culture of measurement. After all, with measurement comes responsibility; what one doesn't see one doesn't need to fix.
• Foundations want to leave their options open.
• A foundation's strategy may be illusive and confusing. The contributors to this include
  • An inherent ambivalence about power and control over grantees. Foundations can be reluctant to tell grantees what to do because
    
    • they wish to be detached and objective and not involved in funded projects
    • they don't have staff or resources to support funded projects
    • they have respect for the independence of grantees
  • A lack of a market that drives foundations to be more effective and outcome oriented.
  • A tendency to prefer being detached and objective and not involved in funded projects

A third area that contributes to foundations' difficulty in sticking to strategy is a lack of emphasis on results that lead to a change or impact. A reason offered for this is that the quality of grantee operations may be based on foundation's perception of a grantee's organizational efficiency rather than a grantee's social results.

Next Time: Evaluation, Proposal Vetting, and Consultants and Their Relationship to Foundation Effectiveness

During the last decade the partnership model has really advanced and become central to activities in all sorts of fields. The interagency and public-private nature of broad partnerships naturally provides benefits for implementation of anything from health initiatives to local and regional growth initiatives.

For many years we have been coaching partnerships dealing with rural development and we believe the partnership model will spread even more in existing and into new fields. Successful partnerships obviously open up opportunities that no single actor can accomplish. Below, we mention a couple of the key factors for success.

When reading these tips we would like you to bear in mind one risk with broad partnerships that rarely is acknowledged; while a partnership gets strong through a broad base of key stakeholders the output from the partnership work may turn out to be the “least common denominator”. In critical situations or situations that require very forward-looking, innovative and/or unconventional solutions, a least common denominator might not be enough. In those cases strategic and dedicated interventions may be needed to advance the cause. But more on that in a later post.

The following tips are general for most kind of partnerships and are based on our coaching and evaluation of partnerships.

1. Pinpoint and make explicit the real needs the partnership potentially could address and harmonize every stakeholder’s expectations against those. A shared perspective on the underlying rationales for potential collaboration items is fundamental for a successful partnership.
2. Make sure to implement a process that makes everyone’s voice heard on equal terms. Very often an informal power structure creates an imbalance already from start in a partnership. The first measure for success is to establish a neutral forum.
3. Pinpoint and aim for outcomes that lie close and are concrete. Unite your efforts first around achievable short term outcomes that help the learning and partnership to grow before aiming at long term outcomes or visions. Aiming for concrete short term results is a god way to get the dogs to learn how to pull the sled in the same direction.
4. Make explicit how every stakeholders’ unique strengths contribute to the achievement of the outcomes agreed upon. Being explicit about the assumption on how a program expects to work increases the chance for success.
5. Involve the partnership immediately in evaluation planning and engage stakeholders continuously in the learning process as well as data collection if possible. Reflect collectively on outcomes relative expectations and aims. The shared learning is the biggest boost for enhanced success.

In November we teamed up with Kairos Learning to deliver two online hour-long courses, Building Accountability Into Your Team and Activating Your Program's Strategy and Evaluation.

Below you'll find the recording of the second virtual class,  Activating Your Program's Strategy and Evaluation, which presents an approachable perspective on setting strategy for achieving goals in education, nonprofit and community programs. 

Watch the video to learn:

• A simple yet effective technique for setting strategy and goals for your work
• How to effectively answer the question "Is my project reaching its goals?"
• How to confidently demonstrate to funding sources that your programs are effective

Let us hear your comments on this webinar -- what do you think of this topic? What else should we be covering?

[By the way, the chat window in the recording is not very readable. But this won't prevent you from enjoying the slides and the synchronized audio. If a higher resolution video becomes available, we'll post it.]

I appreciated Madeline Smith’s remarks (may require login) at TCI 2009, wherein she outlined her thoughts on how regional clusters can become learning clusters. Clusters, she noted, are complex animals wherein you have “good companies, strong research universities and supportive policy makers, and you sort of mix them all together and magically economic growth pops out the other end.” Ultimately clusters are relationship-based entities, and they are susceptible to the pitfalls that strike most socio-economic initiatives. This includes, as Madeline noted in her talk, changes in the external environment, changing customer expectations, demands to learn to adapt and evolve, and a need for internal systems that allow for incorporation of formative feedback. One can see why it’s a challenge for clusters to achieve the Holy Grail of organizational effectiveness: status as a learning organization.

In addition to the challenges outlined by Madeline, I would argue that a necessary component to becoming a learning cluster is judicious and systematic use of strategy and evaluation. But one takeaway I have from my hallway conversations in Jyväskylä is that neither is happening as well as it should within the cluster community.

What’s keeping clusters from rushing headlong into strategy and evaluation? I’d like to leverage something that Madeline brought up in her talk. In discussing barriers to learning, Madeline talked about how “our current world view restricts our ability to learn” and that everyone brings a cognitive bias and their own cognitive frameworks to a project.

I’d like to suggest that a good deal of the variance in why clusters don’t engage in strategic planning and program evaluation is explained by the world view of cluster stakeholders, or to be more specific, the competing world views of the stakeholders.

It’s quite easy to imagine how this might be the case. The concept of strategic planning and the ideas behind program evaluation, depending on one’s disciplinary background, are defined, applied, and conducted in different ways. Thus the cognitive biases and frameworks of those from business, government, education, and the social sector tacitly and explicitly collide in ways that can lead to inaction, adherence to status quo, and lackluster results in the face of rich resources.

For instance, as Allison and Kay have noted, strategic planning is a term that is often used interchangeably with “long-range planning,” “business planning,” and “operational planning.” In the case of evaluation, its not uncommon to have the term “program evaluation” be confused with “monitoring” or “surveillance,” or “reporting,” all terms which are antonyms to the beneficial, learning-focused nature of quality program evaluation. As Mari Jose Aranguren and her colleagues noted in their presentation (may require login), there are severe difficulties in evaluating clusters due to their implicit characteristics: a mixture of tangible and intangible objectives; the complexity of determining cause-effect relationships. (By the way, I think the application of participatory evaluation frameworks, which Mari and her colleagues are doing, could have great promise in advancing program evaluation within clusters.)

I’d be curious to hear from others who are in the throes of bringing robust strategy and evaluation techniques to clusters. What has been challenging? What seems to be working?

The theme of the TCI annual global conference Oct 2009 in Jyväskylä, Finland is “Learning Clusters”. We are joining business leaders, cluster coaches, funders and researchers who will investigate ways to enhance competitiveness on a global market.  We will partner with VINNOVA, Sweden, to present insights on what’s really essential in a cluster leadership. Our workshop with VINNOVA “How to significantly shorten time-to-impact and create tangible change”, is designed to demonstrate a hands-on approach for leaders on how to establish and maintain a Learning Cluster.

VINNOVA used OpenEye’s visualization technique to uncover how a program can be designed to have the best possible impact. In the process VINNOVA discovered that any program that has a social dimension, which indeed all programs or clusters have, needs to be continuously reinvented. Why?  Because, the area in which we are working will change continuously. It will change both as a result of what we do and as a result of changes in the world around us. So, in order to stay on course we must be prepared to review our inputs and actions over and over. From our experience the message is clear – The basis of a successful design for impact is a good learning strategy.

At the TCI conference, we will talk about how strategic planning and evaluation work significantly better when they go hand in hand. Consequently, we argue that it is necessary to evaluate from DAY 1 in order to learn, revise and reinvent the strategy focusing on the factors that will influence your progress starting tomorrow.

Evaluation is becoming an increasingly important theme in cluster development. More people are becoming aware of the importance of getting early confirmation of progress rather than waiting to see if the ultimate economic goals of cluster policies, such as wage growth and number of new employees, are met. As a cluster goes through several stages in its lifecycle, from embryonic to declining, the evaluation strategy not only has to look at different factors in the different stages but also adapt to what’s unique about a cluster. There are no generic models or standard solutions that can guarantee success. The perfect cluster model is unique for each cluster.

VINNOVA realized the need to pinpoint exactly what the needs and problems are in their programming and continuously keep focus on the evolving nature of them. Being successful in a context where there is a social dimension requires continuous innovation based on what really happens. Success is deliberate change and impact is the result of a careful design. Focus on activities very often will make you see what you want to see. If you focus on what you would like to accomplish you will soon see that activities are simply “tests” that will give you insight on what works and what doesn’t work on your road to success. This is the paradigm shift that we hope to bring to the attention of the conference delegates. It’s about leaving the traditional role as an administrator of a cluster or a program behind you in order to become an impact designer.

If you won't be able to come to the conference, follow us on Twitter: www.twitter.com/OpenEyeGroup

Stewart I. Donaldson is one of the leading figures in the area of theory-driven evaluation. In this book from 2007 he presents several interesting cases where program theory has been used. The reason for this is that Donaldson believes that there is a need for more literature and cases around this mature theory. We definitely agree and hope that this book will contribute to have society developers, strategists and others to read about how this approach effectively empowers development projects.
We recommend the book and would also like to point out two things that Donaldson mentions in his book

Program theory has evolved and been widely accepted to become a strategic tool, not only as a standalone theory but as an integral part of many modern and popular evaluation and strategy practices. We constantly try to push the approach to become a fundamental component for the strategy work of an initiative. We hope more and more developers understand the strengths of doing so.

Donaldson points out how hard it is to involve necessary stakeholders in the process. we have experience of that too, but our recommendation is to talk about this work as the strategic tool rather than evaluation. Evaluation rarely gets enough attention in the beginning by stakeholders, so try to change the way you talk about the process in order to have them set aside necessary time to participate. We have been very successful with this, managing to get groups together even when stakeholders have been spread out all over the country.

Read, reflect and continue the exchange of experience with us.