IRIS Feedback
We’re interested in your feedback. Please leave your comments about the IRIS initiative and framework here.
Creating a Common Language for Assessing Social and Environmental Impact
Creating a Common Language for Assessing Social and Environmental Impact
IRIS FeedbackWe’re interested in your feedback. Please leave your comments about the IRIS initiative and framework here.
Many of the indicators throughout this framework are inputs, outputs, and activity-related metrics. Will there be an attempt to incorporate outcome indicators?
USAID has been refining their approach to measuring outcomes ever since the Agency undertook reengineering under the Government Performance Results Act of 1993. Much good work has been done in this area, toolkits developed (I co-authored the USAID Performance Management Toolkit in 2001 while at PwC Consulting, later IBM BCS, see http://www.usaid.gov/policy/ads/200/200sbn.pdf ), measures tested and improved, etc. It would be worth the time of the IRIS team to meet with the agency’s Program and Policy Coordination bureau.
On the domestic front, much interesting work on outcomes has been done by Harry Hatry, the Urban Institute, and the Center for What Works. In particular, see their Outcome Indicators Project (http://www.urban.org/center/cnp/projects/outcomeindicators.cfm )
The Glossary entry for Education is empty. Is this a bug or has the glossary not yet been developed?
http://iris-standards.org/framework/glossary-education
What is the plan for the growth of the governance and standardization process; what is its structure? The FAQ gives the following information, but I am wondering how the nomination and participation process will evolve beyond the founding consortium.
# How will the taxonomy be updated and revised over time?
* A governing body will be created that will oversee the evolution of the IRIS taxonomy. This body will likely be housed at the Global Impact Investing Network (GIIN) with one or two full time employees working to establish a system that allows for the revision of the taxonomy over time as comments and feedback are obtained from the broad stakeholder universe.
There is a huge need for many in the sector to get a grip on outputs, i.e., bottom line impacts of social enterprises. E.g., if a social enterprise has the objective of helping people out of poverty, they need a clear definition of what poverty is (e.g., a certain income level), a measurement of the baseline they start from and a (regular) measurement of the impact their offering of products or services has with its clients.
Such outputs should be properly defined and be included in the system and, more important, one needs to find a way to measure those outputs efficiently.
Having a (social impact)rating agency go out to the field to assess bottom line impact of a social enterprise sounds far too costly to me.
One idea would be to use IRIS to make social enterprises aware of the need of such measures, define them properly and set standards for how to gather and report the information. A (costly) rating agency would then just need to assess if the standards were adhered to, but the data gathering would happen in the respective country at much lower cost.
The indicators are interesting and I think that the definitions are a good starting point, but my question is:
How many people are actually measuring these indicators and how are they doing it?
Are there any reports out there related to this taxonomy????
Are there people that are willing to take a look at what we are doing and how we are doing it? This would be very beneficial to our institution.
We have been measuring most of the indicators for our clients and would like to know if we are in the correct path….
Also I believe that we should use this tool to communicate and get ideas and thoughts to how things should get started and how we can do this without the same mistakes…
Folks,
As mentioned on the webinar today, I wanted to restate my suggestion to create the ability for membership organizations that have many IRIS users to aggregate their data so an organizational impact report could be developed. This may be as simple as allowing them to check a box in the general overview information.
Thanks,
Randall
Environmental program reporting has been traditionally flawed and programs subsequently ineffective because it leaves out science measures on species types and diversity, water flow and quality, diversity and nutritional quality of food types eaten, forest management programs, etc.
The types of measures in this section are too abstract and barren to roll-up into decision-making spreadsheets, analysis or other types of data used to bridge scale.
I saw this at work in a World Bank portfolio review of global forest projects. The case studies offered detail level on measurable (and actionable) criteria/factors. At the next level up many key features were lost, subsumed by the abstraction process, so became invisible, despite the fact that they were crucial to important outcomes. By the time the review reached the Board level, it was impossible to trace how Board decisions could be re-routed down to the local level for the intended high portfolio performance to occur.
You run this same risk.
Environmental program reporting has been traditionally flawed and programs subsequently ineffective because it leaves out science measures on species types and diversity, water flow and quality, diversity and nutritional quality of food types eaten, forest management programs, etc.
The types of measures in this section are too abstract and barren to roll-up into decision-making spreadsheets, analysis or other types of data used to bridge scale.
I saw this at work in a World Bank portfolio review of global forest projects. The case studies offered detail level on measurable (and actionable) criteria/factors. At the next level up many key features were lost, subsumed by the abstraction process, so became invisible, despite the fact that they were crucial to important outcomes. By the time the review reached the Board level, it was impossible to trace how Board decisions could be re-routed down to the local level for the intended high portfolio performance to occur.
You run this same risk.
Thank you for the webinar today. The moderator gave a good, clear overview of the framework.
It appears that the framework is intended to create a new asset class and may well accomplish that feat. (Congrats on that achievement–not a small one. I’m sure that much work went into development.) However, is the framework also intended to assist investors in their analysis of whether a project will maximize social impact? Or is it intended to assist investors in selecting between potential projects on the basis of whether the data predict a likelihood of the project going to scale? If it’s either of the latter, I agree with Matts’ comment above and think that the metrics should continue to develop beyond “outputs” to include documented behavior change (”outcomes”), as well as actual, measured social impacts.
Two techniques might be useful in developing such metrics (which perhaps could be an add-on to IRIS). One would be to apply any proposed metric to the original documentation for what are now successful social enterprises, i.e., those that can document social impact on a large scale. If the proposed metric doesn’t identify such an organization based on its initial business plan, for example, then it may be under-inclusive and thus not really useful to an early-stage investor that wants to support an organization that can go to scale. How would IRIS have assessed Acumen Fund’s initial business plan? Would it have meaningfully differentiated between Acumen Fund and some competing fund that sought an investment? And would that, in any way, have predicted which fund would succeed, either in impact or in scale? I don’t see that the framework addressed these important issues.
If, instead, the framework is primarily geared toward expansion capital, then it perhaps should be labeled as such, e.g., IRIS Social VC Reporting, IRIS Output Indicator, or something similar.
Second (and in the event that the framework is intended to work across the range of early-stage, pilot, business development stage, and expansion stage enterprises), one could backcast a proposed outcome or social impact metric based on an explicit theory of change. For instance, a social enterprise called the OneAcre Fund in Kenya appears to be having an enormous impact on the lives of rural farmers. Currently, they cover about 70% of their costs, but they are on an expansion curve that suggests that they will reach full financial sustainability in the next three to four years, as they reach about 30,000 farming families. By “walking back the cat” to see how they themselves linked documented impacts to behavior change, and then examining how they linked that behavior change to the organization’s earlier “outputs,” one could create a true social impact reporting structure that might be bolted on to IRIS, and that would allow investors to make some pretty robust, cost-effectiveness decisions about how they deploy their capital in a way that is likely to achieve large-scale social impact.
If this is not done, however, I think that IRIS runs the risk of getting burned. After all, it’s easy to imagine a case in which an investment fund uses IRIS to CYA on some outgrower agricultural deal, only to find out later–despite the fact that the “jobs created” box had been ticked (and the women employed, minorities employed, etc.)–that most of those jobs proved to be little more than indentured servitude in practice. That’s an extreme example, admittedly, but the point remains: If meaningful impact reporting were as simple as tracking basic financial data and “outputs,” e.g., counting the number of bednets shoved out the door, then reporting would be easy and IRIS would be more than sufficient. But it’s not sufficient. It seems to stop short just when it could be of most use in extracting meaningful data and helping to direct the intelligent allocation of capital based on that data.
I know that IRIS is aware of this general problem, or limitation in the reporting system, and that these are early days for such reporting. I’m also aware of, and respectful of, the fact that you’re trying to tip the larger financial markets into substantial change, and that that in and of itself, if successful, could vastly increase social impact on the whole. So I appreciate the design decisions you’ve faced thus far and am glad that this effort is going forward.
Nevertheless, the framework, at this point, seems most applicable to the world of project accounting and large foundation grant management, and not as useful to early-stage funds, including social angel funds, for example, that make investment decisions based on potential social impacts, cost-effectiveness, and the internal business structures that are most likely to lead to successful expansion.
Two meta-level suggestions:
- I think there is some redundance in the sector-specific metrics, and some that can be moved up to the meta-level layer. These include specifically the metrics on type of product or service offered, and on target clients.
One of the things we do at our organisation is work with programmes that try and build or rebuild the social cohesion in the developing world. For example this can include working in conflict and post conflict zones, disaster areas and areas that need urban regeneration. It would be great to include another category for Social Cohesion in the sector specific indicators as well.
I agree with some of the comments already made earlier that it is important for the reporting fund manager to know how to use the reporting. It’s fine to do this very elaborately during the investment process. During portfolio management it would be good to align these reporting standards to general reporting standards for fund managers. Our fund reports monthly to the board, and quarterly to its investors. For the quarterly reports we choose several social indicators we report on which are relevant for the business, where the most important part is to see how the organization develops on those indicators: how many employees compared to last year/quarter, average salary compared to last year, amount of customers reached in suburban/rural areas, compared to last year, etc.
As stated during the recent Webinar, questions regarding the practical adoption and use of IRIS will have to be worked through and clarified any time soon in order for organizations to identify the value of the taxonomy for their own systems and the sector as a whole.
In order to allow for comparative analysis and benchmarking, the taxonomy will have to eventually weigh indicators and/or identify a set of “compulsory” for enterprises to report on. Otherwise, enterprises might opt for “cherry-picking”, which will undermine the potential for comparison of the data.
How “intermediary organizations” such as social investment exchanges will liaise with data and fit into the information flow is still to be clarified.
The challenge remains to unlock capital funds into profitable high-social impact investment opportunities by creating a solid track record and a common ground for investors and social enterprises to engage. The role of social exchanges, could be crucial in introducing and advocating for the use of social output metrics to their natural clients: fund managers and social enterprises.
I agree with most comments, which speak to the need to move towards real IMPACT monitoring and assessment, certainly by identifying and looking at those initiatives, which have the greatest experience and success. I do however, acknowledge IRIS as a step in that direction, especially with regards to the power of broad consultation in advocating for the need to consider social and financial return combined at a global scale.
As part of the set of financial indicators I suggest to add a category ‘Composition of ownership’ and a subdivision in external owners (percentage and type) and internal owners (directors, management and employees). What is important to find out whether internal ownership exceeds 50 percent of the company.
With regard to the governance of the company I would suggest to add one or more criteria with regard to stakeholder consultation and stakeholder (dis)agreement - and ways/procedures to solve any material disagreement with stakeholders. One can also decide to add this as a seperate subset of criteria under the heading of the meta-layers or under the operational criteria.
What makes impact reporting valuable and interesting is to allow companies to report best practices. Those best practices can deal with the issue of reporting as such or with specific achievements in the impact arena. To allow companies to report on their best practices as part of their Impact Reporting will provide some flesh on the bone.
However useful, IRIS is not the only framework that allows companies and investors to report their both positive and negative impact on society, culture, nature, etc. What we are trying to establish is to work together with and make use of existing frameworks that are seen as valuable by stakeholders. Good examples in the agricultural arena come from the Sustainable Agriculture Standard (operated by Rainforest Alliance), Utz Certified, GlobalGAP (although not as elaborate), etc. It helps to establish trust and credibility if companies and investors use these frameworks and are able to account for this use somewhere in the IRIS framework.
a rather profane geedback: the resolution of the framework graphic is insufficient to be able to read the text. I would be very grateful if you could send me a higher resolution of this. Sorry to be a bore..
Franz
How do outputs help a venture understand and respond to their client’s needs? Outputs show if a project has been implemented as planned, but what if a flawed model as been implemented? For example, measuring the number of mosquito nets distributed (an output) doesn’t give information on how and if the nets are actually being used. In reality the nets could still be unopened, or they could be hanging above a bed without their sides down. It is critical to understand these outcomes in order to identify areas for improvement in one’s business model. There is a real danger in making output data collection the standard for the development field.
At the William Davidson Institute we have developed and implemented an outcome survey tool and a straightfoward process to collect data at the project level.
Heather:
You’re exactly right.
To assess capacity and effectiveness of an organization (as opposed to the relative merits of what it does or aims to do), governance is absolutely key. Measuring governance effectiveness is difficult.
It appears that no one has given this aspect of the framework much thought.
What are the next steps to make this part of the framework one that institutional shareholders and other investors (and their advisers) are likely to find credible and useful?
Providing data to allow investors, either directly or through funds, to select and monitor SMEs (other than MFIs) for their triple bottom line potential and actual outcomes, plus their scaleability is, as I understand it, and from points that other commentators have made, a major aim of IRIS.
While there are sector specific indicators for energy, water and for agriculture & artisans and for microfinance itself where the investee vehicle will presumably be an SME, there appears to be no cross-sector SME category, or perhaps better a set of indicators applying to all SMEs that is clearly visible within the framework. Is this a weakness? A diversified SME portfolio is a common aim of many funds, and for example basic aspects of governance such as women (or other previously excluded minorities) in positions of leadership, having some transparency through employee attendance at regular open meetings, or evidence of social capital formation such as building stability into difficult employment categories, young men, young women, minority language groups, linkages with local institutions such as schools etc. or more mainstream financial and operational performance aspects could be shown. At the unit of the investee - the SME -, but including upstream and downstream effects, is there not a case for grouping certain non sector-specific points?