NAARM

PME in NARS: Strategic Context

N H Rao

National Academy of Agricultural Research Management,

Rajendranagar, Hyderabad
http://naarm.ernet.in
1

Agenda

NAARM

Perspective on PME (Priority Setting, Monitoring and Evaluation) of

Research projects

PME in context of changing dynamics of agricultural research

Example: PME in NAIP

Scientific Research Project

NAARM

Project:
One-time endeavour to create a unique product or service for beneficial change or
added value
Involves using inputs to carry out planned activities in specified ways and time
schedules to achieve desired outputs and outcomes/impacts
Scientific Research: generating
new knowledge and solutions to
problems by asking questions
and systematically seeking
information to answer the
questions
A scientific research project
supported by stakeholders is a
basic mechanism of doing
science

(fig adapted from: : Institute of Good Governance, 2002)

Project support by stakeholders

Based on a project
proposal for support

NAARM
Stakeholders differ in perspectives
Researchers will focus on
scientific issues (mainly on the
disciplines of their own interest)
and on global research trends
Decision makers in public
institutions will focus on issues of
major public concern
Research users (government
departments, business, NGOs,
farmers, consumers) will focus on
their practical and economic
concerns
Society will focus on governance,
access to technologies and
impacts

PME provides the strategic framework and tools to balance project support (resources)
with good science and its application for desired uses and impacts
4

The P in PME ?:

NAARM

P = priority setting prioritizing investments in different contexts

requires answers to:
What should we do?
to maximize impact of investments
to strengthen national research system
to increase harmonization with global research

How do we decide what should be done ?

National/International policies: food security,
inclusive growth, environment
Foresight: current and future S&T, population,
economy, environment, society
Promise of research excellence; new knowledge,
potential use, team calibre
Organization: Vision, Mission, Strategy, resources
5

Policy
Policy =

NAARM
Principles
Guidelines
Rules
Mechanisms

to guide stakeholders in decision making

- priorities, objectives, strategies,
instruments, and accountability

Policy Hierarchy
National economic policy
(vision of societys future)

National agricultural policy

Agricultural research policy
(organization-vision-missionstrategy)
IP policy

Public policy is driven by:

vision of societys future and guided by the basic principles of effectiveness,
transparency, equity, consistency, comprehensiveness and public good
evolutionary process (influenced by legal/judicial processes)

Foresight : Rising population and middle class

drive future demand

NAARM

Middle class: households with daily expenditures between $10 and $100 per person in PPP terms

Source: Global Harvest GAP report, 2014

Foresight: Building resilience to Climate change

NAARM

1C warming of atmosphere
triples variance of
weather
2001-10 decade of
extremes (WMO, 2013)
Increases Agriculture/food
security risk
integrated multi-model
assessments to quantify and
manage climate change risks
Source:
Winkler et
al, 2011

CMIP5 - access to gridded

data sets of model
projections

Promise of research excellence and use (innovation)

NAARM

Research excellence judged by peer review (concept, method, content))

Research to innovation :
research can take place anywhere (universities, public institutions) , but
innovation generally takes place in firms (private sector)
Requires bridging two distinct valleys:

Technological Valley of Death

Commercialization Valley of Death

Public Private Partnerships (PPP) for more effective use of

resources and accelerating innovation

Private sector in
agricultural R&D :

The size of business (2006)

R&D intensity (% sales):

Crop protection:

6-7%

Seed:10-15%
Animal health:

8-9%

Machinery: 2-3%
Animal breeding: 7-8%

NAARM

Increasing share of private investment in research in

biotechnology

NAARM

US $ million
Total (R&D R
asin%
of
R&D
sales)

Company

Crop
protection

Seed/
biotechnology

Bayer

730

110

Syngenta

500

310

840 (11%) 126

810 (11%) 122

Monsanto

40

490

530 (10%)

80

Pioneer

215

312

527 (11%)

79

BASF

340

93

433 (10%)

51

ICAR (XI FYP)

400-500

Note: relatively high share of D in R&D

Adapted from : Spielman, 2007 and K Sharma, 2014

India: Public and private agricultural R&D investments

NAARM

millions of 2005 USD

Source; IFPRI (India), 2014

India: public private performance in seed sector

field crop varieties released
(20052010 )

Source; IFPRI
(India), 2014

NAARM

New vegetable hybrids in

India, 19982005

Foresight: Emerging technologies

NAARM

Knowledge becoming an increasingly important factor of

production in agriculture
Convergence of molecular biology, nanotechnology, computer
science, measurement technologies, mechanization, ecosystems
science, simulation, big data analytics
new technologies are enabling in nature
integrate into all sciences
enable in-depth understanding of agriculture at system level
enable mechanization
Most initiatives from private sector

Societal expectations: performance / governance

NAARM

Two dimensions:
internal dimension: management
processes that determine performance
(outputs)
external dimension: governance processes
that ensure accountability (outcomes)
Track resources and decisions
Translate outputs into outcomes
Translate stakeholder interests into
organizational programmes and activities

15

The M&E in PME ?: Measuring work and performance

NAARM

Monitoring (M) (supply side assessment of performance):

How should we do it - systematic documentation as Plan/Project
How do we know weve done it - links between plan, resources,
activities and expected research outputs

Evaluation (E) (demand side assessment of performance):

so what ?
what difference did the project make - research outcomes/impacts
increasing emphasis in prioritization on expected outcomes

P connects research projects to Organizational Strategy

&
M&E connect Projects with Implementation of strategy
16

M&E Framework basis for performance assessment

and accountability

NAARM

M&E Causal chain linking projects to:

inputs - activities outputs - outcomes - impact
Objectives

Goals

Time
Impact
Long term
effects of
changes
(+/-)

Monitoring
How the project works
(internal process perspective)
(fig adapted from: : Institute of Good Governance, 2002)

Evaluation
how the project makes a difference
(external stakeholder perspective)
17

Project Outputs

NAARM
Directly relate to the project
objectives and research
process
Outputs may include
- intangible things
knowledge, knowhow
- tangible things
publications, patents/IP,
products, databases
- information for action
Intermediate deliverables that
add up to outputs

18

Project Outcomes example: NAIP component 2

NAARM

19

Increasing importance of data as output/outcome

NAARM

Research cannot flourish if data are not preserved and made accessible. All
concerned must act accordingly.
More and more often these days, a research projects success is measured
not just by the publications it produces, but also by the data it makes
available to the wider community. Pioneering archives such as GenBank have
demonstrated just how powerful such legacy data sets can be for generating
new discoveries especially when data are combined from many
laboratories and analysed in ways that the original researchers could not
have anticipated.
All but a handful of disciplines still lack the technical, institutional and
cultural frameworks required to support such open data access - leading to a
scandalous shortfall in the sharing of data by researchers. This deficiency
urgently needs to be addressed by funders, universities and the researchers
themselves.

20

ICAR 2050
Vision

NAARM

In 2050 some 1.6 billion people have adequate, nutritious, safe and healthy food,
and adequate fibre within limits of the natural system
Mission
Harness power of science and education with a human touch for higher and
sustainable agricultural production.
Strategic Focus
farmer first
green revolution 2 while enhancing natural resources input intensive to
knowledge intensive agriculture for multiplying resource use efficiencies
(four fold increase in land productivity ; three fold increase in water productivity; doubling of
energy use efficiency; six fold increase in labour productivity)

climate resilience
create globally competitive human resources (frontier sciences initiatives)
premium on innovation: transform NARS to NAIS (PPP, linkages)

Agricultural Innovation System

NAARM
innovation
knowledge introduced and
utilized in an economic or
social process (Spielman,
2005)

Process of transforming NARS to NAIS:

Interactive: complex backward and forward linkages between agricultural
production, agro-industry and consumers
multidimensional engages economic, environmental, social and political
dimensions of production and utilization of knowledge

NAIP is the pilot

22

NAIP The transition sought

NAARM

Food self sufficiency orientation to market orientation of agricultural research

Transforming organization principle of NARS from Research Systems to
Innovation systems: research embedded in wider relationships across the
production-consumption continuum (interactive process requiring new and
more flexible and demand-oriented organization of research and institutional
policy)
Greater accountability to public: decisions, expenditures, impacts, regulation
simply changing mandates and adding new interests will not work
required - institutional reengineering for new capacities, flexibility, research
partnerships with market links .,

23

NAIP: Objective and Priority Areas

NAARM

The general objective of the project is to contribute to the sustainable

transformation of Indian agricultural sector from a food self-sufficiency
orientation to a market orientation in support of poverty alleviation and
income generation
Priority areas:

Build capacity of ICAR for organizational change

Promote production to consumption systems to enhance productivity,

nutrition, profitability, income, and employment

Improve livelihood security of rural people living in selected disadvantaged

regions through technology-led innovation system

Build capacity for basic & strategic research to meet present and future
technology development challenges

24

NAIP Implementation: organization and strategy

NAARM

Organization

Component 1 (objective /priority area 1): Institutional reengineering, capacity

building

Component 2 (Objective/priority area 2): Production to consumption systems

Component 3 (Objective /priority area 3): Sustainable rural livelihoods

security in disadvantaged areas

Component 4 (Objective priority area 4): Basic and strategic research (BSR)

Strategy

Research in consortia mode through public-private partnerships - to

accelerate collaborative development and application of agricultural
innovations (public research organizations, farmers, private sector and other
stakeholders)

Concentrate investment in a few large projects

25

NAIP: project selection and implementation

NAARM

Project selection criteria (by each component)

Relevance

Quality (science excellence, collaboration/partnership, team capacities)

Performance (expected outputs & outcomes)

Post project Sustainability

Three phases of project implementation:

Initial phase of six months for pre-project activities: needs analysis,

orientation and sensitization of stakeholders, baseline survey/bench marks,
identification of targets, and output and outcome indicators

M&E during Project implementation monitoring systems (for activities,

outputs, outcomes)

Final phase of 6 months for post-project activities: report writing,

information on outputs and outcomes, documenting success stories,
dissemination, planning for post-project follow-up activities (uptake plan)
26

Component 1: organizational change

NAARM

Projects:

Information, communication & dissemination systems (ICDS)

Knowledge portals
Knowledge systems for research, education and technology transfer
e-learning portals

Communication & public awareness (DKMA)

Business planning & development (BPD)

Learning & capacity building (L & CB)

Policy analysis & visioning support to NARS (VPAGe)

Remodeling financial & procurement systems (FMS/ERP)

27

Component 2: Production to Consumption Systems

Research projects with focus on:

NAARM

Fig Source:
Haarwich et al, 2003

End to end commodity chains

value addition: adding and
accumulating value from one link
to next in the value chain

innovations at each link

for improved competitiveness

Product development and

technology commercialization
Projects: sorghum, meat,
jasmine, grapes, ..
28

Component 3: Sustainable Rural Livelihood Security

Fig Source:
DFID

NAARM

Development & validation of sustainable

farming systems based on on-farm research
Transferring promising technologies under
NATP and other sources including ITK
emphasized

Research projects focus:

rural livelihood systems in
disadvantaged regions

regional / theme-based consortia

partnerships with NGOs, Govt Depts
Regions selected from 150 disadvantaged
29
districts

Component 4: Basic and Strategic Research

NAARM

Focus areas
Challenge problems of national and long-term importance requiring innovative
research at the frontiers of science

Nanotechnology

Biotechnology

Gene discovery and allele mining

Bioprospecting - marine biota for bioactive molecules and products

QTL identification, cloning of QTL genes and use in MAS

Biosecurity

NRM/IPM

Enhancement of nutrient use and uptake efficiency in plants/animals

Carbon pool conservation & enhancement

Mitigation and adaptation strategies for managing climate change

Precision agriculture

Implementation: Individual institutions, network or consortium

30

M&E Framework basis for performance assessment

NAARM

Comprehensive Baseline / Benchmark survey (socio-economic including gender,

technical, socio-environmental) to be conducted within 6 months of formation of
consortia

M&E focus on measurement of outputs and outcomes with reference to benchmark

survey and on the quality of the management processes in the consortia

(fig adapted from: : Institute of Good Governance, 2002)

31

Governance Structure for the National Agricultural Innovation Project

NAARM

National Steering Committee

Project Management Committee (PMC)

O&M Programme Committee

Research Programme Committee (RPC)

O&M Advisory Group

Project Implementation Unit

Administration

Finance

Procurement

Project 1

TAG 2

TAG 3

TAG 4

National Director

National
Co-ordinator
(O&M)
Component 1

Project 2

TAG = Technical Advisory Group

National
Coordinator
(Value Chain)
Component 2

National
Coordinator
(LivelihoodImprovement)
Component 3

As in Component 2

Project N
CAC 1

CAC 2

CAC N

CIC 1

CIC 2

CIC N

National
Coordinator
(Basic & Strategic
Research)
Component 4

As in Component 2

CAC = Consortium Advisory Committee

Cons 1

CIC = Consortium Implementation Committee

Cons 2

Cons N

32

M&E framework: Performance assessment

NAARM

Performance assessment:
relates inputs to:
outputs (productivity, efficiency)
outcomes (direct and indirect consequences of outputs)
includes not only defining and measuring performance but also judging
and improving performance
allows assessments of value added by individuals/teams/organization to
society through the use of performance indicators

Performance measurement through M&E makes accountability possible

33

M&E framework as an interface among stakeholders

NAARM

consolidated source of information showcasing project progress

allows learning from each others experiences, building on expertise and
knowledge
generates (written) reports that contribute to transparency and accountability
allows for lessons to be shared more easily
reveals mistakes and offers paths for learning and improvements
provides a basis for questioning and testing assumptions
provides a means to learn from experience and incorporate into policy and
practice
provides a way to assess the link between implementers and beneficiaries
and decision-makers
adds to the retention and development of institutional memory
provides a more robust basis for raising funds and influencing policy
34

M&E in NAIP - Results Framework

NAARM

35

Focus in monitoring

NAARM

Project management
Financial management
Procurement
Recruitment
Decision processes

Research implementation and outputs

Knowledge management activities

Capacity building activities

Development activities

36

M&E Framework - output indicators - include

process and product

NAARM

expenditure - procurement, recruitment

project management decision-making processes, internal procedures evolved for
conflict resolution, quality and timeliness of financial management and procurement, IP
management, credit/benefit sharing, leadership
Product development: improved crop varieties/animal breeds/tree species - both
numbers and adoption rates
Improved management practices by farmers both numbers and adoption rates
Publications and reportsVariety and number
Training eventsVariety and number
Workshop events variety and number
Dissemination (technology transfer) eventsVariety and number
Professional recognitionsVariety and number

37

M&E Framework - Outcome indicators

NAARM

Area expansion (crops) and/ or increase in number (animals)

Enhanced profitability to the producers and other participants in the value

chain

Enhanced gainful employment

Income generation/ enhancement

Improvement in food consumption/nutrition

Improvement in access to credit

Enhancement of Environment

38

NAIP Results Framework output indicators

NAIP objective:
Accelerated collaborative
development and application
of agricultural innovations

Component 1:
ICAR as the catalyzing agent
for management of change in
the Indian NARS

NAARM

The number of partnerships between

public research system, private sector
and other stakeholders
Increase in agricultural innovations by
end of the project
Increase in the number of knowledge
products and P.A. messages
Business development units in five ICAR
institutes/SAUs
Enhanced capacity for policy analysis,
planning and advice
Enhanced financial and procurement
management capacity

39

Expected outputs

Component 2:
Research on
production to
consumption systems

Component 3:
Research on
sustainable rural
livelihood security

Component 4:

NAARM
Number of technologies jointly developed
between consortia partners
Improved quality management mechanisms
Number of public-private partnerships
Number of technologies tested in
interaction with target groups
Increased rural employment opportunities
Number of farmer organizations engaged in
collaborative research
Number of publications of Indian agricultural
scientists in high quality journals

Basic and strategic

research in the frontier Number of patents obtained
areas of agricultural
science

40

Levels of Monitoring and Evaluation in NAIP

NAARM

1.

Half yearly/Annual Reports

2.

CAC/PAC

3.

Bank led Implementation Support Review missions

4.

NC(O&M) assisted by an external M&E consultant at the national

level

5.

External reviews

41

Impact assessment

NAARM

Longer term perspective than M&E

Toward the end of the project

Standardized methodology to enhance credibility

Consortia teams + NCAP

Not at random, but for interesting cases

More emphasis on economic and social contributions of the

sub-projects than on feedback
document outstanding
successes .

42

NAIP/ PWC: Outcome assessment component 2 (Sep 2012)

NAARM

43

NAIP/ PWC: Outcome assessment component 3(Sep 2012)

NAARM

44

Eg: Outcome assessment: Component 3 project

NAARM

Introduction of Redgram (BSMR-736) Transplanting/ Dibbling method

and Demonstration of Integrated Pest Management (
University of Agricultural Sciences , Raichur (Lead center)

45

Operation of PME cells: suggestions (indicative)

NAARM

Priority setting (project proposals integrated into organizational strategy):

Periodic technology foresight assessments to identify priority areas validation by
PMC; external reviews; documentation & dissemination
New Project proposals: include conformity statements with priorities and measurable
indicators and timelines
New Projects submitted to PME cell and processed for approval by PMC
Externally funded projects institutionalized in RPFs
Positive, proactive approach towards realizing institutional vision/mission
PMC to meet often to promote and process new projects
Monitoring (current projects):
Benchmark: Institutional, year-wise programmes and targets of current FYP
Principal mechanism : RPF2/RPP2 at IRC; IRC at six months
Focus: activities, processes, outputs
Database of outputs: technologies, publications, IP, other
Monitoring Indicators

Evaluation (post project)

Technology assessments
Technology transfer: Networking with ITMUs, BPDU/ZTMC, KVKs
Outcome assessments (tangible (indicators), intangibles)

Can PME activities

be integrated into a
formal project
(RPF)?
outsource some
aspects of E ?
46

NAARM

Thank you

47