Analysis of ranking exercises in Participatory Rural Appraisal ( PRA ) : a case study from the smallholder rubber sector

Rubber Research Institute of Sri Lanka (RRISL) is an organization, which caters to both smallholder and the estate sectors by means of developing appropriate technologies to increase productivity of the rubber sector. Low adoption rates were evident for many recommended technologies, especially in the smallholder sector, since their perceptions are not always taken into account in recommendation of technologies. As a result, Participatory Rural Appraisal (PRA) has gained a considerable interest among the researchers recently, especially when dealing with smallholder rubber farmers. Thus, it is vital to introduce statistical concepts into PRA, which is an area where statistical applications are very rare. Therefore, many participatory studies report only data summaries as results. This paper addresses this issue and illustrations are presented based on participatory studies carried out with smallholder rubber farmers. The paper is focused on analysis of commonly used participatory tools; pair-wise ranking and matrix ranking, using appropriate statistical techniques.


Introduction
Participatory Rural Appraisal (PRA) is a family of approaches and methods to enable local people to present, share and analyze their knowledge of life and situations to plan, act, monitor and evaluate the situations.There has been an increasing interest on PRA as it has been successful in many occasions in planning and decision support since its development in late 1980s.Extensive reviews on evolution, methods and practices of PRA can be found in Chambers (1994), Chambers et al. (1989) and Riley and Fielding (2001).
Research on rubber is mainly confined to identification of problems by researchers and development of recommendations by the researchers themselves.
This results in low adoption levels in most of the recommendations, which are not affordable to farmers, especially at small-scale level.There have been several adaptive research trials in farmers' fields conducted by the Rubber Research Institute of Sri Lanka (RRISL).A better success would have been achieved by collaboration with the farmers; by means of understanding their perceptions, local knowledge and specific problems.Approaches like PRA can be of immense importance to collect such valuable information in terms of knowledge and experience gathered in generations by local communities.
In most cases, studies involving PRA methods typically generate qualitative data.Only several tools generate quantitative data.Often nonstatisticians believe that statistics is confined only to quantitative data generated from standard designs (Riley and Fielding, 2001).This belief led many participatory researchers to keep away from statistics in interpretation of their results.Statistical analysis offers a systematic way of describing results and determining their reliability when appropriate methods are applied to the situations.Hence, the choice of the appropriate test is important in obtaining a valid result.
PRA tools like preference ranking and scoring, wealth ranking, and pair-wise ranking generate data of ordinal scale of measurement, which can be successfully handled by appropriate non-parametric methods.One such example on analyzing preference ranking studies is found in a study reported by Wijesuriya et al. (2003).The main objective of this study was to employ appropriate statistical methods for pair-wise ranking and matrix ranking exercises and to illustrate the methodology using examples from participatory studies carried out with smallholders in Moneragala area.

Illustrations of statistical analysis on PRA tools Illustration 1: Pair-wise ranking
Illustration 1 employs data from a pair-wise ranking exercise carried out in Polgahapitiya in the Moneragala district to study the farmers' perception on factors affecting growth and yield of rubber trees.This study was done in connection with a project "Strategies for enhancing the productivity in smallholder rubber units of nontraditional rubber growing areas of Sri Lanka" (Wijesuriya et al., 2004).
Pair-wise ranking can be used to prioritize different reasons responsible for a particular problem one wishes to study.An effective Semi Structured Interview (SSI) is a prerequisite for this tool.The participants are requested to list out the reasons responsible for an identified specific problem.These listed reasons are then arranged in a matrix in rows and columns.In the next step, they are requested to consider two of the reasons at once, and select the most important reason and to fill in the elements of the matrix with its number (identity).Once they finish all possible combinations,

Where
and Rj are the 2 mean scores to be c pared and n i and n j are the number of treatment groups.For this illustration, k = conside nking using a the number of occurrences is counted for each reason.The resulting matrix is presented in Fig. 1.Finally, the ranking is done based on the number of occurrences, as indicated in column 'Rank' in Fig. 1.This matrix provides only an idea about the priority order of identified effects on growth and yield of rubber.

Data analysis
Number of times a certain factor being preferred over another is given as the 'Total' in Fig. 1.This situation can be regarded as binary outcomes where we can assign '1' for 'being preferred over another' and '0' for 'not being preferred over another' for further analysis.This will create a data set as given in Table 1, where the illustration has 12 factors to compare among them, and therefore there are 11 binary outcomes.
The data in Table 1 can be analyzed by the Kruskal-Wallis (K-W) test, which is applicable for testing the differences among several treatments (in this case 'factors').Further, multiple comparisons can be made according to the method suggested by Siegel and Castellan (1988).The analysis was done using Genstat version 7.
The mean scores obtained for each treatment are presented in Table 2, in descending order.The K-W test signified that at least 2 median values are significantly different (χ 2 = 48.72,P< 0.0001).Consequently, multiple comparisons were done using the following inequality.Ri om sample sizes of i th and j th groups respectively.
where α = experiment wise error rate and k = 12 and therefore, α* = 0.05/12(11) = 0.000379.The corresponding Z value for α* is 3.4.Subsequently, the value of the right hand side of the equation 1 was calculated as 50.1 for this illustration.
According to the results in Table 2, factors; 12 and 3 were red as the most important factors by the participants.The next group comprises of factors; 7, 11, 5, 9, 1 and 8, which can be regarded as important factors.The factors which found to be least important were; 2, 4, 10 and 6.This kind of approach is superior to ranks as it provides more information about the categories, which are of practical use to the researchers who make use of this information for further development approaches.

Factors
Matrix ranking is useful in finding the order of priority of different alternatives against pre-determined criteria.SSI is again a pre requisite for defining alternatives and criteria for a matrix ranking exercise.In general, the matrix has alternatives in columns and criteria in rows.Each criterion was given equal weights (25) in this exercise, but criteria may be given different weights.Then the participants were given a chance to weight each alternative considering a single criterion at once.Finally the marks for each alternative are summed and ranking was done to decide the priority order.The resulting matrix is presented in Table 3.

Data analysis
The Friedman's test is used to analyse the data given in Table 3.The criteria were considered as 'blocks' and the crops as 'treatments'.The analysis was done employing the statistical package, Genstat version 7, and the subsequent multiple comparisons were carried out as suggested by Siegel and Castellan (1988).
The order of priority of crops based on the criteria suggested by farmers is presented in Fig. 3.The highest priority was given to banana, followed by vegetables, Allocasia spp., mixed crops, pineapple, cassava, lime and orange.Sum of ranks produced by the Friedman test is given in Table 3.The computed test statistic (Q) was 20.32 (P=0.005)suggesting at least one treatment comparison is significantly different.
Afterwards, multiple comparisons were made using the following inequality.For this illustration, k = 8 and therefore, α* = 0.05/8(7) = 0.000893.The corresponding Z value = 3.11 and the critical value for comparisons is 28.5.Based on this critical value, the crops can be categorized into 3 groups as suggested in Table 4.The farmers have considered banana, vegetables and Allocasia spp. as the most suitable crops and orange as the least important.The rest of the crops common in this area were considered as moderately important (Table 4).

Conclusions
Results obtained from pair-wise ranking and matrix ranking exercises done in PRA can be effectively improved through respective tests, Kruskal-Wallis (KW) and Friedman's test followed by multiple comparisons and subsequently can be used for grouping.As an alternative to multiple comparisons, the total number of occurrences under a certain factor is preferred over the others in pair-wise ranking and the total marks obtained for different options in matrix ranking can

Fig. 1 .
Fig. 1.The pair-wise ranking matrix prepared by the participants at Polgahapitiya in the Moneragala district

*Fig. 2 .
Fig. 2. Dendrogram indicating the sequence of importance and the categories of importance (A) High, (B) Moderate and (C) Low R i = Rank sum of i th group R j = Rank sum of j th group n = No. of blocks k α = experiment wise error rate and k= number of treatment groups

Table 3 .
Results of the Matrix ranking exercise in Tanwatta in the Moneragala district

Table 4 .
Sums of ranks for different crops bc Low * Means with the common letters are not significantly different