When it comes to community-level decision making it is appropriate to utilize expert data based-methods, as the respective subject domains are, mostly, weakly structured ones. At the same time, during decision-making, opinion of the target territorial community members should be taken into consideration alongside expert data. The paper outlines an original method for formal description of weakly structured community-level problems, which uses both expert information and opinion of respondents from among community members. It represents a hybrid approach, incorporating elements of both traditional expert data-based methods and social surveys (questionnaires). The main goal (problem) is formulated by a decision-maker or research organizer. It is then decomposed by experts into sub-goals or factors that are crucial for its achievement, and these factors and their weights are estimated by respondents who are ordinary community members. The method includes the following conceptual steps: hierarchical decomposition of the problem, direct estimation of importance of factors that influence the problem, estimation of lowest-level “non-decomposable” factors by respondents in Likert agreement scale, and rating of the factors based on respondents' estimates through linear convolution (weighted summing). The obtained ratings provide the basis for defining toppriority activities that should be performed in order to solve the problem, and for subsequent distribution of limited resources among these activities. Experimental results, obtained in the process of actual research of public space quality, illustrate the method's application, and confirm its high efficiency. The advantages of the suggested method are efficiency and, at the same time, ease of use. In contrast to traditional expert data-based methods, it does not require any preliminary coaching sessions to be held with the respondents. The method is intended for decision-making support at the level of territorial communities (urban, rural, district, neighborhood, and others) in the spheres, directly related to the interests of community members. Target users of the method include local self-government bodies, media, public and volunteer organizations, activists, and other interested parties..
Weakly structured nature is inherent for many fields of human activity. As we know
from numerous sources, the characteristic features of a weakly structured subject
domain are as follows (see, for instance, [
At the same time, in order for decisions made in any subject domain (a weaklystructured one as well) to be efficient, they should be informed and wellsubstantiated. Thanks to consideration and systematization of all available information, the level of trustworthiness of the decision-maker (DM) increases, while the possibility of erroneous and incompetent decisions being made is reduced. So, in order to set priorities and plan respective activities, a DM needs to be able to analyze and formally describe weakly structured subject domains. Consequently, the problem of analysis and formal description of these domains retains its high relevance. With the listed properties of weakly structured subject domains in mind, we should acknowledge that expert data-based methods and technologies are a powerful (and often, the only) mathematical tool for decision-making support in these domains. Experts (ideally – narrow-profile specialists) should be engaged, first, to outline a set of factors which are crucial for a given domain, and identify the nature of interrelations between these factors, and, second, to provide numeric (quantitative, cardinal), or at least, ordinal (rank) estimates of the relative weights of factors and alternative decision variants, from which the DM will have to select an optimal one.
In the most common case, decision-making means either choosing one of several
alternative decision variants from a given set, or ranking/rating of these variants.
Optimality of a decision variant (alternative) is defined based on some specific global
(aggregate) efficiency criterion, which can reflect the degree of achievement of a
certain main goal. Such general efficiency criterion is, usually, defined based on the
realities of a specific situation. It provides the “starting point” in the process of formal
analysis and decomposition (break-down) of the subject domain into particular
aspects, as well as expert estimation of decision variants. In a way, the global efficiency
criterion plays the role of a target function from mathematical optimization theory [
The result of expert decomposition of the main criterion (goal) into sub-criteria
(sub-goals) is a hierarchy of criteria, which characterize the subject domain.
Expert data-based decision-support methods are listed and described in multiple
publications. We can mention classical works by Kendall [
Acknowledged multi-criteria expert estimation methods include the
aforementioned AHP, TOPSIS, ELECTRE, CTDEA methods; the most popular ranking
aggregation methods include Borda rule, Condorced rule, Kemeny’s median, and others;
and when it comes to pair-wise comparisons, the common approaches include, again,
AHP/ANP [
Specific applications of expert data-based methods, particularly those using the
hierarchical approach to problem decomposition, are rather numerous. Dozens of
specific applications are described in the proceedings of the International
Symposiums for the Analytic Hierarchy Process (ISAHP) [
In modern-time Ukraine, there are several spheres, calling for expert data usage in
decision-making process. In this context we can, again, mention the technological
forecasting problems [
Another subject domain, which is relevant for Ukraine, is decision-making at the
level of communities (village, raion, urban, territorial, etc). After launching of
decentralization (particularly, budget decentralization), administrative-territorial
organization reform, and formation of unified territorial communities (see [
While the arsenal of available approaches and methods is seemingly wide, in this particular case we are talking about a specific type of problems and specific conditions of expert examination. So, the question is: which of the listed approaches (or their components) should be applied in community-level decision-making to ensure that community members’ opinion is taken into account?
The key feature of the aforementioned class of problems is their weakly structured nature. Formal problem statement is possible only when the goal, which the DM or other interested party is trying to achieve, is clearly defined. A community often finds it hard even to formulate a specific problem, not to mention identification of factors, which could influence its solution.
“What should be done with the waterfront area?”, “how can we reorganize the park?”, “what is the best way to arrange the system of water supply and water disposal in the settlement?”, “what should we do with an old village cultural center (club), museum, library?” etc. Such typical community-level problem examples have several features in common. First, as it has been said, they immediately concern the representatives of a given community and reflect their interests. Second, they do not include any recommendations as to how the issue under consideration can be resolved. That is, the only input data is some problem to be solved, or some main goal to be achieved (let us denote it as G , and let us denote the main criterion of efficiency of its achievement as C0 ).
In view of weakly structured nature of the problems, it would be reasonable to
hierarchically decompose the problem into specific components. Relevance and
effectiveness of application of hierarchical approaches are shown, for example, in the
works of Saaty ([
So, at the initial phase it is suggested to decompose (break-down) the main goal or
problem into factors, which play decisive roles in its achievement (solution), as it is
done in the listed methods, such as AHP [
Both in AHP and CTDEA, when a hierarchy is built, the weights of impact factors (criteria) are defined, that is every edge of the hierarchy graph is assigned a certain weight.
In [
Based on these considerations, it is suggested to delegate the hierarchy building process to the experts in the given subject domain (as it is done in AHP or CTDEA), while introducing several additional requirements:
1) We should forbid input of cycles (loops, where criteria influence themselves)
into the hierarchy graph. In the ideal case the hierarchy should represent a tree-type
graph, that is, one node should have only one “ancestor” (hierarchy graphs of this
type are addressed, for example, in [
2) Bottom-level criteria should be formulated not as concepts (for example,
“quality of family leisure in the park, estimated in the scale from 1 to n”), but as positive
statements, with which a respondent (not an expert, in the general case!) might agree
or disagree. For example: “the park is a good place to spend quality time with a
family”; response variants: “totally agree”, “agree”, “disagree”, “totally disagree”, “don’t
know”. That is, we should provide respondents with an opportunity to estimate
“atomic” bottom-level factors in Likert’s scale [
3) Estimation of relative weights or impacts of factors (criteria) should be
delegated to respondents from among target community members, so these estimates
should be direct ones. In the process of weight estimation, in order to ensure
vividness, the estimates are to be provided in graphic (and not verbal or numeric) format
(this requirement is also based on the peculiarities of obtaining data from respondents,
described in [
Based on the listed requirements, we can suggest the following step-by-step algorithm of formal description of a weakly structured problem. 3
1) The DM or expert examination organizer formulates the main problem or goal G and chooses the experts (at least one expert) in the respective subject domain.
2) Experts build a hierarchy of criteria (see Fig. 1 below), which are crucial for the given problem or goal: {Ci : i 0..n}. The bottom-level criteria (which do not have ancestors in the hierarchy graph) {Ci : k 1..l} are formulated as positive statek ments, which the respondents will estimate in Likert’s agreement-disagreement scale.
3) A set of respondents {rj : j 1..m} is chosen from among the members of the given community.
4) Respondents estimate (directly, in clear, preferably, graphic format) the weights of impact factors at each hierarchy level. As a result, we get a set of impact coefficients (or relative respondent.
weights) {wi( j) : i 1..n; j 1..m} , provided by every
Impact wi0 of criterion Ci upon the main criterion C0 is defined as shown in [11]
(case of a hierarchy of “tree” or “network” type) and [
ni np wi0 ws (1)
p1 s1 where ni is the quantity of all possible paths from node Ci to node C0 in the criteria hierarchy graph; p is the number of a particular path; np is the length of the path number p , leading from Ci to C ; s is the number of the node within the path; 0 ws is the impact of criterion Cs upon its immediate ancestor in the path number p in the hierarchy graph.
We should note that if the hierarchy graph is a tree (every node (vertex) has no more than one ancestor, as on Fig. 1), the number of summands in formula (1) equals 1, as there is only one path, leading from any criterion Ci to the main criterion C . 0 5) Respondents select the estimates of bottom-level criteria in Likert’s scale (as shown above). Scales grades are assigned the respective numeric equivalents for further aggregation, i.e., convolution (for instance, “6 – totally agree”, “5 – agree”, “4 – rather agree than disagree”, “3 – rather disagree than agree”, “2 – disagree”, “1 – totally disagree”, “0 – don’t know or don’t care”). As a result, we get a certain number of individual judgments of respondents for criteria from the bottom level of the hierarchy: {qi(kj) ; k 1..l; j 1..m} , where ik are numbers of the bottom-level criteria, l is their total number (quantity), and j is the respondent’s number.
6) Estimates according to every criterion are aggregated through weighted summation (convolution) of estimates, provided by all respondents. As a result, we get the ratings of all bottom-level criteria Qi ; k
m
Qik j1 wi(k j0)qi(kj) (2) where wi(kj0) is the relative impact of bottom-level criterion Cik upon the main criterion C0 , calculated according to formula (1) based on values of impacts of all intermediate-level criteria, provided by the respondent number j .
Based on these ratings, potential and priorities of the problem solution are identified, while community interests are taken into account. Highest-rated factors represent the top-priority aspects, which the community is satisfied with, while lowest-rated factors represent aspects, which do not satisfy the community, or are insignificant in the eyes of community members.
7) In order to check the consistency of respondents’ judgments, we can ask them to write small verbal reviews, in which they should try to outline (once again, this time, verbally) positive and negative aspects, characterizing the subject domain.
We should stress that we are talking about inner consistency of judgments of each respondent: verbal review should be consistent with estimates, provided in Likert’s scale at previous steps (see step 5). Mutual incompatibility of the estimates is not a problem, i.e. judgments of different respondents can differ, and it is quite natural, because in our case respondents only express their opinions, and do not try to estimate some objective values.
As judgments are provided in Likert scale, and it is only inner consistency of the
estimates that is verified, traditional consistency measures (such as Kendall’s rank
correlation [
8) In order to ensure transparency, tag (word) clouds can be generated (based on verbal reviews), which will also, in a way, represent the ratings of positive and negative aspects of the issue under consideration. In order to build tag clouds based on word frequency analysis of verbal review texts, publicly available online software tools can be used (such as WordItOut, Worlde, WordArt, WordCloud, TagUl, Many Eyes, Tagxedo, etc).
The final result of the algorithm is a formal analytic description of the subject domain. Such a description allows interested parties to clearly identify the key positive and negative aspects in the given subject domain, taking public opinion into account, and, thus, define its potential, prospects, and top-priority problems, at which resources should be targeted. 4
As an example of application of the suggested weakly structured subject domain description method, let us consider an actual research, which took place in summer of 2018, in one of the raion centers of Kyiv oblast (Ukraine). The main task was to study the quality of a public location (the territory of the state local history museum) in order to facilitate its further transformation and improvement. The study envisioned a set of tasks: 1) ensure communication (productive contact) between the community and the town management; 2) identify strong and weak points of the location; 3) identify priorities of the community (i.e. what was important for the residents); 4) evaluate the extent to which the location meets the needs and expectations of its users; 5) stimulate new ideas concerning location improvement; 6) define priorities of location development.
The focus-group from among the community members included 11 respondents, featuring local residents, museum employees, civil servants, journalists, artists.
The main goal was to improve the quality of the location; respectively, the generalized location quality was the main criterion.
Criteria hierarchy was built based on the SpaceShaper [
The hierarchy of criteria, which influence the quality of the location, built in
“Solon” DSS [
The list of criteria is presented in Table 1. As we can see, the topmost (first) level of the hierarchy consists only of the main criterion “the overall quality (efficiency) of the location”. Its immediate sub-criteria (descendants) are functionality (which, in turn, includes accessibility, ease of use, and interests of residents), characteristics (including order, environment, and look/appearance/design), and value of the location (for the community and an individual respondent respectively). The bottom (fourth) level features 41 criterion, formulated as positive statements, with which respondents can agree or disagree (for instance, sub-criteria of accessibility are: “it is easy to get here”, “the place is open whenever I come here” etc).
Every respondent received a questionnaire form, in which (s)he had to specify his(her) occupation and location usage mode (frequency and purpose of visits), provide weights of criteria of the second and third levels of the hierarchy, and estimate the bottom-level criteria in Likert’s agreement-disagreement scale.
Estimation of weights of intermediate-level criteria was performed using sector diagrams (pie charts) (see Fig. 2). The choice of this particular method for weight estimation results, primarily, from simplicity and transparency considerations. As a result of estimation, every second-level criterion was assigned an integer-value weight from 0 to 12 (sum of all second-level criterion weights equals 12), while every third-level criterion was assigned a weight from 0 to 144 (sum of all third-level criterion weights equals 144). Weights of these criteria were estimated by each respondent. It should be stressed, that this particular method of criterion weight estimation was chosen for the specific study. In the general case, depending on specific hierarchy structure, and the degree of process automation, other weight input methods can be used, providing they are clear and understandable. Bottom-level criteria were presented to respondents in the form of tables (in accordance to SpaceShaper methodology, their weights are considered equal). Respondents had to express their judgment on each of the 41 bottom-level criterion in Likert’s scale. An example of the table from the questionnaire, filled out by every respondent, is shown in Table 2.
Once the surveys were completed, the respondents were offered to describe in their own words the strong and weak points of the location, as well as their vision of the ideal condition of the location (so-called “letter from the future”). Aggregation of survey data was performed as follows.
Verbal values were replaced by numeric equivalents (as shown in the previous section): 6 – totally agree”, “5 – agree”, “4 – rather agree than disagree”, “3 – rather disagree than agree”, “2 – disagree”, “1 – totally disagree”, “0 – don’t know or don’t care”.
After that rating of every bottom-level criterion was calculated through weighted summation (convolution) of estimates provided by all respondents (according to formula (2)).
In general case bottom-level ratings lie within the range between 0 and Rmax :
Rmax k mqmax wmax k , (3) where k is the number of bottom-level criterion; m is the quantity of respondents; qmax is the maximum value of numeric equivalent of a scale grade, which can be chosen by a respondent, while wmax k is a maximum possible criterion weight value. lly ree taoT isagd In the case of our particular problem the number of respondents is m 11 ; the range of bottom-level criterion weights is l {wik Z [0;144]; k1 wik wmax 144; l 41; ik {12..52}} , i.e. bottomlevel criteria from Table 1 with numbers 12 to 52 can be re-numbered from 1 to 41; their weights are expressed by integer values from 0 to 144, and their sum equals 144. The range of numeric equivalents of Likert scale grades lies between “0” (“don’t know or don’t care”) and “6” (“totally agree”), i.e in formula (3) qmax 6 . Respectively, bottom-level criterion ratings will belong to the range from 0 to 9504 (according to (3) Rmax 11 6 144 9504 ). If all ratings need to fall within the range between 0 and 1, they can be normalized.
Bottom-level criterion ratings, obtained through aggregation of respondents’ estimates, are shown on Fig. 3. For the sake of convenience these criteria are numbered from 1 to 41 (as described above).
We should note that absolute rating value does not play any significant role. It is the ratios (or differences) between ratings of different criteria that matter. Besides that, important information can be obtained from the ratios of ratings within each subgroup (functionality, characteristics, value of location). Fig.4 displays the relative ratings of criteria, which belong to “functionality” subgroup.
Similarly, the respective ratings for each intermediate-level criterion were calculated. Rating of a criterion which has descendants in the hierarchy graph is calculated as the weighted sum of ratings of its immediate sub-criteria: (4) where Qi is the rating of criterion Ci ; Qi( j) is the rating of this criterion calculated based on estimates provided by respondent number j ; m is the total number of respondents; v is the number of immediate sub-criteria of Ci in the hierarchy graph; wi(kj,i) is the weight of impact of sub-criterion number ik upon criterion Ci , estimated by respondent number j ; Qi(k j) is the rating of the sub-criterion number ik , calculated based on estimates provided by respondent number j . For instance, in the hierarchy on Fig. 1 the sub-criteria of “Functionality” ( C ) are “Accessibility” ( C4 ), 1 “Use ( C5 ), and “Interests of community” ( C6 ). So, the rating of “Functionality” is calculated as the sum of ratings of these sub-criteria. Aggregate relative ratings of third-level criteria are shown on Fig. 5, illustrating respondents’ opinion about location’s compliance with the parameters listed in the survey. As we can see from the diagram, the weakest points of the location are that (according to the respondents) it does not meet the needs of the community and is rarely used.
Based on textual analysis of verbal reviews and “letters from the future”, tag clouds were built. It is interesting to note that the drawbacks, mentioned by respondents in verbal reviews, confirm the ratings, shown on Fig. 5 (i.e., respondents’ judgments are rather consistent): according to the respondents, low level of location usage and its inability to serve community interests, are the main flaws of the location.
Functionality estimate
Fig. 5. Generalized ratings of third-level criteria Aggregate survey results allow us to make several important conclusions.
1. Location has a great potential for further improvement and development. Community representatives consider themselves capable of active participation in transformation of the public space.
2. Museum space and adjacent territory are not the focal point of active community life. With the exception of museum employees, community residents rarely visit the location.
3. The main advantages of the public space are convenient downtown location, coziness, presence of greenery, open territory, and esthetic attractiveness of the historical museum building. The main drawbacks (and, consequently the main points for location development) include poor understanding of its designation by the community, lack of cultural events, activities, entertainment, management initiatives, creativity; alerting condition of trees on the territory of the museum.
4. While accessibility (convenient location), attractiveness for the community, and environmental potential are the strong points of the space, inclusiveness (consideration of interests of all potential user categories), full-fledged utilization of location’s capacity, design/look, and infrastructure are the top-priority development aspects.
5. Transformation and sustainable development of the location should be based on the results of the conducted study, particularly, on aggregate judgments of the respondents from among community members. Location development activities and projects, implemented by local authorities, NGOs, activists, volunteers, based on public opinion, will be successful and get support from the community.
As of now, public organizations in cooperation with local activists and authorities have already accomplished several projects along the lines of the study results. 5
As we can see, the described approach is a “hybrid” one in a way that it combines the elements of both decision theory and sociology (surveying, agreement scale usage).
The following features of the approach are common with the available decision
support methods:
— usage of hierarchic problem decomposition (as in AHP [
Now let us list the main differences of the approach from the existing methods.
— the key task is only to describe the subject domain, i.e. to form a system of linked
criteria (not to compare alternatives or projects according to these criteria, as it is
done in AHP or CTDEA);
— the way of criterion formulation. “Atomic” bottom-level criteria, which do not
have descendants in the hierarchy graph, are formulated as positive statements (and
not definitions, as in traditional methods), with which a respondents can agree or
disagree;
— as a result, instead of direct estimates or pair-wise comparisons, Likert’s
agreement scale is used;
— ease of problem decomposition: hierarchy graph is a “tree” [
So, we should stress once more that the key feature of the method is the combination of expert and sociological mindsets, which ensures, on the one hand, ease of use, and on the other – high efficiency of the method.
The results of the method’s work are the ratings of activity scopes, providing the basis
for further prioritization and, potentially, for allocation of limited resources [
The method’s advantages are ease-of-use and understandability for a community member, efficiency and transparency, universality and flexibility (for each new subject domain a new unique hierarchy can be built), vividness of subject domain description process and representation of results.
The method’s key disadvantages are the possibility of manipulations (experts can formulate criteria in some biased way, however, this is the issue of ethical principles of these experts and the DM), and of emergence of “lobbies” among community members (according to profession, age, mindset, gender, wealth, social status, etc).
A separate problem concerns manual input and processing of data (when MS Excel is the only software tool used for data aggregation). Ideally, the process of formal description of subject domain should be almost fully automated. Certain steps of the above-listed algorithm are already automated within the existing DSS. Particularly, “Consensus-2” DSS [42] includes means for registration of experts who are inputting the hierarchy, and for input of the hierarchy itself. “Solon” DSS [11, 41] includes tools for hierarchy input, as well as for calculation of relative impacts of criteria. Thus, the functions, delegated to experts, are already automated, while functions, delegated to respondents and to the research organizer (knowledge engineer, who has to aggregate the data and obtain recommendations for the DM using the DSS tools) still require automation. In order to simplify the process of surveying and aggregation of survey data is would be reasonable to automate: — completion of the surveys (for example, using tablets or similar gadgets); — submission of survey data to DSS knowledge base in remote mode; — calculation of criterion ratings using the mathematical tools of a DSS. Recognition and frequency analysis of verbal review texts, as well as tag cloud building are a bit more difficult to automate, however improvement of this algorithm step is no less relevant than automation of other steps. 6
It has been shown that community-level problems represent an example of weakly structured subject domains. During their formal description we should consider both expert data and community members’ opinion. In view of the need to consider public opinion during community-level decision-making, it is unreasonable to apply existing decision support methods and technologies in their classical form.
An efficient, yet simple, method has been suggested for formal description and analysis of community-level problems, taking public opinion into consideration. The method allows a DM, a local authority, an NGO, community representatives, volunteers, activists, media, or any other interested parties to get a clear understanding of a specific subject domain, which will provide the basis for prioritizing of future steps.
Experimental results have been obtained, based on conducted research of quality of a specific public space. The research confirms both efficiency and ease-of-use of the suggested approach. Based on the research results, specific recommendations concerning improvement of the target public space (location) have been worked out.
The described method should be used for community-level decision-making – in villages, raions, towns, neighborhoods — in spheres, immediately concerning the respective community members. Particular decisions might concern such aspects as planning and improvement of road and transport networks, domestic waste disposal, water supply and disposal, planning and improvement of territories, reintegration of public spaces into active community life, neutralization of negative information impacts, etc.
The method is an efficient decision support tool, which should be used by local self-government bodies, civil organizations, volunteers, activists, and any other interested parties.
Further studies will be dedicated to search for new applications of the method, and to automation of particular steps of the described procedure of analysis of weakly structured subject domains.