An expert is a specialist (professional), whose assessments and judgments the decision maker considers useful to take into account when making decisions.
Let's give some explanations. Naturally, the opinions of a novice therapist and a world-famous cardiologist are not comparable in terms of the level of assessment of the patient’s condition and recommendations for his treatment in a fairly difficult situation. However, if the patient needs emergency help, and a specialist of high professional level for one reason or another cannot be invited, then the decision will be made by whoever is nearby, even if he is less qualified. In this situation, it is he who turns out to be the expert.
Thus, the formation of the composition of the expert commission depends on:
· specific situation decision making;
· the ability of the organizers of the examination to attract highly qualified specialists for the work;
· opportunities for specialists to take part in the work of the expert commission.
It should also be noted that at present there is no generally accepted unified methodology for assessing the qualities of an expert, therefore opinions about the professional level of a specialist often differ significantly. In real practice, the decision maker either strives to form an opinion about the professional level of the candidate expert, or entrusts this to those who are entrusted with organizing the examination. Therefore, in the absence of opportunities and experience in organizing and conducting examinations, it makes sense to turn to the services of independent examination centers, information and analytical centers, etc., the main tasks of which are to analyze the situation, assess the objects of examination, prepare and evaluate alternative options decisions.
When assessing the qualities of an expert, it is necessary to take into account his professional knowledge, experience and effectiveness of work as part of expert commissions. There are many ways to assess the qualities of an expert, each of which can be successfully used in a particular case. They are divided into three main groups:
· a priori;
· a posteriori;
· test.
A priori methods are called methods for assessing the qualities of an expert that do not use information about the results of his participation in previous examinations.
A posteriori methods are called methods for assessing the qualities of an expert based on information about the results of his participation in previous examinations.
Test methods are methods for assessing the qualities of an expert that involve conducting a special test.
A priori methods for assessing an expert. This group includes, first of all, the most common methods of self-assessment, the essence of which is that the expert himself evaluates his qualities. The following methods can be used:
· self-esteem on one of the point scales (3, 5, 10 or 100 points);
· self-assessment using verbal-numeric scales, which, along with the numerical values of the gradations, contain their qualitative description;
· self-assessment on a verbal scale, in which the expert gives a verbal assessment of his knowledge and experience, using qualitative gradations of a pre-developed scale;
· self-assessment using the differential method, in which the expert evaluates his qualities using two main private criteria: a criterion characterizing his familiarity with the main sources of information in this field (for example, special domestic and foreign periodicals, patent information, in-house information, etc.); etc.) and a criterion characterizing familiarity with the object of examination (for example, knowledge of industry specifics, a specific enterprise, familiarity with product samples, etc.).
In this case, the expert’s comprehensive self-assessment is calculated using the formula:
TO k = ( TO and + ß TO h) / 2,
Where TO k - comprehensive self-assessment of the expert, TO and - awareness coefficient (in fractions of unity), TO z is the coefficient of familiarity with the problem, ß is the weighting coefficient.
The issues of measuring indicators and types of scales will be discussed in more detail in the next topic.
The accuracy of self-assessment methods is somewhat controversial. On the one hand, no one better than the expert himself represents the totality of knowledge and experience that he possesses. On the other hand, a person makes the biggest mistake when assessing his own capabilities.
Another group of a priori methods for assessing the qualities of an expert includes mutual assessment methods, the use of which assumes that experts evaluate each other. The most common of these is the expert list method. The procedure for its use is as follows. Each expert compiles a list of specialists whom he considers competent in this field. Then, based on these lists, the expert’s competence coefficient is calculated - the ratio of the number of lists in which a given specialist is present to the total number of compiled lists.
The a priori methods for assessing the qualities of an expert also include the most common documentary or questionnaire method. In it, to assess certain qualities of a specialist, it is proposed to use objective characteristics that have documentary evidence, for example, such as: work experience, academic degree, academic title, position held, number of publications, citation index, etc. It should be borne in mind that, based only on these objective factors, it is impossible to adequately assess the suitability and usefulness of an expert’s participation in the work of a particular expert group.
A posteriori estimation methods. These methods involve the use of the results of his participation in previous surveys when assessing the qualities of an expert. With their help, to a certain extent, such qualities as conformism, opportunism, and expert competence can be identified. In particular, to assess the level of competence of a specialist, the method of paired comparisons can be used. Its essence lies in the fact that the expert is sequentially presented with pairs of compared evaluation objects and he selects from each pair the most preferable object from his point of view. After all pairs are presented, analysts have information about the expert’s comparative preferences regarding the objects being assessed. In this case, a situation may arise when, in a direct comparison, the first object is preferable to the second, the second is preferable to the third, but, at the same time, the third object turns out to be preferable to the first, i.e. an obvious contradiction arises. In real practice, such contradictions (of course, in longer chains of comparison) do not happen so rarely. Naturally, the fewer contradictions in an expert’s assessments, the more justified his competence is.
The a posteriori method also involves assessing the reliability of the expert’s judgments. As a criterion for assessing such reliability, the reliability coefficient is used - the relative frequency of cases in which the expert attributed the greatest probability to subsequently confirmed events. When using this method, the average value of the reliability coefficient of the expert commission is also calculated and the individual coefficients of the experts are compared with it.
Another method of a posteriori assessment is the method of deviation from the resulting group estimate. This method is based on calculating the deviation coefficient - Co.
TOoi = D oi / D max,
Where To oi- coefficient of deviation of judgments of the i-th expert, D oi- deviation of the individual assessment of the i-th expert from the resulting assessment, D max - the maximum possible deviation of the expert's assessment from the resulting assessment.
Test methods for assessing the qualities of an expert. Their essence is the execution by the subject expert of some previously prepared task. These methods are quite well known, because are widely used to determine the professional suitability of specialists in various fields of activity. The advantage of these methods is that they allow not only to establish whether an expert has a certain professional level, but also to identify the skills and experience necessary for productive participation in the work of the expert commission. It must, however, be remembered that conducting test experiments requires compliance with the following important rules:
· the test must be developed specifically for specific objects of expert assessment;
· the true values of the estimated parameters (correct answers) must be known analytical group, conducting the test, but are unknown to the expert being tested;
· a scale should be developed to determine the accuracy of assessments given by an expert;
· the probability of accidentally guessing the true estimate should be very small.
The use of test methods makes it possible to evaluate such an important professional quality of an expert as the reproducibility of expert assessments. To do this, several test experiments are carried out, close to the conditions of real examination. In this case, the time interval between them should be sufficient for the subject to have time to forget the results of the previous one. Then the obtained estimates are compared with each other. More stable (reproducible) expert assessments indicate that he is more professional competence, and about greater suitability for participation in the expert commission.
Expert assessments often arise in practice, for example, when tasting food products. In general, they are typical for sociological surveys, for example, a quality control manager conducts a survey of customers in a supermarket. Using expert method An order scale is often used to assess quality. The issue of comparison is resolved according to the principle “better is worse”, “more is less”. More detailed information how many times better or worse is often not required.
Pairwise comparison (objects are compared to each other in pairs). When constructing an order scale or so-called ranked series, experts use the method pairwise comparison. In tab. Figure 1 shows an example of ranking six objects using pairwise comparison. This is the result of the work of one expert who assessed objects in a certain way. Preference for one object over another is indicated by 1, the opposite situation is indicated by 0.
Ranked series(order scale) for objects, the comparative assessment of which is given in table. 1 will look like:
Q4 If you use several experts, you can get a more accurate result. You can use more advanced criteria, for example, advantage is determined by a score of 1, worse quality is determined by a score of -1, and equivalent quality is determined by a score of 0. The mechanism for compiling a ranked series remains the same. Psychologists have proven that pairwise comparison underlies any choice (i.e. you choose products by comparing them in pairs), however, the order scale is often drawn up in advance (not a ranked series) and reference points are fixed in it, which are called points. This is how the twelve-point scale of earthquake intensity MSK - 64, the Mohs mineralogical scale, the five-point scale for assessing knowledge, points in figure skating, etc. appeared. Table 2 shows the Mohs mineral hardness scale as an example. Object number Bottom line Table 1. Ranking of six objects by pairwise comparison Table 2. Mohs scale Each subsequent mineral leaves a scratch on the previous one, i.e. is more solid. The measurement results obtained by pairwise comparison can be refined by the method of successive approximation. When forming an expert group, it is advisable to conduct testing, mutual assessment of experts and check the consistency of opinions. Testing consists of experts solving problems with results known to the testing organizers, but unknown to the experts, and testing, using the Fisher criterion, the hypothesis that the assessments of different experts belong to the same general population of assessments. Self-assessment consists of each expert answering questions in a specially designed questionnaire within a limited time. This test is carried out on a computer and then scored. Experts can evaluate each other, but this requires a trusting environment and experience of working together. The consistency of expert opinion can be assessed by the value of the concordance coefficient: Where S- the sum of squared deviations of all rank estimates for each object of examination from the average value; The concordance coefficient varies in the range 0<W<1, причем 0 - полная несогласованность, 1 - полное единодушие. It is necessary to determine the degree of consistency in the opinions of five experts, the results of which ranked seven objects are shown in Table 3. To determine the degree of consistency, a special measure is used - the Kendall concordance coefficient (from the Latin. concordare- bring into line, organize). Examination object number Expert assessment Sum of ranks Deviation from the average Square deviation Table 3. Data for assessing the consistency of opinions of five experts We estimate the arithmetic mean number of ranks: Q avg = (21 + 15 + 9 + 28 + 7 + 25 + 35)/7 = 20. Then we estimate the sum of squared deviations from the mean: S = 630. We determine the value of the concordance coefficient: W = 12 * 630 / 25 * (343 - 7) = 0,9. Is it a lot or a little? If we carry out the corresponding calculations in STATISTICA, then you can get the following table of results: Rice. 1. Results of analysis in STATISTICA From this table it follows that different expert opinions in this example are insignificant: p<0.00014. Influence on the results of the examination of the quantitative composition of experts. As the number of experts in the group increases, the accuracy of the measurement increases, which is typical for repeated measurements. Number of experts n, ensuring a given measurement accuracy, can be established by knowing the law of distribution of expert opinions and the maximum permissible standard error of estimation Sx Expert methods
- assessment methods carried out by a group of experts under conditions of uncertainty or risk. Expert methods
are used to determine the nomenclature of quality indicators, their weight coefficients, to measure quality indicators and evaluate them using the organoleptic method. Assessment of quality indicators by measuring, registration, and calculation methods is used to determine complex quality indicators of various levels of the hierarchy. Designed for expert assessment of goods in cases where other previously listed methods are inapplicable or uneconomical. Expert methods are based on making heuristic decisions, the basis for which is the knowledge and experience accumulated by experts in a particular field in the past. Expert methods have certain advantages and disadvantages. The advantages are that they allow decisions to be made when objective methods are not receptive. Other advantages include their recoverability. The scope of application of these methods is not only the assessment of the quality of goods, but also the study of technological cycle operations, decision making, management, forecasting. Expert methods used by qualified experts allow accurate assessment of goods. The experiments conducted indicate that with the correct expert assessment methodology, the error in the results is 5-10%, which is acceptable for measurement methods. The results of expert assessments obtained experimentally in various groups of experts showed their fairly high restoreability. The disadvantages of expert methods include subjectivity, limited application, and high costs of their implementation. The subjectivism of expert methods is a consequence of the fact that expert assessment is carried out by each expert individually and represents, according to E.P. Raikhman and G.G. Azgaldov, “nothing more than his psychological reaction to the physical and chemical characteristics of the product.” However, it should be borne in mind that an expert assessment consists of the opinions of several experts, each of whom is not only a specialist in a certain field of knowledge, but also a consumer. Therefore, expert assessment to a certain extent reflects the opinions of consumers, which is impossible to achieve with other methods. Due to a significant amount of subjectivity, expert methods have certain limitations. their use is rational in two cases: firstly, when the goals set for experts cannot be solved by other methods; secondly, when available alternative methods give less accurate and reliable results or are associated with greater costs. To eliminate this drawback, expert methods when conducting commodity examination are combined with other methods. Most often, expert and organoleptic methods are used together. Moreover, when developing organoleptic scoring scales, choosing a range of quality indicators, and determining weight coefficients, expert methods are indispensable.
Expert methods are divided into three subgroups: 1) methods of group questioning of experts; 2) mathematical and statistical methods for processing expert assessments; 3) methods for expert assessment of quality indicators. Each group of expert methods is in turn divided into types and varieties. The classification of expert methods is presented in Fig. 1.3. Rice. 1.3. V Methods of group questioning of experts - methods based on conducting a survey of a group of experts with subsequent analysis and processing of the information received from them. The purpose of these methods is to obtain group expert judgment to make final decisions. The basis for the choice is the need to make complex decisions in a situation of uncertainty or to draw up a scientifically based forecast, requiring the participation of a group of independent and competent specialists in a narrow field or many fields of knowledge (for example, knowledge of a homogeneous group of products or all food products). The main advantages of group expert assessment lie in the possibility of a comprehensive analysis of quantitative and qualitative aspects of the problems of determining and/or forecasting individual characteristics of goods or their totality. Interaction between experts allows one to significantly increase the amount of total information owned by a group of experts in comparison with the information of any member of the group. In addition, the number of factors taken into account during the group assessment and influencing the effectiveness of the decision made is greater than the sum of the factors taken into account by one expert. With a group assessment, there is less error in making basic decisions and indicators that are not significant for solving the problem. Therefore, an important advantage of group assessment is the ability to obtain a generalized result. Disadvantages of group assessments include: difficulty in obtaining reliable and consistent assessments; receiving different answers to the same question with a large difference in opinions due to different competences of experts; receiving unambiguous answers does not guarantee their validity and reliability, and this cannot be verified during an examination; a greater amount of incorrect information from a group of experts than from an individual expert can lead to significant errors in the final results; the possibility of confrontation, when individual experts, due to uncertainty or other reasons, may agree with the opinion of the majority. Despite these shortcomings, it has been experimentally established that, if certain requirements are met, group assessment is more reliable than individual assessment. These requirements include: an acceptable distribution of grades; group reliability; preparation of the examination. The effectiveness of the examination depends on the accuracy and reliability of the results obtained, that is, on the methods used and on the qualifications of the expert. Choosing an expert is a difficult task; most often, several personal properties are taken into account: competence - professional and qualimetric, the expert’s interest in the results of the examination, attitude to the matter, objectivity. Traits such as risk appetite and other psychological characteristics are almost not taken into account. A systematic approach to assessing the quality of an expert has not yet been developed; existing assessment methods are divided into five groups: Heuristic; Statistical - assessments that are obtained as a result of analyzing expert assessments with determination of deviations from average values; Test - assessments obtained as a result of test tasks performed by experts; Documentary - assessments of competence obtained from the analysis of individual documentary data from examinations conducted by an expert; Combined - estimates obtained during the analysis of data obtained by combining the listed methods. Heuristic
assessments include self-assessment and assessment made by a team of experts. Self-esteem is most often subjective; To reduce subjectivity, a point scale is used for individual properties of the expert. Self-assessment is carried out accordingly according to the types of goods and quality indicators, for example, the aesthetics of products; the expert evaluates himself through a questionnaire. The questionnaire includes the frequency of familiarization with modern domestic and foreign literature, with modern product samples, and with the results of sociological surveys. As a type of self-assessment, the method of assessment based on reasoning and familiarization with the products that are being analyzed is used. The assessment is carried out using a questionnaire to determine the competence coefficient. By using test scores
it is possible to assess such important properties of an expert as qualimetric and professional competence, objectivity. Qualimetric competence when conducting organoleptic assessment consists of repeatedly interviewing experts to determine weight coefficients several times and determining the reliability of the assessments. Professional competence is tested in the form of control of the ability to use various types of rating scales - order, ratios, intervals, as well as the ability to distinguish a significant number of characteristics, gradations when assessing the properties of the product that is being analyzed. With the participation of experts in collective assessment and discussion, the phenomenon of conformity is observed, that is, the expert falls under the influence of the conclusions of other experts, which negatively affects the formation of an objective assessment of the expert. Therefore, the ability to adhere to one’s opinion and defend one’s conclusions is of positive importance during the examination by a collective method and the formation of an objective assessment. Expert methods
quality assessments are based on the use of expert judgment. They are used when it is impossible or impractical or uneconomical to use measurement or calculation methods. This occurs when there is a lack of information, the need to use and develop special technical means, when assessing aesthetic quality indicators, and the like. Expert methods can be combined with other methods or used as an independent type during the assessment of the quality of regulatory documentation for products and products, determining the nomenclature of indicators and their weight coefficients, when selecting basic samples and quality indicators, during the determination and measurement of quality indicators using the organoleptic method, during time for assessing single and complex quality indicators determined by measurement or calculation method. The basis for expert quality assessment is the objective social usefulness of the product, which reflects its modernity. The opinion about the quality of products expressed by a qualified expert meets public needs and coincides with the opinion of the mass consumer. However, when choosing a product, the consumer takes a conservative position and accepts new products cautiously, and sometimes does not perceive them at all. The expert opinion on the quality of the product summarizes the totality of opinions of product consumers. Research has shown that the recorded assessments of product quality by experts coincide with the assessments of product quality that were received during a mass consumer survey. Determining the accuracy of expert research for compliance with the examination methodology showed that it is 5-10%. In relation to the quality of products and their consumer properties, expert methods are used in the following cases: Determination of the nomenclature of indicators during the examination; Selecting criteria for evaluating a product or product and placing indicators according to the principle of hierarchy; Determining the weighting coefficients of quality indicators to determine the level of quality of a product or product; Study of indicators by the organoleptic method using methods of their quantitative expression; Assessment of quality indicators using measuring, registration, and calculation methods to determine complex quality indicators of various levels of the hierarchy. The most commonly used expert methods are: Leading expert (single); Commissions; combined. The leading expert method makes it possible to quickly conduct an examination, reduce the time for the approval and discussion procedure, and statistical processing of data by group members. However, the results of an examination conducted by one expert depend on the level of his professional knowledge, personality, and level of competence. The expert commission method involves the participation of a group of specialists who conduct analysis and assessment. This method allows you to obtain reliable, objective results, but requires a significant investment of time in preparing and organizing the examination. The number of experts who form the commission depends on the required accuracy and reliability of the examination results. The expert commission consists of two groups - working and expert. The working group prepares, organizes and conducts expert assessment of product quality, and subsequent analysis of its results. The working group includes an organizer, a consultant who has professional knowledge about the product, and technical workers evaluate it. An expert group may consist of several subgroups, each of which specializes in solving relevant problems - determining the range of indicators, assessing individual groups of indicators, etc. The combined method, which is based on the sequential use of the work of a leading expert and a small expert commission, is used in some cases. There are no expert methods in the General Classification. However, classification is used depending on the ratio of the amount of data obtained by an expert or analytical method, the method of obtaining information from an expert and some other factors. Depending on the method of obtaining information from an expert, the following methods are distinguished: Collective; Individual. With the collective method, the technical worker conducts a survey of the entire group of experts at once, with the individual method - each expert separately. To obtain reliable results, it is necessary to clearly and correctly set the goal and objectives for the expert; with the group method this is difficult, but possible with a constantly formed group. In the individual method, interviews, interview questionnaires, questionnaires, and mixed questionnaires are used. Surveys can be conducted in person or by correspondence. In the face-to-face method, the expert expresses his judgment to the person conducting the survey. During an absentee survey, there is no contact between them and the expert fills out the questionnaire or survey card. During the interview, the technical worker makes a note in the form of a conversation, which goes through the program and a certain list of questions. During an interview-questionnaire, the list of questions is more specific, targeted, and the sequence of questions is strictly defined. The questionnaire is filled out in the presence of an expert. The questionnaire differs in that the expert fills out the questionnaire independently, having an explanatory note on how to fill it out. A mixed survey involves a preliminary explanation from the expert on filling out the questionnaire with clarification of the task. In terms of information content, interview methods have the greatest potential, interviews - questionnaires, questionnaires. The greatest independence of judgment is characteristic of the survey method. To solve complex situations of uncertainty or during the formation of a scientific and technical forecast, examination requires the participation of a group of erudite specialists who are knowledgeable in many fields of knowledge. The main advantage of participatory assessment is the possibility of comprehensive analysis of quantitative and qualitative aspects of problems. There are problems that cannot be solved without the participation of specialists. It is assumed that the opinion of a group of experts is more reliable than the opinion of an individual, that is, two groups of equally competent experts are more likely to find an objective solution. Combining operations for preparing and conducting expert surveys, as well as technical operations, made it possible to create several expert methods that have received the greatest recognition and distribution. These include Delphi, PATTERN and combined methods. Delphi method
(in some sources - Delphi) - a method of questioning experts, based on sequentially implemented procedures that are aimed at forming a group opinion on procedures with insufficient information. The Delphi method was developed at the American research company RAND Corporation by V. Helmer, N. Dolkey and T. Gordon. It was used for military scientific and technical forecasting of the future. The term Delphi comes from the name of a town in Greece where the oracles lived at the temple of the god Apollo. Features of the Delphi method are: refusal to work together among experts; anonymity of assessments; adjustable feedback; group response. Refusal from joint work of experts and anonymity are achieved by the fact that each expert expresses his opinion in the questionnaire, without group discussion. Other technical methods of individual questioning are also used, for example, answers to questions are entered by experts into a computer. This allows you to reduce the discrepancy between individual assessments and obtain a group answer that correctly reflects the opinion of each expert. The anonymity of the survey makes it possible to reduce the conformist authoritarian influence of individual dominant experts; regulated feedback reduces the influence of individual and group interests. The introduction of feedback also increases the criterion of objectivity and reliability of assessments. When using this method for the purpose of expert assessment of the quality of consumer goods, the following disadvantages turn out to be: complexity of interviewing experts and filling out questionnaires, labor-intensive assessment due to a large number of quality indicators (sometimes up to 20-40) and filling out several questionnaires (3-10), cumbersomeness records of explanations due to the lack of direct contact between the organizer and the experts. The method is promising for obtaining a group expert assessment and in-depth analysis of events in situations of uncertainty. PATTERN method
- a method of questioning experts, based on building a hierarchical structure - a tree of goals - and deciding on these goals after open discussion. The name of the method consists of the first letters of English words meaning "Aiding planning by quantifying technical data." The method was developed by the American company Honkuell to evaluate projects for new weapons systems. The method has analogues: PROFILE, in France - the KPI method, PROPLEN and the like. The PATTERN method involves several stages. Stage I - stating the main problem that needs to be solved and dividing it into a number of secondary problems of the first, second, etc. order, which are then divided into narrower tasks. The division continues until simple elements are obtained that can be evaluated by experts. As a result of this division, a hierarchical structure of interconnected primary, secondary problems and tasks, called a goal tree, is obtained. Stage II - determination, with the help of experts, of the coefficients of weight (or significance) of each task in relation to the main goal, while the experts make a decision after an open discussion in the expert group. Such an open discussion, along with a positive factor - the interaction of experts who strive to make a positive decision - also has negative consequences due to conformism, i.e. distortion of the actual opinion of experts due to suggestion or adaptation to the opinion of the majority. Stage III - use of a computer to process the received data and analyze it. The advantages of the PATTERN method are the simplification of the expert survey procedure. Disadvantages: lack of justification for the optimal number of members of the expert group, as well as methods for selecting competent specialists to the expert group; processing survey results without taking into account differences as individual experts; absence of barriers to experts' conformity; insufficient development and uncertainty of the principles for constructing a goal tree. Since the Delphi and PATTERN methods have significant shortcomings and do not fully meet the goals of peer review, Oe. L. Raikhman and G. G. Azgaldov proposed a combined method in which the positive features of other expert methods were used and their disadvantages were eliminated. Combined method
- a method based on a combination of individual and collective expert assessments. The advantages of the combined method are sufficient flexibility, which allows you to eliminate errors when interviewing experts and increase the reliability of the examination results, a clear definition of the strategy by classifying tasks by degree of importance and operations for their implementation, and high reproducibility of the results. The disadvantages of the method include its extensive operation, which requires a significant investment of time and money. However, this disadvantage is compensated by increased reliability and recovery of results. To assess the quality of goods, the combined method has a general algorithm of expert operations: 1. Preparatory stage: Formation of a working group; Formation of an expert group; Classification of products and consumers; Construction of a structural diagram of quality indicators. 2. Stage of obtaining individual expert assessments: Choosing a procedure for assigning assessments by experts; Choosing a method for obtaining information from an expert and preparing documents necessary for the survey; Expert survey. 3. Stage of obtaining collective expert assessments: Generalizations of individual expert assessments; Determining the consistency of individual expert assessments; Determining the objectivity of collective expert assessments. For each stage there are corresponding tasks. During the preparatory stage, tasks such as determining the functions and structure of the working group, its quantitative composition, and the responsibilities of individual members are solved. The principles for forming an expert group are determined and developed: regarding the number of experts, their professional training, etc. At the second stage, the technique of interviewing experts is determined, the presence of contact between experts, the method of transmitting information and the form of expert assessments are assessed. Experts can determine ratings on their own or after discussing with other experts or consulting anonymous experts. Assessments are justified, quantified and dichotomous, where answers are given in the form of “yes”, “no” or 0-1. Rational use of information received from experts is possible if it is converted into a form convenient for analysis, preparation and decision-making. The possibilities of converting information into appropriate forms depend on the specific features of the object, the completeness of data about it, reliability, the level of decision-making, as well as on the adopted criterion, depending on the problem being studied. One element common to many expert methods is the weighting factor. Weight factor is a quantitative characteristic of the degree of significance of a particular indicator for assessing quality. Determination of weight coefficients for quality indicators is carried out using the expert method. Weight coefficients are designed to increase the reliability of expert assessment of the quality of goods. Each indicator occupies a certain place in the nomenclature of quality indicators in terms of importance. Experts rank indicators by degree of importance based on professional knowledge and skills. In addition, any qualified expert seeks to evaluate the quality indicators of the examined product from the perspective of the mass consumer. If an expert has the ability to compare and evaluate possible options for action, giving each of them a certain number, then he has a certain system or scale of preferences. Correct use of scales is essential to ensure the accuracy of expert assessments. The following types of scales are distinguished: nominal, ordinal, interval, ratios.
But the order scale has the greatest advantages over others due to the relative simplicity of expert assessment of quality indicators by degree of significance. Characteristics of the scales are given in table. 1.1. Nominal name scale
used to distinguish one object from another. Objects must be numbered, but the numbers indicate the object, not its quantitative characteristic. This is a simpler type of measurement in which numbers or symbols are used only to classify objects. The scale can be used for digital coding of individual properties in questionnaires and for determining weight coefficients. Ordinal scale (ranks)
- an assessment method in which the parameters, indicators or objects that are being evaluated are arranged in order of increase or decrease in the indicator of the parameter (indicator) or properties of the object. A classic example of evaluation using an ordinal scale is the assessment of the hardness of minerals using the Pug scale (the relative hardness scale consists of 10 hardness standards, with the hardness of talc taken as 1, and diamond - 10). This method can be used to determine the color intensity of flour, the aroma of fruit juices, the bouquet of wines, and the consistency of cheeses. The ordinal scale has advantages when used to determine weight coefficients, since it simplifies the process of ordering quality indicators by importance for consumers. Table 1.1.
Types of scales and their characteristics Scale type Definition of scale Relationships specified on the scale Nominal A simple type of measurement in which numbers or symbols are used only to classify objects. Equivalence (=) Ordinal (ranks) Objects of one class are in a corresponding relationship with 3 objects of another class (more than, more advantages, stronger, etc.). If [A]>[B] for some objects of classes A and B, then there is a partially ordered scale. Equivalence (=). Greater than (>). Interval An ordinal scale, which is divided into known distances between any two numbers on the scale, the zero point of the scale and the unit of measurement are chosen arbitrarily. Equivalence (=) Greater than (>). The ratio of any two intervals is known. Relationships Interval scale using true zero point, the ratio of any two points is independent of the unit of measurement. Equivalence (=). Greater than (>). The ratio of any two intervals is defined. The relationship between any two points is defined. When determining the weight coefficients of quality indicators, experts first evaluate the most important of these indicators (in their opinion) and assign a certain number to it, for example 1. All further indicators are evaluated in decreasing or increasing order by degree of importance. After this, the data of all experts is averaged for each indicator. In the practice of assessing the quality of goods, a method for determining weight coefficients, called the “fixed amount method,” is often used. Its essence lies in the fact that experts assign a weight coefficient for the indicators included in the top-level indicator, and the sum of these coefficients must be equal to a predetermined number. The “fixed amount method” is advisable to use only with a small number of indicators. Practical experience shows that when expertly assessing the quality of goods, it is advisable to use the following procedure for determining weight coefficients. 1. Preliminary ranking by experts of indicators of a homogeneous group. Rank 1 is assigned to the most important indicator, 2 to the next most important, and so on. If indicators are equal in importance, then they are assigned the same ranks. The number of indicators in a homogeneous group should be 4 or more. If the number is smaller, ranking is not carried out. 2. Determination by experts of indicator weight coefficients. An indicator of the 1st rank is assigned a weight coefficient of 10. The weight coefficient of the next most important indicator is determined as the proportion of the importance of the first indicator. When determining the third and subsequent indicators, their importance in comparison with the previous ones is taken into account. As a result of these sequential actions, the expert determines the weight coefficients of individual and then complex quality indicators. 3. Familiarization of experts with the values of weight coefficients (their justifications) assigned by other experts. Justification of weight coefficients is a very labor-intensive operation, therefore it is used with a limited number of indicators (about 10-15). Otherwise, experts are asked to provide justification only for some indicators at their discretion. 4. Averaging the values of weight coefficients determined by all experts. Conducted by technical workers by calculating the arithmetic average or weighted average. In the latter case, a comprehensive assessment of the expert’s quality is taken into account. The disadvantages of the scale are the inaccuracy of ranking estimates due to the lack of interval equations, the impossibility of calculating even the arithmetic mean. Interval scale.
This is an assessment method in which the essential characteristic is the difference between the values of the evaluated parameters, which can be expressed by the number of units provided for on this scale. Using such a scale, objects are ranked, and in certain units it is determined how much one object is larger than another. An example of an interval scale is the Celsius scale, which is divided into 100 equal intervals and is used to characterize product properties that are associated with temperature conditions, for example, the frost resistance of synthetic leather, the minimum temperature of the freezer in the refrigerator. Relationship scale
allows you to achieve the highest level of measurement. This is an evaluation method that uses a unit of measurement, it is used for most parameters that represent physical quantities: size, weight, density, force, voltage, frequency, and the like. The results of measurements on a scale have the properties of numbers that can be subjected to statistical processing. An example of such a scale is the Kelvin temperature scale, which begins at absolute zero. When comparing the interval and ratio scales, it was determined that the latter produces more accurate results. In addition, interval scale scores can be used to calculate weighted averages, the calculations of which are typical for experimental surveys. The ratio scale can be the basis for the expert method; the order scale can be used with sufficient justification. In the practice of expert assessment, two main types of scales are used: dimensional and dimensionless. Data from dimensionless scales are expressed in fractions of a unit, percentage, and points. Mathematical and statistical methods for processing expert assessments are methods designed to increase the reliability of the results of assessing the quality of goods by experts. They are divided into four subgroups: ranking, direct evaluation, sequential preferences and paired comparisons. Ranging
- a method based on the arrangement of objects of examination in ascending or descending order. Designed to solve many practical problems when the objects that determine the final results cannot be directly measured. In addition, individual objects characterized by different natures are incompatible, since they do not have a common measure of comparison. The basis for ranking is the need to organize any object in time and space, as well as in accordance with the quality being measured without making precise measurements. And finally, in a situation where the quality that is being measured cannot, in principle, be measured for practical and theoretical reasons. The ranking procedure consists of an expert placing objects in the most rational order and assigning them a certain rank in the form of a natural number. In this case, the most important object receives rank 1, and rank n - the least important. The result is an order scale in which the number of ranks is equal to the number of objects. If two objects have the same ranks, then they are assigned so-called standardized ranks, which are calculated as the average of the sum of the places of objects with the same ranks. For example, six objects are assigned the following ranks: Objects 2 and 5 shared 2nd and 3rd places. their standardized rank will be equal to (2 + 3) /2 = 2,5. Objects 3, 4 and 6 shared 4th, 5th and 6th places, and their standardized rank is 5: (4 4-5 + b)/3 = 5. The result is the following ranking: The ranking method is rarely used in its pure form. Most often it is combined with the direct assessment method or its modifications (ranking by sum of assessments, combined method, etc.). Direct assessment method
is that the range of change of any quantitative variable is divided into several intervals, each of which is assigned a certain score, for example from 0 to 10. The rating scale can be positive and negative, for example, from +3 to -3.Expert must include each object in a certain interval depending on its value. The number of intervals into which the entire range of quality changes is divided may be different for different experts. Individual experts are allowed to evaluate qualitatively different factors with the same number. In some cases, it is more convenient to select the most preferable factor by first assessing and then ranking. Total rank estimates can be normalized, this allows us to establish a closer connection between the ratings that experts provided to individual objects. For this purpose, assessments for all objects are summed up and subsequently each of them is divided by the resulting amount. The normalized scores calculated in this way can be ranked again. When conducting an examination by several experts, they strive to obtain an average assessment for each object. For this purpose, the normalized assessments of each object are summed up, the resulting amount is divided by the number of experts. The second way to determine the relationship between factor assessments is that the most important factor is given a score (weight is set) beyond a certain number of 1 or 10, the following factors are assessed as a share of the most important factor. The advantage of the method is that it simplifies the process of selecting assessments, because you do not need to compare the entire series each time, but only take into account the value of the first and previous most important assessment. Scores are averaged by calculating the arithmetic mean. Sequential Advantage Method
- based on comparison of an individual object with the sum of subsequent objects to establish its importance. used in measuring the level of quality and assessing the activities of scientific organizations. The method has such a main advantage compared to other methods, which makes it possible to compare and measure qualitatively different factors. The method was developed by V. Churchman and G. Ackoff and is intended for making comparisons subject to certain tolerances. The order in which results are presented or how they are grouped does not affect the benefits. The procedure for sequential comparisons is as follows. The expert is presented with a number of objects (indicators, factors, results) that need to be assessed according to their relative importance (significance), and he makes a ranking. The most important object is assigned a score of 1, the remaining objects are given scores below 1 to 0 in order of their relative importance. The expert then determines whether an item rated 1 is more important than the sum of the latter factors. If the importance of an object is high, then it increases the score so that it is greater than the sum of all others. If the value of an object is lower than the sum of all others, it adjusts the estimates. Thus, the procedure used is to systematically test the estimates by comparing them successively. It is advisable to use the method of sequential preferences if the number of objects being compared does not exceed 7. With a larger number of objects, they must be divided into subgroups that include 6 objects. In cases where this is not possible, the paired comparison method should be used. Paired comparison method
- based on comparing the objects of examination in pairs to establish the most important in each pair. It is used to identify advantages among a significant number of factors, problems, and indicators. Experts can simply make comparisons stating the superiority of one factor over another. It is possible to use a special preference scale, where each degree of advantage has its own specific rating. The method of paired comparisons can also be used to establish the total ranks of factors. To facilitate the procedure, matrices of paired equations are compiled in which all objects (factors) are written in the same order twice: in the top row and the leftmost column. Each expert must mark the intersection of the row and column for the two factors being compared. Depending on which factor is the most important, this score will be equal to 1 or 0, respectively. Dashes or zeros are placed in the main diagonal of such a matrix (Table 1.2). Table 1.2.
Each pair of factors can be compared once or twice. There are various options for partial pairwise comparison: choosing a preferred object from pre-grouped pairs; partial pairwise comparison of one group of objects with all others, while the remaining factors are compared with some others; establishing the total ranks of factors. The paired comparison method is sometimes combined with a preliminary ranking of objects, and paired comparison is used to clarify the advantages of individual objects. In this case, an additional matrix is constructed, which indicates the proportion of cases when one factor turns out to be more significant than another, in the total number of estimates received. Methods for expert assessment of product quality indicators- these are methods for determining the actual values of single and complex quality indicators. Designed to determine the values of the quality indicator by calculation or heuristically in cases where the use of measurement methods is impossible or uneconomical due to excessive costs of their use or long test times. For example, when determining the taste and smell of food products, only organoleptic methods are used. Measuring methods do not provide accurate, reliable estimates, despite the increased costs. For differential and comprehensive assessment of samples that differ significantly in quality, it is recommended to determine the value of a single P indicator as follows: where P;5 is the base (reference value). Another, more accurate method is based on the study of indicators in order to determine the types of dependence and, i.e., in order to develop formulas for calculating estimates of indicators: Evaluation of single quality indicators
begins with determining the permissible intervals of their change (P; - P;) P; - the best value of the indicator, exceeding which is impractical or impossible. The principles for assigning the maximum permissible value of an indicator depend on the purpose of quality assessment, and it is necessary that this principle be the same for all indicators. You achieve increased reliability of expert assessments by dividing complex operations into simple ones, which constitute a multi-stage procedure for assessing acceptable values of an indicator. The transition to each next level is carried out after making agreed decisions to the previous one. The expert procedure for determining acceptable values of quality indicators consists of a number of operations: issuing questionnaires and explanatory notes to experts, which list quality indicators and describe the principles for selecting acceptable indicator values; experts filling out questionnaires and indicating specific product models whose values they consider to be maximum permissible; familiarizing each expert with the assessments indicated by other experts and discussing them; conducting the second (sometimes third and fourth) round of questioning; Averaging assessment results. If there is a significant difference of opinion, an additional round of voting is held. The indicator value is taken as the maximum if at least 70% of the votes are cast for it. If this condition is not met, the average of 50% of the largest rimax values is taken as the maximum permissible value, and the average of 50% of the largest values is taken as the minimum permissible value. The values are used by experts when determining assessments of quality indicators K. To determine by experts the type of dependencies (i) between the values of indicators G; and their estimates of K often use the “principal method of points.” The need to use it is due to the fact that dividing the assessment procedure into several stages simplifies the expert’s work and allows him to give estimates to some characteristic points, based on which it is possible to build a model of the actual value. "Principal point method"
Depending on their number, it has several varieties. "Method of three main points" - based on the separation of the values of indicators G; for maximum, minimum, average values and determining the values of P estimates; at these points. The scale interval between the maximum and minimum points is preset (scales 0-1 or 0-10). The expert’s task also includes determining the dependence trend in the interval between the main points and constructing a graph. After this, from the graphical dependence you can move on to the analytical formula for calculating estimates of the quality indicator Ki. The “Three Main Points Method” allows you to develop only an approximate assessment model. “Method of seven main points” is a method of assessing on a seven-point scale for assessing indicators, the values of which are determined experimentally or by calculation, as well as by an organoleptic method. The seven-point scale is uniform, that is, when moving from one quality class to another, the score changes by one point. These scales are widely used, especially in organoleptic assessment. To obtain more accurate results, you should proceed to determining the type of relationship between grades and scores. To facilitate the expert’s work, five graphs are provided in the explanatory note to the questionnaire. The expert selects the curve (or combination of curves) that best, in his opinion, reflects the nature of the dependence And
Then each quality class is assigned a score in accordance with the nature of the dependence and the values of the quality indicators. In this case, it is advisable to use numbers in the range 0-10, multiples of 0.5, with the “highest quality” class receiving a score of 10. Thus, the graph constructed by the expert characterizes the relationship between the absolute values of indicators Pi and their estimates of K;, and for indicators assessed by organoleptic methods, between quality classes and their estimates. In conclusion, the results obtained are discussed, processed and analyzed. For indicators that are determined by measurement and calculation methods, it is advisable to give an analytical description of the curves, which allows you to calculate an estimate for any values of the indicators. The use of the “principal point method” makes it possible to group and classify indicators by type of dependence. Determination of complex quality indicators is carried out by two types of methods: methods of comprehensive assessment of the quality of product samples; methods for constructing models of complex quality indicators. Methods for comprehensive quality assessment have two varieties - the express method and methods of moving through levels without preparation and with preparation. Express methods for comprehensive assessment of the quality of product samples
are based on determining a complex quality indicator by analyzing the values of individual individual indicators and appearance without preliminary assessment and taking into account weighting factors. When using these methods, it is necessary to take into account that the maximum number of indicators to be assessed, even for a highly qualified expert, is 7-9 indicators located at the same level of hierarchy, which form a fairly homogeneous group. In addition, experts must take into account the importance of individual indicators using weighting coefficients, the relationship between them, and also consider the quality of the product as a system. Method of moving through levels without preparation
is a set of operations that are carried out sequentially, with a gradual increase in level. In this case, the analysis begins from the bottom level of the indicator tree. Taking into account the values of the indicators of the lower level, the expert gives an assessment of the indicators of the higher level. These operations are repeated with increasing levels until the upper level is reached - a comprehensive (generalized) quality assessment. Method of moving through levels with preparation
is based on preliminary determination by experts of the weighting coefficients of quality indicators and their assessments. When assigning complex assessments, the expert knows the average values of weight coefficients and estimates of individual indicators. The procedure for determining complex indicators is similar to the procedure for the method of moving through levels without preparation. Formalization of the peer review process
consists in finding the relationship between the values of quality indicators G; (or their estimates K) and a quality indicator of a higher level, i.e. in determining the type of decision function that experts use when assigning complex indicators. At the same time, the decisive function, like any model, simplifies the object of study, since not all indicators and connections between them are taken into account. The initial data for determining the type of function can be the results of assessing the quality of various samples using express methods or methods of moving through levels. Then the assessments assigned by the experts are summarized in a common matrix, each row of which is a set of assessments of individual indicators of the sample and complex expert assessments. Based on this, machine algorithms can be developed and programs can be compiled to find decisive functions using a computer. 1. Expert method for assessing the quality of goods: definition, features in the examination of food and non-food products. Expert method 2.Environmental friendliness and safety of goods: definition, significance, types of safety that characterize environmental friendliness and safety. Environmental properties- the ability of the product not to have a harmful impact on the environment during operation. Increasingly, environmental pollution is putting humanity's existence on the brink of disaster. Because of this, the degree of importance of environmental properties increases sharply (for example, for flour, starch, washing powder, vehicles and packaging are reliable). All goods pollute the environment to varying degrees at various stages of distribution. Thus, the greatest environmental pollution occurs during the production of goods or raw materials, materials or semi-finished products. Indicators of the environmental properties of goods are various types of pollution that worsen the safety of the environment. The Federal Law “On Technical Regulation” combines the safety of the population and the environment into safety for life, health of consumers, animals and plants, and for the property of legal entities and individuals. Safety- a very important and widely interpreted property of many objects: goods, processes, services, as well as the environment. Therefore, in Art. 2 of the Federal Law “On Technical Regulation” provides a general definition: Safety is a state in which the risk of harm or damage is limited to an acceptable level. This is the absence of risk to life, health, during the operation of goods.” Product safety is a mandatory requirement and must be regulated by technical regulations. All Consumer Products must have this quality.. Unlike other consumer properties, deterioration or loss leads to loss of functional or social purpose; exceeding the permissible level of safety indicators transfers the product to the dangerous category. In accordance with the Federal Law “On Technical Regulation”, the following are distinguished: types of security
: 1) chemical safety - no risk that can be caused by toxic substances. (Toxic elements - heavy metal salts, food additives, packaging dyes, arsenic, mercury, copper, iron are prohibited); 2) rational safety - absence of harm that can be caused by radioactive elements (building materials: code, cement); 3) mechanical safety - absence of harm that can be caused due to impacts, friction, punctures (mainly for non-food products - clothing, shoes); 4) electrical, magnetic, e/magnetic safety - absence of risk that can be caused by exposure to electric, magnetic, e/magnetic fields during the operation of complex technical goods (electrical goods, for example, ovens); 5) thermal safety (temperature increases) - heating devices must have this; 6) sanitary and hygienic safety (biodamage): a) diseases are caused by microorganisms; b) insects, rodents - zoological damage; 7) fire safety - the absence of unacceptable risk to life and health during storage and operation of goods as a result of fire or spontaneous combustion. 3. Using the example of high-quality acceptance of any group of goods in a specific trading enterprise (store, base), show the following: Availability of a certificate of conformity for safety and quality for the product under study; Violation of what quality factors (raw materials, technological processes, design, packaging, labeling, storage) caused the product to be rejected; According to which ND (full designation and name of the standard or specification) the quality acceptance was carried out and what specific requirements were violated (it is necessary to provide the wording of the corresponding paragraph of the ND where these requirements are reflected); Indicate the acceptance and rejection numbers for the product under study. Acceptance of a batch of textile goods (woolen blankets) was carried out in the “Manufactured goods” store of individual entrepreneur G.A. Belova. According to Russian legislation, the sale of most types of goods to consumers is impossible without a document confirming their compliance with safety requirements. Thus, certification for a number of products is a mandatory condition for their introduction to the market for the purpose of subsequent sale. Blankets are among the goods subject to mandatory certification, since their quality can affect the safety of people. A certificate of conformity is a document confirming the product’s compliance with safety requirements and quality established for these products in technical regulations, codes of practice, current standards and other documentation. The blankets had a sanitary-epidemiological conclusion and a certificate of conformity in the GOST R certification system. Acceptance of goods was carried out in accordance with GOST 20566-75 “Fabrics and piece goods” textile products. Acceptance rules and sampling methods" for compliance of the quality of blankets with the requirements of GOST 9382-78 "Pure wool and half-wool blankets. General technical requirements."According to paragraph 3 of GOST GOST 20566-75, 100% of the products are subjected to quality control of piece products in appearance, compliance of packaging and marking with the requirements of NTD. During an external inspection of a batch of blankets consisting of 20 pieces, one blanket with raw edges was found - the edges of the blanket are not trimmed with tape or overcast. Thus, the requirement of paragraph 1.11 of GOST 9382-78 “Pure wool and half-woolen blankets” was violated. General technical requirements”, according to which the cut edges of blankets must be covered with tape or firmly overcast over the edge on special overcasting machines with cotton threads or threads made of chemical fibers. This discrepancy arose as a result of a violation of the technological process of making the blanket - operations on processing the edges were not performed. Among the methods used in quality management to solve many management problems, expert methods are very often used. At first, expert methods were used mainly to solve problems related to forecasting in the field of science and technology, and then they began to be used in other areas, including quality management. This is due to the large number of contradictions that arise in this type of management, which in many cases cannot be resolved by objective methods. Expert methods are also used in cases of: The inability to use modeling and description of controlled objects using formalized mathematical methods; Lack of sufficiently reliable information; Information uncertainty of managed objects; Development of medium- and long-term forecasts of the influence of laws and regulations on quality management; Existence of trends in the development of quality management and the market environment; The presence of extreme situations in quality management. In such cases, the use of professional experience and the intuition of expert specialists, formed on its basis, acquires inestimable importance. The essence of expert methods, both when solving quality management problems and when using them in decision-making practice in other areas of science, technology and management, is to average the opinions (judgments) of specialist experts on the issues under consideration in various ways. The expert method of assessing the quality level of technical products is used in cases where it is impossible or very difficult to apply methods for objectively determining the values of single or complex quality indicators using methods such as instrumental, empirical or calculation. The expert method (or expert method, i.e. the method of expert assessments) is a combination of several different methods, which are varieties and modifications of the examination method. Well-known varieties of the expert method are used wherever the basis of the decision is the collective decision of competent people (experts). For example, decisions of various councils, conferences, meetings, commissions, as well as examiners when assessing students’ knowledge, etc. -- all these are decisions made by expert methods. Expert methods for assessing product quality can be used in immediately forming a general assessment (without detail) of the level of product quality, as well as in solving many specific issues related to determining the quality indicators of something. Therefore, expert methods are applied: For a general (generalized) assessment of product quality; When classifying the products being assessed; When determining the range of quality indicators for the products being assessed; When determining the weight coefficients of product quality indicators; When assessing product quality indicators using the organoleptic method; When choosing basic samples and dimensionless values of basic quality indicators; When determining the final comprehensive quality indicator based on a set of individual and complex (generalized and group) indicators; During product certification and certification. An expert method for assessing the level of product quality cannot be used if the quality can be assessed by other analytical or experimental methods with greater accuracy or at lower costs. The results of a general expert assessment of such a complex set of properties as product quality have elements of uncertainty and unreasonableness. Therefore, expert assessment of product quality as a whole is preliminary, not saturated with information, and only in a first approximation, it roughly characterizes the quality of the product being assessed. Based on such an expert quality assessment, it is obviously not possible to make any engineering decisions. This method can, for example, be used in commercial transactions when there is no specific (quantified) information about the level of quality of the purchased products, etc. However, it should be noted that the expert method for assessing many quality indicators of technical and other products is the only possible one, it is used quite widely and appropriate methods have been developed for this. The object of examination (expert assessments) in our case is consumer properties in their totality, i.e. quality. The criteria by which quality assessment is carried out are divided into general and specific. General criteria include values, ideas and norms that have developed in society. Specific criteria for an expert are real requirements for the quality of products of this type, established in regulatory, technical and other mandatory documents. In the form of specific criteria, there is also a set of basic values of quality indicators that characterize the planned or designed product. The characteristics of actually existing high-quality products, manufactured in the country or abroad, are also specific criteria for experts. In order to increase the reliability, accuracy, reliability and reproducibility of expert assessments, the examination is carried out by making a group decision by competent people. To assess the level of product quality, an expert commission is created, consisting of an expert and working group. The expert group includes highly qualified and specially trained workers in the field of creation and operation of the products being evaluated: researchers, designers, technologists, designers, commodity experts, economists, etc. The number of experts included in the group depends on the required accuracy of average estimates and should range from seven to twenty people. During a correspondence survey, there is no upper limit on the number of experts interviewed. The expert group (commission) uses an expert method to obtain information about the quality indicators of the products being assessed. In this case, the expert group can make decisions based on averaging experts’ assessments or by conducting expert votes (the “commission” method). In order to reduce subjectivity in the expert method, it is recommended to conduct several rounds of surveys of experiments. The expert method of “commissions” is that it uses a kind of voting. First, experts give ratings independently of each other. Then, after an open discussion of the assigned ratings, the experts again independently evaluate each quality parameter. Subsequently, an expert assessment is calculated from the adjusted individual assessments. This work is carried out by a working group of the expert commission. In addition, the working group organizes a procedure for interviewing experts, analyzes the results obtained and draws up a conclusion of the expert commission. It is desirable that for assessments of products of the same type, the expert commission is formed from permanent experts and members of the working group. This is due to the fact that in the process of work of a relatively permanent commission, work experience is accumulated, its members are trained, general approaches and principles are developed, and this increases the efficiency of the expert commission. The list and sequence of the main stages of the expert commission’s work is as follows: 1) appointment of persons responsible for organizing and carrying out work on expert assessment of product quality; 2) formation of expert and working groups; 3) development of classification and determination of the nomenclature of quality indicators of the products being assessed; 4) preparation of questionnaires and explanatory notes for interviewing experts; 5) assessment and survey of experts; 6) processing of expert assessments; 7) analysis and registration of the results of expert assessment of the quality (or quality indicators) of products. In the practice of expert quality assessment, in particular in the expert assessment of consumer properties of products, complex and operational examinations are mainly used. A comprehensive examination is carried out for a comprehensive study and assessment of the quality of groups of homogeneous products mass-produced by industry. In this regard, during the examination, a systematic, integrated approach to the analysis and evaluation of products is implemented. During a comprehensive examination, one receives not only a more complete description of the object being assessed, but also certain scientific, methodological and regulatory material used in conducting other types of examination. Operational examination is based on data obtained during previous comprehensive examinations. This technique allows you to significantly reduce the volume and time of expert work with sufficient depth and validity of expert opinions. The most common expert methods for classification based on preference assessment: 1. Rank method 2. Direct assessment method 3. Comparison method: Paired comparison Sequential matching Using the method of ranks, the studied objects of an organized system are ranked (ordered) depending on their relative importance. In this case, the most preferred object is usually assigned the first rank, and the least last, equal in absolute value to the number of ordered objects Table 1 - Determination of the resulting rank of ranking objects Thus, this method allows you to determine the place of the object under study among other objects. The advantage of the rank method is its simplicity. The disadvantages are: Inability to rank with sufficient accuracy the number of objects whose number exceeds 15-20; It does not answer the question of how far in importance the objects under study are from each other. This method is used in the practice of studying control systems, despite its simplicity, quite rarely. The direct evaluation method involves ordering the objects under study depending on their importance by assigning points to each of them. In this case, the most important object is assigned (rated) the highest number of points on the accepted scale. The most common rating scale range is from 0 to 1, 0 to 5, 0 to 10, 0 to 100. In the simplest case, the rating can be 0 or 1. Sometimes the rating is carried out in verbal form. For example, “very important”, “important”, “unimportant”, etc., which is also sometimes translated into a point scale (3, 2, 1, respectively) for greater convenience in processing survey results. The use of this method is used only if the experts are confident that they are fully informed about the properties of the object under study, which often does not happen. Table 2 - Determination of the results of direct assessment of objects Ranking object No. Expert No. Sum of object ranks Resulting object rank Object weight Based on the results of expert assessments, the location of any object can be determined using formula (1), where Bi is the importance of the i-th object (i=123….n) calculated on the basis of experts (j=1,2,3,….k) Aij= rating (in points) given to the i-th object by the j-th expert The matching method is carried out by pairwise comparison and sequential comparison. In pairwise comparison, the expert compares the objects under study according to their importance in pairs, identifying the most important one in each pair of objects. The expert presents all possible pairs of objects in the form of a record of each combination (object 1 - object 2, object 2 - object 3, etc.) or in the form of a matrix. As a result of comparing the objects in each pair, the expert expresses an opinion about the importance of one or another object, that is, he gives preference to one of them. Sometimes experts come to the conclusion that each of the objects in the pair is equivalent. Ordering in each pair of objects, of course, does not immediately provide ordering of all the objects under consideration, so subsequent processing of the comparison results is necessary. It is most convenient to carry out pairwise comparisons and their processing using matrices as a tool. In some cases, with a large number of objects under study, the results of paired comparisons are influenced by psychological factors, that is, preference is sometimes given not to the object that is actually preferable to others, but to the one that is written first in the list of pairs or is located in the matrix above the one being compared. Therefore, sometimes, to exclude psychological influence, a double pair comparison is carried out, that is, a pair comparison is made again, but only with the reverse arrangement of objects and, accordingly, the objects in each pair. The method of paired comparisons is very simple and it allows you to study a larger number of objects (compared, for example, with the rank method) and with greater accuracy. The essence of the sequential comparison method is as follows. The expert arranges all studied objects in order of their importance (like the rank method). Each object is preliminarily assigned a certain number of points, for example, on a scale from 0 to 1 (as an assessment method). Moreover, the most important object is given a score equal to 1, and all others are given in decreasing order of their importance, that is, from 1 to 0. Next, the expert decides whether the importance of an object with rank 1 will be greater than the sum of the points of all other objects. If so, then the score of the first object increases until this condition is met, and if not, then the expert reduces this value to such a numerical value that it becomes less than the sum of the scores of all other objects. The values of the ratings of the second, third and subsequent objects in importance are determined sequentially in the same way as the rating of the first most important object. The sequential comparison method is the most labor-intensive for experts. This is especially noticeable when the number of objects being studied is more than six or seven. The processing of collected expert opinions is carried out both quantitatively (numerical data) and qualitatively (substantive information). Various methods are used for this. It should be noted that in the presence of numerical data, to resolve issues with sufficient information material, methods of averaging expert judgments are mainly used. However, even with the available numerical data, but with insufficient information on the issue at hand, along with quantitative methods for processing expert data, methods of qualitative analysis and synthesis are also used. When using the considered expert methods, the opinions of experts often do not completely coincide. Therefore, it is necessary to quantitatively evaluate the degree of consistency of expert opinions and determine the reasons for the discrepancy in judgments. The measure of consistency is, of course, determined on the basis of statistical data from the entire group of experts. The consistency of expert opinions is determined using the concordance coefficient: where C is the sum of squared deviations of the sums of ranks for each object from the average sum of ranks for all objects (N) and experts (K). The general advantages of expert methods are the speed of obtaining results without the presence of a regulatory framework in the management system, the ability to evaluate a particular object when it is impossible to measure its characteristics with quantitative objective methods. The disadvantages of expert methods are their subjectivity and the corresponding possible errors in the examination results, significant costs for attracting experienced experts to participate in expert work, the influence of authoritative members of the expert group and corporate interests on the opinion of individual experts. The application of all the considered expert methods, despite their shortcomings, shows their effectiveness in research and design of control systems. Moreover, the greatest effect is achieved by using several methods simultaneously. expert quality scale
The influence of the composition of experts on the results of the examination
n- number of experts;
m- number of objects of examination.Example
The expert method for determining quality indicators is based on taking into account the opinions of specialist experts. An expert is a specialist competent in solving a specific problem. This method is used in cases where quality indicators cannot be determined by other methods due to insufficient information, the need to develop special technical means, etc.
The expert method is a combination of several different methods that represent its modifications. Well-known varieties of the expert method are used where the basis of the decision is the collective decision of competent people (experts). An expert's qualifications are determined not only by knowledge of the subject of discussion. The specific capabilities of the expert are taken into account. For example, in the food industry, when assessing the quality of food products, the expert’s ability to perceive taste, smell, as well as his state of health is taken into account. Experts assessing aesthetic and ergonomic quality indicators must be knowledgeable in the field of artistic design.
When using the expert method to assess quality, a working and expert group is formed. The working group organizes the procedure for interviewing experts, collects questionnaires, processes and analyzes expert assessments.
The expert group is formed from highly qualified specialists in the field of creation and use of the products being assessed: commodity experts, marketers, designers, constructors, technologists, etc. It is desirable that the expert group is formed not for one examination, but as a permanently functioning body with a fairly stable composition of experts.
