Higher Education Quality: What is the perception of the players involved? Conclusions of a Comparative Analysis
Gilli Juan José; López Armengol Martín A.; Colombo María de la Paz; Mollo Brisco Gabriela
“Visión de Futuro” Año 13, Volumen Nº 20, Nº 1, Enero - Junio 2016
URL de la Revista: http://revistacientifica.fce.unam.edu.ar/
URL del Documento: http://revistacientifica.fce.unam.edu.ar/index.php?option=com_content&view=article&id=409&Itemid=88
ISSN 1668 – 8708 – Versión en Línea
ISSN 1669 – 7634 – Versión Impresa
E-mail: revistacientifica@fce.unam.edu.ar
________________________________________________________________________________

 

Higher Education Quality: What is the perception of The players involved? Conclusions of a Comparative Analysis

(*) Mollo Brisco Gabriela F.; (**)  Gilli Juan José; (***) López Armengol Martín A.; (****) Colombo María de la Paz

(*)Facultad de Ciencias Económicas
Universidad de la Plata
Buenos Aires – Argentina
gabriela.mollo@econo.unlp.edu.ar

(**) Facultad de Ciencias Económicas
Universidad de la Plata
Buenos Aires – Argentina
juanjose.gilli@econo.unlp.edu.ar

(***) Facultad de Ciencias Económicas
Universidad de la Plata
Buenos Aires – Argentina
martin.lopezarmengol@econo.unlp.edu.ar

(****) Facultad de Ciencias Económicas
Universidad de la Plata
Buenos Aires – Argentina
paz.colombo@econo.unlp.edu.ar

Reception Date: 07/23/2015 - Approval Date: 11/24/2015

ABSTRACT

In education there is an ongoing paradigm that challenges the present system to generate significant processes of change. However, student and educator underlying conceptions regarding the learning and teaching processes must be analyzed in advance. Our first approach to this topic is to address one of its specific aspects: Education Quality.
In view of its relevance, research findings about Education Quality, in general terms, and those regarding higher education, in particular, have increased as from the nineties. Nevertheless, the main obstacles, even in these days, are to define the concept of “Quality“ meeting the expectations of all the parties involved in the education process, and to achieve consensus about the key determinants of influence within the sphere of university.
The aim of this paper is to make a comparative analysis of Economic Sciences student and educator opinions at Argentinean universities regarding the meaning of the aforementioned concept. In addition, our goal is to determine which factors affect Education Quality in a positive way.

KEY WORDS: Education Quality; Education Management; University Players; University Students.

INTRODUCTION

The 21st century has seen a new society paradigm arise, a model where information understood as knowledge appears as the founding block for the economic, political, and social development of this new era. Peter Drucker (1993) is renowned as one of the pioneers in perceiving the transformation of a manufacturing, industrial society into a service society, where knowledge becomes a utility and the key workers will be knowledge workers. Under these premises, Universities are positioned as the backbone that supports this new societal model and, therefore, ensuring the quality of higher education becomes a particularly critical issue.
As a starting point, it should be noted that it is hard to associate the term Quality,as it is usually understood, to a well-delimited and accurately defined category. University quality is, first and foremost, a complex and multi-dimensional concept that cannot be separated from its relation to the objectives and players of the university system.
The ECUALE Network, formed by professors from universities in Argentina, Spain, and Mexico, has been working for 5 years in this area, through various research works. All the studies carried out by the Network are based on the idea that university teaching quality cannot be improved if the factors that interact with it are unknown.
The objective of this article is to make an empirical contribution to the study of higher education quality in Argentina. The basic premise of the investigations whose summarized results are presented in this article considers that educational quality, core component in the education of university students, has factors that define it and players (stakeholders) that make it come to being. It is then essential, in order to design and implement public and institutional policies that promote educational quality, that we know and acknowledge how professors and students (main players in the teaching and learning process) perceive this quality and the factors that affect it.
In this sense, we are presenting a comparative analysis of the opinions of educators and students regarding the meaning of the concept of university teaching quality for them, as well as their perceptions in relation to the factors that they believe favorably affect it at a national level.
It has been considered that the perceptions of these two groups of players will allow reaching a qualitative approximation to measuring and assessing the concept under study, which would certainly be useful for the design of policies aimed at improving this aspect of the university educational system. In this context, it would seem fair and reasonable to assume that both university educators and students have a certain degree of knowledge and analysis capabilities that would enable them to correctly interpret which are the variables that act as quality factors in the teaching and learning process. However, and despite this, the responses gathered must be understood as the individual opinions of qualified people on the causes behind the process mentioned above and its results.

DEVELOPMENT

Conceptual context

As a starting point, it should be noted that it is hard to associate the term Quality,  as it is usually understood, to a well-delimited and accurately defined category. University quality is, first and foremost, a complex and multi-dimensional concept that cannot be separated from its relation to the objectives and players of the university system.
The most complete and accepted definition of the meanings that the term quality can have in a university context is that of Harvey and Green (1993). Below, we present a brief summary of the quality concepts developed by these authors and their main characteristics.

Table 1: Conceptions of Quality in Higher Education and their characteristics

Source: Adapted from Barrenetxea, 2005

The five notions of quality proposed by Harvey and Green (1993) can be ordered in a plane generated by two dimensions (Figure 1). The horizontal axis represents the extent to which each notion of quality is derived from an element that is either internal to the institution or somehow external (not belonging) to it. The vertical axis proposes a different division of the concepts of quality. The lower part of this axis gathers the traditional definitions on Quality Management (both specifically for universities as well as in the more general sense of business management), while the upper part shows a more modern view of the phenomenon.


Figure 1: Quality concepts (Harvey and Green, 1993) in a two-dimensional space
Source: López Armengol, M. A. and Colombo, M. P (coords.), 2011

Naturally, these five notions of quality are not entirely incompatible, especially when trying to acknowledge its presence within the symbolic universe of the players (in this case, those acting at universities). That is, it should not be surprising that the same individual may uphold various ways of understanding quality.
Now, the mere description of the meanings for the term quality in the university context does not explain the reason for such variety. In this sense, the most widely accepted hypothesis is that known as materialistic thesis. This premise (Houston, 2008; Srikanthan and Dalrymple, 2007, 2003) suggests that the reason why there are different quality concepts is that there are multiple interests groups (stakeholders) associated to universities. For each interest group, the university has a different sense and purpose (linked to their own interests), and values the quality of its results based on that particular point of view.
Houston (2008), for instance, specifically considers a tension between three spaces external to the university that may require from it different, and even opposing, commitments – society in general, the productive sector, and the educational system. This in addition to the coexistence of at least five groups within the university organization – students, educators, university managers, technical support staff, and administrative staff, each of them with its own expectations regarding values, operation principles, methods and procedures, and, naturally, the results that the institution should achieve. If this is the case, the conceptualization of the term quality faces an even greater obstacle than that of achieving technical consensus on what to call things, since, as stated by Becher (1999), there would be background policy issues to the concept. That is, there is more at stake than an agreement to refer to a fact; this is more about legitimizing the viewpoints and interests of one group rather than those of the other, competing groups.


Figure 2: University stakeholders
Source: Houston (2008:63)

Methodological context

The purpose of this article is to present the compared results of two research works carried out within the ECUALE Network: the one carried out for the ECUALE Project (Studies on Quality at Latin American and Spanish Universities) financed by the AECID (Spanish Agency for International Development Cooperation), and the one carried out for Project 11/E116 (Quality in Higher Education. A Study on Student Perception) financed by the Educational Incentives Program of the Ministry of Education of Argentina.
The ECUALE Project is based on the perceptions of the educators teaching classroom courses in the area of Economic Sciences of public universities in Argentina. On the other hand, Project 11/E116 is based on analyzing students in the area of Social Sciences of the National University of La Plata. However, in the case of this paper in particular, and in order to achieve a methodology that is more accurate, only students belonging to the School of Economic Sciences were considered. Our sample then included 121 educators from the first project, and 133 students from the second one.
The survey strategy used in both cases was a closed, self-administered questionnaire. In the case of the ECUALE Project, it was an online survey. It was sent out to each individual e-mail address of each educator, accompanied by a personalized message signed by the authorities of the School of Economic Sciences of the National University of La Plata that included a brief explanation of the project and the invitation to participate. In the case of Project 11/E116, surveys were administered in person through the educators teaching each of the courses selected.
The Likert format was used to create the questionnaire. This is a psychometric scale that is commonly used in questionnaires, and is the most widely used scale used in research surveys. When responding to an item of a questionnaire created using this technique, respondents specify their level of agreement or disagreement with a given statement (element or item). A Likert-type item is a statement presented to the subjects for them to assess based on their subjective judgment. In general, they are requested to express their level of agreement or disagreement by choosing one of (usually) five possible responses or levels referring to the statement.
The questionnaires used for both research studies were as similar as possible to each other in order to facilitate subsequent comparisons. The questionnaires were organized in three basic, general blocks:
The first block was oriented to the analysis of the most widely accepted concept of quality in university education. To that end, the level of agreement of the respondents with several quality conceptions was queried, as well as their perception regarding the correspondence between their university and those meanings. To do this, the concepts proposed by Harvey and Green presented above were implemented. Specifically, the following meanings of the erm quality were used for the projects:

Figure 3 shows a schematic representation of the relationship established between the operational variables defined above and the original concepts that were presented in Table 1.


Figure 3: Quality concepts (Harvey and Green, 1993) and the items designed to implement them
Source: López Armengol, M. A. and Colombo, M. P (coords.), 2011

The second block of the questionnaire focused on the analysis of the factors affecting quality improvement in university education. This block included two questions. The first one was aimed at inquiring on the opinion of educators and students regarding the significance of the factors mentioned for improving university education, while the second one asked about their perception of the presence of these factors at their university and academic unit.
Finally, the third block in the questionnaires encompassed what is commonly known as control variables (age, gender, etc.).
Based on the results obtained in both studies, this paper will focus on their comparison. In this sense, Bereday's model (1968) will be the comparative education method used, since it is an already classic method in comparative pedagogy. While acknowledging the significance of the first two stages (Description and Interpretation), this article will focus on the Juxtaposition and Comparison stages.
It should be noted that the Juxtaposition stage is not concerned with individual data and reports (as is the case with the Description and Interpretation) stages. This is an eminently relational stage that confronts different studies on the object being compared based on what could be considered as parallel sets. It should be noted that this is a comparison stage in itself during which not only are similarities and differences observed, but the parallel sets are confronted to reach an inherently comparative situation for the problem.
The Comparison stage is an assessment, results, and achievement phase that is a consequence of the analyses carried out in the previous stages and, in particular, the Juxtaposition stage. That is, if in the previous phase the parallel sets were confronted and the comparison chart was clearly established, the main objective now would be that of assessing and drawing conclusions, separating the essential from the accidental. In conclusion, it could be said that this is more of a summary stage than an analysis one. 

Results

In this section, the main results obtained in the studies will be discussed in relation to the concepts of quality that take precedence among Argentinean university educators and students in Economic Sciences. It should be noted that this is a non-conditional analysis, meaning that no conclusions can be drawn regarding the incidence of certain characteristics on the results, and its causality can only be established as a hypothesis. In this context, the study will be based on the sum of the relative frequencies (high and very high responses in Likert's scale) of the responses obtained (hereinafter, HRF).

The Concept of Quality in University Education

As already mentioned, the studies carried out queried university students and educators on their opinion regarding alternative ways for defining the concept of quality. As a first step, they were asked to express their level of personal agreement with each of the seven definitions of quality detailed above (see Methodological Context). The results obtained in this regard are presented in Table 2 below.

Table 2: Agreement of educators and students with seven statements defining the concept of quality in university education

Source: Authors

Based on the analysis of the data presented in Table 2, it can be observed that both educators and students identify more with the conceptions built from internal references, rather than with those notions that can be considered as external to the educational context. The conceptualizations of quality as transformative power in university education are clearly the ones to which both educators and students feel mainly related. There is a small difference in favor of students, which can be taken as a sign of the active role that they seek within the teaching and learning process. In this sense, Harvey and Green (1993) state that the learner has to take possession of the learning process and assume responsibility in determining the style and delivery method for such learning. The transformative process provides the opportunity of self-strengthening with positive consequences in the decision-making process itself. By empowering the students, quality is assessed in terms of process democratization and not just results.
Thus, based on the previous considerations, it would seem that all survey respondents, regardless of their role, consider that there is an undeniable need to promote an education that focuses on a comprehensive human development of the students, considering not only cognitive dimensions but also social and appraisive ones. An education that fosters critical and independent thinking and, at the same time, the ability for life-long learning by developing capabilities that allow students to move forward in their own learning and education as individuals committed to society in general and their communities.
Based on everything mentioned above, it could be said that educators and students mainly consider quality within a comprehensive educational approach (as pointed out by Malkova quoted by Bosco Bernal (1992)). This approach focuses on the multidimensional development of students as subjects with potential for learning that a properly oriented teaching process should be able to set free and develop. It also considers that these subjects that learn should have an active participation in building their own learning and not simply reproduce the knowledge of others.
Going deeper into this analysis, it is relevant to add some considerations regarding the agreement of each of the surveyed groups with the statements or concepts proposed at an individual level. The responses obtained in this sense are presented in Figure 4.


Figure 4: Agreement with the concepts of quality for the students vs. the educators (HRF)
Source: Authors

The most interesting observation that can be derived from this figure is that, regardless of the values adopted for each of the definitions, it would seem that the views of educators and students share the same orientation.
There are two specific conceptualizations that stand out due to the different views of both groups. The first one is the definition that suggests that quality is related to meeting the expectations of students and the organizations where graduates work. This definition was remarkably more valued by students than educators, which could be attributed to the effect on students of choosing a course of studies that will allow them to quickly and appropriately transition to the job market. However, and regardless any possible reason for such choice, this difference has to be taken into account because it points to the needs and expectations of students in relation to their professional training.
The other concept that shows a significant difference between both parallel groups under study is the one suggesting that quality consists in achieving efficiency which, curiously enough, is the meaning that both groups place well below all other definitions. This conceptualization was taken from business sciences bibliography, and it would seem that it is not well accepted by survey respondents as a valid concept when considering university education in Argentina. However, and despite this, students would seem to be more prone to assess this educational quality in relation to the return of an investment made, which could possibly be influenced by the educational process through which they are currently going. This can however only be considered as a hypothesis and, as such, would require more specific future research.
As a final and summarizing point for this section, it should be noted that educators and students seemed to share the same view for quality in university education. Based on the opinions gathered from both groups, it can be said that survey respondents agree with Zabalza (2007) in that, in order to make quality education possible, a connection should be achieved with the individual's growth and improvement and that, following the thoughts of this author, being increasingly informed, knowledgeable and competent professionals would lack any sense if this does not translate into being a better person.

Quality Determinants

Incidence of factors that determine university quality according to students and educators
In this section, we will discuss the responses given by survey participants when asked about the degree to which certain factors affect university education quality.
Even though the questionnaires used in each of the studies are not exactly alike, most of the issues under study were analyzed for both groups, which makes possible the direct comparison between both study groups. Thus, there are 16 factors, grouped in three different categories that are used in this analysis: those pertaining to educators, those pertaining to students, and those pertaining to the university. Specifically, the factors mentioned above are the following:

Category 1: Educator-related factors

Category 2: Student-related factors

Category 3: University-related factors

Table 3 presents the results grouped by analysis categories.

Table 3: Quality-determining factors by category

Source: Authors

If a categorical analysis is performed, it can be seen that, both for students and educators, student- and educator-related factors are those mostly considered to determine higher education quality. Following them, far behind, are university-related factors. This means that both educators and students are, based on their own perception, the key elements for quality university education.
In the following paragraphs the various factors in each of the categories proposed are analyzed.

From the interpretation of these data, it is clear that survey respondents acknowledge the significance of some characteristics of the educators for education quality. From the responses obtained, it can be concluded that these players continue to trust the characteristics that are traditionally linked to being a good educator: knowledge of the subject matter, communication skills, and motivation (see Figure 5). These characteristics are not only those most valued within this category, but they are also those occupying the highest ranks among all factors.


Figure 5: Educator-related factors that determine quality (HRF)
Source: Authors

Now, oddly enough, despite being one of the two most highly valued categories by both study groups, it includes two of the factors that were considered to be of little importance (one by each group). In addition to this, these two factors are on opposing sides for both groups – the factor considered to be unimportant for one group is well positioned by the other, and vice-versa. Specifically, these factors are: updated pedagogy training of the educator and participation of the educator in research activities. These differences expose different views of the educator role that do not share the exact same characteristics for both groups. The students seek educators that are committed to teaching and teaching-related training activities, whereas the educators prioritize training activities that are more relevant to their professional specialization. 

This group of factors, ranked by students on the same level as educator-related factors, and barely below it by educators, is the most homogeneous category when considering the individual components. Students and educators are consistent in these items, and all of the factors suggested were assigned similar values (see Figure 6).


Figure 6: Student-related factors that determine quality (HRF)
Source: Authors

This was the least valued category both by students and educators. However, there are some differences between both groups, namely: there is consistency in student appraisal of the category in itself and each of its individual components (see Figure 7). In other terms, all of the elements included in this category are among the lowest scoring ones. This could partly be due to the distance between students and the institution as such, since students interact with the university through the educators and only indirectly with what they consider to be the institution itself.
In the case of the educators, on the contrary, there is no evidence of this inner uniformity in the category. Specifically, the second highest ranking factor for this group of survey respondents is found in this subset of elements: strictness when selecting educators. Educators consider that quality is largely dependent on the adequate operation of the institutions in relation to ensuring a correct selection of their faculty. That is, they consider that it is themselves who are the core component of quality and, as such, the institution must ensure that only the best educators can access the positions. On the other hand, this category also holds those factors that educators consider as unimportant and that are mainly related to the administrative aspects with which the institution can or must provide them for the educators to be able to perform their roles (existence of appropriate support services and efficiency for processing paperwork and administrative processes).
The little value assigned by educators to these two factors corroborates what Mintzberg (1994) states in his book Designing Effective Organizations dealing with the behavior of professionals in professional bureaucracies. In this type of organizations, the group referred to as professionals (in the context of universities, the educators) focus exclusively on themselves and their relation with their customers (in this case, the students). Their main concern is freedom of action within the institution, and for this reason, controls and regulations tend to be strongly rejected. However, these professionals need a strong support staff that will allow them to carry out their tasks without worrying about those issues that they consider to be of little relevance (despite them being essential for performing their institutional tasks). 


Figure 7: University-related factors that determine quality (HRF)
Source: Authors

Finally, and as a conclusion to this section, it should be pointed out that both groups were in agreement in relation to one of the factors that less affects higher education quality: strictness when selecting students. In this sense, it could be argued that this is linked to the idea, deeply-rooted in the Argentinean academic community, that the university educational system should be open and inclusive.

Priority quality-determining factors

In addition to the analysis of the opinions detailed in the previous section on the significance of the factors that affect education quality, survey respondents from both groups were also asked about their perception regarding the presence of some of those elements in their academic unit.
Based on these responses, a priority measurement was built for each concept, calculated as the difference between the significance or incidence level and the corresponding presence or development level. Thus, those factors that were valued as important but are not quite developed at the institutions will show as those with the highest priority. The results obtained for the factors that were considered as the most significant in both surveys are presented.
From a methodological point of view, it was considered that the different players were not capable of assessing the presence of factors pertaining to their own group in their own academic units. Thus, the analysis will be carried out taking the following into account:

As regards the first item, the priority of factors related to the institution, it should be noted that the educators were more critical in relation to their presence or development level in their universities and academic units, and therefore, the corresponding priorities to tackle or work towards attaining them is naturally higher.

That is, regarding this category, educators would seem to be harsher in their opinions than students. This reinforces what was mentioned above in relation to the concepts presented by Henry Mintzberg regarding professional bureaucracies (see Table 4).
In this context, the element that emerges as the one with the highest priority is that of strictness when selecting educators. This means that educators consider that institutions should focus their efforts to improve on this aspect, which is consistent with the discussion in the previous section.
On the other hand, both educators and students agree that the following are priority factors within this category: ongoing training provided by the corresponding school or university to its educators and coordination between secondary and university contents. Both issues are perceived as significant, but their actual development would seem lacking or insufficient in their corresponding academic units.

Table 4: Educator and student perceptions regarding priority factors associated to the university

Source: Authors

The results from the second group proposed, student perceptions regarding priority factors associated to educators, are presented in Table 5. It shows that the students consider that educator-related factors would not have a high priority at their institution, since none of the individual components is rated above 40%. In other words, even though for students this is the most significant category of factors to achieve quality, as discussed in the previous section, this group also considers that the individual elements are better developed.

Table 5: Student perceptions regarding priority factors associated to educators

Source: Authors

Finally, Table 6 shows the opinions of educators regarding the priority of student-related factors. As it can be seen, and opposing the opinions of the students described above, educators think that all of the factors in this category should be considered as priority factors at their universities – the difference between their incidence and their presence is always above 65% for this group. Educators believe that these elements are essential for education quality but, in spite of this, are not currently present in their academic units, meaning that a more active intervention would be required.

Table 6: Educator perceptions regarding priority factors associated to students

Source: Authors

It is thus evident that educators and students have opposing views as regards the scope of their respective roles and responsibilities in relation to quality. It is for this reason that this issue should be analyzed and discussed to achieve a more complete and accurate view of the reality at Argentinean universities.

CONCLUSION

Throughout this article, the results from two research projects carried out in Argentina (ECUALE Project and Project 11/E116) have been discussed. Specifically, the differences between two fundamental players in the educational process have been considered in relation to higher education quality: educators and students from the area of Economic Sciences. 
It is undeniable that we live in a knowledge-society, and it is within this context that universities take the lead role that the industries once had during the industrial-society era. Workers in our era, knowledge workers as Peter Ducker (1993) proposes, use their intellect rather than their physical strength, and it is for this reason that the discussion on issues pertaining to higher education in general and university educational quality in particular cannot be set aside from any national agenda.
Specifically, the intended added value of this article would be the possibility of offering empirical information on the existence, or lack of, differing views between both study groups in relation to the core aspects that revolve around quality in education. As final summary, the following paragraphs will list a few of the most significant differences observed from the analysis of the data gathered and presented throughout the article:
First, and in relation to how quality in higher education is understood, it was seen that both students and educators link the concept of quality with the possibility of developing a critical spirit in the students, providing them with skills that will allow autonomous learning, and helping them become socially committed individuals. This is no minor issue, since having a consensus as regards the meaning of a concept is essential to be able to work on that concept. Thus, this first result reveals that both groups understand university quality in the same way.
For students and educators, quality would not involve answering to the needs of the market or externally imposed regulations (despite belonging to the discipline of economic sciences), but it should rather be understood from a more modern and internal perspective. This is also interesting, because Argentinean universities are currently very involved with assessment and accreditation processes to demonstrate compliance with certain minimum quality standards, and yet both groups build their idea of quality as the transformative efficiency that education should have on students, rather than quality based on these external indicators.
Secondly, those factors with a strongest impact on quality for students and educators were considered. It can be concluded that there is a shared imaginary by both groups in that the most relevant issues that affect quality are linked to the faculty, both regarding their specific knowledge as well as their communication skills and/or their motivation. That is, nowadays, educators are still the center of attention when it comes to quality in education. These are followed by student-related factors, which both study groups identify as key elements for quality with high values and close to those in the first category. In the last place we find the group of factors that are associated with universities as institutions. This shows that both for educators and students, the playfield is not important, only the players that are in it actually playing.
Finally, based on the analysis of the priority areas that require work to improve quality, it is observed that both groups strongly believe that universities are facing the huge challenge of reducing the gap between what is perceived as important and what is actually offered by these educational institutions. 
Even though the factors pertaining to the institution were the least valued by both groups, educators consider that there are several aspects in this category that are lacking in their academic units.
However, and despite this, the most relevant aspect of this issue comes from analyzing the cross-considerations of educators and students. The latter do not perceive any current significant shortcomings associated to professor-related, quality-determining elements. That is, they consider that the faculty has the necessary qualities to ensure an adequate quality for higher education. This is exactly the opposite of what educators believe regarding student-related factors – they think that these elements have none of the qualities that they should have in order to improve quality in their academic units.
Thus, taking into account all the issues discussed throughout this article proposing a comparative analysis of the opinions of students and educators regarding quality in higher education, we hope we were able to contribute to exposing some of the hidden imaginaries behind each study group. Such exposure is merely the first step towards a more constructive dialogue (one that considers alternative views, concerns, and limitations) aimed at outlining, designing and implementing consensual educational quality policies that are shared by the various organizational levels and players in the university educational process.

BIOGRAPHICAL ABSTRACT

Please refer to articles Spanish Biographical abstract.

BIBLIOGRAPHY REFERENCES

Please refer to articles in Spanish Bibliography References.