Measuring attitudes towards ethnic minority patients: the revalidated REMP-3 instrument for graduate healthcare practitioners

Abstract

Objective

Measuring intercultural attitudes can aid in understanding and addressing persistent inequities in healthcare. Instead of creating new instruments, several sources call for a more rigorous revalidation of existing instruments towards a more broad population. As an example of such an existing instrument, the EMP-3 (Ethnic Minority Patients) focuses on the attitudes of physicians towards ethnic minority patients. Starting from a robust theoretical underpinning and a rigorous methodological setup, the present study revalidates the EMP-3 instrument for physicians towards the REMP-3 instrument for graduate healthcare practitioners.

Methods

We assessed the reliability and validity of the old EMP-3, which we then updated to a new REMP-3 instrument. We used structural equation modeling to model the framework of intercultural effectiveness on two waves of independent data, N₂₀₂₁ = 368 and N₂₀₂₂ = 390. Within this framework, we tested the new REMP-3 instrument as an operationalization of intercultural attitudes. We conducted a confirmatory factor analysis on the first wave, after which we made adaptations to the original EMP-3 instrument to obtain a new REMP-3 instrument. The new REMP-3 instrument was then cross-validated using the data of the second wave.

Results

The new REMP-3 instrument is a psychometric upgrade compared to the EMP-3. The REMP-3 now has a cross-validated structure, with three subscale dimensions (i.e., task perception, background perception and the perceived need to communicate) and an overarching higher-order, full-scale dimension. Both the subscales as well as the full instrument show acceptable to good internal consistency reliability, with a reduced number of items from eighteen to ten. As theoretically predicted, the REMP-3 also functions as a measure of intercultural attitudes in an intercultural competence framework.

Conclusion

Ultimately, the REMP-3 instrument can contribute to more equity in healthcare by concisely and reliably assessing and monitoring attitudes in healthcare practitioners. This attitude assessment represents the potential of learning new skills and knowledge to address interactions with ethnic minority patients, which is especially useful during training situations like an internship.

Background

According to the World Health Organization, equity in healthcare embodies the absence of unfair, avoidable or remediable differences among groups of people [1]. Recent review research indicates that ethnic minority patients are one of those groups that still experience healthcare disparities in important areas like emergency healthcare [2], safety of healthcare [3], diabetes treatment [4], and organ transplants [5].

To improve this equity in healthcare, research already focuses on investigating intercultural attitudes of healthcare practitioners towards ethnic minority patients. For instance, the study of such attitudes can aid in understanding equity related problems ranging from general social and ethnic differences in healthcare systems [6] to more specific problems like racism in healthcare [7].

However, empirical data collection on healthcare practitioners’ intercultural attitudes towards ethnic minority patients remains inconsistent [8], with limited attention towards sufficient methodological rigor [9, 10]. Often, research exclusively focuses on standalone self-report instruments, without the validating power of an underlying theoretical framework or without linking attitude instruments results to real life outcomes. For instance, Osmancevic and colleagues scrutinized 44 studies and 21 instruments in a systematic review, of which only three instruments¹ showed sufficient levels of psychometric quality to assess the intercultural competence of nurses [10,11,12,13]. Although we considered these instruments as valid candidates for the present study, we decided against targeting these instruments for mainly two reasons. First, we assessed the instruments’ items as too nurse-specific and thus less suited to generalize towards a more broad target population. And second, the instruments are not exclusive attitude instruments, but also include items that cover other constructs like motivation and knowledge. Such composite instruments are less compatible with the setup of the present study as we operationalize each component of intercultural competence separately by using the framework of intercultural competence by Leung and colleagues [14].

The framework of Leung and colleagues can provide a robust theoretical underpinning for intercultural competence (revalidation) studies [14,15,16], as the framework facilitates a more structured approach. Figure 1 shows a summary of the framework. Intercultural competence is described as a combination of three components that represent dispositions on (a) personality traits, (b) attitudes and (c) knowledge and skills needed to perform effectively in intercultural interactions [17]. First, intercultural personality traits represent persevering personal dispositions that can explain effectiveness in intercultural situations [18, 19]. Second, intercultural attitudes represent world views individuals have regarding the ethnocentric – ethnorelative continuum [20]. An ethnocentric view represents a world view in which the own culture is seen as superior, while an ethnorelative view acknowledges the co-existence of many different cultures, all of equal value. And third, intercultural capabilities represent the skills and knowledge an individual has acquired to address intercultural situations [21].

Fig. 1

The framework of intercultural competence. Note. The figure is adapted from Schelfhout et al., 2022. The framework features four hypotheses that need confirming in order for the data to fit the framework. H1 = A high disposition on intercultural traits predicts a more ethnorelative disposition, H2 = A high disposition on intercultural traits predicts a higher cultural intelligence, H3 = A more ethnorelative disposition predicts a higher cultural intelligence. H4 = A higher cultural intelligence predicts more intense cultural contacts

The three components interact in a specific way: Traits trigger attitudes and capabilities, attitudes trigger capabilities, and capabilities lead to intercultural effectiveness. As Fig. 1 shows, intercultural attitudes function as a central component of intercultural competence [22]. Due to this central function, attitudes directly affect the acquisition of capabilities like knowledge and skills [23]. Indeed, a more ethnorelative view in a healthcare practitioner will facilitate the learning of skills and knowledge, while a more ethnocentric view will hamper such learning. An intercultural attitude measure during training situations like (pre -or postgraduation) internships can thus prove key to explore to which extent attitudes can affect the learning of knowledge and skills. In conclusion, the framework posits that the central attitudes do not exert a direct influence on the outcomes of intercultural effectiveness. Instead, their impact is mediated indirectly through the acquisition of capabilities like skills and knowledge (see also Fig. 1).

Osmancevic and colleagues advise that selecting an intercultural competence instrument should depend on the purpose of the study and the targeted population [10]. For the present study, we thus aim at revalidating an instrument that could specifically measure intercultural attitudes in a more broad population of graduate healthcare practitioners. For this purpose, the present study chooses to target the EMP-3 instrument, which measures the perceptions and attitudes of physicians towards ethnic minority patients [24]. This EMP-3 instrument featured three subscales on (a) task perception (b) the attitude towards physician–patient communication and (c) perception of minority patients’ need for communication (see Table 1). The EMP-3 instrument showed validity as the subscales correlated with proven scales like the Jefferson’s Empathy scale [25] and the Patient Practitioner Orientation scale [26]. However, De Maesschalck and colleagues called for further revalidation studies to investigate the use of the EMP-3 instrument in more broad groups of healthcare practitioners [24].

Table 1 Item List for EMP-3 and REMP-3

Despite the instrument’s obvious qualities, we also observe that the original EMP-3 study underexplored four important aspects regarding the validity of the results that hold back the instrument’s further dissemination. First, the original EMP-3 intends to cover attitudes of physicians towards ethnic minority patients. Yet, the original EMP-3 does not have a physician–specific nature as the current instrument does not involve specific procedures, exclusively known to physicians. As such, the instrument could be suited for use in a broader population of healthcare practitioners like nurses and medical assistants. Second, the original EMP-3 instrument study does not report the correlations between the subscales, while the possibility of a higher-order factor is also not investigated. A higher-order structure should at least be investigated in the present study to further strengthen the internal structure of the instrument. Third, the instrument is not based on a formal theoretical framework. For construct validity purposes, the present study therefore proposes to integrate instrument as a measure of intercultural attitudes into the framework of Leung and colleagues [23]. As an instrument that measures intercultural attitudes, a revalidated EMP-3 could therefore be integrated as a central component of intercultural competence (see also Fig. 1), in between traits and capabilities. The hypothesized relations of the framework have to be tested in order to evaluate if the framework holds given the data used. Fourth and finally, the original study did not link the EMP-instrument to any specific medical outcomes like the effectiveness of the care or the patient experience to show criterion validity. As the present study aims for a use of the instrument in a broader population, a general outcome like the intensity of intercultural contacts is suited for the present study’s revalidation purposes. Indeed, an individual that has a higher intercultural competence, generally shows a higher intensity of intercultural contacts [27].

The present study revalidates the original EMP-III instrument towards a more broad REMP-3 instrument that measures the intercultural attitudes of graduate healthcare practitioners. For this revalidation, we fit the present study’s empirical data on traits, attitudes (i.e., the EMP-III results) and capabilities to the intercultural competence framework of Leung and colleagues [23]. Four effects need to be tested in order to evaluate if intercultural attitudes as measured by the REMP-3 can indeed function as the central component of intercultural competence (see also Fig. 1). As suggested by Schelfhout and colleagues [22], the hypotheses are directly drawn from the theoretical model (see also Fig. 1).

First, a high disposition on intercultural traits predicts more ethnorelative attitudes or less ethnocentric attitudes. For instance, Talay & De Coninck reported that openness, agreeableness, and honesty-humility were negatively associated with ethnocentricity in the form of refugee prejudice [28].

H1

A high disposition on intercultural traits predicts a more ethnorelative disposition.

Second, a high disposition on intercultural traits should predict higher intercultural capabilities like cultural intelligence. For instance, Li and colleagues showed that open mindedness is positively related to cultural intelligence but only when individuals also have a high disposition on agreeableness [29]. Instead of focusing on single traits, the authors therefore advocated to use an integrative approach by taking a full spectrum of traits into account when studying the effects on a construct like cultural intelligence. Also in healthcare specifically, researchers are already aware of the effects intercultural traits can have on intercultural intelligence. For instance, Wang and colleagues used the open mindedness trait as a controlling covariate to test the effects of an overseas intervention on the cultural intelligence of undergraduate healthcare students [30].

H2

A high disposition on intercultural traits predicts a higher cultural intelligence.

Third, a more ethnorelative attitude should predict higher intercultural capabilities. For instance, Majda and colleagues reported that emergency healthcare professionals with a positive (or ethnorelative) attitude towards culturally divergent people showed a higher cultural intelligence [31].

H3

A more ethnorelative disposition predicts a higher cultural intelligence.

Fourth and final, higher intercultural capabilities should predict more intercultural effectiveness. For instance, Schwarzenthal and colleagues demonstrated that a higher intercultural intelligence can result in beneficial outcomes like increased intercultural contacts and cooperation [27].

H4

A higher cultural intelligence predicts more intense cultural contacts.

On a final note, the framework by Leung and colleagues does not predict a direct effect of attitudes on intercultural effective behavior [23].

Methods

Data

The data were obtained from the regionwide Flemish (i.e., large region in Belgium with about 6.8 million inhabitants) EdisTools (i.e., E-antidiscrimination Tools) project that aims to chart and remedy discrimination in four key domains of human interaction: education, healthcare, housing and work. As such, the project focuses on the interaction between the ethnic majority service providers (e.g., healthcare practitioners) and the ethnic minority clients (e.g., patients). For the present study, participants were recruited from the pre-Master Program in Management and Organization of Healthcare and from the pre-Master Program in Health Promotion, both from the Faculty of Medicine and Health Sciences of a large Flemish university. The main target audience consists of graduate healthcare practitioners that want to master their healthcare training to further degree. The data was collected online in two waves. The first wave ran from February 2021 to May 2021 (N₂₀₂₁ = 368, age M = 22.69, age SD = 4.37), where about 29% of the participants identified with a male gender orientation, 70% identified with a female gender orientation and 1% identified with a different gender orientation. About 68% of the participants indicated they already had some form of experience in working as a healthcare practitioner, ranging from one week to twenty-three years. Such a diverse graduate student population with already some experience in various subfields of healthcare seems appropriate for the present study, as the REMP-3 aims to become an instrument that can measure intercultural attitudes in a broad population of graduate healthcare practitioners. As an additional illustration, some participants added a job description to their accumulated experience, indicating a wide scope of working environments, ranging from an internship or general nursing aid, to very specific descriptions like operating room assistant or psychiatric nurse. The vast majority (97%) of the participants indicated they were born in the country were the study was conducted. About 6% indicated they had a father who was born in another country, about 7% indicated they had a mother that was born in another country and about 8% indicated they had a grandmother that was born in another country.

The second wave ran from February 2022 to May 2022 (N₂₀₂₂ = 390, age M = 22.71, age SD = 4.48), where about 23% of the participants identified with a male gender orientation and 77% identified with a female gender orientation. One participant indicated a different gender orientation. About 65% of the participants indicated they had some form of experience in working as a healthcare practitioner, ranging from one week to thirty years. Analogue to the first wave, some participants added a job description to their accumulated experience, indicating a wide scope of working environments. The vast majority (i.e., 95%) of the participants indicated they were born in the country were the study was conducted. About 10% indicated they had a father who was born in another country, about 9% indicated they had a mother that was born in another country and about 11% indicated they had a grandmother that was born in another country. Note that both datasets are independent (i.e., each student only participated in one wave).

Measures

Intercultural traits

We measures intercultural traits using the Short Form Multicultural Personality Questionnaire or SF-MPQ [32, 33]. We used this questionnaire, as the SF - MPQ features a specific intercultural iteration of the Big Five personality traits, that explains intercultural effectiveness above and beyond the original Big Five. For instance, the SF-MPQ was recently administered to a population of Western and non-Western, male and female students [33]. Results showed that all five subscales (i.e., cultural empathy, flexibility, social initiative, emotional stability and open mindedness) could be reliably used in both comparative as well as longitudinal designs, for Western and non –Western students alike. Five traits are measured, including cultural empathy (CE), flexibility (FX), social initiative (SI), emotional stability (ES) and open mindedness (OM). Each trait subscale features eight items (e.g., Pays attention to the emotions of others, CE), measured on a five-point Likert scale, anchored between totally not applicable (1) to completely applicable (5). For reliability, we refer to the Results section.

Intercultural attitudes

Intercultural attitudes are measured using the original attitude towards Ethnic Minority Patients instrument or EMP-3 [24]. Participants had to indicate to which extent they agreed with a set of eighteen statements, on a five-point Likert scale from totally not agree (1) to totally agree (5). The original three subscales are measured including (1) ten items on task perception or TP (e.g., Physicians should be aware of the cultural identity of each patient), (2) six items on the attitude towards physician – patient communication or PPC (e.g., Patients’ social background determines the way they communicate with physicians) and (3) two items on the perception of minority patients’ need for communication or PNC (e.g., Minority patients prefer a paternalistic consulting style, reverse coded). For reliability, we again refer to the Results section.

Intercultural capabilities

We measured intercultural capabilities using cultural intelligence or CQ [27]. Multiple studies already use CQ to validate intercultural capabilities as a part of the intercultural competence framework [15, 22]. Although a multimodal measure is possible, the present study only uses the overarching scale for reasons of parsimony to limit the risk of inflating the results our SEM analyses. CQ is measured using 24 items (e.g., If there is a misunderstanding between people from different cultures, I try to clear it up) on a five-point Likert scale, anchored between strongly disagree (1) and strongly agree (5). For reliability, we again refer to the Results section.

Intercultural effectiveness

We measured intercultural effectiveness using the intensity of intercultural contacts or IOIC, which is an adaptation from the operationalization of Schwarzenthal and colleagues [27]. Literature already features studies that use this concise measure to validate intercultural competence constructs [15]. Indeed, the measure has one question and probes for the intensity of intercultural contacts (i.e., How would you characterize your contacts with people that have a migration background? ). The responses were anchored on a four-point Likert scale ranging from (1) I only have anonymous contacts with people that have a migration background, over (2) I have vague acquaintances with people that have a migration background and (3) I have friends or close colleagues/fellow students with a migration background, to (4) I have close relatives or close friends with a migration background. The analyses with the continuous Likert scale were also repeated with an ordinal scale, rendering analogue correlation patterns.

Analyses

We have included two measures of internal consistency reliability common to literature. First, the Cronbach’s alpha is used as a measure of internal consistency, with α > 0.70 indicating an acceptable reliability and α > 0.80 indicating a good reliability [34]. However, Cronbach’s alpha is sensitive to the length of a (sub)scale and the alpha does not measure homogeneity as such [35, 36]. As the present study features (sub)scales with a lower number of items (i.e.,< 11 items), we have followed the suggestions of Clark and Watson to include the average inter-item correlation (AIIC) as a measure of internal consistency reliability as the AIIC is independent of scale length and does cover homogeneity [37]. Clark and Watson recommend that the AIIC should fall into the 0.15 − 0.50 range, with more general scales (e.g., CQ) showing a relatively lower AIIC and more specific scales showing a relatively higher AIIC (e.g., MPQCE) [37].

The revalidation and framework integration analyses are conducted using structural equation models or SEM [38], using the lavaan package [39]. Evaluating SEM is usually executed by using a battery of fit indices. For the present study, a battery of three indices common to literature are included to complement the conservative chi-squared test including the Comparative Fit Index or CFI (> 0.90 for an adequate fit, > 0.95 for a good fit), the Root Mean Square Error of Approximation or RMSEA (90% confidence interval or CI should have a lower bound no higher than 0.05 and a higher bound lower than 0.08) and the Standardized Root Mean square Residual or SRMR (< 0.08 for a good fit). For a complete discussion on fit indices, we refer to measuring model fit by David Kenny [40]. Note that the cutoff values are not absolute, as an evaluation of goodness-of-fit should always regard a full index pattern. The SEM analyses consist of two major parts: a confirmatory factor analyses or CFA on the original EMP-3 subscales and a latent SEM analyses on the framework of intercultural competence. For both parts, an adequately fitting model is derived from the data from the first wave (N₂₀₂₁) and cross-validated on the data from the second wave (N₂₀₂₂). The construction and finetuning of the models were executed using modification indices (MI), that indicate how the fit of a model can change if variables or regressions are added or omitted [39]. The Results section also contains a summarizing table featuring all variables, including the original EMP-3 instrument and the revalidated REMP-3 instrument respectively. On a final note, analyses were also controlled for the effects of age (year of birth), gender (male − 1, female − 0) and (months of ) healthcare experience. To avoid inflation of model fit by adding a multitude of additional variables, we opted to act conservatively and therefore conducted the analyses for each hypothesis separately, moving away from SEM towards linear regression analyses.

Results

The variable descriptive statistics for both datasets N₂₀₂₁ and N₂₀₂₂ are highly similar regarding means, standard deviations and reliability (see also Additional File 1 for additional analyses). For reasons of parsimony, Table 2 therefore reports the pooled variable descriptive statistics of N₂₀₂₁ and N₂₀₂₂. Table 2 thus reveals that the reliability of the original three EMP-3 subscales is at least acceptable. However, the reliability of a presumed overarching construct is lackluster as the AIIC is too low (i.e., < 0.15). Moreover, we also observe a negative correlation between the PPC and PNC subscale, which is problematic as both scales intend to measure different components of the same intercultural attitude construct and thus should be positively correlated. The EMP-3 and its subscales do seem to be correlated to a number of MPQ scales and CQ, as is also predicted by the hypotheses. The (negative) correlations between the subscales and the reliability are further assessed using a CFA. The hypothesized correlations are further assessed using a full SEM on the framework of intercultural competence.

Table 2 Variable summary and correlation matrix

Confirmatory factor analyses

We conducted a CFA on the data of the first wave (N₂₀₂₁ = 368), and cross-validated the results on the data of the second wave (N₂₀₂₂ = 390). First, a model on the data of 2021 was constructed in which all items were loaded on one overarching latent EMP-3 construct. The model showed a poor fit, with χ²(135, N₂₀₂₁ = 368) = 839.19, p < .001, CFI = 0.43, RMSEA = 0.12 with 90% CI [0.11, 0.13] (i.e., the null-hypothesis of an RMSEA below 0.05 is rejected) and SRMR = 0.12. Second, we constructed a model analogous to De Maesschalck and colleagues [24] in which the EMP-3 items are loaded on their respective (i.e., three) latent subscales (see also Table 1). Again, the model showed a poor fit with χ²(132, N₂₀₂₁ = 368) = 487.00, p < .001, CFI = 0.71, RMSEA = 0.09, p < .001 with 90% CI [0.08, 0.09] and SRMR = 0.12. Note that the relation between the latent constructs of PPC and PNC showed a negative loading of -0.27. As a consequence, the model did not converge any more if we added an overarching (i.e., over the three subscales) EMP-3 higher order latent construct.

Following these results, we decided to remove the PPC subscale. Moreover, we also observed that items 1 and 9 loaded poorly on the TP scale. We therefore decided to remove these items as well. Further inspecting the MI of the last tested model, items 8 and 10 were related closely to the extent we considered a new subscale Background Perception or BP. Finally MI also indicated that item 3R had a stronger loading on PNC compared to TP.

Taking these results into account, we constructed a new Model₂₀₂₁ 1 with items 2, 4, 5, 6 and 7 loading on the TP latent subscale, items 8 and 10 loading on the BP subscale and items 3R, 17R and 18R loading on the PNC subscale. Finally, we also added an overarching latent construct representing a higher order REMP-3 factor for intercultural attitudes. The model showed an adequate to good fit, with χ²(27, N₂₀₂₁ = 368) = 53.88, p = .002, CFI = 0.96, RMSEA = 0.05, p = .41, 90% CI [0.03, 0.07] (i.e., the null-hypothesis of an RMSEA below 0.05 cannot be rejected) and SRMR = 0.04. Figure 2 shows the final Model₂₀₂₁ 1. To cross-validate the CFA, we applied the model structure from Model₂₀₂₁ 1 to the 2022-wave data. Model₂₀₂₂ 1 again showed an adequate to good fit, with χ²(27, N₂₀₂₂ = 390) = 60.97, p < .001, CFI = 0.96, RMSEA = 0.06, p = .26, 90% CI [0.04, 0.08] and SRMR = 0.04. Figure 3 shows the final Model₂₀₂₂ 1. In sum, our CFA analyses indicated that Model 1 provides cross-validated evidence for the REMP-3 instrument as a measure of intercultural attitudes.

Fig. 2

Model₂₀₂₁ 1: CFA of REMP-3 using the data from wave 2021. Note. (R)EMP-3 = (Revalidated) attitude towards Ethnic Minority Patients, TP = Task Perception, BP = Background Perception, PNC = Perceived Need for Communication. The observed variables are depicted as squares, the latent variables are depicted as circles. The items are annotated using the numbers from Table 1. Items annotated with an R are scored reversely

Fig. 3

Model₂₀₂₂ 1: CFA of REMP-3 using the data from wave 2022. Note. (R)EMP-3 = (Revalidated) attitude towards Ethnic Minority Patients, TP = Task Perception, BP = Background Perception, PNC = Perceived Need for Communication. The observed variables are depicted as squares, the latent variables are depicted as circles. The items are annotated using the numbers from Table 1. Items annotated with an R are scored reversely

Structural equation modeling: the Intercultural Framework of Intercultural competence

We started from Model₂₀₂₁ 1 and added the hypothesized relations as already introduced in Fig. 1, supplemented with the possible direct effects of MPQ and REMP-3 scales on the IOIC outcome variable. Model₂₀₂₁ 2 showed an adequate to good fit, with χ²(90, N₂₀₂₁ = 368) = 154.40, p < .001, CFI = 0.93, RMSEA = 0.04, p = .79 with 90% CI [0.03, 0.06] and SRMR = 0.04. The model showed an explained variance for CQ of R² = 0.32 and for IOIC of R² = 0.14. We further explored the somewhat lower CFI. Analyses thus revealed that the baseline model for Model₂₀₂₁ 2 (i.e., the model it is compared against to obtain relative fit measures like a CFI) has a low RMSEA = 0.139. Literature shows that in case of a low RMSEA (< 0.158) of the base model, incremental fit indices like the CFI have a practical maximum of 0.95 and should be interpreted with care [41]. Cross-validating the model, we applied the model structure from Model₂₀₂₁ 2 to the 2022-wave data. Model₂₀₂₂ 2 again showed an adequate to good fit, with χ²(90, N₂₀₂₂ = 390) = 151.60, p < .001, CFI = 0.94, RMSEA = 0.04, p = .88 with 90% CI [0.03, 0.05] and SRMR = 0.04. The model showed an explained variance for CQ of R² = 0.39 and for IOIC of R² = 0.14. The RMSEA = 0.149 of the baseline model was again lower than 0.158. Table 3 shows the final Model₂₀₂₂ 2. As Model₂₀₂₂ 2 does still feature a lot of non-significant effects, these effects could artificially improve the fit indices. As a control, we removed these non-significant effects in Model₂₀₂₁ 3 (see Fig. 4) and Model₂₀₂₂ 3 (see Fig. 5). Both models still showed an adequate to good fit, with χ²(76, N₂₀₂₁ = 368) = 138.87, p < .001, CFI = 0.93, RMSEA = 0.05, p = .62, 90% CI [0.04, 0.06] and SRMR = 0.04 for Model₂₀₂₁ 3 (note that the RMSEA = 0.155 of the baseline model was again lower than 0.158) and χ²(76, N₂₀₂₂ = 390) = 125.81, p < .001, CFI = 0.95, RMSEA = 0.04, p = .88, 90% CI [0.03, 0.05] and SRMR = 0.04 for Model₂₀₂₂ 3. For Model₂₀₂₂ 3 the baseline model reached an RMSEA = 0.164, which is above the 0.158 threshold, explaining the now slightly improved CFI value. In sum, the final revalidated REMP-3 has three subscales (i.e., TP, BP and PNC) and a higher-order factor that can be integrated as a measure of intercultural attitude into the framework of intercultural competence.

Table 3 SEM Model₂₀₂₂ 2 and hypotheses Testing

Fig. 4

Model₂₀₂₁ 3: CFA of REMP-3 using the data from wave 2021. Note. (R)EMP-3 = (Revalidated) attitude towards Ethnic Minority Patients, TP = Task Perception, BP = Background Perception, PNC = Perceived Need for Communication. MPQ = Multicultural Personality Questionnaire, CE = Cultural Empathy, OM = Open Mindedness, ES = Emotional Stability, CQ = Cultural Intelligence, IOIC = Intensity Of Intercultural Contacts. The observed variables are depicted as squares, the latent variables are depicted as circles. The items are annotated using the numbers from Table 1. Items annotated with an R are scored reversely.

Fig. 5

Model₂₀₂₂ 3: CFA of REMP-3 using the data from wave 2022. Note. (R)EMP-3 = (Revalidated) attitude towards Ethnic Minority Patients, TP = Task Perception, BP = Background Perception, PNC = Perceived Need for Communication. MPQ = Multicultural Personality Questionnaire, CE = Cultural Empathy, OM = Open Mindedness, ES = Emotional Stability, CQ = Cultural Intelligence, IOIC = Intensity Of Intercultural Contacts. The observed variables are depicted as squares, the latent variables are depicted as circles. The items are annotated using the numbers from Table 1. Items annotated with an R are scored reversely.

Hypothesis testing

The hypothesis testing was conducted conservatively, as we opted to use the cross-validated Model₂₀₂₂ 2 described in Table 3. In such a way, the effects of one variable are controlled for the effects of the other (relevant) variables, while the full model is constructed on a different dataset.

First, we have found at least partial evidence for H1, as a high disposition on the two intercultural traits of CE and OM predicts a more ethnorelative disposition on the REMP-3 (i.e., a higher score). Note that the ES disposition showed a reverse effect. Such an effect is not uncommon, but is addressed in the Discussion. Second, we have found at least partial evidence for H2, as a high disposition on the intercultural traits of CE and OM predicts a higher cultural intelligence, measured by CQ. Third, we have found evidence for H3, as a more ethnorelative disposition measured by the REMP-3 predicts a higher cultural intelligence measured by CQ, while controlling for the effects of intercultural traits. Finally, we have found evidence for H4 as a higher cultural intelligence as measured by CQ predicts more intense cultural contacts as measured by IOIC, showing criterion validity.

In sum, all hypotheses were (at least partially) confirmed. These conclusions are further supported by an acceptable to good internal consistency reliability as shown by Table 2. As such, we conclude that the REMP-3 functions as a valid and reliable measure of the central intercultural attitudes component in the framework of intercultural competence (see Fig. 1).

Controlling for gender, age and experience

Thirty-four participants chose to not disclose information on their age or experience, three participants indicated a different gender orientation. These participants were not included in the analyses. Analyses were conducted on the pooled dataset of N₂₀₂₁ and N₂₀₂₂. All effects were standardized. Regressing the REMP-3 score on age, gender and healthcare experience rendered a significant linear model, F(3, 717) = 13.29, p < .001, R² = 0.05, with a non-significant effect of experience (β = 0.01, p = .93), a possible trending effect of age (β = 0.09, p = .12), with younger participants obtaining higher scores, and a significant effect of gender (β = − 0.22, p < .001), with a lower average score for the male gender. As experience rendered a non-significant effect, experience was removed from further control analyses. For H1, the REMP-3 score is regressed on MPQCE, MPQOM, MPQES, age and gender. The linear model was significant, F(5, 749) = 28.04, p < .001, R² = 0.16, with significant effects of MPQCE (β = 0.16, p < .001), MPQOM (β = 0.23, p < .001), MPQES (β = − 0.09, p = .02), age (β = 0.09, p = .01) and gender (β = − 0.18, p < .001). Although age and gender have a significant effect, both variables do not change the conclusions for H1. For H2 and H3, CQ was regressed on MPQCE, MPQOM, REMP-3, age and gender. The linear model was again significant, F(5, 749) = 64.91, p < .001, R² = 0.30, with significant effects of MPQCE (β = 0.17, p < .001), MPQOM (β = 0.34, p < .001) and REMP-3 (β = 0.22, p < .001). The effects of age (β = 0.02, p = .49) and gender (β = 0.02, p < .49) were not significant (even in the absence of MPQCE, MPQOM and REMP-3) and thus do not change the conclusions for H2 and H3. Finally, for H4, IOIC was regressed on CQ, OM, age and gender. The linear model was again significant, F(5, 749) = 23.29, p < .001, R² = 0.14, with significant effects of MPQOM (β = 0.29, p < .001) and CQ (β = 0.12, p = .003). The effect of age was not significant (β = 0.00, p = .98), while the effect of gender was trending (β = 0.07, p = .06). The effects of age and gender do not change the conclusions for H4.

Discussion

Answering the call for more, and more rigorous, revalidation of existing instruments [8,9,10], the present study aimed to revalidate the Ethnic Minority Patients attitude measure for physicians (EMP-3) for use in graduate healthcare practitioners [24]. To this extent, the present study aimed to integrate a revalidated EMP-3 instrument or REMP-3 into the framework of intercultural competence as a measure of intercultural attitudes. To assess this reintegration, a number of hypotheses were drawn from intercultural competence literature regarding the interplay between traits, attitudes, capabilities and effectiveness. Important to note, all results were cross-validated using a second, independent sample of graduate healthcare practitioners.

More specifically, the content of the EMP-3 for physicians was revalidated towards the REMP-3 for a broad graduate healthcare practitioner population by removing one of three subscales and two poor loading items (see also Table 1). The Task Perception (TP) and the Perception of Needs in Communication (PNC) were retained, while the attitude towards Physician – Patient Communication (PPC) was removed entirely as the subscale correlated negatively with the PNC. Apart from the statistical arguments, we consider the PPC subscale somewhat ambiguous, as the items probe for both the current situation as well as the ideal situation. For sure, item 14 “The communication between physicians and patients is facilitated when they share the same social background.” is based on empirical findings [42], but is also contradictory to an ideal world in which social background is no longer relevant. Participants can experience a conflict between actual and ideal situations, which could explain the statistical findings regarding the PPC subscale. Considering the results and the possible explanation, we decided to remove the PPC subscale in the present study’s iteration of the REMP-3 instrument. We assume that this removal of the PPC subscale is warranted, as the SEM analyses indicate a better fit of the instrument data without the PPC subscale, while also allowing a higher order factor. However, future research can reconsider the use of the subscale by testing our hypothetical explanation of actual versus ideal situation conflict in participants. Research can compare both situation hypotheses by adjusting the question that is posed to participants accordingly.

Moreover, the revalidated REMP-3 instrument was successfully integrated as an ethnorelative measure of intercultural attitudes (and world views) into the framework of intercultural competence [22, 23]. Largely in line with (healthcare) literature, a higher disposition on intercultural traits predicts a more ethnorelative attitude [28] and a higher cultural intelligence [29]. A more ethnorelative attitude also predicts to a higher cultural intelligence [31], while higher cultural intelligence predicts more intense intercultural contacts [27]. Note that intercultural attitudes do not have a direct effect on more intense cultural contacts. This result is actually in line with literature, as the framework does not presume a direct link between attitudes and real life outcomes.

Despite not having a direct effect on real life outcomes, intercultural attitudes are at a central position of intercultural competence, as an ethnorelative disposition positively affects the ability of enlarging intercultural capabilities by acquiring new skills and knowledge. We estimated this positive effect at r = .36. The emerging literature on intercultural competence framework validation studies reports similar correlations of r = .40 [15] and r = .37 [22]. Important to note, these studies used a different operationalization of intercultural attitudes, which provides additional evidence that the effects of the REMP-3 measurements are not instrument-specific. Rather, these REMP-3 effects appear to represent genuine effects of an ethnorelative intercultural attitude.

Furthermore, an appropriate empathic and open minded trait disposition also facilitates an ethnorelative disposition, while too much emotional stability seems to inhibit an ethnorelative disposition, especially regarding task perception. As an explanation, we consider that a lower emotional stability in a graduate healthcare practitioner can also indicate a higher level of involvement towards patient care and the tasks that need performing. Indeed, the involvement regarding the patient’s condition can trigger an emotional reaction. This involvement hypothesis for emotional stability is further supported by the negative correlation between emotional stability and cultural empathy (see Table 2). It is also not uncommon to observe effects that are not featuring in the original framework. Similar unexpected, minor effects were also observed in earlier empirical tests of the framework of intercultural competence [22].

Theoretical implications

Showing ample construct validity, the REMP-3 is a straightforward psychometric upgrade compared to the original EMP-3 (see also Tables 1 and 2). First, the REMP-3 now has a clear and cross-validated content and structure, with three subscale dimensions and an overarching higher order full scale dimension. Second, both the subscales as well as the full scale show acceptable to good internal consistency reliability, even though the total number of items is reduced from eighteen to ten. Third, the REMP-3 has stronger correlations to the related constructs of intercultural competence, while even showing a positive correlation with the intensity of intercultural contacts, which the original EMP-3 does not. As a result, the quality of the REMP-3 structure was sufficient to empirically integrate and cross-validate the REMP-3 as a measure of intercultural attitudes into the theoretical framework of intercultural competence [23].

Practical implications

Practically, the cross-validated REMP-3 is suited to question graduate healthcare practitioners on their attitudes towards ethnic minority patients in a concise manner (i.e., only ten items). As intercultural attitudes are at a central position of intercultural competence, measuring the attitudes of graduate healthcare practitioners can give an indication to which extent the potential to learn new intercultural capabilities is facilitated (i.e., in case of a more ethnorelative disposition) or hampered (i.e., in case of a more ethnocentric disposition) by the practitioner’s attitudes [31]. Awareness and change of intercultural attitudes can thus prove to be key in order to understand and address issues like ethnic differences and racism in healthcare systems through learning processes [6, 7].

As an example, supervisors can systematically evaluate and monitor the progress of healthcare internships or trial periods by administering the REMP-3 assessment at predetermined intervals, such as before the start of the internship and after its conclusion. This systematic approach allows for a structured comparison of the intern’s performance over time. In conjunction with other evaluative methods, such as direct observations of interns in clinical settings (e.g., general practices or hospitals), the REMP-3 results enable supervisors to assess whether there has been a measurable shift in the intern’s attitudes. Furthermore, the impact of these attitudinal changes on the intern’s effectiveness in intercultural patient interactions can be analysed, offering further insights into their acquired skills and knowledge.

Limitations and Future Research

The present study revalidates the REMP-3 for use in a broad population of graduate healthcare practitioners. However, the participants for the present study were all still studying albeit with various amounts of experience in a wide range of medical settings, which does seem appropriate for the present study’s goals. Still, future studies should focus on replicating the results of the present study in more specific samples of healthcare practitioners, preferably over a wide range of work settings like general nursery, psychiatric nursery and surgery assistance. We are cautiously optimistic that the REMP-3 results will be replicated, especially as the results were cross-validated on independent data samples while showing no effects of previous healthcare experience on the REMP-3 score. We attribute this non-effect to the more general approach of the instrument, further strengthened by using non-specific, non-medical questionnaires to operationalize intercultural competence [21] and intercultural effectiveness [27].

The present study does show that younger (i.e., age) and female (i.e., gender) practitioners can score higher on the REMP-3. However, age and gender do not seem to have an effect on acquiring intercultural capabilities as an ethnorelative attitude combined with cultural empathy and an open mind are predictive of the already learned cultural capabilities. We do acknowledge that the research line regarding the effects of covariates like gender, age and experience on the components of intercultural competence is quite complex and requires more research. Such future research on components like ethnorelativism as measured by the REMP-3, should always include a full framework of intercultural competence to assess the validity of the (sub)scales, as the validity of an instrument’s results is function of both the instrument as well as the population [43].

Conclusions

The cross-validated REMP-3 attitude instrument concisely and reliably measures intercultural attitudes of a broad population of healthcare practitioners towards ethnic minority patients. The instrument can be used in full or split out in three subscales of task perception, background perception and the perception of needs in communication. The REMP-3 instrument is therefore suited to assess and monitor the attitude of graduate healthcare practitioners, even over longer periods of time like the stages of an internship. Future research can evaluate the impact of this attitude assessment and monitoring in other settings like general practices and with a different, more experienced target population including but not limited to nurses and medical assistants. Ultimately, the REMP-3 instrument can contribute to more equity in healthcare by assessing and monitoring attitudes in healthcare practitioners, as these attitudes indicate the potential of acquiring new skills and knowledge to properly address interactions with ethnic minority patients.