- Research
- Open access
- Published:
Quantitative evaluation of two-way referral policies based on PMC index model
International Journal for Equity in Health volume 24, Article number: 8 (2025)
Abstract
Objective
Perfect policy system support is the key to effective work, and policy evaluation can provide theoretical basis for policy revision. This study aims to quantitatively evaluate the two-way referral policy in China and thus improve the policy system of two-way referral.
Methods
This research focuses on 14 two-way referral policies published from 2007 to 2021. Basing on the focus of the two-way referral policy identified by text-mining methods, we constructed a PMC index model containing 9 primary and 43 secondary variables to conduct a quantitative evaluation of the two-way referral policy.
Results
All document types are work notices. The two-way referral policy issued by relevant departments have gradually increased since 2015.The average PMC index of the 14 documents was 5.84, and the policy with the highest PMC index score was the policy P14 issued by Jiangsu province. There are 6 excellent policies, 8 good policies, and no qualified grade policy. The difference in PMC index scores mainly shows the three dimensions of policy clarity, policy content and policy coverage.
Conclusions
This study indicates that the quality of two-way referral policy is generally good, which reveals the factors affecting the policy operability and contributes to expand the research perspective of two-way referral policy evaluation.We also analyzed the advantages and disadvantages of the existing policy of two-way referral through policy evaluation, and provided guidance for subsequent policy revisions.
Introduction
The reform of China’ s health system focuses on the hierarchical diagnosis and treatment system, the modern hospital management system, the universal medical insurance system, the drug supply guarantee system and the comprehensive supervision system, among which the hierarchical diagnosis and treatment system is the first of China’ s “five basic medical systems”. The basic mode of hierarchical diagnosis and treatment is first diagnosis at primary medical institutions, two-way referral, classification and treatment of acute and chronic conditions, and linkage between higher level and subordinate medical institutions [1]. Among which two-way referral is an important embodiment of the operation effect of hierarchical diagnosis and treatment [2]. Patient referral rates in other countries are generally between 10 and 30%, but less than 5% in China [3].
China’ s public medical institutions are divided into first-level hospitals, second-level hospitals and third-level hospitals. The higher the level of hospitals, the higher the difficulty of the diseases that can be treated. As tertiary hospitals have high-quality medical resources such as doctors with superb medical technology and advanced medical equipment, patients will go to tertiary hospitals directly as long as they get sick [4]. To relieve the pressure on tertiary hospitals, China further optimized the referral policy in 2015. According to the types of diseases, the reimbursement rate of medical insurance is differentiated from primary hospitals to tertiary hospitals, so as to restrict patients’ medical treatment behavior. The essence of two-way referral is to guide patients to different levels of medical institutions according to the severity of patients’ disease [5], and to reasonably divert patients who were concentrated in tertiary medical institutions before.
A sound policy document can point out the direction of action for health management reform, while policy evaluation can evaluate the advantages and disadvantages of existing policies and provide guidance for subsequent policy revisions [6]. The domestic and foreign studies on two-way referral focus on the aspects of two-way referral mode, implementation status, problem countermeasures, and efficiency of referral [7]. The analysis of two-way referral policy mainly focuses on literature review [8], and there are few studies on the analysis of policy quantification. The more commonly used tool in policy evaluation is PMC index model, which can scientifically and intuitively compare multiple policies [9]. In this study, we established the PMC index model, quantified the policies related to two-way referral, and explored the potential improvement points in its policy system, which provided a theoretical basis for the construction of a reasonable and effective two-way referral mode.
Methods
Data source
We took “two-way referral” and “referral” as the theme words, combined with the authority of policy sources, and searched in the CNKI government document database, the official website of the health commission of national and provincial (including autonomous regions and municipalities), and the Peking University Law database. Since the official establishment of the two-way referral system was 1997, the search publication time was limited to January 1, 1997-December 31, 2023. The inclusion criteria for the document are as following: the document type is one of the notice, opinion, method and scheme; the document issuing agencies are at the central government, provinces, municipalities directly under the Central Government and special economic zones; the subject of the document is the two-way referral. As illustrated in Tables 1 and 14 documents are included in this study, and the policy text is coded according to P1-P14.
Policy text mining and variable selection
We used ROST CM 6.0 to clarify the high-frequency words in the two-way referral policy document through text mining, eliminate the words that lack statistical significance such as “two-way referral”, “following” and “according”, and finally determined 60 high-frequency words with high frequency and reflect the focus of two-way referral (Table 2). As shown in Table 3, based on text mining and combined with other literature, 9 first level variables and 43 secondary variables were selected in this study.
The PMC index model
The PMC index model, proposed by Ruiz Estrada [16], is a common method to evaluate policy documents. It can present the advantages and disadvantages of various policies from multiple dimensions, and put forward a more scientific and effective path for policy improvement [17]. The PMC index is calculated in four steps [12]. Firstly, Multi-input-output tables were constructed using primary and secondary variables, as shown in Table 4; Secondly, the secondary variable is assigned. If the index reflected by the secondary variable is involved in the policy text, the secondary variable is assigned to 1, otherwise, the value is 0. Thirdly, the primary variable score is calculated, which is equal to the mean of all its secondary variables combined. Finally, the sum of the primary variable scores was calculated for each policy. Taking into account the relevant policy quality classification standard [18], the policy is divided into four grades: perfect policy (8–9 points), excellent policy (6-7.99 points), acceptable policy (4-5.99 points) and qualified policy (0-3.99 points).
Results
Publishing characteristics of the two-way referral policy
According to the included policy text, all document types are work notices. As illustrated in Fig. 1, the first document in line with the theme of two-way referral was released in 2007, and the number of two-way referral policy releases has gradually increased since 2015. 2017 was the year with the most publications, followed by 2015 and 2021, both with two policy releases. The core issuing agency of the two-way referral documents was the Health Commission, and 12 out of the 14 documents were issued by the health commission of each province and city. Only one document is joint publication, while the rest are independent publication, lacking the cooperation of multiple departments.
Network analysis of the two-way referral policy
The social semantic network analysis of the 14 included policy texts was conducted by ROSTCM.6 software. As shown in Fig. 2, the two-way referral policy follows the two subjects of medical institutions and patients. The types of two-way referral patients can be seen from the high-frequency words such as “rehabilitation”, “chronic disease”, “frequently-occurring disease” and “complicated”. From the words “clear diagnosis”, “principle”, “condition” and “evaluation”, we can know that the key to two-way referral lies in confirmed patients. From the words “higher level”, “subordinate” and “up and down linkage”, we can see the linkage working mechanism of two-way referral. From the words “process”, “sign a contract” and “make an appointment”, we can find that the two-way referral emphasizes the standardization of referral.
Overall evaluation of the PMC index for two-way referral policy
As shown in Table 5, the average PMC index of 14 policies is 5.84. Six policies are rated as excellent, eight policies are acceptable, and no qualified policies. The design of two-way referral policy is generally reasonable. The highest PMC index score is policy P14, and the lowest score is policy P2. As indicated in Fig. 3, through the comparative analysis of all policies through the radar chart, it can be found that the difference of PMC index scores mainly shows three dimensions: policy clarity, policy content and policy coverage. From the perspective of policy evaluation indicators, among the 9 primary variable PMC index scores, the policy evaluation and policy clarity scores are high; the policy coverage score is low. Among the regional distribution, the two-way referral policies issued by Jiangsu, Yunnan and Hainan provinces had high scores, while the policies issued by Xinjiang Uygur Autonomous Region had the lowest scores.
Discussion
The development process of two-way referral policy
In 1997, relevant departments proposed the establishment of two-way referral system for the first time [19]. This research found that the relatively standard two-way referral document was only released in 2007, with a stagnation period of nearly 10 years, which is consistent with the research conclusions of other scholar [20]. Most of the studies on hierarchical diagnosis and treatment in China were after the new health care reform in 2009 [21]. In 2015, The General Office of the State Council issued the Guiding Opinions on Promoting the Construction of the Hierarchical Diagnosis and Treatment System, requiring it to gradually improve the hierarchical diagnosis and treatment system by 2017, and comprehensively improve the hierarchical diagnosis and treatment capacity by 2020, which is of positive significance to the implementation of two-way referral in various regions [22]. As illustrated in Fig. 1, the release of two-way referral policy texts increased significantly since 2015. Although a series of two-way referral documents was developed in response to national policies, the 14 policy texts included in this study show that their PMC index ranges from 4.99 to 6.86 and there is room for improvement.
Focus of two-way referral policy
As shown in Fig. 2, the smooth flow of two-way referral involves communities, medical institutions, medical staff, patients and other stakeholders. The policy formulation needs to comprehensively consider “willing to referral”, “whether to referral” and “how to referral” [23]. At present, the included policy texts mainly focus on the issue of “how to transfer”. According to the PMC index score, most policies define the scope of referral implementation and have clear referral procedures. Meanwhile, the division of rights and responsibilities among medical institutions and health departments at all levels is also standardized. For example, the Hainan Provincial Policy (P1) mentions: the Health Bureau organizes and implements two-way referral, secondary medical institutions undertake professional training tasks, and the community is responsible for popularizing the concept of scientific medical treatment to the public. The policies included in this study have had relatively little discussion on the issue of “ability to referral”, with only 43% of policies mentioning personnel training. However, general practitioners act as the gatekeepers of primary diagnosis and two-way referral whose service ability is closely related to the effect of hierarchical diagnosis and treatment. Therefore, the future policy formulation are supposed to emphasize this content.
Factors affecting the feasibility of two-way referral policy
Among the regional distribution, the two-way referral policies issued by Xinjiang Uygur Autonomous Region had the lowest scores, which may be related to the level of economic development and special geographical location of Xinjiang Uygur Autonomous Region [24]. There are many landforms, such as mountains, gobi and desert in Xinjiang Uygur Autonomous Region, and the winter snow season is long there, which causes some difficulties for the implementation of two-way referral work. As a result, the two-way referral policy formulated by relevant agency only stipulates some basic problems. Establishing a specific and continuous assessment mechanism is essential for providing continuous referral services [3]. Lack of clear referral guidance is an important reason affecting the hospital’s referral work [8]. As shown in Fig. 3, it can be found that the difference of PMC index scores mainly shows three dimensions: policy clarity, policy content and policy coverage. For the evaluation of the effectiveness of two-way referral work, there are few quantitative indicators in the policy, and each indicator lacks quantitative provisions, which reduces the feasibility in the policy assessment. In the two-way referral policy, the lowest PMC index scores were found in P2 and P3. Due to the lack of instructions on referral principles, standards, and protocols, P2 scored poorly in the policy content dimension. Due to the lack of policy publicity, system improvement and personnel training content, P3 scored low in the policy guarantee dimension. SMART principle (specific, measurable, attainable, relevant, time-based) was often used for designing practical indicators. When developing two-way referral policies in the future, we could consider adding indicators including the referral rate, referral speed, referral service quality, health care cost service efficiency and referral residents’ perception. Besides, the policy should also reflect the assessment period, indicators, methods and effectiveness significance of two-way referral work, which is also of great significance for promoting the development of two-way referral work.
The relationship between long-term policies and short-term policies
From the perspective of policy timeliness (X2), most policies belong to long-term policies, and only two documents are short-term policies (P9 and P11). This may be because long-term policies, as a macro guidance document, have the characteristics of overall, universal and comprehensive, which is of great significance for promoting the development of healthcare undertakings. However, promoting two-way referral should not only rely on long-term policies, but also need short-term policies. The validity period of the short-term policies is often within 1–3 years, with good flexibility. Short-term policies can be improved under the guidance of the long-term policies, as the basis for testing the phased completion effect of the long-term policy goals, which is more practical. Thus, detailed and executable implementation plans should be designed under the guidance of long-term policies [25].
Policy coverage and two-way referral
The PMC index of policy coverage score (X9) was low, indicating that less attention was paid to the hierarchical diagnosis and treatment system in the two-way referral document. In 2016, WHO proposed the framework of integrated medical and health service system. The integrated medical and health service system has reshaped the functional orientation of communities, rehabilitation nursing institutions and medical institutions, providing patients with integrated life cycle services such as health promotion, disease prevention and treatment. It is necessary to build a patient-centered and community-based integrated medical service system to improve the accessibility and satisfaction of residents’ medical treatment. Some scholars point out that taking the medical alliances as the carrier, expanding the coverage of remote collaboration network, signing up family doctors, and strengthening community construction can effectively promote two-way referral.
Conclusion
There is no quantitative study of two-way referral policy before. Basing on text mining and PMC index model, this study quantitatively evaluated 14 two-way referral policies from 2007 to 2021 issued by the Chinese government. We also explored the potential improvement points in two-way referral policies, which provided a theoretical basis for the construction of a reasonable and effective two-way referral mode. The two-way referral policy is overall reasonable which mainly revolves around patients and medical institutions. In the process of policy formulation, relevant departments tend to focus more on the foundation, content, rights and obligations of the objects involved, and clarity of policies, while paying less attention to the timeliness, coverage and feedback of policies. To optimize the two-way referral policy, we suggest increasing the short-term policy with a time span of less than 3 years, and clarifying the applicable institutional level and evaluation indicators when formulating the policy, such as: referral rate, referral speed, referral service quality, and views of referral residents, so as to enrich the policy content. Furthermore, enhancing cross-departmental collaboration through the establishment of referral centers is also a notable issue because referral involves different levels of medical institutions.
Data availability
The datasets used and analysed during the current study are available from the corresponding author on reasonable request.
Abbreviations
- PMC-Index:
-
Policy Modeling Consistency Index
References
Luo D, Zhu X, Qiu X, et al. Healthcare preferences of chronic disease patients under China’s hierarchical medical system: an empirical study of Tianjin’s reform practice[J]. Sci Rep. 2024;14(1):11631.
Wei Z, Bai J, Feng R. Optimization referral rate design for hierarchical diagnosis and treatment system based on accessibility-utilization efficiency bi-objective collaboration: a case study of China[J]. Soc Sci Med. 2023;322:115827.
Zhao D, Chen S, Jin S, et al. Predictors of referral behaviour and intention amongst physicians in a medical consortium based on the theory of planned behaviour: a cross-sectional study in China[J]. Front Public Health. 2023;11:1159207.
Lu C, Zhang Z, Lan X. Impact of China’s referral reform on the equity and spatial accessibility of healthcare resources: a case study of Beijing[J]. Soc Sci Med. 2019;235:112386.
Greenfield G, Foley K, Majeed A. Rethinking primary care’s gatekeeper role[J]. BMJ,2016,354:i4803.
Liu L, Chen J, Wang C et al. Quantitative evaluation of China’s basin ecological compensation policies based on the PMC index model[J]. Environ Sci Pollut Res Int 2023,30(7):17532–45.
Aboelkhir H, Elomri A, Elmekkawy TY et al. A bibliometric analysis and visualization of decision support systems for Healthcare Referral Strategies[J]. Int J Environ Res Public Health,2022,19(24).
Kane S, Jiang H, Tian Y et al. Making effective referrals happen: a theory-informed policy analysis[J]. Health Policy Plan 2021,35(10):1309–17.
Hong S, Wang T, Fu X, et al. Research on quantitative evaluation of digital economy policy in China based on the PMC index model[J]. PLoS ONE. 2024;19(2):e298312.
Estrada MAR. Policy modeling: definition, classification and evaluation[J]. J Policy Model 2011,33(4):523–36.
Guo D, Qi L, Song X. Quantitative evaluation of the medicine innovation policy in China: based on the PMC-Index model[J]. Front Public Health. 2024;12:1403320.
Yang Y, Tang J, Li Z, et al. How effective is the health promotion policy in Sichuan, China: based on the PMC-Index model and field evaluation[J]. BMC Public Health. 2022;22(1):2391.
Yan W. Quantitative evaluation study of medical security treatmentpolicy in Hainan Province - based on pmc index model[D]. Hai Nan Medical University; 2023.
Yang C, Yin S, Cui D, et al. Quantitative evaluation of traditional Chinese medicine development policy: a PMC index model approach[J]. Front Public Health. 2022;10:1041528.
Jiaxin C, Tingting W, Manxia F, et al. Quantitative evaluation for payment policy of diagnosis-related groups in China based on PMC index mode[J]. China Pharm. 2022;33(13):1559–64.
Ruiz Estrada MA, Yap SF, Nagaraj S. Beyond the Ceteris Paribus Assumption: modeling demand and supply assuming Omnia Mobilis[J]. Social Science Electronic Publishing; 2010.
Wei Q, Wang X, Zhang G et al. Internet Healthcare Policy Analysis, Evaluation, and Improvement Path: Multidimensional Perspectives[J]. Healthcare (Basel),2023,11(13).
Li S, Shen J, Sun F, et al. Quantitative evaluation of ecological compensation policies for the watershed in China: based on the improved policy modeling consistency Index[J]. Volume 29. Environmental Science and Pollution Research; 2022. pp. 66659–74. 44.
Yu W, Li M, Nong X et al. Practices and attitudes of doctors and patients to downward referral in Shanghai, China[J]. BMJ Open,2017,7(4):e12565.
Xiaojia C. Analysis on the implementation situation of two-way referral system in China[J]. China Econ,2022(02):241–2.
Li L, Fu H. China’s health care system reform: Progress and prospects[J]. Int J Health Plann Manage 2017,32(3):240–53.
Shi H, Cheng Z, Liu Z, et al. Does a new case-based payment system promote the construction of the ordered health delivery system? Evidence from a pilot city in China[J]. Int J Equity Health. 2024;23(1):55.
Blank L, Baxter S, Woods HB et al. Referral interventions from primary to specialist care: a systematic review of international evidence[J]. Br J Gen Pract 2014,64(629):e765–74.
Feng CY, Liang RH, Jiang XM. Analysis of the Government Health expenditure in the First Decade of Chinese New Medical Reform (2009–2018): Xinjiang Uygur Autonomous Region as an Example[J]. Risk Manag Healthc Policy. 2020;13:387–95.
Yang J, Siri JG, Remais JV et al. The Tsinghua–Lancet Commission on Healthy cities in China: unlocking the power of cities for a healthy China[J]. The Lancet,2018,391(10135):2140–84.
Acknowledgements
Not applicable.
Funding
The research was funded by Research and Develop Program, West China Hospital of Stomatology Sichuan University, grant number RD-01-202409, Philosophy and Social Sciences Planning Project of Guangdong Province in 2023: Research on the mechanism of national voluntary epidemic prevention behavior based on Repast-fsQCA in the context of COVID-19 “Class B and B control”, grant number GD23CGL11, Natural Science Foundation of Guangdong Province in 2022: Construction and application of COVID-19 control model PSR-SOR-Haddon in Guangdong Province, grant number 2022A1515011112, and Guangzhou Municipal Science and Technology Program key projects: Research and development of common diseases diagnosis and treatment system for middle-aged and elderly people based on artificial intelligence, grant number 202103000037.
Author information
Authors and Affiliations
Contributions
Conceptualization, J.G. and G.S.; Data curation, J.G., J.D. and W.X.; Formal analysis, J.G.; Writing - original draft, all authors; Writing - Review & Editing, L.S., S.L., O.X. and G.S.
Corresponding authors
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Gong, J., Shi, L., Deng, J. et al. Quantitative evaluation of two-way referral policies based on PMC index model. Int J Equity Health 24, 8 (2025). https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12939-024-02373-3
Received:
Accepted:
Published:
DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12939-024-02373-3