1Department of Preventive Medicine, Korea University College of Medicine, Seoul, Korea
2Department of Laboratory Medicine and Genetics, Soonchunhyang University Bucheon Hospital, Soonchunhyang University College of Medicine, Bucheon, Korea
3Department of Family Medicine, Kangdong Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Korea
4Division of Korean Cancer Management Guideline Network, National Cancer Center, Goyang, Korea
5Department of Cancer Control, Graduate School of Cancer Science and Policy, National Cancer Center, Goyang, Korea
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
In the case of clinical practice guideline (CPG), the need for the prospective registration of protocols has been proposed several times. However, the registration of CPG protocols is not yet active. The objective of this study was to summarize the experience of the CPG protocol registration program in Korea.
Methods
This study was performed in the following order: 1) formation of a methodological expert group; 2) CPG protocol template development; 3) CPG protocol preparation and expert review; 4) exploration of the knowledge and attitude of the guideline developers toward CPG protocol.
Results
The final version of the CPG protocol templates consists of four parts (planning, development, finalization, and timetable). The protocols for 18 cancers were submitted by 14 medical societies. conflicts of interest (n = 14, 77.8%), guideline development group (GDG; n = 9, 50%), scope of CPG (n = 9, 50%), and key questions (n = 8, 44.4%) were the under-reported areas in the submitted protocols. The GDGs (n = 13, 72.7%) was the most misreported areas of the protocol. CPG developers generally agreed on the advantages of protocol registration but responded that it was difficult to understand the concepts in the protocol and fill them with appropriate content. The areas where CPG developers responded that they felt difficulty were recommendation grade (n = 9, 75.0%), GDG composition (n = 7, 58.3%), and determining key questions (n = 7, 58.3%).
Conclusions
The CPG protocol registration program was planned and piloted in Korea, and it could be said that it is feasible. It is necessary to evaluate the developed CPG later and determine whether protocol registration affects the quality of CPG through indices such as transparency and clarity of CPG.
Evidence-based medicine requires the core process of generating, synthesizing, and applying research evidence. This core process is conducted through systematic review and clinical practice guideline (CPG) development process. To proceed effectively with this process, a transparent plan (protocol) should be established in advance, and the actual process should be undertaken according to these plans [1]. These protocols should be disclosed in advance through registration or publication, and through these activities, reporting bias, such as selective reporting of results or publication bias, can be reduced [2].
In the case of clinical trials or systematic literature reviews, the activity of preparing a protocol in advance and registering it is already being established. For systematic reviews, PROSPERO (https://www.crd.york.ac.uk/prospero/) and Open Science Framework (https://osf.io/prereg/) are major registration platforms. In the case of clinical trials, the International Clinical Trials Registry Platform (https://www.who.int/clinical-trials-registry-platform/network), managed by the World Health Organization, and ClinicalTrials.gov (https://clinicaltrials.gov/), managed by the National Library of Medicine, are major prospective registration platforms [3].
In the case of CPGs, the need for the prospective registration of protocols has been proposed several times [4], and registration through the Guidelines International Network has partially progressed [5]. However, the registration of CPG protocols is not yet active.
The Korean National Cancer Center (KNCC) started a project to develop Korean cancer guidelines for 18 types of cancer together with the Korean Academy of Medical Sciences (KAMS). As part of the preparation process, it was decided to prepare a development protocol for each CPGs in advance and to create a registration platform for protocols.
This study aims to summarize the experience of the CPG protocol registration program in Korea. The CPG protocol registration program comprises preparing the CPG protocol according to a protocol template and providing feedback after expert reviews.
Methods
This study was performed in the following order: 1) formation of a methodological expert group; 2) CPG protocol template development; 3) CPG protocol preparation and expert review; 4) exploration of the knowledge and attitude of the guideline developer toward CPG protocol.
1. Cancer CPG development project of the National Cancer Center of Korea
The National Cancer Center of Korea formed a cancer CPG development project group. The National Cancer Center sent an official letter to Korean cancer societies to announce a CPG development support plan to develop a cancer CPG.
2. Formation of a methodological expert group
Three experts (S.Y.K., Y.K.L, and H.J.K), who participated as methodologists in the cancer treatment CPG project, established an evidence-based CPG development manual and decided to develop a template for CPG protocol.
3. Development of the CPG protocol template
Three experts (S.Y.K., Y.K.L, H.J.K) developed a protocol template so that developers from each academic society could properly establish protocols according to the development manual. With the help of this template, the CPG developer can create a protocol according to the planning, development, and finalization elements of the CPG development process suggested in the manual.
The draft protocol template was presented in the workshop for the developers and methodology advisors for the Korean Cancer Guideline. The protocol template, version 1.0, was distributed to developers and methodology advisors. Three experts elaborated the protocol template to version 2.0 to improve its editorial visibility.
4. CPG protocol preparation and expert review
Training on guideline development methodology and protocol preparation was provided to Guideline Development Group (GDG) members. Developers wrote and submitted protocols according to the developed protocol template. The 18 submitted CPG protocols were reviewed by 5 experts (1 protocol was reviewed by 2 experts). Feedback was provided by combining the opinions of two experts. The feedback content entailed 1) under-reporting or non-reporting, and 2) misreporting. A briefing session was held to provide feedback on the protocol, and a revised (final) version was submitted. A website for uploading CPG protocols will be built on the National Cancer Center website, and it will be opened soon.
5. Exploration of the knowledge and attitude of the guideline developer toward CPG protocol
A questionnaire was developed to explore the knowledge and attitude of the guideline developer toward CPG protocol, and a survey was conducted through a Google forms. After explaining the purpose of the survey, we investigated the general characteristics of respondents, whether they agree on the advantages of CPG protocol registration, the degree of difficulty in preparing the protocol, and the specific content that was hard to fill up.
Results
1. CPG protocol templates
The final version of the CPG protocol templates (Supplementary Material 1) comprises four parts (planning, development, finalization, and timetable. Part 1 (Planning) requires developers to document their advance planning for eight items: guideline development group (GDG), scope and purpose of the CPG, key questions, target users and healthcare settings, terms of reference, funding sources, and conflict of interest. Part 2 (Development) has four items: selection of relevant literature, evidence synthesis, from evidence to recommendation, and making recommendations. Part 3 (Finalization) contains an external review, an endorsement, and a reporting and authorship. Part 4 is the CPG development process timeline.
2. General characteristics of the planned CPG
CPG protocols for 18 cancers were submitted by 14 medical societies (Table 1). Table 2 shows the characteristics of the planned CPGs. All the 18 CPG GDGs included a steering committee, guideline panel, and conflict of interest committee; The median numbers of members of the steering committee, guideline panel, and conflict of interest committee were 8 (range 4–14), 34 (14–48), and 8 (4–14), respectively. The number of specialties involved in GDG had a median value of 4 (1–9). The largest number of chair's specialties was surgery field. De novo was mostly adopted for the CPG development method.
3. Expert feedback on the submitted protocol
Tables 3 and 4 show the feedback provided by five experts for the submitted protocol. Table 3 shows the under-reporting areas in the CPG protocols. Among the planned phase areas, feedback was made on conflicts of interest (n = 14, 77.8%), GDG (n = 9, 50%), scope of CPG (n = 9, 50%), and key questions (n = 8, 44.4%). In the development stage, literature search (n = 12, 66.6%), evidence level (n = 8, 44.4%), and recommendation grade (n = 8, 44.4%) were the main feedback areas. In the finalization stage, external review (n = 8, 44.4%) was the most common feedback area.
Table 4 shows the misreporting areas in the CPG protocols. Among the planned phase areas, feedback was made on GDGs (n = 13, 72.7%), purpose of CPG (n = 6, 33.3%), scope of CPG (n = 6, 33.3%), determining outcomes (n = 6, 33.3%). In the development phase, risk of bias assessment (n = 6, 33.3%) and level of evidence (n = 6, 33.3%) were the main feedback areas. In the finalization stage, external review (n = 4, 22.2%) and endorsement (n = 3, 16.7%) were the common feedback areas.
4. Exploration of the knowledge and attitude of the developers toward the CPG protocol
Individual CPG panels that participated in protocol preparing were asked about their knowledge and attitude toward protocol. Protocol writers generally agreed with what was known as an advantage of protocol registration, and the agreement rate was higher for preventing publication bias, and for improving transparency of CPG (Table 5). Protocol writers responded that it was difficult to understand the concepts in the protocol and fill them with appropriate content. The areas where CPG developers responded that they felt difficulty were recommendation grade (n = 9, 75.0%), GDG composition (n = 7, 58.3%), and determining key questions (n = 7, 58.3%).
Discussion
KNCC and KAMS decided to implement a CPG protocol registration project as part of a project to develop 18 types of cancer CPGs. The registration program went through the process of forming an expert team, developing a protocol template, reviewing and providing feedback on submitted CPG protocols, and exploring CPG developers' knowledge and attitudes toward the protocol. A review of CPG protocols found that conflicts of interest, literature search, guideline development group (GDG), and CPG scope definition were areas of underreporting, with GDGs being the most misreported area of the protocol. CPG developers generally agreed on the advantages of protocol registration, but responded that they had difficulty writing content about recommendation ratings, GDG composition, and key questions.
The research field in which protocol registration was first initiated was clinical trial. Clinicaltrials.gov was first established in 2000, after the United States Congress passed the Requiring Trial Registration Act. Protocol registration for clinical trials is known to reduce the risk of publication bias, such as non-reporting, partial reporting, and selective reporting of study results [6]. However, according to a meta-analysis of protocol registration for clinical trials, only 20% of randomized controlled trials registered protocols in advance [7].
In the case of systematic literature reviews, protocol registration is also known to improve the quality of systematic review reporting [8], improve transparency of content, [9] and help research design and performance [10].
While there were proposals for CPG protocol registration, the CPG protocol registration program has never been implemented. Therefore, the CPG registration program as established in this study is, to the best of our knowledge, the first case in the world. Since transparency and clarity are also very important in CPG development, protocol registration is essential in this area.
As shown in this study, several issues must be addressed in the CPG protocol registration program. The first is the attitude of CPG developers. As shown in this study, the CPG developers exhibit a mixed attitude toward protocol registration. They agreed with the advantages of protocol registration but responded that it was difficult to understand the concepts in the protocol and fill them with appropriate content. This can be resolved by reducing the content of the protocol template or by developing a protocol-writing manual. Second, there are many cases of underreporting and misreporting in the contents of the protocol. This may be related to the difficulty of preparing the protocol that developers responded to. For this part, it may be necessary to clarify the contents of the protocol template, to have an educational program, and to provide an example of template.
In conclusion, the CPG protocol registration program was planned and piloted in Korea. Consequently, the CPG protocol registration could be completed without much difficulty. It is necessary to evaluate the developed CPG later and determine whether protocol registration affects the quality of CPG through indices such as transparency and clarity of CPG.
Notes
Conflict of Interest
Soo Young Kim has been an editor of the Journal of Evidence-based Practice since 2025. However, he was not involved in the peer reviewer selection, evaluation, or decision process of this article. No other potential conflicts of interest relevant to this article were reported.
Funding
This study was supported by national cancer center grant (2112570), National Cancer Center, Republic of Korea.
Data Availability Statement
All data generated or analyzed during this study are included in this published article and its supplementary information files.
Ethics Approval and Consent to Participate
Not applicable.
Author Contributions
Conceptualization: Kim SY, Lee YK, Kim Y. Data curation: Kim SY, Kim HJ. Funding acquisition: Wang KC, Gwak HS. Methodology: Kim SY, Kim HJ, Lee YK. Writing - original draft: Kim SY, Kim HY, Lee YK. Writing - review & editing: Kim SY, Kim HJ, Lee YK, Wang KC, Gwak HS, Kim Y.
Acknowledgments
We would like to thank Professor Seungsoo Sheen, Ajoo University and Dong-Ah Park who were reviewed submitted protocols. In addition, we would like to thank the members of KAMS, CPG executive committee and Korean Cancer Guidelines Project Group, for providing many comments on the research.
2. Shinohara K, Tajika A, Imai H, Takeshima N, Hayasaka Y, Furukawa TA. Protocol registration and selective outcome reporting in recent psychiatry trials: New antidepressants and cognitive behavioural therapies. Acta Psychiatr Scand 2015; 132: 489-98.
3. Nagendrababu V, Duncan HF, Dummer PM. International Endodontic Journal policy on mandatory prospective (a priori) protocol registration for clinical trials and systematic reviews. Int Endod J 2021; 54: 1685-6.
4. Chen Y, Guyatt GH, Munn Z, Florez ID, Marušić A, Norris SL, et al. Clinical practice guidelines registry: Toward reducing duplication, improving collaboration, and increasing transparency. Ann Intern Med 2021; 174: 705-7.
8. Dos Santos MBF, Agostini BA, Bassani R, Pereira GKR, Sarkis-Onofre R. Protocol registration improves reporting quality of systematic reviews in dentistry. BMC Med Res Methodol 2020; 20: 57.
10. Xu C, Cheng LL, Liu Y, Jia PL, Gao MY, Zhang C. Protocol registration or development may benefit the design, conduct and reporting of dose-response meta-analysis: Empirical evidence from a literature survey. BMC Med Res Methodol 2019; 19: 78.
