Introduction
Chronic hepatitis B virus (HBV) infection (CHB) remains a global health challenge, with an estimated 240 to 400 million infected individuals worldwide.1–3 HBV nucleos(t)ide analogues (NUCs) introduced over the past two decades can help reduce this burden of disease; suppression of viral replication with HBV NUCs for 3-5 years can reverse HBV-associated liver fibrosis and reduce the incidence of hepatocellular carcinoma (HCC).4–6
Entecavir (ETV) is a third-generation NUC approved for treating hepatitis B e-antigen (HBeAg)-positive, HBeAg-negative, and lamivudine-experienced adults with CHB, based on phase 3 results demonstrating histologic, virologic, and biochemical benefits.7–9 Virologic breakthrough was rare after 5 years of ETV treatment in NUC-naive patients; consistent with this durable antiviral effect, HBV disease progression was reduced.10–12 ETV safety was favorable in randomized trials and long-term follow-up studies; no association between ETV and risk of specific adverse events was identified with therapy of up to 5 years.13 However, in 2-year preclinical carcinogenicity studies, benign and malignant tumors involving lung, liver, and brain were observed in ETV-exposed mice and rats.14 Excepting lung tumors, which were limited to male mice, rodent tumors occurred only at significantly higher ETV exposures than those occurring with approved doses in humans.
These findings prompted initiation of the REALM study ((Randomized, Observational Study of Entecavir to Assess Long-Term Outcomes Associated with Nucleoside/Nucleotide Monotherapy for Patients with Chronic HBV Infection; NCT00388674) of HBV-associated and non-HBV-associated clinical outcomes in adults with CHB who received ETV or non-ETV HBV NUCs for up to 10 years.15,16 Planned enrollment was 12,500 patients in up to 500 research centers globally. To ensure data quality and consistency from this large and geographically diverse investigator group during the extended follow-up, an independent event adjudication committee (EAC) was established to review investigator-reported clinical outcome events (COEs). EACs have proven valuable in randomized trials in other fields, particularly those with substantial risk of misclassifying events that are crucial for study outcome assessment.17–21 However, independent EACs have seldom been used in studies of liver disease; to our knowledge, REALM is the first HBV therapeutic study to utilize an EAC for evaluating COEs. Herein, we report the structure and processes utilized by the REALM EAC, the COEs submitted for EAC review, outcomes of their assessments, and key lessons learned.
Methods
Study design
Eligible patients were adults with HBeAg-positive or -negative CHB, who, in their physician’s opinion, were eligible for monotherapy with an approved HBV NUC. Patients could be HBV treatment-naive or experienced, with or without compensated or decompensated cirrhosis, or coinfected with hepatitis C and/or hepatitis delta viruses. Patients were ineligible if coinfected with human immunodeficiency virus, had expected liver transplant-free survival of less than 1 year, a history of malignant neoplasm or dysplastic liver nodule, prior ETV use, or intention to receive interferon monotherapy.
Patients were randomly assigned 1:1 to receive ETV or other standard-of-care HBV NUC that was selected at each investigator’s discretion and dosed per the product label. Concurrent interferon-alfa-2b or pegylated interferon-alfa-2a treatment was allowed. After receiving the first dose of study therapy, patients’ HBV treatment regimens could be modified by switching to or adding alternate HBV NUC(s) or by terminating treatment altogether. Patient observation continued regardless of such modifications.
Data collection
After randomization, patients were followed for up to 10 years after enrollment of the first patient. With full enrollment anticipated to take 3 years, individual patients were expected to receive 7 to 10 years of follow-up. Participating sites monitored for and reported treatment-related serious adverse events (SAEs) and COEs; COEs included malignant neoplasms (non-HCC, HCC), deaths, and non-HCC liver-related manifestations of HBV disease progression, which included development or progression of compensated or decompensated cirrhosis. Patients were assessed for COEs through twice-yearly in-person visits and two interim telephone interviews. Although the specific means for evaluating patients was at the investigators’ discretion, site guidance was provided regarding COEs to be monitored and diagnostic criteria.
EAC structure and review process
Due to the complexity of COE definitions and the unblinded study design, an EAC comprising two subcommittees (hepatology, oncology) was established to adjudicate all investigator-reported COEs per criteria outlined in the EAC charter (Supplemental Appendix). The EAC was co-chaired by a hepatologist and an oncologist and composed of experts in those fields; each member also served on the EAC subcommittee relevant to their clinical expertise. The hepatology subcommittee reviewed non-HCC liver-related events of HBV disease progression and liver-related deaths; the oncology subcommittee reviewed events of non-HCC malignant neoplasms and malignancy-related deaths. The subcommittees shared responsibility for reviewing new reports of HCC, dysplastic liver nodules, and non-liver/non-malignancy-related deaths. EAC members and the sponsor remained blinded to HBV therapies received throughout the study. Full committee meetings were initially held quarterly; however, due to increasing case volume, beginning in 2009, meetings were held every other month. Full committee meetings were teleconferences, except for one face-to-face meeting annually, which typically occurred before the American Association for the Study of Liver Diseases (commonly known as AASLD) or European Association for Study of the Liver (commonly known as EASL) meeting.
Each reported COE was reviewed independently by two subcommittee members, who recommended an adjudicated diagnosis (Fig. 1). With concordant reviews, the relevant Chair completed an adjudication case report form (CRF) which certified the adjudicated diagnosis and date of diagnosis, after which these data were entered into the clinical database. Discordant reviews prompted full committee review; the final decision was based on majority votes. The sponsor’s Global Pharmacovigilance Group reviewed treatment-related SAE reports. If SAE details were consistent with a study-defined COE, the investigator was asked to re-report the event as a COE. If an event was reported as both an SAE and a COE, and subsequently confirmed to meet COE criteria, the SAE report was withdrawn.
Investigators reported events to the sponsor and/or a contracted clinical research organization (CRO) on standardized COE CRFs in the form of diagnostic questionnaires and COE workbooks. These documents formed the core contents of a case packet created for each reported COE and submitted for EAC review. The diagnostic questionnaire contained the reported COE and date of diagnosis; COE workbooks contained the investigator-reported event term, the primary evaluation method supporting the diagnosis (e.g. histology, imaging, laboratory assessments) and test date, and any secondary diagnostic information.
Following sponsor/CRO review of submitted CRFs, sites were contacted as necessary using standardized data request forms (DRFs) for further case information and source documents required for EAC review and adjudication. Specific DRFs were developed for each type of COE; DRFs requested case information, such as liver imaging or biopsy data for cirrhosis, ascitic fluid neutrophil counts for spontaneous bacterial peritonitis, or upper endoscopy results for variceal bleeding. The CRO provided an important link between investigators and the EAC for acquisition of event documentation.
Additional COE capture methods for EAC adjudication
During the study, at least annually, the EAC reviewed evolving literature addressing HBV and HCC outcomes, including society practice guidelines, to assess the need to modify EAC charter diagnostic criteria. The charter was amended twice during the study, in October 2009 and May 2014. The first amendment added liver stiffness measurement by transient elastography (FibroScanÒ) as a supportive finding for diagnosis of cirrhosis. The second amendment allowed definitive diagnosis of cirrhosis based on FibroScanÒ findings alone, and HCC diagnosis based on detection of a characteristic focal lesion (arterial phase enhancement with venous phase washout) by a single contrast-enhanced, cross-sectional imaging procedure. Additional steps undertaken to maximize COE data capture are described below.
HCC events: Following the 2014 EAC charter amendment that resulted from updated AASLD practice guidelines for HCC,22,23 the EAC reassessed investigator-reported HCC events that were assessed previously as “unable to adjudicate” using the original charter criteria. Reporting investigative sites were requested to provide any new relevant data, and the full EAC assessed whether the revised diagnostic criteria could now support an adjudicated diagnosis of HCC. Transarterial chemoembolism reports were also reviewed for relevant angiography findings, and dysplastic liver nodule reports were added to case packets of subsequently reported HCC events to support HCC diagnosis and timing.
Non-HCC HBV disease progression events: Initially, diagnosis of cirrhosis was based on liver histology and/or ultrasound. During the study, use of ultrasound-based elastography to assess liver fibrosis and cirrhosis became common practice. The EAC charter was modified twice, adding ultrasound-based elastography data as probable and definitive criteria for diagnosis of cirrhosis. However, the EAC did not retrospectively re-evaluate cirrhosis events that were previously assessed as “unable to adjudicate” because the technology was not used consistently before the charter amendment.
Deaths: The EAC assigned a primary cause of death to events with more than one investigator-reported cause and, to the extent permitted by supporting data, a primary cause of death to events reported with an “unknown” cause. Most death reports were submitted with source documents, e.g. hospital discharge summaries and/or death certificates, allowing the EAC to affirm the death and assess the primary cause. However, some deaths were reported only with the death CRF and lacked source documents. Most such cases were reported by Asian sites, and the absence of source documents was often related to the death occurring at an outlying medical facility, where the investigator lacked attending privileges. In such cases, investigators were asked to verify the death in writing and, where possible, provide the most likely cause.
Non-HCC malignancy events: Some events of non-HCC malignancies were reported with general cancer diagnoses, even when supporting pathology reports allowed greater specificity regarding tumor type. To enhance the accuracy of data capture and analyses for these events, the EAC oncology chair and subcommittee members were trained on MedDRA coding of malignant and premalignant tumors. Heightened attention to investigator-reported diagnoses versus those on pathology reports and other source documents was reinforced. Secondarily, the sponsor independently reviewed investigator reports and adjudicated tumor diagnoses prior to addition to the study database. The EAC oncology chair was asked to reassess cases of concern, particularly histology data, and amend adjudicated diagnoses where appropriate.
Results
Patient disposition
Among the 12,485 randomized patients, 6216 and 6162 initiated treatment with ETV or a non-ETV nuc, respectively (Fig. 2).15 Patients continued their randomly assigned treatment prior to switching to an alternate HBV treatment for a median 98.7 months (range: 0-114.4) with ETV or 94.0 months (range: 0-113.1) with a non-ETV NUC; median cumulative duration of either HBV therapy was 99.1 and 96.7 months, respectively. The results of the REALM study have been published elsewhere, and revealed no significant differences in clinical endpoints between ETV and non-ETV nucleos(t)ide analogue treatment, and virologic response independent of treatment group was associated with a reduced risk of liver-related HBV disease progression (hazard ratio (HR): 0.09, 95% confidence interval (CI): 0.038-0.221) and HCC (HR: 0.03, 95% CI: 0.009-0.113).24
COE outcomes
EAC-reviewed and -adjudicated COEs of liver-related HBV disease progression (HCC, non-HCC HBV disease progression, or liver-related death) were reported in 442 patients (7.1%) in the ETV group and 464 patients (7.5%) in the non-ETV group (Table 1).
Table 1.Summary of EAC-reviewed COEs
| EAC-reviewed COEs, number of patients | ETV, N=6216 | Non-ETV, N=6162 |
| Deaths | 240 | 264 |
| Liver-related | 46 | 48 |
| HCC | 290 | 316 |
| Non-HCC malignant neoplasms | 109 | 91 |
| Non-HCC HBV disease progression | 202 | 202 |
Overall, 504 patients died, 502 of whom received an EAC-adjudicated diagnosis regarding cause of death; two events were assessed as unable to adjudicate due to lack of data verifying the death. Among these 502 patients, 328 (65%) were adjudicated with cause of death as reported by the investigator, and the remaining 174 (35%) were determined to have an alternate cause of death (Table 2). The most common causes of death in the ETV and non-ETV groups, respectively, were HCC (n=43 and 69), liver-related conditions (n=46 and 48), and non-HCC malignancies (n=17 and 15). Among the 179 adjudicated deaths with discrepant diagnoses between the reporting investigator and the EAC, most often the cause of death was reported as being due to a specific event such as HCC (n=66), a liver-related condition (n=30) or a malignancy (n=12) but was adjudicated as unknown due to inadequate evidence. The EAC adjudicated a specific cause of death for 6 patients with a reported unknown cause.
Table 2.Summary of COE reviews by the EAC in the REALM study
| Patients with events, n (%) | ETV, N=6216 | Non-ETV, N=6162 |
| Reported deaths | 240 (4) | 264 (4) |
| EAC reviewed | 240 (4) | 264 (4) |
| Adjudicated as death | 238 (4) | 264 (4) |
| Adjudicated as reported | 156 (3) | 169 (3) |
| Adjudicated as reported with alternative diagnosis date | 0 | 3 (<0.1) |
| Adjudicated with alternate diagnosis | 82 (1) | 92 (1) |
| Unable to adjudicate | 2 (<1) | 0 |
| Reported HCC events | 289 (5) | 316 (5) |
| EAC reviewed | 290 (5) | 316 (5) |
| Adjudicated as HCC | 241 (4) | 263 (4) |
| Adjudicated as reported | 122 (2) | 146 (2) |
| Adjudicated as reported with alternative diagnosis date | 119 (2) | 117 (2) |
| Adjudicated with alternate diagnosis | 1 (<1) | |
| Unable to adjudicate | 51 (1) | 53 (1) |
| Pre-existing events | 0 | 1 (<1) |
| Reported non-HCC malignant neoplasm events | 109 (2) | 91 (1) |
| EAC reviewed | 109 (2) | 91 (1) |
| Adjudicated as non-HCC malignant neoplasm | 95 (2) | 81 (1) |
| Adjudicated as reported | 35 (1) | 24 (<1) |
| Adjudicated as reported with alternative diagnosis date | 48 (1) | 38 (1) |
| Adjudicated with alternative diagnosis | 12 (<1) | 19 (<1) |
| Adjudicated as alternate event | 1 (<1) | 0 |
| Unable to adjudicate | 13 (<1) | 10 (<1) |
| Reported non-HCC events of HBV disease progression | 208 (3) | 207 (3) |
| EAC reviewed | 202 (3) | 202 (3) |
| Adjudicated as non-HCC events of HBV disease progression | 137 (2) | 146 (2) |
| Adjudicated as reported | 87 (1) | 88 (1) |
| Adjudicated as reported with alternative diagnosis date | 34 (1) | 44 (1) |
| Adjudicated as reported with alternative disease progression manifestation | 16 (<1) | 12 (<1) |
| Adjudicated with alternate diagnosis | 0 | 3 (<1) |
| Pre-existing events | 21 (<1) | 8 (<1) |
| Unable to adjudicate | 44 (1) | 48 (1) |
Among the 606 EAC-reviewed HCC events, 504 were adjudicated as HCC. One event was adjudicated as pre-existing; none were assessed with an alternate diagnosis. The EAC could not adjudicate 104 HCC events, predominantly due to inadequate case documentation. The EAC pursued various avenues to limit the number of HCC events assessed as unable to adjudicate. All HCC reports reviewed by the EAC after the 2014 charter amendment were assessed using the revised criteria for HCC diagnosis. In addition, using the revised diagnostic criteria, the EAC reassessed 31 previous HCC events that could not be adjudicated because only a single imaging study had been conducted; four (13%) were reassessed as meeting criteria for HCC.
The EAC reviewed 404 investigator-reported events of non-HCC HBV disease progression, 202 in each treatment group. Among these, 283/404 were adjudicated as a new event of disease progression, either as reported (n=175), as reported with a modified date of diagnosis (n=78), or with the disease progression manifestation changed (n=28). Of the remaining 124/404 events, 29 were adjudicated as pre-existing (most commonly events of cirrhosis or ascites), 3 received an alternate diagnosis, and 92 were assessed as unable to adjudicate due to inability to satisfy EAC diagnostic criteria.
The EAC reviewed 200 events of non-HCC malignancy. Among these, 145 were adjudicated as reported, or as reported but with an alternate diagnosis date. Thirty-one were adjudicated with an alternate diagnosis of non-HCC malignancy, providing greater specificity regarding tumor type. One was adjudicated as an alternate event (endometrial hyperplasia), and the remaining twenty-three were assessed as unable to adjudicate due to inability to satisfy EAC diagnostic criteria. All reported non-HCC malignancies assessed as unable to adjudicate lacked histologic data or other means for confirming the reported diagnosis.
Concordance between investigator and EAC assessments of HBV-associated COEs was high overall (85.0%; 1465/1724 COEs). However, concordance varied substantially by COE type. Concordance was highest for deaths (99.6% agreement; 502/504 COEs), followed by non-HCC malignancies (88.0% agreement; 176/200 COEs), HCC (83.3% agreement; 504/605 COEs), and non-HCC HBV disease progression (68.2% agreement; 283/415 COEs). In addition, EAC adjudication resulted in modification of the COE (e.g., hepatic decompensation), COE manifestation (e.g., ascites), and/or COE diagnosis date in 26.7% cases (461/1724 COEs).
Discussion
This is the first prospective observational cohort study of CHB therapy that has employed an EAC to affirm study endpoints. The standardized adjudication process was custom-tailored to meet the challenging aspects of study design. These include the diverse clinical outcomes assessed, the global scope that involved 24 countries and 299 sites in the Americas, Europe and Asia-Pacific, a large cohort of approximately 12,500 patients, and a long (10-year) study duration.
The sponsor and CRO worked closely with the EAC to establish operational and administrative processes in accordance with the EAC charter, and to ensure that the EAC was provided with the means to perform its role efficiently. Despite the long study duration, attrition of EAC membership during the study was very limited.
The EAC reviewed and adjudicated cases independently of the sponsor and remained blinded to treatment-specific results throughout the study. The sponsor performed annual data assessments to share interim results of adjudicated COEs with the EAC, the data monitoring committee, and regulatory authorities; however, these interim assessments were based on pooled treatment results and study blinding was maintained.
EAC participation in clinical studies is well established in other fields, particularly cardiovascular medicine,17–21 to promote consistency, objectivity, accuracy, and reliability of study outcomes. This is accomplished through standardized application of prespecified clinical event definitions that reflect best evidence from published literature and consensus among expert physicians, medical societies, and regulators. Event adjudication may also contribute to patient protection through collection of safety data used by oversight committees and regulatory agencies in real-time clinical event monitoring.
However, event adjudication is resource-intensive, time-consuming, and potentially subject to bias. To promote consistency and accuracy of event adjudication, the REALM EAC applied several strategies representing best practice that may support future observational cohort studies of CHB infection. The most pertinent challenges faced in the adjudication experience, the measures employed to address these challenges, and lessons learned are summarized below.
Optimizing efficiency
As the study progressed, substantial differences between sites were observed in the quality and availability of primary data needed for COE adjudication. To address this issue, standardized DRFs for each COE category and category subtype were used to obtain additional data deemed critical for EAC adjudication. This element of the adjudication process enhanced the quality and efficiency of EAC case review and limited the need for additional site queries. Guidance in the application of COE criteria and event reporting were made available to investigators, and study coordinators made more frequent visits to sites where data collection was problematic.
The large volume of study data presented logistical challenges associated with acquisition and distribution of documents for EAC review, assessment, and documentation. These challenges were addressed through the rigorous, step-wise EAC process established for COE reporting, source document submissions, data querying and retrieval of responses, and use of adjudication CRFs specific to each COE type. A secure web-based portal provided efficient storage and recall of COE documentation. Through the portal, EAC members accessed event-related documents and COE case packets for review, and EAC chairs filed completed adjudication CRFs.
Adjudication efficiency was enhanced by periodic assessment to ensure adequate EAC membership and appropriate distribution of case assignments across the EAC, including hepatology and oncology experts. Although oncologists were initially responsible primarily for reviewing reported non-HCC malignancies and malignancy-related deaths, as the case load increased, the oncologists also assumed shared responsibility with hepatologists for reviewing HCC events.
Assignment of adjudicated diagnoses
Event adjudication entailed determining the diagnosis and the date of diagnosis of investigator-reported events, and assessment of whether reported cases met EAC charter-defined COE criteria. Investigator-reported dates of diagnosis were verified by primary review of radiographic, laboratory, and other source documents from the REALM database.
Due to international enrollment (299 site investigators, 24 countries, 4 continents), the committee faced challenges in resolving significant geographic variability in language, clinical practice standards, and definitions of liver-related events. The rigorous assembly of COE-related information, using standardized DRFs for COE category and type, and adherence to EAC charter-defined criteria for COEs facilitated report consistency. Furthermore, international representation in the composition of the EAC helped promote consistency in event adjudication across global regions.
Event capture and prespecified data analyses
Data analyses rely heavily on the quality and consistency of source data. In this study, primary and key secondary endpoint analyses were based on adjudicated COE data; hence, minimizing the number of unadjudicated events was critical. After initial establishment of diagnostic criteria, adjustments were implemented to address subsequent changes that emerged in clinical practice guidelines, diagnostic testing, and treatment over the 10-year study period. Ultimately, the EAC charter was amended only twice, to incorporate new guideline criteria for HCC diagnosis and add ultrasound-based elastography to diagnostic criteria for cirrhosis. EAC reassessment of cases following charter updates ensured uniform application of charter criteria across time.
Adjudicating cause of death was the situation that most frequently required full-committee discussion to resolve discordant reviews. Inadequate hospital documentation was a frequent source of uncertainty, particularly in regions where death certificates were not regularly available. In addition, in patients with cirrhosis and HCC, distinguishing liver-related death from HCC-related death required careful evaluation of primary documents to assess primary contributing factors to fatality. For example, a fatal variceal hemorrhage associated with portal hypertension, although generally liver-related, could be adjudicated as HCC-related if malignant invasion of the portal vein was documented. Furthermore, the EAC would request primary medical documents for subjects preceding study enrollment to distinguish clinical events as pre-existing, recurrent, or new.
As described in Methods, a secondary, independent review process was implemented to ensure that adjudicated diagnoses of non-HCC malignancies were as specific as possible, based on supporting pathology data. Of the 176 reported non-HCC malignancies that were adjudicated as a non-HCC malignancy (ETV: n=95; non-ETV: n=81), 31 of these events (ETV: n=12; non-ETV: n=19) were assigned an alternate tumor diagnosis by the EAC. In all cases, this was based on selection by the reviewing EAC oncologist of a more specific tumor diagnosis regarding the organ site and/or tissue diagnosis.
Maintaining consistency in COE adjudication over the duration of the study was essential. Monthly teleconferences and annual face-to-face meetings supported regular engagement of EAC members and minimized turnover. The teleconferences provided a live record of adjudication decisions and approach, and helped to promote homogeneity of the adjudication process over time. The EAC charter, adjudication process, and committee performance was reviewed annually.
The relatively frequent modifications of some categories of investigator-reported COEs demonstrate the impact of external adjudication on endpoint assessment in observational cohort studies. Most differences between investigator and EAC assessments of death were related to inadequate documentation to support investigator conclusions regarding cause of death. In contrast, discrepancies regarding HCC, non-HCC malignant neoplasms, and non-HCC events of HBV disease progression were more commonly related to pre-existing diagnoses, alternative dates of diagnosis, unacceptable diagnostic methods, or alternative disease progression manifestations.
In summary, if supported by an effective administrative infrastructure, EACs can enhance the consistency and validity of clinical outcome assessment and strengthen regulatory review of therapeutic interventions in liver disease. The REALM study represents possibly the first application of EAC adjudication in HBV therapeutic research, and is unique due to its large cohort, geographic scope, outcome assessments bridging hepatology and oncology, and long-term follow-up over 10 years. The challenges and accomplishments of this EAC may provide useful insights for future cohort studies in liver disease.