The importance of benchmarking in biomarker discovery and validation

By Avishek Majumder & Priya Chetty on June 7, 2018

Benchmarking is a process of comparing and contrasting best existing methods to new emerging techniques and methodologies. Benchmarking in biomarker discovery is a method of setting a baseline for identification and classification of new protein and molecules profiling technologies. Biomarkers are the best aid in understanding the cause, progression, diagnosis, regression and outcome of treatment of a disease. A number of biomarkers are currently very popular in diagnosis and management of cardiovascular diseases, infections, immunological and genetic disorders as well as in cancer (Perera and Weinstein, 2000). Thus, benchmarking the existing molecular biomarkers help in discovery and development of new biomarkers for diseases. Moreover, to ensure its meaningfulness scientifically and clinically, benchmarking analysis is a crucial step.

Process of biomarker benchmarking

Benchmarking the parameters for biomarker discovery is the absolute need for increasing the uptake of biomarkers in the clinical setting. There requires a consensus among the regulators and researcher to define the benchmark scores, which is still lagging in case of a biomarker. The process of biomarker benchmarking involves three principal phases as shown in the figure below.

Biomarker benchmarking process (McShane, 2017)
Biomarker benchmarking process (McShane, 2017)

The stages are:

  • Analytical validation (Stage 1): addresses the analysis of biomarker evidence based on an analytical performance of an essay.
  • Quantification (Stage 2): where the evidence between the biomarker and diseased condition is assessed.
  • Utilization (Stage 3): lastly, this step occurs which includes a conceptual analysis of the biomarker based on the proposed use and applicability of obtained evidence.

Thus, benchmarking procedure is fundamental since it leads to a targeted therapy by improving the clinical diagnosis. The assessment of biomarkers has also been a serious concern since the beginning of molecular research.

Tools for benchmarking biomarkers

Tools like protein mass spectrometry (MS), are the technology of choice for identifying potentially clinically useful molecular patterns in cancer, heart diseases and other chronic ailments. With enhanced assays and screening techniques the discovery has become easier. However using a varied range of discovery techniques has also created complex bottlenecks that prevent the progression of biomarkers into clinical validation phase (Morrow and de Lemos, 2007). The popular techniques like mass spectrometry, crystallography and immunoassays are widely used in benchmarking biomarkers. In the recent years and near future the same techniques have been or will be replaced by machine learning, data mining and other computational techniques.

Challenges of benchmarking biomarkers

Over the last two decades, significant problems have risen in discovery process of biomarkers. However, some of the major reasons that pose a challenge to biomarker benchmarking are:

  • Relatedness of surrogate endpoint and clinical endpoint: This is defined as a biomarker that is intended to replace a clinical endpoint or abnormality (Ransohoff, 2004). To determine the biomarkers as surrogate endpoints and to predict a relevant clinical outcome is a big challenge. With respect to therapeutic and pathophysiological conditions it is important to maintain consistency and accuracy to epidemiological assessments.
  • Reliability or reproducibility of a biomarker: This is crucial in benchmarking to avoid potentially harmful or worthless tests in clinical usage. This is because if the biomarker is not reliable, this can lead to misclassification of exposure or disease.
  • Assessing clinical validity: This validity is normally expressed in terms of diagnostic accuracy, which measures the degree to which the biomarker can identify diseased patients. Wrongful validation of biomarker is disadvantageous because it is a critical step in clinical integration (Ransohoff, 2004).

Hurdles in biomarker validation

One of the main reasons for biomarker discovery getting stalled at the clinical stages comprises the failure of biomarkers to demonstrate clinically acceptable effect change. Statistically, empowered sample sets help in decent diagnostic biomarker identification. However, studies lead to complications due to the regulatory and ethical issues in conducting trials in human sets. Critical issues for validation of biomarkers comprise the following (Henry and Hayes, 2006):

  • risk categorization
  • screening
  • diagnosis
  • prognosis
  • prediction of therapeutic response and
  • monitoring.

Another obstacle includes the validation of biomarkers which depends on confirmation by different labs in various locations. However, problems exist with consistency in biological sample preparation and different antibodies used for quantification. This eventually leads to wrong conclusions about the validity of biomarkers. Finally, statistical analysis conveniently helps the clinicians and scientists to provide evidence of statistical significance for the usefulness of biomarkers. If an analysis is weak due to human or coding errors in software, it results in drawing wrong conclusions during biomarker validation and qualification.

Clinical aspects of biomarker validation

A potential biomarker candidate is required to demonstrate the association between the clinical endpoint and the pre-treatment samples obtained from patients. The acceptance of clinical biomarkers depends on their accuracy, independence and clinical utility (Bosze et al., 2000). Accuracy determines the strength of prediction of the biomarkers. Independence means when placed in a multivariate model, the biomarker should retain a predictive value when compared against other predictive biomarkers. A clinical utility should address the improvement of patient care and outcomes.

Furthermore, some of the challenges associated with limiting the success rate of prospective markers comprise of:

  • poor study design
  • complicated statistical analysis
  • lack of reproducibility and
  • the inability to translate the bench work into the clinical use.

Thus, the development of clinically relevant diagnostic biomarkers begins with a predefined roadmap. To begin with, biomarker development must comprise of a reproducible assay and should define the biomarker distribution in the assigned population. Subsequently, these results should be compared against the “gold standard” for diagnosis. At this stage, the clinical biomarkers could be classified into three types: risk, diagnosis and prognosis biomarkers.

Biomarker validation and testing

Principal stages in the biomarker validation and testing (Hayes, 2015)
Principal stages in the biomarker validation and testing (Hayes, 2015)

There are three principal stages in validation and testing:

  1. The discovery phase involves high-resolution mass spectroscopy experiments with a small number of samples.
  2. The shortlisted candidate biomarkers are validated in the second stage before proceeding to clinical application.
  3. The clinical stage is the more complicated stage that involves many approvals to confirm the proposed role of the biomarkers in the diseased states.

Benchmarking the parameters for biomarker discovery is the absolute need for increasing the uptake of biomarkers in the clinical setting. Thus, benchmarking helps drive advances in risk, diagnosis, and prognosis of diseases such that they could be powerful molecules in individualizing and optimizing therapy.


  • Bosze, P., et al. 2000. Conseusus statements on prognostic factors in epithelial ovarian carcinoma. Report of the Consensus Meeting organized by the European Society of Gynaecological Oncology, ESGO. Eur J Gynaecol Oncol, 21, 513-26.
  • Hayes, D. F. 2015. Biomarker validation and testing. Mol Oncol, 9, 960-6.
  • Henry, N. L. & Hayes, D. F. 2006. Uses and abuses of tumor markers in the diagnosis, monitoring, and treatment of primary and metastatic breast cancer. Oncologist, 11, 541-52.
  • McShane, L. M. 2017. In Pursuit of Greater Reproducibility and Credibility of Early Clinical Biomarker Research. Clin Transl Sci, 10, 58-60.
  • Pererra, F. P. & Weinstein, I. B. 2000. Molecular epidemiology: recent advances and future directions. Carcinogenesis, 21, 517-24.
  • Ransohoff, D. F. 2004. Rules of evidence for cancer molecular-marker discovery and validation. Nat Rev Cancer, 4, 309-14.

Priya is the co-founder and Managing Partner of Project Guru, a research and analytics firm based in Gurgaon. She is responsible for the human resource planning and operations functions. Her expertise in analytics has been used in a number of service-based industries like education and financial services.

Her foundational educational is from St. Xaviers High School (Mumbai). She also holds MBA degree in Marketing and Finance from the Indian Institute of Planning and Management, Delhi (2008).

Some of the notable projects she has worked on include:

  • Using systems thinking to improve sustainability in operations: A study carried out in Malaysia in partnership with Universiti Kuala Lumpur.
  • Assessing customer satisfaction with in-house doctors of Jiva Ayurveda (a project executed for the company)
  • Predicting the potential impact of green hydrogen microgirds (A project executed for the Government of South Africa)

She is a key contributor to the in-house research platform Knowledge Tank.

She currently holds over 300 citations from her contributions to the platform.

She has also been a guest speaker at various institutes such as JIMS (Delhi), BPIT (Delhi), and SVU (Tirupati).