Small Compound Screening Overview
Introduction
This report is designed to be an introduction to small compound high throughput screening (HTS) at the Target Discovery Institute, Nuffield Department of Medicine, University of Oxford. The TDI benefits from a broad range of experience across a wide range of screens and our scientists have extensive knowledge of high throughput screening. We are available for consultation at the earliest stages of assay conception and we will work closely with each investigator through all stages of assay development, optimization, automation and screening.
HTS Transfer Protocols
All assays being transferred to the Janus for HTS will be subjected to a thorough validation process before the production screening beings.
This validation process will consist of:
- Initial Consultation
- Stability and Process study
- Liquid Handling Validation
- Plate Uniformity Assessment
- Control Validation, Z’calculation, Assessment of edge effects and drift within the body of the screen
- Replicate Experiment
- We require as a minimum a 2 replicate study over 2 different days for biological reproducibility and robustness
- Pilot Screen
- a small number of plates containing compounds of varied pharmacologically activity and the chosen controls
- Production Runs
1. Initial Consultation
The initial consultation with the HTS facility manager is an opportunity to discuss your ideas for a potential screen during the conceptual stage or as you begin the bench-top development.
We have experience across a diverse range of screening platforms and formats and have run a wide variety of screens in our facility. We are able to advise you on what might be the most effective approach (both in terms of time and cost) for the question you would like to investigate.
We have also developed a number of novel screening approaches designed to answer specific criteria and we welcome the opportunity to develop additional novel screens. We would encourage you to discuss your proposed assay at the earliest stages of development as we can assist you with the following issues:
- There are several types and many approaches to HTS. Each type of screen has merits and demerits (i.e. - easy to set up but requires more consumables), we will discuss these in full to determine the most appropriate screening regime for you.
- Your choice of reagents - Some reagents have limited stability, may be very expensive compared to alternatives, susceptible to pH or temperature changes, etc. make them unsuitable for HTS.
- Specific Phenotype Optimization - Optimizing the phenotype under investigation is the single most crucial step to successful screening. We can help you devise a specific protocol to target the best cell density, cell cycle stage, target gene knock-down, minimise cytotoxicity and optimise the most appropriate assay reagents
- We can begin to discuss logistics and timings of your assay as well as the projected cost.
An initial consultation at the earliest stage of assay development will ensure a smooth transition from bench-top to HTS.
2. Stability and Process
The stability and process study is the initial step of the HTS assay validation. It will be necessary to examine all reagents used in the assay for the following properties:
- Stability during production.
- Storage stability - for any un-used reagents you may wish to store for future assays.
- Emergency stability - the stability of assay reagents at room temperature in the unlikely event of a robotics failure
- Time-course experiments to determine the range of acceptable times for each reagent in process. This is of particular importance for cell health which may be subject to changes over time as plates are seeded throughout the course of the experiment.
This study may be run concurrently with the final steps of the liquid handling validation process and will be the responsibility of the assay developer.
3. Liquid Handling Validation
Once your proposed screen assay has met the criteria for moving from the bench-top to the Janus Liquid Handling Work Station, the programming of all assay steps on the liquid handling robotics will become the responsibility of the high throughput developer (TDI operator).
- Initially, we will meet to thoroughly discuss the processing steps in the protocol. Any questions or clarification should be addressed here so that adjustments may be made to the protocol at the earliest opportunity.
- Within a couple of days a written time-line for the development of the screen will be produced and must be agreed by the assay scientist(s) and ourselves.
- Once the High Throughput liquid handling protocol writing is complete, we will validate all steps of the protocol together using coloured dyes to track all liquid handling steps. You will thoroughly evaluate the protocol at this point. There will invariably be alterations from your protocol and any variations will be approved by you.
4. Plate Uniformity Assessment
Once your proposed screen assay has passed the Liquid Handling Validation step, a Plate Uniformity Assessment will be conducted (Detailed below in this report). This is a series of experiments designed to address:
- Drift – this usually seen as a left right shift across the body of the screen.
- Edge Effect – this is usually seen along the perimeters of the plate. As a standard approach in a 384 well plate we leave the outer row and columns empty of compounds. For 96 well plates, edge effects are generally minimised by a larger volume in each well.
* In general, Drift or Edge Effects <20% are considered acceptable and random effects seen on <20% are also considered acceptable. If HTS protocol adjustments are necessary to address unacceptable Drift or Edge Effects, they will be made by the HTS developer and approved by you.
5. Replicate Experiment - Final Assay Validation
This is a set of experiments designed to validate the HTS procedure immediately before the Production runs. Along with the Plate Uniformity Assessment, the Replicate Experiment will ensure:
- A set of experiments, a "dry run", using all components of the assay will be performed. We will seed plates and carry out all steps of the assay to produce a Z' Factor [1], to determine if the screen produces data with sufficient sensitivity and specificity to accurately differentiate between positive "hits" and negative samples.
- Once an acceptable Z’ has been found (this is usually in excess of 0.3 for cell based HTS) a minimum of two concentration pilot studies will be used to validate the procedure using Janus Liquid Handling robots (detailed below in this report).
- Inter- and Intra-Plate Variability will be assessed using a standard coefficient of variance calculation (CV) for the body of the screen - an acceptable CV range will fall within 10%.
- All Plate and Assay controls will be assessed.
If all aspects of the HTS assay are within acceptable range for the above criteria, we will proceed to the Production run.
6. Production Run
- Run the HTS assay. This is where your HTS assay produces reams of data to be written up in high impact scientific journals.
- Post-Production evaluation and report. See appendix 1 for HTS Final Report layout.
Types of Small Compound High Throughput Screens
High throughput screening assays can be divided into two main types:
1. Protein based biochemical screens - These screens use purified proteins, substrates and small compound inhibitors in buffered solutions to produce an optical (fluorescent, luminescent, etc.) readout that monitors enzymatic or binding activity using high throughput plate-readers. Small compound inhibitors are ranked on their ability to reduce the protein function, and by extension the optical signal, in this type of screen.
Advantage: Very high throughput, small volume reactions reduce reagent costs and simple readouts. Target of the inhibitor is defined.
Disadvantage: Small compound may not be water soluble, membrane permeable and may be promiscuous or toxic. Single protein inhibition may not cause desired phenotypic change in cells or tissues due to redundant pathways.
2. Cell based screens- These small compound screens utilize cells plated in 96 or 384 well plates to produce a visual phenotypic change in the cells which can be quanitified. The three general measurement types (although others exist) of a cell based screen are:
Uniform well readouts - These include cell viability assays such as Resazurin turn over and may include simple reporter gene assays. These assays usually employ high throughput platereaders for measurement nd are generally luminescence, absorbance or fluorescence based.
High-Content Imaging Screens - These screens are designed to probe changes to a cellular phenotype (i.e. foci formation screens, nuclear and cellular morphology, localization of proteins, etc). HCI screens employ specialized high content imagers to produce high content images which can be used to measure phenotypic changes. These assays are content (and data) rich and can provide in-depth analysis of cellular changes. In addition, they can also be "multi-plexed" with multiple probes for DNA, membrane and specific protein staining to provide large-scale information on several changes to a cellular phenotype. They can also provide cell cycle information over the course of the assay.
The high-content images generated for these types of screen are held indefinitely and may be re-interrogated at a later stage for more in-depth analysis as information becomes available. Plates which contain fixed cells are also stored where possible in case plates are required for further staining and analysis.
Reporter gene systems - These may take the form of high throughput FACS based assays producing readouts of GFP, luciferase, etc. internalized signal; or a more simplified high throughput approach utilising plate-readers for measurement generally in the form of luminescence, absorbance or fluorescent signal
Advantages: Cell based assays produce phenotypic changes affecting pathways directly associated with disease states in which the target protein or pathway may not beknown. Small compound cell based screens ensure compound solubility, membrane permeability, non-toxic and effectiveness at low, therapeutically relevant concentrations.
Disadvantages: Lower throughput than protein based assays, much more technically difficult and much longer duration (days/weeks vs. minutes). Target of inhibitors are not definitively known.
Criteria for Developing Robust Small Compound HTS
The HTS conditions should be optimized to satisfy several criteria:
- Since small compound screens are dependent on efficient inhibition (IC50/90) or excitation (EC50/9) optimization of compound concentration is crucial to the success of HTS. See the Two Step Janus Transfect ion Protocol below for information on how we optimize conditions in TDI.
- Use only healthy and robust cells - self explanatory. All cell lines must be proven to be Mycoplasma free at the time of screening. In general we ask that all cells entering the TDI be certified mycoplasma free prior to entering our incubators.
- A suitable negative control must be included on all plates. In the case of small compound screening this in most cases will be a concentration of DMSO equal to that found in the compound wells. This gives an accurate picture of potential toxicity issues for DMSO.
- The inhibition/excitation should produce a robust phenotypic change with a measurable Z' Factor (usually above 0.3) for the control samples. No screen will be progress to production without a robust and reproducible dynamic range (see below).
- Cast a wide net during the primary screen - establish the most efficient concentration conditions (we recommend screening at 10mM and 2mM for the primary screen, this ensures we are screening well above the likely IC/EC90. The goal of the primary screen is maximal sensitivity. At this stage, false positives (FP’s) are much better than false negatives (FN’s)
To help ensure all criteria are satisfied during the development of an HTS and before production runs commence, we employ a two step assay validation protocol using the PerkinElmer Janus liquid handling workstation. The assay validation protocols are detailed below.
Plate Layout and Controls for Small Compound Screens
A number of different controls (both positive and negative) are necessary to obtain meaningful and reliable results from small compound screening.
They are summarized below:
Positive Controls
PC: This is the positive control for your assay. The PC should induce your screening phenotype and CAN BE used in the statistical analysis of your data for evaluating hits. The PC should target known genes in your pathway and it is very important to test several potential PC to find one that produces the desired phenotypic change at the levels you require.
Negative Controls
NC - This negative control and is usually DMSO at a concentration mirroring that found in the compound wells. For robust assays this is usually between 0.5 and 2%. Above this cells begin to show some toxic effects. The NC serves as the baseline for effects of solvent alone on cells.
EC50/IC50 and EC90/IC90 Controls
EC50/IC50 and EC90/IC90 Controls – these are added to plates where specific phenotypes are being identified. In most cases these are not used in statistical analysis, but are used as reference wells to compare the body of the data against.
Typical plate layout for small compound screens at TDI
The primary screens in a 96-well and 384-well format should have 16-24 negative wells per plate for the negative control to be used as the negative reference for selecting strong to moderate strength phenotypic assay response. [2]Both the False Negative Rate (FNR) and the False Positive Rate (FPR) are lowest at 24 wells. However, a good trade-off between cost and benefit can be found at sample sizes between 4-11 samples. Since the aim of many primary screens is to capture c0ompounds with strong to moderate strength, sample negative control size employed by our primary screens will be between 8-11 samples.
96 Well Plate Format
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
|
A |
+ve |
Comp 1 |
Comp 9 |
Comp 17 |
Comp 25 |
Comp 33 |
Comp 41 |
Comp 49 |
Comp 57 |
Comp 65 |
Comp 73 |
+ve |
B |
+ve |
Comp 2 |
Comp 10 |
Comp 18 |
Comp 26 |
Comp 34 |
Comp 42 |
Comp 50 |
Comp 58 |
Comp 66 |
Comp 74 |
+ve |
C |
EC/IC50 |
Comp 3 |
Comp 11 |
Comp 19 |
Comp 27 |
Comp 35 |
Comp 43 |
Comp 51 |
Comp 59 |
Comp 67 |
Comp 75 |
EC/IC50 |
D |
EC/IC50 |
Comp 4 |
Comp 12 |
Comp 20 |
Comp 28 |
Comp 36 |
Comp 44 |
Comp 52 |
Comp 60 |
Comp 68 |
Comp 76 |
EC/IC50 |
E |
EC/IC90 |
Comp 5 |
Comp 13 |
Comp 21 |
Comp 29 |
Comp 37 |
Comp 45 |
Comp 53 |
Comp 61 |
Comp 69 |
Comp 77 |
EC/IC90 |
F |
EC/IC90 |
Comp 6 |
Comp 14 |
Comp 22 |
Comp 30 |
Comp 38 |
Comp 46 |
Comp 54 |
Comp 62 |
Comp 70 |
Comp 78 |
EC/IC90 |
G |
-ve |
Comp 7 |
Comp 15 |
Comp 23 |
Comp 31 |
Comp 39 |
Comp 47 |
Comp 55 |
Comp 63 |
Comp 71 |
Comp 79 |
-ve |
H |
-ve |
Comp 8 |
Comp 16 |
Comp 24 |
Comp 32 |
Comp 40 |
Comp 48 |
Comp 56 |
Comp 64 |
Comp 72 |
Comp 80 |
-ve |
384 Well Plate Format
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
13 |
14 |
15 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
|
A |
|
+ve |
Comp 1 |
Comp 17 |
Comp 33 |
Comp 49 |
Comp 65 |
Comp 81 |
Comp 97 |
Comp 113 |
Comp 129 |
Comp 145 |
Comp 161 |
Comp 177 |
Comp 193 |
Comp 209 |
Comp 225 |
Comp 241 |
Comp 257 |
Comp 273 |
Comp 289 |
Comp 305 |
+ve |
|
B |
|
+ve |
Comp 2 |
Comp 18 |
Comp 34 |
Comp 50 |
Comp 66 |
Comp 82 |
Comp 98 |
Comp 114 |
Comp 130 |
Comp 146 |
Comp 162 |
Comp 178 |
Comp 194 |
Comp 210 |
Comp 226 |
Comp 242 |
Comp 258 |
Comp 274 |
Comp 290 |
Comp 306 |
+ve |
|
C |
|
+ve |
Comp 3 |
Comp 19 |
Comp 35 |
Comp 51 |
Comp 67 |
Comp 83 |
Comp 99 |
Comp 115 |
Comp 131 |
Comp 147 |
Comp 163 |
Comp 179 |
Comp 195 |
Comp 211 |
Comp 227 |
Comp 243 |
Comp 259 |
Comp 275 |
Comp 291 |
Comp 307 |
+ve |
|
D |
|
+ve |
Comp 4 |
Comp 20 |
Comp 36 |
Comp 52 |
Comp 68 |
Comp 84 |
Comp 100 |
Comp 116 |
Comp 132 |
Comp 148 |
Comp 164 |
Comp 180 |
Comp 196 |
Comp 212 |
Comp 228 |
Comp 244 |
Comp 260 |
Comp 276 |
Comp 292 |
Comp 308 |
+ve |
|
E |
|
EC/IC50 |
Comp 5 |
Comp 21 |
Comp 37 |
Comp 53 |
Comp 69 |
Comp 85 |
Comp 101 |
Comp 117 |
Comp 133 |
Comp 149 |
Comp 165 |
Comp 181 |
Comp 197 |
Comp 213 |
Comp 229 |
Comp 245 |
Comp 261 |
Comp 277 |
Comp 293 |
Comp 309 |
EC/IC50 |
|
F |
|
EC/IC50 |
Comp 6 |
Comp 22 |
Comp 38 |
Comp 54 |
Comp 70 |
Comp 86 |
Comp 102 |
Comp 118 |
Comp 134 |
Comp 150 |
Comp 166 |
Comp 182 |
Comp 198 |
Comp 214 |
Comp 230 |
Comp 246 |
Comp 262 |
Comp 278 |
Comp 294 |
Comp 310 |
EC/IC50 |
|
G |
|
EC/IC50 |
Comp 7 |
Comp 23 |
Comp 39 |
Comp 55 |
Comp 71 |
Comp 87 |
Comp 103 |
Comp 119 |
Comp 135 |
Comp 151 |
Comp 167 |
Comp 183 |
Comp 199 |
Comp 215 |
Comp 231 |
Comp 247 |
Comp 263 |
Comp 279 |
Comp 295 |
Comp 311 |
EC/IC50 |
|
H |
|
EC/IC50 |
Comp 8 |
Comp 24 |
Comp 40 |
Comp 56 |
Comp 72 |
Comp 88 |
Comp 104 |
Comp 120 |
Comp 136 |
Comp 152 |
Comp 168 |
Comp 184 |
Comp 200 |
Comp 216 |
Comp 232 |
Comp 248 |
Comp 264 |
Comp 280 |
Comp 296 |
Comp 312 |
EC/IC50 |
|
I |
|
EC/IC90 |
Comp 9 |
Comp 25 |
Comp 41 |
Comp 57 |
Comp 73 |
Comp 89 |
Comp 105 |
Comp 121 |
Comp 137 |
Comp 153 |
Comp 169 |
Comp 185 |
Comp 201 |
Comp 217 |
Comp 233 |
Comp 249 |
Comp 265 |
Comp 281 |
Comp 297 |
Comp 313 |
EC/IC90 |
|
J |
|
EC/IC90 |
Comp 10 |
Comp 26 |
Comp 42 |
Comp 58 |
Comp 74 |
Comp 90 |
Comp 106 |
Comp 122 |
Comp 138 |
Comp 154 |
Comp 170 |
Comp 186 |
Comp 202 |
Comp 218 |
Comp 234 |
Comp 250 |
Comp 266 |
Comp 282 |
Comp 298 |
Comp 314 |
EC/IC90 |
|
K |
|
EC/IC90 |
Comp 11 |
Comp 27 |
Comp 43 |
Comp 59 |
Comp 75 |
Comp 91 |
Comp 107 |
Comp 123 |
Comp 139 |
Comp 155 |
Comp 171 |
Comp 187 |
Comp 203 |
Comp 219 |
Comp 235 |
Comp 251 |
Comp 267 |
Comp 283 |
Comp 299 |
Comp 315 |
EC/IC90 |
|
L |
|
EC/IC90 |
Comp 12 |
Comp 28 |
Comp 44 |
Comp 60 |
Comp 76 |
Comp 92 |
Comp 108 |
Comp 124 |
Comp 140 |
Comp 156 |
Comp 172 |
Comp 188 |
Comp 204 |
Comp 220 |
Comp 236 |
Comp 252 |
Comp 268 |
Comp 284 |
Comp 300 |
Comp 316 |
EC/IC90 |
|
M |
|
-ve |
Comp 13 |
Comp 29 |
Comp 45 |
Comp 61 |
Comp 77 |
Comp 93 |
Comp 109 |
Comp 125 |
Comp 141 |
Comp 157 |
Comp 173 |
Comp 189 |
Comp 205 |
Comp 221 |
Comp 237 |
Comp 253 |
Comp 269 |
Comp 285 |
Comp 301 |
Comp 317 |
-ve |
|
N |
|
-ve |
Comp 14 |
Comp 30 |
Comp 46 |
Comp 62 |
Comp 78 |
Comp 94 |
Comp 110 |
Comp 126 |
Comp 142 |
Comp 158 |
Comp 174 |
Comp 190 |
Comp 206 |
Comp 222 |
Comp 238 |
Comp 254 |
Comp 270 |
Comp 286 |
Comp 302 |
Comp 318 |
-ve |
|
O |
|
-ve |
Comp 15 |
Comp 31 |
Comp 47 |
Comp 63 |
Comp 79 |
Comp 95 |
Comp 111 |
Comp 127 |
Comp 143 |
Comp 159 |
Comp 175 |
Comp 191 |
Comp 207 |
Comp 223 |
Comp 239 |
Comp 255 |
Comp 271 |
Comp 287 |
Comp 303 |
Comp 319 |
-ve |
|
P |
|
-ve |
Comp 16 |
Comp 32 |
Comp 48 |
Comp 64 |
Comp 80 |
Comp 96 |
Comp 112 |
Comp 128 |
Comp 144 |
Comp 160 |
Comp 176 |
Comp 192 |
Comp 208 |
Comp 224 |
Comp 240 |
Comp 256 |
Comp 272 |
Comp 288 |
Comp 304 |
Comp 320 |
-ve |
|
Optimization using Janus Liquid Handling Robots (two Step Optimization):
Optimizing the screen is important to determine the most appropriate concentrations of compounds to screen the cells against. Below is a protocol for optimization using Janus Liquid Handling Robots (Two Step Optimization), from which we choose the best conditions for our small compound screens.
Step 1: Once the bench top assay has been optimised, we will create a liquid handling protocol which mirrors closely the steps used in the bench top screening. This will require optimisation of our MDT and Varispan liquid handling heads and in most cases will require a number of cell plates to be provided to ensure liquid handling does not interrupting the cell layer.
Once the liquid handling has been optimised, a number of plates will be run with the optimised protocol to ensure reproducibility of the screen. This will require a number of plates set up with the Positive, Negative and the desired EC/IC concentrations. When preparing to run a screen, a statistically relevant measure is needed to determine if the screen produces data with sufficient sensitivity to accurately differentiate between positive "hits" and negative samples.
Three measurements are commonly used: signal-to-background ratio, coefficient of variation, and the Z' Factor. The measure we use is the Z' Factor (see below). The Optimization Step 2 procedure will include two or more plates fully loaded with positive and negative controls to fully test the outputs "robustness". The calculated Z' Factor will be a number between 0 and 1 with the best, most robust assays approaching 1.
To interpret the Z'-factor, use these guidelines:
- A Z-factor of 1, ideal. This is approached when you have a huge dynamic range with very small standard deviations.
- A Z-factor between 0.5 and 1.0 is an excellent assay.
- A Z-factor between 0 and 0.5 is marginal.
- A Z-factor less than 0 means that the signals from the positive and negative controls overlap, making the assay essentially useless for screening purposes.[1]
In addition to the Z' Factor, which will measure the "robustness" of the assay, there are several methods we can use to visualize the reproducibility of the output. These visualization techniques are utilized to identify systematic sources of error or data with poor reproducibility due to poor assay design or implementation.
From here a decision will be made whether to continue optimisation or progress directly to a targeted pilot study.
Step 2: Once the most appropriate parameters have been established from Step 1, we test the assay using these conditions to ensure the screen has met the established criteria before moving forward to a full scale screen.
Each pilot screen plate will be run at 2 concentrations (usually 10mM and 2mM, mirroring the conditions in the final screen) along with a Resazurin plate, which will allow us to determine potential toxic effects of the compounds.
Raw data Heat Maps:
The above is an example of a 384 well small compound screen plate with no edge effects (Carter, Sharma, and Ebner, unpublished data)
Replicate Correlation Plot:
(Lunden, Ebner, unpublished data)
Plate-Well Scatter Plot:
These are visual examples of Edge Effect and Drift from Eli Lilly and Company and the National Institutes of Health Chemical Genomics Centre guidelines for running HTS assays.
Screening without a POsitive Control
Not all screens have a positive control to accurately measure the phenotypic change a positive hit in your screen might produce. Often, finding those hits is the reason for running a novel screen and if there was a positive control, there would be no reason to run the screen! In these situations, extra care must be applied to develop the screen. In most cases, if the assay is performing well, positive hits follow.
Statistical Analysis:
A well designed, highly sensitive and robust high throughput screen requires strict quality control and accurate measurements. These requirements work in unison to produce a screen that yields hits that can be confirmed in secondary screens as being biologically active. Design and sensitivity has been addressed previously in this guide. Here, we will address quality control and statistical analysis of raw data.
Plate-to-Plate Variability
Because systematic error can decrease the overall performance of a screen, controls are always included on assay plates to help identify plate-to-plate variability and establish background levels of an assay. It is important in HTS to have the ability to compare all of the plates in a production run to each other. To do this, normalization of raw data is employed. Below are a couple of processing methods with a summary of their equations:
Below, the Z Score, Enhanced Z Score and B Score methods excludes the plate positive and negative control measurements and instead uses the samples themselves as controls under the assumption that the majority of samples are inactive and the data is normally distributed.
*Enhanced Z Score is more resistant to outliers (positive hits) than Z Score.
*The B Score is a variant of the Z score that uses MAD to account for plate-to-plate variability as well as a two-way median polish to minimize row and column affects.
*The 3D-B Score is a variant of the B score but applies a third dimension (stacked plates) to minimize row, column and plate affects. (B. Evers, unpublished data)
We have used several of the above tools to normalize our HTS data sets with success. It is standard practice in the HTS field to use several normalizing methods to analyse your data sets before choosing the method that works best for your screen. Additional assistance can be obtained from The Computational Biology Research Group (CBRG) which provides computing support for bioinformatics analysis at the University of Oxford:
http://www.molbiol.ox.ac.uk/CBRG_home.shtml
*A note on control placement on high throughput assay plates: Most commercial high throughput libraries are produced with the outside columns empty to accommodate the placement of assay controls. When possible, it is recommended that control columns be moved to the interior of the plate to minimize positional bias (edge effects).
Replicates
Once the assay protocol and liquid handling procedures have been optimized, the only way to minimize experimental variability further is to increase the number of replicates and then average the resulting measurements. This serves two functions; estimates based on repeated measurements are less variable than single measurement and additional replicates reduce the number of false positives without increasing the false negatives. Of course, additional replicates incur additional cost to a primary screen. However, in our experience, these additional cost are compensated by the reduced false negative hit rate, a stronger statistical evaluation of the results (a way to estimate variability of measurements), and reduced cost associated with "cherry-picking" when repeating primary hit results. Additionally, high throughput screens we have developed at the Target Discovery Institute where lower Z' factors were obtained were successfully developed and produced validated hits by running the assay in triplicate.
We recommend a primary (pilot) screen be run in triplicate. Here, we define replicates as three independent repeats measured under the same experimental conditions. The additional costs are mostly incurred in the plates since in most assays; working stocks are produced from concentrated stocks (greater storage life of concentrated stocks) and are sufficient to produce three replicates. Expensive antibodies or signal producing reagents are increased as well in triplicate primary screens but this is usually a fraction of the additional cost.
Sources of Experimental Error in HTS and Reducing Their Effects
Automation of scientific assays to the HTS platform introduces the potential for experimental and responsive variability not often observed in smaller experiments performed on the bench-top. Sources of systematic error found in HTS can be:
- Temperature and evaporation gradients "edge effects"
- Liquid handling malfunctions - "drift" or repeated pipetting errors
- Batch processing errors
- Non-homogeneous cell seeding
- Differential reagent degradation over time (stability)
- Solubility
- Variation or gradient temperatures in the incubators
- The effect of each compound on cell growth
Techniques for reducing variability in High Throughput Screening
Screens in the TDI use several techniques for reducing assay variability between plates and between wells. To mitigate the effects of systematic error and to assist in the troubleshooting of observed error, we have implemented a stringent regime of:
- Implementing a liquid handling procedure that to the best of the developer's ability treats all plates exactly the same. The reagents must be added and incubated in the same order and over the same amount of time. This is the developer's responsibility.
- For cell based assays, the plates must receive the cells in good condition and at the same passage number. For assays with several steps or more replicates, unfortunately this is very difficult since cells must be kept in suspension for a substantial length of time as the plates are processed. Where possible we aim to bulk up and freeze down a large batch of cells which can be re-sussed as required for the duration of the screen. Often there can be two to three hours between the first plate and the last plate in a production run so this can lead to substantial variability if not controlled for.
- For all cell based assays, the condition of the cells going into the screen plates is paramount. Cells must be actively dividing and under 80% confluent. We have implemented a standardized cell seeding protocol to ensure the cells are fit for the screen. We plate a known number of cells that will be used in the screen assay into T75 or T175 flasks three days before the assay knowing they should be at the correct growth stage and number necessary to seed all of the plates needed for the run. This is a crucial control if a production run is processed over the course of days or weeks.
- When possible, inhibitors or activators which produce a phenotypic change or inhibition in the screen should be included in the "body" of the screen and on multiple plates. These known positive hits must come from the library vendor and will provide information on the assay sensitivity, library condition, precision and reproducibility, and could help diagnose any sources of experimental variability.
- Recording the location and time the plates are in the incubator (it is good practice to independently verify the incubator functioning within its proper temperature, humidity and gas settings). This is more of a trouble shooting step to identify potential sources of error as a production run is progressing.[4]
- Production runs should be completed in as short a window as possible (days or weeks) to minimizing the effect of passage number on the screen. Also, the reagents used in a production screen should all originate from the same lot to reduce lot to lot variability of your reagents.
- After the cells have been seeded, the cell plates are allowed to rest at room temperature for 20 minutes to allow the cells to settle and begin reattaching to the plate surface.[5]
- When possible, additional media changes can be used for screens of longer duration (more than three days) to minimize gas gradients affecting differential cell growth across a plate.
Pre-analysis Metrics
In large high throughput screening production, it is important to monitor the quality of the data being produced during the production run to ensure the procedures are working as expected and the data is of sufficient quality to allow statistically relevant conclusions to be made.
We employ the Z Factor (a slight variant of above Z' Factor, see above) during the assay as it assesses screen samples of screened plates as they are produced.
Additionally, strictly standardized mean difference (SSMD) can be used to assess screen data which is statistically more rigorous.[6]
Identifying Positive Hits
The main goal of any high throughput screen is to identify positive hits, that is, samples that are meaningfully differentiated from the negative controls. Once all the plates have been normalized, the final task of the primary screen is to identify these "positive hits".
There are many positive hit identification techniques available and in many cases, several are used and collated do produce a hit list ranging from strong to weak positives. These techniques include:
1. Mean of normalized data + Standard Deviation - For simplicity, this is the most heavily used positive hit scoring technique and one we use most frequently at the Target Discovery Institute. Depending on the screen, the + standard deviation can be set to -2 (2) or -3 (3).
2. Median of normalized data + MAD - A more Robust technique to identify positive hits.
Once the positive hit list has been produced, the capacity for follow-up study often dictates the number of positive hits passed on for secondary screen evaluation. For example re-ordering of small compounds can be cost prohibitive so sometimes only the strongest or most interesting positive hits are further evaluated.
POst Assay "Off-Target" Filtering
Once the primary screen is completed, additional filtering will be necessary to ensure the positive hits are real and not "Off-target effects". The steps below can be used to confirm primary hits:
Step 1: All positive hits will be re-evaluated for reproducibility (Filter 2) in the same assay (repeat primary screen with positive hits) to reconfirm positive hits.
Step 2: Repeat the remaining hits on a second cell line (if possible). The mechanism of sensitivity or loss of function might be restricted to specific cell line. Although not specifically an "off-target" strategy, this is an excellent follow-up assay and filter aid.
Reporting Results
Cell Viability Assays
A quick review of Cell Viability assays can be found on the following web site:
http://www.rndsystems.com/product_detail_objectname_cell_viability.aspx
As an assay standard we use Resazurin salts in PBS (10µg/mL) for our cell viability assays due to very low cost of the reagent and high sensitivity. However, this may not be the most appropriate measure if cells are metabolically compromised – this may be an important issue in transformed cell lines. This is something which will be discussed in our intial meetings. Alternatives to Resazurin are available but must be balanced for cost.
HTS Review Papers and Resources
Below are a number of papers (and Web sites) that provide excellent reviews of HTS in specific fields of research. This list is representative and is not meant to replace a full literature search!
NIH Chemical Genomics Center: General guidelines for HTS
http://www.ncbi.nlm.nih.gov/books/NBK53196/
Current Trends in HTS:
Mayr, L.M. and D. Bojanic, Novel trends in high-throughput screening. Curr Opin Pharmacol, 2009. 9(5): p. 580-8.[7]
Statistics in HTS:
Malo, N., et al., Statistical practice in high-throughput screening data analysis. Nat Biotechnol, 2006. 24(2): p. 167-75.[8]
Cardiovascular Research HTS review:
Etzion, Y. and A.J. Muslin, The application of phenotypic high-throughput screening techniques to cardiovascular research. Trends Cardiovasc Med, 2009. 19(6): p. 207-12. [11]
Cell Based Pharmacogenomics:
Welsh, M., et al., Pharmacogenomic discovery using cell-based models. Pharmacol Rev, 2009. 61(4): p. 413-29.[12]
Small compound High Content Imaging Screen review:
Carpenter, A.E., Image-based chemical screening. Nat Chem Biol, 2007. 3(8): p. 461-5.[13]
Small compound cell-based HTS target identification:
Rix, U. and G. Superti-Furga, Target profiling of small molecules by chemical proteomics. Nat Chem Biol, 2009. 5(9): p. 616-24.[14]
Saxena, C., et al., An immuno-chemo-proteomics method for drug target deconvolution. J Proteome Res, 2008. 7(8): p. 3490-7.[15]
An, F and Tolliday, N Cell-based assays for high-throughput screening. Molecular Biotechnology. 2010 2(45), 180-186
Hertzberg, R and Pope A High-Throughput Screening. New Technology for the 21st Century. Current Opinion in Chemical Biology, 4(4), 445-451
Bioluminescent Assays for High-Throughput Screening:
Mayr, L.M. and D. Bojanic, Novel trends in high-throughput screening. Curr Opin Pharmacol, 2009. 9(5): p. 580-8.[16]
Biosensors:
Morris, M.C., Fluorescent biosensors of intracellular targets from genetically encoded reporters to modular polypeptide pTDIes. Cell Biochem Biophys, 2010. 56(1): p. 19-37.[17]
Kurzawa, L. and M.C. Morris, Cell-cycle markers and biosensors. Chembiochem, 2010. 11(8): p. 1037-47.[18]
List of Academic Screening Facilities World-wide: