Level: Advanced
Els Pattyn, Non-Clinical Efficacy and Safety Statistician, Ablynx, a Sanofi company
Immunogenicity assays to detect anti-drug antibodies (ADA) in subject samples provide data on drug safety and efficacy and are required for approval by health authorities such as EMA and FDA. Such assays are usually qualitative or semi-quantitative and thus require cut-points as threshold values for distinguishing positive and negative samples.
Establishing appropriate cut-points is crucial to ensuring acceptable sensitivity of the assay to detect ADA and as such requires particularly complex statistical calculations. Since one study can have multiple immunogenicity read-outs, these cut-point calculations become very laborious.
We developed a JMP script for cut-point determination and assessment, which follows a pre-determined decision tree developed based on industry guidance, white papers and scientific best-practice. The script is designed to be appropriate for validation and use in GxP studies.
End-users can select decision trees applicable for their needs, such as type of assay and study. The application accepts Excel files to upload data and makes outcome-dependent decisions – for example, adapting the effects included in the mixed-effects model based on the study-design, selecting the most appropriate normalization/transformation, calculating analyst-specific cut-points in case of significant analyst-specific differences, and adapting down-stream analysis in cases where no second-tier confirmatory data is available.
In summary, this JMP script allows immunogenicity cut-points to be calculated quickly and efficiently in a standardized way, including automated reporting that is suitable for regulatory submissions.
Speaker | Transcript |
| Els Pattyn | Hello, my name is Els Pattyn and I'm working as a non clinical efficacy and safety statistician of Sanofi. So I'm happy to present you in JMP an end user tool we have developed for immunogenicity assessments. |
| First short introduction of the company I work for. So originally I was employed by Ablynx. Ablynx is a biopharmaceutical company which is based in Ghent, Belgium. | |
| And we are engaged in the discovery and development of nanobodies. Nanobodies are camelid heavy chain entities, as you can see here, and you can discriminates. (OK, I will first my pointer...set my pointer. I don't think you see anything.) | |
| Okay, here we are. So nanobodies are entities from camelid heavy chain antibodies and they are smaller than the conventional antibodies. Conventional antibodies have to have two heavy chains, two large chains and camelid antibody you only have the heavy chain. | |
| So they are small, they can also be used modular you can have multiple nanobodies and have multiple specificities and they're also easier to manufacture. | |
| We were | |
| founded in 2001 and then 17 years later, we had our first compounds in the clinic. It was Cablivi. It was and | |
| it is targeted for a rare blood disease aTTP. And 2018 was another important year as then we got acquired by Sanofi, which is a large biopharmaceutical company | |
| with headquarters in France. So then at that time, we were more than 450 employees, but with the acquisition of Sanofi and we are part of a fairly large company. | |
| Sanofi itself has more than ??? employees with around 15,500 employees committed to R & D. And you see | |
| we are spread across the three...across three continents. | |
| And also therapeutic platform of Sanofi is very wide. It's only...it's not only the nanobodies, but it's really a lot and it's still growing. | |
| Then about the tool we have developed. | |
| It's an immunogenicity tool. | |
| About the context of immunogenicity. Immunogenicity is the ability of a substance to provoke an immune response. | |
| So often it's wanted. It's when your immune system has an appropriate response to a pathogen, but it can also...can also be wanted response towards a therapeutic agents and that's done in the case of a vaccine. | |
| That it can also be an unwanted response, a response to a therapeutic antigen and then it's called anti drug antibodies or ADAs. | |
| And these ADAs can be a | |
| you can have allergic reactions or even anaphylactic shock. | |
| Loss of efficacy. | |
| You can have antibodies that have neutralizing capacity so that you lose your activity, so it will be no surprise that it has special attention to the regulatory entities. | |
| Several guidelines issued by the EMEA and the FDA regarding the ADAs and how to analyze it. | |
| So completes ADAs determinations fills several steps. It's a multi tiered approach. | |
| You have a screening assays where you determine your sensitivity, then you have confirmatory assay, where you look for where it's specific or not. | |
| And you also have characterization assays, and that's done to determine whether your antibodies have neutralizing capacity. And sometimes you want to determine which isotypes it targets. | |
| So what we have to do is to determine good points, so that we can determine what's positive to discriminate the positives versus the negatives, and that has to be determined on so-called blank or naive population. | |
| So the blank population, of course, should be representative; it should be free of outliers also not exist...have pre existing antibodies and incorporate all sorts of variability. | |
| You have biological variabilities, so the different subjects that you test, subject ids, but you also have variability...technical vulnerability so that's of your analysts, so of your run, | |
| plate to plate credibility. And when you want to assess it in a proper way, you prefer to do it by mixed effect model with a REML model so that you can have these elements as random effects in your model. | |
| So there was within Sanofi, a | |
| a... | |
| a need to have an end user tool for good point setting, so the aims were that we could have a harmonized and standardized approach across all sites; | |
| to have a user friendly end user tool with the state of the art analysis, so it should be a statistical package because we want to have...to use...make use of the mixed effect model. | |
| It should also operate in a regulator...regulated environment, because we have also lots of GxP studies and it should also include uniform and automatic reporting. | |
| So that's where we started, | |
| where we explored whether we could do it with JMP and that's by its its language, it's JSL language. | |
| You all know it can easily be retrieved when you have a graph. You can just click save scripts, and then you have the script. So that's an easy start. | |
| And another...and what is also an asset is that it is programmable towards an outcome dependent decision. So we started with retrieving the scripts from every graph and every analysis we wanted to do, and then it became really a huge analysis. | |
| We have one parental code and see here, and we have different | |
| child codes that are called by the parental codes. | |
| And it is, | |
| as I said, it should be, this should be programmable towards outcome depending decision, so the code gets really more difficult by | |
| audience, because we have to do a lot of statistical analysis, assess whether you have statistical significance or not. | |
| We have different normalizations we want to assess. | |
| So | |
| that's around some extra aspects of the code, how it look like. | |
| So | |
| before giving a demonstration, here there are two, in fact two steps we want to perform the analysis. First, there is an Excel template where we populate...where the raw data should be populated. | |
| And then the JSL has to be launched. So it's done, only the parental code that has to be launched. | |
| I will go over these two steps. So | |
| we have one | |
| Excel template. I'll open it so that you can have a look. | |
| So here, you see an overview, where we can populate it and here is data for upload in the Excel file. | |
| As I have told, | |
| it can be an ADA assay or an NAb essay and depending on whether it's NAb or ADA assay, the header names are different. So here, you can say it's an ADA assay, it's either without or with confirmatory data. | |
| The code is adapted that it can either | |
| accept...that it can either accept summarized data or otherwise raw data, replicated data. So say it's here, replicated data that you can answer the number of replicates. Say, for instance it's three replicates. | |
| And in order to to determine if it's a ??? point we also related to negative control...to the negative control of a plate, and that should be multiple negative controls on one plate. For instance, 2, and I say, I call it Start and End, so when you populate this | |
| then we make a worksheet. And then here, you have your Excel templates that you have to populate and it's ready for upload and have the correct naming for uploads. | |
| So, once you have done that, then you can go to JMP. | |
| I already have JMP open. And what you have to do, then, is to click on the parental JSL codes and automatically an interaction menu appears. And then you have to fill in the fields to be filled out, so yeah. | |
| Just do here and test for JMP. | |
| Then there are five types of assays, so let's say we have now an ADA assay and here you can upload this file for uploads. I have here input data, some dummy data to upload. Then we select it and on here, there are some | |
| numbers you can change. The default numbers are filled out, but if you want another percent's thresholds as acceptance criteria, you can change it. | |
| And if your data has an order specificity done three decimal places, you can change it, and I think here...there's no decimal places in the data, so I change it to zero. So then | |
| you have a second introduction window here. It's an ADA assay so that's typical for the ADA assay. You have an NAb assay, | |
| you have another window. So here, you have to select whether it's clinical or nonclinical because good points, I think, is different there. I have here a confirmatory data and I want to analyze in a clinical setting, so I click it here. | |
| Now you have different outlier.... | |
| outline removal approaches that you can click. | |
| And also, there is another level of flexibility that you can enter. When you...just to make it user friendly, we have one BTD approach. BTD is | |
| an internal guideline and when you have...you want this approach, then there will be no other window that is opening. | |
| And when you want to have extra options then you can click it here, and you can have extra output or calculate, for instance, run specific differences on a specific | |
| difference. So when you click here, you have extra options, but I want to stick to the general upload approach. And then you can also have an upfront exclusion. We want to do it here and, once you have entered that, then the analysis starts. | |
| So it's a whole chain of analysis. It's an outlier removal approach and it's an iterative way. It checks so that when you have removed outliers, first analytical, than biological or if there are other outliers. So here, you see a series of analysis that's been performed. | |
| So it takes a while, because this data has | |
| screening data and also also confirmatory data. So now the calculations are | |
| going on. | |
| And then, at the end, there is a report. That's what you see here, | |
| that's still assembling the report now. | |
| And it's now outputting the data sets in PDF formats. | |
| Generating the outputs so now, and now, it's finished. So in the result parts, | |
| all results are automatically output here. That's what you see here. You have the data sets, you have a report and a journal, and also a PDF file. And then data sets, the different data sets that were outputs. And here you see that you have full report with the analysis settings, the methodology. | |
| Close it. And you have your descriptives. | |
| And of course what is nice | |
| for JMP is that you have your interactivity when you want to follow | |
| this subject, for instance. I have to double click on 411. | |
| You can follow it throughout all all the analysis so it's all linked. So, then you have here, then for my information of the screening cut point factors. It assesses different transformations, different normalizations, then it evaluates the most appropriate blank population. | |
| You have an output of all | |
| outliers, whether they were analytical or biological. | |
| And identities of the outliers. Yeah you have...so you see, you have all | |
| analysis that has to be performed, all analysis relevant for the settings. Same also for confirmatory cut points. And here is ADA scoring, whether the score is negative, positive, reactive. So | |
| by this, the whole analysis is done and you have the final conclusions. | |
| Under the data tables outputs. | |
| And the same, what we see here, the same, we also have, of course, in a PDF format. | |
| That can be then submitted for... | |
| submitted | |
| for...it's validated. We are able to validate it so everything we saw there, you see again here in a PDF format. | |
| So | |
| so, to conclude, what was the effect, what was the aim to do. | |
| We were able to generate | |
| codes to do an analysis very quickly and efficiently. Normally it took days for the analysis and for the reporting, now we can we have our... | |
| our analysis and report in 10 minutes, so it's an automated in a standardized way. | |
| It's an automatic reporting and it's also it's performed in a validated environment that's suitable for submission. | |
| So yeah I want to talk...to thank my colleagues, my colleagues of the non clinical efficacy and safety teams. I highlighted people that were involved in | |
| the development of the end user tool. | |
| And also yeah it was, of course, | |
| a collaboration between different teams, so I want to thank all people that were involved in that. |