Stanford Center for Biomedical Informatics Research Colloquia - Shared screen with speaker view
Nigam Shah
Just to the panelists: FYI - https://tinyurl.com/ML-panel has my thoughts on deploying models along three axes:(1) the philosophy or conceptual steps [i.e. the delivery science],(2) the process or sequence of actions [i.e. the SOP, MINIMAR checklist for training datasets, etc],(3) the mechanics or nuts and bolts [i.e. what database, what API, what server, send email or use Epic API etc].
Nigam Shah
The first slide will be shown (by Jonathan) when introducing the discussants.
Josh Knowles
Jomol Mathew
Is it possible to be get the slides or the recording?
jonathan chen
Sessions is being recorded. Should be posted online in a few days at https://bmir.stanford.edu/education/colloquia.html
Jomol Mathew
Thanks a lot
Somalee Datta
Matt: One of the things we have in our data is insurance type e.g. medicare or hmo or captitated. It may be possible to check for equity using that. One of our other collaborators, Stephen Lin, has been asking these questions.
Matthew Lungren
thats a great idea
Matthew Lungren
we have an ongoing project on that topic with mit and emory - lots of interesting findings with our models
tina hernandez boussard
Using insurance is a great start. However, you have to be careful using these proxy variables when you start looking into bias and fairness! ;)
Jomol Mathew
Which parts of this happen on the school side vs the hospital side? Or do you have a joint team across school and hospital
Nigam Shah
@Jomol, we are moving towards having joint teams. Having our IT unified under TDS has been a good step forward.
tina hernandez boussard
MINIMAR (MINimum Information for Medical AI Reporting): Developing reporting standards for artificial intelligence in health careTina Hernandez-Boussard, Selen Bozkurt, John P A Ioannidis , Nigam H ShahPMID: 32594179 PMCID: PMC7727333
Nigam Shah
https://tinyurl.com/ML-panel has details on all three axes we talked about (the philosophy, the best practice processes, and the mechanics). The url should be viewable to anyone.
kate vitale
NIH’s AllofUs Research Program is another (explicitly) diverse EHR, survey and genetic database getting off the ground that might be of interesting for some folks
deendayal dinakarpandian
What are some ways to quantify the expected benefit/cost of adding an AI option to an existing workflow?
wendi knapp
yes Thanks!!
Nigam Shah
@Deen — check out https://www.medrxiv.org/content/10.1101/2020.07.10.20149419v1
Nigam Shah
@Wendi — Developments in public datasets are patchy. For COVID there is N3C by NIH and the Covidresearchdatabase by Datavant.Reg communicating up the org, the use case has to be distilled down to ROI and value statment.Finally, gettting data out from Vendor systems is something ot be handled at contracting. Get data access included in the purchasing.
Arjun Bhatt
Dr. Lungren mentioned this worry of model drift, where the relationship between variables that the model is relying upon might change over time. To what extent can these changes be “compensated” for via model retraining? And to make this a bit more concrete, what specific sources of drift might present more difficulties to a scientist trying to build a model that will stand the test of time?
wendi knapp
great point distribution
wendi knapp
yes a realistic view
Nigam Shah
@Priya — GOSSIS is a nice (and free!) alternative to things such as the APACHE IV score.
Nigam Shah
Great step, and in the right direction.
Joseph Paul Cohen
If the issue is shift then why not define things to prediction that we know won’t shift over time?
wendi knapp
thanks all!