riskcalc.org

A Repository of Risk Calculators for Medical Decision Making

Alex Zajichek

Research Data Scientist, Cleveland Clinic

June 12, 2025

A Little Background

QHS is a department of over 100 biostatisticians, data scientists, programmers, etc. that collaborate on and supply quantitative support to research activities at Cleveland Clinic

From clinical trials and study design to precision medicine and population health across many disease areas

My work focuses on clinical prediction modeling and observational statistical analysis, primarily using EHR and/or registry data

Free repository of risk calculators for individualized medical decision making; intended for clinician/patient use
Embedded with predictive models from published research studies; mostly regression
Each calculator is a separate Shiny application
Average about 10-15K active users/month (per Google Analytics)

Added source code to public QHS GitHub repository and links in the apps themselves (Example)

Built an R package to accelerate building risk calculators with the standard layout (work in progress)

How can we more formally use GitHub for source management, through CI/CD. Currently we just “put” it there, but it’s not actually connected to the live platform
We run into problems with version as different people develop calculators in different contexts, using new packages, R versions, etc. Sometimes have to retrofit applications to work on server so everything doesn’t break. Pipe operator a great example of this. Containerize things…
Currently hardcode model equations into code to stay lightweight, or save as RData object, but no real standardization. Want to generalize the concept of “risk calculator” to be agnostic to methodology, to accommodate ML and other frameworks, but need to have a modular way to do it consistently.
Often in academia models are developed, papers are published, risk calcluator is built and released and then “forgotten”. Becomes an artifcact and then we move on to the next thing. Given these are supposed to be living, breathing data products, would be good to monitor them accordingly to assess performance

One stop shop for clinical risk prediction; community involvement
API development for model access
More direct research ecosystem for methodological advancement; clinical integration

These are some possibilities we’ve discussed
Academia saturated with prediction models, don’t know what to use.
How we garner community development to submit models, coalesce, rank/upvote models so the best prediction is given for the given context.
Invoke community model validation. Display validation levels of models.
Want to be able to give access to models to researchers and organizations to use in their workflows
How can we advance methodology for clinical risk prediction, in the context of using these types of tools for medical decision making? (e.g., pool models/information to take advantage of independent information w/o redundancy, use of prior information, etc.)
Academia disconnected from clinical workflows; how to partner to be more directly informed by clinical constraints (focus on the tool, not just the model) (bridging research and clinical)
Being able to adapt to local environments

Michael W. Kattan, PhD
Department Chair (2004-2024)
Quantitative Health Sciences
Cleveland Clinic

Creator of riskcalc.org