R for HTA 2024 Workshop - Robert Smith & Tom Ward - AssertHE

Summary

The speaker introduces the 'CT H' package developed for model review in health economics, emphasizing the shift from spreadsheets to script-based languages. The package includes checks for model validation and a visualization tool for understanding the model's function network. It also features a unique summary generation using large language models to explain individual functions. The presentation highlights the importance of communication, efficiency, and adaptability in model development, inviting contributions and feedback on the open-source tool.

Takeaways

• 🙌 The presentation introduces the CT H package developed for model review, emphasizing the transition from spreadsheet software to script-based programming languages like R.
• 📝 The speaker expresses gratitude to contributors who have helped in the development of the package, including those who have tested bugs and contributed to the manuscript.
• 🔄 The overarching aim is to shift the modeling pipeline to a more integrated system for statistical work, model development, visualization, reporting, and application creation.
• 📈 The importance of usability, communication, efficiency, adaptability, and future-proofing in model development is highlighted, with a focus on the need for intuitive interfaces and effective communication of complex clinical pathways.
• 🤖 The potential of large language models in health economic modeling is mentioned as a future direction, emphasizing the need for models to be adaptable and incorporate advanced technologies.
• 🔍 The assertHR package is introduced as a tool to help modelers review and build models, offering a series of checks and functionalities to ensure models are working as intended.
• 📊 The package includes functionality for visualizing the structure of a model through a function network plot, which helps in understanding how different functions within a model relate to each other.
• 🛠️ The visualization tool allows for the identification of functions, their documentation, corresponding tests, and code coverage, providing a quick overview of the model's structure and testing status.
• 🔗 The use of interactive HTML outputs for model visualization is discussed, enabling easy sharing and communication of the model structure with stakeholders.
• 🔑 The potential for generating summaries of model functions using large language models is explored, offering a way to understand individual components of a model in layman's terms.
• 🔍 The script concludes with a call for contributions and feedback on the CT H package, emphasizing the open-source nature of the project and the community's role in its development and improvement.

Q & A

What is the primary goal of the CT H package being discussed?

The primary goal of the CT H package is to shift the modeling pipeline from spreadsheet software like Excel to script-based programming languages, enabling a more integrated and efficient process for statistical work, model development, data visualization, reporting, and app creation.

Why is there a focus on moving away from Excel to script-based programming languages in model development?

The focus is on improving usability, communication, efficiency, adaptability, and future-proofing. Script-based languages allow for better code management, testing, and integration with other tools, which is essential for complex health economic models.

What does the speaker mean by 'shift the modeling pipeline'?

Shifting the modeling pipeline refers to transitioning from using multiple disparate tools like Excel, VBA, and PowerPoint to a unified system where all aspects of model development, analysis, and reporting are handled within script-based programming languages.

What is the significance of the assertHR package in the context of this presentation?

The assertHR package is significant as it provides a set of tools for modelers to review and build models more effectively. It includes functionality for conducting checks on model components and visualizing the structure of a model through function networks.

How does the assertHR package help in reviewing models?

The assertHR package helps in reviewing models by providing a series of checks that can be applied to model components, ensuring they meet certain criteria, and by visualizing the model's function network, which aids in understanding the relationships and flow between different parts of the model.

What is the role of the 'visualize project' function in the assertHR package?

The 'visualize project' function in the assertHR package allows users to plot the function network of a model, which helps in understanding how different functions within the model are interconnected and how they contribute to the overall model structure.

How can the assertHR package be used to improve communication about a model's development?

The assertHR package can be used to improve communication by generating summaries of model functions, identifying test coverage, and providing an interactive visualization of the model's structure. This makes it easier for stakeholders with varying levels of technical knowledge to understand the model.

What is the potential future direction for the assertHR package mentioned in the script?

The potential future direction includes expanding the testing functionality, integrating a chat-bot for interactive questions about model functions, fine-tuning a large language model for better function summaries, and possibly submitting the package for peer review and publication.

How does the speaker address concerns about the limitations of the assertHR package in verifying model accuracy?

The speaker acknowledges that the assertHR package is not a substitute for full verification checks and that it may not catch all errors, such as incorrect transition probabilities. However, it is intended to nudge modelers towards better testing practices and provide a starting point for identifying potential issues.

What is the current status of the assertHR package in terms of availability and contribution?

The assertHR package is currently available as an open-source development version on GitHub. The speaker encourages contributions, testing on various models, and feedback to improve the package further.