--- title: "Getting Started with ElicitationWizard" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Getting Started with ElicitationWizard} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>", eval = FALSE ) ``` ## Overview ElicitationWizard is an R package that provides an interactive Shiny application for eliciting Bayesian prior distributions using large language models (LLMs). It queries multiple LLM "experts" in parallel, parses their responses into named probability distributions, and aggregates them via opinion pooling or the Delphi method. --- ## Installation Install the package from CRAN: ```{r install-cran} install.packages("ElicitationWizard") ``` Or install the development version from GitHub: ```{r install-github} # install.packages("pak") pak::pak("JefferyAyiti/Automated_Elicitation") ``` ### Dependencies The following R packages are required and will be installed automatically: | Package | Purpose | |--------------|---------------------------------------------------| | `shiny` | Web application framework | | `bslib` | Bootstrap theming (flatly theme) | | `shinyjs` | Button enable/disable during async calls | | `ellmer` | LLM API client (`chat_openai_compatible()`) | | `future` | Parallel async execution | | `promises` | Non-blocking promise handling | | `commonmark` | Markdown rendering | --- ## API Requirements ElicitationWizard connects to any **OpenAI-compatible API endpoint**. You need: - A **Base URL** — the root of the API (e.g. `https://your-endpoint/v1`) - An **API key** — for authentication We recommend [SAIA (Scalable AI Accelerator)](https://docs.hpc.gwdg.de/services/ai-services/saia/index.html), which provides OpenAI-compatible access to a range of open models including those used by this package. Other compatible providers include: - **Groq** — `https://api.groq.com/openai/v1` - **Together AI** — `https://api.together.ai/v1` - **OpenRouter** — `https://openrouter.ai/api/v1` These can be provided in two ways: **Option 1 — Environment variables (recommended):** ```{r env-vars} Sys.setenv(BASE_URL = "https://your-endpoint/v1") Sys.setenv(API_KEY = "sk-...") ``` Or add them to your `.Renviron` file for persistence across sessions: ``` BASE_URL=https://your-endpoint/v1 API_KEY=sk-... ``` **Option 2 — Enter directly in the app** via the "API Connection" panel in the sidebar. --- ## Launching the App ```{r launch} library(ElicitationWizard) run_elicitation() ``` --- ## Interface Overview The app has a **sidebar** for configuration and a **main panel** with four tabs. ### Sidebar | Panel | Description | |----------------------|--------------------------------------------------------------------------------| | **API Connection** | Base URL and API key; collapsible; pre-filled from environment variables | | **Clinical Context** | Free-text description of the disease, population, and trial setting | | **Result Expectations** | Model structure and the parameter(s) to elicit priors for | | **Expert Configuration** | Number of LLM experts (1–4) and model assignment per expert | | **Elicit Priors** | Button to run elicitation; disabled while requests are in flight | ### Main Panel Tabs | Tab | Description | |----------------------|--------------------------------------------------------------------------| | **Expert Responses** | Full markdown-rendered responses from each expert, with LaTeX support | | **Prior Summary** | Parsed distributions in a table + density overlay plots | | **Opinion Pooling** | Linear pooled distribution across all valid experts | | **Delphi Method** | Multi-round iterative elicitation with peer feedback | --- ## Available Models ### Elicitation Models These models act as the LLM "experts": | Label | Model ID | |-------------------|----------------------------------------| | MedGemma-27b-it | `medgemma-27b-it` | | Gemma-3-27b-it | `gemma-3-27b-it` | | Deepseek-R1-L-70b | `deepseek-r1-distill-llama-70b` | | Llama-3.3-70b | `llama-3.3-70b-instruct` | | Qwen-3-I-30b | `qwen3-30b-a3b-instruct-2507` | | Mistral-3-675B | `mistral-large-3-675b-instruct-2512` | ### Stan Code Generation Models Used by the "Generate Stan Code" buttons: | Label | Model ID | |----------------------|-----------------------------------| | Qwen-Coder (default) | `qwen3-coder-30b-a3b-instruct` | --- ## Supported Distribution Families The parser recognises the following families in expert responses: Normal, Log-Normal, Beta, Gamma, Exponential, Half-Normal, Cauchy, Half-Cauchy, Uniform, Weibull, Poisson, Student-t, Inverse-Gamma, Log-Logistic, Pareto > **Note:** Dirichlet and Binomial are recognised but not supported by the > density pipeline and will be marked as invalid. --- ## Next Steps See `vignette("tutorial", package = "ElicitationWizard")` for a step-by-step walkthrough of a complete prior elicitation session using the default Poisson-Gamma clinical trial example.