Nivedita Bhadra

Large Language Models and Working with LLM APIs

Nivedita — Wed, 05 Aug 2026 21:00:00 GMT

1 Introduction

A language model is a system that estimates how likely a given sequence of words is, and uses that estimate to predict what comes next. Given “I am going to the”, a language model doesn’t retrieve a stored answer — it assigns a probability to every candidate next word based on patterns learned from text, and the most likely candidates (store, gym, office) reflect what tends to follow that phrase in the data it learned from.

A Large Language Model (LLM) is the same idea at a much larger scale: a neural network, almost always built on the Transformer architecture, trained on a very large text corpus to perform this next-token prediction task, and then adapted so that the resulting model can follow instructions and hold a conversation rather than just continue text.

This notebook builds that idea computationally — training a small next-token prediction model from scratch, verifying it reproduces the “going to the ___” example directly rather than asserting it — and then works through what an LLM API actually is, building a working (if self-contained) API client that demonstrates authentication, request structure, token accounting, and rate limiting. Where real hosted models are involved, this notebook uses the current OpenAI and Google Gen AI SDKs to verify their setup code is correct and importable, since no network access to those providers’ endpoints is available in this environment; the toy API built below is what handles the actual executed generation examples.

2 Next-Token Prediction, Built From Scratch

The mechanism underlying every LLM can be demonstrated with a tiny model trained on a handful of sentences. A bigram model — predicting the next word from only the single word before it — is a drastic simplification of what a real Transformer does (which conditions on the entire preceding context, not just one word), but the underlying operation is the same: count how often each word follows a given word in training data, and turn those counts into a probability distribution over what comes next.

import re
from collections import defaultdict, Counter
import numpy as np

training_sentences = [
    "i am going to the store",
    "i am going to the gym",
    "i am going to the gym",
    "i am going to the office",
    "i love eating pizza",
    "i love eating salad",
    "the capital of france is paris",
    "she is going to the gym today",
    "he loves eating pizza for dinner",
]

def tokenize(text):
    return re.findall(r"[a-z]+", text.lower())

def train_bigram_model(sentences):
    model = defaultdict(Counter)
    for sentence in sentences:
        tokens = [""] + tokenize(sentence) + [""]
        for prev_tok, next_tok in zip(tokens, tokens[1:]):
            model[prev_tok][next_tok] += 1
    return model

bigram_model = train_bigram_model(training_sentences)
print("Learned continuations of 'the':", dict(bigram_model["the"]))

Learned continuations of 'the': {'store': 1, 'gym': 3, 'office': 1, 'capital': 1}

The raw counts are converted to a probability distribution the same way a real model’s output logits are: via softmax, with a temperature parameter controlling how sharply the distribution favors the highest-count option.

def next_token_probabilities(model, prev_token, temperature=1.0):
    counts = model[prev_token]
    if not counts:
        return {}
    tokens = list(counts.keys())
    logits = np.log(np.array(list(counts.values()), dtype=float))
    scaled = logits / temperature
    exp_scaled = np.exp(scaled - scaled.max())
    probs = exp_scaled / exp_scaled.sum()
    return dict(zip(tokens, probs))

probs = next_token_probabilities(bigram_model, "the", temperature=1.0)
for token, p in sorted(probs.items(), key=lambda x: -x[1]):
    print(f"{token:10s} {p:.1%}")

gym        50.0%
store      16.7%
office     16.7%
capital    16.7%

This reproduces the classic “I am going to the ___” example exactly: gym is the most probable continuation given the training data, followed by store, office, and capital at equal lower probability, matching how often each followed “the” during training rather than being asserted.

#pip install matplotlib numpy gradio openai google-genai

import matplotlib.pyplot as plt

tokens = list(probs.keys())
values = list(probs.values())

fig, ax = plt.subplots(figsize=(6, 3.5))
bars = ax.bar(tokens, values, color="#4C72B0")
ax.set_ylabel("Predicted probability")
ax.set_title('Next-token prediction after "I am going to the ___"')
ax.set_ylim(0, max(values) * 1.3)
for bar, p in zip(bars, values):
    ax.text(bar.get_x() + bar.get_width() / 2, p + 0.01, f"{p:.0%}", ha="center")
plt.tight_layout()
plt.show()

Fontconfig warning: ignoring UTF-8: not a valid region tag
Matplotlib is building the font cache; this may take a moment.

3 The Evolution of Language Modeling Architectures

Type	Description	Example
N-gram models	Predict the next token from a fixed-size window of preceding tokens (e.g., the single previous word, as in the bigram model above)	Trigram models
RNN / LSTM	Sequence models that carry a hidden state forward, giving them memory of earlier context beyond a fixed window	LSTM-based language models
Transformers	Process an entire sequence in parallel via self-attention, allowing every token to condition on every other token directly rather than through a chain of hidden states	BERT, the GPT family

The bigram model above sits at the first row of this table — real LLMs use the third. The practical consequence of the shift to Transformers is that context is no longer bottlenecked through a single fixed-size window or a sequentially-updated hidden state, which is part of why modern models can condition coherently on very long inputs.

4 What Makes a Model “Large”

Three factors are usually meant by “large”:

Parameters — the learned weights in the network. Early GPT-generation models were in the tens to low hundreds of billions of parameters; current frontier models use architectures (commonly mixture-of-experts) where only a fraction of total parameters activate per token, so raw parameter count alone is no longer a reliable proxy for capability across model families.
Context window — how many tokens the model can condition on in a single request. This has grown substantially: from roughly 2K–8K tokens in early GPT-3-era models to context windows in the hundreds of thousands to millions of tokens in current frontier models.
Training corpus — the scale of text (and increasingly code, images, and other modalities) the model was trained on, now typically measured in trillions of tokens.

5 How an LLM Is Built: Three Phases

Training (pretraining). The model repeatedly predicts a masked or next token across a massive unlabeled corpus, adjusting its parameters whenever its prediction is wrong — mechanically the same process demonstrated with the toy bigram model above, just at a vastly larger scale and with a Transformer architecture instead of bigram counts.

Fine-tuning. A pretrained model is further trained on a smaller, curated dataset to specialize its behavior — for instruction-following and conversational behavior in general-purpose assistants, or for a narrower domain such as legal or medical text.

Inference. The trained model is used to generate output for new prompts, with no further weight updates. This is what happens every time a request is sent to a model through an API — which is the subject of the rest of this notebook.

6 Why LLMs Matter in Practice

LLMs are foundation models underlying a wide range of applications: text generation and summarization, translation, question answering over documents, conversational assistants, and code generation and explanation. Their practical value comes from being general-purpose — the same underlying model handles translation, drafting, and code review without separate task-specific training — which is what makes API access to a single model useful across many different applications rather than needing a different specialized model per task.

7 The Current LLM Landscape

Model names and version numbers in this space turn over quickly — training material describing “the current GPT model” or “the current Gemini model” from even a year or two earlier is a common source of outdated code, since API calls reference a specific model name that may since have been retired. As of mid-2026:

Family	Developer	Current generation	Notably superseded
GPT	OpenAI	GPT-5.x family (multiple capability tiers)	GPT-3, GPT-4, GPT-4o — all retired from ChatGPT by early 2026
Claude	Anthropic	Sonnet, Opus, and Haiku tiers at the current generation, alongside a Mythos tier above Opus	Claude 1/2, Claude 3 family
Gemini	Google DeepMind	Gemini 3.x family (Pro and Flash tiers)	Gemini 1.0, Gemini 1.5 — including 1.5 Flash, used throughout older tutorials on this topic
Llama	Meta	Llama 4 (Scout and Maverick generally available; Behemoth remains in training)	Llama 1, 2, 3
DeepSeek	DeepSeek	DeepSeek V4 (Pro and Flash)	DeepSeek V2, V3, V3.1, V3.2, and the R1 reasoning model

Two structural points are worth noting independent of any specific version number. First, the trend across every major lab has been toward multiple capability tiers within a generation (a fast/cheap tier and a higher-capability tier) rather than a single flagship model, so “the current GPT model” is now a family rather than one name. Second, several providers have converged on hybrid or unified reasoning behavior — a single model that can operate in a fast mode or a slower, more deliberate reasoning mode — rather than shipping reasoning as an entirely separate model line, which was still the case in the GPT-o1/o3 and DeepSeek-R1 generation.

8 What Is an LLM API?

An API (Application Programming Interface) is a defined way for one piece of software to request something from another over a network — a client sends a structured request, a server processes it and returns a structured response. An LLM API applies this pattern to a hosted language model: the request typically includes a prompt, a model name, and generation settings (temperature, maximum output length); the response includes the generated text along with metadata such as token counts.

Working with an LLM API in practice means handling four things: authentication (proving the request is authorized, via an API key), request construction (the prompt and generation parameters), the response format (extracting the generated text from a structured response object), and usage accounting (tracking tokens consumed, since most providers bill and rate-limit by token count).

The rest of this section builds a working — if self-contained — version of all four, since this environment has no network access to a real provider’s endpoint. The mechanics demonstrated (key checking, request/response structure, token counting, rate limiting) are exactly what a real API client handles; only the actual language generation is a toy stand-in.

9 Building a Minimal LLM API, End to End

The pieces below assemble into a single call_llm_api() function that behaves like a real hosted API: it checks an API key, tracks a per-key call count against a rate limit, tokenizes the prompt, generates a continuation using the bigram model trained earlier, and returns a response object with a token-usage breakdown — the same shape as a real provider’s response.

import random
import time

# --- a minimal "account" store, standing in for a real provider's auth system ---
VALID_API_KEYS = {"demo-key-abc123"}
_call_log = defaultdict(list)
RATE_LIMIT_PER_MINUTE = 100  # generous default so later demo cells aren't blocked by earlier ones

class AuthenticationError(Exception):
    pass

class RateLimitError(Exception):
    pass

def _check_rate_limit(api_key):
    now = time.time()
    _call_log[api_key] = [t for t in _call_log[api_key] if now - t < 60]
    if len(_call_log[api_key]) >= RATE_LIMIT_PER_MINUTE:
        raise RateLimitError(
            f"Rate limit exceeded: {RATE_LIMIT_PER_MINUTE} calls per minute for this key."
        )
    _call_log[api_key].append(now)

def _last_known_token(tokens, model):
    """Walk backward through the prompt's tokens to find the most recent one
    the model actually saw during training; falls back to  if none did."""
    for tok in reversed(tokens):
        if tok in model:
            return tok
    return ""

def _generate(prompt, max_tokens, temperature, seed):
    rng = random.Random(seed)
    tokens = tokenize(prompt)
    current = _last_known_token(tokens, bigram_model)
    generated = []
    for _ in range(max_tokens):
        probs = next_token_probabilities(bigram_model, current, temperature=temperature)
        if not probs or "" in probs and rng.random() < probs.get("", 0):
            break
        candidates, weights = zip(*probs.items())
        next_tok = rng.choices(candidates, weights=weights, k=1)[0]
        if next_tok == "":
            break
        generated.append(next_tok)
        current = next_tok
    return " ".join(generated)

def call_llm_api(prompt, api_key, model="toy-llm-mini", max_tokens=10, temperature=0.7, seed=0):
    if api_key not in VALID_API_KEYS:
        raise AuthenticationError("Invalid API key.")
    _check_rate_limit(api_key)

    prompt_tokens = len(tokenize(prompt))
    completion_text = _generate(prompt, max_tokens, temperature, seed)
    completion_tokens = len(tokenize(completion_text))

    return {
        "model": model,
        "choices": [{"text": completion_text}],
        "usage": {
            "prompt_tokens": prompt_tokens,
            "completion_tokens": completion_tokens,
            "total_tokens": prompt_tokens + completion_tokens,
        },
    }

response = call_llm_api("I am going to the", api_key="demo-key-abc123", seed=1)
response

{'model': 'toy-llm-mini',
 'choices': [{'text': 'gym today'}],
 'usage': {'prompt_tokens': 5, 'completion_tokens': 2, 'total_tokens': 7}}

Two failure modes worth confirming actually work, since they’re central to using any real API: an invalid key should be rejected, and exceeding the rate limit should raise rather than silently succeed.

# Invalid key
try:
    call_llm_api("Hello", api_key="not-a-real-key")
except AuthenticationError as e:
    print("Caught expected error:", e)

# Rate limit: temporarily tighten the limit and use a dedicated key, so this
# demonstration doesn't consume the quota for "demo-key-abc123" used elsewhere
# in this notebook.
VALID_API_KEYS.add("demo-key-for-rate-limit-test")
rate_limit_test_key = "demo-key-for-rate-limit-test"

_original_limit = RATE_LIMIT_PER_MINUTE
RATE_LIMIT_PER_MINUTE = 3
for i in range(4):
    try:
        call_llm_api("i love eating", api_key=rate_limit_test_key, seed=i)
        print(f"Call {i+1}: succeeded")
    except RateLimitError as e:
        print(f"Call {i+1}: blocked -- {e}")
RATE_LIMIT_PER_MINUTE = _original_limit

Caught expected error: Invalid API key.
Call 1: succeeded
Call 2: succeeded
Call 3: succeeded
Call 4: blocked -- Rate limit exceeded: 3 calls per minute for this key.

Both behave as a real API would: the wrong key is rejected before any generation happens, and the fourth call within the same minute is blocked once the limit is reached, without needing to actually wait or hit a real server to demonstrate it.

9.1 Role-Based Prompting

A prompt that opens by assigning the model a persona or role is a common pattern for steering tone and framing. Since the toy model above only knows the handful of training sentences given to it, its output won’t reflect the requested persona the way a real LLM would — but the request/response mechanics are identical, which is what this section demonstrates.

def build_role_prompt(role, user_input):
    return f"You are {role}. User: {user_input}\nAssistant:"

prompt = build_role_prompt(
    role="a senior data scientist explaining concepts to a general audience",
    user_input="the gym"
)
print("Prompt sent to the API:\n", prompt, "\n")

response = call_llm_api(prompt, api_key="demo-key-abc123", max_tokens=8, seed=2)
print("Response:", response)

Prompt sent to the API:
 You are a senior data scientist explaining concepts to a general audience. User: the gym
Assistant: 

Response: {'model': 'toy-llm-mini', 'choices': [{'text': 'today'}], 'usage': {'prompt_tokens': 16, 'completion_tokens': 1, 'total_tokens': 17}}

9.2 A Chatbot Loop

A terminal chatbot is usually written with a blocking input() call inside a while True: loop, which works interactively but can’t run inside an executed notebook cell (there is no one there to type a response). The loop logic is identical either way; the only change here is replacing live keyboard input with a small pre-written list of turns, so the whole exchange runs and is captured in the output.

def run_chatbot(turns, api_key):
    print("Chatbot session started.\n")
    for user_input in turns:
        print(f"You: {user_input}")
        result = call_llm_api(user_input, api_key=api_key, max_tokens=6,
                               seed=hash(user_input) % 1000)
        bot_text = result["choices"][0]["text"]
        print(f"Bot: {bot_text}\n")
    print("Chat ended.")

scripted_turns = [
    "i am going to the",
    "i love eating",
    "the capital of france",
]

run_chatbot(scripted_turns, api_key="demo-key-abc123")

Chatbot session started.

You: i am going to the
Bot: office

You: i love eating
Bot: salad

You: the capital of france
Bot: is going to the gym

Chat ended.

10 Setting Up the Current Real SDKs

The two client libraries below are the current, verified-importable SDKs for OpenAI and Google’s Gemini API. Constructing a client doesn’t require network access (it only stores configuration), so that step is executed directly; the actual generation call is shown immediately after as reference code, since it requires a real API key and a live network connection to the provider, neither of which this environment has.

One correction relative to older material on this topic: google.generativeai — the package used throughout many existing Gemini tutorials, including earlier versions of this one — is fully deprecated. Google archived that repository and consolidated all Gemini access into a single unified SDK, google-genai, imported as from google import genai. Code written against the old package will not receive updates and, per Google’s own deprecation notice, should be migrated.

# Current Gemini SDK -- client construction only, no network call
from google import genai

gemini_client = genai.Client(api_key="replace-with-a-real-key")
print("Gemini client ready:", type(gemini_client).__name__)

# The actual generation call (reference only -- requires a real key and network access):
#
# response = gemini_client.models.generate_content(
#     model="gemini-3.5-flash",
#     contents="Tell me a fun fact about space.",
# )
# print(response.text)

Gemini client ready: Client

# Current OpenAI SDK -- client construction only, no network call
from openai import OpenAI

openai_client = OpenAI(api_key="replace-with-a-real-key")
print("OpenAI client ready:", type(openai_client).__name__)

# The actual generation call (reference only -- requires a real key and network access):
#
# response = openai_client.chat.completions.create(
#     model="gpt-5.6-terra",
#     messages=[{"role": "user", "content": "Tell me a fun fact about space."}],
# )
# print(response.choices[0].message.content)

OpenAI client ready: OpenAI

11 Wrapping the API in a Simple Web Interface

Gradio turns a Python function into a browser-based UI with minimal code. gr.Interface can be constructed and inspected without calling .launch() — launching starts a live local server, which isn’t appropriate inside an executed notebook cell, but constructing the interface and calling its underlying function directly confirms the wiring is correct.

import gradio as gr

def chatbot_interface(user_input):
    result = call_llm_api(user_input, api_key="demo-key-abc123", max_tokens=8,
                           seed=hash(user_input) % 1000)
    return result["choices"][0]["text"]

iface = gr.Interface(
    fn=chatbot_interface,
    inputs="text",
    outputs="text",
    title="Toy LLM API Chatbot",
    description="Type a prompt; the response comes from the toy bigram model built earlier.",
)

print("Interface constructed:", type(iface).__name__)
print("Direct call through the interface function:", iface.fn("i am going to the"))

# To actually serve this in a browser: iface.launch()

/opt/homebrew/Cellar/jupyterlab/4.6.1/libexec/lib/python3.14/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Interface constructed: Interface
Direct call through the interface function: office

12 Best Practices

Never hard-code an API key in source. Load it from an environment variable or a .env file excluded from version control, as in the python-dotenv pattern.
Handle authentication and rate-limit errors explicitly, as demonstrated above — a production integration should catch these rather than let the whole application crash on a transient limit.
Track token usage from the response object, not by estimating it — the usage field in a real response is the authoritative count for billing and context-window purposes.
Pin a specific model version rather than an alias that silently repoints to a newer model, if reproducible behavior matters for your use case.
Separate the model-calling code from the UI layer, as done here (the call_llm_api() function is identical whether it’s called directly, from a scripted loop, or from the Gradio interface).
Verify current model names before deploying, given how quickly they change — this notebook’s own landscape table will itself be out of date within a year or so.

13 Limitations

Hallucination. A model can generate fluent, confident, and factually wrong output; this is a property of how generation works, not a bug specific to any one provider.
No persistent memory by default. Each API call is stateless unless the calling application resends prior context, exactly as demonstrated by the fact that call_llm_api() above has no memory of previous calls on its own.
No grounded understanding. A language model manipulates learned statistical patterns over tokens; it does not have verified knowledge of facts in the way a lookup against a trusted database would.
Context window limits. Very long conversations or documents can exceed what a model can attend to in a single request, requiring truncation, summarization, or retrieval (as covered in a companion notebook on LangChain and RAG).
Cost and token limits. API usage is typically billed per token, and usage accounting (as built above) is necessary to keep this under control.
Data privacy. Prompts sent to a third-party API may be logged according to that provider’s data policy — sensitive information warrants caution before inclusion in a prompt.
Bias. A model can reproduce patterns present in its training data, including skewed or unbalanced representations of certain topics or groups.

14 Summary

A language model estimates a probability distribution over the next token given preceding context, and an LLM is this same mechanism at large scale, built on a Transformer architecture and trained in three phases: pretraining on a large corpus, fine-tuning toward instruction-following behavior, and inference on new prompts at request time. All three were demonstrated computationally above, at small scale, with a trained bigram model standing in for a full Transformer.

An LLM API exposes a hosted model over a standard request/response interface, and working with one in practice comes down to four concerns: authentication, request construction, response parsing, and usage accounting — all four were built and verified directly in this notebook via a self-contained toy API, since no network access to a real provider was available. The current SDKs for both major hosted providers (OpenAI, Google’s unified google-genai) were verified as correctly importable and constructible, with the actual generation calls documented as reference code for use with a real API key. The model landscape itself — which specific model name to put in that call — is the part of this material most likely to go stale fastest, and was flagged as such rather than treated as a fixed fact.

15 References

Vaswani et al., “Attention Is All You Need” (2017) – arXiv:1706.03762
Brown et al., “Language Models are Few-Shot Learners” (2020) – arXiv:2005.14165
Chowdhery et al., “PaLM: Scaling Language Modeling with Pathways” (2022) – arXiv:2204.02311
OpenAI API documentation – platform.openai.com/docs
Google Gen AI SDK documentation – ai.google.dev/gemini-api/docs

A note on the Gemini 1.5 technical report. Earlier material on this topic points to Google’s Gemini 1.5 technical report as reference documentation. That generation has since been superseded by Gemini 3.x, and the gemini-1.5-flash model name used throughout older tutorials (including the walkthrough this notebook is based on) no longer reflects the current model lineup. The Gemini API documentation link above always resolves to current model information rather than a fixed historical snapshot.

16 Try It Yourself

Task 1: Call an LLM API for text generation. Using either the toy call_llm_api() built above, or a real provider’s SDK once you have a key, send the prompt "the capital of france" and inspect both the generated text and the token usage in the response.

Task 2: Build a small prompt playground. Write a function that accepts a persona and a user message, builds a role-based prompt with build_role_prompt(), sends it through call_llm_api(), and returns the response text — then try it with two different personas on the same input and compare tone and framing (in a real model; the toy model here won’t vary by persona, since it has no notion of one).

16.1 Solutions

Task 1

result = call_llm_api("the capital of france", api_key="demo-key-abc123", max_tokens=6, seed=7)
print("Generated text:", result["choices"][0]["text"])
print("Token usage:", result["usage"])

Generated text: is paris
Token usage: {'prompt_tokens': 4, 'completion_tokens': 2, 'total_tokens': 6}

The model correctly continues toward “paris” (the only completion it ever saw following “the capital of france” during training), and the usage dictionary reports 4 prompt tokens (the, capital, of, france) plus however many completion tokens were generated — confirming the token-accounting path works end to end, not just the generation path.

Task 2

def prompt_playground(persona, user_message, api_key="demo-key-abc123", seed=0):
    prompt = build_role_prompt(persona, user_message)
    result = call_llm_api(prompt, api_key=api_key, max_tokens=8, seed=seed)
    return prompt, result["choices"][0]["text"]

personas = [
    "a formal technical writer",
    "a casual, enthusiastic tutor",
]

for i, persona in enumerate(personas):
    prompt, output = prompt_playground(persona, "i love eating", seed=i)
    print(f"Persona: {persona}")
    print(f"Prompt sent: {prompt}")
    print(f"Response:    {output}\n")

Persona: a formal technical writer
Prompt sent: You are a formal technical writer. User: i love eating
Assistant:
Response:    salad

Persona: a casual, enthusiastic tutor
Prompt sent: You are a casual, enthusiastic tutor. User: i love eating
Assistant:
Response:    pizza for dinner

As expected, the toy model’s output doesn’t actually shift with persona — it has no representation of tone or formality, only bigram counts from nine training sentences — but the prompt construction and API call path is exactly what would carry a persona’s influence through to a real model’s output. Swapping call_llm_api() for openai_client.chat.completions.create() or gemini_client.models.generate_content() with the same build_role_prompt() output is the only change needed to run this against a real model.

t-test or ANOVA? Choosing the Right Test for Before/After Treatment Across Multiple Groups

Nivedita — Tue, 04 Aug 2026 08:05:30 GMT

1 Introduction

A question that comes up constantly in applied statistics: there are several groups — say, three treatment arms — and each subject is measured before and after treatment. Should the comparison use a t-test or an ANOVA?

This question, as posed, is missing information. The correct test depends on two things simultaneously: the number of groups, and the number of time points measured within each subject. Misjudging either one produces a test that is underpowered, statistically invalid, or answering a different question than the one intended.

This tutorial develops the decision logic, demonstrates why naive approaches fail, and works through a complete example using a mixed-design (repeated-measures) ANOVA — the test that fits a multi-group before/after design.

2 What a t-test Can and Cannot Test

A t-test compares two means. Two variants are relevant to this design.

An independent-samples t-test compares two separate groups measured once each — for example, a treatment group versus a control group, both assessed after treatment. The two sets of observations arise from different subjects.

A paired t-test compares two measurements taken from the same subjects — for example, a biomarker measured before and after treatment within a single group. Pairing removes between-subject variability from the comparison, which increases statistical power relative to treating the two measurements as independent.

Q: Why can’t a single t-test handle a multi-group before/after design directly? A: Each t-test variant models exactly one source of variation — either differences between groups or differences between time points — but not both together. A before/after study with three treatment arms contains both sources of variation at once, and neither t-test variant has a mechanism for modeling their combination.

3 Why Not Run Several t-tests?

A common workaround is to run a paired t-test separately within each group, or an independent t-test at each time point, and compare the pattern of results informally. This approach fails for two reasons.

First, running multiple significance tests on related data inflates the family-wise Type I error rate, since each additional test carries its own probability of a false positive. With three groups tested separately, the effective error rate across the family of tests exceeds the nominal 0.05 threshold intended for a single test.

Second, and more fundamentally, separate t-tests cannot test the question a multi-group before/after design is built to answer: did the treatment effect — the before-to-after change — differ across groups? A set of individual paired t-tests indicates whether each group changed on its own, but provides no formal test of whether the magnitude of change differed between groups. That comparison is the group × time interaction, and testing it requires a model that includes both factors jointly.

4 The Mixed-Design ANOVA

A mixed-design ANOVA (also termed a between-within ANOVA) models two factors simultaneously:

Time (or condition) is a within-subjects factor: before and after values come from the same subjects and are therefore correlated.
Group is a between-subjects factor: treatment arms consist of different, independent subjects.

The model partitions variance into three components: a main effect of time (whether values changed overall), a main effect of group (whether groups differ overall), and a group × time interaction (whether the magnitude of change depended on group membership).

Q: Which of these three effects answers the original research question? A: In a treatment-comparison study, the interaction term is typically the effect of interest. A significant interaction indicates that groups did not all change by the same amount, which is the statistical signature of a treatment effect that differs from whatever change occurred in the comparison arms.

5 Decision Guide

Design	Test
Two groups, one time point	Independent-samples t-test
One group, two time points	Paired t-test
Two groups, two time points (before/after)	Mixed ANOVA, or a t-test on change scores
More than two groups, two time points	Mixed ANOVA
More than two time points, with or without multiple groups	Repeated-measures or mixed ANOVA

6 Worked Example: A Biomarker Across Three Groups

Consider three groups — A, B, and C — with a biomarker recorded for each subject under two conditions: control and treatment. This is a 3 × 2 mixed design: group is between-subjects, condition is within-subjects, since each subject contributes both a control and a treatment value.

The simulation below generates 15 subjects per group, with the treatment effect itself set to differ by group: group A shows a minimal response, group B a moderate response, and group C a strong response. This is the pattern a mixed ANOVA is designed to detect via the group × condition interaction.

set.seed(42)
n_per_group <- 15  # subjects per group

# Simulate a biomarker with a group-specific baseline and a group-specific
# treatment response (this is what creates a group x condition interaction)
simulate_group <- function(group_id, baseline, effect) {
  subject   <- paste0(group_id, "_", seq_len(n_per_group))
  control   <- rnorm(n_per_group, mean = baseline, sd = 5)
  treatment <- control + rnorm(n_per_group, mean = effect, sd = 3)
  data.frame(
    subject   = rep(subject, 2),
    group     = group_id,
    condition = rep(c("control", "treatment"), each = n_per_group),
    biomarker = c(control, treatment)
  )
}

df <- rbind(
  simulate_group("A", baseline = 50, effect = 2),   # minimal response
  simulate_group("B", baseline = 50, effect = 8),   # moderate response
  simulate_group("C", baseline = 50, effect = 15)   # strong response
)

df$group     <- factor(df$group)
df$condition <- factor(df$condition, levels = c("control", "treatment"))
df$subject   <- factor(df$subject)

head(df, 6)

A data.frame: 6 × 4
	subject	group	condition	biomarker

1	A_1	A	control	56.85479
2	A_2	A	control	47.17651
3	A_3	A	control	51.81564
4	A_4	A	control	53.16431
5	A_5	A	control	52.02134
6	A_6	A	control	49.46938

aggregate(biomarker ~ group + condition, data = df, mean)

A data.frame: 6 × 3
group	condition	biomarker

A	control	52.42117
B	control	48.28969
C	control	51.12821
A	treatment	53.37999
B	treatment	56.58443
C	treatment	66.59590

Group A moves from approximately 52.4 to 53.4 (a minimal shift), group B moves from approximately 48.3 to 56.6, and group C moves from approximately 51.1 to 66.6. The response grows across groups, as specified in the simulation.

# Mixed ANOVA: condition is within-subjects (each subject has a control AND
# a treatment value), group is between-subjects. Error(subject/condition)
# tells R that condition is nested within subject.
model <- aov(biomarker ~ group * condition + Error(subject / condition), data = df)
summary(model)


Error: subject
          Df Sum Sq Mean Sq F value  Pr(>F)   
group      2  770.3   385.2   7.431 0.00173 **
Residuals 42 2177.0    51.8                   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Error: subject:condition
                Df Sum Sq Mean Sq F value   Pr(>F)    
condition        1 1527.8  1527.8  279.79  < 2e-16 ***
group:condition  2  789.4   394.7   72.28 2.52e-14 ***
Residuals       42  229.4     5.5                     
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

The output contains two error strata, corresponding to the between-subjects and within-subjects portions of the design.

The subject stratum tests the main effect of group (): the three groups differ overall, averaging control and treatment together.

The subject:condition stratum tests the main effect of condition () and, critically, the group × condition interaction ().

Q: Which result answers the original question — did treatment work differently by group? A: The interaction term. Its significance indicates that the magnitude of the control-to-treatment change is not the same across groups A, B, and C. A significant main effect of condition alone would only indicate that biomarker levels changed on average; the interaction indicates that the treatment effect itself depends on group membership. Three separate paired t-tests, one per group, would indicate whether each group changed individually, but would provide no formal test of whether those three changes differed from one another — which is ordinarily the scientific question of interest in a treatment-comparison study.

The table of F-statistics makes the interaction’s significance clear, but doesn’t show what that interaction actually looks like. The plot below does: each line traces one group’s mean biomarker level from control to treatment, with error bars showing the standard error of the mean. A mixed ANOVA with no interaction would produce three roughly parallel lines — all groups shifting by about the same amount. What the simulation shows instead is divergence: group C’s line rises steeply, group B’s rises more modestly, and group A’s is nearly flat.

# Group means and standard errors by condition, for the interaction plot
summary_df <- aggregate(biomarker ~ group + condition, data = df,
                         FUN = function(x) c(mean = mean(x), se = sd(x) / sqrt(length(x))))
summary_df <- do.call(data.frame, summary_df)
names(summary_df) <- c("group", "condition", "mean", "se")
summary_df

A data.frame: 6 × 4
group	condition	mean	se

A	control	52.42117	1.323583
B	control	48.28969	1.286907
C	control	51.12821	1.045192
A	treatment	53.37999	1.793629
B	treatment	56.58443	1.587070
C	treatment	66.59590	1.105771

group_colors <- c(A = "#4C72B0", B = "#DD8452", C = "#55A868")

plot(NULL, xlim = c(0.8, 2.2),
     ylim = range(c(summary_df$mean - summary_df$se, summary_df$mean + summary_df$se)) + c(-2, 2),
     xaxt = "n", xlab = "", ylab = "Biomarker level (mean \u00B1 SE)",
     main = "Treatment effect by group: a group x condition interaction")
axis(1, at = c(1, 2), labels = c("Control", "Treatment"))

for (g in levels(summary_df$group)) {
  sub <- summary_df[summary_df$group == g, ]
  sub <- sub[order(sub$condition), ]
  x <- c(1, 2)
  lines(x, sub$mean, col = group_colors[g], lwd = 2.5, type = "b", pch = 19, cex = 1.3)
  arrows(x, sub$mean - sub$se, x, sub$mean + sub$se, angle = 90, code = 3,
         length = 0.05, col = group_colors[g])
}

legend("topleft", legend = levels(summary_df$group), col = group_colors, lwd = 2.5, pch = 19,
       title = "Group", bty = "n")

Non-parallel lines are the visual signature of an interaction — and the degree of non-parallelism here (group C pulling sharply away from A and B) is exactly what produced the very small p-value on the group:condition term above. This plot and the ANOVA table are two views of the same result: one gives the formal test, the other shows what the tested pattern actually looks like in the data, which is often the more persuasive figure for a paper or presentation.

7 Assumptions and Alternatives

The mixed-design ANOVA carries the standard ANOVA assumptions of normally distributed residuals and homogeneity of variance across groups, together with sphericity for the within-subjects factor when more than two time points are present. Sphericity is automatically satisfied with only two levels, as in a single before/after comparison.

Where these assumptions are doubtful, two alternatives are available. A linear mixed-effects model, fit with lme4 or nlme, accommodates unbalanced data and missing observations more gracefully than classical ANOVA while producing an equivalent interaction test. For small samples or markedly non-normal outcomes, a nonparametric alternative such as the Friedman test, combined with rank-based group comparisons, can be used, though these approaches test somewhat different hypotheses and offer less flexibility for interaction effects.

8 Conclusion

“t-test or ANOVA” is not, by itself, a well-posed question. The determining factor is the number of independent sources of variation present in the design. A before/after comparison within a single group constitutes one factor; a comparison across multiple groups introduces a second. When both are present together, as in a multi-group treatment study, the appropriate test is one built to model two factors and their interaction jointly — the mixed-design ANOVA. Substituting a t-test in this setting does not merely reduce statistical power; it silently discards the interaction test that the design was intended to answer.

Generative AI and Prompt Engineering

Nivedita — Mon, 03 Aug 2026 21:00:00 GMT

A weak instruction and an engineered one reach the same model, but persona, context, example, and output format push the engineered version toward a single high-probability answer instead of a diffuse guess.

1 Introduction

Generative AI denotes a class of systems that produce new content — text, images, audio, video, or code — rather than only classifying or scoring existing data. A model learns the statistical structure of a training corpus and then samples new instances consistent with that structure, as opposed to mapping fixed inputs onto a fixed set of output labels.

This tutorial covers the material in three layers. First, the conceptual layer: what generative models are, how large language models (LLMs) are trained, and how this differs from conventional machine learning. Second, the mechanical layer: how a prompt actually becomes an output, via token-level sampling — treated computationally, in R, using a toy model, so the abstraction is not left as a black box. Third, the applied layer: prompt engineering as a design discipline, common prompting strategies, and a survey of the current tool landscape, closing with practical guidance and known limitations.

Model and product names change quickly in this field. Where a specific product is named below, the description reflects its status as of mid-2026; several products referenced in earlier training material on this topic (GPT-3, GPT-4, Google Bard, DALL-E, Amazon CodeWhisperer) have since been superseded or renamed, and this is noted explicitly where relevant.

2 What Generative AI Produces

Three domains illustrate the breadth of the category:

Text. Current-generation LLMs — OpenAI’s GPT-5 family, Anthropic’s Claude models, Google’s Gemini, and Meta’s open-weight Llama models — generate essays, code, and dialogue conditioned on a prompt.

Images. Diffusion and autoregressive image models (Midjourney, Stable Diffusion, OpenAI’s GPT Image line, Adobe Firefly) convert a text description into a corresponding image.

Audio and video. Tools such as Suno (music), ElevenLabs (voice), and Runway (video) apply the same generate-from-description paradigm to other modalities.

The common thread is not the modality but the training objective: the model is optimized to produce plausible continuations or reconstructions of its training distribution, not to output a predetermined answer.

3 Generative AI versus Conventional Machine Learning

The distinction is best drawn along five axes rather than treated as a single sharp boundary — a fraud-detection classifier and an LLM are both “machine learning” in the broad sense, but they differ systematically in what they are built to do.

ml_vs_genai <- data.frame(
  aspect       = c("Objective", "Typical output", "Typical training data",
                    "Dominant learning paradigm", "Common architectures", "Example task"),
  machine_learning = c("Predict a label or value from input features",
                        "Class label, score, or continuous value",
                        "Structured / tabular",
                        "Supervised (labeled data)",
                        "Linear/logistic regression, tree ensembles, SVM",
                        "Flagging a fraudulent transaction"),
  generative_ai = c("Produce a new instance consistent with a learned distribution",
                     "Text, image, audio, or video",
                     "Large unstructured corpora (text, image, audio)",
                     "Self-supervised pretraining, often followed by fine-tuning",
                     "Transformer-based architectures (decoder-only LLMs, diffusion models)",
                     "Drafting a paragraph explaining a result"),
  stringsAsFactors = FALSE
)

print(ml_vs_genai, row.names = FALSE)

                     aspect                                machine_learning
                  Objective    Predict a label or value from input features
             Typical output         Class label, score, or continuous value
      Typical training data                            Structured / tabular
 Dominant learning paradigm                       Supervised (labeled data)
       Common architectures Linear/logistic regression, tree ensembles, SVM
               Example task               Flagging a fraudulent transaction
                                                         generative_ai
         Produce a new instance consistent with a learned distribution
                                          Text, image, audio, or video
                       Large unstructured corpora (text, image, audio)
            Self-supervised pretraining, often followed by fine-tuning
 Transformer-based architectures (decoder-only LLMs, diffusion models)
                              Drafting a paragraph explaining a result

4 Large Language Models

An LLM is trained to predict the next token in a sequence given the tokens that precede it. Applied at scale over a large text corpus, this single objective — next-token prediction — is sufficient to produce a model that captures grammar, factual associations, and long-range discourse structure, without ever being given an explicit label for any of those properties. This is what makes the training self-supervised: the “label” at each position is simply the next token in the existing text, so no separate annotation step is required.

Pretraining establishes the model’s general language competence over a broad corpus. Fine-tuning (and closely related techniques such as instruction tuning and reinforcement learning from human feedback) subsequently adapts a pretrained model toward following instructions reliably or performing well in a narrower domain — for instance, adapting a general-purpose model toward clinical or legal terminology.

Why does the distinction between pretraining and fine-tuning matter practically? A pretrained-only model tends to continue text rather than answer a question — asked “What is the capital of Finland?”, it may generate a plausible-looking follow-up question rather than a direct answer. Instruction tuning is what makes a model behave like an assistant that responds directly to a request, which is the behavior most users interact with in products such as ChatGPT or Claude.

4.1 Why LLMs matter in practice

Communication and translation — real-time translation and multilingual content generation, without a separately trained model per language pair.
Scalable content production — drafting at a volume and speed not practical for manual writing alone, subject to the fact-checking caveat discussed later.
Personalization — tailoring phrasing, reading level, or emphasis to a stated audience.
Routine-task automation — drafting emails, summarizing long documents, generating boilerplate code.
Accessibility — text-to-speech and speech-to-text pipelines that lower barriers for users with visual or motor impairments.

5 Prompt Engineering as a Design Problem

A prompt can be decomposed into a small number of largely independent components: a persona (the role the model should adopt), the instruction itself, contextual background, optional worked examples, and an output-format specification. Building these programmatically, rather than writing each prompt by hand, makes explicit what changes between an under-specified request and a well-constructed one — and is closer to a controlled comparison, in the experimental-design sense, than to freeform writing.

build_prompt <- function(instruction, persona = NULL, context = NULL,
                          examples = NULL, output_format = NULL) {
  parts <- character(0)
  if (!is.null(persona))       parts <- c(parts, sprintf("Act as %s.", persona))
  if (!is.null(context))       parts <- c(parts, sprintf("Context: %s", context))
  if (!is.null(examples))      parts <- c(parts, sprintf("Examples:\n%s", paste(examples, collapse = "\n")))
  parts <- c(parts, sprintf("Task: %s", instruction))
  if (!is.null(output_format)) parts <- c(parts, sprintf("Respond as: %s", output_format))
  paste(parts, collapse = "\n\n")
}

weak_prompt <- build_prompt(
  instruction = "Explain what a Manhattan plot shows."
)

engineered_prompt <- build_prompt(
  instruction = "Explain what a Manhattan plot shows.",
  persona = "a statistical genetics instructor addressing first-year PhD students",
  context = "The audience has taken introductory statistics but has not yet covered GWAS.",
  output_format = "three short paragraphs, ending with one sentence on how a significance threshold is chosen"
)

cat(weak_prompt, "\n\n---\n\n", engineered_prompt, sep = "")

Task: Explain what a Manhattan plot shows.

---

Act as a statistical genetics instructor addressing first-year PhD students.

Context: The audience has taken introductory statistics but has not yet covered GWAS.

Task: Explain what a Manhattan plot shows.

Respond as: three short paragraphs, ending with one sentence on how a significance threshold is chosen

6 From Prompt to Output: Simulating Token Sampling

Given a fixed prompt, an LLM does not return a single deterministic string. At each position it assigns a score (logit) to every candidate next token, converts these scores into a probability distribution via the softmax function, and samples from that distribution. The temperature parameter rescales the logits before this conversion: low temperature sharpens the distribution toward the highest-scoring token (output closer to deterministic), while high temperature flattens it (more variable output). This is exactly why the same prompt, sent twice, can produce two different — but each individually plausible — responses.

This mechanism can be simulated directly with a toy vocabulary, without calling any external model.

softmax <- function(logits, temperature = 1) {
  scaled <- logits / temperature
  exp_scaled <- exp(scaled - max(scaled))  # numerically stable
  exp_scaled / sum(exp_scaled)
}

vocab  <- c("significant", "suggestive", "negligible", "confounded", "spurious")
logits <- c(4.2, 3.1, 0.8, 0.5, 0.2)

temperatures <- c(0.2, 0.7, 1.0, 1.5)
prob_table <- sapply(temperatures, function(t) softmax(logits, t))
colnames(prob_table) <- paste0("T=", temperatures)
rownames(prob_table) <- vocab

round(prob_table, 3)

A matrix: 5 × 4 of type dbl
	T=0.2	T=0.7	T=1	T=1.5
significant	0.996	0.817	0.710	0.575
suggestive	0.004	0.170	0.236	0.276
negligible	0.000	0.006	0.024	0.060
confounded	0.000	0.004	0.018	0.049
spurious	0.000	0.003	0.013	0.040

The entropy of each distribution quantifies this concentration effect directly: low temperature yields low entropy (the model is close to deterministic), and high temperature moves the distribution toward uniform over the vocabulary.

entropy <- function(p) -sum(p * log(p))

data.frame(
  temperature = temperatures,
  entropy     = round(apply(prob_table, 2, entropy), 3),
  top_token   = vocab[apply(prob_table, 2, which.max)]
)

A data.frame: 4 × 3
	temperature	entropy	top_token

T=0.2	0.2	0.026	significant
T=0.7	0.7	0.537	significant
T=1	1.0	0.800	significant
T=1.5	1.5	1.118	significant

7 Zero-Shot, Few-Shot, and Chain-of-Thought Prompting

The three standard prompting strategies differ in how much structure is supplied before the task itself.

Zero-shot prompting — the model performs the task with no worked examples, relying entirely on knowledge acquired during training. Well suited to translation, general Q&A, and straightforward classification.

Few-shot prompting — the prompt includes several input–output examples before the actual task, which anchors the model to a specific desired format or style. Useful when the output must follow a particular structure — consistent-style content generation, pattern-based classification, structured summarization.

Chain-of-thought (CoT) prompting — the model is guided through explicit intermediate reasoning steps before producing a final answer. This is the most reliable strategy for multi-step arithmetic, logic, and other tasks where an unstructured single-shot answer is prone to skipping a step.

These can be built as three thin wrappers around the same build_prompt() function defined above, which makes the structural difference between the strategies explicit rather than a matter of writing style.

zero_shot <- build_prompt(
  instruction = "Classify the following variant consequence as high, moderate, or low impact: stop_gained"
)

few_shot <- build_prompt(
  instruction = "Classify the following variant consequence as high, moderate, or low impact: stop_gained",
  examples = c(
    "missense_variant -> moderate impact",
    "synonymous_variant -> low impact",
    "frameshift_variant -> high impact"
  )
)

chain_of_thought <- build_prompt(
  instruction = "Classify the following variant consequence as high, moderate, or low impact: stop_gained",
  context = paste(
    "Reason step by step: (1) identify what the consequence term means at the",
    "protein level, (2) assess whether it truncates or disrupts the reading frame,",
    "(3) map that assessment to an impact category, (4) state the final classification."
  )
)

cat("ZERO-SHOT\n", zero_shot, "\n\n",
    "FEW-SHOT\n", few_shot, "\n\n",
    "CHAIN-OF-THOUGHT\n", chain_of_thought, sep = "")

ZERO-SHOT
Task: Classify the following variant consequence as high, moderate, or low impact: stop_gained

FEW-SHOT
Examples:
missense_variant -> moderate impact
synonymous_variant -> low impact
frameshift_variant -> high impact

Task: Classify the following variant consequence as high, moderate, or low impact: stop_gained

CHAIN-OF-THOUGHT
Context: Reason step by step: (1) identify what the consequence term means at the protein level, (2) assess whether it truncates or disrupts the reading frame, (3) map that assessment to an impact category, (4) state the final classification.

Task: Classify the following variant consequence as high, moderate, or low impact: stop_gained

8 A Heuristic for Prompt Specificity

The qualitative distinction between a “weak” and a “well-engineered” prompt can be given a rough quantitative proxy: count the structural elements present — an explicit role, contextual background, at least one worked example, and an explicit output format. This is not a validated metric, but it makes concrete what “well-engineered” is standing in for, and gives a way to compare prompts before sending either to a model.

specificity_score <- function(prompt) {
  has_role    <- grepl("^Act as", prompt)
  has_context <- grepl("Context:|Reason step by step", prompt)
  has_example <- grepl("Examples:|->", prompt)
  has_format  <- grepl("Respond as:", prompt)
  word_count  <- lengths(strsplit(prompt, "\\s+"))

  data.frame(
    role = has_role, context = has_context,
    example = has_example, output_format = has_format,
    word_count = word_count,
    structure_score = has_role + has_context + has_example + has_format
  )
}

prompts <- list(weak = weak_prompt, engineered = engineered_prompt,
                 zero_shot = zero_shot, few_shot = few_shot, cot = chain_of_thought)

do.call(rbind, lapply(prompts, specificity_score))

A data.frame: 5 × 6
	role	context	example	output_format	word_count	structure_score

weak	FALSE	FALSE	FALSE	FALSE	7	0
engineered	TRUE	TRUE	FALSE	TRUE	46	3
zero_shot	FALSE	FALSE	FALSE	FALSE	13	0
few_shot	FALSE	FALSE	TRUE	FALSE	26	1
cot	FALSE	TRUE	FALSE	FALSE	52	1

The structure_score column separates the five prompts cleanly: the weak prompt scores zero, the engineered version scores three, and the three named strategies each add exactly one structural element beyond a bare instruction — context for chain-of-thought, worked examples for few-shot, nothing beyond the instruction itself for zero-shot.

9 Applying LLMs to Academic and Professional Tasks

Current general-purpose LLMs support four broad capabilities relevant to this kind of work: natural language understanding (following multi-part instructions), text generation (drafting essays, reports, and code), information retrieval across general topics (subject to the currency limitations discussed later), and step-by-step problem solving.

Applied by subject, three patterns recur:

Quantitative subjects (mathematics, statistics) — worked, stepwise solutions and clarification of terminology; chain-of-thought prompting is particularly effective here.
Conceptual subjects (natural sciences) — explanation at a specified level of background knowledge, and generation of illustrative examples or experiment ideas.
Interpretive subjects (literature, argumentation) — thematic analysis and drafting assistance, where the underlying claims still require the user’s own verification and judgment.

At the workflow level rather than the subject level, three uses account for most practical value: simplifying research (summarizing long material into key points), generating ideas (structured brainstorming and outlining), and producing explanations calibrated to a stated audience (via analogy or simplification).

9.1 A worked example: reusing `build_prompt()` for a recurring task

A common pattern in practice is to fix most of a prompt’s structure in advance and defer one piece of task-specific data to a follow-up turn. This reuses the same build_prompt() constructor already defined, applied to a resume-drafting task with an explicit “wait for further input” instruction.

resume_prompt <- build_prompt(
  instruction = paste(
    "Draft a resume for a Data Science role with 4 years of experience across",
    "two prior positions. Use standard sections: Contact Information,",
    "Professional Summary, Work Experience, Education, Skills, Certifications.",
    "Wait for a job description to be supplied before finalizing content."
  ),
  persona = "a resume writer specializing in data science roles",
  context = "The candidate has 4 years of experience across 2 companies.",
  output_format = "a standard resume layout with the sections listed above"
)

cat(resume_prompt)

Act as a resume writer specializing in data science roles.

Context: The candidate has 4 years of experience across 2 companies.

Task: Draft a resume for a Data Science role with 4 years of experience across two prior positions. Use standard sections: Contact Information, Professional Summary, Work Experience, Education, Skills, Certifications. Wait for a job description to be supplied before finalizing content.

Respond as: a standard resume layout with the sections listed above

This pattern — templating the reusable structure while leaving the variable input (here, the job description) to be supplied afterward — generalizes to any workflow where the same task recurs with different data each time, for example generating a plain-language summary of a coloc.abf result for different gene–trait pairs.

10 The Generative AI Tool Landscape

The tool landscape in this field moves quickly, so this section is organized by category, with model or product names given as illustrative examples current as of mid-2026 rather than a fixed catalog. A few notable changes relative to material from a year or two prior: Google’s Bard was renamed Gemini in February 2024; Amazon’s CodeWhisperer was rebranded as Amazon Q Developer in April 2024; OpenAI retired the DALL-E brand in 2026, consolidating image generation under the GPT Image line; and video models such as Runway have moved through several major generations (Gen-2 → Gen-4.5) since earlier overviews of this space were written.

tool_landscape <- data.frame(
  category = c(
    "Text generation", "Text generation", "Text generation", "Text generation",
    "Image generation", "Image generation", "Image generation",
    "Code generation", "Code generation", "Code generation",
    "Audio & music", "Audio & music",
    "Video generation", "Video generation",
    "Multimodal", "Multimodal",
    "Open-weight / local"
  ),
  example_tool = c(
    "ChatGPT (OpenAI)", "Claude (Anthropic)", "Gemini (Google)", "Llama (Meta, open-weight)",
    "Midjourney", "Stable Diffusion (Stability AI)", "GPT Image (OpenAI)",
    "GitHub Copilot", "Amazon Q Developer", "StarCoder (Hugging Face / BigCode)",
    "Suno (music)", "ElevenLabs (voice)",
    "Runway", "Synthesia",
    "Claude / GPT / Gemini (vision-enabled)", "Multimodal open models (e.g. IDEFICS-style)",
    "Ollama / LM Studio for local inference"
  ),
  primary_use = c(
    "General-purpose conversational assistant", "General-purpose assistant, long-document handling",
    "Conversational assistant integrated with Google services", "Open-weight models for self-hosted or fine-tuned use",
    "Stylized, artistic image generation", "Open-source, locally runnable image generation",
    "Native image generation integrated into a chat assistant",
    "In-editor code completion and generation", "AWS-integrated coding and cloud-resource assistant",
    "Open-source code-focused language model",
    "Text-to-song generation", "Realistic voice synthesis and cloning",
    "Text- and image-conditioned video generation", "Script-to-video with avatar presenters",
    "Combined text, image, and document reasoning", "Open research models combining vision and language",
    "Running open-weight LLMs on local hardware"
  ),
  stringsAsFactors = FALSE
)

print(tool_landscape, row.names = FALSE)

            category                                example_tool
     Text generation                            ChatGPT (OpenAI)
     Text generation                          Claude (Anthropic)
     Text generation                             Gemini (Google)
     Text generation                   Llama (Meta, open-weight)
    Image generation                                  Midjourney
    Image generation             Stable Diffusion (Stability AI)
    Image generation                          GPT Image (OpenAI)
     Code generation                              GitHub Copilot
     Code generation                          Amazon Q Developer
     Code generation          StarCoder (Hugging Face / BigCode)
       Audio & music                                Suno (music)
       Audio & music                          ElevenLabs (voice)
    Video generation                                      Runway
    Video generation                                   Synthesia
          Multimodal      Claude / GPT / Gemini (vision-enabled)
          Multimodal Multimodal open models (e.g. IDEFICS-style)
 Open-weight / local      Ollama / LM Studio for local inference
                                              primary_use
                 General-purpose conversational assistant
        General-purpose assistant, long-document handling
 Conversational assistant integrated with Google services
     Open-weight models for self-hosted or fine-tuned use
                      Stylized, artistic image generation
           Open-source, locally runnable image generation
 Native image generation integrated into a chat assistant
                 In-editor code completion and generation
       AWS-integrated coding and cloud-resource assistant
                  Open-source code-focused language model
                                  Text-to-song generation
                    Realistic voice synthesis and cloning
             Text- and image-conditioned video generation
                   Script-to-video with avatar presenters
             Combined text, image, and document reasoning
       Open research models combining vision and language
               Running open-weight LLMs on local hardware

A caution about tool inventories in a fast-moving field. Any fixed list like the one above is a snapshot, not a durable reference — model names, ownership, and even entire product lines (DALL-E being the clearest recent example) change on a timescale of months. Treat categories as more stable than the specific example filling each row, and verify current status before citing a specific tool in published material.

11 Best Practices

The techniques covered above condense into a short set of practical habits:

State the task precisely rather than open-endedly — specificity is the single largest lever on output quality.
Assign a persona when the task benefits from a particular voice or expertise framing.
Supply context the model cannot otherwise infer (audience, constraints, prior decisions).
Treat the first response as a draft — iterative refinement is standard practice, not a sign of a failed first attempt.
Verify factual claims in the output against a primary source before relying on them.
Match the tool to the modality of the task — a text model is not the right tool for an image-generation request, and vice versa.
Avoid placing sensitive personal or proprietary information into a prompt sent to a third-party service.
Save prompt templates that work well for recurring tasks, following the reusable-constructor pattern demonstrated above.

12 Limitations of Current Generative AI Systems

Fluency is not the same as correctness. A model can produce confident, well-formatted, and factually wrong output; the sampling mechanism described above optimizes for plausibility given training data, not for truth.
No grounded fact-checking by default. Absent a retrieval or search step, a response reflects patterns learned during training rather than a verified lookup.
Bias inheritance. Systematic patterns present in training data can be reproduced in output, including underrepresentation or skewed framing of certain groups or viewpoints.
Knowledge currency. A model’s default knowledge is bounded by its training cutoff; anything after that date requires an explicit retrieval or search capability, not the base model alone.
Miscalibrated confidence. Language models are not reliably self-aware of their own uncertainty; a wrong answer is often phrased with the same confidence as a correct one.
Data handling. Prompts submitted to a hosted service may be logged or used according to that provider’s data policy; sensitive data warrants caution.
Not a substitute for domain judgment. These systems are best used as a drafting and exploration aid, with a domain expert retaining responsibility for the final claim or decision.

13 Summary

Generative AI is distinguished from conventional machine learning by its objective (producing new content consistent with a learned distribution, rather than predicting a label) and by its typical training regime (self-supervised pretraining on large unstructured corpora, often followed by instruction fine-tuning). LLMs are the text-generation instance of this category, and the process by which a fixed prompt becomes a specific output is governed by token-level sampling — softmax over logits, tuned by temperature — which is directly simulable and was demonstrated above without reference to any specific vendor’s model.

Prompt engineering is best treated as a compositional design problem: persona, instruction, context, examples, and output format are largely independent factors that can be manipulated and combined programmatically, and zero-shot, few-shot, and chain-of-thought prompting are three points along a single axis of increasing structural scaffolding rather than unrelated techniques. This gives a concrete basis — the build_prompt() constructor and the specificity_score() heuristic developed in this notebook — for evaluating and improving a prompt before sending it to any model, current or future.

14 References

Vaswani et al., “Attention Is All You Need” (2017) — arXiv:1706.03762
Brown et al., “Language Models are Few-Shot Learners” (2020) — arXiv:2005.14165
Ouyang et al., “Training Language Models to Follow Instructions” (2022) — arXiv:2203.02155
“A Survey of Prompt Engineering Techniques” (2023) — arXiv:2302.11382
“An Overview of Large Language Models” (2023) — arXiv:2303.18223
Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in LLMs” (2022) — arXiv:2201.11903

15 Try It Yourself

Extend specificity_score() to also flag prompts that specify a maximum output length, and re-score the five example prompts from earlier in this notebook.
Modify the toy vocabulary and logits in the sampling section to model a near-tied scenario between two tokens, and compare the resulting entropy curve across temperatures.
Write a build_prompt() call for a task from your own tutorial pipeline — for example, a plain-language summary of a coloc.abf result for a general audience — and score it before and after adding context and an output format.
Pick one row of the tool landscape table and verify its current status against the vendor’s own documentation; note anything that has changed since this notebook was written.

16 Solutions

The four exercises above are worked below, reusing the functions and objects already defined earlier in this notebook.

16.1 Solution 1: Extending `specificity_score()` with a length constraint

The extension follows the same pattern as the existing structural checks: search the prompt text for a marker phrase, here treating any mention of a numeric limit alongside a length-related word (word, sentence, paragraph) as evidence that an output-length constraint was specified.

specificity_score_v2 <- function(prompt) {
  has_role    <- grepl("^Act as", prompt)
  has_context <- grepl("Context:|Reason step by step", prompt)
  has_example <- grepl("Examples:|->", prompt)
  has_format  <- grepl("Respond as:", prompt)
  has_length  <- grepl("\\d+\\s*(word|sentence|paragraph)", prompt, ignore.case = TRUE)
  word_count  <- lengths(strsplit(prompt, "\\s+"))

  data.frame(
    role = has_role, context = has_context,
    example = has_example, output_format = has_format,
    length_constraint = has_length,
    word_count = word_count,
    structure_score = has_role + has_context + has_example + has_format + has_length
  )
}

# Re-score the five prompts already defined earlier in the notebook
do.call(rbind, lapply(prompts, specificity_score_v2))

A data.frame: 5 × 7
	role	context	example	output_format	length_constraint	word_count	structure_score

weak	FALSE	FALSE	FALSE	FALSE	FALSE	7	0
engineered	TRUE	TRUE	FALSE	TRUE	FALSE	46	3
zero_shot	FALSE	FALSE	FALSE	FALSE	FALSE	13	0
few_shot	FALSE	FALSE	TRUE	FALSE	FALSE	26	1
cot	FALSE	TRUE	FALSE	FALSE	FALSE	52	1

None of the five original example prompts specified a numeric output length, so length_constraint is FALSE throughout and the structure_score values are unchanged from the earlier table. Adding a length-constrained prompt confirms the new check actually fires:

length_constrained_prompt <- build_prompt(
  instruction = "Explain what a Manhattan plot shows.",
  output_format = "no more than 3 sentences"
)

specificity_score_v2(length_constrained_prompt)

A data.frame: 1 × 7
role	context	example	output_format	length_constraint	word_count	structure_score

FALSE	FALSE	FALSE	TRUE	TRUE	14	2

16.2 Solution 2: A near-tied vocabulary and its entropy curve

The original vocabulary had one dominant token (significant, logit 4.2) well ahead of the rest. Here two tokens are given nearly identical logits, which should push entropy higher at every temperature relative to the original, since the distribution can no longer collapse onto a single clear winner even at low temperature.

vocab_tied  <- c("significant", "suggestive", "negligible", "confounded", "spurious")
logits_tied <- c(3.0, 2.9, 0.8, 0.5, 0.2)  # top two logits nearly tied

prob_table_tied <- sapply(temperatures, function(t) softmax(logits_tied, t))
colnames(prob_table_tied) <- paste0("T=", temperatures)
rownames(prob_table_tied) <- vocab_tied

round(prob_table_tied, 3)

A matrix: 5 × 4 of type dbl
	T=0.2	T=0.7	T=1	T=1.5
significant	0.622	0.511	0.463	0.398
suggestive	0.378	0.443	0.419	0.373
negligible	0.000	0.022	0.051	0.092
confounded	0.000	0.014	0.038	0.075
spurious	0.000	0.009	0.028	0.062

comparison <- data.frame(
  temperature       = temperatures,
  entropy_original  = round(apply(prob_table, 2, entropy), 3),
  entropy_near_tied = round(apply(prob_table_tied, 2, entropy), 3)
)
comparison$entropy_increase <- round(comparison$entropy_near_tied - comparison$entropy_original, 3)
comparison

A data.frame: 4 × 4
	temperature	entropy_original	entropy_near_tied	entropy_increase

T=0.2	0.2	0.026	0.663	0.637
T=0.7	0.7	0.537	0.893	0.356
T=1	1.0	0.800	1.098	0.298
T=1.5	1.5	1.118	1.320	0.202

The near-tied vocabulary has strictly higher entropy than the original at every temperature, and the gap is largest at low temperature (T=0.2): with a single dominant token, low temperature nearly always resolves to that token, but with two tokens close in score, even a sharpening transformation cannot fully separate them. As temperature increases, both distributions approach the same uniform-entropy ceiling, so the two curves converge — the effect of a near-tie is strongest exactly where it is least intuitive, at low temperature, not high.

16.3 Solution 3: A `coloc.abf` summary prompt, before and after engineering

A minimal version of this task and an engineered version, scored with specificity_score() for direct comparison.

coloc_weak <- build_prompt(
  instruction = "Summarize this coloc.abf result in plain language."
)

coloc_engineered <- build_prompt(
  instruction = "Summarize this coloc.abf result in plain language: PP4 = 0.87, PP3 = 0.09, PP0-PP2 negligible.",
  persona = "a genetic epidemiologist writing for a non-specialist collaborator",
  context = paste(
    "PP4 is the posterior probability that the GWAS and eQTL signals share a",
    "single causal variant; PP3 is the probability of two distinct causal",
    "variants at the same locus."
  ),
  output_format = "two sentences, avoiding statistical jargon where possible"
)

rbind(
  specificity_score(coloc_weak),
  specificity_score(coloc_engineered)
)

A data.frame: 2 × 6
role	context	example	output_format	word_count	structure_score

FALSE	FALSE	FALSE	FALSE	8	0
TRUE	TRUE	FALSE	TRUE	65	3

As with the earlier examples, adding persona, context, and an output-format constraint raises the structure score from 0 to 3; the qualitative difference this makes to an actual model response would be that the weak version leaves both the intended audience and the meaning of PP4/PP3 unstated, while the engineered version fixes both before the request reaches the model.

16.4 Solution 4: Verifying one row of the tool landscape table

Taking the Runway row (Video generation) as the example. The table entry describes Runway generically as a “text- and image-conditioned video generation” tool, without pinning a version number — which was a deliberate choice, since version-specific claims are exactly what goes stale fastest in this space. Checking against Runway’s own release material as of mid-2026:

Runway’s current flagship is Gen-4.5, released December 2025, which added native text-to-video generation on top of the image-conditioned workflow introduced in Gen-4 (March 2025).
Earlier overviews of this space (including the source material this notebook was built from) describe Runway at the Gen-2 stage, which is now three major generations behind.
The general category description in the table — “text- and image-conditioned video generation” — still holds without modification; only a specific version number would need updating.

This illustrates the intended use of the caution note attached to the table: the category-level description was written to survive this kind of check, while any row that had included a specific version number would already need a correction after roughly a year.

The barbeque question: will AI’s data center problem end like the genome did?

Nivedita Bhadra — Mon, 27 Jul 2026 21:00:00 GMT

It happened somewhere between the second batch of skewers and the point where someone finally admitted the coals needed more time. Someone at the table—half paying attention to the grill, half to the group—said the thing that derailed the evening in the best way:

“You know, twenty years ago, sequencing a single human genome felt impossible. Now it’s routine. Is AI’s data center problem going to go the same way?”

Forks paused. It was a good question, the kind that sounds simple until you actually try to answer it.

Smoke from a barbeque grill splitting into a DNA double helix on one side and a stack of data center server racks on the other

The $3 Billion Genome

Rewind to 2003. The Human Genome Project had just wrapped after thirteen years and roughly $3 billion, reading out the three billion base pairs of human DNA one laborious Sanger-sequencing run at a time. It was a moonshot—international, decade-spanning, celebrated with press conferences.

Fast forward, and something remarkable happened to that cost curve. It didn’t just fall—it collapsed. Sequencing a human genome went from tens of millions of dollars to a few thousand, and today, in highly optimized settings, the cost is approaching only a few hundred dollars. That transformation didn’t happen because Sanger sequencing became incrementally better every year. It happened because the entire approach was reinvented. Next-generation sequencing arrived with a fundamentally different chemistry, reading millions of DNA fragments in parallel instead of one at a time. Genome sequencing didn’t simply become cheaper—it was replaced by something fundamentally better.

That was the story someone was telling around the grill. And the question hanging in the smoke was this:

Is AI’s compute crunch waiting for its own next-generation sequencing moment?

The Electron Problem

Here’s where the table split into two camps.

The optimists argued that something very genome-like is already happening in AI. Training costs aren’t falling simply because chips are getting faster—they’re falling because of genuine algorithmic breakthroughs. Over the past decade, researchers have repeatedly shown that smarter model architectures, improved optimization methods, and more efficient training strategies can dramatically reduce the amount of computation required to reach a given level of performance.

DeepSeek, for example, demonstrated that architectural redesign—not simply buying more GPUs—can substantially reduce training and inference costs while maintaining frontier-level capability. Hardware is evolving alongside these algorithms: every new generation of AI accelerators delivers meaningful improvements in throughput and energy efficiency, steadily pushing the cost of computation downward.

It sounds strikingly similar to the genome story: a stubborn bottleneck overcome not by brute force, but by better ideas.

Then the skeptic at the table—there’s always one—pointed out the catch, and it changed the entire shape of the discussion.

A genome has a finish line.

There are about three billion base pairs, full stop. Once sequencing became inexpensive, researchers didn’t need to read more DNA from each person—the computational target stayed fixed. Demand for sequencing certainly exploded, with millions of genomes now being sequenced around the world, but the amount of DNA contained in a human genome never changed. Every efficiency improvement made sequencing cheaper, faster, and more accessible without moving the goalposts.

AI has no comparable finish line.

When training becomes ten times cheaper, nobody simply trains yesterday’s model for one-tenth the cost and goes home. Instead, researchers build larger models, run more experiments, deploy more applications, and serve vastly more users. Economists have a name for this pattern: Jevons Paradox. Making something more efficient often doesn’t reduce total consumption—it increases it, because lower cost creates entirely new uses that were previously impractical.

Industry analysis increasingly points in this direction. Efficiency gains in AI are likely to be accompanied by an explosion in experimentation, deployment, and inference at scale. Even if each unit of computation becomes cheaper, the total demand for computation may continue to grow.

There’s another wrinkle as well.

Genome sequencing benefited from a paradigm shift that fundamentally replaced the previous technology. Modern computing also experienced enormous efficiency gains during the cloud-computing revolution, when organizations moved workloads from inefficient on-premises server rooms into highly optimized hyperscale data centers.

Frontier AI, however, largely begins inside those already-optimized facilities. The easy infrastructure gains have mostly been captured before today’s AI boom. The next improvements must come from genuinely harder engineering problems: better chip architectures, smarter model designs, improved scheduling, higher hardware utilization, more efficient networking, advanced cooling, and entirely new approaches to computation itself.

So, Same Story or Different One?

By the time the coals were dying down, the table had more or less landed here: the mechanism rhymes.

Paradigm-breaking innovations—not incremental scaling—are what truly bend cost curves. AI is already experiencing several of them through new architectures, sparse models, improved training methods, specialized hardware, and increasingly sophisticated software engineering. In that sense, the genome story is repeating.

But the ending may be very different.

Genome sequencing is fundamentally a fixed-length problem. Every human genome contains roughly the same amount of DNA to read. Intelligence, by contrast, is an open-ended problem. Every time AI becomes cheaper to train or deploy, researchers don’t simply stop. They build more capable systems, create entirely new applications, expand inference to billions of users, and tackle problems that previously seemed out of reach. The goalposts keep moving.

So the honest answer isn’t that AI will have one dramatic “next-generation sequencing” moment after which the data-center problem disappears.

It’s more likely that AI will experience many such moments. Each breakthrough will make computation dramatically more efficient than before. But unlike genome sequencing, those gains may be continually absorbed by expanding demand, leaving society with ever more capable AI systems—yet not necessarily fewer data centers.

Even researchers who study this professionally disagree about how the balance will play out. Some believe algorithmic innovation and specialized hardware will continue to outpace demand growth. Others argue that energy availability, manufacturing capacity, and physical limits on computing will eventually become the dominant constraints. The debate remains very much alive.

Which, if you think about it, is a pretty good note to end a barbeque conversation on: not a tidy answer, but a much better question than the one you started with.

Written after (and inspired by) an actual argument over the grill about whether we’re solving AI’s biggest infrastructure problem—or simply discovering how big “solved” needs to become.

Multivariate Concepts in Statistical Genetics: From LD Score Regression to Genomic SEM

Nivedita Bhadra — Mon, 27 Jul 2026 21:00:00 GMT

The model this tutorial builds up to fitting: a single shared genetic factor F1, estimated not from individual-level data but from a genetic covariance matrix built out of GWAS summary statistics alone.

Genomic SEM LDSC Structural Equation Modeling R

1 Multivariate Concepts in Statistical Genetics

In many areas of statistical genetics, we are not only interested in whether a single trait is heritable. We are also interested in whether multiple traits share part of their genetic architecture.

For example, psychiatric traits such as major depression, anxiety, post-traumatic stress disorder, alcohol use disorder, bipolar disorder, and schizophrenia are not genetically isolated. They often show genetic overlap. This means that some genetic variants may influence more than one phenotype.

This tutorial introduces the main multivariate concepts needed to understand Genomic SEM. We will move step by step from basic ideas such as polygenicity and LD Score Regression to genetic covariance, genetic correlation, genome-wide structural models, and multivariate GWAS.

1.0.1 1. Why Do We Need Multivariate Genetic Models?

Complex traits are usually highly polygenic. This means that they are influenced by thousands of genetic variants, each with a very small effect.

A simple single-gene model is usually not appropriate for complex psychiatric or behavioral traits. Instead, risk emerges from the combined effect of many variants.

This creates two important problems:

We need methods that can summarize genome-wide genetic signal.
We need methods that can estimate how much genetic signal is shared across traits.

This is where LD Score Regression and Genomic SEM become useful.

1.0.2 2. From Genetic Overlap to Genetic Architecture

Genetic overlap means that the same genetic variants contribute to variation in more than one phenotype.

However, genetic overlap can appear in different forms.

Two traits may share:

A broad common genetic factor Several smaller domain-specific factors A mixture of shared and trait-specific genetic effects SNPs that influence one trait much more than another

Genomic SEM allows us to model these possibilities explicitly.

For example, if four psychiatric traits are genetically correlated, we can test whether this correlation is well explained by one common factor.

A simplified model could be:

$ F_1 MDD, PTSD, ANX, ALCH $

where is a shared genetic liability factor.

1.0.3 3. Why Traits Do Not Need to Come From the Same Sample

One powerful feature of LD Score Regression and Genomic SEM is that traits do not need to be measured in the same individuals.

For example, we may have:

A GWAS of major depression from one cohort A GWAS of anxiety from another cohort A GWAS of PTSD from another cohort A GWAS of alcohol use disorder from another cohort

These GWAS may come from different studies, different consortia, and different individuals.

The connection between them is the genome.

Because each GWAS reports SNP-level effects across the genome, we can compare the pattern of SNP effects across traits. If SNPs that increase one trait also tend to increase another trait, this creates evidence of shared genetic architecture.

This is why summary statistics are so useful. We do not always need access to individual-level data to study genetic overlap.

1.0.4 2. Polygenicity and Linkage Disequilibrium

Suppose a phenotype is influenced by a few true causal variants. In GWAS, we do not observe only those causal variants. We also observe many nearby SNPs that are correlated with them.

This correlation among SNPs is called linkage disequilibrium, or LD.

A SNP with high LD is correlated with many nearby SNPs. Therefore, high-LD SNPs are more likely to tag causal variants. This means they are more likely to show association signal even if they are not causal themselves.

1.0.4.1 2.1 LD Score

The LD score of SNP (j) is defined as:

$ _j = i r{ij}^2 $

where:

is the LD score of SNP (j)
is the squared correlation between SNP (i) and SNP (j)

A high LD score means that the SNP is correlated with many other SNPs.

1.0.5 4. The Intuition Behind LD Score Regression

LD Score Regression works because highly polygenic traits create a predictable relationship between LD and GWAS signal.

Imagine there are true causal variants in the genome.

A SNP does not need to be causal to show association. If it is correlated with a causal variant through LD, it can still pick up association signal.

This means:

High-LD SNPs are correlated with many nearby SNPs. Therefore, they have more chances to tag causal variants. Low-LD SNPs are correlated with fewer nearby SNPs. Therefore, they usually only show strong signal if they are causal or very close to causal variants.

For a highly polygenic trait, many causal variants are spread across the genome. As a result, SNPs with higher LD scores tend to have larger GWAS chi-square statistics on average.

This is the core logic of LD Score Regression.

1.0.6 5. Why the LDSC Intercept Matters

In GWAS, test statistics can be inflated for several reasons.

Some inflation reflects true polygenic signal. This is expected when many SNPs contribute to the trait.

However, some inflation can come from confounding, such as:

Population stratification Cryptic relatedness Sample overlap Technical artifacts

LD Score Regression separates these components.

The slope of the regression captures polygenic signal.

The intercept captures inflation that is not explained by LD score.

This is why LD Score Regression is useful: it helps distinguish true polygenic signal from confounding.

1.0.7 3. SNP Heritability Using LD Score Regression

LD Score Regression is based on a simple idea:

If a trait is highly polygenic, SNPs with higher LD scores should, on average, have larger GWAS test statistics.

In other words, we regress the GWAS chi-square statistics on LD scores.

The expected chi-square statistic for SNP (j) can be written as:

$ E[_j^2 | _j] = _j + Na + 1 $

where:

is the GWAS sample size
is SNP heritability
is the number of SNPs
is the LD score
captures confounding such as population stratification
the intercept captures inflation not due to polygenic signal

The slope of this regression is used to estimate SNP heritability.

1.0.7.1 3.1 Why Sample Size Matters

Imagine two traits have the same GWAS p-values, but different sample sizes:

Trait 1:
Trait 2:

If the association statistics are similar, the trait with the smaller sample size may imply stronger per-sample genetic signal. This is why LD Score Regression accounts for sample size when estimating heritability.

1.0.8 4. Genetic Covariance and Genetic Correlation

LD Score Regression can also be extended to two traits.

Instead of regressing one trait’s chi-square statistics on LD scores, we examine the product of GWAS (Z)-statistics across two traits.

For traits (Y_1) and (Y_2), we use:

$ Z_{1j}Z_{2j} $

If SNPs with high LD scores tend to have similar effects across both traits, this suggests genetic overlap.

1.0.8.1 4.1 Genetic Covariance

Genetic covariance measures the extent to which two traits share genetic influences.

The cross-trait LD Score Regression equation is:

$ E[z_{1j}z_{2j}|_j] = j + + {Ns} $

Conceptually:

The slope estimates genetic covariance.
The intercept captures sample overlap and confounding.
Traits do not need to be measured in the same individuals.

This is very useful because many GWAS summary statistics are publicly available and come from independent cohorts.

1.0.8.2 4.2 Genetic Correlation

Genetic correlation is the standardized form of genetic covariance:

$ r_g = {} $

A genetic correlation close to 1 means two traits share much of their genetic architecture.

A genetic correlation close to 0 means little shared genetic influence.

A negative genetic correlation means genetic variants that increase one trait tend to decrease the other.

1.0.9 8. Why Shared Genetic Architecture Requires New Models

1.0.10 5. From LD Score Regression to Genomic SEM

Genomic SEM stands for Genomic Structural Equation Modeling.

It provides a flexible framework for fitting structural equation models to genetic covariance matrices estimated from GWAS summary statistics.

The main advantage is that we can model genetic relationships among traits even when those traits were measured in different samples.

1.0.11 6. The Two-Stage Framework of Genomic SEM

Genomic SEM has two main stages.

1.0.11.1 Stage 1: Estimate Genetic Covariance Structure

Using multivariable LD Score Regression, we estimate:

A genetic covariance matrix
A sampling covariance matrix

The genetic covariance matrix is usually called (S).

The diagonal elements of (S) are SNP heritabilities.

The off-diagonal elements are genetic covariances.

A simplified matrix looks like this:

$ S =

1.0.11.2 Stage 2: Fit a Structural Equation Model

After estimating (S), we fit a model to explain the covariance structure.

For example, we may fit a common factor model where depression, anxiety, PTSD, and alcohol use disorder load on a shared internalizing factor.

1.0.12 9. Genome-Wide SEM vs Multivariate GWAS

It is important to distinguish between genome-wide SEM and multivariate GWAS.

1.0.12.1 Genome-Wide SEM

Genome-wide SEM uses the genetic covariance matrix across traits.

It asks:

What is the genetic relationship among these traits?

For example:

Do the traits load on one common factor? Are there multiple factors? Does one genetic trait predict another? How well does the model fit?

This stage does not test SNP effects directly.

1.0.12.2 Multivariate GWAS

Multivariate GWAS adds SNP effects to the model.

It asks:

Which SNPs are associated with the latent genetic factor?

For example:

$ F_1 SNP $

This tests whether each SNP predicts the shared genetic factor.

So, genome-wide SEM models the covariance structure, while multivariate GWAS tests SNP-level effects on that structure.

1.0.13 7. Practical Setup in R

We will now follow the practical structure from the workshop.

The main steps are:

Load GenomicSEM
Munge GWAS summary statistics
Run LD Score Regression
Specify and run a genome-wide Genomic SEM model
Prepare summary statistics for multivariate GWAS
Run userGWAS
Interpret factor GWAS and QSNP results

install.packages("devtools")

devtools::install_github(
  "GenomicSEM/GenomicSEM"
)
# Load GenomicSEM
require(GenomicSEM)


The downloaded binary packages are in
    /var/folders/3f/6pzn2nyn32d7wthyxkzsqyxcdrtvym/T//RtmpPbLU4V/downloaded_packages

rlang        (1.1.6  -> 1.2.0    ) [CRAN]
lifecycle    (1.0.4  -> 1.0.5    ) [CRAN]
glue         (1.8.0  -> 1.8.1    ) [CRAN]
cli          (3.6.5  -> 3.6.6    ) [CRAN]
vctrs        (0.6.5  -> 0.7.3    ) [CRAN]
utf8         (1.2.5  -> 1.2.6    ) [CRAN]
pillar       (1.10.2 -> 1.11.1   ) [CRAN]
magrittr     (2.0.3  -> 2.0.5    ) [CRAN]
cpp11        (0.5.2  -> 0.5.5    ) [CRAN]
bit          (4.0.5  -> 4.6.0    ) [CRAN]
tzdb         (0.4.0  -> 0.5.0    ) [CRAN]
tibble       (3.2.1  -> 3.3.1    ) [CRAN]
hms          (1.1.3  -> 1.1.4    ) [CRAN]
bit64        (4.0.5  -> 4.8.2    ) [CRAN]
colorspace   (2.1-1  -> 2.1-2    ) [CRAN]
gridBase     (NA     -> 0.4-7    ) [CRAN]
sfsmisc      (NA     -> 1.1-24   ) [CRAN]
R.oo         (1.26.0 -> 1.27.1   ) [CRAN]
data.table   (1.17.4 -> 1.18.4   ) [CRAN]
mnormt       (2.1.1  -> 2.1.2    ) [CRAN]
vroom        (1.6.5  -> 1.7.1    ) [CRAN]
clipr        (0.8.0  -> 0.8.1    ) [CRAN]
proxy        (0.4-27 -> 0.4-29   ) [CRAN]
Rcpp         (1.0.14 -> 1.1.1-1.1) [CRAN]
simsalapar   (NA     -> 1.0-13   ) [CRAN]
mgsub        (NA     -> 2.0.0    ) [CRAN]
dplyr        (1.1.4  -> 1.2.1    ) [CRAN]
R.utils      (2.12.3 -> 2.13.0   ) [CRAN]
splitstac... (NA     -> 1.4.8.1  ) [CRAN]
stringr      (1.5.1  -> 1.6.0    ) [CRAN]
lavaan       (0.6-19 -> 0.6-21   ) [CRAN]
gdata        (3.0.0  -> 3.0.1    ) [CRAN]
readr        (2.1.5  -> 2.2.0    ) [CRAN]
e1071        (1.7-14 -> 1.7-17   ) [CRAN]

The downloaded binary packages are in

    /var/folders/3f/6pzn2nyn32d7wthyxkzsqyxcdrtvym/T//RtmpPbLU4V/downloaded_packages

── R CMD build ─────────────────────────────────────────────────────────────────

✔  checking for file ‘/private/var/folders/3f/6pzn2nyn32d7wthyxkzsqyxcdrtvym/T/RtmpPbLU4V/remotes2dbf3fb76124/GenomicSEM-GenomicSEM-0a63ac0/DESCRIPTION’

─  preparing ‘GenomicSEM’:

✔  checking DESCRIPTION meta-information

─  excluding invalid files

   Subdirectory 'man' contains invalid file names:

     ‘decisiontree.png’

─  checking for LF line-endings in source and make files and shell scripts

─  checking for empty or unneeded directories

   Omitted ‘LazyData’ from DESCRIPTION

─  building ‘GenomicSEM_0.0.5.tar.gz’

   Warning: invalid uid value replaced by that for user 'nobody'

   Warning: invalid gid value replaced by that for user 'nobody'

library(GenomicSEM)

1.0.14 8. Step 1: Munge Summary Statistics

Munging is the process of cleaning and harmonizing GWAS summary statistics.

It usually includes:

Aligning alleles
Filtering to HapMap3 SNPs
Checking SNP IDs
Formatting effect sizes
Formatting standard errors
Preparing files for LD Score Regression

In this practical, the workshop uses four traits:

ALCH: Alcohol use disorder
PTSD: Post-traumatic stress disorder
MDD: Major depressive disorder
ANX: Anxiety disorder

# Example file names
files <- c(
  "ALCH_withrsID.txt",
  "SORTED_PTSD_EA9_ALL_study_specific_PCs1.txt",
  "MDD_withNeff.txt",
  "ANX_withNeff.txt"
)

# HapMap3 SNP reference file
hm3 <- "eur_w_ld_chr/w_hm3.snplist"

# Trait names
trait.names <- c("ALCH", "PTSD", "MDD", "ANX")

# Effective sample size
N <- c(NA, 5831.346, NA, NA)

# Run munge
# This is commented out because the workshop already provides prepared files.
# munge(files = files, hm3 = hm3, trait.names = trait.names, N = N)

1.0.15 9. Step 2: Run Multivariable LD Score Regression

After munging, we run LD Score Regression.

This estimates:

SNP heritability for each trait
Genetic covariance between traits
Genetic correlation between traits
Sampling covariance matrix

The output is later used by Genomic SEM.

# Munged summary statistics
traits <- c(
  "ALCH.sumstats.gz",
  "PTSD.sumstats.gz",
  "MDD.sumstats.gz",
  "ANX.sumstats.gz"
)

# Sample prevalence
sample.prev <- c(.5, .5, .5, .5)

# Population prevalence
population.prev <- c(.159, .3, .15, .20)

# LD score folders
ld <- "eur_w_ld_chr/"
wld <- "eur_w_ld_chr/"

# Trait names
trait.names <- c("ALCH", "PTSD", "MDD", "ANX")

# Run LDSC
# This is commented out because the workshop provides LDSC_INT.RData.
# LDSC_INT <- ldsc(
#   traits = traits,
#   sample.prev = sample.prev,
#   population.prev = population.prev,
#   ld = ld,
#   wld = wld,
#   trait.names = trait.names
# )

# Save LDSC output
# save(LDSC_INT, file = "LDSC_INT.RData")

1.0.16 10. Step 3: Load the LDSC Output

The workshop practical already provides the LDSC object.

This object contains the genetic covariance matrix and sampling covariance matrix.

load("LDSC_INT.RData")

# Genetic covariance matrix
LDSC_INT$S

# Sampling covariance matrix
LDSC_INT$V

A matrix: 4 × 4 of type dbl
ALCH	PTSD	MDD	ANX
0.13938790	0.05977947	0.05943021	0.08500236
0.05977947	0.23937808	0.05799439	0.11428679
0.05943021	0.05799439	0.08503281	0.12667327
0.08500236	0.11428679	0.12667327	0.23329361

A matrix: 10 × 10 of type dbl
6.041603e-04	4.630259e-05	1.329209e-05	2.110818e-05	1.699375e-04	1.928842e-05	5.939947e-06	3.078558e-06	6.866515e-06	1.292371e-05
4.630259e-05	1.683117e-03	5.491624e-06	1.012613e-04	7.541207e-05	1.794540e-05	9.955430e-05	2.224531e-07	3.119102e-06	1.406092e-05
1.329209e-05	5.491624e-06	3.692467e-05	4.865869e-05	8.966010e-05	1.773068e-06	2.035833e-05	6.028298e-06	9.777927e-06	7.764292e-06
2.110818e-05	1.012613e-04	4.865869e-05	2.627033e-04	1.244898e-05	-7.693747e-08	1.510451e-05	6.091000e-06	1.858721e-05	3.524720e-05
1.699375e-04	7.541207e-05	8.966010e-05	1.244898e-05	1.135989e-02	4.536925e-05	3.350365e-04	-1.606011e-05	5.419991e-05	1.212088e-06
1.928842e-05	1.794540e-05	1.773068e-06	-7.693747e-08	4.536925e-05	1.336556e-04	1.255636e-04	7.752931e-06	1.314640e-05	3.972090e-05
5.939947e-06	9.955430e-05	2.035833e-05	1.510451e-05	3.350365e-04	1.255636e-04	8.250458e-04	-4.762316e-07	5.242814e-06	3.447767e-05
3.078558e-06	2.224531e-07	6.028298e-06	6.091000e-06	-1.606011e-05	7.752931e-06	-4.762316e-07	1.219319e-05	1.921343e-05	1.590191e-05
6.866515e-06	3.119102e-06	9.777927e-06	1.858721e-05	5.419991e-05	1.314640e-05	5.242814e-06	1.921343e-05	5.109221e-05	5.444954e-05
1.292371e-05	1.406092e-05	7.764292e-06	3.524720e-05	1.212088e-06	3.972090e-05	3.447767e-05	1.590191e-05	5.444954e-05	2.843342e-04

1.0.17 11. Understanding the Genetic Covariance Matrix

The matrix LDSC_INT$S contains genetic variances and covariances.

The diagonal values are SNP heritabilities.

The off-diagonal values are genetic covariances.

For example, if the covariance between MDD and ANX is high, this suggests that depression and anxiety share genetic influences.

# Print genetic covariance matrix
print(LDSC_INT$S)

           ALCH       PTSD        MDD        ANX
[1,] 0.13938790 0.05977947 0.05943021 0.08500236
[2,] 0.05977947 0.23937808 0.05799439 0.11428679
[3,] 0.05943021 0.05799439 0.08503281 0.12667327
[4,] 0.08500236 0.11428679 0.12667327 0.23329361

1.0.18 12. Understanding the Sampling Covariance Matrix

The matrix LDSC_INT$V contains uncertainty around the estimates in S.

This is important because some GWAS are more highly powered than others.

It also models dependencies caused by sample overlap between GWAS studies.

# Print sampling covariance matrix
print(LDSC_INT$V)

              [,1]         [,2]         [,3]          [,4]          [,5]
 [1,] 6.041603e-04 4.630259e-05 1.329209e-05  2.110818e-05  1.699375e-04
 [2,] 4.630259e-05 1.683117e-03 5.491624e-06  1.012613e-04  7.541207e-05
 [3,] 1.329209e-05 5.491624e-06 3.692467e-05  4.865869e-05  8.966010e-05
 [4,] 2.110818e-05 1.012613e-04 4.865869e-05  2.627033e-04  1.244898e-05
 [5,] 1.699375e-04 7.541207e-05 8.966010e-05  1.244898e-05  1.135989e-02
 [6,] 1.928842e-05 1.794540e-05 1.773068e-06 -7.693747e-08  4.536925e-05
 [7,] 5.939947e-06 9.955430e-05 2.035833e-05  1.510451e-05  3.350365e-04
 [8,] 3.078558e-06 2.224531e-07 6.028298e-06  6.091000e-06 -1.606011e-05
 [9,] 6.866515e-06 3.119102e-06 9.777927e-06  1.858721e-05  5.419991e-05
[10,] 1.292371e-05 1.406092e-05 7.764292e-06  3.524720e-05  1.212088e-06
               [,6]          [,7]          [,8]         [,9]        [,10]
 [1,]  1.928842e-05  5.939947e-06  3.078558e-06 6.866515e-06 1.292371e-05
 [2,]  1.794540e-05  9.955430e-05  2.224531e-07 3.119102e-06 1.406092e-05
 [3,]  1.773068e-06  2.035833e-05  6.028298e-06 9.777927e-06 7.764292e-06
 [4,] -7.693747e-08  1.510451e-05  6.091000e-06 1.858721e-05 3.524720e-05
 [5,]  4.536925e-05  3.350365e-04 -1.606011e-05 5.419991e-05 1.212088e-06
 [6,]  1.336556e-04  1.255636e-04  7.752931e-06 1.314640e-05 3.972090e-05
 [7,]  1.255636e-04  8.250458e-04 -4.762316e-07 5.242814e-06 3.447767e-05
 [8,]  7.752931e-06 -4.762316e-07  1.219319e-05 1.921343e-05 1.590191e-05
 [9,]  1.314640e-05  5.242814e-06  1.921343e-05 5.109221e-05 5.444954e-05
[10,]  3.972090e-05  3.447767e-05  1.590191e-05 5.444954e-05 2.843342e-04

1.0.19 10. Factor Identification in Genomic SEM

Latent factors do not have a natural scale. Therefore, we must identify the scale of the factor.

There are two common approaches.

Option 1: Unit Variance Identification

In this approach, the variance of the latent factor is fixed to 1.

Example:

Model <- ” F1 =~ NAMDD + PTSD + ALCH + ANX F1 1F1 ”

This means the factor has a standardized variance.

In Genomic SEM, this is often equivalent to using:

std.lv = TRUE Option 2: Unit Loading Identification

In this approach, one factor loading is fixed to 1.

Example:

Model <- ” F1 =~ 1*MDD + PTSD + ALCH + ANX ”

Here, the scale of the factor is defined by MDD.

Both approaches are valid, but they lead to different parameter scaling. The interpretation of the model should always consider how the factor was identified.

1.0.20 13. Lavaan Model Syntax Used in Genomic SEM

Genomic SEM uses lavaan-style model syntax.

Here are the most important operators:

Syntax	Meaning
`A ~ B`	A is regressed on B
`A ~~ A`	Variance of A
`A ~~ B`	Covariance between A and B
`F1 =~ A + B + C`	Latent factor F1 loads on A, B, and C
`A ~~ 1*B`	Fix covariance between A and B to 1
`A ~~ a*B`	Label covariance as parameter `a`

This syntax allows us to specify many different genetic models.

1.0.21 14. Common Factor Model

A common factor model assumes that several observed traits share one underlying latent genetic factor.

For example:

$ F1 MDD, PTSD, ALCH, ANX $

Here, F1 represents a shared internalizing genetic factor.

# Define covariance structure
covstruc <- LDSC_INT

# Common factor model
INT.model <- "
F1 =~ MDD + PTSD + ALCH + ANX
"

# Use unit variance identification
std.lv <- TRUE

# Run model
IntResults <- usermodel(
  covstruc = covstruc,
  model = INT.model,
  std.lv = std.lv
)

[1] "Running primary model"
[1] "Calculating Standardized Results"
[1] "Calculating SRMR"
elapsed 
  0.175

1.0.22 15. Genetic Multiple Regression

Genomic SEM is not limited to factor models. It can also estimate genetic regression models.

For example, suppose we want to know whether the genetic component of educational attainment is associated with schizophrenia and bipolar disorder.

A simplified model could be:

[ EA_g = b_1SCZ_g + b_2BIP_g + u]

This means that the genetic component of educational attainment is regressed on the genetic components of schizophrenia and bipolar disorder.

In lavaan-style syntax, this could be written as:

model <- ” EA ~ SCZ + BIP SCZ BIP ”

This allows us to ask whether one trait explains unique genetic variation in another trait after accounting for additional genetically correlated traits.

1.0.23 16. GWAS-by-Subtraction

GWAS-by-subtraction is one of the important applications of Genomic SEM.

The idea is to separate shared and residual genetic components.

For example, educational attainment and cognitive performance are genetically correlated. But educational attainment is influenced by both cognitive and non-cognitive factors.

A Genomic SEM model can estimate:

A cognitive genetic factor shared with cognitive performance A non-cognitive genetic component of educational attainment

Conceptually:

$ EA = Cognitive + NonCognitive $

This allows researchers to perform GWAS on the residual non-cognitive component.

In simple terms, GWAS-by-subtraction asks:

What genetic signal remains in one trait after removing the genetic signal shared with another trait?

This is useful when we want to isolate more specific genetic pathways.

1.0.24 15. View Model Results

The results table contains:

Parameter estimates
Standard errors
Standardized estimates
p-values

The factor loadings tell us how strongly each trait relates to the latent genetic factor.

IntResults$results

A data.frame: 9 × 9
	lhs	op	rhs	Unstand_Est	Unstand_SE	STD_Genotype	STD_Genotype_SE	STD_All	p_value

5	F1	=~	MDD	0.283806747	0.021002745436444	0.97326125	0.0720249151052966	0.97326125	1.313540e-41
6	F1	=~	PTSD	0.221278068	0.0402049336545347	0.45226835	0.0821745168725034	0.45226834	3.717880e-08
3	F1	=~	ALCH	0.205225639	0.024321802328951	0.54969157	0.0651453167330757	0.54969157	3.230100e-17
4	F1	=~	ANX	0.445784749	0.032456114148652	0.92294074	0.0671962594562505	0.92294072	6.265550e-43
1	ALCH	~~	ALCH	0.097270350	0.025899146459423	0.69783919	0.185806270387686	0.69783918	1.728330e-04
9	PTSD	~~	PTSD	0.190414108	0.106977265864464	0.79545336	0.446896663140437	0.79545335	7.508426e-02
8	MDD	~~	MDD	0.004486541	0.0109566843706871	0.05276254	0.12885240976165	0.05276254	6.821876e-01
2	ANX	~~	ANX	0.034569558	0.0301103378331493	0.14818043	0.129066285535458	0.14818043	2.509289e-01
7	F1	~~	F1	1.000000000		1.00000000		1.00000000	NA

1.0.25 16. View Model Fit

Model fit tells us whether the proposed model explains the observed genetic covariance matrix well.

Important fit statistics include:

Chi-square: lower is better
AIC: useful for comparing models
CFI: higher is better
SRMR: lower is better

As a rough guide:

CFI > 0.90 suggests acceptable fit
CFI > 0.95 suggests good fit
SRMR < 0.10 suggests acceptable fit
SRMR < 0.05 suggests good fit

IntResults$modelfit

A data.frame: 1 × 6
	chisq	df	p_chisq	AIC	CFI	SRMR

df	1.283452	2	0.526383	17.28345	1	0.03621695

1.0.26 17. Interpreting the Common Factor Model

A strong loading of MDD on F1 means that major depression strongly reflects the shared genetic factor.

A weaker loading of ALCH on F1 means that alcohol use disorder may share some genetic risk with the factor, but also has more trait-specific genetic influences.

The residual variance of each trait represents genetic variation not explained by the common factor.

1.0.27 18. Alternative Model: Correlated Traits Model

Instead of forcing all traits to load on a common factor, we could estimate pairwise genetic covariances directly.

This is useful when we do not want to assume a latent factor.

correlated.model <- "
MDD ~~ PTSD
MDD ~~ ALCH
MDD ~~ ANX
PTSD ~~ ALCH
PTSD ~~ ANX
ALCH ~~ ANX
"

CorResults <- usermodel(
  covstruc = LDSC_INT,
  model = correlated.model,
  std.lv = FALSE
)

CorResults$results
CorResults$modelfit

[1] "Running primary model"
[1] "Calculating Standardized Results"
[1] "Calculating SRMR"
elapsed 
  0.104 
[1] "Model fit statistics are all printed as NA as you have specified a fully saturated model (i.e., df = 0)"

A data.frame: 10 × 9
	lhs	op	rhs	Unstand_Est	Unstand_SE	STD_Genotype	STD_Genotype_SE	STD_All	p_value

7	MDD	~~	PTSD	0.05799439	0.011560951	0.4064906	0.08103228	0.4064906	5.264780e-07
4	MDD	~~	ALCH	0.05943021	0.006076568	0.5458853	0.05581520	0.5458853	1.369039e-22
5	MDD	~~	ANX	0.12667327	0.007147882	0.8993739	0.05074961	0.8993739	2.847008e-70
8	PTSD	~~	ALCH	0.05977947	0.041025806	0.3272633	0.22459619	0.3272633	1.450836e-01
9	PTSD	~~	ANX	0.11428679	0.028723611	0.4836180	0.12154733	0.4836180	6.925092e-05
2	ALCH	~~	ANX	0.08500236	0.016208124	0.4713755	0.08988118	0.4713755	1.567668e-07
1	ALCH	~~	ALCH	0.13938790	0.024579672	1.0000000	0.17634007	1.0000000	1.420817e-08
10	PTSD	~~	PTSD	0.23937808	0.106582777	1.0000000	0.44524869	1.0000000	2.470812e-02
6	MDD	~~	MDD	0.08503281	0.003491874	1.0000000	0.04106502	1.0000000	5.572026e-131
3	ANX	~~	ANX	0.23329361	0.016862213	1.0000000	0.07227893	1.0000000	1.561049e-43

A data.frame: 1 × 6
	chisq	df	p_chisq	AIC	CFI	SRMR

df	NA	0	NA	NA	NA	NA

1.0.28 19. Alternative Model: Regression Model

Genomic SEM can also fit genetic regression models.

For example, we can ask whether the genetic component of anxiety predicts the genetic component of depression.

regression.model <- "
MDD ~ ANX + PTSD + ALCH
ANX ~~ PTSD
ANX ~~ ALCH
PTSD ~~ ALCH
"

RegResults <- usermodel(
  covstruc = LDSC_INT,
  model = regression.model,
  std.lv = FALSE
)

RegResults$results
RegResults$modelfit

[1] "Running primary model"
[1] "Calculating Standardized Results"
[1] "Calculating SRMR"
elapsed 
  0.115 
[1] "Model fit statistics are all printed as NA as you have specified a fully saturated model (i.e., df = 0)"

A data.frame: 10 × 9
	lhs	op	rhs	Unstand_Est	Unstand_SE	STD_Genotype	STD_Genotype_SE	STD_All	p_value

7	MDD	~	ANX	0.51330125	0.068314744	0.85021835	0.11315470	0.85021835	5.744864e-14
8	MDD	~	PTSD	-0.03483028	0.094099604	-0.05843942	0.15788349	-0.05843942	7.112762e-01
6	MDD	~	ALCH	0.12827888	0.075097732	0.16423830	0.09614929	0.16423830	8.760680e-02
4	ANX	~~	PTSD	0.11428679	0.028723611	0.48361796	0.12154733	0.48361796	6.925093e-05
2	ANX	~~	ALCH	0.08500236	0.016208124	0.47137552	0.08988118	0.47137551	1.567668e-07
9	PTSD	~~	ALCH	0.05977947	0.041025806	0.32726331	0.22459619	0.32726331	1.450836e-01
1	ALCH	~~	ALCH	0.13938790	0.024579672	1.00000001	0.17634007	1.00000001	1.420816e-08
10	PTSD	~~	PTSD	0.23937808	0.106582777	0.99999999	0.44524869	0.99999999	2.470812e-02
5	MDD	~~	MDD	0.01440758	0.005525258	0.16943560	0.06497795	0.16943560	9.118313e-03
3	ANX	~~	ANX	0.23329361	0.016862213	1.00000000	0.07227893	1.00000000	1.561050e-43

A data.frame: 1 × 6
	chisq	df	p_chisq	AIC	CFI	SRMR

df	NA	0	NA	NA	NA	NA

1.0.29 20. Multivariate GWAS in Genomic SEM

So far, we estimated a genome-wide model using the genetic covariance matrix.

Now we move to multivariate GWAS.

The goal is to test SNP effects on a latent factor.

Instead of asking:

Is this SNP associated with MDD?

we ask:

Is this SNP associated with the shared genetic factor underlying MDD, PTSD, ALCH, and ANX?

This can increase power and improve interpretation when traits share genetic architecture.

1.0.30 21. Steps for Multivariate GWAS

Multivariate GWAS in Genomic SEM has four main steps:

Munge summary statistics
Run LD Score Regression
Prepare summary statistics using sumstats
Run multivariate GWAS using userGWAS

The first two steps are the same as before. If we already ran LDSC for the same traits, we do not need to repeat it.

1.0.31 22. Step 3 for Multivariate GWAS: Prepare SNP Summary Statistics

The sumstats() function prepares SNP-level summary statistics for userGWAS.

In the workshop practical, reduced chromosome 4 files are used for demonstration.

# Files must be in the same order as the LDSC traits
files <- c("ALCH4.txt", "PTSD4.txt", "MDD4.txt", "ANX4.txt")

# Reference file for allele frequency
ref <- "reference.1000G.ch4.txt"

# Trait names
trait.names <- c("ALCH", "PTSD", "MDD", "ANX")

# Whether SEs are on logistic scale
se.logit <- c(FALSE, TRUE, TRUE, TRUE)

# Whether linear probability model correction is needed
linprob <- c(TRUE, FALSE, FALSE, FALSE)

# Sample size argument
N <- c(NA, NA, NA, NA)

# Prepare summary statistics
INT_sumstats <- sumstats(
  files = files,
  ref = ref,
  trait.names = trait.names,
  se.logit = se.logit,
  linprob = linprob,
  N = N
)

The preparation of 4 summary statistics for use in Genomic SEM began at: 2026-06-10 12:44:42.44117
Please note that the files should be in the same order that they were listed for the ldsc function
Reading in reference file
Applying MAF filer of 0.01 to the reference file.

All files loaded into R!
Preparing summary statistics for file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ALCH4.txt
Interpreting the SNP column as the SNP column.
Interpreting the A1 column as the A1 column.
Interpreting the A2 column as the A2 column.
Interpreting the Z column as the effect column.
Interpreting the P column as the P column.
Interpreting the WEIGHT column as the N column.
0 rows were removed from the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ALCH4.txt summary statistics file due to entries that were duplicated for rsID. These are removed as they likely reflect multiallelic variants.
Merging file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ALCH4.txt with the reference file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/reference.1000G.ch4.txt
1000 rows present in the full /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ALCH4.txt summary statistics file.
0 rows were removed from the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ALCH4.txt summary statistics file as the rsIDs for these SNPs were not present in the reference file.
The effect column was determined NOT to be coded as an odds ratio (OR) for the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ALCH4.txt summary statistics file based on the median of the effect column being close to 0.
An transformation used to back out logistic betas for binary traits is being applied for: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ALCH4.txt
No INFO column, cannot filter on INFO, which may influence results
1000 SNPs are left in the summary statistics file /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ALCH4.txt after QC and merging with the reference file.
Preparing summary statistics for file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/PTSD4.txt
Interpreting the MARKERNAME column as the SNP column.
Interpreting the ALLELE1 column as the A1 column.
Interpreting the ALLELE2 column as the A2 column.
Interpreting the EFFECT column as the effect column.
Interpreting the P.VALUE column as the P column.
Cannot find N column, try renaming it to N in the summary statistics file for:/Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/PTSD4.txt
Interpreting the STDERR column as the SE column.
Interpreting the DIRECTION column as the DIRECTION column.
0 rows were removed from the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/PTSD4.txt summary statistics file due to entries that were duplicated for rsID. These are removed as they likely reflect multiallelic variants.
Merging file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/PTSD4.txt with the reference file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/reference.1000G.ch4.txt
1000 rows present in the full /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/PTSD4.txt summary statistics file.
0 rows were removed from the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/PTSD4.txt summary statistics file as the rsIDs for these SNPs were not present in the reference file.
The effect column was determined NOT to be coded as an odds ratio (OR) for the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/PTSD4.txt summary statistics file based on the median of the effect column being close to 0.
No INFO column, cannot filter on INFO, which may influence results
Performing transformation under the assumption that the effect column is either an odds ratio or logistic beta (please see output above to determine whether it was interpreted as an odds ratio) and the SE column is a logistic SE (i.e., NOT the SE of the odds ratio) for:/Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/PTSD4.txt
1000 SNPs are left in the summary statistics file /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/PTSD4.txt after QC and merging with the reference file.
Preparing summary statistics for file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/MDD4.txt
Found an NEFF column for sample size. 

Please note that this is likely effective sample size and should only be used for liability h^2 conversion for binary traits and that it should reflect the sum of effective sample sizes across cohorts.

Be aware that some NEFF columns reflect half of the effective sample size; the function will automatically double the column names if recognized [check above in .log file to determine if this is the case].
If the Neff value is halved in the summary stats, but not recognized by the munge function, this should be manually doubled prior to running munge.
Interpreting the MARKERNAME column as the SNP column.
Interpreting the A1 column as the A1 column.
Interpreting the A2 column as the A2 column.
Interpreting the LOGOR column as the effect column.
Interpreting the P column as the P column.
Interpreting the NEFF column as the N column.
Interpreting the MAF column as the MAF column.
Interpreting the STDERRLOGOR column as the SE column.
0 rows were removed from the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/MDD4.txt summary statistics file due to entries that were duplicated for rsID. These are removed as they likely reflect multiallelic variants.
Merging file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/MDD4.txt with the reference file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/reference.1000G.ch4.txt
1400 rows present in the full /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/MDD4.txt summary statistics file.
20 rows were removed from the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/MDD4.txt summary statistics file as the rsIDs for these SNPs were not present in the reference file.
The effect column was determined NOT to be coded as an odds ratio (OR) for the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/MDD4.txt summary statistics file based on the median of the effect column being close to 0.
5rows were removed from the/Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/MDD4.txtsummary statistics file due to effect values estimated at exactly 0 as this causes problems for matrix inversion necessary for later Genomic SEM analyses.
No INFO column, cannot filter on INFO, which may influence results
Performing transformation under the assumption that the effect column is either an odds ratio or logistic beta (please see output above to determine whether it was interpreted as an odds ratio) and the SE column is a logistic SE (i.e., NOT the SE of the odds ratio) for:/Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/MDD4.txt
1375 SNPs are left in the summary statistics file /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/MDD4.txt after QC and merging with the reference file.
Preparing summary statistics for file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ANX4.txt
Found an NEFF column for sample size. 

Please note that this is likely effective sample size and should only be used for liability h^2 conversion for binary traits and that it should reflect the sum of effective sample sizes across cohorts.

Be aware that some NEFF columns reflect half of the effective sample size; the function will automatically double the column names if recognized [check above in .log file to determine if this is the case].
If the Neff value is halved in the summary stats, but not recognized by the munge function, this should be manually doubled prior to running munge.
Interpreting the SNP column as the SNP column.
Interpreting the ALLELE1 column as the A1 column.
Interpreting the ALLELE2 column as the A2 column.
Interpreting the EFFECT column as the effect column.
Interpreting the P column as the P column.
Interpreting the NEFF column as the N column.
Interpreting the STDERR column as the SE column.
Interpreting the DIRECTION column as the DIRECTION column.
0 rows were removed from the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ANX4.txt summary statistics file due to entries that were duplicated for rsID. These are removed as they likely reflect multiallelic variants.
Merging file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ANX4.txt with the reference file: /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/reference.1000G.ch4.txt
1000 rows present in the full /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ANX4.txt summary statistics file.
0 rows were removed from the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ANX4.txt summary statistics file as the rsIDs for these SNPs were not present in the reference file.
The effect column was determined NOT to be coded as an odds ratio (OR) for the /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ANX4.txt summary statistics file based on the median of the effect column being close to 0.
No INFO column, cannot filter on INFO, which may influence results
Performing transformation under the assumption that the effect column is either an odds ratio or logistic beta (please see output above to determine whether it was interpreted as an odds ratio) and the SE column is a logistic SE (i.e., NOT the SE of the odds ratio) for:/Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ANX4.txt
1000 SNPs are left in the summary statistics file /Users/nbhadra/Documents/aSchork/ISG_2026/Scripts/script_day6/GenomicSEM_2026/ANX4.txt after QC and merging with the reference file.
After merging across all summary statistics using listwise deletion, performing QC, and merging with the reference file, there are 1000 SNPs left in the final multivariate summary statistics file
Sumstats finished running at 2026-06-10 12:44:44.381982
Running sumstats for all files took 0 minutes and 1.94081211090088 seconds
Please check the log file ALCH_PTSD_MDD_ANX_sumstats.log to ensure that all columns were interpreted correctly and no warnings were issued for any of the summary statistics files.

1.0.32 23. Why `linprob` Is Needed

Sometimes binary traits are analyzed as continuous outcomes, or the summary statistics contain only Z-statistics.

In these cases, Genomic SEM needs additional information to reconstruct the appropriate beta and standard error.

In this practical, linprob = TRUE is used for ALCH.

1.0.33 24. Step 4: Run userGWAS

Now we run a multivariate GWAS.

The model is:

$ F1 =~ MDD + PTSD + ALCH + ANX $

and then:

$ F1 SNP $

This tests whether each SNP predicts the shared latent genetic factor.

# Covariance structure from LDSC
covstruc <- LDSC_INT

# SNP-level summary statistics
SNPs <- INT_sumstats

# Multivariate GWAS model
model <- "
F1 =~ MDD + PTSD + ALCH + ANX
F1 ~ SNP
"

# Save only the SNP effect on F1
sub <- "F1~SNP"

# Practical uses serial processing
parallel <- FALSE

# Calculate QSNP
Q_SNP <- TRUE

# Run multivariate GWAS
INT_GWAS <- userGWAS(
  covstruc = covstruc,
  SNPs = SNPs,
  model = model,
  sub = sub,
  parallel = parallel,
  Q_SNP = Q_SNP
)

[1] "Please note that an update was made to userGWAS on Sept 1 2023  so that the default behavior is to fix the measurement model using the fix_measurement argument."
[1] "Starting GWAS Estimation"
Running Model: 1
Running Model: 1000
elapsed 
 26.264

1.0.34 25. View the First Rows of the Multivariate GWAS Output

The output contains:

SNP effect on the factor
Standard error
p-value
QSNP statistic
QSNP p-value
Warnings and errors

INT_GWAS[[1]][1:5, ]

A data.frame: 5 × 24
	SNP	CHR	BP	MAF	A1	A2	lhs	op	rhs	free	⋯	Pval_Estimate	chisq	chisq_df	chisq_pval	AIC	Q_SNP	Q_SNP_df	Q_SNP_pval	error	warning
											⋯
1	rs10030871	4	68786	0.0765408	T	C	F1	~	SNP	6	⋯	0.8257532	4.905573	8	0.7676193	18.90557	3.622121	3	0.3052654	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
2	rs6599368	4	69567	0.0755467	A	T	F1	~	SNP	6	⋯	0.8059251	4.839517	8	0.7745831	18.83952	3.556062	3	0.3135635	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
3	rs7678633	4	69713	0.0755467	G	A	F1	~	SNP	6	⋯	0.7979215	4.901617	8	0.7680379	18.90162	3.618164	3	0.3057569	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
4	rs13130581	4	70392	0.0725646	A	G	F1	~	SNP	6	⋯	0.9840311	5.029483	8	0.7544203	19.02948	3.746031	3	0.2902263	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
5	rs13125929	4	71566	0.0725646	T	C	F1	~	SNP	6	⋯	0.9069168	5.787662	8	0.6710031	19.78766	4.504209	3	0.2119151	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se

1.0.35 26. Check Warnings and Errors

Before interpreting results, always check warnings and errors.

A value of 0 usually means the SNP was estimated successfully.

# Warnings
table(INT_GWAS[[1]]$warning)

# Errors
table(INT_GWAS[[1]]$error)


lavaan->lav_lavaan_step02_options():  \n   the following argument(s) override(s) the options in slotOptions: se 
                                                                                                           1000


   0 
1000

1.0.36 27. Genome-Wide Significant Factor Hits

The standard genome-wide significance threshold is:

$ 5 ^{-8} $

We can check how many SNPs are significantly associated with the latent factor.

table(INT_GWAS[[1]]$Pval_Estimate < 5e-8)


FALSE 
 1000

1.0.37 28. QSNP: Testing SNP-Level Heterogeneity

QSNP tests whether the SNP effect fits the common pathway model.

If a SNP acts mainly through the common factor, then its effects on individual traits should be consistent with the factor structure.

If the SNP has a strong trait-specific effect, the common pathway model may not fit well.

A significant QSNP value suggests SNP-level heterogeneity.

This means the SNP may affect one or more traits differently than expected from the common factor model.

QSNP can be used in two ways:

As a quality-control metric As a biological result

As a QC metric, it can help identify SNPs that do not cleanly represent the common factor.

As a biological result, it can highlight SNPs that differentiate related traits.

1.0.38 29. QSNP as a QC Metric

Suppose a SNP is significantly associated with an internalizing factor.

If QSNP is not significant, this suggests that the SNP effect is consistent with the shared factor.

But if QSNP is highly significant, the SNP may not be a clean factor SNP.

It may be driven mainly by one phenotype.

For example, a SNP may look associated with an internalizing factor, but the effect may actually be much stronger for depression than anxiety or PTSD.

In that case, QSNP warns us that the SNP may have trait-specific effects.

1.0.39 30. QSNP as a Biological Result

QSNP is not only a warning. It can also be scientifically interesting.

A significant QSNP result may reveal SNPs that distinguish traits that are otherwise genetically correlated.

For example, depression and anxiety may share a broad genetic factor, but some variants may be more specific to depression.

Similarly, schizophrenia and bipolar disorder may share genetic liability, but some variants may help differentiate them.

Therefore, QSNP can be used to study genetic specificity within broader shared genetic architecture.

1.0.40 31. Multiple-Factor Models and QSNP

In models with multiple factors, QSNP can be calculated for each SNP-factor relationship.

This is useful because a SNP may fit one factor well but show heterogeneity for another factor.

For example, a SNP may operate cleanly through a substance-use factor but show disorder-specific effects within an internalizing factor.

This makes QSNP especially useful in complex psychiatric models where several correlated factors are estimated together.

# Genome-wide significant QSNP hits
table(INT_GWAS[[1]]$Q_SNP_pval < 5e-8)

# Suggestive QSNP hits
table(INT_GWAS[[1]]$Q_SNP_pval < 1e-5)


FALSE 
 1000


FALSE  TRUE 
  926    74

1.0.41 29. Interpreting QSNP

QSNP can be interpreted in two ways.

First, it can be used as a quality-control metric. If a SNP has a strong factor association but also a very significant QSNP value, the SNP may not represent the common factor cleanly.

Second, QSNP can be a result of interest. It may identify SNPs that differentiate traits from one another.

For example, a SNP may be associated with depression but not anxiety, even though both traits load on the same internalizing factor.

View(INT_GWAS[[1]])

A data.frame: 1000 × 24
SNP	CHR	BP	MAF	A1	A2	lhs	op	rhs	free	⋯	Pval_Estimate	chisq	chisq_df	chisq_pval	AIC	Q_SNP	Q_SNP_df	Q_SNP_pval	error	warning
										⋯
rs10030871	4	68786	0.0765408	T	C	F1	~	SNP	6	⋯	0.8257532	4.905573	8	0.7676193	18.90557	3.622121	3	0.30526541	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs6599368	4	69567	0.0755467	A	T	F1	~	SNP	6	⋯	0.8059251	4.839517	8	0.7745831	18.83952	3.556062	3	0.31356354	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs7678633	4	69713	0.0755467	G	A	F1	~	SNP	6	⋯	0.7979215	4.901617	8	0.7680379	18.90162	3.618164	3	0.30575689	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13130581	4	70392	0.0725646	A	G	F1	~	SNP	6	⋯	0.9840311	5.029483	8	0.7544203	19.02948	3.746031	3	0.29022634	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13125929	4	71566	0.0725646	T	C	F1	~	SNP	6	⋯	0.9069168	5.787662	8	0.6710031	19.78766	4.504209	3	0.21191512	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs6839041	4	72048	0.2693840	T	G	F1	~	SNP	6	⋯	0.9978168	3.541053	8	0.8959803	17.54105	2.257601	3	0.52069205	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs6851329	4	72303	0.0725646	A	G	F1	~	SNP	6	⋯	0.9886240	4.880091	8	0.7703120	18.88009	3.596639	3	0.30844301	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs10027065	4	72939	0.0755467	T	C	F1	~	SNP	6	⋯	0.7715593	4.943569	8	0.7635902	18.94357	3.660117	3	0.30058151	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs4690284	4	73508	0.1451290	C	T	F1	~	SNP	6	⋯	0.1241614	2.966177	8	0.9364621	16.96618	1.682725	3	0.64078118	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13119939	4	73981	0.0725646	G	A	F1	~	SNP	6	⋯	0.9614800	5.132825	8	0.7432898	19.13283	3.849373	3	0.27819520	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs6599371	4	74015	0.0755467	A	G	F1	~	SNP	6	⋯	0.8128254	4.837074	8	0.7748395	18.83707	3.553622	3	0.31387385	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13125667	4	74238	0.0725646	G	A	F1	~	SNP	6	⋯	0.9843619	5.225879	8	0.7331836	19.22588	3.942427	3	0.26774849	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs10001263	4	74508	0.0755467	G	A	F1	~	SNP	6	⋯	0.7706384	5.281466	8	0.7271125	19.28147	3.998014	3	0.26167870	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs11722512	4	74639	0.1819090	A	T	F1	~	SNP	6	⋯	0.9216326	7.046880	8	0.5315833	21.04688	5.763430	3	0.12370462	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13109067	4	74815	0.0755467	A	G	F1	~	SNP	6	⋯	0.8393225	4.908892	8	0.7672680	18.90889	3.625440	3	0.30485367	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13148549	4	74983	0.0755467	T	G	F1	~	SNP	6	⋯	0.7773565	5.186565	8	0.7374625	19.18656	3.903112	3	0.27211808	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13114862	4	75102	0.0765408	A	G	F1	~	SNP	6	⋯	0.8046513	4.687451	8	0.7903995	18.68745	3.403999	3	0.33342816	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs60994255	4	77446	0.1580520	G	A	F1	~	SNP	6	⋯	0.9382913	6.899311	8	0.5475343	20.89931	5.615860	3	0.13187081	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs59464888	4	77449	0.1640160	G	A	F1	~	SNP	6	⋯	0.8781014	5.984696	8	0.6489462	19.98470	4.701244	3	0.19502698	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13435282	4	79340	0.1928430	G	A	F1	~	SNP	6	⋯	0.6215750	7.730510	8	0.4602277	21.73051	6.447058	3	0.09177397	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13434621	4	79535	0.1938370	C	G	F1	~	SNP	6	⋯	0.5791460	6.954466	8	0.5415533	20.95447	5.671014	3	0.12876046	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13145153	4	79719	0.0765408	A	G	F1	~	SNP	6	⋯	0.6384795	4.872118	8	0.7711529	18.87212	3.588666	3	0.30944321	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs59603164	4	80001	0.1749500	C	T	F1	~	SNP	6	⋯	0.8474933	8.281415	8	0.4064747	22.28142	6.997963	3	0.07196273	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs7664056	4	80132	0.0765408	A	G	F1	~	SNP	6	⋯	0.6660829	4.539084	8	0.8055102	18.53908	3.255634	3	0.35386546	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs28716751	4	81303	0.0815109	T	C	F1	~	SNP	6	⋯	0.3150613	2.516640	8	0.9609505	16.51664	1.233188	3	0.74505586	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs2187874	4	82321	0.1640160	G	T	F1	~	SNP	6	⋯	0.7757893	6.046779	8	0.6419918	20.04678	4.763327	3	0.18997076	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs60400620	4	84610	0.1918490	A	C	F1	~	SNP	6	⋯	0.6593662	7.660052	8	0.4673615	21.66005	6.376598	3	0.09465829	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs61447907	4	84792	0.1918490	C	T	F1	~	SNP	6	⋯	0.6546304	7.596372	8	0.4738560	21.59637	6.312919	3	0.09733980	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs7667153	4	85422	0.1918490	G	A	F1	~	SNP	6	⋯	0.6540012	7.607574	8	0.4727104	21.60757	6.324123	3	0.09686277	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs61792225	4	85526	0.1739560	C	T	F1	~	SNP	6	⋯	0.7963943	8.833127	8	0.3565683	22.83313	7.549675	3	0.05629572	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋱	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮
rs6834433	4	100294905	0.2077530	T	C	F1	~	SNP	6	⋯	0.8712175	13.430376	8	0.0978747603	27.43038	12.1469250	3	6.896436e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs6857143	4	100294928	0.2077530	A	G	F1	~	SNP	6	⋯	0.9057906	13.365897	8	0.0998650553	27.36590	12.0824435	3	7.106014e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs6834464	4	100294945	0.2077530	T	G	F1	~	SNP	6	⋯	0.8381863	13.759848	8	0.0882436578	27.75985	12.4763917	3	5.917295e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13121469	4	100295047	0.1172960	C	T	F1	~	SNP	6	⋯	0.4026174	26.646167	8	0.0008136034	40.64617	25.3627149	3	1.296593e-05	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs56279505	4	100295240	0.0964215	T	A	F1	~	SNP	6	⋯	0.8475897	31.241304	8	0.0001272647	45.24130	29.9578521	3	1.408519e-06	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs1121534	4	100295415	0.2087480	T	C	F1	~	SNP	6	⋯	0.8676112	13.834399	8	0.0861851697	27.83440	12.5509449	3	5.715560e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs6827898	4	100295863	0.1172960	G	A	F1	~	SNP	6	⋯	0.3756755	24.804186	8	0.0016778555	38.80419	23.5207303	3	3.144944e-05	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs10030511	4	100296248	0.4473160	C	T	F1	~	SNP	6	⋯	0.5111428	6.108862	8	0.6350390678	20.10886	4.8254096	3	1.850371e-01	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs10032906	4	100296355	0.2067590	G	A	F1	~	SNP	6	⋯	0.9844021	16.448897	8	0.0363875630	30.44890	15.1654448	3	1.680586e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs34980886	4	100296402	0.1172960	T	A	F1	~	SNP	6	⋯	0.3479156	24.096953	8	0.0022075521	38.09695	22.8135003	3	4.416375e-05	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs10000067	4	100296476	0.2067590	T	C	F1	~	SNP	6	⋯	0.9678549	16.609195	8	0.0344457602	30.60920	15.3257420	3	1.558415e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs76088139	4	100296678	0.2067590	C	A	F1	~	SNP	6	⋯	0.9823261	16.046919	8	0.0417134522	30.04692	14.7634672	3	2.030356e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs35208813	4	100296684	0.4483100	T	C	F1	~	SNP	6	⋯	0.4735511	5.846655	8	0.6644040158	19.84666	4.5632037	3	2.067215e-01	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs35796885	4	100296748	0.4483100	T	C	F1	~	SNP	6	⋯	0.4841398	6.394731	8	0.6031062592	20.39473	5.1112786	3	1.638278e-01	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs12651165	4	100296791	0.2067590	C	T	F1	~	SNP	6	⋯	0.9046283	16.548481	8	0.0351695859	30.54848	15.2650294	3	1.603614e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs66503521	4	100297205	0.2067590	C	T	F1	~	SNP	6	⋯	0.9029744	16.037486	8	0.0418467104	30.03749	14.7540322	3	2.039379e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs12502498	4	100297551	0.3240560	T	C	F1	~	SNP	6	⋯	0.4844888	5.787124	8	0.6710632139	19.78712	4.5036703	3	2.119631e-01	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs28818226	4	100297591	0.2067590	T	C	F1	~	SNP	6	⋯	0.9857570	16.463681	8	0.0362042963	30.46368	15.1802287	3	1.668932e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs9991913	4	100297896	0.2067590	G	T	F1	~	SNP	6	⋯	0.9932682	16.589447	8	0.0346796579	30.58945	15.3059960	3	1.572975e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs9992022	4	100297998	0.2067590	G	C	F1	~	SNP	6	⋯	0.9696586	16.470744	8	0.0361170551	30.47074	15.1872930	3	1.663391e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs13110291	4	100298393	0.1182900	A	G	F1	~	SNP	6	⋯	0.3829837	23.222449	8	0.0030900849	37.22245	21.9389962	3	6.716566e-05	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs56899508	4	100298398	0.1182900	G	T	F1	~	SNP	6	⋯	0.3767382	23.128751	8	0.0032028187	37.12875	21.8452984	3	7.024885e-05	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs9994612	4	100298399	0.2067590	G	A	F1	~	SNP	6	⋯	0.9960079	16.491560	8	0.0358610431	30.49156	15.2081083	3	1.647172e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs283406	4	100298471	0.0894632	G	A	F1	~	SNP	6	⋯	0.2162775	1.778214	8	0.9870725654	15.77821	0.4947615	3	9.200408e-01	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs10017136	4	100298681	0.2067590	A	C	F1	~	SNP	6	⋯	0.9678550	16.543299	8	0.0352320187	30.54330	15.2598444	3	1.607534e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs9997820	4	100299450	0.4502980	G	A	F1	~	SNP	6	⋯	0.4744628	6.772480	8	0.5613676486	20.77248	5.4890277	3	1.392964e-01	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs9997653	4	100299453	0.3260440	C	A	F1	~	SNP	6	⋯	0.4702161	5.658205	8	0.6854562084	19.65821	4.3747501	3	2.237381e-01	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs1908962	4	100299641	0.4483100	C	A	F1	~	SNP	6	⋯	0.5317039	6.267869	8	0.6172541670	20.26787	4.9844168	3	1.729418e-01	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs1908963	4	100299664	0.2067590	T	C	F1	~	SNP	6	⋯	0.9495890	16.835809	8	0.0318649312	30.83581	15.5523550	3	1.400596e-03	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se
rs1837932	4	100299978	0.4483100	T	C	F1	~	SNP	6	⋯	0.5111001	6.685218	8	0.5709443237	20.68522	5.4017663	3	1.446336e-01	0	lavaan->lav_lavaan_step02_options(): the following argument(s) override(s) the options in slotOptions: se

1.0.42 31. Practice Section: Anthropometric Traits

The workshop also provides an additional LDSC object for anthropometric traits.

These include:

BMI: Body Mass Index
WHR: Waist-Hip Ratio
Waist: Waist Circumference
Hip: Hip Circumference
CO: Childhood Obesity
Height: Height
BL: Birth Length
BW: Birth Weight
IHC: Infant Head Circumference

The goal is to practice specifying your own model.

# Load anthropometric LDSC object
load("Anthro_LDSC.RData")

# Define covariance structure
covstruc <- anthro

# Check available traits
colnames(anthro$S)

'BMI'
'WHR'
'CO'
'Waist'
'Hip'
'Height'
'IHC'
'BL'
'BW'

1.0.43 32. Example Anthropometric Factor Model

One possible model is a general body-size factor.

This is only an example. In real research, you should write down your model before running it.

Your.Model <- "
BodySize =~ BMI + WHR + Waist + Hip
"

std.lv <- TRUE

YourResults <- usermodel(
  covstruc = covstruc,
  model = Your.Model,
  std.lv = std.lv
)

YourResults$results
YourResults$modelfit

[1] "Running primary model"
[1] "Calculating Standardized Results"
[1] "Calculating SRMR"
elapsed 
  0.099 
[1] "The S matrix was smoothed prior to model estimation due to a non-positive definite matrix. The largest absolute difference in a cell between the smoothed and non-smoothed matrix was  5.03885789198688e-05 As a result of the smoothing, the largest Z-statistic change for the genetic covariances was  0.00627650037966632 . We recommend setting the smooth_check argument to true if you are going to run a multivariate GWAS."

A data.frame: 9 × 9
	lhs	op	rhs	Unstand_Est	Unstand_SE	STD_Genotype	STD_Genotype_SE	STD_All	p_value

2	BodySize	=~	BMI	0.32835242	0.00834346434861948	0.9261942	0.0235346832851601	0.9261941	< 5e-300
5	BodySize	=~	WHR	0.17364843	0.00899442936403122	0.5823406	0.0301633569935819	0.5823405	4.76178089813194e-83
4	BodySize	=~	Waist	0.37150337	0.00849900513137077	1.0554343	0.0241454997322284	1.0554343	< 5e-300
3	BodySize	=~	Hip	0.29921533	0.00998707625847693	0.8166573	0.0272580315696167	0.8166573	3.23530305105103e-197
1	BMI	~~	BMI	0.01786761	0.00214989324612072	0.1421646	0.0171056871098916	0.1421646	9.49574545542005e-17
9	WHR	~~	WHR	0.05876384	0.00500826060659902	0.6608797	0.0563247279072853	0.6608795	8.59540394910649e-32
8	Waist	~~	Waist	-0.01411697	0.0014484156789879	-0.1139415	0.0116904301220693	-0.1139415	1.90955940051609e-22
7	Hip	~~	Hip	0.04471200	0.00409589113627207	0.3330708	0.0305112937271513	0.3330708	9.6335396382315e-28
6	BodySize	~~	BodySize	1.00000000		1.0000000		1.0000000	NA

A data.frame: 1 × 6
	chisq	df	p_chisq	AIC	CFI	SRMR

df	4808.273	2	0	4824.273	0.9790462	0.07075376

1.0.44 33. Example Two-Factor Model

We may hypothesize that body mass and early growth are partly distinct genetic dimensions.

For example:

AdultBody: BMI, WHR, Waist, Hip
EarlyGrowth: BL, BW, IHC

TwoFactor.Model <- "
AdultBody =~ BMI + WHR + Waist + Hip
EarlyGrowth =~ BL + BW + IHC

AdultBody ~~ EarlyGrowth
"

TwoFactorResults <- usermodel(
  covstruc = covstruc,
  model = TwoFactor.Model,
  std.lv = TRUE
)

TwoFactorResults$results
TwoFactorResults$modelfit

[1] "Running primary model"
[1] "Calculating Standardized Results"
[1] "Calculating SRMR"
elapsed 
  0.139 
[1] "The S matrix was smoothed prior to model estimation due to a non-positive definite matrix. The largest absolute difference in a cell between the smoothed and non-smoothed matrix was  7.68229238531509e-05 As a result of the smoothing, the largest Z-statistic change for the genetic covariances was  0.00971267593380176 . We recommend setting the smooth_check argument to true if you are going to run a multivariate GWAS."

A data.frame: 17 × 9
	lhs	op	rhs	Unstand_Est	Unstand_SE	STD_Genotype	STD_Genotype_SE	STD_All	p_value

1	AdultBody	=~	BMI	0.32355145	0.00848257151119238	0.9126525	0.0239270666230049	0.9126526	< 5e-300
4	AdultBody	=~	WHR	0.17014707	0.00915015430923173	0.5705696	0.0306840495730172	0.5705696	3.52749230117905e-77
3	AdultBody	=~	Waist	0.37262394	0.00839616539135095	1.0585040	0.0238508087911616	1.0585043	< 5e-300
2	AdultBody	=~	Hip	0.30767486	0.00991987368769025	0.8397060	0.0270733262889365	0.8397059	3.27994599611528e-211
10	EarlyGrowth	=~	BL	0.31883086	0.0424505344230068	0.7863983	0.104704413507432	0.7863989	5.88371366564811e-14
11	EarlyGrowth	=~	BW	0.25182147	0.0369779692381788	0.7456392	0.109491090900721	0.7456389	9.75711480693518e-12
12	EarlyGrowth	=~	IHC	0.38776135	0.0639504168355385	0.8067723	0.133054274151609	0.8067722	1.33216727234899e-09
6	AdultBody	~~	EarlyGrowth	0.20561960	0.0457670518741615	0.2056203	0.0457670449740265	0.2056203	7.03116016834168e-06
8	BMI	~~	BMI	0.02099732	0.00236172074867183	0.1670651	0.0187911396306323	0.1670652	6.07311168507216e-19
17	WHR	~~	WHR	0.05997650	0.00508748828560331	0.6744501	0.0572099863458111	0.6744503	4.44686078822693e-32
16	Waist	~~	Waist	-0.01492436	0.00155917314029345	-0.1204312	0.0125816586271371	-0.1204313	1.04884008907025e-21
14	Hip	~~	Hip	0.03959078	0.00385032587902093	0.2948940	0.0286792741025182	0.2948940	8.45512132535067e-25
15	IHC	~~	IHC	0.08065102	0.058447583363325	0.3491186	0.253010103779767	0.3491186	0.167621642770187
7	BL	~~	BL	0.06272152	0.0309055815553739	0.3815762	0.188018927129747	0.3815767	0.0424118546489747
9	BW	~~	BW	0.05064431	0.0212845979481534	0.4440231	0.186611198922835	0.4440227	0.0173414200434353
5	AdultBody	~~	AdultBody	1.00000000		1.0000000		1.0000000	NA
13	EarlyGrowth	~~	EarlyGrowth	1.00000000		1.0000000		1.0000000	NA

A data.frame: 1 × 6
	chisq	df	p_chisq	AIC	CFI	SRMR

df	4994.695	13	0	5024.695	0.9822088	0.09017272

1.0.45 34. Comparing Models

We can compare models using fit statistics such as AIC, CFI, and SRMR.

Lower AIC suggests a better-fitting model among competing models.

Higher CFI and lower SRMR suggest better absolute fit.

YourResults$modelfit
TwoFactorResults$modelfit

A data.frame: 1 × 6
	chisq	df	p_chisq	AIC	CFI	SRMR

df	4808.273	2	0	4824.273	0.9790462	0.07075376

A data.frame: 1 × 6
	chisq	df	p_chisq	AIC	CFI	SRMR

df	4994.695	13	0	5024.695	0.9822088	0.09017272

1.0.46 35. Important Practical Advice

Genomic SEM is flexible, but this flexibility should be used carefully.

Do not simply try many models until one gives a significant result.

A better workflow is:

Define your research question.
Write down your hypothesized model.
Run the model.
Check model fit.
Interpret parameters cautiously.
If using data-driven exploration, clearly report it as exploratory.

This is especially important in multivariate genetic modeling because many reasonable models may appear plausible.

1.0.47 36. Summary

In this tutorial, we covered the main concepts behind multivariate genetic analysis using Genomic SEM.

We learned that complex traits are highly polygenic and that LD Score Regression uses the relationship between LD score and GWAS signal to estimate SNP heritability.

We then extended this idea to genetic covariance and genetic correlation across traits.

Finally, we introduced Genomic SEM as a framework for fitting structural models to genetic covariance matrices and for running multivariate GWAS on latent genetic factors.

The key idea is simple but powerful:

Instead of studying each trait separately, Genomic SEM allows us to model shared and trait-specific genetic architecture across multiple traits.

This makes it especially useful for psychiatric, behavioral, anthropometric, and medical traits where genetic overlap is common.

1.0.48 37. Key Takeaways

Complex traits are usually highly polygenic.
LD scores summarize how much LD each SNP has with nearby SNPs.
LD Score Regression estimates SNP heritability from GWAS summary statistics.
Cross-trait LD Score Regression estimates genetic covariance and genetic correlation.
Genomic SEM fits structural equation models to genetic covariance matrices.
A common factor model can capture shared genetic liability across traits.
Multivariate GWAS can test SNP effects on latent genetic factors.
QSNP tests whether SNP effects are consistent with the common factor model or show trait-specific heterogeneity.

Heritability and Genetic Correlation: From First Principles to Summary Statistics

Nivedita Bhadra — Sun, 26 Jul 2026 21:00:00 GMT

The core duality this tutorial builds toward: HE regression works in the space of individuals (relatedness), LDSC works in the space of markers (LD score) – but in both, heritability is just a regression slope. These panels are plotted from the tutorial’s own simulated data.

Heritability LDSC GREML R

Every quantitative genetics question eventually runs into the same two words: heritability and genetic correlation. How much of height is genetic? Do depression and obesity share genetic causes? Is a GWAS signal confounded by population structure?

Answering any of these requires estimating variance and covariance — not of the phenotype itself, but of its genetic component. This tutorial builds that machinery from the ground up: starting with individual-level methods you could run by hand on a handful of relatives, and ending with the summary-statistics methods (LD Score Regression) that power essentially every modern GWAS consortium paper.

What you’ll learn

What “heritability” actually means — and why there are three different definitions

How to estimate heritability from relatives, using Haseman-Elston regression

Why binary disease traits need special handling (the liability threshold model)

How LD Score Regression estimates heritability from GWAS summary statistics alone

What genetic correlation is, and the four different reasons two traits can share genetic architecture

The practical pitfalls that make genetic correlation estimates misleading if you’re not careful

0.1 Why Do We Need a Formal Definition of Heritability at All?

Francis Galton noticed over a century ago that relatives resemble each other more than random pairs of people, and that identical twins resemble each other more than fraternal twins. A large meta-analysis of fifty years of twin studies later put a number on this: an average heritability estimate of 49% across thousands of traits, with 69% of studies supporting a purely additive genetic model (Polderman et al., 2015).

But “heritability” turns out to be a slippery word. Before we can estimate it, we need to agree on exactly what quantity we’re estimating.

1 Part 1 — Three Definitions of Heritability

An estimand is the true underlying population parameter we’re actually trying to estimate. It sounds like a pedantic distinction, but different heritability methods target genuinely different estimands — and comparing numbers across methods without realizing this is a common source of confusion.

1.1 Fixed-Effect Realised Variance

This asks: of the phenotypic variance in this specific sample, how much is explained by the genotypes we actually observed? It’s a property of the realized data, not an abstract population parameter.

1.2 Random-Individual Expected Variance

Here, genetic effects are treated as random draws from a distribution, and — the Genetic Relationship Matrix (GRM) — captures how genetically similar each pair of individuals is. This is the estimand targeted by methods like GREML/GCTA (Part 3).

1.3 Random-Marker Expected Variance

Here it’s the SNP effects that are random, each drawn with variance (spread evenly across SNPs). The estimand is directly. This is the estimand targeted by LD Score Regression (Part 5).

1.4 Why This Distinction Matters in Practice

GREML and LDSC are both routinely described as estimating “SNP heritability,” but they’re formally targeting different estimands under different assumptions about what’s random (individuals vs. markers). In practice they tend to agree well for polygenic traits with good data — but when they disagree, the difference isn’t necessarily a bug in one of the methods; it can reflect which estimand each one is actually built to target.

Key takeaways

“Heritability” has at least three distinct formal definitions, differing in what’s treated as random (nothing, individuals, or markers).

GREML targets the random-individual estimand; LD Score Regression targets the random-marker estimand.

Keep this in mind before treating heritability estimates from different methods as directly interchangeable.

2 Part 2 — Haseman-Elston Regression: Heritability from First Principles

Before GWAS existed, before genome-wide SNP arrays existed, geneticists were already estimating heritability — from relatives. Haseman-Elston (HE) regression, developed by J.K. Haseman and R.C. Elston in 1972, is the conceptual ancestor of nearly every method in this tutorial, including LD Score Regression itself.

2.1 The Core Premise

If a trait is influenced by genetics, relatives who are more genetically similar should have more similar phenotypes.

HE regression turns this intuition into a simple ordinary least squares (OLS) regression — no maximum likelihood, no iterative optimization, just a linear regression of phenotypic similarity on genetic similarity.

2.2 Setting Up the Model

Let and be the (mean-centered) phenotypes of individuals and , so . The basic additive model is , giving:

Assuming no shared environment, phenotypic covariance between two relatives is driven entirely by the proportion of genome they share identical by descent, :

2.3 Version 1 (1972): Squared Differences

Haseman and Elston’s original approach measured phenotypic similarity using the squared difference . Expanding and substituting the variance/covariance definitions above:

This gives a linear regression , where the intercept estimates (twice total phenotypic variance) and the slope estimates . So: regress squared phenotypic differences on genetic relatedness, then divide the slope by to recover .

2.4 Version 2 (2000): Cross-Products — Simpler and More Powerful

Elston and colleagues later proposed a cleaner alternative: use the cross-product of mean-centered phenotypes, , instead of the squared difference.

Why is this better? Squared differences mix together the covariance (the genetic signal we actually want) with each individual’s own variance (which includes environmental noise). Cross-products isolate the covariance directly. Since :

so directly. This gives an even simpler regression:

Now the intercept is expected to be exactly zero (assuming no shared environment), and the slope estimates directly — no dividing by required.

2.5 A Simulated Illustration

# Simulate a simple additive-genetic scenario across pairs of relatives
set.seed(1)
n_pairs <- 2000

# Relatedness (pi_ij): 0.5 for siblings, 0.25 for half-sibs/grandparent, etc.
pi_ij <- sample(c(0.0, 0.125, 0.25, 0.5), n_pairs, replace = TRUE,
                 prob = c(0.25, 0.25, 0.25, 0.25))

sigma_g2 <- 0.4   # true additive genetic variance
sigma_e2 <- 0.6   # true environmental variance

# Simulate mean-centered phenotype pairs consistent with this relatedness structure
y_i <- rnorm(n_pairs, 0, sqrt(sigma_g2 + sigma_e2))
noise <- rnorm(n_pairs, 0, sqrt((1 - pi_ij) * sigma_g2 + sigma_e2))
y_j <- pi_ij * y_i + noise   # illustrative construction, not a full pedigree simulator

# Cross-product HE regression
C_ij <- y_i * y_j
he_fit <- lm(C_ij ~ pi_ij)
summary(he_fit)   # slope should recover something in the neighborhood of sigma_g2


Call:
lm(formula = C_ij ~ pi_ij)

Residuals:
    Min      1Q  Median      3Q     Max 
-4.7523 -0.5138 -0.1174  0.3520 10.3084 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -0.02636    0.03741  -0.705    0.481    
pi_ij        1.09987    0.13514   8.139 6.93e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.089 on 1998 degrees of freedom
Multiple R-squared:  0.03209,   Adjusted R-squared:  0.03161 
F-statistic: 66.24 on 1 and 1998 DF,  p-value: 6.93e-16

This is a simplified illustration of the logic, not a full quantitative-genetics simulator (real HE regression is normally run on GRM values estimated from genome-wide SNP data, not a handful of discrete relatedness categories) — but it captures the essential mechanic: regress phenotype cross-products on genetic relatedness, and the slope is your heritability estimate.

Key takeaways

HE regression turns “genetically-similar relatives resemble each other phenotypically” into a simple linear regression.

The original (1972) squared-difference version requires dividing the slope by .

The revised (2000) cross-product version is both simpler and more statistically powerful, because it isolates covariance without mixing in individual-level variance.

HE regression is the conceptual foundation for LD Score Regression, covered in Part 5.

3 Part 3 — GREML, GCTA, and the Genetic Relationship Matrix

HE regression works well conceptually, but modern genome-wide SNP data calls for a slightly different formalization: GREML (Genomic REstricted Maximum Likelihood), implemented in the widely-used GCTA software (Yang et al.).

3.1 From OLS to a Mixed Model

Instead of a simple linear regression, GREML fits a random-effects mixed model:

where is the Genetic Relationship Matrix (GRM) — the genome-wide analog of the pairwise relatedness from HE regression, but estimated directly from SNP genotypes rather than known pedigree relationships. This targets the random-individual expected variance estimand from Part 1: .

3.2 Building the GRM

The GRM quantifies genetic similarity between every pair of individuals in a sample, estimated from genome-wide genotype data rather than known family relationships — which is what makes it applicable to unrelated individuals in large cohorts like biobanks, not just close relatives.

	Person A	Person B	Person C
A	1.00	0.10	0.02
B	0.10	1.00	0.08
C	0.02	0.08	1.00

The diagonal (self-relatedness) is close to 1; off-diagonal entries reflect how much genome-wide genetic material each pair shares. The core logic is the same intuition as HE regression: if genetics influences the trait, individuals who are more genetically similar (higher GRM entries) should be more phenotypically similar too — GREML just estimates this via restricted maximum likelihood rather than OLS, over a full genome-wide GRM rather than a handful of relatedness categories.

3.3 Why GREML Matters

GREML/GCTA was one of the methods that first demonstrated “missing heritability” could be recovered using genome-wide SNP data on unrelated individuals — showing that much of the heritability twin studies had estimated really was captured by common SNPs, just not by the small number of genome-wide-significant hits any single GWAS had found. This distinction between “heritability captured by genome-wide-significant SNPs” and “heritability captured by all measured SNPs” (the latter usually much larger) remains one of the central themes of statistical genetics.

Key takeaways

GREML uses a genome-wide Genetic Relationship Matrix (GRM) in a random-effects mixed model, estimated via restricted maximum likelihood.

It targets the random-individual expected variance estimand: .

GREML requires individual-level genotype data, unlike the summary-statistics methods covered later in this tutorial.

4 Part 4 — Binary Traits: The Liability Threshold Model and PCGC

Everything in Parts 2–3 assumed a continuous, normally-distributed phenotype. Most disease traits, though, are binary: case or control. This creates a real problem for naive heritability estimation.

4.1 The Ascertainment Problem

Case-control GWAS deliberately oversample cases. A disease might have a true population prevalence of 1%, but researchers typically recruit a sample that’s 50% cases — otherwise there wouldn’t be enough cases to have any statistical power at all.

If you plug 0/1 case-control status directly into HE regression, you get a heritability estimate on the observed scale (). Because of the deliberate oversampling, this number is heavily biased and not biologically meaningful — it reflects your sampling design as much as the underlying genetics.

4.2 The Liability Threshold Model

The standard fix: assume the binary trait is driven by an unobserved, continuous, normally-distributed liability.

Everyone has a liability score, whether or not they’re a case.
If your liability crosses a threshold , you become a case ().
The threshold is set by the population prevalence — rarer diseases have a higher threshold.

The quantity we actually want isn’t the heritability of the observed 0/1 scale — it’s the heritability of the underlying continuous liability scale, .

4.3 PCGC: Fixing the Regression Directly

Golan et al. (2014) developed PCGC (“Phenotype Correlation–Genotype Correlation”) to solve this within the HE cross-product framework directly, rather than computing and applying an after-the-fact correction (which turns out to be mathematically unreliable under severe ascertainment).

Let be the height of the standard normal density at the liability threshold , the case proportion in the ascertained sample, and the true population prevalence. PCGC shows that the expected phenotypic covariance in an ascertained sample is:

The fix is elegant: scale the observed cross-products by this prevalence-derived constant before regressing on the GRM (), exactly as in ordinary HE regression. The result is a clean, directly-interpretable estimate of liability-scale heritability.

4.4 Why This Matters in Practice

Comparing raw observed-scale estimates across two case-control studies of the same disease — say, one with 30% cases and one with 50% cases — is comparing apples to oranges, since the observed-scale number depends on the ascertainment ratio, not just the underlying biology. Always convert to the liability scale before comparing heritability estimates across binary-trait studies, especially ones with different sampling designs or disease prevalence assumptions.

Key takeaways

Case-control ascertainment biases naive (observed-scale) heritability estimates for binary traits.

The liability threshold model reframes disease as an unobserved continuous trait crossing a threshold set by population prevalence.

PCGC folds the correction directly into the HE cross-product regression, using a prevalence-based scaling factor, rather than a flawed post-hoc transformation.

Liability-scale , not observed-scale , is the number that’s comparable across studies with different ascertainment.

5 Part 5 — LD Score Regression: Heritability from Summary Statistics Alone

Everything so far — HE regression, GREML, PCGC — requires individual-level genotype data: actual genotypes for actual people. But most large GWAS consortia only release summary statistics: one effect size, standard error, and p-value per SNP, with no individual-level data at all (for privacy and logistical reasons). LD Score Regression (LDSC), introduced by Bulik-Sullivan, Finucane, and colleagues in 2015, estimates heritability from summary statistics alone — no individual genotypes required.

5.1 The Intuition: LD Amplifies Detectable Association

Recall that SNPs correlated with a causal variant through linkage disequilibrium pick up some of that variant’s association signal — this is the same LD logic that underlies GWAS itself and PRS construction. The LDSC insight extends this one step further:

A SNP that tags (is correlated with) more of the genome should show a larger marginal association signal on average — simply because it’s more likely to be picking up signal from a nearby causal variant.

5.2 From Fisher’s Polygenic Model to an Estimating Equation

Assume a polygenic model (Fisher, 1918): , with — effect sizes drawn with variance spread evenly across SNPs. From GWAS, we observe marginal effect estimates . Taking the expectation of and working through the algebra:

where is the LD score of SNP — the sum of its squared correlations with every other SNP in the reference panel. This is a simple linear regression: regress each SNP’s GWAS statistic against its LD score.

Slope → estimates SNP heritability .
Intercept → captures confounding, including population stratification. This is exactly what LD score regression was originally designed to detect: true polygenic signal should scale with LD, but confounding inflation (like population stratification) inflates test statistics roughly uniformly regardless of LD — so it shows up in the intercept, not the slope.

Worked example from the original literature: a schizophrenia GWAS (Psychiatric Genomics Consortium) showed (substantial genomic inflation) but an LDSC intercept of only 1.06 — meaning the overwhelming majority of that inflation was consistent with genuine polygenic architecture, not confounding. This kind of decomposition is one of LDSC’s most valuable practical contributions: separating “real polygenicity” from “something’s wrong with your GWAS.”

5.3 A Simulated Illustration

# Illustrative LDSC-style simulation
set.seed(42)
M <- 5000          # number of SNPs
N <- 50000         # GWAS sample size
h2_true <- 0.3     # true heritability
a_true  <- 0.01    # confounding/stratification inflation

# Simulate LD scores (in reality, computed from a reference panel like 1000 Genomes)
ld_score <- rgamma(M, shape = 2, scale = 20)

# Simulate chi-squared statistics under the LDSC expectation, plus noise
expected_chisq <- (N * h2_true / M) * ld_score + N * a_true + 1
chisq_obs <- rgamma(M, shape = 2, rate = 2 / expected_chisq)  # noisy draws around expectation

ldsc_fit <- lm(chisq_obs ~ ld_score)
summary(ldsc_fit)

# Recovering h2 and the intercept from the fitted slope/intercept
slope_hat     <- coef(ldsc_fit)[["ld_score"]]
intercept_hat <- coef(ldsc_fit)[["(Intercept)"]]
h2_hat <- slope_hat * M / N

cat("Estimated h2:", round(h2_hat, 3), "\n")
cat("Estimated intercept (stratification proxy):", round(intercept_hat, 3), "\n")


Call:
lm(formula = chisq_obs ~ ld_score)

Residuals:
   Min     1Q Median     3Q    Max 
-907.6 -318.0  -97.4  198.9 4064.4 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 486.5597    11.0465   44.05   <2e-16 ***
ld_score      3.4683     0.2255   15.38   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 452.4 on 4998 degrees of freedom
Multiple R-squared:  0.04519,   Adjusted R-squared:  0.045 
F-statistic: 236.6 on 1 and 4998 DF,  p-value: < 2.2e-16

Estimated h2: 0.347 
Estimated intercept (stratification proxy): 486.56

The point of this simulation isn’t the exact numbers — it’s the mechanic: regress statistics on LD scores, and the slope (rescaled by ) is your heritability estimate.

5.4 The Duality Between HE Regression and LDSC

These two methods are more closely related than they might look:

	HE regression (sample space)	LDSC (marker space)
Matrix	→ (GRM: relatedness between people)	→ (LD matrix: correlation between SNPs)
Regress	phenotype cross-products on the GRM	GWAS statistics on LD scores
Underlying question	Do genetically-similar people have similar phenotypes?	Do SNPs that tag more of the genome show bigger effects?

By the rules of linear algebra, these two spaces contain the same total information about genetic variance — they’re dual views of the same underlying question. But summarizing the full LD matrix down to one number per SNP () throws away the specific shape of how SNPs correlate with each other — information the full GRM in HE regression preserves. This is exactly why HE regression, using individual-level data, is statistically more powerful than LDSC using summary statistics alone. LDSC trades some statistical power for the enormous practical advantage of not needing individual-level genotypes.

Key takeaways

LDSC estimates heritability from GWAS summary statistics alone, by regressing each SNP’s statistic on its LD score.

The slope of this regression estimates heritability; the intercept estimates confounding/stratification inflation — letting you decompose genomic inflation into “real polygenicity” vs. “something’s wrong.”

LDSC and HE regression are dual views of the same underlying variance information — marker-space vs. sample-space — with HE regression retaining more statistical power at the cost of requiring individual-level data.

6 Part 6 — Relaxing LDSC’s Assumptions

Standard LDSC is a method-of-moments estimator — it needs only and , with no strict distributional (Gaussian) assumption on effect sizes. But it does assume something stronger than it first appears: a uniform genetic architecture, where every SNP is expected to explain exactly the same amount of phenotypic variance, regardless of its LD, minor allele frequency, or biological function.

6.1 Where the Uniformity Assumption Actually Hides

It’s not just the term — it’s baked into how the genotype matrix is defined. LDSC standardizes genotypes:

where is the raw allele count and is the minor allele frequency. Standardizing forces every SNP’s column to have variance exactly 1, so for every SNP regardless of its frequency. The consequence: dividing by inflates the allelic-scale effect-size variance of rare variants specifically.

This wasn’t an oversight — the LDSC authors were fully aware of it, and chose it deliberately for two reasons: (1) it makes a clean correlation matrix, simplifying the algebra considerably; (2) it’s empirically defensible, since effect-size variance genuinely does tend to increase as allele frequency decreases, consistent with negative selection acting more strongly against common large-effect variants.

6.2 Relaxing the Frequency Assumption: The α Parameter

Is the standard uniformity assumption the right fit for every trait? Not necessarily. We can generalize it with a parameter :

α	Interpretation
(LDSC default)	Variance independent of MAF — common and rare variants explain equal expected variance
	Effect size (not variance) independent of MAF — common variants explain more overall variance
	Strong negative selection — rare variants dominate the variance

Introducing changes the LD score itself into a weighted version, , which then plugs into the same regression machinery as before.

How is α estimated? Via profile likelihood: define a grid of candidate values, compute the expected GWAS statistics under each, compare to the actual observed statistics, and pick the that maximizes model likelihood. Empirically, across many complex traits the true tends to fall between and — suggesting the default may somewhat overestimate the contribution of rare variants for many traits.

6.3 Relaxing the LD Assumption: LDAK and SumHer

A different critique: if ten SNPs sit in near-perfect LD, they likely all tag the same underlying causal variant — but standard LDSC gives each of them full weight, effectively inflating that region’s apparent contribution. LDAK (Speed & Balding) introduces an LD-based weight per SNP — SNPs in dense LD regions get down-weighted, SNPs in low-LD regions (more independent signal) get up-weighted: .

6.4 Relaxing the Functional Assumption: Stratified LDSC (s-LDSC)

A third critique: a SNP sitting inside an active promoter or coding region is plausibly more likely to be causal than one in an intergenic “desert” — biology isn’t uniform across the genome. Stratified LD Score Regression (Finucane et al., 2015) partitions the variance across overlapping functional annotation categories instead of assuming one genome-wide :

where if SNP belongs to annotation category (0 otherwise), and is the per-SNP heritability contribution of category . The regression itself generalizes to a multiple regression: , where measures how much of category is tagged by SNP .

The widely-used “baseline model” is simply a specific choice of about 24 core functional annotations (plus flanking windows around them) used as the default category set.

What can you do with this? Two useful quantities fall out directly: the heritability of category itself, , and whether category is “punching above its weight” — comparing to (the category’s share of the genome). If a category explains disproportionately more heritability than its size would predict, that’s heritability enrichment — a signal that the category is biologically important, and a concept that carries forward directly into methods like SBayesRC (Part 8).

6.5 All Four Approaches, Side by Side

Every method below is really the same method-of-moments regression machinery, applied under a different assumption about :

Method	Variance assumption
Standard LDSC	Uniform:
SumHer / LDAK	Scaled by LD weights (and optionally MAF via α)
α-modeled LDSC	Scaled by MAF:
s-LDSC	Sum of functional-annotation contributions:

The statistical framework stays fixed; what changes is the biological model of genetic architecture baked into the variance assumption.

6.6 A Brief Aside: Non-Additive (Dominance) Heritability

Everything so far assumes purely additive genetic effects. Dominance variance () captures whether the heterozygote genotype deviates from the exact midpoint of the two homozygotes. Modeling this requires an orthogonal encoding of additive and dominance genotype codes (achievable under Hardy-Weinberg equilibrium), which then permits two independent LD score regressions — one additive, one dominance — since dominance LD decays at the square of additive LD (much faster with physical distance).

Applied across roughly 1,100 traits in UK Biobank, dominance heritability turned out to be small — typically under 5% of total genetic variance, and often statistically indistinguishable from zero. This doesn’t mean dominance never matters biologically at individual loci — but at the population-variance level, additive effects absorb most of what a two-degree-of-freedom model can explain. The standard additive-only GWAS assumption holds up well for most complex traits.

Key takeaways

Standard LDSC’s “uniform architecture” assumption is baked into genotype standardization, and can be relaxed in several directions.

The α parameter relaxes the MAF assumption; LDAK/SumHer relax the LD-density assumption; stratified LDSC (s-LDSC) relaxes the functional/annotation assumption.

All of these remain method-of-moments regressions — only the assumed variance structure of SNP effects changes.

Non-additive (dominance) heritability is generally small for complex traits, validating the standard additive GWAS model.

7 Part 7 — Genetic Correlation: Do Two Traits Share Genetic Architecture?

Everything so far has focused on a single trait. But one of the most common questions in statistical genetics involves two traits: do depression and obesity share genetic causes? Are psychiatric disorders genetically related to each other? Does elevated cholesterol share genetic architecture with heart disease? Answering these requires genetic correlation.

7.1 Motivation

The goal is to determine the shared genetics of two traits — schematically, whether the genetic effects underlying Trait A and Trait B overlap. This matters for two main reasons: uncovering causal pathways (e.g. cholesterol → heart disease) and informing categorization of related conditions (e.g. clustering psychiatric disorders by shared genetic architecture rather than by symptom overlap alone).

7.2 Four Reasons Two Traits Can Be Genetically Correlated

A single variant or gene influencing multiple traits is called pleiotropy, and it comes in several distinct flavors — each with a different biological and causal interpretation:

Vertical pleiotropy (): the variant affects trait , which in turn causally affects trait . The genetic correlation here reflects a genuine causal chain.

Horizontal pleiotropy ( and , independently): the same variant affects both traits through separate biological pathways, with no causal link between and themselves.

Pleiotropy via an intermediate phenotype ( and ): an “endophenotype” mediates the shared effect — for example, latent factors underlying clusters of psychiatric disorders (Grotzinger et al., 2025).

Spurious pleiotropy (via LD): two distinct causal variants, and , happen to sit in LD with each other (or with a shared marker). This looks like pleiotropy in the data but reflects two independent causal mechanisms that are simply co-inherited — an artifact of genome structure, not shared biology.

Mendelian Randomization can distinguish vertical from horizontal pleiotropy — this is exactly the tool genetic correlation itself cannot provide, since only tells you whether shared genetic architecture exists, not its direction or mechanism.

7.3 From Variance/Covariance/Correlation to Heritability/Genetic Covariance/Genetic Correlation

The mathematical structure of genetic correlation is a direct analogy to ordinary variance and covariance:

General statistical concept	Genetics equivalent
	— heritability
	— genetic covariance
	— genetic correlation

So the whole problem of estimating genetic correlation reduces to estimating heritability (variance) and genetic covariance (covariance) — as the saying goes, all you really need is covariance.

7.4 Cross-Trait LD Score Regression

Cross-trait LDSC extends the single-trait -on-LD-score regression from Part 5 to a product of Z-scores across two GWAS:

where are Z-scores for the two traits at SNP ; are the two GWAS sample sizes; is genetic covariance; is the number of reference SNPs; is the LD score; is the phenotypic correlation between the traits; and is the number of individuals overlapping between the two studies.

The slope of this regression estimates genetic covariance (which is then rescaled into the correlation ); the intercept absorbs any phenotypic correlation induced by sample overlap between the two GWAS.

Single-trait LDSC is a special case of this equation. Set trait 1 = trait 2 (so , , complete overlap), and the cross-trait equation reduces exactly to the standard single-trait form from Part 5, . Genetic correlation and heritability estimation are, mathematically, the same regression viewed from two different angles.

Robustness to sample overlap. A reassuring property: cross-trait LDSC remains fairly robust even when the two GWAS share participants (). Overlap tends to inflate the intercept roughly uniformly across variants, rather than biasing the slope — meaning the estimate itself stays comparatively reliable even in the presence of sample overlap, unlike many other cross-trait methods.

7.5 A Simulated Illustration

# Illustrative cross-trait LDSC simulation
set.seed(7)
M  <- 5000
N1 <- 40000; N2 <- 40000
rho_g_true <- 0.3   # true genetic covariance
rho_true   <- 0.0   # phenotypic correlation from any sample overlap
Ns_true    <- 0      # no overlapping samples in this illustration

ld_score <- rgamma(M, shape = 2, scale = 20)

expected_z1z2 <- (sqrt(N1 * N2) * rho_g_true / M) * ld_score +
                 (rho_true * Ns_true) / sqrt(N1 * N2)

z1z2_obs <- expected_z1z2 + rnorm(M, sd = 2)   # add regression noise

cross_ldsc_fit <- lm(z1z2_obs ~ ld_score)
summary(cross_ldsc_fit)

rho_g_hat <- coef(cross_ldsc_fit)[["ld_score"]] * M / sqrt(N1 * N2)
cat("Estimated genetic covariance:", round(rho_g_hat, 3), "\n")


Call:
lm(formula = z1z2_obs ~ ld_score)

Residuals:
    Min      1Q  Median      3Q     Max 
-6.4236 -1.3460 -0.0022  1.3030  6.7866 

Coefficients:
            Estimate Std. Error  t value Pr(>|t|)    
(Intercept) 0.052421   0.049026    1.069    0.285    
ld_score    2.398684   0.001003 2391.175   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.984 on 4998 degrees of freedom
Multiple R-squared:  0.9991,    Adjusted R-squared:  0.9991 
F-statistic: 5.718e+06 on 1 and 4998 DF,  p-value: < 2.2e-16

Estimated genetic covariance: 0.3

Exactly as with single-trait LDSC, the mechanic is what matters: regress the product of Z-scores across two traits on LD score, and the slope gives genetic covariance.

Key takeaways

Genetic correlation asks whether two traits share genetic architecture — quantified as , the correlation of their genetic components.

Pleiotropy underlying genetic correlation comes in several distinct flavors: vertical, horizontal, mediated by an intermediate phenotype, or spurious (via LD) — each implies a different biological story.

Cross-trait LDSC extends single-trait LDSC to a product-of-Z-scores regression; single-trait LDSC is the special case where a trait is compared against itself.

Genetic correlation, unlike Mendelian Randomization, cannot on its own distinguish why two traits are correlated — only whether they are.

8 Part 8 — Interpreting Genetic Correlation, and Its Limitations

Having a formula for is only half the job — interpreting it correctly, and knowing when to distrust it, matters just as much.

8.1 Reading the Sign and Magnitude

ranges conceptually from to :

Positive : genetic effects on the two traits point in the same direction.
Negative : genetic effects point in opposite directions.
Zero : either the genetic effects are essentially uncorrelated genome-wide, or positive and negative local genetic correlations are canceling out across the genome. This second possibility is worth remembering — a genome-wide near zero doesn’t rule out strong correlation concentrated in specific regions (this is what dedicated local genetic correlation methods, like LAVA or SUPERGNOVA, are built to detect).

Real examples from the literature (Bulik-Sullivan, Finucane et al., 2015): positive genetic correlations cluster among metabolic disorders, lipid/heart-disease traits, growth-related traits, and psychiatric disorders; negative correlations appear between HDL cholesterol and other metabolic/cardiovascular traits, and between years of education and risk factors like smoking, BMI, and LDL cholesterol.

One more practical note: cross-trait LDSC generally has wider standard errors than either individual-level methods or single-trait LDSC’s own heritability estimates — a noisy-looking estimate isn’t necessarily a red flag on its own, it may just reflect the inherently lower precision of the cross-trait regression.

8.2 Four Limitations to Keep in Mind

1. Traits must be sufficiently heritable. If a trait’s own heritability is too low, the genetic correlation regression can mechanically produce — a nonsensical value that nonetheless falls directly out of the math when the underlying signal-to-noise ratio is too poor. As a practical rule of thumb, treat estimates involving any trait whose heritability Z-score is below 4 as too noisy to trust.

2. The method assumes polygenicity. LDSC-based genetic correlation works best for traits with many small-effect causal variants spread across the genome. It’s less reliable for traits dominated by a handful of large-effect loci, since the underlying regression logic depends on polygenic averaging across many SNPs.

3. Reverse causation can masquerade as genetic correlation. A positive genetic correlation between obesity and depression, for instance, is equally consistent with obesity causally influencing depression risk, or depression causally influencing obesity risk (Speed et al., 2019). Genetic correlation is symmetric and directionless by construction — it cannot tell you which way, if either, causality runs. That question requires Mendelian Randomization, not .

4. Assortative mating can inflate estimates. If people non-randomly choose partners who resemble them on a given trait (assortative mating, with correlation between mates), the equilibrium genetic correlation exceeds the “true” underlying genetic correlation: whenever (and only when there’s no assortative mating, ). This is a subtle confound worth remembering specifically for traits known to be assortatively mated on — height and educational attainment are classic examples.

8.3 Extension: Genomic SEM

Once you can estimate genetic correlations pairwise between many traits, a natural next step is to model the structure of that correlation across all of them simultaneously. Genomic SEM (Grotzinger et al., 2019) is exactly this — a two-step framework:

Estimate the full genetic covariance matrix across many traits, using pairwise cross-trait LDSC.
Decompose that covariance matrix into latent genetic factors — structural equation modeling (see our companion SEM tutorial) applied to a genetic covariance matrix, rather than a raw phenotypic one.

This is precisely how researchers have identified shared latent genetic factors underlying clusters of psychiatric disorders (Grotzinger et al., 2025) — the same factor =~ indicator measurement-model logic from classical SEM, just with the genetic covariance matrix standing in for the raw phenotypic covariance matrix.

Key takeaways

Interpret ’s sign as the direction of shared genetic effects, but remember a near-zero genome-wide estimate can still mask strong local correlation.

Genetic correlation requires reasonably heritable, polygenic traits to be reliable, and is symmetric — it cannot establish which trait causes the other (that’s Mendelian Randomization’s job).

Assortative mating is a distinct, easy-to-overlook confound that can inflate estimates above their “true” value.

Genomic SEM extends pairwise genetic correlation into a full structural model across many traits at once, using the same measurement-model logic as ordinary SEM.

9 Part 9 — Practical Checklist, and Where This Leads Next

9.1 Before You Trust an LDSC Heritability or Genetic Correlation Estimate

A short, practical checklist worth running through before reporting or relying on any LDSC-based result:

Ancestry-matched LD reference. LD scores must come from a reference panel matching the GWAS population’s genetic ancestry — the same ancestry-matching concern that recurs throughout fine-mapping and PRS.
Sufficient sample size. LDSC gets noisy at low — a common rule of thumb is for standard LDSC and for stratified LDSC.
Sufficient polygenicity. The method works poorly if very few SNPs actually affect the trait; watch out for a small number of unusually large-effect variants distorting the regression.
Liability-scale conversion for binary traits. Never compare raw observed-scale estimates across studies with different case-control ascertainment — convert to the liability scale first (Part 4).
Custom annotations in s-LDSC need to actually work correctly under block-jackknife resampling if you’re building your own annotation categories.
Genomic control caution. If genomic control has already been applied to input summary statistics, estimates can be distorted — but estimates are comparatively robust to this.
Low-heritability trait pairs. Treat / estimates involving any trait with an Z-score below 4 as too noisy to trust, regardless of how the point estimate looks.

9.2 The Common Thread

Looking back across this whole tutorial, there’s really one continuous story: relaxing the assumption that every SNP matters equally.

Each step in this progression adds a more realistic piece of biology — first genome-wide relatedness instead of pedigree-based relatedness, then allowing MAF-dependence, then LD-density weighting, then functional-annotation-dependence. But every one of these methods shares a structural limitation: they’re all method-of-moments estimators, working “in expectation” over aggregate variance components. They can tell you the total (or category-partitioned) heritability very well — but they cannot tell you the posterior probability that any individual SNP is causal. If the true underlying architecture is spike-and-slab, LDSC-family methods simply can’t distinguish the spikes (null SNPs) from the slabs (causal SNPs).

9.3 The Bridge to Bayesian Methods

This is exactly the motivation for moving to a fully Bayesian framework — methods like SBayesRC (covered in depth in our companion PRS tutorial), which estimate a posterior effect size for every individual SNP, using:

Eigendecomposition of the LD matrix within pseudo-independent LD blocks, to make the computation tractable at genome scale — a direct computational descendant of the LD-score-based tricks used throughout this tutorial.
A four-component mixture prior (one zero-effect “spike,” three non-zero-variance “slabs”), extending the uniform-variance assumption of standard LDSC into an explicit, estimable distribution of effect sizes.
Functional annotations modifying each SNP’s prior probability of a non-zero effect — the same heritability-enrichment logic from stratified LDSC (Part 6), now feeding directly into a Bayesian prior rather than just partitioning aggregate variance after the fact.

In other words: everything in this tutorial builds toward being able to ask, and eventually answer, one final question — not just “how much of this trait is genetic?”, but “which specific variants are responsible?”

Final key takeaways

Heritability estimation methods form a continuous progression: individual-level (HE regression, GREML) → summary-statistics-based (LDSC and its many extensions) → fully Bayesian, variant-level (SBayesRC).

Genetic correlation is the natural two-trait extension of the same LD Score Regression machinery used for single-trait heritability.

Every method here — including cross-trait LDSC — comes with real, well-documented failure modes; checking sample size, ancestry matching, polygenicity, and liability-scale conversion before trusting a result is not optional.

LDSC-family methods excel at aggregate (genome-wide or per-category) variance estimation, but structurally cannot resolve individual-variant causality — which is precisely the gap that Bayesian polygenic methods were built to fill.

Survival Analysis: Deriving the Kaplan-Meier Estimator and the Cox Model From Scratch

Nivedita Bhadra — Sat, 25 Jul 2026 21:00:00 GMT

The shape every method in this tutorial is built to explain: a survival curve that steps down at each event, with uncertainty that widens as fewer people remain at risk.

Kaplan-Meier Cox Model Survival Analysis R

1 Introduction

Suppose a new cancer treatment is introduced, and researchers want to know whether it helps patients live longer. They recruit volunteers, follow them for several years, and record when each patient experiences an event such as disease recurrence or death.

At first glance, this might seem like a straightforward prediction problem. Why not simply calculate the average survival time or fit a standard regression model?

The challenge is that real-world studies are not so straightforward. Some patients leave the study early, some are still alive when the study ends, and others are lost to follow-up. For these individuals, we know that the event has not yet occurred, but we do not know exactly when it eventually will. Ignoring these partially observed cases discards valuable information, while treating them as complete observations leads to biased conclusions.

Survival analysis helps to solve this problem.

Survival analysis is designed to analyze time-to-event data—that is, data in which the outcome of interest is not simply whether an event occurs, but when it occurs. Although the name originated in medical research, where the event was often death or disease recurrence, the methodology applies to various situations involving the timing of an event.

Example cases:

How long do cancer patients survive after treatment?
When does a patient experience disease relapse?
How long until a machine fails in a manufacturing plant?
When does a customer cancel a subscription service?
How long before a software system crashes?
When does a genetic mutation become fixed in a population?

What makes these problems unique is that the timing of the event carries just as much information as the event itself.

A defining feature of survival data is censoring. Not every participant experiences the event during the observation period. For example, if a clinical trial ends after five years and a patient is still alive, we only know that their survival time is at least five years—their true survival time remains unknown. Classical statistical methods such as linear regression are not designed to handle these incomplete observations appropriately, whereas survival analysis incorporates them naturally, allowing researchers to use all available information without introducing systematic bias.

Over the past several decades, survival analysis has become one of the most important statistical tools in medicine, epidemiology, public health, engineering, economics, reliability analysis, and increasingly in data science and machine learning. Whether evaluating new therapies, estimating equipment reliability, modeling customer retention, or studying disease progression, the central question remains the same:

Given what we know today, how does the probability of experiencing an event change over time?

In this tutorial, we will build an intuitive understanding of the two foundational methods in survival analysis:

Kaplan–Meier estimation, which estimates the probability of surviving over time without assuming a specific statistical model.
The Cox proportional hazards model, which quantifies how predictors such as age, treatment, or genetic factors influence the instantaneous risk of experiencing the event.

We will try understand the statistical ideas and the underlying mathematics behind them , and implement each technique in R using some simple examples.

2 Basic Survival Quantities

Before learning the Kaplan–Meier estimator or the Cox proportional hazards model, it is important to understand the three mathematical quantities that form the foundation of survival analysis.

Suppose we follow an individual from the start of a study until an event occurs—for example, death, disease relapse, machine failure, or customer churn. The time until this event is called the survival time or event time.

We denote this event time by the random variable .

Because different individuals experience the event at different times, we treat as a random variable, meaning that its exact value is uncertain before we observe it.

Survival analysis is built around three closely related functions:

The survival function, which tells us the probability that an individual remains event-free beyond a given time.
The hazard function, which measures the instantaneous risk of experiencing the event at a particular moment, assuming the individual has survived up to that time.
The cumulative hazard function, which accumulates this risk over time.

Together, these three quantities describe the same underlying process from different perspectives.

2.1 1. Survival Function

The most intuitive quantity is the survival function.

It answers the question:

What is the probability that an individual survives beyond time ?

Mathematically,

where

is the event time,
is the cumulative distribution function (CDF), representing the probability that the event has already occurred by time .

As time increases, the survival probability can only stay the same or decrease.

2.2 2. Hazard Function

While the survival function describes the probability of remaining event-free, researchers are often interested in something slightly different:

Among individuals who have survived until time , how likely is the event to occur immediately afterward?

This idea is captured by the hazard function, sometimes called the instantaneous event rate or instantaneous risk.

It is defined as

Notice the conditioning.

The hazard is not the probability of dying at time . Instead, it measures the instantaneous rate at which events occur among those who are still at risk just before time .

This distinction is one of the most important concepts in survival analysis.

2.3 3. Relationship Between Hazard and Survival

Although the survival and hazard functions describe different ideas, they contain exactly the same information.

Starting from the definition of conditional probability,

where

is the probability density function (PDF),
is the survival function.

Since

we obtain

This equation tells us that the hazard is simply the rate at which the logarithm of the survival probability decreases over time.

2.4 4. Cumulative Hazard Function

Rather than looking only at the instantaneous hazard, it is often useful to consider the total accumulated hazard up to time .

This is called the cumulative hazard function:

Integrating the previous relationship gives

or equivalently,

This elegant identity shows that the survival function and cumulative hazard function are simply two different mathematical representations of the same underlying phenomenon.

2.5 Why These Quantities Matter

These definitions are more than mathematical formalities—they motivate the two most widely used methods in survival analysis.

Kaplan–Meier estimation focuses on estimating the survival function directly from observed data.
The Cox proportional hazards model focuses on modeling the hazard function and how it changes with patient characteristics such as age, treatment, or genetic factors.

Although the two methods approach the problem from different directions, they are linked through the relationship

Understanding this connection makes it much easier to see how the Kaplan–Meier estimator and the Cox model fit together within a single statistical framework.

3 The Kaplan–Meier Estimator

The Kaplan–Meier estimator is the most widely used non-parametric method for estimating the survival function from observed data.

Its goal is straightforward:

Given a group of individuals, what is the probability of remaining event-free over time?

Unlike parametric survival models, the Kaplan–Meier estimator makes no assumptions about the underlying distribution of survival times. Instead, it estimates the survival curve directly from the observed data while naturally accounting for censored observations.

3.1 The Data We Observe

In an ideal study, we would know the exact event time for every individual. In practice, however, this is rarely possible. Some participants experience the event during follow-up, while others are still event-free when the study ends or are lost to follow-up. For these individuals, we only know that the event occurs after their last observed time.

To accommodate this, survival analysis records two pieces of information for each individual:

The observed follow-up time
Whether the event occurred during follow-up

Mathematically, for the -th individual, we observe

where

is the true event time,
is the censoring time,
is the observed follow-up time, equal to whichever occurs first.

We also define an event indicator

Thus, each observation consists of the pair .

3.2 Constructing the Risk Set

The Kaplan–Meier estimator builds the survival curve one event time at a time.

Suppose the distinct observed event times are

Immediately before each event time , we define two important quantities:

: the number of individuals at risk just before .
: the number of events occurring exactly at .

An individual is considered at risk if they have not experienced the event or been censored before time . Equivalently,

Notice that individuals who are censored after still belong to the risk set at time , because they were under observation until that point.

3.3 Deriving the Kaplan–Meier Estimator

The key idea is simple.

To survive beyond time , an individual must survive every earlier event time. Therefore, the overall survival probability can be written as the product of conditional survival probabilities:

At each event time ,

individuals are at risk,
experience the event.

The empirical probability of experiencing the event at that instant is therefore

so the empirical probability of surviving beyond that event time is

Multiplying these conditional survival probabilities across all event times gives the Kaplan–Meier estimator:

This estimator is called the product-limit estimator because it is obtained by multiplying together successive conditional survival probabilities.

3.4 Why Does the Survival Curve Look Like a Staircase?

The Kaplan–Meier estimator is a step function.

The survival probability changes only when an event occurs.

Censored observations do not produce downward steps because no event has occurred. Instead, censored individuals are simply removed from the risk set for subsequent event times, reducing the denominator in later calculations.

Consequently,

observed events produce downward steps,
censoring affects future risk sets but does not immediately change the estimated survival probability.

3.5 Implementing the Kaplan–Meier Estimator in R

The following example implements the Kaplan–Meier estimator directly from its mathematical definition.

We begin with a small toy dataset containing both observed events (status = 1) and censored observations (status = 0). For each distinct event time, we calculate

the number of individuals at risk (),
the number of observed events (),
the updated survival probability using the product-limit formula.

Finally, we compare our manual calculation with the result produced by the survfit() function from R’s survival package. Both methods should produce identical survival estimates, demonstrating that the Kaplan–Meier estimator implemented by survfit() is simply an efficient implementation of the mathematical definition derived above.


library(survival)
toy <- data.frame(
  time   = c(4, 6, 6, 8, 9, 10, 11, 14, 14, 18),
  status = c(1, 1, 0, 1, 0, 1, 0, 1, 1, 0)   # 1 = event, 0 = censored
)

toy

# Distinct event times
event_times <- sort(unique(toy$time[toy$status == 1]))

# Manual Kaplan–Meier calculation
km_manual <- data.frame(
  time = event_times,
  n = NA,
  d = NA,
  S = NA
)

S <- 1

for (i in seq_along(event_times)) {

  tj  <- event_times[i]

  # Number at risk immediately before tj
  n_j <- sum(toy$time >= tj)

  # Number of events at tj
  d_j <- sum(toy$time == tj & toy$status == 1)

  # Product-limit update
  S <- S * (1 - d_j / n_j)

  km_manual$n[i] <- n_j
  km_manual$d[i] <- d_j
  km_manual$S[i] <- S
}

km_manual

# Compare with survfit()
fit <- survfit(Surv(time, status) ~ 1, data = toy)

comparison <- data.frame(
  time = event_times,
  manual = km_manual$S,
  survfit = summary(fit)$surv
)

comparison

A data.frame: 10 × 2
time	status

4	1
6	1
6	0
8	1
9	0
10	1
11	0
14	1
14	1
18	0

A data.frame: 5 × 4
time	n	d	S

4	10	1	0.9000000
6	9	1	0.8000000
8	7	1	0.6857143
10	5	1	0.5485714
14	3	2	0.1828571

A data.frame: 5 × 3
time	manual	survfit

4	0.9000000	0.9000000
6	0.8000000	0.8000000
8	0.6857143	0.6857143
10	0.5485714	0.5485714
14	0.1828571	0.1828571

The comparison table shows that the manually computed survival probabilities exactly match those returned by survfit(). This confirms that the Kaplan–Meier estimator is simply the cumulative product of conditional survival probabilities evaluated at each observed event time.

The manually computed survival probabilities match those returned by survfit() exactly at every event time, confirming that the Kaplan–Meier estimator implemented in R is simply an efficient implementation of the product-limit formula derived above.

Why does the survival curve drop by different amounts at each event time?

The size of each downward step depends on the proportion of individuals who experience the event, not simply on the number of events. At event time , the drop is determined by

where is the number of events and is the number of individuals still at risk. Consequently, two events among only three individuals at risk produce a much larger decrease in the survival probability than one event among nine individuals at risk.

3.6 Visualizing the Kaplan–Meier Survival Curve

The Kaplan–Meier estimate is conventionally displayed as a step function. The curve drops at observed event times and remains unchanged between events. Vertical marks indicate censored observations: these individuals were still event-free at their last recorded follow-up time, but their subsequent outcomes are unknown.



km_fit <- survfit(Surv(time, status) ~ 1, data = lung)

plot(
  km_fit,
  xlab = "Time",
  ylab = "Estimated survival probability",
  main = "Kaplan–Meier Survival Curve",
  mark.time = TRUE,
  conf.int = TRUE,
  lwd = 2
)

The estimated survival probability declines over follow-up as deaths occur. The widening confidence interval toward the right-hand tail reflects the decreasing number of individuals remaining under observation. ### Quantifying Uncertainty: Greenwood’s Formula

The Kaplan–Meier estimator provides a point estimate of the survival probability. As with any statistical estimate, however, we also want to know how precise that estimate is.

The uncertainty of the Kaplan–Meier estimator is estimated using Greenwood’s formula, which provides an estimate of its variance:

Taking the square root gives the estimated standard error:

Each observed event contributes one term to the summation. As follow-up continues and fewer individuals remain in the risk set, the uncertainty of the survival estimate generally increases. This is why confidence intervals around Kaplan–Meier curves typically become wider toward the end of a study.

3.7 Implementing Greenwood’s Formula

Just as we manually computed the Kaplan–Meier estimator, we can also compute its standard error directly from Greenwood’s formula.

At each event time we:

Update the Kaplan–Meier survival estimate.
Add the Greenwood variance contribution to a running cumulative sum.
Multiply the accumulated variance term by the current survival estimate to obtain the standard error.

The implementation below follows these three steps directly.

# Manual Greenwood standard error
S <- 1
cum_var_term <- 0

km_manual$se <- NA

for (i in seq_along(event_times)) {

  n_j <- km_manual$n[i]
  d_j <- km_manual$d[i]

  # Update Kaplan–Meier estimate
  S <- S * (1 - d_j / n_j)

  # Greenwood variance contribution
  cum_var_term <- cum_var_term +
    d_j / (n_j * (n_j - d_j))

  # Standard error
  km_manual$se[i] <- S * sqrt(cum_var_term)

}

km_manual[, c("time", "S", "se")]

A data.frame: 5 × 3
time	S	se

4	0.9000000	0.09486833
6	0.8000000	0.12649111
8	0.6857143	0.15149402
10	0.5485714	0.17243785
14	0.1828571	0.15998445

We can now compare both the manually calculated survival probabilities and their standard errors with the values returned by survfit().

comparison <- data.frame(
  time = summary(fit)$time,
  manual_surv = km_manual$S,
  survfit_surv = summary(fit)$surv,
  manual_se = km_manual$se,
  survfit_se = summary(fit)$std.err
)
comparison

A data.frame: 5 × 5
time	manual_surv	survfit_surv	manual_se	survfit_se

4	0.9000000	0.9000000	0.09486833	0.09486833
6	0.8000000	0.8000000	0.12649111	0.12649111
8	0.6857143	0.6857143	0.15149402	0.15149402
10	0.5485714	0.5485714	0.17243785	0.17243785
14	0.1828571	0.1828571	0.15998445	0.15998445

The manually calculated survival probabilities and standard errors should match those returned by survfit() to numerical precision. This demonstrates that the Kaplan–Meier estimator and Greenwood’s standard error are not separate algorithms, but rather direct implementations of the mathematical formulas derived above.

4 The Cox Proportional Hazards Model

The Kaplan–Meier estimator is an excellent tool for estimating the survival probability of a group over time. However, it has one important limitation: it cannot simultaneously evaluate the effect of multiple variables on survival.

Suppose we want to answer questions such as:

Does age increase the risk of death?
Is a new treatment more effective than the standard treatment?
Does smoking shorten survival after accounting for age?
Do specific genetic variants influence disease progression?

These questions involve covariates—variables that may influence the time until an event occurs. While Kaplan–Meier curves can compare a few predefined groups (for example, treated versus untreated patients), they cannot quantify the effect of several predictors simultaneously.

To address this problem, Sir David Cox introduced the Cox proportional hazards model in 1972. Rather than modeling the survival probability directly, the Cox model describes how covariates influence the hazard, or instantaneous risk, of experiencing the event.

4.1 The Cox Model

For an individual with covariate vector , the hazard function is written as

where

is the hazard for individual ,
is the baseline hazard, representing the hazard when all covariates are at their reference values,
is the vector of observed covariates (such as age, sex, or treatment),
is the vector of regression coefficients describing how each covariate affects the hazard.

The exponential function ensures that the hazard remains positive while allowing covariates to increase or decrease the risk multiplicatively.

4.2 Hazard Ratios

One of the most attractive features of the Cox model is that we can compare two individuals without knowing the baseline hazard.

Suppose individuals A and B have covariate vectors and . Their hazard ratio is

Notice that the baseline hazard cancels completely.

This means that the Cox model estimates relative risk, rather than absolute risk.

For example, if

then one individual has twice the instantaneous risk of experiencing the event compared with another individual who differs by one unit in that covariate.

4.3 Why Is It Called the Proportional Hazards Model?

The hazard ratio above does not depend on time.

Regardless of whether the baseline hazard increases, decreases, or fluctuates during follow-up, the ratio between two individuals remains constant:

This assumption is known as the proportional hazards assumption, and it gives the model its name.

Importantly, this is an assumption, not a mathematical guarantee. In practice, it should always be assessed after fitting the model, most commonly using Schoenfeld residuals, which we will discuss later.

4.4 Cox’s Key Insight: The Partial Likelihood

At first glance, estimating the regression coefficients appears difficult because the baseline hazard is completely unknown.

Cox’s key insight was that we do not actually need to estimate the baseline hazard in order to estimate the regression coefficients.

Instead of modeling the full likelihood, he considered the probability that, among all individuals still at risk at an observed event time, the individual who actually experienced the event was the one who failed.

Suppose individual experiences the event at time , and let denote the risk set—the individuals still under observation immediately before that event time.

The conditional probability that individual experiences the event is

Remarkably, the unknown baseline hazard cancels from this expression, just as it did in the hazard ratio.

Multiplying these conditional probabilities across all observed event times produces the partial likelihood:

Unlike an ordinary likelihood, the partial likelihood contains only the regression coefficients and no baseline hazard, making estimation much simpler.

4.5 Estimating the Regression Coefficients

To estimate the regression coefficients, we maximize the log partial likelihood numerically.

This produces the estimate

which quantifies the effect of each covariate on the hazard.

Modern statistical software performs this optimization automatically. In R, the coxph() function from the survival package fits the Cox proportional hazards model using this partial likelihood approach.

To better understand how the method works, the following example first implements the log partial likelihood directly from its mathematical definition and optimizes it using optim(). We then compare the manually estimated regression coefficients with those returned by coxph() using the lung dataset and two covariates: age and sex.

4.6 Implementing the Cox Partial Likelihood by Hand

We now implement the Cox partial likelihood directly from its mathematical definition. We use the lung dataset from the survival package and model survival using age and sex.

In the original dataset:

status = 1 indicates censoring and status = 2 indicates death.
sex = 1 indicates male and sex = 2 indicates female.

We recode both variables into a more conventional binary form:

status = 0 for censored and status = 1 for an observed event.
sex = 0 for male and sex = 1 for female.


lung2 <- na.omit(lung[, c("time", "status", "age", "sex")])

lung2$status <- lung2$status - 1   # 0 = censored, 1 = event
lung2$sex    <- lung2$sex - 1      # 0 = male, 1 = female

The function below calculates the negative log partial likelihood. For each distinct event time, it identifies:

the individuals still in the risk set,
the individuals experiencing the event,
the linear predictor for everyone in the risk set.

It then adds each event’s contribution to the log partial likelihood:


# Negative log partial likelihood
neg_log_pl <- function(beta, data) {

  X <- as.matrix(data[, c("age", "sex")])
  time <- data$time
  status <- data$status

  event_times <- sort(unique(time[status == 1]))

  log_partial_likelihood <- 0

  for (tj in event_times) {

    # Individuals still at risk immediately before tj
    risk_idx <- which(time >= tj)

    # Individuals experiencing the event at tj
    event_idx <- which(time == tj & status == 1)

    # Linear predictors for the risk set
    linear_predictor <- as.vector(
      X[risk_idx, , drop = FALSE] %*% beta
    )

    # Add one contribution for each event at tj
    for (i in event_idx) {

      log_partial_likelihood <-
        log_partial_likelihood +
        sum(X[i, ] * beta) -
        log(sum(exp(linear_predictor)))
    }
  }

  # optim() minimizes, so return the negative value
  -log_partial_likelihood
}

We use optim() to minimize the negative log partial likelihood. The Hessian matrix returned by the optimizer provides an estimate of the covariance matrix of the regression coefficients.


fit_manual <- optim(
  par = c(0, 0),
  fn = neg_log_pl,
  data = lung2,
  method = "BFGS",
  hessian = TRUE
)

beta_manual <- setNames(
  fit_manual$par,
  c("age", "sex")
)

se_manual <- setNames(
  sqrt(diag(solve(fit_manual$hessian))),
  c("age", "sex")
)

round(beta_manual, 4)
round(se_manual, 4)

age: 0.017
sex: -0.5126

age: 0.0092
sex: 0.1675

4.7 Handling Tied Event Times

In real survival datasets, multiple individuals may experience the event at exactly the same recorded time. These are called tied events.

The manual implementation above uses the Breslow approximation: all events occurring at the same time share the same risk-set denominator.

To make the comparison technically consistent, we must therefore instruct coxph() to use the same method:

fit_cox <- coxph(
  Surv(time, status) ~ age + sex,
  data = lung2,
  ties = "breslow"
)

summary(fit_cox)

Call:
coxph(formula = Surv(time, status) ~ age + sex, data = lung2, 
    ties = "breslow")

  n= 228, number of events= 165 

         coef exp(coef)  se(coef)      z Pr(>|z|)   
age  0.017013  1.017158  0.009222  1.845  0.06506 . 
sex -0.512565  0.598957  0.167462 -3.061  0.00221 **
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

    exp(coef) exp(-coef) lower .95 upper .95
age     1.017     0.9831    0.9989    1.0357
sex     0.599     1.6696    0.4314    0.8316

Concordance= 0.603  (se = 0.025 )
Likelihood ratio test= 14.08  on 2 df,   p=9e-04
Wald test            = 13.44  on 2 df,   p=0.001
Score (logrank) test = 13.69  on 2 df,   p=0.001

We can compare the manually estimated coefficients and standard errors with those returned by coxph():

comparison <- data.frame(
  term = c("age", "sex"),
  manual_beta = as.numeric(beta_manual),
  coxph_beta = as.numeric(coef(fit_cox)),
  manual_se = as.numeric(se_manual),
  coxph_se = as.numeric(sqrt(diag(vcov(fit_cox))))
)

comparison[, -1] <- round(comparison[, -1], 4)

comparison

A data.frame: 2 × 5
term	manual_beta	coxph_beta	manual_se	coxph_se

age	0.0170	0.0170	0.0092	0.0092
sex	-0.5126	-0.5126	0.1675	0.1675

The manual and coxph() estimates should agree closely because both calculations now use the same partial likelihood and the same method for handling tied event times.

The fitted coefficients are expressed on the log-hazard scale. Exponentiating them gives the corresponding hazard ratios:

exp(beta_manual)
exp(coef(fit_cox))

age: 1.01715863844201
sex: 0.598955202302099

age: 1.01715843259764
sex: 0.598957406233886

For age, the hazard ratio represents the multiplicative change in the hazard associated with a one-year increase in age, holding sex constant.

For sex, the hazard ratio compares females (sex = 1) with males (sex = 0), holding age constant. A hazard ratio below 1 indicates a lower estimated hazard for females than for males.

The manually estimated coefficients and standard errors agree with those returned by coxph() to several decimal places. Because both implementations use the Breslow method for tied event times, the very small remaining differences are due only to numerical optimization and floating-point precision.

4.8 Interpreting the Cox Coefficients

The coefficients produced by a Cox model are expressed on the log-hazard scale. Exponentiating a coefficient converts it into a hazard ratio:

A hazard ratio greater than 1 indicates a higher estimated hazard, while a hazard ratio below 1 indicates a lower estimated hazard, holding the other covariates fixed.

For age, the estimated coefficient is approximately

Exponentiating gives

This means that each additional year of age is associated with an estimated 1.7% increase in the instantaneous hazard of death, holding sex constant.

The effect appears small because the coefficient describes a one-year difference. Over a ten-year age difference, the corresponding hazard ratio is

Thus, a patient who is ten years older has an estimated hazard approximately 19% higher, assuming the proportional hazards model is appropriate and the other covariates remain fixed.

For sex, coded as

0 = male,
1 = female,

the estimated coefficient is approximately

The corresponding hazard ratio is

Therefore, female patients in this cohort have an estimated hazard of death approximately 0.60 times that of male patients, after adjusting for age. Equivalently, this corresponds to an estimated 40% lower hazard:

These interpretations describe associations within this dataset. They should not automatically be interpreted as causal effects.

4.9 Standard Errors from the Information Matrix

Estimating the coefficients is only part of the analysis. We also need to quantify their uncertainty.

Let

denote the Cox log partial likelihood. The observed information matrix is

The estimated covariance matrix of the fitted coefficients is obtained by inverting the observed information matrix at the estimated parameter values:

The standard error of each coefficient is then the square root of the corresponding diagonal element:

In our implementation, optim() minimizes the negative log partial likelihood. Therefore, when hessian = TRUE, the returned Hessian is already the observed information matrix:

information_matrix <- fit_manual$hessian

covariance_matrix <- solve(information_matrix)

se_manual <- sqrt(diag(covariance_matrix))

This means the same optimization procedure provides both

the coefficient estimates, through the location of the minimum, and
their estimated uncertainty, through the curvature of the objective function around that minimum.

A sharply curved likelihood produces smaller standard errors and more precise coefficient estimates. A flatter likelihood produces larger standard errors, indicating greater uncertainty about the fitted effects.

5 Residual Diagnostics in the Cox Model

In ordinary linear regression, residuals are easy to define—they are simply the difference between the observed and predicted values of the response variable.

Survival analysis is different. Because many observations are censored, we often do not know the true event time for every individual. Consequently, there is no single “raw residual” analogous to that used in linear regression.

Instead, the Cox proportional hazards model uses several specialized residuals, each designed to answer a different diagnostic question. Among the most commonly used are:

Martingale residuals, used to assess model fit and detect nonlinearity in covariate effects.
Deviance residuals, a transformation of martingale residuals that is more symmetric and useful for identifying outliers.
Schoenfeld residuals, used to assess the proportional hazards assumption.
Scaled Schoenfeld residuals, commonly used in formal tests of proportional hazards, such as the cox.zph() test in R.

5.1 Martingale Residuals

The martingale residual compares what was actually observed with what the fitted Cox model expected for each individual.

It is defined as

where

is the event indicator (1 = event, 0 = censored),
is the estimated baseline cumulative hazard evaluated at the observed follow-up time,
is the individual’s covariate vector,
is the vector of estimated regression coefficients.

The estimated baseline cumulative hazard is usually obtained using the Breslow estimator,

where

is the number of events at time , and
is the risk set immediately before time .

Notice that the martingale residual is simply

Observed events − Expected cumulative hazard

making it conceptually similar to an observed-minus-expected residual used in many statistical models.

5.2 Interpreting Martingale Residuals

Martingale residuals have some unusual mathematical properties.

They are bounded above by 1.
They are unbounded below.
Their distribution is often highly skewed.

A positive residual indicates that fewer events were expected than observed for that individual.

A large negative residual indicates that the individual survived substantially longer than the model predicted based on their covariates.

Because of their asymmetry, martingale residuals are not ideal for identifying outliers directly. Instead, they are primarily used to assess whether the functional form of a continuous covariate is appropriate.

For example, after fitting a Cox model with age as a linear predictor, one can plot the martingale residuals against age. If the residuals exhibit a systematic curved pattern rather than random scatter, this suggests that the effect of age may not be linear and that a transformation or spline term could provide a better fit.

In R, martingale residuals can be obtained directly from a fitted Cox model using

martingale_residuals <- residuals(fit_cox, type = "martingale")

In the next section, we will examine Schoenfeld residuals, which play a different role: testing whether the proportional hazards assumption underlying the Cox model is satisfied.

5.3 Computing Martingale Residuals by Hand

We can reproduce the martingale residuals returned by coxph() by calculating the Breslow baseline cumulative hazard directly.

First, we extract the fitted coefficients and calculate each individual’s relative hazard:

beta_hat <- coef(fit_cox)

X <- as.matrix(
  lung2[, c("age", "sex")]
)

relative_hazard <- as.vector(
  exp(X %*% beta_hat)
)

Next, we estimate the baseline cumulative hazard using the Breslow estimator:

# Distinct observed event times
event_times <- sort(
  unique(lung2$time[lung2$status == 1])
)

# Breslow baseline cumulative hazard
H0 <- numeric(length(event_times))
cumulative_hazard <- 0

for (j in seq_along(event_times)) {

  tj <- event_times[j]

  # Number of events at tj
  d_j <- sum(
    lung2$time == tj &
    lung2$status == 1
  )

  # Individuals in the risk set at tj
  risk_set <- lung2$time >= tj

  # Breslow increment
  hazard_increment <-
    d_j / sum(relative_hazard[risk_set])

  cumulative_hazard <-
    cumulative_hazard + hazard_increment

  H0[j] <- cumulative_hazard
}

The cumulative baseline hazard is a step function. For each individual, we therefore need the value corresponding to the most recent event time at or before their observed follow-up time.

Before the first event, the cumulative hazard must equal zero.

# Locate the final event time at or before each observed time
interval_index <- findInterval(
  lung2$time,
  event_times
)

# Evaluate H0(t) at each individual's observed follow-up time
H0_at_Y <- ifelse(
  interval_index == 0,
  0,
  H0[interval_index]
)

The expected cumulative number of events for individual (i) is

The martingale residual is therefore

martingale_manual <-
  lung2$status -
  H0_at_Y * relative_hazard

We can compare the manual calculation with the residuals returned by coxph():

martingale_coxph <- residuals(
  fit_cox,
  type = "martingale"
)

comparison <- data.frame(
  manual = martingale_manual,
  coxph = as.numeric(martingale_coxph),
  difference = martingale_manual -
    as.numeric(martingale_coxph)
)

head(
  transform(
    comparison,
    manual = round(manual, 4),
    coxph = round(coxph, 4),
    difference = round(difference, 8)
  )
)

A data.frame: 6 × 3
	manual	coxph	difference

1	0.0062	0.0062	0
2	-0.5030	-0.5030	0
3	-3.1272	-3.1272	0
4	0.5335	0.5335	0
5	-2.3474	-2.3474	0
6	-4.2477	-4.2477	0

The two sets of residuals should agree to numerical precision, provided that fit_cox was fitted using the Breslow method:

5.4 Schoenfeld Residuals

Whereas martingale residuals assess the fit of individual observations, Schoenfeld residuals are used to assess the proportional hazards assumption. At each observed event time (t_j), the Schoenfeld residual is defined as the difference between the covariate value of the individual who experienced the event and the risk-set-weighted average covariate value:

There is one Schoenfeld residual for each event and each covariate, rather than one residual per individual. If the proportional hazards assumption holds, these residuals should show no systematic trend when plotted against time.

5.5 Computing Schoenfeld Residuals in R

The survival package computes Schoenfeld residuals directly from a fitted Cox model. The residual matrix has one row for each observed event and one column for each covariate.


sch <- residuals(fit_cox, type = "schoenfeld")

dim(sch)
head(sch)

A matrix: 6 × 2 of type dbl
	age	sex
5	0.9382653	0.7269349
11	9.9412847	-0.2707258
11	16.9412847	-0.2707258
11	2.9412847	-0.2707258
12	10.1431920	-0.2759338
13	12.2083432	-0.2777061

In practice, the proportional hazards assumption is usually assessed by plotting the scaled Schoenfeld residuals against event time and testing whether their slope differs significantly from zero. In R, the cox.zph() function performs both the graphical diagnostic and the corresponding statistical test.

Deviance residuals, Cox–Snell residuals, and DFBETAs are additional diagnostic measures derived from the same martingale and score-function framework. They are useful for identifying outliers, assessing overall model fit, and evaluating influential observations, and are beyond the scope of this tutorial.

5.6 Comparing Survival Between Groups

Before performing the log-rank test, the group-specific Kaplan–Meier curves provide a visual summary of the survival difference.



km_sex <- survfit(Surv(time, status) ~ sex, data = lung2)

plot(
  km_sex,
  xlab = "Time",
  ylab = "Estimated survival probability",
  main = "Kaplan–Meier Curves by Sex",
  mark.time = TRUE,
  lwd = 2,
  col = c("steelblue", "firebrick")
)

legend(
  "bottomleft",
  legend = c("Male", "Female"),
  col = c("steelblue", "firebrick"),
  lwd = 2,
  bty = "n"
)

The separation between the curves gives an immediate visual impression of the group difference, while the log-rank test formally evaluates whether the survival experiences differ over follow-up.

6 The Log-Rank Test

The log-rank test is the most widely used statistical test for comparing survival curves between two or more groups. Unlike the Cox proportional hazards model, it does not estimate regression coefficients or require covariates. Instead, it tests the null hypothesis that the groups have the same underlying survival experience.

At each observed event time, the test compares the observed number of events in each group with the expected number of events under the null hypothesis of equal survival.

Mathematically,

where

is the number of observed events in Group 1 at event time ,
is the number of individuals at risk in Group 1 just before ,
is the total number of events at ,
is the total number of individuals at risk at .

The variance of the observed-minus-expected statistic is

The log-rank test statistic is then

which follows approximately a chi-squared distribution with one degree of freedom under the null hypothesis.

Why does this look familiar?

The log-rank test is built from exactly the same risk-set bookkeeping used by the Kaplan–Meier estimator. At each event time, we count how many individuals remain at risk () and how many events occur (). The only difference is that the log-rank test performs these calculations separately for each group and compares the observed and expected numbers of events over the entire follow-up period.

6.1 Implementing the Log-Rank Test by Hand

To illustrate the calculation, we divide the lung dataset into two groups using the median age:

old: age above the median,
young: age at or below the median.

At each observed event time, the code calculates the numbers at risk and the observed events in the older group, then accumulates the observed count, expected count, and variance across follow-up.


lung3 <- na.omit(lung[, c("time", "status", "age")])

lung3$status <- lung3$status - 1
lung3$agegrp <- ifelse(
  lung3$age > median(lung3$age),
  "old",
  "young"
)

event_times <- sort(
  unique(lung3$time[lung3$status == 1])
)

O1 <- 0
E1 <- 0
V  <- 0

for (tj in event_times) {

  # Individuals at risk immediately before tj
  risk_idx <- which(lung3$time >= tj)

  n_j  <- length(risk_idx)
  n1j  <- sum(lung3$agegrp[risk_idx] == "old")
  n2j  <- n_j - n1j

  # Total events and events in the older group at tj
  d_j <- sum(
    lung3$time == tj &
    lung3$status == 1
  )

  d1j <- sum(
    lung3$time == tj &
    lung3$status == 1 &
    lung3$agegrp == "old"
  )

  # Accumulate observed and expected events
  O1 <- O1 + d1j
  E1 <- E1 + (n1j / n_j) * d_j

  # Accumulate the variance
  if (n_j > 1) {
    V <- V +
      (n1j * n2j * d_j * (n_j - d_j)) /
      (n_j^2 * (n_j - 1))
  }
}

Z2 <- (O1 - E1)^2 / V

c(
  O1 = O1,
  E1 = round(E1, 3),
  Var = round(V, 3),
  Z2 = round(Z2, 3)
)

O1: 85
E1: 76.237
Var: 40.821
Z2: 1.881

Here, (O_1) is the total number of observed events in the older group, (E_1) is the number expected under equal survival, and (Z^2) is the log-rank chi-squared statistic.

We can compare the manual result with R’s survdiff() function:





fit_logrank <- survdiff(
  Surv(time, status) ~ agegrp,
  data = lung3
)

fit_logrank

c(
  manual_Z2 = Z2,
  survdiff_Z2 = fit_logrank$chisq
)

Call:
survdiff(formula = Surv(time, status) ~ agegrp, data = lung3)

               N Observed Expected (O-E)^2/E (O-E)^2/V
agegrp=old   111       85     76.2     1.007      1.88
agegrp=young 117       80     88.8     0.865      1.88

 Chisq= 1.9  on 1 degrees of freedom, p= 0.2

manual_Z2: 1.88099912472048
survdiff_Z2: 1.88099912472049

The two chi-squared statistics should agree to numerical precision.

7 A Full Proportional-Hazards Diagnostic Workflow

The residuals introduced above are the raw material. Used together, in a fixed order, they form a standard diagnostic workflow for a fitted Cox model: does the model fit at all (Cox–Snell), are there poorly-fit or influential individuals (deviance), and does the proportional hazards assumption actually hold (cox.zph).

7.1 Deviance residuals

Martingale residuals are useful but skewed: they range over , so a handful of extreme negative values can dominate a plot. Deviance residuals are a variance-stabilizing transform of the martingale residual :

Why bother transforming at all? is approximately symmetric around 0 for a well-fitting model, the way ordinary residuals are in linear regression. That makes them far more useful than martingale residuals for spotting individual outliers by eye.

dev_manual <- sign(martingale_manual) *
  sqrt(pmax(0, -2 * (martingale_manual + lung2$status * log(lung2$status - martingale_manual))))
dev_coxph <- residuals(fit_cox, type = "deviance")

round(head(cbind(manual = dev_manual, coxph = dev_coxph)), 4)

A matrix: 6 × 2 of type dbl
	manual	coxph
1	0.0062	0.0062
2	-0.4371	-0.4371
3	-2.5009	-2.5009
4	0.6767	0.6767
5	-1.5095	-1.5095
6	-2.9147	-2.9147

The manually computed deviance residuals match those returned by coxph(). Plotting deviance residuals against the linear predictor (= X) is a standard diagnostic for identifying potential outliers and assessing model fit.

Plotting the deviance residuals against the fitted linear predictor provides a simple visual check for unusual observations or systematic departures from the fitted Cox model.


lp <- predict(fit_cox, type = "lp")

plot(
  lp,
  dev_coxph,
  pch = 16,
  col = adjustcolor("steelblue", 0.6),
  xlab = "Linear predictor",
  ylab = "Deviance residual",
  main = "Deviance Residuals vs Linear Predictor"
)

abline(h = 0, lty = 2, col = "grey40")

As you can see, the residuals are scattered roughly symmetrically around zero, with no obvious trend or a small number of extreme outlying observations.

No individual sits far outside the band formed by the rest, and there is no visible curvature — no sign that a transformation of age or sex is needed.

7.2 Cox–Snell residuals

Cox–Snell residuals check overall model fit rather than individual fit. For a correctly specified model, the residual

should behave like a censored sample from an distribution — because is just , and the cumulative hazard of any survival time evaluated at itself is exponential with rate 1.

How is that checked in practice? Treat as a new survival dataset and estimate its own Kaplan–Meier cumulative hazard, . If the original model fits well, this cumulative hazard should trace the 45° line , since that is the cumulative hazard of a unit-rate exponential.

Conveniently, , so no separate calculation is needed beyond the martingale residuals already computed.

coxsnell <- lung2$status - martingale_manual   # R_i = delta_i - M_i
cs_fit <- survfit(Surv(coxsnell, lung2$status) ~ 1)

plot(cs_fit$time, -log(cs_fit$surv), type = "s",
     xlab = "Cox-Snell residual", ylab = "Estimated cumulative hazard",
     main = "Cox-Snell Residual Plot")
abline(0, 1, col = "red", lty = 2)

The step function tracks the reference line closely apart from some flattening in the right tail, where only a handful of individuals remain at risk and the empirical cumulative hazard becomes noisy by construction. This is the expected pattern for an adequately fitting Cox model, rather than evidence of poor fit.

7.3 Testing Proportional Hazards with `cox.zph()`

Schoenfeld residuals were introduced earlier as raw per-event quantities. cox.zph() turns them into a formal test by examining whether the scaled Schoenfeld residuals vary systematically with a transformation of event time. By default, it uses a Kaplan–Meier transformation of time and tests whether the estimated time-dependent trend is zero.

What does a significant result mean?

A small p-value for a covariate indicates that its estimated effect on the hazard changes over follow-up time. In other words, the proportional hazards assumption may be violated for that covariate. The GLOBAL test evaluates the proportional hazards assumption for all covariates jointly.

zph_fit <- cox.zph(fit_cox)
zph_fit

       chisq df    p
age    0.208  1 0.65
sex    2.599  1 0.11
GLOBAL 2.761  2 0.25

par(mfrow = c(1, 2))
plot(zph_fit)

Neither covariate shows evidence of a violation of the proportional hazards assumption (age: (p = 0.65); sex: (p = 0.11); global test: (p = 0.25)). In both panels, the smoothed curves remain approximately horizontal and fluctuate around zero throughout follow-up. The proportional hazards assumption therefore appears reasonable for this model, and the estimated hazard ratios can be interpreted as approximately constant over the observed follow-up period.

7.4 Putting the workflow together

For a fitted Cox model, the same three checks apply in the same order every time:

Cox–Snell residuals — does the model fit overall? Compare their cumulative hazard to the 45° line.
Deviance residuals — which individuals fit badly? Plot against the linear predictor and scan for outliers or curvature.
cox.zph() — does proportional hazards hold, covariate by covariate? A significant result flags exactly which covariate needs a time-varying effect or stratification.

8 Summary

Kaplan–Meier estimation and Cox proportional hazards regression are built on the same counting-process framework but estimate different quantities.

Kaplan–Meier estimates the survival function (S(t)) directly as a product of conditional survival probabilities at each observed event time, with Greenwood’s formula providing its standard error.
Cox proportional hazards regression models the hazard as [ h(t X)=h_0(t)(^T X), ] and estimates () by maximizing the partial likelihood, which eliminates the unspecified baseline hazard by conditioning on the risk set at each event time.

On top of the fitted Cox model, a complete diagnostic workflow was developed from three residual types:

Cox–Snell residuals assess overall model fit by comparing the estimated cumulative hazard with the 45° reference line.
Deviance residuals identify potential outliers and departures from the assumed functional form of the covariates.
cox.zph(), based on scaled Schoenfeld residuals, formally tests the proportional hazards assumption for each covariate and for the model as a whole.

Every quantity derived in this tutorial—including the Kaplan–Meier estimator (S(t)), its standard error, the Cox regression coefficients (), their standard errors, martingale, deviance, and Cox–Snell residuals, and the log-rank test statistic—was implemented directly from its mathematical definition in base R and verified against the corresponding functions in the survival package (survfit(), coxph(), survdiff(), and cox.zph()), with agreement to at least three decimal places throughout.

9 References

Kaplan EL, Meier P. Nonparametric Estimation from Incomplete Observations. Journal of the American Statistical Association. 1958;53(282):457–481. https://www.jstor.org/stable/2281868
Cox DR. Regression Models and Life-Tables. Journal of the Royal Statistical Society: Series B. 1972;34(2):187–220. https://www.jstor.org/stable/2985181
Therneau TM, Grambsch PM. Modeling Survival Data: Extending the Cox Model. Springer; 2000. https://link.springer.com/book/10.1007/978-1-4757-3294-8

Phylogenetic Generalized Least Squares (PGLS)

Nivedita Bhadra — Tue, 21 Jul 2026 21:00:00 GMT

1 The problem ordinary regression ignores

Suppose you want to test whether brain size scales with body size across mammals, or whether flower color predicts pollinator visitation across plant species. The natural instinct is to collect one data point per species and run an ordinary least squares (OLS) regression. This is wrong, and has been known to be wrong since Felsenstein’s foundational 1985 paper.

The reason is simple. Species are not independent samples from some population; they are the tips of a phylogenetic tree. Two closely related species — say, a chimpanzee and a bonobo — resemble each other not because of some causal relationship between the traits under study, but because they inherited both traits from a recent common ancestor. Standard regression assumes residuals are independently and identically distributed. When your data points are structured by a phylogeny, that assumption is violated, and violated systematically: closely related species will tend to have correlated residuals. The practical consequence is that OLS understates its own uncertainty. Effective sample size is smaller than the number of species you counted, degrees of freedom are inflated, and p-values are optimistic — sometimes dramatically so (Felsenstein 1985; Freckleton et al. 2002).

Phylogenetic Generalized Least Squares (PGLS) addresses this directly. Rather than discarding the phylogeny or trying to “correct for” it after the fact, PGLS builds the expected pattern of non-independence into the regression itself, as a covariance structure on the residuals.

2 From independent contrasts to PGLS

Felsenstein’s original solution was phylogenetically independent contrasts (PICs): transform the raw trait values into a set of statistically independent contrasts computed along the branches of the tree, then run a standard regression on the contrasts, forced through the origin (Felsenstein 1985). This works well but is somewhat inflexible — it handles only Brownian motion evolution, offers no natural way to include multiple predictors or categorical covariates, and doesn’t extend cleanly to generalized linear models.

Grafen (1989) reframed the problem as one of generalized least squares regression, showing that PICs are mathematically equivalent to a GLS fit in which the residual covariance matrix is derived from the tree. Martins and Hansen (1997) generalized this further, and Pagel (1997, 1999) introduced scaling parameters — most importantly λ (lambda) — that let the strength of the phylogenetic signal in the residuals be estimated from the data rather than assumed. Freckleton, Harvey, and Pagel (2002) demonstrated that this λ-based approach is robust and statistically powerful even when the phylogeny is incompletely known. This is the method now generally called PGLS, and it is the standard tool in modern phylogenetic comparative biology (Symonds and Blomberg 2014).

3 The model, formally

An ordinary least squares regression assumes

that is, residuals are independent and identically distributed with covariance matrix . PGLS replaces this with

where is an matrix encoding the expected covariance between species’ residuals, derived from the phylogeny. Under a Brownian motion model of evolution, the covariance between two species is proportional to the amount of shared branch length between them and the root: closely related species (which share a longer path from the root) get a higher covariance, tips connected only near the root get a lower one. The generalized least squares estimator is then

which is just OLS after “whitening” the data by the inverse of the phylogenetic covariance structure. In practice, is rarely used raw; it is common to allow it to be scaled by Pagel’s , which multiplies the off-diagonal (shared ancestry) elements of the correlation matrix by while leaving the diagonal untouched. recovers the full Brownian expectation; collapses the model back to OLS, since it implies no phylogenetic signal in the residuals at all. is estimated by maximum likelihood alongside the regression coefficients, so the data themselves tell you how much phylogenetic correction is warranted (Pagel 1999; Freckleton et al. 2002).

4 Worked example in R

The example below is fully self-contained: it simulates a tree and trait data with a known phylogenetic structure, so PGLS can be shown recovering the correct relationship where OLS does not.

4.1 Packages

ape provides the phylogenetic infrastructure — tree objects, trait simulation, and the correlation structures (corBrownian, corPagel, corMartins, corGrafen) that plug directly into nlme::gls(). This combination is the base implementation of PGLS in R (Paradis and Schliep 2019; Pinheiro and Bates 2000). The caper package (Orme et al. 2013) offers a popular alternative interface (caper::pgls) built specifically for comparative datasets, and phytools (Revell 2012) provides complementary tools for tree manipulation and visualization; either integrates well with the workflow below.

install.packages(c("ape", "nlme"))   # if not already installed
library(ape)   # tree handling and correlation structures
library(nlme)  # gls(), the workhorse for PGLS


The downloaded binary packages are in
    /var/folders/vm/xtvv8jb542s04t61c12lsnvr0000gn/T//RtmpgOJBH7/downloaded_packages

4.2 Simulating a tree and correlated trait data

set.seed(123)

# 1. A random ultrametric tree with 40 tips ("species")
tree <- rcoal(40)
tree$tip.label <- paste0("sp", 1:40)

# 2. A predictor trait evolving under Brownian motion along the tree
x <- rTraitCont(tree, model = "BM", sigma = 1)

# 3. A response trait built as a linear function of x (true slope = 0.8),
#    plus its own phylogenetically structured error term
phylo_error <- rTraitCont(tree, model = "BM", sigma = 0.5)
y <- 0.8 * x + phylo_error

dat <- data.frame(species = tree$tip.label, x = x, y = y)
rownames(dat) <- dat$species
head(dat)

A data.frame: 6 × 3
	species	x	y

sp1	sp1	1.533555	1.119959
sp2	sp2	1.578064	1.176384
sp3	sp3	1.524573	1.050779
sp4	sp4	1.752620	1.301077
sp5	sp5	1.429647	2.070559
sp6	sp6	1.223539	1.970735

plot(tree, show.tip.label = FALSE, main = "Simulated 40-tip phylogeny")

Because both x and the error term were generated under Brownian motion on the same tree, the residuals of any regression of y on x will themselves carry phylogenetic signal — exactly the situation PGLS is designed for.

4.3 Naive OLS (ignoring the phylogeny)

m_ols <- lm(y ~ x, data = dat)
summary(m_ols)$coefficients

A matrix: 2 × 4 of type dbl
	Estimate	Std. Error	t value	Pr(>\|t\|)
(Intercept)	-0.1581428	0.09203221	-1.718342	9.387331e-02
x	1.1539112	0.08223846	14.031284	1.304257e-16

OLS estimates the slope well above the true value of 0.8, with a tight standard error and an extremely small p-value. This is precisely the failure mode Felsenstein warned about: shared ancestry inflates the apparent strength and certainty of the relationship.

4.4 PGLS assuming Brownian motion

corBM <- corBrownian(1, phy = tree, form = ~species)
m_bm <- gls(y ~ x, data = dat, correlation = corBM, method = "ML")
summary(m_bm)$tTable

A matrix: 2 × 4 of type dbl
	Value	Std.Error	t-value	p-value
(Intercept)	-0.3596555	0.4085034	-0.8804222	3.841651e-01
x	0.8462925	0.0751379	11.2631903	1.131397e-13

Once the Brownian covariance structure is built into the residuals, the slope estimate sits much closer to the true value (0.8), and the standard error is appropriately larger, reflecting the smaller effective sample size once phylogenetic redundancy is accounted for.

4.5 PGLS with Pagel’s λ estimated from the data

corPagel_str <- corPagel(value = 0.5, phy = tree, form = ~species)
m_lambda <- gls(y ~ x, data = dat, correlation = corPagel_str, method = "ML")
summary(m_lambda)$tTable

A matrix: 2 × 4 of type dbl
	Value	Std.Error	t-value	p-value
(Intercept)	-0.3579659	0.41925874	-0.8538067	3.985625e-01
x	0.8421314	0.07516831	11.2032762	1.323633e-13

m_lambda$modelStruct$corStruct

Correlation structure of class corPagel representing
  lambda 
1.000392

Here λ is estimated at essentially 1, correctly recovering the fact that the simulated residual error really was generated under pure Brownian motion. In real data, λ will rarely be exactly 0 or 1; it is precisely this intermediate value that tells you how much phylogenetic correction the data support, rather than forcing an all-or-nothing assumption.

4.6 Comparing models

AIC(m_ols, m_bm, m_lambda)

A data.frame: 3 × 2
	df	AIC

m_ols	3	49.61137
m_bm	3	-35.29059
m_lambda	4	-33.89102

Both PGLS models vastly outperform OLS by AIC, confirming that the phylogenetic structure in the residuals is real and that ignoring it costs a great deal of model fit. The Brownian and λ models are nearly indistinguishable here (as expected, since λ ≈ 1), but in general, letting λ be estimated rather than fixed at 1 is the safer default, since it nests both extremes.

5 Applying this to your own data

Match your tips to your data. The tree’s tip labels and your data frame’s row names (or a species-name column) must correspond exactly; mismatches are the most common source of silent errors.
Prune the tree to your species. Use ape::keep.tip() (or drop.tip()) to reduce a large reference phylogeny down to just the species in your dataset before fitting.
Check for polytomies and zero-length branches. These can make some correlation structures numerically unstable; ape::multi2di() resolves polytomies at random for testing purposes, though ideally branch lengths should come from a proper phylogenetic analysis.
Don’t assume Brownian motion by default. Fit λ (or, alternatively, Pagel’s κ or δ, or an Ornstein-Uhlenbeck model via ape::corMartins) and let the likelihood tell you which structure the data support, rather than hard-coding an assumption.
Diagnose the fitted model as you would any GLS: examine standardized residuals for outliers or remaining structure, and consider whether the trait itself might need a transformation before modeling.

6 Common pitfalls

Fitting PGLS with method = "REML" when comparing models with different fixed effects via likelihood ratio tests or AIC; use method = "ML" for that purpose, and switch to REML only when your final model’s fixed effects are settled.
Treating a λ estimate of 0 as evidence that phylogeny “doesn’t matter” without checking the confidence interval on λ; with modest sample sizes, that interval is often wide.
Applying PGLS to trait pairs measured on different subsets of species without pruning both the tree and the data consistently.
Forgetting that PGLS corrects for correlated residuals due to shared ancestry — it does not, by itself, tell you whether a relationship is causal or convergent.

7 References

Felsenstein, J. (1985). Phylogenies and the comparative method. The American Naturalist, 125(1), 1–15.

Freckleton, R. P., Harvey, P. H., & Pagel, M. (2002). Phylogenetic analysis and comparative data: A test and review of evidence. The American Naturalist, 160(6), 712–726.

Grafen, A. (1989). The phylogenetic regression. Philosophical Transactions of the Royal Society of London B, 326(1233), 119–157.

Martins, E. P., & Hansen, T. F. (1997). Phylogenies and the comparative method: A general approach to incorporating phylogenetic information into the analysis of interspecific data. The American Naturalist, 149(4), 646–667.

Orme, D., Freckleton, R., Thomas, G., Petzoldt, T., Fritz, S., Isaac, N., & Pearse, W. (2013). caper: Comparative Analyses of Phylogenetics and Evolution in R. R package.

Pagel, M. (1999). Inferring the historical patterns of biological evolution. Nature, 401(6756), 877–884.

Paradis, E., & Schliep, K. (2019). ape 5.0: An environment for modern phylogenetics and evolutionary analyses in R. Bioinformatics, 35(3), 526–528.

Pinheiro, J., & Bates, D. (2000). Mixed-Effects Models in S and S-PLUS. Springer.

Revell, L. J. (2012). phytools: An R package for phylogenetic comparative biology (and other things). Methods in Ecology and Evolution, 3(2), 217–223.

Symonds, M. R. E., & Blomberg, S. P. (2014). A primer on phylogenetic generalised least squares. In L. Z. Garamszegi (Ed.), Modern Phylogenetic Comparative Methods and Their Application in Evolutionary Biology (pp. 105–130). Springer.

From Raw Reads to GWAS-Ready Data

Nivedita Bhadra — Thu, 09 Jul 2026 21:00:00 GMT

The full path this tutorial walks, end to end: every stage below is a real, copy-pasteable command, from a sequencer’s raw output to a matrix ready for association testing.

GATK PLINK Regenie HPC

How to use this

Each numbered step has a “What’s happening” explanation before the commands — read that first, then run the commands, then move on. No need to rush any of it. The tutorial is written for: - Part -1: environment setup on an Apple Silicon Mac (Lima Linux VM), since Linux-native bioinformatics tools don’t run natively on macOS ARM64 - Part 0: reference data staging, done once - Parts 1-3: the pipeline itself, runnable on your Mac at chr20/small-cohort scale now, and identically on real HPC later (just swap out the setup — every downstream command is the same either way)

Assume once you’re set up: - Reference: GRCh38 (Homo_sapiens_assembly38.fasta), matching known-sites VCFs from GATK resource bundle - Conda/mamba environments per tool group, to avoid dependency conflicts (set up in Part -1) - Everything here is designed to scale from “one trio on a laptop” to “thousands of WGS samples on a cluster” — the commands don’t change, only whether you prefix them with sbatch

Part -1 — Setting up your environment (Apple Silicon Mac, step by step)

What’s happening here, conceptually first: almost every tool in this pipeline (GATK, bwa-mem2, samtools, plink) was built and tested for Linux. macOS is Unix-like but not Linux — different system libraries, different binary format. On Intel Macs this mostly didn’t matter because the CPU instruction set matched Linux x86_64 builds. On Apple Silicon (M-series), the CPU architecture itself (ARM64) is different, so pre-built Linux binaries won’t run at all, and macOS-native ARM64 builds of these tools are incomplete or missing on bioconda. The fix is to run a real Linux virtual machine on your Mac — not emulation of individual programs, but an actual Linux kernel and userspace — so every tool installs and runs exactly as it would on your future HPC. This also means everything you learn about the environment transfers directly.

Step 1: Install Lima (the VM manager)

# Homebrew is macOS's package manager — if you don't have it:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Lima runs a lightweight Linux VM with automatic file-sharing back to macOS
brew install lima

What this does: Lima creates a small Linux VM (using Apple’s native vz virtualization framework on M-series, so it’s fast — not slow software emulation) and automatically mounts your home directory inside it, so files you create feel local either way.

Step 2: Start a Linux VM sized for this work

limactl start --cpus 12 --memory 32 --disk 150 --arch aarch64 --vm-type vz default

What each flag does: - --cpus 12 — give the VM 12 of your performance cores; leaves headroom for macOS itself - --memory 32 — 32GB to the VM out of your 48GB total; leaves the rest for macOS + any GUI tools you’re running alongside - --disk 150 — reference genomes, VCFs, and intermediate BAMs eat disk fast; 150GB is a safe starting allocation for chr20-scale work (you’ll want more for full-genome runs) - --arch aarch64 — native ARM64 Linux, matching your M5 Pro’s actual architecture (no translation layer) - --vm-type vz — Apple’s native virtualization (faster than the older QEMU-based default on Apple Silicon)

# Drop into a shell inside the VM — this is now a real Linux machine
limactl shell default

Everything from here on runs inside this Lima shell, not in your normal macOS terminal. Your prompt will change to reflect this. To leave, just exit; to come back later, limactl start default (if stopped) then limactl shell default again.

Step 3: Install Miniforge/mamba inside the VM

What’s happening: conda/mamba is a package + environment manager built specifically for scientific software — it resolves complex dependency chains (a specific version of GATK needing a specific Java version needing specific system libraries) that would be painful to install by hand. mamba is a faster drop-in replacement for conda’s solver. We install it fresh inside the Linux VM (not on macOS) since that’s where all the Linux binaries will actually run.

curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh"
bash Miniforge3-Linux-aarch64.sh -b -p $HOME/miniforge3
source $HOME/miniforge3/bin/activate
conda init bash
# close and reopen the Lima shell (or `source ~/.bashrc`) for this to take effect

Step 4: Create isolated environments per tool group

What’s happening: different bioinformatics tools sometimes need conflicting versions of shared libraries (e.g. different Python or htslib versions). Rather than fighting version conflicts in one giant environment, we create separate named environments — one activated at a time — so each tool group gets exactly the dependencies it wants without breaking another.

mamba create -n ngs-core -c bioconda -c conda-forge \
  fastp fastqc multiqc bwa-mem2 samtools sambamba gatk4 bcftools \
  vcftools plink plink2 mosdepth -y

mamba create -n ngs-sv -c bioconda manta smoove truvari -y

mamba create -n ngs-phase -c bioconda shapeit4 beagle eagle -y

mamba create -n ngs-gwas -c bioconda -c conda-forge regenie -y

Why split this way: ngs-core covers Part 1 + most of Part 2 (alignment, calling, basic QC). ngs-sv isolates the structural-variant callers, which sometimes pin older/different library versions. ngs-phase and ngs-gwas isolate the Part 3 tools similarly. You’ll conda activate ngs-core (etc.) before running the relevant commands — one environment active at a time.

Before trusting any of this, verify the key tools actually resolved to working binaries (aarch64 bioconda coverage isn’t 100% for every package):

conda activate ngs-core
bwa-mem2 version
gatk --version
samtools --version

If any of these fail to install or run, the fallback is forcing that specific package through the linux-64 (Intel Linux) channel, which the VM’s Linux kernel can still execute via qemu-user-static emulation — slower for that one tool, but everything else stays native:

mamba create -n ngs-core-x64 --platform linux-64 -c bioconda -c conda-forge <tool-that-failed> -y

Step 5: Confirm your resource allocation matches what you planned

nproc                # should show 12 (or whatever you allocated)
free -h              # should show ~32GB total
df -h $HOME          # confirm your disk allocation

Step 6: A sane working directory layout

What’s happening: genomics pipelines generate a lot of intermediate files (raw reads → trimmed reads → BAM → dedup BAM → recalibrated BAM → GVCF → joint VCF → filtered VCF → phased VCF → imputed VCF, per sample, per chromosome). Without a deliberate layout you will lose track of what’s what within a day. This structure mirrors what you’ll see on real HPC filesystems.

mkdir -p ~/ngs_work/{refs,raw,work/{trimmed,aligned,dedup,bqsr,gvcf,joint,filtered,phased,imputed},qc,scripts}
cd ~/ngs_work

From here, Part 0 (reference staging) and Part 1 onward all run inside this Lima Linux shell, with ngs-core (or the relevant environment) activated.

Part 0 — Staging reference data (run once, reuse across all samples)

What’s happening: every downstream step needs a common coordinate system to align reads to and a set of “known truth” variant sites to calibrate against. This section downloads exactly those shared resources once, so every sample you process afterward references the identical files — consistency here is what makes results comparable across samples and, eventually, across collaborating cohorts like FinnGen partners.

mkdir -p refs/{genome,vqsr,phasing,giab}
cd refs

# --- GRCh38 reference + GATK resource bundle (public, no GCP account required) ---
gsutil -m cp gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta \
  gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai \
  gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict \
  genome/

gsutil -m cp \
  gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dbsnp138.vcf.gz \
  gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dbsnp138.vcf.gz.tbi \
  gs://gcp-public-data--broad-references/hg38/v0/Mills_and_1000G_gold_standard.indels.hg38.vcf.gz \
  gs://gcp-public-data--broad-references/hg38/v0/Mills_and_1000G_gold_standard.indels.hg38.vcf.gz.tbi \
  gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.known_indels.vcf.gz \
  gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.known_indels.vcf.gz.tbi \
  vqsr/

# --- VQSR training/truth resources ---
gsutil -m cp \
  gs://gcp-public-data--broad-references/hg38/v0/hapmap_3.3.hg38.vcf.gz \
  gs://gcp-public-data--broad-references/hg38/v0/1000G_omni2.5.hg38.vcf.gz \
  gs://gcp-public-data--broad-references/hg38/v0/1000G_phase1.snps.hg38.vcf.gz \
  vqsr/

# --- Index BWA-MEM2 reference (do this once, it's slow) ---
bwa-mem2 index genome/Homo_sapiens_assembly38.fasta

# --- 1000 Genomes high-coverage phased panel (for SHAPEIT4/Eagle/Beagle) ---
for chr in $(seq 1 22); do
  wget -P phasing/ \
    "http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000G_2504_high_coverage/working/20220422_3202_phased_SNV_INDEL_SV/1kGP_high_coverage_Illumina.chr${chr}.filtered.SNV_INDEL_SV_phased_panel.vcf.gz"
done

# --- Genetic maps (Beagle/Eagle format, GRCh38) ---
wget -P phasing/ https://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh38.map.zip
unzip phasing/plink.GRCh38.map.zip -d phasing/

# --- GIAB truth set (HG002) for validating your pipeline against known-truth calls ---
# Browse the current directory tree first — URL paths shift periodically:
# https://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/
# Grab the GRCh38-aligned AJ-trio (HG002/3/4) FASTQ or CRAM set from there.

# --- Sanity checksum everything before trusting it ---
find . -type f \( -name "*.vcf.gz" -o -name "*.fasta" \) -exec md5sum {} \; > checksums.txt

If gsutil isn’t available on your HPC (common — many clusters block outbound GCS auth), use the plain HTTPS mirror instead: prefix each path with https://storage.googleapis.com/gcp-public-data--broad-references/... and swap gsutil -m cp for wget.

What each of these files is actually for, since it matters later when something goes wrong: - Homo_sapiens_assembly38.fasta (+.fai/.dict) — the coordinate system itself. Every position you’ll ever report (“chr20:1234567”) is only meaningful relative to this exact file. Mixing reference builds (GRCh37 vs GRCh38) between steps is one of the most common and hardest-to-notice pipeline bugs. - dbsnp138.vcf.gz — a catalog of previously observed variant positions, used to annotate/flag known vs novel sites and as a training resource - Mills...indels and known_indels — curated “trustworthy” indel sites, used specifically by BQSR to know which mismatches are real biology (leave alone) vs sequencing error (recalibrate away) - hapmap, 1000G_omni2.5, 1000G_phase1.snps — high-confidence SNP truth sets used later by VQSR to learn what a real variant’s statistical signature looks like, versus an artifact’s - the 1000G phased panel + genetic maps — used in Part 3 for phasing and imputation, which need a large reference cohort’s known haplotype structure to infer missing/uncertain genotypes in your own samples - GIAB HG002 — the one sample in this whole tutorial where you actually know the right answer, which is why it’s used for validation exercises later

PART 1 — Raw FASTQ → Analysis-Ready BAM/CRAM

1.1 Sanity-check and QC raw reads

What’s happening: a sequencer doesn’t output “the genome” — it outputs millions of short, overlapping, imperfect text reads (FASTQ format: a DNA sequence + a per-base quality score for each read). Before you spend hours aligning bad data, you check whether the raw reads themselves are trustworthy: are quality scores dropping off, is there leftover adapter sequence the sequencer failed to strip, is the GC content distribution what you’d expect from human DNA (a skew can mean contamination). fastp then actively cleans the reads — trimming adapters and low-quality bases — before anything touches the reference genome.

# Per-sample QC
fastqc sample_R1.fastq.gz sample_R2.fastq.gz -o qc/ -t 8

# Adapter trimming + quality filtering + auto-detection of adapters (fastp is faster than trimmomatic and gives a JSON report)
fastp \
  -i sample_R1.fastq.gz -I sample_R2.fastq.gz \
  -o trimmed_R1.fastq.gz -O trimmed_R2.fastq.gz \
  --detect_adapter_for_pe \
  --qualified_quality_phred 20 \
  --length_required 36 \
  --thread 8 \
  --json fastp_sample.json --html fastp_sample.html

# Aggregate QC across a whole batch — this is what you'll actually eyeball daily
multiqc qc/ fastp_*.json -o multiqc_report/

What to flag before proceeding: per-base quality drop-off, adapter content >5%, GC content bimodality (contamination), duplication rate pre-alignment (library complexity proxy).

1.2 Alignment (BWA-MEM2 — faster drop-in replacement for BWA-MEM, same output)

What’s happening: alignment is the step where each of those millions of short reads gets placed at its most likely position of origin on the reference genome. bwa-mem2 does this using an FM-index (a compressed, searchable representation of the whole genome built by bwa-mem2 index) to rapidly find candidate matching regions, then does a proper local alignment (allowing for mismatches/small indels from real biological variation or sequencing error) to pick the best position and produce a CIGAR string describing exactly how the read lines up. The output SAM/BAM format is essentially “for every read: where it landed, how well it matched, and how confident we are.” The read group (-R string) isn’t decoration — it’s metadata GATK relies on downstream to know which reads came from the same physical sequencing run, which matters for error-model calibration in BQSR and for correctly attributing genotypes to the right sample.

# Index reference once
bwa-mem2 index Homo_sapiens_assembly38.fasta

# Align with proper read group (mandatory for GATK downstream — get this wrong and BQSR/HaplotypeCaller will silently misbehave)
bwa-mem2 mem -t 16 \
  -R "@RG\tID:sample1_L001\tSM:sample1\tPL:ILLUMINA\tLB:lib1\tPU:flowcell1.lane1" \
  Homo_sapiens_assembly38.fasta \
  trimmed_R1.fastq.gz trimmed_R2.fastq.gz \
  | samtools sort -@ 8 -o sample1.sorted.bam -
samtools index sample1.sorted.bam

For multi-lane samples, align each lane separately with distinct ID/PU, same SM, then merge:

samtools merge -@ 8 sample1.merged.bam sample1_L001.sorted.bam sample1_L002.sorted.bam
samtools index sample1.merged.bam

1.3 Mark duplicates

What’s happening: PCR amplification during library prep, and sometimes the sequencer itself (optical duplicates), can produce multiple reads that are actually copies of the exact same original DNA fragment rather than independent observations. If you count these as independent evidence, you’ll inflate confidence in variant calls that are really just one PCR-duplicated read counted five times. This step doesn’t delete duplicates — it flags them (a SAM flag bit) so downstream tools like HaplotypeCaller know to down-weight or ignore them, while the raw data is preserved for auditing.

# GATK's MarkDuplicatesSpark parallelizes well; sambamba markdup is a lighter/faster alternative for large WGS batches
gatk MarkDuplicatesSpark \
  -I sample1.merged.bam \
  -O sample1.dedup.bam \
  -M sample1.dedup_metrics.txt \
  --spark-master local[16]

# Alternative (often faster at scale):
sambamba markdup -t 16 sample1.merged.bam sample1.dedup.bam

1.4 Base Quality Score Recalibration (BQSR)

What’s happening: the sequencer’s own per-base quality scores are a machine’s estimate of its own error rate, and that estimate is systematically biased in predictable ways — by position in the read, by the specific sequence context, by which machine cycle produced the base. BQSR builds an empirical error model by comparing observed mismatches against the known truth sites (dbSNP/Mills/known indels you staged in Part 0) — any mismatch at a known-variant site is assumed to be real biology, while mismatches everywhere else are assumed to be sequencing error, and the model learns the actual error rate per context. ApplyBQSR then rewrites each base’s quality score using this corrected model. Downstream variant callers weight evidence by these quality scores directly, so this step measurably improves variant calling accuracy — it’s not cosmetic.

gatk BaseRecalibrator \
  -I sample1.dedup.bam \
  -R Homo_sapiens_assembly38.fasta \
  --known-sites Homo_sapiens_assembly38.dbsnp138.vcf.gz \
  --known-sites Mills_and_1000G_gold_standard.indels.hg38.vcf.gz \
  --known-sites Homo_sapiens_assembly38.known_indels.vcf.gz \
  -O sample1.recal.table

gatk ApplyBQSR \
  -I sample1.dedup.bam \
  -R Homo_sapiens_assembly38.fasta \
  --bqsr-recal-file sample1.recal.table \
  -O sample1.final.bam

1.5 Convert to CRAM (mandatory at scale — ~40-60% smaller than BAM, and this matters a lot once you’re storing thousands of WGS samples)

What’s happening: BAM stores every read’s full sequence even though most of it matches the reference exactly. CRAM instead stores only the differences from the reference (which you supply via -T) plus a reference-independent fallback for unmapped/unusual reads, then compresses that much smaller representation further. The tradeoff is that a CRAM file is meaningless without the exact reference FASTA that produced it — which is exactly why Part 0 pinned one canonical reference file for the whole pipeline.

samtools view -T Homo_sapiens_assembly38.fasta -C -@ 8 \
  -o sample1.final.cram sample1.final.bam
samtools index sample1.final.cram

1.6 Post-alignment QC gates — run these before anyone trusts the sample

What’s happening: this is the checkpoint where you decide, quantitatively, whether a sample is good enough to keep processing or needs to be flagged/excluded/re-sequenced. Each metric answers a distinct question: coverage/depth (samtools coverage, mosdepth) — did we sequence deeply enough at each position to call genotypes confidently, and are there suspicious dropouts that might indicate a deletion or a capture failure; samtools stats/flagstat — did alignment itself go well (mapping rate, proper pairing, insert size distribution matching the library prep expectation); contamination (VerifyBamID2) — is this BAM secretly a mixture of two individuals’ DNA (a real risk in any wet-lab pipeline, from sample swaps to cross-contamination), measured by checking whether allele frequencies at known-variant sites look like they came from one clean genome or a mixture.

# Coverage / depth
samtools coverage sample1.final.cram > sample1.coverage.txt
mosdepth --by 500 sample1 sample1.final.cram   # fast, windowed depth — great for spotting CNV-scale dropouts

# Insert size, alignment stats
samtools stats sample1.final.cram > sample1.stats.txt
samtools flagstat sample1.final.cram > sample1.flagstat.txt

# Contamination check (critical for WGS/WES QC gates — cross-sample or cross-species contamination)
gatk CollectFingerprintingDetailMetrics \
  -I sample1.final.cram -R Homo_sapiens_assembly38.fasta \
  -O sample1.fingerprint --HAPLOTYPE_MAP hapmap_3.3.hg38.map

VerifyBamID2 --SVDPrefix resource/1000g.phase3 \
  --Reference Homo_sapiens_assembly38.fasta --BamFile sample1.final.cram

Standard gate thresholds to codify into a QC pipeline: mean coverage (≥30x WGS / ≥50-100x on-target WES), %reads mapped ≥95%, duplication rate <20%, contamination (FREEMIX) <3%, sex-check concordance (via chrX/Y coverage ratio) against reported sex.

PART 2 — Variant Calling, Joint Genotyping, SV Calling, and VCF-Level QC

2.1 Per-sample GVCF calling (GATK’s scalable design — call once per sample, joint-genotype later without re-calling)

What’s happening: HaplotypeCaller doesn’t just look at one position at a time — it locally reassembles the reads in each region into candidate haplotypes (via a local de Bruijn-like graph), then uses a statistical model to score how well each candidate haplotype explains the observed reads, which is what lets it call indels accurately and not just simple substitutions. Critically, we run it in GVCF mode (-ERC GVCF), which records a confidence estimate at every position — including ones with no variant — rather than only variant sites. This is the design that makes cohort-scale calling tractable: if you called each sample against a fixed set of “known” variant sites, you’d never discover a variant that’s rare or private to your cohort; GVCF mode defers the “is this actually a variant across the cohort” decision to the joint-genotyping step next, while still only running the expensive per-read reassembly once per sample.

gatk HaplotypeCaller \
  -I sample1.final.cram \
  -R Homo_sapiens_assembly38.fasta \
  -O sample1.g.vcf.gz \
  -ERC GVCF \
  --native-pair-hmm-threads 8

For WES, restrict to the capture region:

gatk HaplotypeCaller \
  -I sample1.final.cram -R Homo_sapiens_assembly38.fasta \
  -L capture_targets.interval_list -ip 100 \
  -O sample1.g.vcf.gz -ERC GVCF

2.2 Joint genotyping across a cohort (this is where FinnGen-scale joint calling happens)

What’s happening: GenomicsDBImport merges each sample’s per-position confidence records (from the GVCFs) into an efficient, queryable on-disk database — think of it as a sparse matrix of “sample × genomic position → evidence,” built specifically so it scales to thousands of samples without exploding in size or IO cost, which flat CombineGVCFs doesn’t do well past a few hundred samples. GenotypeGVCFs then makes the actual joint calling decision: at every position where any sample showed evidence of a variant, it looks across all samples simultaneously to decide the final genotype for each individual — a site that looked marginal in one sample alone can become a confident call once you see the same allele recurring across the cohort, which is exactly the statistical leverage joint calling gives you over calling each sample in isolation.

# Consolidate GVCFs — GenomicsDBImport scales far better than CombineGVCFs for thousands of samples
gatk GenomicsDBImport \
  --genomicsdb-workspace-path cohort_db_chr1 \
  -L chr1 \
  --sample-name-map cohort.sample_map.txt \
  --batch-size 50 \
  --reader-threads 8

# sample_map.txt format: sample_idpath/to/sample.g.vcf.gz  (one per line)

gatk GenotypeGVCFs \
  -R Homo_sapiens_assembly38.fasta \
  -V gendb://cohort_db_chr1 \
  -O cohort_chr1.vcf.gz

Run per-chromosome in parallel across the cluster, then merge:

bcftools concat -Oz -o cohort.joint.vcf.gz cohort_chr*.vcf.gz
bcftools index -t cohort.joint.vcf.gz

2.3 Variant Quality Score Recalibration (VQSR) — the actual filtering step that matters most for downstream GWAS validity

What’s happening: raw joint-called variants include real biology and a substantial number of technical artifacts (mapping errors, systematic sequencing biases, assembly mistakes in repetitive regions). A single hard threshold on any one annotation (e.g. “reject anything with QD < 2”) is crude because artifacts don’t look identical across the genome. VQSR instead trains a Gaussian mixture model on several per-variant annotations simultaneously (QD = quality normalized by depth, MQ = mapping quality, FS/SOR = strand bias measures, ReadPosRankSum = whether the variant allele clusters suspiciously at read ends) using the truth/training sites from Part 0 as positive examples of “this is what real variants look like.” Every variant in your callset then gets a score for how well it resembles the truth-set variants’ statistical fingerprint, and ApplyVQSR cuts at a sensitivity threshold (e.g. 99.5% — “keep the score cutoff that retains 99.5% of the training truth sites”) rather than an arbitrary single-metric threshold. This is genuinely more powerful than hard filtering because it learns the joint distribution of what real variants look like across all metrics at once, not each metric independently.

# SNPs
gatk VariantRecalibrator \
  -R Homo_sapiens_assembly38.fasta -V cohort.joint.vcf.gz \
  --resource:hapmap,known=false,training=true,truth=true,prior=15.0 hapmap_3.3.hg38.vcf.gz \
  --resource:omni,known=false,training=true,truth=false,prior=12.0 1000G_omni2.5.hg38.vcf.gz \
  --resource:1000G,known=false,training=true,truth=false,prior=10.0 1000G_phase1.snps.hg38.vcf.gz \
  --resource:dbsnp,known=true,training=false,truth=false,prior=2.0 Homo_sapiens_assembly38.dbsnp138.vcf.gz \
  -an QD -an MQ -an MQRankSum -an ReadPosRankSum -an FS -an SOR \
  -mode SNP \
  -O cohort.snps.recal --tranches-file cohort.snps.tranches

gatk ApplyVQSR \
  -R Homo_sapiens_assembly38.fasta -V cohort.joint.vcf.gz \
  --recal-file cohort.snps.recal --tranches-file cohort.snps.tranches \
  --truth-sensitivity-filter-level 99.5 -mode SNP \
  -O cohort.snps.filtered.vcf.gz

# Indels (same pattern, different annotations/resources)
gatk VariantRecalibrator \
  -R Homo_sapiens_assembly38.fasta -V cohort.snps.filtered.vcf.gz \
  --resource:mills,known=false,training=true,truth=true,prior=12.0 Mills_and_1000G_gold_standard.indels.hg38.vcf.gz \
  --resource:dbsnp,known=true,training=false,truth=false,prior=2.0 Homo_sapiens_assembly38.dbsnp138.vcf.gz \
  -an QD -an FS -an ReadPosRankSum -an MQRankSum -an SOR \
  -mode INDEL -O cohort.indels.recal --tranches-file cohort.indels.tranches

gatk ApplyVQSR \
  -R Homo_sapiens_assembly38.fasta -V cohort.snps.filtered.vcf.gz \
  --recal-file cohort.indels.recal --tranches-file cohort.indels.tranches \
  --truth-sensitivity-filter-level 99.0 -mode INDEL \
  -O cohort.final.vcf.gz

For cohorts too small for reliable VQSR (roughly <30 samples), fall back to hard filtering with VariantFiltration using the same annotation thresholds as tranches approximate — worth knowing both paths since production cohorts often need to bootstrap from small pilot batches.

2.4 Structural variants (SNVs/indels are not the whole story — this is explicitly in scope per the JD)

What’s happening: HaplotypeCaller is built for small variants (SNVs and indels typically under ~50bp) because its local reassembly approach breaks down for larger rearrangements. Structural variants — deletions, duplications, insertions, inversions, translocations spanning hundreds to millions of bases — need different evidence entirely: read pairs whose insert size or orientation doesn’t match expectation, split reads that align partially to two different genomic locations, and coverage depth changes. Manta scans for exactly these signal types and assembles breakpoints from them; smoove (built on lumpy+svtyper+duphold) is a lighter, faster alternative tuned for running consistently across large cohorts, followed by a merge/genotype step so that an SV detected in one sample gets properly genotyped (present/absent/heterozygous) across every other sample in the cohort, the SV equivalent of joint genotyping in section 2.2.

# Manta — good general-purpose SV caller (deletions, duplications, insertions, inversions, translocations)
configManta.py --bam sample1.final.cram --referenceFasta Homo_sapiens_assembly38.fasta \
  --runDir manta_sample1
manta_sample1/runWorkflow.py -j 8

# smoove — lighter-weight, good for cohort-scale SV joint calling
smoove call --outdir sv_out/ --name sample1 --fasta Homo_sapiens_assembly38.fasta \
  --exclude exclude.cnvnator_100bp.GRCh38.20170403.bed -p 1 --genotype sample1.final.cram

smoove merge --name cohort --fasta Homo_sapiens_assembly38.fasta sv_out/*.genotyped.vcf.gz
smoove genotype -d -x -p 1 --name cohort-joint --outdir genotyped/ \
  --fasta Homo_sapiens_assembly38.fasta --vcf cohort.merged.sites.vcf.gz sample1.final.cram

2.5 VCF-level QC (bcftools + vcftools — the daily-driver tools for harmonization QC)

What’s happening: this is sample-level QC done after variant calling rather than on the raw reads — sometimes a sample only reveals a problem once you can see its genotypes in the context of the whole cohort. Per-sample missingness (how many sites failed to get a confident genotype call) flags samples with poor overall data quality. Heterozygosity rate is a classic contamination/inbreeding proxy — unusually high heterozygosity can indicate sample contamination (looks like a mixture of two genomes), unusually low can indicate inbreeding or a technical artifact. Ts/Tv ratio (transition vs transversion substitution rate) has a well-known expected value for real human variation (~2.0-2.1 genome-wide, ~3.0 in exomes) — a callset that deviates substantially is a signal something upstream (often VQSR filtering) went wrong. The KING kinship table detects unexpected relatedness or accidental sample duplicates before they contaminate GWAS results, since undetected relatedness violates the independence assumption most association tests rely on.

# Site-level metrics
bcftools stats cohort.final.vcf.gz > cohort.stats.txt

# Per-sample missingness, heterozygosity, Ts/Tv — classic sample-QC exclusion criteria
vcftools --gzvcf cohort.final.vcf.gz --missing-indv --out cohort_qc
vcftools --gzvcf cohort.final.vcf.gz --het --out cohort_qc
vcftools --gzvcf cohort.final.vcf.gz --TsTv-summary --out cohort_qc

# Relatedness / duplicate-sample detection — essential before any GWAS
plink2 --vcf cohort.final.vcf.gz --make-king-table --out cohort_king

PART 3 — Harmonization, Phasing, Imputation, and Building the GWAS-Ready Matrix

3.1 Normalize and harmonize variant representation (the step everyone underestimates — left-alignment/normalization mismatches silently break joint analyses across cohorts, e.g. FinnGen + external partners)

What’s happening: the same real-world variant can be written multiple equivalent ways in VCF format — a multi-allelic site (one position, several alternate alleles) can be listed as one record or split into several, and indels near repetitive sequence can be left-aligned differently depending on which tool produced the call. If two cohorts represent the identical variant differently, a naive position-based merge will treat them as two different variants and silently lose the overlap. bcftools norm -m -any splits multi-allelic sites into separate biallelic records and left-aligns indels against the reference to a single canonical representation, which is what makes downstream joins across samples, cohorts, and reference panels actually work. The chromosome-naming step handles the other classic mismatch — some tools/panels use chr1, others use 1 for the same chromosome, and a merge between mismatched naming conventions fails or silently drops everything.

bcftools norm -f Homo_sapiens_assembly38.fasta -m -any cohort.final.vcf.gz -Oz -o cohort.norm.vcf.gz
bcftools index -t cohort.norm.vcf.gz

# Harmonize chromosome naming (chr1 vs 1 — the single most common cross-cohort merge failure)
bcftools annotate --rename-chrs chr_name_conv.txt cohort.norm.vcf.gz -Oz -o cohort.harm.vcf.gz

3.2 Convert to PLINK format and apply standard GWAS-QC filters

What’s happening: PLINK’s binary format (.bed/.bim/.fam) is far more compact and faster to compute on than VCF for the kind of matrix-wide operations GWAS tools need, so this is the standard conversion point. The four filters here are the textbook GWAS quality gates, each guarding against a different failure mode: --geno 0.02 drops variants missing genotype calls in >2% of samples (usually a sign the site is technically unreliable, not biologically interesting); --mind 0.02 drops samples missing >2% of variants (a poor-quality sample, analogous to the per-sample missingness check in 2.5 but now enforced as a hard exclusion); --maf 0.01 drops variants below 1% minor allele frequency, since very rare variants have too little statistical power for standard GWAS and are also where genotyping/calling errors masquerade as “variants” most easily; --hwe 1e-6 flags departures from Hardy-Weinberg equilibrium, which in a large random-mating population is often a signal of genotyping error rather than real biology (true selection/population-structure departures exist but are rarer than artifacts at this threshold).

plink2 --vcf cohort.harm.vcf.gz --make-bed --out cohort_plink --set-all-var-ids @:#:\$r:\$a

plink2 --bfile cohort_plink \
  --geno 0.02 --mind 0.02 --maf 0.01 --hwe 1e-6 \
  --make-bed --out cohort_qc1

# LD-based pruning for PCA/kinship input
plink2 --bfile cohort_qc1 --indep-pairwise 200 50 0.2 --out prune
plink2 --bfile cohort_qc1 --extract prune.prune.in --make-bed --out cohort_pruned

3.3 Population structure and relatedness (needed both for QC exclusions and as GWAS covariates)

What’s happening: GWAS association tests assume, at minimum, that samples are unrelated and that allele frequency differences aren’t confounded with the trait through ancestry alone (population stratification — a classic false-positive source where an allele is just more common in one ancestral group that also happens to differ in the trait for unrelated cultural/environmental reasons). KING-robust kinship estimates relatedness directly from genotype sharing patterns in a way that’s robust even in the presence of population structure, letting you catch cryptic relatedness before it inflates your test statistics. PCA on the LD-pruned genotypes captures the main axes of ancestry-driven genetic variation in your cohort; the top PCs get included as covariates in the GWAS model in Part 3.7 specifically to soak up and control for that stratification signal.

# KING-robust kinship for relatedness/family structure — standard in FinnGen-style pipelines
plink2 --bfile cohort_pruned --make-king-table --out cohort_kinship

# PCA for ancestry / population-stratification covariates
plink2 --bfile cohort_pruned --pca 10 --out cohort_pca

3.4 Phasing (statistical, reference-based haplotype phasing — precursor to imputation)

What’s happening: standard genotyping tells you which two alleles a person carries at each position (e.g. heterozygous A/G) but not which chromosome copy each allele sits on — that’s phase, and imputation algorithms specifically need phased haplotypes to work (they extend known haplotype blocks, not independent genotypes). SHAPEIT4/Eagle2 solve this statistically: given your cohort’s genotypes plus a large reference panel of already-phased haplotypes (the 1000G panel from Part 0), they find the most probable haplotype assignment by looking for chunks of your sample’s genotype pattern that match known haplotype segments in the reference — real chromosomes are mosaics of a surprisingly limited number of ancestral haplotype blocks, which is exactly the structure phasing algorithms exploit.

# SHAPEIT4 — accurate and fast for large biobank-scale cohorts
shapeit4 --input cohort_qc1.chr20.vcf.gz \
  --map genetic_map_chr20.b38.txt \
  --region chr20 \
  --reference 1000GP.chr20.phased.vcf.gz \
  --output cohort.chr20.phased.vcf.gz \
  --thread 16

# Eagle2 is the common alternative, especially paired with Beagle/Minimac imputation pipelines
eagle --vcfRef 1000GP.chr20.bcf --vcfTarget cohort_qc1.chr20.vcf.gz \
  --geneticMapFile genetic_map_hg38.txt.gz \
  --outPrefix cohort.chr20.eagle_phased --numThreads 16

3.5 Imputation to a reference panel (Beagle5 shown; swap for Minimac4 if using the Michigan/TOPMed panel conventions)

What’s happening: your directly-called genotypes cover only the positions your sequencing/calling actually captured well. Imputation fills in genotypes at additional positions present in the (much larger, deeply-sequenced) reference panel, by matching your phased haplotype segments against the panel’s haplotypes and inferring which panel haplotype your sample most likely shares at each additional position — essentially borrowing statistical power from the reference panel’s depth to extend your effective variant coverage far beyond what you directly sequenced. Every imputed genotype comes with an uncertainty estimate (DR2/INFO score — how confidently the algorithm could make that inference), which is why the very next line filters on it: low-confidence imputed calls are worse than not having the variant at all, since they inject noise into a GWAS as if it were real signal.

java -Xmx32g -jar beagle.jar \
  gt=cohort.chr20.phased.vcf.gz \
  ref=1000GP.chr20.bref3 \
  map=plink.chr20.GRCh38.map \
  out=cohort.chr20.imputed \
  nthreads=16

# Post-imputation QC: filter on imputation INFO score — this threshold is a direct input to your downstream SBayesR/GCTB work
bcftools view -i 'INFO/DR2>0.8' cohort.chr20.imputed.vcf.gz -Oz -o cohort.chr20.imputed.qc.vcf.gz

3.6 Assemble the analysis-ready GWAS matrix

What’s happening: simple bookkeeping at this point — stitching the per-chromosome imputed files back into one genome-wide dataset and converting to PLINK2’s .pgen format, which stores dosages (fractional genotype probabilities from imputation, not just hard 0/1/2 calls) far more efficiently than VCF at cohort scale. The final MAF filter here is a second, post-imputation pass — imputation itself can introduce very-rare spurious variants at the edges of panel coverage, so re-applying a frequency floor right before association testing is standard practice.

# Concatenate imputed chromosomes, convert to PLINK2 pgen for efficient large-scale storage
bcftools concat -Oz -o cohort.imputed.allchr.vcf.gz cohort.chr*.imputed.qc.vcf.gz
plink2 --vcf cohort.imputed.allchr.vcf.gz --make-pgen --out cohort_gwas_ready

# Final MAF/INFO filters typically applied right before association testing
plink2 --pfile cohort_gwas_ready --maf 0.01 --make-pgen --out cohort_gwas_final

3.7 Run the GWAS (regenie — the standard for biobank-scale mixed-model GWAS; handles related individuals and population structure natively)

What’s happening: naive linear/logistic regression of trait-on-genotype, one variant at a time, breaks down at biobank scale because related individuals and residual population structure inflate false positives (the same confound PCA/kinship in 3.3 partially addresses, but not completely). Regenie’s two-step design handles this more robustly: Step 1 fits a whole-genome ridge regression using directly-genotyped variants to build a polygenic prediction of the trait per individual, which effectively captures and removes the confounding structure (relatedness + polygenic background) before any single-variant test happens. Step 2 then tests each imputed variant for association with the trait residual left after subtracting that Step-1 prediction — dramatically better calibrated than testing raw trait values directly, especially for related samples or rare-variant/case-control-imbalanced traits, which is why --firth (a bias-corrected logistic regression) is included for binary traits.

# Step 1: whole-genome ridge regression on genotyped (not imputed) variants
regenie --step 1 --bed cohort_qc1 \
  --phenoFile pheno.txt --covarFile covars_with_PCs.txt \
  --bsize 1000 --bt --out cohort_step1

# Step 2: association testing on imputed dosages
regenie --step 2 --pgen cohort_gwas_final \
  --phenoFile pheno.txt --covarFile covars_with_PCs.txt \
  --bsize 400 --bt --firth --approx \
  --pred cohort_step1_pred.list \
  --out cohort_gwas_results

3.8 Where GCTB/SBayesR picks up from here

The cohort_gwas_results summary statistics (chr, pos, effect allele, beta, se, p, N) plus an LD reference panel built from cohort_pruned (or a matched external panel) are exactly the two inputs your SBayesR workflow expects — same COJO-format conventions you’ve already worked through with GCTB. This is the natural handoff point from “production sequencing pipeline” into the polygenic-score/methods work mentioned as a growth area in the role.

Quick reference: end-to-end command chain (one sample, condensed)

fastp -i R1.fq.gz -I R2.fq.gz -o t1.fq.gz -O t2.fq.gz
bwa-mem2 mem -R "@RG\tID:s1\tSM:s1\tPL:ILLUMINA" ref.fa t1.fq.gz t2.fq.gz | samtools sort -o s1.bam -
gatk MarkDuplicatesSpark -I s1.bam -O s1.dedup.bam
gatk BaseRecalibrator -I s1.dedup.bam -R ref.fa --known-sites known.vcf.gz -O recal.table
gatk ApplyBQSR -I s1.dedup.bam -R ref.fa --bqsr-recal-file recal.table -O s1.final.bam
gatk HaplotypeCaller -I s1.final.bam -R ref.fa -ERC GVCF -O s1.g.vcf.gz
gatk GenomicsDBImport --genomicsdb-workspace-path db -V s1.g.vcf.gz -L chr20
gatk GenotypeGVCFs -R ref.fa -V gendb://db -O joint.vcf.gz
gatk VariantRecalibrator ... && gatk ApplyVQSR ... -O final.vcf.gz
bcftools norm -f ref.fa -m -any final.vcf.gz -Oz -o norm.vcf.gz
plink2 --vcf norm.vcf.gz --make-bed --geno 0.02 --maf 0.01 --hwe 1e-6 --out qc
shapeit4 --input qc.vcf.gz --reference ref_panel.vcf.gz --output phased.vcf.gz
beagle gt=phased.vcf.gz ref=panel.bref3 out=imputed
regenie --step 1 ... && regenie --step 2 ... --out gwas_results

What’s deliberately out of scope here (per your ask)

GCP-specific orchestration (Cromwell-on-GCP, Life Sciences API, Terra) — the pipeline above is portable to any of those later; the logic doesn’t change, only the scheduler does
Docker/containerization — swap mamba activate calls for singularity exec if your HPC mandates containers; commands inside stay identical

Challenges — work through these in order

1. Single-sample, single-chromosome, start to finish. Take GIAB HG002 chr20 FASTQs only (small enough to run on a laptop or one HPC node in under an hour). Run Part 1 end to end. Check samtools flagstat — you should see >99% mapped. Check VerifyBamID2 FREEMIX — should be near 0 since it’s a clean reference sample.

2. Validate against truth. GIAB publishes a high-confidence VCF + BED for HG002. Call variants with HaplotypeCaller on chr20, then run hap.py (Illumina’s benchmarking tool) against the GIAB truth set. Get your precision/recall numbers. This is the single most useful exercise for understanding what “good” variant calling actually looks like — most tutorials skip this step entirely.

3. Break the read group on purpose. Re-run alignment with a malformed -R string (missing SM tag) and watch HaplotypeCaller fail. Understanding why GATK is strict about read groups will save you hours in production.

4. Trio joint-calling. Pull HG002/HG003/HG004 (the GIAB Ashkenazi trio), joint-call all three with GenomicsDBImport/GenotypeGVCFs, then run bcftools +mendelian or gatk CalculateGenotypePosteriors to check Mendelian inheritance consistency. Any violations point to either de novo variants or pipeline errors — you’ll need to reason about which.

5. Small-cohort VQSR failure mode. Try running VariantRecalibrator on just 3-5 samples. It will likely fail or produce garbage tranches — this is expected. Now implement the hard-filtering fallback instead and compare filtered variant counts between the two approaches.

6. Sex-check discrepancy hunt. Compute chrX/chrY coverage ratios across a batch of samples (real or simulated) and cross-reference against reported sex metadata. Deliberately mislabel one sample and see if your QC catches it.

7. Full chr20-only pipeline through GWAS. Chain Parts 1→3 on chr20 for ~10-20 samples (mix real GIAB + simulate additional samples by downsampling public 1000G CRAMs). Get all the way to a regenie output file. This is the exercise that actually proves you understand the full chain, not just individual steps.

8. Orchestrate it. Once each step works manually, wrap the whole chain in Nextflow (nf-core/sarek is the production-grade reference implementation for exactly this pipeline — read its source even if you don’t run it) or Snakemake. This is the actual day-to-day form this work takes in a production setting, not standalone scripts.

9. The hard one — SV/CNV concordance. Run both Manta and smoove on the same sample, compare their outputs with truvari, and reconcile the discrepancies. SV calling disagreement between callers is normal and understanding why two reasonable tools disagree is a genuinely advanced skill.

Jupyter Notebook vs VS Code — which to run this in

Neither is where the pipeline itself should live, and that’s worth being direct about upfront:

The core pipeline (Parts 1–3) should be shell scripts or a workflow manager (Snakemake/Nextflow), not notebook cells. These are long-running, cluster-scheduled, multi-hour-to-multi-day jobs with dependencies between steps. Notebook kernels aren’t built for that — they don’t manage job arrays, don’t handle SLURM dependencies, and a kernel restart or SSH disconnect can silently kill a job that isn’t actually running through the scheduler. Debugging a failed step also gets harder in a notebook, since you lose clean stdout/stderr logs per stage.
VS Code (with the Remote-SSH extension) is the better fit for building and iterating on the pipeline itself — editing bash/Snakemake/Nextflow files directly on the HPC, using the integrated terminal to submit sbatch jobs, and tailing .log/.out files live. This is genuinely how most people doing this kind of work day-to-day operate.
Jupyter earns its place downstream, not upstream — for the QC/exploration layer: plotting multiqc output, visualizing PCA/kinship results, inspecting hap.py benchmark tables, plotting Manhattan/QQ plots from your regenie output. That’s real exploratory analysis where cell-by-cell iteration and inline plots genuinely help. Keep it strictly for that layer.
If you want notebook-style reproducibility and proper job control, look at Papermill (parameterized notebook execution triggered from a script) or JupyterLab with the SLURM kernel/magic extensions — but even then, treat it as a QC/reporting layer sitting on top of a script-driven pipeline, not the pipeline’s execution engine itself.

Practical setup: VS Code Remote-SSH into your HPC login node for all pipeline development and job submission, plus a Jupyter kernel (via srun --pty into a compute node, or JupyterHub if your cluster runs one) purely for the plotting/QC notebooks that consume the pipeline’s outputs.

Natural next study targets given the JD

Long-read/pangenome integration: minimap2 + PBSV/Sniffles for long-read SV calling, and the pangenome graph tooling (vg, minigraph-cactus) as the field moves off single linear references
Cromwell/WDL or Nextflow for turning this into a scalable, reproducible workflow — this is likely how “building and maintaining scalable workflows on HPC and cloud” is actually implemented day-to-day

LangChain and Retrieval-Augmented Generation

Nivedita — Sat, 04 Jul 2026 21:00:00 GMT

1 Introduction

A language model on its own answers from what it learned during training: it has no access to a specific organization’s documents, no memory of earlier turns in a conversation, and no way to call an external tool. LangChain is an open-source framework for closing that gap — it provides standard interfaces for connecting a model to external data, giving it persistent conversational state, and composing several model calls into a single workflow.

This notebook covers LangChain’s core abstractions and Retrieval-Augmented Generation (RAG) computationally, with every code example actually executed rather than described. Two constraints shape how that’s done. First, LangChain moved fast between 2023 and 2025: the framework reached its 1.0 release in October 2025, and several APIs commonly shown in older tutorials — LLMChain, SimpleSequentialChain, SequentialChain, and the classic ConversationBufferMemory family — have since been removed outright, not merely deprecated. This is verified directly below rather than asserted. Second, running real chains against a hosted model requires an API key and network access to that provider, neither of which is assumed here; instead, LangChain’s own deterministic fake models (FakeListLLM, FakeListChatModel) are used where a model call is needed, and a small self-contained retrieval pipeline is built by hand for the RAG section. This keeps every output in this notebook reproducible without external credentials, while the mechanics demonstrated — prompt composition, chaining, memory, and retrieval — are identical to what a real provider integration would do.

2 Checking the Current LangChain API Before Using It

Before building anything, it’s worth confirming what’s actually available in the installed version, since this determines which patterns below are current and which older material (including common tutorials) would fail outright.

#pip install langchain langchain-core langgraph numpy

import langchain
import langchain_core

print("langchain version:     ", langchain.__version__)
print("langchain_core version:", langchain_core.__version__)

# The pre-2024 chain and memory classes commonly shown in tutorials
for module_name, names in [
    ("langchain.chains", ["LLMChain", "SimpleSequentialChain", "SequentialChain"]),
    ("langchain.memory", ["ConversationBufferMemory", "ConversationSummaryMemory"]),
]:
    try:
        mod = __import__(module_name, fromlist=names)
        print(f"{module_name}: importable")
    except ModuleNotFoundError as e:
        print(f"{module_name}: NOT importable -> {e}")

langchain version:      1.3.14
langchain_core version: 1.5.3
langchain.chains: NOT importable -> No module named 'langchain.chains'
langchain.memory: NOT importable -> No module named 'langchain.memory'

Both langchain.chains and langchain.memory fail to import in the installed version — these modules were removed, not just deprecated. LLMChain was marked deprecated in LangChain 0.1.17 (April 2024) with removal scheduled for 1.0; ConversationBufferMemory and the related memory classes followed the same path starting in 0.3.1. The 1.0 release (October 2025) carried out that removal. Any code written against these classes — including the pattern shown in older training material on this topic — needs to be rewritten against the current API, which is what the rest of this notebook does.

3 Current Architecture

LangChain’s package layout reflects a deliberate split introduced as the framework matured:

langchain-core — the foundational abstractions: the Runnable interface, prompt templates, output parsers, and the LangChain Expression Language (LCEL), which is the | (pipe) syntax used to compose components.
langchain — higher-level, pre-built chains and utilities built on top of langchain-core.
langchain-community and provider packages (langchain-openai, langchain-google-genai, and so on) — integrations with specific model providers, vector stores, and document loaders, kept as optional dependencies so a project only installs what it needs.
LangGraph — a separate, closely integrated framework for stateful, multi-step agent workflows, now the recommended way to build anything involving persistent memory or multi-turn tool use.

Every composable piece in this architecture — a prompt, a model, a retriever, an output parser — implements the same Runnable interface, which is why they can all be connected with the same | operator regardless of type. This single unifying abstraction has effectively replaced the older “six components” framing (Model I/O, Data Connection, Chains, Agents, Memory, Callbacks) used in pre-2024 documentation of this framework; the underlying concerns — models, prompts, retrieval, memory, and tool-using agents — are all still present, just organized around Runnables and LCEL rather than as six separate categories.

4 Chains

A chain connects a sequence of steps — prompt formatting, a model call, output parsing, possibly a retrieval step — into a single callable pipeline, so an application doesn’t have to manually manage passing state from one step to the next. In the current API, “building a chain” means composing Runnable objects with the | operator; there is no separate Chain class to instantiate for the common cases.

4.1 The simplest chain: prompt to model to parser

The most basic pattern takes a prompt template, fills it with a variable, sends it to a model, and extracts the text from the response. Below, FakeListLLM stands in for a real provider – it returns a fixed, pre-specified response regardless of input, which is exactly what makes this reproducible without an API key.

from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.language_models.fake import FakeListLLM

llm = FakeListLLM(responses=["GenomeCraft Analytics"])

prompt = PromptTemplate.from_template(
    "What is a good name for a company that makes {product}?"
)

chain = prompt | llm | StrOutputParser()
result = chain.invoke({"product": "genomics data pipelines"})
print(result)

GenomeCraft Analytics

4.2 Sequential composition

Where an older API would use SimpleSequentialChain to feed the output of one chain into the next, the current pattern uses RunnablePassthrough.assign() to build a dictionary that carries both the original input and each intermediate result forward.

from langchain_core.runnables import RunnablePassthrough

llm_name = FakeListLLM(responses=["GenomeCraft Analytics"])
llm_description = FakeListLLM(responses=[
    "GenomeCraft Analytics builds reproducible pipelines for large-scale genomic data processing and interpretation."
])

name_prompt = PromptTemplate.from_template(
    "What is a good name for a company that makes {product}?"
)
description_prompt = PromptTemplate.from_template(
    "Write a 20-word description for the company: {company_name}"
)

name_chain = name_prompt | llm_name | StrOutputParser()
description_chain = description_prompt | llm_description | StrOutputParser()

overall_chain = (
    {"company_name": name_chain}
    | RunnablePassthrough.assign(description=description_chain)
)

result = overall_chain.invoke({"product": "genomics data pipelines"})
result

{'company_name': 'GenomeCraft Analytics',
 'description': 'GenomeCraft Analytics builds reproducible pipelines for large-scale genomic data processing and interpretation.'}

4.3 Multiple inputs and outputs

The older SequentialChain class was used when a workflow needed more than one input or output variable, and access to every intermediate result rather than just the final one. The equivalent LCEL pattern nests RunnablePassthrough.assign() calls, so each new field is added to a running dictionary without discarding the earlier ones – this reproduces a four-step translate to summarize to detect-language to follow-up workflow, with all four outputs available at the end.

translate_llm = FakeListLLM(responses=[
    "The GamersTech laptops impress with their exceptional performance and elegant design."
])
summarize_llm = FakeListLLM(responses=[
    "GamersTech laptops balance strong gaming performance with a sleek, portable design."
])
language_llm = FakeListLLM(responses=["French"])
followup_llm = FakeListLLM(responses=[
    "Merci pour ce retour tres positif sur nos ordinateurs portables GamersTech."
])

review = (
    "Les ordinateurs portables GamersTech impressionnent par ses performances "
    "exceptionnelles et son design elegant."
)

translate_chain = (
    PromptTemplate.from_template("Translate the following review to English:\n\n{review}")
    | translate_llm | StrOutputParser()
)
summarize_chain = (
    PromptTemplate.from_template("Summarize the following review in one sentence:\n\n{english_review}")
    | summarize_llm | StrOutputParser()
)
language_chain = (
    PromptTemplate.from_template("What language is the following review written in?\n\n{review}")
    | language_llm | StrOutputParser()
)
followup_chain = (
    PromptTemplate.from_template(
        "Write a follow-up response to this summary, in the specified language.\n\n"
        "Summary: {summary}\nLanguage: {language}"
    )
    | followup_llm | StrOutputParser()
)

pipeline = (
    RunnablePassthrough.assign(english_review=translate_chain)
    | RunnablePassthrough.assign(summary=summarize_chain, language=language_chain)
    | RunnablePassthrough.assign(followup_message=followup_chain)
)

output = pipeline.invoke({"review": review})
for key, value in output.items():
    print(f"{key}:\n  {value}\n")

review:
  Les ordinateurs portables GamersTech impressionnent par ses performances exceptionnelles et son design elegant.

english_review:
  The GamersTech laptops impress with their exceptional performance and elegant design.

summary:
  GamersTech laptops balance strong gaming performance with a sleek, portable design.

language:
  French

followup_message:
  Merci pour ce retour tres positif sur nos ordinateurs portables GamersTech.

4.4 Routing between sub-chains

A router chain sends an input to one of several specialized sub-chains depending on what kind of input it is – for instance, routing a support query to a billing-specific prompt or a technical-specific prompt. The current equivalent is RunnableBranch, which evaluates a sequence of (condition, chain) pairs and runs the first one that matches.

from langchain_core.runnables import RunnableBranch

billing_llm = FakeListLLM(responses=["This looks like a billing question -- routing to the billing team."])
technical_llm = FakeListLLM(responses=["This looks like a technical question -- routing to the engineering team."])
general_llm = FakeListLLM(responses=["Routing to general support."])

billing_chain = PromptTemplate.from_template("{query}") | billing_llm | StrOutputParser()
technical_chain = PromptTemplate.from_template("{query}") | technical_llm | StrOutputParser()
general_chain = PromptTemplate.from_template("{query}") | general_llm | StrOutputParser()

router = RunnableBranch(
    (lambda x: "invoice" in x["query"].lower() or "charge" in x["query"].lower(), billing_chain),
    (lambda x: "error" in x["query"].lower() or "crash" in x["query"].lower(), technical_chain),
    general_chain,  # default
)

for query in ["Why was I charged twice on my invoice?",
              "The pipeline crashes with a segmentation fault.",
              "What are your support hours?"]:
    print(query, "->", router.invoke({"query": query}))

Why was I charged twice on my invoice? -> This looks like a billing question -- routing to the billing team.
The pipeline crashes with a segmentation fault. -> This looks like a technical question -- routing to the engineering team.
What are your support hours? -> Routing to general support.

5 Memory

A language model call is stateless by default: nothing from one call is automatically available to the next unless it’s explicitly included in the prompt. “Memory” in this context means a mechanism for storing prior turns and re-injecting them, so a conversation feels continuous rather than resetting at every message.

Four memory strategies recur across older LangChain material, distinguished by what they store:

Buffer memory – stores the full conversation verbatim.
Buffer window memory – stores only the most recent k exchanges, discarding older ones.
Token buffer memory – keeps as much recent conversation as fits within a token budget, verbatim.
Summary memory – replaces older turns with a running summary, generated by the model itself.

Each represents a different trade-off between fidelity (how much detail is preserved) and cost (how many tokens are spent re-sending history on every call). All four are implemented as classes in langchain.memory in older versions – a module that, as confirmed above, no longer exists in the installed version. The current recommended approach uses LangGraph’s checkpointing system, which persists conversation state outside the chain itself and supports the buffer/window/summary trade-off through how the state is read back rather than through separate classes for each strategy.

from langgraph.graph import StateGraph, MessagesState, START, END
from langgraph.checkpoint.memory import InMemorySaver
from langchain_core.language_models.fake_chat_models import FakeListChatModel
from langchain_core.messages import HumanMessage

chat_model = FakeListChatModel(responses=[
    "Hello! It's nice to meet you.",
    "Your name is Sarah -- you told me a moment ago.",
])

def call_model(state: MessagesState):
    response = chat_model.invoke(state["messages"])
    return {"messages": [response]}

builder = StateGraph(MessagesState)
builder.add_node("model", call_model)
builder.add_edge(START, "model")
builder.add_edge("model", END)

graph = builder.compile(checkpointer=InMemorySaver())
config = {"configurable": {"thread_id": "session-1"}}

turn_1 = graph.invoke({"messages": [HumanMessage("Hi, my name is Sarah.")]}, config=config)
print("Turn 1:", turn_1["messages"][-1].content)

turn_2 = graph.invoke({"messages": [HumanMessage("What is my name?")]}, config=config)
print("Turn 2:", turn_2["messages"][-1].content)

persisted_state = graph.get_state(config)
print("\nMessages held in the checkpoint:", len(persisted_state.values["messages"]))

Turn 1: Hello! It's nice to meet you.
Turn 2: Your name is Sarah -- you told me a moment ago.

Messages held in the checkpoint: 4

The second call correctly answers “Sarah” because the full message history was retrieved from the checkpointer and passed back into the model, not because the fake model has any memory of its own – FakeListChatModel simply returns its next scripted response regardless of input. The thread_id in the config is what scopes the stored history to this particular conversation; a different thread_id would start from empty state.

6 Retrieval-Augmented Generation (RAG)

A model’s knowledge is fixed at training time. RAG addresses this by inserting a retrieval step before generation: given a query, first find the most relevant passages from an external document collection, then pass both the query and those passages to the model so its answer is grounded in retrieved material rather than in training data alone.

The pipeline has two stages:

Retriever – converts the query into a vector and finds the most similar vectors in a pre-built index of document chunks.
Generator – a language model that conditions its answer on both the original query and the retrieved chunks.

This addresses three specific weaknesses of using a language model alone: the model’s knowledge can be extended to include material it was never trained on (private documents, content newer than its training cutoff), answers can be traced back to a specific retrieved source, and the retrieval step reduces (though does not eliminate) the tendency to generate a plausible-sounding but unsupported answer.

6.1 Common applications

RAG is the standard architecture behind several application categories: chatbots that answer from internal company documents, legal or financial Q&A tools that summarize relevant clauses, research assistants that retrieve from academic or clinical literature, and support agents that answer from product documentation.

7 Vector Databases

A vector database is built to store and search high-dimensional numeric vectors – the embeddings produced by an embedding model – rather than the exact-match or range queries a conventional database is optimized for. Retrieval works by approximate nearest-neighbor search: given a query vector, find the stored vectors closest to it by some distance measure, typically cosine similarity.

Several tools recur in this space, each with a different operating model rather than a strict quality ranking: Chroma (lightweight, easy to run locally, common for prototyping), FAISS (a similarity-search library rather than a managed database, built for large-scale offline or self-hosted use), Pinecone and Qdrant (managed or self-hostable services built for production-scale, low-latency search), Weaviate (a vector-native database with built-in classification and hybrid search), and Redis (a general-purpose store that added vector search as a capability rather than being purpose-built for it). Which one fits a given project depends on scale, latency requirements, and whether self-hosting or a managed service is preferred – not on one being categorically “better” than the others.

8 Essential RAG Components, Built by Hand

The remaining pieces of a RAG pipeline – chunking, embedding, and similarity search – can be demonstrated directly without a hosted embedding model, using term-frequency vectors instead of dense neural embeddings. The mechanics (vectorize each chunk, vectorize the query the same way, rank by cosine similarity) are identical to what a real embedding model does; only the quality of the vectors themselves differs.

8.1 Chunking

Long documents are split into smaller pieces before embedding, because a single embedding vector for an entire document would blur together many different topics, making retrieval far less precise. Here, each chunk is already a short single-topic sentence, so this step is trivial – in practice a chunker would split a long document into passages of roughly a few hundred words each, often with a small overlap between consecutive chunks so a sentence spanning a chunk boundary isn’t lost entirely from either side.

document_chunks = [
    "GWAS identifies associations between genetic variants and traits across the genome.",
    "Fine-mapping narrows a GWAS locus down to the most likely causal variant.",
    "Polygenic risk scores aggregate many small-effect variants into a single predictive score.",
    "Retrieval-augmented generation grounds a language model's answer in retrieved documents.",
    "Vector databases store embeddings and support fast approximate nearest neighbor search.",
]
document_chunks

['GWAS identifies associations between genetic variants and traits across the genome.',
 'Fine-mapping narrows a GWAS locus down to the most likely causal variant.',
 'Polygenic risk scores aggregate many small-effect variants into a single predictive score.',
 "Retrieval-augmented generation grounds a language model's answer in retrieved documents.",
 'Vector databases store embeddings and support fast approximate nearest neighbor search.']

8.2 A minimal embedding model

Each chunk is converted to a vector using term-frequency counts weighted by inverse document frequency (TF-IDF) – common words shared across every chunk contribute little to the vector, while words distinctive to a particular chunk dominate it. This is a much cruder representation than a trained neural embedding model (it has no notion of synonyms or word order), but the retrieval mechanism built on top of it – cosine similarity ranking – is exactly the same mechanism a production vector database uses on dense embeddings.

import numpy as np
import re
from collections import Counter

STOPWORDS = {"a", "an", "the", "is", "are", "in", "on", "of", "to", "and", "or",
             "with", "how", "does", "do", "use", "s", "its", "this", "that",
             "for", "by", "into", "across"}

def tokenize(text):
    tokens = re.findall(r"[a-z0-9]+", text.lower())
    return [t for t in tokens if t not in STOPWORDS]

def build_vocabulary(chunks):
    vocab = sorted(set(tok for chunk in chunks for tok in tokenize(chunk)))
    return {tok: i for i, tok in enumerate(vocab)}

def inverse_document_frequency(chunks, vocab):
    n_chunks = len(chunks)
    doc_freq = np.zeros(len(vocab))
    for chunk in chunks:
        for tok in set(tokenize(chunk)):
            doc_freq[vocab[tok]] += 1
    return np.log((n_chunks + 1) / (doc_freq + 1)) + 1

def embed(text, vocab, idf):
    vec = np.zeros(len(vocab))
    for tok, count in Counter(tokenize(text)).items():
        if tok in vocab:
            vec[vocab[tok]] = count * idf[vocab[tok]]
    norm = np.linalg.norm(vec)
    return vec / norm if norm > 0 else vec

vocabulary = build_vocabulary(document_chunks)
idf = inverse_document_frequency(document_chunks, vocabulary)
chunk_embeddings = [embed(chunk, vocabulary, idf) for chunk in document_chunks]

print(f"Vocabulary size: {len(vocabulary)}")
print(f"Embedding dimension per chunk: {chunk_embeddings[0].shape[0]}")

Vocabulary size: 46
Embedding dimension per chunk: 46

8.3 Similarity search

The query is embedded with the same vocabulary and IDF weights, then ranked against every stored chunk by cosine similarity – since both vectors are already normalized to unit length, cosine similarity reduces to a plain dot product.

def cosine_similarity(a, b):
    return float(np.dot(a, b))

def retrieve(query, chunks, chunk_embeddings, vocab, idf, top_k=2):
    query_embedding = embed(query, vocab, idf)
    scores = [cosine_similarity(query_embedding, emb) for emb in chunk_embeddings]
    ranked = sorted(zip(scores, chunks), reverse=True)
    return ranked[:top_k]

query = "How does RAG use a vector database to ground an LLM's answer?"
top_matches = retrieve(query, document_chunks, chunk_embeddings, vocabulary, idf, top_k=5)

for score, chunk in top_matches:
    print(f"{score:.3f}  {chunk}")

0.236  Retrieval-augmented generation grounds a language model's answer in retrieved documents.
0.224  Vector databases store embeddings and support fast approximate nearest neighbor search.
0.000  Polygenic risk scores aggregate many small-effect variants into a single predictive score.
0.000  GWAS identifies associations between genetic variants and traits across the genome.
0.000  Fine-mapping narrows a GWAS locus down to the most likely causal variant.

The two chunks about RAG and vector databases score well above the three unrelated genetics chunks, which score exactly zero – they share no distinctive vocabulary with the query once stopwords are removed. This is the retrieval step of a RAG pipeline in miniature: the same query-embed-and-rank mechanism, at a scale of five chunks instead of millions, and with TF-IDF vectors standing in for a trained embedding model’s dense vectors.

8.4 Assembling the full pipeline

The retrieved chunks are then handed to a generator, exactly as in the earlier chain examples – a prompt template that includes the retrieved context, piped into a model.

rag_llm = FakeListLLM(responses=[
    "RAG grounds an LLM's answer by first retrieving relevant passages -- often from a "
    "vector database using similarity search -- and then conditioning generation on both "
    "the query and those retrieved passages, rather than relying on the model's training "
    "data alone."
])

rag_prompt = PromptTemplate.from_template(
    "Answer the question using only the context below.\n\n"
    "Context:\n{context}\n\nQuestion: {question}"
)

rag_chain = rag_prompt | rag_llm | StrOutputParser()

retrieved_context = "\n".join(chunk for _, chunk in top_matches[:2])
answer = rag_chain.invoke({"context": retrieved_context, "question": query})
print(answer)

RAG grounds an LLM's answer by first retrieving relevant passages -- often from a vector database using similarity search -- and then conditioning generation on both the query and those retrieved passages, rather than relying on the model's training data alone.

9 Best Practices

Build against LCEL, not the legacy Chain classes. As demonstrated above, LLMChain and its relatives are gone from the current package, so code written against them fails immediately rather than merely warning.
Choose memory strategy by conversation length and cost, not by default. Full buffer memory is simplest but grows unbounded; window or summary strategies trade some fidelity for a bounded cost.
Ground factual or domain-specific responses in retrieval. A RAG step reduces (not eliminates) the risk of a confident, unsupported answer.
Match the vector store to the deployment, not the other way around. A locally-run prototype and a production service under load have different requirements, and the “best” vector database differs accordingly.
Test each component of a pipeline independently before composing it. A Runnable can be invoked and inspected on its own before being piped into a longer chain.
Keep prompts concise and retrieval focused. Retrieving more chunks than the model needs increases cost and can dilute the most relevant context.
Treat this fast-moving ecosystem as fast-moving. As this notebook’s own deprecation check demonstrates, code and tutorials more than a year or so old are a reasonable place to start but should be verified against the current API before being relied on.

10 Limitations

Retrieval quality bounds answer quality. A generator conditioned on irrelevant or outdated retrieved chunks will produce a fluent but ungrounded answer regardless of how good the underlying model is.
Full conversation memory has a cost ceiling. Storing complete history without bound eventually exceeds context limits or becomes prohibitively expensive to resend on every call.
Composed pipelines are harder to test. A single model call is straightforward to evaluate; a multi-step chain with retrieval, memory, and several model calls has many more places a failure can originate from.
Latency compounds across steps. Each additional retrieval or model call in a pipeline adds to total response time.
The framework itself changes quickly. As shown directly above, APIs that were standard as recently as a year or two ago have since been removed, not just superseded – any documentation or tutorial, including this one eventually, is a snapshot rather than a permanent reference.

11 Summary

LangChain provides a common Runnable interface – composed with the | operator via LCEL – that unifies prompts, models, retrievers, and output parsers into a single composition system, and this has superseded the older, separate Chain classes (LLMChain, SimpleSequentialChain, SequentialChain) entirely; this notebook confirmed their removal directly rather than assuming it. Memory has undergone the same shift, from the deprecated ConversationBufferMemory family to LangGraph’s checkpointing system, which persists conversation state outside the chain and was demonstrated here with a minimal two-turn example. Retrieval-Augmented Generation was built end-to-end at small scale – chunking, a hand-built TF-IDF embedding, cosine-similarity retrieval, and a generation step conditioned on the retrieved context – which exercises the same retrieve-then-generate mechanism a production system built on a real embedding model and vector database would use, just with a transparent, dependency-free stand-in for the embedding step itself.

12 References

Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (2020) – arXiv:2005.11401
Karpukhin et al., “Dense Passage Retrieval for Open-Domain Question Answering” (2020) – arXiv:2004.04906
Vaswani et al., “Attention Is All You Need” (2017) – arXiv:1706.03762
Gao et al., “Retrieval-Augmented Generation for Large Language Models: A Survey” (2023) – arXiv:2312.10997
LangChain documentation – docs.langchain.com

A correction to a common citation. Material on this topic sometimes cites arXiv:2301.12652 as “A Survey on Retrieval-Augmented Generation” by Karpukhin et al. (2023). That identifier actually belongs to a different paper (REPLUG, on retrieval-augmented black-box language models), and Karpukhin et al.’s well-known 2020 paper is on dense passage retrieval, not a RAG survey. The reference list above cites the actual survey (Gao et al., 2023) and the correctly attributed Karpukhin et al. paper separately.

13 Try It Yourself

Modify the router chain’s conditions so a query containing “refund” is also routed to the billing chain, and confirm the routing with a new test query.
Extend the sequential pipeline (translate to summarize to detect language to follow-up) with a fifth step that scores the summary’s length against a target, reusing the RunnablePassthrough.assign() pattern.
Add two more document chunks to the RAG retrieval example – one relevant to the existing query, one clearly not – and confirm the ranking places them where expected.
Look up LangChain’s current documentation for one component used in this notebook (LCEL, LangGraph checkpointers, or RunnableBranch) and note anything that has changed since this notebook was written.

14 Solutions

Worked solutions to the four exercises above. Each was run against the same installed LangChain version confirmed at the top of this notebook.

14.1 1. Routing “refund” queries to billing

Adding a third or condition to the billing branch’s lambda is enough – no new branch is needed, since a refund question is a billing question.

from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.language_models.fake import FakeListLLM
from langchain_core.runnables import RunnableBranch

billing_llm = FakeListLLM(responses=["This looks like a billing question -- routing to the billing team."])
technical_llm = FakeListLLM(responses=["This looks like a technical question -- routing to the engineering team."])
general_llm = FakeListLLM(responses=["Routing to general support."])

billing_chain = PromptTemplate.from_template("{query}") | billing_llm | StrOutputParser()
technical_chain = PromptTemplate.from_template("{query}") | technical_llm | StrOutputParser()
general_chain = PromptTemplate.from_template("{query}") | general_llm | StrOutputParser()

router = RunnableBranch(
    (lambda x: "invoice" in x["query"].lower() or "charge" in x["query"].lower() or "refund" in x["query"].lower(), billing_chain),
    (lambda x: "error" in x["query"].lower() or "crash" in x["query"].lower(), technical_chain),
    general_chain,  # default
)

for query in ["Why was I charged twice on my invoice?",
              "The pipeline crashes with a segmentation fault.",
              "What are your support hours?",
              "Can I get a refund for last month's subscription?"]:
    print(query, "->", router.invoke({"query": query}))

Why was I charged twice on my invoice? -> This looks like a billing question -- routing to the billing team.
The pipeline crashes with a segmentation fault. -> This looks like a technical question -- routing to the engineering team.
What are your support hours? -> Routing to general support.
Can I get a refund for last month's subscription? -> This looks like a billing question -- routing to the billing team.

The new query about a refund is correctly routed to the billing chain alongside the existing invoice/charge queries.

14.2 2. Scoring summary length against a target

A fifth RunnablePassthrough.assign() step adds a length_score field, following the exact pattern the rest of the pipeline already uses – the new step reads summary from the running dictionary and reports how far its word count is from a target.

from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.language_models.fake import FakeListLLM
from langchain_core.runnables import RunnablePassthrough, RunnableLambda

translate_llm = FakeListLLM(responses=[
    "The GamersTech laptops impress with their exceptional performance and elegant design."
])
summarize_llm = FakeListLLM(responses=[
    "GamersTech laptops balance strong gaming performance with a sleek, portable design."
])
language_llm = FakeListLLM(responses=["French"])
followup_llm = FakeListLLM(responses=[
    "Merci pour ce retour tres positif sur nos ordinateurs portables GamersTech."
])

review = (
    "Les ordinateurs portables GamersTech impressionnent par ses performances "
    "exceptionnelles et son design elegant."
)

translate_chain = (
    PromptTemplate.from_template("Translate the following review to English:\n\n{review}")
    | translate_llm | StrOutputParser()
)
summarize_chain = (
    PromptTemplate.from_template("Summarize the following review in one sentence:\n\n{english_review}")
    | summarize_llm | StrOutputParser()
)
language_chain = (
    PromptTemplate.from_template("What language is the following review written in?\n\n{review}")
    | language_llm | StrOutputParser()
)
followup_chain = (
    PromptTemplate.from_template(
        "Write a follow-up response to this summary, in the specified language.\n\n"
        "Summary: {summary}\nLanguage: {language}"
    )
    | followup_llm | StrOutputParser()
)

# New fifth step: score the summary's length against a target word count,
# reusing the RunnablePassthrough.assign() pattern from the rest of the pipeline.
TARGET_WORDS = 12

def score_summary_length(inputs):
    word_count = len(inputs["summary"].split())
    return {
        "word_count": word_count,
        "target_words": TARGET_WORDS,
        "delta": word_count - TARGET_WORDS,
    }

length_score_step = RunnableLambda(score_summary_length)

pipeline = (
    RunnablePassthrough.assign(english_review=translate_chain)
    | RunnablePassthrough.assign(summary=summarize_chain, language=language_chain)
    | RunnablePassthrough.assign(followup_message=followup_chain)
    | RunnablePassthrough.assign(length_score=length_score_step)
)

output = pipeline.invoke({"review": review})
for key, value in output.items():
    print(f"{key}:\n  {value}\n")

review:
  Les ordinateurs portables GamersTech impressionnent par ses performances exceptionnelles et son design elegant.

english_review:
  The GamersTech laptops impress with their exceptional performance and elegant design.

summary:
  GamersTech laptops balance strong gaming performance with a sleek, portable design.

language:
  French

followup_message:
  Merci pour ce retour tres positif sur nos ordinateurs portables GamersTech.

length_score:
  {'word_count': 11, 'target_words': 12, 'delta': -1}

The generated summary comes in one word under the 12-word target – delta: -1. All five outputs, including the original input, remain available in the final dictionary, which is the main advantage of the nested-assign() pattern over a plain linear chain.

14.3 3. Adding two more chunks to the retrieval example

One new chunk is written to be relevant to the existing query (about cosine similarity ranking), and one is written to be clearly unrelated (about heritability, matching the theme of the other irrelevant chunks already in the set).

import numpy as np
import re
from collections import Counter

STOPWORDS = {"a", "an", "the", "is", "are", "in", "on", "of", "to", "and", "or",
             "with", "how", "does", "do", "use", "s", "its", "this", "that",
             "for", "by", "into", "across"}

def tokenize(text):
    tokens = re.findall(r"[a-z0-9]+", text.lower())
    return [t for t in tokens if t not in STOPWORDS]

def build_vocabulary(chunks):
    vocab = sorted(set(tok for chunk in chunks for tok in tokenize(chunk)))
    return {tok: i for i, tok in enumerate(vocab)}

def inverse_document_frequency(chunks, vocab):
    n_chunks = len(chunks)
    doc_freq = np.zeros(len(vocab))
    for chunk in chunks:
        for tok in set(tokenize(chunk)):
            doc_freq[vocab[tok]] += 1
    return np.log((n_chunks + 1) / (doc_freq + 1)) + 1

def embed(text, vocab, idf):
    vec = np.zeros(len(vocab))
    for tok, count in Counter(tokenize(text)).items():
        if tok in vocab:
            vec[vocab[tok]] = count * idf[vocab[tok]]
    norm = np.linalg.norm(vec)
    return vec / norm if norm > 0 else vec

def cosine_similarity(a, b):
    return float(np.dot(a, b))

def retrieve(query, chunks, chunk_embeddings, vocab, idf, top_k=2):
    query_embedding = embed(query, vocab, idf)
    scores = [cosine_similarity(query_embedding, emb) for emb in chunk_embeddings]
    ranked = sorted(zip(scores, chunks), reverse=True)
    return ranked[:top_k]

document_chunks = [
    "GWAS identifies associations between genetic variants and traits across the genome.",
    "Fine-mapping narrows a GWAS locus down to the most likely causal variant.",
    "Polygenic risk scores aggregate many small-effect variants into a single predictive score.",
    "Retrieval-augmented generation grounds a language model's answer in retrieved documents.",
    "Vector databases store embeddings and support fast approximate nearest neighbor search.",
    # New: one relevant to the query below, one clearly not.
    "Cosine similarity ranks retrieved chunks by comparing their embedding vectors to the query vector.",
    "Heritability partitions phenotypic variance into genetic and environmental components.",
]

vocabulary = build_vocabulary(document_chunks)
idf = inverse_document_frequency(document_chunks, vocabulary)
chunk_embeddings = [embed(chunk, vocabulary, idf) for chunk in document_chunks]

query = "How does RAG use a vector database to ground an LLM's answer?"
top_matches = retrieve(query, document_chunks, chunk_embeddings, vocabulary, idf, top_k=len(document_chunks))

for score, chunk in top_matches:
    print(f"{score:.3f}  {chunk}")

0.261  Retrieval-augmented generation grounds a language model's answer in retrieved documents.
0.170  Vector databases store embeddings and support fast approximate nearest neighbor search.
0.165  Cosine similarity ranks retrieved chunks by comparing their embedding vectors to the query vector.
0.000  Polygenic risk scores aggregate many small-effect variants into a single predictive score.
0.000  Heritability partitions phenotypic variance into genetic and environmental components.
0.000  GWAS identifies associations between genetic variants and traits across the genome.
0.000  Fine-mapping narrows a GWAS locus down to the most likely causal variant.

The new relevant chunk lands third, just behind the two chunks it shares vocabulary with (“vector”, “database”), and the new irrelevant chunk scores exactly zero alongside the other genetics chunks, since it shares no non-stopword vocabulary with the query. The ranking places both new chunks exactly where expected.

14.4 4. What’s changed in the current documentation

Checked two components used in this notebook against LangChain’s current reference docs:

RunnableBranch is unchanged. The current reference lists it as available since v0.1 with no deprecation notice, so the routing pattern used above is still the recommended one, not a stopgap.
LangGraph checkpointing has grown a companion concept since this notebook’s core pattern was written. The current persistence docs now distinguish two systems: checkpointers (like InMemorySaver, used above) for short-term, thread-scoped memory, and a separate Store (for example InMemoryStore) for long-term, cross-thread memory such as user preferences or facts that should persist beyond a single conversation thread. The docs also now explicitly flag InMemorySaver as suitable for debugging and testing only, recommending PostgresSaver for production use, a distinction not called out above.

For the two-turn example in this notebook, both points are consistent with what’s shown – InMemorySaver is exactly right for a scoped, in-memory demo, and there was no cross-thread memory need that would have called for a Store.

Twins, Variance, and the Logic of Heritability

Nivedita Bhadra — Sun, 21 Jun 2026 21:00:00 GMT

The one fact this whole tutorial runs on: MZ and DZ twins share C and E equally, but differ in exactly how much of A they share – which is enough, on its own, to separate genetic from environmental variance.

Heritability ACE Model Twin Studies R

Why do some siblings look and behave almost identically, while others in the same family seem to have little in common?

Height, cognitive ability, disease risk — nearly every measurable human trait varies from person to person.

How much of that variation comes from genetic differences, and how much comes from the environment people happen to grow up in?

That question is what heritability tries to answer. This tutorial builds the idea up from scratch: what heritability actually means, why twins are uniquely suited to estimate it, how that logic becomes a fitted statistical model, and how the same idea extends beyond twins to unrelated people using DNA directly.

1 Heritability Is a Statement About Variance, Not About You

A heritability of 70% does not mean 70% of your height is genetic.

Heritability describes how much of the variation across a population is attributable to genetic differences. It says nothing about any one individual.

Phenotypic variance splits into two pieces:

where is the total observed variance in a trait, is the share attributable to genetic differences between people, and is the share attributable to environmental differences.

Heritability is simply the genetic fraction of that total:

A trait can be highly heritable and still change dramatically across generations. Human height sits around , yet average height rose substantially over the 20th century as nutrition and healthcare improved. High heritability describes current variance, not future immutability — and it is specific to the population it was measured in. A population with little environmental variation will show a higher than one with large environmental disparities, for the exact same trait and the exact same genetics, simply because differs between them.

2 Why Twins Are a Natural Experiment

Directly separating from in an ordinary sample is impossible — you never observe the same person raised in two different genetic or environmental worlds.

Twins offer a workaround, because two twin types differ in exactly one respect.

Monozygotic (MZ) twins  →  share ~100% of their segregating genetic variants
Dizygotic (DZ) twins    →  share ~50% of their segregating genetic variants, on average

Both twin types, raised together, are assumed to share their rearing environment equally.

If a trait is influenced by genetics, MZ twins — who share all their genes — should resemble each other more closely than DZ twins, who share only half. The size of that gap between MZ and DZ similarity is where an estimate of heritability comes from.

3 From Twin Correlations to Variance Components

Twin models split phenotypic variance into three parts rather than two:

A  =  additive genetic variance        (variants add up across the genome)
C  =  shared (common) environment      (identical for both twins in a pair)
E   =  unique environment               (everything that differs within a pair,
                                          including measurement error)

Given the sharing rules above, the expected correlation between twins works out to:

(All variances here are on the standardized scale, so .)

Two equations, two unknowns — the difference between them isolates directly.

4 Falconer’s Method

Subtracting the two correlation equations gives a strikingly simple estimator, due to Falconer (1960):

No model-fitting software required — just two correlations and arithmetic. Let’s simulate twin data with known true variance components and check whether this recovers them.

simulate_twins <- function(n, a, c, e, shared_genetic_fraction) {
  # a, c, e are given as SDs; shared_genetic_fraction = 1 for MZ, 0.5 for DZ
  A_shared  <- rnorm(n, 0, a)
  A1_unique <- rnorm(n, 0, a * sqrt(1 - shared_genetic_fraction))
  A2_unique <- rnorm(n, 0, a * sqrt(1 - shared_genetic_fraction))
  A1 <- sqrt(shared_genetic_fraction) * A_shared + A1_unique
  A2 <- sqrt(shared_genetic_fraction) * A_shared + A2_unique

  C_shared <- rnorm(n, 0, c)
  E1 <- rnorm(n, 0, e)
  E2 <- rnorm(n, 0, e)

  data.frame(twin1 = A1 + C_shared + E1, twin2 = A2 + C_shared + E2)
}

set.seed(1)
n_pairs <- 2000
a2_true <- 0.50; c2_true <- 0.20; e2_true <- 0.30    # must sum to 1
a <- sqrt(a2_true); c <- sqrt(c2_true); e <- sqrt(e2_true)

mz <- simulate_twins(n_pairs, a, c, e, shared_genetic_fraction = 1.0)
dz <- simulate_twins(n_pairs, a, c, e, shared_genetic_fraction = 0.5)

r_mz <- cor(mz$twin1, mz$twin2)
r_dz <- cor(dz$twin1, dz$twin2)

cat(sprintf("MZ correlation: %.3f\n", r_mz))
cat(sprintf("DZ correlation: %.3f\n", r_dz))

MZ correlation: 0.708
DZ correlation: 0.439

h2_falconer <- 2 * (r_mz - r_dz)
c2_falconer <- r_mz - h2_falconer
e2_falconer <- 1 - r_mz

data.frame(
  component = c("A (genetic)", "C (shared env)", "E (unique env)"),
  true_value = c(a2_true, c2_true, e2_true),
  falconer_estimate = round(c(h2_falconer, c2_falconer, e2_falconer), 3)
)

A data.frame: 3 × 3
component	true_value	falconer_estimate

A (genetic)	0.5	0.538
C (shared env)	0.2	0.170
E (unique env)	0.3	0.292

Falconer’s arithmetic recovers all three true values closely, using nothing more than two correlations.

That simplicity is also its ceiling. Falconer’s method gives no standard errors, no way to test whether is actually needed at all, and no way to compare competing models formally. For that, twin research moved to full likelihood-based modeling — the same logic, expressed as a fitted statistical model instead of a shortcut formula.

5 Why Move to a Fitted Model?

Falconer’s estimator is a fixed, one-shot calculation. A structural equation model (SEM) treats , , and as parameters to be estimated by maximum likelihood, given the raw twin data and the covariance structure implied by MZ/DZ sharing:

Fitting these by likelihood, rather than reading them off two correlations, buys three things at once: standard errors on , , and ; the ability to constrain or and formally test whether that constrained model fits significantly worse; and a natural way to add covariates or handle missing data. This is exactly what dedicated twin-modeling software like OpenMx does under the hood — the version below implements the same likelihood directly, so the mechanics stay visible.

# Bivariate normal log-likelihood for a set of twin pairs, given a 2x2 covariance matrix (mean 0)
biv_normal_loglik <- function(dat, Sigma) {
  n <- nrow(dat)
  detS <- Sigma[1,1]*Sigma[2,2] - Sigma[1,2]^2
  Sinv <- matrix(c(Sigma[2,2], -Sigma[1,2], -Sigma[1,2], Sigma[1,1]), 2, 2) / detS
  x <- as.matrix(dat)
  quad <- rowSums((x %*% Sinv) * x)
  -0.5 * n * log(detS) - 0.5 * sum(quad) - n * log(2*pi)
}

# Negative log-likelihood for the ACE model (and constrained sub-models)
neg_ll_ACE <- function(par, fix_c = NA, fix_a = NA) {
  a2 <- if (!is.na(fix_a)) fix_a else plogis(par[1])
  c2 <- if (!is.na(fix_c)) fix_c else plogis(par[2]) * (1 - a2)
  e2 <- 1 - a2 - c2
  if (e2 <= 0) return(1e10)

  Sigma_mz <- matrix(c(1, a2 + c2, a2 + c2, 1), 2, 2)
  Sigma_dz <- matrix(c(1, 0.5*a2 + c2, 0.5*a2 + c2, 1), 2, 2)

  -(biv_normal_loglik(mz, Sigma_mz) + biv_normal_loglik(dz, Sigma_dz))
}

5.1 Fitting ACE, AE, and CE

Three models, nested inside each other: the full ACE model, and two restricted versions that force one component to zero.

fit_ACE <- function() {
  opt <- optim(c(0, 0), neg_ll_ACE, method = "Nelder-Mead")
  a2 <- plogis(opt$par[1]); c2 <- plogis(opt$par[2]) * (1 - a2); e2 <- 1 - a2 - c2
  list(a2 = a2, c2 = c2, e2 = e2, negLL = opt$value, df = 3)
}

fit_AE <- function() {
  f <- function(par) neg_ll_ACE(c(par[1], -1e6), fix_c = 0)
  opt <- optim(0, f, method = "Brent", lower = -10, upper = 10)
  a2 <- plogis(opt$par[1]); e2 <- 1 - a2
  list(a2 = a2, c2 = 0, e2 = e2, negLL = opt$value, df = 2)
}

fit_CE <- function() {
  f <- function(par) neg_ll_ACE(c(-1e6, par[1]), fix_a = 0)
  opt <- optim(0, f, method = "Brent", lower = -10, upper = 10)
  c2 <- plogis(opt$par[1]); e2 <- 1 - c2
  list(a2 = 0, c2 = c2, e2 = e2, negLL = opt$value, df = 2)
}

ace <- fit_ACE(); ae <- fit_AE(); ce <- fit_CE()

data.frame(
  model = c("ACE", "AE", "CE"),
  a2 = round(c(ace$a2, ae$a2, ce$a2), 3),
  c2 = round(c(ace$c2, ae$c2, ce$c2), 3),
  e2 = round(c(ace$e2, ae$e2, ce$e2), 3),
  neg2LL = round(2 * c(ace$negLL, ae$negLL, ce$negLL), 1)
)

A data.frame: 3 × 5
model	a2	c2	e2	neg2LL

ACE	0.508	0.190	0.301	20975.0
AE	0.709	0.000	0.291	21001.4
CE	0.000	0.574	0.426	21173.7

The full ACE fit lands almost exactly on the true simulated values (0.50 / 0.20 / 0.30) — maximum likelihood recovers the same answer Falconer’s shortcut did, as it should when the model is correctly specified.

Dropping (the AE model) doesn’t just zero it out — the genetic component jumps up to absorb the shared-environment variance that’s no longer available to explain the resemblance between twins. That’s a useful diagnostic on its own: watch what happens to the other components when you force one to zero.

5.2 Is the Extra Complexity of ACE Actually Justified?

A likelihood ratio test answers this directly: is the improvement in fit from adding back in bigger than you’d expect from noise alone?

lrt_stat <- 2 * ae$negLL - 2 * ace$negLL
lrt_df   <- ace$df - ae$df
lrt_p    <- 1 - pchisq(lrt_stat, lrt_df)

cat(sprintf("LRT comparing AE against full ACE: chi-sq = %.2f, df = %d, p = %.5f\n",
            lrt_stat, lrt_df, lrt_p))

aic <- c(ACE = 2*ace$negLL + 2*ace$df,
         AE  = 2*ae$negLL  + 2*ae$df,
         CE  = 2*ce$negLL  + 2*ce$df)
cat("\nAIC by model (lower is better):\n")
print(round(aic, 1))

LRT comparing AE against full ACE: chi-sq = 26.37, df = 1, p = 0.00000

AIC by model (lower is better):
    ACE      AE      CE 
20981.0 21005.4 21177.7

Both diagnostics agree: forcing to zero makes the model fit significantly worse, and AIC prefers the full ACE model over either restricted alternative. In a real analysis, this is exactly the workflow — fit the full model and each restriction, then let the likelihood ratio test and AIC decide which components the data actually support, rather than assuming a particular model in advance.

This is the concrete payoff of moving from Falconer’s formula to a fitted model: a formal, quantitative answer to “does shared environment matter here?” — not just a point estimate.

6 Beyond Twins: Heritability From Unrelated People

Twin models lean on a pair of fixed relatedness values — MZ pairs share (on average) all their genetic variance, DZ pairs share half. Modern genomics can go further: instead of assuming relatedness from pedigree structure, measure it directly from genotypes at hundreds of thousands of SNPs, for people who aren’t related at all.

The genetic relationship matrix (GRM) between two individuals and , using standardized genotypes, is:

where is the number of SNPs and denotes a genotype standardized to mean 0 and variance 1. Full siblings average close to 0.5 here, just like the pedigree expectation for DZ twins — but now it’s measured, not assumed, and it varies continuously even among people with no known family relationship at all.

The Haseman–Elston regression (Haseman and Elston 1972) turns this into a heritability estimate with a beautifully simple idea:

Pairs of people who are more genetically similar should also be more phenotypically similar — in direct proportion to how related they are.

Regressing pairwise phenotypic similarity on GRM values, the slope of that line is an estimate of SNP heritability. This is the logic that underlies GREML and GCTA (Yang et al. 2010) applied at genome-wide scale — the toy version below applies it to a small simulated cohort of unrelated individuals.

set.seed(2)
n_ind <- 300
n_snps <- 2000
maf <- runif(n_snps, 0.05, 0.5)

genotypes <- sapply(maf, function(p) rbinom(n_ind, 2, p))
geno_std  <- scale(genotypes)                 # standardize each SNP column
GRM       <- (geno_std %*% t(geno_std)) / n_snps

h2_true <- 0.5
genetic_effect    <- rnorm(n_snps, 0, sqrt(h2_true / n_snps))
genetic_component  <- as.numeric(scale(geno_std %*% genetic_effect)) * sqrt(h2_true)
env_component      <- rnorm(n_ind, 0, sqrt(1 - h2_true))
phenotype          <- genetic_component + env_component

pheno_outer <- outer(phenotype, phenotype)
idx <- upper.tri(GRM)

he_fit <- lm(pheno_outer[idx] ~ GRM[idx])
round(coef(summary(he_fit)), 4)

A matrix: 2 × 4 of type dbl
	Estimate	Std. Error	t value	Pr(>\|t\|)
(Intercept)	-0.0008	0.0043	-0.1892	0.8499
GRM[idx]	0.6392	0.1922	3.3250	0.0009

The regression slope — the Haseman–Elston estimate of SNP heritability — lands in the right neighborhood of the true value of 0.5, estimated entirely from unrelated individuals and their genotypes, with no twins or known family structure anywhere in sight.

This is the same conceptual move that let human genetics scale heritability estimation from small twin registries to biobanks of hundreds of thousands of unrelated people (Yang et al. 2010) — replace an assumed relatedness value (0, 0.5, or 1, from pedigree structure) with a measured one (any value in between, from genotypes), and the same regression logic still applies.

7 Key Takeaways

Heritability describes variance across a population, not a property of any one person, and it changes with the population’s environmental context.
Twin designs work because MZ and DZ pairs differ in exactly one thing — how much genetic variance they share — while (by assumption) sharing their environment equally.
Falconer’s method turns two correlations into estimates of A, C, and E with simple arithmetic, and recovers the truth well in simulation.
A fitted SEM (the ACE model) reproduces Falconer’s estimates while adding what arithmetic alone can’t: standard errors, and formal likelihood ratio / AIC comparisons between nested models like ACE, AE, and CE.
The same logic extends past twins entirely — the Haseman–Elston regression estimates heritability from measured genetic relatedness (a GRM) among people who share no known family relationship at all.

8 References

Falconer, D. S. (1960). Introduction to Quantitative Genetics. Oliver and Boyd.

Neale, M. C., & Cardon, L. R. (1992). Methodology for Genetic Studies of Twins and Families. Kluwer Academic Publishers.

Haseman, J. K., & Elston, R. C. (1972). The investigation of linkage between a quantitative trait and a marker locus. Behavior Genetics, 2(1), 3–19.

Yang, J., Benyamin, B., McEvoy, B. P., Gordon, S., Henders, A. K., Nyholt, D. R., Madden, P. A., Heath, A. C., Martin, N. G., Montgomery, G. W., Goddard, M. E., & Visscher, P. M. (2010). Common SNPs explain a large proportion of the heritability for human height. Nature Genetics, 42(7), 565–569.

Polderman, T. J. C., Benyamin, B., de Leeuw, C. A., Sullivan, P. F., van Bochoven, A., Visscher, P. M., & Posthuma, D. (2015). Meta-analysis of the heritability of human traits based on fifty years of twin studies. Nature Genetics, 47(7), 702–709.

Genetic Association Testing for Rare Variants

Nivedita Bhadra — Fri, 19 Jun 2026 21:00:00 GMT

The tutorial’s central result in one picture: the same 12-variant gene, with the same causal architecture, gives two completely different answers depending on whether the test sums effects (burden) or sums their squares (SKAT).

SKAT Burden Test Rare Variants R

Rare variants are, by definition, seen in only a handful of people. That single fact breaks almost every assumption a standard GWAS relies on, and it forces a completely different statistical toolkit.

This tutorial builds that toolkit from the ground up. We start with why single-variant testing fails, work through burden tests and SKAT by deriving their score statistics by hand, and finish with the saddlepoint approximation that makes rare-variant testing usable at biobank scale. Every statistic below is implemented in base R and checked against either a built-in R function or a Monte Carlo simulation.

1 Why Single-Variant Tests Fail for Rare Variants

If GWAS works for common variants, why not just run the same test on rare ones? Because you’ll rarely even observe the variant. A single-variant test needs carriers in your sample before it can say anything at all.

Large-effect variants on protein function are disproportionately rare, because natural selection removes them from the population before they become common. This is the central reason rare variants matter: the alleles most likely to have a real, interpretable effect on a trait are exactly the ones a standard genotyping array will barely capture.

The scale of the problem is easiest to see through a simple question: how many people do you need before you’re likely to observe even one carrier?

1.1 1.1 How many samples until you see one copy?

Assume random mating, so the two chromosomes carried by each of individuals are independent draws with minor allele frequency . The probability of seeing zero copies of the variant is , so:

# Sample size needed to observe >= 1 copy of a variant with 99.9% probability
maf <- c(0.1, 0.01, 0.001, 0.0001)
required_n <- log(0.001) / (2 * log(1 - maf))
data.frame(MAF = maf, Required_N = ceiling(required_n))

A data.frame: 4 × 2
MAF	Required_N

1e-01	33
1e-02	344
1e-03	3453
1e-04	34538

Going from a MAF of 1% to 0.01% multiplies the required sample size a hundredfold. Single-variant testing simply runs out of carriers long before it runs out of interesting biology. And yet rare variants are collectively far from rare: each person carries roughly 200 rare coding variants, so a gene-level view of these variants is both necessary and has plenty of signal to work with.

2 Collapsing Variants Into a Gene-Level Burden

If no single variant has enough carriers, what do we test instead? We stop asking about one variant and start asking about a gene. Define a carrier as anyone with at least one rare variant in the region, and test whether carriers and non-carriers differ in disease risk.

This converts a sparse, high-dimensional problem (many rare variants, few carriers each) into a simple 2x2 question: are carriers more likely to be cases than non-carriers?

# A toy burden contingency table: carriers of a rare-variant burden vs disease status
burden_table <- matrix(c(14, 6, 5, 15), nrow = 2, byrow = TRUE,
                       dimnames = list(Status = c("Case", "Control"),
                                       Burden  = c("Carrier", "Non-carrier")))
burden_table

fisher.test(burden_table)

A matrix: 2 × 2 of type dbl
	Carrier	Non-carrier
Case	14	6
Control	5	15


    Fisher's Exact Test for Count Data

data:  burden_table
p-value = 0.01039
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
  1.457116 35.737819
sample estimates:
odds ratio 
  6.614723

The odds ratio of association follows directly from the table:

With small carrier counts, a chi-squared test is unreliable, so fisher.test reaches for the exact hypergeometric distribution instead of a large-sample approximation. It’s worth confirming exactly what that function is computing.

3 Fisher’s Exact Test, From First Principles

What is fisher.test actually computing under the hood? The exact probability of observing a table at least as extreme as yours, conditional on the row and column totals staying fixed, under the hypergeometric distribution.

For a 2x2 table with cell counts , the probability of the observed table conditional on its margins is:

The two-sided p-value sums this probability over every table with the same margins that is at least as unlikely as the one observed — not just the fraction for the observed table alone.

# Verify the hypergeometric formula by hand against R's Fisher test
a <- 14; b <- 6; c <- 5; d <- 15
row1 <- a + b; row2 <- c + d
col1 <- a + c; col2 <- b + d
total <- row1 + row2

p_observed <- choose(row1, a) * choose(row2, c) / choose(total, col1)
cat("P(observed table) =", p_observed, "\n")

possible_a <- max(0, col1 - row2):min(row1, col1)
probs <- sapply(possible_a, function(x) {
  choose(row1, x) * choose(row2, col1 - x) / choose(total, col1)
})
p_two_sided <- sum(probs[probs <= p_observed + 1e-8])
cat("Two-sided p-value (by hand) =", p_two_sided, "\n")

P(observed table) = 0.004577422 
Two-sided p-value (by hand) = 0.0103867

0.0104 matches R’s reported 0.01039 to rounding. This is a useful sanity check to run once: it confirms fisher.test isn’t summing probabilities for tables that happen to be less extreme in only one direction, a mistake that’s easy to make when re-implementing exact tests by hand.

R.A. Fisher, who introduced this test alongside the concept of the null hypothesis, was explicit that a null hypothesis can only ever be rejected by data, never confirmed by it — a framing worth keeping in mind for every p-value that follows.

A single 2x2 table is a fine illustration, but it throws away information: every rare variant gets collapsed into one indicator, regardless of how rare it individually is, and regardless of whether its effect on the trait is protective or damaging. A regression framework fixes both problems.

4 A Regression Framework for Burden Testing

For a continuous trait, the natural model regresses the phenotype on every rare variant in the region:

For a binary trait, the same idea holds on the log-odds scale:

Testing separate coefficients still burns degrees of freedom and statistical power. The burden test collapses them into a single weighted sum:

Why weight variants at all, rather than just summing raw genotype counts? Because not every rare variant is equally rare. A weighting scheme that upweights rarer variants reflects the prior belief that rarer variants are more likely to be under stronger negative selection, and therefore more likely to matter.

A common choice, due to Wu et al. (2011), is , which assigns steeply increasing weight as MAF approaches zero.

5 Simulating a Rare-Variant Test Gene

To make the rest of this tutorial concrete, we simulate a 12-variant gene in 4,000 individuals: four variants that increase disease risk, two that are protective, and six with no effect at all. This bidirectional architecture — some variants increase risk, others decrease it — is deliberately chosen, because it’s the setting where burden and variance-component tests are expected to behave differently.

set.seed(2026)

n <- 4000                      # sample size
q <- 12                        # number of rare variants in the gene
maf <- runif(q, min = 0.0005, max = 0.01)  # rare variant MAFs

# Genotype matrix: 0/1/2 copies of the minor allele, drawn under Hardy-Weinberg
G <- sapply(maf, function(p) rbinom(n, 2, p))
colnames(G) <- paste0("v", seq_len(q))

# One covariate (e.g. a normalized age term)
X <- cbind(1, scale(rnorm(n)))
colnames(X) <- c("Intercept", "Covariate")

# Causal architecture: variants 1-4 increase risk, variants 5-6 are protective,
# the remaining 6 are non-causal noise. Effects are deliberately bidirectional,
# a setting where burden tests are known to lose power.
true_beta <- c(1.3, 1.1, 1.0, 0.9, -1.4, -1.2, rep(0, q - 6))

alpha <- c(-2.2, 0.15)   # covariate effects under the null model of disease risk
linpred <- X %*% alpha + G %*% true_beta
pi_true <- 1 / (1 + exp(-linpred))
y <- rbinom(n, 1, pi_true)

cat("Cases:", sum(y), " Controls:", sum(y == 0), "\n")
cat("MAFs:", round(maf, 4), "\n")

saveRDS(list(G = G, X = X, y = y, maf = maf, true_beta = true_beta),
        "sim_data.rds")

Cases: 408  Controls: 3592 
MAFs: 0.0071 0.0058 0.0018 0.0032 0.0058 7e-04 0.0049 0.0087 0.0029 0.006 6e-04 0.0071

A roughly 1:9 case-to-control ratio, which is typical of a moderately prevalent disease in a biobank cohort. This imbalance will matter a great deal later on.

With the gene simulated, we can now compute the Wu weight for each variant from its true MAF.

sim <- readRDS("sim_data.rds")
maf <- sim$maf

# Wu weights: w_j = Beta density of MAF under Beta(1, 25), upweighting rarer variants
w <- dbeta(maf, 1, 25)
data.frame(variant = colnames(sim$G), MAF = round(maf, 4), weight = round(w, 2))

A data.frame: 12 × 3
variant	MAF	weight

v1	0.0071	21.05
v2	0.0058	21.75
v3	0.0018	23.92
v4	0.0032	23.14
v5	0.0058	21.76
v6	0.0007	24.56
v7	0.0049	22.20
v8	0.0087	20.28
v9	0.0029	23.32
v10	0.0060	21.63
v11	0.0006	24.67
v12	0.0071	21.08

The rarest variant here (v11, MAF = 0.0006) gets almost 25% more weight than the most common one in the set (v8, MAF = 0.0087). This is a modest difference at these MAFs, but the gap widens sharply as MAF approaches the lower end of the exome-wide spectrum.

6 Three Classical Tests, and Why Genetics Prefers One of Them

Testing whether in a maximum-likelihood framework can be done three ways:

Wald test: fit the full (alternative) model, and measure how far the estimated effect is from zero, scaled by its standard error.
Likelihood ratio test (LRT): fit both the null and alternative models, and measure the drop in log-likelihood between them.
Score test: fit only the null model, and evaluate the gradient of the log-likelihood at that null.

All three measure the same underlying distance between the null hypothesis and the maximum-likelihood estimate, just along different geometric axes. As sample size grows, they converge to the same distribution:

If they’re asymptotically the same, why does the choice matter for rare-variant testing? Computation. A modern biobank study tests tens of thousands of genes, often under a mixed model that corrects for relatedness and population structure. Fitting the full alternative model separately for every gene is computationally intractable. The score test only requires fitting the null model once — testing each gene afterward is just matrix multiplication against a fixed set of residuals.

This is the entire reason burden tests, SKAT, and SKAT-O are all built as score tests rather than Wald or LRT tests.

7 Deriving the Burden Score Statistic

Recall the weighted burden model, with burden score for individual :

The score function is the gradient of the log-likelihood with respect to . By the chain rule:

The terms cancel exactly, which is why the score function has such a clean form. Evaluated at the null (, so is replaced by its null-model prediction ):

Squaring gives the burden test statistic used in tools like SAIGE-GENE:

Intuitively: is large whenever people who carry more of the weighted burden also have larger-than-expected residuals — i.e., the gene’s cumulative dosage tracks disease status.

8 Testing the Burden Model in R

We fit the null model once (covariates only), compute residuals, and turn the derivation above directly into code.

sim <- readRDS("sim_data.rds")
G <- sim$G; X <- sim$X; y <- sim$y; maf <- sim$maf
w <- dbeta(maf, 1, 25)

## --- Step 1: fit the null model (covariates only, no genotypes) ---
null_model <- glm(y ~ X[, -1], family = binomial())
pi_hat <- fitted(null_model)
resid  <- y - pi_hat

## --- Step 2: build the weighted burden score B_i = sum_j w_j * G_ij ---
B <- as.vector(G %*% w)

## --- Step 3: burden score statistic U(0) = sum_i resid_i * B_i ---
U0 <- sum(resid * B)

## --- Step 4: variance of U(0) under the null ---
## Var(U) = B' V B, where V = diag(pi_hat * (1 - pi_hat))
V <- pi_hat * (1 - pi_hat)
var_U0 <- sum(V * B^2)

## --- Step 5: score test statistic and p-value ---
Q_burden <- U0^2 / var_U0
p_burden_score <- pchisq(Q_burden, df = 1, lower.tail = FALSE)

cat("Burden score U(0)      :", round(U0, 4), "\n")
cat("Var[U(0)]              :", round(var_U0, 4), "\n")
cat("Burden score statistic :", round(Q_burden, 4), "\n")
cat("Burden score p-value   :", format.pval(p_burden_score, digits = 4), "\n\n")

## --- Compare to a Wald test from a direct logistic regression on the burden score ---
alt_model <- glm(y ~ X[, -1] + B, family = binomial())
wald_p <- summary(alt_model)$coefficients["B", "Pr(>|z|)"]
cat("Wald test p-value (glm):", format.pval(wald_p, digits = 4), "\n")

Burden score U(0)      : 109.4711 
Var[U(0)]              : 22029.38 
Burden score statistic : 0.544 
Burden score p-value   : 0.4608 

Wald test p-value (glm): 0.4362

Two things stand out. First, the score and Wald p-values are close (0.46 and 0.44 land in the same non-significant territory), exactly the asymptotic equivalence discussed above. Second, and more importantly: the burden test completely misses this gene, despite it having six genuinely causal variants.

That’s not a bug in the implementation. It’s the burden test doing exactly what it’s designed to do, in a setting it’s poorly suited for.

9 When Burden Tests Fail: The Variance-Component Alternative

Why did the burden test fail here, specifically? Because four risk-increasing and two protective variants partially cancel out inside the same weighted sum. A burden test is only well-powered when all causal variants push risk in the same direction.

The naive fix — summing single-variant score statistics before squaring — has exactly this flaw:

If variant A contributes and variant B contributes , they cancel to zero and the association vanishes entirely, even though both variants are genuinely causal.

The fix, due to Wu et al. (2011) building on the C-alpha test of Neale et al. (2011), is to square each variant’s contribution before summing:

This is the SKAT (sequence kernel association test) statistic. Because residuals are squared before aggregation, risk-increasing and protective variants both contribute positively — nothing cancels.

10 Implementing SKAT From Scratch

is a quadratic form in the vector of per-variant score statistics, and under the null it follows a mixture of weighted distributions rather than a single chi-squared distribution — one weight per eigenvalue of the statistic’s covariance kernel. We build that kernel directly, residualizing the genotype matrix on the covariates first, then get a p-value using the four-moment chi-squared approximation of Liu, Tang & Zhang (2009).

sim <- readRDS("sim_data.rds")
G <- sim$G; X <- sim$X; y <- sim$y; maf <- sim$maf
w <- dbeta(maf, 1, 25)

## --- Null model (same as for the burden test) ---
null_model <- glm(y ~ X[, -1], family = binomial())
pi_hat <- fitted(null_model)
resid  <- y - pi_hat
V <- pi_hat * (1 - pi_hat)

## --- Step 1: per-variant score statistics S_j = sum_i G_ij * resid_i ---
S <- as.vector(t(G) %*% resid)

## --- Step 2: SKAT statistic Q_S = sum_j w_j^2 * S_j^2 ---
Q_skat <- sum(w^2 * S^2)
cat("SKAT statistic Q_S:", round(Q_skat, 3), "\n")

## --- Step 3: null distribution of Q_S is a mixture of weighted chi-sq(1)'s.
## Its "kernel" matrix comes from projecting G onto the space orthogonal to X
## (so covariates are properly accounted for), then reweighting by w and V.
Gt <- G - X %*% solve(t(X) %*% X) %*% (t(X) %*% G)     # residualize G on X
W  <- diag(w)
K  <- W %*% t(Gt) %*% diag(V) %*% Gt %*% W               # q x q kernel matrix
lambda <- eigen(K, symmetric = TRUE, only.values = TRUE)$values
lambda <- lambda[lambda > 1e-8]
cat("Eigenvalues of the kernel matrix:", round(lambda, 2), "\n")

## --- Step 4: Liu, Tang & Zhang (2009) moment-matching chi-square approximation ---
liu_pvalue <- function(Q, lambda) {
  c1 <- sum(lambda)
  c2 <- sum(lambda^2)
  c3 <- sum(lambda^3)
  c4 <- sum(lambda^4)
  s1 <- c3 / c2^1.5
  s2 <- c4 / c2^2

  if (s1^2 > s2) {
    a     <- 1 / (s1 - sqrt(s1^2 - s2))
    delta <- s1 * a^3 - a^2
    df    <- a^2 - 2 * delta
  } else {
    a     <- 1 / s1
    delta <- 0
    df    <- c2^3 / c3^2
  }
  muQ    <- c1
  sigmaQ <- sqrt(2 * c2)
  muX    <- df + delta
  sigmaX <- sqrt(2 * (df + 2 * delta))

  Qnorm <- (Q - muQ) / sigmaQ * sigmaX + muX
  pchisq(Qnorm, df = df, ncp = delta, lower.tail = FALSE)
}

p_skat <- liu_pvalue(Q_skat, lambda)
cat("SKAT p-value (Liu et al. 2009 approximation):", format.pval(p_skat, digits = 4), "\n")

SKAT statistic Q_S: 45891.8 
Eigenvalues of the kernel matrix: 3155.72 2340.21 2230.61 1851.55 1809.8 1770.95 1683.92 1596.49 1357.32 865.01 511.66 502.97 
SKAT p-value (Liu et al. 2009 approximation): 0.01069

SKAT recovers the association (p = 0.011) that the burden test missed entirely (p = 0.46), on the exact same data. This is the textbook case for variance-component tests: causal variants with mixed effect directions.

It’s worth checking this analytic p-value against a direct simulation, since the moment-matching approximation is, after all, an approximation.

sim <- readRDS("sim_data.rds")
G <- sim$G; X <- sim$X; y <- sim$y; maf <- sim$maf
w <- dbeta(maf, 1, 25)

null_model <- glm(y ~ X[, -1], family = binomial())
pi_hat <- fitted(null_model)
V <- pi_hat * (1 - pi_hat)

Q_obs <- {
  resid <- y - pi_hat
  S <- as.vector(t(G) %*% resid)
  sum(w^2 * S^2)
}

set.seed(1)
B <- 20000
Q_null <- numeric(B)
for (b in seq_len(B)) {
  y_perm <- rbinom(length(y), 1, pi_hat)        # simulate under the fitted null
  resid_perm <- y_perm - pi_hat
  S_perm <- as.vector(t(G) %*% resid_perm)
  Q_null[b] <- sum(w^2 * S_perm^2)
}

p_mc <- mean(Q_null >= Q_obs)
cat("Monte Carlo p-value (", B, "null draws ):", p_mc, "\n")

Monte Carlo p-value ( 20000 null draws ): 0.01505

0.015 against the analytic 0.011: close enough to trust the moment-matching approximation for this gene, and a good habit to keep whenever a new test statistic goes into a pipeline.

11 SKAT-O: Combining Burden and Variance-Component Information

Neither test is uniformly better. Burden tests are more powerful when every causal variant pushes the trait in the same direction; SKAT is more powerful when effects are mixed or only a subset of variants are causal. SKAT-O (Lee et al., 2012) sidesteps the choice by taking a weighted combination of both statistics:

and searching over a grid of values, with a correction for having tested several of them. recovers pure SKAT; recovers the pure burden test.

sim <- readRDS("sim_data.rds")
G <- sim$G; X <- sim$X; y <- sim$y; maf <- sim$maf
w <- dbeta(maf, 1, 25)

null_model <- glm(y ~ X[, -1], family = binomial())
pi_hat <- fitted(null_model)
resid  <- y - pi_hat
V <- pi_hat * (1 - pi_hat)

B_score <- as.vector(G %*% w)
U0      <- sum(resid * B_score)
var_U0  <- sum(V * B_score^2)
Q_B     <- U0^2 / var_U0                     # burden statistic (as a chi-sq_1)

S       <- as.vector(t(G) %*% resid)
Q_S     <- sum(w^2 * S^2)                    # SKAT statistic

liu_pvalue <- function(Q, lambda) {
  c1 <- sum(lambda); c2 <- sum(lambda^2); c3 <- sum(lambda^3); c4 <- sum(lambda^4)
  s1 <- c3 / c2^1.5; s2 <- c4 / c2^2
  if (s1^2 > s2) {
    a <- 1 / (s1 - sqrt(s1^2 - s2)); delta <- s1 * a^3 - a^2; df <- a^2 - 2 * delta
  } else {
    a <- 1 / s1; delta <- 0; df <- c2^3 / c3^2
  }
  muQ <- c1; sigmaQ <- sqrt(2 * c2); muX <- df + delta; sigmaX <- sqrt(2 * (df + 2 * delta))
  Qnorm <- (Q - muQ) / sigmaQ * sigmaX + muX
  pchisq(Qnorm, df = df, ncp = delta, lower.tail = FALSE)
}

Gt <- G - X %*% solve(t(X) %*% X) %*% (t(X) %*% G)
K  <- diag(w) %*% t(Gt) %*% diag(V) %*% Gt %*% diag(w)
lambda <- eigen(K, symmetric = TRUE, only.values = TRUE)$values
lambda <- lambda[lambda > 1e-8]

rhos <- c(0, 0.1, 0.2, 0.4, 0.6, 0.8, 1)      # rho = 1 recovers the pure burden test
p_by_rho <- sapply(rhos, function(rho) {
  Q_rho <- (1 - rho) * Q_S + rho * Q_B * sum(w^2)   # rescale so burden/SKAT are comparable
  liu_pvalue(Q_rho, (1 - rho) * lambda + rho * sum(w^2) * mean(lambda) / sum(w^2))
})

data.frame(rho = rhos, p_value = format.pval(p_by_rho, digits = 4))
cat("\nMinimum p-value across rho grid:", format.pval(min(p_by_rho), digits = 4),
    "at rho =", rhos[which.min(p_by_rho)], "\n")
cat("(SKAT-O then applies its own correction for testing multiple rho values;\n",
    " shown here to illustrate the interpolation, not as a calibrated final p-value.)\n")

A data.frame: 7 × 2
rho	p_value

0.0	0.01069
0.1	0.01989
0.2	0.03765
0.4	0.13442
0.6	0.41188
0.8	0.84245
1.0	0.99938


Minimum p-value across rho grid: 0.01069 at rho = 0 
(SKAT-O then applies its own correction for testing multiple rho values;
  shown here to illustrate the interpolation, not as a calibrated final p-value.)

For this gene, (pure SKAT) is best, which is exactly what we’d expect given the bidirectional effect design. Note that this implementation illustrates the interpolation qualitatively rather than reproducing Lee et al.’s exact null distribution, which accounts for the correlation between the burden and SKAT statistics across the whole grid through a more involved numerical integration.

12 Which Variants Go Into the Test?

A gene-based test is only as good as the variant set fed into it. In practice, variants are grouped by predicted functional consequence before testing:

Loss-of-function (LoF): nonsense, frameshift, splice-disrupting
Damaging missense: amino-acid-changing variants predicted deleterious by tools like CADD, PolyPhen, or SIFT
Synonymous: typically included as a negative control, since these shouldn’t affect protein function

Different genes show enrichment for different variant classes depending on their biology, so most modern rare-variant pipelines run several MAF cutoffs (e.g., <1%, <0.1%, <0.01%) crossed with several annotation categories for each gene, rather than committing to one variant set up front.

13 The Hidden Problem at Biobank Scale: Case-Control Imbalance

Everything above relies on the Central Limit Theorem: the null distribution of a score statistic is assumed to converge to a symmetric Normal. That assumption quietly depends on having enough carriers and a reasonably balanced number of cases and controls. Biobank-scale binary traits routinely have neither.

We can see this directly by simulating the null distribution of a single-variant score statistic under three regimes.

set.seed(7)
n <- 5000

simulate_null_scores <- function(maf, case_rate, n = 5000, B = 50000) {
  pi_hat <- rep(case_rate, n)               # null model: no covariates, intercept only
  g <- rbinom(n, 2, maf)
  V <- pi_hat * (1 - pi_hat)
  var_S <- sum(V * g^2)
  sd_S  <- sqrt(var_S)

  scores <- replicate(B, {
    y <- rbinom(n, 1, pi_hat)
    sum(g * (y - pi_hat))
  })
  list(scores = scores / sd_S, g = g)   # standardized score statistic
}

common_balanced <- simulate_null_scores(maf = 0.30, case_rate = 0.50)
rare_balanced   <- simulate_null_scores(maf = 0.005, case_rate = 0.50)
rare_imbalanced <- simulate_null_scores(maf = 0.005, case_rate = 0.01)

report <- function(label, scores) {
  cat(sprintf("%-28s skewness = %6.3f   empirical tail P(Z>3) = %.5f (Normal approx: %.5f)\n",
              label, mean(((scores - mean(scores)) / sd(scores))^3),
              mean(scores > 3), pnorm(3, lower.tail = FALSE)))
}

report("Common variant, balanced",   common_balanced$scores)
report("Rare variant, balanced",     rare_balanced$scores)
report("Rare variant, imbalanced",   rare_imbalanced$scores)

Common variant, balanced     skewness =  0.006   empirical tail P(Z>3) = 0.00120 (Normal approx: 0.00135)
Rare variant, balanced       skewness = -0.018   empirical tail P(Z>3) = 0.00090 (Normal approx: 0.00135)
Rare variant, imbalanced     skewness =  1.292   empirical tail P(Z>3) = 0.02412 (Normal approx: 0.00135)

The common-variant and rare-but-balanced cases match the Normal approximation closely, exactly as CLT theory promises. The rare-and-imbalanced case is a different story: skewness jumps to 1.29, and the true tail probability is nearly 18-fold larger than the Normal approximation would suggest.

That’s not a small correction. At genome-wide or exome-wide significance thresholds (typically or stricter after multiple testing correction), an 18-fold underestimate of the p-value in the tail translates directly into false positives.

14 The Saddlepoint Approximation: Fixing the Tail

If the Normal approximation only uses the mean and variance, what would a better approximation use? Every moment of the distribution, via its cumulant generating function (CGF), and — critically — it approximates the distribution at the specific point in the tail you care about, rather than at the center.

The CGF is the log of the moment generating function, . Its derivatives at recover the ordinary moments (variance, skewness, kurtosis), but its real advantage is computational: because our score statistic is a sum of independent per-individual contributions, for the total is just the sum of each individual’s CGF, whereas the moment generating functions would need to be multiplied — a much less numerically stable operation once you’re summing thousands of terms.

The saddlepoint approximation (SPA) uses the full CGF to re-center the approximation exactly at the observed test statistic, rather than at the mean. It solves for a ‘tilting’ parameter satisfying , then uses the curvature of at that point to get an accurate tail probability — the Lugannani-Rice formula. This is the method behind SAIGE-GENE+’s improved type-I error control at rare, imbalanced traits.

set.seed(7)
n <- 5000
maf <- 0.005
case_rate <- 0.01
pi_hat <- rep(case_rate, n)
g <- rbinom(n, 2, maf)

sigma <- sqrt(sum(pi_hat * (1 - pi_hat) * g^2))

## CGF of a single term X_i = g_i * (Y_i - pi_i), Y_i ~ Bernoulli(pi_i)
K_i  <- function(t, g_i, p_i) log((1 - p_i) * exp(-t * g_i * p_i) + p_i * exp(t * g_i * (1 - p_i)))
K1_i <- function(t, g_i, p_i) { h <- 1e-5; (K_i(t + h, g_i, p_i) - K_i(t - h, g_i, p_i)) / (2 * h) }
K2_i <- function(t, g_i, p_i) { h <- 1e-4; (K_i(t + h, g_i, p_i) - 2 * K_i(t, g_i, p_i) + K_i(t - h, g_i, p_i)) / h^2 }

K_total  <- function(t) sum(sapply(seq_len(n), function(i) K_i(t, g[i], pi_hat[i])))
K1_total <- function(t) sum(sapply(seq_len(n), function(i) K1_i(t, g[i], pi_hat[i])))
K2_total <- function(t) sum(sapply(seq_len(n), function(i) K2_i(t, g[i], pi_hat[i])))

## Test statistic set at 3 standard deviations - the "borderline significant" region
q_obs <- 3 * sigma
cat("sigma =", round(sigma, 3), " q_obs (3 sd) =", round(q_obs, 3), "\n")

## Solve the saddlepoint equation K'(t_hat) = q_obs
that <- uniroot(function(t) K1_total(t) - q_obs, interval = c(1e-6, 5))$root

## Lugannani-Rice saddlepoint approximation for P(S >= q_obs)
w <- sign(that) * sqrt(2 * (that * q_obs - K_total(that)))
u <- that * sqrt(K2_total(that))
p_spa <- pnorm(w, lower.tail = FALSE) + dnorm(w) * (1 / u - 1 / w)

## Compare to the naive Normal approximation using only mean & variance
p_normal <- pnorm(q_obs, mean = 0, sd = sigma, lower.tail = FALSE)

## Monte Carlo "ground truth"
B <- 2000000
mc_scores <- replicate(B, sum(g * (rbinom(n, 1, pi_hat) - pi_hat)))
p_mc <- mean(mc_scores >= q_obs)

cat("Saddlepoint t_hat        :", round(that, 4), "\n")
cat("SPA p-value              :", format.pval(p_spa, digits = 4), "\n")
cat("Normal-approx p-value    :", format.pval(p_normal, digits = 4), "\n")
cat("Monte Carlo p-value      :", format.pval(p_mc, digits = 4), "(", B, "draws )\n")

sigma = 0.704  q_obs (3 sd) = 2.111 
Saddlepoint t_hat        : 1.6964 
SPA p-value              : 0.01289 
Normal-approx p-value    : 0.00135 
Monte Carlo p-value      : 0.01394 ( 2e+06 draws )

The saddlepoint approximation (0.0129) tracks the 2-million-draw Monte Carlo truth (0.0139) closely. The Normal approximation (0.00135), by contrast, is off by roughly an order of magnitude at exactly the same test statistic. This single comparison is the entire justification for why SAIGE-GENE+, and every serious modern rare-variant tool, replaces the Normal approximation with SPA before reporting a p-value.

15 Choosing a Method in Practice

No single test is correct for every study design. The right choice depends on relatedness structure, sample size, trait type, and case-control balance.

Setting	Recommended approach
Quantitative trait, unrelated samples, large N	SKAT, SMMAT, or EmmaX-SKAT
Quantitative trait, related samples (any N)	SAIGE-GENE+, Regenie, or STAAR
Binary trait, balanced (case:control > 1:10), unrelated	SKAT
Binary trait, balanced, related samples	SMMAT, EmmaX-SKAT, or SAIGE-GENE+
Binary trait, imbalanced (case:control < 1:10)	SAIGE-GENE+, Regenie, or STAAR (all use SPA)

As a rule of thumb: the more imbalanced or related your samples, the more you need a method built around a mixed model with saddlepoint-corrected p-values, rather than a method that assumes independence and a Normal null.

16 Summary

Rare variants are individually hard to detect, but collectively common enough to test as a group.
Burden tests and SKAT differ in one key assumption: whether causal effects within a gene point in the same direction.
Both are implemented as score tests specifically because biobank-scale analysis makes fitting the full alternative model, gene by gene, computationally infeasible.
SKAT-O interpolates between burden and SKAT rather than forcing a choice between them.
At rare variant frequencies and with case-control imbalance, the Normal approximation that score tests rely on breaks down badly — the saddlepoint approximation is what makes modern tools like SAIGE-GENE+ trustworthy at genome-wide significance thresholds.

17 References

Foundational methods

Li, B. & Leal, S. M. (2008). Methods for detecting associations with rare variants for common diseases: application to analysis of sequence data. American Journal of Human Genetics, 83, 311–321.
Madsen, B. E. & Browning, S. R. (2009). A groupwise association test for rare mutations using a weighted sum statistic. PLoS Genetics, 5, e1000384.
Price, A. L. et al. (2010). Pooled association tests for rare variants in exon-resequencing studies. American Journal of Human Genetics, 86, 832–838.
Neale, B. M. et al. (2011). Testing for an unusual distribution of rare variants. PLoS Genetics, 7, e1001322.
Wu, M. C. et al. (2011). Rare-variant association testing for sequencing data with the sequence kernel association test. American Journal of Human Genetics, 89, 82–93.
Lee, S., Wu, M. C. & Lin, X. (2012). Optimal tests for rare variant effects in sequencing association studies. Biostatistics, 13, 762–775.
Lee, S. et al. (2012). Optimal unified approach for rare-variant association testing with application to small-sample case-control whole-exome sequencing studies. American Journal of Human Genetics, 91, 224–237.
Liu, H., Tang, Y. & Zhang, H. H. (2009). A new chi-square approximation to the distribution of non-negative definite quadratic forms in non-central normal variables. Computational Statistics & Data Analysis, 53, 853–856.

Scalable software and applications

Zhou, W. et al. (2020). Scalable generalized linear mixed model for region-based association tests in large biobanks and cohorts. Nature Genetics, 52, 634–639.
Zhou, W., Bi, W., Zhao, Z. et al. (2022). SAIGE-GENE+ improves the efficiency and accuracy of set-based rare variant association tests. Nature Genetics, 54, 1466–1469.
Backman, J. D. et al. (2021). Exome sequencing and analysis of 454,787 UK Biobank participants. Nature, 599, 628–634.
Karczewski, K. J. et al. (2022). Systematic single-variant and gene-based association testing of thousands of phenotypes in 394,841 UK Biobank exomes. Cell Genomics, 2, 100168.

Statistical Fine-Mapping: From GWAS Signals to Causal Variants

Nivedita Bhadra — Mon, 15 Jun 2026 21:00:00 GMT

The result this tutorial builds up to, from real simulated data: with three true causal variants hiding in one correlated region, ABF collapses onto a single dominant signal while SuSiE separates out all three.

Fine-Mapping SuSiE GWAS R

0.1 What Is Fine-Mapping?

GWAS has identified thousands of genomic regions associated with disease, but it typically doesn’t pinpoint the causal variant. Instead, GWAS flags a region containing many correlated variants, because of Linkage Disequilibrium (LD) — the non-random correlation between nearby genetic variants:

SNP A ---- SNP B ---- SNP C
              ^
           causal

If SNP B is causal, A and C often look significant too, simply because they’re inherited together.

GWAS asks: Which region is associated with the trait? Fine-mapping asks: Which specific variant is most likely causing it?

A locus flagged by GWAS might contain 10, 100, or 1,000 SNPs with nearly identical p-values — GWAS alone cannot separate them.

0.2 The Goal: Posterior Inclusion Probabilities

Fine-mapping estimates for every variant in a region — a Posterior Inclusion Probability (PIP) — rather than just a p-value.

results <- data.frame(SNP = c("rs1","rs2","rs3","rs4","rs5"),
                       PIP = c(0.01, 0.04, 0.88, 0.05, 0.02))

rs1  0.01   rs2  0.04   rs3  0.88   rs4  0.05   rs5  0.02

rs3, with an 88% posterior probability, is the strongest causal candidate.

	GWAS	Fine-mapping
Focus	p-values, association testing, genome-wide discovery	posterior probabilities, causal inference, variant prioritization
Role	first step	second step

0.3 Why It Matters

A schizophrenia GWAS locus with 200 SNPs is too expensive to validate experimentally in full. Fine-mapping can shrink the candidate list to 3 variants, or even 1 — dramatically cutting the cost of functional genomics, drug target discovery, eQTL interpretation, colocalization, and precision medicine.

0.4 The Modern Statistical Genetics Pipeline

This backbone underlies large-scale projects like UK Biobank, FinnGen, the Psychiatric Genomics Consortium, and GTEx.

0.5 What Data Do You Need?

Three common scenarios determine what’s possible:

Individual-level data — genotypes and phenotypes for every participant. Ideal: LD can be computed directly from the sample.
Single-cohort GWAS summary statistics — only beta, SE, p-value. Requires an external LD reference panel (1000 Genomes, UK Biobank, TOPMed).
Meta-analysis summary statistics — combined across cohorts with different LD structures; the hardest case, often requiring specialized methods (FastMap, CARMA, SLALOM).

0.6 The Central Role of LD

Two SNPs with are almost perfectly correlated — statistically indistinguishable. High LD → large credible sets, low resolution. Low LD → small credible sets, high resolution. This is why ancestry-matched LD reference panels matter so much.

Foundational concepts for the rest of this tutorial: Linkage Disequilibrium, Causal Variants, Posterior Inclusion Probability, Credible Sets, LD Reference Panels, Summary Statistics, Individual-Level Data.

Key takeaways. Fine-mapping identifies likely causal variants within a GWAS locus. Because neighboring SNPs are correlated through LD, GWAS alone can’t determine which variant is responsible. Fine-mapping combines GWAS statistics with LD and Bayesian inference to assign causality probabilities, producing PIPs and credible sets — the bridge between GWAS discovery and downstream eQTL mapping, colocalization, functional validation, and drug target identification.

1 Linkage Disequilibrium, PIPs, and Credible Sets

Three concepts form the mathematical backbone of every fine-mapping method: Linkage Disequilibrium (LD), Posterior Inclusion Probabilities (PIPs), and Credible Sets.

1.1 Linkage Disequilibrium

LD measures the correlation between genetic variants — when two SNPs are inherited together more often than chance predicts, they’re in LD. If , knowing one variant almost perfectly predicts the other.

Why LD exists: during recombination, nearby variants are less likely to be separated, so physical proximity → co-inheritance → correlation → LD. The closer two SNPs sit, the stronger their LD tends to be.

Two common measures: (0–1, measures historical recombination) and (0–1, measures correlation; means no correlation, means perfect correlation). Fine-mapping primarily uses .

Example 4×4 LD matrix, where SNP1–SNP2 and SNP3–SNP4 form two separate, tightly-correlated blocks:

ld <- data.frame(rbind(
  c(1.00, 0.92, 0.15, 0.05),
  c(0.92, 1.00, 0.18, 0.07),
  c(0.15, 0.18, 1.00, 0.85),
  c(0.05, 0.07, 0.85, 1.00)
))

1.2 Posterior Inclusion Probability (PIP)

pip <- data.frame(SNP = c("rs1","rs2","rs3","rs4"),
                   PIP = c(0.03, 0.07, 0.82, 0.08))

rs3 has an 82% probability of being causal.

PIPs are not p-values. A p-value measures evidence against the null hypothesis; a PIP measures probability of causality. does not mean “99.999999999999999% chance causal” — that’s a common and incorrect conflation.

Multiple causal variants are common. Many loci contain more than one causal SNP (e.g., variants 20, 39, and 68 all influencing the same trait) — exactly the scenario that motivated methods like SuSiE.

1.3 Credible Sets

A credible set is a group of variants that collectively contain the causal SNP with high probability — most commonly a 95% credible set: a 95% probability the causal SNP lies somewhere in the set, not a 95% probability for each individual member.

Constructing one: sort by PIP, then accumulate until crossing the threshold.

SNP	PIP	Cumulative
rs1	0.60	0.60
rs2	0.25	0.85
rs3	0.08	0.93
rs4	0.04	0.97
rs5	0.03	1.00

The 95% credible set is {rs1, rs2, rs3, rs4} — the smallest prefix exceeding 95%.

Interpreting size: a single-SNP set (e.g. {rs25}) means excellent resolution; a 3-SNP set is reasonable; a 50-SNP set means poor resolution.

1.4 What Determines Credible Set Size?

Sample size — larger N → higher power → smaller credible sets.
LD structure — high LD → large credible sets (SNPs statistically indistinguishable); low LD → small credible sets.
Effect size — larger effects → higher PIPs; smaller effects → lower PIPs.

Ideal: one SNP at PIP = 0.98, credible set = {that SNP}. Difficult: five SNPs each around PIP 0.16–0.20, credible set = all five, reflecting genuine uncertainty rather than method failure.

1.5 Why Fine-Mapping Is Bayesian

Bayesian methods directly answer “how likely is each SNP to be causal?” — exactly the question fine-mapping needs. This is why nearly every modern method (ABF, SuSiE, FINEMAP, CARMA, CAVIAR) is Bayesian.

Key takeaways. LD is the correlation structure among nearby variants and the primary reason GWAS can’t identify causal variants directly. Fine-mapping combines LD with GWAS statistics to estimate PIPs. Because uncertainty remains, methods report credible sets — groups of variants that jointly contain the causal SNP with high probability. Sample size, effect size, and LD structure strongly influence both PIPs and credible set size.

2 Bayesian Fine-Mapping and Approximate Bayes Factors (ABF)

GWAS cannot tell us which of several significant SNPs is causal — it only tells us how unlikely the data would be under no association. Fine-mapping needs something different: . This is where Bayesian statistics comes in, and Approximate Bayes Factors (ABF) — Jon Wakefield’s method — is one of the earliest and still most widely used solutions.

2.1 Frequentist vs. Bayesian Thinking

The frequentist approach tests and computes — a p-value. The Bayesian approach compares “SNP is causal” against “SNP is not causal” and computes directly — exactly the quantity we want.

2.2 Bayes Factors

A Bayes Factor compares two models: . BF = 1 means no preference; BF = 10 means the data are 10× more likely under the causal model; BF = 100 is strong evidence for causality.

“Approximate” because exact Bayes Factors are expensive to compute. Wakefield’s approximation needs only the effect estimate () and its standard error (SE) — both routinely available in GWAS summary statistics, with no individual-level data required.

2.3 Wakefield’s ABF Formula

For each SNP, compute and . Choose a prior variance (commonly 0.01–0.04, representing your prior belief about plausible effect sizes — smaller expects smaller effects). Then:

2.4 From Bayes Factors to PIPs

Normalize the BFs across all SNPs in the region: .

import numpy as np, pandas as pd

beta = np.array([0.05, 0.15, 0.09])
se   = np.array([0.03, 0.03, 0.03])
W = 0.04

z = beta / se
V = se**2
r = W / (W + V)
lbf = (np.log(1-r) + r*(z**2)) / 2
bf = np.exp(lbf)
pip = bf / bf.sum()

results = pd.DataFrame({"Beta": beta, "SE": se, "Z": z, "BF": bf, "PIP": pip})

For example, three SNPs with BFs 3, 50, and 10 (total 63) give PIPs of 0.048, 0.794, and 0.159 — SNP2 is the strongest candidate.

2.5 The Single-Causal-Variant Assumption

ABF assumes exactly one causal variant per locus — a simplification that breaks down whenever a locus actually has 2, 3, 5, or more independent causal signals (variants 20, 39, and 68, say). ABF then splits and dilutes the true signals across correlated SNPs, since it can only ever “explain” the locus with one variant. This limitation directly motivated SuSiE, FINEMAP, CAVIAR, and CARMA.

2.6 Strengths and Limitations

Strengths: fast, simple, requires only summary statistics, no LD matrix needed, computationally efficient for large GWAS. Limitations: single-causal-variant assumption, cannot separate multiple signals, can be confused by strong LD, lower resolution in complex regions.

ABF also underlies coloc.abf, used for testing whether GWAS and eQTL signals share a causal variant (Part 12) — carrying the same single-variant assumption into colocalization.

Key takeaways. ABF was among the first practical Bayesian fine-mapping methods, converting GWAS summary statistics into PIPs by normalizing Bayes Factors across SNPs. It’s fast and requires only summary statistics, but its core limitation — assuming a single causal variant per locus — motivated the development of multi-signal methods like SuSiE.

3 SuSiE: Sum of Single Effects

ABF assumes one causal variant per locus, but real GWAS regions routinely contain 2, 3, 5+ independent causal variants. SuSiE — one of the most widely used fine-mapping methods today — was built to handle exactly this.

3.1 The Core Idea

Instead of fitting one signal, SuSiE models the phenotype as a sum of single effects:

where is the phenotype, the genotype matrix, the SNP effects, and noise. Rather than fitting all SNP effects simultaneously, SuSiE decomposes the model into separate single-effect components — each one identifying one potential causal signal (e.g. Signal 1 → variant 39, Signal 2 → variant 20, Signal 3 → variant 68) — then combines them.

The key parameter L sets the maximum number of causal signals to search for (e.g. L = 4 allows up to 4 independent signals).

3.2 What SuSiE Produces

For every SNP, a PIP (e.g. rs20 = 0.95, rs39 = 0.98, rs68 = 0.92 — all three likely causal, unlike a single p-value). And for each independent signal, a credible set — sometimes tight ({rs20} alone, excellent resolution), sometimes broader ({rs20, rs21, rs22}, meaning one of these is probably causal but the data can’t distinguish which).

3.3 Running SuSiE in R

Illustrative syntax – the objects below (GWAS_df, N, in_sample_LD, y) are built from scratch in the worked example later in this tutorial.

library(susieR)

susie_results <- susie_rss(
  bhat = GWAS_df$beta_marginal,
  shat = GWAS_df$se_marginal,
  n = N,
  R = in_sample_LD,
  var_y = var(y),
  L = 4,
  estimate_residual_variance = TRUE
)

Unlike ABF, SuSiE requires an LD matrix — it has to know which SNPs are correlated in order to separate distinct signals.

susie_plot(susie_results, y = "PIP")

3.4 When SuSiE Works Best

Large sample size (more information), lower LD (better separation between causal variants), strong effects (higher PIPs), and an accurate, ancestry-matched LD matrix.

Common beginner mistake: assuming the lead SNP (smallest p-value) is automatically the causal SNP. SuSiE often shows lead SNP ≠ causal SNP, because the association signal can be distributed across correlated variants.

3.5 SuSiE vs. ABF

Feature	ABF	SuSiE
Bayesian	Yes	Yes
Uses summary statistics	Yes	Yes
Uses LD matrix	No	Yes
Multiple signals	No	Yes
Credible sets	Limited	Yes
Modern standard	No	Yes

SuSiE became popular for combining accurate PIPs, multiple-signal detection, credible sets, computational efficiency, summary-statistics compatibility, and natural integration with colocalization.

Key takeaways. SuSiE (Sum of Single Effects) is a Bayesian method that models several independent causal signals simultaneously, rather than assuming a single causal SNP. It needs GWAS summary statistics plus an LD matrix, and produces PIPs and credible sets per signal — now one of the most widely used fine-mapping methods because it handles the complexity of real genetic loci.

4 Running a Complete Fine-Mapping Analysis Step-by-Step

With the concepts and methods in hand, here’s a complete, self-contained workflow — simulate a locus with known causal variants, run GWAS, then compare ABF against SuSiE.

library(MASS)     # data simulation
library(susieR)   # fine-mapping
library(ggplot2)  # visualization

Simulate a genomic region of p <- 100 SNPs with moderate LD (off-diagonal 0.3, diagonal 1):

p <- 100
LD <- matrix(0.3, nrow = p, ncol = p)
diag(LD) <- 1

Simulate genotypes for N <- 1000 individuals using the LD matrix as the covariance structure:

set.seed(4)
N <- 1000
X <- mvrnorm(n = N, mu = rep(0, p), Sigma = LD)
# dim(X) -> 1000 x 100

Define L <- 3 true causal SNPs, chosen at random (e.g. variants 68, 39, 1 — the workshop’s exact setup):

set.seed(1)
L <- 3
causal_ind <- sample(1:p, L, replace = FALSE)

Assign effect sizes for a target heritability, e.g. h2g <- 0.1:

h2g <- 0.1
per_snp_h2g <- h2g / L
set.seed(5)
effect_sizes <- rnorm(L, mean = 0, sd = sqrt(per_snp_h2g))

beta <- rep(0, p)
beta[causal_ind] <- effect_sizes
genetic_effect <- X %*% beta

Simulate the phenotype by adding environmental noise:

var_g <- var(genetic_effect)
sigma_squared <- ifelse(1 - var_g > 0.05, 1 - var_g, 0.1)
set.seed(105)
epsilon <- rnorm(N, mean = 0, sd = sqrt(sigma_squared))
y <- as.numeric(genetic_effect + epsilon)

Run GWAS — one univariate regression per SNP, producing a full summary-statistics table:

GWAS_df <- data.frame(beta = numeric(p), se = numeric(p), z = numeric(p), pval = numeric(p))

for (i in 1:p) {
  fit <- summary(lm(y ~ X[,i] - 1))
  beta_hat <- fit$coefficients[1, "Estimate"]
  se_hat   <- fit$coefficients[1, "Std. Error"]
  z <- beta_hat / se_hat
  p_val <- pchisq(z^2, df = 1, lower.tail = FALSE)
  GWAS_df[i, ] <- c(beta_hat, se_hat, z, p_val)
}

Plotting -log10(pval) against SNP position gives a regional Manhattan plot; the tallest peaks are candidate causal variants.

Apply ABF fine-mapping using the Wakefield formula from Part 3, implemented as a reusable function:

run_abf <- function(beta, stderr, W = 0.04) {
  z <- beta / stderr
  V <- stderr^2
  r <- W / (W + V)
  lbf <- (log(1-r) + r*(z^2)) / 2
  lbf_max <- max(lbf)
  denom <- lbf_max + log(sum(exp(lbf - lbf_max)))
  exp(lbf - denom)
}

GWAS_df$PIP_ABF <- run_abf(GWAS_df$beta, GWAS_df$se)

ABF will often strongly prioritize just one signal, even when several causal variants exist.

Run SuSiE, using an in-sample LD matrix computed directly from the simulated genotypes:

R <- cov(scale(X))

susie_results <- susie_rss(
  bhat = GWAS_df$beta, shat = GWAS_df$se, n = N, R = R,
  var_y = var(y), L = 4, estimate_residual_variance = TRUE
)

GWAS_df$PIP_SuSiE <- susie_results$pip
head(GWAS_df[order(-GWAS_df$PIP_SuSiE), ])

HINT: For estimate_residual_variance = TRUE, please check that R is the "in-sample" LD matrix; that is, the correlation matrix obtained using the exact same data matrix X that was used for the other summary statistics. Also note, when covariates are included in the univariate regressions that produced the summary statistics, also consider removing these effects from X before computing R.

A data.frame: 6 × 6
	beta	se	z	pval	PIP_ABF	PIP_SuSiE

1	-0.23391752	0.02955646	-7.914261	2.487253e-15	9.999993e-01	1.000000e+00
39	0.11271078	0.03016101	3.736970	1.862512e-04	4.604485e-11	1.000000e+00
68	-0.17209840	0.02969970	-5.794617	6.847711e-09	6.692374e-07	9.999861e-01
51	-0.11439249	0.03180954	-3.596169	3.229380e-04	2.881971e-11	2.692740e-06
48	-0.10034456	0.03015366	-3.327774	8.754293e-04	1.120259e-11	2.578888e-06
32	-0.08948059	0.03040323	-2.943127	3.249148e-03	3.467398e-12	1.484153e-06

A PIP_SuSiE plot typically shows several distinct peaks, corresponding to the multiple simulated causal variants — unlike ABF’s single dominant peak.

Examine credible sets, the most important SuSiE output:

susie_results$sets$cs
# $L1: 1    $L2: 39   $L3: 68

$L1: 1
$L2: 39
$L3: 68

Three independent signals, correctly located at variants 39, 20, and 68.

4.1 ABF vs. SuSiE, Head to Head

Where ABF may identify only variant 1, SuSiE correctly recovers variants 1, 39, and 68 — because it allows multiple causal variants. This is precisely why SuSiE has become the preferred modern approach.

Key takeaways. A complete fine-mapping analysis starts from GWAS summary statistics and an LD matrix. ABF converts these into posterior probabilities under a single-causal-variant assumption; SuSiE extends this to multiple independent signals in the same locus. The outputs — PIPs and credible sets — feed directly into downstream eQTL and colocalization analyses.

5 Understanding Credible Sets in Practice

After running SuSiE, researchers often jump straight to PIPs — but the most informative output is usually the credible set itself, since most real GWAS loci don’t resolve to a single variant.

5.1 Why We Need Them

A single SNP at PIP = 0.98 makes life easy. But most loci look more like:

SNP	PIP
rs39	0.32
rs40	0.28
rs41	0.22
rs42	0.10
rs43	0.08

Now which SNP is truly causal is genuinely unclear — rather than pretending to know, Bayesian fine-mapping represents that uncertainty explicitly via the credible set.

A 95% credible set means there’s a 95% probability the causal SNP lies somewhere in the set — not a 95% probability attached to each member individually.

5.2 Worked Construction

Step	Add	Cumulative
1	rs39 (0.45)	0.45
2	rs40 (0.25)	0.70
3	rs41 (0.15)	0.85
4	rs42 (0.08)	0.93
5	rs43 (0.04)	0.97

The 95% credible set = {rs39, rs40, rs41, rs42, rs43}. This does not mean all five are causal — it means one (or more) of them likely is, but the data can’t distinguish which.

Resolution spectrum: ideal = {rs39} alone (excellent, near-certain localization); moderate = {rs39, rs40, rs41} (still useful — 3 variants to validate instead of 500); poor = 50–100 SNPs (data can’t isolate the causal variant at all).

5.3 Why Large Credible Sets Occur

High LD — SNPs with are statistically indistinguishable.
Small sample size — less information, more diffuse PIPs, larger sets. (N=500 vs N=50,000 makes a dramatic difference.)
Weak genetic effects — small is harder to localize than large .

Excellent fine-mapping: rs39 = 0.97, everything else ≈ 0.01 → credible set = {rs39}, near-perfect localization. Difficult fine-mapping: five SNPs each around 0.13–0.25 → credible set = all five, strong genuine uncertainty.

5.4 Multiple Credible Sets

Modern loci often contain multiple signals — Signal 1: CS1={rs20}, Signal 2: CS2={rs39, rs40}, Signal 3: CS3={rs68} — exactly what SuSiE was designed to discover. Each credible set corresponds to a separate causal signal:

susie_results$sets$cs
# $L1: 1    $L2: 39   $L3: 68

$L1: 1
$L2: 39
$L3: 68

5.5 Coverage Trade-Off

Most studies use 95% coverage, but 80%, 90%, and 99% are all used. Higher coverage (99%) gives more certainty but larger sets; lower coverage (80%) gives smaller sets but greater risk of missing the true causal SNP entirely.

5.6 Credible Sets ≠ Lead SNPs

A “lead SNP” (smallest p-value, or highest PIP) is a single point estimate; the credible set captures the uncertainty around it. These are not interchangeable, and conflating them is a common mistake.

What a fine-mapping study should report: lead SNP, PIP, credible set, credible set size, method used, LD reference panel. Example:

Lead variant rs39, PIP 0.94, 95% credible set {rs39, rs40}, size 2.

5.7 Biological Interpretation

Credible set size = 1 → strong candidate for CRISPR/reporter/functional validation. Credible set size = 50 → needs more data (larger GWAS, better LD panel, eQTL analysis, colocalization, functional annotations) before it can be refined further.

Key takeaways. Credible sets quantify uncertainty by reporting a group of variants that jointly contain the causal SNP with a specified probability (typically 95%), rather than pretending to identify a single answer. Small sets mean strong resolution; large sets reflect high LD, limited sample size, or weak effects. Credible sets are often more informative than lead SNPs precisely because they represent uncertainty explicitly.

6 The Impact of Sample Size, Effect Size, and LD on Resolution

“Why is my credible set so large? Why are my PIPs so low? Why can’t I identify the causal SNP?” The answer almost always comes down to three factors: sample size, linkage disequilibrium, and effect size.

Resolution refers to how precisely a causal variant can be localized. Excellent: 95% credible set = 1 SNP. Poor: 95% credible set = 50 SNPs. The goal is always small credible sets with high PIPs.

6.1 Factor 1: Sample Size

The single most important factor. Standard error scales as , so larger → smaller SE → larger Z-scores → higher PIPs.

Example: true . At : SE ≈ 0.08, Z ≈ 1.25 (weak evidence). At : SE ≈ 0.01, Z ≈ 10 (very strong evidence). Increasing sample size 10× often improves fine-mapping resolution dramatically — sharper, less noisy signal, smaller credible sets.

6.2 Factor 2: Linkage Disequilibrium

Fine-mapping is easiest when SNPs are weakly correlated. At , each SNP behaves independently and a causal signal is easy to pinpoint. At , neighboring SNPs look almost statistically identical (e.g. rs39 at and rs40 at ) — the model simply cannot tell them apart, producing large credible sets.

Resolution spectrum by LD: low LD → {rs39} (excellent); moderate LD → {rs39, rs40, rs41} (acceptable); very high LD → 25 SNPs (poor). Even a huge sample size () can’t fully overcome severe LD in a locus — more data helps, but doesn’t eliminate the fundamental statistical limitation.

Ancestry matters here: European-ancestry LD blocks tend to be long (harder to resolve); African-ancestry LD blocks tend to be shorter (often better resolution) — one reason multi-ancestry fine-mapping has become popular.

6.3 Factor 3: Effect Size

Large effects () → large Z-score → high PIP → often a single-SNP credible set. Small effects () → small Z-score → diffuse PIPs → large credible set. Traits differ in genetic architecture: LDL cholesterol often has large-effect variants; educational attainment is dominated by tiny effects — so some traits are inherently easier to fine-map than others. Lower heritability ( vs. ) weakens signals the same way.

6.4 Interaction Between Factors

Best case: large sample size + low LD + large effects → tiny credible sets, high PIPs. Worst case: small sample size + high LD + tiny effects → huge credible sets, low PIPs.

6.5 Diagnosing Poor Fine-Mapping

When results look weak, check: Is sample size large enough? Is LD too high? Are effect sizes too small? Is the LD reference panel the right ancestry?

Common beginner mistake: interpreting a 50-SNP credible set as “fine-mapping failed.” Usually the method is working correctly — the data simply doesn’t contain enough information to distinguish the variants. Fine-mapping quantifies uncertainty; it doesn’t eliminate it.

Key takeaways. Fine-mapping resolution is driven primarily by sample size, LD, and effect size. Larger samples and stronger effects sharpen PIPs; high LD makes neighboring variants statistically indistinguishable and enlarges credible sets. No method can fully overcome severe LD or limited information — understanding these limits is essential for realistic interpretation and avoiding overconfidence in causal variant claims.

7 Fine-Mapping with Individual-Level Data vs. Summary Statistics

The first question in any fine-mapping project is: what data do I actually have? The answer determines which methods you can use, how accurate your results will be, how LD gets estimated, and whether colocalization is even possible.

7.1 Two Main Types of Data

Individual-level data — genotypes, phenotypes, and covariates per person:

Person	SNP1	SNP2	SNP3	Phenotype
1	0	1	2	3.2
2	1	1	0	1.7
3	2	0	1	4.5

Summary statistics only — no individual genotypes:

SNP	Beta	SE	P-value
rs1	0.12	0.03	1e-8
rs2	0.10	0.03	5e-7
rs3	0.02	0.04	0.40

Large biobanks (UK Biobank, FinnGen, deCODE, iPSYCH) usually release only summary statistics — raw genotype sharing raises privacy, storage, and regulatory concerns.

7.2 Individual-Level Data Workflow (Gold Standard)

R <- cov(scale(X))   # X = genotype matrix from your own samples

This produces in-sample LD — LD computed from exactly the individuals in your GWAS — the most accurate LD possible, and considered the gold standard.

7.3 Summary Statistics Workflow

Common reference panels: 1000 Genomes, UK Biobank, TOPMed, HRC. You must match the reference panel’s ancestry to your GWAS cohort — e.g. European-ancestry GWAS → 1000 Genomes Europeans.

7.4 Why LD Matching Is Critical

If your GWAS is Finnish but you estimate LD from an African-ancestry panel, the LD structure will genuinely differ — e.g. true GWAS LD between rs39/rs40 might be while the mismatched reference shows . That mismatch can cause false signals, incorrect PIPs, wrong credible sets, and failed colocalization — one of the most common causes of unreliable fine-mapping. SuSiE in particular needs good LD, since it relies on knowing exactly which SNPs are correlated to separate multiple signals.

Methods requiring LD: SuSiE, FINEMAP, CAVIAR, CARMA. Methods that don’t: ABF (needs only beta and SE) — one reason it remains useful as a fallback.

7.5 Single-Cohort vs. Meta-Analysis

Single-cohort (e.g. FinnGen or UK Biobank alone): ancestry-matched LD → fine-mapping is generally straightforward.

Meta-analysis (European + Finnish + Japanese + African cohorts combined): each cohort has different LD, so a single LD matrix no longer represents the pooled data accurately. This motivated specialized meta-analysis methods. Common strategies: fine-map each cohort separately then compare/integrate results, or use methods purpose-built for heterogeneous LD (FastMap and related approaches).

7.6 Multi-Ancestry Fine-Mapping

Different populations have different LD block lengths — combining ancestries can dramatically improve resolution, since a variant tightly linked to several others in a European LD block might sit alone in an African LD block. Example: a European-only GWAS gives a credible set of 40 SNPs, but adding African-ancestry data narrows it to 5.

7.7 Data-Type Ranking

Tier	Data
Best	Individual-level data + in-sample LD
Good	Summary statistics + matched LD panel
Risky	Summary statistics + poorly matched LD
Worst	Summary statistics + no LD information

Practical advice: use in-sample LD whenever possible; always match ancestry between GWAS and LD panel; avoid mixing populations unless using methods explicitly designed for multi-ancestry analysis.

This all matters even more for colocalization later: coloc.susie depends on accurate fine-mapping, and poor LD estimation can cause both false and missed colocalization. Good LD = good fine-mapping = good colocalization.

Key takeaways. Data availability determines your entire fine-mapping strategy. Individual-level data gives the best results because LD is computed directly from the study sample. Summary-statistics-only analyses require an external, ancestry-matched LD reference panel — mismatch here is one of the most common sources of unreliable results in modern fine-mapping.

8 Major Fine-Mapping Methods — ABF, CAVIAR, FINEMAP, SuSiE, CARMA, FastMap

The core concepts (LD, PIPs, credible sets, Bayesian inference, multiple causal variants) are now in place — the natural next question is which method to actually use. Choice depends on available data, LD information, study design, and computational resources.

Historical evolution: — each generation addressed limitations of the last.

Method	Main idea	Inputs	Multiple signals?	Needs LD?	Notes
ABF	Assume one causal variant; compute Bayes Factors per SNP, normalize to posteriors	Beta, SE	No	No	Fast, simple, minimal inputs; can’t separate multiple signals; best when no trustworthy LD exists or for quick exploratory passes. Basis of `coloc.abf`.
CAVIAR	Models several causal SNPs simultaneously via Z-scores + LD	Z-scores, LD matrix	Yes	Yes	One of the first methods beyond ABF; accounts for LD and multiple variants, but computationally expensive in large regions. Historically important — inspired later methods.
FINEMAP	Bayesian model selection: searches causal-variant configurations (e.g. {rs39}, {rs20+rs39}, {rs20+rs39+rs68}) and finds which best explains the data	Summary statistics, LD matrix	Yes	Yes	Fast, accurate, scalable for large GWAS/biobank studies; widely used; needs good LD or performance degrades.
SuSiE	Sum of single effects — models phenotype as several independent signals rather than one	Summary statistics + LD, or individual-level data	Yes	Yes	Excellent multiple-signal handling, interpretable credible sets, computationally efficient, integrates naturally with colocalization. Sensitive to LD mismatch. The current default for most researchers.
CARMA	CAusal variant identification with Allelic heterogeneity and outliers — explicitly models outlier/artifact SNPs (genotyping errors, imputation artifacts)	Summary statistics, LD matrix	Yes	Yes	Robust to problematic data; more computationally intensive and harder to interpret. Best for uncertain data quality or large meta-analyses prone to artifacts.
FastMap	Efficient approximations built for biobank-scale data (millions of individuals, thousands of loci)	Summary statistics, LD	Yes	Yes	Very fast, highly scalable; newer and less extensively validated than SuSiE/FINEMAP. Best for massive biobank or meta-analysis fine-mapping.

8.1 Choosing a Method

Your situation	Recommended method
Only beta/SE, no LD matrix	ABF
Summary statistics + good LD	SuSiE or FINEMAP
Suspected outliers/artifacts	CARMA
Very large-scale (biobank) analysis	FastMap
Planning colocalization afterward	SuSiE (integrates with `coloc.susie`)

Current consensus: ask most statistical geneticists for the default fine-mapping method today and the answer is SuSiE — it offers multi-signal modeling, credible sets, PIPs, strong theoretical grounding, and mature software support. FINEMAP remains extremely popular too, often run alongside SuSiE.

Practical recommendation summary: individual-level data → SuSiE; summary statistics + good LD → SuSiE or FINEMAP; no LD available → ABF; suspected data artifacts → CARMA; massive biobank analyses → FastMap.

All these methods answer the same underlying question — “which variants are most likely causal?” — differing mainly in their assumptions, computational strategy, treatment of multiple signals, and robustness to imperfect data. The shared output vocabulary — PIPs and credible sets — is the common language of modern fine-mapping.

Key takeaways. ABF is simple and useful without LD but assumes a single causal variant. CAVIAR pioneered multi-variant modeling. FINEMAP and SuSiE are the current workhorses for multi-signal fine-mapping with credible sets. CARMA adds robustness to outliers; FastMap trades some validation maturity for speed at biobank scale. In practice, SuSiE and FINEMAP dominate when reliable LD is available.

9 The Fine-Mapping Decision Tree — Choosing the Correct Pipeline

For beginners, the hard part isn’t running fine-mapping software — it’s choosing the right workflow before you even start. Rather than picking a method first, start by asking: what data do we actually have?

9.1 Step 1: Do You Have Individual-Level Data?

That means genotypes and phenotypes for each participant — PLINK files (.bed/.bim/.fam) or VCF + phenotype files. If yes, follow Branch A; if no, follow Branch B.

9.2 Branch A: Individual-Level Data

The ideal case (UK Biobank, FinnGen, iPSYCH, All of Us, when individual-level access is granted) — you can compute in-sample LD directly:

Single cohort: straightforward — in-sample LD, then SuSiE. Multiple cohorts (European + African + Asian, each with full genotypes): either (1) fine-map each cohort separately then compare — often recommended — or (2) run multi-ancestry fine-mapping, since differing LD structures across ancestries can improve resolution.

9.3 Branch B: Summary Statistics Only

Most researchers land here. The critical follow-up question: is a reliable, ancestry-matched LD panel available?

B1 — good LD available: match ancestry between GWAS and reference panel (mismatch distorts PIPs and credible sets) → use SuSiE or FINEMAP.
B2 — no reliable LD: SuSiE/FINEMAP aren’t appropriate without LD → fall back to ABF, accepting its single-causal-variant assumption as a compromise: .
B3 — suspected data problems (imputation errors, outliers, meta-analysis artifacts): , built specifically for robustness to problematic variants.

9.4 The Meta-Analysis Branch

Combining cohorts (UK Biobank + FinnGen + deCODE + Biobank Japan) into one meta-analysis means multiple LD structures exist simultaneously — but fine-mapping methods assume one LD matrix, creating a mismatch. Two strategies: (1) cohort-specific fine-mapping — fine-map FinnGen, UK Biobank, and deCODE separately, then integrate; often preferred — or (2) specialized meta-analysis methods like FastMap, designed to account for heterogeneous LD.

9.5 The Colocalization Branch

Once you have a GWAS credible set and an eQTL credible set in the same region, the next question is whether they share a causal variant. If you used ABF → coloc.abf (, single-variant assumption). If you used SuSiE → coloc.susie (), generally preferred since it allows multiple causal variants and aligns naturally with modern fine-mapping.

9.6 The Complete Decision Tree

Individual-level data?
 ├─ Yes → Compute in-sample LD → SuSiE
 └─ No  → Summary statistics → Good LD available?
              ├─ Yes → SuSiE / FINEMAP
              └─ No  → ABF
                          ↓
                   Credible Sets → Colocalization
                                      ├─ coloc.abf
                                      └─ coloc.susie

9.7 Worked Examples

Scenario	Data	Recommendation
FinnGen study	Summary stats + FinnGen LD panel	SuSiE
Public GWAS Catalog result	Beta + SE only, no LD	ABF
UK Biobank genotypes	Individual-level data	In-sample LD → SuSiE
Psychiatric Genomics Consortium meta-analysis	Multi-cohort meta-analysis	Cohort-specific fine-mapping, or FastMap

Common beginner mistake: picking a method (SuSiE, say) before checking whether the required inputs (a trustworthy LD matrix) actually exist. Always start from the data you have, not the method you’ve heard of.

Recommended default workflow: check for individual-level data first; if unavailable, check for a matched LD panel; if that’s also unavailable, use ABF as a fallback; treat CARMA and FastMap as targeted solutions for artifact-prone or biobank-scale data respectively.

Key takeaways. The correct fine-mapping pipeline is determined by data type, not by method popularity. Individual-level data → in-sample LD → SuSiE is the gold-standard path. Summary-statistics-only analyses hinge on whether a well-matched LD reference panel exists; without one, ABF is the fallback. Meta-analyses and suspected data-quality issues call for specialized approaches (FastMap, CARMA respectively). This decision tree should be the first step of any fine-mapping project, before any software is run.

10 Integrating Fine-Mapping with eQTL Data — Identifying Candidate Genes

Fine-mapping narrows a locus down to a small set of candidate causal SNPs — but it doesn’t tell us which gene is affected. Most GWAS hits are non-coding, often sitting 50–100 kb from the nearest gene, so proximity alone doesn’t reveal the mechanism. This is where eQTL (expression quantitative trait locus) analysis comes in.

10.1 What Is an eQTL?

A genetic variant that influences gene expression: . For example, rs12345 might increase expression of Gene A while decreasing expression of Gene B — creating a biological mechanism linking DNA variation to disease.

Why this matters: suppose a schizophrenia GWAS finds rs12345 with PIP = 0.95 — excellent fine-mapping, but we still don’t know which gene is responsible. If rs12345 also turns out to be an eQTL for CACNA1C, we now have a concrete hypothesis: .

Data sources: GTEx, the eQTL Catalogue, PsychENCODE, CommonMind Consortium, eQTLGen.

Cis- vs. trans-eQTLs: most studies focus on cis-eQTLs (variant within ~1 Mb of the gene it regulates); trans-eQTLs (a chromosome 1 SNP affecting a chromosome 12 gene, say) are generally harder to detect.

10.2 The Fine-Mapping + eQTL Workflow

Suppose fine-mapping gives a 95% credible set of {rs39, rs40, rs41}. Checking an eQTL database might show rs39 → Gene A, rs40 → no eQTL, rs41 → Gene B — immediately turning two SNPs into two strong gene candidates. Fine-mapping first is essential: starting from 1,000 raw GWAS SNPs and checking eQTLs for all of them would produce an unmanageable number of candidate genes; narrowing to 5 SNPs first makes the eQTL lookup tractable.

The “nearest gene” trap: a common beginner mistake is assuming the nearest gene is the causal gene. It often isn’t — a GWAS SNP may sit near Gene A but actually regulate Gene B much farther away. eQTL data resolve this ambiguity.

10.3 Fine-Mapping the eQTL Itself

eQTL studies can (and should) be fine-mapped the same way GWAS is: . If GWAS fine-mapping gives CS = {rs39} and eQTL fine-mapping independently gives CS = {rs39}, that’s highly suggestive — both analyses point at the same variant, though it’s not yet proof (that requires formal colocalization, Part 12).

10.4 Tissue Specificity

A variant may be an eQTL in brain but not in blood or liver. For schizophrenia, relevant tissues include prefrontal cortex, hippocampus, and neurons — brain eQTLs are typically far more informative than blood eQTLs for a brain-relevant trait. A standard psychiatric-genomics workflow: .

10.5 Fine-Mapping + eQTL = Gene Prioritization

Fine-mapping answers “which SNP?”; eQTL analysis answers “which gene?” Together they give gene prioritization — but even when a GWAS SNP and an eQTL SNP look similar, that alone doesn’t prove they’re the same causal variant, because LD can create misleading overlap. That gap is exactly what colocalization (Part 12) formally resolves.

Key takeaways. Fine-mapping identifies likely causal variants but not the affected gene. eQTL analysis fills that gap by linking variants to gene expression. Intersecting fine-mapped GWAS credible sets with eQTL results prioritizes candidate genes and generates mechanistic hypotheses — but overlap alone doesn’t prove a shared causal variant, since LD can create misleading co-occurrence. That limitation motivates formal colocalization methods.

11 Colocalization — Does a GWAS Signal Share a Causal Variant With an eQTL Signal?

A GWAS signal and an eQTL signal overlapping in the same region doesn’t necessarily mean the same variant drives both. Colocalization is the formal statistical test for whether two association signals are driven by the same underlying causal variant.

The ambiguity: suppose a schizophrenia GWAS implicates rs39, and an eQTL study implicates rs40 for Gene A expression, with . Two explanations are equally consistent with strong LD alone:

Shared causal variant (true colocalization): rs39 drives both Gene A expression and disease risk directly.
Distinct variants (not colocalization): rs39 drives disease risk, rs40 (a different variant) drives Gene A expression — strong LD just makes them look similar.

11.1 The Five Colocalization Hypotheses

The coloc framework evaluates five mutually exclusive hypotheses, with posterior probabilities PP.H0–PP.H4 summing to 1:

Hypothesis	Meaning
H0	No association with either trait
H1	Association with Trait 1 (GWAS) only
H2	Association with Trait 2 (eQTL) only
H3	Both traits associated, but different causal variants (“linkage”)
H4	Both traits associated, one shared causal variant — the result most researchers hope for

Example: PP.H0=0.00, PP.H1=0.00, PP.H2=0.01, PP.H3=0.05, PP.H4=0.94 → 94% probability of a shared causal variant, strong evidence for colocalization.

Interpreting PP.H4: >0.80 strong evidence, >0.90 very strong, >0.95 extremely strong. Conversely, PP.H3 = 0.95 means both traits are genuinely associated, but through different variants — a very different biological conclusion.

11.2 `coloc.abf`

The original method — summary-statistics only, no LD matrix required. Inputs: beta, SE, and sample size for both traits. Major assumption: one causal variant per locus — the same limitation as ABF fine-mapping. This becomes a real problem when a locus genuinely has multiple signals: in one demonstration, a GWAS causal variant (39) overlapped an eQTL locus with two signals (39, shared, and 20, private) — coloc.abf strongly favored H3 (different variants) even though a shared variant truly existed, because it could only test one variant at a time.

11.3 `coloc.susie`

Built to fix exactly this: instead of comparing one signal against one signal, it compares credible set against credible set (CS1 vs CS1, CS2 vs CS2, …), using SuSiE fine-mapping results from both traits. Workflow: and , both feeding into coloc.susie. In the same multi-signal example, coloc.susie correctly recovers PP.H4 ≈ 1.0 for the shared signal — advantages: handles multiple causal variants, is credible-set aware, and works on complex loci. This makes it the modern preferred choice whenever reliable LD is available.

11.4 Biological Interpretation and Its Limits

If a GWAS colocalizes with brain eQTL data for CACNA1C at PP.H4 = 0.97, that’s strong evidence the same variant influences both CACNA1C expression and disease risk — a compelling mechanistic hypothesis. But even PP.H4 = 0.99 does not prove causality — it only means the data are consistent with one shared variant. Confirming the mechanism still requires functional studies, CRISPR experiments, or animal models.

Key takeaways. Colocalization formally tests whether two association signals share a causal variant, via five posterior probabilities (H0–H4), with H4 = shared variant and H3 = distinct variants. coloc.abf needs only summary statistics but assumes one causal variant per locus; coloc.susie overcomes this by comparing fine-mapped credible sets and is now the preferred default when good LD is available. Colocalization bridges GWAS findings to gene regulatory mechanisms — but even strong PP.H4 is evidence, not proof, of a causal mechanism.

12 Practical Interpretation of Colocalization Results

Running a colocalization analysis is easy; interpreting it correctly is much harder. Seeing PP.H4 = 0.85 and immediately concluding “gene identified!” skips several important checks.

12.1 Worked Examples

Scenario	PP.H0	PP.H1	PP.H2	PP.H3	PP.H4	Interpretation
Strong colocalization	0.00	0.01	0.02	0.03	0.94	Very strong evidence for a shared causal variant — textbook success
Strong evidence against colocalization	0.00	0.00	0.00	0.96	0.04	Both traits associated, but different variants — one of the most common real-world outcomes
Ambiguous region	0.05	0.10	0.10	0.35	0.40	Inconclusive — neither H3 nor H4 dominates; more data needed

12.2 Conventional Thresholds

PP.H4	Evidence
< 0.50	Weak
0.50–0.80	Moderate
≥ 0.80	Strong
≥ 0.90	Very strong
≥ 0.95	Exceptional

These are conventions, not strict rules — context matters.

12.3 Look Beyond PP.H4 Alone

Focusing only on PP.H4 is risky: PP.H3 = 0.45 and PP.H4 = 0.50 technically makes H4 the largest single probability, but there’s nearly as much support for “different variants” — the result is genuinely uncertain. A useful diagnostic is the ratio : for PP.H3=0.10, PP.H4=0.90, this gives 0.90 — excellent support for sharing. Low ratios flag ambiguous H3-vs-H4 competition even when PP.H4 looks superficially high.

12.4 Fine-Mapping Quality Drives Colocalization Quality

A GWAS credible set of 50 SNPs and an eQTL credible set of 60 SNPs create enormous combined uncertainty — colocalization may be inconclusive no matter how it’s run. By contrast, GWAS = {rs39} and eQTL = {rs39} makes colocalization almost trivially easy to interpret (PP.H4 ≈ 1). Poor fine-mapping upstream directly degrades colocalization downstream.

12.5 Four Common Mistakes

Assuming visual overlap = colocalization. A GWAS peak and an eQTL peak sitting in the same region doesn’t mean “same signal” — strong LD alone can create that appearance. This is exactly why formal colocalization methods exist.
Ignoring LD mismatch. If the GWAS uses European samples and the eQTL uses African samples, differing LD can yield different credible sets even when the underlying biology is identical. Always consider ancestry.
Treating colocalization as proof of causality. Even PP.H4 = 0.99 only suggests shared genetic regulation, not that “Gene A causes disease” — that claim needs functional studies, CRISPR experiments, animal models, or perturbation experiments.
Ignoring tissue context. A brain eQTL colocalization (PP.H4 = 0.95) is far more biologically relevant for schizophrenia than a blood eQTL colocalization (PP.H4 = 0.05) for the same locus — tissue selection matters enormously.

12.6 Sensitivity Analysis

Because coloc’s Bayesian priors (, , — beliefs about how likely a SNP affects trait 1, trait 2, or both) influence the posterior, a good study tests multiple prior settings:

Illustrative syntax – dataset1/dataset2 are placeholders for your own formatted summary-statistics lists; see the coloc package documentation for the exact list format each expects.

coloc.abf(dataset1, dataset2, p1 = 1e-4, p2 = 1e-4, p12 = 1e-5)

If PP.H4 = 0.82 under one prior but 0.35 under another, the result is unstable and should be reported cautiously.

12.7 A Complete Worked Example

Strong evidence that the same variant influences both gene expression and depression risk — a biologically meaningful hypothesis. A publication should report: gene, tissue, PP.H4, PP.H3, method used, and both credible set sizes — e.g. CACNA1C, prefrontal cortex, coloc.susie, PP.H4 = 0.96, GWAS CS size 2, eQTL CS size 1.

12.8 Hierarchy of Evidence and Practical Checklist

Before trusting a result, check: are both traits genuinely associated (H1/H2/H3/H4)? Is PP.H4 high (preferably >0.8)? Is PP.H3 correspondingly low? Were the credible sets small? Was LD properly ancestry-matched? Is the tissue biologically relevant? Were sensitivity analyses performed across priors?

Key takeaways. Interpret colocalization using the full set of posterior probabilities, not PP.H4 in isolation — a high PP.H4 alongside a non-trivial PP.H3 is genuinely ambiguous. Fine-mapping quality directly limits colocalization quality: poorly resolved credible sets create irreducible uncertainty. Colocalization is strong evidence, not proof, of a shared mechanism — and proper interpretation requires attention to ancestry matching, tissue relevance, and prior sensitivity.

13 End-to-End Statistical Genetics Pipeline — From GWAS to Biological Discovery

GWAS, fine-mapping, eQTL analysis, and colocalization were each covered individually — but in practice they’re chained into a single integrated pipeline whose goal is to move from association to biological mechanism.

13.1 The Fundamental Problem

A GWAS hit on chromosome 6 for schizophrenia tells us something important is here — but not which variant, which gene, or which mechanism. That gap is what the modern post-GWAS pipeline exists to close.

13.2 The Complete Workflow

This framework underlies studies from UK Biobank, FinnGen, the Psychiatric Genomics Consortium, GTEx, and PsychENCODE.

Step by step, with a running example (rs39, schizophrenia):

GWAS — 500,000 individuals, 10 million SNPs, model , output beta/SE/p-value per SNP. rs39 at — strong association, not yet causality. Because of LD, rs39–rs42 may all look significant; GWAS alone can’t say which is causal.
Fine-mapping — SuSiE/FINEMAP/CARMA/ABF, using GWAS summary stats + LD matrix, narrow this down: rs39, PIP = 0.94, 95% credible set = {rs39, rs40}. What started as ~100 candidate SNPs is now 2 — a dramatic improvement in tractability.
Gene prioritization — the nearest gene is often not the causal gene, so functional genomics data must be integrated.
eQTL analysis — rs39 influences CACNA1C expression in brain tissue, giving a testable hypothesis: . Fine-mapping answers “which SNP?”; eQTL answers “which gene?”
Fine-mapping the eQTL — independently fine-map the eQTL signal too: GWAS CS = {rs39}, eQTL CS = {rs39}. Highly suggestive, but still not proof.
Colocalization — coloc.abf or coloc.susie formally test whether both signals share a causal variant across the five hypotheses (H0: no association; H1: trait 1 only; H2: trait 2 only; H3: different variants; H4: shared variant). PP.H4 = 0.96 → strong evidence for a shared causal variant, often the key result of the whole study.

Putting it together: GWAS finds rs39 → fine-mapping gives PIP = 0.98 → eQTL analysis links rs39 to CACNA1C → colocalization gives PP.H4 = 0.97. This supports — a biologically meaningful hypothesis distilled from an initial 10 million tested SNPs down to 1 variant, 1 gene, 1 mechanism.

13.3 Real Example: Psychiatric Genetics

using resources like PsychENCODE, GTEx, and CommonMind.

13.4 Beyond eQTLs: Multi-Omics Integration

The same fine-mapping + colocalization framework extends to other molecular QTLs: sQTLs (splicing), pQTLs (protein), mQTLs (methylation), caQTLs (chromatin accessibility). A modern multi-omics workflow:

Single-cell data identifies which cell types express the target gene (e.g. Gene A highly expressed in excitatory neurons but not microglia), adding biological context. TWAS (transcriptome-wide association) complements this: . Mendelian Randomization goes a step further, testing whether altered expression is causally related to disease () — stronger evidence than colocalization alone.

13.5 Hierarchy of Evidence

Each layer adds evidence toward the ultimate chain: .

Common beginner misconception: stopping after GWAS. GWAS is usually just the starting point — the real biological discoveries typically emerge during fine-mapping, eQTL integration, colocalization, and functional interpretation. A modern statistical geneticist’s typical project: .

Final key takeaways. The journey from GWAS to biological discovery is a chain of interconnected steps: GWAS identifies associated regions, fine-mapping narrows them to likely causal variants, eQTL analysis links variants to gene expression, and colocalization tests whether the same variant drives both expression and disease risk. Modern statistical genetics increasingly layers in single-cell genomics, TWAS, Mendelian Randomization, and multi-omics data to build a fuller mechanistic picture. The ultimate objective is to move from a significant SNP to a causal gene, a relevant cell type, a biological pathway, and eventually therapeutic insight.

Mendelian Randomization

Nivedita Bhadra — Mon, 15 Jun 2026 21:00:00 GMT

One of the fundamental goals of science is to determine whether one variable causes another.

In medicine, epidemiology, genetics, economics, and social sciences, researchers constantly observe relationships between variables. Observing a relationship, however, is not the same as proving a causal effect.

Consider a few familiar examples:

Individuals with a higher body mass index (BMI) tend to have a higher risk of coronary heart disease.
Smokers are more likely to develop lung cancer.
Elevated C-reactive protein (CRP) levels are associated with hypertension.
Individuals who exercise regularly tend to live longer.

These observations are informative, but none of them establishes causality on its own.

Understanding the distinction between correlation and causation is one of the most important ideas in statistics and epidemiology. It is also the motivation behind Mendelian Randomization (MR), an approach that uses genetic variants to investigate causal relationships.

What you’ll learn in this series

How to distinguish correlation from causation

Why observational associations can be misleading

What confounding and reverse causation mean

How Mendelian Randomization uses genetic variants as instruments

When MR is a useful tool for causal inference

How to run a complete two-sample MR analysis in R, from instrument selection to sensitivity analysis

Who this is for. Researchers and students who want an accessible, code-first introduction to Mendelian Randomization with a genetics focus. Familiarity with basic statistics and epidemiology is helpful but not required.

Why MR matters. Mendelian Randomization helps researchers address causal questions when randomized controlled trials are not feasible. It uses the natural randomization of genetic inheritance to infer whether an exposure may have a causal effect on an outcome.

1 Part 1 — Correlation, Causation, and Why We Need MR

1.1 What Is Correlation?

Correlation describes the extent to which two variables vary together.

When one variable increases and another tends to increase as well, the variables have a positive correlation. When one increases while the other decreases, they have a negative correlation.

Correlation is most commonly measured with the Pearson correlation coefficient, , where .

Correlation	Interpretation
	Perfect positive relationship
	No linear relationship
	Perfect negative relationship

Height and weight, for example, are usually positively correlated, since taller individuals generally weigh more than shorter individuals. But a correlation coefficient never tells us whether one variable causes the other.

1.2 The Central Question

Suppose researchers observe that individuals with elevated CRP levels tend to have higher blood pressure. This raises an obvious question:

Does CRP cause high blood pressure?

Several explanations are equally consistent with the data:

CRP causes high blood pressure.
High blood pressure causes elevated CRP.
Some other factor influences both CRP and blood pressure.
The relationship is partly causal and partly confounded.

Observational data alone cannot distinguish between these possibilities.

1.3 Correlation Does Not Imply Causation

Correlation does not imply causation.

Two variables can be strongly correlated even when neither directly causes the other. There are two classic reasons this happens: reverse causation and confounding.

1.3.1 Reverse Causation

Reverse causation occurs when the true direction of the relationship is the opposite of what we initially assume. Researchers may observe

and conclude that the biomarker causes disease, when the true relationship runs the other way:

Elevated inflammatory markers, for instance, are often observed in people who already have disease — the disease raised the biomarker, not the other way around. Reverse causation is especially problematic in observational studies, because measurements are often taken after disease processes have already begun.

1.3.2 Confounding

Confounding occurs when a third variable influences both the exposure and the outcome. For example, general health-consciousness influences both exercise habits and diet quality. A researcher might observe a strong association between exercise and diet quality, but the relationship is largely explained by an underlying tendency toward healthy behavior overall, not a direct causal link between the two.

A classic example: ice cream and drowning. Across a year of data, ice cream sales and drowning incidents both rise in summer and are positively correlated. It would clearly be wrong to conclude

Instead, temperature is the confounder driving both:

This simple example illustrates why associations alone are insufficient for establishing causality.

1.4 Why Observational Studies Are Challenging

Most epidemiological studies are observational: researchers observe naturally occurring variation in exposures such as smoking, alcohol use, obesity, physical activity, diet, and blood biomarkers, without controlling who receives which exposure.

Observational studies are valuable because they are inexpensive and can involve very large populations. But their estimates are frequently affected by confounding, reverse causation, measurement error, and selection bias — so observed associations may not reflect true causal effects.

1.4.1 The Counterfactual Problem

A fundamental challenge in causal inference is that we cannot observe alternative realities. For an individual with obesity, we observe their actual outcome,

but we can never simultaneously observe what would have happened to that same person had they not been obese. This unobservable alternative is called the counterfactual, and because it can never be directly observed, causal inference is inherently difficult.

1.5 Randomized Controlled Trials

The gold standard for causal inference is the randomized controlled trial (RCT). Participants are randomly assigned to a treatment group or a control group, and randomization balances both measured and unmeasured confounders across the two groups:

If the treatment and control groups differ in outcomes after randomization, that difference is more likely to represent a true causal effect.

1.5.1 Why Not Use RCTs for Everything?

Randomized trials are powerful, but often impossible, unethical, or impractical. We cannot randomly assign people to smoke, to be obese, to drink heavily, or to have high blood pressure. For many important public health questions, randomized experiments simply cannot be performed, so scientists need alternative methods for investigating causality.

1.6 Nature’s Randomized Experiment

Genetics provides a natural form of randomization. During reproduction, genetic variants are transmitted from parents to offspring according to Mendel’s laws, and because this transmission is largely random, genetic variants are generally assigned before birth, before disease onset, before lifestyle choices are made, and before most environmental exposures occur.

This natural randomization creates an opportunity to study causal relationships using genetic information. The central idea is simple:

If a genetic variant influences an exposure, and that exposure truly causes an outcome, then the genetic variant should also be associated with the outcome.

This insight is the foundation of Mendelian Randomization.

1.7 Looking Ahead

We saw that observational associations can arise from confounding, reverse causation, or other bias, and why randomized controlled trials are the gold standard for causal inference despite being infeasible for many important questions. The next part introduces Mendelian Randomization itself, and shows how naturally occurring genetic variation can serve as an instrumental variable for investigating causal relationships.

Key takeaways

Correlation measures association, not causation.

Observational studies are vulnerable to confounding and reverse causation.

Strong associations do not necessarily indicate causal effects.

Randomized controlled trials reduce bias through randomization, but are not always feasible or ethical.

Genetic variants provide a natural source of randomization that Mendelian Randomization exploits.

2 Part 2 — What Is Mendelian Randomization?

In Part 1 we discussed why establishing causality is difficult, and why RCTs — the gold standard — are not always feasible or ethical. This raises an obvious question:

Can we obtain evidence about causality without performing a randomized experiment?

Mendelian Randomization (MR) attempts to answer this question using genetic variation.

2.1 The Central Idea Behind Mendelian Randomization

MR uses genetic variants as proxies for modifiable exposures. The key insight is that genetic variants are assigned at conception according to Mendel’s laws of inheritance. Because they are determined before birth, genetic variants cannot be influenced by later disease processes, are generally unaffected by lifestyle choices, and are usually independent of many environmental confounders. This makes them useful instruments for studying causality.

2.1.1 A Motivating Example

Suppose researchers observe that individuals with elevated CRP levels tend to have higher blood pressure. Observational data alone cannot tell us whether

MR attempts to answer this using genetic variants associated with CRP.

2.2 Mendelian Randomization as Nature’s Experiment

In an RCT, random assignment of treatment balances confounders:

In Mendelian Randomization, we replace treatment assignment with genetic inheritance. Instead of randomly assigning individuals to a treatment, nature randomly assigns genetic variants during reproduction:

Example: smoking and lung cancer. Suppose one genotype is associated with heavier smoking and another with lighter smoking. If smoking truly causes lung cancer, then people carrying the smoking-promoting genotype should also show a higher risk of lung cancer. The genetic variant acts as a proxy for smoking behavior, and comparing groups defined by genotype is less vulnerable to many of the traditional confounders that plague observational comparisons of smokers and non-smokers.

2.3 What Is an Instrumental Variable?

An instrumental variable is a variable that helps estimate a causal effect in the presence of confounding. In MR, the genetic variant is the instrument. The exposure might be BMI, smoking, cholesterol, blood pressure, CRP, or alcohol consumption; the outcome might be coronary heart disease, stroke, diabetes, cancer, or depression. The causal framework is:

with the genetic variant influencing the outcome only through the exposure.

The basic MR model has three components:

Exposure — a modifiable factor of interest (BMI, smoking, LDL cholesterol, CRP, physical activity, …)
Outcome — a disease or trait (coronary heart disease, stroke, type 2 diabetes, Alzheimer’s disease, …)
Instrument — a genetic variant associated with the exposure (SNPs associated with BMI, cholesterol, smoking behavior, …)

2.4 The Three Core Assumptions of Mendelian Randomization

Every MR study depends on three fundamental assumptions. Violating any of them can bias the causal estimate.

2.4.1 Assumption 1: Relevance

The genetic variant must be associated with the exposure:

If a SNP has no relationship with BMI, for example, it cannot tell us anything about whether BMI affects disease. Strong instruments produce more reliable MR estimates; weak instruments can produce unstable, biased results.

2.4.2 Assumption 2: Independence

The genetic variant should not be associated with confounding variables such as income, education, diet, exercise, or socioeconomic status:

If the genetic variant is correlated with confounders, it no longer behaves like a randomized experiment.

2.4.3 Assumption 3: Exclusion Restriction

The genetic variant must influence the outcome only through the exposure. The desired pathway is

and there should be no direct pathway

that bypasses the exposure. This is usually the hardest assumption to verify.

2.5 Understanding Pleiotropy

A major threat to MR is pleiotropy — when a genetic variant influences multiple traits. If a SNP affects both BMI and blood pressure, and we’re studying whether BMI affects blood pressure, the direct SNP → blood pressure effect creates a problem: it violates the exclusion restriction. Later parts in this series discuss methods such as MR-Egger that attempt to detect and correct for pleiotropy.

2.6 Why Genetic Variants Are Useful Instruments

Genetic variants have three attractive properties:

Fixed at conception. Genotypes are established before birth, and disease cannot alter them, which greatly reduces reverse causation.
Usually precede disease. This gives a natural temporal ordering: .
Less influenced by lifestyle. Unlike environmental exposures, genotypes are not modified by diet, exercise, smoking, or medication use, which helps reduce confounding.

A conceptual example. Suppose a SNP is associated with higher CRP levels. Researchers estimate , then test whether the same SNP is associated with blood pressure. If the SNP influences CRP, the SNP is associated with blood pressure, and the three MR assumptions hold, this provides evidence that CRP may causally influence blood pressure — the central logic of Mendelian Randomization.

2.7 Strengths and Limitations of Mendelian Randomization

Compared with traditional observational studies, MR offers reduced confounding (genotypes are largely independent of environmental factors), reduced reverse causation (disease cannot change inherited variants), the ability to leverage existing large public GWAS summary statistics, and ethical feasibility for questions that could never be studied with a randomized trial.

MR is not a perfect substitute for an RCT, however. Important challenges include weak instruments, population stratification, horizontal pleiotropy, selection bias, and measurement error. For this reason, MR studies typically perform multiple sensitivity analyses to assess robustness — a theme this series returns to repeatedly.

2.8 Looking Ahead

Now that we understand the basic MR framework, we’re ready to estimate causal effects. The next part introduces the Wald Ratio, the first quantitative MR estimator, based on the relationship between SNP-exposure and SNP-outcome effects.

Key takeaways

Mendelian Randomization uses genetic variants as instrumental variables — naturally randomized proxies for exposures.

Every MR study rests on three assumptions: relevance, independence, and exclusion restriction.

Pleiotropy — a variant affecting multiple traits — is a major threat to valid causal inference in MR.

Genetic variants can provide evidence about causal relationships when randomized experiments are impossible.

3 Part 3 — The Wald Ratio and Single-SNP Mendelian Randomization

In Part 2 we established that a valid MR analysis rests on three assumptions: the genetic variant is associated with the exposure, is independent of confounders, and influences the outcome only through the exposure. Now we can ask a practical question:

How do we actually estimate a causal effect using a genetic variant?

The simplest MR estimator is the Wald Ratio.

3.1 From Instrumental Variables to Causal Effects

Recall the basic MR framework: . If the SNP influences the exposure, the exposure influences the outcome, and the MR assumptions hold, the SNP should also be associated with the outcome. This gives us two measurable quantities:

where is the genetic variant, is the exposure, and is the outcome. These two quantities are the foundation of the Wald Ratio.

3.2 The Intuition Behind the Wald Ratio

Imagine a SNP increases BMI by 0.5 units per allele, and the same SNP increases coronary heart disease (CHD) risk by 0.1 units per allele. The natural question is: how much does CHD increase per unit increase in BMI? We estimate this by dividing the SNP-outcome effect by the SNP-exposure effect — exactly what the Wald Ratio does.

3.3 The Wald Ratio Formula

This ratio estimates the causal effect of the exposure on the outcome.

Why does it work? If the true causal effect of on is , and the SNP affects the outcome only through the exposure, then

The SNP acts as a naturally randomized perturbation of the exposure, and the resulting change in the outcome carries information about the causal effect.

3.4 A Numerical Example

Suppose a GWAS reports for SNP → BMI (each additional effect allele increases BMI by 0.20 units) and for SNP → CHD. The Wald estimate is

A one-unit increase in BMI is associated with a 0.40-unit increase in CHD risk, assuming the MR assumptions hold.

This is the same logic behind the earlier CRP–blood pressure example: an MR estimate built from a genetic instrument can differ substantially from the naive observational association, precisely because it strips out confounding and reverse causation.

3.5 Estimating Uncertainty

Every statistical estimate carries uncertainty, and the Wald Ratio is no exception. Given and , the uncertainty in the ratio is commonly approximated with the delta method:

More complete versions incorporate uncertainty in both the numerator and denominator; most MR software handles this automatically.

Once we have and , a 95% confidence interval follows the usual form:

For example, an estimate of with a standard error of gives a 95% CI of roughly . Because zero is not contained in the interval, the estimate is statistically significant at the 5% level.

3.6 The Importance of Instrument Strength

The Wald Ratio relies heavily on its denominator, . If the SNP has only a very weak association with the exposure, the ratio becomes unstable. For example, and gives — a tiny denominator producing an extreme, unreliable estimate. This is known as the weak instrument problem, and using only one SNP typically provides limited statistical power on top of this instability.

3.7 Why Multiple SNPs Are Better

Modern GWAS often identify dozens, hundreds, or even thousands of SNPs associated with a single exposure — 50 for BMI, 100 for cholesterol, 200 for smoking, and so on. Each SNP produces its own Wald Ratio estimate. Rather than relying on any single SNP, we can combine information across many instruments, which increases precision, power, and robustness. The most widely used approach for combining multiple instruments is Inverse Variance Weighted (IVW) Mendelian Randomization, covered in Part 4.

Key takeaways

The Wald Ratio is the simplest MR estimator, using a single SNP as an instrumental variable.

The causal estimate is the SNP-outcome effect divided by the SNP-exposure effect.

Strong instruments are essential; weak instruments produce unstable results.

Modern MR studies combine many SNPs using IVW and related approaches rather than relying on one SNP.

4 Part 4 — Multiple-SNP MR and the Inverse Variance Weighted (IVW) Method

In Part 3 we introduced the Wald Ratio, which uses a single genetic variant as an instrument. Modern MR studies rarely rely on just one SNP — researchers typically use dozens, hundreds, or thousands of variants associated with the exposure. This raises the question:

How can we combine information from multiple genetic instruments into a single, more precise causal estimate?

The most widely used answer is the Inverse Variance Weighted (IVW) Method.

4.1 Why One SNP Is Usually Not Enough

A single SNP usually explains only a tiny proportion of variation in the exposure — one BMI-associated SNP might explain less than 0.1% of BMI variance — which limits precision. Any measurement error in the SNP-exposure or SNP-outcome estimate directly affects the causal estimate, and if that one SNP violates an MR assumption, the entire analysis is biased. For these reasons, modern MR studies use many genetic instruments together.

4.2 Combining Multiple Instruments

Suppose a GWAS identifies five SNPs associated with BMI:

SNP	SNP → BMI ()	SNP → CHD ()	Wald Ratio
SNP1	0.20	0.08	0.40
SNP2	0.15	0.06	0.40
SNP3	0.10	0.05	0.50
SNP4	0.25	0.09	0.36
SNP5	0.18	0.07	0.39

Now we have five separate estimates of the same causal effect, and the challenge is combining them.

4.3 Why Not Simply Average Them?

A simple average treats every SNP equally, but not every SNP is estimated with the same precision. A SNP estimated from a GWAS with a small standard error carries more reliable information than one estimated with a large standard error, so more precise SNPs should contribute more heavily to the combined estimate. This motivates inverse variance weighting, a standard meta-analysis technique: more precise estimates receive larger weights, since smaller variance means greater precision:

Illustration. Two SNPs both estimate a causal effect of 0.40, but SNP1 has variance 0.01 (weight ) and SNP2 has variance 0.04 (weight ). SNP1 receives four times more weight because it’s estimated more precisely.

4.4 The IVW Formula

Given per-SNP Wald Ratio estimates with weights , the IVW estimate is a weighted average:

The numerator is the weighted sum of SNP-specific causal estimates; the denominator is the total weight. Precise instruments get more influence, noisy ones get less.

A numerical example. Suppose three SNPs give:

SNP	Estimate	Variance	Weight
SNP1	0.40	0.01	100
SNP2	0.50	0.02	50
SNP3	0.35	0.05	20

A one-unit increase in the exposure is associated with a 0.424-unit increase in the outcome, assuming the MR assumptions are satisfied.

4.5 An Alternative View of IVW

IVW can also be understood as a weighted regression of on , weighted by inverse variance, with the slope of that regression equal to the causal estimate. This interpretation becomes useful later when we discuss MR-Egger regression, which is the same regression with the intercept unconstrained.

With a single SNP, ; with multiple SNPs, is a weighted average of Wald Ratios. IVW is therefore a natural extension of the Wald Ratio, not a different estimator.

4.6 Why IVW Became the Standard Method

IVW became the most widely used MR estimator because it offers higher statistical power (using many SNPs substantially increases power), greater precision (combining information reduces uncertainty), a straightforward causal interpretation, and computational simplicity.

4.7 The Assumptions Behind IVW

IVW relies on the same three instrumental variable assumptions from Part 2: relevance (each SNP is associated with the exposure), independence (each SNP is independent of confounders), and exclusion restriction (each SNP affects the outcome only through the exposure, and through no other pathway).

4.8 The Problem of Horizontal Pleiotropy

IVW performs well when every instrument is valid. But if some SNPs affect the outcome through pathways other than the exposure — directly, alongside — bias can occur. This is horizontal pleiotropy, one of the major threats to Mendelian Randomization, and Part 7 covers it in detail.

4.9 Fixed Effects and Random Effects IVW

Two versions of IVW are used in practice. Fixed effects IVW assumes all SNPs estimate exactly the same causal effect, attributing differences among SNP estimates entirely to sampling variation. Random effects IVW allows some additional heterogeneity among SNP estimates, and is often preferred when mild pleiotropy or heterogeneity is suspected.

4.10 Strengths and Limitations of IVW

IVW offers high statistical power, increased precision, use of all available instruments, straightforward interpretation, and wide acceptance in genetic epidemiology. Its limitations are that it is sensitive to pleiotropy, requires valid instruments, can become biased when assumptions are violated, and does not automatically detect directional pleiotropy. Because of these limitations, IVW is usually accompanied by additional sensitivity analyses.

4.11 Looking Ahead

IVW is often the starting point of an MR analysis, but researchers must still investigate whether its assumptions are reasonable — especially horizontal pleiotropy. Part 5 addresses a related practical question: where do the SNP-exposure and SNP-outcome estimates actually come from? The answer is Two-Sample Mendelian Randomization.

Key takeaways

Modern MR studies usually use multiple SNPs, each producing a Wald Ratio estimate.

IVW combines these estimates by inverse-variance weighting — more precise instruments get more weight.

IVW can be viewed as a weighted average of Wald Ratios, or as a weighted regression through the origin.

IVW is the standard MR estimator, but is sensitive to horizontal pleiotropy and requires sensitivity analyses.

5 Part 5 — Two-Sample Mendelian Randomization

In Part 4 we combined multiple genetic instruments using IVW, assuming we already had estimates of and for every variant. This raises an obvious question:

Where do these estimates actually come from?

In modern MR studies, they typically come from two separate Genome-Wide Association Studies (GWAS). This approach is called Two-Sample Mendelian Randomization, and it is the standard framework for contemporary MR analyses.

5.1 Revisiting the MR Framework

To estimate a causal effect we need SNP-exposure and SNP-outcome associations. The natural question is:

Must these associations come from the same individuals?

The answer is no — and this is one of the major strengths of modern MR.

5.2 One-Sample Mendelian Randomization

Historically, MR was performed within a single dataset, measuring genotype, exposure, and outcome in the same individuals:

Person	Genotype	BMI	CHD
1	AA	24	No
2	AG	28	Yes
3	GG	31	Yes

This is one-sample MR. Its advantages are complete data on all variables in the same individuals, flexible modeling with access to individual-level data, and the ability to adjust directly for covariates. Its limitations are the expense of collecting genotype, exposure, and outcome data on the same individuals, the resulting small sample sizes and limited power compared to large GWAS consortia, and restricted access to individual-level genetic data. These limitations motivated two-sample MR.

5.3 Two-Sample Mendelian Randomization

Two-sample MR uses summary statistics from two separate GWAS: an exposure GWAS provides associations, and an outcome GWAS provides associations. The participants in the two studies do not need to overlap.

Exposure GWAS example (BMI):

SNP	Beta ()
rs1	0.12
rs2	0.08
rs3	0.15

Outcome GWAS example (same SNPs, a disease outcome):

SNP	Beta ()
rs1	0.05
rs2	0.03
rs3	0.06

Researchers no longer need individual-level genotype, exposure, and outcome data in the same participants — publicly available GWAS summary statistics can simply be combined, which dramatically increases statistical power. Thousands of GWAS summary statistic datasets are now available through resources such as UK Biobank, FinnGen, the GIANT Consortium, CARDIoGRAMplusC4D, and the Psychiatric Genomics Consortium.

A practical example. To study , we might use a BMI GWAS with and a CHD GWAS with ; the participants don’t need to overlap, and we simply extract SNP associations from both.

5.4 The Assumption of Population Similarity

Although the two samples need not contain the same individuals, they should ideally represent similar populations. A European-ancestry exposure GWAS paired with a European-ancestry outcome GWAS is fine; pairing a European-ancestry exposure GWAS with an East Asian-ancestry outcome GWAS is potentially problematic, since ancestry differences can affect SNP effects and introduce bias.

5.5 Sample Overlap

Can the same individuals appear in both GWAS? Yes — but excessive overlap can sometimes create bias. No overlap (fully independent participant sets) is the ideal situation. Partial overlap, common in modern biobanks, is often acceptable, especially with strong instruments. Complete overlap makes the analysis behave more like one-sample MR, and bias can become more problematic when instruments are weak.

An important property of two-sample MR is that weak instrument bias behaves differently than in one-sample MR: in one-sample MR, weak instruments tend to bias estimates toward the observational association, while in two-sample MR they generally bias estimates toward the null — a property that makes two-sample MR particularly attractive.

5.6 Why Summary Statistics Are Sufficient

One of the most remarkable features of two-sample MR is that individual-level data are not required — for each SNP we only need an effect size, standard error, effect allele, and other allele, all of which are routinely reported in GWAS summary statistics. This is why so many MR studies can now be conducted with freely available public data.

5.7 The Typical Two-Sample MR Workflow

Select genetic instruments from the exposure GWAS.
Extract SNP-outcome associations from the outcome GWAS.
Harmonize alleles between the two datasets.
Calculate SNP-specific Wald Ratios.
Combine estimates using IVW.
Perform sensitivity analyses.

5.8 Strengths and Limitations of Two-Sample MR

Its strengths are access to extremely large sample sizes, publicly available data, increased statistical power, and broad applicability across thousands of exposure–outcome pairs. Its limitations include the need for population similarity between the two GWAS, the risk of sample overlap bias, the practical challenge of harmonization (Part 6), and the fact that pleiotropy can still violate MR assumptions regardless of sample structure.

5.9 Looking Ahead

Before SNP associations from two different studies can be combined, we must verify that effect alleles, reference alleles, strand orientation, and palindromic SNPs are all correctly aligned. This process — harmonization — is one of the most important practical steps in MR, and is the subject of Part 6.

Key takeaways

Two-sample MR uses separate GWAS for the exposure and the outcome; participants need not overlap.

Modern MR relies primarily on publicly available summary statistics, enabling very large sample sizes.

Exposure and outcome GWAS should ideally come from similar populations; excessive sample overlap can introduce bias.

Two-sample MR has become the standard framework for contemporary MR studies.

6 Part 6 — Harmonization of Exposure and Outcome Data

In Part 5 we saw that two-sample MR combines information from separate exposure and outcome GWAS. To estimate a causal effect we need SNP-exposure and SNP-outcome effects for the same genetic variants — but before those effects can be combined, we must ensure they refer to the same allele. This process is called harmonization.

Harmonization looks like a simple data-cleaning step, but it is one of the most important stages of an MR analysis: a mistake here can completely reverse the interpretation of a causal effect.

6.1 Why Harmonization Is Necessary

Recall the Wald Ratio, . This calculation assumes both effect estimates refer to exactly the same allele. If they don’t, the causal estimate is wrong.

Every SNP has two alleles — for example rs123 might be A/G, rs456 might be C/T. One allele is designated the effect allele (the allele the GWAS effect size is reported for); the other is the non-effect or other allele. If a GWAS reports SNP rs123 with effect allele A and beta 0.10, that means each additional copy of A increases the trait by 0.10 units.

6.1.1 Why Allele Alignment Matters

Suppose the exposure GWAS reports rs123 with effect allele A and beta 0.10, while the outcome GWAS reports the same SNP with effect allele G and beta . At first glance these look like different effects — but the two studies are simply using different reference alleles. The outcome estimate must be converted so both studies refer to the same allele before we can compute a Wald Ratio.

Correct harmonization. Exposure GWAS: EA = A, OA = G, beta = 0.10. Outcome GWAS: EA = G, OA = A, beta = . Since the exposure effect allele (A) is the outcome’s other allele, we flip the sign of the outcome beta and align to A: outcome becomes EA = A, beta = . Now both datasets refer to allele A.

What happens if we skip this step? If we naively divide by , we get — a negative causal effect. After correct harmonization, the ratio becomes . A simple allele mismatch can completely reverse the biological interpretation of a study.

6.2 DNA Strand Orientation

DNA has two complementary strands, with pairing rules A–T and C–G, so a SNP can be reported on either strand. A/G on the forward strand is T/C on the reverse strand — both represent the same underlying variant, which creates an additional harmonization challenge. If one GWAS reports rs123 as A/G and another reports it as T/C, these may be the same SNP viewed from opposite strands, and harmonization software must account for this.

Strand flipping resolves this by translating alleles: A↔︎T and C↔︎G, ensuring exposure and outcome datasets refer to the same biological allele.

6.3 Palindromic SNPs

Some SNPs create a special challenge: those with allele pairs A/T or C/G are called palindromic, because flipping the strand of A/T gives T/A — which looks identical from the opposite direction. This creates ambiguity: given an exposure GWAS reporting EA = A, OA = T and an outcome GWAS reporting EA = T, OA = A for the same palindromic SNP, we cannot immediately tell whether the alleles are correctly aligned.

Using allele frequencies to resolve ambiguity. If the exposure GWAS reports an effect allele frequency (EAF) of 0.10 and the outcome GWAS reports 0.11, the similar frequencies suggest the same allele is being referenced. If instead the frequencies were 0.10 versus 0.90, that would indicate a strand mismatch. Many MR software packages use allele frequency comparisons like this during harmonization — though when EAF is close to 0.50, orientation cannot be inferred reliably, and these ambiguous palindromic SNPs are usually removed from the analysis.

6.4 Harmonization Workflow

A typical harmonization procedure follows five steps:

Match SNP identifiers — confirm the same rsID exists in both datasets.
Compare effect alleles — determine whether exposure and outcome datasets use the same allele.
Flip effect sizes if necessary — reverse the sign of beta when alleles differ.
Check strand orientation — resolve forward/reverse strand differences.
Remove ambiguous SNPs — exclude palindromic SNPs that can’t be confidently aligned.

6.5 Harmonization in TwoSampleMR

The TwoSampleMR package automates this entire process:

Sys.getenv("OPENGWAS_JWT")

'eyJhbGciOiJSUzI1NiIsImtpZCI6ImFwaS1qd3QiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJhcGkub3Blbmd3YXMuaW8iLCJhdWQiOiJhcGkub3Blbmd3YXMuaW8iLCJzdWIiOiJuaXZlZGl0YS5ob21lQGdtYWlsLmNvbSIsImlhdCI6MTc4NTA5NTYzMCwiZXhwIjoxNzg2MzA1MjMwfQ.ncPxT99pTGyra1p3Yv0cc-vQgRaIBT7lGI2F2LbSdKdXNnZYNZEy0L-EzM8fMtkVy_zg_CqnJ4wYWhbD-kSGtq8GtlnDh7RxJTy5DOWo3dhOBn6gM4WuyWy87Mc_k8vlLAPcO1UCimsiv2St7lN5d1I1vyQfOm8c2Oq2oZp3P0C6tQG6smD5CdcH8i31OyLbIl-15cuLYvzaGXMdUbPDWodv4Rx8urYjsoFGVWbIF-MNilg4g4-y7wmd7I4bIzObAp_KFT0ty1w0I9-YKhcSJH2ERCOcxDfEJgtNa8R9txCrRee4rNldJo2STE0UCpnVE3O84R34pgil5ImORmYp6w'

#install.packages("remotes")
#remotes::install_github("MRCIEU/TwoSampleMR")
library(TwoSampleMR)

exposure_dat <- extract_instruments(outcomes = "ieu-a-2")

outcome_dat <- extract_outcome_data(
  snps = exposure_dat$SNP,
  outcomes = "ieu-a-7"
)

harmonised_dat <- harmonise_data(
  exposure_dat,
  outcome_dat
)

TwoSampleMR version 0.7.9 


 [>] New authentication requirements: https://mrcieu.github.io/ieugwasr/articles/guide.html#authentication.

 [>] Major upgrades to our servers completed to improve service and stability.

 [>] We need your help to shape our emerging roadmap!

     Please take 2 minutes to give us feedback -

     https://forms.office.com/e/eSr7EFAfCG

Extracting data for 78 SNP(s) from 1 GWAS(s)

Querying id chunk 1 of 1

Querying variant chunk 1 of 2

Querying variant chunk 2 of 2

Harmonising Body mass index || id:ieu-a-2 (ieu-a-2) and Coronary heart disease || id:ieu-a-7 (ieu-a-7)

Removing the following SNPs for being palindromic with intermediate allele frequencies:
rs1558902

This function aligns alleles, flips effect sizes, resolves strand issues, and removes problematic SNPs.

Example workflow:

exposure_dat <- extract_instruments(
  outcomes = "ieu-a-2"
)

outcome_dat <- extract_outcome_data(
  snps = exposure_dat$SNP,
  outcomes = "ieu-a-7"
)

harmonised_dat <- harmonise_data(
  exposure_dat,
  outcome_dat
)

Extracting data for 78 SNP(s) from 1 GWAS(s)

Querying id chunk 1 of 1

Querying variant chunk 1 of 2

Querying variant chunk 2 of 2

Harmonising Body mass index || id:ieu-a-2 (ieu-a-2) and Coronary heart disease || id:ieu-a-7 (ieu-a-7)

Removing the following SNPs for being palindromic with intermediate allele frequencies:
rs1558902

After harmonization, effect alleles match, beta estimates are aligned, and Wald Ratios can be calculated safely.

6.6 Common Harmonization Errors

Several mistakes recur in MR studies: allele mismatch (exposure and outcome effects referring to different alleles), strand mismatch (forward and reverse strands not aligned), incorrect SNP matching (the wrong variants merged together), and ignoring palindromic SNPs (retaining ambiguous SNPs that should have been excluded). Any of these can produce incorrect causal estimates.

After harmonization, it’s good practice to inspect the number of SNPs retained versus removed, allele frequencies, strand flips performed, and palindromic SNP exclusions — unexpected patterns here often signal problems in the input data.

6.7 Why Harmonization Is So Important

The statistical methods used in MR can be sophisticated, but even the most advanced estimator cannot correct for incorrectly aligned alleles. A perfectly implemented IVW analysis run on improperly harmonized data will still produce misleading results.

Harmonization is not a minor preprocessing step. It is a critical component of Mendelian Randomization.

6.8 Looking Ahead

Now that exposure and outcome datasets are aligned, we can examine one of the most important threats to MR: horizontal pleiotropy, which occurs when genetic variants influence the outcome through pathways other than the exposure, violating MR’s core assumptions.

Key takeaways

Harmonization ensures exposure and outcome effect estimates refer to the same allele before combining them.

Getting harmonization wrong can silently reverse the sign of a causal estimate.

Strand orientation and palindromic SNPs are the two trickiest sources of ambiguity.

harmonise_data() in TwoSampleMR automates allele alignment, strand resolution, and exclusion of unresolvable SNPs.

7 Part 7 — Horizontal Pleiotropy

Now that exposure and outcome datasets can be correctly aligned (Part 6), we turn to one of the biggest threats to Mendelian Randomization: horizontal pleiotropy.

In genetics, pleiotropy — a single variant influencing more than one phenotype — is extremely common. A single SNP might influence BMI, type 2 diabetes, blood pressure, and cholesterol simultaneously, by affecting appetite, insulin resistance, and physical activity all at once. Pleiotropy is normal biology; the real question is whether it violates MR’s assumptions.

7.1 Revisiting the MR Framework

The ideal MR model is — for example, , with no alternative pathways. Under these conditions, MR can validly estimate a causal effect.

The exclusion restriction assumption states that the genetic variant influences the outcome only through the exposure:

This is exactly where pleiotropy becomes important.

7.2 Two Types of Pleiotropy

7.2.1 Vertical Pleiotropy

Vertical pleiotropy occurs when a SNP affects downstream traits through the exposure — for example, . The SNP affects CHD through a biological chain of events mediated entirely by the exposure pathway, so this does not violate the exclusion restriction. This is exactly the type of causal pathway MR is designed to capture, and vertical pleiotropy is generally not considered a problem for MR.

7.2.2 Horizontal Pleiotropy

Horizontal pleiotropy occurs when a SNP influences the outcome through a pathway other than the exposure. For example, if and, simultaneously, directly, and we’re studying , the direct SNP effect creates an alternative pathway that violates the exclusion restriction.

The outcome is now influenced by two routes — through the exposure, and through an independent pleiotropic pathway — making it impossible to determine how much of the SNP-outcome association is truly mediated by the exposure. The observed SNP-outcome effect becomes:

A numerical example. Suppose (SNP → BMI) and the true causal effect (BMI → CHD), so the expected SNP-outcome effect via the exposure alone is . Now suppose the SNP also directly affects CHD with a pleiotropic effect of . The observed SNP-outcome effect becomes , and the Wald Ratio is — far from the true causal effect of , purely because of horizontal pleiotropy.

7.3 Balanced vs. Directional Pleiotropy

Balanced pleiotropy occurs when some SNPs have positive pleiotropic effects and others have negative ones that roughly cancel out — e.g. averaging to approximately zero. In this case, IVW estimates may remain relatively unbiased.

Directional pleiotropy occurs when pleiotropic effects consistently point in the same direction — e.g. , averaging to a clearly positive value. This systematically biases the MR estimate and is much more problematic than balanced pleiotropy.

A valid instrument satisfies all MR assumptions ( only); an invalid instrument violates at least one, most commonly through horizontal pleiotropy.

7.4 Why Pleiotropy Is So Common

Modern GWAS have repeatedly shown that many genetic variants influence multiple traits simultaneously — obesity, diabetes, blood pressure, and lipids, for instance, often share genetic architecture. This widespread genetic sharing is sometimes called biological pleiotropy, and as a result:

Pleiotropy should generally be expected rather than considered unusual.

7.5 Detecting Pleiotropy

Pleiotropy cannot always be observed directly, but several statistical methods attempt to detect it: MR-Egger regression, Cochran’s Q test, MR-PRESSO, leave-one-out analysis, weighted median estimation, and weighted mode estimation. These methods form the basis of MR sensitivity analyses, covered in the following parts.

Heterogeneity as a warning sign. If multiple SNPs are all valid instruments, they should estimate approximately the same causal effect — e.g. Wald Ratios of 0.32, 0.29, 0.34, 0.31 across four SNPs are highly consistent. But if one SNP produces a wildly different estimate (say 1.80 among otherwise consistent 0.3-ish values), that may indicate horizontal pleiotropy, a data problem, or an invalid instrument. Large heterogeneity often motivates further investigation.

7.6 Why We Need Robust MR Methods

IVW assumes every instrument is valid. In reality, some SNPs may be pleiotropic, invalid, or violate MR assumptions outright. Researchers therefore developed robust methods that remain informative even when some instruments are invalid — MR-Egger, weighted median, and weighted mode — each making different assumptions about the nature of the pleiotropy present. These are the subjects of Parts 8–10.

7.7 The Big Picture

Pleiotropy is not merely a technical nuisance — it is one of the central challenges in causal inference using genetics. The modern MR toolkit exists largely because some degree of pleiotropy is almost inevitable. The goal is not necessarily to eliminate pleiotropy, but to detect it, quantify it, and assess whether it meaningfully changes the causal conclusions.

7.8 Looking Ahead

The most famous method for addressing horizontal pleiotropy is MR-Egger regression, which extends the IVW framework to allow for directional pleiotropy and provides a statistical test for pleiotropic bias. That’s the subject of Part 8.

Key takeaways

Pleiotropy occurs when a genetic variant influences multiple traits.

Vertical pleiotropy generally does not violate MR assumptions; horizontal pleiotropy does.

Balanced pleiotropy is usually less problematic than directional pleiotropy.

Detecting pleiotropy — rather than assuming it away — is a central goal of MR sensitivity analysis.

8 Part 8 — MR-Egger Regression: Detecting and Adjusting for Directional Pleiotropy

Part 7 established that horizontal pleiotropy is one of the biggest threats to Mendelian Randomization. IVW assumes every genetic instrument is valid, but if some SNPs influence the outcome through pathways other than the exposure, the IVW estimate can become biased. This raises a natural question:

Can we detect whether pleiotropy is affecting our MR analysis?

MR-Egger regression was developed to answer this. Unlike IVW, MR-Egger allows genetic variants to have direct effects on the outcome, and provides a statistical framework for detecting directional pleiotropy.

8.1 Why IVW Can Fail

IVW essentially fits a weighted regression through the origin:

The intercept is fixed at zero, which means IVW assumes there is no average pleiotropic effect across SNPs. If this is violated, the causal estimate may be biased.

8.2 The Idea Behind MR-Egger

MR-Egger extends the IVW regression by allowing a non-zero intercept:

where is the intercept, is the causal estimate, and is random error. The intercept term is the key innovation.

8.3 Why the Intercept Matters

The intercept represents the average pleiotropic effect across all instruments. If , there is no evidence of directional pleiotropy; if , there is evidence that SNPs are affecting the outcome through pathways other than the exposure. The intercept becomes a diagnostic tool for pleiotropy itself.

Visualizing IVW vs. MR-Egger. Plotting on the x-axis and on the y-axis, each SNP becomes a point. IVW forces the regression line through the origin ; MR-Egger allows the line to cross the y-axis at , giving it the flexibility to account for directional pleiotropy.

8.4 Slope and Intercept

The slope of the MR-Egger regression, , is interpreted just like the IVW estimate — for example, suggests a one-unit increase in the exposure causes a 0.40-unit increase in the outcome, assuming the MR-Egger assumptions hold.

The intercept is often more interesting than the slope. Suppose : this suggests that, on average, SNPs have positive direct effects on the outcome not mediated through the exposure — evidence for directional pleiotropy. Researchers formally test against ; a significant result suggests pleiotropic bias.

Recall from Part 7 that under balanced pleiotropy the average pleiotropic effect is approximately zero, so the MR-Egger intercept should also be approximately zero. Under directional pleiotropy, the average effect is non-zero, and the intercept should be significantly different from zero — exactly what MR-Egger is designed to detect.

8.5 The InSIDE Assumption

MR-Egger introduces a new assumption: Instrument Strength Independent of Direct Effect, or InSIDE. It states:

The strength of the SNP-exposure association is independent of the direct pleiotropic effect.

In other words, strong instruments should not systematically have larger pleiotropic effects than weak instruments — mathematically, direct effects and instrument strengths should be uncorrelated.

This matters because the MR-Egger slope can remain unbiased even when every SNP is pleiotropic — but only if InSIDE holds. If InSIDE is violated, both the intercept and the slope may become misleading, and unfortunately InSIDE usually cannot be tested directly.

8.6 Comparing IVW and MR-Egger

Feature	IVW	MR-Egger
Allows non-zero intercept	No	Yes
Detects directional pleiotropy	No	Yes
Higher statistical power	Yes	No
More robust to pleiotropy	No	Yes

8.7 When Should MR-Egger Be Used?

MR-Egger is particularly useful when many instruments are available, pleiotropy is suspected, or researchers want a dedicated sensitivity analysis for the IVW assumptions. It has become one of the standard robustness checks in modern MR.

8.8 Strengths and Limitations of MR-Egger

Its strengths are that the intercept test provides direct evidence of pleiotropic bias, it can remain informative even when all SNPs are somewhat pleiotropic, and it’s one of the most widely used sensitivity analyses in MR. Its limitations are lower statistical power (wider confidence intervals than IVW), heavy dependence on the untestable InSIDE assumption, sensitivity to a small number of outlying SNPs, and a general requirement for a large number of instruments to perform well.

8.9 Why MR-Egger Is Usually a Sensitivity Analysis

Most MR studies report IVW, MR-Egger, weighted median, and weighted mode side by side, then compare results. If all methods point in the same direction, confidence in the causal conclusion increases; if they disagree substantially, further investigation is warranted.

8.10 The Big Picture

MR-Egger exists because researchers recognized that pleiotropy is widespread in genetics. Rather than assuming it away, MR-Egger attempts to detect and adjust for it — one of the most important methodological developments in modern MR.

8.11 Looking Ahead

Although robust to some forms of pleiotropy, MR-Egger often has low statistical power. Part 9 introduces the weighted median estimator, which can provide consistent causal estimates even when up to half of the instruments are invalid.

Key takeaways

MR-Egger extends IVW by allowing a non-zero intercept.

The intercept tests for directional pleiotropy; the slope estimates the causal effect.

A significant intercept suggests pleiotropic bias in the IVW estimate.

MR-Egger relies on the untestable InSIDE assumption and typically has lower power than IVW.

It is normally used as a sensitivity analysis rather than a primary estimator.

9 Part 9 — Weighted Median Mendelian Randomization

Part 8 introduced MR-Egger, a method that can detect and partially account for directional pleiotropy. MR-Egger was an important advance, but it has real limitations: lower statistical power, wider confidence intervals, dependence on the untestable InSIDE assumption, and sensitivity to outlying SNPs.

Researchers therefore sought a method that stays robust when some genetic instruments are invalid, while retaining reasonable statistical power. The weighted median estimator is one of the most successful answers, and has become one of the most widely used sensitivity analyses in MR.

9.1 Why Do We Need Another Method?

IVW performs well when all instruments are valid, but suppose we have 100 SNPs and 10, 20, or 30 of them are invalid. IVW can become biased, because every SNP contributes to the final estimate — even a relatively small number of problematic instruments can distort the result. Researchers wanted a method that could tolerate some invalid instruments.

9.2 The Core Idea

The weighted median estimator rests on a simple observation:

If most instruments are valid, the middle estimate should still be reliable.

Rather than averaging SNP estimates, the weighted median focuses on the center of the distribution, which makes it much less sensitive to extreme values and invalid instruments.

Revisiting Wald Ratios. Suppose five SNPs give estimates of 0.30, 0.35, 0.40, 0.42, and 2.50. The last value is dramatically larger than the rest — possibly due to horizontal pleiotropy, data errors, a weak instrument, or a genuinely invalid instrument. IVW will be pulled toward this outlier; the weighted median is far less affected.

9.3 What Is a Median?

A median is the middle value in an ordered list. For the median is . Unlike a mean, the median resists extreme values: for the mean is , but the median is still .

In MR, we don’t simply take the plain median of SNP estimates — each SNP has a different level of precision, and more precise SNPs should contribute more information. So we use a weighted median: the estimate where 50% of the total weight lies on either side.

A simple example.

SNP	Estimate	Weight
SNP1	0.20	10
SNP2	0.25	20
SNP3	0.30	30
SNP4	0.35	25
SNP5	1.50	15

Total weight is 100; the weighted median corresponds to the point where cumulative weight reaches 50. SNP5 has an extreme estimate, but because it carries only 15% of the total weight, it has limited influence on the final result.

9.4 The 50% Rule

The weighted median estimator has a remarkable property: it produces a consistent causal estimate provided that

More than 50% of the total weight comes from valid instruments.

This is often called the 50% valid instrument assumption. Even if 40% of SNP weight comes from invalid instruments and 60% from valid ones, the weighted median can still recover the correct causal effect — substantially more robust than IVW, which is biased by any invalid instrument contributing to the average.

Comparing IVW and weighted median. Suppose 90 SNPs estimate 0.30 and 10 pleiotropic SNPs estimate 2.00. IVW will be pulled upward because all SNPs contribute to the average, but the weighted median remains close to 0.30, provided the majority of weight comes from valid instruments.

9.5 Comparing IVW, MR-Egger, and Weighted Median

Feature	IVW	MR-Egger	Weighted Median
All instruments valid	Required	Not required	Not required
Detects pleiotropy	No	Yes	No
Robust to invalid SNPs	Limited	Moderate	Strong
Statistical power	High	Low	Moderate

Scenario 1. IVW = 0.52, MR-Egger = 0.40, Weighted Median = 0.42: IVW is noticeably larger while MR-Egger and the weighted median agree — suggestive of some pleiotropic bias in IVW.

Scenario 2. IVW = 0.45, MR-Egger = 0.44, Weighted Median = 0.43: all methods agree closely, providing little evidence of major pleiotropic bias and strong support for the causal effect.

The weighted median is particularly useful when a small number of SNPs produce extreme estimates — for instance, four SNPs clustered around 0.30 alongside one SNP at 5.00. IVW would be pulled toward that outlier; the weighted median stays close to 0.30.

9.6 When Does Weighted Median Fail?

It’s not perfect. Problems occur when more than 50% of the weight comes from invalid instruments, when most SNPs share similar pleiotropic effects, or when strong directional pleiotropy affects the majority of instruments. In these cases, the weighted median can also become biased.

9.7 Why Weighted Median Became Popular

The weighted median occupies an attractive middle ground: more robust than IVW, and more precise with greater statistical power than MR-Egger. As a result, it’s now routinely included in MR sensitivity analyses. A typical MR study might report IVW = 0.48, MR-Egger = 0.39, Weighted Median = 0.41; researchers examine the direction, magnitude, and confidence intervals of each, and consistency across methods strengthens confidence in causal conclusions.

9.8 Looking Ahead

The weighted median still assumes that more than 50% of instrument weight is valid. Part 10 introduces the weighted mode estimator, which takes a different perspective entirely:

The largest cluster of SNP estimates is most likely to represent the true causal effect.

Key takeaways

The weighted median is a robust MR method based on the median, weighted by precision, of SNP-specific causal estimates.

It remains consistent if more than 50% of the total weight comes from valid instruments.

It is less sensitive to outliers than IVW, and generally has more power than MR-Egger.

It cannot directly detect pleiotropy, and is typically reported alongside IVW and MR-Egger as a sensitivity analysis.

10 Part 10 — Weighted Mode Mendelian Randomization

Part 9 introduced the weighted median estimator, robust as long as more than 50% of total instrument weight comes from valid SNPs. Researchers asked a natural follow-up question:

What if fewer than 50% of the instruments are valid?

In some situations the majority of SNPs may not be valid instruments, yet a cluster of SNPs may still identify the true causal effect. This idea motivated mode-based Mendelian Randomization, commonly known as the weighted mode estimator.

10.1 Why Another Robust Method?

Recall the assumptions behind the methods so far: IVW requires all instruments to be valid; MR-Egger allows pleiotropy but relies on InSIDE; weighted median requires more than 50% of total weight to be valid. But what if only 30% or 40% of SNPs are valid — what if valid instruments simply don’t form a majority? Can we still recover the correct causal effect? The weighted mode estimator attempts to answer yes.

10.2 The Core Idea

The weighted mode estimator rests on a simple intuition:

Valid instruments should estimate approximately the same causal effect.

If multiple SNPs are valid, their Wald Ratio estimates should cluster around the true causal effect; invalid instruments may scatter in many different directions. Therefore:

The largest cluster of SNP estimates is likely to represent the true causal effect.

This cluster defines the mode.

10.3 What Is a Mode?

In statistics, the mode is the most frequently occurring value — for the mode is , since it appears most often. The weighted mode estimator extends this idea to clusters of MR estimates rather than single repeated values.

From Wald Ratios to clusters. Suppose SNP-specific estimates are 0.30, 0.31, 0.29, 0.32, 1.50, 1.40, and . Three distinct clusters emerge: Cluster A = , Cluster B = , Cluster C = . The largest cluster is A, so the weighted mode estimate is approximately .

10.4 The Plurality-Valid Assumption

Weighted median requires that more than 50% of total weight come from valid instruments. Weighted mode requires only that the largest cluster of SNPs consist of valid instruments — the plurality-valid assumption.

Majority vs. plurality. A majority means more than 50% — e.g. 60% valid SNPs versus 40% invalid. A plurality means the largest group, but not necessarily more than 50% — e.g. 40% valid SNPs, 35% “invalid type A,” and 25% “invalid type B.” Here valid instruments don’t form a majority, but they do form the largest cluster, and the weighted mode estimator can still work.

This is powerful: the weighted mode can succeed in situations where both IVW and weighted median fail, because it requires only the largest cluster to be valid — a much weaker and often more realistic assumption.

10.5 Comparing the Four Main MR Estimators

Feature	IVW	MR-Egger	Weighted Median	Weighted Mode
Requires all SNPs valid	Yes	No	No	No
Detects pleiotropy	No	Yes	No	No
Robust to invalid instruments	Limited	Moderate	Strong	Strong
Statistical power	High	Lower	Moderate	Moderate
Key assumption	All valid	InSIDE	>50% valid weight	Largest cluster valid

Example. IVW = 0.55, MR-Egger = 0.38, Weighted Median = 0.41, Weighted Mode = 0.40: IVW is noticeably larger while the three robust methods agree closely — a pattern that may indicate pleiotropic bias affecting IVW. When all four methods agree closely instead, that provides strong support for a stable causal estimate.

10.6 Why Weighted Mode Is Useful — and Its Limitations

Weighted mode tolerates invalid instruments, doesn’t require the InSIDE assumption, and doesn’t require a majority of valid SNPs — it can remain informative when other methods struggle. Its limitations: results are sensitive to how clusters are identified, it typically has lower precision (larger standard errors) than IVW, it is less familiar to many researchers than IVW or MR-Egger, and it performs poorly if SNP estimates form a broad continuous distribution rather than clear clusters.

10.7 Why Weighted Mode Is Usually a Sensitivity Analysis

As with MR-Egger and weighted median, most studies report weighted mode alongside IVW, MR-Egger, and weighted median rather than as a standalone primary analysis. Agreement across all four increases confidence that an observed causal effect is not being driven by violations of MR assumptions.

10.8 The Robust MR Toolbox

At this point we have four major estimators: IVW (primary analysis), MR-Egger (detects directional pleiotropy), weighted median (robust if >50% of weight is valid), and weighted mode (robust if the largest cluster is valid). Together these form the foundation of modern MR sensitivity analysis.

Even when some instruments are invalid, the true causal signal often emerges as the dominant pattern among the valid instruments.

10.9 Looking Ahead

Having covered the four most widely used MR estimators, Part 11 turns to how researchers combine them systematically to evaluate the robustness of a causal conclusion — including Cochran’s Q statistic, heterogeneity testing, and leave-one-out analysis.

Key takeaways

The weighted mode estimator identifies the largest cluster of SNP-specific causal estimates.

It relies on the plurality-valid assumption, weaker than the weighted median’s 50% rule.

It does not require the InSIDE assumption and can remain consistent even when fewer than 50% of instruments are valid.

It is one of the most robust MR estimators, typically used alongside IVW, MR-Egger, and weighted median.

11 Part 11 — Sensitivity Analyses in Mendelian Randomization

Parts 4 through 10 introduced four major MR estimators: IVW, MR-Egger, weighted median, and weighted mode. Each attempts to estimate the causal effect of an exposure on an outcome, but each relies on different assumptions. This raises the obvious question:

Which estimate should we trust?

In practice, MR rarely relies on a single estimator. Instead, researchers run a battery of sensitivity analyses to evaluate whether their conclusions are robust to violations of MR’s assumptions — among the most important components of a modern MR study.

11.1 Why Sensitivity Analyses Are Necessary

Recall the three core MR assumptions: relevance, independence, and exclusion restriction. Unfortunately, we cannot directly verify all of them — some SNPs may be invalid instruments, horizontal pleiotropy may be present, and measurement errors may occur. As a result:

A single MR estimate is rarely sufficient.

Sensitivity analyses help determine whether conclusions remain stable under different assumptions. The goal is not necessarily to prove that an estimate is correct, but to ask: how sensitive is the result to potential violations of MR assumptions? If multiple methods agree, confidence increases; if they disagree substantially, caution is warranted.

11.2 The Four Core MR Estimators, Side by Side

Most MR studies report all four: IVW as the primary analysis, MR-Egger to test for directional pleiotropy, weighted median as robust when more than 50% of weight is valid, and weighted mode as robust when the largest cluster of SNPs is valid. Together they provide complementary perspectives on the causal effect.

Scenario 1 — strong agreement. IVW = 0.42, MR-Egger = 0.40, Weighted Median = 0.43, Weighted Mode = 0.41. All methods point the same direction with similar effect sizes — little evidence of major violations, strengthening confidence in the conclusion.

Scenario 2 — moderate differences. IVW = 0.48, MR-Egger = 0.39, Weighted Median = 0.42, Weighted Mode = 0.40. Estimates differ somewhat but direction is consistent — possible mild pleiotropy; the causal conclusion may still be reasonable, but with greater uncertainty.

Scenario 3 — strong disagreement. IVW = 0.70, MR-Egger = 0.10, Weighted Median = 0.15, Weighted Mode = 0.12. Large disagreement suggests a strong possibility of pleiotropic bias and likely violation of IVW’s assumptions — researchers should investigate further before making causal claims.

11.3 Heterogeneity in Mendelian Randomization

If every SNP is a valid instrument, each should estimate approximately the same causal effect — e.g. 0.31, 0.29, 0.33, 0.30 across four SNPs. If instead estimates differ dramatically — 0.31, 0.29, 1.75, — this variability is called heterogeneity, and it may indicate horizontal pleiotropy, invalid instruments, data quality problems, population differences, or outlying SNPs. Heterogeneity is an important warning signal.

11.3.1 Cochran’s Q Statistic

The most common heterogeneity test in MR is Cochran’s Q, which asks: do SNP-specific causal estimates vary more than expected by chance? The null hypothesis is that all SNPs estimate the same causal effect; the alternative is that at least some differ.

A Q-test p-value of suggests little evidence of heterogeneity; a p-value suggests strong evidence of heterogeneity, prompting investigation of possible pleiotropy. Importantly, heterogeneity is a warning sign, not proof of invalidity — possible explanations include true biological complexity, measurement error, pleiotropy, and population structure. It should trigger further investigation rather than immediate rejection of the analysis.

11.3.2 The MR-Egger Intercept Test

Recall from Part 8 that MR-Egger estimates , and the intercept measures average directional pleiotropy. The null hypothesis is (no directional pleiotropy); the alternative is (directional pleiotropy present). For example, an intercept of with shows no evidence of pleiotropy, while an intercept of with suggests pleiotropic bias.

11.3.3 Leave-One-Out Analysis

Another popular sensitivity analysis is the leave-one-out procedure: given 20 SNPs, run IVW using all 20, then remove SNP1 and re-run, remove SNP2 and re-run, and so on until every SNP has been excluded once. This identifies influential SNPs — for example, if the full analysis gives but removing SNP7 drops the estimate to , SNP7 is clearly driving the result and deserves closer inspection.

The ideal pattern is that all leave-one-out estimates are similar; the problematic pattern is that one or two SNPs dramatically alter the estimate, potentially signaling horizontal pleiotropy, data problems, or invalid instruments.

11.3.4 Funnel Plots, Scatter Plots, and Forest Plots

A funnel plot plots each SNP’s causal estimate against its precision. Under balanced pleiotropy the plot should be symmetric; under directional pleiotropy it can become asymmetric.

A scatter plot places on the x-axis and on the y-axis, with each point a SNP, and IVW/MR-Egger regression lines overlaid — useful for visualizing instrument consistency, outliers, and pleiotropic effects.

A forest plot displays individual SNP estimates, their confidence intervals, and the combined MR estimate, helping identify outlying SNPs, instrument consistency, and heterogeneity. Forest plots are widely used in published MR studies.

11.4 Triangulation of Evidence

One of the most important ideas in causal inference is triangulation: rather than relying on a single result, researchers examine IVW, MR-Egger, weighted median, weighted mode, heterogeneity tests, and leave-one-out analyses together. When all lines of evidence support the same conclusion, confidence increases substantially.

A reassuring example. IVW = 0.45, MR-Egger = 0.42, Weighted Median = 0.43, Weighted Mode = 0.41; MR-Egger intercept ; Cochran’s Q ; leave-one-out stable. Interpretation: little evidence of pleiotropy or heterogeneity, strong support for a causal effect.

A concerning example. IVW = 0.80, MR-Egger = 0.10, Weighted Median = 0.18, Weighted Mode = 0.15; MR-Egger intercept ; Cochran’s Q ; one SNP drives the leave-one-out result. Interpretation: strong evidence of pleiotropy, the IVW estimate is likely biased, and causal conclusions should be interpreted cautiously.

11.5 How Published MR Studies Are Typically Reported

A standard MR paper usually includes the IVW, MR-Egger, weighted median, and weighted mode estimates; Cochran’s Q; the MR-Egger intercept test; a leave-one-out analysis; and scatter, forest, and funnel plots. These analyses collectively evaluate robustness.

11.6 The Big Picture

Sensitivity analyses are not optional extras — they are an essential component of Mendelian Randomization. The goal is not simply to produce a causal estimate, but to determine whether that estimate remains credible under a variety of assumptions and potential violations. Strong causal claims require strong supporting evidence, and sensitivity analyses provide it.

11.7 Looking Ahead

We’ve now covered the major MR estimators and sensitivity analyses used in modern Mendelian Randomization. Part 12 brings everything together in a complete, practical workflow using the TwoSampleMR package in R.

Key takeaways

Sensitivity analyses evaluate the robustness of MR conclusions; agreement across methods strengthens confidence.

Heterogeneity (via Cochran’s Q) and the MR-Egger intercept both signal possible pleiotropy or invalid instruments.

Leave-one-out analysis identifies individual SNPs that disproportionately drive a result.

Funnel, scatter, and forest plots provide complementary visual diagnostics.

Modern MR studies rely on multiple complementary analyses rather than a single estimator.

12 Part 12 — A Complete Mendelian Randomization Workflow Using `TwoSampleMR`

The previous parts covered the theoretical foundations of MR: correlation versus causation, instrumental variables, the Wald Ratio, IVW, two-sample MR, harmonization, horizontal pleiotropy, MR-Egger, weighted median, weighted mode, and sensitivity analyses. This final part brings everything together in a complete, practical MR analysis using the TwoSampleMR package in R.

The goal is not merely to run software, but to see how each step connects back to the assumptions and concepts covered throughout this series.

12.1 The Overall Workflow

A standard MR analysis consists of six major steps:

12.2 Step 1: Install and Load Required Packages

# install.packages("remotes")
# remotes::install_github("MRCIEU/TwoSampleMR")
# install.packages("data.table")
# install.packages("dplyr")

library(TwoSampleMR)
library(data.table)
library(dplyr)


Attaching package: ‘data.table’


The following object is masked from ‘package:base’:

    %notin%



Attaching package: ‘dplyr’


The following objects are masked from ‘package:data.table’:

    between, first, last


The following objects are masked from ‘package:stats’:

    filter, lag


The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union

12.3 Step 2: Define the Scientific Question

Every MR analysis begins with a causal hypothesis:

Does BMI causally increase the risk of Coronary Heart Disease (CHD)?

Exposure: BMI. Outcome: Coronary Heart Disease.

12.4 Step 3: Obtain Genetic Instruments

The first task is identifying SNPs associated with the exposure — — which become the instrumental variables. Using the IEU OpenGWAS database:

# Browse available studies
ao <- available_outcomes()

# Search for BMI datasets
ao %>%
  filter(grepl("body mass index", trait, ignore.case = TRUE)) %>%
  head()

A GwasInfo: 6 × 27
id	trait	coverage	ncase	group_name	year	mr	author	sex	qc_prior_to_upload	⋯	ncontrol	covariates	subcategory	category	ontology	doi	study_design	consortium	sd
										⋯
ebi-a-GCST90103751	Body mass index	NA	NA	public	2022	1	Wong HS	NA	NA	⋯	NA	NA	NA	NA	NA	NA	NA	NA	NA
ebi-a-GCST90095039	Body mass index	NA	NA	public	2022	1	Fernndez-Rhodes L	NA	NA	⋯	NA	NA	NA	NA	NA	NA	NA	NA	NA
ebi-a-GCST90095034	Body mass index	NA	NA	public	2022	1	Fernndez-Rhodes L	NA	NA	⋯	NA	NA	NA	NA	NA	NA	NA	NA	NA
ebi-a-GCST90029007	Body mass index	NA	NA	public	2018	1	Loh PR	NA	NA	⋯	NA	NA	NA	NA	NA	NA	NA	NA	NA
ebi-a-GCST90025994	Body mass index	NA	NA	public	2021	1	Barton AR	NA	NA	⋯	NA	NA	NA	NA	NA	NA	NA	NA	NA
ebi-a-GCST90018947	Body mass index	NA	NA	public	2021	1	Sakaue S	NA	NA	⋯	NA	NA	NA	NA	NA	NA	NA	NA	NA

Suppose the BMI dataset of interest is ieu-a-2. Extract instruments:

exposure_dat <- extract_instruments(
  outcomes = "ieu-a-2"
)

extract_instruments() selects SNPs that reach genome-wide significance, , against BMI. The output contains SNP IDs, effect alleles, beta coefficients, standard errors, and p-values.

Inspect the instruments:

head(exposure_dat)

A data.frame: 6 × 15
	id.exposure	chr.exposure	pos.exposure	SNP	effect_allele.exposure	other_allele.exposure	eaf.exposure	beta.exposure	se.exposure	pval.exposure	samplesize.exposure	exposure	mr_keep.exposure	pval_origin.exposure	data_source.exposure

1	ieu-a-2	16	53803574	rs1558902	A	T	0.4500	0.0809	0.0030	1.12980e-156	336974	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd
2	ieu-a-2	18	57838401	rs663129	A	G	0.2833	0.0549	0.0034	3.02970e-57	332575	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd
3	ieu-a-2	2	632348	rs13021737	G	A	0.8750	0.0604	0.0039	5.43876e-54	333169	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd
4	ieu-a-2	4	45182527	rs10938397	G	A	0.4333	0.0399	0.0030	1.41710e-40	337092	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd
5	ieu-a-2	1	177889480	rs543874	G	A	0.2667	0.0497	0.0037	2.28718e-40	339078	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd
6	ieu-a-2	6	50865820	rs943005	T	C	0.1000	0.0444	0.0038	4.52376e-31	339197	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd

Column	Description
`SNP`	Variant ID
`beta.exposure`	SNP → Exposure effect
`se.exposure`	Standard error
`effect_allele.exposure`	Effect allele
`other_allele.exposure`	Reference allele
`pval.exposure`	GWAS p-value

12.5 Step 4: Obtain Outcome Associations

Now extract the same SNPs from the outcome GWAS — suppose ieu-a-7 corresponds to Coronary Heart Disease:

outcome_dat <- extract_outcome_data(
  snps = exposure_dat$SNP,
  outcomes = "ieu-a-7"
)

Extracting data for 78 SNP(s) from 1 GWAS(s)

Querying id chunk 1 of 1

Querying variant chunk 1 of 2

Querying variant chunk 2 of 2

This provides associations. Inspect the outcome data:

head(outcome_dat)

A data.frame: 6 × 16
	SNP	chr	pos	beta.outcome	se.outcome	samplesize.outcome	pval.outcome	eaf.outcome	effect_allele.outcome	other_allele.outcome	outcome	id.outcome	originalname.outcome	outcome.deprecated	mr_keep.outcome	data_source.outcome

1	rs657452	1	49589847	-0.007670	0.0093844	184305	0.4137470	0.566124	G	A	Coronary heart disease \|\| id:ieu-a-7	ieu-a-7	Coronary heart disease	Coronary heart disease \|\| \|\|	TRUE	igd
2	rs7531118	1	72837239	0.017675	0.0097331	184305	0.0693745	0.480633	C	T	Coronary heart disease \|\| id:ieu-a-7	ieu-a-7	Coronary heart disease	Coronary heart disease \|\| \|\|	TRUE	igd
3	rs17381664	1	78048331	0.016188	0.0102099	184305	0.1128490	0.349349	C	T	Coronary heart disease \|\| id:ieu-a-7	ieu-a-7	Coronary heart disease	Coronary heart disease \|\| \|\|	TRUE	igd
4	rs11165643	1	96924097	0.005150	0.0092844	184305	0.5791050	0.553284	T	C	Coronary heart disease \|\| id:ieu-a-7	ieu-a-7	Coronary heart disease	Coronary heart disease \|\| \|\|	TRUE	igd
5	rs7550711	1	110082886	-0.048354	0.0305635	184305	0.1136310	0.027650	T	C	Coronary heart disease \|\| id:ieu-a-7	ieu-a-7	Coronary heart disease	Coronary heart disease \|\| \|\|	TRUE	igd
6	rs543874	1	177889480	0.007600	0.0117296	184305	0.5170290	0.189189	G	A	Coronary heart disease \|\| id:ieu-a-7	ieu-a-7	Coronary heart disease	Coronary heart disease \|\| \|\|	TRUE	igd

Important columns include beta.outcome, se.outcome, effect_allele.outcome, and pval.outcome.

12.6 Step 5: Harmonization

Before calculating causal estimates, alleles must be aligned (Part 6):

Exposure and outcome effects must refer to the same allele.

harmonised_dat <- harmonise_data(
  exposure_dat,
  outcome_dat
)

head(harmonised_dat)

Harmonising Body mass index || id:ieu-a-2 (ieu-a-2) and Coronary heart disease || id:ieu-a-7 (ieu-a-7)

Removing the following SNPs for being palindromic with intermediate allele frequencies:
rs1558902

A data.frame: 6 × 36
	SNP	effect_allele.exposure	other_allele.exposure	effect_allele.outcome	other_allele.outcome	beta.exposure	beta.outcome	eaf.exposure	eaf.outcome	remove	⋯	se.exposure	pval.exposure	samplesize.exposure	exposure	mr_keep.exposure	pval_origin.exposure	data_source.exposure	action	SNP_index	mr_keep
											⋯
1	rs10132280	A	C	A	C	-0.0221	-0.012169	0.3333	0.282164	FALSE	⋯	0.0033	1.40088e-11	338856	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd	2	1	TRUE
2	rs1016287	C	T	C	T	-0.0228	-0.014087	0.6750	0.699031	FALSE	⋯	0.0033	4.35512e-12	339033	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd	2	1	TRUE
3	rs10182181	G	A	G	A	0.0309	0.018295	0.5000	0.473525	FALSE	⋯	0.0029	8.07049e-26	338829	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd	2	1	TRUE
4	rs1032524	C	T	C	T	0.0182	0.020795	0.5083	0.512710	FALSE	⋯	0.0029	5.63599e-10	333818	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd	2	1	TRUE
5	rs10733682	G	A	G	A	-0.0188	-0.004541	0.5750	0.491116	FALSE	⋯	0.0030	2.45499e-10	336886	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd	2	1	TRUE
6	rs10840100	G	A	G	A	0.0206	0.014599	0.7250	0.607618	FALSE	⋯	0.0030	6.66653e-12	339135	Body mass index \|\| id:ieu-a-2	TRUE	reported	igd	2	1	TRUE

This automatically matches SNPs, aligns alleles, resolves strand issues, removes problematic SNPs, and excludes ambiguous palindromic variants where necessary.

12.7 Step 6: Run Mendelian Randomization

mr_results <- mr(harmonised_dat)
mr_results

Analysing 'ieu-a-2' on 'ieu-a-7'

A data.frame: 5 × 9
id.exposure	id.outcome	outcome	exposure	method	nsnp	b	se	pval

ieu-a-2	ieu-a-7	Coronary heart disease \|\| id:ieu-a-7	Body mass index \|\| id:ieu-a-2	MR Egger	77	0.5480371	0.18668642	4.417059e-03
ieu-a-2	ieu-a-7	Coronary heart disease \|\| id:ieu-a-7	Body mass index \|\| id:ieu-a-2	Weighted median	77	0.5020780	0.06963858	5.604734e-13
ieu-a-2	ieu-a-7	Coronary heart disease \|\| id:ieu-a-7	Body mass index \|\| id:ieu-a-2	Inverse variance weighted	77	0.4795636	0.06453223	1.074690e-13
ieu-a-2	ieu-a-7	Coronary heart disease \|\| id:ieu-a-7	Body mass index \|\| id:ieu-a-2	Simple mode	77	0.3494593	0.16648719	3.913634e-02
ieu-a-2	ieu-a-7	Coronary heart disease \|\| id:ieu-a-7	Body mass index \|\| id:ieu-a-2	Weighted mode	77	0.4539777	0.14820840	3.029458e-03

By default, TwoSampleMR reports IVW, MR-Egger, weighted median, and weighted mode. A typical output:

Method	Estimate	SE	P-value
IVW	0.42	0.08	0.001
MR-Egger	0.38	0.15	0.02
Weighted Median	0.40	0.10	0.003
Weighted Mode	0.39	0.12	0.005

All four methods point toward a positive causal effect, which strengthens confidence in the result.

12.8 Step 7: Heterogeneity Testing

heterogeneity_results <- mr_heterogeneity(harmonised_dat)
heterogeneity_results

A data.frame: 2 × 8
id.exposure	id.outcome	outcome	exposure	method	Q	Q_df	Q_pval

ieu-a-2	ieu-a-7	Coronary heart disease \|\| id:ieu-a-7	Body mass index \|\| id:ieu-a-2	MR Egger	143.8278	75	3.055122e-06
ieu-a-2	ieu-a-7	Coronary heart disease \|\| id:ieu-a-7	Body mass index \|\| id:ieu-a-2	Inverse variance weighted	144.1213	76	3.997391e-06

A Cochran’s Q p-value of indicates little evidence of heterogeneity; a p-value would instead suggest possible pleiotropy or invalid instruments.

12.9 Step 8: Test for Directional Pleiotropy

pleiotropy_results <- mr_pleiotropy_test(harmonised_dat)
pleiotropy_results

A data.frame: 1 × 7
id.exposure	id.outcome	outcome	exposure	egger_intercept	se	pval

ieu-a-2	ieu-a-7	Coronary heart disease \|\| id:ieu-a-7	Body mass index \|\| id:ieu-a-2	-0.001908813	0.004879647	0.6967744

An intercept p-value of indicates little evidence of directional pleiotropy; a p-value of would suggest it may be present.

12.10 Step 9: Leave-One-Out Analysis

loo_results <- mr_leaveoneout(harmonised_dat)
mr_leaveoneout_plot(loo_results)

Warning message:

“Removed 1 row containing missing values or values outside the scale range (`geom_point()`).”

[[1]]

If all leave-one-out estimates stay similar, the result is stable; if removing one SNP dramatically changes the estimate, that SNP warrants further investigation.

12.11 Step 10: Scatter Plot

scatter_plot <- mr_scatter_plot(mr_results, harmonised_dat)
scatter_plot

$`ieu-a-2.ieu-a-7`

The plot shows SNP-exposure effects, SNP-outcome effects, and the IVW and MR-Egger regression lines together.

12.12 Step 11: Forest Plot

single_snp_results <- mr_singlesnp(harmonised_dat)
mr_forest_plot(single_snp_results)

Warning message:

“Removed 1 row containing missing values or values outside the scale range (`geom_point()`).”

[[1]]

This helps identify outliers and heterogeneity across individual SNPs.

12.13 Step 12: Funnel Plot

mr_funnel_plot(single_snp_results)

[[1]]

Symmetric plots suggest balanced pleiotropy; asymmetry may indicate directional pleiotropy.

12.14 A Complete Example Pipeline

library(TwoSampleMR)

# Extract instruments
exposure_dat <- extract_instruments(
  outcomes = "ieu-a-2"
)

# Extract outcomes
outcome_dat <- extract_outcome_data(
  snps = exposure_dat$SNP,
  outcomes = "ieu-a-7"
)

# Harmonize
harmonised_dat <- harmonise_data(
  exposure_dat,
  outcome_dat
)

# Main MR
mr_results <- mr(harmonised_dat)

# Heterogeneity
mr_heterogeneity(harmonised_dat)

# Pleiotropy
mr_pleiotropy_test(harmonised_dat)

# Leave-one-out
loo <- mr_leaveoneout(harmonised_dat)

# Visualizations
mr_scatter_plot(mr_results, harmonised_dat)
mr_leaveoneout_plot(loo)

Extracting data for 78 SNP(s) from 1 GWAS(s)

Querying id chunk 1 of 1

Querying variant chunk 1 of 2

Querying variant chunk 2 of 2

Harmonising Body mass index || id:ieu-a-2 (ieu-a-2) and Coronary heart disease || id:ieu-a-7 (ieu-a-7)

Removing the following SNPs for being palindromic with intermediate allele frequencies:
rs1558902

Analysing 'ieu-a-2' on 'ieu-a-7'

A data.frame: 2 × 8
id.exposure	id.outcome	outcome	exposure	method	Q	Q_df	Q_pval

ieu-a-2	ieu-a-7	Coronary heart disease \|\| id:ieu-a-7	Body mass index \|\| id:ieu-a-2	MR Egger	143.8278	75	3.055122e-06
ieu-a-2	ieu-a-7	Coronary heart disease \|\| id:ieu-a-7	Body mass index \|\| id:ieu-a-2	Inverse variance weighted	144.1213	76	3.997391e-06

A data.frame: 1 × 7
id.exposure	id.outcome	outcome	exposure	egger_intercept	se	pval

ieu-a-2	ieu-a-7	Coronary heart disease \|\| id:ieu-a-7	Body mass index \|\| id:ieu-a-2	-0.001908813	0.004879647	0.6967744

$`ieu-a-2.ieu-a-7`

Warning message:

“Removed 1 row containing missing values or values outside the scale range (`geom_point()`).”

[[1]]

A note on ieugwasr API changes. Running extract_instruments() against the live OpenGWAS API can currently fail with an error such as unused argument (x_api_source = x_api_source_header()), because a mismatch between the installed TwoSampleMR and ieugwasr package versions changes the function signature for tophits(). If you hit this, update both packages to their latest GitHub versions (remotes::install_github("MRCIEU/ieugwasr") and remotes::install_github("MRCIEU/TwoSampleMR")) before re-running the pipeline above.

12.15 How Results Are Usually Reported

A typical conclusion might read:

Genetically predicted BMI was positively associated with coronary heart disease risk. The IVW estimate suggested a significant causal effect. Sensitivity analyses using MR-Egger, weighted median, and weighted mode methods produced similar estimates. There was little evidence of directional pleiotropy based on the MR-Egger intercept test, and leave-one-out analyses indicated that no single SNP drove the observed association.

This is the style commonly seen in published MR studies.

12.16 Common Mistakes

Using weak instruments — weak SNPs produce unstable estimates.
Ignoring harmonization — incorrect allele alignment can reverse causal conclusions.
Reporting only IVW — robust methods should also be presented.
Ignoring pleiotropy tests — sensitivity analyses are essential.
Overstating causality — MR provides evidence for causality, but remains subject to its assumptions and limitations.

12.17 The Big Picture

A complete MR study is much more than calculating a single causal estimate. Researchers must select instruments, harmonize data, estimate causal effects, test assumptions, evaluate robustness, and interpret results carefully. The strength of MR comes not from a single method, but from combining multiple complementary analyses into a coherent causal inference framework.

12.18 Final Summary of the Series

Across these twelve parts we covered: correlation versus causation, Mendelian Randomization, instrumental variables, the Wald Ratio, IVW estimation, two-sample MR, harmonization, horizontal pleiotropy, MR-Egger, weighted median, weighted mode, sensitivity analyses, and the complete MR workflow. Together, these concepts form the foundation of modern Mendelian Randomization and causal inference in statistical genetics.

Key takeaways

TwoSampleMR provides a complete framework for Mendelian Randomization analyses in R.

Instrument selection comes first; harmonization is essential before estimation.

IVW is typically the primary analysis, with MR-Egger, weighted median, and weighted mode providing robustness checks.

Sensitivity analyses (heterogeneity, pleiotropy tests, leave-one-out) evaluate whether MR assumptions are being violated.

Modern MR studies rely on multiple complementary methods rather than a single estimate — and careful interpretation matters as much as the statistical estimation itself.

The Covariance Matrix in Association Testing

Nivedita Bhadra — Sun, 14 Jun 2026 21:00:00 GMT

The object at the center of every case in this tutorial: a covariance matrix’s diagonal describes each observation’s own variance, but it’s the off-diagonal terms – the part “independence” assumes away – that decide whether a p-value can be trusted.

Covariance Mixed Models GRM R

Very often atatistical models fail because we assume observations are independent when they are not. Repeated measurements from the same patient, genetic similarity between individuals, longitudinal studies, family data, spatial observations, and multivariate phenotypes all introduce correlation. Ignoring these dependencies produces misleading standard errors, inflated false-positive rates, and unreliable inference.

The mathematical object that describes these dependencies is the covariance matrix. It summarizes not only how variable each observation is, but also how every pair of observations varies together. Once that covariance structure is specified, many seemingly different statistical methods—from ordinary least squares to generalized least squares, linear mixed models, generalized estimating equations, phylogenetic regression, and modern GWAS methods—become variations of the same underlying framework.

In one sentence: a covariance matrix tells a statistical test which observations are allowed to be treated as independent, and every association test is, underneath, a decision about that matrix.

1 What a covariance matrix actually is

For a random vector , the covariance matrix collects every pairwise covariance:

The diagonal holds variances, . The off-diagonal holds the covariances between pairs.

Why not just track variances? Because variance alone tells you how much a single variable moves. It says nothing about whether two variables move together. Two traits can each have large variance and still be uncorrelated, or have small variance and be almost perfectly redundant. The off-diagonal is where all the structure lives.

A quick simulated illustration: three correlated traits.

set.seed(1)

Sigma_true <- matrix(c(
  1.0, 0.7, 0.1,
  0.7, 1.0, 0.2,
  0.1, 0.2, 1.0
), nrow = 3)

Y <- MASS::mvrnorm(n = 500, mu = c(0, 0, 0), Sigma = Sigma_true)
colnames(Y) <- c("trait1", "trait2", "trait3")

round(cov(Y), 2)

A matrix: 3 × 3 of type dbl
	trait1	trait2	trait3
trait1	1.04	0.70	0.09
trait2	0.70	0.99	0.26
trait3	0.09	0.26	1.13

The sample covariance recovers the input structure: trait1 and trait2 move together (0.7), trait3 is nearly independent of trait1 (0.1).

2 Why the covariance matrix matters for statistical testing

Every standard test statistic (t-test, F-test, GWAS Wald test) is built assuming a specific for the residuals or the samples. The two default assumptions are:

Independence: for .
Homoscedasticity: for all .

Together these collapse to , the identity matrix scaled by a constant. Ordinary least squares, the standard GWAS regression, and the standard t-test all silently assume this.

What breaks when that assumption is wrong? Nothing about the point estimate. The regression coefficient stays unbiased. What breaks is the standard error, because the standard OLS variance formula is only correct when . If samples are actually correlated, the true variance of is Plugging in the wrong (diagonal) gives standard errors that are too small, meaning inflated test statistics, meaning false positives.

This is the single mechanism behind population stratification in GWAS, pseudoreplication in phylogenetic comparative methods, and the multiple-testing correlation problem in fine-mapping. Different names, same incorrect assumption.

3 Association testing under different covariance structures

The general form of an association test is a generalized least squares (GLS) problem:

which reduces to ordinary least squares when . Every case below is the same equation with a different plugged in.

3.1 Case 1 — independent samples (the textbook GWAS test)

If individuals are unrelated and drawn from a single population, is a reasonable approximation, and a simple linear model per variant is valid:

n <- 1000
genotype <- rbinom(n, 2, 0.3)
phenotype <- 0.3 * genotype + rnorm(n)

fit <- lm(phenotype ~ genotype)
summary(fit)$coefficients

A matrix: 2 × 4 of type dbl
	Estimate	Std. Error	t value	Pr(>\|t\|)
(Intercept)	-0.001616378	0.04417016	-0.03659433	9.708158e-01
genotype	0.327975887	0.04923014	6.66209569	4.454492e-11

No hidden structure, no correction needed. The standard GWAS Wald test is designed for such cases.

3.2 Case 2 — cryptic relatedness and population stratification

In reality cohorts are never fully unrelated, and subgroups can differ in both allele frequency and phenotype mean. This induces off-diagonal covariance between individuals that correlates with genotype, which is exactly the condition under which OLS standard errors become invalid.

We simulate two subpopulations with different means and a genotype frequency difference:

set.seed(2)
n <- 1000
pop <- rep(c("A", "B"), each = n / 2)

# allele frequency differs by population -> stratification
freq <- ifelse(pop == "A", 0.1, 0.4)
genotype <- rbinom(n, 2, freq)

# phenotype differs by population for reasons unrelated to genotype
phenotype <- ifelse(pop == "A", 0, 1) + rnorm(n, sd = 0.5)

naive <- lm(phenotype ~ genotype)
summary(naive)$coefficients["genotype", ]

Estimate: 0.36051672383449
Std. Error: 0.0312442835871719
t value: 11.5386458719287
Pr(>|t|): 5.30984934684389e-29

Here genotype picks up a spurious association purely because it tags population membership, not because it affects the phenotype. The fix is to model the covariance directly, either with a genomic relationship matrix (GRM) in a mixed model, or with fixed-effect covariates (PCs) that absorb the structure. The mixed-model version:

where is the GRM: estimates genome-wide relatedness between individuals and from marker data. This is exactly built from data instead of assumed to be diagonal.

# toy GRM from a small marker panel, standardised as in GCTA/GEMMA.
# Crucially, these markers must actually carry the population signal --
# a marker panel with the same allele frequency in both groups would
# produce a GRM blind to the very structure we're trying to correct for.
marker_freq_A <- runif(200, 0.05, 0.15)
marker_freq_B <- runif(200, 0.25, 0.45)
markers <- sapply(seq_len(200), function(j) {
  f <- ifelse(pop == "A", marker_freq_A[j], marker_freq_B[j])
  rbinom(n, 2, f)
})
markers_std <- scale(markers)
K <- tcrossprod(markers_std) / ncol(markers_std)

# correcting the naive test with the top structure-capturing PCs of K
pcs <- eigen(K)$vectors[, 1:2]
corrected <- lm(phenotype ~ genotype + pcs)
summary(corrected)$coefficients["genotype", ]

Estimate: 0.0332531649087993
Std. Error: 0.0267829897746121
t value: 1.24157777711286
Pr(>|t|): 0.214684689209704

Adding the components that summarise absorbs the stratification signal and pulls the genotype effect back toward the null. Tools like GEMMA, SAIGE, and regenie fit the full mixed model rather than a PC approximation, but the logic is identical: replace the identity covariance with .

3.3 Case 3 — non-independent taxa: PGLS

This scenario also appears in other fields. In a comparative dataset across species, closely related species resemble each other for reasons that have nothing to do with the trait being tested — shared evolutionary history, not shared biology of interest. Ordinary regression treats each species as an independent data point, so the effective sample size is inflated and p-values are anti-conservative.

Phylogenetic generalised least squares replaces with a covariance matrix derived from the phylogeny, typically under Brownian motion: is the shared branch length from the root to the most recent common ancestor of species and .

library(ape)
library(nlme)

set.seed(3)
tree <- rtree(30)
trait_x <- rTraitCont(tree, model = "BM")
trait_y <- 0.5 * trait_x[tree$tip.label] + rTraitCont(tree, model = "BM", sigma = 0.5)

d <- data.frame(species = tree$tip.label, x = trait_x[tree$tip.label], y = trait_y)

naive_ols <- lm(y ~ x, data = d)
pgls_fit <- gls(y ~ x, data = d, correlation = corBrownian(phy = tree, form = ~species))

summary(naive_ols)$coefficients["x", ]
summary(pgls_fit)$tTable["x", ]

Estimate: 0.887154063958009
Std. Error: 1.95739468827265
t value: 0.453232078983979
Pr(>|t|): 0.653874585786409

Value: 1.3394774022645
Std.Error: 1.3131699537816
t-value: 1.02003354433076
p-value: 0.316445575574491

Same data, two different assumed covariance matrices, two different standard errors on the same point estimate. This is the direct cousin of Case 2: relatedness there is genetic and pairwise (the GRM), relatedness here is phylogenetic and hierarchical (branch-length covariance), but the correction mechanism — swap for the real — is the same equation.

3.4 Case 4 — correlated variants: joint testing and colocalization

The covariance problem also shows up across variants rather than across samples. Nearby SNPs in linkage disequilibrium (LD) have correlated genotypes, so their marginal association test statistics are correlated even under the null. Ignoring this inflates the effective number of independent tests and misleads fine-mapping.

The LD matrix (correlation between genotypes at variants and ) plays the role of here. A joint (multi-SNP) test statistic is:

where is the vector of marginal Wald z-scores. This is the same GLS logic again, just applied to summary statistics instead of raw phenotypes:

set.seed(4)
# simulate LD between 5 variants
R <- matrix(0.3, 5, 5); diag(R) <- 1
z <- as.numeric(MASS::mvrnorm(1, mu = c(0, 0, 3, 0, 0), Sigma = R))

# naive: treat as independent
chisq_naive <- sum(z^2)
pchisq(chisq_naive, df = 5, lower.tail = FALSE)

# correct: account for LD covariance
chisq_joint <- as.numeric(t(z) %*% solve(R) %*% z)
pchisq(chisq_joint, df = 5, lower.tail = FALSE)

0.0670792649358373

0.0175917963452512

coloc.abf and conditional/joint (COJO) analyses both hinge on getting this right; a mismatched LD reference panel is the single most common cause of spurious colocalization or fine-mapping results.

4 The pattern across all four cases

Case	What violates independence	Covariance matrix used	Fix
Unrelated individuals	none		plain OLS/Wald test
Cryptic relatedness / stratification	shared ancestry between individuals	GRM	linear mixed model
Cross-species comparison	shared phylogenetic history	Brownian-motion tree covariance	PGLS
Variants in LD	correlated genotypes across markers	LD matrix	joint/conditional test, colocalization

Every row is the same generalized least squares equation with a different . Therefore, whenever a test looks “too significant” relative to intuition, the first thing to check is not the effect size but whether the covariance structure of the data was estimated correctly.

5 Further reading

Yang, Lee, Goddard & Visscher (2011). GCTA: A tool for genome-wide complex trait analysis. American Journal of Human Genetics, 88(1), 76–82. https://doi.org/10.1016/j.ajhg.2010.11.011

Freckleton, Harvey & Pagel (2002). Phylogenetic analysis and comparative data: a test and review of evidence. American Naturalist, 160(6), 712–726. https://doi.org/10.1086/343873

Yang, Ferreira, Morris, et al. (2012). Conditional and joint multiple-SNP analysis of GWAS summary statistics identifies additional variants influencing complex traits. Nature Genetics, 44(4), 369–375. https://doi.org/10.1038/ng.2213

Ridge and Lasso Regression

Nivedita Bhadra — Tue, 09 Jun 2026 21:00:00 GMT

Ridge and lasso are the two workhorse methods for fitting a linear model when you have many predictors, some of them correlated, and ordinary least squares (OLS) starts to behave badly. Both add a penalty term to the regression objective; they differ in exactly one detail — whether that penalty is squared or absolute — and that single difference changes everything about how the two methods behave.

This tutorial builds both from scratch: the math, the geometry behind why lasso zeroes out coefficients and ridge doesn’t, and a full R implementation of each, checked against closed-form solutions and OLS along the way.

1 Why Plain Linear Regression Breaks Down

If OLS already minimizes the residual sum of squares, why would we want anything else? Because “minimizes the training residual” and “gives a trustworthy, stable estimate” are not the same promise. OLS keeps that promise only when predictors are roughly independent and there’s plenty of data relative to the number of predictors. Neither is guaranteed in practice.

Real predictor sets are rarely independent. Two variables measuring related things — income and years of education, or two neighboring genetic markers in the same region of a chromosome — carry overlapping information. When predictors are highly correlated, OLS has to arbitrarily decide how to split credit between them, and that split is extremely sensitive to the specific noise in your sample.

We can see this directly. Simulate a 10-predictor design with two “correlated blocks” of 4 predictors each (correlation 0.85 within each block) plus 2 independent predictors, only three of which actually affect the outcome.

set.seed(42)

## Build a 10-predictor design matrix with two correlated "LD-like" blocks
## of 4 predictors each (within-block correlation 0.85) plus 2 independent
## noise predictors -- deliberately mimicking correlated genotype blocks.
p <- 10
n <- 200

Sigma <- diag(p)
block1 <- 1:4; block2 <- 5:8
Sigma[block1, block1] <- 0.85
Sigma[block2, block2] <- 0.85
diag(Sigma) <- 1

## Sample correlated predictors via the Cholesky factor of Sigma
L <- chol(Sigma)
Z <- matrix(rnorm(n * p), n, p)
X <- Z %*% L
X <- scale(X)                      # standardize columns
colnames(X) <- paste0("X", 1:p)

## True effects: only the first predictor in each correlated block, plus
## one of the independent noise predictors, actually affect the outcome
true_beta <- c(3, 0, 0, 0, -2, 0, 0, 0, 1.5, 0)
y <- as.vector(X %*% true_beta + rnorm(n, sd = 3))

cat("Correlation within block 1 (X1-X4):\n")
print(round(cor(X[, block1]), 2))
cat("\nTrue beta:", true_beta, "\n")

saveRDS(list(X = X, y = y, true_beta = true_beta, Sigma = Sigma), "reg_data.rds")

Correlation within block 1 (X1-X4):
     X1   X2   X3   X4
X1 1.00 0.85 0.82 0.84
X2 0.85 1.00 0.84 0.85
X3 0.82 0.84 1.00 0.84
X4 0.84 0.85 0.84 1.00

True beta: 3 0 0 0 -2 0 0 0 1.5 0

X1–X4 and X5–X8 form two tight correlated blocks, exactly the kind of structure that causes trouble. Only X1, X5, and X9 have a real effect on the outcome — everything else is either a correlated “twin” of a real predictor, or pure noise.

1.1 1.1 OLS instability under correlated predictors

Refit OLS 300 times on fresh draws of the outcome (same design matrix each time) and look at how much the estimated coefficients bounce around.

sim <- readRDS("reg_data.rds")
X <- sim$X; true_beta <- sim$true_beta
n <- nrow(X); p <- ncol(X)

set.seed(1)
B <- 300
ols_coefs <- matrix(NA, B, p)
for (b in seq_len(B)) {
  y_b <- as.vector(X %*% true_beta + rnorm(n, sd = 3))   # fresh noise draw, same X
  ols_coefs[b, ] <- coef(lm(y_b ~ X - 1))
}
colnames(ols_coefs) <- colnames(X)

cat("Standard deviation of OLS estimates for each predictor:\n")
print(round(apply(ols_coefs, 2, sd), 2))

Standard deviation of OLS estimates for each predictor:
  X1   X2   X3   X4   X5   X6   X7   X8   X9  X10 
0.49 0.51 0.46 0.52 0.52 0.52 0.55 0.52 0.21 0.22

The correlated-block predictors (X1–X8) have roughly double the standard deviation of the independent ones (X9, X10) — 0.5 versus 0.2 — purely because of correlation, not because they’re individually less informative. OLS is unbiased here (the boxes are centered near the true values), but “unbiased and wildly variable” is not a comforting property when you’re trying to trust a specific estimate from a specific sample.

Regularization trades away a little of that unbiasedness in exchange for a large reduction in variance — a good trade whenever variance is the bigger problem, which it usually is with correlated or high-dimensional predictors.

2 Ridge Regression: Shrinking with an L2 Penalty

What does ridge regression actually change about the objective function? It adds a penalty proportional to the squared size of the coefficients, so the optimizer is rewarded for keeping them small, not just for fitting the training data.

controls the strength of the penalty: recovers plain OLS, and as every coefficient is forced toward zero. Because the penalty is smooth and quadratic, this objective has a closed-form solution:

Compare this to the OLS solution, : ridge simply adds to the diagonal of before inverting. This is precisely the fix for the numerical instability that correlated predictors cause — becomes close to singular (nearly non-invertible) when columns are highly correlated, and adding keeps it comfortably invertible (this is why ridge is sometimes called Tikhonov regularization, its original name in the applied-math literature).

sim <- readRDS("reg_data.rds")
X <- sim$X; y <- sim$y; true_beta <- sim$true_beta
n <- nrow(X); p <- ncol(X)

ridge_fit <- function(X, y, lambda) {
  p <- ncol(X)
  solve(t(X) %*% X + lambda * diag(p), t(X) %*% y)
}

lambda_grid <- exp(seq(log(0.01), log(500), length.out = 60))
ridge_path <- t(sapply(lambda_grid, function(l) ridge_fit(X, y, l)))
colnames(ridge_path) <- colnames(X)

## Sanity check against the OLS solution as lambda -> 0
cat("Ridge at lambda~0     :", round(ridge_path[1, ], 3), "\n")
cat("OLS (lm)              :", round(coef(lm(y ~ X - 1)), 3), "\n")

saveRDS(list(lambda_grid = lambda_grid, ridge_path = ridge_path), "ridge_path.rds")

Ridge at lambda~0     : 2.808 -0.636 0.445 0.293 -1.273 -0.078 -0.88 0.226 1.832 0.307 
OLS (lm)              : 2.809 -0.636 0.445 0.293 -1.273 -0.078 -0.88 0.226 1.832 0.307

As it must: with essentially at zero, ridge and OLS agree to three decimal places. Now trace out the full path as grows.

p <- ncol(X)
matplot(log(lambda_grid), ridge_path, type = "l", lty = 1, lwd = 2,
        col = rainbow(p, end = 0.85),
        xlab = expression(log(lambda)), ylab = expression(hat(beta)),
        main = "Ridge coefficient paths")
abline(h = 0, lty = 3, col = "grey40")
legend("topright", legend = colnames(X), col = rainbow(p, end = 0.85), lty = 1, lwd = 2, cex = 0.7)

Every coefficient shrinks smoothly and continuously toward zero as increases, but none of them ever actually reach zero (except in the limit). That single observation — smooth shrinkage, never exactly zero — is the entire practical difference between ridge and lasso, and it comes directly from using a squared penalty instead of an absolute one. We’ll see exactly why in the geometry section below.

3 Lasso: An L1 Penalty and the Sparsity It Creates

Lasso (Tibshirani, 1996) replaces the squared penalty with an absolute one:

Why doesn’t this have a closed-form solution the way ridge does? The absolute value function isn’t differentiable at zero. Calculus-based closed forms rely on setting a smooth derivative to zero everywhere; lasso’s objective has a kink exactly where the interesting behavior (a coefficient becoming zero) happens.

Instead, lasso is solved iteratively. Coordinate descent (Friedman, Hastie & Tibshirani, 2010) optimizes one coefficient at a time, holding all others fixed, and cycles through all predictors repeatedly until nothing changes. Fixing every for , the one-dimensional update for turns out to be a simple soft-thresholding operation:

shrinks its input toward zero by , and clips it to exactly zero if the input wasn’t larger than to begin with. That clipping-to-exactly-zero step is where lasso’s sparsity comes from — it’s baked directly into the per-coordinate update rule.

soft_threshold <- function(z, gamma) sign(z) * pmax(abs(z) - gamma, 0)

lasso_fit <- function(X, y, lambda, tol = 1e-8, max_iter = 5000) {
  p <- ncol(X); n <- nrow(X)
  beta <- rep(0, p)
  Xy <- t(X) %*% y
  XtX <- t(X) %*% X

  for (iter in seq_len(max_iter)) {
    beta_old <- beta
    for (j in seq_len(p)) {
      r_j <- Xy[j] - sum(XtX[j, -j] * beta[-j])
      beta[j] <- soft_threshold(r_j, n * lambda / 2) / XtX[j, j]
    }
    if (max(abs(beta - beta_old)) < tol) break
  }
  beta
}

## Verify the coordinate-descent update against the univariate closed form:
## for a single predictor (no correlated neighbours), lasso reduces to soft-
## thresholding the OLS coefficient directly.
x1 <- X[, 1]
beta_ols_uni <- sum(x1 * y) / sum(x1^2)
lambda_test <- 0.3
beta_lasso_uni <- soft_threshold(sum(x1 * y), n * lambda_test / 2) / sum(x1^2)
cat("Univariate OLS beta          :", round(beta_ols_uni, 4), "\n")
cat("Univariate soft-thresholded  :", round(beta_lasso_uni, 4), "\n\n")

lambda_grid <- exp(seq(log(0.005), log(3), length.out = 60))
lasso_path <- t(sapply(lambda_grid, function(l) lasso_fit(X, y, l)))
colnames(lasso_path) <- colnames(X)

cat("Lasso at lambda~0 (should match OLS):", round(lasso_path[1, ], 3), "\n")
cat("OLS (lm)                             :", round(coef(lm(y ~ X - 1)), 3), "\n")

saveRDS(list(lambda_grid = lambda_grid, lasso_path = lasso_path), "lasso_path.rds")

Univariate OLS beta          : 2.9352 
Univariate soft-thresholded  : 2.7844 

Lasso at lambda~0 (should match OLS): 2.8 -0.614 0.434 0.289 -1.267 -0.067 -0.868 0.199 1.831 0.305 
OLS (lm)                             : 2.809 -0.636 0.445 0.293 -1.273 -0.078 -0.88 0.226 1.832 0.307

Both checks hold: the univariate soft-thresholded estimate is a shrunken version of the univariate OLS estimate (2.78 vs. 2.94, exactly what soft-thresholding by a positive amount should do), and the full multivariate lasso path recovers OLS almost exactly as . Now the path itself:

p <- ncol(X)
matplot(log(lambda_grid), lasso_path, type = "l", lty = 1, lwd = 2,
        col = rainbow(p, end = 0.85),
        xlab = expression(log(lambda)), ylab = expression(hat(beta)),
        main = "Lasso coefficient paths")
abline(h = 0, lty = 3, col = "grey40")
legend("topright", legend = colnames(X), col = rainbow(p, end = 0.85), lty = 1, lwd = 2, cex = 0.7)

This is the signature lasso picture: coefficients don’t just shrink, they drop to exactly zero, one by one, in a staircase pattern as increases. Compare this directly to the ridge path above — same data, same range in spirit, completely different qualitative behavior.

4 The Geometry Behind Why Lasso Zeroes Out Coefficients

Both penalized objectives have an equivalent “constrained optimization” formulation: minimize the residual sum of squares subject to a budget on the total penalty. For lasso, the budget is on — geometrically, a diamond (in two dimensions) centered at the origin. For ridge, the budget is on — a circle.

Why does a diamond produce exact zeros but a circle doesn’t? Picture the OLS solution as the center of a series of concentric ellipses (contours of equal residual sum of squares — every point on the same ellipse fits the training data equally well). The penalized solution is wherever the smallest ellipse first touches the constraint boundary. A diamond has corners sitting exactly on the coordinate axes; an ellipse is disproportionately likely to first touch the diamond at one of those corners, which is precisely the point where one coordinate is zero. A circle has no corners anywhere, so there’s nothing to make touching at an axis special.

We can make this exact rather than hand-wavy, using two genuinely correlated predictors and finding where an actual RSS contour touches an actual diamond and an actual circle.

set.seed(11)
n <- 200
rho <- 0.6
Sigma2 <- matrix(c(1, rho, rho, 1), 2, 2)
L <- chol(Sigma2)
Z <- matrix(rnorm(n * 2), n, 2)
X2 <- scale(Z %*% L)
true_beta2 <- c(2, 1.4)
y2 <- as.vector(X2 %*% true_beta2 + rnorm(n, sd = 1.5))

beta_hat <- as.vector(solve(t(X2) %*% X2, t(X2) %*% y2))
XtX <- t(X2) %*% X2
rss_val <- function(b) { d <- b - beta_hat; as.numeric(t(d) %*% XtX %*% d) }

t_budget <- 0.5   # a tight penalty budget, so the constraint clearly binds

## Grid-search along each edge of the L1 diamond for the true RSS-minimizing point
diamond_vertices <- rbind(c(t_budget, 0), c(0, t_budget), c(-t_budget, 0), c(0, -t_budget), c(t_budget, 0))
s_grid <- seq(0, 1, length.out = 2000)
best_rss_l1 <- Inf; best_pt_l1 <- NULL
for (e in 1:4) {
  v1 <- diamond_vertices[e, ]; v2 <- diamond_vertices[e + 1, ]
  pts <- t(sapply(s_grid, function(s) (1 - s) * v1 + s * v2))
  r <- apply(pts, 1, rss_val)
  if (min(r) < best_rss_l1) { best_rss_l1 <- min(r); best_pt_l1 <- pts[which.min(r), ] }
}

## Grid-search around the L2 circle for the true RSS-minimizing point
theta_grid <- seq(0, 2 * pi, length.out = 4000)
circle_pts <- cbind(t_budget * cos(theta_grid), t_budget * sin(theta_grid))
r_circle <- apply(circle_pts, 1, rss_val)
best_pt_l2 <- circle_pts[which.min(r_circle), ]

cat("Diamond (lasso) touching point:", round(best_pt_l1, 3),
    "-- exactly zero on one axis:", any(abs(best_pt_l1) < 1e-6), "\n")
cat("Circle (ridge) touching point :", round(best_pt_l2, 3), "\n")

Diamond (lasso) touching point: 0.5 0 -- exactly zero on one axis: TRUE 
Circle (ridge) touching point : 0.369 0.337

The diamond-constrained solution lands exactly at a corner: , not approximately, exactly. The circle-constrained solution lands at a generic point on the boundary with both coordinates nonzero. Same data, same RSS ellipses, same size budget — the only thing that changed is the shape of the constraint region, and that alone accounts for the sparsity.

5 Choosing by Cross-Validation

Neither the coefficient path nor the geometry tells you which to actually use. That’s chosen empirically, by holding out folds of data and measuring prediction error.

ridge_fit <- function(X, y, lambda) solve(t(X) %*% X + lambda * diag(ncol(X)), t(X) %*% y)

set.seed(99)
K <- 10
n <- nrow(X)
folds <- sample(rep(1:K, length.out = n))

cv_error <- function(fit_fn, lambda_grid) {
  errs <- matrix(NA, K, length(lambda_grid))
  for (k in 1:K) {
    train <- folds != k; test <- folds == k
    for (li in seq_along(lambda_grid)) {
      beta_k <- fit_fn(X[train, , drop = FALSE], y[train], lambda_grid[li])
      pred <- X[test, , drop = FALSE] %*% beta_k
      errs[k, li] <- mean((y[test] - pred)^2)
    }
  }
  colMeans(errs)
}

ridge_lambda_grid <- exp(seq(log(0.01), log(500), length.out = 40))
lasso_lambda_grid  <- exp(seq(log(0.005), log(3), length.out = 40))

ridge_cv <- cv_error(ridge_fit, ridge_lambda_grid)
lasso_cv <- cv_error(lasso_fit, lasso_lambda_grid)

ridge_lambda_min <- ridge_lambda_grid[which.min(ridge_cv)]
lasso_lambda_min <- lasso_lambda_grid[which.min(lasso_cv)]

cat("Ridge lambda.min:", round(ridge_lambda_min, 3), " CV MSE:", round(min(ridge_cv), 3), "\n")
cat("Lasso lambda.min:", round(lasso_lambda_min, 3), " CV MSE:", round(min(lasso_cv), 3), "\n")

saveRDS(list(ridge_lambda_grid = ridge_lambda_grid, ridge_cv = ridge_cv,
             lasso_lambda_grid = lasso_lambda_grid, lasso_cv = lasso_cv,
             ridge_lambda_min = ridge_lambda_min, lasso_lambda_min = lasso_lambda_min),
        "cv_results.rds")

Ridge lambda.min: 7.792  CV MSE: 9.491 
Lasso lambda.min: 0.185  CV MSE: 9.45

Both curves have the same characteristic shape: error is high when is too small (overfitting; barely different from OLS) or too large (underfitting; everything shrunk toward a useless zero model), with a minimum somewhere in between. Here ridge and lasso land on almost identical best-case prediction error (9.49 vs. 9.45) on this data — which won’t always be true, but is a reasonable outcome when, as here, most of the true signal is concentrated in a few predictors alongside genuine noise.

6 The Bias-Variance Tradeoff, Made Concrete

Regularization is often described as “trading bias for variance.” That’s not just a slogan — we can measure both quantities directly by repeatedly simulating fresh data, fitting ridge at each , and tracking how far the average prediction is from the truth (bias) versus how much predictions bounce around across simulations (variance).

true_beta <- sim$true_beta
lambda_grid <- exp(seq(log(0.01), log(500), length.out = 30))

set.seed(5)
B <- 200
x_new <- X[1:20, ]                    # a fixed held-out "test" design to evaluate prediction error on
true_signal <- as.vector(x_new %*% true_beta)

bias2 <- numeric(length(lambda_grid))
variance <- numeric(length(lambda_grid))
mse <- numeric(length(lambda_grid))

for (li in seq_along(lambda_grid)) {
  preds <- matrix(NA, B, nrow(x_new))
  for (b in seq_len(B)) {
    y_b <- as.vector(X %*% true_beta + rnorm(n, sd = 3))
    beta_b <- ridge_fit(X, y_b, lambda_grid[li])
    preds[b, ] <- as.vector(x_new %*% beta_b)
  }
  mean_pred <- colMeans(preds)
  bias2[li]    <- mean((mean_pred - true_signal)^2)
  variance[li] <- mean(apply(preds, 2, var))
  mse[li]      <- mean((preds - matrix(true_signal, B, length(true_signal), byrow = TRUE))^2)
}

cat("Bias^2 range   :", round(range(bias2), 3), "\n")
cat("Variance range :", round(range(variance), 3), "\n")
cat("lambda at min total MSE:", round(lambda_grid[which.min(mse)], 3), "\n")

Bias^2 range   : 0.002 6.827 
Variance range : 0.059 0.626 
lambda at min total MSE: 0.198

At small , bias is negligible but variance dominates total error. As grows, variance drops sharply while bias climbs slowly at first, then steeply. The total error curve (black) is the sum of the two, and its minimum sits well away from — meaning the “unbiased” OLS solution is not the one with the lowest expected prediction error. A little deliberate bias buys a large variance reduction, right up until the penalty gets so strong that bias takes over.

7 Ridge vs. Lasso, Side by Side

Putting it all together: fit OLS, ridge (at its CV-selected ), and lasso (at its CV-selected ) on the same data, and compare all three against the true coefficients.

cvres <- readRDS("cv_results.rds")

beta_ols   <- coef(lm(y ~ X - 1))
beta_ridge <- ridge_fit(X, y, cvres$ridge_lambda_min)
beta_lasso <- lasso_fit(X, y, cvres$lasso_lambda_min)

compare <- rbind(true_beta, beta_ols, as.vector(beta_ridge), beta_lasso)
rownames(compare) <- c("True", "OLS", "Ridge (CV)", "Lasso (CV)")
colnames(compare) <- colnames(X)
print(round(compare, 2))

             X1    X2   X3   X4    X5    X6    X7   X8   X9  X10
True       3.00  0.00 0.00 0.00 -2.00  0.00  0.00 0.00 1.50 0.00
OLS        2.81 -0.64 0.45 0.29 -1.27 -0.08 -0.88 0.23 1.83 0.31
Ridge (CV) 2.39 -0.37 0.47 0.39 -1.13 -0.17 -0.77 0.11 1.76 0.31
Lasso (CV) 2.55  0.00 0.20 0.10 -1.18  0.00 -0.71 0.00 1.77 0.23

Lasso sets X2, X6, and X8 to exactly zero — three of the seven truly-null predictors — while ridge shrinks every coefficient toward zero without eliminating any of them. Neither recovers the true model exactly (the correlated “twin” predictors within each block make that a genuinely hard problem), but lasso’s sparsity gives a noticeably more interpretable result here, at comparable predictive accuracy to ridge.

8 When to Reach for Which

Situation	Better choice
You believe most predictors have some small effect (a “dense” true model)	Ridge
You believe only a handful of predictors truly matter (a “sparse” true model)	Lasso
Predictors come in tightly correlated groups, and you want to keep all of them	Ridge (spreads weight across the group rather than picking one arbitrarily)
You want automatic variable selection alongside estimation	Lasso
(more predictors than observations)	Both work; lasso additionally gives a sparse, interpretable model
You want the properties of both	Elastic net — , tuning between them (Zou & Hastie, 2005)

One caveat about lasso and correlated predictors worth internalizing from the geometry section: when two predictors are highly correlated, lasso tends to arbitrarily pick one and zero out the other, rather than splitting credit between them the way ridge does. That’s a feature if you want a small, interpretable model, and a liability if the identity of which correlated predictor gets picked matters for your interpretation.

9 Summary

Both ridge and lasso add a penalty on coefficient size to the OLS objective, trading a small amount of bias for a often much larger reduction in variance.
Ridge’s squared (L2) penalty has a closed-form solution and shrinks every coefficient smoothly, never to exactly zero.
Lasso’s absolute-value (L1) penalty has no closed form, requires an iterative solver like coordinate descent, and produces exact sparsity — some coefficients become precisely zero.
The sparsity difference is entirely geometric: a diamond-shaped constraint region has corners on the coordinate axes; a circular one doesn’t.
The regularization strength is chosen empirically via cross-validation, by directly measuring which value minimizes out-of-sample prediction error.

10 References

Hoerl, A. E. & Kennard, R. W. (1970). Ridge regression: biased estimation for nonorthogonal problems. Technometrics, 12, 55–67.
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B, 58, 267–288.
Zou, H. & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B, 67, 301–320.
Friedman, J., Hastie, T. & Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33, 1.
Hastie, T., Tibshirani, R. & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer.

Polygenic Risk Scores (PRS): From GWAS to Genetic Prediction

Nivedita Bhadra — Mon, 08 Jun 2026 21:00:00 GMT

The whole tutorial in one picture: millions of small GWAS effects, combined by a PRS method, collapse into a single number per person – which is only useful once you can place that person in a risk distribution.

GWAS PRS SBayesRC PLINK

0.1 Why Polygenic Risk Scores?

Mendelian diseases (Huntington’s, cystic fibrosis, sickle cell anemia) are driven by mutations in a single gene. Most common diseases aren’t — height, BMI, coronary artery disease, schizophrenia, and depression are each influenced by thousands of variants, each with a tiny individual effect (polygenicity). No single SNP is informative enough to predict risk on its own, which raises the question: how do we combine thousands of tiny genetic effects into a useful predictor? A Polygenic Risk Score (PRS) — also called a Polygenic Score (PGS) — is the answer: a weighted sum of genetic variants across the genome, condensing an individual’s inherited predisposition into a single number.

0.2 What Is a PRS?

where indexes the individual, indexes the SNP, is the number of SNPs, is the individual’s risk-allele count at SNP , and is that SNP’s estimated effect size — usually taken directly from GWAS summary statistics. Positive increases risk; negative decreases it.

Worked example. Given effects , , , and an individual with genotype counts 2, 1, 0:

0.3 Why PRS Works Despite Tiny, Often Non-Causal SNPs

Most SNPs used in a PRS aren’t themselves causal — they still improve prediction because of Linkage Disequilibrium (LD), the tendency of nearby SNPs to be co-inherited. Even without observing the true causal SNP directly, an observed SNP correlated with it still carries predictive information. This is the same principle underlying GWAS itself.

Analogy: a genetic credit score. Just as a bank doesn’t judge creditworthiness from one transaction but aggregates many small signals, a PRS aggregates many small genetic signals into one composite score.

The discovery dataset produces GWAS effect sizes; the target dataset provides the genotypes () of new individuals the score will be applied to. The GWAS effect sizes are simply applied to the target genotypes using the PRS formula above.

0.4 What a PRS Does and Doesn’t Mean

A PRS is not a diagnosis and does not directly measure disease. It represents relative inherited genetic predisposition compared with others in the population — environmental factors still play a major role in whether disease actually develops.

Applications: cardiovascular medicine (identifying individuals with elevated genetic risk for early intervention), and more broadly across common disease research.

Limitations: prediction accuracy is fundamentally capped by the trait’s heritability, GWAS discovery sample size, and how well the discovery population’s ancestry matches the target population (Part 9 covers this in depth).

Key takeaways. A PRS is a weighted sum of genotype counts and GWAS effect sizes across many SNPs. It works even for non-causal SNPs because of LD with true causal variants. A PRS reflects relative genetic predisposition, not a diagnosis, and its accuracy is bounded by heritability and by how well discovery and target populations match.

1 Evaluating Polygenic Risk Scores

A PRS is only useful if it actually predicts the trait — so how well it predicts must be measured rigorously, on held-out data, using metrics that suit the outcome type.

1.1 Quantitative Traits: R² and Incremental R²

For continuous traits (height, BMI, LDL), the standard metric is R², the proportion of phenotypic variance explained. Fit a null model with covariates only, then a full model adding the PRS:

This isolates the variance explained specifically by genetics, over and above known covariates.

from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score

null_model.fit(X_cov, y)
r2_null = r2_score(y, null_model.predict(X_cov))

full_model.fit(X_full, y)
r2_full = r2_score(y, full_model.predict(X_full))

incremental_r2 = r2_full - r2_null

1.2 Disease Traits: AUC, Pseudo-R², Liability-Scale R²

Binary case/control outcomes need logistic regression and different metrics:

Pseudo-R² (McFadden, Nagelkerke, Cox-Snell) — analogous to R² but derived from a logistic model; interpretation differs from linear R², but larger is still better.
AUC (Area Under the ROC Curve) — the probability that a randomly chosen case receives a higher predicted risk than a randomly chosen control.

AUC	Interpretation
0.50	Random guessing
0.60	Weak
0.70	Moderate
0.80	Strong
0.90	Excellent

from sklearn.metrics import roc_auc_score
auc = roc_auc_score(y_true, prs_scores)

Real-world AUCs vary widely by trait and method: basic clumping-and-thresholding PRS for coronary artery disease or type 2 diabetes often land around 0.57–0.60, while well-optimized, large-biobank PRS have reported AUCs as high as 0.79–0.81 for coronary artery disease. There’s no single “typical” number — it depends heavily on trait architecture, discovery sample size, and how the PRS was built.

Liability-scale R² — case/control samples are usually enriched for cases relative to the true population prevalence, so raw R² on the observed scale can be misleading. The liability threshold model treats disease as arising once an underlying continuous liability (genetics + environment + noise) crosses a threshold; liability-scale R² estimates the proportion of that underlying liability explained by the PRS. This metric is especially common in psychiatric genetics.

1.3 Risk Stratification and Odds Ratios

Individuals are commonly split into PRS groups — deciles are typical — and disease prevalence compared across groups. Example: bottom decile odds ratio = 1.0 (reference), top decile odds ratio = 3.5, meaning individuals in the top PRS decile have 3.5× the disease odds of those in the bottom decile. A useful PRS shows monotonically increasing risk from lowest to highest group.

1.4 Avoiding Overfitting: Discovery, Tuning, and Target Sets

A major pitfall: building a PRS, testing many parameter choices, picking the best-performing one, and evaluating on the same individuals used for tuning — this inflates the apparent accuracy, because the model has effectively already “seen” the answer. The standard fix is three independent datasets:

Dataset	Role
Discovery	Runs the GWAS, produces SNP effect estimates
Tuning	Optimizes parameters (p-value thresholds, clumping settings, hyperparameters)
Target	Used once, to report final unbiased prediction performance

If individuals overlap between discovery and target sets, prediction accuracy is artificially inflated. These three samples must remain independent — this is the gold standard for reporting PRS performance.

Key takeaways. Quantitative traits are evaluated with incremental R²; disease traits with AUC, pseudo-R², liability-scale R², and odds ratios. Risk stratification (typically by decile) gives an intuitive picture of prediction performance. Overfitting from re-using tuning data is one of the most common and serious mistakes in PRS evaluation — discovery, tuning, and target datasets must stay independent.

2 Clumping and P-Value Thresholding (C+PT)

Given GWAS summary statistics, how do we decide which SNPs actually go into a PRS? Naively including every SNP creates problems: many SNPs are highly correlated via LD, many have noisy or negligible effect estimates, and millions of SNPs together introduce substantial statistical noise. Clumping and P-value Thresholding (C+PT) was one of the earliest and most influential answers to this, and despite newer Bayesian methods, it remains a widely used baseline.

2.1 Why Not Just Use Every SNP?

A GWAS hit on chromosome 6 might show 5 SNPs all with p-values around – — not 5 independent discoveries, but the same underlying signal viewed through SNPs inherited together via LD. Including all of them in a PRS effectively double- (or quintuple-) counts the same genetic information, producing redundant information, inflated variance, overfitting, and reduced prediction accuracy.

2.2 The Two-Step Procedure

Step 1 — Clumping removes redundant SNPs, keeping the most significant SNP in each LD region and discarding its correlated neighbors:

Sort SNPs by p-value (most significant first).
Select the most significant remaining SNP.
Remove all SNPs within a specified window (e.g. 250 kb) whose LD with it exceeds a threshold (e.g. ).
Move to the next remaining SNP and repeat until all SNPs are processed.

threshold	Interpretation
0.1	Strict — removes more SNPs
0.2	Moderate
0.5	Relaxed

Step 2 — P-value thresholding. After clumping, still-many SNPs must be filtered by significance. Researchers typically test several thresholds — , , , , 0.05, 0.1, 0.5, 1.0 — and pick whichever gives the best held-out prediction. Restricting to genome-wide-significant SNPs alone () is tempting but usually suboptimal for highly polygenic traits: many true causal variants never individually reach that threshold, so overly strict cutoffs discard real signal.

The tuning process. Compare incremental across thresholds on a tuning set:

Threshold	Incremental R²
5e-8	0.03
1e-6	0.05
1e-4	0.08
0.01	0.11
0.05	0.09

Here 0.01 wins. This produces the classic hump-shaped curve: too few SNPs misses real signal (low accuracy), a moderate number captures the optimal balance (peak accuracy), and too many SNPs lets noise dominate (accuracy falls again).

2.3 Scoring and Practical Use

The final score uses the same formula as before, restricted to the selected SNP set:

plink \
  --bfile target_data \
  --score selected_snps.txt 1 2 3 \
  --out prs_scores

(Column 1 = SNP ID, column 2 = effect allele, column 3 = effect size — PLINK multiplies genotype counts by effects and sums.)

2.4 Strengths and Limitations

Strengths: simple, computationally cheap, works directly from summary statistics, easy to reproduce — hence its continued use as the standard baseline. Limitations: discards potentially useful SNPs during clumping, handles LD only indirectly (via pruning, not explicit modeling), treats each SNP’s effect independently rather than jointly, and requires threshold tuning (adding complexity and overfitting risk).

These limitations motivated whole-genome regression and Bayesian methods — BLUP, SBLUP, LDpred, BayesR, BayesC, SBayesR, SBayesRC — which use all SNPs, model LD directly, and apply shrinkage. C+PT nonetheless remains the essential conceptual starting point for understanding them.

Key takeaways. C+PT selects SNPs in two steps: clumping removes LD-redundant SNPs, and p-value thresholding filters by significance, typically tuned across several candidate cutoffs. Prediction accuracy follows a hump-shaped curve against threshold stringency. C+PT is simple and widely used, but discards information and ignores joint LD structure — limitations that motivated modern Bayesian PRS methods.

3 Best Linear Unbiased Prediction (BLUP)

C+PT discards many SNPs, ignores joint LD structure, treats SNPs independently, and requires threshold tuning. BLUP represents a conceptual shift: use every SNP simultaneously rather than selecting a subset. Understanding BLUP matters because modern Bayesian methods (BayesR, SBayesR) are direct extensions of it.

3.1 The Infinitesimal Model

BLUP rests on one of the oldest ideas in quantitative genetics — the infinitesimal model: every SNP contributes to the trait, each with a very small effect, drawn from a shared normal distribution . Most SNP effects cluster near zero, symmetric around it, with very large effects rare. This is a surprisingly good approximation for many highly polygenic traits (height, BMI, schizophrenia — GWAS repeatedly show thousands of associated variants each).

“Best Linear Unbiased Prediction” breaks down as: Best = smallest prediction error variance among linear unbiased estimators; Linear = prediction is a linear combination of SNP effects (); Unbiased = predictions aren’t systematically too high or too low.

3.2 The Mixed Model Framework

are fixed effects (known covariates like age, sex, principal components, batch); is the genetic component, treated as a random effect: , where is the Genetic Relationship Matrix (GRM) — pairwise genetic similarity between individuals — and is genetic variance. Rather than estimating each SNP’s effect independently, BLUP models their collective contribution via this relationship structure: more genetically similar individuals are expected to be more phenotypically similar.

3.3 Shrinkage

BLUP doesn’t trust raw GWAS estimates fully — it pulls them toward zero, more aggressively for small/noisy estimates than large ones. Example: GWAS effects of 0.40, 0.05, 0.01 might become BLUP effects of 0.25, 0.03, 0.005. Because small-sample GWAS estimates can be inflated by chance, shrinkage separates “mostly signal” from “signal + noise,” reducing overfitting and improving out-of-sample prediction.

PRS calculation is otherwise unchanged — — the difference from C+PT is simply that every SNP contributes, not a selected subset.

3.4 Advantages and Limitations

Advantages: uses all SNPs (no discarding), no arbitrary p-value thresholds to tune, handles polygenicity naturally, and shrinkage reduces overfitting.

Limitations: assumes every SNP has a non-zero effect (unrealistic — many SNPs likely have none), assumes a single normal distribution for all effects (real traits likely have null, small-, medium-, and large-effect SNP classes that a single distribution can’t represent), and struggles to incorporate sparse architectures or functional annotations.

Feature	C+PT	BLUP
Uses all SNPs	No	Yes
Requires tuning	Yes	No
Models LD directly	Partially	Better
Shrinkage	No	Yes
Computational complexity	Low	Moderate

BLUP’s single-distribution assumption motivated researchers to ask: what if many SNPs truly have zero effect, effect sizes come from multiple distributions, or biological annotations could be incorporated? Answering these questions led to Bayesian methods (BayesA, BayesB, BayesC, BayesR, SBayesR, SBayesRC) — direct generalizations of the BLUP framework that relax its restrictive assumptions.

Key takeaways. BLUP is a whole-genome regression method using all SNPs simultaneously, under the infinitesimal-model assumption that effects are normally distributed. It relies on a Genetic Relationship Matrix and applies shrinkage to reduce overfitting, eliminating the need for clumping or p-value thresholds. Its core limitation — one shared normal distribution for every SNP effect — motivated the Bayesian methods that followed.

4 Bayesian Methods and MCMC

BLUP’s key limitation is assuming every SNP’s effect is drawn from one shared normal distribution — unrealistic, since many SNPs likely have zero effect while a few have outsized ones. Bayesian methods relax this by treating SNP effects as random variables with more flexible priors — the foundation for BayesA, BayesB, BayesC, BayesR, and their modern summary-statistics descendants SBayesR and SBayesRC.

4.1 The Bayesian Philosophy

Classical (frequentist) statistics treats a parameter like as a fixed but unknown true value. Bayesian statistics instead asks for its full probability distribution, combining:

Prior beliefs — what we assume before seeing data, e.g. , encoding “most effects are small, positive and negative are equally likely, large effects are rare.”
The likelihood — how probable the observed data is under a given parameter value.

via Bayes’ theorem: . If the prior expects but the GWAS data strongly suggests , the posterior lands somewhere in between (e.g. ) — naturally shrinking noisy GWAS estimates toward more plausible values, which often improves prediction.

4.2 BayesC and the Spike-and-Slab Prior

BayesC was the first influential model of this kind: some SNPs have exactly zero effect (, probability ), others are drawn from a normal distribution (, probability ). This is a spike-and-slab prior — a “spike” at zero for null SNPs, a “slab” (broader distribution) for real effects. Rather than forcing every SNP to contribute, BayesC lets the data determine which SNPs matter, reducing noise and improving shrinkage — especially when the true genetic architecture is sparse (e.g. 100 causal variants among 1,000,000 measured SNPs, which BLUP handles poorly).

4.3 Why MCMC Is Needed

Bayesian posteriors for a million SNPs at once have no closed-form solution — the parameter space is far too high-dimensional to compute directly. Markov Chain Monte Carlo (MCMC) solves this by sampling from the posterior instead of solving it analytically. It combines two ideas: a Markov chain (each new state depends only on the current state, not the full history) and Monte Carlo sampling (random draws used to approximate otherwise intractable quantities, in the spirit of estimating by randomly throwing darts at a circle inscribed in a square). Over many iterations, MCMC spends more time in high-posterior-probability regions and less in low-probability ones, and the accumulated samples reconstruct the posterior distribution.

Gibbs sampling is the standard MCMC algorithm here: rather than updating SNP effects (), genetic variance (), residual variance (), and the inclusion probability () all simultaneously, it updates them one at a time, cycling through thousands of iterations:

Update β → Update σ²β → Update σ²e → Update π → repeat

Burn-in. Early samples reflect arbitrary starting values rather than the true posterior, so they’re discarded — e.g. of 50,000 total iterations, the first 10,000 might be burn-in, leaving 40,000 retained samples for inference.

Convergence diagnostics. A trace plot (parameter value vs. iteration) that fluctuates stably around a constant mean suggests convergence; a plot with a clear upward/downward trend suggests the chain hasn’t stabilized yet.

Extracting estimates. After convergence, the posterior mean becomes the final SNP effect estimate, and posterior variance quantifies uncertainty (large variance = low confidence). A SNP’s Posterior Inclusion Probability (PIP) is simply the fraction of retained iterations in which it was included — e.g. included in 45,000 of 50,000 samples gives PIP = 0.90.

PIP	Interpretation
0.01	Almost certainly irrelevant
0.10	Weak evidence
0.50	Moderate evidence
0.90	Strong evidence
0.99	Very strong evidence

4.4 Strengths, Weaknesses, and the Path Forward

MCMC provides full posterior distributions, honest uncertainty estimates, and PIPs — real advantages over point-estimate methods. But it’s also computationally expensive (1 million SNPs × 50,000 iterations can take hours to days and substantial memory), can converge slowly, and can be hard to diagnose. This computational burden directly motivated SBayesR and SBayesRC, which achieve similar Bayesian flexibility using summary statistics and efficient sparse-matrix methods instead of raw individual-level MCMC over the full genotype matrix.

Key takeaways. Bayesian methods treat SNP effects as random variables with flexible priors rather than a single shared distribution. BayesC’s spike-and-slab prior lets some SNPs have exactly zero effect. Because the resulting posteriors have no closed-form solution, MCMC (typically Gibbs sampling) approximates them via iterative random sampling, discarding early “burn-in” draws and using posterior means and inclusion probabilities as final estimates. MCMC’s computational cost motivated the summary-statistics-based methods that followed.

5 SBayesR: Bayesian Prediction from Summary Statistics

MCMC over individual-level genotypes is powerful but computationally expensive at biobank scale. SBayesR (summary-data-based BayesR) reformulates the BayesR model to run on GWAS summary statistics plus an LD reference panel, instead of raw genotypes — making Bayesian polygenic prediction tractable at scale. It’s now one of the most widely used methods for building PRS from GWAS summary data, and the foundation for SBayesRC.

5.1 Why SBayesR Was Developed

Each earlier method had a specific gap: C+PT discards SNPs and ignores joint LD; BLUP forces every SNP into a single effect-size distribution; individual-level BayesC models sparsity well but needs raw genotype data and is computationally expensive at scale. Researchers wanted Bayesian flexibility, summary-statistics compatibility, and scalability together — SBayesR was the answer, implemented in the GCTB software.

5.2 The Mixture Prior

BLUP assumes one shared distribution, , for every SNP. BayesR (and SBayesR) instead uses a mixture of normal distributions with a point mass at zero:

where is the proportion of SNPs in class and scales that class’s variance. The standard BayesR/SBayesR configuration uses four components — — representing zero effect, small effect, medium effect, and large effect classes respectively (component 1 is a literal point mass at zero; components 2–4 scale by 0.01×, 0.1×, and 1× respectively). A typical starting mixture might set — most SNPs assigned near-zero effect, a small number carrying most of the real signal — and SBayesR estimates the actual proportions from the data during MCMC, which itself provides insight into the trait’s genetic architecture.

5.3 Recovering Joint Effects from Marginal GWAS Statistics

GWAS summary statistics report marginal effects — each SNP tested one at a time, ignoring correlation with its neighbors. The relationship to the true joint effects is approximately , where is the vector of observed GWAS effects, is the LD matrix, and is the joint effect vector SBayesR aims to recover. If rs1, rs2, rs3 are strongly correlated, a naive analysis might spread signal across all three; modeling them jointly through lets SBayesR assign most of the signal to one and shrink the others toward zero — improving both interpretability and prediction.

5.4 Making This Computationally Tractable

A full LD matrix for 1 million SNPs has entries — far too large to handle directly. Because most SNPs are only correlated with nearby variants, the vast majority of LD matrix entries are effectively zero, so SBayesR stores a sparse representation (reduced memory, faster computation), and can further use an eigen-decomposition to reduce the computational burden during MCMC sampling. These optimizations are what make biobank-scale SBayesR analyses feasible at all.

5.5 Running SBayesR

gctb \
  --sbayes R \
  --ldm ukbEURu_hm3_sparse \
  --gwas-summary trait.ma \
  --out trait_sbayesr

Internally, GCTB reads the summary statistics, loads the LD matrix, runs MCMC (updating SNP effects, mixture-component membership, genetic variance, and residual variance each iteration), and outputs posterior mean SNP effects to trait_sbayesr.snpRes and MCMC/parameter diagnostics to trait_sbayesr.parRes. These posterior effects are typically less noisy than raw GWAS effects, and are combined with target genotypes using PLINK’s --score, exactly as in earlier methods.

5.6 Strengths and Limitations

Strengths: uses all SNPs, models LD directly (joint rather than marginal effects), a flexible mixture prior that better matches real genetic architecture, works from summary statistics alone, and scales to biobank-sized analyses.

Limitations: treats every SNP as equally likely to be causal a priori — no biological information about coding regions, regulatory elements, or conservation is used; results depend on how well the LD reference panel matches the GWAS population; and it remains more computationally demanding than C+PT.

Feature	C+PT	BLUP	SBayesR
Uses all SNPs	No	Yes	Yes
Models LD	Partial	Better	Yes (jointly)
Summary statistics only	Yes	No	Yes
Mixture prior	No	No	Yes
Sparse architecture	No	No	Yes

That first limitation — ignoring biology — is exactly what SBayesRC addresses next.

Key takeaways. SBayesR extends BayesR to work from GWAS summary statistics and an LD reference panel, using a four-component mixture prior (a zero-effect spike plus three non-zero variance classes) instead of BLUP’s single normal distribution. Sparse LD matrices and eigen-decomposition make this tractable at biobank scale. It doesn’t yet use any biological/functional information — the motivation for SBayesRC.

6 SBayesRC: Adding Functional Annotations

SBayesR treats every SNP as equally likely to be causal before seeing the data — but decades of molecular biology say that’s not realistic. A SNP sitting in a protein-coding exon and a SNP sitting in an intergenic region shouldn’t necessarily get the same prior probability of being causal, even with identical GWAS p-values. SBayesRC extends SBayesR by letting functional annotations inform those priors.

6.1 Functional Annotations and Heritability Enrichment

Functional annotations describe biological properties of genomic regions — coding sequence, promoters, enhancers, evolutionarily conserved regions, open chromatin (DNase hypersensitivity sites), histone marks, and more. Each SNP can be tagged with one or more of these categories, forming an annotation matrix (SNP × annotation, 0/1 entries).

The key concept is heritability enrichment: some annotation categories explain disproportionately more heritability than their share of the genome would suggest. Illustratively:

Annotation	SNP Fraction	Heritability Fraction	Enrichment
Coding	2%	10%	5×
Conserved	5%	20%	4×
Intergenic	50%	20%	0.4×

An enrichment of 5 means that annotation category contributes five times more heritability than its size alone would predict — flagging it as more likely to harbor causal variants, and letting SBayesRC assign it a higher prior probability of non-zero effect during inference.

6.2 The Model Change

In SBayesR, the mixture-component probabilities are the same for every SNP. In SBayesRC, becomes a function of each SNP’s annotations — so two SNPs with identical GWAS evidence can receive different prior probabilities of being causal, depending on their biological context. This lets SBayesRC combine statistical evidence and biological plausibility, rather than statistical evidence alone.

6.3 Running SBayesRC

gctb \
  --sbayes RC \
  --ldm ukbEURu_hm3_sparse \
  --annot annotations.txt \
  --gwas-summary trait.ma \
  --out trait_sbayesrc

The only new input compared with SBayesR is the annotation file. Large-scale implementations commonly draw on baselineLD-style annotation sets spanning coding regions, conservation, regulatory elements, histone modifications, and gene expression — often dozens to hundreds of categories. Typical outputs add trait_sbayesrc.enrich, summarizing annotation-specific heritability enrichment, alongside the usual .snpRes and .parRes files.

6.4 Advantages and Limitations

Because it draws on both statistical and biological evidence, SBayesRC often modestly but consistently outperforms SBayesR across traits. That said, it inherits new failure modes: annotation quality directly affects performance (poor annotations can hurt more than help); it estimates more parameters, adding computational cost; annotations may not transfer cleanly across ancestries; and even the best current annotation sets capture only a fraction of real regulatory biology.

Feature	SBayesR	SBayesRC
Summary statistics	Yes	Yes
LD modeling	Yes	Yes
Mixture priors	Yes	Yes
Functional annotations	No	Yes
Heritability enrichment	No	Yes

Key takeaways. SBayesRC extends SBayesR by letting functional annotations modify each SNP’s prior probability of being causal, using heritability enrichment to identify biologically important genomic regions. Combining statistical and biological evidence often modestly improves prediction accuracy over SBayesR alone — though results still depend on annotation quality and completeness, and may not generalize equally across ancestries.

7 Practical PRS Analysis: PRSice, GCTB, PLINK

A complete PRS pipeline, connecting the theory above to actual tools:

7.1 Required Datasets and Files

Three datasets, as in Part 2: discovery (produces GWAS summary statistics), target (genotypes, phenotypes, covariates for evaluation), and an LD reference for methods that need it (SBayesR, SBayesRC, LDpred) — commonly UK Biobank, 1000 Genomes, or HapMap3.

A typical project layout:

GWAS/         trait.ma
Target/       target.bed, target.bim, target.fam
Covariates/   covariates.txt
Phenotypes/   phenotype.txt
LD/           ukb_ldm/

7.2 Step 1 — QC Summary Statistics and Target Genotypes

Check for missing SNP IDs, duplicate variants, strand ambiguity, allele mismatches, and incorrect sample sizes in the summary statistics — poor-quality summary stats reliably wreck downstream prediction.

plink \
  --bfile target \
  --geno 0.02 \
  --mind 0.02 \
  --maf 0.01 \
  --make-bed \
  --out target_QC

7.3 Step 2 — C+PT via PRSice

PRSice implements clumping, thresholding, PRS construction, and evaluation in one tool:

Rscript PRSice.R \
  --prsice PRSice_linux \
  --base trait.ma \
  --target target_QC \
  --pheno phenotype.txt \
  --cov covariates.txt \
  --stat BETA \
  --beta \
  --out PRS_output

--base is the GWAS summary file, --target the target genotypes, --pheno/--cov the phenotype and covariates for evaluation. Outputs include PRS_output.best (best p-value threshold), PRS_output.summary (incremental R², SNP counts), and PRS_output.all_score (scores at every threshold tested).

7.4 Step 3 — SBayesR and SBayesRC via GCTB

GCTB needs an LD reference matrix in addition to summary statistics — pre-computed sparse LD matrices derived from UK Biobank reference panels (e.g. ukbEURu_hm3_sparse) are commonly used:

gctb \
  --sbayes R \
  --ldm ukbEURu_hm3_sparse \
  --gwas-summary trait.ma \
  --out trait_sbayesr

For SBayesRC, add an annotation file and switch --sbayes R to --sbayes RC:

gctb \
  --sbayes RC \
  --ldm ukbEURu_hm3_sparse \
  --annot annotations.txt \
  --gwas-summary trait.ma \
  --out trait_sbayesrc

Both produce a .snpRes file with posterior SNP effects, generally less noisy than raw GWAS betas.

7.5 Step 4 — Scoring and Evaluation

plink \
  --bfile target_QC \
  --score score.txt 1 2 3 \
  --out target_PRS

producing per-individual scores (FID IID SCORE1_SUM). Evaluate exactly as in Part 2 — compare a full model (Phenotype ~ Sex + Age + PCs + PRS) against a null model (Phenotype ~ Sex + Age + PCs) and take the incremental :

null_model.fit(X_cov, y);  r2_null = r2_score(y, null_model.predict(X_cov))
full_model.fit(X_full, y); r2_full = r2_score(y, full_model.predict(X_full))
incremental_r2 = r2_full - r2_null

Then stratify by risk group (e.g. bottom 10% / middle 80% / top 10%) and compare disease prevalence across them.

Illustrative method comparison (values are trait-dependent, not universal constants):

Method	Incremental R²
C+PT	0.08
BLUP	0.10
SBayesR	0.13
SBayesRC	0.14

SBayesRC often — not always — achieves the highest accuracy among these; the actual ranking depends on the trait, sample size, and annotation quality available.

7.6 Common Problems and Best Practices

Common problems: allele mismatches between summary statistics and target genotypes (the single most frequent source of errors — always harmonize alleles first); an LD reference whose ancestry doesn’t match the GWAS population (degrades accuracy); sample overlap between discovery and target sets (inflates apparent performance); and poor-quality input summary statistics (garbage in, garbage out).

Best practices: thorough QC at every step, ancestry-matched LD references, strict independence between discovery/tuning/target samples, evaluation only on held-out data, and comparing multiple methods rather than trusting one blindly.

Key takeaways. PRSice runs C+PT end-to-end; GCTB runs SBayesR (--sbayes R) and SBayesRC (--sbayes RC, plus --annot); PLINK’s --score converts posterior SNP effects into individual-level PRS. Allele mismatches and LD reference mismatch are the most common practical failure points — careful QC and ancestry matching matter as much as method choice.

8 Interpreting PRS: What Do They Really Mean?

A high PRS does not mean “you will get the disease,” and a low PRS does not mean “you’re protected.” Both are common misreadings. A PRS is a statistical predictor, not a diagnosis — understanding its actual meaning matters for responsible use in research, medicine, and public communication.

8.1 Relative Position, Not Absolute Fate

A PRS represents an individual’s genetic predisposition relative to others in the same population — the operative word is relative. A height-PRS analogy: knowing Person C’s PRS is +2.4 and Person A’s is −2.1 doesn’t tell you their exact heights, but does suggest Person C is likely taller than Person A. The same logic carries over to disease risk.

8.2 Relative Risk vs. Absolute Risk

These two are frequently conflated. If average disease risk is 10% and the top PRS decile has “3× higher risk” (a relative risk statement), the absolute risk for that group becomes roughly 30% — meaningfully elevated, but nowhere near certainty. Even high-risk individuals usually don’t develop the disease. This distinction is essential when communicating PRS results to anyone outside a statistics-fluent audience.

8.3 Percentiles Make Raw Scores Interpretable

A raw score like “PRS = 1.42” means almost nothing by itself — interpretation improves enormously once it’s expressed as a percentile within a reference population (e.g. 99th percentile = higher than 99% of people). Most PRS distributions are approximately normal, with most individuals clustering near the middle and extreme scores becoming rarer toward the tails.

Illustrative risk stratification example (coronary artery disease):

Group	Disease Rate
Bottom 10%	3%
Average	7%
Top 10%	18%

Higher PRS tracks with higher disease frequency, but the majority of even the top-decile group still doesn’t develop the disease. Odds ratios (e.g. top decile 3.5× the odds of bottom decile) summarize this pattern compactly, but odds and probability aren’t the same thing and shouldn’t be conflated.

8.4 PRS Is Not Destiny

Phenotype = Genetics + Environment + Random effects, roughly speaking — even highly heritable diseases have substantial non-genetic contributors. For type 2 diabetes, diet, physical activity, obesity, smoking, and sleep all matter alongside genetic risk: a high-PRS individual may never develop disease, and a low-PRS individual isn’t guaranteed protection.

Heritability sets a hard ceiling. If a trait’s heritability is , only 40% of phenotypic variation is genetic in origin — even a theoretically perfect PRS cannot explain the other 60%, because that variance simply isn’t genetic. A high PRS suggests increased inherited susceptibility and elevated relative risk; it does not imply certainty of disease, an immediate clinical diagnosis, or the presence of symptoms.

8.5 Population Portability

A PRS trained on a European-ancestry GWAS and applied to an African-ancestry population typically loses substantial predictive accuracy — driven by differing allele frequencies, differing LD patterns (so tag SNPs correlated with a causal variant in one population may be poorly correlated with it in another), and differing environmental exposures. This portability problem is one of the major open challenges in the field (Part 10 covers it further).

8.6 Clinical and Ethical Considerations

Potential clinical uses include early screening, targeting preventive interventions, and risk communication as part of personalized medicine — but clinical implementation is still an active area of research, not settled practice. PRS should be one input among many (family history, lifestyle, environmental exposures, medical records, biomarkers), not a standalone basis for major decisions. Open ethical questions include data privacy, potential for discrimination (e.g. insurance, employment), how to communicate risk responsibly, and equitable distribution of benefit across ancestries currently underrepresented in GWAS.

8.7 Common Misinterpretations, Corrected

Misinterpretation	Reality
High PRS = disease	False — it’s elevated relative risk, not certainty
Low PRS = protected	False — non-genetic factors still matter
Genetics = destiny	False — environment and chance both contribute substantially
PRS works equally in every population	False — accuracy varies by ancestry match to the discovery GWAS

Key takeaways. A PRS measures relative genetic predisposition, not a diagnosis. Relative risk and absolute risk are different quantities and shouldn’t be conflated; percentiles are usually more interpretable than raw scores. Prediction accuracy is capped by heritability and degrades when applied across ancestries different from the discovery GWAS. Responsible use treats PRS as one input among several, not a standalone predictor of fate.

9 The Future of PRS: Multi-Ancestry, Functional Genomics, Precision Medicine

Despite real progress — from simple weighted sums to SBayesRC’s Bayesian, LD-aware, annotation-informed models — today’s PRS methods remain far from perfect. Major open challenges: limited cross-ancestry portability, missing heritability, gene-environment interactions, rare variants, and clinical implementation.

Prediction accuracy today varies substantially by trait: it’s generally strongest for traits like height, and more modest for BMI, coronary artery disease, type 2 diabetes, schizophrenia, and especially depression — reflecting real differences in genetic architecture, GWAS sample size, and environmental contribution across traits, not just tool limitations.

9.1 The Multi-Ancestry Problem

Most large GWAS to date have been conducted in European-ancestry cohorts, so PRS trained on them predict well within European-ancestry populations but substantially worse when applied to other ancestries. Three main drivers: differing allele frequencies (a SNP common in one population may be rare in another, changing its statistical power to contribute to prediction); differing LD patterns (a tag SNP correlated with a causal variant in one population may be weakly correlated with it in another, breaking the LD-based logic PRS relies on); and differing environmental exposures (diet, healthcare access, socioeconomic conditions), meaning even identical genetics can translate to different outcomes.

Multi-ancestry PRS methods, combining GWAS data across ancestral groups and modeling both shared and population-specific genetic effects, are an active area of development aimed at improving generalization — though this remains one of the field’s largest unsolved problems, and the underrepresentation of non-European ancestries in existing GWAS is a structural, not just methodological, limitation.

9.2 Larger Samples, Rarer Variants

Prediction accuracy scales with discovery GWAS sample size. Major biobanks now include roughly: UK Biobank (~500,000 participants), FinnGen (~500,000), the Million Veteran Program (over 1,000,000), and All of Us (targeting over 1,000,000) — with future resources likely to grow substantially larger still.

Most current PRS focus on common variants; rare variants may carry larger individual biological effects but are harder for standard GWAS to detect reliably, since statistical power drops sharply as allele frequency falls. Integrating common, rare, and structural variation into a single predictive framework is an active research direction.

9.3 Functional Genomics and Multi-Omics

Beyond the coding/regulatory annotations SBayesRC already uses, future models may incorporate gene expression (eQTLs), chromatin accessibility, single-cell-resolved cell-type-specific effects, and epigenetic marks (DNA methylation, histone modifications) — moving toward genuinely multi-omics prediction. Related approaches like TWAS (transcriptome-wide association studies), PrediXcan, and FUSION predict disease via genetically-predicted gene expression as an intermediate step (SNPs → predicted expression → disease), rather than jumping directly from SNPs to phenotype — potentially yielding more biologically interpretable predictions.

Gene-environment interaction (G×E). Standard PRS models assume genetic and environmental contributions are additive/independent, but reality is often more complex — a genetic predisposition to obesity, for instance, may be amplified by a high-calorie diet and sedentary lifestyle, or dampened by exercise and healthy nutrition. Modeling these interactions explicitly, rather than assuming independence, is an active area of methods development.

9.4 Machine Learning, and What Actually Moves the Needle

Neural networks, graph neural networks, transformer architectures, and other deep-learning approaches are being explored for genomic prediction, offering potential to capture nonlinear effects and integrate heterogeneous data types. So far, though, evidence suggests that for most current PRS applications, better data tends to matter more than more complex models — larger, more diverse, better-QC’d GWAS typically move prediction accuracy further than swapping in a more sophisticated architecture on the same underlying data.

9.5 Clinical Translation

Potential applications span cardiovascular disease (earlier intervention for high-risk individuals), cancer screening (risk-stratified programs), psychiatry (earlier identification of vulnerable individuals), and preventive medicine generally. Real obstacles to clinical adoption remain: calibration (predicted risks must actually match observed outcomes), equity (prediction must work reasonably across diverse populations, not just those well-represented in discovery GWAS), interpretability (clinicians need understandable, actionable outputs), and ethics (privacy and fairness). These are as important to solve as any statistical improvement.

The long-term vision — often called precision medicine — combines individual genome, environment, and medical history into personalized healthcare; PRS is likely to become one input among several in that broader picture, not a stand-alone diagnostic tool.

9.6 How the Field Got Here

Era	Approach
Early	Candidate gene studies
GWAS era	Single-SNP association analysis
PRS era	C+PT
Whole-genome era	BLUP
Bayesian era	BayesR
Summary-statistics era	SBayesR
Functional genomics era	SBayesRC

Each generation incorporated more information and more realistic assumptions about genetic architecture than the last.

Final key takeaways. PRS is now central to statistical genetics, and accuracy keeps improving as datasets grow — but multi-ancestry portability, missing heritability, gene-environment interactions, and rare variants remain substantial open problems. Deep learning may help at the margins, but data quality and quantity currently matter more than model sophistication. Clinical implementation requires careful validation on calibration, equity, and interpretability, not just prediction accuracy. Precision medicine — integrating genetics, environment, and clinical data — is a long-term goal the field is still working toward, not a present reality.

A Complete GWAS Practical Tutorial Using PLINK, GCTA, R, and METAL

Nivedita Bhadra — Sun, 07 Jun 2026 21:00:00 GMT

A Manhattan plot: each point is a SNP, plotted by genomic position (x-axis) and significance (y-axis). Towers that cross the genome-wide threshold are the signals a GWAS is built to find.

PLINK GCTA METAL R Population Genetics

A note on how to use this tutorial. PLINK, GCTA, and METAL are external command-line tools – install them separately and run the shown commands against your own genotype files. They are not part of this notebook’s Python kernel and are not executed here. To make the core statistical concepts concrete without requiring real genotype data, several sections include small, genuinely executed Python simulations (clearly marked) that reproduce the shape of the real output – a heterozygosity distribution, a stratified PCA plot, QQ plot patterns, a Manhattan plot, and a GRM heatmap – using synthetic data, not results from the commands above.

Genome-wide association studies, or GWAS, are used to identify genetic variants associated with traits or diseases.

A GWAS usually tests hundreds of thousands to millions of SNPs across the genome.

For each SNP, we ask:

Is genetic variation at this SNP statistically associated with the phenotype?

In this tutorial, we walk through a complete GWAS workflow using:

PLINK for genotype quality control
PCA for population structure
GCTA fastGWA for mixed-model association testing
R for QQ plots and Manhattan plots
METAL for meta-analysis
GRM inspection for relatedness checks

The goal is not only to run commands, but to understand why each step matters.

0.0.1 What You Will Learn

How to prepare genotype and phenotype files for GWAS
How to perform sample and SNP quality control with PLINK
How to compute principal components and build a GRM
How to run mixed-model GWAS with GCTA fastGWA
How to visualize results with QQ and Manhattan plots
How to perform a simple meta-analysis with METAL
How to inspect relatedness and heterozygosity in GWAS data

0.0.2 Who Should Read This Tutorial

This tutorial is intended for researchers and students who want a practical GWAS workflow using standard tools in statistical genetics. Prior experience with command-line tools, PLINK, and basic genomics concepts is helpful but not required.

0.0.3 Overview of the GWAS Pipeline

The complete workflow is:

Prepare genotype and phenotype files
Perform genotype quality control
Remove low-quality SNPs
Remove low-quality individuals
Check heterozygosity outliers
Compute principal components
Build a genetic relationship matrix
Run GWAS without PC adjustment
Run GWAS with PC adjustment
Compare QQ plots and Manhattan plots
Meta-analyze two GWAS results
Inspect heterogeneity
Identify the top SNP
Inspect relatedness using the GRM

0.0.4 Why Quality Control is Necessary?

Before running GWAS, we must clean the genotype data.

Poor genotype quality can create false associations.

Common problems include:

SNPs missing in many individuals
Individuals missing many genotypes
Very rare variants with unstable estimates
SNPs violating Hardy-Weinberg equilibrium
Sample contamination
Inbreeding
Unexpected relatives
Population stratification

A GWAS is only as reliable as the data used to run it.

0.0.5 Data Used in This Tutorial

We assume two simulated studies:

study1.bed
study1.bim
study1.fam

study2.bed
study2.bim
study2.fam

study1_pheno.txt
study2_pheno.txt

study1_covariates.txt
study2_covariates.txt

Each study contains:

around 2,000 individuals
around 50,000 SNPs
one phenotype
covariates including sex, age, and PCs

0.0.6 PLINK File Formats

PLINK binary genotype data are stored in three files.

File	Meaning
`.bed`	Binary genotype data
`.bim`	SNP information
`.fam`	Individual information

The .bim file contains SNP-level information:

chromosome  SNP_ID  genetic_distance  base_pair_position  allele1  allele2

The .fam file contains individual-level information:

FID  IID  father_ID  mother_ID  sex  phenotype

The .bed file stores the actual genotype matrix in compressed binary format.

0.0.7 Create Working Directory

mkdir GWAS
cd GWAS
# Place your own study1.{bed,bim,fam}, study2.{bed,bim,fam},
# phenotype (study1_pheno.txt, study2_pheno.txt), and
# covariate (study1_covariates.txt, study2_covariates.txt) files here.

Check files:

ls

Inspect the genotype files:

head study1.fam
head study1.bim

Inspect phenotype and covariate files:

head study1_pheno.txt
head study1_covariates.txt

0.0.8 Question: Is the Phenotype Quantitative or Case-Control?

Look at the phenotype file:

head study1_pheno.txt

If the phenotype is continuous, such as height, BMI, or simulated trait value, then it is quantitative.

If the phenotype is coded as 0/1 or 1/2 for disease status, then it is case-control.

In this practical, the phenotype is treated as a quantitative phenotype because fastGWA is run using a linear mixed model.

0.0.8.1 Software Used

We use:

Tool	Purpose
PLINK 1.9	QC and genotype processing
GCTA	GRM construction and fastGWA
R	Plotting and file preparation
qqman	QQ plots and Manhattan plots
METAL	Meta-analysis

1 Part 1: Quality Control in PLINK

Quality control is one of the most important parts of GWAS.

The order matters.

We usually clean SNPs first and individuals second.

Why?

If many SNPs are poor quality, individuals may appear to have high missingness just because those SNPs failed. So we first remove bad SNPs, then evaluate individual-level quality.

1.0.1 Step 1.1: Get an Overview of the Data

Run:

plink --bfile study1 --freq --out study1_freqs

This command:

reads study1.bed, study1.bim, and study1.fam
reports number of individuals
reports number of SNPs
computes allele frequencies

Output files:

study1_freqs.frq
study1_freqs.log

Inspect:

head study1_freqs.frq

The .frq file contains allele frequency information.

Important columns usually include:

Column	Meaning
CHR	Chromosome
SNP	SNP ID
A1	Allele 1
A2	Allele 2
MAF	Minor allele frequency
NCHROBS	Number of observed chromosomes

1.0.2 Step 1.2: SNP Missingness

Some SNPs fail genotyping in many individuals.

This is measured by SNP missingness.

Run:

plink --bfile study1 --missing --out study1_miss

This creates:

study1_miss.imiss
study1_miss.lmiss

File	Meaning
`.imiss`	Missingness per individual
`.lmiss`	Missingness per SNP

Inspect SNP missingness:

head study1_miss.lmiss

The important column is:

F_MISS

If:

F_MISS = 0.05

then the SNP is missing in 5% of individuals.

1.0.3 Remove SNPs With Missingness > 2%

We keep SNPs with call rate at least 98%.

plink --bfile study1 \
  --geno 0.02 \
  --make-bed \
  --out study1_geno

Explanation:

Flag	Meaning
`--bfile study1`	Input PLINK binary dataset
`--geno 0.02`	Remove SNPs missing in more than 2% of individuals
`--make-bed`	Write new PLINK binary files
`--out study1_geno`	Output prefix

Output:

study1_geno.bed
study1_geno.bim
study1_geno.fam

1.0.4 Step 1.3: Minor Allele Frequency Filtering

Minor allele frequency, or MAF, is the frequency of the less common allele.

Rare SNPs are difficult to test in small samples because:

there are few minor allele carriers
standard errors become large
genotype errors can have large influence
power is low

We remove SNPs with MAF below 1%.

plink --bfile study1_geno \
  --maf 0.01 \
  --make-bed \
  --out study1_maf

This removes SNPs with:

MAF < 0.01

1.1 Should the MAF Threshold Change in a Larger Study?

Yes, possibly.

In a much larger sample, such as 100,000 or 500,000 individuals, we may have enough power to analyze rarer variants.

For example:

Sample Size	Reasonable MAF threshold
2,000	1% or 5%
50,000	0.5% or 1%
500,000	0.1% may be possible

However, rare variant analysis often requires additional care, such as burden tests or sequence-level QC.

1.1.1 Step 1.4: Hardy-Weinberg Equilibrium

Hardy-Weinberg equilibrium gives expected genotype frequencies under random mating.

For allele frequency:

the expected genotype frequencies are:

Strong deviation from HWE can indicate:

genotyping error
population stratification
inbreeding
selection
true disease association in case-control data

Remove SNPs with HWE p-value below:

plink --bfile study1_maf \
  --hwe 1e-6 \
  --make-bed \
  --out study1_qc_snps

1.2 Why Test HWE Only in Controls for Case-Control Studies?

In case-control studies, a truly disease-associated SNP may deviate from HWE among cases.

If we test HWE in all individuals or in cases, we may accidentally remove true disease signals.

Therefore, for case-control GWAS, HWE filtering is often performed in controls only.

1.2.1 Step 1.5: Individual Missingness

After SNP QC, we check missingness per individual.

Some individuals may have poor DNA quality and many missing genotypes.

Remove individuals missing more than 2% of genotypes:

plink --bfile study1_qc_snps \
  --mind 0.02 \
  --make-bed \
  --out study1_qc_mind

Explanation:

Flag	Meaning
`--mind 0.02`	Remove individuals with missingness > 2%

1.2.2 Step 1.6: Heterozygosity Check

Heterozygosity is the proportion of SNPs where an individual carries two different alleles.

Individuals with unusually high heterozygosity may indicate:

sample contamination
DNA mixture
technical artifacts

Individuals with unusually low heterozygosity may indicate:

inbreeding
long runs of homozygosity
poor genotype calling

Compute heterozygosity:

plink --bfile study1_qc_mind \
  --het \
  --out study1_het

Inspect:

head study1_het.het

Important columns:

Column	Meaning
O(HOM)	Observed homozygous genotypes
E(HOM)	Expected homozygous genotypes
N(NM)	Number of non-missing genotypes
F	Inbreeding coefficient estimate

2 Plot Heterozygosity in R

Switch to R.

setwd("YOUR_WORKING_DIRECTORY")

het <- read.table("study1_het.het", header = TRUE)

het$het_rate <- (het$N.NM. - het$O.HOM.) / het$N.NM.

hist(
  het$het_rate,
  breaks = 50,
  main = "Heterozygosity Rate",
  xlab = "Heterozygosity"
)

upper <- mean(het$het_rate) + 3 * sd(het$het_rate)
lower <- mean(het$het_rate) - 3 * sd(het$het_rate)

abline(v = upper, col = "red", lty = 2)
abline(v = lower, col = "red", lty = 2)

3 Identify Heterozygosity Outliers

outliers <- het[
  het$het_rate < lower | het$het_rate > upper,
  c("FID", "IID")
]

cat("Heterozygosity outliers:", nrow(outliers), "\n")

write.table(
  outliers,
  "study1_het_outliers.txt",
  row.names = FALSE,
  col.names = FALSE,
  quote = FALSE,
  sep = "\t"
)

4 Remove Heterozygosity Outliers

Back in the terminal:

plink --bfile study1_qc_mind \
  --remove study1_het_outliers.txt \
  --make-bed \
  --out study1_qc

Now study1_qc is the final QC’d Study 1 dataset.

5 Step 1.7: Summarize QC Results

Check final SNP and individual counts:

plink --bfile study1_qc \
  --freq \
  --out study1_qc_freqs

You should summarize:

QC Step	Output Prefix	What Was Removed
Raw data	study1	None
SNP missingness	study1_geno	SNPs with missingness > 2%
MAF filter	study1_maf	SNPs with MAF < 1%
HWE filter	study1_qc_snps	SNPs with HWE p < 1e-6
Individual missingness	study1_qc_mind	Individuals with missingness > 2%
Heterozygosity	study1_qc	Heterozygosity outliers

6 Repeat QC for Study 2

Now repeat the same commands for Study 2.

7 Study 2: Overview

plink --bfile study2 --freq --out study2_freqs

8 Study 2: SNP Missingness

plink --bfile study2 --missing --out study2_miss

head study2_miss.lmiss

plink --bfile study2 \
  --geno 0.02 \
  --make-bed \
  --out study2_geno

9 Study 2: MAF Filtering

plink --bfile study2_geno \
  --maf 0.01 \
  --make-bed \
  --out study2_maf

10 Study 2: HWE Filtering

plink --bfile study2_maf \
  --hwe 1e-6 \
  --make-bed \
  --out study2_qc_snps

11 Study 2: Individual Missingness

plink --bfile study2_qc_snps \
  --mind 0.02 \
  --make-bed \
  --out study2_qc_mind

12 Study 2: Heterozygosity

plink --bfile study2_qc_mind \
  --het \
  --out study2_het

In R:

het2 <- read.table("study2_het.het", header = TRUE)

het2$het_rate <- (het2$N.NM. - het2$O.HOM.) / het2$N.NM.

hist(
  het2$het_rate,
  breaks = 50,
  main = "Study 2 Heterozygosity Rate",
  xlab = "Heterozygosity"
)

upper2 <- mean(het2$het_rate) + 3 * sd(het2$het_rate)
lower2 <- mean(het2$het_rate) - 3 * sd(het2$het_rate)

abline(v = upper2, col = "red", lty = 2)
abline(v = lower2, col = "red", lty = 2)

outliers2 <- het2[
  het2$het_rate < lower2 | het2$het_rate > upper2,
  c("FID", "IID")
]

cat("Study 2 heterozygosity outliers:", nrow(outliers2), "\n")

write.table(
  outliers2,
  "study2_het_outliers.txt",
  row.names = FALSE,
  col.names = FALSE,
  quote = FALSE,
  sep = "\t"
)

Back in terminal:

plink --bfile study2_qc_mind \
  --remove study2_het_outliers.txt \
  --make-bed \
  --out study2_qc

Check final Study 2 data:

plink --bfile study2_qc \
  --freq \
  --out study2_qc_freqs

# Illustrative simulation -- NOT real output from the commands above.
# Reproduces the shape of a typical study1_het.het heterozygosity distribution
# and the +/- 3 SD outlier rule described in the text.

import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng(42)

n = 2000
het_rate = rng.normal(0.30, 0.02, n)
outlier_idx = rng.choice(n, 15, replace=False)
het_rate[outlier_idx] += rng.choice([-1, 1], 15) * rng.uniform(0.08, 0.15, 15)

mean_h, sd_h = het_rate.mean(), het_rate.std()
upper, lower = mean_h + 3 * sd_h, mean_h - 3 * sd_h
outliers = np.where((het_rate > upper) | (het_rate < lower))[0]

fig, ax = plt.subplots(figsize=(7, 4.2))
ax.hist(het_rate, bins=50, color="#2f6f6b", alpha=0.75, edgecolor="#fbfaf7")
ax.axvline(upper, color="#b9812c", linestyle="--", linewidth=1.6)
ax.axvline(lower, color="#b9812c", linestyle="--", linewidth=1.6)
ax.set_title("Heterozygosity Rate Across Individuals (Simulated)")
ax.set_xlabel("Heterozygosity rate")
ax.set_ylabel("Number of individuals")
plt.tight_layout()
plt.show()

print(f"Flagged {len(outliers)} heterozygosity outliers out of {n} individuals")

Flagged 15 heterozygosity outliers out of 2000 individuals

12.0.1 QC Interpretation

At the end of QC, we should ask:

How many SNPs were removed due to missingness?
How many SNPs were removed due to low MAF?
How many SNPs were removed due to HWE deviation?
How many individuals were removed due to missingness?
How many individuals were removed due to heterozygosity outliers?
Are the remaining sample sizes reasonable?
Are the remaining SNP counts reasonable?

12.0.2 Why This QC Pipeline Matters

Skipping SNP missingness filtering can leave unreliable variants.

Skipping MAF filtering can produce unstable rare-variant tests.

Skipping HWE filtering can leave genotyping artifacts.

Skipping individual missingness filtering can leave poor-quality samples.

Skipping heterozygosity checks can leave contaminated or inbred samples.

In GWAS, even small QC problems can create genome-wide false positives.

12.0.3 End of Part 1

At this point we have:

inspected genotype files
computed allele frequencies
removed poorly genotyped SNPs
removed rare variants
removed HWE outliers
removed low-quality individuals
removed heterozygosity outliers
produced final QC’d datasets:

study1_qc
study2_qc

Next we will compute principal components to detect and correct population structure.

12.1 Part 2: Principal Components Analysis (PCA) and Population Stratification

12.1.0.1 Why Do We Need PCA?

Suppose we perform a GWAS for a disease.

Imagine:

Group A has ancestry from Northern Europe
Group B has ancestry from Southern Europe

Now suppose:

Disease prevalence differs between groups
Allele frequencies differ between groups

A SNP can appear associated with disease simply because ancestry differs.

This creates a false positive association.

This phenomenon is called:

12.1.1 Population Stratification

Population stratification is one of the largest sources of false positives in GWAS.

A GWAS can produce apparently significant associations even when no causal effect exists.

12.1.1.1 Example

Suppose:

Population	Disease Rate	Allele Frequency
Population A	10%	0.20
Population B	40%	0.80

Even if the SNP has nothing to do with disease:

cases contain more individuals from Population B
controls contain more individuals from Population A

The SNP becomes associated with disease.

The association is completely spurious.

13 How PCA Helps

Principal Components Analysis identifies major axes of genetic variation.

Instead of looking at:

50,000 SNPs

we summarize ancestry using:

PC1
PC2
PC3
...
PC10

These PCs can then be included as covariates in the GWAS model.

The PCs absorb ancestry differences.

This dramatically reduces false positives.

14 Mathematical Intuition

Suppose our genotype matrix is:

with dimensions:

where:

N = individuals
M = SNPs

For example:

PCA decomposes the genotype matrix into orthogonal directions of variation.

Mathematically:

where:

U contains individual scores
D contains singular values
V contains SNP loadings

The first principal component captures the largest source of genetic variation.

Often:

PC1 reflects ancestry
PC2 reflects ancestry
PC3 reflects finer population structure

# Illustrative simulation -- NOT real output from PLINK --pca.
# Simulates genotypes for two diverged populations plus an admixed group,
# to make the "How PCA Helps" discussion above concrete.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA

rng = np.random.default_rng(42)
n_per_group, n_snps = 300, 300

def simulate_genotypes(n_ind, n_snps, allele_freqs):
    return rng.binomial(2, allele_freqs, size=(n_ind, n_snps))

freqs_pop_a = rng.uniform(0.05, 0.5, n_snps)
freqs_pop_b = np.clip(freqs_pop_a + rng.normal(0, 0.25, n_snps), 0.01, 0.99)
freqs_admixed = (freqs_pop_a + freqs_pop_b) / 2

geno_a = simulate_genotypes(n_per_group, n_snps, freqs_pop_a)
geno_b = simulate_genotypes(n_per_group, n_snps, freqs_pop_b)
geno_admixed = simulate_genotypes(150, n_snps, freqs_admixed)

geno = np.vstack([geno_a, geno_b, geno_admixed])
labels = (["Population A"] * n_per_group + ["Population B"] * n_per_group +
          ["Admixed"] * 150)

geno_std = (geno - geno.mean(axis=0)) / (geno.std(axis=0) + 1e-8)
pcs = PCA(n_components=2).fit_transform(geno_std)

fig, ax = plt.subplots(figsize=(6.5, 5.5))
colors = {"Population A": "#2f6f6b", "Population B": "#b9812c", "Admixed": "#4a5a68"}
for grp in ["Population A", "Population B", "Admixed"]:
    mask = np.array(labels) == grp
    ax.scatter(pcs[mask, 0], pcs[mask, 1], s=18, alpha=0.7, color=colors[grp], label=grp)
ax.set_title("PCA on Simulated Genotypes: Population Stratification")
ax.set_xlabel("PC1")
ax.set_ylabel("PC2")
ax.legend(frameon=False)
plt.tight_layout()
plt.show()

15 Why LD Pruning is Necessary

Before PCA we must remove highly correlated SNPs.

Otherwise:

large LD blocks dominate the PCA
certain chromosomes become overrepresented
ancestry signals become distorted

Therefore we first perform:

15.1 LD Pruning

16 Linkage Disequilibrium Refresher

Linkage disequilibrium (LD) measures correlation between nearby SNPs.

For two SNPs:

measures their correlation.

Values:

r²	Interpretation
0	Independent
1	Perfect correlation

PCA works best when SNPs are approximately independent.

17 Step 2.1 LD Pruning

Run:

plink --bfile study1_qc \
      --indep-pairwise 50 5 0.2 \
      --out study1_prune

18 Understanding the Command

--indep-pairwise 50 5 0.2

means:

Parameter	Meaning
50	Window size
5	Step size
0.2	r² threshold

PLINK:

examines 50 SNP windows
shifts by 5 SNPs
removes SNPs with:

19 Output Files

study1_prune.prune.in
study1_prune.prune.out

19.0.1 SNPs Retained

wc -l study1_prune.prune.in

These SNPs will be used for PCA.

19.0.2 SNPs Removed

wc -l study1_prune.prune.out

These SNPs were excluded because of LD.

20 Why Not Use All SNPs?

Imagine a chromosome region containing:

500 highly correlated SNPs

Without pruning:

that region contributes 500 times
another region contributes only once

The PCA becomes biased.

LD pruning gives each genomic region approximately equal influence.

21 Step 2.2 Compute Principal Components

Now compute PCs using only pruned SNPs.

plink --bfile study1_qc \
      --extract study1_prune.prune.in \
      --pca 10 \
      --out study1_pca

22 Output Files

study1_pca.eigenvec
study1_pca.eigenval

The eigenvectors contain PC scores.

Inspect:

head study1_pca.eigenvec

Example:

FID IID PC1 PC2 PC3 ...

Each row represents one individual.

22.0.1 Variance Explained

Inspect:

cat study1_pca.eigenval

The first eigenvalue corresponds to:

PC1

The second eigenvalue corresponds to:

PC2

Larger eigenvalues indicate stronger axes of variation.

22.0.1.1 Visualizing Principal Components

Switch to R.

pcs <- read.table(
  "study1_pca.eigenvec",
  header = FALSE
)

colnames(pcs) <- c(
  "FID",
  "IID",
  paste0("PC",1:10)
)

23 PC1 vs PC2

plot(
  pcs$PC1,
  pcs$PC2,
  xlab="PC1",
  ylab="PC2",
  main="Study 1: PC1 vs PC2",
  pch=20,
  col="steelblue"
)

24 Interpretation

Each point is one individual.

Individuals close together are genetically similar.

Individuals far apart are genetically different.

25 Possible Outcomes

25.1 Scenario 1: Single Cloud

*******
*********
*******

Interpretation:

relatively homogeneous population
little population structure

25.2 Scenario 2: Two Clusters

****      ****
****      ****

Interpretation:

two ancestry groups
strong population stratification

25.3 Scenario 3: Gradient

****
  ****
      ****

Interpretation:

continuous ancestry variation
admixture

25.3.0.1 Real Example

If we analyzed:

Europeans
Africans
East Asians

PC1 and PC2 often separate groups almost perfectly.

The resulting plot contains three distinct clusters.

25.3.1 Quantifying Variance Explained

Create a scree plot.

eig <- scan(
  "study1_pca.eigenval"
)

var_exp <- eig/sum(eig)

plot(
  var_exp,
  type="b",
  pch=19,
  xlab="Principal Component",
  ylab="Variance Explained",
  main="Scree Plot"
)

25.3.2 Interpretation

Usually:

PC1 explains most variance
PC2 explains less
later PCs explain progressively less

A sharp drop indicates the important ancestry dimensions.

25.3.3 Visualizing Multiple PCs

PC1 vs PC3:

plot(
  pcs$PC1,
  pcs$PC3,
  pch=20,
  col="darkgreen",
  xlab="PC1",
  ylab="PC3"
)

PC2 vs PC3:

plot(
  pcs$PC2,
  pcs$PC3,
  pch=20,
  col="firebrick",
  xlab="PC2",
  ylab="PC3"
)

Sometimes structure only appears in later PCs.

25.3.4 Why PCs Become Covariates

Suppose:

is phenotype.

Instead of fitting:

we fit:

The PCs absorb ancestry effects.

This reduces false associations.

25.3.5 How Many PCs Should Be Used?

Common choices:

Dataset	Typical PCs
Small GWAS	5–10
UK Biobank	10–20
Highly diverse cohort	20–40

There is no universal answer.

Researchers often inspect scree plots and genomic inflation.

26 PCA and Relatedness

Close relatives can distort PCA.

For example:

siblings
parent-child pairs
cousins

may create artificial clusters.

Best practice:

Identify unrelated individuals.
Compute PCs on unrelateds.
Project PCs onto relatives.

Tools commonly used:

PLINK2
flashPCA
EIGENSOFT

For this practical we compute PCs directly because the number of relatives is small.

27 Repeat PCA for Study 2

Perform the same steps.

LD pruning:

plink --bfile study2_qc \
      --indep-pairwise 50 5 0.2 \
      --out study2_prune

Compute PCs:

plink --bfile study2_qc \
      --extract study2_prune.prune.in \
      --pca 10 \
      --out study2_pca

Plot:

pcs2 <- read.table(
  "study2_pca.eigenvec",
  header=FALSE
)

colnames(pcs2) <- c(
  "FID",
  "IID",
  paste0("PC",1:10)
)

plot(
  pcs2$PC1,
  pcs2$PC2,
  pch=20,
  col="darkorange",
  xlab="PC1",
  ylab="PC2",
  main="Study 2: PC1 vs PC2"
)

28 Summary

In this section we learned:

Population stratification creates false positive GWAS hits.
PCA identifies ancestry differences.
LD pruning is required before PCA.
Principal components summarize genetic variation.
PC1 and PC2 often represent ancestry.
PCs are added as covariates in GWAS.
PCA substantially reduces confounding.

At this point we have:

QC’d genotype data
ancestry estimates
principal components

Next we will build a Genetic Relationship Matrix (GRM) and run a mixed-model GWAS using GCTA fastGWA.

29 Part 3: Genetic Relationship Matrices (GRMs) and Mixed-Model GWAS with GCTA fastGWA

30 Why Ordinary GWAS Can Fail

Suppose we perform a simple GWAS.

For each SNP we fit:

where:

(Y) is the phenotype
SNP is genotype dosage (0,1,2)
(_1) is the SNP effect

This works well if:

individuals are unrelated
there is no population structure

Unfortunately, real cohorts violate both assumptions.

Examples:

siblings
cousins
parent-offspring pairs
population substructure

These create correlation among observations.

Ordinary regression assumes observations are independent.

Violation of this assumption leads to:

inflated test statistics
false positives
incorrect p-values

31 Relatedness Creates Correlated Phenotypes

Imagine two siblings.

They share approximately:

of their genome.

If a trait has a genetic component, siblings tend to have similar phenotypes.

Their observations are therefore not independent.

A standard GWAS treats them as independent.

This underestimates uncertainty and inflates significance.

32 The Solution: Mixed Models

Instead of fitting:

we fit:

where:

(X) = fixed effects
(g) = polygenic random effect
() = residual noise

The random effect captures genetic similarity among individuals.

33 What is a Genetic Relationship Matrix?

A Genetic Relationship Matrix (GRM) measures genetic similarity between all pairs of individuals.

Suppose we have:

individuals.

The GRM is an:

matrix.

Example:

Interpretation:

Individual 1 and 2 are siblings
Individual 3 is unrelated

34 Relationship Values

Typical GRM values:

Relationship	Expected GRM
Same person	1.0
Parent-child	0.5
Full siblings	0.5
Half siblings	0.25
Grandparent-grandchild	0.25
First cousins	0.125
Unrelated	~0

34.0.1 How the GRM is Computed

Suppose:

is genotype dosage for SNP (j) in individual (i).

Genotypes:

Genotype	Dosage
AA	0
Aa	1
aa	2

The GRM entry between individuals (i) and (k) is:

where:

(M) = number of SNPs
(p_j) = allele frequency

This standardizes each SNP before averaging.

35 Visualizing a GRM

Suppose:

Individuals:
A
B
C
D

A GRM heatmap might look like:

      A    B    C    D

A   1.0 0.5 0.0 0.0
B   0.5 1.0 0.0 0.0
C   0.0 0.0 1.0 0.2
D   0.0 0.0 0.2 1.0

A and B are siblings.

C and D are distant relatives.

36 Why Not Use the Full GRM Directly?

Suppose:

The GRM contains:

entries.

This becomes enormous.

Computational cost grows rapidly.

37 Sparse GRMs

fastGWA solves this problem.

Instead of storing every relationship:

0.001
0.002
0.003

it stores only meaningful relationships.

Example threshold:

Any relationship below:

is replaced by zero.

38 Full vs Sparse GRM

Full GRM:

Everyone related to everyone.

Sparse GRM:

Only close relatives retained.

Advantages:

smaller memory footprint
faster computation
scalable to biobank-sized datasets

39 Why PCs Are Still Necessary

This is one of the most important concepts in the workshop.

A sparse GRM captures:

siblings
cousins
close relatives

It does NOT capture:

ancestry
population structure

because distant relationships are set to zero.

Therefore:

Sparse GRM
≠
Population Stratification Correction

PCs are still required.

40 What Corrects What?

Component	Corrects
PCs	Population stratification
Sparse GRM	Relatedness
Mixed model	Polygenic background

All are needed.

41 Step 3.1 Build the Full GRM

Using GCTA:

gcta64 \
  --bfile study1_qc \
  --make-grm \
  --out study1_grm \
  --thread-num 4

Output:

study1_grm.grm.bin
study1_grm.grm.id
study1_grm.grm.N.bin

42 What These Files Mean

File	Purpose
.grm.bin	GRM values
.grm.id	Individual IDs
.grm.N.bin	Number of SNPs used

43 Step 3.2 Create Sparse GRM

Convert the full GRM:

gcta64 \
  --grm study1_grm \
  --make-bK-sparse 0.05 \
  --out study1_sp_grm

Threshold:

0.05

Relationships below 0.05 become zero.

44 Why 0.05?

Approximately:

Closer than third cousins

Recommended by GCTA developers.

Balances:

accuracy
speed

45 Preparing Covariates

GCTA requires:

No header
FID IID covariates

46 Covariates Without PCs

cov <- read.table(
  "study1_covariates.txt",
  header=TRUE
)

write.table(
  cov[,c(
      "FID",
      "IID",
      "sex",
      "age"
  )],
  "study1_covar_noPCs.txt",
  row.names=FALSE,
  col.names=FALSE,
  quote=FALSE,
  sep="\t"
)

47 Covariates With PCs

write.table(
  cov[,c(
      "FID",
      "IID",
      "sex",
      "age",
      paste0("PC",1:10)
  )],
  "study1_covar_withPCs.txt",
  row.names=FALSE,
  col.names=FALSE,
  quote=FALSE,
  sep="\t"
)

48 Prepare Phenotype File

pheno <- read.table(
  "study1_pheno.txt",
  header=TRUE
)

write.table(
  pheno,
  "study1_pheno_gcta.txt",
  row.names=FALSE,
  col.names=FALSE,
  quote=FALSE,
  sep="\t"
)

49 GWAS Without PC Adjustment

First run the model WITHOUT PCs.

gcta64 \
 --fastGWA-mlm \
 --bfile study1_qc \
 --grm-sparse study1_sp_grm \
 --pheno study1_pheno_gcta.txt \
 --qcovar study1_covar_noPCs.txt \
 --out study1_noPCs \
 --thread-num 4

50 Why Do This?

To see the effect of population stratification.

The sparse GRM corrects:

relatedness

but not:

ancestry

51 GWAS With PC Adjustment

Now run the proper model.

gcta64 \
 --fastGWA-mlm \
 --bfile study1_qc \
 --grm-sparse study1_sp_grm \
 --pheno study1_pheno_gcta.txt \
 --qcovar study1_covar_withPCs.txt \
 --out study1_withPCs \
 --thread-num 4

Now we correct:

relatedness
ancestry

simultaneously.

52 Understanding fastGWA Output

Inspect:

head study1_withPCs.fastGWA

Important columns:

Column	Meaning
CHR	Chromosome
SNP	SNP ID
POS	Base-pair position
A1	Effect allele
A2	Other allele
AF1	Effect allele frequency
BETA	Effect size
SE	Standard error
P	P-value

53 Interpreting Beta

Example:

BETA = 0.25

means:

Each additional copy of the effect allele increases the phenotype by 0.25 units.

54 Interpreting Standard Error

Smaller:

SE

means more precise estimates.

Larger sample sizes generally reduce SE.

55 Interpreting P-values

The null hypothesis:

Small p-values suggest association.

Typical threshold:

56 Why Genome-Wide Significance Is So Stringent

We test:

Hundreds of thousands
to
Millions
of SNPs

Multiple testing becomes severe.

Therefore:

is the accepted threshold.

57 Repeat for Study 2

Repeat:

Build full GRM
Build sparse GRM
Create covariates
Prepare phenotype file
Run fastGWA without PCs
Run fastGWA with PCs

Output:

study2_noPCs.fastGWA

study2_withPCs.fastGWA

58 Summary

In this section we learned:

Why ordinary regression fails in related samples.
What a GRM measures.
How a GRM is computed.
Why sparse GRMs are used.
Why PCs are still required.
How fastGWA works.
How to run mixed-model GWAS.
How to interpret fastGWA output.

At this point we have completed the association analysis itself.

Next we will visualize the results using:

QQ plots
Manhattan plots
Genomic inflation factor λ

to determine whether our GWAS results are trustworthy.

59 Part 4: Visualizing GWAS Results with QQ Plots and Manhattan Plots

60 Why Visualization Matters

Running a GWAS produces a table containing thousands or millions of p-values.

A table is difficult to interpret.

Visualization helps answer important questions:

Is there population stratification?
Is there inflation of test statistics?
Are there genuine associations?
How many loci reach genome-wide significance?
Are significant SNPs clustered in genomic regions?

Two plots dominate GWAS visualization:

QQ plots
Manhattan plots

These plots appear in almost every GWAS publication.

61 Loading GWAS Results

Switch to R.

library(qqman)

res_noPCs <- read.table(
  "study1_noPCs.fastGWA",
  header = TRUE
)

res_withPCs <- read.table(
  "study1_withPCs.fastGWA",
  header = TRUE
)

62 Preparing Data for qqman

The qqman package expects:

SNP
CHR
BP
P

columns.

Create helper function:

prep <- function(df){

  data.frame(
    SNP = df$SNP,
    CHR = df$CHR,
    BP = df$POS,
    P = df$P
  )

}

qq_noPCs <- prep(res_noPCs)

qq_withPCs <- prep(res_withPCs)

63 Understanding the Null Hypothesis

For every SNP we test:

Under the null hypothesis:

SNP has no effect
p-values follow a Uniform(0,1) distribution

Therefore:

Most p-values should be large.
Few p-values should be small.

If the null is true everywhere:

Observed p-values
≈
Expected p-values

64 What is a QQ Plot?

QQ stands for:

Quantile-Quantile

A QQ plot compares:

observed p-values
expected p-values under the null

Instead of plotting p-values directly, we use:

because small p-values become easier to see.

65 Expected Pattern Under the Null

If all SNPs are null:

Observed ≈ Expected

The points fall on the diagonal.

Visually:

|
|      /
|     /
|    /
|   /
|__/________

66 Creating QQ Plots

par(mfrow=c(1,2))

qq(
  qq_noPCs$P,
  main="QQ Plot: No PC Adjustment"
)

qq(
  qq_withPCs$P,
  main="QQ Plot: With PC Adjustment"
)

67 Interpreting QQ Plots

There are three common patterns.

68 Pattern 1: Perfect Null

All points on diagonal

Interpretation:

no inflation
no true signal

69 Pattern 2: Global Inflation

Points rise above diagonal everywhere

Interpretation:

Possible causes:

population stratification
batch effects
cryptic relatedness
poor QC

This is usually bad.

70 Pattern 3: Tail Deviation

Most points on diagonal
Only tail rises

Interpretation:

true genetic associations
well-controlled GWAS

This is the desired pattern.

# Illustrative simulation -- NOT real fastGWA output.
# Reproduces the three QQ plot patterns described above: a perfect null,
# global inflation, and a well-controlled GWAS with true signal in the tail.

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

rng = np.random.default_rng(42)
n_snp = 20000

def qq_coords(pvals):
    obs = -np.log10(np.sort(pvals))
    exp = -np.log10(np.linspace(1 / len(pvals), 1, len(pvals)))
    return exp, obs

p_null = rng.uniform(0, 1, n_snp)

lambda_inflate = 1.8
chisq_true = stats.chi2.rvs(df=1, size=n_snp, random_state=42)
p_inflated = 1 - stats.chi2.cdf(chisq_true * lambda_inflate, df=1)

p_signal = rng.uniform(0, 1, n_snp)
n_hits = 25
p_signal[:n_hits] = rng.uniform(1e-12, 1e-8, n_hits)

fig, axes = plt.subplots(1, 3, figsize=(13, 4.3))
titles = ["Perfect Null", "Global Inflation", "True Signal in Tail"]
datasets = [p_null, p_inflated, p_signal]
for ax, title, pvals in zip(axes, titles, datasets):
    exp, obs = qq_coords(pvals)
    lim = max(exp.max(), obs.max()) * 1.05
    ax.plot([0, lim], [0, lim], color="#ddd8cd", linewidth=1.2, linestyle="--")
    ax.scatter(exp, obs, s=4, color="#2f6f6b", alpha=0.5)
    ax.set_title(title, fontsize=12)
    ax.set_xlabel("Expected $-log_{10}(p)$")
    ax.set_xlim(0, lim)
    ax.set_ylim(0, lim)
axes[0].set_ylabel("Observed $-log_{10}(p)$")
fig.suptitle("QQ Plot Patterns (Simulated)", fontweight="bold", y=1.02)
plt.tight_layout()
plt.show()

71 Why PC Adjustment Matters

Compare:

No PCs

versus

With PCs

If PCs successfully correct stratification:

inflation decreases
QQ plot approaches diagonal

This demonstrates why PCA was necessary.

72 The Genomic Inflation Factor (λ)

A QQ plot is visual.

Lambda provides a numerical summary.

Definition:

For 1 degree of freedom:

73 Computing Lambda

lambda <- function(p){

  chisq <- qchisq(
    1-p,
    df=1
  )

  median(chisq) /
  qchisq(
    0.5,
    df=1
  )

}

cat(
  "Lambda (No PCs):",
  lambda(qq_noPCs$P),
  "\n"
)

cat(
  "Lambda (With PCs):",
  lambda(qq_withPCs$P),
  "\n"
)

74 Interpreting Lambda

λ	Interpretation
1.00	Ideal
1.02	Very good
1.05	Usually acceptable
>1.10	Investigate inflation
>1.20	Likely problematic

75 Important Caveat

Many beginners think:

λ > 1
means
bad GWAS

This is not always true.

Large studies often have:

thousands of real associations
highly polygenic traits

True signal can also increase λ.

Therefore:

QQ plots should always be interpreted together with λ.

76 Manhattan Plots

QQ plots summarize the whole GWAS.

Manhattan plots show where signals occur.

Each SNP is plotted according to:

genomic position
significance

77 Why the Name “Manhattan”?

Significant loci appear as towers.

These resemble skyscrapers in Manhattan.

# Illustrative simulation -- NOT real fastGWA output.
# Simulates genome-wide p-values across 10 chromosomes with one true locus,
# to show what a real association "tower" looks like against the null background.

import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng(7)
n_chr, snps_per_chr = 10, 1500
true_chr, true_pos_center = 4, 700

chrom, pos, pval = [], [], []
cum_offset = 0
chrom_offsets = []

for c in range(1, n_chr + 1):
    p = np.sort(rng.integers(1, 250_000_000, snps_per_chr))
    pv = rng.uniform(0, 1, snps_per_chr)
    if c == true_chr:
        window = slice(max(0, true_pos_center - 15), true_pos_center + 15)
        pv[window] = rng.uniform(1e-10, 5e-8, len(pv[window]))
    chrom.extend([c] * snps_per_chr)
    pos.extend(p + cum_offset)
    pval.extend(pv)
    chrom_offsets.append(cum_offset + p.max() / 2)
    cum_offset += p.max() + 20_000_000

chrom, pos, pval = np.array(chrom), np.array(pos), np.array(pval)
neglog_p = -np.log10(pval)

fig, ax = plt.subplots(figsize=(11, 4))
colors_cycle = ["#2f6f6b", "#4a5a68"]
for c in range(1, n_chr + 1):
    mask = chrom == c
    ax.scatter(pos[mask], neglog_p[mask], s=5, color=colors_cycle[c % 2], alpha=0.7)
ax.axhline(-np.log10(5e-8), color="#b9812c", linestyle="--", linewidth=1.3, label="Genome-wide (5e-8)")
ax.axhline(-np.log10(1e-5), color="#4a5a68", linestyle=":", linewidth=1.1, label="Suggestive (1e-5)")
ax.set_xticks(chrom_offsets)
ax.set_xticklabels(range(1, n_chr + 1))
ax.set_xlabel("Chromosome")
ax.set_ylabel("$-log_{10}(p)$")
ax.set_title(f"Manhattan Plot (Simulated Data with One True Locus on Chr {true_chr})")
ax.legend(frameon=False, fontsize=9)
plt.tight_layout()
plt.show()

78 Constructing Manhattan Plots

The x-axis:

Chromosomal position

The y-axis:

Small p-values become tall peaks.

79 Plotting Results

par(mfrow=c(2,1))

manhattan(
  qq_noPCs,
  main="No PC Adjustment",
  suggestiveline=-log10(1e-5),
  genomewideline=-log10(5e-8)
)

manhattan(
  qq_withPCs,
  main="With PC Adjustment",
  suggestiveline=-log10(1e-5),
  genomewideline=-log10(5e-8)
)

80 Understanding the Threshold Lines

The blue line:

is the suggestive threshold.

The red line:

is the genome-wide significance threshold.

81 Why 5×10⁻⁸?

Historically:

Approximately one million independent tests occur in a European GWAS.

Using Bonferroni correction:

This became the standard threshold.

82 What Does a True Signal Look Like?

A true causal locus rarely appears as a single SNP.

Instead:

Many nearby SNPs become significant

because of linkage disequilibrium.

Result:

Tower

rather than:

Single isolated point

83 Example

Good signal:

      *
     ***
    *****
   *******

Suspicious signal:

A single isolated SNP often indicates:

genotyping error
poor imputation
technical artifact

84 Comparing No-PC and PC-Adjusted Results

Ask:

Do significant loci remain?
Do some peaks disappear?
Does inflation decrease?

Possible outcome:

No PCs:
Many peaks

With PCs:
Fewer peaks

Interpretation:

Many initial hits were likely due to stratification.

85 Identifying Top Hits

Find the most significant SNPs.

top_hits <- res_withPCs[
  order(res_withPCs$P),
]

head(
  top_hits[
    ,
    c(
      "CHR",
      "SNP",
      "POS",
      "BETA",
      "SE",
      "P"
    )
  ]
)

86 Volcano Plot (Optional)

Although uncommon in GWAS, a volcano plot can visualize:

effect size
significance

plot(
  res_withPCs$BETA,
  -log10(res_withPCs$P),
  pch=20,
  col=rgb(0,0,1,0.3),
  xlab="Beta",
  ylab="-log10(P)",
  main="Volcano Plot"
)

87 Why Manhattan Plots Are More Popular

Volcano plots ignore:

Genomic position

Manhattan plots preserve:

Chromosome
Position
LD structure

making them more informative.

88 Common GWAS Visualization Mistakes

88.1 Mistake 1

Reporting Manhattan plots without QQ plots.

You cannot assess inflation.

88.2 Mistake 2

Reporting λ without QQ plots.

You lose context.

88.3 Mistake 3

Ignoring isolated significant SNPs.

True loci usually form peaks.

88.4 Mistake 4

Comparing Manhattan plots from different studies without checking sample size.

Larger studies naturally produce stronger signals.

89 Study 2 Visualization

Repeat:

res2_noPCs <- read.table(
  "study2_noPCs.fastGWA",
  header=TRUE
)

res2_withPCs <- read.table(
  "study2_withPCs.fastGWA",
  header=TRUE
)

Create:

QQ plots
λ values
Manhattan plots

Compare:

Study 1
Study 2

before meta-analysis.

90 Summary

In this section we learned:

Why GWAS visualization is essential.
How QQ plots detect inflation.
How λ is calculated.
Why λ > 1 is not always problematic.
How Manhattan plots display genome-wide associations.
Why true loci form towers.
How PC adjustment affects GWAS results.

At this point we have:

QC’d data
principal components
mixed-model GWAS results
visualized associations

Next we will combine Study 1 and Study 2 using METAL and perform GWAS meta-analysis.

90.0.1 Part 5: GWAS Meta-Analysis Using METAL

90.0.2 Why Meta-Analysis?

Suppose we run GWAS in:

Study 1 (N = 2,000)
Study 2 (N = 2,500)

Each study may lack sufficient power to detect small genetic effects.

Many complex traits are highly polygenic.

Most SNP effects are extremely small.

For example:

may require tens of thousands of samples for reliable detection.

Instead of analyzing studies separately, we can combine evidence across studies.

This process is called:

90.0.2.1 Meta-Analysis

90.0.3 Advantages of Meta-Analysis

Meta-analysis:

increases sample size
increases statistical power
improves effect-size estimation
identifies consistent signals
avoids sharing individual-level genotype data

This last point is particularly important.

Many large consortia share only:

effect sizes
standard errors
p-values

rather than raw genotype data.

90.0.3.1 Historical Perspective

Many famous GWAS discoveries were found through meta-analysis.

Examples include:

GIANT Consortium (height, BMI)
Psychiatric Genomics Consortium (PGC)
CARDIoGRAM
DIAGRAM
Alzheimer’s Disease Genetics Consortium

Modern GWAS often combine:

10
20
50
100+
cohorts

90.0.3.1.1 Fixed-Effect Meta-Analysis

The simplest model assumes:

Every study estimates the same underlying genetic effect.

Suppose:

Study 1 estimates:

with standard error:

Study 2 estimates:

with standard error:

90.0.3.2 Inverse Variance Weighting

The combined effect is:

where:

Studies with smaller standard errors receive larger weights.

90.0.3.3 Why Larger Studies Get More Weight

Large studies generally have:

smaller standard errors
more precise estimates

Therefore:

This is exactly what we want.

More reliable studies contribute more strongly.

90.0.4 What Does METAL Do?

METAL is one of the most widely used GWAS meta-analysis programs.

Input:

Summary statistics

Output:

Combined summary statistics

METAL does not require:

genotype data
phenotype data
individual-level covariates

Only GWAS summary statistics are needed.

90.0.4.1 Preparing GWAS Results

We previously generated:

study1_withPCs.fastGWA

study2_withPCs.fastGWA

These files contain:

SNP
chromosome
position
beta
standard error
p-value

90.0.4.2 Inspecting Results

head study1_withPCs.fastGWA

head study2_withPCs.fastGWA

90.0.4.3 Required Columns

METAL typically requires:

Column	Meaning
SNP	Variant identifier
A1	Effect allele
A2	Other allele
BETA	Effect size
SE	Standard error
P	P-value

90.0.4.4 Harmonization

Before meta-analysis:

effect alleles must match.

For example:

Study 1:

A = effect allele
G = reference allele

Study 2:

G = effect allele
A = reference allele

If not corrected:

effect estimates will point in opposite directions.

This can completely invalidate results.

90.0.4.5 Example

Study 1:

Study 2:

The apparent disagreement may be entirely due to allele coding.

Always harmonize alleles.

90.0.4.6 Creating a METAL Script

Create:

metal_script.txt

91 Contents

SCHEME STDERR

MARKER SNP

ALLELE A1 A2

EFFECT BETA

STDERR SE

PVAL P

PROCESS study1_withPCs.fastGWA

PROCESS study2_withPCs.fastGWA

OUTFILE meta_results .

ANALYZE

QUIT

92 Understanding the Commands

92.1 SCHEME STDERR

Use inverse-variance weighting.

Weights:

92.2 MARKER

Specifies SNP identifier column.

SNP

92.3 EFFECT

Specifies effect-size column.

BETA

92.4 STDERR

Specifies standard error column.

SE

92.5 PROCESS

Loads a study.

PROCESS study1
PROCESS study2

92.6 ANALYZE

Runs the meta-analysis.

93 Running METAL

metal metal_script.txt

Output:

meta_results1.tbl

94 Inspecting Meta-Analysis Results

head meta_results1.tbl

Common columns:

Column	Meaning
MarkerName	SNP ID
Allele1	Effect allele
Allele2	Other allele
Effect	Combined beta
StdErr	Combined SE
P-value	Meta-analysis p-value
Direction	Sign of effect in each study

95 Understanding Direction

Example:

++

Both studies:

positive effect

Example:

--

Both studies:

negative effect

Example:

+-

Studies disagree.

This may indicate:

heterogeneity
allele issues
random variation

96 Why Meta-Analysis Increases Power

Suppose:

Study 1:

Study 2:

Neither reaches:

alone.

Combined analysis may produce:

This is why meta-analysis dominates modern GWAS.

97 Comparing GWAS and Meta-Analysis

Load results in R.

study1 <- read.table(
  "study1_withPCs.fastGWA",
  header=TRUE
)

study2 <- read.table(
  "study2_withPCs.fastGWA",
  header=TRUE
)

meta <- read.table(
  "meta_results1.tbl",
  header=TRUE
)

98 Number of Significant Hits

sum(
  study1$P < 5e-8
)

sum(
  study2$P < 5e-8
)

sum(
  meta$P.value < 5e-8
)

99 Interpretation

Typically:

Meta-analysis
>
Individual studies

in terms of discoveries.

100 Manhattan Plot of Meta-Analysis

Prepare data.

library(qqman)

meta_plot <- data.frame(
  SNP = meta$MarkerName,
  CHR = meta$Chromosome,
  BP = meta$Position,
  P = meta$P.value
)

manhattan(
  meta_plot,
  main="Meta-analysis Manhattan Plot",
  suggestiveline=-log10(1e-5),
  genomewideline=-log10(5e-8)
)

101 QQ Plot of Meta-Analysis

qq(
  meta$P.value,
  main="Meta-analysis QQ Plot"
)

102 Heterogeneity

One of the most important concepts in meta-analysis is:

102.1 Heterogeneity

Suppose:

Study 1:

Study 2:

These are consistent.

Now suppose:

Study 1:

Study 2:

These are inconsistent.

This is heterogeneity.

103 Cochran’s Q Statistic

Many meta-analysis tools evaluate:

Large Q values indicate disagreement among studies.

104 I² Statistic

Another common metric:

Interpretation:

I²	Meaning
0%	No heterogeneity
25%	Low
50%	Moderate
75%	High

105 Why Heterogeneity Matters

Heterogeneity may arise from:

ancestry differences
environmental differences
phenotype definitions
technical differences
gene-environment interaction

106 Mystery Phenotype Investigation

The workshop asks:

Can we identify the phenotype from the strongest GWAS hit?

Procedure:

Find the top SNP.

top_snp <- meta[
  which.min(meta$P.value),
]

top_snp

Record:

rsID

Search:

GWAS Catalog
Ensembl
dbSNP
Open Targets Genetics

Compare previously reported traits.

This often provides clues regarding phenotype identity.

107 Example Resources

Useful databases:

108 Summary

In this section we learned:

Why GWAS meta-analysis is necessary.
How inverse-variance weighting works.
How METAL combines studies.
Why allele harmonization matters.
How to interpret meta-analysis results.
How heterogeneity is assessed.
Why meta-analysis increases power.
How to identify candidate phenotypes using top SNPs.

At this point we have completed:

QC
PCA
GRM construction
fastGWA analysis
QQ plots
Manhattan plots
Meta-analysis

The final section will focus on:

inspecting relatedness directly from the GRM
identifying relatives
interpreting GRM values
understanding cryptic relatedness
best practices for large-scale GWAS.

108.0.1 Part 6: Understanding Relatedness, GRMs, and Best Practices for GWAS

108.0.1.1 Why Relatedness Matters

One of the fundamental assumptions of classical statistical tests is:

Observations are independent.

In genetic studies this assumption is often violated.

Individuals may be:

siblings
parent-child pairs
cousins
twins
members of the same pedigree

These relationships introduce correlation.

If ignored, GWAS statistics become inflated.

108.0.1.2 What is Cryptic Relatedness?

Cryptic relatedness means:

Individuals are genetically related, but the relationship is unknown or unrecorded.

Examples:

unknown cousins
undocumented family relationships
pedigree errors
duplicate samples

Large biobanks often contain thousands of related individuals.

108.0.1.3 Historical Perspective

Early GWAS often removed relatives.

Modern GWAS typically retain relatives and use mixed models.

Why?

Because removing relatives wastes valuable samples.

Mixed models allow us to:

retain individuals
model relatedness directly
increase statistical power

108.0.1.4 Revisiting the Genetic Relationship Matrix

The GRM contains pairwise genetic similarity.

Suppose we have:

individuals.

The GRM contains:

entries.

Each cell represents:

How genetically similar are two individuals?

108.0.1.5 GRM Interpretation

Suppose:

ID1 ID2 Relationship

produces:

0.50

Interpretation:

Likely:

parent-child
full siblings

Suppose:

0.25

Interpretation:

Likely:

half siblings
grandparent-grandchild
avuncular relationships

Suppose:

0.125

Interpretation:

Likely:

first cousins

Suppose:

0.00

Interpretation:

Essentially unrelated.

108.0.1.6 Typical Relatedness Thresholds

GRM Value	Interpretation
>0.95	Duplicate sample / identical twin
~0.50	Parent-child or sibling
~0.25	Second-degree relative
~0.125	First cousin
~0	Unrelated

These values are approximate.

Real data fluctuate around expectations.

108.0.1.7 Reading the GRM

We previously generated:

study1_grm

using:

gcta64 \
  --bfile study1_qc \
  --make-grm \
  --out study1_grm

Convert GRM to text:

gcta64 \
  --grm study1_grm \
  --grm-cutoff 0 \
  --make-grm-gz \
  --out study1_grm_text

This produces:

study1_grm_text.grm.gz

Inspect:

zcat study1_grm_text.grm.gz | head

108.0.1.8 Understanding GRM Columns

Typical output:

ID1
ID2
N_SNP
GRM

Meaning:

Column	Description
ID1	Individual 1
ID2	Individual 2
N_SNP	Number of SNPs used
GRM	Relatedness estimate

108.0.1.9 Finding Close Relatives

Extract individuals with:

zcat study1_grm_text.grm.gz \
| awk '$4 > 0.05'

These are genetically related pairs.

108.0.1.10 Counting Related Pairs

zcat study1_grm_text.grm.gz \
| awk '$4 > 0.05' \
| wc -l

This gives:

Number of related pairs

108.0.1.11 Identifying Duplicates

Potential duplicates:

zcat study1_grm_text.grm.gz \
| awk '$4 > 0.95'

Interpretation:

Possible:

duplicated sample
monozygotic twins
sample labeling error

These should be investigated.

108.0.1.12 Visualizing the GRM

Switch to R.

library(data.table)

grm <- fread(
  "study1_grm_text.grm.gz"
)

108.0.1.13 Heatmap of Relatedness

library(ggplot2)

ggplot(
  grm,
  aes(
    x=V1,
    y=V2,
    fill=V4
  )
)+
geom_tile()+
scale_fill_gradient(
  low="white",
  high="red"
)+
theme_minimal()+
labs(
  title="Genetic Relationship Matrix"
)

108.0.1.14 Interpretation

Bright red regions indicate:

families
clusters of relatives

White regions indicate:

unrelated individuals

# Illustrative simulation -- NOT a real GRM from GCTA.
# Simulates a small GRM for 20 individuals including two sibling pairs,
# one parent-child pair, and one first-cousin pair, to make the heatmap
# and "finding close relatives" discussion concrete.

import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng(42)
n_ind = 20
grm = np.zeros((n_ind, n_ind))
np.fill_diagonal(grm, 1.0)
base_noise = rng.normal(0, 0.015, (n_ind, n_ind))
grm += (base_noise + base_noise.T) / 2
np.fill_diagonal(grm, 1.0)

sib_pairs = [(0, 1), (4, 5)]
for i, j in sib_pairs:
    grm[i, j] = grm[j, i] = 0.5 + rng.normal(0, 0.02)
grm[8, 9] = grm[9, 8] = 0.5 + rng.normal(0, 0.02)          # parent-child
grm[12, 13] = grm[13, 12] = 0.125 + rng.normal(0, 0.02)    # first cousins

fig, ax = plt.subplots(figsize=(6, 5.3))
im = ax.imshow(grm, cmap="YlGnBu", vmin=0, vmax=1)
ax.set_title("Simulated Genetic Relationship Matrix (GRM)")
ax.set_xlabel("Individual")
ax.set_ylabel("Individual")
cbar = fig.colorbar(im, ax=ax, fraction=0.046, pad=0.04)
cbar.set_label("Relatedness")
plt.tight_layout()
plt.show()

close_pairs = [(i, j, round(grm[i, j], 3))
               for i in range(n_ind) for j in range(i + 1, n_ind)
               if grm[i, j] > 0.05]
print("Detected close-relative pairs (GRM > 0.05):", close_pairs)

Detected close-relative pairs (GRM > 0.05): [(0, 1, np.float64(0.496)), (4, 5, np.float64(0.504)), (8, 9, np.float64(0.516)), (12, 13, np.float64(0.117))]

108.0.1.15 Relatedness Distribution

hist(
  grm$V4,
  breaks=100,
  main="GRM Distribution",
  xlab="Relatedness"
)

108.0.1.16 Expected Pattern

Most individuals should be:

Near zero

A small number should show:

0.125
0.25
0.5

indicating relatives.

108.0.2 Relationship Categories

Create categories.

grm$relationship <- cut(
  grm$V4,
  breaks=c(
    -Inf,
    0.05,
    0.125,
    0.25,
    0.5,
    Inf
  ),
  labels=c(
    "Unrelated",
    "Distant",
    "First Cousin",
    "Second Degree",
    "First Degree"
  )
)

table(
  grm$relationship
)

108.0.2.1 Why Relatedness Creates False Positives

Suppose:

siblings share 50% of genome
siblings share environmental exposures

Phenotypes become correlated.

A SNP inherited within families may appear associated with the trait simply because family members resemble each other.

This creates inflation.

108.0.3 Classical Solution

Older GWAS often removed relatives.

Example:

plink \
  --bfile study1_qc \
  --rel-cutoff 0.125 \
  --make-bed \
  --out study1_unrelated

This removes one individual from each related pair.

109 Disadvantages

You lose data.

Example:

20,000 samples

may become:

14,000 samples

after removing relatives.

Power decreases.

110 Modern Solution

Use:

Linear Mixed Models

Examples:

fastGWA
BOLT-LMM
SAIGE
REGENIE

These methods:

keep relatives
model relatedness directly

This is why modern biobanks rarely remove all relatives.

111 Comparing Approaches

Approach	Relatives Removed?
PLINK Linear Regression	Usually yes
Mixed Models	No
fastGWA	No
BOLT-LMM	No
REGENIE	No

112 Why Biobanks Need Mixed Models

Consider:

112.0.1 UK Biobank

Approximately:

500,000 individuals

Contains:

Tens of thousands
of related individuals

Removing all relatives would waste enormous amounts of data.

Mixed models solve this problem.

113 GWAS Best Practices Checklist

Before running GWAS:

113.1 Sample QC

✔ Missingness

✔ Sex checks

✔ Heterozygosity

✔ Relatedness

113.2 SNP QC

✔ Missingness

✔ MAF filtering

✔ HWE filtering

113.3 Population Structure

✔ PCA

✔ Ancestry inspection

113.4 Association Analysis

✔ Mixed model

✔ GRM

✔ PCs as covariates

113.5 Visualization

✔ QQ plot

✔ Lambda

✔ Manhattan plot

113.6 Replication

✔ Independent cohort

✔ Meta-analysis

113.6.1 Common Beginner Mistakes

113.6.2 Mistake 1

Running GWAS without QC.

113.6.3 Mistake 2

Ignoring population structure.

113.6.4 Mistake 3

Ignoring relatedness.

113.6.5 Mistake 4

Using only p-values.

Effect sizes matter.

113.6.6 Mistake 5

Reporting isolated SNPs without checking LD.

113.6.7 Mistake 6

Assuming genome-wide significance proves causality.

GWAS identifies association, not causation.

113.6.7.1 What Happens After GWAS?

A significant GWAS hit is only the beginning.

Typical follow-up analyses include:

Fine mapping
Colocalization
eQTL analysis
TWAS
Polygenic Risk Scores
Mendelian Randomization
Functional annotation

113.6.8 Complete GWAS Workflow

You have now completed the entire GWAS pipeline:

Raw Genotypes
      ↓
Quality Control
      ↓
LD Pruning
      ↓
PCA
      ↓
GRM Construction
      ↓
Mixed Model GWAS
      ↓
QQ Plot
      ↓
Manhattan Plot
      ↓
Meta-analysis
      ↓
Relatedness Inspection
      ↓
Biological Interpretation

A GWAS is much more than running a regression for millions of SNPs.

Every stage exists for a reason:

QC prevents technical artifacts.
PCA controls ancestry differences.
GRMs model genetic similarity.
Mixed models account for relatedness.
QQ plots diagnose inflation.
Manhattan plots reveal genomic loci.
Meta-analysis increases power.

When all of these pieces work together, GWAS becomes one of the most powerful tools in modern human genetics, enabling the discovery of thousands of genetic variants associated with disease, behavior, physiology, and molecular traits.

Genetic Drift and Fixation: Understanding Evolution Through Randomness

Nivedita Bhadra — Mon, 01 Jun 2026 21:00:00 GMT

When people think about evolution, they often think about natural selection.

Natural selection explains how advantageous genetic variants become more common because they improve survival or reproduction.

However, natural selection is not the only force driving evolution.

Even in the complete absence of selection, allele frequencies can change over time simply because populations are finite.

This phenomenon is called genetic drift.

Genetic drift is one of the most fundamental concepts in population genetics because it explains:

Random changes in allele frequencies
Loss of genetic diversity
Founder effects
Population bottlenecks
Neutral evolution
Fixation and extinction of alleles

Understanding drift is essential for modern genomics, GWAS, evolutionary biology, conservation genetics, and statistical genetics.

In this tutorial we will study:

What genetic drift is
Why fixation occurs
The Wright-Fisher model
Important theoretical results
Simulation of allele-frequency trajectories
Verification of classical population genetics theorems

0.1 Learning Objectives

By the end of this tutorial you should be able to:

Explain genetic drift intuitively
Derive the Wright-Fisher model
Understand fixation and extinction
Simulate neutral evolution
Interpret allele frequency trajectories
Understand why small populations drift faster than large populations
Connect drift to modern genomic studies

0.2 Prerequisites

This tutorial assumes familiarity with:

Basic probability
Binomial distributions
Python programming
Elementary genetics

No prior knowledge of population genetics is required.

0.3 What is an Allele?

A gene can exist in different versions called alleles.

For example:

Gene	Alleles
Eye color gene	Blue, Brown
SNP rs12345	A, G

Suppose a locus contains two alleles:

and

Let denote the frequency of allele A.

Then

is the frequency of allele a.

For example:

Allele	Frequency
A	0.7
a	0.3

The frequencies always sum to one.

0.4 Infinite vs Finite Populations

Imagine an infinitely large population.

If:

No mutation
No migration
No selection
Random mating

then allele frequencies remain constant forever.

This idealized situation is the basis of the Hardy-Weinberg equilibrium.

Real populations are never infinite.

Because populations are finite, each generation represents a random sample from the previous generation.

This sampling introduces randomness.

That randomness is genetic drift.

0.5 An Intuitive Example

Imagine a population containing:

50 copies of allele A
50 copies of allele a

The allele frequency is:

Now imagine forming the next generation by randomly selecting individuals.

By chance you may obtain:

55 copies of A
45 copies of a

Now:

Nothing biological happened.

No selection occurred.

The change is purely due to chance.

This is genetic drift.

0.5.1 Formal Definition

Genetic drift is the random fluctuation of allele frequencies due to finite population sampling.

Unlike natural selection:

Genetic Drift	Natural Selection
Random	Non-random
Stronger in small populations	Can operate in any population
Does not require fitness differences	Requires fitness differences
Can fix harmful alleles	Favors beneficial alleles

0.6 What is Fixation?

An allele is fixed when its frequency becomes:

This means every chromosome in the population carries that allele.

Example:

Allele	Frequency
A	1.0
a	0.0

At this point:

Genetic variation disappears
No alternative allele remains

0.7 What is Allele Loss?

The opposite situation occurs when:

The allele disappears entirely from the population.

Example:

Allele	Frequency
A	0
a	1

In a drift-only model:

fixation is absorbing
extinction is absorbing

Once reached, these states remain forever.

0.8 The Wright-Fisher Model

The Wright-Fisher model is the classical mathematical model of genetic drift.

Assumptions:

Constant population size
Diploid individuals
Random mating
No mutation
No migration
No selection
Non-overlapping generations

Suppose:

is the number of diploid individuals.

There are:

gene copies.

0.9 Sampling the Next Generation

Suppose allele A has frequency:

in generation:

The number of copies of allele A in the next generation follows:

The allele frequency becomes:

This simple equation forms the foundation of modern population genetics.

0.10 Important Theorem 1: Drift Has No Direction

For a neutral allele:

Interpretation:

On average, allele frequencies do not systematically increase or decrease.

Drift is unbiased.

However, individual populations may behave very differently.

0.11 Important Theorem 2: Variance of Drift

The variance is:

This equation reveals an important biological fact:

Drift becomes stronger as population size decreases.

0.12 Why Small Populations Drift Faster

Suppose:

For

the variance becomes:

For

the variance becomes:

The smaller population experiences much larger fluctuations.

0.13 Important Theorem 3: Fixation Probability

One of the most beautiful results in population genetics states:

For a neutral allele:

where

is the starting frequency.

Examples:

Initial Frequency	Fixation Probability
0.1	10%
0.2	20%
0.5	50%
0.8	80%

0.14 New Mutations

A new mutation begins as a single copy.

In a diploid population:

Therefore:

Example:

For

Only 0.05%.

Most neutral mutations disappear.

0.15 Important Theorem 4: Heterozygosity Decay

Drift reduces genetic diversity.

Expected heterozygosity after t generations:

This explains why isolated populations lose diversity over time.

# Setting Up Python


import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

np.random.seed(42)

0.16 Simulating the Wright-Fisher Model

We now implement the model directly.

def simulate_drift(
    N=100,
    p0=0.5,
    generations=100
):

    p = p0

    trajectory = [p]

    for _ in range(generations):

        count_A = np.random.binomial(
            2*N,
            p
        )

        p = count_A/(2*N)

        trajectory.append(p)

        if p == 0 or p == 1:

            trajectory.extend(
                [p]*(generations-len(trajectory)+1)
            )

            break

    return np.array(trajectory)


# Simulating One Population


trajectory = simulate_drift(
    N=100,
    p0=0.5,
    generations=200
)


# Visualizing Drift


plt.figure(figsize=(12,6))

plt.plot(
    trajectory,
    linewidth=2
)

plt.axhline(
    0,
    linestyle="--"
)

plt.axhline(
    1,
    linestyle="--"
)

plt.xlabel("Generation")
plt.ylabel("Allele Frequency")

plt.title(
    "Single Wright-Fisher Trajectory"
)

plt.ylim(
    -0.05,
    1.05
)

plt.show()

0.17 Interpreting the Plot

Notice:

Frequency fluctuates randomly
No directional trend exists
Changes occur despite no selection
Eventually fixation or extinction may occur

This randomness is the hallmark of genetic drift.

0.18 Simulating Multiple Populations

The true power of drift appears when we simulate many populations with identical starting conditions.


def simulate_many_populations(
    N=100,
    p0=0.5,
    generations=200,
    n_replicates=50
):

    trajectories=[]

    for _ in range(n_replicates):

        traj = simulate_drift(
            N,
            p0,
            generations
        )

        trajectories.append(traj)

    return np.array(trajectories)

trajectories = simulate_many_populations(
    N=100,
    p0=0.5,
    generations=200,
    n_replicates=50
)

plt.figure(figsize=(12,7))

for traj in trajectories:

    plt.plot(
        traj,
        alpha=0.3
    )

plt.xlabel("Generation")
plt.ylabel("Allele Frequency")

plt.title(
    "50 Independent Populations Starting at p=0.5"
)

plt.ylim(
    -0.05,
    1.05
)

plt.show()

0.19 Interpretation

All populations began with: .

Yet they diverged dramatically.

Some populations moved upward.

Some moved downward.

Some approached fixation.

Some approached extinction.

This illustrates a central lesson:

Evolutionary outcomes can differ substantially even when populations start under identical conditions.

0.20 Summary

In this first part we learned:

What genetic drift is
Why finite populations matter
What fixation means
The Wright-Fisher model
Classical theorems of drift
Why small populations drift faster
How to simulate allele-frequency trajectories

In Part 2 we will move beyond a single SNP and systematically investigate:

Population size effects
Initial allele frequency effects
Fixation probability verification
Multiple SNP simulations
Sample size effects
Allele frequency spectra

0.21 Experimental Exploration of Genetic Drift

In Part 1, we simulated a single SNP evolving under neutral genetic drift.

Now we move toward a more realistic population genetics framework by asking:

What happens when population size changes?
What happens when initial allele frequency changes?
Can we experimentally verify the fixation theorem?
What happens when thousands of SNPs evolve simultaneously?
How does sample size affect allele frequency estimates?
How does drift reshape the allele frequency spectrum?

These experiments mirror questions encountered in modern statistical genetics and genomics.

0.22 Experiment 1: Effect of Population Size

Recall the variance theorem:

The denominator contains population size:

Therefore:

Small populations should drift rapidly.
Large populations should drift slowly.

0.23 Simulating Different Population Sizes


population_sizes = [
    20,
    50,
    100,
    500,
    1000
]

plt.figure(figsize=(12,7))

for N in population_sizes:

    traj = simulate_drift(
        N=N,
        p0=0.5,
        generations=200
    )

    plt.plot(
        traj,
        linewidth=2,
        label=f"N={N}"
    )

plt.xlabel("Generation")
plt.ylabel("Allele Frequency")

plt.title(
    "Effect of Population Size on Genetic Drift"
)

plt.legend()

plt.ylim(
    -0.05,
    1.05
)

plt.show()

0.24 Interpretation

Notice the dramatic difference.

For:

the allele frequency may rapidly reach fixation or extinction.

For:

the frequency remains close to its initial value.

This directly verifies the variance theorem.

0.25 Quantifying Drift Variability

Instead of looking at one trajectory, we can measure variance across many replicate populations.

population_sizes = [
    20,
    50,
    100,
    500,
    1000
]

variances=[]

for N in population_sizes:

    final_freqs=[]

    for _ in range(1000):

        traj = simulate_drift(
            N=N,
            p0=0.5,
            generations=100
        )

        final_freqs.append(
            traj[-1]
        )

    variances.append(
        np.var(final_freqs)
    )

plt.figure(figsize=(10,6))

plt.plot(
    population_sizes,
    variances,
    marker="o",
    linewidth=3
)

plt.xlabel("Population Size")
plt.ylabel("Variance of Final Allele Frequency")

plt.title(
    "Drift Variance Decreases With Population Size"
)

plt.show()

0.26 Experiment 2: Initial Allele Frequency

The fixation theorem states:

where, is the initial allele frequency.

We first visualize trajectories starting from different frequencies.

starting_frequencies = [
    0.1,
    0.3,
    0.5,
    0.7,
    0.9
]

plt.figure(figsize=(12,7))

for p0 in starting_frequencies:

    traj = simulate_drift(
        N=100,
        p0=p0,
        generations=200
    )

    plt.plot(
        traj,
        label=f"p0={p0}"
    )

plt.xlabel("Generation")
plt.ylabel("Allele Frequency")

plt.title(
    "Effect of Initial Allele Frequency"
)

plt.legend()

plt.show()

0.27 Interpretation

Alleles beginning near have a much greater chance of fixation.

Alleles beginning near

have a much greater chance of loss.

0.28 Experiment 3: Verifying the Fixation Theorem

One of the most elegant results in population genetics states .

We now verify this experimentally.

def estimate_fixation_probability(
    p0,
    N=100,
    generations=1000,
    n_replicates=1000
):

    fixation_count=0

    for _ in range(n_replicates):

        traj = simulate_drift(
            N=N,
            p0=p0,
            generations=generations
        )

        if traj[-1] == 1:
            fixation_count += 1

    return fixation_count/n_replicates

p0_values = np.arange(
    0.05,
    1.0,
    0.05
)

estimated_probs=[]

for p0 in p0_values:

    estimated_probs.append(
        estimate_fixation_probability(
            p0=p0,
            N=100,
            generations=1000,
            n_replicates=500
        )
    )

plt.figure(figsize=(8,8))

plt.plot(
    p0_values,
    p0_values,
    linewidth=3,
    label="Theory"
)

plt.scatter(
    p0_values,
    estimated_probs,
    s=70,
    label="Simulation"
)

plt.xlabel("Initial Frequency")
plt.ylabel("Fixation Probability")

plt.title(
    "Experimental Verification of the Fixation Theorem"
)

plt.legend()

plt.show()

0.29 Interpretation

The simulation closely follows the theoretical prediction:

This is one of the most important theorems in neutral evolution.

0.30 Experiment 4: Multiple SNP Simulation

Real genomes contain millions of SNPs.

Instead of tracking a single locus, we simulate many SNPs simultaneously.

# Creating 1,000 Independent SNPs


n_snps = 1000

initial_freqs = np.random.uniform(
    0.05,
    0.95,
    n_snps
)

final_freqs=[]

for p0 in initial_freqs:

    traj = simulate_drift(
        N=100,
        p0=p0,
        generations=200
    )

    final_freqs.append(
        traj[-1]
    )


# Visualizing Frequency Changes

plt.figure(figsize=(12,6))

plt.hist(
    initial_freqs,
    bins=30,
    alpha=0.6,
    label="Initial Frequencies"
)

plt.hist(
    final_freqs,
    bins=30,
    alpha=0.6,
    label="Final Frequencies"
)

plt.xlabel("Allele Frequency")
plt.ylabel("Number of SNPs")

plt.title(
    "Genetic Drift Reshapes the Frequency Spectrum"
)

plt.legend()

plt.show()

0.31 Interpretation

Initially, frequencies were nearly uniform.

After drift:

More SNPs accumulate near 0
More SNPs accumulate near 1
Intermediate frequencies become less common

This is a hallmark of neutral evolution.

0.32 Experiment 5: SNP Loss and Fixation

How many SNPs become fixed or lost?



lost=0
fixed=0

for p in final_freqs:

    if p == 0:
        lost += 1

    elif p == 1:
        fixed += 1

print("Lost SNPs:", lost)
print("Fixed SNPs:", fixed)


# Visualization


categories = [
    "Lost",
    "Fixed",
    "Segregating"
]

counts = [
    lost,
    fixed,
    len(final_freqs)-lost-fixed
]

plt.figure(figsize=(8,6))

plt.bar(
    categories,
    counts
)

plt.ylabel("Number of SNPs")

plt.title(
    "Fate of SNPs After Drift"
)

plt.show()

Lost SNPs: 313
Fixed SNPs: 285

0.33 Experiment 6: Number of SNPs Matters

Now repeat the experiment using increasingly larger numbers of SNPs.


snp_counts = [
    100,
    500,
    1000,
    5000
]

fixed_counts=[]
lost_counts=[]

for n_snps in snp_counts:

    fixed=0
    lost=0

    freqs=np.random.uniform(
        0.05,
        0.95,
        n_snps
    )

    for p0 in freqs:

        traj=simulate_drift(
            N=50,
            p0=p0,
            generations=300
        )

        if traj[-1]==1:
            fixed += 1

        elif traj[-1]==0:
            lost += 1

    fixed_counts.append(fixed)
    lost_counts.append(lost)

plt.figure(figsize=(10,6))

plt.plot(
    snp_counts,
    fixed_counts,
    marker="o",
    linewidth=3,
    label="Fixed"
)

plt.plot(
    snp_counts,
    lost_counts,
    marker="o",
    linewidth=3,
    label="Lost"
)

plt.xlabel("Number of SNPs")
plt.ylabel("Count")

plt.title(
    "Genome-Wide Consequences of Drift"
)

plt.legend()

plt.show()

0.34 Interpretation

As the number of SNPs increases:

More variants become fixed
More variants become lost
Overall diversity declines

0.35 Experiment 7: Sampling Error Versus Genetic Drift

In GWAS we rarely observe the entire population.

Instead we estimate allele frequencies from samples.

Suppose the true frequency is .

0.36 Simulating Sampling Error


true_frequency = 0.30

sample_sizes = [
    20,
    50,
    100,
    500,
    1000,
    5000
]

sampling_std=[]

for n in sample_sizes:

    estimates=[]

    for _ in range(5000):

        count = np.random.binomial(
            n,
            true_frequency
        )

        estimates.append(
            count/n
        )

    sampling_std.append(
        np.std(estimates)
    )

plt.figure(figsize=(10,6))

plt.plot(
    sample_sizes,
    sampling_std,
    marker="o",
    linewidth=3
)

plt.xlabel("Sample Size")
plt.ylabel("Standard Deviation")

plt.title(
    "Sampling Error Decreases With Sample Size"
)

plt.show()

0.37 Interpretation

Larger studies produce more precise allele frequency estimates.

This randomness is not genetic drift.

Instead it is statistical sampling error.

0.38 Genetic Drift vs Sampling Error

Genetic Drift	Sampling Error
Biological process	Measurement process
Occurs across generations	Occurs during data collection
Changes true frequency	Changes estimated frequency
Evolutionary phenomenon	Statistical phenomenon

Understanding this distinction is essential in GWAS and population genetics.

0.39 Experiment 8: Distribution of Frequency Estimates


sample_size = 50

estimates=[]

for _ in range(10000):

    count=np.random.binomial(
        sample_size,
        true_frequency
    )

    estimates.append(
        count/sample_size
    )

plt.figure(figsize=(10,6))

plt.hist(
    estimates,
    bins=40
)

plt.axvline(
    true_frequency,
    linestyle="--"
)

plt.xlabel("Estimated Frequency")
plt.ylabel("Count")

plt.title(
    "Sampling Distribution of Allele Frequency Estimates"
)

plt.show()

0.40 Interpretation

Even though the true frequency remains fixed at:

individual samples produce different estimates.

This is exactly why larger cohorts improve statistical power.

0.41 Summary

In this section we experimentally demonstrated:

Small populations drift faster.
Large populations are more stable.
Fixation probability equals starting frequency.
Drift pushes SNPs toward fixation and extinction.
Diversity decreases across the genome.
Sample size affects precision of allele-frequency estimation.
Sampling error and genetic drift are fundamentally different processes.

In Part 3 we will move to:

Genome-wide simulations
Heterozygosity decay
Effective population size
Bottlenecks
Founder effects
Population structure
GWAS implications
Modern statistical genetics applications

0.42 Genome-Wide Consequences of Genetic Drift

So far we have studied drift at the level of individual SNPs.

Real genomes contain millions of variants.

Over evolutionary time, drift acts simultaneously on all of them.

The cumulative effect produces:

Loss of genetic diversity
Population differentiation
Founder effects
Population bottlenecks
Changes in allele frequency spectra

Many patterns observed in modern genomics can be understood through drift alone.

0.43 Heterozygosity

One of the most important measures of genetic diversity is heterozygosity.

For a biallelic SNP:

where:

(p) = frequency of allele A
(1-p) = frequency of allele a

0.44 Maximum Heterozygosity

The maximum occurs when:

because:

The minimum occurs when:

because:

At fixation, all diversity disappears.

0.45 Theoretical Decay of Heterozygosity

One of the classical results of population genetics is:

where:

(H_0) = initial heterozygosity
(N) = population size
(t) = generations

0.46 Simulating Heterozygosity Decay


def heterozygosity_decay(
    H0,
    N,
    generations
):

    t = np.arange(
        generations+1
    )

    Ht = H0 * (
        1 - 1/(2*N)
    )**t

    return t, Ht

population_sizes = [
    20,
    100,
    500,
    1000
]

plt.figure(figsize=(12,6))

for N in population_sizes:

    t,Ht = heterozygosity_decay(
        H0=0.5,
        N=N,
        generations=500
    )

    plt.plot(
        t,
        Ht,
        linewidth=3,
        label=f"N={N}"
    )

plt.xlabel("Generation")
plt.ylabel("Expected Heterozygosity")

plt.title(
    "Loss of Genetic Diversity Through Drift"
)

plt.legend()

plt.show()

0.47 Interpretation

Small populations lose diversity rapidly.

Large populations retain diversity for much longer periods.

This is one reason conservation biologists worry about endangered populations.

0.48 Effective Population Size

A critical concept in population genetics is:

the effective population size.

0.48.1 Census Size vs Effective Size

Census size:

represents the number of individuals.

Effective population size:

represents the size of an ideal population that experiences the same amount of drift.

Usually:

0.49 Why is Effective Population Size Smaller?

Real populations violate Wright-Fisher assumptions:

Unequal family sizes
Population structure
Sex imbalance
Bottlenecks
Selection

All of these increase drift.

0.50 Why Statistical Geneticists Care About Ne

Many quantities depend on:

including:

LD decay
Drift strength
Coalescent times
Polygenic adaptation
Demographic inference

0.51 Population Bottlenecks

A bottleneck occurs when population size suddenly decreases.

Examples include:

Natural disasters
Famine
Disease outbreaks
Founder migrations

0.52 Bottleneck Simulation


def simulate_bottleneck(
    p0=0.5,
    generations_before=50,
    generations_bottleneck=20,
    generations_after=100,
    N_before=1000,
    N_bottleneck=20,
    N_after=1000
):

    p = p0

    trajectory=[p]

    schedule = (
        [N_before]*generations_before +
        [N_bottleneck]*generations_bottleneck +
        [N_after]*generations_after
    )

    for N in schedule:

        count=np.random.binomial(
            2*N,
            p
        )

        p=count/(2*N)

        trajectory.append(p)

    return np.array(trajectory)

traj = simulate_bottleneck()

plt.figure(figsize=(12,6))

plt.plot(
    traj,
    linewidth=3
)

plt.axvspan(
    50,
    70,
    alpha=0.3,
    label="Bottleneck"
)

plt.xlabel("Generation")
plt.ylabel("Allele Frequency")

plt.title(
    "Population Bottleneck"
)

plt.legend()

plt.show()

0.53 Interpretation

Most frequency changes occur during the bottleneck period.

This happens because:

temporarily becomes very small.

0.53.1 Founder Effects

A founder effect is a special case of drift.

A small group leaves a large population and establishes a new population.

The new population contains only a subset of the original genetic variation.

0.53.2 Founder Effect Simulation


source_population = np.random.uniform(
    0.05,
    0.95,
    5000
)

founder_population = np.random.choice(
    source_population,
    size=200,
    replace=False
)

plt.figure(figsize=(12,6))

plt.hist(
    source_population,
    bins=30,
    alpha=0.5,
    label="Source Population"
)

plt.hist(
    founder_population,
    bins=30,
    alpha=0.5,
    label="Founder Population"
)

plt.xlabel("Allele Frequency")
plt.ylabel("Count")

plt.title(
    "Founder Effect"
)

plt.legend()

plt.show()

0.54 Interpretation

Even before evolution occurs, the founder population already differs genetically.

This is purely due to sampling.

0.55 Drift Across an Entire Genome

Now simulate:

10,000 SNPs
500 generations


n_snps = 10000

initial_freqs = np.random.uniform(
    0.01,
    0.99,
    n_snps
)

final_freqs=[]

for p0 in initial_freqs:

    traj = simulate_drift(
        N=100,
        p0=p0,
        generations=500
    )

    final_freqs.append(
        traj[-1]
    )


# Fate of Variants


fixed = np.sum(
    np.array(final_freqs)==1
)

lost = np.sum(
    np.array(final_freqs)==0
)

segregating = (
    n_snps-fixed-lost
)

print("Fixed:", fixed)
print("Lost:", lost)
print("Segregating:", segregating)

plt.figure(figsize=(8,6))

plt.bar(
    ["Lost","Fixed","Segregating"],
    [lost,fixed,segregating]
)

plt.ylabel("Number of SNPs")

plt.title(
    "Genome-Wide Fate of Variants"
)

plt.show()

Fixed: 4523
Lost: 4665
Segregating: 812

0.56 Interpretation

After sufficient evolutionary time:

Many variants disappear
Many become fixed
Only a subset remain polymorphic

0.57 Genetic Drift and Population Structure

Different populations experience different drift histories.

Suppose two populations split.

Each evolves independently.

Their allele frequencies diverge over time.

0.58 Simulating Divergence


shared_freqs = np.random.uniform(
    0.05,
    0.95,
    1000
)

population_A=[]
population_B=[]

for p0 in shared_freqs:

    trajA = simulate_drift(
        N=100,
        p0=p0,
        generations=100
    )

    trajB = simulate_drift(
        N=100,
        p0=p0,
        generations=100
    )

    population_A.append(
        trajA[-1]
    )

    population_B.append(
        trajB[-1]
    )

plt.figure(figsize=(8,8))

plt.scatter(
    population_A,
    population_B,
    alpha=0.5
)

plt.xlabel("Population A")
plt.ylabel("Population B")

plt.title(
    "Population Divergence Through Drift"
)

plt.show()

0.59 Interpretation

The populations began identically.

Drift alone created differences.

This is the foundation of population structure.

0.60 Connection to FST

Population differentiation is often measured using:

which quantifies how much allele frequencies differ between populations.

Drift increases:

over time.

0.61 Why GWAS Researchers Care About Drift

Modern GWAS datasets contain individuals from populations with different demographic histories.

Drift can create allele-frequency differences unrelated to disease.

0.61.1 Example

Suppose:

Population A:

Population B:

for a SNP.

If disease prevalence also differs between populations, a naive GWAS may falsely associate the SNP with disease.

0.61.2 Confounding

This phenomenon is called:

Population stratification.

Drift is one of its major causes.

0.62 How GWAS Controls Drift Effects

Modern studies use:

Principal Components Analysis (PCA)
Linear Mixed Models
Genetic Relationship Matrices
Ancestry Adjustment

to account for population structure.

0.63 Connection to FinnGen

Finnish populations experienced:

Founder events
Bottlenecks
Relative isolation

Consequently:

Some rare variants became common
Certain disease alleles increased in frequency

This is known as the Finnish Disease Heritage.

0.64 Connection to UK Biobank

UK Biobank contains:

Multiple ancestry groups
Geographic structure
Historical demographic differences

These produce subtle allele-frequency differences caused partly by drift.

0.65 Connection to Polygenic Scores

Polygenic scores depend on allele frequencies.

Since drift changes frequencies:

Prediction accuracy changes across populations
Transferability decreases
Ancestry matching becomes important

Genetic drift is random evolution caused by finite populations.
Drift eventually causes fixation or extinction.
Small populations drift faster than large populations.
Heterozygosity decreases through time.
Effective population size determines drift strength.
Bottlenecks accelerate drift.
Founder effects reshape genetic variation.
Drift creates population structure.
Drift contributes to FST.
Drift influences GWAS, polygenic scores, and demographic inference.

0.66 Summary

One of the most surprising lessons in population genetics is that evolution does not always require selection.

Randomness alone can dramatically reshape genomes.

Given enough time, drift can:

Eliminate variants
Fix variants
Reduce diversity
Differentiate populations

Many genomic patterns observed today arise not because some variants were better than others, but simply because finite populations are subject to chance.

Understanding genetic drift is therefore essential for evolutionary biology, statistical genetics, genomics, and modern precision medicine.

Nivedita Bhadra

Large Language Models and Working with LLM APIs

1 Introduction

2 Next-Token Prediction, Built From Scratch

3 The Evolution of Language Modeling Architectures

4 What Makes a Model “Large”

5 How an LLM Is Built: Three Phases

6 Why LLMs Matter in Practice

7 The Current LLM Landscape

8 What Is an LLM API?

9 Building a Minimal LLM API, End to End

9.1 Role-Based Prompting

9.2 A Chatbot Loop

10 Setting Up the Current Real SDKs

11 Wrapping the API in a Simple Web Interface

12 Best Practices

13 Limitations

14 Summary

15 References

16 Try It Yourself

16.1 Solutions

t-test or ANOVA? Choosing the Right Test for Before/After Treatment Across Multiple Groups

1 Introduction

2 What a t-test Can and Cannot Test

3 Why Not Run Several t-tests?

4 The Mixed-Design ANOVA

5 Decision Guide

6 Worked Example: A Biomarker Across Three Groups

7 Assumptions and Alternatives

8 Conclusion

Generative AI and Prompt Engineering

1 Introduction

2 What Generative AI Produces

3 Generative AI versus Conventional Machine Learning

4 Large Language Models

4.1 Why LLMs matter in practice

5 Prompt Engineering as a Design Problem

6 From Prompt to Output: Simulating Token Sampling

7 Zero-Shot, Few-Shot, and Chain-of-Thought Prompting

8 A Heuristic for Prompt Specificity

9 Applying LLMs to Academic and Professional Tasks

9.1 A worked example: reusing build_prompt() for a recurring task

10 The Generative AI Tool Landscape

11 Best Practices

12 Limitations of Current Generative AI Systems

13 Summary

14 References

15 Try It Yourself

16 Solutions

16.1 Solution 1: Extending specificity_score() with a length constraint

16.2 Solution 2: A near-tied vocabulary and its entropy curve

16.3 Solution 3: A coloc.abf summary prompt, before and after engineering

16.4 Solution 4: Verifying one row of the tool landscape table

The barbeque question: will AI’s data center problem end like the genome did?

The $3 Billion Genome

The Electron Problem

So, Same Story or Different One?

Multivariate Concepts in Statistical Genetics: From LD Score Regression to Genomic SEM

1 Multivariate Concepts in Statistical Genetics

1.0.1 1. Why Do We Need Multivariate Genetic Models?

1.0.2 2. From Genetic Overlap to Genetic Architecture

1.0.3 3. Why Traits Do Not Need to Come From the Same Sample

1.0.4 2. Polygenicity and Linkage Disequilibrium

1.0.4.1 2.1 LD Score

1.0.5 4. The Intuition Behind LD Score Regression

1.0.6 5. Why the LDSC Intercept Matters

1.0.7 3. SNP Heritability Using LD Score Regression

1.0.7.1 3.1 Why Sample Size Matters

1.0.8 4. Genetic Covariance and Genetic Correlation

1.0.8.1 4.1 Genetic Covariance

1.0.8.2 4.2 Genetic Correlation

1.0.9 8. Why Shared Genetic Architecture Requires New Models

1.0.10 5. From LD Score Regression to Genomic SEM

1.0.11 6. The Two-Stage Framework of Genomic SEM

1.0.11.1 Stage 1: Estimate Genetic Covariance Structure

1.0.11.2 Stage 2: Fit a Structural Equation Model

1.0.12 9. Genome-Wide SEM vs Multivariate GWAS

1.0.12.1 Genome-Wide SEM

1.0.12.2 Multivariate GWAS

1.0.13 7. Practical Setup in R

9.1 A worked example: reusing `build_prompt()` for a recurring task

16.1 Solution 1: Extending `specificity_score()` with a length constraint

16.3 Solution 3: A `coloc.abf` summary prompt, before and after engineering

1.0.32 23. Why `linprob` Is Needed