Causal Bayesian networks

A causal Bayesian network is a probabilistic graphical model whose arrows are meant to represent causal structure, not just statistical association. That one sentence contains the whole idea.

A Bayesian network already gives you a compact way to represent a joint probability distribution over many variables. A causal Bayesian network adds a stronger claim: if there is an arrow from to , the model is saying that changing could change , at least through the mechanism represented by the graph. This is why the same graph can be read in three different ways at once:

as a factorization of a joint distribution,
as a map of conditional independences,
as a model of what happens under interventions such as .

The key object is a directed acyclic graph, or DAG. Directed means the edges have arrows. Acyclic means you cannot follow arrows and come back to where you started. So a causal Bayesian network can represent “Smoking causes tar buildup, which raises cancer risk,” but not a same-time feedback loop unless that loop is unfolded over time.

Three big questions organize the subject:

Graph semantics. What exactly do nodes, arrows, paths, and missing arrows mean?
Causal inference. How do we go from observed data to quantities like rather than just ?
Modeling limits. When do hidden confounders, non-identifiability, or strong assumptions such as faithfulness stop the graph from answering the question we want?

A useful first mental picture is this: a causal Bayesian network is a machine for separating seeing from doing. Statistics alone tells you what tends to occur together. Causality asks what would happen if you actively changed one variable. The graph is the bridge between those two.

Graphs, variables, and the causal Markov idea

A causal Bayesian network is built from a set of random variables. Each node in the graph is one variable: smoking status, blood pressure, treatment received, exam score, rainfall, or anything else the model tracks. An arrow means is a direct cause of in the model.

That last phrase matters. It does not mean is the only cause of , or that the effect is large, or that it always happens. It means that once the model is set up, is allowed to depend directly on through its local mechanism.

The graph must be a DAG. The acyclic part is not just cosmetic. It guarantees that the model has an ordering: some variables come earlier, others later, and every node depends only on its parents , never on its own descendants. That lets the joint distribution factorize cleanly as

This is the causal Markov factorization.

Here is the intuition. Suppose a node has parents “Genetics” and “Smoking.” Then instead of specifying how that node depends on every variable in the whole system, you only specify if those are its parents. The graph says: once you know the direct causes, extra information about non-descendants adds nothing further. That is the causal Markov idea in words.

The trap here is to think the graph is just a picture. It is really a set of commitments:

Local dependence. Each variable is generated from its parents.
Global factorization. The whole joint distribution breaks into local pieces.
Conditional independence implications. Missing edges mean some dependences are absent once the right variables are conditioned on.

In practice, this is why DAGs scale. A full joint distribution over many variables is huge. A graph lets you replace one giant object with many smaller local conditionals.

How the graph encodes a joint distribution

Take the classic example:

Rain Sprinkler
Rain Wet grass
Sprinkler Wet grass

This says rain influences whether the sprinkler is used, and both rain and sprinkler affect whether the grass is wet.

Without a graph, the joint distribution over these three variables is just one object:

With the graph, it becomes

That is the whole payoff. The DAG turns one high-dimensional joint into local pieces attached to nodes.

Why this reduces complexity

Imagine each variable is binary. Then the unconstrained joint over three variables needs probabilities for all combinations, minus one because probabilities sum to . But with the graph:

is one number,
needs two conditional probabilities,
needs four.

You still have to define the model, but the structure tells you where dependence lives. For larger systems, that matters enormously.

Local tables and densities

For discrete variables, the local pieces are often conditional probability tables. For continuous variables, they may be conditional densities or parametric models such as linear-Gaussian relationships. The graph itself does not force a single parametric form. It only tells you which arguments each local mechanism may depend on.

A good way to read a DAG is node by node:

Find a node.
List its parents.
Write one local conditional for that node given its parents.
Multiply all those local conditionals together.

If you can do that, you can write the joint distribution from the graph.

Direct causes, indirect causes, and causal paths

One arrow means one step of direct influence. If , then is a direct cause of in the model. If , then may still affect , but now part of that effect is indirect, carried through .

A child-friendly analogy helps here. Think of dominoes. If domino knocks over domino , and knocks over domino , then affects even without touching it directly.

This gives three basic path ideas:

Parent / child. If , then is a parent of and is a child of .
Ancestor / descendant. If there is a directed path from to , then is an ancestor of and is a descendant of .
Path. A path is any sequence of connected nodes, ignoring arrow direction for the moment when reasoning about association.

Why ignore arrow direction on a path? Because statistical association can travel along a path in ways that depend on structure and conditioning, not just on the existence of a directed causal flow. That is exactly why later we need d-separation.

What one arrow does not mean:

it does not mean the effect is deterministic,
it does not mean there are no omitted causes,
it does not mean every association between and is causal,
it does not mean there is no mediation through other variables as well.

A very common mistake is to see an arrow and think “ explains all of .” No. It means is one direct ingredient in the mechanism for .

This distinction becomes crucial when drawing adjustment sets. If a variable lies on a causal path from treatment to outcome, adjusting for it may remove part of the effect you actually want to estimate. If it lies on a back-door path created by common causes, adjusting for it may be exactly what saves you.

Conditional independence and d-separation

Most of the practical power of a causal Bayesian network comes from knowing which paths carry association and which paths are blocked. This is where the graph stops being a picture and starts becoming a reasoning tool.

The key idea is conditional independence. Two variables are conditionally independent given a set of other variables if, once those variables are known, learning one tells you nothing more about the other. In DAGs, the graphical test for this is d-separation.

You only need to master three local patterns.

1. Chain:

Here, can influence through . If you do not condition on , the path is open. If you do condition on , the path is blocked.

Example: Rain Wet road Traffic. If you already know whether the road is wet, extra information about rain tells you less about traffic through that path.

2. Fork:

Now is a common cause of and . Again, if you do not condition on , the path is open. If you do condition on , the path is blocked.

Example: Genetics Smoking and Genetics Cancer. Smoking and cancer may be associated partly because of their shared cause. Conditioning on genetics can remove that back-door association.

3. Collider:

This is the weird one, and the one learners most often get backward. A collider blocks the path by default. But conditioning on the collider, or on one of its descendants, can open the path.

Example: Skill Admission Wealth. In the general population, skill and wealth might be unrelated. But among admitted students, the two can become associated because admission selects for one or the other. That is collider bias.

The rule of d-separation

A path is blocked if it contains:

a chain or fork where the middle node is conditioned on, or
a collider where neither the collider nor any of its descendants is conditioned on.

If every path between and is blocked by a conditioning set , then and are d-separated by , and the model implies a conditional independence between them.

This is the toolkit behind adjustment, confounding control, and selection-bias diagnosis.

The collider case is the trap: conditioning does not always remove bias. Sometimes it creates it.

Why conditioning can help or hurt

People often learn one rule too early: “control for more variables.” In causal work, that rule is false.

When conditioning helps

Conditioning helps when you block a non-causal path that creates spurious association.

Suppose

Here is a confounder: a common cause of treatment and outcome . If you compare people with different values of without adjusting for , you mix together the effect of with differences due to . Conditioning on blocks that fork path and can reduce bias.

When conditioning hurts

Now suppose

Here is a collider. Before conditioning, the path between and is blocked. After conditioning on , it opens. You create an association that was not there.

A classic selection version is Berkson's bias: if hospital admission depends on either disease or disease , then among hospitalized patients the diseases can appear negatively associated even if they are independent in the general population.

The practical lesson

Before adjusting for a variable, ask:

Is it a common cause that blocks a back-door path?
Is it a mediator on the causal path I want to measure?
Is it a collider or affected by a collider?
Is it a descendant of treatment, which may introduce post-treatment bias?

The trap here is to think “more conditioning means more fairness.” Graphically, some variables are medicine and others are poison.

Interventions, do-notation, and causal effects

A causal Bayesian network earns the word causal because it tells you how to represent an intervention. The symbol is , read as “set to by intervention.” This is different from merely observing that .

That distinction is the heart of the subject:

Observation: means we looked and found units with .
Intervention: means we actively set to .

These are equal only under special conditions, typically when there is no confounding between and .

Graph surgery: cutting incoming arrows

In a causal DAG, an intervention on is represented by removing all arrows into , then fixing to the chosen value.

Why remove incoming arrows? Because once you intervene, is no longer being generated by its usual causes. If a doctor assigns a treatment, that assignment breaks the old mechanism that would have determined treatment naturally.

So if the observational model factorizes as

then under , the factor for is replaced by the fixed setting , and its original parent-driven mechanism is cut away.

Causal effect measures

A few quantities appear constantly:

Total causal effect. How much does changing change the distribution of overall?
Average causal effect. For binary treatment, often
Controlled direct effect. The effect of changing while holding a mediator fixed.
Natural direct and indirect effects. Finer decompositions that usually need stronger assumptions.

If you remember only one thing from this section, remember this:

answers “what do we see among cases with ?”
answers “what would happen if we made ?”

Those are not the same question.

Back-door adjustment

The back-door criterion is the cleanest graphical recipe for identifying a causal effect from observational data. The name comes from the kind of path it blocks: a path from to that starts by going into , usually through a common cause.

Suppose you want the causal effect of treatment on outcome . A set of variables is a valid adjustment set if, roughly speaking:

it blocks every back-door path from to ,
it does not include descendants of .

Why exclude descendants of ? Because they may lie downstream of the treatment and conditioning on them can distort the very effect you want.

A simple example

Let

Exercise Weight loss
Motivation Exercise
Motivation Weight loss

Here motivation is a confounder. If highly motivated people both exercise more and lose more weight for other reasons too, then the observational quantity mixes causal effect with confounding.

Adjusting for motivation gives the back-door formula:

For continuous , replace the sum with an integral.

This formula says: estimate the outcome difference within levels of the confounder, then average back over the population distribution of the confounder.

Intuition in plain words

Back-door adjustment works because it tries to simulate what randomization would have done. Within each stratum of , the treatment groups are more comparable. If blocks all the non-causal paths, then the remaining association between and can be interpreted causally.

The trap here is to adjust for variables that feel predictive rather than variables that block confounding. Prediction and causal identification are not the same task.

Front-door adjustment and instrumental-style ideas

Back-door adjustment is powerful, but it fails when important confounders are unobserved. The front-door criterion is one of the classic ways around that.

Front-door intuition

Suppose

there is hidden confounding between and ,
but affects only through the mediator ,
and there is no unblocked back-door path from to ,
and all back-door paths from to are blocked by conditioning on .

Then the effect of on can still be identified through the mediator . This is front-door adjustment.

A canonical story is: smoking affects tar deposits in lungs, and tar affects cancer. Even if there is hidden confounding between smoking and cancer, observing the mediator tar can sometimes let you recover the causal effect because the treatment-to-mediator and mediator-to-outcome pieces are separately learnable.

The logic is subtle but powerful: instead of trying to compare treated and untreated units directly, you decompose the causal path through an observed mediator.

Instrumental-style thinking

An instrumental variable is a variable that shifts treatment but affects outcome only through , and is not confounded with the outcome except through treatment. Instrumental strategies are related in spirit: they seek identification through special structural features rather than ordinary covariate adjustment.

This comes with strong assumptions:

Relevance: must affect .
Exclusion: must not affect except through .
Independence: must not share hidden causes with .

These are often harder to defend than back-door assumptions, but sometimes they are the only path available.

Structural causal models and where Bayesian networks fit

A causal Bayesian network is often best understood as the graphical face of a structural causal model.

A structural causal model says each variable is generated by an equation:

where:

are the parents of in the graph,
is the local mechanism,
is an exogenous noise term, meaning influence that comes from outside the modeled system.

The DAG tells you which variables appear as arguments in each structural equation. The structural equations tell you how those causes actually combine.

Why this is broader than a bare DAG

A graph alone gives you qualitative structure:

who can directly affect whom,
which independences are implied,
which interventions might be identifiable.

A structural model gives more:

the data-generating mechanisms,
the effect of interventions by replacing equations,
the basis for counterfactual questions.

This is the cleaner mental model: the DAG is the wiring diagram; the structural equations are the machine inside the box.

Interventions in structural form

If the original system has

then an intervention replaces the first equation with

That replacement generates a new distribution over the remaining variables. This is the structural version of “cutting incoming arrows.”

Some causal questions can be answered from graph structure plus observational data. Others need stronger assumptions about these mechanisms, hidden variables, or functional forms. This is why causal Bayesian networks are powerful but not magical.

Counterfactuals and potential outcomes in DAG language

Interventional questions ask what happens if we set a variable. Counterfactual questions go one level deeper: for this particular unit, what would have happened under a different treatment?

Example:

factual: this patient took the drug and recovered,
counterfactual: would this same patient have recovered if they had not taken it?

That is not directly observed. It is inferred through a model.

DAG language and potential outcomes

In potential-outcomes notation, one often writes:

for the outcome if treatment were set to ,
for the outcome if treatment were set to .

The average treatment effect is

In structural causal models, these potential outcomes can be interpreted as counterfactual variables produced by interventions on the structural equations. So the DAG tradition and the potential-outcomes tradition are not enemies. They are often two languages for closely related ideas.

Average versus unit-level questions

This distinction matters:

Average causal effects ask about population-level differences.
Counterfactuals often ask unit-level “what if” questions.

A DAG plus assumptions can identify average effects even when individual counterfactual pairs such as are never jointly observable.

The trap here is to think counterfactuals are optional philosophy layered on top. They are the language needed for questions of attribution, mediation, fairness, and explanation at the individual level.

Learning causal Bayesian networks from data

So far, the graph was given. In practice, people often try to learn it from data. This means learning two things:

parameters of local conditional distributions,
structure of the graph itself.

Parameter learning is usually the easier part once the graph is known. Structure learning is the hard part.

Two broad strategies

Constraint-based methods

These methods test conditional independence relations in the data and try to build a graph consistent with them. The classic example is the PC algorithm. The logic is: if the graph implies independences, and the data seem to show them, remove or orient edges accordingly.

Score-based methods

These methods search over candidate DAGs and assign each a score trading off fit and complexity. The classic example is GES (Greedy Equivalence Search). The goal is to find a graph, or equivalence class of graphs, with a high score.

Why observational data usually cannot give a unique DAG

Here is the deep limitation: multiple DAGs can encode the same set of conditional independences. Such DAGs are in the same Markov equivalence class.

A famous tiny example:

These can imply the same observational independences, so observational data alone may not distinguish them. What is often identifiable is not one fully directed DAG, but a partially directed graph representing the whole equivalence class.

Two DAGs are Markov equivalent when they share the same skeleton and the same v-structures.

Why this matters in practice

This is why causal discovery from observational data is fragile. It relies on assumptions such as:

Causal sufficiency: no hidden common causes among modeled variables.
Faithfulness: observed independences are due to graph structure, not accidental parameter cancellation.
Correct variable set and measurement quality.

If these fail, learned graphs may be wrong, underdetermined, or misleading.

Still, structure learning is useful. It can narrow possibilities, suggest hypotheses, and combine with interventions or domain knowledge to recover more orientation than observational data alone permits.