Earlier work contains, among other things ideas, facts, scenes a faire, non-original elements, functionally dictated aspects, elements that have few or no alternative modes of expression.
Earlier work also contains copyright-protected elements.
Defendant bases later work in part on earlier work.

Derivative work?

Defendant takes copyright-protected elements.
Defendant may or may not have a valid defense.
Taking is substantial.

Later work is prima facie an infringing derivative work, subject to possible defenses.

Defendant takes copyright-protected elements.
Has no valid defense.

Later work is infringing derivative work. Defendant is liable.