Contextual MDPs WM (in preparation)

Formalization and proof of Proposition #1 from the paper.