Given the MMDP in figure 1, a coordination problem exists at [s.
As such, I would claim that one must model communication using actions in an MMDP that have effects not on the underlying system state but on the "mental state" of the agents involved.
Coordination problems arise at specific states of an MMDP but must be considered in the context of the sequential decision problem as a whole.
By expanding the state space of the original MMDP to account for this, we recover the usual value function definition.
The expanded MMDP is essentially the cross-product of the original MMDP and the FSM defining the protocol.
This model is able to associate a well-defined sequential value to MMDP states by incorporating the mechanism state where necessary.
The value of imposing a given coordination protocol can be defined using the value function induced in a given MMDP when the protocol is adopted.