Combine first 3 ingredients in a small saucepan. Bring to a simmer over medium heat,stirring constantly. Remove from heat;stir in vanilla.

Pour 3/4 cup brewed coffee into each mug. Add 1/4 cup milk mixture to each mug,stirring well. Serve hot.