Place the tea bags, cinnamon, ginger and allspice in the coffee filter of a drip coffeemaker. Add water; brew according to manufacturer's directions.

Meanwhile, in a small saucepan, combine the milk, brown sugar and creamer. Cook and stir over medium heat until heated through and sugar is dissolved. Pour milk mixture into mugs; stir in tea. Dollop with whipped topping and sprinkle with nutmeg if desired.