Preheat oven to 350 degrees F (175 degrees C). Spray 12 muffin cups with cooking spray.

Heat olive oil in a skillet over medium heat; cook and stir asparagus, green bell pepper, and onion in the hot oil until softened, 5 to 10 minutes.

Whisk eggs, milk, salt, and black pepper together in a bowl. Mix cooked vegetables and Cheddar cheese into egg mixture. Spoon about 1/4 cup mixture into each muffin cup.

Bake in the preheated oven until frittatas are set in the middle and lightly browned, about 20 minutes.