Heat the olive oil in a medium dutch oven over medium heat. Add the carrots, celery and leeks and sprinkle with the salt. Cook, stirring occasionally, until the leeks soften, about 5 minutes.

Add the thyme, lemon zest, bay leaves and chicken to the pot. Pour in the broth and 2 cups water. Bring to a simmer and cook, uncovered, until the chicken is cooked through and tender, 20 to 25 minutes.

Remove the chicken to a cutting board and let cool slightly. Shred or cut the meat into bite-size pieces

Return the soup to a simmer and add the orzo. Cook until the orzo is just tender, 8 to 10 minutes, returning the chicken to the pot in the last few minutes to heat through. Remove the soup from the heat and discard the lemon zest and bay leaves. Season with salt to taste. Stir in the lemon juice and dill.