In stand mixer bowl, add 1 cup flour, yeast, sugar, salt, ginger, lemon and orange zest; mix well. In saucepan or microwave-safe dish, heat water, orange juice, and milk to 120-130°F. Add warm liquids to flour mixture. Add butter. Using a paddle attachment, blend on low speed for 30 seconds; beat on medium speed for 4 minutes.
Switch to dough hook attachment. Gradually stir in enough remaining flour to make a firm dough. Knead until smooth and elastic, about 5 to 7 minutes.(NOTE: For hand mixing: following above steps, mix ingredients in large mixing bowl using a wooden spoon or dough whisk. Knead on floured surface until smooth and elastic.) Place dough in lightly oiled bowl and turn to grease top. Cover; let rise until indentation remains after poking dough with finger down to second knuckle, about doubled in size.
Prepare pan: Grease one 9x5-inch bread pan; set aside.
Turn dough onto lightly floured surface; punch down to remove air bubbles. Roll or pat into a 14x7-inch rectangle. Starting with shorter side, roll up tightly, pressing dough into roll. Pinch edges and taper ends to seal. Place in prepared pan. Cover; let rise until indentation remains after touching.
Preheat oven to 375°F.
Bake for 30 to 40 minutes. Remove from pan; cool on wire rack.