Preheat oven to 350 degrees. Use a 9×5-inch loaf pan and spray with non-stick spray or line with foil and spray with non-stick spray to get out easily for slicing.
Mix brown sugar and cinnamon together in a bowl. Set aside.
In another medium-sized bowl, beat white sugar and butter together using an electric mixer until smooth and creamy.
Beat in eggs, 1 at a time, until blended in; add in vanilla extract.
Combine & whisk flour and baking powder together in another bowl and add into creamed butter mixture and stir until blended.
Mix milk into batter until smooth.
Pour half the batter into the prepared loaf pan; add half the apples and half the brown sugar/cinnamon mixture.
Lightly pat apple mixture into batter.
Pour the remaining batter over apple layer and top with remaining apples and brown sugar/cinnamon mixture.
Lightly pat apples into batter; swirl brown sugar mixture through apples using knife or spoon.
Bake in the preheated oven until a toothpick inserted in the center of the loaf comes out clean, approximately 50-60 minutes.
To make glaze, mix powdered sugar and milk or cream together until well mixed.
Let cool for about 15 minutes before drizzling with glaze.