I hear you Low-level but try this...
Pick a day when the wind is blowing. Climb higher than you are accustomed to flying, set airspeed to your best rate of climb speed, power for a medium climbing turn and sit back and watch while you do 360's at a set speed and power. You are not trying to hold station over the ground, simply flying 360's at a fixed speed. Bet you your VSI will remain constant throughout. Doesn't climb slower 'downwind'. Downwind refers to a position relative to the ground in our minds only. Your machine can't differentiate - and why should it? If you did exactly the same on a calm day the results would be the same. The machine is doing exactly the same thing relative to the airmass it is in - independent of the ground.
(This of course leaves inertia out of the discussion - we don't weigh enough for it to matter)
The problem is, our minds play tricks on us, suddenly we're descending but we are moving at the same speed (could be a downdraught) so we add power and pull the nose up - we can't stall so its not drilled into us as it is fixed wing pilots - our stress level goes up, we're thinking 10 steps ahead while the airspeed continues to decline. LAST thing you wanna do is aim the machine at the thing you are trying to miss (usually the ground but trees and wires are good to miss too) so you engage the turbo pulling back even more. Kaplaks. I have been tempted into it and have spoken to several people who have survived the impact. None have been able to honestly say that they knew what their airspeed was at the time. Did it matter?
One of my wise past instructors suggested one should aim for the thing you are trying to miss. No point in missing it by miles and losing airspeed, best chance of obtaining the optimum airspeed is to aim at it, then of course miss it .. and the object beyond ... and so on. Adequate airspeed is your only hope.
Um,... wandered off on a tangent there, forgive me.
Low-level, please do it and report back to us. Would like to hear if you will agree then?
Rgds
Len