Would this hurt a charger? (Over powering it)

explaining further... For example, my Accucel6 is a 5W discharger. It can handle a discharge current of max 1A or as needed less if the voltage is above 5V to limit the power to 5W. All of that 5W will be dissipated as heat inside the charger, and the cooling of the charger cannot handle more than 5W. 

They are counting on 90% efficiency of the switching stuff. So when charging at 50W, 10% will be lost as heat inside the charger, again 5W. This is not a coincidence....

As to wether higher or lower voltages are best, I don't think you can simply say: The higher the better (within specs).

For example, if your charger is rated at 24V, connecting it to the 24V of a truck might damage it. First because of the charger in the truck might go above 24V to charge the 24V battery in the truck but also because of hard-to-measure transients.

Also, if the charger allows 30V input, and you're charging a 5V battery at 5A, the high-side-fet will be on for about 1/5th of the time, while the rest of the time the inductor will discharge through a diode at 5.6V. Now, the input is loaded on average with about 1A, but still the peaks need to be above 5A. this will make things difficult on the  30V powersupply if it too happens to be a switching powersupply. Also, to prevent all this current-differential having to come from the source, there is a capacitor on the input. This capacitor will also experience part of the ripple current of about 5A. And 5A is a lot for such a capacitor.

If the input voltage is only 5.5V, the high side fet will be on most of the time, leading to less stress on the input capacitor and catch-diode....

So if power-dissipated in the input capacitor or diode are the reaons for breaking down of the charger, then you happen to need a voltage that is as small as possible (but large enough for the charger to be able to step-down instead of step-up....)

But if the (average) current at the input is somehow determining the useful life of the charger, then you're right that the input voltage should be as high as possible, but safely within range of the specs.... 

In short: It needs to be within specs, it's difficult (for us) to know exactly if high or low in that range is best.