It looks like raising the 'drag threshold' on Unity's EventSystem Component makes it easier to press buttons without triggering the ScrollRect. You could also attach a DragEventSender to the button, so that only a horizontal drag triggers it, making an accidental scroll less likely.
~ Declan.