It appears that there is a threshold that must be met when scrolling the content of a world space ScrollView which is not adjusted based on the size of the scroll view. This makes it impossible to scroll sufficiently small ScrollViews (if there are no scrollbars).
In Client Sim, there is no such threshold.
In-Game example:
Editor example: