If I have seen further it is by standing on the shoulders of giants.

(Newton in a letter to Robert Hooke)

From this we can calculate the giants to be at most 14.3 m tall, assuming they are human-shaped. This is because for seeing further Newton’s eyes must be higher than the eyes of the giant, i.e. his eye height, standing, must be larger than the shoulder-eye-distance of the giants. Sir Isaac is reported to have been five feet six inches (UK) which is about 167.6 cm. Using present-day median values for eye height and shoulder height (see below) for approximate proportions, his eyes were at 155.1 cm. Using this as the shoulder-eye-distance for the giant, by proportions it follows that the giant is at most about 9.198-times taller than Newton, that is about 14.3 meters tall.

FInishing, it is is worth considering that as giants are taller they are probably also proportionately wider and thicker than Newton, so that at maximum they are 778 times heavier than him. When proportionately scaling up body sizes the weight scales by cubes but bone cross section area, which determines their maximal load, only by squares. If giants are subject to biological limits of bone strength, then their bones have at worst only a tenth of the relative strength of Newton’s. Thus such giants can probably best bear their body weight (and Newton’s) when standing neck-deep under water. That however would defeat the purpose.

Detailed calculation:

AVERAGE HUMAN (50th percentiles, in cm)
eye height = 163.26
shoulder height = 144.18
shoulder-eye-distance = 19.08
total height = 175.49
EH:TH = 0.93031
SE:TH = 0.10872414

NEWTON (see here and more here)
with some likelihood five feet six inches  = 167.64 cm, then eye height by proportion = 155.9572

GIANT
shoulder-eye-distance < 155.9572 then total height by proportion <1434.43.

Footnote: The giants and shoulders metaphor has been used at least since scholasticism.