@Flanken.
I think there is a misconception in your explanation there. As I'm certain you know already (but others may not) F-stops are not fixed in size per se but are always relative to the entrance pupil diameter, so if a lens has a focal length of say 200 mm and its entrance pupil diameter is 100 mm, the aperture is 2.0 (200/100), hence the physical size of fast lenses can vary enormously depending on the camera size / sensor they are being designed for. Naturally the focal length has a major impact the longer you go !
How this applies to smaller cameras (and sensor sizes) is easily understood when I give as an example the old Olympus 3040 I used to have which had a f1.8 zoom lens but a minuscule sensor (so you can see the f-stop is related to the size of the entrance pupil not the sensor size) therefore the entrance pupil can be made smaller and still achieve a fast lens.
I do not know why your Olympus DX 300/2.8 was so heavy but this must be related to the relative size of the entrance pupil they were able to achieve (possibly for a cost saving they did not re-engineer older 35mm designs) and that re-engineering for the relatively small difference in size not worth the expense.
There is a both a definite weight and size saving when working with APS-C cameras as can be seen when comparing lenses, and of course this follows with 4/3 and so on down in sensor size, otherwise they would not be able to manufacture fast lenses for these cameras.
Just look at some of the best lenses ever made (Pentax) where the simply wonderful 77/1.8, 31/1.9 and 43/1.8 are tiny/very small lenses (and of course Pentax were the first to make the slower but minuscule pancake lenses too) compared to their full frame (or even APS-C) counterparts in Canon/Nikon lands.
Most probably the cost of engineering and manufacturing a completely new line of APS-C lenses, which would be expensive and less attractive to people who have not spent as much on their cameras as those buying pro or enthusiast-level full frame cameras, is not financially viable so full frame designs are re-engineered to be made suitable for APS-C but with less weight and size saving than would otherwise be possible, the physical sizes between full frame and APS-C cameras not being so great therefore, the savings not worth the expense.
Just my musings