There are two kinds of field of view, true and apparent (tFOV & aFOV). tFOV is what angle of the real world the binocular shows you while aFOV is how wide the image circle looks when you look through the binoculars. While there are some intricacies in converting between aFOV and tFOV (simple vs tangent formula, distortion etc) for simplicity here we will just say that two binoculars with the same magnification will have the same aFOV for a tFOV. Now here is where it gets a bit complicated.
Both the tFOV and aFOV are set by the field stop diameter of the eyepiece. Inside the binoculars in the focal plane of the eyepiece is a circular aperture that sets how much of the image from the objective lens is projected through the eyepiece to your eye. This aperture alone sets the field of view. If it is wider the FOV is wider. The tFOV is set by the field stop diameter and objective focal length (notice it is the focal length of the objective, not the diameter! Although larger diameter objectives tend to have longer focal lengths) given by the equation:
tFOV=2arctan(r/fo)
where r is the radius of the field stop and fo is the focal length of the objective.
The aFOV is set by the field stop diameter and eyepiece focal length given by the equation:
aFOV=2arctan(r/fe).
You may notice that the field of view does not depend on objective diameter, prism size, or anything else, only objective and eyepiece focal length and the field stop diameter. The magnification is also set by the ratio of the objective and eyepiece focal lengths so hypothetically for two binoculars with the same objective and eyepiece focal lengths they would have the same magnification but could have different field of views if they had different field stop diameters (I am typing this in bed while very tired so let me know if this doesn’t make sense).
As for the aforementioned mention of prisms being related to the field of view, wider fields of view require a larger image to be formed at the field stop. In order to transmit this larger image from the objective lens to the field stop (at the appropriate orientation) larger prisms must be used or else you will clip out some of the light cone resulting in vignetting.