That’s not quite true - images are only shared if you attach the image to federated content, such as a post or comment. Then yes other instances will cache the image.
If you never do that, and just upload an image accidentally like OP then it will not be federated AFAIK.
How hard can it be to fix?
Well, the first thing a server does with your image is to send it to a lot of other, completely unrelated servers.
That’s not quite true - images are only shared if you attach the image to federated content, such as a post or comment. Then yes other instances will cache the image.
If you never do that, and just upload an image accidentally like OP then it will not be federated AFAIK.