Last year, Richard Zhang presented published a paper on making conv nets shift-invariant again. I experimented a bit with his proposed layers that implement some sort of anti-aliasing. The effectiveness seems to depend a bit on the architecture used (but this is a general problem with all backbones over-engineered for ImageNet…). I definitely recommend experimenting with Richard’s layers proposed!