Introducing CMNIST - building challenging MNIST-like datasets

MNIST - the world’s most famous computer vision benchmark dataset. From a human perspective, the only thing we should detect on MNIST is shape. However, it seems like neural networks are much more sensitive to texture than shape. I debugged many neural networks that ran on videos, and texture sensitivity was the number one failure case. Therefore, I introduce CMNIST. CMNIST combines existing MNIST-like datasets to generate arbitrary challenging datasets using 784 pixels only. I hope this helps to develop better neural network architectures instead of overfitting to existing datasets.

Version 2

Release planned for Q2/2023

WIP

Version 1

• Download technical paper as pdf
• CMNIST generator script [GitHub]
• CMNIST generator script [GitLab]

How to cite CMNIST

If you use any of the CMNIST subsets in your research, please use the following bibtex entry:

@Misc{wenkel2019cmnist,
  author = {Simon Wenkel},
  title  = {Concatenated MNIST (CMNIST). Making 784 pixels challenging again.},
  year   = {2019},
  url    = {https://www.simonwenkel.com/publications/articles/pdf/20190924_CMNIST.pdf},
}