The problem with email image spam classification is known from the year 2005. There are several approaches to this task. Lately, those approaches use convolutional neural networks (CNN). We propose a novel approach to the image spam classification task. Our approach is based on CNN and transfer learning, namely Resnet v1 used for semantic feature extraction and one layer Feedforward Neural Network for classification. We have shown that this approach can achieve state-of-the-art performance on publicly available datasets. 99% F1-score on two datasets [dredze 2007, Princeton] and 96% F1-score on the combination of these datasets. Due to the availability of GPUs, this approach may be used for just-in-time classification in anti-spam systems handling huge amounts of emails. We have observed also that mentioned publicly available datasets are no longer representative. We overcame this limitation by using a much richer dataset from a one-week long real traffic of the freemail provider Email.cz. The training data annotation was created by user labeling of the emails. The image spam (and image ham even more) tackles privacy issues. We overcame it by publishing extracted feature vectors with associated classes (instead of images itself). This data does not violate privacy issues. We have published Email.cz image spam dataset v1 via the AcademicTorrents platform and propose a system, which achieves up to 96% F1-score with presented model architecture on this novel dataset. Providing our dataset to the community may help others with solving similar tasks.