Transfer learning on MNIST

Course Discussions AI Deep Learning by IIT Roorkee
#1

Hello,
I was trying to apply transfer learning on the MNIST dataset from keras.datasets. The mnist dataset is grayscale with only single channel (28,28,1). But, as large networks like VGG or inception need input images in size (224,224,3), I was trying to resize the whole training data to that particular size.

  1. To convert the images from grayscale to RGB I used cv2.COLOR_GRAY2RGB. I was successful in transforming the whole dataset.
    code snippet for reference:

def fixColor(image):
return(cv2.cvtColor(image, cv2.COLOR_GRAY2RGB))

new_train = []
for i in range(len(input_train)):
new_train.append(fixColor(input_train[i]))

Now, the next function takes input in numpy array format, so I had to convert this list(new_train) to numpy. This was successful with:

from numpy import array
a = array( new_train )

  1. Now, to change the height and width from (28,28) to (224,224) I decided to use image.smart_resize (from tensorflow.keras.preprocessing.image). It worked well when I trying to experiment with small examples. But when I tried transforming the whole training dataset, the kernal crashes, as the RAM being used shoots up. I tried both on local system and google colab to no avail.
    code snippet:

new_train2 = [] # a new list as the append operation is more efficient in list compared to numpy array
for i in range(len(a)):
new_train2.append(image.smart_resize(a[i],(224,224)))

I tried many different hacks but nothing worked completely. Could anyone kindly help me with this issue?

Entire code :

import tensorflow as tf
from tensorflow import keras
from keras.datasets import mnist
import numpy as np
import cv2
from tensorflow.keras.preprocessing import image

(input_train, target_train), (input_test, target_test) = mnist.load_data()

def fixColor(image):
return(cv2.cvtColor(image, cv2.COLOR_GRAY2RGB))

new_train = []
for i in range(len(input_train)):
new_train.append(fixColor(input_train[i]))

from numpy import array
a = array( new_train )

new_train2 = []
for i in range(len(a)):
new_train2.append(image.smart_resize(a[i],(224,224)))

Thank you.

#2

Hi, try a greyscale to multi-channel conversion: (28, 28) -->> (28, 28, 3) and then feed it to the VGG16 pre-trained model. Let me know if it worked.

#3

Hi, I’ve converted grayscale to multi-channel using cv2.COLOR_GRAY2RGB as mentioned above.

I tried VGG16 pre_trained model with input shape (28,28,3)… I’m getting the below error…

ValueError: Negative dimension size caused by subtracting 2 from 1 for ‘{{node sequential_13/vgg16/block5_pool/MaxPool}} = MaxPoolT=DT_FLOAT, data_format=“NHWC”, explicit_paddings=[], ksize=[1, 2, 2, 1], padding=“VALID”, strides=[1, 2, 2, 1]’ with input shapes: [?,1,1,512].

#4

Try the following:

train_X= np.array(train_data.iloc[:,1:])
test_X= np.array(test_data.iloc[:,1:])
train_Y= np.array (train_data.iloc[:,0])
test_Y = np.array(test_data.iloc[:,0])

train_X=np.dstack([train_X] * 3)
test_X=np.dstack([test_X]*3)
train_X.shape,test_X.shape

train_X = train_X.reshape(-1, 28,28,3)
test_X= test_X.reshape (-1,28,28,3)
train_X.shape,test_X.shape

from keras.preprocessing.image import img_to_array, array_to_img
train_X = np.asarray([img_to_array(array_to_img(im, scale=False).resize((48,48))) for im in train_X])
test_X = np.asarray([img_to_array(array_to_img(im, scale=False).resize((48,48))) for im in test_X])
#train_x = preprocess_input(x)

train_X = train_X / 255.
test_X = test_X / 255.
train_X = train_X.astype('float32')
test_X = test_X.astype('float32')

train_Y_one_hot = to_categorical(train_Y)
test_Y_one_hot = to_categorical(test_Y)

train_X,valid_X,train_label,valid_label = train_test_split(train_X, train_Y_one_hot, test_size=0.2, random_state=13)

IMG_WIDTH = 48
IMG_HEIGHT = 48
IMG_DEPTH = 3
BATCH_SIZE = 16

train_X = preprocess_input(train_X)
valid_X = preprocess_input(valid_X)
test_X  = preprocess_input (test_X)

conv_base = VGG16(weights='<<put in the path to the weight file>>',
                  include_top=False, 
                  input_shape=(IMG_HEIGHT, IMG_WIDTH, IMG_DEPTH)
                 )
#5

Hi, I’m getting this error for train_test_split function.

ValueError: Found input variables with inconsistent numbers of samples: [48000, 60000]

I checked the shapes of both train_X and train_Y_one_hot. They are (60000, 48, 48, 3) and (60000, 10) respectively. What’s the issue?

#6

You are running into that error because your X and Y don’t have the same length (which is what train_test_split requires). The code that I gave was for Fashion MNIST, try to modify it according to your requirement for MNIST.

#7

I’ve edited the code and it ran successfully with Dense layers. But the accuracy is extremely poor. val_accuracy: 0.1050

I want to use Conv2d layers instead of Dense layers. But when I use them, it shows
“Input 0 of layer conv2d_2 is incompatible with the layer: : expected min_ndim=4, found ndim=2. Full shape received: (None, 512)”
How should I use Convolutional layers along with transfer learning?

#8

Try the following notebook, it is for Fashion MNIST, so you may have to modify it:

https://www.kaggle.com/anandad/classify-fashion-mnist-with-vgg16