The DQN uses Imagenet learned weights for AlexNet to initialize the layers. Following link can be used to download the imagenet.npy file.

Download imagenet.npy

Once downloaded, place it in

models/imagenet.npy

I guess I forgot to mention this part. Thanks for pointing it out.