The DQN uses Imagenet learned weights for AlexNet to initialize the layers. Following link can be used to download the imagenet.npy file.
Once downloaded, place it in
models/imagenet.npy
I guess I forgot to mention this part. Thanks for pointing it out.