Rajnath Singh refuses