The Timing profile of the Neural network

The code was profiled using the NVIDIA cuda profiler which gives the execution time of the individual kernel calls.

It looks like the cross correlations are taking the most time. The parallelization is done on the filter pixels which is the smallest size compared to the sensitivities and feature map sizes. CudaFFT might be a good option for the same.