While it can be done by using deep learning framework profiler or nvidia-smi output(http://forums.fast.ai/t/visualizing-gpu-memory-usage/1103/5) for all GPU instances but looking for any existing tool to visualise GPU memory just like ambari provide a nice dashboard for CPU memory (https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.1.5/bk_ambari-operations/content/viewing_the_cluster_dashboard.html).
you can perhaps implement it with <a href="https://developer.nvidia.com/perfworks" rel="nofollow">https://developer.nvidia.com/perfworks</a>