When I look into the 'tf_core_gpu_kernels.lib' (one can open a. Don't know how to check every compilation step, though. `tf_core_gpu_kernels_generated_softmax_op_gpu.cu.cc.). The build directory contains many files related to 'softmax' (e.g. The data type of the out layer is float32 (out/Softmax -> (,)).Ĭreating the softmax-op in C++ and placing it on GPU explicitly throws this error message:Ĭannot assign a device for operation 'tsoftmax': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.Ī call to tensorflow::LogAllRegisteredKernels() showed also that Softmax is only available for CPU! This placement is also confirmed by high CPU/low GPU utilization, and also apparent from profiling the application. Out/Softmax: (Softmax): /job:localhost/replica:0/task:0/device:CPU:0 Out/BiasAdd: (BiasAdd): /job:localhost/replica:0/task:0/device:GPU:0 Out/MatMul: (MatMul): /job:localhost/replica:0/task:0/device:GPU:0 When I use the C++ program and load the freezed graph (following closely the LoadGraph() function in - ReadBinaryProto() and session->Create()), and log again the device placements, I find that the Softmax is placed on CPU (all others ops are on GPU):ĭense_6/MatMul: (MatMul): /job:localhost/replica:0/task:0/device:GPU:0ĭense_6/BiasAdd: (BiasAdd): /job:localhost/replica:0/task:0/device:GPU:0ĭense_6/Relu: (Relu): /job:localhost/replica:0/task:0/device:GPU:0 To save the graph, the freeze_graph script is used (the script producing the log above loads again the freezed graph in. Out/Softmax: (Softmax): /job:localhost/replica:0/task:0/gpu:0 Out/BiasAdd: (BiasAdd)/job:localhost/replica:0/task:0/gpu:0 Out/BiasAdd: (BiasAdd): /job:localhost/replica:0/task:0/gpu:0 Out/MatMul: (MatMul): /job:localhost/replica:0/task:0/gpu:0 In python (tensorflow-gpu) all ops use the GPU (using log_device_placement):

The model is trained in Python (using the Keras framework), then saved and inference is done using a C++ program that facilitates a CMake build of TensorFlow (following basically those instructions: ). Tensorflow places softmax op on cpu instead of gpu I have a tensorflow model with multiple inputs and several layers, and a final softmax layer.

Inferred latch verilog