I'm not sure why I originally did this, but there's no good reason to
put pointers to the cuda context and stream in the priv struct. They
are directly available in the device context that is already being
stored there.