We develop a variational inference (VI) scheme for the recently proposed Bayesian kernel support vector machine (SVM) and a stochastic version (SVI) for the linear Bayesian SVM. We compute the SVM’s posterior, paving the way to apply attractive Bayesian techniques, as we exemplify in our experiments by means of automated model selection.