This paper introduces a Bayesian inversion approach to estimating steady state ocean circulation and tracer fields. It is based on a quasi-horizontal flow model and a PDE solver for the forward problem of computing solutions to the tracer field advection-diffusion equations. A typical feature of existing ocean circulation inverse methods is a preprocessing stage in which the tracer data are interpolated over a regular grid and the interpolation error is ignored in the subsequent inversion. Our approach only uses interpolated data at those grid points that have neighboring hydrographic stations. By exploiting physically-based models in an integrated fashion, the method provides a statistically unified inversion and tracer field reconstruction with minimal data smoothing. Solving the problem consists of finding information about the circulation and tracer fields in the presence of a number of assumptions (prior information); the resulting posterior probability distribution summarizes what we can know about these fields. We develop a Markov chain Monte Carlo simulation procedure to extract information from the (analytically intractable) posterior distribution of all the parameters in the model; uncertainty about the "solution" is represented by variation in the output of the simulation runs. Our approach is aimed at finding the time-averaged quasi-horizontal flow and tracer fields for an abyssal neutral density layer in the South Atlantic.