Synthetic population for Bologna: maximum-entropy approach via persistent contrastive divergence on heterogeneous ISTAT data

Generating a synthetic population is a very new and increasingly important problem for downstream applications. A synthetic population is required for lack of census data, and conserving individual privacy. This thesis builds a synthetic population of 500,000 inhabitants for the Comune of Bologna, each described by K=34 attributes, all based on aggregate tables acquired from ISTAT database. The already existing approaches, fail to obtain the necessary diversity of the synthetic population, generating problems in the downstream applications. Whereas more complicated approaches become numerically hard to solve for large number of attributes. For this project the joint space of 34 categorical attributes contains 2.6 × 10^19 possible individual profiles. We address the first and the second problem with GibbsPCDSolver, a stochastic maximum-entropy solver based on Persistent Contrastive Divergence. The most important problem that this project addresses as the central theme –where most existing approaches refuse to confront– is dealing with real-world heterogeneous data. The real ISTAT tables are published at five different geographical levels, and contradict each other. No joint distribution can satisfy all published targets at once, so any solver necessarily leaves residual error somewhere. For his problem a reliability-weighting scheme has been proposed. A separate difficulty arises from the requirement that no individual be logically impossible. We demonstrate the effect directly and repair it with a Metropolis–Hastings move that transposes an entire block of attributes in one step. We support this reading with four diagnostic tools developed in the thesis. The resulting population contains no logically impossible individual and retains full diversity which is precisely the property that reweighting methods such as raking lose, and which downstream agent-based simulations of Bologna require.