In this paper we present a distributed regression framework to model data with different contexts. Different context is defined as the change of the underlying laws of probability in the distributed sources. Most state of the art methods do not take into account the different context and assume that the data comes from the same statistical distribution. We propose an aggregation scheme for models that are in the same neighborhood in terms of statistical divergence. We conduct experiments with synthetic data sets to validate our proposal. Our proposed algorithm outperforms other models that follow a traditional approach.