(1) Background: loneliness is a problem that becomes increasingly acute in old age, with greater repercussions among socially disadvantaged groups such as indigenous and Afro-descendant older adults. The aim of this research is to analyze the psychosocial variables related to loneliness in old age. (2) Methods: a multi-ethnic sample was involved, with the participation of eight indigenous peoples and Afro-descendant tribal people (n = 1.348). Various gerontological scales previously validated among the Chilean population (De Jong Gierveld Loneliness Scale, Brief Resilient Coping Scale, Health Problems Questionnaire, and Family APGAR questionnaire) and a model are contrasted, establishing the relationship between psychosocial variables and loneliness. (3) Results: Structural equation modeling (SEM) showed the existence of indirect relationships between health problems, via family functioning and resilience, and loneliness. Resilience and family functioning were directly related to loneliness (WLSMV-χ2 (df = 345) = 875.106, p < 0.001; CFI = 0.992; TLI = 0.991; RMSEA = 0.034 [C.I. 90% = 0.031–0.037]). (4) Conclusions: loneliness has cross-culturally affected older Chilean people living in rural areas and it appears that the COVID-19 pandemic has had a negative effect on well-being. This study proves that loneliness is related to several psychosocial variables that can be intervened.