People with schizophrenia have difficulties recognizing other people's expressions, emotional states, and intentions; however, much less is known about their ability to perceive and understand social interactions. We used scenes depicting social situations to compare responses from 90 volunteers (healthy controls [HC], schizophrenia [SZ], and bipolar disorder [BD] outpatients from the Hospital del Salvador in Valparaíso, Chile) to the question: “What do you think is happening in the scene?” Independent blind raters assigned a score of 0 (absent), 1 (partial), or 2 (present) for each item based on whether the description identifies a) the context, b) the people, and c) the interaction depicted in the scenes. Regarding the context of the scenes, the SZ and BD groups scored significantly lower than the HC group, with no significant difference between the SZ and BD groups. Regarding the identification of the people and the interactions, the SZ group scored lower than the HC and BD groups, with no significant difference between the HC and BD groups. An ANCOVA was used to examine the relationship between diagnosis, cognitive performance, and the results of the social perception test. The diagnosis had an effect on context (p = .001) and people (p = .0001) but not on interactions (p = .08). Cognitive performance had a significant effect on interactions (p = .008) but not on context (p = .88) or people (p = .62). Our main result is that people with schizophrenia may have significant difficulties perceiving and understanding social encounters between other people.