Emotional space refers to a multi-dimensional emotional model that describes a group of subjective feelings or emotions. Since the existing discrete emotional space is mainly aimed at human’s primary emotions, it cannot describe the complex emotions evoked when watching movies. In order to solve this problem, an emotional fusion space for videos was constructed by selecting movies and TV dramas with rich emotional semantics as the research objects. Firstly, emotional words based on movie and TV drama videos are acquired and analyzed by using subjective evaluation and semantic analysis methods. Then, the emotional word vectors obtained from the above analysis are fused, reduced dimension by t-distributed stochastic neighbor embedding (t-SNE) algorithm, and clustered by bisecting K-means clustering algorithm to get a discrete emotional space for movie and TV drama videos. This emotional fusion space can obtain different categories by changing the value of the emotion classification number without re-labeling and calculation.