Abstract Every day, we encounter numerous photographs of faces, whether on social media, television, or smartphones. From an evolutionary perspective, faces are particularly salient and tend to be remembered, likely due to their relevance for survival and social interaction. However, not all face photographs have the same likelihood of being remembered. While some research has explored the features that contribute to face memorability, this area remains largely uncharted. Here, we propose a fast approach to control the memorability of face images. Our approach begins by identifying a hyperplane in the latent space of StyleGAN that separates high and low memorable images. We then adjust the memorability of an image by shifting it along the positive or negative direction of this hyperplane’s normal vector. Analyzing the StyleGAN augmented latent space reveals the impact of individual face attributes on an image’s memorability. A unique aspect of our work is that it is successful on both real and synthesized face images. Furthermore, our method is not restricted to face photographs; we show its success in modifying the memorability of images of various categories. Finally, we propose a new multi-level memorability modification technique, which assists in modifying the memorability scores while maintaining the identity of the face images. Similar content being viewed by others Data availability The codes and models trained and/or analyzed during the current study are publicly available in https://github.com/mamyou96/FaceMem.git. References Isola, P., Xiao, J., Torralba, A. & Oliva, A. What makes an image memorable? In CVPR 2011 145–152 (IEEE, 2011). Bainbridge, W. A., Isola, P. & Oliva, A. The intrinsic memorability of face photographs. J. Exp. Psychol. Gen. 142(4), 1323–34 (2013). Isola, P., Xiao, J., Parikh, D., Torralba, A. & Oliva, A. What makes a photograph memorable?. Pattern Anal. Mach. Intell. IEEE Trans. 36, 1469–1482 (2014). Newman, A. et al. Multimodal