Sensory neuroscientists must carefully choose or design experimental stimuli in order to test hypotheses. Recently, scientists have leveraged computational models and automatic differentiation to synthesize model-optimized stimuli, such as model metamers -- stimuli that are physically distinct but that produce identical model responses. However, there exist many different stimuli that satisfy the constraints of the stimulus-generation process, but that may give rise to different scientific interpretations. Here, we propose adding a penalty term to the objective function used to generate the stimuli. This biases the synthesis procedure, allowing researchers to preferentially search for stimuli with certain properties. We demonstrate the use of several penalty functions on a simple LGN-inspired model to increase perceptual diversity among synthesized model metamers. By carefully choosing their penalty functions, researchers can better design stimulus sets to address their scientific question.