With the boom of online fashion market, content-based fashion search assisting users to retrieve desired fashion items with an uphold query image, plays a crucial role in a wide range of commercial environments, including fashion community websites and online shopping platforms. Nevertheless, in some scenarios, users may not be satisfied with all attributes of the available query image, and thus would like to change certain attributes (e.g., change collar from v-neck to round) to get the desired fashion item. Towards this end, in this work, we aim to investigate the practical task of fashion search with attribute manipulation, which allows users to replace the unwanted attributes in the available query image with the desired ones and thus achieve a flexible fashion search. Thanks to the remarkable success of the generative adversarial network in various image generation tasks, we aim to enhance the fashion search with attribute manipulation by directly generating the prototype image of the target item. In particular, we seamlessly integrate the prototype image generation and fashion search process in a consolidated GAN framework. In this framework, the generator works on producing the prototype image that meets user's attribute manipulation requirement over the query image, and the discriminator is devised to distinguish the correct attribute manipulation and meanwhile learn a distance metric with an adversarial strategy to facilitate fashion search. Extensive experiments on two real-world datasets have demonstrated the effectiveness of our proposed model.