Segmenting structure-of-interest is a fundamental problem in medical image analysis. Numerous automatic segmentation algorithms have been extensively studied for the task. However, misleading image information and the complex organ structures with high curvature boundaries may cause under- or over-segmentation for the deformable models. Learning based approaches can alleviate this issue, while they usually require a large number of representative training samples for each use case, which may not be available in practice. On the other hand, manually correcting segmentation errors produces good results and doctors would like such tools to improve accuracy in local areas. Therefore, we propose a 3D editing framework to interactively and efficiently refine the segmentation results, by editing the mesh directly. Specifically, the shape editing framework is modeled by integrating the Laplacian coordinates, image context information and user specified control points. We employ a local patch based optimization to enhance the supplement force field near the control points to improve correction accuracy. Our method requires few (and intuitive) user inputs, and the experimental results show competitive performance of our interactive refinement compared to other state-of-the-art methods.