This is quite possible but not easy at all; you'll to write several things to allow custom edit in the
. I think modal form is not a good idea as you can do much better by providing a custom in-place editor. However, a modal form is also possible: you in-line editor will be the button you want; and this button will show your modal form.
I tried to describe a road map on this topic here: How to get response when click PropertyGrid
Again, this is not easy, so I'm sorry I cannot afford developing code specially for your case. It would be far beyond the format of CodeProject Questions-and-Answers. You can get the idea, go to help pages on the types I list in my Answer and write your own code, ask another question if and when you face any problems.
See also: Setting value in textbox when it is taken from combobox..