The code you had above would partially work. It would probably start multiple processes, and also not handle the external app being closed remotely.
See the code below which does what you want, but also caters for the remote process being closed externally.
In this example I am using the windows calculator. Create a new project with a windows form and a button on that form, paste in the code.
Public Class Form1
Private theApp As System.Diagnostics.Process
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
Dim theAppStartInfo As New ProcessStartInfo("calc.exe")
If theApp IsNot Nothing Then
theApp.CloseMainWindow()
Else
theApp = New Process()
theApp.StartInfo = theAppStartInfo
theApp.Start()
AddHandler theApp.Exited, AddressOf theApp_Exited
theApp.EnableRaisingEvents = True
theApp.SynchronizingObject = Me.Button1
End If
End Sub
Private Sub theApp_Exited(ByVal sender As Object, ByVal e As System.EventArgs)
MsgBox("The process has exited, ExitCode: " + theApp.ExitCode.ToString)
theApp = Nothing
End Sub
End Class