Well if I understand correctly this may help you.
To close a specific form
Private Sub ExitToolStripMenuItem_Click(sender As System.Object, e As System.EventArgs) Handles ExitToolStripMenuItem.Click
Me.Close()
End Sub
To close all forms and exit application call
Application.Exit()
From your main form.
Note: Calling Me.Close() from your main form will usually close all forms including itself and shutdown app as well. If this is not what you want to do you will need to call Me.Hide() on your main form instead.
Does this help?