how to avoid asp.net application open same browser in different tabs in same system

In general, this is not possible – you can't tell whether the user has closed a previous tab or not. If the site is behind a login, you can invalidate previous sessions by the same user on login, so they can't operate it in two different browsers at once, though you should think extremely carefully before doing this as it is generally a big annoyance for the user (if they want to open two sessions there is probably a good reason for that).

For multi-step transactions you should store the transient information in the page, or in a database entry keyed by something stored in the page, not in the session. You can protect such transactions from being double-entered by checking if there are any already open, but if you do this you must be extremely careful with cleanup since you won't get a notification if the user's browser closes and if you don't clean up past transaction entries they will never be able to start a new transaction. Again, you should not enforce only one transaction at a time unless there is a very good reason; if the user wants to do two things at once you should let him.