tell setup.exe if its first run or not

One way is to check if there is any registry parameter present which is created by the setup file during its first run.

for example:
Let say your setup.exe runs for the first time. It check if registry parameter "HKLM\Software\MySoftware\Installed" is present or not. If not present then it is its first run and setup.exe creates this registry parameter as a last step of installation. Now when setup.exe runs for the second time it checks this registry parameter, it will be present as setup.exe has created this registry parameter during its first run.