You can use the
destoywi[
^] command to permanently remove any Work Item from TFS. I have the below code snippet in a bat file that I use when required.
@ECHO OFF
ECHO Please type the IDs :
set /p id=
cd c:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE
witadmin destroywi /collection:"http://<your tfs="" server="">:8080/tfs/<your tfs="" collection="" name="">" /noprompt /id:%id%
PAUSE</your></your>