Why not try a little search with the keywords 'how to find prime numbers java'.
To make it easy to delete values form any position in an array, you should look into using dynamic arrays such as linked lists.
If this is school work, you might need to create your own:
Linked Lists[
^]
You can aslo look into Java collections:
Java Collections Tutorial[
^]