Rollback multiple row

Start a transaction before you do any of the inserts. See MySQL :: MySQL 5.7 Reference Manual :: 13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Syntax[^]

Wrap the entire set of inserts within a Try-Catch block and include a Rollback in the Catch block.

E.g.

Dim cmd As New MySqlCommand()
Dim trn As MySqlTransaction

trn = yourConnection.BeginTransaction(IsolationLevel.ReadCommitted)
cmd.Transaction = trn

Try
   cmd.CommandText = "insert into table (A) values (A)"
   cmd.ExecuteNonQuery()
   cmd.CommandText = "insert into tableB (B) values (B)"
   cmd.ExecuteNonQuery()
   For value As Integer = 0 To 5   
      cmd.CommandText = "insert into tableC (C) values (value)"
      cmd.ExecuteNonQuery()
   NEXT

   trn.Commit()  'all values will have been inserted

Catch e As Exception

   trn.Rollback() 'no values will have been inserted

End Try

NOTE: I haven't included any code for the connection, nor proper handling of the error once the rollback has occurred (you might want to let your user know about the error).
This code is untested and is only intended to show you the principle - there may be minor errors.

You also tagged your question with storedproc so if you want to do this all within a MySQL stored procedure instead of VB have a look at the solution from wchiquito on this link[^]