You are not using the listbox in a correct way.
You can do the following:
Let the Listbox reflect the state of your model.
In that case your model class should have a Selected property.
look at:
Basic-MVVM-Listbox-Binding-In-WPF[
^]
or
Make a usercontrol.
The usercontrol holds your data and the Xaml.
Add the usercontrol to the listboxItems.
then you can use
private void ActivateGroupButton_Click(object sender, RoutedEventArgs e)
{
Usercontrol(((Button)Sender).Parent) to have access to your data and controls.
}