To Change App Config
Configuration config = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None);
config.AppSettings.Settings["test"].Value = "blah";
but In your case as you are using Add-In I think Save JSON/XML in AppData would be more applicable and fissile.
How to do it...
get AppData Folder
var folderName = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData),"OutlookAddin");
Create folder if don't exists now create your desire XML File or JSON file there and Read and write. I would suggest you to use serialization to create or modify those configuration details at AppData folder.
How to Serialize Class: Object Serialization using C#
using JSON : JSON Serialization and Deserialization in ASP.NET
] ( Just watch how to serialize a .Net class to JSON)
I was busy with some deployment at my end that's why I did not reply your answer sorry for delay.
If you need any further clarification please feel free to ask.