Two things:
1) The stored character needs the right font in order to display it correctly.
Depending on how you are displaying it, unless you use the correct font you will probably get a question mark instead.
2) Try sending it as a NVARCHAR string instead of a VARCHAR:
Dim Insert_Symbol As New SqlCommand("Insert into Currency.dbo.Currency_Symbol (Currency_Symbol) values (N'" & CountryCurrency_Symbol & "')", SQLCon)
may help. Or better, pass it as a parameter via a parameterized query instead and let the system sort it out. Concatenating strings to form SQL commands is not a safe way to do things: it leaves you wide open to accidental or deliberate SQL Injection attack which can destroy your entire database. Always use Parameterized queries instead.
When you concatenate strings, you cause problems because SQL receives commands like:
SELECT * FROM MyTable WHERE StreetAddress = 'Baker's Wood'
The quote the user added terminates the string as far as SQL is concerned and you get problems. But it could be worse. If I come along and type this instead: "x';DROP TABLE MyTable;--" Then SQL receives a very different command:
SELECT * FROM MyTable WHERE StreetAddress = 'x';DROP TABLE MyTable;
Which SQL sees as three separate commands:
SELECT * FROM MyTable WHERE StreetAddress = 'x';
A perfectly valid SELECT
DROP TABLE MyTable;
A perfectly valid "delete the table" command
And everything else is a comment.
So it does: selects any matching rows, deletes the table from the DB, and ignores anything else.
So ALWAYS use parameterized queries! Or be prepared to restore your DB from backup frequently. You do take backups regularly, don't you?