What you need is a "state" class you can pass along. This state class can hold a reference to the EndPoint passed to the UdpClient. I'm not sure how well the example holds up with BeginReceive (especially with calling it again to continue listening) as I've only really used BeginReceiveFrom - which works similar.
Something like this:
Private Class UdpState
Public SendingSocket As UdpClient
Public ReceivingEndpoint As EndPoint
Then you declare and start your receive like this:
Private Sub Form1_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
Dim ep As New IPEndPoint(IPAddress.Any, 5070)
Dim u As New UdpClient(ep)
Dim s As New UdpState()
s.SendingSocket = u
s.ReceivingEndpoint = ep
u.BeginReceive(New AsyncCallback(AddressOf Packet_Received), s)
And finally in the callback to get the ip address
Private Sub Packet_Received(ByVal ar As IAsyncResult)
Dim s As UdpState = CType(ar.AsyncState, UdpState)
Dim IP_Remote_End As IPEndPoint = TryCast(s.ReceivingEndpoint, IPEndPoint)
I pulled this example from the MSDN page for the BeginReceive method (http://msdn.microsoft.com/en-us/library/system.net.sockets.udpclient.beginreceive.aspx) which happened to closely correspond to my own code for the BeginReceiveFrom method.
Hope that helps.