Click here to Skip to main content
15,884,176 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
Hi, all!
I am writing proxy server on asynchronous sockets for educational purposes and stuck at the point, that my browser treat some web-sites' response different way if i don't use the proxy, for some of them i don't see response at all, though i see in console, that request was sent and response was received.
Here is the code of server.
Could you advise, what is wrong and what i can do to improve server?

Quote:
C#
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;

namespace AsyncTcpServer
{
    internal class AsyncTcpServer
    {
        private bool syncmode;
        private Socket listenSocket;
        private const int port_ = 9999;
        private const int bufferSendSize = 8192;
        private const int bufferSize = 65536;//1048576;
        private LingerOption lo;
        private AutoResetEvent autoEvent;

        private static void Main(string[] args)
        {
            // Create the socket which listens for incoming connections.
            var server = new AsyncTcpServer();
            ThreadPool.SetMinThreads(100, 100);
            ThreadPool.SetMaxThreads(3000, 3000);
            server.StartServer();
            Console.WriteLine("Press smth to stop server");
            Console.Read();
        }

        private void StartServer()
        {
            autoEvent = new AutoResetEvent(true);
            var addressList = Dns.GetHostEntry(Environment.MachineName).AddressList;
            foreach (var ipAddress in addressList)
            {
                Console.WriteLine(ipAddress);
            }
            int choice;
            Console.WriteLine("Choose ip");
            int.TryParse(Console.ReadLine(), out choice);
            Console.WriteLine("Choose sync mode or not");
            bool.TryParse(Console.ReadLine(), out syncmode);
            var localEndPoint = new IPEndPoint(addressList[choice], port_);

            // Get host related information.
            listenSocket = new Socket(localEndPoint.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
            lo = new LingerOption(false, 0);
            listenSocket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.Linger, lo);
            // Get endpoint for the listener.
            if (localEndPoint.AddressFamily == AddressFamily.InterNetworkV6)
            {
                // Set dual-mode (IPv4 & IPv6) for the socket listener.
                // 27 is equivalent to IPV6_V6ONLY socket option in the winsock snippet below,
                // based on http://blogs.msdn.com/wndp/archive/2006/10/24/creating-ip-agnostic-applications-part-2-dual-mode-sockets.aspx
                listenSocket.SetSocketOption(SocketOptionLevel.IPv6, (SocketOptionName)27, false);
                listenSocket.Bind(new IPEndPoint(IPAddress.IPv6Any, localEndPoint.Port));
            }
            else
            {
                // Associate the socket with the local endpoint.
                Console.WriteLine(localEndPoint);
                listenSocket.Bind(localEndPoint);
            }

            // Start the server.
            listenSocket.Listen(500);
            if (syncmode)
            {
                listenSocket.BeginAccept(EndAccept, listenSocket);
            }
            else
            {
                for (int i = 0; i < 200; i++)
                {
                    listenSocket.BeginAccept(EndAccept, listenSocket);
                } 
            }
        }

        private void EndAccept(IAsyncResult result)
        {
            if (syncmode)
            {
                autoEvent.WaitOne();    
            }
            
            Console.WriteLine("ExecuteRequest");
            var server = result.AsyncState as Socket;
            if (server != null)
            {
                try
                {
                    var myClient = server.EndAccept(result);
                    myClient.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.Linger, lo);
                    var token = new Token(myClient, bufferSendSize);

                    myClient.BeginReceive(token.Buffer, 0, token.Buffer.Length, SocketFlags.None, EndReceive, token);
                }
                catch (SocketException e)
                {
                    Console.WriteLine(e.Message);
                    autoEvent.Set();
                    listenSocket.BeginAccept(EndAccept, listenSocket);
                }

            }
            else
            {
                StartServer();
            }
        }

        private void EndReceive(IAsyncResult result)
        {
            var token = result.AsyncState as Token;
            if (token == null) return;
            
            var myClient = token.Connection;
            var length = myClient.EndReceive(result);

            Console.WriteLine("Process request from {0}", myClient.RemoteEndPoint as IPEndPoint);
            byte[] httpRequest = new byte[length];
            Array.Copy(token.Buffer, httpRequest, length);
            token.Buffer = httpRequest;
            var endPoint = ProcessReceive(token);
            if (endPoint == null)
            {
                return;
            }
            // connecting to remote socket
            var rerouting = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
            {
                SendBufferSize = bufferSendSize,
                ReceiveBufferSize = bufferSize
            };
            rerouting.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.Linger, lo);
            token.RemoteConnection = rerouting;
            rerouting.BeginConnect(endPoint, EndConnect, token);       
        }

        private IPEndPoint ProcessReceive(Token token)
        {
            // searching host and port
            var client = token.Connection;
            var regex = new Regex(@"Host: (((?<host>.+?):(?<port>\d+?))|(?<host>.+?))\s+", RegexOptions.Multiline | RegexOptions.IgnoreCase);
            var match = regex.Match(System.Text.Encoding.UTF8.GetString(token.Buffer));
            string host = match.Groups["host"].Value;
            var endPoint = client.RemoteEndPoint as IPEndPoint;
            if (host.Equals(string.Empty))
            {
                listenSocket.BeginAccept(EndAccept, listenSocket);
                return null;
            }
            Console.WriteLine("requesting {0} from {1}", host, endPoint);

            int port = 0;
            // use 80 port as default
            if (!int.TryParse(match.Groups["port"].Value, out port))
            {
                port = 80;
            }

            // receive ip from host
            const string ipPattern = @"^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$";
            bool isIp = Regex.IsMatch(host, ipPattern);
            
            // creating access point
            IPAddress ip;
            if (isIp)
            {
                ip = IPAddress.Parse(host);
            }
            else
            {
                IPHostEntry hostEntry = Dns.GetHostEntry(host);
                ip = hostEntry.AddressList[hostEntry.AddressList.Count() - 1];
            }
            return new IPEndPoint(ip, port);
        }

        private void EndConnect(IAsyncResult result)
        {
            var token = result.AsyncState as Token;
            if (token == null) return;
            try
            {
                var rerouting = token.RemoteConnection;
                rerouting.EndConnect(result);
                Console.WriteLine("Connected to {0}", token.RemoteConnection.RemoteEndPoint as IPEndPoint);
                rerouting.BeginSend(token.Buffer, 0, token.Buffer.Length, SocketFlags.None, EndSend, token);          
            }
            catch (SocketException e)
            {
                token.Dispose();
                Console.WriteLine(e.Message);
                autoEvent.Set();
                listenSocket.BeginAccept(EndAccept, listenSocket);
            }
            
        }

        private void EndSend(IAsyncResult result)
        {
            var token = result.AsyncState as Token;
            if (token == null) return;
            try
            {
                var length = token.RemoteConnection.EndSend(result);
                Console.WriteLine("Sent{1} bytes to {0}", token.RemoteConnection.RemoteEndPoint as IPEndPoint, length);
                var newtoken = new Token(token, bufferSize);
                token.RemoteConnection.BeginReceive(newtoken.Buffer, 0, newtoken.Buffer.Length, SocketFlags.None, EndReceiveRemote, newtoken);
            }
            catch (SocketException e)
            {
                token.Dispose();
                Console.WriteLine(e.Message);
                autoEvent.Set();
                listenSocket.BeginAccept(EndAccept, listenSocket);
            }
        }

        private void EndReceiveRemote(IAsyncResult result)
        {
            var token = result.AsyncState as Token;
            if (token == null) return;
            try
            {
                var length = token.RemoteConnection.EndReceive(result);
                Console.WriteLine("Received {1} bytes from {0}", token.RemoteConnection.RemoteEndPoint as IPEndPoint, length);
                var httpResponse = token.Buffer;
                if (httpResponse == null || httpResponse.Length <= 0) return;
                if (!syncmode)
                {
                    autoEvent.WaitOne();   
                }
                token.Connection.BeginSend(token.Buffer, 0, length, SocketFlags.None, EndSendToClient, token); 
            }
            catch (SocketException e)
            {
                token.Dispose();
                autoEvent.Set();
                Console.WriteLine(e.Message);
                listenSocket.BeginAccept(EndAccept, listenSocket);
            }
        }

        private void EndSendToClient(IAsyncResult result)
        {
            var token = result.AsyncState as Token;
            if (token == null) return;
            var length = token.Connection.EndSend(result);
            Console.WriteLine("Sent{1} bytes to {0}", token.Connection.RemoteEndPoint as IPEndPoint, length);
            token.Dispose();
            autoEvent.Set();
            listenSocket.BeginAccept(EndAccept, listenSocket);
        }
    }
}

Posted
Comments
Sergey Alexandrovich Kryukov 4-Feb-14 17:19pm    
First of all, why do you think any server could ever be asynchronous? I think it's rather should be multithreaded...
—SA
[no name] 5-Feb-14 2:36am    
I've read some articles where it was described, that asynchronous approach works faster, only that.
Also, I've noticed that in debug mode it works more correctly.
Sergey Alexandrovich Kryukov 5-Feb-14 2:38am    
I cannot agree with that.
—SA
[no name] 5-Feb-14 4:41am    
Thanks for response. I have a few more questions.
Please, advise:
1. Is it a right way to left request accepting logic asynchronous to avoid "while(true)" constructions and make onAccepted logic such as Threadpool.Queueuserworkitem?
2. Shall i use anything like AutoResetEvent/Mutex to lock listening socket while responding to client?
Sergey Alexandrovich Kryukov 5-Feb-14 11:28am    
1) I don't seriously consider asynchronous communications.
2) Why? If you can tell me a reason, we can discuss it. They two are very different. You only need lock. You see, I suggested to have two threads and talk to the clients in a queue. If a new client connected, it adds its remote socket to the queue. Hence, the queue is a shared object. And the shared object should... what...? Hence, mutual exclusion. But why Mutex if you have lock?
—SA

1 solution

The discussion in comments looks going well beyond the format of Quick Questions & Answers. You really need to design the server and client parts thoughtfully from the very beginning; so focusing on just your questions may not be so helpful. For some very relevant ideas, please see my past answers:
an amateur question in socket programming[^],
Multple clients from same port Number[^].

Please think about it and then ask follow-up questions if you have any.

I don't see if you are read to embrace throwing out the idea of asynchronous communications. It should be done asynchronous by using thread, each of those should use synchronous, blocking network operations. Why? To me, this is quite clear. I shared some explanations in my past answers, please read them:
Asynchronous and Concurrent WCF service Calls[^],
TcpListener, TcpClient, Ping in Visual Basic 2005[^],
problem in multithreading ? !!![^].

—SA
 
Share this answer
 
Comments
[no name] 5-Feb-14 18:05pm    
Thank you a lot! Plenty of useful information.
I understood the point of not using async API. I'm fine with that, though may be in some cases it is the simplest way.

If you don't mind, let's get back to my latest comment in comments above.
Basicly, i need http proxy server only, my client would be any web-browser, that has option to set proxy server and i think, that tcp level is certainly what i need since i need to transfer request exactly from client to requested resource and response back.
The remaining architecture question for me - is a time gain in using concurrent threads for chains "send data to remote server - receive response from remote server" noticeable?

I mean technical implementation of such example:
let's consider simple example and consider that receiving client request is small(1 unit) operation, receiving http response from remote resource - large(2 unit)
so we have
using main thread and one more
time to receive from client 1 unit
time to send to remote server 1
time to receive from remote server 2
time to send to client 2

time to receive from client 1
time to send to remote server 1
time to receive from remote server 2
time to send to client 2

time to receive from client 1
time to send to remote server 1
time to receive from remote server 2
time to send to client 2

total 18

when performing operations with remote server in concurrent threads as soon, as data from clients is available
we get this

1 received first request body
1 1 received second request body + sent first request to remote server
1 1 received third request body + sent second request to remote server
1 2 sent third request to remote server + received first response
2 received second response
2 2 received third response + sent first response to client
working
2 sent second response to client
working
2 sent third response to clien

total time - 10
Sergey Alexandrovich Kryukov 5-Feb-14 19:40pm    
You are welcome.
To get another step, I would need to step back and understand the purpose of this "proxy server for educational purposes". What should be its ultimate purpose? How do you see the topmost level of architecture? It's too early to discuss timing.

Fundamentally, the biggest concern is: the HTTP is predominantly client-server. Everything happens on HTTP request, so HTTP response should be short and really synchronous. Also, HTTP is stateless. All this limits the functionality in draconian way (for now, let's forget about advanced Web technology like server push, such as implemented by SignalR). To me, it tells me that if you need some asynchronous behavior, you would need a separate tier, which would be "stateful" and essentially multithreaded. As I understand, you are not thinking in this direction and hope to stay with just one Web proxy, but such proxy, from the user's standpoint, would behave exactly as an HTTP server and implement HTTP protocol. Or didn't I understand you correctly? You just cannot get such asynchronous behavior using just the Wen client, due to limitations of HTTP protocol. The problem is not "time gain", as it is understood in performance-related issues. This is not performance, this is the architecture related to scenarios...

—SA
[no name] 6-Feb-14 5:10am    
The ultimate purpose of this server is some simple optimization of content for mobile devices (like image сompression, cutting off/minifying css and javascript, force gzip encoding etc.). But first of all it should work stable like a simple web-proxy for any client. Educational purpose only means, that it will not have any production load (just my university graduation project), nevertheless, i want to complete it a proper way.

Yes, i hope to stay with just one web proxy and from user's standpoint it would behave exactly like a HTTP server and implement HTTP protocol.
I've tried using HTTP listener and web requests for this purpose, but it gives some strange results(for example using no proxy some resources like tut.by redirect my mobile to m.tut.by, but if i use my proxy it gives no redirection, though i pass to web request the whole lot of information i get in http listener from my mobile browser).
Sergey Alexandrovich Kryukov 6-Feb-14 11:02am    
Maybe you over-complicate this problem then. Is looks like you can process requests in the same thread on the fly; the attempt to implement asynchronous behavior (even based on thread) may not be needed. Needs some thinking...
—SA
[no name] 6-Feb-14 11:38am    
Maybe. But on-the-fly processing of requests in the same thread is not a good choice, because it significantly slows the workflow of browser page loading.
When i run proxy server locally, i expect not to feel much difference in time of resource loading with and without proxy, e.g. if some page is loaded literally in no time without proxy, and with proxy i see loading for 15 sec - that means that something is strongly wrong.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900