Re: File Transfer Queue Architecture - C# Discussion Boards

So are you saying that there's no benefit in sending multiple files at once? If so, what's the right way to do it?

Thanks

If it's not broken, fix it until it is.
Everything makes sense in someone's mind.
Ya can't fix stupid.

No, I'm saying that you're not going to get the boost in speed you think you're going to get.

A guide to posting questions on CodeProject

Click this: Asking questions is a skill.
Seriously, do it.

Dave Kreskowiak

So it sounds like it's best to just send one file at a time?

If it's not broken, fix it until it is.
Everything makes sense in someone's mind.
Ya can't fix stupid.

(A cry in the wilderness)

What?

If it's not broken, fix it until it is.
Everything makes sense in someone's mind.
Ya can't fix stupid.

Exactly!

It's the easiest to implement.

If you're trying to get someone to make the decision for you, it's not going to happen. It's up to you to evaluate how much work this is going to be for you and whether or not it's going to be worth it.

A guide to posting questions on CodeProject

Click this: Asking questions is a skill.
Seriously, do it.

Dave Kreskowiak

I'd be inclined to simplify that.

Start with a general-purpose async throttle, inspired by code from Stephen Toub's blog[^]:

C#

public sealed class Throttle
{
    private readonly Queue<TaskCompletionSource<IDisposable>> _waiters = new Queue<TaskCompletionSource<IDisposable>>();
    private readonly WaitCallback _releaseCoreCallback;
    private readonly Task<IDisposable> _releaserTask;
    private readonly Releaser _releaser;
    private readonly int _maxCount;
    private int _currentCount;

    public Throttle(int initialCount)
    {
        if (initialCount <= 0) throw new ArgumentOutOfRangeException();

        _maxCount = _currentCount = initialCount;

        _releaser = new Releaser(this);
        _releaserTask = Task.FromResult((IDisposable)_releaser);
        _releaseCoreCallback = ReleaseCore;
    }

    public Task<IDisposable> WaitAsync()
    {
        lock (_waiters)
        {
            if (_currentCount > 0)
            {
                // We have a spare slot - let the caller continue immediately:
                _currentCount--;
                return _releaserTask;
            }
            
            // The caller needs to wait for a slot to be released:
            var waiter = new TaskCompletionSource<IDisposable>();
            _waiters.Enqueue(waiter);
            return waiter.Task;
        }
    }

    private void Release()
    {
        TaskCompletionSource<IDisposable> toRelease = null;
        lock (_waiters)
        {
            if (_waiters.Count > 0)
            {
                // Another thread is waiting - let it continue now:
                toRelease = _waiters.Dequeue();
            }
            else if (_currentCount < _maxCount)
            {
                // Release the slot back to the pool:
                _currentCount++;
            }
            else
            {
                // Someone called "Dispose" twice:
                throw new SemaphoreFullException();
            }
        }
        
        if (toRelease != null)
        {
            // Release the next waiter on a background thread
            // to avoid a possible StackOverflowException:
            ThreadPool.QueueUserWorkItem(_releaseCoreCallback, toRelease);
        }
    }

    private void ReleaseCore(object state)
    {
        ((TaskCompletionSource<IDisposable>)state).SetResult(_releaser);
    }

    private sealed class Releaser : IDisposable
    {
        private readonly Throttle _throttle;

        public Releaser(Throttle throttle)
        {
            _throttle = throttle;
        }

        public void Dispose()
        {
            _throttle.Release();
        }
    }
}

That gives you a simple way to restrict the number of async methods which can access a resource simultaneously.

Your FTPQueue class then becomes:

C#

public static class FTPQueue
{
    private const int MAX_FILE_UPLOADS = 3;
    private static readonly Throttle UploadThrottle = new Throttle(MAX_FILE_UPLOADS);
    
    public static async Task FileReceived(string fileName)
    {
        Console.WriteLine("Received file {0}", fileName);
        
        using (await UploadThrottle.WaitAsync())
        {
            Console.WriteLine("Uploading file {0}", fileName);
            bool result = await TransferFile(fileName);
            if (result)
            {
                Console.WriteLine("Finished uploading {0}", fileName);
            }
            else
            {
                Console.WriteLine("Failed to upload {0}", fileName);
            }
        }
    }
    
    private static async Task<bool> TransferFile(string fileName)
    {
        await Task.Delay(10000);
        return true;
    }
}

"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer

Interesting.. I'll take a look.

Thanks

If it's not broken, fix it until it is.
Everything makes sense in someone's mind.
Ya can't fix stupid.

Hi All,

I have built a windows forms application using VS2005.

There is a textbox and button in the form. I have created a shortkey [Ctrl+u] on using the shortcut a file browser will open successfully. After selecting the files the text "u" is being printed in the textbox.

Please guide me to resolve this issue.

Thanks in advance.

Thanks
Prabhanjan

Without seeing the relevant fragments of your code? We can't do anything to help you: we can't see your screen, access your HDD, or read your mind - we only get exactly what you type to work with.

Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...

Change the code so that the u is not printed in the textbox.

There are only 10 types of people in the world, those who understand binary and those who don't.

How did you "wire-up" your short-cut key? WPF? Windows Forms? ...

In C#, Asked about the code of the button Decrease Indent and button Increase Indent for RichTextBox ? do you know share with me ?

See indent for RichTextBox - Google Search[^].

Good morning to all.

So about 8 minutes ago I discovered the existence of something called ConfigR, available through nuget. The idea is you write your config files in c# rather than xml which allows for strong typing, this is the compiled at runtime to a real object.

Or something like that.

I'm going to investigate further, but I have an immediate question - what are the security implications of this and does this open the door to script injection like attacks? Anyone had any experience with this thing?

Regards,
Rob Philpott.

According to the project site[^]:

Quote:
you can write any C# you like in your 'configuration file'

That would seem to suggest that the configuration file can be used to inject code into your process.

But if an attacker can modify your config file, then they already have access to your system, and they can probably do a lot worse. Unsure | :~

"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer

I can convert image to binary string from
[HERE]

But This binary string how to convert the that image ?

This does not have a quick answer.

Usually an image is represented with an array of pixels where each item has a a value explaining that pixel for a 256x256 pixel image you'll have an array with size 65536 items. Assuming this is the case here, you still need to know what kind of values are accepted in that images, 8 bits, 12 bits, 24 bits (RGB) or 32 bits (ARGB) eg. Even if you know that you need to know which encoding is used to generate the string and if that string also contains any headers (like a bmp has), exif information etc...

The best thing to do is to try and figure out what encoding is used to generate the string and try to get the bytes back. From there you can work further.

It's hard to give anything more then this without more information, because of the many possibilities.
Try some stuff out and come back if you have more detailed questions.

V.

(MQOTD rules and previous solutions)

Hi All

i'm working on gps tracking system

now i'm working on receiving data from gps devices throw tcp port

using tcpListner class then arrange the data then save it to sql server

i have created a console application to do so then convert it to windows service using NSSM

it is working as expected but after about 30 minutes it consumes all the server memory and CPU

it is only 4 devices connected (Test Phase)

The Code

C#

using System;
using System.Threading;
using System.Net.Sockets;
using System.Text;
using System.Text.RegularExpressions;
using System.Diagnostics;
using System.Data.SqlTypes;
using System.Data.SqlClient;
using System.Data;
using Microsoft.SqlServer.Types;
using System.Linq;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            TcpListener serverSocket = new TcpListener(8889);
            TcpClient clientSocket = default(TcpClient);
            int counter = 0;

            serverSocket.Start();
            //  Console.WriteLine(" >> " + "Server Started");

            counter = 0;
            while (true)
            {
                counter += 1;
                clientSocket = serverSocket.AcceptTcpClient();
                //  Console.WriteLine(" >> " + "Client No:" + Convert.ToString(counter) + " started!");
                handleClinet client = new handleClinet();
                client.startClient(clientSocket, Convert.ToString(counter));
            }

            clientSocket.Close();
            serverSocket.Stop();
            //   Console.WriteLine(" >> " + "exit");
            Console.ReadLine();
        }
    }

    //Class to handle each client request separatly
    public class handleClinet
    {

        static void WriteLog(string message, EventLogEntryType type)
        {

            using (EventLog eventLog = new EventLog("Application"))
            {
                eventLog.Source = "Application";
                eventLog.WriteEntry(message, type, 101, 1);
            }
        }
        static int InsideDangerArea(double Lat, double Lng)
        {

            string point = "POINT(" + Lng + "  " + Lat + ")";

            string ConnStr = "Data Source =.; Initial Catalog = GPS_Tracking;Integrated Security = True";
            using (SqlConnection conn = new SqlConnection(ConnStr))
            {
                conn.Open();
                using (SqlCommand comm = new SqlCommand("Select id from T_Geofncies", conn))
                {
                    DataTable dt = new DataTable();
                    dt.Load(comm.ExecuteReader());

                    foreach (DataRow dr in dt.Rows)
                    {
                        string Query = "  DECLARE @g geometry; DECLARE @h geometry; SET @g = (select(points) from T_Geofncies where id=" + dr["id"].ToString() + " );";
                        Query += " SET @h = geometry::STGeomFromText('" + point + "', 4326); SELECT @g.STContains(@h);";
                        comm.CommandText = Query;
                        int Val = Convert.ToInt32(comm.ExecuteScalar());
                        if (Val == 1)
                        {
                            conn.Close();
                            conn.Dispose();
                            return Convert.ToInt32(dr["id"]);
                        }

                    }
                }
                conn.Close();
                conn.Dispose();
            }
            return 0;
        }

        static int OutsideSafeArea(double Lat, double Lng)
        {

            string point = "POINT(" + Lng + "  " + Lat + ")";

            string ConnStr = "Data Source =.; Initial Catalog = GPS_Tracking;Integrated Security = True";
            using (SqlConnection conn = new SqlConnection(ConnStr))
            {
                conn.Open();
                using (SqlCommand comm = new SqlCommand("Select id from T_SafeArea", conn))
                {
                    DataTable dt = new DataTable();
                    dt.Load(comm.ExecuteReader());

                    foreach (DataRow dr in dt.Rows)
                    {
                        string Query = "  DECLARE @g geometry; DECLARE @h geometry; SET @g = (select(points) from T_SafeArea where id=" + dr["id"].ToString() + " );";
                        Query += " SET @h = geometry::STGeomFromText('" + point + "', 4326); SELECT @g.STContains(@h);";
                        comm.CommandText = Query;
                        int Val = Convert.ToInt32(comm.ExecuteScalar());
                        if (Val == 1)
                        {
                            conn.Close();
                            conn.Dispose();
                            return Convert.ToInt32(dr["id"]);
                        }

                    }
                }
                conn.Close();
                conn.Dispose();
            }
            return 0;
        }
        static SqlGeography GetGeographyFromText(String pText)
        {
            SqlString ss = new SqlString(pText);
            SqlChars sc = new SqlChars(ss);
            try
            {
                return SqlGeography.STPointFromText(sc, 4326);
            }
            catch (Exception ex)
            {
                throw ex;
            }
        }
        TcpClient clientSocket;
        string clNo;
        public void startClient(TcpClient inClientSocket, string clineNo)
        {
            this.clientSocket = inClientSocket;
            this.clNo = clineNo;
            Thread ctThread = new Thread(doChat);
            ctThread.Start();
        }
        private void doChat()
        {
            string ConnStr = "Data Source =.; Initial Catalog = GPS_Tracking;Integrated Security = True";

            int requestCount = 0;
            // byte[] bytesFrom = new byte[10025];
            string dataFromClient = null;
            Byte[] sendBytes = null;
            string serverResponse = null;
            string rCount = null;
            requestCount = 0;

            while ((true))
            {
                try
                {
                    requestCount = requestCount + 1;
                    NetworkStream networkStream = clientSocket.GetStream();
                    int i;
                    int size = (int)clientSocket.ReceiveBufferSize;
                    // Loop to receive all the data sent by the client.
                    Byte[] bytes = new Byte[size];
                    string data = "";
                    string IMEI;
                    while ((i = networkStream.Read(bytes, 0, bytes.Length)) != 0)
                    {

                        try
                        {
                            data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);

                            string[] tokens = data.Split(new[] { "GPRMC" }, StringSplitOptions.None);
                            var longest = Regex.Matches(data, @"\d+").Cast().OrderByDescending(m => m.Length).First();


                            IMEI = longest.ToString();
                            if (IMEI.Length > 15)
                                IMEI = IMEI.Substring(1);



                            foreach (string item in tokens)
                            {
                                try
                                {
                                    string[] Values = item.Split(',');                                   // Console.WriteLine("Received: {0}", data);
                                    string time = Values[1];
                                    // Console.WriteLine("Time= " + time);
                                    string lat;
                                    string lng;
                                    string speed;
                                    string date;

                                    lat = Values[3];
                                    lng = Values[5];
                                    speed = Values[7];
                                    date = Values[9];

                                    string NewDString = date.Substring(2, 2) + date.Substring(0, 2) + date.Substring(4, 2);

                                    //  Console.WriteLine("IMEI= " + IMEI);

                                    // Alternate choice: If the string has been input by an end user, you might  
                                    // want to format it according to the current culture: 
                                    // IFormatProvider culture = System.Threading.Thread.CurrentThread.CurrentCulture;
                                    string myDate = (NewDString + time).Insert(2, "-").Insert(5, "-").Insert(8, " ").Insert(11, ":").Insert(14, ":");


                                    double latDeg = Convert.ToDouble(Convert.ToDouble(lat).ToString().Substring(0, 2));
                                    double latMin = Convert.ToDouble(Convert.ToDouble(lat).ToString().Substring(2));

                                    double lngDeg = Convert.ToDouble(Convert.ToDouble(lng).ToString().Substring(0, 2));
                                    double lngmin = Convert.ToDouble(Convert.ToDouble(lng).ToString().Substring(2));

                                    double latmap = latDeg + (latMin / 60);
                                    //  OldLat=
                                    double lngmap = lngDeg + (lngmin / 60);
                                    //if ((Math.Round(latmap, 3) != Math.Round(OldLat, 3) && Math.Round(lngmap, 3) != Math.Round(OldLng, 3)) || idleRecord > 30)
                                    //{
                                    using (SqlConnection conn = new SqlConnection(ConnStr))
                                    {
                                        conn.Open();
                                        using (SqlCommand cmd = conn.CreateCommand())
                                        {
                                            // DbCommand also implements IDisposable

                                            // create command with placeholders
                                            cmd.CommandText =
                                               "INSERT INTO T_Tracking " +
                                               "([IMEI], [TrackTime],  [Longitude], [Lattitude],  [speed],[MapPoint],[SafeAreaID],[GeoFenceID]) " +
                                               "VALUES(@IMEI, @TrackTime,  @Longitude, @Lattitude,  @speed,@MapPoint,@SafeAreaID,@GeoFenceID)";


                                            SqlParameter p_IMEI = new SqlParameter("@IMEI", IMEI);
                                            cmd.Parameters.Add(p_IMEI);

                                            SqlParameter p_TrackTime = new SqlParameter("@TrackTime", myDate);
                                            cmd.Parameters.Add(p_TrackTime);

                                            SqlParameter p_Longitude = new SqlParameter("@Longitude", lngmap);
                                            cmd.Parameters.Add(p_Longitude);

                                            SqlParameter p_Lattitude = new SqlParameter("@Lattitude", latmap);
                                            cmd.Parameters.Add(p_Lattitude);

                                            SqlParameter p_Speed = new SqlParameter("@speed", speed);
                                            cmd.Parameters.Add(p_Speed);
                                            SqlParameter p_Points = new SqlParameter("@MapPoint", System.Data.SqlDbType.Udt);
                                            p_Points.UdtTypeName = "geometry";
                                            p_Points.Value = GetGeographyFromText("Point(" + lngmap + "  " + latmap + ") ");
                                            cmd.Parameters.Add(p_Points);
                                            SqlParameter P_Safe = new SqlParameter("@SafeAreaID", OutsideSafeArea(latmap, lngmap));
                                            cmd.Parameters.Add(P_Safe);
                                            SqlParameter P_GeoFence = new SqlParameter("@GeoFenceID", InsideDangerArea(latmap, lngmap));
                                            cmd.Parameters.Add(P_GeoFence);
                                            // execute
                                            cmd.ExecuteNonQuery();

                                        }
                                        //}
                                        //else
                                        //    idleRecord = idleRecord + 1;

                                    }
                                }
                                catch (Exception exp) { WriteLog(exp.ToString(), EventLogEntryType.Error); }
                            }

                            
                        }
                        catch { }
                    }
                  
                }
                catch (Exception ex)
                {
                    // Console.WriteLine(" >> " + ex.ToString());
                }
            }
        }
    }
}

C#

what optimization can be done ?

Imagine on live version it will be about 1000 device

Thank you for the help

There's a bit much here to look at so I haven't completely gone through the logic, but it seems that you're making clients on new threads and those clients never kill themselves so they're basically leaked.

i guess you are right
please check that code part

C#

while (true)
            {
                counter += 1;
                clientSocket = serverSocket.AcceptTcpClient();
               
                handleClinet client = new handleClinet();
                client.startClient(clientSocket, Convert.ToString(counter));
            }
 
            clientSocket.Close();
            serverSocket.Stop();

it will never go to the close part

I don't see you close the connection or the NetworkStream. So, it may be one of the issues.

i'm using

C#

using (SqlConnection conn = new SqlConnection(ConnStr))
           {
               conn.Open();
               using (SqlCommand comm = new SqlCommand("Select id from T_SafeArea", conn))
               {



               }
               conn.Close();
               conn.Dispose();
           }

you are right about closing the network stream and the clients
could you point out where in the code to do so