Re: Is streamreader slow compared to other text import functions? - C# Discussion Boards

Re: Is streamreader slow compared to other text import functions?

3-May-13 3:42

I'm building a vcf editor and there are approximately 4k contacts in the particular file I'm using.

I've gotten to contact 1000 in a little over 10 minutes, so I'm wondering what a faster way to go about parsing these would be?

The file is only 1.2mb and the function is below. Maybe the slow speed is in the gui update and not stream reader, not sure?

using (StreamReader reader = new StreamReader(filePath))
                {
                    List<string> vcfFile = new List<string>();
                    string readLine;
                    while ((readLine = reader.ReadLine()) != null)
                    {
                        vcfFile.Add(readLine); // Add to list.
                    }
                    int count = 0;
                    foreach (string line in vcfFile)
                    {
                        if (line == "BEGIN:VCARD")
                        {
                            count = count + 1;
                            tbxOutput.Text += (count + ".) ").ToString();
                        }
                        tbxOutput.Text += line + System.Environment.NewLine;
                        if (line == "END:VCARD")
                        {
                            tbxOutput.Text += System.Environment.NewLine + System.Environment.NewLine;
                        }
                        Application.DoEvents();
                    }
                }

GuyThiebaut3-May-13 4:21

GuyThiebaut

3-May-13 4:21

I take it the UI is freezing while this runs in the foreground?

If you are not runnning this as a separate thread, run it as a background worker and update the textbox via a delegate.
Also get rid of the Application.DoEvents(); too as it is considered to be bad practise-> clickety[^].

“That which can be asserted without evidence, can be dismissed without evidence.”

― Christopher Hitchens

Re: Is streamreader slow compared to other text import functions?

turbosupramk33-May-13 4:51

Re: Is streamreader slow compared to other text import functions?

3-May-13 4:51

Thanks for the reply.

I temporarily used the application.doevents() because I couldn't tell if it was doing anything.

I have since thrown this in, but it is still very very slow, especially after it gets past the first few hundred entries. The initial few hundred entries are at a speed I would expect.

I will try the workthread as well.

if (count % 50 == 0)
                        {
                            Application.DoEvents();
                            tbxOutput.SelectionStart = tbxOutput.Text.Length;
                            tbxOutput.ScrollToCaret();
                            tbxOutput.Refresh();
                        }

Pete O'Hanlon3-May-13 4:24

Pete O'Hanlon

3-May-13 4:24

Part of your slow down is in the continual appending to tbxOutput.Text. Rather than using a TextBox, use a ListBox instead. Alternatively, read it all into a StringBuilder and blast it out in one go with a ToString(). And yes, move this into a background thread.

I was brought up to respect my elders. I don't respect many people nowadays.

CodeStash - Online Snippet Management | My blog | MoXAML PowerToys | Mole 2010 - debugging made easier

Re: Is streamreader slow compared to other text import functions?

turbosupramk33-May-13 5:13

Re: Is streamreader slow compared to other text import functions?

3-May-13 5:13

Great idea, that really sped things up to about 20 seconds!

turbosupramk33-May-13 5:14

Re: Is streamreader slow compared to other text import functions?

3-May-13 5:14

This code is very quick in case someone else can learn from my mistake, thank you both for helping me through this.

private void vcfProcessor(object filePathObj)
        {
            try
            {
                string filePath = filePathObj.ToString();
                using (StreamReader reader = new StreamReader(filePath))
                {
                    List<string> vcfFile = new List<string>();
                    string readLine;
                    while ((readLine = reader.ReadLine()) != null)
                    {
                        vcfFile.Add(readLine);
                    }
                    int count = 0;
                    foreach (string line in vcfFile)
                    {
                        if (line == "BEGIN:VCARD")
                        {
                            count = count + 1;
                            lbxOutput.Items.Add(count + ".) ").ToString();
                        }

                        lbxOutput.Items.Add(line + System.Environment.NewLine);

                        if (line == "END:VCARD")
                        {
                            lbxOutput.Items.Add(System.Environment.NewLine);
                        }
                        if (count % 50 == 0)
                        {
                            tbxContactNumber.Text = count.ToString();
                            Application.DoEvents();
                        }
                    }
                    tbxContactNumber.Text = count.ToString();
                }


            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message + System.Environment.NewLine + ex.Source + System.Environment.NewLine + ex.TargetSite + System.Environment.NewLine + ex.StackTrace + System.Environment.NewLine + System.Environment.NewLine);
            }
        }

DaveyM693-May-13 13:47

DaveyM69

3-May-13 13:47

As others have said, perform this function on a background thread, update your controls using Invoke/BeginInvoke and get rid of the DoEvents

Dave

Binging is like googling, it just feels dirtier.
Please take your VB.NET out of our nice case sensitive forum.
Astonish us. Be exceptional. (Pete O'Hanlon)

BTW, in software, hope and pray is not a viable strategy. (Luc Pattyn)

Event/Delegate for progress bar issue

MichCl3-May-13 3:13

3-May-13 3:13

I'm trying to add an interface, but it's causing havoc with my ProgressBar update events (My progress bar event is null so my progress bar never gets updated). I have the following older link about it: [^]

The progress bar works where it doesn't use events in Generic, but it's not working where I'm using the events in CR5_new and CR5_Comm. As you can see below, when I get to RaiseProgressChange in my CR5_new class, the event is null. I can't figure out why it's null. I'm not very good with events, and throwing in the interface (the iCR_Comm interface is new) doesn't help, since I'm not very good at interfaces either. I removed unrelated code, so hopefully this post isn't missing important definitions. I realize that this doesn't look familiar to you and is a lot of info, but any help would be appreciated. If anyone is mean, I'm not going to upvote that comment. There are very specific project related reasons that these are all separate vs projects. (re-use, multiple CRx's, multiple CRx_comm's, and numerous dynamic data classes that give extra control info to the Generic class.) Everything works (programming, write, read, through CR5_Comm class), so don't focus on that. It's just the events that update the progress bar that don't work. Names of classes have been changed enough that they don't give away our information, but still resemble my code.

My code looks like this now (all separate vs projects):

vs project: crTool
frmUserForm.cs:

using USB_Comm;
namespace CRWriter
{
  public partial class frmUser:Form
  {
  public frmUser(string dllSelected, TemplateHandlerApplication.Templates.TEMPL[] template, string dll, Site.Site.SITE cert0)
  {
         programDll = dllSelected;
         templateArr = template;
         itsDll = dll;
         cert = cert0;
         InitializeComponent();
         CreateSubForms();   //start here
  }
  private void CreateSubForms()
  {
    cb = new CrComm();
    int errValue = cb.FindUsbHid(ref HWndBoxID); //see how many boxes there are
    for (int i = 0; i < cb.cbInfos.Length; i++)
    {
       iCRComm cb1 = null;
        if (controls[i] == null)
        {   
           CRComm_Factory.CRCommFactory factory = new  CRComm_Factory.CRCommFactory();
           int type = factory.DetermineCR_Type(programDll);
           try
           {
              cb1 = factory.GetCRComm(type);//iCrComm cb1 = factory.GetCR(type);
              controls[i] = gen.GetProgramControl(cb1, programDll, templateArr, itsDll, cert, i, cb1.GetBoxID(i), this);
            }
...
         }
      }
  }
 }
}

vs project: Generic
GenericPC.cs

using USB_Comm;
namespace GenericCW
{
  public partial class GenericPC: UserControl
  {
    public GenericPC() {InitializeComponent();}
    public Control GetPC(iCRComm cbInstance, string dllSelected, TemplateHandlerApp.Templates.TEMPL[] template, string dll, Site.Site.SITE cert0, int slaveIndex, int boxID, Form theParent)
    {
           //cr = new CR5_new.CR5(slaveIndex);
           cr = factory.GetCR(type, slaveIndex);
           cr.ProgressChanged += new ProgressChangeHandler2(updateProgressBar);
    }
    private int writeResultCheck(byte []  writeDat, byte [] statusDat, byte [] dataDumpWriteCheck)
   {
        for (int count = 0; ((count < max) && (!done)); count++)
        {
            if(((count % 24) == 0) || (count == max-1))
               updateProgressBar(count, SlaveIndex); //this works
            }
        }
    }
    private void updateProgressBar(int n, int id)
    {
       if (id == SlaveIndex) //make sure it's our progress bar and not the other one
       {
           pb_Progress.BeginInvoke(
              new Action(() =>
              {
                 pb_Progress.Value = n;
              }
           ));
       }
    }
  }
}

vs project: cr5_new
CR5_new.cs:

using USB_Comm;
namespace CR5_new
{
    public class CR5:iCR
    {
        [STAThread]
        static void Main()
        {
        }
        public event ProgressChangeHandler2 ProgressChanged;
        //constructor
        public CR5(int slaveIndex)
        {
           SlaveIndex = slaveIndex;
        }
        public int ProcessTagWrite(ref byte[] WriteDat, bool isFastProgramMode)
        {
           AttemptWrites();
        }
        private int AttemptWrites()
        {
           for (int i = 0; i < (max); i++)
           {
                //update Progress Bar every 28
                if (((i % 28) == 0) || (i == max-1))
                   RaiseProgressChange(i, SlaveIndex);
           }
         }
        private void RaiseProgressChange(int progress, int id)
        {
            if (ProgressChanged != null) //this is null
            {
                ProgressChanged(progress, id);  //not getting here
            }
        }

vs project: icr
iCR.cs:

using USB_Comm;
public interface iCR
{
            event ProgressChangeHandler2 ProgressChanged;
...
}

vs project: icrComm
iCRComm.cs:

public delegate void ProgressChangeHandler2(int progress, int id);
public interface iCRComm
{
    event ProgressChangeHandler2 ProgressChanged;
    int ReadTag(ref byte[] dataDumpWriteCheck);
}

vs project: cr5_comm_new
CR5_Comm.cs:

namespace USB_Comm
{
   public class CrComm:iCRComm
   {
        public event ProgressChangeHandler2 ProgressChanged;

        private void RaiseProgressChange(int progress, int id)
        {
            if (ProgressChanged != null)
            {
                ProgressChanged(progress, id);  //prog bar not updating from here either
            }
        }
        public int ReadTag(ref byte[] byteData, int usbIndex)
        {
            for (int cell = 0; cell < max; cell++)
            {
                //only update progress bar every 28
                if (((cell % 28) == 0) || (cell == max-1))
                {
                    RaiseProgressChange(cell, usbIndex);
                }
            }
         }
    }
  }
}
}

Eddy Vluggen3-May-13 7:21

3-May-13 7:21

public Control GetPC(iCRComm cbInstance, string dllSelected, TemplateHandlerApp.Templates.TEMPL[] template, string dll, Site.Site.SITE cert0, int slaveIndex, int boxID, Form theParent)
{
       //cr = new CR5_new.CR5(slaveIndex);
       cr = factory.GetCR(type, slaveIndex);
       cr.ProgressChanged += new ProgressChangeHandler2(updateProgressBar);
}

What type is "cr"? And what does the factory return? Btw, your naming isn't very helpful here. Any way we could translate what you're trying to achieve to a simple console-app? Makes it easier to focus on the problem itself, without having to worry about the rest of the app.

Why does the CR5_new class have a Main method? It's not the entry-point for your app, is it?

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

MichCl3-May-13 7:31

3-May-13 7:31

Thanks for the input, Eddy. I guess it's a good idea to try and put it in a simple console app. It might be crucial that I'm jumping through separate vs projects etc, so I left it as-is. I'm not sure why CR5 has a main. It's mostly just a dll and not a form, unlike a lot of my other classes.

cr is an iCR type. It looks like this in the factory:

public iCR GetCR(CRType type, int slaveIndex)
{
            iCR cr = null;
            switch (type)
            {
                case CRType.CR5:
                    cr = new CR5_new.CR5(slaveIndex);
                    break;
                case CRType.CR6:
                    cr = new CR6.CR6(slaveIndex);
                    break;
                default:
                    throw new ArgumentException(string.Format("A CR of type {0} cannot be found", Enum.GetName(typeof(CRType), type)));
            }
            return cr;
}

Eddy Vluggen3-May-13 8:29

3-May-13 8:29

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleApplicationXXIV
{
    enum CRType { CR5, CR6 }
    public delegate void ProgressChangeHandler2(int progress, int id);
    public interface iCR
    {
        event ProgressChangeHandler2 ProgressChanged;
    }
    public class CR5 : iCR
    {
        public int SlaveIndex { get; set; }
        public event ProgressChangeHandler2 ProgressChanged;
        public CR5(int slaveIndex)
        {
            SlaveIndex = slaveIndex;
        }
        public int ProcessTagWrite() //(ref byte[] WriteDat, bool isFastProgramMode)
        {
            AttemptWrites();
            return 0;
        }
        private void AttemptWrites()
        {
            for (int i = 0; i < (10); i++)
            {
                RaiseProgressChange(i, SlaveIndex);
            }
        }
        private void RaiseProgressChange(int progress, int id)
        {
            if (ProgressChanged != null) //this is null
            {
                ProgressChanged(progress, id);  //not getting here
            }
        }
    }
    class GenericPC
    {
        public object GetPC()
        {
            var cr = Program.GetCR(CRType.CR5, 0);
            cr.ProgressChanged += new ProgressChangeHandler2(updateProgressBar);
            (cr as CR5).ProcessTagWrite();
            return cr;
        }

        private void updateProgressBar(int n, int id)
        {
            Console.WriteLine(String.Format("Progress: {0}\tid:{1}", n, id));
        }
    }
    class Program
    {
        static void Main(string[] args)
        {
            var npc = new GenericPC();
            npc.GetPC();
            Console.ReadKey();
        }

        public static iCR GetCR(CRType type, int slaveIndex)
        {
            iCR cr = null;
            switch (type)
            {
                case CRType.CR5:
                    cr = new CR5(slaveIndex);
                    break;
                case CRType.CR6:
                    cr = null; //new CR6(slaveIndex);
                    break;
                default:
                    throw new ArgumentException(string.Format("A CR of type {0} cannot be found", Enum.GetName(typeof(CRType), type)));
            }
            return cr;
        }
    }
}

First things first; place the cursor on the word ProgressChangeHandler2 and press F2 (or Ctrl-R, Ctrl-R) and rename that to something without a number Big Grin | :-D

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

MichCl3-May-13 8:39

3-May-13 8:39

Good start. The Cr5_Comm class is missing from that. Good idea for ProgressChangeHandler to lose the 2. If you don't have the iCR_Comm interface, it's like I never made my changes at all.

Eddy Vluggen3-May-13 8:53

3-May-13 8:53

To use the other interface, the CR5 class would need to inherit from it and implement it.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleApplicationXXIV
{
    enum CRType { CR5, CR6 }
    public delegate void ProgressChangeHandler2(int progress, int id);
    public interface iCRComm
    {
        event ProgressChangeHandler2 ProgressChanged;
        int ReadTag(ref byte[] dataDumpWriteCheck);
    }
    public class CR5 : iCRComm
    {
        public int SlaveIndex { get; set; }
        public event ProgressChangeHandler2 ProgressChanged;
        public CR5(int slaveIndex)
        {
            SlaveIndex = slaveIndex;
        }
        public int ReadTag(ref byte[] dataDumpWriteCheck)
        {
            Console.WriteLine(String.Format("Data: {0}", dataDumpWriteCheck));
            return 0;
        }
        private void RaiseProgressChange(int progress, int id)
        {
            if (ProgressChanged != null) //this is null
            {
                ProgressChanged(progress, id);  //not getting here
            }
        }
    }
    class GenericPC
    {
        public iCRComm GetPC()
        {
            var cr = Program.GetCR(CRType.CR5, 0);
            cr.ProgressChanged += new ProgressChangeHandler2(updateProgressBar);
            byte[] b = null;
            cr.ReadTag(ref b);
            return cr;
        }

        private void updateProgressBar(int n, int id)
        {
            Console.WriteLine(String.Format("Progress: {0}\tid:{1}", n, id));
        }
    }
    class Program
    {
        static void Main(string[] args)
        {
            var npc = new GenericPC();
            npc.GetPC();
            Console.ReadKey();
        }

        public static iCRComm GetCR(CRType type, int slaveIndex)
        {
            iCRComm cr = null;
            switch (type)
            {
                case CRType.CR5:
                    cr = new CR5(slaveIndex);
                    break;
                case CRType.CR6:
                    cr = null; //new CR6(slaveIndex);
                    break;
                default:
                    throw new ArgumentException(string.Format("A CR of type {0} cannot be found", Enum.GetName(typeof(CRType), type)));
            }
            return cr;
        }
    }
}

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

MichCl3-May-13 9:22

3-May-13 9:22

Can CR5 inherit from both iCR and iCRComm?

MichCl3-May-13 9:25

3-May-13 9:25

Your definition and use of iCRComm is what my iCR interface is. It's still not doing what my iCRComm does. Thanks for helping!!

Eddy Vluggen3-May-13 9:40

3-May-13 9:40

Yes; although I don't see why one wouldn't simply modify the original interface to include the new method. Remember that the factory is there for a good reason.

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

MichCl3-May-13 9:48

3-May-13 9:48

Yes, but I have two factories. One for CR5 interface and one for CRComm interface. It may not be obvious from what I posted, but they have very different uses. Generic needs to use CR5 for writes. CR5Comm is used all over the place and definitely isn't the same/can't be condensed into one with CR5. You're changing the use of everything too much for it to be helpful. Maybe we need to go back to the original VS project definitions of everything? I know it's difficult, but it's like that for a reason. Even where you put the delegate doesn't translate to anywhere in my code.

Eddy Vluggen4-May-13 0:41

4-May-13 0:41

MichCl wrote:
It may not be obvious from what I posted, but they have very different uses.

I'd expect anything out that factory to behave similar; otherwise you'd be checking whether you're working on a CR5 or a CR6 everywhere in code, making things even more complicated.

MichCl wrote:
Even where you put the delegate doesn't translate to anywhere in my code.

As far as the compiler is concerned, it's located in "some" namespace. As long as it's referenced, it can be used.

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

MichCl6-May-13 3:40

6-May-13 3:40

Like I said. There are two different factories. One for comm, one for crx. I get two different things from them. The fact that I don't have a check for CR5 or CR6, is because my cr factory returns an instance of iCR that is an instance of either cr5 or cr6. When I make a call from my instance of icr that is returned, it will automatically go to the implementation of cr5 or cr6, whichever I instantiated in the factory.

I realize the compiler doesn't care about the delegate location and usage, but I do. The structure/fix that you are suggesting doesn't translate even slightly to my code, which is in separate vs projects. I can't put my delegate where you have it, so it's not a fix for my problem. Thanks for the help!

Eddy Vluggen6-May-13 7:52

6-May-13 7:52

MichCl wrote:
The structure/fix that you are suggesting doesn't translate even slightly to my code, which is in separate vs projects.

Aight, can you add in the project names in the original post? Make 'em fake ones if required Smile | :)

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

MichCl6-May-13 9:19

6-May-13 9:19

Thanks! I updated the original post with vs project names.

MichCl7-May-13 2:09

7-May-13 2:09

Did you see the changes I made to the original post?

Eddy Vluggen7-May-13 3:16

7-May-13 3:16

I did, but I won't be able to work on it until I am home Smile | :)

MichCl7-May-13 3:55