saving file into sql database - C# Discussion Boards

20-Mar-07 0:01

look up DataTable in Object Browser.

You can use DataTable.Rows[x].Cells[y].Value or something of the likes.

public object BufferOverFlow<br />
        {<br />
__get { return BufferOverFlow; }<br />
__set { BufferOverFlow = value; }<br />
}

What event fired??

Janu_M19-Mar-07 21:53

Janu_M

19-Mar-07 21:53

Hi i developed a windows application, after completion of the processing it has to write the results in to a file. I wrote code for this form_closing event. It is working fine when i closes the application. My problem is

If the system is shutting down with out closing the application it is not writing data in to the file. I dont know what event is fired when the application is closed by shutdown.

It is very important to store the results into the file.

Please Help me.

Thanks in Advance.
Ramu

joon _vh.19-Mar-07 22:06

19-Mar-07 22:06

Here's a small tutorial:

Making a Windows Forms App Respond to System Shutdown[^]

It looks like it is the answer to your problem.

public object BufferOverFlow<br />
        {<br />
__get { return BufferOverFlow; }<br />
__set { BufferOverFlow = value; }<br />
}

Janu_M20-Mar-07 20:25

Janu_M

20-Mar-07 20:25

Thank You Very Much.

This helps a lot for me.

Thank You
Ramu

AB777119-Mar-07 22:22

AB7771

19-Mar-07 22:22

just use the form closing event and use the FormCloseEventArgs object
and check if( e.Closereason == CloseReason.WindowsShutDown)

hope that is useful for u...

Thanks & Regards,
Pramod
"Everyone is a genius at least once a year"

saving file into sql database

Janu_M20-Mar-07 20:26

Janu_M

20-Mar-07 20:26

Thank You very Much.

This helps me a lot.

Thank You
Ramu.

dabuskol19-Mar-07 21:39

dabuskol

19-Mar-07 21:39

Hi,

I want to save/upload a file in sql database BUT WITHOUT using the upload object from c#, just a straight forward code. I already have the path of the file itself with fuill permission. Does anybody know how to do it?

Thanks

Dabsukol

Re: saving file into sql database

joon _vh.19-Mar-07 22:48

Fastest way to write a text file

19-Mar-07 22:48

Will this do the trick?

Adding binary data to a table[^]

public object BufferOverFlow<br />
        {<br />
__get { return BufferOverFlow; }<br />
__set { BufferOverFlow = value; }<br />
}

MSBuild Question

Member 390685019-Mar-07 20:53

Member 3906850

19-Mar-07 20:53

Code is simple as below:

Process process = Process.Start("MSBUILD.EXE", "D:\\Common\\Common.sln /p:Configuration=Debug");
process.WaitForExit();

When I ran above code in C#, I got "ExitCode = 1" error, couldn't figure out why. Anybody please help?

Thanks a bunch.

Hazy

blackjack215019-Mar-07 20:51

blackjack2150

19-Mar-07 20:51

Hi.

I need to write a large text file from my application. Large means something like 200+ Megabytes.

What is the fastest way to do it? Using the static methods of the "File" class or using streams?? I am aware that either way, it'll take a long, long tine, but ...time is precious.

And another related question. It helps in terms of performance to write as much data as possible at a time, doesn't it?

Thanks.

Russell Jones19-Mar-07 22:08

Russell Jones

19-Mar-07 22:08

blackjack2150 wrote:

And another related question. It helps in terms of performance to write as much data as possible at a time, doesn't it?

kind of depends what you mean by performance. but often yes.

1. The user starts the file save procedure and starts looking at email. The PC will look faster if your process is slightly less fast so smaller packets that don't block the disk queue might help.

2. The hard disk has issues. The program continually tries to write and fails. Because your packets are huge it fails more often as it's hard to find unbroken space that big and it has to attempt each write many times. (This would more likely be a problem on a network share for instance).

3. The hard disk is fast, is 0% fragmented and there's more ram and CPU power in the box than you could shake a stick at. This machine isn't going to slow down for anything so you might as well max the packet size and get the job done fast.

Do you need this file save to complete before the user continues? If not i would suggest sticking the save process in a background thread and maybe put a progress bar in the UI.

Russ

blackjack215019-Mar-07 22:33

blackjack2150

19-Mar-07 22:33

User interaction is not important for me. In fact is almost absent. I've converted my program to a console application, as I noticed that during the execution the UI got blocked. I know there are ways to avoid this, but I decided that it's not worth the effort.

If it helps I'll briefly describe what the program does:

I have a folder containing more than 7000 xml files which are news articles. I must read everyone of them and write to a file every distinct word in these xmls. It adds up to about 42000 words. I another file I must write the frequency of appearance of every word in every xml file, so there are 42000*7000 = a s...load of numbers to be written.

So, the user just starts the program, sits back and begins to read a book on programming. Smile | :)

It takes about 5 minutes to read the files and store the words in memory, another 5 to write the 42000 words to the first file, and writing the last file....well God knows, as I never had the patience to wait that much.
This file should have: (7000 * 42000 * 2) / 1024 /1024 ~= 560 MB. OMG | :OMG:

The data structures I use are just basic arrays and matrixes. Does the word "matrixes" exist? Or should it be matrices?

So, what do you suggest?

m@u19-Mar-07 23:18

m@u

19-Mar-07 23:18

Hi
well that does not sound like a I/O Performance problem. writing to streams is pretty fast.
i have an application that writes every day a 70GB Zip - File and it doesnt take more than 4 hours. and it gets slower the bigger the file gets. the first gb takes about 15 minutes.

is the whole data processing finished when your app writes the first file?
if not, i would say, the creation of the data used for your second file takes so long and not the write process of it..

greets
m@u

PIEBALDconsult23-Feb-10 10:46

PIEBALDconsult

23-Feb-10 10:46

It seems to me that you can write the first file as you go, unless you want to write them alphabetically or something.

If you have a reasonable idea of the maximum number of digits for the frequency, you could write the second file as you go as well, just keep a Dictionary containing the word, the current count, and the offset to the position of the frequency.

blackjack2150 wrote:
The data structures I use are just basic arrays and matrixes

I recommend a Dictionary.

blackjack2150 wrote:
matrices

That is correct.

joon _vh.19-Mar-07 22:22

19-Mar-07 22:22

I mocked up a quick perf. test. You can run it yourself to see the results (in VS 2005).

Just make an empty console app. and past this code:

    class Program<br />
    {<br />
        static string printme =<br />
            "Fastest way to write a text file Fastest way to write a text file Fastest way to write a text file";<br />
<br />
        static void Main(string[] args)<br />
        {<br />
            // start the first stopwatch <br />
<br />
            Stopwatch sw = new Stopwatch();<br />
            sw.Start();<br />
<br />
            // run the first test (stream)<br />
<br />
            test1();<br />
<br />
            // stop stopwatch and print<br />
<br />
            sw.Stop();<br />
            Console.WriteLine(sw.Elapsed);<br />
<br />
            // reset <br />
<br />
            sw.Reset();<br />
<br />
            // start second stopwatch<br />
            <br />
            sw.Start();<br />
<br />
            // run second test<br />
<br />
            test2();<br />
<br />
            // stop stopwatch and print<br />
<br />
            sw.Stop();<br />
            Console.WriteLine(sw.Elapsed);<br />
<br />
            // hold for laughs<br />
<br />
            Console.Read();<br />
        }<br />
<br />
<br />
        /// <summary><br />
        /// Method that writes 5000 lines of text with a streamwriter<br />
        /// </summary><br />
        static void test1()<br />
        {<br />
            TextWriter tw = new StreamWriter("test1.txt");<br />
            for (int i = 0; i < 50000; i++)<br />
            {<br />
                tw.WriteLine(i + " " + printme);<br />
            }<br />
            tw.Close();<br />
        }<br />
<br />
        /// <summary><br />
        /// Method that writes 5000 lines of text with the static methods from the File class<br />
        /// </summary><br />
        static void test2()<br />
        {<br />
            for (int i = 0; i < 50000; i++)<br />
            {<br />
                 File.AppendAllText("test2.txt",i + " " + printme + "\n\r");<br />
            }<br />
        }<br />
    }

I hope my logic is right, and if it is, you will hopefulyl see the same result as me. Streamwriter completes in a few seconds, and File.AppendAllText... takes forever.
There is also AppendText, but that returns a streamwriter and gives you the same result as test1.

Hope I helped.

public object BufferOverFlow<br />
        {<br />
__get { return BufferOverFlow; }<br />
__set { BufferOverFlow = value; }<br />
}

Guffa19-Mar-07 22:30

Guffa

19-Mar-07 22:30

blackjack2150 wrote:
What is the fastest way to do it? Using the static methods of the "File" class or using streams?? I am aware that either way, it'll take a long, long tine, but ...time is precious.

I don't think that there is such a big difference. Most of the static methods in the File class are using streams anyway, so then the difference is just how you use the stream.

There might be a difference in writing a text stream, and encoding the text first and writing it as a binary stream, so that might be worth testing.

blackjack2150 wrote:
It helps in terms of performance to write as much data as possible at a time, doesn't it?

That depends. The difference is hardly noticable unless the buffer size is very small or very big.

You should be aware of the large objects heap, though. Any object that is larger than 85 kb is allocated in the largs objects heap instead of the regular heap. The large objects heap grows when needed but never shrinks, so if you allocate a lot of large objects the program may allocate a lot of memory that is not returned to the system until the program ends.

---
single minded; short sighted; long gone;

joon _vh.19-Mar-07 23:49

19-Mar-07 23:49

Guffa wrote:
There might be a difference in writing a text stream, and encoding the text first and writing it as a binary stream, so that might be worth testing.

Here is some code. Again, make a console app and paste this testcode. A Bytestream is obviously faster, but I can't seem to debug it properly (it generates a character that shouldn't be there at the beginning of each write, so the file is bigger).

<br />
        static string printme =<br />
            "Fastest way to write a text file Fastest way to write a text file Fastest way to write a text file";<br />
<br />
        private static int linecount = 50000;<br />
<br />
        static void Main(string[] args)<br />
        {<br />
            // start the first stopwatch <br />
<br />
            Stopwatch sw = new Stopwatch();<br />
            sw.Start();<br />
<br />
            // run the first test (stream)<br />
<br />
            test1();<br />
<br />
            // stop stopwatch and print<br />
<br />
            sw.Stop();<br />
            Console.WriteLine(sw.Elapsed);<br />
<br />
            // reset <br />
<br />
            sw.Reset();<br />
<br />
            // start second stopwatch<br />
            <br />
            sw.Start();<br />
<br />
            // run second test<br />
<br />
            test2();<br />
<br />
            // stop stopwatch and print<br />
<br />
            sw.Stop();<br />
            Console.WriteLine(sw.Elapsed);<br />
<br />
            // hold for laughs<br />
<br />
            Console.Read();<br />
        }<br />
<br />
<br />
        /// <summary><br />
        /// Method that writes 5000 lines of text with a streamwriter<br />
        /// </summary><br />
        static void test1()<br />
        {<br />
            TextWriter tw = new StreamWriter("test1.txt");<br />
            for (int i = 0; i < linecount; i++)<br />
            {<br />
                tw.WriteLine(i + " " + printme);<br />
            }<br />
            tw.Close();<br />
        }<br />
<br />
        /// <summary><br />
        /// Method that writes 5000 lines of text with a bytestream<br />
        /// </summary><br />
        static void test2()<br />
        {<br />
            FileStream fs = File.Create("test2.txt");<br />
            BinaryWriter bw = new BinaryWriter(fs);<br />
            for (int i = 0; i < linecount; i++)<br />
            {<br />
                string toprint = i + " " + printme + "\r\n";<br />
                bw.Write(toprint);<br />
            }<br />
        }<br />
    }<br />

public object BufferOverFlow<br />
        {<br />
__get { return BufferOverFlow; }<br />
__set { BufferOverFlow = value; }<br />
}

Guffa20-Mar-07 2:10

Guffa

20-Mar-07 2:10

A BinaryWriter is not used to write text to a binary file, it's used to write data in binary format to a binary file. When you write a string using a BinaryWriter, it writes the length of the string, then the binary representation of the string. The string is not encoded either, so it will be in UTF-16, which is the internal format for strings.

You don't need a writer to write a binary file. A FileStream can do that.

static void test2() {
   using(FileStream fs = File.Create("test2.txt")) {
      for (int i = 0; i < linecount; i++) {
         string toprint = i.ToString() + " " + printme + "\r\n";
         byte[] buffer = Encoding.UTF8.GetBytes(toprint);
         bw.Write(buffer, 0, buffer.Length);
      }
   }
}

---
single minded; short sighted; long gone;

joon _vh.20-Mar-07 2:50