Re: Using Binary Serialization and GZip ? - C# Discussion Boards

BillWoodruff17-Jul-13 18:11

17-Jul-13 18:11

Hi Dave,

I am delighted to know you are familiar with such constructions (the more common form, I believe, is "do the necessary") [^]. I'm not sure whether, technically, these (now archaic) forms qualify as calques, and/or Anglicisms.

Have you lived in India, or is there a family connection with the Raj ?

Hope my response to you did not seem patronizing Smile | :)

yours, Bill

“Human beings do not live in the objective world alone, nor alone in the world of social activity as ordinarily understood, but are very much at the mercy of the particular language which has become the medium of expression for their society. It is quite an illusion to imagine that one adjusts to reality essentially without the use of language and that language is merely an incidental means of solving specific problems of communication or reflection." Edward Sapir, 1929

Re: Help Regarding OPEN XML,

Dave Kreskowiak18-Jul-13 2:09

Dave Kreskowiak

18-Jul-13 2:09

BillWoodruff wrote:
Have you lived in India, or is there a family connection with the Raj ?

Oh God, no! I just went and did the homework to find out my it's so popular in India, and, to a lesser extent, South Asia English.

BillWoodruff wrote:
Hope my response to you did not seem patronizing

No, no. We're good! Laugh | :laugh:

A guide to posting questions on CodeProject[^]

Dave Kreskowiak

Re: Help Regarding OPEN XML,

BillWoodruff17-Jul-13 17:28

BillWoodruff

17-Jul-13 17:28

Hi Meetnneel,

Please describe what you have done so far, post the code you have written so far, and discuss what specific problems you are having with it.

yours, Bill

Using Binary Serialization and GZip ?

BillWoodruff17-Jul-13 3:14

BillWoodruff

17-Jul-13 3:14

I am successfully using .NET's BinaryFormatter to serialize a very complex object (an instance, at run-time) into a MemoryStream, and I can save that to a file, and then de-serialize the file back into an instance of the object.

Then, I wanted to add compressing the MemoryStream using GZip [1], before saving it to a file.

I am, without compile- or run- time error, using .NET's GZip facility to GZip the created MemoryStream, and then save it to a disk file.

To summarize the save-to-file sequence:

1. use BinaryFormatter to serialize the object to a MemoryStream

2. use GZip to compress the MemoryStream

3. write the Gzipped MemoryStream to disk

I've already learned the hard way that there is a quirk in using GZip in .NET in compressing [^].

I verify that the file is written, and it appears to have been "shrunk" by the GZip facility.

But, when I try to reverse the process, to essentially deserialize (recreate) the object by:

1. reading the Gzipped saved file into a MemoryStream

2. using GZip's Decompress on the MemoryStream

3. using BinaryFormatter to deserialize the MemoryStream, which I then cast into the Type of the original object: fail.

The point I am stuck at is expressed in this code:

// un GZip

// at this point 'newMemoryStream holds the result
// of opening the saved serialized/Gzipped file

using (GZipStream deCompressionStream = new GZipStream(new MemoryStream(), CompressionMode.Decompress))
{
   // here 'deCompressionStream is 'CanRead = 'true, CanWrite = 'false
   // Length property examined at break-point indicates some Exception
   // so you can't write anything into 'deCompressionStream
   // like this:
   // newMemoryStream.CopyTo(deCompressionStream);

   // yes, this will not fail, but doesn't make sense
   // it's inconsistent with the way that Gzipping compression works !
   deCompressionStream.CopyTo(newMemoryStream);
}

BinaryFormatter bFormatter = new BinaryFormatter();

// if you try this using the "doesn't make sense"
// attempt above:
// you'll get a 'stream was not readable' error

object obj = bFormatter.Deserialize(newMemoryStream);

Thanks in advance for any advice.

yours, Bill

[1] I set out to explore using GZip as a way to increase my general knowledge of .NET's stream facilities, and after learning how to successfully use Mehdi Gholam's fastBinaryJSON work here on CP, and using the MiniLZO class in Mehdi's Raptor Document Store.

Re: Using Binary Serialization and GZip ?

Richard Deeming17-Jul-13 4:02

Richard Deeming

17-Jul-13 4:02

You're passing an empty MemoryStream to the GZipStreams constructor. You need to pass the stream containing the previously compressed data instead.

using (var fileStream = File.OpenRead(@"Path\To\Your\File.bin"))
using (var deCompressionStream = new GZipStream(fileStream, CompressionMode.Decompress))
{
   var bFormatter = new BinaryFormatter();
   object obj = bFormatter.Deserialize(deCompressionStream);
}

"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer

Re: Using Binary Serialization and GZip ?

BillWoodruff17-Jul-13 15:07

BillWoodruff

17-Jul-13 15:07

Richard, thanks very much for your example !

Your deserialization code is one of the first variations I tried, and it fails to deserialize with an error: "The magic number in GZip header is not correct. Make sure you are passing in a GZip stream."

Clearly my Gzipping is not being done properly. This is confirmed by this experiement: if I modify your code like this:

using (var fileStream = File.OpenRead(filePath))
using (var deCompressionStream = new GZipStream(fileStream, CompressionMode.Decompress))
{
    var bFormatter = new BinaryFormatter();

    // put a break-point here: examine obj
    // this works !
    object obj = bFormatter.Deserialize(fileStream);

    // for testing only
    return null;
}

That confirms to me I did not GZip properly.

I tried several variations of GZipping found on MSDN, and StackOverFlow: almost all these examples use source.CopyTo(destination) in some form, within the using statements bracketing the use of GZipStream, evidently for the reason there is a known problem with GZipping not being completed fully until after it exits the 'using statement closure, as referenced here: [^].

So, the burden shifts back to finding the correct way for Gzipping.

thanks again for your time, Bill

Re: Using Binary Serialization and GZip ?

Eddy Vluggen17-Jul-13 5:07

Eddy Vluggen

17-Jul-13 5:07

You're copying streams which isn't required; they're based on the decorator-pattern. Below example should work (albeit the GZipped version will be bigger since it's a small class)

using System;
using System.Runtime.Serialization.Formatters.Binary;
using System.IO;
using System.IO.Compression;

[Serializable()]
class SumTestClass
{
    public int TestInt { get; set; }
    public string TestString { get; set; }
    public SumTestClass()
    {
        TestInt = 42;
        TestString = "Hello World";
    }
}

static class Program
{
    static BinaryFormatter bf = new BinaryFormatter();
    static void Main(string[] args)
    {
        object thingyToSerialize = new SumTestClass();

        // ..or use a memorystream :)
        using (var ms = new FileStream("C:\\Temp\\Test.zip", FileMode.Create)) 
        using (var zs = new GZipStream(ms, CompressionMode.Compress, leaveOpen: true))
        {
            bf.Serialize(zs, thingyToSerialize);
            Console.WriteLine("Stuff saved");
        }
        
        using (var ms = new FileStream("C:\\Temp\\Test.zip", FileMode.Open)) 
        using (var zs = new GZipStream(ms, CompressionMode.Decompress, leaveOpen: true))
        {
            SumTestClass testResult = bf.Deserialize(zs) as SumTestClass;
            Console.WriteLine("String: {0}\nInt: {1}", testResult.TestString, testResult.TestInt);
        }

        Console.ReadLine();
    }
}

Hope it helps a bit, yell if it doesn't Smile | :)

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

Re: Using Binary Serialization and GZip ?

BillWoodruff17-Jul-13 16:52

BillWoodruff

17-Jul-13 16:52

Thanks, Eddy !

Your code works with a MemoryStream that has been Gzipped, Serialized, and written to a file.

The 'leaveopen parameter ... in both gzip, and un-gzip ... appears to be redundant: my implementation of your code works with, or without, them.

It compresses a 9mb object=>source-stream to a little over 3mb disk file, whether Compression setting is 'Fastest, or 'Optimal.

"Raising the stakes" from 100k to 1 million total objects yielded a 98+ megabyte MemoryStream that Gzipped to a 30.6mb file with both 'Optimal, and 'Fastest compression. That does make me question the efficiency of Gzip in .NET.

But (hypothesis), perhaps the complex object I am Gzipping and Serializing is, by its nature not highly compressible ?

It is much better to have working code you don't quite understand, than to have code you believe you understand, and expect to work, that does not work Smile | :)

When I feel like I fully understand how I "strayed onto the wrong path" for so many hours, yesterday, and understand exactly why your code works, I feel I should write up a Tip-Trick for CP.

I didn't realize that by using a ".gz" extension to the files I created I was creating something that could be opened with WinRar !

yours, Bill

~

"This isn't right. This isn't even wrong." Wolfgang Pauli"

Re: Using Binary Serialization and GZip ?

Eddy Vluggen18-Jul-13 6:32

Eddy Vluggen

18-Jul-13 6:32

You're welcome Smile | :)

BillWoodruff wrote:
"Raising the stakes" from 100k to 1 million total objects yielded a 98+ megabyte MemoryStream that Gzipped to a 30.6mb file with both 'Optimal, and 'Fastest compression. That does make me question the efficiency of Gzip in .NET.

Alternatively, one could launch an external process and wait for the command to finish. And suddenly, one can easily switch out the ZIP and try an ARJ, or a LHArc Smile | :)

BillWoodruff wrote:
I feel I should write up a Tip-Trick for CP.

Please do

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

Re: Using Binary Serialization and GZip ?

BillWoodruff18-Jul-13 17:00

BillWoodruff

18-Jul-13 17:00

Eddy Vluggen wrote:
Alternatively, one could launch an external process and wait forthe command to finish. And suddenly, one can easily switch out the ZIP and try an ARJ, or a LHArc

I have thought about experimenting with launching something like 7-Zip via Process.Start, as suggested (but never detailed) on the 7-Zip discussion forums, but I wonder if, in using any externally launched process, the (possibly required) transfer of the stream to be compressed, or the creation of a "temp" file to hold the stream, before invoking the external process' facilities, would add an "overhead" that would negate its benefit.

I have SharpCompress, found here on CodeProject [^], working, but am not seeing any radical improvement in the current test case over .NET's built-in GZip.

Yesterday I spent a few hours trying to get SharpZipLib [^] working in VS 2012, FrameWork 4.5, without success. But, I'll try that again today.

thanks, again, Bill

~
“This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal

modified 19-Jul-13 1:10am.

Re: Using Binary Serialization and GZip ?

Eddy Vluggen19-Jul-13 7:30

Eddy Vluggen

19-Jul-13 7:30

BillWoodruff wrote:
would add an "overhead" that would negate its benefit.

To answer that, one needs to investigate both overhead and benefit. Yes, it does introduce a slight overhead - but not very much if you hide the console-window. Consider it an isolated, switchable, heavyweight instance of a thread.

BillWoodruff wrote:
SharpCompress, SharpZipLib

Let's also add DotNetZip [^]. And LZMA[^].

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

Re: Using Binary Serialization and GZip ?

harold aptroot18-Jul-13 7:17

harold aptroot

18-Jul-13 7:17

See http://www.virtualdub.org/blog/pivot/entry.php?id=335[^] for how much it sucks.
There are various 3rd party deflate libs that suck less, I recommend SharpZipLib (nice license, too).

Though personally I recommend not using the built-in serialization in the first place and just dumping your raw data in a stream. It's a little more work up-front, but it's more flexible, a lot faster, and your data format will actually make sense (which is not always important of course). You can still put it through Deflate of course, I tend to do that (easier than explicitly storing sparse arrays and such).

Re: Using Binary Serialization and GZip ?

BillWoodruff18-Jul-13 17:04

BillWoodruff

18-Jul-13 17:04

That's an excellent analysis of GZip's limitations, in the article you cite, on virtualdub.com !

As I just commented to Eddy (post above): "Yesterday I spent a few hours trying to get SharpZipLib working in VS 2012, FrameWork 4.5, without success. But, I'll try that again today."

thanks, Bill

~
“This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal

Re: Using Binary Serialization and GZip ?

Ennis Ray Lynch, Jr.17-Jul-13 7:11

Ennis Ray Lynch, Jr.

17-Jul-13 7:11

GZip Stream writes information to the stream after it is "closed" when creating the compressed zip stream. Do not wrap your compression in the using block. Call flush before closing out the stream when compressing. Then your decompress code may work.

Need custom software developed? I do custom programming based primarily on MS tools with an emphasis on C# development and consulting.

"And they, since they Were not the one dead, turned to their affairs" -- Robert Frost

"All users always want Excel" --Ennis Lynch

Re: Using Binary Serialization and GZip ?

BillWoodruff17-Jul-13 17:02

BillWoodruff

17-Jul-13 17:02

Thanks, Ennis, I was aware of that issue, and in the OP, I cited a response on StackOverFlow:

"I've already learned the hard way that there is a quirk in using GZip in .NET in compressing [^]:"

That post cites a Microsoft BCL blog: [^].

yours, Bill

“This isn't right. This isn't even wrong." Wolfgang Pauli

WebBrowser and swf textbox and button

Member 843530717-Jul-13 2:11

Member 8435307

17-Jul-13 2:11

How can I select flash textbox, past text, and click button in C# WebBrowser?

TextBox and Button: s7.postimg.org/oh2w84js7/button.jpg

Form where button is:

HTML

<form name="working" action="object_do.php" method="POST"><br><table class="wb" width="100%"><tbody><tr><td class="wbwhite" align="center" colspan="2">blahblahblah</td></tr><tr><td class="wb" width="140"><img src="work_codes/15903-34/1701507--381682.jpeg" width="250" height="60" border="0" name="imgcode"></td><td class="wblight" valign="center" width="100%">   <input type="hidden" size="4" name="code" autocomplete="off"><input type="hidden" value="5" name="id"><input type="hidden" value="5" name="id2"><script type="text/javascript">swfobject.registerObject("2e7f13a242369aa0a133ee3dced5bf5d", "9.0.0", "http://***/swffiles/expressInstall.swf");</script>
    <div>   
                <object id="2e7f13a242369aa0a133ee3dced5bf5d" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" width="147" height="22" style="visibility: visible;">
                    <param name="movie" value="swffiles/459017015079176508wworkcode.swf?ver=1701507bf3d40fb076319ff7e6c2fc7a4898acdd88">
                <param name="FlashVars" value="params=1701507|5|f3d40fb076319ff7e6c2fc7a4898acdd|">
                <param name="bgcolor" value="#f5f3ea">
                <param name="allowScriptAccess" value="always"><param name="wmode" value="opaque"><!--[if !IE]>-->
BUTTON AND TEXT BOX----------->     <object type="application/x-shockwave-flash" data="swffiles/459017015079176508wworkcode.swf?ver=1701507bf3d40fb076319ff7e6c2fc7a4898acdd88" width="147" height="22">
                <param name="FlashVars" value="params=1701507|5|f3d40fb076319ff7e6c2fc7a4898acdd|">
                <param name="bgcolor" value="#f5f3ea">
                <param name="allowScriptAccess" value="always"><param name="wmode" value="opaque"><!--<![endif]-->
                    <div>
                        <p><a href="http://www.adobe.com/go/getflashplayer"><img src="http://www.adobe.com/images/shared/download_buttons/get_flash_player.gif" alt="Get Adobe Flash player"></a></p>
                    </div>
                    <!--[if !IE]>-->
                    </object>
                    <!--<![endif]-->
                </object>
            </div></td></tr><tr><td class="wblight" colspan="2"><br>   blahblahblah "blah".</td></tr></tbody></table></form>

Re: WebBrowser and swf textbox and button

Amir Mohammad Nasrollahi29-Jul-13 6:33

Amir Mohammad Nasrollahi

29-Jul-13 6:33

it is inconceivable copying from flash file because it is compiled and how you want do that?
it's my opinion Smile | :)

-Amir Mohammad Nasrollahi

/* LIFE RUNS ON CODE */

viewing and printing a pdf on different browser

tasumisra17-Jul-13 0:25

tasumisra

17-Jul-13 0:25

Hi Guys,

Recently i am working on an assignment where i need to find a 3rd party software to download and print a pdf on different browser including (IE6,7,8,9) and Mozila,crome ,Opera etc..

i found a few but not sure which one would be suitable

http://www.debenu.com/kb/setup-c-and-visual-studio-for-use-with-debenu-quick-pdf-library/[^]

http://www.verypdf.com/app/pdf-editor/pdf-viewer-ocx-control-activex.html[^]

http://www.foxitsoftware.com/products/sdk/print-manager/pricing.php[^]

can you please help me in selecting one .. may be the one can be used with windows based application as well.

Regards,
Tasu

vikas da

Re: viewing and printing a pdf on different browser

Bernhard Hiller17-Jul-13 21:43

Bernhard Hiller

17-Jul-13 21:43

I fail to understand the reason behind that requirement. Adobe Acrobat reader is available for many browsers (perhaps no more supporting extremely outdated legacy browsers like IE6 or NetScape 4) on many platforms - even Linux. And most browsers are already configured to use that. Why do you insist on soemthing different?

Re: viewing and printing a pdf on different browser

Amir Mohammad Nasrollahi29-Jul-13 6:40

Amir Mohammad Nasrollahi

29-Jul-13 6:40

good

/* LIFE RUNS ON CODE */

Re: viewing and printing a pdf on different browser

Amir Mohammad Nasrollahi29-Jul-13 6:40

Amir Mohammad Nasrollahi

29-Jul-13 6:40

as what "Bernhard Hiller" says adobe acrobat is free and suported by many browsers but from your selections list, verypdf can be better because of prices and good support (tasted by one of my friends)

/* LIFE RUNS ON CODE */

Problem in creating Shortcut to desktop and uninstall program in user all program

Sneha Bisht16-Jul-13 21:24

Sneha Bisht

16-Jul-13 21:24

Hi,
i face a problem to create a setup of window application in vs2010.
for create a setup i use vs window installer and done step by step as i seen many examples in Google( how to create setup for window application).
But in my case shortcut create in desktop, but when i click on the shortcut it will running look like as it install project again.
same uninstall project create in all programs but when i click on them nothing happens.

please help me.
Thanks.

Re: Problem in creating Shortcut to desktop and uninstall program in user all program

Jay Nardev17-Jul-13 22:03

Jay Nardev

17-Jul-13 22:03

Create shortcut of executable by right click on it in Application Folder in “File System” Section of Setup Project. And then cut the shortcut of file and paste it into user desktop folder and build the project.

It should work.

Windows Service: Timer vs Thread

BBatts16-Jul-13 6:15

BBatts

16-Jul-13 6:15

Hello All,

I was wanting to start a discussion about best practice when creating threads in a Windows Service.

I have a service that has multiple process running at the same time (6-10). Is it better to use timers or threads? I personally prefer threads because I can end them all with a single line of code where as if have to end each timer individually. Plus with threads, you don't have to worry about it calling it self until you want it to.

Does anybody have any other thoughts on this? What's a better way to go and why? Which is lighter on PC usage and memory? etc etc

Thanks in advance,
Bryan

Re: Windows Service: Timer vs Thread

jschell16-Jul-13 8:18

jschell

16-Jul-13 8:18

BBatts wrote:
Is it better to use timers or threads?

When the timer fires it uses a thread.

BBatts wrote:
Plus with threads, you don't have to worry about it calling it self until you
want it to.

Huh? If the thread is running and not blocked then it is fact running. If you weren't saying that I am not sure what you are saying because the execution flow is always in one thread or another.

BBatts wrote:
because I can end them all with a single line of code

Not sure what that means exactly. At least sometimes one can't just end threads arbitrarily and sometimes one must end them in a certain order. But the same applies to timers but only to the extent that they too are using threads.

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.