.NET (Core and Framework)

Memory utilization of MemoryStream

21-Feb-14 21:06

You'll need to save the code page in a separate column next to the VARCHAR, or turn that into an NVARCHAR. You can't "determine" the codepage from a random string of bytes.

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

elix54520-Feb-14 9:18

elix545

20-Feb-14 9:18

Why MemoryStream object uses double the memory?

For example, whether the capacity of MemoryStream is 1GB in Task Manager I see that is consuming 2GB.

   Module Module1

    Sub Main()
        Dim ObjMemoryStream As System.IO.MemoryStream
        ObjMemoryStream = New System.IO.MemoryStream()
        System.Console.WriteLine("Capacity: {0}", ObjMemoryStream.Capacity.ToString())
        System.Console.WriteLine("Length: {0}", ObjMemoryStream.Length.ToString())

        Dim j As Integer

        For i As Integer = 0 To 1073741824 Step 1
            If (i Mod 256) = 0 Then j = 0 Else j = i Mod 256
            ObjMemoryStream.WriteByte(j)
        Next i

        System.Console.WriteLine("Length: {0}", ObjMemoryStream.Length.ToString())

        Dim ObjFileStream As System.IO.FileStream
        ObjFileStream = System.IO.File.Create("C:\TEST.TXT", 1024)

        Dim ObjBinaryWriter As System.IO.BinaryWriter
        ObjBinaryWriter = New System.IO.BinaryWriter(ObjFileStream, System.Text.Encoding.ASCII)

        ObjMemoryStream.Position = 0

        ObjBinaryWriter.Write(ObjMemoryStream.ToArray())

        ObjBinaryWriter.Flush()
        ObjBinaryWriter.Close()

        System.Console.ReadKey()

    End Sub

End Module

Pete O'Hanlon20-Feb-14 9:35

Pete O'Hanlon

20-Feb-14 9:35

First of all, you don't see the MemoryStream in isolation as far as memory consumption goes. There's the rest of the program, including all the assemblies that get loaded in - they all take up memory. Secondly, don't rely on Task Manager to determine memory profile. Task manager is a notoriously poor way of judging memory usage. Part of the reason for this is that .NET grabs more memory than it needs - this is to give the application room "to grow into" as it runs. Memory allocation is an expensive operation, so having your application grab just the bytes it needs while it's running would be a waste of time.

Dave Kreskowiak20-Feb-14 9:36

Dave Kreskowiak

20-Feb-14 9:36

That's how much memory the MemoryStream has set aside for use. Since you didn't specify a capacity, it starts off with 256 bytes by default. As the stream is used and the current capacity is exceeded, it reallocates it's internal buffer and sets a new size larger than the current capacity. That new size is always twice the last size it used.

Arrays in .NET are immutable. Once created, you cannot change the size of an array. That limitation applies to everything in the .NET Framework, including the buffer used by MemoryStream. In order to increase the size, a new array of some size must be created and all the elements of the previous array copied to it, then the previous array is destroyed.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak

elix54520-Feb-14 10:08

elix545

20-Feb-14 10:08

I think that object are storing the the data using two memory byte, please check the next example:

Module Module1

    Sub Main()         
        Dim ObjMemoryStream As System.IO.MemoryStream
        ObjMemoryStream = New System.IO.MemoryStream(1073741825)
        System.Console.WriteLine("Capacity: {0}", ObjMemoryStream.Capacity.ToString())
        System.Console.WriteLine("Length: {0}", ObjMemoryStream.Length.ToString())

        Dim j As Integer

        For i As Integer = 0 To 1073741824 Step 1
            If (i Mod 256) = 0 Then j = 0 Else j = i Mod 256
            ObjMemoryStream.WriteByte(j)
        Next i

        System.Console.WriteLine("Capacity: {0}", ObjMemoryStream.Capacity.ToString())
        System.Console.WriteLine("Length: {0}", ObjMemoryStream.Length.ToString())

        Dim ObjFileStream As System.IO.FileStream
        ObjFileStream = System.IO.File.Create("C:\TEST.txt", 1024)

        Dim ObjBinaryWriter As System.IO.BinaryWriter
        ObjBinaryWriter = New System.IO.BinaryWriter(ObjFileStream, System.Text.Encoding.ASCII)

        ObjMemoryStream.Position = 0

        ObjBinaryWriter.Write(ObjMemoryStream.ToArray())

        ObjBinaryWriter.Flush()
        ObjBinaryWriter.Close()

        System.Console.ReadKey()
    End Sub


End Module

Dave Kreskowiak20-Feb-14 13:11

Dave Kreskowiak

20-Feb-14 13:11

No, it's not. What you're storing is irrelevant. If you keep exceeding the capacity of the MemoryStream, it will keep allocating new arrays, twice as large as the last, until it can't grow any more (2GB).

These are the sizes the MemoryStream went through after you created it and keep adding more and more data to it:

Oh, look at that. It managed to hit 2GB in size right after you added 1073741824 bytes of information to it. It's allocating bigger and bigger blocks of memory because it has no way of knowing how big to make the array to that it fits your data perfectly.

If you want to keep it from getting to 2GB of memory, allocate the MemoryStream with the expected size as a parameter, say 1.5GB:

Dim ms As New MemoryStream(1610612736)

Now, if you exceed that, the MemoryStream will try to reallocate itself using twice the size you initially specified and that will lead to an OutOfMemory exception.

You can NOT look in Task Manager to see how much memory your app is using. It's telling you how much memory the .NET CLR as RESERVED for your app. This will always be larger than what your app is actually using.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak

elix54520-Feb-14 14:25

elix545

20-Feb-14 14:25

Thank you very mush. Thumbs Up | :thumbsup:

Question about VB and Databases

JonB26-Feb-14 5:19

JonB

26-Feb-14 5:19

I assume you're looking at the process size when you hit the ReadKey().

Your ObjMemoryStream will be 1GB + 1 byte big (you start from 0).

Then you evaluate ObjMemoryStream.ToArray(). That will create an array (Help: "This method returns a copy of the contents of the MemoryStream as a byte array"), also of size 1GB-odd. (You don't know when .NET will garbage-collect that, and process sizes often don't shrink anyway.)

Don't convert your memory stream to an array! Try something like ObjMemoryStream.CopyTo() instead.

Gantt Chart

Member 1057034317-Feb-14 0:39

Member 10570343

17-Feb-14 0:39

Can anyone help me to develope Gantt chart in resources view type. I want to show the various task assign to employee in a particular duration. Please share me code for the same. Frown | :(

-Angel

ASP.NET Membership

Meetmca16-Feb-14 18:17

Meetmca

16-Feb-14 18:17

Can anyone suggest how to work with asp.net membership configuration and database?

Re: ASP.NET Membership

Richard MacCutchan16-Feb-14 22:26

Richard MacCutchan

16-Feb-14 22:26

Possibly, but your question would be better placed for an answer if you posted in the http://www.codeproject.com/Forums/12076/ASP-NET.aspx[^].

Veni, vidi, abiit domum

CaseLost14-Feb-14 17:05

CaseLost

14-Feb-14 17:05

Im no sure if im in the correct discussion but im am trying to create a stupid and simple project. i have create a database and i connected it with VB. i have create a menustrip with "table" and "data".
the "table" has a sub-menu that says "Inventory show" and "Customer_Information show". When you click on the "Inventory show" in the sub-menu...how do you write the code to show only "Inventory" (from the database) in the form? sorry if i explained confusing, my english is off a little bit.

Public Class Form1

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
'TODO: This line of code loads data into the 'GradebookDataSet._02_Customer_Information' table. You can move, or remove it, as needed.
        Me._02_Customer_InformationTableAdapter.Fill(Me.GradebookDataSet._02_Customer_Information)
        'TODO: This line of code loads data into the 'GradebookDataSet._01_Inventory' table. You can move, or remove it, as needed.
        Me._01_InventoryTableAdapter.Fill(Me.GradebookDataSet._01_Inventory)

    End Sub

    Private Sub InventoryToolStripMenuItem_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles InventoryToolStripMenuItem.Click

'im try to just show the table "inventory" in the form when you click on the sub-menu
'what do I have to write on here?

  End Sub

       Private Sub Customer_InformationToolStripMenuItem_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles InventoryToolStripMenuItem.Click

     
'im try to just show the table "Customer_Information" in the form when you click on 'the sub-menu
'what do I have to write on here?
  

    End Sub

    Private Sub ExitToolStripMenuItem_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles ExitToolStripMenuItem.Click
        End
    End Sub
End Class

Re: Question about VB and Databases

Eddy Vluggen15-Feb-14 23:33

15-Feb-14 23:33

CaseLost wrote:
'im try to just show the table "inventory" in the form when you click on the sub-menu
'what do I have to write on here?

Depends on what you want to do with that data. If you have a datagridview, you could load the data and assign it to it's datasource-property.

CaseLost wrote:
how do you write the code to show only "Inventory" (from the database) in the
form?

Not so fast; you'll need something to display it ON; a form, a grid, labels. Then you'll need to load your data (using SQL, restricting the data using a WHERE clause). After that, you can put your data on that form and display it.

If you haven't displayed anything from a database before, I suggest you put this project on hold and try some examples first. There's an example here[^] Smile | :)

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

sql query syntax in .net

Member 1050621512-Feb-14 3:52

Member 10506215

12-Feb-14 3:52

Hello guys.i need help with this query. i am not very good at this kind of query.
wh is wrong with it? missing ant ) or ' ?
Hereis my code:

strsql = "update membercars set "
                strsql = strsql & "make= '" & txtmake.Text & "', "
                strsql = strsql & "model= '" & txtmodel.Text & "', "
                strsql = strsql & "color= '" & txtcolor.Text & "', "
                strsql = strsql & "tag= '" & txttag.Text & "', "
                strsql = strsql & "memberid= " & Val(txtserial.Text) & " where serial = " & Val(txtcarserial.Text)

                strsql = strsql & " and serial = " & Val(txtserial.Text) & ") "

Dave Kreskowiak12-Feb-14 4:30

Dave Kreskowiak

12-Feb-14 4:30

EVERYTHING is wrong with it.

Google for "SQL Injection attack" to find out why what you're doing is wrong and then Google for "C# parameterized queries" for all kinds of examples on how to build your query without using string concatenation.

Oh, and you have a closing ")", but no opening "(".

A guide to posting questions on CodeProject[^]

Dave Kreskowiak

thatraja12-Feb-14 4:31

thatraja

12-Feb-14 4:31

You have added an unwanted closing paranthesis at end(last line), remove that.

And your query/code is poorly format, bad practices & not secure one. Use parametrized queries. Check these
Using parameterized SQL queries[^]
Give me parameterized SQL, or give me death[^]

thatraja

Richard Deeming12-Feb-14 5:22

Richard Deeming

12-Feb-14 5:22

Here's another SQL Injection reference, in case you weren't convinced by the other messages:
http://www.troyhunt.com/2013/07/everything-you-wanted-to-know-about-sql.html[^]

To fix your code:

SQL

command.CommandText = "update membercars set make = @make, model = @model, color = @color, tag = @tag, memberid = @memberid where serial = @serial";
command.Parameters.AddWithValue("@make", txtmake.Text)
command.Parameters.AddWithValue("@model", txtmodel.Text)
command.Parameters.AddWithValue("@color", txtcolor.Text)
command.Parameters.AddWithValue("@tag", txttag.Text)
command.Parameters.AddWithValue("@memberid", txtserial.Text)
command.Parameters.AddWithValue("@serial", txtcarserial.Text)

"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer

Eddy Vluggen12-Feb-14 6:56

12-Feb-14 6:56

This is your query btw (changed layout for readabillity);

SQL

update membercars 
   set make= '1'
     , model= 'super'
     , color= 'green'
     , tag= 'Youre it!'
     , memberid= txtserial
 where serial = txtcarserial
   and serial = txtserial

If txtcarserial and txtserial do not match, no records will be returned. The SQL query syntax is the same, regardless of the calling language.

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]

Member 1050621512-Feb-14 22:01

Member 10506215

12-Feb-14 22:01

Thank you so much guys. It worked and it's much easier than the method that i wrote before.

cmd.Connection = mcon
              cmd.CommandText = "update membercars set make=@make, model=@model, color=@color, tag= @tag where memberid=@memberid;"
              cmd.Parameters.AddWithValue("@make", txtmake.Text)
              cmd.Parameters.AddWithValue("@model", txtmodel.Text)
              cmd.Parameters.AddWithValue("@color", txtcolor.Text)
              cmd.Parameters.AddWithValue("@tag", txttag.Text)
              cmd.Parameters.AddWithValue("@memberid", txtserial.Text)
              cmd.ExecuteNonQuery()

Just one more question. I am writing an insert query.

cmd.CommandText = "insert into membercars(make, model, plate, color, tag) values (@make, @model, @plate, @color, @tag) where memberid=@memberid;"
      cmd.Parameters.AddWithValue("@make", txtmake.Text)
      cmd.Parameters.AddWithValue("@model", txtmodel.Text)
      cmd.Parameters.AddWithValue("@plate", txtplate.Text)
      cmd.Parameters.AddWithValue("@color", txtcolor.Text)
      cmd.Parameters.AddWithValue("@tag", txttag.Text)
      cmd.Parameters.AddWithValue("@memberid", txtserial.Text)
      cmd.ExecuteNonQuery()

it's giving me an eror: incorrect syntax near where. what's the error?

Eddy Vluggen12-Feb-14 22:28

12-Feb-14 22:28

Member 10506215 wrote:
what's the error?

An INSERT does not contain a WHERE-clause. What would/should the "where" do? Syntax can be found here[^].

Get a good book on SQL; this looks like trial and error.

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]