How can I avoid memory leak in EntityFramework at object creation?

Question

0.00/5 (No votes)

See more:

Hello everybody,

I'm quite newbie to EntityFramework and I have encountered a problem, when I add multiple items to database.
This is not the real application, but for learning purposes I have created a test app, that kinda simulate the behaviour of the real application.

The real application is based on producer-consumer pattern, where the consumer saves the received array of bytes somewhere - with some metadata -, which is created upon user interaction, so I don't know when and how many comes.

To simulate this pattern, this is the code I created:

C#

private void Button_Click_1(object sender, RoutedEventArgs e)
       {
           string filename = @"path-to-a-jpg";

           //approx. 15 megabytes of Bmp image, to fill up the DB more quickly
           byte[] data = System.Drawing.Image.FromFile(filename).ToByteArray();

           Visitor someone;

           for (int i = 0; i < 100; i++)
           {
               using (ZooContext ctx = new ZooContext())
               {
                   someone = new Visitor()
                   {
                       Name = "test100 " + i,
                       Img = data
                   };


                   ctx.Visitors.Add(someone);

                   ctx.SaveChanges();//around 81 it crashes with out of memory exception

                   someone = null;
               }

               GC.Collect();//tried to force garbage collection, but it didn't help
           }
       }

Class, that is stored in the DB (it would be nice, to keep the lazy loading enabled):

C#

public class Visitor
    {
        public int ID { get; set; }

        public string Name { get; set; }

        public byte[] Img { get; set; }

        public virtual ICollection<Animal> FavouriteAnimals { get; set; }
    }

My DbContext is this class:

C#

public class ZooContext : DbContext
    {
        public ZooContext()
        {
            //we disable databese initialization
            Database.SetInitializer<ZooContext>(new System.Data.Entity.NullDatabaseInitializer<ZooContext>());

        }

        public DbSet<Animal> Animals { get; set; }

        public DbSet<Visitor> Visitors { get; set; }


        protected override void OnModelCreating(DbModelBuilder modelBuilder)
        {
            //i just need the singular class names as tabel names
            modelBuilder.Conventions.Remove<PluralizingTableNameConvention>();

            modelBuilder.Entity<Visitor>()
                .HasMany(v => v.FavouriteAnimals)
                .WithMany(a => a.Visitors)
                .Map(m =>
                {
                    m.MapLeftKey("Visitor_ID");
                    m.MapRightKey("Animal_ID");
                    m.ToTable("Visitor_Animal");
                });


            base.OnModelCreating(modelBuilder);
        }

    }

After adding approx. 80 items it throws an OutOfMemory exception and at that point the task manager shows around 1200 megabytes of used memory, for vshost process containing my application.
I'm testing it on a 32-bit computer with 3GBs of RAM, but I suppose that the memory overhead should be much more less.

By the way, retrieving from such a huge DB is OK, because it's OK for me, that the retrieved objects are not tracked and querying for images has little memory overhead even, my DB is around 11GB now :).

I forgot to add, that I'm using SQLite with System.Data.SQLite provider.

Thanks in advance! I appreciate any suggestion.

Posted 2-Sep-15 5:55am

macika123

Updated 2-Sep-15 6:14am

v2

Add a Solution

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Dave Kreskowiak · Answer 1 · 2015-09-02T06:31:00

Instead of using a context "globally" through the class, create an instance of the context inside your for loop, do your database work for that record and save, then dispose of the context instance. Repeat for every iteration of the loop.

The problem is that you set someone to null but the Visitor instance does not go away. The reason is because the EF change tracker is still holding a reference to the objects you added and is even tracking original values for the properties in them.

Disposing the instance of the context and creating a new one on every pass also kills the change tracker inside the context and drops the references to the objects you added, freeing them up for the garbage collector to do its thing.

You don't need the call to GC.Collect at all.

One of the problems with your design is that if you tried to get all of Visitor records for some query you'd be returning about 15MB of data for EACH RECORD. That isn't a scalable solution. If you want the data on every Visitor but not the pictures you con't do that with your current design. See this[^] for an example of splitting the table into two classes. This allows you to work with all of the data without digging up all of the image data.

I used a similar technique for images in my current project. I store all the image metadata and thumbnail data using one class and the full size image data using a separate class, both of which are mapped to the same database table. Works like a charm.