Running out of memory

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Running out of memory

Mike
Good morning, my name is Mike.

I am in the process of testing NLog's viability for our application, which is basically a plugin architecture consisting of many different AppDomains.

In the course of stress testing, I've found that after about 75 AppDomains are loaded with the following DLL, the system runs out of memory.

    public class Class1
    {
        private static Logger log;

        public void Sleep(int ConfigNumber)
        {
            List<double> wasteOfTime = new List<double>();
            for (int i = 0; i < 1000000; i++)
                wasteOfTime.Add(1182828.1262);
            wasteOfTime.Clear();
            Thread.Sleep(10000);
            log.Info(ConfigNumber + " here!");
           
        }
    }

Using this config:

   <targets>
      <target name="logfile" xsi:type="File" fileName="file.log"/>
    </targets>

    <rules>
      <logger name="*" minLevel="Info" writeTo="logfile" />
    </rules>

If I run the exact same code, minus the logging, I can reach up to ~200 before I encounter any issues.

Is there a special approach I should take to adapt NLog for multiple AppDomain scenarios more efficiently?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Running out of memory

Jarek Kowalski
Administrator

Couple questions:

1. Is this NLog 1.0 or 2.0 Beta1 ?

2. How many logger instances do you have? Are you creating them dynamically by any chance?

Jarek

NLog Blog
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Running out of memory

Mike
This is NLog 2.0 Beta1.

Each time the DLL is loaded into a new AppDomain, a new Logger gets instantiated dynamically.  Unless there happens to be a way to make use of the Logger from the primary AppDomain?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Running out of memory

Mike
Here is the rest of my setup, may be easier to just give you my source rather than talking about it:


Main Form
_______________________________________________________________________
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Reflection;
using System.Text;
using System.Threading;
using System.Windows.Forms;
using NLog;
// using Utilities;  <- This is a private DLL of mine.

namespace NLogTester
{
    public partial class Form1 : Form
    {
    //    private static Logger log = LogManager.GetCurrentClassLogger();
        List<WorkThread> threads = new List<WorkThread>();

        public Form1()
        {
            InitializeComponent();
        }

        private void btnGo_Click(object sender, EventArgs e)
        {
      //      log.Info("Button clicked");
            for (int i = 0; i < 150; i++)
            {
                WorkThread thread = new WorkThread();
                thread.Start(i);
                threads.Add(thread);
            }
        }

        private void btnStop_Click(object sender, EventArgs e)
        {
            foreach (WorkThread thread in threads)
                thread.keepRunning = false;
            threads.Clear();
        }
    }

    class WorkThread
    {
        SmartFile sf = new SmartFile();
        Thread myThread;
        public bool keepRunning = true;
        private static Logger log = LogManager.GetCurrentClassLogger();
        private static Random rand = new Random();
        int myNumber;
        public void Start(int uniqueNumber)
        {
            ThreadStart start = new ThreadStart(WorkerLoop);
            myThread = new Thread(start);
            myThread.Name = "SleepyDLL, Config #" + uniqueNumber;
            myNumber = uniqueNumber;
            myThread.Start();
        }

        public void WorkerLoop()
        {
            try
            {
                AppDomainSetup setup = new AppDomainSetup();

                setup.ApplicationBase = Application.StartupPath;
                AppDomain domain = AppDomain.CreateDomain("SleepyDLL " + myNumber, null, setup);

                Loader loader = (Loader)domain.CreateInstanceAndUnwrap(Assembly.GetExecutingAssembly().FullName, "NLogTester.Loader");
                loader.Reflect(sf.GetAppPath() + "SleepyDLL.dll");

                while (keepRunning)
                {

                    int sleeptime = rand.Next(2000);
                    loader.InvokeDll("SleepyDLL", myNumber);
                    Thread.Sleep(1000);
                }
                AppDomain.Unload(domain);
            }
            catch (Exception ex)
            {
                log.Error(ex.ToString());
            }
        }


    }
}
-----------------------------------------------------------------------
Loader.cs (Loads the appdomain with the specified DLL)
_______________________________________________________________________
using NLog;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Text;

namespace NLogTester
{
    class Loader : MarshalByRefObject
    {
        //private static Logger log = LogManager.GetCurrentClassLogger();

        Object activator;
        MethodInfo methodInfo;

        public override object InitializeLifetimeService()
        {
            return null;
        }

        public void Reflect(String assemblyPath)
        {
            try
            {
                Assembly assem = AppDomain.CurrentDomain.Load(AssemblyName.GetAssemblyName(assemblyPath));
                List<Type> availableTypes = new List<Type>();
                availableTypes.AddRange(assem.GetTypes());

                Type interfaceType = availableTypes.Find(delegate(Type t)
                {
                    return t.IsInterface;
                });

                String methodName = interfaceType.GetMethods()[0].Name;
                Type interfaceImplementingType = availableTypes.Find(delegate(Type t)
                {
                    return t.GetInterfaces().Contains(interfaceType);
                });

                methodInfo = interfaceImplementingType.GetMethod(methodName);
                activator = Activator.CreateInstance(interfaceImplementingType);
            }
            catch (Exception)
            {
                //log.Error(ex.ToString());
            }
        }

       public void InvokeDll(String dllName, int configNumber)
        {
            Object[] args = new object[1];
            args[0] = configNumber;
            try
            {
                methodInfo.Invoke(activator, args);
            }
            catch (Exception)
            {
                //log.Error("Error in invoke of " + dllName + ".dll, Config #" + configNumber + ": " + ex.ToString());
            }
        }
}
----------------------------------------------------------------------------
SleepyDLL (Compile seperately)
______________________________________________________________________________
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using NLog;

namespace SleepyDLL
{
    public interface ISleepy
    {
        void Sleep(int ConfigNumber);
    }
    public class Class1 : ISleepy
    {
        private static Logger log = LogManager.GetCurrentClassLogger();
        public Class1()
        {
        }

        public void Sleep(int ConfigNumber)
        {
        //    List<double> wasteOfTime = new List<double>();
        //    for (int i = 0; i < 1000000; i++)
        //        wasteOfTime.Add(1182828.1262);
        //    wasteOfTime.Clear();
            Thread.Sleep(10000);
            log.Info(ConfigNumber + " here!");
        }
    }
}
------------------------------------------------------------------------------
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Running out of memory

sbogy
I have the same problem. Using single instance of Nlog v.2.0.0.0. Software of kind 24/7.
After 1-2 days of working it's throwing OutOfMemoryException
 at System.IO.FileStream.Write(Byte[] array, Int32 offset, Int32 count)
   at NLog.Internal.FileAppenders.RetryingMultiProcessFileAppender.Write(Byte[] bytes)
   at NLog.Targets.FileTarget.WriteToFile(String fileName, Byte[] bytes, Boolean justData)
   ...
   at NLog.Logger.WriteToTargets(LogLevel level, IFormatProvider formatProvider, String message, Object[] args)
   at NLog.Logger.Info(String message, Int32 argument)

"String message" parametr is very simple ~ like "was writen N records".

help?
Loading...