Meta Trader 4 oftenly frozen when launching c# DLL with [DLLExport]

[Googol-Winchester]

Hi, thanks for your kindly effort first.

I want to attach a C# DLL to the native C++ software (not definitely sure) Meta Trader 4, it will be normal when it calls any other native C++ DLL, but oftenly got FROZEN when I assigned my C# DLL with this [DLLExport] package.

I had tried everything I can do, but issue is still there.

It won't happen every time but almost 5 / 10 when the phase of just loading DLL, not even calling any functions. But if lucky sometimes it works, nothing strange will happen.

I googled some articles, they said some situation named "DLL Loader Lock", if calling .NET functions before the native C++ software had really launched whole DLL, there will be big probability DEADLOCK happen.

But I am not sure if it is the issue of DLL Loader Lock? I have modified my C# code do not call any .NET functions before the MT4 call my DLL to Initialize DLL after the DLL had loaded, even not have any static object initialization before the first call. But not have any luck.

Please help, thanks very much.

[Googol-Winchester]

I mean maybe it caused such problem?

https://msdn.microsoft.com/zh-tw/library/windows/desktop/dn633971(v=vs.85).aspx

[Googol-Winchester]

sometimes when it load the Debug mode C# DLL failed, it show this:

2018.03.24 23:00:09.809 Unhandled exception 0xE0434352

[3F]

@Googol-Lien Hard to say what's wrong with your library because of very abstract details.

Well anyway, the entry point of the .net modules should contain simply initialization of the .net clr env through calling _CorDllMain or _CorExeMain (This moment contains native opcodes for specific architecture to finally init VM).

~
https://github.com/dotnet/coreclr/blob/20275aa647c5733bc5b1929cba3fd1094c67fb1d/src/dlls/mscorpe/ceefilegenwriter.cpp#L848

Unhandled exception 0xE0434352

Well, it seems we can exclude the first guess about deadlocks :)

Try to isolate problem via very simple code. Start from empty functions. Check calling convention to work properly with the stack. And try to debug at runtime via debugger.

Moreover, I think the problem may be simply with loading of some additional modules for you library. Check this out.

[Googol-Winchester]

@3F That’s nice you can confirm to exclude deadlock issue for me, that’s important for me after so much frustrated trying to guess which part goes wrong, deeply appreciated to your efforts and reply.

I can make sure the problem must happen when loading DLL phase because the script language MQL4 of Meta Trader 4 will receive 1st event named OnInit() without touch any other code when MT4 have been ready and loaded all hooked DLL.

Sometimes it even doesn’t call any OnInit() of 6 symbols which hooked to the same one C# DLL and got frozen not going initialization. (put print out “Start to init” on the 1st line of OnInit() in my MQL4 script to confirm it) So, no any call into the C# DLL yet at that moment, nor any MQL4 function called, except DLL loader and static objects’ constructors. (You can consider that as MT4 has launched 6 threads to hook the C# DLL isolately.)

Just now, I switch the target platform into .NET Framework 4.7.1, keep only one static read only object has been assigned by “new Object()” for using by Monitor.Enter() in my C# code, and all other objects or data for multithread created after lock the first object when the MT4 call my own DLL initializing function, for the reason to keep it simple when the DLL being loading, seems it becomes more stable.

The frozen issue always happens when the computer just been waked up after hours sleep or sometimes system seems too busy, so I can not confirm it is solved after just executed normally few times. (You know how frustrated every time I thought it had been solved, but some symbol windows are normal, some frozen, after trying exit-relaunch cycles running normally. The issue is just like a ghost.)

Moreover, I think the problem may be simply with loading of some additional modules for you library. Check this out.

I have used the DllImport to get functions of windows native DLL like user32.dll and wrapped, export some functions using DllExport attribute in my C# DLL to let MT4 call, do you mean this part could be problem?

But there was already such issue when I just have some simple functions in the C# DLL only.

I am not sure if this issue has been solved, I have changed settings by your dllexport configuration tool, maybe it help, too, but I don’t really understand all meanings of those parameters so just take a chance by trying my luck.

However, MT4 is not a very good platform having its own issue, either. After your help and modification of my C# codes, if there still any issue, hard to say maybe it just caused by some little bugs of MT4 itself.

[3F]

@Googol-Lien Unhandled exception is more related for problem with additional libraries. But if still you're watching mainly frozen MetaTrader, let's see ...

Initial thought, and I'm still sure, that current problem does not directly related neither for DllExport nor for .NET platform (its initialization I mean). And I'm also excluding deadlocks for DllMain as I said above.

The second thing it's a threads in your managed code and its possible deadlocks:

• Did you try to use STA thread instead of MTA ? Try to switch on single-threaded apartment model.

The MTA problem especially will be noticed with invoking Show() method for any System.Windows.Forms.Form instances. So you can also play with this to finally isolate an problem.

To use STA you can try like this:
https://github.com/3F/DllExport/blob/master/Wizard/UI/App.cs#L43

var thread = new Thread(() => ...your code...);
thread.SetApartmentState(ApartmentState.STA);  // <<<
thread.Start();
thread.Join();

[Googol-Winchester]

@3F I believe Unhandled exception must be caused by my fault not to lock any object but only use EnterCriticalSection() to create object when loading the C# DLL.

My bad luck the MT4 frozen again without any calling function of my C# DLL with 7 symbols' sub-windows hooked to it. All of them holding on DLL loading stage, and I just turn on my computer and login after about 1 hour.

The other MT4 (2 MT4 opened, different folder with different files of C# DLL) has 1 window only, it runs total normally.

There are 2 paragraph code I have used MTA modes, I will try to make them as STA mode, although they seems not really issue with such problem, because no one line could be executed before the MT4 trigger the 1st event to tell my MQL script it is ready.

But I only put these codes below in my loading stage (Before the Init() function, I did not let MT4 call any other function or trigger any new tread):

using System;
using System.Text;
using System.Threading;

namespace AngryWater
{
	//--------------------------------------------------------------------
	//	DLL 共用全域變數類別
	//--------------------------------------------------------------------
	public class TPGlobalVar
	{
		private static readonly TPGlobalVar _lockObj = new TPGlobalVar();
		private static Object _newsFile;
		private static ManualResetEvent _debugWinObject;
		private static AngryWaterDebug _debugWin;
		private static Thread _debugWinThread;
		private static bool _bDebugShowAgain;
		private static bool _bDebugShowing;
		private static bool _bDebugWinRun;
		private static string _newsDataStr;									//已經消化過的新聞資料字串
		private static int _newsCount;										//讀取到的新聞項目
		private static bool _newsDownloadQueue;								//是否已經排定下載新聞檔案工作
		private static Thread _newsThread;									//正在下載新聞的執行緒
		private static DateTime _newsTime;									//新聞資料餵入的時間

		//--------------------------------------------------------------------
		//鎖定新聞檔案讀寫權限
		public static bool LockNewsFile( int milliSecsTimeOut = Timeout.Infinite )
		{
			bool bLock = false;

			try
			{
				bLock = Monitor.TryEnter( _newsFile, milliSecsTimeOut );
			}
			catch {}

			return bLock;
		}

		//釋放新聞檔案讀寫權限
		public static void ReleaseNewsFile()
		{
			try
			{
				Monitor.Exit( _newsFile );
			}
			catch {}
		}

		//確認除錯視窗有打開
		public static AngryWaterDebug ValidateDebugWindow()
		{
			Monitor.Enter( _lockObj );

			if ( _debugWin == null )
			{
				_debugWin = new AngryWaterDebug();
				_bDebugWinRun = true;
				_bDebugShowAgain = true;

				_debugWinThread = new Thread( WinThreadProc );
				_debugWinThread.SetApartmentState( ApartmentState.MTA );
				_debugWinThread.Start();
			}
			else if ( !_bDebugShowing && !_bDebugShowAgain )
			{
				_bDebugShowAgain = true;
				_debugWinObject.Set();
			}

			Monitor.Exit( _lockObj );

			return _debugWin;
		}

		public static void DebugOut( string msg )
		{
			AngryWaterDebug dwin = ValidateDebugWindow();

			dwin.Send( msg );
		}

		public static void SetDebugWinShowAgain()
		{
			Monitor.Enter( _lockObj );

			_bDebugShowAgain = true;
			_debugWinObject.Set();

			Monitor.Exit( _lockObj );
		}

		public static void SetDebugWinClose()
		{
			Monitor.Enter( _lockObj );

			_bDebugWinRun = false;
			_debugWinObject.Set();

			Monitor.Exit( _lockObj );
		}

		public static void SetNewsData( string newsDataStr, int newsCount )
		{
			if ( LockNewsFile() )
			{
				_newsDataStr = newsDataStr;
				_newsCount = newsCount;

				if ( _newsDataStr == null )
				{
					_newsTime = new DateTime( 1970, 1, 1 );		//MQL4 時間的 0
				}
				else
				{
					_newsTime = DateTime.Now;
				}

				ReleaseNewsFile();
			}
		}

		public static DateTime GetNewsTime()
		{
			DateTime time = DateTime.MinValue;

			if ( LockNewsFile() )
			{
				time = _newsTime;

				ReleaseNewsFile();
			}

			return time;
		}

		public static int GetNewsCount()
		{
			int newsCount = 0;

			if ( LockNewsFile() )
			{
				newsCount = _newsCount;

				ReleaseNewsFile();
			}

			return newsCount;
		}

		public static string GetNewsData()
		{
			string newsData = null;

			if ( LockNewsFile() )
			{
				newsData = _newsDataStr;

				ReleaseNewsFile();
			}

			return newsData;
		}

		public static bool IsTryingDownloadNews()
		{
			bool bRet;

			Monitor.Enter( _lockObj );

			bRet = _newsDownloadQueue;

			Monitor.Exit( _lockObj );

			return bRet;
		}

		public static void SetNewsTryingDownload( bool bTrying )
		{
			Monitor.Enter( _lockObj );

			_newsDownloadQueue = bTrying;

			Monitor.Exit( _lockObj );
		}

		public static bool SetNewsDownloadThread( Thread newsThread )
		{
			bool bRet = true;

			Monitor.Enter( _lockObj );

			if ( _newsThread != null && Thread.CurrentThread != _newsThread )
			{
				//已有別的執行緒佔用，且不是目前的執行緒
				bRet = false;
			}
			else
			{
				_newsThread = newsThread;
			}

			Monitor.Exit( _lockObj );

			return bRet;
		}

		public static void ClearNewsData()
		{
			if ( LockNewsFile() )
			{
				_newsDataStr = null;
				_newsCount = 0;
				_newsTime = new DateTime( 1970, 1, 1 );		//MQL4 時間的 0

				ReleaseNewsFile();
			}
		}

		//--------------------------------------------------------------------
		private static void WinThreadProc()
		{
			bool bShowing;

			while ( _bDebugWinRun )
			{
				Monitor.Enter( _lockObj );

				if ( _bDebugShowAgain )
				{
					if ( _debugWin != null && !_debugWin.Visible ) _bDebugShowing = true;

					_bDebugShowAgain = false;
				}

				bShowing = _bDebugShowing;

				Monitor.Exit( _lockObj );

				if ( bShowing )
				{
					_debugWin.ShowDialog();

					Monitor.Enter( _lockObj );

					_bDebugShowing = false;

					Monitor.Exit( _lockObj );
				}

				_debugWinObject.WaitOne();
			}
		}

		public static void Init()
		{
			Thread.BeginCriticalRegion();

			Monitor.Enter( _lockObj );

			_newsFile = new Object();
			_debugWinObject = new ManualResetEvent( false );
			_newsTime = new DateTime( 1970, 1, 1 );		//MQL4 時間的 0

			Monitor.Exit( _lockObj );

			Thread.EndCriticalRegion();
		}

		static TPGlobalVar()
		{
		}

		~TPGlobalVar()
		{
			if ( _debugWin != null ) _debugWin.Dispose();
			if ( _newsThread != null ) _newsThread.Join();
		}
	}
}

[Googol-Winchester]

@3F These codes are script of my MQL, they looks just C++ like, I guess you won't have any problem to understand what are they doing.

MT4 even not trigger this definite 1st event to print out "==== PREPARE TO INIT EA ====" before DLLInit(); but just got frozen of everyone script which hook to my C# DLL.

The DLLInit(); do nothing but the TPGlobalVar.Init(); I list on last reply.

When it goes normally, nothing weird even Threads with MTA mode going fine, but when the issue coming, all MQL scripts just hanging there without calling this OnInit() after / during it loading my C# DLL (the only one DLL):

//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
{
	GetLastDeInitReason();

 	if ( !IsDllsAllowed() )
	{
		AlertComment( WindowExpertName() + ": 'Allow DLL Imports' Option are DISABLE. EA could not run normally without some DLL functions." );
		AlertComment( WindowExpertName() + ": Please Let the Check Box of 'Allow DLL Imports' on the 'Common' tab Checked to enable it when loading EA." );
		return INIT_FAILED;
	}

	Print( _Symbol, " : ==== PREPARE TO INIT EA ====" );

	DLLInit();

	return OnInitCall();
}

[3F]

@Googol-Lien

Did you try to write something simple to exclude many potential problems from your code ?

Try to load empty library that will display some message when DLLInit() is executed. Try even MessageBox.Show() because I don't remember MTA problem for this, or stdout, files, etc.

You can also try to attach debugger to MT4, then finally enter into your code with initialization.

In general, you're trying to make the full/end working plugin with all features what you want, but I will not help for this of course. Thus try to exclude any unrelated things. We need only connection for MT4. Then you will just try to solve problems with your bugs in code if it will be finally reached. So please use decomposition for your big task!

And for example, I see the following possible problems:

1. _debugWin.ShowDialog(); because of MTA if it's a Windows.Forms. Only STA as I said.
2. finalazer with your _newsThread.Join()
3. TPGlobalVar _lockObj = new TPGlobalVar(); do not use this class for sync threads, simply additional new Object() etc.
4. mainly, lot of confusing/ambiguous sync by my view

MT4 even not trigger this definite 1st event to print out "==== PREPARE TO INIT EA ====" before DLLInit();

That is, Print(...) line will be executed normally if you will not use DLLInit(); ?

The DLLInit(); do nothing but the TPGlobalVar.Init(); I list on last reply.

Show also code of DLLInit() if you can.

[Googol-Winchester]

@3F Hi, sorry if my case bothered you. I put my code here last reply does not mean I thought you will help me for my C# DLL, if it make you misunderstood.

I am not saying I have done perfect codes, but the frozen issue was there since I just put the 1st simple function like "int dosomething() { return 123; }" when the C# DLL just built up.

And yes, I always think there may be some potential problems in my code, since the first time MT4 got frozen, so as I said, I tried EVERYTHING I can do included the simplest code as you said for the past month. But issue is still there, so I try to contact you for better solution.

BTW, I said MT4 frozen does not mean the whole software frozen, but only the symbol window which did not call OnInit() event of MQL script function will be frozen. Other symbols sub-windows will work normally if they got lucky to pass that issue. Even the frozen symbol windows are not crashed or no movement, I can still see the price line moving or resize windows, but if I try to close the sub-windows or exit the whole MT4, then the MT4 will wait them and frozen, too.

That is, Print(...) line will be executed normally if you will not use DLLInit(); ?

As last reply what I said, if it is lucky passed the mysterious issue, everything is fine, even you said not to do the MTA thing is running normally without any problem.

-- BUT -- if the issue happened, not a piece of code of my DLL will be touched by MT4, no matter if I have put the DLLInit(); in the MQL script or not, MQL4 will not even call the 1st line in the OnInit() of my MQL4, even I put only the code like "int OnInit() { Print( "Hello" ); }", no "Hello" will be printed if the C# DLL hooked by the "#import" keyword and bad luck the issue happened.

Only if when I removed the "#import" keyword of MQL4 not to import my C# DLL, then the issue totally gone. If I put "#import" keyword to acquire my C# DLL, no matter MQL4 has called any function or not, the issue will come back again SOMETIMES.

That's why I consider the issue is hidden in the process of loading DLL or MT4 itself having some bug with CLR things. As I said, MT4 is not so stable.

I understand what you may think about my report and how I deal with it. I can say I am a game coder since 6502 assembly to Visual Studio 6 C++, and over 5 years Visual Studio 2008 C# script coding experience of Unity3D, I had built my own 3D network game engine even script compiler, I am not so unfamiliar with multi-threads things and debug huge project like 800 thousand lines of C++ game project.

So, please be advised, I really tried EVERYTHING included the STA, MTA things without luck, then come here for some little help.

If you ask me about my testing result, the frozen issue will happen with both mode, even UNKNOWN thread mode, too, but MTA running faster without any other problem (I do not mean the frozen issue, but crashed or others).

My DLLInit(); is as simple as I said if you want to peek:

		[DllExport( "_DLL_INIT", CallingConvention = CallingConvention.StdCall )]
		static public void Init()
		{
			TPGlobalVar.Init();
		}

This issue is so hard to debug because it comes SOMETIMES only, and always happen in loading DLL phase, even only import it but never call any DLL function.

If I got lucky, no any bug or problem what you considered or my guess will happen, so I can exclude the MTA problem if you think it is the issue. Because issue is there even I removed all multi-threading code or other "not only return 1 integer" functions.

All multi-threading problem will be easy to catch because I can see MT4 has called some functions, I know where could be problem, but this issue is not like this.

Anyway, since last reply, I had tried something else, like to remove EnterCriticalSection(), ExitCriticalSection() pair functions when I create the TPGlobalVar(), and the issue seems gone in past hours. I had tried reboot my computer, sleep for hours, launch and exit again and again with full functions in the DLL, not even once issue happened.

But I can not still confirm it is totally solved, this issue is so weird maybe it will come back 2 days later, that's the situation in the past month every time I think it was solved.

Thanks for your big help, please don't worry, I will come back to close this issue if the ghost do not show again in whole next week, or I will try everything you told me AGAIN even I had tried them over at least 3 times since last month, and come back to report my testing result.

Cheers, you did very kind efforts for people, really appreciated. :)

[Googol-Winchester]

@3F Just now, the ghost issue came back again. So I tried to do some tests, and did launch-exit MT4 cycle for dozens times when every time I made some little change to make sure where is the problem.

1. Only use the keyword "#Import" to hook the C# DLL, call nothing: Passed, no issue. And seems MT4 won't load anything if the MQL script did not call any function of C# DLL.

2. Removed all native C++ DLL of windows like "user32.dll" from my C# DLL: Passed, no issue.

3. Removed all codes where calling C# DLL functions in my MQL script, but only keep the first call to the init() of my C# DLL: Passed, no issue. I believe this test only call the 3 parts of codes below, 1 static Object has been newed, Init() has been called, then an empty constructor:

private static readonly Object _lockObj = new Object();

		public static void Init()
		{
			Monitor.Enter( _lockObj );

			if ( _newsFile == null )
			{
				_newsFile = new Object();
				_debugWinObject = new ManualResetEvent( false );
				_newsTime = new DateTime( 1970, 1, 1 );		//MQL4 時間的 0
			}

			Monitor.Exit( _lockObj );
		}

		static TPGlobalVar()
		{
		}

4. Added back those wrapped functions from the windows native C++ DLL, and only call the first call Init(): Passed, no issue. So native C++ DLL may not cause any problem.

After passed all these tests, I believe there is nothing issue when loading C# DLL. :) I am sorry about to bother you and deeply appreciated to your help.

Now I will do more detail debug test of other functions, but I don't think it is any issue of the package of [DLLExport].

FYI, I am very sure the MTA thread mode dose not cause any problem or issue, I don't know why there are some rumors said MTA mode will cause MT4 crashed or frozen? Actually, every time I change it as STA mode, the whole MT4 will be frozen just like crashed even worse than my issue, the whole MT4 dead can not be clicked anything more. So that's the reason why I use it as MTA mode to download file and read XML file, or control my debug output window form.

NO ANY ISSUE has been happened with those threads running MTA mode in my every past test, I can definitely exclude this rumor. The MT4 never be frozen when those MTA threads have been created and executed. (Maybe MT4 has been changed in recent version updates.)

Now I am trying to debug other part like internet connection without creating new thread to do but using the thread from MT4 calling:

				socket.Connect( ipEndPoint );
				socket.Send( ntpData );
				socket.Receive( ntpData );
				socket.Close();

Because this paragraph of codes just connect to time server to get UTC time through UDP policy and running fast, so I didn't make these codes with new thread. This is the most suspicious part of my code which may cause the frozen issue.

Thanks again.

[Googol-Winchester]

@3F Hi, Denis, I just come back with my apology and appreciation.

I think I can confirm this issue was caused the network time access by UDP connection now.

I have modified my code into MTA mode multi-threaded to connect and access network time, now everything looks 555 after dozens times testing. 👍

Although I still not understand the detail why the issue happened, but I am sure its happening time: I put a static variable in the MQL script to access network time, so C# DLL will be called before the OnInit() event triggered. So the calling thread of MT4 into the C# DLL may be freezed when socket connection is successful / failure. (Not very sure but I guess it is more possible when it is failure and thread had been timeout or something else goes wrong.)

Anyway, thank you for your help and very sorry to bother you so much time.

You really did a great work. 💯

(PS: MTA threads really works fine with your package and my C# DLL, so don't worry about the STA mode.)

[3F]

I am very sure the MTA thread mode dose not cause any problem or issue, I don't know why there are some rumors said MTA mode will cause MT4 crashed or frozen?

about MT4 don't know, but the MTA problem is very old known issue of the System.Windows.Form components when you want to use Windows.Forms for MTA threads.

You can start reading, for example, from CA2232:
https://msdn.microsoft.com/en-us/library/ms182351.aspx

 ... the application uses the multithreaded apartment model, 
     which is not supported for Windows Forms.

And unfortunately, it's not a rumors :) Try to change STA to MTA for mentioned wrapper above: https://github.com/3F/DllExport/blob/master/Wizard/UI/App.cs#L43 then try to run our Wizard via msbuild or like -action Configure. Part of this also affects for debugging via VS debugger. VS IDE may also be crashed as I remember :)

Yep, deadlocks.

Well, I just checked MT4 (downloaded from first results in search), and I can say the following: even without test of the .net libraries I had a few moments with frozen main window of this amazing app and 3 infinite loops (between MetaEditor4 <-> MT4) that fully stopped my virtual machine o_o VM: enabled 2 CPUs + 8GB RAM; So, seems it's more related for MT4 ... <_<

But yes, I don't see MTA problems for this case:

#import "MT4test.dll"
void    sayHello();
#import
  
void OnInit()
{
   sayHello();
}

[DllExport(CallingConvention.StdCall)]
public static void sayHello()
{
	new Form() { Text = "Hello from .net clr" }
		.ShowDialog();
}